Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Enable JS emit for noCheck and noCheck for transpileModule #58364

Merged
merged 7 commits into from
May 30, 2024

Conversation

weswigham
Copy link
Member

@weswigham weswigham commented Apr 29, 2024

This PR enables noCheck for JS emit, and tests all js emit tests with noCheck, just as we do declaration emit tests with noCheck. It also enables noCheck for transpileModule by default.

@typescript-bot typescript-bot added Author: Team For Uncommitted Bug PR for untriaged, rejected, closed or missing bug labels Apr 29, 2024
@weswigham
Copy link
Member Author

@typescript-bot perf test this faster

@typescript-bot
Copy link
Collaborator

typescript-bot commented Apr 29, 2024

Starting jobs; this comment will be updated as builds start and complete.

Command Status Results
perf test this faster ✅ Started 👀 Results

@typescript-bot
Copy link
Collaborator

@weswigham
The results of the perf run you requested are in!

Here they are:

tsc

Comparison Report - baseline..pr
Metric baseline pr Delta Best Worst p-value
Compiler-Unions - node (v18.15.0, x64)
Errors 30 30 ~ ~ ~ p=1.000 n=6
Symbols 62,154 62,157 +3 (+ 0.00%) ~ ~ p=0.001 n=6
Types 50,273 50,284 +11 (+ 0.02%) ~ ~ p=0.001 n=6
Memory used 194,036k (± 1.01%) 206,867k (± 0.54%) 🔻+12,831k (+ 6.61%) 205,758k 207,973k p=0.005 n=6
Parse Time 1.30s (± 0.42%) 1.29s (± 1.24%) ~ 1.27s 1.31s p=0.055 n=6
Bind Time 0.72s 0.72s ~ ~ ~ p=1.000 n=6
Check Time 9.58s (± 0.31%) 10.16s (± 0.56%) 🔻+0.58s (+ 6.05%) 10.12s 10.25s p=0.005 n=6
Emit Time 2.62s (± 0.78%) 4.18s (± 0.72%) 🔻+1.56s (+59.52%) 4.14s 4.23s p=0.005 n=6
Total Time 14.23s (± 0.24%) 16.35s (± 0.40%) 🔻+2.12s (+14.87%) 16.28s 16.43s p=0.005 n=6
angular-1 - node (v18.15.0, x64)
Errors 5 5 ~ ~ ~ p=1.000 n=6
Symbols 945,172 945,778 +606 (+ 0.06%) ~ ~ p=0.001 n=6
Types 408,068 408,353 +285 (+ 0.07%) ~ ~ p=0.001 n=6
Memory used 1,222,051k (± 0.01%) 1,249,000k (± 0.00%) +26,950k (+ 2.21%) 1,248,916k 1,249,055k p=0.005 n=6
Parse Time 8.05s (± 0.36%) 8.04s (± 0.42%) ~ 8.01s 8.10s p=0.629 n=6
Bind Time 2.21s (± 0.62%) 2.22s (± 0.57%) ~ 2.21s 2.24s p=0.357 n=6
Check Time 36.43s (± 0.33%) 39.06s (± 0.40%) 🔻+2.64s (+ 7.24%) 38.83s 39.25s p=0.005 n=6
Emit Time 17.34s (± 0.86%) 22.40s (± 0.53%) 🔻+5.06s (+29.18%) 22.28s 22.58s p=0.005 n=6
Total Time 64.03s (± 0.31%) 71.72s (± 0.23%) 🔻+7.69s (+12.01%) 71.47s 71.86s p=0.005 n=6
mui-docs - node (v18.15.0, x64)
Errors 5 5 ~ ~ ~ p=1.000 n=6
Symbols 1,955,870 1,956,035 +165 (+ 0.01%) ~ ~ p=0.001 n=6
Types 676,290 676,309 +19 (+ 0.00%) ~ ~ p=0.001 n=6
Memory used 1,754,563k (± 0.00%) 1,758,450k (± 0.00%) +3,887k (+ 0.22%) 1,758,404k 1,758,494k p=0.005 n=6
Parse Time 6.71s (± 0.32%) 6.71s (± 0.42%) ~ 6.69s 6.75s p=0.934 n=6
Bind Time 2.31s (± 0.45%) 2.30s (± 1.27%) ~ 2.25s 2.33s p=1.000 n=6
Check Time 56.87s (± 0.50%) 57.59s (± 0.32%) +0.72s (+ 1.27%) 57.26s 57.82s p=0.005 n=6
Emit Time 0.13s (± 3.87%) 0.14s (± 4.51%) ~ 0.13s 0.15s p=0.091 n=6
Total Time 66.02s (± 0.46%) 66.75s (± 0.26%) +0.72s (+ 1.09%) 66.44s 66.98s p=0.005 n=6
self-build-src - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 1,215,708 1,216,393 +685 (+ 0.06%) ~ ~ p=0.001 n=6
Types 257,646 257,862 +216 (+ 0.08%) ~ ~ p=0.001 n=6
Memory used 2,323,836k (± 0.03%) 2,332,963k (± 0.02%) +9,127k (+ 0.39%) 2,332,530k 2,333,620k p=0.005 n=6
Parse Time 5.94s (± 0.73%) 5.93s (± 1.07%) ~ 5.86s 6.00s p=0.689 n=6
Bind Time 2.21s (± 1.01%) 2.21s (± 1.05%) ~ 2.18s 2.25s p=0.936 n=6
Check Time 39.91s (± 0.28%) 42.90s (± 0.50%) 🔻+2.99s (+ 7.50%) 42.63s 43.24s p=0.005 n=6
Emit Time 3.13s (± 4.95%) 3.13s (± 1.27%) ~ 3.07s 3.18s p=0.630 n=6
Total Time 51.23s (± 0.47%) 54.21s (± 0.45%) 🔻+2.98s (+ 5.82%) 54.01s 54.61s p=0.005 n=6
self-build-src-public-api - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 1,215,708 1,216,393 +685 (+ 0.06%) ~ ~ p=0.001 n=6
Types 257,646 257,862 +216 (+ 0.08%) ~ ~ p=0.001 n=6
Memory used 2,398,433k (± 0.02%) 2,407,755k (± 0.02%) +9,322k (+ 0.39%) 2,406,735k 2,408,266k p=0.005 n=6
Parse Time 5.11s (± 0.64%) 5.10s (± 0.69%) ~ 5.07s 5.15s p=0.335 n=6
Bind Time 1.67s (± 0.80%) 1.68s (± 0.63%) ~ 1.66s 1.69s p=0.358 n=6
Check Time 34.45s (± 0.32%) 36.55s (± 0.42%) 🔻+2.10s (+ 6.11%) 36.26s 36.68s p=0.005 n=6
Emit Time 2.66s (± 0.97%) 2.68s (± 3.69%) ~ 2.54s 2.80s p=0.870 n=6
Total Time 43.92s (± 0.17%) 46.03s (± 0.42%) 🔻+2.11s (+ 4.81%) 45.64s 46.14s p=0.005 n=6
self-compiler - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 256,204 256,501 +297 (+ 0.12%) ~ ~ p=0.001 n=6
Types 103,653 103,740 +87 (+ 0.08%) ~ ~ p=0.001 n=6
Memory used 424,273k (± 0.01%) 428,151k (± 0.01%) +3,879k (+ 0.91%) 428,112k 428,210k p=0.005 n=6
Parse Time 2.76s (± 1.59%) 2.77s (± 0.40%) ~ 2.76s 2.78s p=0.869 n=6
Bind Time 1.10s (± 1.86%) 1.09s (± 0.37%) ~ 1.09s 1.10s p=1.000 n=6
Check Time 15.49s (± 0.33%) 16.45s (± 0.44%) 🔻+0.96s (+ 6.19%) 16.36s 16.54s p=0.005 n=6
Emit Time 1.15s (± 1.53%) 1.16s (± 2.07%) ~ 1.12s 1.19s p=0.193 n=6
Total Time 20.49s (± 0.29%) 21.47s (± 0.31%) 🔻+0.98s (+ 4.80%) 21.41s 21.57s p=0.005 n=6
ts-pre-modules - node (v18.15.0, x64)
Errors 35 35 ~ ~ ~ p=1.000 n=6
Symbols 224,824 224,980 +156 (+ 0.07%) ~ ~ p=0.001 n=6
Types 93,390 93,429 +39 (+ 0.04%) ~ ~ p=0.001 n=6
Memory used 369,357k (± 0.03%) 372,388k (± 0.01%) +3,032k (+ 0.82%) 372,344k 372,458k p=0.005 n=6
Parse Time 2.83s (± 1.12%) 2.86s (± 1.83%) ~ 2.82s 2.96s p=0.809 n=6
Bind Time 1.59s (± 1.01%) 1.55s (± 3.98%) ~ 1.43s 1.59s p=0.213 n=6
Check Time 15.71s (± 0.29%) 16.78s (± 0.36%) 🔻+1.07s (+ 6.82%) 16.71s 16.88s p=0.005 n=6
Emit Time 0.00s 0.00s ~ ~ ~ p=1.000 n=6
Total Time 20.13s (± 0.34%) 21.20s (± 0.28%) 🔻+1.07s (+ 5.30%) 21.11s 21.27s p=0.005 n=6
vscode - node (v18.15.0, x64)
Errors 4 4 ~ ~ ~ p=1.000 n=6
Symbols 2,799,024 2,804,031 +5,007 (+ 0.18%) ~ ~ p=0.001 n=6
Types 950,803 952,904 +2,101 (+ 0.22%) ~ ~ p=0.001 n=6
Memory used 2,926,597k (± 0.01%) 3,078,559k (± 0.00%) 🔻+151,962k (+ 5.19%) 3,078,514k 3,078,624k p=0.005 n=6
Parse Time 13.34s (± 0.22%) 13.34s (± 0.29%) ~ 13.29s 13.41s p=0.511 n=6
Bind Time 4.09s (± 0.46%) 4.10s (± 0.24%) ~ 4.09s 4.11s p=0.805 n=6
Check Time 73.01s (± 0.24%) 85.83s (± 0.63%) 🔻+12.82s (+17.56%) 85.32s 86.66s p=0.005 n=6
Emit Time 19.77s (± 2.11%) 48.37s (± 1.00%) 🔻+28.60s (+144.66%) 47.75s 48.89s p=0.005 n=6
Total Time 110.21s (± 0.31%) 151.65s (± 0.65%) 🔻+41.44s (+37.60%) 150.52s 152.93s p=0.005 n=6
webpack - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 265,853 265,879 +26 (+ 0.01%) ~ ~ p=0.001 n=6
Types 108,438 108,441 +3 (+ 0.00%) ~ ~ p=0.001 n=6
Memory used 410,473k (± 0.03%) 412,580k (± 0.02%) +2,107k (+ 0.51%) 412,467k 412,710k p=0.005 n=6
Parse Time 4.80s (± 0.80%) 4.78s (± 0.67%) ~ 4.74s 4.83s p=0.469 n=6
Bind Time 2.07s (± 0.94%) 2.08s (± 0.36%) ~ 2.07s 2.09s p=0.503 n=6
Check Time 21.14s (± 0.23%) 22.10s (± 0.20%) 🔻+0.95s (+ 4.50%) 22.04s 22.15s p=0.005 n=6
Emit Time 0.00s 0.00s ~ ~ ~ p=1.000 n=6
Total Time 28.01s (± 0.28%) 28.96s (± 0.22%) +0.95s (+ 3.37%) 28.87s 29.01s p=0.005 n=6
xstate-main - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 523,981 524,011 +30 (+ 0.01%) ~ ~ p=0.001 n=6
Types 178,708 178,708 ~ ~ ~ p=1.000 n=6
Memory used 461,306k (± 0.02%) 462,269k (± 0.02%) +963k (+ 0.21%) 462,177k 462,387k p=0.005 n=6
Parse Time 3.11s (± 0.52%) 3.14s (± 0.57%) +0.03s (+ 0.86%) 3.11s 3.16s p=0.041 n=6
Bind Time 1.18s (± 0.54%) 1.18s (± 0.44%) ~ 1.18s 1.19s p=0.386 n=6
Check Time 18.18s (± 0.78%) 18.54s (± 0.63%) +0.37s (+ 2.02%) 18.38s 18.69s p=0.005 n=6
Emit Time 0.00s 0.00s ~ ~ ~ p=1.000 n=6
Total Time 22.47s (± 0.64%) 22.87s (± 0.46%) +0.39s (+ 1.75%) 22.72s 23.00s p=0.005 n=6
System info unknown
Hosts
  • node (v18.15.0, x64)
Scenarios
  • Compiler-Unions - node (v18.15.0, x64)
  • angular-1 - node (v18.15.0, x64)
  • mui-docs - node (v18.15.0, x64)
  • self-build-src - node (v18.15.0, x64)
  • self-build-src-public-api - node (v18.15.0, x64)
  • self-compiler - node (v18.15.0, x64)
  • ts-pre-modules - node (v18.15.0, x64)
  • vscode - node (v18.15.0, x64)
  • webpack - node (v18.15.0, x64)
  • xstate-main - node (v18.15.0, x64)
Benchmark Name Iterations
Current pr 6
Baseline baseline 6

Developer Information:

Download Benchmarks


!!!! File file1.js missing from original emit, but present in noCheck emit
//// [file1.js]
export const x = 3;
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No checker error, so the emit happens. 🤷‍♂️

}
declare var k: c | m.c;
-declare var l: c | m.c;
+declare var l: m.c | c;
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Union order swaps - the EmitResolver during js+.d.ts emit happens to create these types in the reverse order of a full typecheck.

!!! related TS2304 multiLinePropertyAccessAndArrowFunctionIndent1.ts:1:18: Cannot find name 'role'.
!!! related TS2304 multiLinePropertyAccessAndArrowFunctionIndent1.ts:2:18: Cannot find name 'Role'.
!!! related TS2503 multiLinePropertyAccessAndArrowFunctionIndent1.ts:4:26: Cannot find namespace 'ng'.
!!! related TS2304 multiLinePropertyAccessAndArrowFunctionIndent1.ts:4:53: Cannot find name 'Role'.
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The top-level return is a grammar error that normally blocks us reporting all these errors. Calculating emit flags for them lazily, however, manifests these errors. Do note: this is also technically a correctness improvement - previously, block scoped bindings and the like inside a top-level return like this wouldn't get transformed according to the target because we'd assume we'd calculated their NodeCheckFlags when in fact we hadn't. Still, these errors are basically silent, even if they're reasonable, hence the warning in the error log. I could probably silence these and get the old behavior back if I really tried, I'm just not sure it's necessary.


!!!! File all.d.ts missing from original emit, but present in noCheck emit
//// [all.d.ts]
declare module "ref/a" {
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Now that this isn't emitDeclarationOnly, this configuration of compiler options is forbidden, thus produces no output again. (Even with noCheck)

@@ -846,7 +846,7 @@ type AnyArr = [...any];
declare const tc4: [...string[], number, number, number];
declare function concat2<T extends readonly unknown[], U extends readonly unknown[]>(t: T, u: U): (T[number] | U[number])[];
-declare const tc5: (2 | 4 | 1 | 3 | 6 | 5)[];
+declare const tc5: (1 | 2 | 3 | 6 | 4 | 5)[];
+declare const tc5: (3 | 2 | 1 | 6 | 4 | 5)[];
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This union order shifts again. Presumably the EmitResolver usages in js emit tweak the order we manifest these number types in.


!!!! File autoAccessor1.js missing from original emit, but present in noCheck emit
//// [autoAccessor1.js]
var __classPrivateFieldGet = (this && this.__classPrivateFieldGet) || function (receiver, state, kind, f) {
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The target error for auto-accessors, Properties with the 'accessor' modifier are only available when targeting ECMAScript 2015 and higher. is a checker error. So when it's suppressed, we happily do emit.

@weswigham weswigham changed the title Enable JS emit for noCheck Enable JS emit for noCheck and noCheck for transpileModule Apr 30, 2024
"use strict";
Object.defineProperty(exports, "__esModule", { value: true });
-exports.a = void 0;
-exports.a = x.c;
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this is a case where the noCheck output is an improvement. As a perf optimization, I made reference marking with an Unspecified hint skip all Ambient nodes - that causes this change. The export declare import statement in this file usually gets transformed into the output js, even though it has a declare modifier. That's likely because the declare modifier on an import is an error, so the alias marking is done unconditionally. In any case, since the input has a syntax/grammar type error, I think it's fine for the output to diverge a bit like this. I could bring it in-line (either by changing normal behavior to elide this, or noCheck to retain it), but I'm not sure it's worth the effort, given the error.

@weswigham
Copy link
Member Author

@typescript-bot perf test this faster

I still need to sprinkle around setting the calculatedFlags for nodes during the full typecheck pass (likely just by setting all of them when the node is visited), so this should still have perf impact, just less than before.

@typescript-bot
Copy link
Collaborator

typescript-bot commented May 14, 2024

Starting jobs; this comment will be updated as builds start and complete.

Command Status Results
perf test this faster ✅ Started 👀 Results

@typescript-bot
Copy link
Collaborator

@weswigham
The results of the perf run you requested are in!

Here they are:

tsc

Comparison Report - baseline..pr
Metric baseline pr Delta Best Worst p-value
Compiler-Unions - node (v18.15.0, x64)
Errors 30 30 ~ ~ ~ p=1.000 n=6
Symbols 62,154 62,155 +1 (+ 0.00%) ~ ~ p=0.001 n=6
Types 50,248 50,251 +3 (+ 0.01%) ~ ~ p=0.001 n=6
Memory used 192,387k (± 0.10%) 205,440k (± 0.44%) 🔻+13,053k (+ 6.78%) 205,031k 207,300k p=0.005 n=6
Parse Time 1.56s (± 0.96%) 1.55s (± 1.38%) ~ 1.52s 1.57s p=0.183 n=6
Bind Time 0.87s (± 1.19%) 0.87s (± 1.53%) ~ 0.86s 0.89s p=0.923 n=6
Check Time 11.32s (± 0.30%) 11.38s (± 0.34%) +0.06s (+ 0.52%) 11.32s 11.42s p=0.035 n=6
Emit Time 3.16s (± 0.77%) 4.79s (± 0.64%) 🔻+1.64s (+51.90%) 4.77s 4.85s p=0.005 n=6
Total Time 16.91s (± 0.21%) 18.59s (± 0.36%) 🔻+1.68s (+ 9.93%) 18.50s 18.68s p=0.005 n=6
angular-1 - node (v18.15.0, x64)
Errors 5 5 ~ ~ ~ p=1.000 n=6
Symbols 944,110 944,377 +267 (+ 0.03%) ~ ~ p=0.001 n=6
Types 407,140 407,284 +144 (+ 0.04%) ~ ~ p=0.001 n=6
Memory used 1,222,041k (± 0.00%) 1,241,149k (± 0.00%) +19,108k (+ 1.56%) 1,241,069k 1,241,223k p=0.005 n=6
Parse Time 8.07s (± 0.71%) 8.08s (± 0.32%) ~ 8.03s 8.10s p=0.935 n=6
Bind Time 2.24s (± 0.47%) 2.23s (± 0.75%) ~ 2.21s 2.25s p=0.742 n=6
Check Time 36.26s (± 0.47%) 36.42s (± 0.40%) ~ 36.20s 36.64s p=0.148 n=6
Emit Time 17.40s (± 0.38%) 21.35s (± 0.69%) 🔻+3.94s (+22.66%) 21.16s 21.53s p=0.005 n=6
Total Time 63.97s (± 0.29%) 68.08s (± 0.34%) 🔻+4.11s (+ 6.43%) 67.68s 68.33s p=0.005 n=6
mui-docs - node (v18.15.0, x64)
Errors 5 5 ~ ~ ~ p=1.000 n=6
Symbols 1,964,276 1,964,276 ~ ~ ~ p=1.000 n=6
Types 819,365 819,365 ~ ~ ~ p=1.000 n=6
Memory used 1,849,721k (± 0.00%) 1,849,735k (± 0.00%) ~ 1,849,667k 1,849,851k p=0.810 n=6
Parse Time 8.05s (± 0.52%) 8.05s (± 0.36%) ~ 8.02s 8.09s p=1.000 n=6
Bind Time 2.74s (± 1.23%) 2.72s (± 0.45%) ~ 2.70s 2.73s p=0.103 n=6
Check Time 69.19s (± 0.30%) 69.30s (± 0.68%) ~ 68.72s 69.78s p=0.574 n=6
Emit Time 0.17s (± 3.32%) 0.17s (± 3.10%) ~ 0.16s 0.17s p=0.640 n=6
Total Time 80.15s (± 0.20%) 80.23s (± 0.60%) ~ 79.64s 80.70s p=0.630 n=6
self-build-src - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 1,221,216 1,221,413 +197 (+ 0.02%) ~ ~ p=0.001 n=6
Types 259,517 259,546 +29 (+ 0.01%) ~ ~ p=0.001 n=6
Memory used 2,337,342k (± 0.03%) 2,337,909k (± 0.03%) ~ 2,337,023k 2,339,287k p=0.298 n=6
Parse Time 5.05s (± 1.19%) 5.00s (± 0.64%) ~ 4.97s 5.05s p=0.128 n=6
Bind Time 1.88s (± 0.87%) 1.89s (± 0.62%) ~ 1.88s 1.91s p=0.462 n=6
Check Time 33.77s (± 0.20%) 33.98s (± 0.22%) +0.20s (+ 0.60%) 33.85s 34.06s p=0.008 n=6
Emit Time 2.66s (± 1.35%) 2.68s (± 1.33%) ~ 2.63s 2.72s p=0.521 n=6
Total Time 43.37s (± 0.25%) 43.56s (± 0.32%) +0.19s (+ 0.43%) 43.37s 43.77s p=0.045 n=6
self-build-src-public-api - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 1,221,216 1,221,413 +197 (+ 0.02%) ~ ~ p=0.001 n=6
Types 259,517 259,546 +29 (+ 0.01%) ~ ~ p=0.001 n=6
Memory used 2,413,862k (± 0.01%) 2,415,295k (± 0.01%) +1,433k (+ 0.06%) 2,414,897k 2,415,795k p=0.005 n=6
Parse Time 7.75s (± 0.72%) 7.64s (± 0.42%) -0.11s (- 1.36%) 7.61s 7.70s p=0.012 n=6
Bind Time 2.50s (± 0.55%) 2.49s (± 0.72%) ~ 2.46s 2.51s p=0.293 n=6
Check Time 49.70s (± 0.23%) 50.01s (± 0.28%) +0.31s (+ 0.62%) 49.83s 50.14s p=0.013 n=6
Emit Time 3.94s (± 3.17%) 3.83s (± 0.94%) ~ 3.78s 3.87s p=0.128 n=6
Total Time 63.89s (± 0.32%) 64.00s (± 0.27%) ~ 63.78s 64.21s p=0.298 n=6
self-compiler - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 256,767 256,845 +78 (+ 0.03%) ~ ~ p=0.001 n=6
Types 104,587 104,616 +29 (+ 0.03%) ~ ~ p=0.001 n=6
Memory used 426,112k (± 0.03%) 426,413k (± 0.01%) +301k (+ 0.07%) 426,362k 426,466k p=0.005 n=6
Parse Time 3.38s (± 0.49%) 3.37s (± 0.58%) ~ 3.34s 3.39s p=0.745 n=6
Bind Time 1.32s (± 0.68%) 1.33s (± 0.31%) +0.01s (+ 0.88%) 1.33s 1.34s p=0.025 n=6
Check Time 17.90s (± 0.18%) 18.01s (± 0.46%) ~ 17.86s 18.10s p=0.053 n=6
Emit Time 1.38s (± 0.85%) 1.37s (± 1.75%) ~ 1.34s 1.40s p=0.410 n=6
Total Time 23.98s (± 0.09%) 24.08s (± 0.47%) ~ 23.88s 24.21s p=0.063 n=6
ts-pre-modules - node (v18.15.0, x64)
Errors 35 35 ~ ~ ~ p=1.000 n=6
Symbols 224,575 224,575 ~ ~ ~ p=1.000 n=6
Types 93,785 93,785 ~ ~ ~ p=1.000 n=6
Memory used 369,901k (± 0.03%) 369,906k (± 0.03%) ~ 369,775k 370,060k p=1.000 n=6
Parse Time 2.83s (± 0.41%) 2.86s (± 2.04%) ~ 2.81s 2.97s p=0.569 n=6
Bind Time 1.58s (± 0.74%) 1.56s (± 3.93%) ~ 1.44s 1.61s p=0.934 n=6
Check Time 15.67s (± 0.39%) 15.74s (± 0.19%) +0.08s (+ 0.49%) 15.70s 15.78s p=0.037 n=6
Emit Time 0.00s 0.00s ~ ~ ~ p=1.000 n=6
Total Time 20.07s (± 0.35%) 20.16s (± 0.25%) +0.09s (+ 0.42%) 20.08s 20.21s p=0.043 n=6
vscode - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 2,823,555 2,825,537 +1,982 (+ 0.07%) ~ ~ p=0.001 n=6
Types 957,785 958,891 +1,106 (+ 0.12%) ~ ~ p=0.001 n=6
Memory used 2,995,829k (± 0.00%) 3,118,275k (± 0.00%) 🔻+122,446k (+ 4.09%) 3,118,194k 3,118,353k p=0.005 n=6
Parse Time 16.92s (± 0.14%) 16.95s (± 0.25%) ~ 16.90s 17.01s p=0.226 n=6
Bind Time 5.01s (± 0.28%) 5.00s (± 0.21%) ~ 4.98s 5.01s p=0.118 n=6
Check Time 88.58s (± 0.49%) 88.50s (± 0.29%) ~ 88.14s 88.89s p=0.423 n=6
Emit Time 28.64s (± 0.47%) 46.72s (± 0.64%) 🔻+18.08s (+63.14%) 46.38s 47.10s p=0.005 n=6
Total Time 139.14s (± 0.34%) 157.17s (± 0.25%) 🔻+18.03s (+12.96%) 156.62s 157.71s p=0.005 n=6
webpack - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 265,866 265,866 ~ ~ ~ p=1.000 n=6
Types 108,401 108,401 ~ ~ ~ p=1.000 n=6
Memory used 410,620k (± 0.02%) 410,632k (± 0.02%) ~ 410,518k 410,736k p=0.810 n=6
Parse Time 4.73s (± 0.82%) 4.76s (± 1.23%) ~ 4.71s 4.85s p=0.468 n=6
Bind Time 2.06s (± 1.18%) 2.07s (± 0.79%) ~ 2.05s 2.09s p=0.806 n=6
Check Time 21.03s (± 0.42%) 21.06s (± 0.63%) ~ 20.91s 21.29s p=0.630 n=6
Emit Time 0.00s 0.00s ~ ~ ~ p=1.000 n=6
Total Time 27.83s (± 0.36%) 27.89s (± 0.38%) ~ 27.74s 28.05s p=0.470 n=6
xstate-main - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 524,639 524,639 ~ ~ ~ p=1.000 n=6
Types 178,906 178,906 ~ ~ ~ p=1.000 n=6
Memory used 462,733k (± 0.02%) 462,688k (± 0.01%) ~ 462,636k 462,723k p=0.471 n=6
Parse Time 2.61s (± 0.29%) 2.60s (± 0.45%) ~ 2.58s 2.61s p=0.383 n=6
Bind Time 0.98s (± 0.52%) 0.98s (± 0.64%) ~ 0.97s 0.99s p=0.386 n=6
Check Time 15.31s (± 0.45%) 15.37s (± 0.47%) ~ 15.27s 15.48s p=0.170 n=6
Emit Time 0.00s 0.00s ~ ~ ~ p=1.000 n=6
Total Time 18.90s (± 0.40%) 18.95s (± 0.36%) ~ 18.85s 19.04s p=0.199 n=6
System info unknown
Hosts
  • node (v18.15.0, x64)
Scenarios
  • Compiler-Unions - node (v18.15.0, x64)
  • angular-1 - node (v18.15.0, x64)
  • mui-docs - node (v18.15.0, x64)
  • self-build-src - node (v18.15.0, x64)
  • self-build-src-public-api - node (v18.15.0, x64)
  • self-compiler - node (v18.15.0, x64)
  • ts-pre-modules - node (v18.15.0, x64)
  • vscode - node (v18.15.0, x64)
  • webpack - node (v18.15.0, x64)
  • xstate-main - node (v18.15.0, x64)
Benchmark Name Iterations
Current pr 6
Baseline baseline 6

Developer Information:

Download Benchmarks

@weswigham
Copy link
Member Author

Much better, no more check regressions, only emit ones, and those aughta get fixed with proper application of .calculatedFlags during the checker walk (though it's admittedly pretty odd that only some projects with emit show the perf regression - maybe those projects have settings such that they don't really look at NodeCheckFlags?).

@weswigham
Copy link
Member Author

@typescript-bot perf test this faster

@typescript-bot
Copy link
Collaborator

typescript-bot commented May 15, 2024

Starting jobs; this comment will be updated as builds start and complete.

Command Status Results
perf test this faster ✅ Started 👀 Results

@typescript-bot
Copy link
Collaborator

@weswigham
The results of the perf run you requested are in!

Here they are:

tsc

Comparison Report - baseline..pr
Metric baseline pr Delta Best Worst p-value
Compiler-Unions - node (v18.15.0, x64)
Errors 30 30 ~ ~ ~ p=1.000 n=6
Symbols 62,154 62,155 +1 (+ 0.00%) ~ ~ p=0.001 n=6
Types 50,248 50,251 +3 (+ 0.01%) ~ ~ p=0.001 n=6
Memory used 192,255k (± 0.03%) 191,841k (± 0.09%) -414k (- 0.22%) 191,736k 192,184k p=0.008 n=6
Parse Time 1.56s (± 1.55%) 1.57s (± 1.24%) ~ 1.53s 1.58s p=0.186 n=6
Bind Time 0.87s (± 0.87%) 0.86s (± 0.73%) ~ 0.85s 0.87s p=0.081 n=6
Check Time 11.31s (± 0.41%) 11.33s (± 0.50%) ~ 11.24s 11.39s p=0.629 n=6
Emit Time 3.15s (± 0.37%) 4.04s (± 0.26%) 🔻+0.90s (+28.48%) 4.03s 4.06s p=0.005 n=6
Total Time 16.88s (± 0.33%) 17.80s (± 0.25%) 🔻+0.92s (+ 5.46%) 17.74s 17.86s p=0.005 n=6
angular-1 - node (v18.15.0, x64)
Errors 5 5 ~ ~ ~ p=1.000 n=6
Symbols 944,110 944,377 +267 (+ 0.03%) ~ ~ p=0.001 n=6
Types 407,140 407,284 +144 (+ 0.04%) ~ ~ p=0.001 n=6
Memory used 1,222,053k (± 0.01%) 1,222,445k (± 0.00%) +392k (+ 0.03%) 1,222,405k 1,222,474k p=0.005 n=6
Parse Time 8.06s (± 0.37%) 8.11s (± 0.26%) +0.04s (+ 0.54%) 8.08s 8.13s p=0.019 n=6
Bind Time 2.24s (± 0.52%) 2.23s (± 0.55%) ~ 2.23s 2.26s p=0.389 n=6
Check Time 36.52s (± 0.59%) 36.51s (± 0.45%) ~ 36.35s 36.83s p=1.000 n=6
Emit Time 17.51s (± 0.42%) 20.36s (± 0.41%) 🔻+2.85s (+16.26%) 20.25s 20.50s p=0.005 n=6
Total Time 64.34s (± 0.42%) 67.21s (± 0.35%) 🔻+2.87s (+ 4.47%) 66.99s 67.66s p=0.005 n=6
mui-docs - node (v18.15.0, x64)
Errors 5 5 ~ ~ ~ p=1.000 n=6
Symbols 1,964,167 1,964,167 ~ ~ ~ p=1.000 n=6
Types 819,279 819,279 ~ ~ ~ p=1.000 n=6
Memory used 1,849,638k (± 0.00%) 1,849,675k (± 0.00%) +37k (+ 0.00%) 1,849,622k 1,849,709k p=0.045 n=6
Parse Time 6.77s (± 0.39%) 6.77s (± 0.31%) ~ 6.74s 6.80s p=0.805 n=6
Bind Time 2.29s 2.29s (± 0.62%) ~ 2.27s 2.31s p=1.000 n=6
Check Time 58.88s (± 0.29%) 58.69s (± 0.48%) ~ 58.30s 59.06s p=0.226 n=6
Emit Time 0.14s (± 3.60%) 0.14s (± 3.60%) ~ 0.14s 0.15s p=1.000 n=6
Total Time 68.08s (± 0.26%) 67.89s (± 0.42%) ~ 67.49s 68.25s p=0.298 n=6
self-build-src - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 1,221,216 1,221,413 +197 (+ 0.02%) ~ ~ p=0.001 n=6
Types 259,517 259,546 +29 (+ 0.01%) ~ ~ p=0.001 n=6
Memory used 2,388,033k (± 3.26%) 2,364,164k (± 2.56%) ~ 2,337,968k 2,487,875k p=0.471 n=6
Parse Time 6.02s (± 0.48%) 5.97s (± 1.19%) ~ 5.85s 6.04s p=0.228 n=6
Bind Time 2.26s (± 0.61%) 2.25s (± 0.78%) ~ 2.22s 2.27s p=0.126 n=6
Check Time 39.70s (± 0.47%) 39.92s (± 0.52%) ~ 39.61s 40.12s p=0.093 n=6
Emit Time 3.15s (± 1.92%) 3.13s (± 2.94%) ~ 3.04s 3.24s p=0.810 n=6
Total Time 51.15s (± 0.25%) 51.27s (± 0.53%) ~ 51.01s 51.61s p=0.575 n=6
self-build-src-public-api - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 1,221,216 1,221,413 +197 (+ 0.02%) ~ ~ p=0.001 n=6
Types 259,517 259,546 +29 (+ 0.01%) ~ ~ p=0.001 n=6
Memory used 2,413,884k (± 0.02%) 2,415,052k (± 0.02%) +1,168k (+ 0.05%) 2,414,104k 2,415,414k p=0.020 n=6
Parse Time 6.22s (± 1.38%) 6.18s (± 0.91%) ~ 6.11s 6.27s p=0.470 n=6
Bind Time 2.03s (± 0.60%) 2.03s (± 0.86%) ~ 2.01s 2.06s p=0.627 n=6
Check Time 40.23s (± 0.40%) 40.34s (± 0.34%) ~ 40.15s 40.56s p=0.199 n=6
Emit Time 3.18s (± 3.62%) 3.10s (± 2.72%) ~ 2.99s 3.25s p=0.378 n=6
Total Time 51.69s (± 0.54%) 51.67s (± 0.37%) ~ 51.43s 51.89s p=0.810 n=6
self-compiler - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 256,767 256,845 +78 (+ 0.03%) ~ ~ p=0.001 n=6
Types 104,587 104,616 +29 (+ 0.03%) ~ ~ p=0.001 n=6
Memory used 426,107k (± 0.03%) 426,389k (± 0.01%) +282k (+ 0.07%) 426,297k 426,434k p=0.008 n=6
Parse Time 4.18s (± 0.75%) 4.17s (± 0.66%) ~ 4.13s 4.20s p=0.871 n=6
Bind Time 1.62s (± 0.65%) 1.61s (± 1.73%) ~ 1.57s 1.64s p=1.000 n=6
Check Time 22.21s (± 0.34%) 22.30s (± 0.45%) ~ 22.19s 22.43s p=0.334 n=6
Emit Time 1.71s (± 1.67%) 1.73s (± 0.94%) ~ 1.71s 1.75s p=0.225 n=6
Total Time 29.71s (± 0.28%) 29.81s (± 0.29%) ~ 29.73s 29.92s p=0.148 n=6
ts-pre-modules - node (v18.15.0, x64)
Errors 35 35 ~ ~ ~ p=1.000 n=6
Symbols 224,575 224,575 ~ ~ ~ p=1.000 n=6
Types 93,785 93,785 ~ ~ ~ p=1.000 n=6
Memory used 369,905k (± 0.01%) 369,934k (± 0.02%) ~ 369,796k 370,009k p=0.378 n=6
Parse Time 2.83s (± 1.06%) 2.84s (± 0.82%) ~ 2.80s 2.86s p=0.683 n=6
Bind Time 1.59s (± 0.69%) 1.58s (± 0.77%) ~ 1.57s 1.60s p=0.507 n=6
Check Time 15.69s (± 0.38%) 15.70s (± 0.13%) ~ 15.68s 15.74s p=0.418 n=6
Emit Time 0.00s 0.00s ~ ~ ~ p=1.000 n=6
Total Time 20.11s (± 0.32%) 20.13s (± 0.16%) ~ 20.10s 20.18s p=0.421 n=6
vscode - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 2,823,941 2,825,923 +1,982 (+ 0.07%) ~ ~ p=0.001 n=6
Types 957,912 959,018 +1,106 (+ 0.12%) ~ ~ p=0.001 n=6
Memory used 2,996,271k (± 0.00%) 2,996,190k (± 0.00%) -81k (- 0.00%) 2,996,136k 2,996,267k p=0.020 n=6
Parse Time 16.94s (± 0.28%) 16.93s (± 0.24%) ~ 16.85s 16.97s p=0.628 n=6
Bind Time 5.01s (± 0.22%) 5.01s (± 0.38%) ~ 4.99s 5.04s p=0.933 n=6
Check Time 88.96s (± 0.40%) 88.54s (± 0.34%) ~ 88.13s 88.97s p=0.093 n=6
Emit Time 28.65s (± 0.59%) 41.55s (± 0.69%) 🔻+12.90s (+45.03%) 41.12s 41.78s p=0.005 n=6
Total Time 139.55s (± 0.31%) 152.03s (± 0.36%) 🔻+12.47s (+ 8.94%) 151.34s 152.55s p=0.005 n=6
webpack - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 265,866 265,866 ~ ~ ~ p=1.000 n=6
Types 108,401 108,401 ~ ~ ~ p=1.000 n=6
Memory used 410,525k (± 0.01%) 410,503k (± 0.02%) ~ 410,453k 410,609k p=0.423 n=6
Parse Time 3.84s (± 1.10%) 3.82s (± 1.39%) ~ 3.76s 3.90s p=0.468 n=6
Bind Time 1.66s (± 1.25%) 1.66s (± 0.62%) ~ 1.64s 1.67s p=0.803 n=6
Check Time 16.89s (± 0.35%) 16.92s (± 0.34%) ~ 16.86s 17.00s p=0.422 n=6
Emit Time 0.00s 0.00s ~ ~ ~ p=1.000 n=6
Total Time 22.39s (± 0.24%) 22.40s (± 0.28%) ~ 22.30s 22.47s p=0.747 n=6
xstate-main - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 524,639 524,639 ~ ~ ~ p=1.000 n=6
Types 178,906 178,906 ~ ~ ~ p=1.000 n=6
Memory used 462,713k (± 0.03%) 462,728k (± 0.01%) ~ 462,660k 462,817k p=0.521 n=6
Parse Time 3.89s (± 0.60%) 3.88s (± 0.41%) ~ 3.86s 3.90s p=0.415 n=6
Bind Time 1.45s (± 1.71%) 1.44s (± 1.73%) ~ 1.42s 1.49s p=0.516 n=6
Check Time 22.41s (± 0.75%) 22.65s (± 0.47%) +0.25s (+ 1.09%) 22.50s 22.82s p=0.013 n=6
Emit Time 0.00s 0.00s ~ ~ ~ p=1.000 n=6
Total Time 27.75s (± 0.50%) 27.97s (± 0.36%) +0.22s (+ 0.80%) 27.84s 28.13s p=0.020 n=6
System info unknown
Hosts
  • node (v18.15.0, x64)
Scenarios
  • Compiler-Unions - node (v18.15.0, x64)
  • angular-1 - node (v18.15.0, x64)
  • mui-docs - node (v18.15.0, x64)
  • self-build-src - node (v18.15.0, x64)
  • self-build-src-public-api - node (v18.15.0, x64)
  • self-compiler - node (v18.15.0, x64)
  • ts-pre-modules - node (v18.15.0, x64)
  • vscode - node (v18.15.0, x64)
  • webpack - node (v18.15.0, x64)
  • xstate-main - node (v18.15.0, x64)
Benchmark Name Iterations
Current pr 6
Baseline baseline 6

Developer Information:

Download Benchmarks

@weswigham
Copy link
Member Author

@typescript-bot perf test this faster

@typescript-bot
Copy link
Collaborator

typescript-bot commented May 15, 2024

Starting jobs; this comment will be updated as builds start and complete.

Command Status Results
perf test this faster ✅ Started 👀 Results

@typescript-bot
Copy link
Collaborator

@weswigham
The results of the perf run you requested are in!

Here they are:

tsc

Comparison Report - baseline..pr
Metric baseline pr Delta Best Worst p-value
Compiler-Unions - node (v18.15.0, x64)
Errors 30 30 ~ ~ ~ p=1.000 n=6
Symbols 62,154 62,155 +1 (+ 0.00%) ~ ~ p=0.001 n=6
Types 50,248 50,251 +3 (+ 0.01%) ~ ~ p=0.001 n=6
Memory used 193,418k (± 0.97%) 193,538k (± 0.98%) ~ 191,747k 195,341k p=0.378 n=6
Parse Time 1.29s (± 1.88%) 1.29s (± 1.88%) ~ 1.25s 1.31s p=1.000 n=6
Bind Time 0.72s 0.72s ~ ~ ~ p=1.000 n=6
Check Time 9.54s (± 0.19%) 9.61s (± 0.24%) +0.07s (+ 0.72%) 9.58s 9.64s p=0.005 n=6
Emit Time 2.65s (± 0.66%) 3.39s (± 0.97%) 🔻+0.74s (+28.09%) 3.33s 3.43s p=0.005 n=6
Total Time 14.20s (± 0.21%) 15.01s (± 0.41%) 🔻+0.81s (+ 5.72%) 14.90s 15.07s p=0.005 n=6
angular-1 - node (v18.15.0, x64)
Errors 5 5 ~ ~ ~ p=1.000 n=6
Symbols 944,110 944,377 +267 (+ 0.03%) ~ ~ p=0.001 n=6
Types 407,140 407,284 +144 (+ 0.04%) ~ ~ p=0.001 n=6
Memory used 1,222,132k (± 0.01%) 1,222,504k (± 0.01%) +372k (+ 0.03%) 1,222,362k 1,222,581k p=0.005 n=6
Parse Time 6.79s (± 0.50%) 6.78s (± 0.54%) ~ 6.74s 6.83s p=1.000 n=6
Bind Time 1.88s (± 0.56%) 1.87s (± 0.48%) ~ 1.86s 1.88s p=0.452 n=6
Check Time 31.32s (± 0.33%) 31.32s (± 0.50%) ~ 31.07s 31.46s p=1.000 n=6
Emit Time 14.74s (± 0.65%) 17.24s (± 0.76%) 🔻+2.50s (+16.96%) 17.10s 17.43s p=0.005 n=6
Total Time 54.73s (± 0.25%) 57.22s (± 0.42%) 🔻+2.49s (+ 4.56%) 56.82s 57.50s p=0.005 n=6
mui-docs - node (v18.15.0, x64)
Errors 5 5 ~ ~ ~ p=1.000 n=6
Symbols 1,964,176 1,964,176 ~ ~ ~ p=1.000 n=6
Types 819,283 819,283 ~ ~ ~ p=1.000 n=6
Memory used 1,849,541k (± 0.00%) 1,849,717k (± 0.00%) +176k (+ 0.01%) 1,849,696k 1,849,745k p=0.005 n=6
Parse Time 8.03s (± 0.19%) 8.04s (± 0.38%) ~ 8.01s 8.10s p=0.805 n=6
Bind Time 2.74s (± 1.34%) 2.73s (± 0.77%) ~ 2.71s 2.76s p=0.807 n=6
Check Time 69.29s (± 0.43%) 69.52s (± 0.37%) ~ 69.17s 69.78s p=0.378 n=6
Emit Time 0.17s (± 3.10%) 0.17s (± 2.42%) ~ 0.16s 0.17s p=0.595 n=6
Total Time 80.23s (± 0.39%) 80.46s (± 0.36%) ~ 80.08s 80.74s p=0.423 n=6
self-build-src - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 1,221,221 1,221,418 +197 (+ 0.02%) ~ ~ p=0.001 n=6
Types 259,523 259,552 +29 (+ 0.01%) ~ ~ p=0.001 n=6
Memory used 2,337,909k (± 0.02%) 2,363,689k (± 2.61%) ~ 2,337,890k 2,489,811k p=0.066 n=6
Parse Time 5.05s (± 0.93%) 5.02s (± 1.36%) ~ 4.95s 5.12s p=0.230 n=6
Bind Time 1.88s (± 0.64%) 1.88s (± 0.89%) ~ 1.85s 1.90s p=0.360 n=6
Check Time 33.87s (± 0.34%) 33.92s (± 0.31%) ~ 33.81s 34.06s p=0.575 n=6
Emit Time 2.63s (± 1.38%) 2.66s (± 2.63%) ~ 2.58s 2.76s p=0.470 n=6
Total Time 43.42s (± 0.30%) 43.50s (± 0.41%) ~ 43.28s 43.79s p=0.423 n=6
self-build-src-public-api - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 1,221,221 1,221,418 +197 (+ 0.02%) ~ ~ p=0.001 n=6
Types 259,523 259,552 +29 (+ 0.01%) ~ ~ p=0.001 n=6
Memory used 2,414,128k (± 0.01%) 2,414,654k (± 0.04%) ~ 2,413,128k 2,415,607k p=0.230 n=6
Parse Time 6.20s (± 0.64%) 6.16s (± 0.97%) ~ 6.09s 6.26s p=0.173 n=6
Bind Time 2.02s (± 1.07%) 2.03s (± 1.47%) ~ 2.00s 2.07s p=0.687 n=6
Check Time 40.12s (± 0.30%) 40.29s (± 0.27%) +0.17s (+ 0.43%) 40.15s 40.40s p=0.031 n=6
Emit Time 3.10s (± 2.29%) 3.11s (± 2.71%) ~ 3.02s 3.22s p=0.748 n=6
Total Time 51.45s (± 0.34%) 51.60s (± 0.29%) ~ 51.36s 51.80s p=0.230 n=6
self-compiler - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 256,767 256,845 +78 (+ 0.03%) ~ ~ p=0.001 n=6
Types 104,587 104,616 +29 (+ 0.03%) ~ ~ p=0.001 n=6
Memory used 426,103k (± 0.03%) 426,358k (± 0.01%) +255k (+ 0.06%) 426,325k 426,383k p=0.005 n=6
Parse Time 2.79s (± 0.30%) 2.78s (± 0.73%) ~ 2.75s 2.81s p=0.171 n=6
Bind Time 1.10s (± 1.06%) 1.11s +0.01s (+ 1.06%) ~ ~ p=0.028 n=6
Check Time 15.19s (± 0.27%) 15.30s (± 0.35%) +0.11s (+ 0.70%) 15.24s 15.40s p=0.006 n=6
Emit Time 1.16s (± 0.44%) 1.14s (± 0.71%) -0.02s (- 1.72%) 1.13s 1.15s p=0.004 n=6
Total Time 20.25s (± 0.23%) 20.34s (± 0.18%) +0.09s (+ 0.43%) 20.29s 20.40s p=0.010 n=6
ts-pre-modules - node (v18.15.0, x64)
Errors 35 35 ~ ~ ~ p=1.000 n=6
Symbols 224,575 224,575 ~ ~ ~ p=1.000 n=6
Types 93,785 93,785 ~ ~ ~ p=1.000 n=6
Memory used 369,831k (± 0.03%) 369,849k (± 0.04%) ~ 369,714k 370,098k p=1.000 n=6
Parse Time 3.52s (± 1.01%) 3.50s (± 1.02%) ~ 3.45s 3.55s p=0.518 n=6
Bind Time 1.93s (± 0.85%) 1.93s (± 1.24%) ~ 1.89s 1.96s p=1.000 n=6
Check Time 19.32s (± 0.30%) 19.34s (± 0.32%) ~ 19.26s 19.43s p=0.747 n=6
Emit Time 0.00s 0.00s ~ ~ ~ p=1.000 n=6
Total Time 24.77s (± 0.23%) 24.77s (± 0.25%) ~ 24.70s 24.87s p=0.630 n=6
vscode - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 2,823,941 2,825,923 +1,982 (+ 0.07%) ~ ~ p=0.001 n=6
Types 957,912 959,018 +1,106 (+ 0.12%) ~ ~ p=0.001 n=6
Memory used 2,996,264k (± 0.00%) 2,996,204k (± 0.00%) ~ 2,996,140k 2,996,288k p=0.065 n=6
Parse Time 16.96s (± 0.22%) 16.94s (± 0.32%) ~ 16.87s 17.03s p=0.226 n=6
Bind Time 5.00s (± 0.23%) 5.00s (± 0.21%) ~ 4.99s 5.02s p=0.615 n=6
Check Time 88.52s (± 0.15%) 88.43s (± 0.16%) ~ 88.24s 88.61s p=0.261 n=6
Emit Time 28.75s (± 1.17%) 41.63s (± 0.31%) 🔻+12.88s (+44.80%) 41.47s 41.81s p=0.005 n=6
Total Time 139.23s (± 0.31%) 152.00s (± 0.14%) 🔻+12.78s (+ 9.18%) 151.66s 152.31s p=0.005 n=6
webpack - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 265,866 265,866 ~ ~ ~ p=1.000 n=6
Types 108,401 108,401 ~ ~ ~ p=1.000 n=6
Memory used 410,600k (± 0.03%) 410,485k (± 0.01%) -115k (- 0.03%) 410,443k 410,539k p=0.031 n=6
Parse Time 3.85s (± 0.93%) 3.85s (± 0.65%) ~ 3.80s 3.87s p=0.806 n=6
Bind Time 1.67s (± 0.33%) 1.67s (± 1.35%) ~ 1.63s 1.70s p=0.663 n=6
Check Time 17.02s (± 0.32%) 16.92s (± 0.26%) -0.10s (- 0.59%) 16.86s 16.98s p=0.013 n=6
Emit Time 0.00s 0.00s ~ ~ ~ p=1.000 n=6
Total Time 22.53s (± 0.13%) 22.44s (± 0.20%) -0.09s (- 0.41%) 22.38s 22.49s p=0.008 n=6
xstate-main - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 524,639 524,639 ~ ~ ~ p=1.000 n=6
Types 178,906 178,906 ~ ~ ~ p=1.000 n=6
Memory used 462,686k (± 0.01%) 462,738k (± 0.02%) ~ 462,665k 462,889k p=0.128 n=6
Parse Time 3.11s (± 0.52%) 3.11s (± 0.47%) ~ 3.09s 3.13s p=0.870 n=6
Bind Time 1.16s (± 0.72%) 1.17s (± 0.44%) ~ 1.16s 1.17s p=0.533 n=6
Check Time 18.31s (± 0.39%) 18.42s (± 0.51%) +0.11s (+ 0.62%) 18.31s 18.56s p=0.045 n=6
Emit Time 0.00s 0.00s ~ ~ ~ p=1.000 n=6
Total Time 22.59s (± 0.26%) 22.70s (± 0.45%) ~ 22.56s 22.85s p=0.054 n=6
System info unknown
Hosts
  • node (v18.15.0, x64)
Scenarios
  • Compiler-Unions - node (v18.15.0, x64)
  • angular-1 - node (v18.15.0, x64)
  • mui-docs - node (v18.15.0, x64)
  • self-build-src - node (v18.15.0, x64)
  • self-build-src-public-api - node (v18.15.0, x64)
  • self-compiler - node (v18.15.0, x64)
  • ts-pre-modules - node (v18.15.0, x64)
  • vscode - node (v18.15.0, x64)
  • webpack - node (v18.15.0, x64)
  • xstate-main - node (v18.15.0, x64)
Benchmark Name Iterations
Current pr 6
Baseline baseline 6

Developer Information:

Download Benchmarks

@@ -5761,10 +5777,11 @@ export interface EmitResolver {
isValueAliasDeclaration(node: Node): boolean;
isReferencedAliasDeclaration(node: Node, checkChildren?: boolean): boolean;
isTopLevelValueImportEqualsWithEntityName(node: ImportEqualsDeclaration): boolean;
getNodeCheckFlags(node: Node): NodeCheckFlags;
hasNodeCheckFlag(node: Node, flags: LazyNodeCheckFlags): boolean;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All in all, I wonder how many other flag checks could be done like this (I did something similar for another flag which helped perf too)

@weswigham weswigham requested a review from jakebailey May 16, 2024 19:55
@jakebailey
Copy link
Member

For fun

@typescript-bot perf test this predictable

@typescript-bot
Copy link
Collaborator

typescript-bot commented May 16, 2024

Starting jobs; this comment will be updated as builds start and complete.

Command Status Results
perf test this predictable ✅ Started 👀 Results

@typescript-bot
Copy link
Collaborator

@jakebailey
The results of the perf run you requested are in!

Here they are:

tsc

Comparison Report - baseline..pr
Metric baseline pr Delta Best Worst p-value
Compiler-Unions - node (v18.15.0, x64)
Errors 30 30 ~ ~ ~ p=1.000 n=6
Symbols 62,154 62,154 ~ ~ ~ p=1.000 n=6
Types 50,248 50,248 ~ ~ ~ p=1.000 n=6
Memory used 192,229k 192,251k +22k (+ 0.01%) ~ ~ p=0.001 n=6
Parse Time 1.55s (± 0.26%) 1.55s (± 0.49%) ~ 1.54s 1.56s p=1.000 n=6
Bind Time 0.85s (± 0.60%) 0.85s (± 0.60%) ~ 0.85s 0.86s p=1.000 n=6
Check Time 11.10s (± 0.17%) 11.04s (± 0.08%) -0.06s (- 0.50%) 11.03s 11.05s p=0.005 n=6
Emit Time 3.38s (± 0.15%) 3.52s (± 0.12%) 🔻+0.14s (+ 4.09%) 3.52s 3.53s p=0.003 n=6
Total Time 16.88s (± 0.09%) 16.96s (± 0.07%) +0.09s (+ 0.51%) 16.95s 16.98s p=0.005 n=6
angular-1 - node (v18.15.0, x64)
Errors 5 5 ~ ~ ~ p=1.000 n=6
Symbols 944,110 944,114 +4 (+ 0.00%) ~ ~ p=0.001 n=6
Types 407,140 407,157 +17 (+ 0.00%) ~ ~ p=0.001 n=6
Memory used 1,222,410k (± 0.00%) 1,222,489k (± 0.00%) +79k (+ 0.01%) 1,222,377k 1,222,540k p=0.043 n=6
Parse Time 6.44s (± 0.15%) 6.44s (± 0.10%) ~ 6.43s 6.45s p=0.733 n=6
Bind Time 1.87s 1.87s (± 0.28%) ~ 1.87s 1.88s p=0.174 n=6
Check Time 33.39s (± 0.08%) 35.10s (± 3.14%) 🔻+1.71s (+ 5.12%) 33.66s 35.88s p=0.005 n=6
Emit Time 16.29s (± 0.75%) 15.60s (± 6.29%) ~ 14.95s 16.87s p=0.376 n=6
Total Time 57.99s (± 0.22%) 59.02s (± 0.23%) +1.02s (+ 1.76%) 58.85s 59.19s p=0.005 n=6
mui-docs - node (v18.15.0, x64)
Errors 5 5 ~ ~ ~ p=1.000 n=6
Symbols 1,964,188 1,964,188 ~ ~ ~ p=1.000 n=6
Types 819,291 819,291 ~ ~ ~ p=1.000 n=6
Memory used 1,849,834k (± 0.00%) 1,849,846k (± 0.01%) ~ 1,849,614k 1,850,270k p=0.687 n=6
Parse Time 10.01s (± 0.14%) 10.01s (± 0.15%) ~ 10.00s 10.04s p=0.243 n=6
Bind Time 2.71s (± 0.20%) 2.71s ~ ~ ~ p=0.071 n=6
Check Time 90.75s (± 0.11%) 92.96s (± 1.55%) ~ 90.49s 94.21s p=0.065 n=6
Emit Time 0.21s (± 5.69%) 0.33s (±53.32%) ~ 0.21s 0.56s p=0.209 n=6
Total Time 103.68s (± 0.09%) 106.02s (± 1.33%) ~ 103.42s 107.14s p=0.066 n=6
self-build-src - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 1,221,231 1,221,427 +196 (+ 0.02%) ~ ~ p=0.001 n=6
Types 259,523 259,552 +29 (+ 0.01%) ~ ~ p=0.001 n=6
Memory used 2,339,017k (± 0.00%) 2,339,875k (± 0.00%) +858k (+ 0.04%) 2,339,829k 2,340,029k p=0.004 n=6
Parse Time 6.96s (± 0.07%) 6.95s (± 0.08%) ~ 6.95s 6.96s p=0.640 n=6
Bind Time 2.58s 2.59s (± 0.47%) ~ 2.58s 2.61s p=0.176 n=6
Check Time 49.70s (± 0.10%) 49.86s (± 0.11%) +0.16s (+ 0.32%) 49.81s 49.95s p=0.005 n=6
Emit Time 4.25s (± 0.19%) 4.25s (± 0.25%) ~ 4.24s 4.27s p=0.510 n=6
Total Time 63.49s (± 0.06%) 63.67s (± 0.11%) +0.18s (+ 0.28%) 63.60s 63.77s p=0.005 n=6
self-build-src-public-api - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 1,221,231 1,221,427 +196 (+ 0.02%) ~ ~ p=0.001 n=6
Types 259,523 259,552 +29 (+ 0.01%) ~ ~ p=0.001 n=6
Memory used 2,413,286k (± 0.00%) 2,414,448k (± 0.00%) +1,162k (+ 0.05%) 2,414,337k 2,414,470k p=0.003 n=6
Parse Time 7.22s (± 0.25%) 7.21s (± 0.25%) ~ 7.19s 7.24s p=0.624 n=6
Bind Time 2.51s 2.53s +0.02s (+ 0.80%) ~ ~ p=0.001 n=6
Check Time 49.81s (± 0.09%) 50.08s (± 0.05%) +0.27s (+ 0.55%) 50.04s 50.12s p=0.005 n=6
Emit Time 4.29s (± 0.12%) 4.24s (± 0.53%) -0.04s (- 0.97%) 4.23s 4.29s p=0.023 n=6
Total Time 63.81s (± 0.08%) 64.05s (± 0.04%) +0.24s (+ 0.38%) 64.03s 64.09s p=0.005 n=6
self-compiler - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 256,768 256,845 +77 (+ 0.03%) ~ ~ p=0.001 n=6
Types 104,587 104,616 +29 (+ 0.03%) ~ ~ p=0.001 n=6
Memory used 425,968k 426,271k +303k (+ 0.07%) ~ ~ p=0.001 n=6
Parse Time 3.12s (± 0.33%) 3.12s (± 0.29%) ~ 3.11s 3.13s p=0.673 n=6
Bind Time 1.57s 1.57s ~ ~ ~ p=1.000 n=6
Check Time 17.91s (± 0.18%) 17.89s (± 0.13%) ~ 17.87s 17.92s p=0.191 n=6
Emit Time 1.37s 1.37s (± 0.30%) ~ 1.37s 1.38s p=0.405 n=6
Total Time 23.97s (± 0.16%) 23.95s (± 0.10%) ~ 23.93s 23.99s p=0.625 n=6
ts-pre-modules - node (v18.15.0, x64)
Errors 35 35 ~ ~ ~ p=1.000 n=6
Symbols 224,575 224,575 ~ ~ ~ p=1.000 n=6
Types 93,785 93,785 ~ ~ ~ p=1.000 n=6
Memory used 369,734k 369,760k +26k (+ 0.01%) ~ ~ p=0.001 n=6
Parse Time 2.81s (± 0.52%) 2.81s (± 0.29%) ~ 2.80s 2.82s p=0.508 n=6
Bind Time 1.42s (± 0.29%) 1.42s ~ ~ ~ p=0.405 n=6
Check Time 15.92s (± 0.08%) 16.04s (± 0.27%) +0.12s (+ 0.74%) 16.00s 16.12s p=0.005 n=6
Emit Time 0.00s 0.00s ~ ~ ~ p=1.000 n=6
Total Time 20.15s (± 0.12%) 20.27s (± 0.17%) +0.12s (+ 0.61%) 20.25s 20.34s p=0.005 n=6
vscode - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 2,824,634 2,824,662 +28 (+ 0.00%) ~ ~ p=0.001 n=6
Types 958,192 958,235 +43 (+ 0.00%) ~ ~ p=0.001 n=6
Memory used 3,000,142k 3,000,070k -72k (- 0.00%) ~ ~ p=0.001 n=6
Parse Time 14.49s (± 0.12%) 14.48s (± 0.16%) ~ 14.46s 14.51s p=0.328 n=6
Bind Time 4.29s (± 0.13%) 4.28s (± 0.10%) -0.02s (- 0.39%) 4.27s 4.28s p=0.003 n=6
Check Time 75.50s (± 0.09%) 76.02s (± 0.14%) +0.52s (+ 0.68%) 75.87s 76.13s p=0.005 n=6
Emit Time 20.11s (± 0.10%) 20.91s (± 0.43%) 🔻+0.81s (+ 4.00%) 20.85s 21.09s p=0.005 n=6
Total Time 114.40s (± 0.05%) 115.69s (± 0.08%) +1.30s (+ 1.13%) 115.57s 115.79s p=0.005 n=6
webpack - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 265,866 265,866 ~ ~ ~ p=1.000 n=6
Types 108,401 108,401 ~ ~ ~ p=1.000 n=6
Memory used 410,531k 410,541k +10k (+ 0.00%) ~ ~ p=0.001 n=6
Parse Time 3.84s (± 0.35%) 3.84s (± 0.32%) ~ 3.82s 3.85s p=0.564 n=6
Bind Time 1.71s (± 0.30%) 1.72s +0.01s (+ 0.39%) ~ ~ p=0.025 n=6
Check Time 16.81s (± 0.06%) 16.84s (± 0.07%) +0.02s (+ 0.14%) 16.82s 16.85s p=0.014 n=6
Emit Time 0.00s 0.00s ~ ~ ~ p=1.000 n=6
Total Time 22.37s (± 0.07%) 22.39s (± 0.07%) ~ 22.37s 22.41s p=0.062 n=6
xstate-main - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 524,639 524,639 ~ ~ ~ p=1.000 n=6
Types 178,906 178,906 ~ ~ ~ p=1.000 n=6
Memory used 462,635k 462,670k +35k (+ 0.01%) ~ ~ p=0.001 n=6
Parse Time 2.54s 2.54s (± 0.20%) ~ 2.54s 2.55s p=0.174 n=6
Bind Time 1.17s (± 0.44%) 1.17s (± 0.35%) ~ 1.16s 1.17s p=0.595 n=6
Check Time 15.33s (± 0.07%) 15.41s (± 0.09%) +0.08s (+ 0.51%) 15.39s 15.43s p=0.004 n=6
Emit Time 0.00s 0.00s ~ ~ ~ p=1.000 n=6
Total Time 19.04s (± 0.04%) 19.11s (± 0.04%) +0.08s (+ 0.42%) 19.11s 19.13s p=0.004 n=6
System info unknown
Hosts
  • node (v18.15.0, x64)
Scenarios
  • Compiler-Unions - node (v18.15.0, x64)
  • angular-1 - node (v18.15.0, x64)
  • mui-docs - node (v18.15.0, x64)
  • self-build-src - node (v18.15.0, x64)
  • self-build-src-public-api - node (v18.15.0, x64)
  • self-compiler - node (v18.15.0, x64)
  • ts-pre-modules - node (v18.15.0, x64)
  • vscode - node (v18.15.0, x64)
  • webpack - node (v18.15.0, x64)
  • xstate-main - node (v18.15.0, x64)
Benchmark Name Iterations
Current pr 6
Baseline baseline 6

tsserver

Comparison Report - baseline..pr
Metric baseline pr Delta Best Worst p-value
Compiler-UnionsTSServer - node (v18.15.0, x64)
Req 1 - updateOpen 2,342ms (± 0.26%) 2,344ms (± 0.20%) ~ 2,336ms 2,349ms p=0.629 n=6
Req 2 - geterr 5,100ms (± 0.09%) 5,120ms (± 0.09%) +20ms (+ 0.39%) 5,114ms 5,127ms p=0.005 n=6
Req 3 - references 282ms (± 0.73%) 283ms (± 0.53%) ~ 280ms 284ms p=0.345 n=6
Req 4 - navto 226ms (± 0.98%) 226ms (± 0.72%) ~ 225ms 229ms p=1.000 n=6
Req 5 - completionInfo count 1,357 1,357 ~ ~ ~ p=1.000 n=6
Req 5 - completionInfo 89ms (± 6.28%) 91ms (± 4.86%) ~ 82ms 93ms p=0.149 n=6
CompilerTSServer - node (v18.15.0, x64)
Req 1 - updateOpen 3,028ms (± 0.43%) 3,035ms (± 0.28%) ~ 3,024ms 3,048ms p=0.378 n=6
Req 2 - geterr 4,786ms (± 9.50%) 4,625ms (± 0.17%) ~ 4,614ms 4,632ms p=0.128 n=6
Req 3 - references 388ms (±10.88%) 374ms (± 8.80%) ~ 359ms 441ms p=0.803 n=6
Req 4 - navto 308ms (±10.55%) 298ms (±10.34%) ~ 278ms 338ms p=1.000 n=6
Req 5 - completionInfo count 1,519 1,519 ~ ~ ~ p=1.000 n=6
Req 5 - completionInfo 111ms (±11.39%) 107ms (±11.12%) ~ 99ms 122ms p=0.640 n=6
xstate-main-1-tsserver - node (v18.15.0, x64)
Req 1 - updateOpen 5,319ms (± 0.09%) 5,317ms (± 0.13%) ~ 5,306ms 5,324ms p=0.809 n=6
Req 2 - geterr 1,140ms (± 0.15%) 1,138ms (± 0.12%) ~ 1,136ms 1,140ms p=0.191 n=6
Req 3 - references 84ms (± 0.62%) 88ms 🔻+4ms (+ 5.18%) ~ ~ p=0.002 n=6
Req 4 - navto 449ms (± 0.19%) 447ms (± 0.09%) -1ms (- 0.30%) 447ms 448ms p=0.007 n=6
Req 5 - completionInfo count 3,413 3,413 ~ ~ ~ p=1.000 n=6
Req 5 - completionInfo 821ms (± 0.10%) 836ms (± 0.09%) +15ms (+ 1.87%) 835ms 837ms p=0.004 n=6
System info unknown
Hosts
  • node (v18.15.0, x64)
Scenarios
  • CompilerTSServer - node (v18.15.0, x64)
  • Compiler-UnionsTSServer - node (v18.15.0, x64)
  • xstate-main-1-tsserver - node (v18.15.0, x64)
Benchmark Name Iterations
Current pr 6
Baseline baseline 6

startup

Comparison Report - baseline..pr
Metric baseline pr Delta Best Worst p-value
tsc-startup - node (v18.15.0, x64)
Execution time 166.41ms (± 0.10%) 168.44ms (± 0.13%) +2.03ms (+ 1.22%) 167.49ms 173.47ms p=0.000 n=600
tsserver-startup - node (v18.15.0, x64)
Execution time 309.42ms (± 0.27%) 309.58ms (± 0.24%) +0.16ms (+ 0.05%) 303.31ms 315.62ms p=0.000 n=600
tsserverlibrary-startup - node (v18.15.0, x64)
Execution time 244.98ms (± 0.09%) 246.22ms (± 0.13%) +1.24ms (+ 0.51%) 245.31ms 253.09ms p=0.000 n=600
typescript-startup - node (v18.15.0, x64)
Execution time 245.85ms (± 0.12%) 246.13ms (± 0.08%) +0.29ms (+ 0.12%) 245.19ms 247.61ms p=0.000 n=600
System info unknown
Hosts
  • node (v18.15.0, x64)
Scenarios
  • tsc-startup - node (v18.15.0, x64)
  • tsserver-startup - node (v18.15.0, x64)
  • tsserverlibrary-startup - node (v18.15.0, x64)
  • typescript-startup - node (v18.15.0, x64)
Benchmark Name Iterations
Current pr 6
Baseline baseline 6

Developer Information:

Download Benchmarks

@jakebailey
Copy link
Member

@typescript-bot perf test this predictable

@typescript-bot
Copy link
Collaborator

typescript-bot commented May 20, 2024

Starting jobs; this comment will be updated as builds start and complete.

Command Status Results
perf test this predictable ✅ Started 👀 Results

@typescript-bot
Copy link
Collaborator

@jakebailey
The results of the perf run you requested are in!

Here they are:

tsc

Comparison Report - baseline..pr
Metric baseline pr Delta Best Worst p-value
Compiler-Unions - node (v18.15.0, x64)
Errors 30 30 ~ ~ ~ p=1.000 n=6
Symbols 62,154 62,154 ~ ~ ~ p=1.000 n=6
Types 50,248 50,248 ~ ~ ~ p=1.000 n=6
Memory used 192,230k 192,252k +22k (+ 0.01%) ~ ~ p=0.001 n=6
Parse Time 1.92s (± 0.42%) 1.92s (± 0.43%) ~ 1.91s 1.93s p=0.718 n=6
Bind Time 1.05s (± 0.49%) 1.06s (± 0.84%) ~ 1.05s 1.07s p=0.190 n=6
Check Time 13.74s (± 0.04%) 13.73s (± 0.14%) ~ 13.71s 13.76s p=0.154 n=6
Emit Time 4.21s (± 0.12%) 4.37s (± 0.17%) +0.16s (+ 3.84%) 4.36s 4.38s p=0.004 n=6
Total Time 20.92s (± 0.08%) 21.08s (± 0.15%) +0.16s (+ 0.75%) 21.06s 21.14s p=0.005 n=6
angular-1 - node (v18.15.0, x64)
Errors 5 5 ~ ~ ~ p=1.000 n=6
Symbols 944,110 944,114 +4 (+ 0.00%) ~ ~ p=0.001 n=6
Types 407,140 407,157 +17 (+ 0.00%) ~ ~ p=0.001 n=6
Memory used 1,222,304k (± 0.02%) 1,222,425k (± 0.01%) ~ 1,222,186k 1,222,604k p=0.297 n=6
Parse Time 7.70s (± 0.13%) 7.69s (± 0.10%) ~ 7.68s 7.70s p=0.300 n=6
Bind Time 2.24s (± 0.18%) 2.24s (± 0.23%) ~ 2.24s 2.25s p=0.595 n=6
Check Time 39.66s (± 2.13%) 40.00s (± 2.78%) ~ 39.25s 41.55s p=0.128 n=6
Emit Time 18.44s (± 5.46%) 19.02s (± 6.09%) ~ 17.77s 20.12s p=0.378 n=6
Total Time 68.03s (± 1.31%) 68.95s (± 0.98%) +0.92s (+ 1.35%) 67.60s 69.31s p=0.030 n=6
mui-docs - node (v18.15.0, x64)
Errors 5 5 ~ ~ ~ p=1.000 n=6
Symbols 1,979,156 1,979,156 ~ ~ ~ p=1.000 n=6
Types 882,049 882,049 ~ ~ ~ p=1.000 n=6
Memory used 1,885,172k (± 0.01%) 1,885,324k (± 0.01%) ~ 1,885,214k 1,885,545k p=0.093 n=6
Parse Time 8.12s (± 0.14%) 8.12s (± 0.22%) ~ 8.09s 8.14s p=0.402 n=6
Bind Time 2.21s 2.21s (± 0.18%) ~ 2.21s 2.22s p=0.405 n=6
Check Time 77.50s (± 2.00%) 77.22s (± 1.87%) ~ 76.12s 79.24s p=0.936 n=6
Emit Time 0.22s (±58.44%) 0.17s (± 5.72%) ~ 0.16s 0.19s p=0.599 n=6
Total Time 88.06s (± 1.84%) 87.72s (± 1.66%) ~ 86.60s 89.75s p=1.000 n=6
self-build-src - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 1,222,650 1,222,846 +196 (+ 0.02%) ~ ~ p=0.001 n=6
Types 259,947 259,976 +29 (+ 0.01%) ~ ~ p=0.001 n=6
Memory used 2,339,801k (± 0.00%) 2,341,049k (± 0.00%) +1,248k (+ 0.05%) 2,341,028k 2,341,139k p=0.004 n=6
Parse Time 6.95s (± 0.28%) 6.97s (± 0.12%) +0.02s (+ 0.36%) 6.96s 6.98s p=0.026 n=6
Bind Time 2.56s (± 0.32%) 2.57s (± 0.16%) +0.01s (+ 0.20%) 2.56s 2.57s p=0.047 n=6
Check Time 49.57s (± 0.02%) 49.92s (± 0.08%) +0.35s (+ 0.71%) 49.88s 49.99s p=0.005 n=6
Emit Time 4.25s (± 0.18%) 4.26s (± 0.10%) +0.01s (+ 0.31%) 4.26s 4.27s p=0.010 n=6
Total Time 63.37s (± 0.02%) 63.75s (± 0.07%) +0.38s (+ 0.59%) 63.69s 63.81s p=0.005 n=6
self-build-src-public-api - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 1,222,650 1,222,846 +196 (+ 0.02%) ~ ~ p=0.001 n=6
Types 259,947 259,976 +29 (+ 0.01%) ~ ~ p=0.001 n=6
Memory used 2,414,188k (± 0.00%) 2,415,473k +1,286k (+ 0.05%) ~ ~ p=0.003 n=6
Parse Time 4.87s (± 0.11%) 4.87s (± 0.17%) ~ 4.87s 4.89s p=0.752 n=6
Bind Time 1.70s (± 0.24%) 1.70s ~ ~ ~ p=0.405 n=6
Check Time 34.32s (± 0.06%) 34.53s (± 0.06%) +0.22s (+ 0.64%) 34.51s 34.56s p=0.005 n=6
Emit Time 2.90s (± 0.70%) 2.92s (± 0.14%) ~ 2.92s 2.93s p=0.056 n=6
Total Time 43.78s (± 0.03%) 44.02s (± 0.04%) +0.24s (+ 0.56%) 44.00s 44.05s p=0.005 n=6
self-compiler - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 257,595 257,672 +77 (+ 0.03%) ~ ~ p=0.001 n=6
Types 104,862 104,891 +29 (+ 0.03%) ~ ~ p=0.001 n=6
Memory used 426,431k 426,585k +154k (+ 0.04%) ~ ~ p=0.001 n=6
Parse Time 3.12s (± 0.39%) 3.12s (± 0.31%) ~ 3.11s 3.13s p=0.498 n=6
Bind Time 1.57s 1.55s (± 0.33%) -0.02s (- 1.06%) 1.55s 1.56s p=0.002 n=6
Check Time 17.87s (± 0.05%) 17.95s (± 0.05%) +0.07s (+ 0.41%) 17.93s 17.95s p=0.004 n=6
Emit Time 1.37s 1.37s ~ ~ ~ p=1.000 n=6
Total Time 23.92s (± 0.06%) 24.00s (± 0.07%) +0.07s (+ 0.29%) 23.97s 24.01s p=0.005 n=6
ts-pre-modules - node (v18.15.0, x64)
Errors 35 35 ~ ~ ~ p=1.000 n=6
Symbols 224,575 224,575 ~ ~ ~ p=1.000 n=6
Types 93,785 93,785 ~ ~ ~ p=1.000 n=6
Memory used 369,748k 369,762k +14k (+ 0.00%) ~ ~ p=0.001 n=6
Parse Time 2.33s (± 0.17%) 2.33s ~ ~ ~ p=0.405 n=6
Bind Time 1.18s (± 0.35%) 1.18s (± 0.35%) ~ 1.18s 1.19s p=1.000 n=6
Check Time 13.54s (± 0.10%) 13.63s (± 0.31%) +0.09s (+ 0.68%) 13.59s 13.69s p=0.005 n=6
Emit Time 0.00s 0.00s ~ ~ ~ p=1.000 n=6
Total Time 17.05s (± 0.07%) 17.15s (± 0.24%) +0.10s (+ 0.56%) 17.11s 17.21s p=0.005 n=6
vscode - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 2,825,722 2,825,750 +28 (+ 0.00%) ~ ~ p=0.001 n=6
Types 958,536 958,579 +43 (+ 0.00%) ~ ~ p=0.001 n=6
Memory used 3,000,779k 3,000,741k -38k (- 0.00%) ~ ~ p=0.001 n=6
Parse Time 12.14s (± 0.11%) 12.14s (± 0.23%) ~ 12.12s 12.19s p=0.459 n=6
Bind Time 3.59s (± 0.14%) 3.59s (± 0.11%) ~ 3.59s 3.60s p=0.114 n=6
Check Time 65.14s (± 0.10%) 65.67s (± 0.14%) +0.53s (+ 0.82%) 65.52s 65.77s p=0.005 n=6
Emit Time 16.94s (± 0.09%) 17.79s (± 0.24%) 🔻+0.85s (+ 5.03%) 17.75s 17.85s p=0.005 n=6
Total Time 97.81s (± 0.08%) 99.19s (± 0.12%) +1.38s (+ 1.42%) 98.99s 99.32s p=0.005 n=6
webpack - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 265,866 265,866 ~ ~ ~ p=1.000 n=6
Types 108,401 108,401 ~ ~ ~ p=1.000 n=6
Memory used 410,533k 410,540k +7k (+ 0.00%) ~ ~ p=0.001 n=6
Parse Time 3.84s (± 0.20%) 3.84s (± 0.35%) ~ 3.82s 3.86s p=0.548 n=6
Bind Time 1.72s (± 0.30%) 1.72s (± 0.24%) ~ 1.71s 1.72s p=0.595 n=6
Check Time 16.79s (± 0.08%) 16.80s (± 0.04%) ~ 16.79s 16.81s p=0.137 n=6
Emit Time 0.00s 0.00s ~ ~ ~ p=1.000 n=6
Total Time 22.34s (± 0.04%) 22.36s (± 0.04%) +0.02s (+ 0.07%) 22.35s 22.37s p=0.015 n=6
xstate-main - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 524,654 524,654 ~ ~ ~ p=1.000 n=6
Types 178,920 178,920 ~ ~ ~ p=1.000 n=6
Memory used 462,627k 462,661k +34k (+ 0.01%) ~ ~ p=0.001 n=6
Parse Time 3.78s (± 0.24%) 3.78s (± 0.36%) ~ 3.77s 3.80s p=0.803 n=6
Bind Time 1.72s 1.72s ~ ~ ~ p=1.000 n=6
Check Time 22.32s (± 0.05%) 22.41s (± 0.07%) +0.09s (+ 0.39%) 22.39s 22.43s p=0.005 n=6
Emit Time 0.00s 0.00s ~ ~ ~ p=1.000 n=6
Total Time 27.82s (± 0.05%) 27.91s (± 0.08%) +0.09s (+ 0.32%) 27.88s 27.95s p=0.005 n=6
System info unknown
Hosts
  • node (v18.15.0, x64)
Scenarios
  • Compiler-Unions - node (v18.15.0, x64)
  • angular-1 - node (v18.15.0, x64)
  • mui-docs - node (v18.15.0, x64)
  • self-build-src - node (v18.15.0, x64)
  • self-build-src-public-api - node (v18.15.0, x64)
  • self-compiler - node (v18.15.0, x64)
  • ts-pre-modules - node (v18.15.0, x64)
  • vscode - node (v18.15.0, x64)
  • webpack - node (v18.15.0, x64)
  • xstate-main - node (v18.15.0, x64)
Benchmark Name Iterations
Current pr 6
Baseline baseline 6

tsserver

Comparison Report - baseline..pr
Metric baseline pr Delta Best Worst p-value
Compiler-UnionsTSServer - node (v18.15.0, x64)
Req 1 - updateOpen 3,490ms (± 0.33%) 3,387ms (± 8.01%) ~ 2,833ms 3,521ms p=1.000 n=6
Req 2 - geterr 7,550ms (± 0.27%) 7,321ms (± 8.02%) ~ 6,122ms 7,577ms p=0.748 n=6
Req 3 - references 415ms (± 0.20%) 418ms (± 0.60%) +3ms (+ 0.72%) 416ms 422ms p=0.011 n=6
Req 4 - navto 340ms (± 0.48%) 340ms (± 0.81%) ~ 336ms 342ms p=0.849 n=6
Req 5 - completionInfo count 1,357 1,357 ~ ~ ~ p=1.000 n=6
Req 5 - completionInfo 124ms (± 5.07%) 127ms (± 6.41%) ~ 121ms 138ms p=0.437 n=6
CompilerTSServer - node (v18.15.0, x64)
Req 1 - updateOpen 3,030ms (± 0.34%) 3,019ms (± 0.47%) ~ 3,004ms 3,034ms p=0.378 n=6
Req 2 - geterr 5,155ms (±11.64%) 4,617ms (± 0.19%) ~ 4,609ms 4,634ms p=0.470 n=6
Req 3 - references 416ms (±10.20%) 388ms (±10.87%) ~ 360ms 443ms p=0.128 n=6
Req 4 - navto 312ms (± 8.76%) 315ms (± 9.36%) ~ 278ms 339ms p=0.622 n=6
Req 5 - completionInfo count 1,519 1,519 ~ ~ ~ p=1.000 n=6
Req 5 - completionInfo 111ms (±11.39%) 114ms (±10.29%) ~ 99ms 122ms p=0.859 n=6
xstate-main-1-tsserver - node (v18.15.0, x64)
Req 1 - updateOpen 7,230ms (±10.08%) 6,485ms (± 4.91%) 🟩-745ms (-10.30%) 6,343ms 7,136ms p=0.045 n=6
Req 2 - geterr 1,584ms (±10.65%) 1,418ms (± 9.44%) 🟩-166ms (-10.47%) 1,359ms 1,691ms p=0.044 n=6
Req 3 - references 124ms (± 7.89%) 126ms (±10.36%) ~ 109ms 135ms p=0.209 n=6
Req 4 - navto 671ms (± 0.12%) 653ms (± 7.50%) ~ 553ms 675ms p=0.373 n=6
Req 5 - completionInfo count 3,413 3,413 ~ ~ ~ p=1.000 n=6
Req 5 - completionInfo 1,221ms (± 0.17%) 1,206ms (± 8.00%) ~ 1,009ms 1,250ms p=0.063 n=6
System info unknown
Hosts
  • node (v18.15.0, x64)
Scenarios
  • CompilerTSServer - node (v18.15.0, x64)
  • Compiler-UnionsTSServer - node (v18.15.0, x64)
  • xstate-main-1-tsserver - node (v18.15.0, x64)
Benchmark Name Iterations
Current pr 6
Baseline baseline 6

startup

Comparison Report - baseline..pr
Metric baseline pr Delta Best Worst p-value
tsc-startup - node (v18.15.0, x64)
Execution time 165.43ms (± 0.10%) 167.45ms (± 0.13%) +2.01ms (+ 1.22%) 166.79ms 172.88ms p=0.000 n=600
tsserver-startup - node (v18.15.0, x64)
Execution time 307.63ms (± 0.12%) 307.64ms (± 0.08%) ~ 304.94ms 309.89ms p=0.693 n=600
tsserverlibrary-startup - node (v18.15.0, x64)
Execution time 296.91ms (± 0.26%) 298.42ms (± 0.31%) +1.51ms (+ 0.51%) 291.62ms 320.74ms p=0.000 n=600
typescript-startup - node (v18.15.0, x64)
Execution time 300.23ms (± 0.26%) 300.41ms (± 0.27%) +0.18ms (+ 0.06%) 293.69ms 309.01ms p=0.000 n=600
System info unknown
Hosts
  • node (v18.15.0, x64)
Scenarios
  • tsc-startup - node (v18.15.0, x64)
  • tsserver-startup - node (v18.15.0, x64)
  • tsserverlibrary-startup - node (v18.15.0, x64)
  • typescript-startup - node (v18.15.0, x64)
Benchmark Name Iterations
Current pr 6
Baseline baseline 6

Developer Information:

Download Benchmarks

@jakebailey
Copy link
Member

The perf results are still seemingly showing a regression in emit time on various projects. Not sure what's going on there but I'll try and profile...

@weswigham
Copy link
Member Author

I mentioned it in our meeting, but the only things not noCheck-flagged (aside from change the format of the API function from "get flags" to "has flag") at this point are this range and this range, each of which solve a bug that only crops up when you do out of order checking (namely: cjs merge symbols not manifesting early enough and type only imports not being marked). You could noCheck-flag both of them if you really wanted, but I feel like it'd make using those API methods a bit footgun-y. But if one of them has outsized (though not that outsized) performance impact, I suppose we have little choice?

@@ -924,6 +930,14 @@ export function emitFiles(resolver: EmitResolver, host: EmitHost, targetSourceFi
forEachChild(node, collectLinkedAliases);
}

function markLinkedReferences(file: SourceFile) {
ts.forEachChildRecursively(file, n => {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can this instead check if file doesnt have TypeChecked flag and do this work and then we can get rid of getDiagnostics all together from getEmitResolver()

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not quite - the TypeChecked flag is, ironically, set on files whose type checking is skipped. Maybe it's a misnomer, but it really just means "Has checkSourceFile been called on this node?". I get your drift though and, yeah, we could probably do a bit of refactoring to use a node flag to track the diagnostic walk state of the containing file. I'll probably put that up as a separate cleanup, though. I also think I want to move around/refactor the API logic for collectLinkedAliases so the API is harder to misuse, and will probably change markLinkedReferences to match, so I'll put all those nonfunctional changes in a separate PR (which should have a 0-test diff).

@jakebailey
Copy link
Member

I'm having a pretty hard time figuring out where the slowdown is; the vscode benchmark takes a long time so two runs are just not comparable, even in predictable mode on a perf locked core.

@typescript-bot perf test this faster

@typescript-bot
Copy link
Collaborator

typescript-bot commented May 24, 2024

Starting jobs; this comment will be updated as builds start and complete.

Command Status Results
perf test this faster ✅ Started 👀 Results

@typescript-bot
Copy link
Collaborator

@jakebailey
The results of the perf run you requested are in!

Here they are:

tsc

Comparison Report - baseline..pr
Metric baseline pr Delta Best Worst p-value
Compiler-Unions - node (v18.15.0, x64)
Errors 30 30 ~ ~ ~ p=1.000 n=6
Symbols 62,154 62,154 ~ ~ ~ p=1.000 n=6
Types 50,248 50,248 ~ ~ ~ p=1.000 n=6
Memory used 192,872k (± 0.79%) 192,460k (± 0.14%) ~ 192,227k 192,869k p=0.298 n=6
Parse Time 1.54s (± 1.90%) 1.56s (± 0.94%) ~ 1.54s 1.58s p=0.292 n=6
Bind Time 0.87s (± 0.63%) 0.87s (± 1.21%) ~ 0.85s 0.88s p=1.000 n=6
Check Time 11.30s (± 0.41%) 11.35s (± 0.37%) ~ 11.29s 11.40s p=0.106 n=6
Emit Time 3.15s (± 0.67%) 3.28s (± 1.10%) 🔻+0.13s (+ 4.18%) 3.25s 3.34s p=0.005 n=6
Total Time 16.86s (± 0.29%) 17.05s (± 0.21%) +0.19s (+ 1.14%) 17.01s 17.10s p=0.005 n=6
angular-1 - node (v18.15.0, x64)
Errors 5 5 ~ ~ ~ p=1.000 n=6
Symbols 944,110 944,114 +4 (+ 0.00%) ~ ~ p=0.001 n=6
Types 407,140 407,157 +17 (+ 0.00%) ~ ~ p=0.001 n=6
Memory used 1,222,082k (± 0.00%) 1,222,179k (± 0.00%) +97k (+ 0.01%) 1,222,119k 1,222,271k p=0.008 n=6
Parse Time 6.79s (± 0.37%) 6.74s (± 0.57%) ~ 6.71s 6.80s p=0.052 n=6
Bind Time 1.88s (± 0.73%) 1.87s (± 0.44%) ~ 1.87s 1.89s p=1.000 n=6
Check Time 31.40s (± 0.40%) 31.31s (± 0.22%) ~ 31.26s 31.45s p=0.147 n=6
Emit Time 14.85s (± 1.05%) 15.28s (± 0.47%) +0.42s (+ 2.86%) 15.21s 15.38s p=0.005 n=6
Total Time 54.91s (± 0.42%) 55.20s (± 0.23%) +0.29s (+ 0.52%) 55.05s 55.38s p=0.045 n=6
mui-docs - node (v18.15.0, x64)
Errors 5 5 ~ ~ ~ p=1.000 n=6
Symbols 1,982,493 1,982,493 ~ ~ ~ p=1.000 n=6
Types 882,095 882,095 ~ ~ ~ p=1.000 n=6
Memory used 1,886,152k (± 0.00%) 1,886,171k (± 0.00%) ~ 1,886,139k 1,886,233k p=0.471 n=6
Parse Time 6.79s (± 0.37%) 6.80s (± 0.39%) ~ 6.76s 6.83s p=0.809 n=6
Bind Time 2.30s (± 0.84%) 2.32s (± 1.35%) ~ 2.28s 2.36s p=0.292 n=6
Check Time 60.82s (± 0.51%) 60.45s (± 0.54%) ~ 60.06s 60.85s p=0.173 n=6
Emit Time 0.14s (± 2.88%) 0.14s ~ ~ ~ p=0.405 n=6
Total Time 70.05s (± 0.46%) 69.71s (± 0.50%) ~ 69.25s 70.15s p=0.230 n=6
self-build-src - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 1,225,003 1,225,199 +196 (+ 0.02%) ~ ~ p=0.001 n=6
Types 260,663 260,692 +29 (+ 0.01%) ~ ~ p=0.001 n=6
Memory used 2,341,920k (± 0.05%) 2,342,420k (± 0.04%) ~ 2,341,598k 2,344,070k p=0.471 n=6
Parse Time 4.95s (± 0.76%) 4.97s (± 0.53%) ~ 4.94s 5.00s p=0.173 n=6
Bind Time 1.91s (± 0.43%) 1.90s (± 0.54%) ~ 1.89s 1.92s p=0.546 n=6
Check Time 33.84s (± 0.36%) 33.96s (± 0.33%) ~ 33.81s 34.11s p=0.148 n=6
Emit Time 2.63s (± 1.51%) 2.69s (± 2.22%) ~ 2.58s 2.74s p=0.092 n=6
Total Time 43.35s (± 0.25%) 43.53s (± 0.39%) ~ 43.31s 43.73s p=0.128 n=6
self-build-src-public-api - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 1,225,003 1,225,199 +196 (+ 0.02%) ~ ~ p=0.001 n=6
Types 260,663 260,692 +29 (+ 0.01%) ~ ~ p=0.001 n=6
Memory used 2,443,367k (± 2.52%) 2,443,368k (± 2.54%) ~ 2,417,448k 2,570,158k p=0.810 n=6
Parse Time 5.18s (± 1.19%) 5.19s (± 0.50%) ~ 5.16s 5.23s p=0.810 n=6
Bind Time 1.69s (± 0.45%) 1.69s (± 0.37%) ~ 1.68s 1.70s p=0.718 n=6
Check Time 34.32s (± 0.37%) 34.32s (± 0.32%) ~ 34.15s 34.43s p=1.000 n=6
Emit Time 2.65s (± 2.24%) 2.71s (± 1.81%) ~ 2.62s 2.77s p=0.126 n=6
Total Time 43.87s (± 0.27%) 43.95s (± 0.29%) ~ 43.74s 44.09s p=0.378 n=6
self-compiler - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 257,738 257,815 +77 (+ 0.03%) ~ ~ p=0.001 n=6
Types 104,935 104,964 +29 (+ 0.03%) ~ ~ p=0.001 n=6
Memory used 426,724k (± 0.03%) 427,059k (± 0.01%) +335k (+ 0.08%) 427,010k 427,128k p=0.005 n=6
Parse Time 3.38s (± 0.81%) 3.33s (± 0.59%) -0.05s (- 1.38%) 3.31s 3.36s p=0.019 n=6
Bind Time 1.32s (± 0.62%) 1.33s (± 1.11%) ~ 1.30s 1.34s p=0.230 n=6
Check Time 18.03s (± 0.25%) 18.00s (± 0.19%) ~ 17.95s 18.04s p=0.335 n=6
Emit Time 1.33s (± 0.88%) 1.33s (± 1.10%) ~ 1.31s 1.35s p=1.000 n=6
Total Time 24.06s (± 0.22%) 23.99s (± 0.19%) ~ 23.91s 24.03s p=0.065 n=6
ts-pre-modules - node (v18.15.0, x64)
Errors 35 35 ~ ~ ~ p=1.000 n=6
Symbols 224,575 224,575 ~ ~ ~ p=1.000 n=6
Types 93,785 93,785 ~ ~ ~ p=1.000 n=6
Memory used 369,809k (± 0.02%) 369,964k (± 0.02%) +155k (+ 0.04%) 369,855k 370,064k p=0.031 n=6
Parse Time 2.85s (± 0.53%) 2.83s (± 0.85%) ~ 2.79s 2.85s p=0.141 n=6
Bind Time 1.58s (± 0.96%) 1.59s (± 0.87%) ~ 1.57s 1.60s p=1.000 n=6
Check Time 15.68s (± 0.61%) 15.74s (± 0.23%) ~ 15.69s 15.77s p=0.064 n=6
Emit Time 0.00s 0.00s ~ ~ ~ p=1.000 n=6
Total Time 20.11s (± 0.57%) 20.15s (± 0.23%) ~ 20.10s 20.21s p=0.228 n=6
vscode - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 2,836,288 2,836,288 ~ ~ ~ p=1.000 n=6
Types 961,202 961,202 ~ ~ ~ p=1.000 n=6
Memory used 3,008,664k (± 0.00%) 3,008,698k (± 0.00%) ~ 3,008,675k 3,008,723k p=0.230 n=6
Parse Time 17.06s (± 0.39%) 17.00s (± 0.20%) -0.07s (- 0.39%) 16.96s 17.06s p=0.029 n=6
Bind Time 5.06s (± 0.36%) 5.08s (± 0.82%) ~ 5.05s 5.16s p=0.357 n=6
Check Time 89.11s (± 0.25%) 90.75s (± 2.67%) ~ 88.96s 94.10s p=0.378 n=6
Emit Time 28.66s (± 0.53%) 27.15s (± 8.87%) ~ 24.04s 28.95s p=0.575 n=6
Total Time 139.88s (± 0.17%) 139.97s (± 0.18%) ~ 139.56s 140.19s p=0.575 n=6
webpack - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 266,087 266,087 ~ ~ ~ p=1.000 n=6
Types 108,518 108,518 ~ ~ ~ p=1.000 n=6
Memory used 410,898k (± 0.02%) 410,869k (± 0.01%) ~ 410,836k 410,894k p=0.687 n=6
Parse Time 4.78s (± 0.43%) 4.75s (± 0.85%) ~ 4.69s 4.81s p=0.077 n=6
Bind Time 2.07s (± 1.12%) 2.07s (± 0.95%) ~ 2.04s 2.09s p=0.746 n=6
Check Time 21.04s (± 0.43%) 21.02s (± 0.35%) ~ 20.91s 21.10s p=0.573 n=6
Emit Time 0.00s 0.00s ~ ~ ~ p=1.000 n=6
Total Time 27.90s (± 0.35%) 27.83s (± 0.31%) ~ 27.73s 27.98s p=0.226 n=6
xstate-main - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 535,745 535,745 ~ ~ ~ p=1.000 n=6
Types 182,753 182,753 ~ ~ ~ p=1.000 n=6
Memory used 470,423k (± 0.01%) 470,432k (± 0.01%) ~ 470,360k 470,525k p=0.689 n=6
Parse Time 3.96s (± 0.39%) 3.95s (± 0.50%) ~ 3.92s 3.97s p=0.460 n=6
Bind Time 1.48s (± 0.70%) 1.50s (± 0.50%) +0.02s (+ 1.24%) 1.49s 1.51s p=0.017 n=6
Check Time 22.59s (± 0.20%) 22.67s (± 0.46%) ~ 22.57s 22.85s p=0.108 n=6
Emit Time 0.00s 0.00s ~ ~ ~ p=1.000 n=6
Total Time 28.04s (± 0.21%) 28.12s (± 0.41%) ~ 28.01s 28.33s p=0.148 n=6
System info unknown
Hosts
  • node (v18.15.0, x64)
Scenarios
  • Compiler-Unions - node (v18.15.0, x64)
  • angular-1 - node (v18.15.0, x64)
  • mui-docs - node (v18.15.0, x64)
  • self-build-src - node (v18.15.0, x64)
  • self-build-src-public-api - node (v18.15.0, x64)
  • self-compiler - node (v18.15.0, x64)
  • ts-pre-modules - node (v18.15.0, x64)
  • vscode - node (v18.15.0, x64)
  • webpack - node (v18.15.0, x64)
  • xstate-main - node (v18.15.0, x64)
Benchmark Name Iterations
Current pr 6
Baseline baseline 6

Developer Information:

Download Benchmarks

return !!(getNodeCheckFlags(node) & flag);
}

function calculateNodeCheckFlagWorker(node: Node, flag: LazyNodeCheckFlags) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there a chance for this to go out of sync with other code that sets these flags?

Copy link
Member Author

@weswigham weswigham May 24, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

100% absolutely, but since we run with noCheck and compare the result to a normal emit for every test we have, it's very likely a test will catch it.

Copy link
Member Author

@weswigham weswigham May 24, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I also do reuse the actual flag-setting functions, just not the traversal to reach and invoke them. So the "should this flag be set" logic should be synced, at least.

@DanielRosenwasser
Copy link
Member

@typescript-bot pack this

@typescript-bot
Copy link
Collaborator

typescript-bot commented May 24, 2024

Starting jobs; this comment will be updated as builds start and complete.

Command Status Results
pack this ✅ Started ✅ Results

@typescript-bot
Copy link
Collaborator

typescript-bot commented May 24, 2024

Hey @DanielRosenwasser, I've packed this into an installable tgz. You can install it for testing by referencing it in your package.json like so:

{
    "devDependencies": {
        "typescript": "https://typescript.visualstudio.com/cf7ac146-d525-443c-b23c-0d58337efebc/_apis/build/builds/161938/artifacts?artifactName=tgz&fileId=956C1558FE35CE6051E3654C542769EBA2201B7AB43750178EFB74C53E41987602&fileName=/typescript-5.5.0-insiders.20240524.tgz"
    }
}

and then running npm install.


There is also a playground for this build and an npm module you can use via "typescript": "npm:@typescript-deploys/pr-build@5.5.0-pr-58364-29".;

@jakebailey
Copy link
Member

I thought I'd try this out on a ts-loader project, so tested out pyright's build before and after. Somehow, it's slower? I suppose this doesn't actually use transpileModule since they're using const enums and everything :(

$ hyperfine 'node --max_old_space_size=8192 ./node_modules/webpack-cli/bin/cli.js --mode development'
Benchmark 1: node --max_old_space_size=8192 ./node_modules/webpack-cli/bin/cli.js --mode development
  Time (mean ± σ):     14.899 s ±  0.221 s    [User: 22.878 s, System: 1.591 s]
  Range (min … max):   14.379 s … 15.200 s    10 runs
 
$ hyperfine 'node --max_old_space_size=8192 ./node_modules/webpack-cli/bin/cli.js --mode development'
Benchmark 1: node --max_old_space_size=8192 ./node_modules/webpack-cli/bin/cli.js --mode development
  Time (mean ± σ):     15.110 s ±  0.164 s    [User: 23.219 s, System: 1.541 s]
  Range (min … max):   14.888 s … 15.395 s    10 runs

@weswigham
Copy link
Member Author

I suppose this doesn't actually use transpileModule since they're using const enums and everything :(

Ya, ts-loader only uses transpileModule if you pass transpileOnly.

@jakebailey
Copy link
Member

jakebailey commented May 30, 2024

In lieu of real-world benchmarks, I used tinybench and tested this before/after on files of various sizes in our codebase, calling:

ts.transpileModule(input, {
    fileName,
    compilerOptions: { target: ts.ScriptTarget.ESNext, module: ts.ModuleKind.Preserve } },
);
Benchmarking src/typescript/typescript.ts (26 lines)...
┌─────────┬───────────────────────┬─────────┬───────────────────┬──────────┬─────────┐
│ (index) │ Task Name             │ ops/sec │ Average Time (ns) │ Margin   │ Samples │
├─────────┼───────────────────────┼─────────┼───────────────────┼──────────┼─────────┤
│ 0       │ 'old transpileModule' │ '1,092' │ 914970.5937785943 │ '±1.49%' │ 10930   │
│ 1       │ 'new transpileModule' │ '1,478' │ 676558.4366416298 │ '±1.69%' │ 14781   │
└─────────┴───────────────────────┴─────────┴───────────────────┴──────────┴─────────┘
35.24% faster, 1.35x faster, 26.06% less time than before
Benchmarking src/compiler/path.ts (1017 lines)...
┌─────────┬───────────────────────┬─────────┬────────────────────┬──────────┬─────────┐
│ (index) │ Task Name             │ ops/sec │ Average Time (ns)  │ Margin   │ Samples │
├─────────┼───────────────────────┼─────────┼────────────────────┼──────────┼─────────┤
│ 0       │ 'old transpileModule' │ '58'    │ 17076565.800341412 │ '±1.07%' │ 586     │
│ 1       │ 'new transpileModule' │ '126'   │ 7927894.958795462  │ '±1.06%' │ 1262    │
└─────────┴───────────────────────┴─────────┴────────────────────┴──────────┴─────────┘
115.40% faster, 2.15x faster, 53.57% less time than before
Benchmarking src/server/project.ts (3064 lines)...
┌─────────┬───────────────────────┬─────────┬───────────────────┬──────────┬─────────┐
│ (index) │ Task Name             │ ops/sec │ Average Time (ns) │ Margin   │ Samples │
├─────────┼───────────────────────┼─────────┼───────────────────┼──────────┼─────────┤
│ 0       │ 'old transpileModule' │ '12'    │ 79510811.21428606 │ '±1.14%' │ 126     │
│ 1       │ 'new transpileModule' │ '31'    │ 31801029.40000019 │ '±1.19%' │ 315     │
└─────────┴───────────────────────┴─────────┴───────────────────┴──────────┴─────────┘
150.03% faster, 2.50x faster, 60.00% less time than before
Benchmarking src/compiler/emitter.ts (6266 lines)...
┌─────────┬───────────────────────┬─────────┬───────────────────┬──────────┬─────────┐
│ (index) │ Task Name             │ ops/sec │ Average Time (ns) │ Margin   │ Samples │
├─────────┼───────────────────────┼─────────┼───────────────────┼──────────┼─────────┤
│ 0       │ 'old transpileModule' │ '7'     │ 135490544.418919  │ '±1.00%' │ 74      │
│ 1       │ 'new transpileModule' │ '15'    │ 65744860.17647067 │ '±1.03%' │ 153     │
└─────────┴───────────────────────┴─────────┴───────────────────┴──────────┴─────────┘
106.09% faster, 2.06x faster, 51.48% less time than before
Benchmarking src/compiler/checker.ts (51965 lines)...
┌─────────┬───────────────────────┬─────────┬────────────────────┬──────────┬─────────┐
│ (index) │ Task Name             │ ops/sec │ Average Time (ns)  │ Margin   │ Samples │
├─────────┼───────────────────────┼─────────┼────────────────────┼──────────┼─────────┤
│ 0       │ 'old transpileModule' │ '0'     │ 1719514998.7000027 │ '±5.94%' │ 10      │
│ 1       │ 'new transpileModule' │ '1'     │ 839784896.9166638  │ '±2.26%' │ 12      │
└─────────┴───────────────────────┴─────────┴────────────────────┴──────────┴─────────┘
104.76% faster, 2.05x faster, 51.16% less time than before

So, for files of a non-trivial size, it sure seems like a good 2x speed boost for transpileModule users.

I have not been able to pin down the perf regression, if there even is one. But I'm pretty sure we should just take this anyhow.

Copy link
Member

@jakebailey jakebailey left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me, though I'm not 100% thrilled with the increase in test time; it looks like it increases CI time from something like 7 minutes to 8.

@weswigham weswigham merged commit bbfc1aa into microsoft:main May 30, 2024
28 checks passed
skeate added a commit to skeate/TypeScript that referenced this pull request Jun 1, 2024
* upstream/main: (37 commits)
  Added NoTruncation flag to completions (microsoft#58719)
  Clone node to remove location even when it has been modified if needed (microsoft#58706)
  Properly account for `this` argument in intersection apparent type caching (microsoft#58677)
  Fix: Include Values of Script Extensions for Unicode Property Value Expressions in Regular Expressions (microsoft#58615)
  In `reScanSlashToken` use `charCodeChecked` not `codePointChecked` (microsoft#58727)
  Shorten error spans for errors reported on constructor declarations (microsoft#58061)
  Mark file as skips typechecking if it contains ts-nocheck (microsoft#58593)
  Fixed an issue with broken `await using` declarations in `for of` loops (microsoft#56466)
  Do not expand type references in keyof and index access (microsoft#58715)
  Improve the performance of isolatedDeclarations quickfix  (microsoft#58722)
  Unwrap `NoInfer` types when narrowing (microsoft#58292)
  Recover from type reuse errors by falling back to inferred type printing (microsoft#58720)
  Fixing self import (microsoft#58718)
  Enable JS emit for noCheck and noCheck for transpileModule (microsoft#58364)
  Revert PR 55371 (microsoft#58702)
  Update dependencies (microsoft#58639)
  Fix baselines after PR 58621 (microsoft#58705)
  Do not infer `yield*` type from contextual `TReturn` (microsoft#58621)
  `await using` normative changes (microsoft#58624)
  Handling statements from a known source file (microsoft#58679)
  ...
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Author: Team For Uncommitted Bug PR for untriaged, rejected, closed or missing bug
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants