Performance fixes for #659

[Ellpeck]

No description provided.

[codecov]

Codecov Report

Attention: Patch coverage is 89.90826% with 11 lines in your changes are missing coverage. Please review.

Project coverage is 67.27%. Comparing base (5617885) to head (de3b7d0).
Report is 1 commits behind head on main.

Files	Patch %	Lines
src/r-bridge/shell.ts	62.50%	4 Missing and 2 partials ⚠️
src/cli/repl/commands/parse.ts	66.66%	1 Missing ⚠️
src/core/print/parse-printer.ts	66.66%	1 Missing ⚠️
src/r-bridge/lang-4.x/ast/parser/json/format.ts	93.33%	0 Missing and 1 partial ⚠️
src/r-bridge/lang-4.x/ast/parser/json/parser.ts	92.30%	1 Missing ⚠️
src/r-bridge/shell-executor.ts	90.90%	0 Missing and 1 partial ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #684      +/-   ##
==========================================
- Coverage   71.37%   67.27%   -4.11%     
==========================================
  Files         218      217       -1     
  Lines        7082     7041      -41     
  Branches     1092     1092              
==========================================
- Hits         5055     4737     -318     
- Misses       1737     2053     +316     
+ Partials      290      251      -39     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[Ellpeck]

Some semi-representative data from this PR, run on my own machine, using the static suite.

Back when we were using xmlparsedata:

Shell init time:                      5:830ms -         7:390ms (median:         6:309ms, mean:         6:333ms, std:         0:354ms)
Retrieval of token map:              10:509ms -        14:268ms (median:        11:205ms, mean:        11:456ms, std:         0:909ms)
AST retrieval:                        6:247ms -       384:829ms (median:        29:179ms, mean:        54:645ms, std:       104:812ms)
AST normalization:                    7:269ms -       446:619ms (median:        49:685ms, mean:        72:833ms, std:       118:882ms)
Dataflow creation:                    4:524ms -       618:351ms (median:        16:612ms, mean:        54:404ms, std:       134:996ms)

New (before this PR, very bad):

Shell init time:                      5:961ms -         9:467ms (median:         6:355ms, mean:         6:537ms, std:         0:674ms)
AST retrieval:                      156:116ms -       576:554ms (median:       173:689ms, mean:       206:244ms, std:       111:386ms)
AST normalization:                    6:851ms -    07.937:445 s (median:        53:126ms, mean:       727:348ms, std:    02.176:340 s)
Dataflow creation:                    3:621ms -       630:165ms (median:        15:563ms, mean:        53:163ms, std:       136:132ms)

This PR. Compared to xmlparsedata, retrieval is still a bit slower, but normalization is faster.

Shell init time:                      5:809ms -         6:723ms (median:         6:297ms, mean:         6:227ms, std:         0:254ms)
AST retrieval:                      152:774ms -       574:988ms (median:       174:446ms, mean:       206:380ms, std:       113:627ms)
AST normalization:                    6:751ms -       280:213ms (median:        40:691ms, mean:        49:418ms, std:        72:849ms)
Dataflow creation:                    3:524ms -       619:933ms (median:        16:780ms, mean:        53:285ms, std:       134:547ms)

[EagleoutIce]

Current drop:

Shell init time:                      3:823ms -         5:594ms (median:         4:182ms, mean:         4:328ms, std:         0:492ms)
AST retrieval:                      125:715ms -       587:729ms (median:       155:680ms, mean:       197:945ms, std:       118:462ms)
AST normalization:                    3:423ms -       193:891ms (median:        13:210ms, mean:        25:803ms, std:        46:899ms)
Dataflow creation:                    3:226ms -       729:369ms (median:        17:122ms, mean:        61:198ms, std:       155:864ms)

[EagleoutIce]

Settle for:

Shell init time:                      3:674ms -         4:843ms (median:         4:829ms, mean:         4:153ms, std:         0:329ms)
AST retrieval:                      125:678ms -       440:728ms (median:       148:740ms, mean:       171:689ms, std:        85:327ms)
AST normalization:                    3:591ms -       169:966ms (median:        12:648ms, mean:        23:988ms, std:        42:998ms)
Dataflow creation:                    3:356ms -       609:993ms (median:        14:210ms, mean:        52:717ms, std:       132:712ms)

[EagleoutIce]

Without jsonlite

Shell init time:                      2:841ms -         3:791ms (median:         2:958ms, mean:         3:542ms, std:         0:252ms)
AST retrieval:                      130:567ms -       980:585ms (median:       171:436ms, mean:       234:780ms, std:       233:575ms)
AST normalization:                    3:957ms -       154:376ms (median:        11:295ms, mean:        21:242ms, std:        38:464ms)
Dataflow creation:                    3:570ms -       605:431ms (median:        15:609ms, mean:        51:217ms, std:       130:292ms)

[EagleoutIce]

This pull request is included in v1.4.2 (see Release v1.4.2 (Dropping xmlparsedata, Benchmark Re-Runs, and Repl Fixes)).