Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[src/dev/build] build Kibana Platform bundles from source #73591

Merged
merged 5 commits into from
Aug 4, 2020

Conversation

spalger
Copy link
Contributor

@spalger spalger commented Jul 29, 2020

Closes #71881

Rather than building the KP bundles from the plugin source files that are copied into the build directory and node_modules which are installed there, we should build the bundles from the source and just write the output of the build process to the build directory. This allows us to move node_modules out of the dependencies list and into devDependencies once we're don't removing the legacy optimizer.

This is done by adding a new config option to the @kbn/optimizer, outputRoot, which defaults to repoRoot but can be customized to mirror the source directory (plus target/public) as the output location for each bundle.

@spalger spalger added Team:Operations Team label for Operations Team v7.10.0 v8.0.0 release_note:skip Skip the PR/issue when compiling release notes labels Jul 29, 2020
@tylersmalley

This comment has been minimized.

@spalger spalger force-pushed the implement/build-plugins-from-src branch 3 times, most recently from d137fb4 to d0fb702 Compare July 30, 2020 01:59
@spalger spalger force-pushed the implement/build-plugins-from-src branch from d0fb702 to b7ddb71 Compare July 30, 2020 18:30
@spalger
Copy link
Contributor Author

spalger commented Aug 3, 2020

So, the difference in module count between master and this PR is that the core and securitySolution bundles in master have two copies of the inherits ponyfill but when installing from source (where the dev dependencies are installed too) the topology of the node_modules directory is different in just this one tiny way... Pretty incredible actually!

The difference in file size is caused by the fact that we're no longer using the babel-ified version of the x-pack source to make bundles from, so we're able to concatenate more modules which reduces the size of the bundles without affecting the module count (now that we count concatenated modules too).

@spalger spalger marked this pull request as ready for review August 3, 2020 22:09
@spalger spalger requested a review from a team as a code owner August 3, 2020 22:09
@elasticmachine
Copy link
Contributor

Pinging @elastic/kibana-operations (Team:Operations)

@kibanamachine
Copy link
Contributor

💚 Build Succeeded

Build metrics

@kbn/optimizer bundle module count

id value diff baseline
core 561 -1 562
securitySolution 2032 -1 2033
total - -2 -

async chunks size

id value diff baseline
apm 3.7MB -16.0KB 3.7MB
beats_management 409.3KB -1.8KB 411.1KB
canvas 1.4MB -2.8KB 1.4MB
crossClusterReplication 462.5KB -1.0KB 463.5KB
enterpriseSearch 265.1KB -34.0B 265.1KB
graph 1.4MB -11.0B 1.4MB
indexLifecycleManagement 275.0KB -1.0B 275.0KB
indexManagement 1.5MB -5.1KB 1.5MB
infra 3.5MB -136.0KB 3.6MB
ingestManager 1.1MB -1.9KB 1.1MB
ingestPipelines 584.9KB -98.0B 584.9KB
licenseManagement 167.5KB -4.0B 167.5KB
logstash 216.1KB -50.0B 216.1KB
maps 3.3MB -674.0B 3.3MB
monitoring 1.1MB -2.5KB 1.1MB
observability 310.3KB -1.9KB 312.2KB
painlessLab 184.2KB -12.0B 184.2KB
remoteClusters 375.4KB -954.0B 376.3KB
rollup 330.5KB -194.0B 330.7KB
security 1.1MB -1.1KB 1.1MB
securitySolution 7.3MB -30.1KB 7.3MB
snapshotRestore 758.2KB -4.0KB 762.2KB
spaces 45.8KB -32.0B 45.8KB
transform 700.6KB -5.5KB 706.1KB
upgradeAssistant 322.3KB -1.3KB 323.7KB
uptime 1.5MB -21.9KB 1.6MB
watcher 602.9KB -2.0KB 604.9KB
total - -237.2KB -

page load bundle size

id value diff baseline
alerts 88.5KB -308.0B 88.8KB
apm 258.0KB +338.0B 257.7KB
beats_management 581.6KB +782.0B 580.9KB
canvas 1.3MB -29.6KB 1.4MB
cloud 5.9KB -485.0B 6.4KB
core 1.2MB -465.0B 1.2MB
crossClusterReplication 192.1KB -763.0B 192.8KB
features 12.9KB -800.0B 13.7KB
fileUpload 725.8KB -705.0B 726.5KB
globalSearch 27.9KB -727.0B 28.6KB
graph 15.7KB +177.0B 15.6KB
indexLifecycleManagement 238.7KB -299.0B 239.0KB
indexManagement 248.1KB -9.3KB 257.4KB
infra 275.1KB -17.2KB 292.3KB
ingestManager 466.0KB +7.2KB 458.9KB
ingestPipelines 32.5KB -144.0B 32.7KB
lens 842.9KB +1.7KB 841.2KB
licenseManagement 26.3KB -912.0B 27.2KB
licensing 20.5KB -1.4KB 21.8KB
lists 162.6KB -106.9KB 269.5KB
logstash 180.1KB -2.3KB 182.4KB
maps 286.9KB +2.7KB 284.2KB
ml 4.3MB -13.9KB 4.3MB
monitoring 297.7KB +2.6KB 295.1KB
painlessLab 161.0KB -5.0B 161.0KB
remoteClusters 35.4KB +94.0B 35.3KB
reporting 291.0KB -1.9KB 292.9KB
rollup 223.1KB -2.3KB 225.3KB
searchprofiler 50.9KB -629.0B 51.5KB
security 283.9KB -4.2KB 288.1KB
securitySolution 804.6KB -65.2KB 869.9KB
snapshotRestore 61.8KB -688.0B 62.5KB
spaces 412.8KB -346.0B 413.2KB
triggers_actions_ui 263.5KB -11.0B 263.5KB
upgradeAssistant 65.2KB +8.1KB 57.1KB
uptime 23.9KB +35.0B 23.9KB
watcher 28.9KB -7.7KB 36.5KB
total - -245.4KB -

History

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

Copy link
Contributor

@tylersmalley tylersmalley left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

@spalger spalger merged commit 409779d into elastic:master Aug 4, 2020
@spalger spalger deleted the implement/build-plugins-from-src branch August 4, 2020 22:34
spalger added a commit to spalger/kibana that referenced this pull request Aug 4, 2020
)

Co-authored-by: spalger <spalger@users.noreply.github.com>
spalger added a commit that referenced this pull request Aug 5, 2020
…) (#74313)

Co-authored-by: spalger <spalger@users.noreply.github.com>
gmmorris added a commit to gmmorris/kibana that referenced this pull request Aug 5, 2020
* master: (74 commits)
  [Discover] Inline noWhiteSpace function (elastic#74331)
  [DOCS] Add Observability topic (elastic#73041)
  skip flaky suite (elastic#74327)
  [Security Solution][Detections] Fixes Severity Override not matching for Elastic Endpoint Security rule (elastic#74317)
  [Security Solution][Exceptions] - Fixes exceptions builder nested deletion issue and adds unit tests (elastic#74250)
  Fixed Alert details does not update page title and breadcrumb (elastic#74214)
  [src/dev/build] build Kibana Platform bundles from source (elastic#73591)
  [Reporting] Shorten asset path to help CLI FS Watcher (elastic#74185)
  Fix TMS not loaded in legacy maps (elastic#73570)
  [Security Solution] styling for notes' panel (elastic#74274)
  [Security Solution][Tech Debt] cleans up ts-ignore issues and some smaller linter issues  (elastic#74268)
  Make the actions plugin support generics (elastic#71439)
  [Security Solution] Keep original note creator (elastic#74203)
  [CI] Fix xpack kibana build dir in xpack visual regression script
  [CI] Fix baseline_capture job by adding parallel process number back
  [Monitoring] Ensure setup mode works on cloud but only for alerts (elastic#73127)
  [Maps] Custom color ramps should show correctly on the map for mvt layers (elastic#74169)
  [kbn/optimizer] remove unused modules (elastic#74195)
  [CI] Add pipeline task queue framework and merge workers into one (elastic#71268)
  Using msearch for tree api endpoint (elastic#73813)
  ...
gmmorris added a commit to gmmorris/kibana that referenced this pull request Aug 5, 2020
* master: (115 commits)
  [Logs UI] Correct trial period duration in anomaly splash screen (elastic#74249)
  [Discover] Inline noWhiteSpace function (elastic#74331)
  [DOCS] Add Observability topic (elastic#73041)
  skip flaky suite (elastic#74327)
  [Security Solution][Detections] Fixes Severity Override not matching for Elastic Endpoint Security rule (elastic#74317)
  [Security Solution][Exceptions] - Fixes exceptions builder nested deletion issue and adds unit tests (elastic#74250)
  Fixed Alert details does not update page title and breadcrumb (elastic#74214)
  [src/dev/build] build Kibana Platform bundles from source (elastic#73591)
  [Reporting] Shorten asset path to help CLI FS Watcher (elastic#74185)
  Fix TMS not loaded in legacy maps (elastic#73570)
  [Security Solution] styling for notes' panel (elastic#74274)
  [Security Solution][Tech Debt] cleans up ts-ignore issues and some smaller linter issues  (elastic#74268)
  Make the actions plugin support generics (elastic#71439)
  [Security Solution] Keep original note creator (elastic#74203)
  [CI] Fix xpack kibana build dir in xpack visual regression script
  [CI] Fix baseline_capture job by adding parallel process number back
  [Monitoring] Ensure setup mode works on cloud but only for alerts (elastic#73127)
  [Maps] Custom color ramps should show correctly on the map for mvt layers (elastic#74169)
  [kbn/optimizer] remove unused modules (elastic#74195)
  [CI] Add pipeline task queue framework and merge workers into one (elastic#71268)
  ...
gmmorris added a commit to gmmorris/kibana that referenced this pull request Aug 5, 2020
* master: (154 commits)
  [ML] Fix initial plugin's bundle size (elastic#74047)
  [Ingest Manager] prevent crash on unhandled rejection from setupIngestManager (elastic#74300)
  [Logs UI] Correct trial period duration in anomaly splash screen (elastic#74249)
  [Discover] Inline noWhiteSpace function (elastic#74331)
  [DOCS] Add Observability topic (elastic#73041)
  skip flaky suite (elastic#74327)
  [Security Solution][Detections] Fixes Severity Override not matching for Elastic Endpoint Security rule (elastic#74317)
  [Security Solution][Exceptions] - Fixes exceptions builder nested deletion issue and adds unit tests (elastic#74250)
  Fixed Alert details does not update page title and breadcrumb (elastic#74214)
  [src/dev/build] build Kibana Platform bundles from source (elastic#73591)
  [Reporting] Shorten asset path to help CLI FS Watcher (elastic#74185)
  Fix TMS not loaded in legacy maps (elastic#73570)
  [Security Solution] styling for notes' panel (elastic#74274)
  [Security Solution][Tech Debt] cleans up ts-ignore issues and some smaller linter issues  (elastic#74268)
  Make the actions plugin support generics (elastic#71439)
  [Security Solution] Keep original note creator (elastic#74203)
  [CI] Fix xpack kibana build dir in xpack visual regression script
  [CI] Fix baseline_capture job by adding parallel process number back
  [Monitoring] Ensure setup mode works on cloud but only for alerts (elastic#73127)
  [Maps] Custom color ramps should show correctly on the map for mvt layers (elastic#74169)
  ...
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
release_note:skip Skip the PR/issue when compiling release notes Team:Operations Team label for Operations Team v7.10.0 v8.0.0
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[build] kbn/optimizer needs ran from source
4 participants