Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

check file changed in loop #1302

Merged

Conversation

tejal29
Copy link
Member

@tejal29 tejal29 commented Jun 6, 2020

small perf improvement.

  1. check if a file is changed in gowalkdir which is faster than filepath.Walk.
  2. Move file deletion logic in the same loop.

This PR reduces the number of times the whole filesystem is walked from 3 to just 1.

@googlebot googlebot added the cla: yes CLA signed by all commit authors label Jun 6, 2020
@tejal29
Copy link
Member Author

tejal29 commented Jun 6, 2020

Performance improvements


Number of Files | Total Build Time Full Snapshot - current | Total Build Time -  Walk performance | % decrease
-- | -- | -- | --
10000 | 13.834973 | 13.579324 | 1.85%
50000 | 47.926338 | 42.460665 | 11.40%
100000 | 96.440554 | 77.774629 | 19.35%
200000 | 187.346273 | 175.198485 | 6.48%
300000 | 341.233622 | 289.063343 | 15.29%
500000 | 1404.410121 | 1122.947834 | 20.04%
700000 | 1772.379601 | 1350.19682 | 23.82%

@tejal29 tejal29 merged commit e73dea4 into GoogleContainerTools:master Jun 6, 2020
@tejal29 tejal29 added the area/performance issues related to kaniko performance enhancement label Aug 22, 2020
@tejal29 tejal29 deleted the check_file_changed_in_loop branch April 26, 2021 16:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/performance issues related to kaniko performance enhancement cla: yes CLA signed by all commit authors
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants