Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

configuring pipeline to avoid changing buildpacks #830

Closed
wants to merge 8 commits into from

Conversation

Shashankft9
Copy link
Member

Changes

  • adds two tekton tasks: backup-source and restore-source to avoid configuring buildpack/builder to not remove the source files
  • using boson go builder

/kind bug

Fixes #827 #809

@knative-prow-robot knative-prow-robot added kind/bug Categorizes issue or PR as related to a bug. needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. labels Feb 14, 2022
@knative-prow-robot
Copy link

@Shashankft9: PR needs rebase.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@knative-prow-robot
Copy link

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: Shashankft9
To complete the pull request process, please assign dprotaso after the PR has been reviewed.
You can assign the PR to them by writing /assign @dprotaso in a comment when ready.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@knative-prow-robot knative-prow-robot added the size/L Denotes a PR that changes 100-499 lines, ignoring generated files. label Feb 14, 2022
@Shashankft9
Copy link
Member Author

cc @zroubalik @lance - if this looks okay, then this should probably be undone: boson-project/packs#8
apologies, I mentioned about it in slack/github issue but forgot to put hold on the PR there.

Copy link
Contributor

@zroubalik zroubalik left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@Shashankft9 do we really need to do this? I am sorry, I haven't had a time to follow the recent stuff. But I'd be definitely more inclined towards changing the builders in case they are removing anything from the source code. Adding new tekton tasks brings a lot of additional complexity.

If these kind of chnages are necessary, then we should include them into a build task, so the source is in a correct state after running the build step.

@Shashankft9
Copy link
Member Author

@zroubalik hey, regarding changing pipeline vs changing buildpacks - I have tried to capture the gist from the slack threads here: #827, wdyt about that?

And yeah, I agree, it would be better to anyway add these 2 tasks as steps in build task.

@zroubalik
Copy link
Contributor

@zroubalik hey, regarding changing pipeline vs changing buildpacks - I have tried to capture the gist from the slack threads here: #827, wdyt about that?

And yeah, I agree, it would be better to anyway add these 2 tasks as steps in build task.

Thanks, I will check that issue, I missed it somehow, sorry.

@Shashankft9
Copy link
Member Author

@zroubalik thoughts on this? should I try the change of putting the new 2 tasks as steps instead - if this is how we would like to proceed considering the issue mentioned above.

@Shashankft9
Copy link
Member Author

Shashankft9 commented Mar 10, 2022

@zroubalik added the build task yaml file in the pipeline resources of the kn-plugin-func repo - with 2 additional steps, backup and restore of source code.
because of this, the documentation will change slightly for the apply of buildpack task, but I can take care of that in another PR, let me know.

@zroubalik
Copy link
Contributor

@zroubalik added the build task yaml file in the pipeline resources of the kn-plugin-func repo - with 2 additional steps, backup and restore of source code. because of this, the documentation will change slightly for the apply of buildpack task, but I can take care of that in another PR, let me know.

Sorry for the delay, though I'd prefer if we change the Go and Rust builders to not delete the function source code and config, so we have consistent experience with all runtimes. WDYT?

@Shashankft9
Copy link
Member Author

so we have consistent experience with all runtimes

thats actually my concern, in the slack thread of buildpacks I have linked in the original issue, i got to learn that there really is no standard in buildpacks today to handle source file deletion, so some do and some don't. So I see two choices, either to change buildpacks that do the source file deletion, maybe in future there will be more OR handle it like this.
I do see your point though, in case if we are going to maintain buildpacks, then it makes sense to do the changes at buildpacks level, but the pipeline change still looks more simpler to me - although, I havent really worked much with CICD systems so I wouldn't know how hard it would be to do the same for systems other than tekton etc.
and then there are security concerns like #357 that @matejvasek briefly talked about in the slack thread.

@matejvasek
Copy link
Contributor

BTW I just noticed that paketo java BP probably does delete sources.

@github-actions
Copy link
Contributor

This Pull Request is stale because it has been open for 90 days with
no activity. It will automatically close after 30 more days of
inactivity. Reopen with /reopen. Mark as fresh by adding the
comment /remove-lifecycle stale.

@github-actions github-actions bot added the lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. label Jun 13, 2022
@lance
Copy link
Member

lance commented Jun 17, 2022

@Shashankft9 I am going to go ahead and close this, since I think the problem has been addressed with a less resource intensive workaround in this PR #1070. Thanks for your contribution in any case!

@lance lance closed this Jun 17, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
kind/bug Categorizes issue or PR as related to a bug. lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. size/L Denotes a PR that changes 100-499 lines, ignoring generated files.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Handling source files deletion in on-cluster-builds
5 participants