Support private repository extensions

[igorbrites]

Feature description 💡

I created a private terrakube-extensions, and now I need to use it on the executors, though it's showing this error to me:

ERROR org.terrakube.executor.service.scripts.ScriptEngineService - https://github.com/MyOrg/terrakube-extensions: Authentication is required but no CredentialsProvider has been registered

How do I pass the credentials? I thought the same GitHub App used to make Dex work was also being used here, but it's not.

Anything else?

Opened as a feature request because idk if it's a bug 😬

[alfespa17]

It is not really a bug, it will be a new feature.

You can customize the git repository but for now it does not support a private repository.

[alfespa17]

The executor component is cloning the directory in this part of the code without adding any kind of authentication as you can see here.

terrakube/executor/src/main/java/org/terrakube/executor/service/scripts/ScriptEngineService.java

Line 107 in adc227f

Git.cloneRepository()

This is the code when you are cloning a private git repository that is used inside a workspace that is really similar

terrakube/executor/src/main/java/org/terrakube/executor/service/workspace/SetupWorkspaceImpl.java

Line 140 in adc227f

Git.cloneRepository()

The parameter that the executor receives to execute a job is done in this part of the code, maybe adding some information about the private extension could work.

terrakube/executor/src/main/java/org/terrakube/executor/service/mode/TerraformJob.java

Line 13 in adc227f

public class TerraformJob {

The API is sending the request with all the parameters to the executor in this part of the code

terrakube/api/src/main/java/org/terrakube/api/plugin/scheduler/job/tcl/executor/ExecutorService.java

Line 63 in adc227f

public boolean execute(Job job, String stepId, Flow flow) {

terrakube/api/src/main/java/org/terrakube/api/plugin/scheduler/job/tcl/executor/ExecutorContext.java

Line 14 in adc227f

public class ExecutorContext {

And finally the Java class that represents the YAML file that you use to define extension is in this part ot the code.

https://github.com/AzBuilder/terrakube/blob/main/api/src/main/java/org/terrakube/api/plugin/scheduler/job/tcl/model/Flow.java

By the way I think you are the first person that want to use a private extension repository

[igorbrites]

Hmmm got it. Could we mount the SSH key we configure on the organization inside the executors and use it to clone the repo? I used https just because the default extensions repo is using it. And the reason I have a private repo is that I needed a MS Teams extension, to alert me about drift detection.

If it gets too complicated, I can try opening a PR on your extensions repo to add the MS Teams extension, but as I'm not a developer, idk if it would even work (I shamelessly copied inspire myself on the SlackApp one to create mine 🙃).

[alfespa17]

Mounting the SSH key in executor won't work because JGIT is not using that keys to clone the repository.

You could fork the extension repository and inside your fork you could create a branch with the new feature for MS teams.

In the helm chart you could customize the parameter like this:

executor:
  properties:
    toolsRepository: "https://github.com/mygithubuser/my-extensions"
    toolsBranch: "my_feature_branch"

Now you can test your new extension in a different repository and when you have it ready you could send the pull request with some sample code and some images so we can update the documentation.

All help is welcome this is an open project 👍

[igorbrites]

I've created a draft on terrakube-extensions repo: AzBuilder/terrakube-extensions#36

I'll test it like you told me to, and then I open the PR for code reviews.

[ehuizar1028]

By the way I think you are the first person that want to use a private extension repository

I would be the 2nd person :)

I am running this on a private GitHub Enterprise, and would like to avoid having this publicly available within our org in order to pull, so an option to add a PAT would be helpful :)

Thanks!