[Bug]: JTE 2.4 doesn't not resolve scriptPath correctly

[NHLJang]

Jenkins Version

2.303.3

JTE Version

2.3

Bug Description

This pull request #252 was told to give JTE the ability to change the scriptPath/Pipeline Template Path, but JTE seems to still try resolving "Jenkinsfile" default name.

I'm using MultibranchPipelineJob and would want to have multiple Template for Build/Nightly/Release multibranch job. Single pipeline_config.groovy could not achieve this IMHO. And it seems we can not have pipeline_config.groovy location to be different between Gorvernance Tier/JTE Configuration repo and project repo, am I correct?

EDIT: Looks like the issue should be rephrased as "scriptPath does not get propagated to parent Governance tier" I assume. Changing scriptPath works for client project, but if we turn off allow_scm_jenkinsfile to use the Template Configuration repo, Jenkinsfile still be the default scriptPath.

Relevant log output

[JTE] Obtained Template 
[JTE] -- scm: git ssh://git@bitbucket.com:7999/my-platform/jenkins-library.git
[JTE] -- file path: Jenkinsfile

Steps to Reproduce

Jenkins Template repository as follows:
- BuildPipelineTemplate
- pipeline_config.groovy

@merge libraries {
common
}
template_methods {
cleanup
publish
extraStep
postBuild
}

Project repository as follows:
- pipeline_config.groovy

jte {
allow_scm_jenkinsfile = false
}
libraries {
docker
}
application_environments {
all {
per_recipent = "my-user@mail.com"
}
}

[steven-terrana]

i'm not sure i'm following.
Would you be able to create a public repository that reproduces the issue you're describing?

[NHLJang]

Hi @steven-terrana,
To be more clear, here is the example of what I've done:
Consumer project which I've setup a multibranchPipeline Jenkins job for. Contains one Jenkinsfile (matter only when allow_scm_jenkinsfile = true) and pipeline_config.groovy file.
JTE library source which acted as both "Pipeline Configugration" source and "Library Source" under "Jenkins Template Engine" section in consumer project's Jenkins job.
If I configure the Build Configuration as above, this is the log:

[JTE] -- file path: pipeline_config.groovy
[JTE] Pipeline Configuration Modifications 
[JTE] Configurations Added:
[JTE] - libraries.common set to [:]
[JTE] - template_methods.prepare_shared_lib set to [:]
[JTE] - template_methods.build set to [:]
[JTE] - template_methods.cleanup set to [:]
[JTE] - template_methods.publish set to [:]
[JTE] - template_methods.extra_step set to [:]
[JTE] - template_methods.mailing set to [:]
[JTE] Configurations Deleted: None
[JTE] Configurations Changed: None
[JTE] Configurations Duplicated: None
[JTE] Configurations Ignored: None
[JTE] Subsequent May Merge:
[JTE] - libraries
[JTE] Subsequent May Override: None
[JTE] Loading Library common 
[JTE] -- scm: git https://github.com/NHLJang/JTE-lib-source.git
[JTE] Template Primitives are overwriting Jenkins steps with the following names: 
[JTE]   1. build
[JTE] Jenkinsfile does not exist. 
[JTE] -- scm: git https://github.com/NHLJang/JTE-lib-source.git
java.lang.Exception: Could not determine pipeline template.

If I would change the name of template file in JTE library source, from "BuildPipelineTemplate" into "Jenkinsfile" (switch to default-script-path branch), then the template would be "resolved", without me touching consumer project's pipeline_config.groovy , or "Build Configuration" scriptPath (still be "BuildPipelineTemplate").

[JTE] Obtained Template (hide)
[JTE] -- scm: git https://github.com/NHLJang/JTE-lib-source.git
[JTE] -- file path: Jenkinsfile

Thanks for your attention 😃

[NHLJang]

This is what I've meant with "scriptPath does not get propagated to parent Governance Tier". I create a use-scm-jenkinsfile branch, where I set allow_scm_jenkinsfile = true and naming a custom Jenkinsfile, scriptPath/file path seems to be resolved correcty :

[JTE] Obtained Repository Jenkinsfile 
[JTE] -- scm: git https://github.com/NHLJang/JTE-consumer-project.git
[JTE] -- file path: BuildPipelineTemplate

So looks like only when I try to refer to BuildPipelineTemplate in parent tier, which is the JTE library source in this case, the scriptPath won't be taken into account, but fallback to "Jenkinsfile" solely.

[steven-terrana]

ah i think i see the problem.

Changing the scriptPath only applies to the path for the Pipeline Template in the application repository (JTE-consumer-project).

If you want to change the path of the Pipeline Template in the Governance Tier(JTE-lib-source) then you'd need to use a Named Pipeline Template by putting BuildPipelineTemplate in a directory called pipeline_templates and then selecting it via the Pipeline Configuration as:

jte{
  pipeline_template = "BuildPipelineTemplate"
}

[NHLJang]

Hi @steven-terrana,
I guess we can. But then the idea of centralizing the templates our projects may use among several purposes (which is the reason the pipeline_templates exists I assume) is not very ideal and straight-forward, don't you think 😃? Because that means we can not directly choose which template will be used for a specific job via Job Config, but instead having extra works with maintaining different branches of Governance Tier repo with different version of pipeline_config content in it, and then, selecting the correct branch on Pipeline Configugration -> SCM for that job, doesn't it?

[krugerm-4c]

I have the exact same problem where I upgraded the plugin to 2.4 and none of my multibranch jobs were working anymore due to the fact that these custom paths were always referenced by default in the source (app project) compared to a separate governance repository with all the JTE resources.

Had to rollback for the time being, but I thought the context of this change was that the Jenkinsfile and pipeline_config.groovy files present in your source repository can have different paths/names, but your actual pipeline code should still be referenced to the governance repository if configured as such.

[steven-terrana]

The Pipeline Template Path configuration for Multibranch projects is not intended to change the way Governance Tier's resolve the pipeline template.

It only changes the paths within the source repository for the Pipeline Template and the Pipeline Configuration.

It's always the case that Governance Tiers are structured as:

Jenkinsfile
pipeline_templates/
  template_A.groovy
  template_B.groovy

where the Pipeline Configuration can declare jte.pipeline_template to be "template_A.groovy" or "template_B.groovy" and if jte.pipeline_template is not set - Jenkinsfile will be used.

I will update the Help Text provided in the Jenkins UI to be more clear for those inputs.
I'll also be marking them as optional configurations that must be explicitly enabled.

[NHLJang]

If you insist, then, I don't see JTE brings up any conveniences for managing different pipelines(s) within one project repository ,which usually be a choice for development strategy, where only one single value for jte.pipeline_template can be configured. Guess I will stick to Shared Library for a while longer and hope for something else in the future.

Thanks.