Change order of spring-profiles using spring.profiles.include and active profiles

[huehnerlady]

After being asked in #15598 to create yet another issue I am happy to do so.

Summary

I have 2 profiles with different values for properties, e.g. dev and local.
The dev properties are the properties used for our development deployment, the local properties is my local version with some tweaks to not interfere with our DEV stage.

I want to have most of the properties in "dev", but then want to override some of these properties, eg. using my local database instead of the deployed one.

so I thought I can achieve this by using spring.profiles.include. But this is not the case, if I use this property, all I get is the dev profiles properties.

in addition you can also see that behaviour in the log, as there is the line
The following profiles are active: local,dev

I expected to see The following profiles are active: dev, local

So what I now need to do is either to add the dev profile in my startup of the service (which is very annoying if you want to use the profile now and then not all the time, in the properties file you can just comment in/out what you need) OR copy/paste all the properties I need from dev to local (which is in my opinion an unnecessary inconvenience).

Steps to reproduce

1. check out my sample project I created using the spring-boot initialiser
2. run ./gradlew bootRun --args='--spring.profiles.active=local' -> there is a log about the order of the profiles and another log to show that the value dev.overrides.local is true as marked in the dev profile, although the local profile says false

Using ./gradlew bootRun --args='--spring.profiles.active=dev,local-without-dev' you can see the behaviour I would expect using the include property.

[philwebb]

Thanks for raising the issue @huehnerlady. We're planning to look at what we can do with the way that configuration properties are loaded across the board. There are quite a few legitimate use-cases like this one.

Unfortunately it's quite hard for us to change the default behavior without breaking someone else who is depending on the current logic.

[huehnerlady]

There are often breaking changes in spring-boot, so I am not 100% sure why this particular one is a challenge?

[philwebb]

We try as much as possible to not make breaking changes in patch releases and we know from experience that fixing issues in ConfigFileApplicationListener without causing side-effects for someone else is very difficult.

It's not that we don't want to provide a solution, it's just I suspect we won't be able to until Spring Boot 2.4.

[huehnerlady]

That is fine, as long as it will be addressed 🙂

[bmaehr]

I have been searching for a simillar problem for some weeks. My two use cases:

• I have a "prod" profile and would like to override only a few properties for a "prod-recovery" profile
• I'm including a "unittest" profile using "spring.profiles.include" in application.properties of the test resources. But I haven't found a possibility to override the values from "unittest" with a second profile when running a specific test.

For my use cases it would be already a solution if spring would honor an @Order annontation when merging the property files from the @PropertySource annotation. Or honor the order of profiles in spring.profiles.include and active profiles.

At the moment it seams to me the order the configuration properties depend only on some kind of random order of class loading. And even worse: The order seams to be different depending on if the application was built on Linux or Windows.

[bmaehr]

Also unexpectced to me: If I have this

    @Configuration
    @Profile("a")
    @PropertySource(value = "classpath:/a.properties")
    static class AProperties {
    }

    @Configuration
    @Profile("b")
    @PropertySource("classpath:/b.properties")
    static class BProperties extends AProperties {
    }

I would expect B takes precedence over A if including (only) B.

[wilkinsona]

@bmaehr This issue is about the ordering behaviour of spring.profiles.include and spring.profiles.active. I don't think they are related to your use of @PropertySource and @Profile, both of which are part of Spring Framework rather than Spring Boot. You may want to open a Spring Framework issue.

[bmaehr]

@wilkinsona I think it is perhaps not possible to solve this issue in spring-boot only, because the underlying problem is in the missing/wrong handling of order of configuration files in spring.

My cases would be already solved if it is possible to define an order for the configurations with "spring.profiles.include" or active profiles. I just tried to add some more cases where order is not correct and possible solutions.

[mbhave]

@huehnerlady We have changed the way we process configuration files and profiles in 2.4.x. In this particular case, there are two options to get to the result you want.

1. We have introduced the concept of importing configuration files. If the only reason the dev profile was being activated was to load the application-dev.properties file, this can now be done with the spring.config.import property. For example, if there's some configuration that needs to be loaded when the local profile is active, it can be done by adding the following to application-local.properties:

spring.config.import=application-dev.properties (the name can even be something like developer.properties)

We have also added support for multi-document properties files. So if you prefer to use a single application.properties file instead of an additional application-local.properties file, it would look like this:

#---
spring.config.activate.on-profile=local
spring.config.import=application-dev.properties

2. If the dev profile needs to be active for reasons other than loading application-dev.properties, it can be done using profile groups in the main application.properties file as follows:

spring.profiles.group.local=dev
#---
spring.config.activate.on-profile=local
dev.overrides.local=false
#---
spring.config.activate.on-profile=dev
dev.overrides.local=true

The order in which profiles are added to the environment will be determined by the ordering of the documents. Later documents will override the properties defined in earlier ones.

For more details on the new processing options, please see the reference guide. Please let us know if these options work for you using the 2.4.0-SNAPSHOTs

[huehnerlady]

@mbhave Many thanks for your answer. as I want to have 2 properties files to be able to gitignore my local one I guess he second option does not work for me, unless it is possible to achieve this with

The following questions came to my mind thinking about how I would use the first option

• Would it be possible there to import multiple configuration files?
• Why is there not the possibility to just give the profile name, why has it to be the whole filename (which can be a lot longer than just profile?
• Will the other possibilities (spring.profiles.include) work along with the new property?
• Will this also work with yml files? I would assume yes? --> Tested it and it works :)

Why did you make the decision not to adapt the way spring.profiles.include works as expected, that you include the profiles not override the active profile?

Having tested the first option it indeed solves my problem, but could it be that now it does ignore the spring.profiles.include property? At least when I use it together with the config import it doesn't seem to work?
If so, this would not solve my problem. I need just the dev profile for importing the properties, but I have other profiles I wanna include for the profiles sake and not the properties, so I do not care in which order they are, but I need them to be included as beans will start according to them

I hope this feedback helps :)

[mbhave]

The groups property will work with multiple property files as well. The groups need to be declared in the main application.properties file but profile specific properties can still be defined in profile-specific files. So you would have an application.properties with

spring.profiles.group.local=dev

and define other properties in application-local.properties and application-dev.properties. This will not guarantee the order that you'd get with multi-document files but since you mentioned that for the profiles you need to activate you do not care in which order they are, it should work for you.

Would it be possible there to import multiple configuration files?

Yes. spring.config.import can be a list.

Why is there not the possibility to just give the profile name, why has it to be the whole filename (which can be a lot longer than just profile?

The imports are not necessarily tied to active profiles. Several locations can be specified under a single spring.config.import key and not just files that look like application-dev.properties.

Will the other possibilities (spring.profiles.include) work along with the new property?

spring.profiles.include has been replaced by spring.profiles.group.* to activate additional profiles if the given profile is active. There were several issues with configuring spring.profiles.include in a profile-specific file/document and it would lead to unexpected results. This is why we require that all profile groups are declared in the main application.properties file (and not in a profile specific section/file) so that we can compute the profiles that are active before loading any profile specific sections.

Will this also work with yml files?

Yes!

A combination of spring.config.import for the dev properties, spring.profiles.group for activating additional profiles and multiple properties file should work for you. If not, please provide a minimal sample that demonstrates the use case that does not work.

[huehnerlady]

This is why we require that all profile groups are declared in the main application.properties

This feels like a step down to me though. I have a private properties file which I do not check in or want to make known in the code in any way. It is my developing local properties file. So How can I then achieve the current - if imperfect - way of declaring further profiles if my developer-profile is not known in the main applications file?

[philwebb]

#22944 has restored the ability to use spring.profiles.include as long as it's not in a profile-specific document. This should allow you to declare profiles from your developer specific file.