Feature flags not enabled when running tests

[jfly]

What is the problem?

I've noticed that feature flags defined in our cdk.json seem to only be in effect when running cdk commands such as cdk synth, cdk diff, etc. They are not enabled when we run our unittests, which is now causing our unittests to fail (specifically because of #3067, but this would happen with any breaking change introduced by a feature flag).

Reproduction Steps

We have the @aws-cdk/aws-kms:defaultKeyPolicies feature flag turned on:

$ grep defaultKeyPolicies cdk.json
    "@aws-cdk/aws-kms:defaultKeyPolicies": true,

Note that defaultKeyPolicies is enabled when I do a cdk synth:

$ cat app.py
#!/usr/bin/env python3

from aws_cdk import core as cdk

app = cdk.App()
flag = "@aws-cdk/aws-kms:defaultKeyPolicies"
enabled = cdk.FeatureFlags.of(app).is_enabled(flag)
print(f"{flag} = {enabled}")

$ cdk synth
@aws-cdk/aws-kms:defaultKeyPolicies = True
Successfully synthesized to ...

But note how it's not enabled when I run a simple test:

$ cat app_test.py
from aws_cdk import core as cdk


def test_flag():
    app = cdk.App()
    flag = "@aws-cdk/aws-kms:defaultKeyPolicies"
    enabled = cdk.FeatureFlags.of(app).is_enabled(flag)
    assert enabled

$ pytest app_test.py
...
============================================= FAILURES ==============================================
_____________________________________________ test_flag _____________________________________________

    def test_flag():
        app = cdk.App()
        flag = "@aws-cdk/aws-kms:defaultKeyPolicies"
        enabled = cdk.FeatureFlags.of(app).is_enabled(flag)
>       assert enabled
E       assert False

app_test.py:8: AssertionError
...

What did you expect to happen?

I expect feature flags defined in cdk.json to be enabled when running tests.

What actually happened?

See "Reproduction Steps" above for details.

CDK CLI Version

1.129.0 (build fb43f89)

Framework Version

No response

Node.js Version

v14.17.5

OS

Linux

Language

Python

Language Version

Python (3.8.10)

Other information

No response

[nija-at]

This is as expected. As part of unit tests we don't pick up the feature flags from cdk.json.

For your unit tests, you need to pass in your own set of feature flags as part of initalizing App.

Here's how we do it in some of our unit tests -

aws-cdk/packages/@aws-cdk/assertions/test/template.test.ts

Lines 27 to 32 in c62377e

	test('default', () => {
	const app = new App({
	context: {
	'@aws-cdk/core:newStyleStackSynthesis': false,
	},
	});

.

If you wish all your unit tests to pass the same context flags as defined in cdk.json, you could make yourself a little utility that does this.

[github-actions]

⚠️COMMENT VISIBILITY WARNING⚠️

Comments on closed issues are hard for our team to see.
If you need more assistance, please either tag a team member or open a new issue that references this one.
If you wish to keep having a conversation with other community members under this issue feel free to do so.