Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[FLINK-35292] Set dummy savepoint path during last-state upgrade #849

Merged
merged 1 commit into from
Jul 23, 2024

Conversation

gyfora
Copy link
Contributor

@gyfora gyfora commented Jul 4, 2024

What is the purpose of the change

Currently the operator always sets the savepoint path even if last-state (HA metadata) must be used.

This can be misleading to users as the set savepoint path normally should never take effect and can actually lead to incorrect state restored if the HA metadata is deleted by the user at the wrong moment.

To avoid this we can set an explicit dummy savepoint path which will prevent restoring from it accidentally.

Brief change log

  • Set dummy savepoint path during last-state upgrade
  • Add/update tests

Verifying this change

Unit tests + manual verification

Does this pull request potentially affect one of the following parts:

  • Dependencies (does it add or upgrade a dependency): no
  • The public API, i.e., is any changes to the CustomResourceDescriptors: no
  • Core observer or reconciler logic that is regularly executed: yes

Documentation

  • Does this pull request introduce a new feature? no

@@ -265,7 +274,7 @@ protected void restoreJob(
throws Exception {
Optional<String> savepointOpt = Optional.empty();

if (spec.getJob().getUpgradeMode() != UpgradeMode.STATELESS) {
if (spec.getJob().getUpgradeMode() == UpgradeMode.SAVEPOINT) {
Copy link
Contributor

@afedulov afedulov Jul 15, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: it seems appropriate to push this logic down into the deploy method. That would also eliminate usage of an optional as a method parameter (something that is debatably considered an antipattern).

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The problem is that this cannot be easily done as what savepoint is passed there will depend on where / why the deploy is called. It is for example different during initial deployments vs upgrades.

While the optional parameter is not great I would like to avoid adding more complexity to the deploy method. It may make sense to actually break up the deploy method in 3 parts, stateless/last-state/savepoint which would get rid of the optional params + the requireHaMetadata flag at the same time at the expense of more methods.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Whether this would actually makes sense from a code readability standpoint still remains to be seen

Copy link
Contributor

@afedulov afedulov left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@gyfora gyfora merged commit 207b149 into apache:main Jul 23, 2024
169 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants