Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix saved objects client upsert functionality #114929

Merged
merged 4 commits into from
Oct 15, 2021

Conversation

jportner
Copy link
Contributor

@jportner jportner commented Oct 13, 2021

Resolves #114918.

Reduce from 2 preflight checks to 1 preflight check.
Also combined two internal functions into a single one.

As far as I can tell, no consumers of multi-namespace objects are actually utilizing upsert yet (the only multi-namespace object at the moment is ML jobs), so I'm just backporting this to 7.16 and leaving it without a release note.

Reduce from 2 preflight checks to 1 preflight check.
Also combined two internal functions into a single one.
@jportner jportner changed the title Fix upsert functionality Fix saved objects client upsert functionality Oct 13, 2021
@jportner jportner added v7.15.2 v7.16.0 v8.0.0 release_note:skip Skip the PR/issue when compiling release notes and removed v7.15.2 labels Oct 13, 2021
Copy link
Contributor Author

@jportner jportner left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Author's notes for reviewers

Comment on lines 186 to 187
/** What namespace(s) the object should exist in, if it needs to be created; practically speaking this will never be undefined */
savedObjectNamespaces: string[] | undefined;
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

practically speaking this will never be undefined

What I mean by this is:
If the object's namespaces field was undefined, then it would not have been found in the first place. But the only way this would be undefined is if the object was found and its source did not include a namespaces field.

At any rate, I thought it would be better not to try to change existing types even though that is the behavior we expect.

* @returns Array of namespaces that this saved object currently includes, or (if the object does not exist yet) the namespaces that a
* newly-created object will include. Value may be undefined if an existing saved object has no namespaces attribute; this should not
* happen in normal operations, but it is possible if the Elasticsearch document is manually modified.
* @throws Will throw an error if the saved object exists and it does not include the target namespace.
*/
private async preflightGetNamespaces(
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

preflightGetNamespaces and preflightCheckIncludesNamespace were very similar, so I combined them into a single preflightCheckNamespaces function.

@jportner
Copy link
Contributor Author

Testing process:

I pulled down #114620 and merged this branch on top. Then I ran the functional tests for SOC.update. The tests passed 🎉

 debg KIBANA_CI_STATS_CONFIG environment variable not found, disabling CiStatsReporter
 debg Loading config file from "/Users/joe/GitHub/kibana-3/test/api_integration/config.js"
 debg Loading config file from "/Users/joe/GitHub/kibana-3/test/common/config.js"
 debg Loading config file from "/Users/joe/GitHub/kibana-3/test/functional/config.js"
 info Config loaded
 debg randomness seed: 1634166443943
 info Starting tests

 └-: apis
   └-> "before all" hook in "apis"
   └-: saved_objects
     └-> "before all" hook in "saved_objects"
     └-: update
       └-> "before all" hook for "should return 200"
       └-> "before all" hook for "should return 200"
         │ debg resolved import for test/api_integration/fixtures/kbn_archiver/saved_objects/basic.json to /Users/joe/GitHub/kibana-3/test/api_integration/fixtures/kbn_archiver/saved_objects/basic.json
         │ info importing 4 saved objects { space: undefined }
         │ succ import success
       └-> should return 200
         └-> "before each" hook: global before each for "should return 200"
         └- ✓ pass  (1.0s) "apis saved_objects update should return 200"
       └-> does not pass references if omitted
         └-> "before each" hook: global before each for "does not pass references if omitted"
         └- ✓ pass  (1.0s) "apis saved_objects update does not pass references if omitted"
       └-> passes references if they are provided
         └-> "before each" hook: global before each for "passes references if they are provided"
         └- ✓ pass  (1.0s) "apis saved_objects update passes references if they are provided"
       └-> passes empty references array if empty references array is provided
         └-> "before each" hook: global before each for "passes empty references array if empty references array is provided"
         └- ✓ pass  (1.0s) "apis saved_objects update passes empty references array if empty references array is provided"
       └-> handles upsert
         └-> "before each" hook: global before each for "handles upsert"
         └- ✓ pass  (2.1s) "apis saved_objects update handles upsert"
       └-: unknown id
         └-> "before all" hook for "should return a generic 404"
         └-> should return a generic 404
           └-> "before each" hook: global before each for "should return a generic 404"
           └- ✓ pass  (28ms) "apis saved_objects update unknown id should return a generic 404"
         └-> "after all" hook for "should return a generic 404"
       └-> "after all" hook for "handles upsert"
         │ debg unloading docs from archive at /Users/joe/GitHub/kibana-3/test/api_integration/fixtures/kbn_archiver/saved_objects/basic.json
         │ info deleting 4 objects { space: undefined }
         │ succ 4 saved objects deleted
       └-> "after all" hook for "handles upsert"
     └-> "after all" hook in "saved_objects"
   └-> "after all" hook in "apis"

6 passing (8.2s)

Copy link
Contributor

@pgayvallet pgayvallet left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for fixing this. Got a few remarks and questions before definitive approval:

src/core/server/saved_objects/service/lib/repository.ts Outdated Show resolved Hide resolved
Comment on lines 1809 to 1811
errorType: 'conflict',
});
savedObjectNamespaces = preflightResult.savedObjectNamespaces;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

incrementCounterInternal also relies on upsert:

I think we need to add the same !preflightResult?.found condition you did in update?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, that was only necessary because update will use the version properties of the existing object (for optimistic concurrency control). If you make an ES update call with the version properties and the upsert field in your request, ES responds with an error.

incrementCounterInternal doesn't need to include the version properties at all because it only updates object attributes using a script.

src/core/server/saved_objects/service/lib/repository.ts Outdated Show resolved Hide resolved
@kibanamachine
Copy link
Contributor

💚 Build Succeeded

Metrics [docs]

✅ unchanged

History

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

Copy link
Contributor

@pgayvallet pgayvallet left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM.

Thanks for making the checkResult changes. I agree this is more verbose for the callers, but just re-reviewing the code, it felt way easier to understand the logic this way.

@jportner jportner added the auto-backport Deprecated - use backport:version if exact versions are needed label Oct 15, 2021
@jportner jportner merged commit 56cb517 into elastic:master Oct 15, 2021
@jportner jportner deleted the issue-114918-fix-broken-upserts branch October 15, 2021 12:11
kibanamachine pushed a commit to kibanamachine/kibana that referenced this pull request Oct 15, 2021
@kibanamachine
Copy link
Contributor

💚 Backport successful

Status Branch Result
7.x

This backport PR will be merged automatically after passing CI.

kibanamachine added a commit that referenced this pull request Oct 15, 2021
Co-authored-by: Joe Portner <5295965+jportner@users.noreply.github.com>
artem-shelkovnikov pushed a commit to artem-shelkovnikov/kibana that referenced this pull request Oct 20, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
auto-backport Deprecated - use backport:version if exact versions are needed release_note:skip Skip the PR/issue when compiling release notes v7.16.0 v8.0.0
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Saved object update w/ upsert doesn't work for multi-namespace objects that don't exist
3 participants