[saved objects] Adds bulkDelete API #139680

TinaHeiligers · 2022-08-29T23:55:07Z

Resolves #30503

Summary

The saved objects service has API's for updating, getting, and resolving many documents in one request but is missing similar functionality for deleting many documents.

There are a few use cases for a new API that would delete documents in bulk:

bulk delete alert saved objects
bulk delete telemetry-related documents
replacement for clearing the kibana index with esArchiver

This PR adds a bulkDelete API with the following characteristics:
Multi-namespace saved objects shared to more than one space need to be deleted by force since the requestor in the active space may not be aware of potential repercussions.

How force in bulk delete behaves:

We implement a force-deletion at a request level:

when force is false (or not added as a query), the document isn't deleted and the API returns an error for that document.
with force as true: will force delete the document.
The requestor must be authorized to delete saved objects in all the spaces from which the object would be removed when using force. This applies to all objects requested in the bulk delete operation.

Once a document has been deleted, the API deletes any legacy URL aliases that may have existed for the object.
The behavior differs from that of deleting a single document with the saved objects delete API in that force is applied to all the objects sent in the request, as opposed to applying to one particular document.

Other notable differences between `delete` and `bulkDelete`:

Errors:

bulkDelete returns errors in the response for each document that couldn't be deleted, whereas delete returns an empty object.
bulkDelete returns a bad request for deleting hidden objects, whereas delete throws a not-found error.

Response:

bulkDelete responds with an array of statuses, containing a summary of the outcome for deleting an object.The response per-object consistd of a success status as either true or false and will always be present. If an object couldn't be deleted (success: false), the corresponding status entry also includes the error for that object.

How to test this:

Testing bulk delete saved objects that don't exist in the current callee's space:

start es and start kibana
create a new space "Non default"
change spaces into "Non default"
load all three sample data fixtures
open the browser's network tab
navigate to Saved Objects application and open the network request for the _find?perPage=50&page=1&fields... request
copy the contents of the saved_objects array from the response
Using a tool of choice (I used postman), issue a _bulk_delete request to saved objects from the default space:

Issue a `_bulk_delete` request to saved objects from the default space

POST localhost:5601/{basepath}/api/saved_objects/_bulk_delete
body (as raw JSON):
[
  {"type": "visualization", "id": "314c6f60-2224-11e8-b802-5bcf64c2cfb4"},
  {"type": "lens", "id": "16b1d7d0-ea71-11eb-8b4b-f7b600de0f7d"},
  ...rest  
]

Observe that the request fails with:

{
    "statuses": [
        {
            "success": false,
            "id": "314c6f60-2224-11e8-b802-5bcf64c2cfb4",
            "type": "visualization",
            "error": {
                "message": "Unable to delete saved object id: 314c6f60-2224-11e8-b802-5bcf64c2cfb4, type: visualization that exists in multiple namespaces, use the \"force\" option to delete all saved objects: Bad Request",
                "statusCode": 400,
                "error": "Bad Request"
            }
        },
        {
            "success": false,
            "id": "16b1d7d0-ea71-11eb-8b4b-f7b600de0f7d",
            "type": "lens",
            "error": {
                "message": "Unable to delete saved object id: 16b1d7d0-ea71-11eb-8b4b-f7b600de0f7d, type: lens that exists in multiple namespaces, use the \"force\" option to delete all saved objects: Bad Request",
                "statusCode": 400,
                "error": "Bad Request"
            }
        }
    ]
}

Send the request again but from the "Non default" space:
POST localhost:5601/{basepath}/s/non-default/api/saved_objects/_bulk_delete

Observer that the request is successful:

{
    "statuses": [
        {
            "success": true,
            "id": "d3d7af60-4c81-11e8-b3d7-01146121b73d",
            "type": "index-pattern"
        },
        {
            "success": true,
            "id": "b05ee7f1-0341-43f1-b345-f6cd3a944254",
            "type": "visualization"
        }
    ]
}

Testing legacy URL alias deletion

There's no easy way to test that bulkDelete also deletes legacy URL aliases that may be present for multi-namespace saved objects.

You can use staging data following the instructions for testing bulkResolve. I found I had to create a new user a system_indices_superuser role add the staging data to kibana.

Checklist

Delete any items that are not applicable to this PR.

Documentation was added for features that require explanation or tutorials.
Unit or functional tests were updated or added to match the most common scenarios

Risk Matrix

Delete this section if it is not applicable to this PR.

Before closing this PR, invite QA, stakeholders, and other developers to identify risks that should be tested prior to the change/feature release.

When forming the risk matrix, consider some of the following examples and how they may potentially impact the change:

Risk	Probability	Severity	Mitigation/Notes
Code should gracefully handle failures for users with different privileges	High	High	Integration tests will verify that saved objects are not deleted when not permitted to delete them in any situation where the request would violate the privileges for the callee.
Consumers need to know the implications of using `force` to delete multi-namespace saved objects	Medium	High	Consumers need to opt-in to using `force` explicitly
Performance regression when deleting multi-namespace saved objects with many legacy URL aliases	Medium	High	Consumers need to be aware that deleting multi-namespace objects also deletes legacy URL aliases, which is done one at a time. THere will be a follow up issue to optimise the `legacyUrlAlias` deletion

…ntation

…ests

TinaHeiligers · 2022-08-30T02:00:25Z

@elasticmachine merge upstream

pgayvallet

Amazing work so far! Overall the implementation looks good, I only have one question about it (this one), the rest are mostly NITs.

As a side note, I found the review of the actual implementation of the SOR.bulkDelete quite... challenging. Maybe we could find a way to extract parts of it to utility methods or something? Note that as long as the implem is working, it's fully optional

packages/core/saved-objects/core-saved-objects-api-browser/src/apis/bulk_delete.ts

packages/core/saved-objects/core-saved-objects-api-server/src/apis/bulk_delete.ts

packages/core/saved-objects/core-saved-objects-api-server/src/saved_objects_repository.ts

packages/core/saved-objects/core-saved-objects-server-internal/src/routes/bulk_delete.ts

src/core/server/integration_tests/saved_objects/service/lib/repository_with_proxy_utils.ts

packages/core/saved-objects/core-saved-objects-api-server-internal/src/lib/repository.ts

…-objects-bulk-delete

packages/core/saved-objects/core-saved-objects-api-server-internal/src/lib/repository.ts

elasticmachine · 2022-09-15T15:55:24Z

Pinging @elastic/kibana-core (Team:Core)

TinaHeiligers

Self review.

TinaHeiligers · 2022-09-15T16:20:08Z

packages/core/saved-objects/core-saved-objects-api-server-internal/src/lib/repository.ts

+   * @returns array BulkDeleteExpectedBulkGetResult[]
+   * @internal
+   */
+  private presortObjectsByNamespaceType(objects: SavedObjectsBulkDeleteObject[]) {


I extracted this from the main body of the bulkDelete implementation to make it easier to follow. The intent is to eventually extract all the pre-flight handling to another module, similar to what's been done for bulkResolve.

TinaHeiligers · 2022-09-15T16:22:36Z

packages/core/saved-objects/core-saved-objects-api-server-internal/src/lib/repository.ts

+  }
+
+  /**
+   * Fetch multi-namespace saved objects


Extracted as a pre-flight check. For now, these helpers are in the main class and can be moved at a later stage to another module.

TinaHeiligers · 2022-09-15T16:25:03Z

packages/core/saved-objects/core-saved-objects-api-server-internal/src/lib/repository.ts

+   * @returns array of objects sorted by expected delete success or failure result
+   * @internal
+   */
+  private getExpectedBulkDeleteMultiNamespaceDocsResults(


Can also be moved to a module at a later stage.

packages/core/saved-objects/core-saved-objects-api-server/src/saved_objects_repository.ts

TinaHeiligers · 2022-09-15T16:38:57Z

...ugins/encrypted_saved_objects/server/saved_objects/encrypted_saved_objects_client_wrapper.ts

+    objects: SavedObjectsBulkDeleteObject[],
+    options?: SavedObjectsBulkDeleteOptions
+  ) {
+    return await this.options.baseClient.bulkDelete(objects, options);


Similar to delete, the bulk operation doesn't return any potentially sensitive data and can be passed directly to the base client.

TinaHeiligers · 2022-09-15T16:40:29Z

x-pack/plugins/security/server/saved_objects/secure_saved_objects_client_wrapper.ts

+      throw error;
+    }
+    const response = await this.baseClient.bulkDelete(objects, options);
+    response?.statuses.forEach(({ id, type, success, error }) => {


Similar to other bulk operations, we need to track the operation for each object being deleted.

TinaHeiligers · 2022-09-15T16:42:45Z

x-pack/test/saved_object_api_integration/common/suites/bulk_delete.ts

+import { expectResponses, getUrlPrefix, getTestTitle } from '../lib/saved_object_test_utils';
+import { ExpectResponseBody, TestCase, TestDefinition, TestSuite, TestUser } from '../lib/types';
+
+export interface BulkDeleteTestDefinition extends TestDefinition {


The tests for the bulk operation were based on those for single delete and modified accordingly.

TinaHeiligers · 2022-09-15T16:48:15Z

x-pack/test/saved_object_api_integration/security_and_spaces/apis/bulk_delete.ts

+    { ...CASES.ALIAS_DELETE_EXCLUSIVE, force: true },
+    { ...CASES.DOES_NOT_EXIST, ...fail404() },
+  ];
+  const hiddenType = [{ ...CASES.HIDDEN, ...fail400() }]; // this behavior diverges from `delete`, which throws 404


As mentioned in the comment, attempting to delete a hidden type, this behavior differs slightly from that of delete, to make it clear the hidden types aren't necessarily "missing" but rather that they cannot be deleted using the api.

…saved_objects_repository.ts

TinaHeiligers · 2022-09-17T16:52:36Z

@elasticmachine merge upstream

x-pack/plugins/spaces/server/saved_objects/spaces_saved_objects_client.ts

…s_client.ts

jeramysoucy

LGTM! I ran all of the tests as I reviewed the code. Fond some very minor nits which I commented on. Manually tested not-found, different space, same space, and force required paths.

jeramysoucy · 2022-09-19T15:53:44Z

packages/core/saved-objects/core-saved-objects-api-server-internal/src/lib/repository.test.ts

+        );
+      });
+
+      it(`returns an for an object when the object's type is hidden`, async () => {


Nit: missing word..."returns an error for..."

jeramysoucy · 2022-09-19T18:36:49Z

src/core/server/integration_tests/saved_objects/service/lib/repository_with_proxy.test.ts

@@ -237,6 +262,15 @@ describe('404s from proxies', () => {
      );
    });

+    it('handles `bulkDelete` requests that are successful when the proxy passes through the product header', async () => {
+      const docsToGet = myBulkDeleteTypeDocs;


Total nit, would expect const docsToDelete

jeramysoucy · 2022-09-19T18:55:12Z

x-pack/plugins/spaces/server/saved_objects/spaces_saved_objects_client.ts

@@ -273,7 +286,7 @@ export class SpacesSavedObjectsClient implements SavedObjectsClientContract {

  async bulkUpdate<T = unknown>(
    objects: Array<SavedObjectsBulkUpdateObject<T>> = [],
-    options: SavedObjectsBaseOptions = {}
+    options: SavedObjectsBulkDeleteOptions = {}


Should not have changed - I think you had already caught this one during our walkthrough.

TinaHeiligers · 2022-09-19T21:31:47Z

packages/core/saved-objects/core-saved-objects-api-server-internal/src/lib/repository.ts

@@ -127,6 +141,7 @@ export interface SavedObjectsRepositoryOptions {
 export const DEFAULT_REFRESH_SETTING = 'wait_for';
 export const DEFAULT_RETRY_COUNT = 3;

+const MAX_CONCURRENT_ALIAS_DELETIONS = 10;


This is the same limit on the number of concurrent alias deletions as used for updating objects' spaces. We could export the constant from here and reuse that as a default everywhere we implement a pMap.

TinaHeiligers · 2022-09-19T21:41:15Z

packages/core/saved-objects/core-saved-objects-api-server-internal/src/lib/repository.ts

+      }).catch((err) => {
+        this._logger.error(`Unable to delete aliases when deleting an object: ${err.message}`);
+      });
+    await pMap(objectsToDeleteAliasesFor, mapper, { concurrency: MAX_CONCURRENT_ALIAS_DELETIONS });


As mentioned in #117056 (comment), there isn't a bulk API for deleting legacy URL aliases for many objects since the es query/script would need to handle different namespaces and delete behavior for each saved object that no longer exists. To minimize the number of changes adding a bulkDelete API introduces, I kept it simple, I'm using a pMap, as is done in updateObjectSpaces.

pgayvallet

Amazing work, LGTM!

The only thing is, as discussed on slack, we should open a follow-up issue to try to improve the performances by trying to achieve 'bulk' removal of the legacy alias during a bulk delete.

pgayvallet · 2022-09-20T11:56:09Z

packages/core/saved-objects/core-saved-objects-api-server-internal/src/lib/repository.test.ts

@@ -2044,6 +2046,517 @@ describe('SavedObjectsRepository', () => {
    });
  });

+  describe('#bulkDelete', () => {


I die a little every time I open this file 😅

We're working on it! Long term plan is to modularize it. I've already started pulling out common code as I've been working on the new extension unit tests which will reside in separate files. Once we merge my changes with the packages refactor we can continue to improve the disaster.

pgayvallet · 2022-09-20T12:03:52Z

packages/core/saved-objects/core-saved-objects-api-server-internal/src/lib/repository.ts

+          namespaces = actualResult!._source.namespaces ?? [
+            SavedObjectsUtils.namespaceIdToString(namespace),
+          ];
+          const useForce = force && force === true ? true : false;


NIT: const useForce = force === true?

pgayvallet · 2022-09-20T12:04:27Z

packages/core/saved-objects/core-saved-objects-api-server-internal/src/lib/repository.ts

+          const useForce = force && force === true ? true : false;
+          // the document is shared to more than one space and can only be deleted by force.
+          if (
+            useForce === false &&


NIT: !useForce && ...

pgayvallet · 2022-09-20T12:05:53Z

packages/core/saved-objects/core-saved-objects-api-server-internal/src/lib/repository.ts

+          }
+          // the following check should be redundant since we're retrieving the docs from elasticsearch but we check just to make sure
+          // @ts-expect-error MultiGetHit is incorrectly missing _id, _source
+          if (docFound && !this.rawDocExistsInNamespace(actualResult, namespace)) {


NIT: we exited previously on the !docFound condition, so docFound can be removed from this condition

pgayvallet · 2022-09-20T12:11:23Z

packages/core/saved-objects/core-saved-objects-api-server-internal/src/lib/repository.ts

+
+      if (rawResponse.result === 'deleted') {
+        // `namespaces` should only exist in the expectedResult.value if the type is multi-namespace.
+        if (namespaces) {


NIT: this condition should always be true here, right? I'm fine keeping, but then we should throw in a else block as we did for if (bulkDeleteResponse === undefined) {

this condition should always be true here, right?

Not necessarily. Only multi-namespace documents have a namespaces array because we're only using mget for those namespace types. Single namespace and agnostic types aren't fetched from es in the mget call.

pgayvallet · 2022-09-20T12:21:52Z

x-pack/plugins/security/server/authorization/privileges/privileges.test.ts

@@ -109,6 +109,7 @@ describe('features', () => {
      actions.savedObject.get('all-savedObject-all-1', 'update'),
      actions.savedObject.get('all-savedObject-all-1', 'bulk_update'),
      actions.savedObject.get('all-savedObject-all-1', 'delete'),
+      actions.savedObject.get('all-savedObject-all-1', 'bulk_delete'),


Oh yea, I forgot about this file too.

TinaHeiligers · 2022-09-20T13:38:26Z

@elasticmachine merge upstream

kibana-ci · 2022-09-20T14:51:26Z

💚 Build Succeeded

Buildkite Build
Commit: cd5c7ff

Metrics [docs]

Public APIs missing comments

Total count of every public API that lacks a comment. Target amount is 0. Run node scripts/build_api_docs --plugin [yourplugin] --stats comments for more detailed information.

id	before	after	diff
`@kbn/core-saved-objects-api-browser`	66	75	+9
`@kbn/core-saved-objects-api-server`	126	137	+11
`@kbn/core-saved-objects-api-server-internal`	49	51	+2
`@kbn/core-saved-objects-server-internal`	66	69	+3
`@kbn/core-usage-data-server`	135	142	+7
total			+32

Page load bundle

Size of the bundles that are downloaded on every page load. Target size is below 100kb

id	before	after	diff
`core`	362.3KB	363.1KB	+803.0B

Unknown metric groups

API count

id	before	after	diff
`@kbn/core-saved-objects-api-browser`	94	106	+12
`@kbn/core-saved-objects-api-server`	289	308	+19
`@kbn/core-saved-objects-api-server-internal`	68	71	+3
`@kbn/core-saved-objects-server-internal`	66	69	+3
`@kbn/core-usage-data-server`	146	153	+7
`core`	2657	2684	+27
total			+71

History

💚 Build #73671 succeeded 7aa18ba
💔 Build #73657 failed 591a7b8
💚 Build #73513 succeeded b0bd664
💚 Build #73134 succeeded b6a3b3f
💚 Build #72842 succeeded 614caf6
💚 Build #72799 succeeded cac082a

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

TinaHeiligers added 10 commits August 29, 2022 09:52

Updates public api types with bulkDelete types

7f98688

Adds bulkDelete server side implementation

1b9c5c4

Adds bulk_delete server-side route, updates core usage data accordingly

254edb3

Adds bulkDelete to client side api, client and contract, adds impleme…

bc5c98f

…ntation

Adds core jest integration tests for bulkDelete

a32022d

Adds bulkDelete to secure_saved_objects_client_wrapper

a07d4d6

adds bulkDelete to secure saved objects client, updates client unit t…

54f5bcb

…ests

Adds bulkDelete to encrypted saved objects wrapper

e1d5383

Cleans up types

4aa5a6c

Adds api integration tests for saved objects bulk delete

95f144a

TinaHeiligers added 2 commits August 29, 2022 18:57

Updates core saved objects usage stats

3cbfe86

updates telemetry schema

1e03ff7

Merge branch 'main' into kbn-30503-saved-objects-bulk-delete

2c8646d

pgayvallet reviewed Aug 30, 2022

View reviewed changes

TinaHeiligers added 6 commits August 31, 2022 12:46

Fix incorrect conditional to apply force flag

941e568

Fix unknown documents deletion

df876c9

Merge branch 'main' of github.com:elastic/kibana into kbn-30503-saved…

51f8216

…-objects-bulk-delete

renames esRequestIndex to help avoid confusion

4ccd6ca

Adds a bulkDelete test assertion for mix of valid and invalid documents

336634a

Handles nits from draft review

40c5454

TinaHeiligers commented Sep 1, 2022

View reviewed changes

packages/core/saved-objects/core-saved-objects-api-server-internal/src/lib/repository.ts Outdated Show resolved Hide resolved

TinaHeiligers commented Sep 1, 2022

View reviewed changes

packages/core/saved-objects/core-saved-objects-api-server-internal/src/lib/repository.ts Outdated Show resolved Hide resolved

Fixes typo

2d79405

TinaHeiligers commented Sep 15, 2022

View reviewed changes

Update packages/core/saved-objects/core-saved-objects-api-server/src/…

614caf6

…saved_objects_repository.ts

Merge branch 'main' into kbn-30503-saved-objects-bulk-delete

b6a3b3f

TinaHeiligers requested review from pgayvallet and legrego September 19, 2022 15:00

TinaHeiligers commented Sep 19, 2022

View reviewed changes

x-pack/plugins/spaces/server/saved_objects/spaces_saved_objects_client.ts Outdated Show resolved Hide resolved

Update x-pack/plugins/spaces/server/saved_objects/spaces_saved_object…

b0bd664

…s_client.ts

jeramysoucy approved these changes Sep 19, 2022

View reviewed changes

TinaHeiligers added 2 commits September 19, 2022 14:11

limit number of concurrent legacy url alias deletions

591a7b8

reimplement error type consistency implementation

0ce4a5d

TinaHeiligers commented Sep 19, 2022

View reviewed changes

handles nits from PR review

7aa18ba

pgayvallet approved these changes Sep 20, 2022

View reviewed changes

kibanamachine and others added 2 commits September 20, 2022 07:38

Merge branch 'main' into kbn-30503-saved-objects-bulk-delete

42ad8ba

Addressed code review nits

cd5c7ff

TinaHeiligers enabled auto-merge (squash) September 20, 2022 14:26

TinaHeiligers merged commit 92ca42f into elastic:main Sep 20, 2022

This was referenced Sep 20, 2022

Reduce performance impact of deleting legacy url aliases #141136

Open

Add docs for bulkDelete saved objects repository API #141143

Closed

TinaHeiligers deleted the kbn-30503-saved-objects-bulk-delete branch September 20, 2022 17:16

shahzad31 mentioned this pull request Sep 21, 2022

[Synthetics] bulkDelete to delete synthetics monitors #141261

Merged

TinaHeiligers mentioned this pull request Sep 21, 2022

Need a replacement for esArchiver.emptyKibanaIndex(); #136562

Open

shahzad31 mentioned this pull request Sep 22, 2022

[Fleet] Update to use savedObjects bulkDelete for package policy delete #141276

Merged

xcrzx mentioned this pull request Dec 16, 2022

[Fleet] Packages with a large number of saved objects in them cause Kibana to crash #147695

Closed

dokmic mentioned this pull request Jan 10, 2023

[Saved Objects Management] Encapsulate saved objects deletion behind an API endpoint #148602

Merged

1 task

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[saved objects] Adds bulkDelete API #139680

[saved objects] Adds bulkDelete API #139680

TinaHeiligers commented Aug 29, 2022 •

edited

Loading

TinaHeiligers commented Aug 30, 2022

pgayvallet left a comment •

edited

Loading

elasticmachine commented Sep 15, 2022

TinaHeiligers left a comment

TinaHeiligers Sep 15, 2022

TinaHeiligers Sep 15, 2022

TinaHeiligers Sep 15, 2022

TinaHeiligers Sep 15, 2022

TinaHeiligers Sep 15, 2022

TinaHeiligers Sep 15, 2022

TinaHeiligers Sep 15, 2022

TinaHeiligers commented Sep 17, 2022

jeramysoucy left a comment

jeramysoucy Sep 19, 2022

jeramysoucy Sep 19, 2022

jeramysoucy Sep 19, 2022

TinaHeiligers Sep 19, 2022

TinaHeiligers Sep 19, 2022

pgayvallet left a comment

pgayvallet Sep 20, 2022

jeramysoucy Sep 20, 2022

pgayvallet Sep 20, 2022

pgayvallet Sep 20, 2022

pgayvallet Sep 20, 2022

pgayvallet Sep 20, 2022

TinaHeiligers Sep 20, 2022 •

edited

Loading

pgayvallet Sep 20, 2022

TinaHeiligers commented Sep 20, 2022

kibana-ci commented Sep 20, 2022

API count

[saved objects] Adds bulkDelete API #139680

[saved objects] Adds bulkDelete API #139680

Conversation

TinaHeiligers commented Aug 29, 2022 • edited Loading

Summary

How force in bulk delete behaves:

Other notable differences between delete and bulkDelete:

How to test this:

Testing legacy URL alias deletion

Checklist

Risk Matrix

TinaHeiligers commented Aug 30, 2022

pgayvallet left a comment • edited Loading

Choose a reason for hiding this comment

elasticmachine commented Sep 15, 2022

TinaHeiligers left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

TinaHeiligers commented Sep 17, 2022

jeramysoucy left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

pgayvallet left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

TinaHeiligers Sep 20, 2022 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

TinaHeiligers commented Sep 20, 2022

kibana-ci commented Sep 20, 2022

💚 Build Succeeded

Metrics [docs]

Public APIs missing comments

Page load bundle

API count

History

TinaHeiligers commented Aug 29, 2022 •

edited

Loading

Other notable differences between `delete` and `bulkDelete`:

pgayvallet left a comment •

edited

Loading

TinaHeiligers Sep 20, 2022 •

edited

Loading