Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[ZDT Migration] support updating mapping for root fields #179258

Closed
pgayvallet opened this issue Mar 22, 2024 · 1 comment · Fixed by #179595
Closed

[ZDT Migration] support updating mapping for root fields #179258

pgayvallet opened this issue Mar 22, 2024 · 1 comment · Fixed by #179595
Assignees
Labels
Feature:Migrations Feature:Saved Objects Team:Core Core services & architecture: plugins, logging, config, saved objects, http, ES client, i18n, etc

Comments

@pgayvallet
Copy link
Contributor

atm the zdt migration algorithm fully ignores the root field mappings after the initial index creation, and never update them. The additive mapping changes are only based on the SO types that have new model versions:

const additiveMappingChanges = generateAdditiveMappingDiff({
types,
meta: currentMappings._meta ?? {},
deletedTypes: context.deletedTypes,
});

/**
* Generates the additive mapping diff we will need to update the index mapping with.
*
* @param types The types to generate the diff for
* @param meta The meta field of the index we're migrating
* @param deletedTypes The list of deleted types to ignore during diff/comparison
*/
export const generateAdditiveMappingDiff = ({
types,
meta,
deletedTypes,
}: GenerateAdditiveMappingsDiffOpts): SavedObjectsMappingProperties => {
const typeVersions = getVirtualVersionMap(types);
const mappingVersion = getVirtualVersionsFromMappingMeta({
meta,
source: 'mappingVersions',
knownTypes: types.map((type) => type.name),
});
if (!mappingVersion) {
// should never occur given we checked previously in the flow but better safe than sorry.
throw new Error(
'Cannot generate additive mapping diff: mappingVersions not present on index meta'
);
}

More important, the version check we're performing to check if we do need to update the mappings is also ignoring the root field mappings, and is only based on comparing the SO type model versions:

const versionCheck = checkVersionCompatibility({
mappings: currentMappings,
types,
source: 'mappingVersions',
deletedTypes: context.deletedTypes,
});

We need to update this logic to support updating our mappings when new root fields are introduced.

This could be done by checking the diff in mappings between the index's and the app's expected mappings (as v2 is doing), or this could be performed by introducing a coreVersion in the model version's meta stored in the index.

@pgayvallet pgayvallet added Team:Core Core services & architecture: plugins, logging, config, saved objects, http, ES client, i18n, etc Feature:Saved Objects Feature:Migrations labels Mar 22, 2024
@elasticmachine
Copy link
Contributor

Pinging @elastic/kibana-core (Team:Core)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Feature:Migrations Feature:Saved Objects Team:Core Core services & architecture: plugins, logging, config, saved objects, http, ES client, i18n, etc
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants