Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add feature to ignore indexes starting with dot on shard limit validation. #4675

Open
nishchay21 opened this issue Oct 4, 2022 · 2 comments
Labels
Cluster Manager enhancement Enhancement or improvement to existing feature or request

Comments

@nishchay21
Copy link
Contributor

nishchay21 commented Oct 4, 2022

Is your feature request related to a problem? Please describe.
Today on the ShardLimitValidator as mention here code only system indexes are ignored from shard limit validation however other plugins which do not inherit system index properties [may be security or hidden indexes] are not excluded. So a feature is required to ignore these indexes from the shard limit validation phase.

Describe the solution you'd like
The high level approach we can follow until all plugin follow a standard method of inheriting the system indices is to exempt indexes starting with a '.' character from shard limit validation. As most of the plugins create an index which starts with '.' so those will be ignored during the shard limit validation phase.

Further, We can use a cluster setting as a feature flag for a gated mechanism where if the feature flag is enabled we ignore the indexes during shard limit validation otherwise handle those indexes as normal indexes.

Describe alternatives you've considered
Move all plugins to follow system indices instead of dot indices as discussed here #3942

@gbbafna
Copy link
Collaborator

gbbafna commented Oct 18, 2022

Thanks @nishchay21 . I think more value can be added if we can ignore such indices from disk based watermark and read-only-delete block . We need to do similar stuff there and ignore datastream indices from these modifications.

@ankitkala
Copy link
Member

Similar to #10331. Probably we can evaluate this with system index plugin.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Cluster Manager enhancement Enhancement or improvement to existing feature or request
Projects
Status: 🆕 New
Development

No branches or pull requests

5 participants