Add custom template settings

[tresoldigiorgio]

Does this issue relate to a new feature or an existing bug?

• Bug
• New Feature

I need the possibility to pass a dictionary of custom template settings that will be added to the default settings in "GetTemplateData" method like this

        private object GetTemplateData()
        {
            if (_options.GetTemplateContent != null)
                return _options.GetTemplateContent();

            var settings = _options.TemplateCustomSettings ?? new Dictionary<string, string>();
            settings.Add("index.refresh_interval", "5s");

            if (_options.NumberOfShards.HasValue && !settings.ContainsKey("number_of_shards"))
                settings.Add("number_of_shards", _options.NumberOfShards.Value.ToString());

            if (_options.NumberOfReplicas.HasValue && !settings.ContainsKey("number_of_replicas"))
                settings.Add("number_of_replicas", _options.NumberOfReplicas.Value.ToString());

            return ElasticsearchTemplateProvider.GetTemplate(
                _options,
                DiscoveredVersion,
                settings,
                _templateMatchString,
                _options.AutoRegisterTemplateVersion);

        }

my use case is to add the setting "index.mapping.total_fields.limit" to increase the limit of 1000 fields.

Thanks

[mivano]

The easiest way now is to supply your own template. Otherwise you will need to add this as an option similar as the number_of_shards etc functionality.

[ffMathy]

This would be super useful.

index.mapping.total_fields.limit has an annoying default.

[mivano]

Although I can agree that is it annoying to run into, it might be there for a reason and having more than 1000 fields can impact the performance. I m not sure how to handle this elegantly, offering a way to increase it, makes it too easy to change the default and maybe safe setting.

[ffMathy]

@mivano I agree with your perspectives. However, in our case, Elasticsearch is used only for logging, and contains data that we could live without in the case of a breakdown.

Also, I believe that responsibility should be in the developer's hands, not yours. No offence, but I don't think it's your job as a library creator to discipline other developers through that library. If you want, you can document the setting describing that it might have a bad effect, or even produce a warning in the logs if it's altered. But forcing them to not use it seems like "babysitting" in a way, if you know what I mean?

C# also allows you to delete files. But it's not like you're gonna delete the user's documents and desktop directories as a developer, because you know that's a bad idea. I could mention countless other libraries that allows you to do something stupid, but because the developer is reasonable, it isn't happening.

To back you up a bit though, I do agree that there are also cases where people choose MD5 for password hashing etc, but once again in my opinion, that should be solved through education and spreading knowledge.

[mivano]

Good point, the custom template gives you now all the freedom, but also requires a lot of knowledge. The total field limit can become pretty annoying, so if this is easily configured, it should be fine.

Care for a PR? The above proposed solution looks pretty flexible.

[ffMathy]

@tresoldigiorgio got time? 😏

[tresoldigiorgio]

I can do it next week.

[mivano]

Any update on this?

[bferdinandus]

Hi, when is the production release of 8.1.0 ready. I really would like to use the TemplateCustomSettings, but I am not allowed to use alpha versions.

[mivano]

Sorry, took some time due to expired nuget keys, but a 8.1 version is now available: https://www.nuget.org/packages/Serilog.Sinks.ElasticSearch/8.1.0