[Lens] New legend inside chart can be more smart in terms of layout

[dej611]

Describe the feature:

The new Legend "inside chart" option in Lens ( #105571 ) offers a new Number of columns control which has been currently limited to 5 columns, preventing the user to create legends overflow for their values.

While the current feature works, it strongly depends on the width of the category label, which can be relatively long, and the user has to manually check what is the best number to fit in there.

For instance in this case of IP values 5 columns is too much:

I would propose a "smart" option for this case, where instead of Number of columns, the user can decide whether the legend should better use either the "Vertical space", or the "Horizontal space": this means that the chart library has to workout the best number of columns in order to display the values and make it harder to have it overflow, even for resized panels.

For instance in the case above with the IP addresses adopting the Horizontal space option should lead to a 4 column in that particular panel size. Resizing the panel should adjust it and probably reduce the columns to 3 and increase the number of rows to 3 as well.

What do you think @stratoula ? cc @markov00 @monfera

[elasticmachine]

Pinging @elastic/kibana-app (Team:KibanaApp)

[monfera]

We've already got a couple of very simple solvers in Charts, eg. a mixed integer, constrained gradient descent function, which could be used for identifying the minimal or maximal value for an acceptable column count; it demands a monotonic cost function (ie. it'll get stuck in a local optimum, if any). There's also a grid search function, though not yet in a PR, which would help with local optima here, and suitable in runtime, as the dimension count is low (1: column count) and the possible values are few (1, 2, 3, ... , approx. 10).

We are also contemplating a simple stochastic optimizer for eventual placement of annotations, eg. whether the chart has an empty, data ink free corner to put a legend of some aspect ratio, and where to put non-overlapping point annotations. But there are still many questions around anything stochastic, glad to chat about it.

Finally, we're on a long term thread of Charts disaggregation, @nickofthyme has made the first giant technical step, which is, conversion of the Charts repo into a monorepo that has charts but also, independently usable components. The solvers should, over time, migrate into the toolbox area so they can be used directly, so a client like Lens, Canvas and Dashboard could also derive optimal inputs to charts (or Vega or whatever) based on its own goals

[markov00]

related to legend redesign elastic/elastic-charts#2096