This should remove the repetition of the ingester ID - a ~25 character string, repeated 128 times (3kb) for each ingester, allocated every 5 seconds.  For a 10 ingester cluster, thats 60KB/s of allocations (1/5s * 10 ingesters * 128 tokens * 10 ingester).  Yes, its N^2.

Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>

Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com>

…luster with RF 3 and only 3 nodes. (#2503)

* Use a real ring with mock KV when testing distributor.

This is to teast out errors in the replication logic.

Signed-off-by: Tom Wilkie <tom@grafana.com>

* Extend distributor test to cover the case RF=3 with 2 ingesters.

Signed-off-by: Tom Wilkie <tom@grafana.com>

* Ensure ring correctly calculates the number of allowed failures when RF=3 and #ingesters=2.

Signed-off-by: Tom Wilkie <tom@grafana.com>

* Add changelog and review feedback.

Signed-off-by: Tom Wilkie <tom@grafana.com>

* Refactor some distributor tests to try and get them to pass.

Signed-off-by: Tom Wilkie <tom@grafana.com>

* Speed up tests but polling more frequently.

Signed-off-by: Tom Wilkie <tom@grafana.com>

* Fix same bug on the write path.

Signed-off-by: Tom Wilkie <tom@grafana.com>

* Tidy up the distributor tests.

Signed-off-by: Tom Wilkie <tom@grafana.com>

* Make test correctly handle RF3 and 2 ingesters - previously was succeeding when it shouldn't

Signed-off-by: Tom Wilkie <tom@grafana.com>

* Update pkg/ring/ring.go

Co-Authored-By: Jacob Lisi <jacob.t.lisi@gmail.com>
Signed-off-by: Tom Wilkie <tom@grafana.com>

Co-authored-by: Jacob Lisi <jacob.t.lisi@gmail.com>

* Added benchmarks on blocks storage

Signed-off-by: Marco Pracucci <marco@pracucci.com>

* Moved ResetTimer()

Signed-off-by: Marco Pracucci <marco@pracucci.com>

Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com>

Signed-off-by: Tyler Reid <tyler.reid@grafana.com>

Signed-off-by: Tyler Reid <tyler.reid@grafana.com>