Added KV Store client that mirrors one backend to another (#1749) · cyriltovena/loki@388b79f · GitHub

Commit

Added KV Store client that mirrors one backend to another (grafana#1749)

Browse files

* Added new KV Store client, MultiClient.

This client is configured with multiple stores, one of them is designated as
primary. All client operations are forwarded to the primary store.

MultiClient also does "mirroring" of values from primary to secondary store.

MultiClient can listen on changes in runtime configuration (via overrides
mechanism), and switch primary store and enable/disable mirroring.

Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com>

* Use Stop, which is now part of kv.Client.

Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com>

* Put back setting of defaultLimits -- used when loading YAML files.

Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com>

* Moved setting of default limits for YAML unmarshal to separate function.

Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com>

* Pass multi-client context as argument.

Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com>

* watchConfigChannel now reacts on context being done as well

Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com>

* Changed Mirroring to *bool.

Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com>

* Ignore mock by yaml.

Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com>

* Renamed mirroring to mirror-enabled to be consistent with MultiConfig.

Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com>

* Renamed 'multi' to 'multi_kv_config' in overrides.yaml.

Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com>

* Forward writes done via CAS function to secondary client.

Mirroring goroutine was removed, and replaced by forwarding writes
done via CAS function to secondary client. Rate limits config was
removed, but there is now timeout for secondary write, to avoid
blocking CAS function for too long, if secondary write is slow
(eg. etcd being down can cause very long writes).

Only WatchKey and WatchPrefix functions now react on change of
primary client.

Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com>

* Added metrics to multi client.

Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com>

* Removed equality check when writing to secondary store.

Without watch-and-mirror functionality, there is no need to check
if value is already present in the secondary store.

Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com>

* Renamed OverridesManager and moved it to its own package.

Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com>

* Make lint happy, 3.

Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com>

* Make lint happy, 4.

Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com>

* Add metric type to variable names, yaml name changes, fixed metric names, removed forgotten log.

Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com>

* Fixed yet one more yaml name.

Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com>

* Fixed tests after changing yaml fields.

Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com>

* Fix bug when default limits are not applied until next overrides reload.

Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com>

* Ignore LoadConfig if LoadPath is empty.

Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com>

* Use channels to communicate config updates.

Instead of spawning new goroutine for each config update,
we now use channels to communicate config updates.

Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com>

* Initialize limits before starting runtimeconfig Manager.

Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com>

* Updated CHANGELOG.md and arguments.md.

Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com>

* Typo

Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com>

* Fix compilation error in ingester_v2_test.go.

Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com>

* Fixed error after rebase.

Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com>

* Fixed error after rebase.

Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com>

* Use logger with component="multikv" to log messages.

Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com>

* Improve log message when runtime config file is not specified.

Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com>

* Don't use memberlist in the example, as it is still experimental.
Addressed other review feedback.

Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com>

Loading branch information

pstibrany authored and gouthamve committed Jan 7, 2020

1 parent 2855375 commit 388b79f

chunk_store_test.go

-Original file line number
+Diff line change
@@ Expand Up @@
     	var limits validation.Limits
     	flagext.DefaultValues(&limits)
     	limits.MaxQueryLength = 30 * 24 * time.Hour
-    	overrides, err := validation.NewOverrides(limits)
+    	overrides, err := validation.NewOverrides(limits, nil)
     	require.NoError(t, err)
     	store := NewCompositeStore()
@@ Expand Down @@

storage/caching_fixtures.go

-Original file line number
+Diff line change
@@ Expand Up / @@ -41,5 +41,5 @@ func defaultLimits() (*validation.Overrides, error) { @@
     	var defaults validation.Limits
     	flagext.DefaultValues(&defaults)
     	defaults.CardinalityLimit = 5
-    	return validation.NewOverrides(defaults)
+    	return validation.NewOverrides(defaults, nil)
     }

storage/factory_test.go

-Original file line number
+Diff line change
@@ Expand Up / @@ -30,7 +30,7 @@ func TestFactoryStop(t *testing.T) { @@
     		},
     	}
-    	limits, err := validation.NewOverrides(defaults)
+    	limits, err := validation.NewOverrides(defaults, nil)
     	require.NoError(t, err)
     	store, err := NewStore(cfg, storeConfig, schemaConfig, limits)
@@ Expand Down @@

0 comments on commit `388b79f`

Please sign in to comment.