Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Full-weight cluster support #207

Open
akvlad opened this issue Aug 31, 2022 · 0 comments
Open

Full-weight cluster support #207

akvlad opened this issue Aug 31, 2022 · 0 comments
Assignees

Comments

@akvlad
Copy link
Collaborator

akvlad commented Aug 31, 2022

Current status

Cluster support created in https://gist.github.com/coelho/c3b7bbb2c95caa61115d93692f9e4ae2 gist and copied into our wiki page.
It's a bit hacky as it needs non-trivial manual approach to run.

Improvement request

Environmental cluster configuration

Create an env variables to specify clustered clickhouse support

CLICKHOUSE_CLUSTER_NAME=clickhouse_cluster
CLUSTERED_CLICKHOUSE_URLS=http://root:pass@clickhouse1;http://root2:pass2@clickhouse2

If these variables are provided, then qryn should run create distributed tables in order to support a clickhouse cluster.
Qryn reader and writer should do requests to the distributed tables instead of "stand alone" ones

Advanced clustering configurations

According to the conversation in #198 :
Provide the special ADVANCED_CLUSTERING_KEY env variable to specify advanced clustering key support.
It can have these values:

  • random for rand() as sharding key
  • fingerprint for fingerprint as sharding key
  • label1;label2;20s as the custom "label-period" sharding key. In this case label1;label2 is semicolon separated label names and 20s is the duration to use for sharding.

Replicated.... tables support and cloud support

Another "cloud" type of clustering is the clustering based on the clickhouse keeper and ReplicatedMerge... tables family. It is also a nice-to-have feature as discussed in #170 .
It's not provided automatically for now.
We need to provide an env variable:
REPLICATED_TABLES=1
If it is provided all the tables with MergeTree family engine should be moved to ReplicatedMergeTree analogues.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant