Design of creating user defined multiple database instances #271

dzhangalibaba · 2018-10-24T05:07:02Z

Detail explanation of the design for creating user defined multiple database instances
Pull request for implementation is multi-DB Part 1: create multiple Redis DB instances based on CONFIG at /etc/sonic/database_config.json sonic-buildimage#2182

Signed-off-by: Dong Zhang d.zhang@alibaba-inc.com

dzhangalibaba · 2018-12-05T18:24:50Z

@lguohan @qiluo-msft @jipanyang
Hi guys, these is the detail design doc for the multiple db instances implementation.
On local in our lab , I made all the changes and the DUT runs for a few days and looks good. If you get a chance , could you start to look at this design and related PR? And the first part of PR code is done at sonic-net/sonic-buildimage#2182 . Let me know what you think online or offline. Thanks.

jipanyang · 2018-12-13T12:51:55Z

you mentioned that to use redis cluster solution, client has to use TCP + PORT instead of unix socket, do you have some rough single client read/write performance benchmark for the two different connections against single redis instance?

dzhangalibaba · 2018-12-13T13:05:34Z

you mentioned that to use redis cluster solution, client has to use TCP + PORT instead of unix socket, do you have some rough single client read/write performance benchmark for the two different connections against single redis instance?

update the benchmark result in DOC

jipanyang · 2018-12-15T12:56:34Z

doc/database/multi_database_instances.md

+     - [x] **generate corresponding runtime ping/PONG check script as well to check if database instances are running later**
+   - [x] **docker ENTRYPOINT : docker_init.sh**
+     - [x] **at this point, we know the DATABASE configuration in database_config.json**
+     - [x] **generate supervisord.conf and all redis.conf before database docker start**


Since it has been agreed that database instance configuration should be available at build time, no supervisord.conf needs to be generated on the fly during running time, could you update the description here?

jipanyang · 2018-12-15T12:59:23Z

doc/database/multi_database_instances.md

+          },
+          "redis_ins_002":{
+              "port": 6381,
+              "databases" : ["APPL_DB","ASIC_DB", "CONFIG_DB"]


Though this is just an example configuration, as a more reasonable db allocation among instances, probably APPL_DB","ASIC_DB", "CONFIG_DB" should be put into different db instances to make parallel db processing possible at critical path.

jipanyang · 2018-12-15T13:24:10Z

doc/database/multi_database_instances.md

+MSET (10 keys): 5550.62 requests per second 
+```
+
+**So I don't think redis cluster is a good way to solve the problem of splitting databases into multiple redis instances in SONiC.**


One concern I have about deploying redis cluster in sonic is that redis cluster replicates data among multiple redis instances asynchronously and doesn't guarantee strong consistency. Not sure whether it is possible that different client may get different data view.

qiluo-msft · 2019-04-09T21:21:26Z

For backward compatibility, please keep the option to specify unix socket for each redis instance. We prefer it for performance issue. #Closed

qiluo-msft

As comments

dzhangalibaba · 2019-04-10T02:09:20Z

For backward compatibility, please keep the option to specify unix socket for each redis instance. We prefer it for performance issue.

we have uinx socket, by default each inistance has a unix socket at "/var/run/instance_name/instance_name.sock",
for example，
redis ： "/var/run/redis/redis.sock"
redis_ins_2: "/var/run/redis_ins_2/redis_ins_2.sock"
...

you mean the user define the unix socket path and name in database_config.json file ? #Closed

qiluo-msft · 2019-04-10T02:12:17Z

Thanks! Reading through the document, I am not aware this design detail.

In reply to: 481505546 [](ancestors = 481505546)

…base_config.json (#2182) this is the first step to moving different databases tables into different database instances in this PR, only handle multiple database instances creation based on user configuration at /etc/sonic/database_config.json we keep current method to create single database instance if no extra/new DATABASE configuration exist in database_config.json file. if user try to configure more db instances at database_config.json , we create those new db instances along with the original db instance existing today. The configuration is as below, later we can add more db related information if needed: { ... "DATABASE": { "redis-db-01" : { "port" : "6380", "database": ["APPL_DB", "STATE_DB"] }, "redis-db-02" : { "port" : "6381", "database":["ASIC_DB"] }, } ... } The detail description is at design doc at sonic-net/SONiC#271 The main idea is : when database.sh started, we check the configuration and generate corresponding scripts. rc.local service handle old_config copy when loading new images, there is no dependency between rc.local and database service today, for safety and make sure the copy operation are done before database try to read it, we make database service run after rc.local Then database docker started, we check the configuration and generate corresponding scripts/.conf in database docker as well. based on those conf, we create databases instances as required. at last, we ping_pong check database are up and continue  Signed-off-by: Dong Zhang d.zhang@alibaba-inc.com

update design to up-to-date

…base_config.json (sonic-net#2182) this is the first step to moving different databases tables into different database instances in this PR, only handle multiple database instances creation based on user configuration at /etc/sonic/database_config.json we keep current method to create single database instance if no extra/new DATABASE configuration exist in database_config.json file. if user try to configure more db instances at database_config.json , we create those new db instances along with the original db instance existing today. The configuration is as below, later we can add more db related information if needed: { ... "DATABASE": { "redis-db-01" : { "port" : "6380", "database": ["APPL_DB", "STATE_DB"] }, "redis-db-02" : { "port" : "6381", "database":["ASIC_DB"] }, } ... } The detail description is at design doc at sonic-net/SONiC#271 The main idea is : when database.sh started, we check the configuration and generate corresponding scripts. rc.local service handle old_config copy when loading new images, there is no dependency between rc.local and database service today, for safety and make sure the copy operation are done before database try to read it, we make database service run after rc.local Then database docker started, we check the configuration and generate corresponding scripts/.conf in database docker as well. based on those conf, we create databases instances as required. at last, we ping_pong check database are up and continue  Signed-off-by: Dong Zhang d.zhang@alibaba-inc.com

dzhangalibaba mentioned this pull request Oct 28, 2018

multi-DB Part 1: create multiple Redis DB instances based on CONFIG at /etc/sonic/database_config.json sonic-net/sonic-buildimage#2182

Merged

stcheng added the enhancement label Nov 2, 2018

dzhangalibaba force-pushed the database_doc branch from 68c22f2 to a1bc19e Compare December 3, 2018 23:47

jipanyang reviewed Dec 15, 2018

View reviewed changes

qiluo-msft requested changes Apr 9, 2019

View reviewed changes

dzhangalibaba force-pushed the database_doc branch from ed69d42 to fe22fa0 Compare August 6, 2019 23:30

dzhangalibaba force-pushed the database_doc branch from fe22fa0 to b24ce54 Compare September 27, 2019 23:14

dzhangalibaba changed the title ~~Design of creating user defined multiple database instances~~ Design of creating user defined multiple database instances [outdated, need update when all PRs are in] Jan 17, 2020

dzhangalibaba added 15 commits February 5, 2020 14:29

multiple user-defined DB inst

3a34372

multiple user-defined DB inst format

9054c84

update png

4e1f382

update next step

bbde7b4

update motivation and cluster effect

b8a9423

update detail design

57869f7

fix type

3414a74

update redis cluster analysis

4dfe3d6

update redis cluster analysis more

38e51dc

update redis cluster analysis typo

c633119

update benchmark

bd596c2

update wb

3ce5098

update format

3b3840c

update design to allow user to config

396a6e6

update typo

9da988c

dzhangalibaba and others added 6 commits February 5, 2020 14:29

update typo as well

1052c57

update typo

d1e2ff3

update doc for keeping database docker entrypoint

293f07a

update doc for added supervisor_database.conf

784a7b6

Update multi_database_instances.md

deb68fc

update design to up-to-date

Update multi_database_instances.md

41f3c94

dzhangalibaba force-pushed the database_doc branch from 886c20b to 41f3c94 Compare February 5, 2020 22:30

dzhangalibaba changed the title ~~Design of creating user defined multiple database instances [outdated, need update when all PRs are in]~~ Design of creating user defined multiple database instances Feb 5, 2020

Update multi_database_instances.md

24f583e

qiluo-msft approved these changes Feb 5, 2020

View reviewed changes

qiluo-msft merged commit 2ac24d2 into sonic-net:master Feb 5, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Design of creating user defined multiple database instances #271

Design of creating user defined multiple database instances #271

dzhangalibaba commented Oct 24, 2018

dzhangalibaba commented Dec 5, 2018

jipanyang commented Dec 13, 2018

dzhangalibaba commented Dec 13, 2018

jipanyang Dec 15, 2018

dzhangalibaba Dec 17, 2018

jipanyang Dec 15, 2018

dzhangalibaba Dec 17, 2018

jipanyang Dec 15, 2018

qiluo-msft commented Apr 9, 2019 •

edited

Loading

qiluo-msft left a comment

dzhangalibaba commented Apr 10, 2019 •

edited by qiluo-msft

Loading

qiluo-msft commented Apr 10, 2019

Design of creating user defined multiple database instances #271

Design of creating user defined multiple database instances #271

Conversation

dzhangalibaba commented Oct 24, 2018

dzhangalibaba commented Dec 5, 2018

jipanyang commented Dec 13, 2018

dzhangalibaba commented Dec 13, 2018

jipanyang Dec 15, 2018

Choose a reason for hiding this comment

dzhangalibaba Dec 17, 2018

Choose a reason for hiding this comment

jipanyang Dec 15, 2018

Choose a reason for hiding this comment

dzhangalibaba Dec 17, 2018

Choose a reason for hiding this comment

jipanyang Dec 15, 2018

Choose a reason for hiding this comment

qiluo-msft commented Apr 9, 2019 • edited Loading

qiluo-msft left a comment

Choose a reason for hiding this comment

dzhangalibaba commented Apr 10, 2019 • edited by qiluo-msft Loading

qiluo-msft commented Apr 10, 2019

qiluo-msft commented Apr 9, 2019 •

edited

Loading

dzhangalibaba commented Apr 10, 2019 •

edited by qiluo-msft

Loading