#Welcome to Kunlun-storage database instance installation guide!
##Intro
Follow this document to install a KunlunBase cluster or a kunlun-storage node in a few steps using a script and a GUI web application, much easier, faster and more convenient than the instructions in this document.
For more information and resources of KunlunBase, such as software, bug reports and features planned/under development, and release notes, please visit our website Download KunlunBase and use it for free at our download site, and refer to all of KunlunBase documentation
For latest release notes, see our online release notes ##Build from source
The same as community mysql or percona-server --- use cmake to configure it then do something like 'make install -j8'.
A typical cmake configuration we often use is as below. Note that we do use jemalloc, a shared object of prebuilt jemalloc is provided and is installed to each kunlun-storage db instance.
cmake .. -DCMAKE_BUILD_TYPE=Release -DCMAKE_INSTALL_PREFIX= -DWITH_SSL=system -DWITH_BOOST= -DWITH_KEYRING_VAULT=0 -DWITH_ZLIB=bundled -DWITH_TOKUDB=NO -DWITH_EDITLINE=bundled -DWITH_LTO=0 -DWITH_ROUTER=0 -DWITH_FEDERATED_STORAGE_ENGINE=1 -DWITH_ARCHIVE_STORAGE_ENGINE=1 -DWITH_NGRAM_PARSER=1 -DWITHOUT_ROCKSDB=1 -DWITH_ZSTD=bundled -DWITH_LZ4=bundled -DWITH_PROTOBUF=bundled -DWITH_LIBEVENT=bundled -DWITH_JEMALLOC=0
After 'make install' completed, in root source directory do 'bash post-install '
##Library dependencies
If you are using a kunlun-storage program built from source on the same Linux distribution and version as where it's being used, simply skip this step because there is no dependency issues.
Add Kunlun-storage/lib into LD_LIBRARY_PATH.
All dynamic shared objects (*.so files) that programs in Kunlun-storage/bin depend on, are provided in Kunlun-storage/lib/deps directory. Try startup mysqld (e.g. mysqld --version) and see if your local Linux distro needs any of the *.so files. If so, copy needed ones into Kunlun-storage/lib.
DO NOT copy everything in deps into lib at once, otherwise your linux OS or any software may not be able to work because of library version mismatches!
Use the ./dba_tools/install-mysql.py script to install an MGR node. To do so, one first needs to prepare an MGR cluster configuration file using the template in dba_tools/mgr_config.json. In this doc below the config file will be called 'my-shards.json'.
One should set the 'onfig' argument to the path of the MGR cluster configuration file.
One should set the 'target_node_index' argument of install-mysql.py to specify the target db instance to install. 'target_node_index' is the target db instance's array index in the 'nodes' array of my-shards.json. Note that one should always install the group replication primary node before installing any of its replicas, and make sure the primary node specified in config file is really currently the primary node of the MGR shard.
Then in Kunlun-storage/dba_tools directory, do below:
python install-mysql.py --config=./my-shards.json --target_node_index=0
There are other optional parameters to install-mysql.py, but they are not used for now or can be generated automatically except the 'dbcfg' argument, as detailed below.
To startup mysql server, use the script 'startmysql.sh' Kunlun-storage/dba_tools: ./startmysql.sh 3306 There are several other useful scripts in Kunlun-storage/dba_tools, use 'imysql.sh port' to connect to a db instance created by install-mysql.py; Use 'monitormysql.sh port' to see current QPS and other performance counters; Use 'stopmysql.sh port' to stop a mysqld server process.
There are multiple db instance config template files in ./dba_tools directory. Among them, ./dba_tools/template.cnf is the default if 'dbcfg' argument isn't specified in install-mysql.py execution, and should always be used for Kunlun-storage db instances. Settings in ./dba_tools/template.cnf are premium for best performance, suitable for high performance mysql servers.
For the rest template files, the ./dba_tools/mysql-template.cnf is suitable for original mysql-8.0.x (x >=18) instances, and ./dba_tools/percona-template.cnf is suitable for original percona-mysql-8.0.18-9. Both of them originated from ./dba_tools/template.cnf, with unsupported arguments commented out and all other settings are the same as ./dba_tools/template.cnf.
And ./dba_tools/template-small.cnf is suitable for Kunlun-storage db instances with trivial resource consumption, e.g. I use it for Kunlun DDC cluster installation testing. One could use another template file as necessary for other purposes.
If one needs other config templates than ./dba_tools/template.cnf, specifiy the 'dbcfg' argument. For example when you have an binary installation of original mysql-8.0.x or a percona-mysql-8.0.x-y, you can copy the Kunlun-storage/dba_tools directory into that installation's root directory, in order to make use of the installation scripts and config templates. And you can create another template file and use it to create db instances as below example.
e.g. python install-mysql.py --config=./mgr_config0.json --target_node_index=0 --dbcfg=./mysql-template-small.cnf
{ "group_uuid":"e6578c3a-547e-11ea-9780-981fd1bd410d", # mgr group uuid. should be unique in a Kunlun DDC cluster for best readability. "nodes": #each node contains configs of a db instance. [ { "is_primary":true, # whether this db instance is a primary node. there must be one and only one primary node in the 'nodes' array. "ip": "127.0.0.1", # ip address to listen on, bind_address and report_host are set to this ip. "port": 6001, # regular listen port "xport": 60010, # mysql X protocol listen port "mgr_port": 60011, # port for mgr communication between an MGR group. not used by clients. "innodb_buffer_pool_size":"64MB", # innodb buffer pool size "data_dir_path":"/data", # innodb data files are stored here "log_dir_path":"/data/log", # binlogs and innodb redo logs and mysqld.err are stored in this path. "user":"abc", # A valid OS user name, the created db instance's all data/log dirs/files will be owned by this user. "election_weight":50 # mgr primary election weight. don't modify it without reading the relevant doc first. } , more shard node (db instance) config objects ] }
This binary is build using below cmake configuration. Note that WITH_JEMALLOC is 0 because there would be error during cmake generation or build phase if it's 1 on some Linux distributions. But a prebuilt jemalloc shared object library file is used when starting up mysqld, see dba_tools/bootmysql.sh for details. Setting WITH_LTO = 1 could produce a little bit of performance gains but it only works for a narrow range of gcc versions.
cmake .. -DCMAKE_BUILD_TYPE=Release -DCMAKE_INSTALL_PREFIX=/home/abc/mysql_installs/kunlun-storage-bin-rel -DWITH_SSL=system -DWITH_BOOST=/home/abc/Downloads/boost_1_70_0 -DWITH_KEYRING_VAULT=0 -DWITH_ZLIB=bundled -DWITH_TOKUDB=NO -DWITH_EDITLINE=bundled -DWITH_LTO=0 -DWITH_ROUTER=0 -DWITH_FEDERATED_STORAGE_ENGINE=1 -DWITH_ARCHIVE_STORAGE_ENGINE=1 -DWITH_NGRAM_PARSER=1 -DWITHOUT_ROCKSDB=1 -DWITH_ZSTD=bundled -DWITH_LZ4=bundled -DWITH_PROTOBUF=bundled -DWITH_LIBEVENT=bundled -DWITH_JEMALLOC=0