kunlun数据库手动集群安装

本操作在Ubuntu 20.04.2 LTS虚拟机单机搭建通过虚拟机内存4G就可以，最好8G

该系统包括： 1.存储分片2个，每个分片3个节点，一个分片用于存储，一个分片用于元数据。 2.计算节点3个. 3.管理节点1个。配置文件见文章末尾。

================================================================================================================= 可能需要部分安装包 1. sudo apt-get install vim python2

2. 计算节点安装缺少库，需要复制自带的库文件到系统目录

sudo cp /home/kunlun/work01/postgresql-11.5-rel/lib/libpq.so.5 /usr/lib/

sudo cp /home/kunlun/work01/postgresql-11.5-rel/lib/libmariadb.so.3 /usr/lib/ 之后删除计算节点的data目录，和计算节点的etc/* 再次安装

3.元数据节点创建，报错 ImportError: No module named mysql.connector
    在postgresql-11.5-rel/resources目录下

tar zxf mysql-connector-python-2.1.3.tar.gz

cd mysql-connector-python-2.1.3/

python2 setup.py build

sudo python2 setup.py install

4.psycopg2安装需要 setuptools

wget https://pypi.python.org/packages/source/s/setuptools/setuptools-0.6c11.tar.gz

tar zxvf setuptools-0.6c11.tar.gz

cd setuptools-0.6c11

python2 setup.py build

sudo python2 setup.py install

5.集群创建，报错 ImportError: No module named psycopg2
    用ppostgresql-11.5-rel/resources目录下psycopg2-2.8.4.tar.gz安装

tar zxf psycopg2-2.8.4.tar.gz

cd psycopg2-2.8.4/

python2 setup.py build

sudo python2 setup.py install

    如果安装错误，先试试下面，再安装一次

sudo apt-get install libpq-dev python-dev 1.sudo apt install python3-pip 1.pip install psycopg2

================================================================================================================= 安装错误，重装注意： ps -ef | grep mysql 和ps -ef | grep postgres 来查看有没有占用的端口进程，有就kill掉，或者重启系统

然后删除以下文件：

1.存储节点的 data目录 #配置JSON文件指定目录，否则报路径已经存在 1.计算节点的 data目录 1.percona-8.0.18-bin-rel/etc/* #否则报端口已经存在 1.postgresql-11.5-rel/etc/*

================================================================================================================= 一：存储节点（mysql）

1.把对应的包从官网上拉下来  http://downloads.zettadb.com/releases/0.8.3/release-binaries (不同版本的路径是不一样的）
    如：cd /home/kunlun/work01/

wget http://zettatech.tpddns.cn:14000/releases/0.8.3/release-binaries/percona-8.0.18-bin-rel.tgz

wget http://zettatech.tpddns.cn:14000/releases/0.8.3/release-binaries/cluster_mgr_rel.tgz

wget http://zettatech.tpddns.cn:14000/releases/0.8.3/release-binaries/postgresql-11.5-rel.tgz tar -zxf 解压三个文件

2.把percona/lib添加到环境变量：

sudo vim /etc/profile

export LD_LIBRARY_PATH=/home/kunlun/work01/percona-8.0.18-bin-rel/lib:$LD_LIBRARY_PATH

export PATH=/home/kunlun/work01/percona-8.0.18-bin-rel/bin:$PATH

export LD_LIBRARY_PATH=/home/kunlun/work01/postgresql-11.5-rel/lib:$LD_LIBRARY_PATH

export PATH=/home/kunlun/work01/postgresql-11.5-rel/bin:$PATH

source /etc/profile

3.在percona/dba_tools下改mgr_config.json，并更改为文件名mysql_shard1.json， mysql_shard2.json
    两个文件，三个节点ip全改成对应服务器的ip
    innodb_log_dir_path，data_dir_path和log_dir_path全改成自己指定的目录，如果已经有了下一步会报错
    user全改成自己当前的用户（如当前用户是kunlun就改成kunlun)

4.在dba_tools下生成主节点0，1，2，如：

$$$ sudo python2 install-mysql.py mgr_config=./mysql_shard1.json target_node_index=0 #,1,2 修改该命令的target_node_index为1和2各生成一次，生成备用节点1和2

$$$ sudo python2 install-mysql.py mgr_config=./mysql_shard2.json target_node_index=0 #,1,2 修改该命令的target_node_index为1和2各生成一次，生成备用节点1和2

5.安装后，第二次启动，在dba_tools下

./startmysql.sh 6001 #6002 6003...

6.登录验证，在dba_tools下

./imysql.sh 6001 #6002 6003...

============================================================================================================== 二.计算节点（postgresql）

5.在postgresql/scripts修改comp-nodes.json文件，并更改为文件名postgresql_comp.json
    修改ip、user、password和datadir（这一步的端口不可以和上面mysql的端口一样，不然会冲突报错）

6.在scripts下执行

$$$ python2 install_pg.py config=postgresql_comp.json install_ids=1 #,2,3 然后各把installable_ids 改成2、3执行，共三次

7.安装后，第二次启动

python2 start_pg.py port=5401 #5402 ...

8.登录验证

psql postgres://abc:abc@127.0.0.1:5401/postgres #5402 ...

9.修改meta-shard.json，shards-config.json, 并更改为文件名postgresql_meta.json， postgresql_shards.json
    打开，修改ip，port，user，password（这里的ip要和mysql的一样，不然会连接不上mysql）
    要和第3步的ip，port对应，user为pgx，password为pgx_pwd（写死的）

10.元数据节点创建，在scripts下执行

$$$ python2 bootstrap.py --config=./postgresql_meta.json --bootstrap_sql=./meta_inuse.sql 在执行该命令的时候要注意mysql有没有在环境变量下，用which mysql看。如果没有就用第2步的来做，把percona/bin放到PATH下

11.集群创建，在scripts下执行

$$$ python2 create_cluster.py --shards_config ./postgresql_shards.json --comps_config ./postgresql_comp.json --meta_config ./postgresql_meta.json --cluster_name clust1 --cluster_owner abc --cluster_biz test 如果出现了 ImportError: No module named psycopg2 该报错信息，就要在postgresql/resource里面解压一个psycopg的包，再进入到这个包里面，用python2 setup.py build来编译，再用sudo python2 setup.py install进行安装（mysql中也有）

12.reload computing node

ps -ef | grep postgres

cd ~/postgresql/bin 1../pg_ctl reload -D /db/datadir/path

================================================================================================================= 集群管理节点：

13.编辑cluster_mgr.cnf

cd cluster_mgr_rel/bin

vim cluster_mgr1.cnf 把里面的port，ip，user，passwd改成postgresql/scripts下postgresql_meta.json的主节点的port，ip，user，passwd

14.运行cluster_mgr文件
    运行方式：./cluster_mgr cluster_mgr1.cnf &
    (cluster_mgr.cnf是刚刚编辑的cluster_mgr.cnf配置文件)（最好放后台运行）

cat cluster_mgr1-*.log #查看运行的LOG是否正常

15.验证集群管理节点构建成功
    运行一个测试脚本，用配置的computing node来连接（回到postgresql/scripts/comp-node.json查看计算节点的配置信息）
    成功运行就是构建成功

================================================================================================================= 附带节点文件

存储分片1，三个节点

mysql_shard1.json
{

"group_uuid":"e6578c3a-547e-11ea-9780-981fd1bd410d",

```
"nodes":
```
```
[
```
```
   {
```
```
      "is_primary":true,
```
```
      "ip": "127.0.0.1",
```
```
      "port": 6001,
```
```
      "xport": 60010,
```
```
      "mgr_port": 60011,
```

      "innodb_buffer_pool_size":"64MB",

      "innodb_log_dir_path":"/home/kunlun/work01/data_mysql/data1_innodb_log",

      "data_dir_path":"/home/kunlun/work01/data_mysql/data1",

      "log_dir_path":"/home/kunlun/work01/data_mysql/data1_log",

```
      "user":"kunlun",
```
```
      "election_weight":50
```
```
   },
```
```
   {
```
```
      "is_primary":false,
```
```
      "ip": "127.0.0.1",
```
```
      "port": 6002,
```
```
      "xport": 60020,
```
```
      "mgr_port": 60021,
```

      "innodb_buffer_pool_size":"64MB",

      "innodb_log_dir_path":"/home/kunlun/work01/data_mysql/data2_innodb_log",

      "data_dir_path":"/home/kunlun/work01/data_mysql/data2",

      "log_dir_path":"/home/kunlun/work01/data_mysql/data2_log",

```
      "user":"kunlun",
```
```
      "election_weight":50
```
```
   },
```
```
   {
```
```
      "is_primary":false,
```
```
      "ip": "127.0.0.1",
```
```
      "port": 6003,
```
```
      "xport": 60030,
```
```
      "mgr_port": 60031,
```

      "innodb_buffer_pool_size":"64MB",

      "innodb_log_dir_path":"/home/kunlun/work01/data_mysql/data3_innodb_log",

      "data_dir_path":"/home/kunlun/work01/data_mysql/data3",

      "log_dir_path":"/home/kunlun/work01/data_mysql/data3_log",

```
      "user":"kunlun",
```
```
      "election_weight":50
```
```
   }
```
```
]
```
}

存储分片2，三个节点

mysql_shard2.json
{

"group_uuid":"e6578c3b-547e-11ea-9780-981fd1bd410d",

```
"nodes":
```
```
[
```
```
   {
```
```
      "is_primary":true,
```
```
      "ip": "127.0.0.1",
```
```
      "port": 6004,
```
```
      "xport": 60040,
```
```
      "mgr_port": 60041,
```

      "innodb_buffer_pool_size":"64MB",

      "innodb_log_dir_path":"/home/kunlun/work01/data_mysql/data4_innodb_log",

      "data_dir_path":"/home/kunlun/work01/data_mysql/data4",

      "log_dir_path":"/home/kunlun/work01/data_mysql/data4_log",

```
      "user":"kunlun",
```
```
      "election_weight":50
```
```
   },
```
```
   {
```
```
      "is_primary":false,
```
```
      "ip": "127.0.0.1",
```
```
      "port": 6005,
```
```
      "xport": 60050,
```
```
      "mgr_port": 60051,
```

      "innodb_buffer_pool_size":"64MB",

      "innodb_log_dir_path":"/home/kunlun/work01/data_mysql/data5_innodb_log",

      "data_dir_path":"/home/kunlun/work01/data_mysql/data5",

      "log_dir_path":"/home/kunlun/work01/data_mysql/data5_log",

```
      "user":"kunlun",
```
```
      "election_weight":50
```
```
   },
```
```
   {
```
```
      "is_primary":false,
```
```
      "ip": "127.0.0.1",
```
```
      "port": 6006,
```
```
      "xport": 60060,
```
```
      "mgr_port": 60061,
```

      "innodb_buffer_pool_size":"64MB",

      "innodb_log_dir_path":"/home/kunlun/work01/data_mysql/data6_innodb_log",

      "data_dir_path":"/home/kunlun/work01/data_mysql/data6",

      "log_dir_path":"/home/kunlun/work01/data_mysql/data6_log",

```
      "user":"kunlun",
```
```
      "election_weight":50
```
```
   }
```
```
]
```
}

计算节点，三个节点

postgresql_comp.json
[
{
```
  "id":1,
```
```
  "name":"comp1",
```
```
  "ip":"127.0.0.1",
```
```
  "port":5401,
```
```
  "user":"abc",
```
```
  "password":"abc",
```

  "datadir":"/home/kunlun/work01/data_post/pg_data_dir1"

},
{
```
  "id":2,
```
```
  "name":"comp2",
```
```
  "ip":"127.0.0.1",
```
```
  "port":5402,
```
```
  "user":"abc",
```
```
  "password":"abc",
```

  "datadir":"/home/kunlun/work01/data_post/pg_data_dir2"

},
{
```
  "id":3,
```
```
  "name":"comp3",
```
```
  "ip":"127.0.0.1",
```
```
  "port":5403,
```
```
  "user":"abc",
```
```
  "password":"abc",
```

  "datadir":"/home/kunlun/work01/data_post/pg_data_dir3"

}
]

元数据节点3个，指向存储节点的一个分片

postgresql_meta.json
[
{
```
  "ip": "127.0.0.1",
```
```
  "port": 6004,
```
```
  "user": "pgx",
```
```
  "password":"pgx_pwd"
```
},
{
```
  "ip": "127.0.0.1",
```
```
  "port": 6005,
```
```
  "user": "pgx",
```
```
  "password":"pgx_pwd"
```
},
{
```
  "ip": "127.0.0.1",
```
```
  "port": 6006,
```
```
  "user": "pgx",
```
```
  "password":"pgx_pwd"
```
}
]

共享分片，就是存储分片的6个节点

postgresql_shards.json
[
{
"shard_name": "shard1",
"shard_nodes":
[
```
   {
```
```
      "ip": "127.0.0.1",
```
```
      "port": 6001,
```
```
      "user": "pgx",
```
```
      "password":"pgx_pwd"
```
```
   },
```
```
   {
```
```
      "ip": "127.0.0.1",
```
```
      "port": 6002,
```
```
      "user": "pgx",
```
```
      "password":"pgx_pwd"
```
```
   },
```
```
   {
```
```
      "ip": "127.0.0.1",
```
```
      "port": 6003,
```
```
      "user": "pgx",
```
```
      "password":"pgx_pwd"
```
```
   }
```
]
},
{
"shard_name": "shard2",
"shard_nodes":
[
```
   {
```
```
      "ip": "127.0.0.1",
```
```
      "port": 6004,
```
```
      "user": "pgx",
```
```
      "password":"pgx_pwd"
```
```
   },   
```
```
   {
```
```
      "ip": "127.0.0.1",
```
```
      "port": 6005,
```
```
      "user": "pgx",
```
```
      "password":"pgx_pwd"
```
```
   },
```
```
   {
```
```
      "ip": "127.0.0.1",
```
```
      "port": 6006,
```
```
      "user": "pgx",
```
```
      "password":"pgx_pwd"
```
```
   }
```
]
}
]

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

kunlun数据库手动集群安装

Clone this wiki locally