Worker setup on Linux

Multi Variant Fishtest: worker setup on Linux

Get username/password for Multi Variant Fishtest

Create an account on Multi Variant Fishtest choosing an username/password:

http://www.variantfishtest.org:6543/signup

Debian/Ubuntu

Install the required packages:

sudo apt update
sudo apt install -y build-essential python3 python3-venv git

Note: the Stockfish tuning branch triggers a GCC 5 bug (the default GCC in Ubuntu 16.04). Read the following paragraph Install several GCC versions on Ubuntu to install a different GCC alongside the default one.

Setup fishtest

It is recommended for security reasons to create a separate user for the fishtest worker since you are essentially running arbitrary code/instructions. This way testing multi variant stockfish can have no impact on your primary accounts and configuration.

sudo useradd -m mvfishtest

Clone the fishtest code and setup a python3 virtual environment:

sudo -i -u mvfishtest git clone --single-branch --branch master https://github.com/ianfab/fishtest.git
sudo -i -u mvfishtest python3 -m venv fishtest/worker/env
sudo -i -u mvfishtest fishtest/worker/env/bin/python3 -m pip install --upgrade pip setuptools wheel requests

Launching the worker

Write this mvfishtest.sh script to run the worker with the output in console:

#!/bin/bash
# usage: mvfishtest.sh  [<n_cores> <username> <password>]
# <n_cores>: number of cores to be used in fishtest. Suggested max value = n. physical cores-1
# <username>: username on fishtest (to be enclosed in quote if contains special characters)
# <password>: password on fishtest (to be enclosed in quote if contains special characters)
# The three parameters are mandatory only for the first execution

if [ $# -gt 0 ]; then
  sudo -i -u mvfishtest fishtest/worker/env/bin/python3 fishtest/worker/worker.py --concurrency $1 $2 $3
else
  sudo -i -u mvfishtest fishtest/worker/env/bin/python3 fishtest/worker/worker.py
fi

For the first execution the three parameters are mandatory:

bash mvfishtest.sh <n_cores> <username> <password>

Starting from the second execution simply run the script or without parameters or with a different number of cores:

bash mvfishtest.sh [<n_cores>]

RHEL/CentOS

If the default version of GCC should not support C++11, you must install the developer toolset, which adds a newer version of GCC. Follow the instructions for your Linux distribution:

https://www.softwarecollections.org/en/scls/rhscl/devtoolset-6/

Use this script to run the worker with the output in console:

#!/bin/bash
# usage: mvfishtest.sh  [<n_cores> <username> <password>]
# <n_cores>: number of cores to be used in fishtest. Suggested max value = n. physical cores-1
# <username>: username on fishtest (to be enclosed in quote if contains special characters)
# <password>: password on fishtest (to be enclosed in quote if contains special characters)
# The three parameters are mandatory only for the first execution

if [ $# -gt 0 ]; then
  sudo -i -u mvfishtest << EOF
source scl_source enable devtoolset-6
python fishtest/worker/worker.py --concurrency $1 "$2" "$3"
EOF
else
  sudo -i -u mvfishtest << EOF
source scl_source enable devtoolset-6
python fishtest/worker/worker.py
EOF
fi

Install several GCC versions on Ubuntu

Launch the following script to have several versions of GCC on Ubuntu alongside the default one: install some GCC versions using the Ubuntu Toolchain Uploads repository and manage the different GCC versions using update-alternatives. This works also for Windows Subsystem for Linux.

#!/bin/bash
# launch this script using "sudo"

# install the default building tools
apt update -y && \
apt install -y build-essential software-properties-common

# add the repository "ubuntu-toolchain-r/test"
apt update -y && \
add-apt-repository -y ppa:ubuntu-toolchain-r/test && \
apt update -y

# install other GCC versions
apt install -y gcc-5 g++-5
apt install -y gcc-6 g++-6
apt install -y gcc-7 g++-7

# configure the alternatives for gcc and g++, setting a higher priority to a newer version (ymmv)
update-alternatives --install /usr/bin/gcc gcc /usr/bin/gcc-5 50 --slave /usr/bin/g++ g++ /usr/bin/g++-5
update-alternatives --install /usr/bin/gcc gcc /usr/bin/gcc-6 60 --slave /usr/bin/g++ g++ /usr/bin/g++-6
update-alternatives --install /usr/bin/gcc gcc /usr/bin/gcc-7 70 --slave /usr/bin/g++ g++ /usr/bin/g++-7

# check or change the alternatives configuration
update-alternatives --config gcc