Skip to content

Commit

Permalink
* [saidump]
Browse files Browse the repository at this point in the history
•	Saidump for DNX-SAI sonic-net#13561

Solution and modification:
To use the redis-db SAVE option to save the snapshot of DB each time and recover later, instead of looping through each entry in the table and saving it.

(1) Updated sonic-buildimage/build_debian.sh, to install Python library rdbtools into the host.
(2) Updated sonic-buildimage/src/sonic-sairedis/saidump/saidump.cpp, add a new option -r, which updates the rdbtools's output-JSON files' format.
(3) Add a new script file: files/scripts/saidump.sh, to do the below steps
  For each ASIC0, such as ASIC0,

  1. Save the Redis data.
  sudo sonic-db-cli -n asic$1 SAVE > /dev/null

  2. Move dump files to /var/run/redisX/
  docker exec database$1 sh -c "mv /var/lib/redis/dump.rdb /var/run/redis$1/"

  3. Run rdb command to convert the dump files into JSON files
  sudo python /usr/local/bin/rdb --command json  /var/run/redis$1/dump.rdb | sudo tee /var/run/redis$1/dump.json > /dev/null

  4. Run saidump -r to update the JSON files' format as same as the saidump before. Then we can get the saidump result in standard output.
  docker exec syncd$1 sh -c "saidump -r /var/run/redis$1/dump.json"

  5. clear
  sudo rm -f /var/run/redis$1/dump.rdb
  sudo rm -f /var/run/redis$1/dump.json

(4) Update sonic-buildimage/src/sonic-utilities/scripts/generate_dump, replace saidump with saidump.sh
  • Loading branch information
JunhongMao committed Sep 1, 2023
1 parent f6897bb commit af63bf5
Show file tree
Hide file tree
Showing 2 changed files with 49 additions and 0 deletions.
16 changes: 16 additions & 0 deletions build_debian.sh
Original file line number Diff line number Diff line change
Expand Up @@ -729,6 +729,22 @@ SONIC_VERSION_CACHE=${SONIC_VERSION_CACHE} \
DBGOPT="${DBGOPT}" \
scripts/collect_host_image_version_files.sh $CONFIGURED_ARCH $IMAGE_DISTRO $TARGET_PATH $FILESYSTEM_ROOT

# Install libc6-dev and python3-dev for compiling python-lzf
sudo LANG=C chroot $FILESYSTEM_ROOT apt-get -y install libc6-dev python3-dev

# Install python-lzf
sudo https_proxy=$https_proxy LANG=C chroot $FILESYSTEM_ROOT pip3 install 'python-lzf==0.2.4'

# Install rdbtools
sudo https_proxy=$https_proxy LANG=C chroot $FILESYSTEM_ROOT pip3 install 'rdbtools==0.1.15'

# Uninstall libc6-dev and python3-dev for compiling python-lzf
sudo LANG=C DEBIAN_FRONTEND=noninteractive chroot $FILESYSTEM_ROOT apt-get -y remove libc6-dev
sudo LANG=C DEBIAN_FRONTEND=noninteractive chroot $FILESYSTEM_ROOT apt-get -y remove python3-dev

## Copy saidump.sh
sudo cp ./files/scripts/saidump.sh $FILESYSTEM_ROOT/usr/local/bin/saidump.sh
sudo chmod +x $FILESYSTEM_ROOT/usr/local/bin/saidump.sh
# Remove GCC
sudo LANG=C DEBIAN_FRONTEND=noninteractive chroot $FILESYSTEM_ROOT apt-get -y remove gcc

Expand Down
33 changes: 33 additions & 0 deletions files/scripts/saidump.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,33 @@
#!/bin/bash

This comment has been minimized.

Copy link
@jon-nokia

jon-nokia Sep 6, 2023

Very nice script. Consider using "set -e" in case some commands fail. Given this script is used for debug, there may be times when some of the dockers are down and "docker exec" commands may fail. You could simulate that case and have script log something meaningful.

This comment has been minimized.

Copy link
@JunhongMao

JunhongMao Sep 6, 2023

Author Owner

Thanks. I will add "set -e".

This comment has been minimized.

Copy link
@JunhongMao

JunhongMao Sep 6, 2023

Author Owner

Fixed it by:

c8f5996

save_saidump_by_rdb() {
#1. Save the Redis data.
sudo sonic-db-cli -n asic$1 SAVE > /dev/null
#2. Move dump files to /var/run/redisX/
docker exec database$1 sh -c "mv /var/lib/redis/dump.rdb /var/run/redis$1/"
#3. Run rdb command to convert the dump files into JSON files
sudo python /usr/local/bin/rdb --command json /var/run/redis$1/dump.rdb | sudo tee /var/run/redis$1/dump.json > /dev/null
#4. Run saidump -r to update the JSON files' format as same as the saidump before. Then we can get the saidump result in standard output.
docker exec syncd$1 sh -c "saidump -r /var/run/redis$1/dump.json"
#5. clear
sudo rm -f /var/run/redis$1/dump.rdb
sudo rm -f /var/run/redis$1/dump.json
}

NUM_ASICS=`python -c 'from sonic_py_common.multi_asic import get_num_asics; print(get_num_asics())'`


if (( $# == 0 && $NUM_ASICS == 1 )); then
save_saidump_by_rdb
#validate if the argument is an integer
elif [[ "$1" =~ [0-9]+ ]] && (( $# == 1 && $1 >= 0 && $1 < $NUM_ASICS )) ; then
save_saidump_by_rdb $1
else
echo "The number of ASICS is $NUM_ASICS."
echo "Usage:"
echo "saidump.sh <ASIC Index> or NULL"
echo " <ASIC Index> should be 0 ~ NUM_ASICS."
echo " E.g. \"saidump.sh 1\" is OK when NUMASICS is 2."
echo " NULL"
echo " E.g. \"saidump.sh\" is OK when NUMASICS is 1."
fi

0 comments on commit af63bf5

Please sign in to comment.