Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[VOQ][saidump] Install rdbtools into the docker base related containers. #16466

Merged
merged 10 commits into from
Nov 8, 2023

Commits on Sep 1, 2023

  1. * [saidump]

    •	Saidump for DNX-SAI sonic-net#13561
    
    Solution and modification:
    To use the redis-db SAVE option to save the snapshot of DB each time and recover later, instead of looping through each entry in the table and saving it.
    
    (1) Updated sonic-buildimage/build_debian.sh, to install Python library rdbtools into the host.
    (2) Updated sonic-buildimage/src/sonic-sairedis/saidump/saidump.cpp, add a new option -r, which updates the rdbtools's output-JSON files' format.
    (3) Add a new script file: files/scripts/saidump.sh, to do the below steps
      For each ASIC0, such as ASIC0,
    
      1. Save the Redis data.
      sudo sonic-db-cli -n asic$1 SAVE > /dev/null
    
      2. Move dump files to /var/run/redisX/
      docker exec database$1 sh -c "mv /var/lib/redis/dump.rdb /var/run/redis$1/"
    
      3. Run rdb command to convert the dump files into JSON files
      sudo python /usr/local/bin/rdb --command json  /var/run/redis$1/dump.rdb | sudo tee /var/run/redis$1/dump.json > /dev/null
    
      4. Run saidump -r to update the JSON files' format as same as the saidump before. Then we can get the saidump result in standard output.
      docker exec syncd$1 sh -c "saidump -r /var/run/redis$1/dump.json"
    
      5. clear
      sudo rm -f /var/run/redis$1/dump.rdb
      sudo rm -f /var/run/redis$1/dump.json
    
    (4) Update sonic-buildimage/src/sonic-utilities/scripts/generate_dump, replace saidump with saidump.sh
    JunhongMao committed Sep 1, 2023
    Configuration menu
    Copy the full SHA
    af63bf5 View commit details
    Browse the repository at this point in the history

Commits on Sep 6, 2023

  1. * [saidump]

        •       Saidump for DNX-SAI sonic-net#13561
    
        Solution and modification:
        To use the redis-db SAVE option to save the snapshot of DB each time and recover later, instead of looping through each entry in the table and saving it.
    
        (1) Updated sonic-buildimage/build_debian.sh, to install Python library rdbtools into the host.
        (2) Updated sonic-buildimage/src/sonic-sairedis/saidump/saidump.cpp, add a new option -r, which updates the rdbtools's output-JSON files' format.
        (3) Add a new script file: files/scripts/saidump.sh, to do the below steps
          For each ASIC0, such as ASIC0,
    
          1. Save the Redis data.
          sudo sonic-db-cli -n asic$1 SAVE > /dev/null
    
          2. Move dump files to /var/run/redisX/
          docker exec database$1 sh -c "mv /var/lib/redis/dump.rdb /var/run/redis$1/"
    
          3. Run rdb command to convert the dump files into JSON files
          sudo python /usr/local/bin/rdb --command json  /var/run/redis$1/dump.rdb | sudo tee /var/run/redis$1/dump.json > /dev/null
    
          4. Run saidump -r to update the JSON files' format as same as the saidump before. Then we can get the saidump result in standard output.
          docker exec syncd$1 sh -c "saidump -r /var/run/redis$1/dump.json"
    
          5. clear
          sudo rm -f /var/run/redis$1/dump.rdb
          sudo rm -f /var/run/redis$1/dump.json
    
        (4) Update sonic-buildimage/src/sonic-utilities/scripts/generate_dump, replace saidump with saidump.sh
    JunhongMao committed Sep 6, 2023
    Configuration menu
    Copy the full SHA
    c8f5996 View commit details
    Browse the repository at this point in the history

Commits on Sep 21, 2023

  1. sonic-net#16466 fix based on review comments:

    Install rdbtools into syncd docker and add saidump.sh into host.
    
    * [saidump]
    • Saidump for DNX-SAI sonic-net#13561
    
    Solution and modification:
    To use the Redis-db SAVE option to save the snapshot of DB each time and recover later, instead of looping through each entry in the table and saving it.
    
    (1) Updated platform/broadcom/docker-syncd-brcm-dnx/Dockerfile.j2, install Python library rdbtools into the syncd containter.
    (2) Updated sonic-buildimage/src/sonic-sairedis/saidump/saidump.cpp, add a new option -r, which updates the rdbtools's output-JSON files' format.
    (3) Updated sonic-buildimage/build_debian.sh, to add a new script file: files/scripts/saidump.sh into the host. This shell file does the below steps:
      For each ASIC0, such as ASIC0,
    
      1. Save the Redis data.
      sudo sonic-db-cli -n asic$1 SAVE > /dev/null
    
      2. Move dump files to /var/run/redisX/
      docker exec database$1 sh -c "mv /var/lib/redis/dump.rdb /var/run/redis$1/"
    
      3. Run rdb command to convert the dump files into JSON files
      docker exec syncd$1 sh -c "rdb --command json /var/run/redis$1/dump.rdb | tee /var/run/redis$1/dump.json > /dev/null"
    
      4. Run saidump -r to update the JSON files' format as same as the saidump before. Then we can get the saidump result in standard output.
      docker exec syncd$1 sh -c "saidump -r /var/run/redis$1/dump.json -m 100"
    
      5. clear
      sudo rm -f /var/run/redis$1/dump.rdb
      sudo rm -f /var/run/redis$1/dump.json
    
    (4) Update sonic-buildimage/src/sonic-utilities/scripts/generate_dump, to check the asic db size and if it is larger than xxx entries, then do with REDIS SAVE, otherwise, to do with old method: looping through each entry of Redis DB.
    JunhongMao committed Sep 21, 2023
    Configuration menu
    Copy the full SHA
    1a414d5 View commit details
    Browse the repository at this point in the history

Commits on Sep 27, 2023

  1. Configuration menu
    Copy the full SHA
    f2215ec View commit details
    Browse the repository at this point in the history
  2. sonic-net#16466 fixing based on review comments:

    To add the rdbtools into base docker.
    Remove sudo in saidump.sh
    JunhongMao committed Sep 27, 2023
    Configuration menu
    Copy the full SHA
    65fc011 View commit details
    Browse the repository at this point in the history

Commits on Sep 30, 2023

  1. sonic-net#16466 fixing based on review comments:

    To add the rdbtools into base docker.
    Move saidump.sh from host to syncd docker container.
    JunhongMao committed Sep 30, 2023
    Configuration menu
    Copy the full SHA
    aaac734 View commit details
    Browse the repository at this point in the history

Commits on Oct 3, 2023

  1. sonic-net#16466 fixing based on review comments:

    To move saidump.sh from this repo to sairedis repo.
    JunhongMao committed Oct 3, 2023
    Configuration menu
    Copy the full SHA
    add5b59 View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    af75a3c View commit details
    Browse the repository at this point in the history

Commits on Oct 12, 2023

  1. Configuration menu
    Copy the full SHA
    a4fc033 View commit details
    Browse the repository at this point in the history

Commits on Oct 28, 2023

  1. Configuration menu
    Copy the full SHA
    8640e46 View commit details
    Browse the repository at this point in the history