Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[VOQ][saidump] Modify generate_dump: replace save_saidump with save_saidump_by_route_size #2972

Merged
merged 12 commits into from
Nov 15, 2023

Commits on Sep 1, 2023

  1. * [saidump]

    •	Saidump for DNX-SAI sonic-net/sonic-buildimage#13561
    
    Solution and modification:
    To use the redis-db SAVE option to save the snapshot of DB each time and recover later, instead of looping through each entry in the table and saving it.
    
    (1) Updated sonic-buildimage/build_debian.sh, to install Python library rdbtools into the host.
    (2) Updated sonic-buildimage/src/sonic-sairedis/saidump/saidump.cpp, add a new option -r, which updates the rdbtools's output-JSON files' format.
    (3) Add a new script file: files/scripts/saidump.sh, to do the below steps
      For each ASIC0, such as ASIC0,
    
      sonic-net#1. Save the Redis data.
      sudo sonic-db-cli -n asic$1 SAVE > /dev/null
    
      sonic-net#2. Move dump files to /var/run/redisX/
      docker exec database$1 sh -c "mv /var/lib/redis/dump.rdb /var/run/redis$1/"
    
      sonic-net#3. Run rdb command to convert the dump files into JSON files
      sudo python /usr/local/bin/rdb --command json  /var/run/redis$1/dump.rdb | sudo tee /var/run/redis$1/dump.json > /dev/null
    
      sonic-net#4. Run saidump -r to update the JSON files' format as same as the saidump before. Then we can get the saidump result in standard output.
      docker exec syncd$1 sh -c "saidump -r /var/run/redis$1/dump.json"
    
      sonic-net#5. clear
      sudo rm -f /var/run/redis$1/dump.rdb
      sudo rm -f /var/run/redis$1/dump.json
    
    (4) Update sonic-buildimage/src/sonic-utilities/scripts/generate_dump, replace saidump with saidump.sh
    JunhongMao committed Sep 1, 2023
    Configuration menu
    Copy the full SHA
    2b43852 View commit details
    Browse the repository at this point in the history
  2. * [saidump]

    •	Saidump for DNX-SAI sonic-net/sonic-buildimage#13561
    
    Solution and modification:
    To use the redis-db SAVE option to save the snapshot of DB each time and recover later, instead of looping through each entry in the table and saving it.
    
    (1) Updated sonic-buildimage/build_debian.sh, to install Python library rdbtools into the host.
    (2) Updated sonic-buildimage/src/sonic-sairedis/saidump/saidump.cpp, add a new option -r, which updates the rdbtools's output-JSON files' format.
    (3) Add a new script file: files/scripts/saidump.sh, to do the below steps
      For each ASIC0, such as ASIC0,
    
      1. Save the Redis data.
      sudo sonic-db-cli -n asic$1 SAVE > /dev/null
    
      2. Move dump files to /var/run/redisX/
      docker exec database$1 sh -c "mv /var/lib/redis/dump.rdb /var/run/redis$1/"
    
      3. Run rdb command to convert the dump files into JSON files
      sudo python /usr/local/bin/rdb --command json  /var/run/redis$1/dump.rdb | sudo tee /var/run/redis$1/dump.json > /dev/null
    
      4. Run saidump -r to update the JSON files' format as same as the saidump before. Then we can get the saidump result in standard output.
      docker exec syncd$1 sh -c "saidump -r /var/run/redis$1/dump.json"
    
      5. clear
      sudo rm -f /var/run/redis$1/dump.rdb
      sudo rm -f /var/run/redis$1/dump.json
    
    (4) Update sonic-buildimage/src/sonic-utilities/scripts/generate_dump, replace saidump with saidump.sh
    JunhongMao committed Sep 1, 2023
    Configuration menu
    Copy the full SHA
    62aab2d View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    5ecb3c1 View commit details
    Browse the repository at this point in the history

Commits on Sep 5, 2023

  1. * [saidump]

        •       Saidump for DNX-SAI sonic-net/sonic-buildimage#13561
    
        Solution and modification:
        To use the redis-db SAVE option to save the snapshot of DB each time and recover later, instead of looping through each entry in the table and saving it.
    
        (1) Updated sonic-buildimage/build_debian.sh, to install Python library rdbtools into the host.
        (2) Updated sonic-buildimage/src/sonic-sairedis/saidump/saidump.cpp, add a new option -r, which updates the rdbtools's output-JSON files' format.
        (3) Add a new script file: files/scripts/saidump.sh, to do the below steps
          For each ASIC0, such as ASIC0,
    
          1. Save the Redis data.
          sudo sonic-db-cli -n asic$1 SAVE > /dev/null
    
          2. Move dump files to /var/run/redisX/
          docker exec database$1 sh -c "mv /var/lib/redis/dump.rdb /var/run/redis$1/"
    
          3. Run rdb command to convert the dump files into JSON files
          sudo python /usr/local/bin/rdb --command json  /var/run/redis$1/dump.rdb | sudo tee /var/run/redis$1/dump.json > /dev/null
    
          4. Run saidump -r to update the JSON files' format as same as the saidump before. Then we can get the saidump result in standard output.
          docker exec syncd$1 sh -c "saidump -r /var/run/redis$1/dump.json"
    
          5. clear
          sudo rm -f /var/run/redis$1/dump.rdb
          sudo rm -f /var/run/redis$1/dump.json
    
        (4) Update sonic-buildimage/src/sonic-utilities/scripts/generate_dump, replace saidump with saidump.sh
    JunhongMao committed Sep 5, 2023
    Configuration menu
    Copy the full SHA
    3504fdc View commit details
    Browse the repository at this point in the history

Commits on Sep 6, 2023

  1. * [saidump]

            •       Saidump for DNX-SAI sonic-net/sonic-buildimage#13561
    
            Solution and modification:
            To use the redis-db SAVE option to save the snapshot of DB each time and recover later, instead of looping through each entry in the table and saving it.
    
            (1) Updated sonic-buildimage/build_debian.sh, to install Python library rdbtools into the host.
            (2) Updated sonic-buildimage/src/sonic-sairedis/saidump/saidump.cpp, add a new option -r, which updates the rdbtools's output-JSON files' format.
            (3) Add a new script file: files/scripts/saidump.sh, to do the below steps
              For each ASIC0, such as ASIC0,
    
              1. Save the Redis data.
              sudo sonic-db-cli -n asic$1 SAVE > /dev/null
    
              2. Move dump files to /var/run/redisX/
              docker exec database$1 sh -c "mv /var/lib/redis/dump.rdb /var/run/redis$1/"
    
              3. Run rdb command to convert the dump files into JSON files
              sudo python /usr/local/bin/rdb --command json  /var/run/redis$1/dump.rdb | sudo tee /var/run/redis$1/dump.json > /dev/null
    
              4. Run saidump -r to update the JSON files' format as same as the saidump before. Then we can get the saidump result in standard output.
              docker exec syncd$1 sh -c "saidump -r /var/run/redis$1/dump.json"
    
              5. clear
              sudo rm -f /var/run/redis$1/dump.rdb
              sudo rm -f /var/run/redis$1/dump.json
    
            (4) Update sonic-buildimage/src/sonic-utilities/scripts/generate_dump, replace saidump with saidump.sh
    JunhongMao committed Sep 6, 2023
    Configuration menu
    Copy the full SHA
    2450c48 View commit details
    Browse the repository at this point in the history

Commits on Sep 19, 2023

  1. Fixup based on the below PR comments.

    sonic-net#2972
    SAI DUMP based on the route table size
    
    * [saidump]
    • Saidump for DNX-SAI sonic-net/sonic-buildimage#13561
    
    Solution and modification:
    To use the Redis-db SAVE option to save the snapshot of DB each time and recover later, instead of looping through each entry in the table and saving it.
    
    (1) Updated platform/broadcom/docker-syncd-brcm-dnx/Dockerfile.j2, install Python library rdbtools into the syncd containter.
    (2) Updated sonic-buildimage/src/sonic-sairedis/saidump/saidump.cpp, add a new option -r, which updates the rdbtools's output-JSON files' format.
    (3) Updated sonic-buildimage/build_debian.sh, to add a new script file: files/scripts/saidump.sh into the host. This shell file does the below steps:
      For each ASIC0, such as ASIC0,
    
      1. Save the Redis data.
      sudo sonic-db-cli -n asic$1 SAVE > /dev/null
    
      2. Move dump files to /var/run/redisX/
      docker exec database$1 sh -c "mv /var/lib/redis/dump.rdb /var/run/redis$1/"
    
      3. Run rdb command to convert the dump files into JSON files
      docker exec syncd$1 sh -c "rdb --command json /var/run/redis$1/dump.rdb | tee /var/run/redis$1/dump.json > /dev/null"
    
      4. Run saidump -r to update the JSON files' format as same as the saidump before. Then we can get the saidump result in standard output.
      docker exec syncd$1 sh -c "saidump -r /var/run/redis$1/dump.json -m 100"
    
      5. clear
      sudo rm -f /var/run/redis$1/dump.rdb
      sudo rm -f /var/run/redis$1/dump.json
    
    (4) Update sonic-buildimage/src/sonic-utilities/scripts/generate_dump, to check the asic db size and if it is larger than xxx entries, then do with REDIS SAVE, otherwise, to do with old method: looping through each entry of Redis DB.
    JunhongMao committed Sep 19, 2023
    Configuration menu
    Copy the full SHA
    9fa769e View commit details
    Browse the repository at this point in the history

Commits on Sep 21, 2023

  1. According to the testing group's advice, change the default ROUTE_TAB…

    …_LIMIT_DIRECT_ITERATION to 24000
    JunhongMao committed Sep 21, 2023
    Configuration menu
    Copy the full SHA
    8a3f93b View commit details
    Browse the repository at this point in the history

Commits on Oct 3, 2023

  1. sonic-net/sonic-buildimage#16466 fixing based on review comments:

    To add the rdbtools into base docker.
    Move saidump.sh from host to syncd docker container.
    JunhongMao committed Oct 3, 2023
    Configuration menu
    Copy the full SHA
    32f5607 View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    64ad586 View commit details
    Browse the repository at this point in the history

Commits on Oct 4, 2023

  1. Configuration menu
    Copy the full SHA
    867a6d1 View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    185bfef View commit details
    Browse the repository at this point in the history

Commits on Oct 12, 2023

  1. Configuration menu
    Copy the full SHA
    86d3efa View commit details
    Browse the repository at this point in the history