Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Reload is inconsistent #72

Open
sngx13 opened this issue Nov 28, 2022 · 4 comments
Open

Reload is inconsistent #72

sngx13 opened this issue Nov 28, 2022 · 4 comments
Assignees
@ketulsuthar

Comments

@sngx13
Copy link

sngx13 commented Nov 28, 2022

Sending e.g device.reload() doesn't always produce expected result. It's sitting and waiting for something at the reload confirm prompt and eventually timeouts and prompt recovery kicks in which effectively sends a new line and allows for reload to proceed.

YAML:

test_bed = """
devices:
  $HOSTNAME:
    os: iosxe
    platform: iosxe
    type: router
    credentials:
      default:
        username: $USERNAME
        password: $PASSWORD
      terminal_server:
        username: $USERNAME
        password: $PASSWORD
      fallback:
        username: cisco
        password: cisco
    connections:
      a:
        protocol: telnet
        ip: $CONSOLE_IP
        port: $CONSOLE_PORT
        login_creds: [terminal_server, default, fallback]
        arguments:
          connection_timeout: 120
          mit: True
          cred_action:
            terminal_server:
              post: sendline()
        settings:
          SENDLINE_AFTER_CRED: terminal_server
          ESCAPE_CHAR_CHATTY_TERM_WAIT: 0.5
          ESCAPE_CHAR_PROMPT_WAIT: 1
          ESCAPE_CHAR_CHATTY_TERM_WAIT_RETRIES: 15
          EXEC_TIMEOUT: 30
          CONFIG_TIMEOUT: 30
          CONFIG_TRANSITION_WAIT: 0.5
          RELOAD_TIMEOUT: 420
          RELOAD_WAIT: 300
          POST_RELOAD_WAIT: 60
          RELOAD_RECONNECT_ATTEMPTS: 5
          SLEEP_PRE_LAUNCH: 0.2
          POST_DISCONNECT_WAIT_SEC: 0
          GRACEFUL_DISCONNECT_WAIT_SEC: 0.2
          CONSOLE_TIMEOUT: 120
"""

LOG output:

2022-11-28 11:47:58,156: %UNICON-INFO: +++ DUT-C1117-4PM logfile /opt/autom8_app/autom8_app/logs/pyats/DUT-C1117-4PM.log +++

2022-11-28 11:47:58,156: %UNICON-INFO: +++ Unicon plugin iosxe (unicon.plugins.iosxe) +++
Trying 80.194.79.121...


2022-11-28 11:47:58,169: %UNICON-INFO: +++ connection to spawn: telnet 80.194.79.121 10014, id: 140694475231472 +++

2022-11-28 11:47:58,169: %UNICON-INFO: connection to DUT-C1117-4PM
Connected to 80.194.79.121.
Escape character is '^]'.

Login: automation
Password:

2022-11-28 11:47:58,282: %UNICON-INFO: Executing post credential command: sendline()



DUT-C1117-4PM#

2022-11-28 11:47:58,396: %UNICON-INFO: Learned hostname(s): 'DUT-C1117-4PM'.

2022-11-28 11:47:58,460: %UNICON-INFO: +++ DUT-C1117-4PM with via 'a': enable +++

2022-11-28 11:47:58,526: %UNICON-INFO: +++ DUT-C1117-4PM with via 'a': executing command 'write erase' +++
write erase
************************************************************************************************************
Erasing Nvram will not clear license trust code.
************************************************************************************************************
Erasing the nvram filesystem will remove all configuration files! Continue? [confirm]
[OK]
Erase of nvram: complete
DUT-C1117-4PM#

2022-11-28 11:48:00,922: %UNICON-INFO: +++ DUT-C1117-4PM with via 'a': reload +++

2022-11-28 11:48:00,985: %UNICON-INFO: +++ reloading DUT-C1117-4PM  with reload_command 'reload' and timeout is 420 seconds +++
reload

System configuration has been modified. Save? [yes/no]: n
WARNING:
Boot variable either does not exist or buffer is too small
This may impact autoboot of the router. Proceed with caution
Do you wish to proceed with reload anyway[confirm]

2022-11-28 11:55:00,991: %UNICON-WARNING: Timeout of 420 seconds has been reached.
Prompt Recovery has commenced. Total timeout occurs in 100 seconds.

2022-11-28 11:55:00,994: %UNICON-INFO: Sending prompt recovery command: b'\r'

Proceed with reload? [confirm]
2022-11-28 11:55:11,007: %UNICON-INFO: Sending prompt recovery command: b'\x15'

2022-11-28 11:55:21,028: %UNICON-INFO: Sending prompt recovery command: b'\x1a'

2022-11-28 11:55:31,036: %UNICON-INFO: Sending prompt recovery command: b'\r'
Rom image verified correctly
@sngx13
Copy link
Author

sngx13 commented Dec 19, 2022 •
edited
Loading

Hi,
It's been a while, have you managed to reproduce this?

DUT-C1111-4P#

2022-12-19 10:55:23,421: %UNICON-INFO: Learned hostname(s): 'DUT-C1111-4P'.

2022-12-19 10:55:23,485: %UNICON-INFO: +++ DUT-C1111-4P with via 'console': enable +++

2022-12-19 10:55:23,550: %UNICON-INFO: +++ DUT-C1111-4P with via 'console': executing command 'write erase' +++
write erase
************************************************************************************************************
Erasing Nvram will not clear license trust code.
************************************************************************************************************
Erasing the nvram filesystem will remove all configuration files! Continue? [confirm]
[OK]
Erase of nvram: complete
DUT-C1111-4P#

2022-12-19 10:55:25,929: %UNICON-INFO: +++ DUT-C1111-4P with via 'console': reload +++

2022-12-19 10:55:25,994: %UNICON-INFO: +++ reloading DUT-C1111-4P  with reload_command 'reload' and timeout is 300 seconds +++
reload

System configuration has been modified. Save? [yes/no]: n
WARNING:
Boot variable either does not exist or buffer is too small
This may impact autoboot of the router. Proceed with caution
Do you wish to proceed with reload anyway[confirm]

2022-12-19 11:00:26,003: %UNICON-WARNING: Timeout of 300 seconds has been reached.
Prompt Recovery has commenced. Total timeout occurs in 100 seconds.

2022-12-19 11:00:26,007: %UNICON-INFO: Sending prompt recovery command: b'\r'

Proceed with reload? [confirm]

2022-12-19 11:00:36,027: %UNICON-INFO: Sending prompt recovery command: b'\x15'

Getting stuck at [confirm] prompt until recovery kicks in.

One interesting observation though. when the following line appears in syslog, reload is always 100% successful: %IOSXE_RP_CFG_NOT-3-BOOT_VAR_NOT_SET: Either the boot variable does not exist or the buffer is too small.Keep it blank.

2022-12-19 12:27:22,902: %UNICON-INFO: Executing post credential command: sendline()


DUT-C1111-4P#

2022-12-19 12:27:23,195: %UNICON-INFO: Learned hostname(s): 'DUT-C1111-4P'.

2022-12-19 12:27:23,259: %UNICON-INFO: +++ DUT-C1111-4P with via 'console': enable +++

2022-12-19 12:27:23,833: %UNICON-INFO: +++ DUT-C1111-4P with via 'console': executing command 'write erase' +++
write erase
************************************************************************************************************
Erasing Nvram will not clear license trust code.
************************************************************************************************************
Erasing the nvram filesystem will remove all configuration files! Continue? [confirm]
[OK]
Erase of nvram: complete
DUT-C1111-4P#

2022-12-19 12:27:26,241: %UNICON-INFO: +++ DUT-C1111-4P with via 'console': reload +++

2022-12-19 12:27:26,306: %UNICON-INFO: +++ reloading DUT-C1111-4P  with reload_command 'reload' and timeout is 400 seconds +++
reload
WARNING:
Boot variable either does not exist or buffer is too small
This may impact autoboot of the router. Proceed with caution
Do you wish to proceed with reload anyway[confirm]
*Dec 19 12:27:25.919: %SYS-7-NV_BLOCK_INIT: Initialized the geometry of nvram
Proceed with reload? [confirm]

*Dec 19 12:27:27.783: %IOSXE_RP_CFG_NOT-3-BOOT_VAR_NOT_SET: Either the boot variable does not exist or the buffer is too small.Keep it blank.
*Dec 19 12:27:31.050: %SYS-5-RELOAD: Reload requested by console. Reload Reason: Reload Command.Rom image verified correctly

@ketulsuthar
Copy link

Hi,

https://pubhub.devnetcloud.com/media/unicon/docs/user_guide/services/generic_services.html#:~:text=as%20shown%20here.-,execute,-%C2%B6

https://pubhub.devnetcloud.com/media/pyats-getting-started/docs/bestpractices/lib.html#:~:text=smaller%20verifications%20together.-,Dialog,-example

can you handle the Dialog the right way?

above link will help you

thanks

@sngx13
Copy link
Author

sngx13 commented Dec 19, 2022 via email

@sngx13
Copy link
Author

sngx13 commented Dec 20, 2022 •
edited
Loading

Where can i find dialog handling? At least the default ones?

2022-12-20 11:01:10,419: %UNICON-INFO: +++ DUT-C1111-4P with via 'console': reload +++

2022-12-20 11:01:10,482: %UNICON-INFO: +++ reloading DUT-C1111-4P  with reload_command 'reload' and timeout is 420 seconds +++
reload

System configuration has been modified. Save? [yes/no]: n
WARNING:
Boot variable either does not exist or buffer is too small
This may impact autoboot of the router. Proceed with caution
Do you wish to proceed with reload anyway[confirm]

This may be down to the fact that default dialog does not expect the warning message and isn't handling this properly:

WARNING: Boot variable either does not exist or buffer is too small
This may impact autoboot of the router. Proceed with caution

Could this be added to next release?

Update
Just confirmed my theory, when boot system bootflash:xxxx is added, problem goes away i.e WARNING message no longer appears and Unicon handles it properly. Can we handle this by default as there are instances when people are not setting boot vars but instead delete the old IOS?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@ketulsuthar ketulsuthar

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@ketulsuthar @sngx13