PyATS fails to connect to APIC using rest.connector.Rest

[moonvulture]

I'm trying to connect to an APIC using PyATS and the rest.connector.Rest method, but no matter what, I get authentication failures. Here is my testbed file:

devices:
  apic:
      os: apic
      type: apic
      custom:
        abstraction:
          order: [os]
      connections:
        rest:
          class: rest.connector.Rest
          protocol: http
          port: 443
          ip: 10.10.10.10
          verify: False
          username: "apic:Tacacs\\my.user.account"
          password: "my_password"

Output from pyATS shell:

>>> testbed = load('aci_tb.yml')
-------------------------------------------------------------------------------            
>>> testbed.devices['apic'].connect(alias='rest', via='rest')
Connecting to 'apic' with alias 'rest'
<Response [401]>
{"totalCount":"1","imdata":[{"error":{"attributes":{"code":"401","text":"TACACS+ Server Authentication DENIED"}}}]}
Request to apic failed. Waiting 10 seconds before retrying

Traceback (most recent call last):
  File "/.robotFramework/lib64/python3.11/site-packages/rest/connector/libs/apic/implementation.py", line 144, in connect
    raise RequestException("Connection to '{ip}' has returned the "
requests.exceptions.RequestException: Connection to '10.10.10.10' has returned the following code '401', instead of the expected status code '200'

And here is the session log from the APIC:

General
Description
From-10.10.10.20-client-type-REST-Failure
Settings
Affected
uni/userext
cause
unknown
clientTag
-
code
generic
Created
2024-04-22T17:49:09.915+00:00
ID
8590532284
Action
special
Session ID
-
severity
info
System Id
1
Trigger
login, session
txId
0
User
remoteuser-my.user.account

I'm running:

genie==24.3
pyats==24.3
rest.connector==24.3

I've tried every combination of credentials and connections that I can think of and I'm still getting 401. The API works with ansible using the rest modules.

[Harishv01]

Kindly give me some time I will check and let you know.

Thank you

[Harishv01]

please try to pass TACACS credentials like this and try

        tacacs:
            login_prompt: "login:"
            password_prompt: "Password:"
            username: "lab"
        passwords:
          tacacs: lab
          enable: lab
          line: lab

Thank you

[moonvulture]

Thanks!

here is my updated testbed

devices:
  apic:
    os: apic
    type: aci
    connections:
      rest:
        class: rest.connector.Rest
        protocol: https
        port: 443
        ip: 10.10.10.10
        verify: False
        tacacs:
          login_prompt: "login:"
          password_prompt: "Password:"
          username: "my.user.account"
        passwords:
          tacacs: 'mypass'
          enable: 'mypass'
          line: 'mypass'

with the same result:

Welcome to pyATS Interactive Shell
==================================
Python 3.11.5 (main, Sep 22 2023, 15:34:29) [GCC 8.5.0 20210514 (Red Hat 8.5.0-20)]

>>> from pyats.topology.loader import load
>>> testbed = load('aci_tb.yml')
-------------------------------------------------------------------------------            
>>> testbed.devices['apic'].connect(alias='rest', via='rest')
Connecting to 'apic' with alias 'rest'
<Response [401]>
{"totalCount":"1","imdata":[{"error":{"attributes":{"code":"401","text":"TACACS+ Server Authentication DENIED"}}}]}
Request to apic failed. Waiting 10 seconds before retrying

[Harishv01]

Okay,Kindly give me some time.I will check with the team and get back to you

[Harishv01]

Can you please provide the debug logs to debug the issue?

Thank you.

[moonvulture]

as requested!

DEBUG:pyats.connections.manager:Connection manager added new connection via rest with alias rest
INFO:rest.connector.libs.apic.implementation:Connecting to 'apic' with alias 'rest'
DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): 10.10.10.10:443
DEBUG:urllib3.connectionpool:https://10.10.10.10:443 "POST /api/aaaLogin.json HTTP/1.1" 401 115
INFO:rest.connector.libs.apic.implementation:<Response [401]>
ERROR:rest.connector.libs.apic.implementation:{"totalCount":"1","imdata":[{"error":{"attributes":{"code":"401","text":"TACACS+ Server Authentication DENIED"}}}]}
WARNING:rest.connector.libs.apic.implementation:Request to apic failed. Waiting 10 seconds before retrying
Traceback (most recent call last):
File "/home/my.user.account/.robotFramework/lib64/python3.11/site-packages/rest/connector/libs/apic/implementation.py", line 144, in connect
raise RequestException("Connection to '{ip}' has returned the "
requests.exceptions.RequestException: Connection to '10.10.10.10' has returned the following code '401', instead of the expected status code '200'
^CTraceback (most recent call last):
File "/home/my.user.account/.robotFramework/lib64/python3.11/site-packages/rest/connector/libs/apic/implementation.py", line 144, in connect
raise RequestException("Connection to '{ip}' has returned the "
requests.exceptions.RequestException: Connection to '10.10.10.10' has returned the following code '401', instead of the expected status code '200'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/nmd_reports/02_DATA_CENTER/robotFramework/test.py", line 13, in
device.connect(alias='rest', via='rest', log_stdout=True)
File "src/pyats/connections/manager.py", line 509, in pyats.connections.manager.ConnectionManager.connect
File "src/pyats/async_/synchronize.py", line 117, in pyats.async_.synchronize.Lockable.locked._wrapped
File "/home/my.user.account/.robotFramework/lib64/python3.11/site-packages/rest/connector/libs/apic/implementation.py", line 153, in connect
time.sleep(retry_wait)

[Harishv01]

Okay,Give me some time i will check and get back to you

Thank you

[Harishv01]

Can you check if the username you passed is correct? I believe there might be an issue with the username ,dots are not supported. correct it and test it. Additionally, could you share the working debug log?

[Harishv01]

Can you check if the username you passed is correct? I believe there might be an issue with the username ,dots are not supported. correct it and test it. Additionally, could you share the working debug log?

[moonvulture]

I'll create a tacacs account with no dots and see if the behavior changes. Interesting note is that I can authenticate with that username using the following resource file for robot framework:

`*** Settings ***
Library pabot.PabotLib
Library RequestsLibrary
Library JSONLibrary
Library Collections

*** Variables ***
${login_json} {"aaaUser" : {"attributes" : {"name" : "%{ACI_USERNAME}", "pwd" : "%{ACI_PASSWORD}"}}}

*** Keywords ***
Get APIC Token
Create Session login %{ACI_URL}
${log_level}= Set Log Level DEBUG
${response}= Wait Until Keyword Succeeds 6x 10s POST On Session login /api/aaaLogin.json data=${login_json}
Set Log Level ${log_level}
${r_token}= Get Value From Json ${response.json()} $..token
Set Parallel Value For Key apic_token ${r_token[0]}

Login APIC
Run Only Once Get APIC Token
${apic_token}= Get Parallel Value For Key apic_token
Create Session apic %{ACI_URL} headers={"Cookie": "APIC-cookie=${apic_token}"}`

[Harishv01]

Create a TACACS account without dots and see whether you will encounter the issue or not and kindly let me know

[Harishv01]

Could you please provide an update on the above, please?

[moonvulture]

Sorry for the delay, I’ll the results posted today.

[Harishv01]

Could you please provide an update on the above, please?

[moonvulture]

ok, looks like it is an issue with the dots in the user account.

aci_tb.yml

devices:
apic:
os: apic
type: apic
custom:
abstraction:
order: [os]
connections:
rest:
class: rest.connector.Rest
protocol: http
port: 443
ip: 10.10.10.10
verify: False
username: "apiuser"
password: "1qaz@WSX3edc$RFV"

pyats shell --testbed-file aci_tb.yml

testbed.devices['apic'].connect(alias='rest', via='rest')
Connecting to 'apic' with alias 'rest'
<Response [200]>
Connected successfully to 'apic'

[Harishv01]

Can I close the ticket now that the issue has been resolved?

[moonvulture]

I agree the workaround works,, but is this not an issue with how implementation.py is passing the credentials?

For example, I can do a curl request to aaaLogin with no issues:
curl -k -d '<aaaUser name="my.user.account" pwd="1qaz@WSX3edc$RFV"/>' -c /etc/telegraf/$cookiefilename -X POST https://10.10.10.10/api/aaaLogin.xml
and I get a bearer token back.

[Harishv01]

In the repo for TACACS, dots are not supported in the pyats. That's why I suggested passing TACACS credentials like the example below

        tacacs:
            login_prompt: "login:"
            password_prompt: "Password:"
            username: "lab"
        passwords:
          tacacs: lab
          enable: lab
          line: lab

Please let me know if you need further assistance or may I close the ticket?

[moonvulture]

I think we have a working solution now. Feel free to close the issue and thanks for the assistance!

[Harishv01]

As you mentioned, we have a working solution because of that you told me to close the ticket. Hence, I am closing the ticket.

Thank you