AML Endpoint Bug Fix and Refactor

[rlundeen2]

Description

Users were encountering a bug with multiple loops when using the AML endpoint. When investigating, there was another issue - it turns out the header of the request had changed (another bug). How it was setup, AMLOnlineEndpointChat was difficult to debug, which made root causing these two other bugs difficult.

This PR fixes both the above bugs and makes the code easier to debug going forward.

• Moved the AML Gandalf demo to its own simpler code notebook aml_endpoints.ipynb
• Created net_utility to standardize network request and backoff logic
• Simplified the logic in aml_online_endpoint_chat
• Removed complete_chat_async as a requirement for the ChatSupport interface. It has perf benefits with a lot of threads but I'd like to get to a stable place first, and to add it back, it would have to be rewritten

Tests

• no new tests required
• new tests added
• existing tests adjusted

Documentation

• no documentation changes needed
• documentation added or edited
• example notebook added or updated

[github-actions]

Test Results

94 tests  +1   94 ✅ +1   9s ⏱️ ±0s
 1 suites ±0    0 💤 ±0 
 1 files   ±0    0 ❌ ±0 

Results for commit 85ff67f. ± Comparison against base commit b37578b.

This pull request removes 8 and adds 9 tests. Note that renamed tests count towards both.

tests.test_aml_online_endpoint_chat ‑ test_complete_chat_async
tests.test_aml_online_endpoint_chat ‑ test_extract_first_response_message_empty_list
tests.test_aml_online_endpoint_chat ‑ test_extract_first_response_message_missing_key
tests.test_aml_online_endpoint_chat ‑ test_extract_first_response_message_normal
tests.test_aml_online_endpoint_chat ‑ test_get_headers_with_empty_api_key
tests.test_common ‑ test_environment_variables_prefers_passed
tests.test_common ‑ test_environment_variables_throws_if_not_set
tests.test_common ‑ test_environment_variables_uses_default

tests.test_aml_online_endpoint_chat ‑ test_complete_chat_bad_json_response
tests.test_common_environment ‑ test_environment_variables_prefers_passed
tests.test_common_environment ‑ test_environment_variables_throws_if_not_set
tests.test_common_environment ‑ test_environment_variables_uses_default
tests.test_common_net_utility ‑ test_get_httpx_client_type[False-Client]
tests.test_common_net_utility ‑ test_get_httpx_client_type[True-AsyncClient]
tests.test_common_net_utility ‑ test_make_request_and_raise_if_error_failure
tests.test_common_net_utility ‑ test_make_request_and_raise_if_error_retries
tests.test_common_net_utility ‑ test_make_request_and_raise_if_error_success

♻️ This comment has been updated with latest results.