Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support HTTP proxies in eland_import_hub_model #688

Merged
merged 6 commits into from
May 2, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
48 changes: 48 additions & 0 deletions docs/guide/machine-learning.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -60,6 +60,12 @@ $ eland_import_hub_model <authentication> \ <1>
<4> Specify the type of NLP task. Supported values are `fill_mask`, `ner`,
`question_answering`, `text_classification`, `text_embedding`, and `zero_shot_classification`.

For more information about the available options, run `eland_import_hub_model` with the `--help` option.

[source,bash]
------------------------
$ eland_import_hub_model --help
------------------------

[discrete]
[[ml-nlp-pytorch-docker]]
Expand Down Expand Up @@ -148,6 +154,23 @@ underscores `__`.

--

[discrete]
[[ml-nlp-pytorch-proxy]]
==== Connect to Elasticsearch through a proxy

Behind the scenes, Eland uses the `requests` Python library, which
https://requests.readthedocs.io/en/latest/user/advanced/#proxies[allows configuring
proxies through an environment variable]. For example, to use an HTTP proxy to connect to
an HTTPS Elasticsearch cluster, you need to set the `HTTPS_PROXY` environment variable
when invoking Eland:

[source,bash]
--------------------------------------------------
HTTPS_PROXY=http://proxy-host:proxy-port eland_import_hub_model ...
--------------------------------------------------

If you disabled security on your Elasticsearch cluster, you should use `HTTP_PROXY`
instead.

[discrete]
[[ml-nlp-pytorch-auth]]
Expand Down Expand Up @@ -191,3 +214,28 @@ eland_import_hub_model --es-api-key <api-key> --url https://<hostname>:<port> ..
eland_import_hub_model --hub-access-token <access-token> ...
--------------------------------------------------
--

[discrete]
[[ml-nlp-pytorch-tls]]
==== TLS/SSL

The following TLS/SSL options for Elasticsearch are available when using the import script:


* Specify alternate CA bundle to verify the cluster certificate:
+
--
[source,bash]
--------------------------------------------------
eland_import_hub_model --ca-certs CA_CERTS ...
--------------------------------------------------
--

* Disable TLS/SSL verification altogether (strongly discouraged):
+
--
[source,bash]
--------------------------------------------------
eland_import_hub_model --insecure ...
--------------------------------------------------
--
1 change: 1 addition & 0 deletions eland/cli/eland_import_hub_model.py
Original file line number Diff line number Diff line change
Expand Up @@ -164,6 +164,7 @@ def get_es_client(cli_args, logger):
"request_timeout": 300,
"verify_certs": cli_args.insecure,
"ca_certs": cli_args.ca_certs,
"node_class": "requests",
}

# Deployment location
Expand Down