Skip to content
This repository has been archived by the owner on Sep 13, 2023. It is now read-only.

Resolve requirements installed from Git or local source #645

Merged
merged 7 commits into from
Mar 23, 2023

Conversation

aguschin
Copy link
Contributor

@aguschin aguschin commented Mar 17, 2023

For a package installed with pip install git+https://... this will pin down the URL to the package to be installed from Git.

For a package installed with pip install local/path this will collect the package and add it as base64-encoded string to .mlem file.

PR also adds --build_arg option for Docker builder to pass args at building time.

TODO:

  • check that nothing breaks because of new fields in requirements: (e.g. source_url)

For k8s deployment this works like:

$ mlem declare deployment kubernetes deployer \
  --image_name myimage --service_type loadbalancer --registry remote \
  --env docker --env.registry remote --registry.host localhost --namespace myns \
  --build_arg.0 GITHUB_TOKEN --build_arg.1 GITHUB_USERNAME=aguschin
💾 Saving deployment to deployer.mlem

Note that one ARG value is set, another isn't:

$ cat deployer.mlem
build_arg:
- GITHUB_TOKEN
- GITHUB_USERNAME=aguschin
env:
  object_type: env
  registry:
    type: remote
  type: kubernetes
image_name: myimage
namespace: myns
object_type: deployment
registry:
  host: localhost
  type: remote
service_type:
  type: loadbalancer
type: kubernetes

Now at mlem deploy --load deployer.mlem MLEM will use GITHUB_TOKEN env var if set. If var has a different name, this can be run like

$ GITHUB_TOKEN=$ENV_VAR_TO_USE mlem deploy --load deployer.mlem

Env var takes precedence, so if you set GITHUB_USERNAME, it's value will be used instead of aguschin.

@aguschin aguschin requested a review from a team as a code owner March 17, 2023 10:33
@aguschin aguschin self-assigned this Mar 17, 2023
@aguschin aguschin temporarily deployed to internal March 17, 2023 10:33 — with GitHub Actions Inactive
@codecov
Copy link

codecov bot commented Mar 17, 2023

Codecov Report

Patch coverage: 68.42% and project coverage change: -0.08 ⚠️

Comparison is base (3bc673e) 85.97% compared to head (3c2ea41) 85.90%.

Additional details and impacted files
@@            Coverage Diff             @@
##             main     #645      +/-   ##
==========================================
- Coverage   85.97%   85.90%   -0.08%     
==========================================
  Files         107      107              
  Lines        9770     9805      +35     
==========================================
+ Hits         8400     8423      +23     
- Misses       1370     1382      +12     
Impacted Files Coverage Δ
mlem/contrib/docker/context.py 81.00% <60.00%> (-0.79%) ⬇️
mlem/core/requirements.py 93.63% <65.21%> (-2.13%) ⬇️
mlem/contrib/docker/base.py 85.00% <100.00%> (ø)
mlem/contrib/kubernetes/base.py 85.57% <100.00%> (+0.28%) ⬆️
mlem/contrib/kubernetes/build.py 100.00% <100.00%> (ø)
mlem/utils/module.py 91.55% <100.00%> (ø)

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

☔ View full report in Codecov by Sentry.
📢 Do you have feedback about the report comment? Let us know in this issue.

@aguschin aguschin temporarily deployed to internal March 17, 2023 11:20 — with GitHub Actions Inactive
@aguschin aguschin temporarily deployed to internal March 17, 2023 16:21 — with GitHub Actions Inactive
@aguschin aguschin temporarily deployed to internal March 21, 2023 04:53 — with GitHub Actions Inactive
@aguschin aguschin temporarily deployed to internal March 21, 2023 12:29 — with GitHub Actions Inactive
@aguschin aguschin temporarily deployed to internal March 21, 2023 13:20 — with GitHub Actions Inactive
@aguschin aguschin merged commit 19b1066 into main Mar 23, 2023
@aguschin aguschin deleted the fix/requirements-parsing branch March 23, 2023 11:06
aguschin added a commit that referenced this pull request Mar 31, 2023
this is for
#647 (comment)

This work similarly to #645:

`mlem build docker` works the same, but I'll provide an example for k8s
deploy here:

```
$ mlem declare deployment kubernetes deployer \
  --image_name myimage --service_type loadbalancer --registry remote \
  --env docker --env.registry remote --registry.host localhost --namespace myns \
  --set_env.0 VAR1 --set_env.1 VAR2=aguschin
💾 Saving deployment to deployer.mlem
```

```yaml
# deployer.mlem
env:
  object_type: env
  registry:
    type: remote
  type: kubernetes
image_name: myimage
namespace: myns
object_type: deployment
registry:
  host: localhost
  type: remote
service_type:
  type: loadbalancer
set_env:
- VAR1
- VAR2=aguschin
type: kubernetes
```

Then in Dockerfile for your image you'll get:
```
...
ENV VAR1=VALUE1
ENV VAR2=aguschin
```

`VALUE1` will be taken from shell/env vars with `os.getenv("VAR1")`. If
it's not set, an Exception will be raised by MLEM.
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
No open projects
Archived in project
Development

Successfully merging this pull request may close these issues.

1 participant