Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Deprecation of the env resolver for variable environment interpolations #573

Closed
odelalleau opened this issue Feb 26, 2021 · 4 comments · Fixed by #606
Closed

Deprecation of the env resolver for variable environment interpolations #573

odelalleau opened this issue Feb 26, 2021 · 4 comments · Fixed by #606
Assignees
Milestone

Comments

@odelalleau
Copy link
Collaborator

odelalleau commented Feb 26, 2021

In OmegaConf 2.1, the env resolver is being deprecated in favor of a new oc.env resolver.

To migrate to this new resolver:

  • Replace ${env:MYVAR} with ${oc.env:MYVAR}
  • Consider whether your config or code may need to be updated due to the following three main differences with oc.env:
    1. Unlike env, oc.env does not convert environment variables to primitives automatically (booleans, integers and floats) and always return the string representation. If you wish to convert from string to a different type, you may either take advantage of automatic conversion for typed nodes (see example below), or use the new oc.decode resolver (see its documentation)
    2. The default value of oc.env is converted to string using str(default) unless it's null (representing Python None). For instance, ${env:OMP_NUM_THREADS,1} will return the string "1" if OMP_NUM_THREADS is not set. And ${env:OMP_NUM_THREADS,null} will return None if if OMP_NUM_THREADS is not set.
    3. Unlike env, oc.env is un-cached. this means that if MYVAR is updated within the program's process, the change will appear when accessing ${oc.env:MYVAR}.
@omry

This comment has been minimized.

@odelalleau

This comment has been minimized.

@omry

This comment has been minimized.

odelalleau added a commit to odelalleau/omegaconf that referenced this issue Mar 15, 2021
* Restore and deprecate the old `env` resolver for backward
  compatibility with OmegaConf 2.0

* The new `oc.env` resolver keeps the string representation of
  environment variables, and does not use the cache

* The new `oc.decode` resolver can be used to parse and evaluate strings
  according to the OmegaConf grammar

Fixes omry#383
Fixes omry#573
Fixes omry#574
odelalleau added a commit to odelalleau/omegaconf that referenced this issue Mar 15, 2021
* Restore and deprecate the old `env` resolver for backward
  compatibility with OmegaConf 2.0

* The new `oc.env` resolver keeps the string representation of
  environment variables, and does not use the cache

* The new `oc.decode` resolver can be used to parse and evaluate strings
  according to the OmegaConf grammar

Fixes omry#383
Fixes omry#573
Fixes omry#574
odelalleau added a commit to odelalleau/omegaconf that referenced this issue Mar 18, 2021
* Restore and deprecate the old `env` resolver for backward
  compatibility with OmegaConf 2.0

* The new `oc.env` resolver keeps the string representation of
  environment variables, and does not use the cache

* The new `oc.decode` resolver can be used to parse and evaluate strings
  according to the OmegaConf grammar

Fixes omry#383
Fixes omry#573
Fixes omry#574
odelalleau added a commit that referenced this issue Mar 18, 2021
* Introduce new `oc.env` and `oc.decode` resolvers

* Restore and deprecate the old `env` resolver for backward
  compatibility with OmegaConf 2.0

* The new `oc.env` resolver keeps the string representation of
  environment variables, does not use the cache, and accepts None as
  default value

* The new `oc.decode` resolver can be used to parse and evaluate strings
  according to the OmegaConf grammar

Fixes #383
Fixes #573
Fixes #574

* Allow `oc.decode` to evaluate interpolations

* Improve documentation of `oc.decode`

In particular it explains that dictionaries and lists are automatically
converted to transient config nodes.

* Update docs/source/usage.rst

Co-authored-by: Omry Yadan <omry@fb.com>

* Update news/573.api_change

Co-authored-by: Omry Yadan <omry@fb.com>

* Update omegaconf/_utils.py

Co-authored-by: Omry Yadan <omry@fb.com>

* Update omegaconf/_utils.py

Co-authored-by: Omry Yadan <omry@fb.com>

* Remove the USERID example

* Show example of quoted string as default value for `oc.env`

* Remove duplicated comments (# #)

* Validate default value of `oc.env` even when not used

* Simplify tests with recwarn

* Use convenience `show()` function in doc to show type and value

* Update doc on string interpolations

* More readable test formatting

* Improve comment formatting

* Restore interpolation examples

* Update docs/notebook/Tutorial.ipynb

Co-authored-by: Omry Yadan <omry@fb.com>

* Update docs/source/usage.rst

Co-authored-by: Omry Yadan <omry@fb.com>

* Update docs/source/usage.rst

Co-authored-by: Omry Yadan <omry@fb.com>

* Rephrasing in doc

* Use `show()` function in doc

* Raise a KeyError instead of ValidationError for missing env variables

* Remove handling of "null" as default in legacy env resolver

* Update news

* Explicit typing for the default value of the `oc.env` resolver

* Use a more appropriate exception type

* Update tests/test_interpolation.py

Co-authored-by: Omry Yadan <omry@fb.com>

* Safer markers for default values

* Fix coverage

* Use more appropriate TypeError

* Refactor: consistent use of _DEFAULT_MARKER_

Co-authored-by: Omry Yadan <omry@fb.com>
@omry omry removed the In progress label Mar 24, 2021
@odelalleau
Copy link
Collaborator Author

odelalleau commented Mar 24, 2021

This is an example of how to take advantage of the automatic conversion for typed nodes (see also the documentation):

import os
from dataclasses import dataclass
from typing import Any

from omegaconf import II, OmegaConf


@dataclass
class Config:
    some_int: int = II("oc.env: SOME_INT, '123'")      # converted to int
    some_bool: bool = II("oc.env: SOME_BOOL, 'True'")  # converted to bool
    some_any: Any = II("oc.env: SOME_ANY, '123'")      # not converted (untyped)


cfg = OmegaConf.structured(Config)

assert cfg.some_int == 123
assert cfg.some_bool is True
assert cfg.some_any == "123"

os.environ["SOME_INT"] = "456"
os.environ["SOME_BOOL"] = "false"
os.environ["SOME_ANY"] = "false"

assert cfg.some_int == 456
assert cfg.some_bool is False
assert cfg.some_any == "false"

shagunsodhani added a commit to shagunsodhani/mjrl_dev that referenced this issue Apr 23, 2022
env resolver has been deprecated. For more details, refers omry/omegaconf#573
pixelb added a commit to pixelb/omegaconf that referenced this issue May 12, 2022
Was slated for removal in 2.2

The deprecation of ${env} was handled in issue omry#573
This addresses issue omry#821
pixelb added a commit to pixelb/omegaconf that referenced this issue May 12, 2022
Was slated for removal in 2.2

The deprecation of ${env} was handled in issue omry#573
This addresses issue omry#821
pixelb added a commit that referenced this issue May 12, 2022
Was slated for removal in 2.2

The deprecation of ${env} was handled in issue #573
This addresses issue #821
facebook-github-bot pushed a commit to facebookresearch/mmf that referenced this issue Jun 22, 2022
Summary:
`env` is deprecated in OmegaConf 2.1 (currently running in fbcode)
in favor of  `oc.env`

`env` is deleted in OC 2.2 (we are upgrading fbcode to omegaconf 2.2)

see omry/omegaconf#573 for more context

Reviewed By: ebsmothers

Differential Revision: D37330921

fbshipit-source-id: 7d48da6d08fe8585825038e0707738b6ecaf0644
RichJackson pushed a commit to AstraZeneca/KAZU that referenced this issue Nov 30, 2022
The old one is deprecated in OmegaConf 2.1 (which our version of
hydra uses): omry/omegaconf#573
RichJackson pushed a commit to AstraZeneca/KAZU that referenced this issue Nov 30, 2022
The old one is deprecated in OmegaConf 2.1 (which our version of
hydra uses): omry/omegaconf#573
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants