[BUG] pkg_resources.DistributionNotFound: The 'urllib3<1.24,>=1.21.1' when python3-urllib3-0:1.25 is installed #59982
Assignees
Labels
Aluminium
Release Post Mg and Pre Si
Bug
broken, incorrect, or confusing behavior
Packaging
Related to packaging of Salt, not Salt's support for package management.
point-release
minor release
Milestone
Comments
sagetherage
added
the
Packaging
|
Looks like I will need to update the python3-requests package and add a python3-certifi package. |
|
Been facing verification failures (testing any |
saltstack-formulas-github
pushed a commit
to netmanagers/salt-image-builder
that referenced
this issue
Apr 20, 2021
Use until packaging resolved, see: * saltstack/salt#59982
|
I'm also seeing another package conflict after running the suggested workaround and trying to do a To repro, start with a pure, latest Amzn Linux 2 AMI, ran the following commands The Adding a version lock for and allows the |
|
For the 3003.1 release we're no longer providing python3-reqeusts or python3-urllib3 from the salt repos as amazonlinux 2 provides those now. It is preferable to downgrade python3-requests rather than python3-urllib3 going forward for salt 3003 or older so that a future upgrade goes smoothly. If you don't want to do pinning, you can just 3003.1 and forward should work installing fresh out of the box without any pinning. Also, because amazon now provides dependencies for python3-psutil you now must before installing salt if you have epel enabled. This is an issue between epel and the amazonlinux 2 repos in general and is an upstream issue we can't really fix properly. It's something either amazon or epel will have to fix. |
|
Hey there - when I checked the release notes for salt-3003.1* I didn't see a mention of this fix. I also notice the *https://docs.saltproject.io/en/master/topics/releases/3003.1.html |
|
Yes, latest will pick up those changes when 3003.1 is released since latest is a symlink to whatever the latest release is. 3003.1 isn't tagged yet. @sagetherage should we throw that in the release notes before we tag? saltstack/salt-pack-py3@c2c71c8 is the commit associated with the dependency fixes on amazon linux 2. |
|
@bryceml yes, can you open a PR to add that to the release notes for 3003.1? |
bryceml
pushed a commit
to bryceml/salt
that referenced
this issue
Jun 7, 2021
garethgreenaway
pushed a commit
that referenced
this issue
Jun 8, 2021
xeacott
pushed a commit
to xeacott/salt
that referenced
this issue
Jun 8, 2021
xeacott
pushed a commit
to xeacott/salt
that referenced
this issue
Jun 8, 2021
xeacott
pushed a commit
to xeacott/salt
that referenced
this issue
Jun 8, 2021
Ch3LL
added a commit
that referenced
this issue
Jun 23, 2021
* Merge 3002.6 bugfix changes (#59822) * Pass `CI_RUN` as an environment variable to the test run. This allows us to know if we're running the test suite under a CI environment or not and adapt/adjust if needed * Migrate `unit.setup` to PyTest * Backport ae36b15 just for test_install.py * Only skip tests on CI runs * Always store git sha in _version.py during installation * Fix PEP440 compliance. The wheel metadata version 1.2 states that the package version MUST be PEP440 compliant. This means that instead of `3002.2-511-g033c53eccb`, the salt version string should look like `3002.2+511.g033c53eccb`, a post release of `3002.2` ahead by 511 commits with the git sha `033c53eccb` * Fix and migrate `tests/unit/test_version.py` to PyTest * Skip test if `easy_install` is not available * We also need to be PEP440 compliant when there's no git history * Allow extra_filerefs as sanitized kwargs for SSH client * Fix regression on cmd.run when passing tuples as cmd Co-authored-by: Alexander Graul <agraul@suse.com> * Add unit tests to ensure cmd.run accepts tuples * Add unit test to check for extra_filerefs on SSH opts * Add changelog file * Fix comment for test case * Fix unit test to avoid failing on Windows * Skip failing test on windows * Fix test to work on Windows * Add all ssh kwargs to sanitize_kwargs method * Run pre-commit * Fix pylint * Fix cmdmod loglevel and module_names tests * Fix pre-commit * Skip ssh tests if binary does not exist * Use setup_loader for cmdmod test * Prevent argument injection in restartcheck * Add changelog for restartcheck fix * docs_3002.6 * Add back tests removed in merge Co-authored-by: Pedro Algarvio <pedro@algarvio.me> Co-authored-by: Megan Wilhite <megan.wilhite@gmail.com> Co-authored-by: Bryce Larson <brycel@vmware.com> Co-authored-by: Pablo Suárez Hernández <psuarezhernandez@suse.com> Co-authored-by: Alexander Graul <agraul@suse.com> Co-authored-by: Frode Gundersen <fgundersen@saltstack.com> * Remove glance state module in favor of glance_image * update wording in changelog * bump deprecation warning to Silicon. * Updating warnutil version to Phosphorous. * Update salt/modules/keystone.py Co-authored-by: Megan Wilhite <megan.wilhite@gmail.com> * Check $HOMEBREW_PREFIX when linking against libcrypto When loading `libcrypto`, Salt checks for a Homebrew installation of `openssl` at Homebrew's default prefix of `/usr/local`. However, on Apple Silicon Macs, Homebrew's default installation prefix is `/opt/homebrew`. On all platforms, the prefix is configurable. If Salt doesn't find one of those `libcrypto`s, it will fall back on the un-versioned `/usr/lib/libcrypto.dylib`, which will cause the following crash: Application Specific Information: /usr/lib/libcrypto.dylib abort() called Invalid dylib load. Clients should not load the unversioned libcrypto dylib as it does not have a stable ABI. This commit checks $HOMEBREW_PREFIX instead of hard-coding `/usr/local`. * Add test case * Add changelog for 59808 * Add changelog entry * Make _find_libcrypto fail on Big Sur if it can't find a library Right now, if `_find_libcrypto` can't find any externally-managed versions of libcrypto, it will fall back on the pre-Catalina un-versioned system libcrypto. This does not exist on Big Sur and it would be better to raise an exception here rather than crashing later when trying to open it. * Update _find_libcrypto tests This commit simplifies the unit tests for _find_libcrypto by mocking out the host's filesystem and testing the common libcrypto installations (brew, ports, etc.) on Big Sur. It simplifies the tests for falling back on system versions of libcrypto on previous versions of macOS. * Fix description of test_find_libcrypto_with_system_before_catalina * Patch sys.platform for test_rsax931 tests * modules/match: add missing "minion_id" in Pillar example The documented Pillar example for `match.filter_by` lacks the `minion_id` parameter. Without it, the assignment won't work as expected. - fix documentation - add tests: - to prove the misbehavior of the documented example - to prove the proper behaviour when supplying `minion_id` - to ensure some misbehaviour observed with compound matchers doesn't occur * Fix for issue #59773 - When instantiating the loader grab values of grains and pillars if they are NamedLoaderContext instances. - The loader uses a copy of opts. - Impliment deepcopy on NamedLoaderContext instances. * Add changelog for #59773 * _get_initial_pillar function returns pillar * Fix linter issues * Clean up test * Bump deprecation release for neutron * Uncomment Sulfur release name * Removing the _ext_nodes deprecation warning and alias. * Adding changelog. * Renaming changelog file. * Update 59804.removed * Initial pass at fips_mode config option * Fix pre-commit * Fix tests and add changelog * update docs 3003 * update docs 3003 - newline * Fix warts in changelog * update releasenotes 3003 * add ubuntu-2004-amd64 m2crypto pycryptodome and tcp tests * add distro_arch * changing the cloud platforms file missed in 1a9b7be * Update __utils__ calls to import utils in azure * Add changelog for 59744 * Fix azure unit tests and move to pytest * Use contextvars from site-packages for thin If a contextvars package exists one of the site-packages locations use it for the generated thin tarball. This overrides python's builtin contextvars and allows salt-ssh to work with python <=3.6 even when the master's python is >3.6 (Fixes #59942) * Add regression test for #59942 * Add changelog for #59942 * Update filemap to include test_py_versions * Fix broken thin tests * Always install the `contextvars` backport, even on Py3.7+ Without this change, salt-ssh cannot target systems with Python <= 3.6 * Use salt-factories to handle the container. Don't override default roster * Fix thin tests on windows * No need to use warn log level here * Fix getsitepackages for old virtualenv versions * Add explicit pyobjc reqs * Add back the passthrough stuff * Remove a line so pre-commit will run * Bugfix release docs * Bugfix release docs * Removing pip-compile log files * Bump requirements to address a few security issues * Address traceback on macOS ``` Traceback (most recent call last): File "setup.py", line 1448, in <module> setup(distclass=SaltDistribution) File "/Users/jenkins/setup-tests/.venv/lib/python3.7/site-packages/setuptools/__init__.py", line 153, in setup return distutils.core.setup(**attrs) File "/opt/salt/lib/python3.7/distutils/core.py", line 108, in setup _setup_distribution = dist = klass(attrs) File "setup.py", line 1068, in __init__ self.update_metadata() File "setup.py", line 1074, in update_metadata attrvalue = getattr(self, attrname, None) File "setup.py", line 1182, in _property_install_requires install_requires += _parse_requirements_file(reqfile) File "setup.py", line 270, in _parse_requirements_file platform.python_version(), _parse_op(op), _parse_ver(ver) File "setup.py", line 247, in _check_ver return getattr(operator, "__{}__".format(op))(pyver, wanted) File "/opt/salt/lib/python3.7/distutils/version.py", line 46, in __eq__ c = self._cmp(other) File "/opt/salt/lib/python3.7/distutils/version.py", line 337, in _cmp if self.version < other.version: TypeError: '<' not supported between instances of 'str' and 'int' ``` * Replace `saltstack.com` with `saltproject.io` on URLs being tested * Add back support to load old entrypoints by iterating instead of type checking Fixes #59961 * Fix issue #59975 * Fix pillar serialization for jinja #60083 * Fix test * Add changelog for #60083 * Update changelog and release for 3003.1 * Remove the changelog source refs * Add connect to IPCMessageSubscriber's async_methods Fixes #60049 by making sure an IPCMessageSubscriber that is wrapped by SyncWrapper has a connect method that runs the coroutine rather than returns a fugure. * Add changelog for #60049 * Update 60049.fixed * Fix coroutine spelling error Co-authored-by: Wayne Werner <waynejwerner@gmail.com> * IPC on windows cannot use socket paths Fixes #60298 * Update Jinja2 and lxml due to security related bugfix releases Jinja2 ------ CVE-2020-28493 moderate severity Vulnerable versions: < 2.11.3 Patched version: 2.11.3 This affects the package jinja2 from 0.0.0 and before 2.11.3. The ReDOS vulnerability of the regex is mainly due to the sub-pattern [a-zA-Z0-9.-]+.[a-zA-Z0-9.-]+ This issue can be mitigated by Markdown to format user content instead of the urlize filter, or by implementing request timeouts and limiting process memory. lxml ---- CVE-2021-28957 moderate severity Vulnerable versions: < 4.6.3 Patched version: 4.6.3 An XSS vulnerability was discovered in the python lxml clean module versions before 4.6.3. When disabling the safe_attrs_only and forms arguments, the Cleaner class does not remove the formaction attribute allowing for JS to bypass the sanitizer. A remote attacker could exploit this flaw to run arbitrary JS code on users who interact with incorrectly sanitized HTML. This issue is patched in lxml 4.6.3. * fix github actions jobs on branch until bullseye comes out * Upgrade to `six==1.16.0` to avoid problems on CI runs ``` 13:59:02 nox > Session invoke-pre-commit was successful. 13:59:02 nox > Running session invoke-pre-commit 13:59:02 nox > pip install --progress-bar=off -r requirements/static/ci/py3.7/invoke.txt 13:59:02 Collecting blessings==1.7 13:59:02 Using cached blessings-1.7-py3-none-any.whl (18 kB) 13:59:02 Collecting invoke==1.4.1 13:59:02 Using cached invoke-1.4.1-py3-none-any.whl (210 kB) 13:59:02 Collecting pyyaml==5.3.1 13:59:02 Using cached PyYAML-5.3.1.tar.gz (269 kB) 13:59:02 Collecting six==1.15.0 13:59:02 Using cached six-1.15.0-py2.py3-none-any.whl (10 kB) 13:59:02 Building wheels for collected packages: pyyaml 13:59:02 Building wheel for pyyaml (setup.py) ... -� �\� �|� �/� �-� �\� �|� �done 13:59:02 Created wheel for pyyaml: filename=PyYAML-5.3.1-cp37-cp37m-linux_x86_64.whl size=546391 sha256=e42e1d66cc32087f4d33ceb81268c86b59f1a97029b19459f91b8d6ad1430167 13:59:02 Stored in directory: /var/jenkins/.cache/pip/wheels/5e/03/1e/e1e954795d6f35dfc7b637fe2277bff021303bd9570ecea653 13:59:02 Successfully built pyyaml 13:59:02 Installing collected packages: six, pyyaml, invoke, blessings 13:59:02 Attempting uninstall: six 13:59:02 Found existing installation: six 1.16.0 13:59:02 Uninstalling six-1.16.0: 13:59:02 ERROR: Could not install packages due to an OSError: [Errno 2] No such file or directory: '/var/jenkins/.cache/pre-commit/repomw8oee1s/py_env-python3/lib/python3.7/site-packages/__pycache__/six.cpython-37.pyc' 13:59:02 13:59:02 nox > Command pip install --progress-bar=off -r requirements/static/ci/py3.7/invoke.txt failed with exit code 1 13:59:02 nox > Session invoke-pre-commit failed. ``` * add changelog for #59982 * Regression test for #56273 * Fix race condition in batch. #56273 * Add changelog for #56273 * Update salt/client/__init__.py Co-authored-by: Pedro Algarvio <pedro@algarvio.me> * Update doc for salt/client * Update changelog/56273.fixed Thoreau said, "Simplify, Simplify" * Update docs * Update docs * Update CHANGELOG.md * Update 3003.1.rst * Fix changelog Co-authored-by: Daniel Wozniak <dwozniak@saltstack.com> Co-authored-by: Pedro Algarvio <pedro@algarvio.me> Co-authored-by: Bryce Larson <brycel@vmware.com> Co-authored-by: Pablo Suárez Hernández <psuarezhernandez@suse.com> Co-authored-by: Alexander Graul <agraul@suse.com> Co-authored-by: Frode Gundersen <fgundersen@saltstack.com> Co-authored-by: Gareth J. Greenaway <gareth@saltstack.com> Co-authored-by: Gareth J. Greenaway <gareth@wiked.org> Co-authored-by: Hoa-Long Tam <hoalong@apple.com> Co-authored-by: krionbsd <krion@freebsd.org> Co-authored-by: Elias Probst <e.probst@ssc-services.de> Co-authored-by: Daniel A. Wozniak <dwozniak@vmware.com> Co-authored-by: Frode Gundersen <frogunder@gmail.com> Co-authored-by: twangboy <slee@saltstack.com> Co-authored-by: twangboy <leesh@vmware.com> Co-authored-by: ScriptAutomate <derek@icanteven.io> Co-authored-by: Wayne Werner <waynejwerner@gmail.com>
|
3003.1 has been released, closing issue. |
|
Thanks, the fix is working fine for Just mentioning -- for our image builder, we still need to keep the workaround for versions less than that, such as |
|
with some quick testing, it appears that setting we would just have to change the .repo files on repo.saltproject.io and change the template in the bootstrap script. @dmurphy18 thoughts on doing that? would there be any downsides? wondering what would be better if amazon releases more python dependencies in the future. |
saltstack-formulas-github
pushed a commit
to netmanagers/salt-image-builder
that referenced
this issue
Jun 23, 2021
Still required for Salt versions that are less than `3003.1`: * saltstack/salt#59982
|
I would think favor theirs since they have an interest in keeping things up to date, and also tend to roll out new stuff. Problem I see, is if we have to make a fix for something which their version will override. |
|
it appears doing it appears only python3-jinja2, python3-markupsafe, and python3-pycurl get installed with a We should find out if an older version of any of these poses a problem. |
|
Looks like at a minimum, we need a newer version of python3-markupsafe based on some tests with those requirements. We therefore can't change the I've tested, and everything seems to work fine if I just delete the |
clrpackages
pushed a commit
to clearlinux-pkgs/salt
that referenced
this issue
Jun 29, 2021
Bryce Larson (5):
add ubuntu-2004-amd64 m2crypto pycryptodome and tcp tests
add distro_arch
changing the cloud platforms file missed in 1a9b7be0e2f300d87924731dc5816fd1000cd22b
fix github actions jobs on branch until bullseye comes out
add changelog for saltstack/salt#59982
Daniel A. Wozniak (17):
Use contextvars from site-packages for thin
Add regression test for #59942
Add changelog for #59942
Update filemap to include test_py_versions
Fix broken thin tests
Fix thin tests on windows
No need to use warn log level here
Fix getsitepackages for old virtualenv versions
Fix issue #59975
Fix pillar serialization for jinja #60083
Fix test
Add changelog for #60083
Add connect to IPCMessageSubscriber's async_methods
Add changelog for #60049
Regression test for #56273
Fix race condition in batch. #56273
Add changelog for #56273
Daniel Wozniak (1):
Update salt/client/__init__.py
Frode Gundersen (5):
update releasenotes 3003
Update docs
Update docs
Update CHANGELOG.md
Update 3003.1.rst
Gareth J. Greenaway (1):
Update 60049.fixed
Megan Wilhite (5):
Update __utils__ calls to import utils in azure
Add changelog for 59744
Fix azure unit tests and move to pytest
Fix coroutine spelling error
Update doc for salt/client
Pedro Algarvio (9):
Always install the `contextvars` backport, even on Py3.7+
Use salt-factories to handle the container. Don't override default roster
Bump requirements to address a few security issues
Address traceback on macOS
Replace `saltstack.com` with `saltproject.io` on URLs being tested
Add back support to load old entrypoints by iterating instead of type checking
IPC on windows cannot use socket paths
Update Jinja2 and lxml due to security related bugfix releases
Upgrade to `six==1.16.0` to avoid problems on CI runs
ScriptAutomate (5):
Bugfix release docs
Bugfix release docs
Removing pip-compile log files
Update changelog and release for 3003.1
Remove the changelog source refs
Wayne Werner (1):
Update changelog/56273.fixed
twangboy (3):
Add explicit pyobjc reqs
Add back the passthrough stuff
Remove a line so pre-commit will run
|
@myii We've updated the repos for 3000.9, 3001.7, and 3002.6 so this should be fixed for those as well now. |
saltstack-formulas-github
pushed a commit
to netmanagers/salt-image-builder
that referenced
this issue
Jun 30, 2021
This reverts commits: * 95e8362 * 4a92088 The workaround is no longer required: * saltstack/salt#59982 (comment)
saltstack-formulas-github
pushed a commit
to netmanagers/salt-image-builder
that referenced
this issue
Jun 30, 2021
## [1.39.3](https://gitlab.com/saltstack-formulas/infrastructure/salt-image-builder/compare/v1.39.2...v1.39.3) (2021-06-30) ### Reverts * **dockerfile.yum:** use Amazon stable builds workaround [skip ci] ([ce6568e](https://gitlab.com/saltstack-formulas/infrastructure/salt-image-builder/commit/ce6568ef28bf15e86b9057fb27629595cc913b2d)), closes [/github.com/saltstack/salt/issues/59982#issuecomment-871528035](https://gitlab.com//github.com/saltstack/salt/issues/59982/issues/issuecomment-871528035)
truzzon
pushed a commit
to truzzon/salt
that referenced
this issue
Aug 10, 2021
* Merge 3002.6 bugfix changes (saltstack#59822) * Pass `CI_RUN` as an environment variable to the test run. This allows us to know if we're running the test suite under a CI environment or not and adapt/adjust if needed * Migrate `unit.setup` to PyTest * Backport ae36b15 just for test_install.py * Only skip tests on CI runs * Always store git sha in _version.py during installation * Fix PEP440 compliance. The wheel metadata version 1.2 states that the package version MUST be PEP440 compliant. This means that instead of `3002.2-511-g033c53eccb`, the salt version string should look like `3002.2+511.g033c53eccb`, a post release of `3002.2` ahead by 511 commits with the git sha `033c53eccb` * Fix and migrate `tests/unit/test_version.py` to PyTest * Skip test if `easy_install` is not available * We also need to be PEP440 compliant when there's no git history * Allow extra_filerefs as sanitized kwargs for SSH client * Fix regression on cmd.run when passing tuples as cmd Co-authored-by: Alexander Graul <agraul@suse.com> * Add unit tests to ensure cmd.run accepts tuples * Add unit test to check for extra_filerefs on SSH opts * Add changelog file * Fix comment for test case * Fix unit test to avoid failing on Windows * Skip failing test on windows * Fix test to work on Windows * Add all ssh kwargs to sanitize_kwargs method * Run pre-commit * Fix pylint * Fix cmdmod loglevel and module_names tests * Fix pre-commit * Skip ssh tests if binary does not exist * Use setup_loader for cmdmod test * Prevent argument injection in restartcheck * Add changelog for restartcheck fix * docs_3002.6 * Add back tests removed in merge Co-authored-by: Pedro Algarvio <pedro@algarvio.me> Co-authored-by: Megan Wilhite <megan.wilhite@gmail.com> Co-authored-by: Bryce Larson <brycel@vmware.com> Co-authored-by: Pablo Suárez Hernández <psuarezhernandez@suse.com> Co-authored-by: Alexander Graul <agraul@suse.com> Co-authored-by: Frode Gundersen <fgundersen@saltstack.com> * Remove glance state module in favor of glance_image * update wording in changelog * bump deprecation warning to Silicon. * Updating warnutil version to Phosphorous. * Update salt/modules/keystone.py Co-authored-by: Megan Wilhite <megan.wilhite@gmail.com> * Check $HOMEBREW_PREFIX when linking against libcrypto When loading `libcrypto`, Salt checks for a Homebrew installation of `openssl` at Homebrew's default prefix of `/usr/local`. However, on Apple Silicon Macs, Homebrew's default installation prefix is `/opt/homebrew`. On all platforms, the prefix is configurable. If Salt doesn't find one of those `libcrypto`s, it will fall back on the un-versioned `/usr/lib/libcrypto.dylib`, which will cause the following crash: Application Specific Information: /usr/lib/libcrypto.dylib abort() called Invalid dylib load. Clients should not load the unversioned libcrypto dylib as it does not have a stable ABI. This commit checks $HOMEBREW_PREFIX instead of hard-coding `/usr/local`. * Add test case * Add changelog for 59808 * Add changelog entry * Make _find_libcrypto fail on Big Sur if it can't find a library Right now, if `_find_libcrypto` can't find any externally-managed versions of libcrypto, it will fall back on the pre-Catalina un-versioned system libcrypto. This does not exist on Big Sur and it would be better to raise an exception here rather than crashing later when trying to open it. * Update _find_libcrypto tests This commit simplifies the unit tests for _find_libcrypto by mocking out the host's filesystem and testing the common libcrypto installations (brew, ports, etc.) on Big Sur. It simplifies the tests for falling back on system versions of libcrypto on previous versions of macOS. * Fix description of test_find_libcrypto_with_system_before_catalina * Patch sys.platform for test_rsax931 tests * modules/match: add missing "minion_id" in Pillar example The documented Pillar example for `match.filter_by` lacks the `minion_id` parameter. Without it, the assignment won't work as expected. - fix documentation - add tests: - to prove the misbehavior of the documented example - to prove the proper behaviour when supplying `minion_id` - to ensure some misbehaviour observed with compound matchers doesn't occur * Fix for issue saltstack#59773 - When instantiating the loader grab values of grains and pillars if they are NamedLoaderContext instances. - The loader uses a copy of opts. - Impliment deepcopy on NamedLoaderContext instances. * Add changelog for saltstack#59773 * _get_initial_pillar function returns pillar * Fix linter issues * Clean up test * Bump deprecation release for neutron * Uncomment Sulfur release name * Removing the _ext_nodes deprecation warning and alias. * Adding changelog. * Renaming changelog file. * Update 59804.removed * Initial pass at fips_mode config option * Fix pre-commit * Fix tests and add changelog * update docs 3003 * update docs 3003 - newline * Fix warts in changelog * update releasenotes 3003 * add ubuntu-2004-amd64 m2crypto pycryptodome and tcp tests * add distro_arch * changing the cloud platforms file missed in 1a9b7be * Update __utils__ calls to import utils in azure * Add changelog for 59744 * Fix azure unit tests and move to pytest * Use contextvars from site-packages for thin If a contextvars package exists one of the site-packages locations use it for the generated thin tarball. This overrides python's builtin contextvars and allows salt-ssh to work with python <=3.6 even when the master's python is >3.6 (Fixes saltstack#59942) * Add regression test for saltstack#59942 * Add changelog for saltstack#59942 * Update filemap to include test_py_versions * Fix broken thin tests * Always install the `contextvars` backport, even on Py3.7+ Without this change, salt-ssh cannot target systems with Python <= 3.6 * Use salt-factories to handle the container. Don't override default roster * Fix thin tests on windows * No need to use warn log level here * Fix getsitepackages for old virtualenv versions * Add explicit pyobjc reqs * Add back the passthrough stuff * Remove a line so pre-commit will run * Bugfix release docs * Bugfix release docs * Removing pip-compile log files * Bump requirements to address a few security issues * Address traceback on macOS ``` Traceback (most recent call last): File "setup.py", line 1448, in <module> setup(distclass=SaltDistribution) File "/Users/jenkins/setup-tests/.venv/lib/python3.7/site-packages/setuptools/__init__.py", line 153, in setup return distutils.core.setup(**attrs) File "/opt/salt/lib/python3.7/distutils/core.py", line 108, in setup _setup_distribution = dist = klass(attrs) File "setup.py", line 1068, in __init__ self.update_metadata() File "setup.py", line 1074, in update_metadata attrvalue = getattr(self, attrname, None) File "setup.py", line 1182, in _property_install_requires install_requires += _parse_requirements_file(reqfile) File "setup.py", line 270, in _parse_requirements_file platform.python_version(), _parse_op(op), _parse_ver(ver) File "setup.py", line 247, in _check_ver return getattr(operator, "__{}__".format(op))(pyver, wanted) File "/opt/salt/lib/python3.7/distutils/version.py", line 46, in __eq__ c = self._cmp(other) File "/opt/salt/lib/python3.7/distutils/version.py", line 337, in _cmp if self.version < other.version: TypeError: '<' not supported between instances of 'str' and 'int' ``` * Replace `saltstack.com` with `saltproject.io` on URLs being tested * Add back support to load old entrypoints by iterating instead of type checking Fixes saltstack#59961 * Fix issue saltstack#59975 * Fix pillar serialization for jinja saltstack#60083 * Fix test * Add changelog for saltstack#60083 * Update changelog and release for 3003.1 * Remove the changelog source refs * Add connect to IPCMessageSubscriber's async_methods Fixes saltstack#60049 by making sure an IPCMessageSubscriber that is wrapped by SyncWrapper has a connect method that runs the coroutine rather than returns a fugure. * Add changelog for saltstack#60049 * Update 60049.fixed * Fix coroutine spelling error Co-authored-by: Wayne Werner <waynejwerner@gmail.com> * IPC on windows cannot use socket paths Fixes saltstack#60298 * Update Jinja2 and lxml due to security related bugfix releases Jinja2 ------ CVE-2020-28493 moderate severity Vulnerable versions: < 2.11.3 Patched version: 2.11.3 This affects the package jinja2 from 0.0.0 and before 2.11.3. The ReDOS vulnerability of the regex is mainly due to the sub-pattern [a-zA-Z0-9.-]+.[a-zA-Z0-9.-]+ This issue can be mitigated by Markdown to format user content instead of the urlize filter, or by implementing request timeouts and limiting process memory. lxml ---- CVE-2021-28957 moderate severity Vulnerable versions: < 4.6.3 Patched version: 4.6.3 An XSS vulnerability was discovered in the python lxml clean module versions before 4.6.3. When disabling the safe_attrs_only and forms arguments, the Cleaner class does not remove the formaction attribute allowing for JS to bypass the sanitizer. A remote attacker could exploit this flaw to run arbitrary JS code on users who interact with incorrectly sanitized HTML. This issue is patched in lxml 4.6.3. * fix github actions jobs on branch until bullseye comes out * Upgrade to `six==1.16.0` to avoid problems on CI runs ``` 13:59:02 nox > Session invoke-pre-commit was successful. 13:59:02 nox > Running session invoke-pre-commit 13:59:02 nox > pip install --progress-bar=off -r requirements/static/ci/py3.7/invoke.txt 13:59:02 Collecting blessings==1.7 13:59:02 Using cached blessings-1.7-py3-none-any.whl (18 kB) 13:59:02 Collecting invoke==1.4.1 13:59:02 Using cached invoke-1.4.1-py3-none-any.whl (210 kB) 13:59:02 Collecting pyyaml==5.3.1 13:59:02 Using cached PyYAML-5.3.1.tar.gz (269 kB) 13:59:02 Collecting six==1.15.0 13:59:02 Using cached six-1.15.0-py2.py3-none-any.whl (10 kB) 13:59:02 Building wheels for collected packages: pyyaml 13:59:02 Building wheel for pyyaml (setup.py) ... -� �\� �|� �/� �-� �\� �|� �done 13:59:02 Created wheel for pyyaml: filename=PyYAML-5.3.1-cp37-cp37m-linux_x86_64.whl size=546391 sha256=e42e1d66cc32087f4d33ceb81268c86b59f1a97029b19459f91b8d6ad1430167 13:59:02 Stored in directory: /var/jenkins/.cache/pip/wheels/5e/03/1e/e1e954795d6f35dfc7b637fe2277bff021303bd9570ecea653 13:59:02 Successfully built pyyaml 13:59:02 Installing collected packages: six, pyyaml, invoke, blessings 13:59:02 Attempting uninstall: six 13:59:02 Found existing installation: six 1.16.0 13:59:02 Uninstalling six-1.16.0: 13:59:02 ERROR: Could not install packages due to an OSError: [Errno 2] No such file or directory: '/var/jenkins/.cache/pre-commit/repomw8oee1s/py_env-python3/lib/python3.7/site-packages/__pycache__/six.cpython-37.pyc' 13:59:02 13:59:02 nox > Command pip install --progress-bar=off -r requirements/static/ci/py3.7/invoke.txt failed with exit code 1 13:59:02 nox > Session invoke-pre-commit failed. ``` * add changelog for saltstack#59982 * Regression test for saltstack#56273 * Fix race condition in batch. saltstack#56273 * Add changelog for saltstack#56273 * Update salt/client/__init__.py Co-authored-by: Pedro Algarvio <pedro@algarvio.me> * Update doc for salt/client * Update changelog/56273.fixed Thoreau said, "Simplify, Simplify" * Update docs * Update docs * Update CHANGELOG.md * Update 3003.1.rst * Fix changelog Co-authored-by: Daniel Wozniak <dwozniak@saltstack.com> Co-authored-by: Pedro Algarvio <pedro@algarvio.me> Co-authored-by: Bryce Larson <brycel@vmware.com> Co-authored-by: Pablo Suárez Hernández <psuarezhernandez@suse.com> Co-authored-by: Alexander Graul <agraul@suse.com> Co-authored-by: Frode Gundersen <fgundersen@saltstack.com> Co-authored-by: Gareth J. Greenaway <gareth@saltstack.com> Co-authored-by: Gareth J. Greenaway <gareth@wiked.org> Co-authored-by: Hoa-Long Tam <hoalong@apple.com> Co-authored-by: krionbsd <krion@freebsd.org> Co-authored-by: Elias Probst <e.probst@ssc-services.de> Co-authored-by: Daniel A. Wozniak <dwozniak@vmware.com> Co-authored-by: Frode Gundersen <frogunder@gmail.com> Co-authored-by: twangboy <slee@saltstack.com> Co-authored-by: twangboy <leesh@vmware.com> Co-authored-by: ScriptAutomate <derek@icanteven.io> Co-authored-by: Wayne Werner <waynejwerner@gmail.com>
garethgreenaway
added a commit
that referenced
this issue
Sep 23, 2021
* Merge 3002.6 bugfix changes (#59822) * Pass `CI_RUN` as an environment variable to the test run. This allows us to know if we're running the test suite under a CI environment or not and adapt/adjust if needed * Migrate `unit.setup` to PyTest * Backport ae36b15 just for test_install.py * Only skip tests on CI runs * Always store git sha in _version.py during installation * Fix PEP440 compliance. The wheel metadata version 1.2 states that the package version MUST be PEP440 compliant. This means that instead of `3002.2-511-g033c53eccb`, the salt version string should look like `3002.2+511.g033c53eccb`, a post release of `3002.2` ahead by 511 commits with the git sha `033c53eccb` * Fix and migrate `tests/unit/test_version.py` to PyTest * Skip test if `easy_install` is not available * We also need to be PEP440 compliant when there's no git history * Allow extra_filerefs as sanitized kwargs for SSH client * Fix regression on cmd.run when passing tuples as cmd Co-authored-by: Alexander Graul <agraul@suse.com> * Add unit tests to ensure cmd.run accepts tuples * Add unit test to check for extra_filerefs on SSH opts * Add changelog file * Fix comment for test case * Fix unit test to avoid failing on Windows * Skip failing test on windows * Fix test to work on Windows * Add all ssh kwargs to sanitize_kwargs method * Run pre-commit * Fix pylint * Fix cmdmod loglevel and module_names tests * Fix pre-commit * Skip ssh tests if binary does not exist * Use setup_loader for cmdmod test * Prevent argument injection in restartcheck * Add changelog for restartcheck fix * docs_3002.6 * Add back tests removed in merge Co-authored-by: Pedro Algarvio <pedro@algarvio.me> Co-authored-by: Megan Wilhite <megan.wilhite@gmail.com> Co-authored-by: Bryce Larson <brycel@vmware.com> Co-authored-by: Pablo Suárez Hernández <psuarezhernandez@suse.com> Co-authored-by: Alexander Graul <agraul@suse.com> Co-authored-by: Frode Gundersen <fgundersen@saltstack.com> * Remove glance state module in favor of glance_image * update wording in changelog * bump deprecation warning to Silicon. * Updating warnutil version to Phosphorous. * Update salt/modules/keystone.py Co-authored-by: Megan Wilhite <megan.wilhite@gmail.com> * Check $HOMEBREW_PREFIX when linking against libcrypto When loading `libcrypto`, Salt checks for a Homebrew installation of `openssl` at Homebrew's default prefix of `/usr/local`. However, on Apple Silicon Macs, Homebrew's default installation prefix is `/opt/homebrew`. On all platforms, the prefix is configurable. If Salt doesn't find one of those `libcrypto`s, it will fall back on the un-versioned `/usr/lib/libcrypto.dylib`, which will cause the following crash: Application Specific Information: /usr/lib/libcrypto.dylib abort() called Invalid dylib load. Clients should not load the unversioned libcrypto dylib as it does not have a stable ABI. This commit checks $HOMEBREW_PREFIX instead of hard-coding `/usr/local`. * Add test case * Add changelog for 59808 * Add changelog entry * Make _find_libcrypto fail on Big Sur if it can't find a library Right now, if `_find_libcrypto` can't find any externally-managed versions of libcrypto, it will fall back on the pre-Catalina un-versioned system libcrypto. This does not exist on Big Sur and it would be better to raise an exception here rather than crashing later when trying to open it. * Update _find_libcrypto tests This commit simplifies the unit tests for _find_libcrypto by mocking out the host's filesystem and testing the common libcrypto installations (brew, ports, etc.) on Big Sur. It simplifies the tests for falling back on system versions of libcrypto on previous versions of macOS. * Fix description of test_find_libcrypto_with_system_before_catalina * Patch sys.platform for test_rsax931 tests * modules/match: add missing "minion_id" in Pillar example The documented Pillar example for `match.filter_by` lacks the `minion_id` parameter. Without it, the assignment won't work as expected. - fix documentation - add tests: - to prove the misbehavior of the documented example - to prove the proper behaviour when supplying `minion_id` - to ensure some misbehaviour observed with compound matchers doesn't occur * Fix for issue #59773 - When instantiating the loader grab values of grains and pillars if they are NamedLoaderContext instances. - The loader uses a copy of opts. - Impliment deepcopy on NamedLoaderContext instances. * Add changelog for #59773 * _get_initial_pillar function returns pillar * Fix linter issues * Clean up test * Bump deprecation release for neutron * Uncomment Sulfur release name * Removing the _ext_nodes deprecation warning and alias. * Adding changelog. * Renaming changelog file. * Update 59804.removed * Initial pass at fips_mode config option * Fix pre-commit * Fix tests and add changelog * update docs 3003 * update docs 3003 - newline * Fix warts in changelog * update releasenotes 3003 * add ubuntu-2004-amd64 m2crypto pycryptodome and tcp tests * add distro_arch * changing the cloud platforms file missed in 1a9b7be * Update __utils__ calls to import utils in azure * Add changelog for 59744 * Fix azure unit tests and move to pytest * Use contextvars from site-packages for thin If a contextvars package exists one of the site-packages locations use it for the generated thin tarball. This overrides python's builtin contextvars and allows salt-ssh to work with python <=3.6 even when the master's python is >3.6 (Fixes #59942) * Add regression test for #59942 * Add changelog for #59942 * Update filemap to include test_py_versions * Fix broken thin tests * Always install the `contextvars` backport, even on Py3.7+ Without this change, salt-ssh cannot target systems with Python <= 3.6 * Use salt-factories to handle the container. Don't override default roster * Fix thin tests on windows * No need to use warn log level here * Fix getsitepackages for old virtualenv versions * Add explicit pyobjc reqs * Add back the passthrough stuff * Remove a line so pre-commit will run * Bugfix release docs * Bugfix release docs * Removing pip-compile log files * Bump requirements to address a few security issues * Address traceback on macOS ``` Traceback (most recent call last): File "setup.py", line 1448, in <module> setup(distclass=SaltDistribution) File "/Users/jenkins/setup-tests/.venv/lib/python3.7/site-packages/setuptools/__init__.py", line 153, in setup return distutils.core.setup(**attrs) File "/opt/salt/lib/python3.7/distutils/core.py", line 108, in setup _setup_distribution = dist = klass(attrs) File "setup.py", line 1068, in __init__ self.update_metadata() File "setup.py", line 1074, in update_metadata attrvalue = getattr(self, attrname, None) File "setup.py", line 1182, in _property_install_requires install_requires += _parse_requirements_file(reqfile) File "setup.py", line 270, in _parse_requirements_file platform.python_version(), _parse_op(op), _parse_ver(ver) File "setup.py", line 247, in _check_ver return getattr(operator, "__{}__".format(op))(pyver, wanted) File "/opt/salt/lib/python3.7/distutils/version.py", line 46, in __eq__ c = self._cmp(other) File "/opt/salt/lib/python3.7/distutils/version.py", line 337, in _cmp if self.version < other.version: TypeError: '<' not supported between instances of 'str' and 'int' ``` * Replace `saltstack.com` with `saltproject.io` on URLs being tested * Add back support to load old entrypoints by iterating instead of type checking Fixes #59961 * Fix issue #59975 * Fix pillar serialization for jinja #60083 * Fix test * Add changelog for #60083 * Update changelog and release for 3003.1 * Remove the changelog source refs * Add connect to IPCMessageSubscriber's async_methods Fixes #60049 by making sure an IPCMessageSubscriber that is wrapped by SyncWrapper has a connect method that runs the coroutine rather than returns a fugure. * Add changelog for #60049 * Update 60049.fixed * Fix coroutine spelling error Co-authored-by: Wayne Werner <waynejwerner@gmail.com> * IPC on windows cannot use socket paths Fixes #60298 * Update Jinja2 and lxml due to security related bugfix releases Jinja2 ------ CVE-2020-28493 moderate severity Vulnerable versions: < 2.11.3 Patched version: 2.11.3 This affects the package jinja2 from 0.0.0 and before 2.11.3. The ReDOS vulnerability of the regex is mainly due to the sub-pattern [a-zA-Z0-9.-]+.[a-zA-Z0-9.-]+ This issue can be mitigated by Markdown to format user content instead of the urlize filter, or by implementing request timeouts and limiting process memory. lxml ---- CVE-2021-28957 moderate severity Vulnerable versions: < 4.6.3 Patched version: 4.6.3 An XSS vulnerability was discovered in the python lxml clean module versions before 4.6.3. When disabling the safe_attrs_only and forms arguments, the Cleaner class does not remove the formaction attribute allowing for JS to bypass the sanitizer. A remote attacker could exploit this flaw to run arbitrary JS code on users who interact with incorrectly sanitized HTML. This issue is patched in lxml 4.6.3. * fix github actions jobs on branch until bullseye comes out * Upgrade to `six==1.16.0` to avoid problems on CI runs ``` 13:59:02 nox > Session invoke-pre-commit was successful. 13:59:02 nox > Running session invoke-pre-commit 13:59:02 nox > pip install --progress-bar=off -r requirements/static/ci/py3.7/invoke.txt 13:59:02 Collecting blessings==1.7 13:59:02 Using cached blessings-1.7-py3-none-any.whl (18 kB) 13:59:02 Collecting invoke==1.4.1 13:59:02 Using cached invoke-1.4.1-py3-none-any.whl (210 kB) 13:59:02 Collecting pyyaml==5.3.1 13:59:02 Using cached PyYAML-5.3.1.tar.gz (269 kB) 13:59:02 Collecting six==1.15.0 13:59:02 Using cached six-1.15.0-py2.py3-none-any.whl (10 kB) 13:59:02 Building wheels for collected packages: pyyaml 13:59:02 Building wheel for pyyaml (setup.py) ... -� �\� �|� �/� �-� �\� �|� �done 13:59:02 Created wheel for pyyaml: filename=PyYAML-5.3.1-cp37-cp37m-linux_x86_64.whl size=546391 sha256=e42e1d66cc32087f4d33ceb81268c86b59f1a97029b19459f91b8d6ad1430167 13:59:02 Stored in directory: /var/jenkins/.cache/pip/wheels/5e/03/1e/e1e954795d6f35dfc7b637fe2277bff021303bd9570ecea653 13:59:02 Successfully built pyyaml 13:59:02 Installing collected packages: six, pyyaml, invoke, blessings 13:59:02 Attempting uninstall: six 13:59:02 Found existing installation: six 1.16.0 13:59:02 Uninstalling six-1.16.0: 13:59:02 ERROR: Could not install packages due to an OSError: [Errno 2] No such file or directory: '/var/jenkins/.cache/pre-commit/repomw8oee1s/py_env-python3/lib/python3.7/site-packages/__pycache__/six.cpython-37.pyc' 13:59:02 13:59:02 nox > Command pip install --progress-bar=off -r requirements/static/ci/py3.7/invoke.txt failed with exit code 1 13:59:02 nox > Session invoke-pre-commit failed. ``` * add changelog for #59982 * Regression test for #56273 * Fix race condition in batch. #56273 * Add changelog for #56273 * Update salt/client/__init__.py Co-authored-by: Pedro Algarvio <pedro@algarvio.me> * Update doc for salt/client * Update changelog/56273.fixed Thoreau said, "Simplify, Simplify" * Update docs * Update docs * Update CHANGELOG.md * Update 3003.1.rst * Ignore configuration for 'enable_fqdns_grains' for AIX, Solaris and Juniper * Added changelog * Let Mac OS Mojave run for 8 hours to avoid timeout * Remove FreeBSD-12.2 * Use Popen for VT * Still allow shell True * Drop shlex split * Add crypto re-init * Fix pre-commit * Do not call close in isalive * Skip tests not valid on windows * Cleanup things that are not really needed * We do not support irix * Fix pre-commit * Remove commented out lines * Add changelog for #60504 * Fix pre-commit issues * pyupgrade does not remove six imports * Fix OSErrors in some test cases * Remove un-needed args processing * Make state_running test more reliable * Removing tmpfs from Fedora 33. * Address leaks in fileserver caused by git backends At this time we do not have the ability to fix the upstream memory leaks in the gitfs backend providers. Work around their limitations by periodically restarting the file server update proccess. This will at least partially address #50313 * Remove un-used import * Fix warts caused by black version * Add changelog * We don't need two changelogs * Also pin the ``pip`` upgrade to be ``<21.2`` * Update the external ipaddress to the latest 3.9.5 version which has some security fixes. Updating the compat.p to use the vendored version if the python version is below 3.9.5 and only run the test_ipaddress.py tests if below 3.9.5. * Adding changelog * Requested changes. * Add shh_timeout to ssh_kwargs * move to with blocks * one with block * reight crypto * add back test file * add changelog * change log file number * add m2crypt support * only check m2crpto * Delete 60571.fixed * add back log * add newline * add newline for log file * Work around pypa/pip#9450 See pypa/pip#10212 * Drop six and Py2 * [3003.2] Add server alive (#60573) * add server alive * rename log * change default alive time * add requested changes * format string * reformat string again * run pre * customize * space * remove EOF dead space * fix pre-commit * run pre Co-authored-by: Megan Wilhite <megan.wilhite@gmail.com> * Changelog for 3003.2 * Man pages update for 3003.2 * Allow CVE entries in `changelog/` * Add security type for towncrier changelog * Add security type for changelog entries pre-commit check * Pin to ``pip>=20.2.4,<21.2`` Refs pypa/pip#9450 * Drop six and Py2 * Fix bug introduced in #59648 Fixes #60046 * Add changelog * Fix doc builds * fix release notes about dropping ubuntu 16.04 * update file client * add changelog file * update changelog * Check permissions of minion config directory * Fix some wording in the messagebox and in comments * Add changelog * Fix extension for changelog * Add missing commas. It also worked, but now is better * docs_3003.3 * fixing version numbers in man pages. * removing newlines. * removing newlines. * Fixing release notes. * Fix changelog file for 3003.2 release * Fix test_state test using loader.context * Re-add test_context test * Allow Local System account, add timestamp * swaping the git-source for vsphere-automation-sdk-python * Remove destroy, handled in context manager Co-authored-by: Daniel Wozniak <dwozniak@saltstack.com> Co-authored-by: Pedro Algarvio <pedro@algarvio.me> Co-authored-by: Bryce Larson <brycel@vmware.com> Co-authored-by: Pablo Suárez Hernández <psuarezhernandez@suse.com> Co-authored-by: Alexander Graul <agraul@suse.com> Co-authored-by: Frode Gundersen <fgundersen@saltstack.com> Co-authored-by: Gareth J. Greenaway <gareth@saltstack.com> Co-authored-by: Gareth J. Greenaway <gareth@wiked.org> Co-authored-by: Hoa-Long Tam <hoalong@apple.com> Co-authored-by: krionbsd <krion@freebsd.org> Co-authored-by: Elias Probst <e.probst@ssc-services.de> Co-authored-by: Daniel A. Wozniak <dwozniak@vmware.com> Co-authored-by: Frode Gundersen <frogunder@gmail.com> Co-authored-by: twangboy <slee@saltstack.com> Co-authored-by: twangboy <leesh@vmware.com> Co-authored-by: ScriptAutomate <derek@icanteven.io> Co-authored-by: Wayne Werner <waynejwerner@gmail.com> Co-authored-by: David Murphy < dmurphy@saltstack.com> Co-authored-by: Joe Eacott <jeacott@vmware.com> Co-authored-by: cmcmarrow <charles.mcmarrow.4@gmail.com> Co-authored-by: Twangboy <shane.d.lee@gmail.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Description
Amazon Linux has updated the python3-urllib3 to 1.25 - this is causing all salt versions to fail after a yum update
Steps to Reproduce the behavior
yum update with python3-urllib3-0:1.25 - any salt version
Versions Report
all salt 300x.x versions
Additional context
work around - lock yum to version python3-urllib3-0:1.23-6.amzn2.noarch
ie:
sudo salt * pkg.install yum-versionlock
sudo salt * cmd.run 'yum versionlock add python3-urllib3-0:1.23-6.amzn2.noarch'
The text was updated successfully, but these errors were encountered: