Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add whitelist and blacklist support. #16

Open
wants to merge 3 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
59 changes: 59 additions & 0 deletions README.rst
Original file line number Diff line number Diff line change
Expand Up @@ -8,3 +8,62 @@ wheel extension::
[buildout]
extensions = buildout.wheel
...

Wheel locations and names can be blacklisted and/or whitelisted.
If a wheel is allowed by these filters, then the wheel will be added to the list
of potential packages considered for installation.

Filtering can be achieved using the following options::

[buildout]
extensions = buildout.wheel
wheel-blacklist-locations =
wheel-whitelist-locations =
wheel-blacklist-names =
wheel-whitelist-names =

Each option accepts a list (new-line, or comma-separated) of regular expressions
which will be matched against the location or full wheel name (including version string).

To prevent any wheels being installed from locations whose url contains ``pypi.python.org``::

[buildout]
extensions = buildout.wheel
wheel-blacklist-locations = pypi.python.org

To limit which wheels will be allowed (packages starting ``foo``,
all packages starting ``bar-1.2.3``, and packages containing ``baz-0.1.1``)::

[buildout]
extensions = buildout.wheel
wheel-whitelist-names =
^foo
^bar-1\.2\.3
baz-0\.1\.1


Whitelists take precedence over blacklists, and names take precedence over locations.

This will allow wheels from any location whose url contains ``pypi``::

[buildout]
extensions = buildout.wheel
wheel-blacklist-locations = pypi.python.org
wheel-whitelist-locations = pypi


No wheels will be allowed from pypi, except for those containing ``foo`` or ``bar``::

[buildout]
extensions = buildout.wheel
wheel-blacklist-locations = pypi.python.org
wheel-whitelist-names = foo,bar


Wheels will be allowed from pypi, but no wheels containing ``foo`` or ``bar``
will be allowed from any location::

[buildout]
extensions = buildout.wheel
wheel-whitelist-locations = pypi.python.org
wheel-blacklist-names = foo,bar
40 changes: 33 additions & 7 deletions src/buildout/wheel/__init__.py
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
import logging
import os
import os.path
import re
import shutil
import sys
import pkg_resources
Expand All @@ -20,6 +21,11 @@

orig_distros_for_location = setuptools.package_index.distros_for_location

filters = {'blacklist_names': [],
'whitelist_names': [],
'blacklist_locations': [],
'whitelist_locations': []}


def unpack_wheel(spec, dest):
WheelInstaller(spec).install_into(dest)
Expand Down Expand Up @@ -126,17 +132,37 @@ def distros_for_location(location, basename, metadata=None):
Here we override setuptools to give wheels a chance.
"""
if basename.endswith('.whl'):
wi = WheelInstaller(basename)
if wi.compatible:
# It's a match. Treat it as a binary
# distro. Buildout will sort it out.
return [wi.distribution(location, metadata)]
# Not a match, short circuit:
return ()
# default allow
allow_install = True
wl_locs = [location for x in filters['whitelist_locations'] if x and re.search(x, location)]
bl_locs = [location for x in filters['blacklist_locations'] if x and re.search(x, location)]
wl_names = [basename for x in filters['whitelist_names'] if x and re.search(x, basename)]
bl_names = [basename for x in filters['blacklist_names'] if x and re.search(x, basename)]
# If whitelists are defined, or loc/name is blacklisted, deny
if filters['whitelist_locations'] or filters['whitelist_names'] or bl_locs or bl_names:
allow_install = False
# if loc/name is in a whitelist, allow
if (wl_locs and not bl_names) or wl_names:
allow_install = True
if allow_install:
wi = WheelInstaller(basename)
if wi.compatible:
# It's a match. Treat it as a binary
# distro. Buildout will sort it out.
return [wi.distribution(location, metadata)]
# Not a match, short circuit:
return ()
return orig_distros_for_location(location, basename, metadata=metadata)


def load(buildout):
config = buildout.get('buildout', None)
if config:
# Options may be comma or white-space separated
filters['whitelist_names'] = config.get('wheel-whitelist-names', '').replace(',', ' ').split()
filters['blacklist_names'] = config.get('wheel-blacklist-names', '').replace(',', ' ').split()
filters['whitelist_locations'] = config.get('wheel-whitelist-locations', '').replace(',', ' ').split()
filters['blacklist_locations'] = config.get('wheel-blacklist-locations', '').replace(',', ' ').split()
setuptools.package_index.distros_for_location = distros_for_location
buildout.old_unpack_wheel = zc.buildout.easy_install.UNPACKERS.get('.whl')
zc.buildout.easy_install.UNPACKERS['.whl'] = unpack_wheel
Expand Down
4 changes: 3 additions & 1 deletion src/buildout/wheel/tests/testwheel.py
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,9 @@
class Buildout(object):
""" Object to pass into the `load()` entry point during tests
"""

@staticmethod
def get(value, default):
return None

class BuildoutWheelTests(unittest.TestCase):

Expand Down