Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[3.11] gh-98706: Sync with importlib_metadata 4.13.0. #98875

Merged
merged 4 commits into from
Nov 5, 2022
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
130 changes: 96 additions & 34 deletions Doc/library/importlib.metadata.rst
Original file line number Diff line number Diff line change
Expand Up @@ -13,29 +13,61 @@

**Source code:** :source:`Lib/importlib/metadata/__init__.py`

``importlib.metadata`` is a library that provides for access to installed
package metadata. Built in part on Python's import system, this library
``importlib_metadata`` is a library that provides access to
the metadata of an installed `Distribution Package <https://packaging.python.org/en/latest/glossary/#term-Distribution-Package>`_,
such as its entry points
or its top-level names (`Import Package <https://packaging.python.org/en/latest/glossary/#term-Import-Package>`_\s, modules, if any).
Built in part on Python's import system, this library
intends to replace similar functionality in the `entry point
API`_ and `metadata API`_ of ``pkg_resources``. Along with
:mod:`importlib.resources` (with new features backported to the
`importlib_resources`_ package), this can eliminate the need to use the older
and less efficient
:mod:`importlib.resources`,
this package can eliminate the need to use the older and less efficient
``pkg_resources`` package.

By "installed package" we generally mean a third-party package installed into
Python's ``site-packages`` directory via tools such as `pip
<https://pypi.org/project/pip/>`_. Specifically,
it means a package with either a discoverable ``dist-info`` or ``egg-info``
directory, and metadata defined by :pep:`566` or its older specifications.
By default, package metadata can live on the file system or in zip archives on
``importlib_metadata`` operates on third-party *distribution packages*
installed into Python's ``site-packages`` directory via tools such as
`pip <https://pypi.org/project/pip/>`_.
Specifically, it works with distributions with discoverable
``dist-info`` or ``egg-info`` directories,
and metadata defined by the `Core metadata specifications <https://packaging.python.org/en/latest/specifications/core-metadata/#core-metadata>`_.

.. important::

These are *not* necessarily equivalent to or correspond 1:1 with
the top-level *import package* names
that can be imported inside Python code.
One *distribution package* can contain multiple *import packages*
(and single modules),
and one top-level *import package*
may map to multiple *distribution packages*
if it is a namespace package.
You can use :ref:`package_distributions() <package-distributions>`
to get a mapping between them.

By default, distribution metadata can live on the file system
or in zip archives on
:data:`sys.path`. Through an extension mechanism, the metadata can live almost
anywhere.


.. seealso::

https://importlib-metadata.readthedocs.io/
The documentation for ``importlib_metadata``, which supplies a
backport of ``importlib.metadata``.
This includes an `API reference
<https://importlib-metadata.readthedocs.io/en/latest/api.html>`__
for this module's classes and functions,
as well as a `migration guide
<https://importlib-metadata.readthedocs.io/en/latest/migration.html>`__
for existing users of ``pkg_resources``.


Overview
========

Let's say you wanted to get the version string for a package you've installed
Let's say you wanted to get the version string for a
`Distribution Package <https://packaging.python.org/en/latest/glossary/#term-Distribution-Package>`_ you've installed
using ``pip``. We start by creating a virtual environment and installing
something into it:

Expand All @@ -54,9 +86,9 @@ You can get the version string for ``wheel`` by running the following:
>>> version('wheel') # doctest: +SKIP
'0.32.3'

You can also get the set of entry points keyed by group, such as
You can also get a collection of entry points selectable by properties of the EntryPoint (typically 'group' or 'name'), such as
``console_scripts``, ``distutils.commands`` and others. Each group contains a
sequence of :ref:`EntryPoint <entry-points>` objects.
collection of :ref:`EntryPoint <entry-points>` objects.

You can get the :ref:`metadata for a distribution <metadata>`::

Expand Down Expand Up @@ -91,7 +123,7 @@ Query all entry points::
>>> eps = entry_points() # doctest: +SKIP

The ``entry_points()`` function returns an ``EntryPoints`` object,
a sequence of all ``EntryPoint`` objects with ``names`` and ``groups``
a collection of all ``EntryPoint`` objects with ``names`` and ``groups``
attributes for convenience::

>>> sorted(eps.groups) # doctest: +SKIP
Expand Down Expand Up @@ -156,7 +188,8 @@ interface to retrieve entry points by group.
Distribution metadata
---------------------

Every distribution includes some metadata, which you can extract using the
Every `Distribution Package <https://packaging.python.org/en/latest/glossary/#term-Distribution-Package>`_ includes some metadata,
which you can extract using the
``metadata()`` function::

>>> wheel_metadata = metadata('wheel') # doctest: +SKIP
Expand All @@ -174,6 +207,13 @@ all the metadata in a JSON-compatible form per :PEP:`566`::
>>> wheel_metadata.json['requires_python']
'>=2.7, !=3.0.*, !=3.1.*, !=3.2.*, !=3.3.*'

.. note::

The actual type of the object returned by ``metadata()`` is an
implementation detail and should be accessed only through the interface
described by the
`PackageMetadata protocol <https://importlib-metadata.readthedocs.io/en/latest/api.html#importlib_metadata.PackageMetadata>`_.

.. versionchanged:: 3.10
The ``Description`` is now included in the metadata when presented
through the payload. Line continuation characters have been removed.
Expand All @@ -187,7 +227,8 @@ all the metadata in a JSON-compatible form per :PEP:`566`::
Distribution versions
---------------------

The ``version()`` function is the quickest way to get a distribution's version
The ``version()`` function is the quickest way to get a
`Distribution Package <https://packaging.python.org/en/latest/glossary/#term-Distribution-Package>`_'s version
number, as a string::

>>> version('wheel') # doctest: +SKIP
Expand All @@ -200,7 +241,8 @@ Distribution files
------------------

You can also get the full set of files contained within a distribution. The
``files()`` function takes a distribution package name and returns all of the
``files()`` function takes a `Distribution Package <https://packaging.python.org/en/latest/glossary/#term-Distribution-Package>`_ name
and returns all of the
files installed by this distribution. Each file object returned is a
``PackagePath``, a :class:`pathlib.PurePath` derived object with additional ``dist``,
``size``, and ``hash`` properties as indicated by the metadata. For example::
Expand Down Expand Up @@ -245,19 +287,24 @@ distribution is not known to have the metadata present.
Distribution requirements
-------------------------

To get the full set of requirements for a distribution, use the ``requires()``
To get the full set of requirements for a `Distribution Package <https://packaging.python.org/en/latest/glossary/#term-Distribution-Package>`_,
use the ``requires()``
function::

>>> requires('wheel') # doctest: +SKIP
["pytest (>=3.0.0) ; extra == 'test'", "pytest-cov ; extra == 'test'"]


Package distributions
---------------------
.. _package-distributions:
.. _import-distribution-package-mapping:

A convenience method to resolve the distribution or
distributions (in the case of a namespace package) for top-level
Python packages or modules::
Mapping import to distribution packages
---------------------------------------

A convenience method to resolve the `Distribution Package <https://packaging.python.org/en/latest/glossary/#term-Distribution-Package>`_
name (or names, in the case of a namespace package)
that provide each importable top-level
Python module or `Import Package <https://packaging.python.org/en/latest/glossary/#term-Import-Package>`_::

>>> packages_distributions()
{'importlib_metadata': ['importlib-metadata'], 'yaml': ['PyYAML'], 'jaraco': ['jaraco.classes', 'jaraco.functools'], ...}
Expand All @@ -271,7 +318,8 @@ Distributions

While the above API is the most common and convenient usage, you can get all
of that information from the ``Distribution`` class. A ``Distribution`` is an
abstract object that represents the metadata for a Python package. You can
abstract object that represents the metadata for
a Python `Distribution Package <https://packaging.python.org/en/latest/glossary/#term-Distribution-Package>`_. You can
get the ``Distribution`` instance::

>>> from importlib.metadata import distribution # doctest: +SKIP
Expand All @@ -291,22 +339,36 @@ instance::
>>> dist.metadata['License'] # doctest: +SKIP
'MIT'

The full set of available metadata is not described here. See :pep:`566`
for additional details.
The full set of available metadata is not described here.
See the `Core metadata specifications <https://packaging.python.org/en/latest/specifications/core-metadata/#core-metadata>`_ for additional details.


Distribution Discovery
======================

By default, this package provides built-in support for discovery of metadata
for file system and zip file `Distribution Package <https://packaging.python.org/en/latest/glossary/#term-Distribution-Package>`_\s.
This metadata finder search defaults to ``sys.path``, but varies slightly in how it interprets those values from how other import machinery does. In particular:

- ``importlib.metadata`` does not honor :class:`bytes` objects on ``sys.path``.
- ``importlib.metadata`` will incidentally honor :py:class:`pathlib.Path` objects on ``sys.path`` even though such values will be ignored for imports.


Extending the search algorithm
==============================

Because package metadata is not available through :data:`sys.path` searches, or
package loaders directly, the metadata for a package is found through import
system :ref:`finders <finders-and-loaders>`. To find a distribution package's metadata,
Because `Distribution Package <https://packaging.python.org/en/latest/glossary/#term-Distribution-Package>`_ metadata
is not available through :data:`sys.path` searches, or
package loaders directly,
the metadata for a distribution is found through import
system `finders`_. To find a distribution package's metadata,
``importlib.metadata`` queries the list of :term:`meta path finders <meta path finder>` on
:data:`sys.meta_path`.

The default ``PathFinder`` for Python includes a hook that calls into
``importlib.metadata.MetadataPathFinder`` for finding distributions
loaded from typical file-system-based paths.
By default ``importlib_metadata`` installs a finder for distribution packages
found on the file system.
This finder doesn't actually find any *distributions*,
but it can find their metadata.

The abstract class :py:class:`importlib.abc.MetaPathFinder` defines the
interface expected of finders by Python's import system.
Expand Down Expand Up @@ -335,4 +397,4 @@ a custom finder, return instances of this derived ``Distribution`` in the

.. _`entry point API`: https://setuptools.readthedocs.io/en/latest/pkg_resources.html#entry-points
.. _`metadata API`: https://setuptools.readthedocs.io/en/latest/pkg_resources.html#metadata-api
.. _`importlib_resources`: https://importlib-resources.readthedocs.io/en/latest/index.html
.. _`finders`: https://docs.python.org/3/reference/import.html#finders-and-loaders
52 changes: 38 additions & 14 deletions Lib/importlib/metadata/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -184,6 +184,10 @@ class EntryPoint(DeprecatedTuple):
following the attr, and following any extras.
"""

name: str
value: str
group: str

dist: Optional['Distribution'] = None

def __init__(self, name, value, group):
Expand Down Expand Up @@ -543,21 +547,21 @@ def locate_file(self, path):
"""

@classmethod
def from_name(cls, name):
def from_name(cls, name: str):
"""Return the Distribution for the given package name.

:param name: The name of the distribution package to search for.
:return: The Distribution instance (or subclass thereof) for the named
package, if found.
:raises PackageNotFoundError: When the named package's distribution
metadata cannot be found.
:raises ValueError: When an invalid value is supplied for name.
"""
for resolver in cls._discover_resolvers():
dists = resolver(DistributionFinder.Context(name=name))
dist = next(iter(dists), None)
if dist is not None:
return dist
else:
if not name:
raise ValueError("A distribution name is required.")
try:
return next(cls.discover(name=name))
except StopIteration:
raise PackageNotFoundError(name)

@classmethod
Expand Down Expand Up @@ -945,13 +949,26 @@ def _normalized_name(self):
normalized name from the file system path.
"""
stem = os.path.basename(str(self._path))
return self._name_from_stem(stem) or super()._normalized_name
return (
pass_none(Prepared.normalize)(self._name_from_stem(stem))
or super()._normalized_name
)

def _name_from_stem(self, stem):
name, ext = os.path.splitext(stem)
@staticmethod
def _name_from_stem(stem):
"""
>>> PathDistribution._name_from_stem('foo-3.0.egg-info')
'foo'
>>> PathDistribution._name_from_stem('CherryPy-3.0.dist-info')
'CherryPy'
>>> PathDistribution._name_from_stem('face.egg-info')
'face'
>>> PathDistribution._name_from_stem('foo.bar')
"""
filename, ext = os.path.splitext(stem)
if ext not in ('.dist-info', '.egg-info'):
return
name, sep, rest = stem.partition('-')
name, sep, rest = filename.partition('-')
return name


Expand Down Expand Up @@ -991,6 +1008,15 @@ def version(distribution_name):
return distribution(distribution_name).version


_unique = functools.partial(
unique_everseen,
key=operator.attrgetter('_normalized_name'),
)
"""
Wrapper for ``distributions`` to return unique distributions by name.
"""


def entry_points(**params) -> Union[EntryPoints, SelectableGroups]:
"""Return EntryPoint objects for all installed packages.

Expand All @@ -1008,10 +1034,8 @@ def entry_points(**params) -> Union[EntryPoints, SelectableGroups]:

:return: EntryPoints or SelectableGroups for all installed packages.
"""
norm_name = operator.attrgetter('_normalized_name')
unique = functools.partial(unique_everseen, key=norm_name)
eps = itertools.chain.from_iterable(
dist.entry_points for dist in unique(distributions())
dist.entry_points for dist in _unique(distributions())
)
return SelectableGroups.load(eps).select(**params)

Expand Down
16 changes: 16 additions & 0 deletions Lib/test/test_importlib/fixtures.py
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@
import pathlib
import tempfile
import textwrap
import functools
import contextlib

from test.support.os_helper import FS_NONASCII
Expand Down Expand Up @@ -296,3 +297,18 @@ def setUp(self):
# Add self.zip_name to the front of sys.path.
self.resources = contextlib.ExitStack()
self.addCleanup(self.resources.close)


def parameterize(*args_set):
"""Run test method with a series of parameters."""

def wrapper(func):
@functools.wraps(func)
def _inner(self):
for args in args_set:
with self.subTest(**args):
func(self, **args)

return _inner

return wrapper
Loading