gh-104212: Explain how to port imp code to importlib

[vstinner]

Explain in What's New in Python 3.12 how to port existing code using the removed imp to the importlib module.

• Issue: Improve the docs regarding the migration from imp to importlib  #104212

---

📚 Documentation preview 📚: https://cpython-previews--105905.org.readthedocs.build/

[vstinner]

Replacing load_dynamic() is non-trivial:

def load_dynamic(name, filename):
    loader = importlib.machinery.ExtensionFileLoader(name, filename)
    spec = importlib.util.spec_from_loader(name, loader)
    module = importlib.util.module_from_spec(spec)
    sys.modules[module.__name__] = module
    loader.exec_module(module)
    return module

If you omit loader.exec_module(), you get an uninitialized extension which lead to funny crashes :-)

[vstinner]

Replace init_builtin():

def init_builtin(name):
    spec = importlib.machinery.BuiltinImporter.find_spec(name)
    if spec is None:
        raise ImportError(f'no built-in module named {name!r}')
    return importlib.util.module_from_spec(spec)

[vstinner]

For removed imp.load_source(), I proposed adding importlib.util.load_source_path(): PR #105755.

[vstinner]

Replace imp.load_package():

import sys
import os.path
import importlib

def load_package(name, path):
    old_path = list(sys.path)
    try:
        sys.path.insert(0, os.path.dirname(path))
        return importlib.import_module(name)
    finally:
        sys.path.clear()
        sys.path.extend(old_path)

I'm not sure about this one. It uses high-level API to call indirectly PathFinder and FileFinder API. I would prefer to not expose them since I consider them as the internal API.

[vstinner]

For removed imp.load_source(), I proposed adding importlib.util.load_source_path(): PR #105755.

I suppose that it can be used to replace imp.load_cached().

Otherwise, here is a recipe to replace imp.load_cached():

import importlib.util
import sys

def load_compiled(name, filename):
    spec = importlib.util.spec_from_file_location(name, filename)
    module = importlib.util.module_from_spec(spec)
    sys.modules[module.__name__] = module
    spec.loader.exec_module(module)
    return module

Test:

import os.path

def create_pyc():
    name = 'script'
    filename = f'/tmp/{name}.py'
    with open(filename, "w") as fp:
        print("print('hello')", file=fp)

    sys.path.insert(0, os.path.dirname(filename))
    mod = importlib.import_module(name)

    del sys.path[0]
    del sys.modules[name]
    os.unlink(filename)
    mod = None

    return importlib.util.cache_from_source(filename)

print("create PYC")
pyc = create_pyc()

print(f"load PYC: {pyc}")
mod = load_compiled('script', pyc)
print(mod)
print(mod.__file__)
print(mod.__cached__)
os.unlink(pyc)

[vstinner]

My recipes always create a new module: they don't try to get a cached module from sys.modules, whereas the imp module respected sys.modules cache.

[miss-islington]

Thanks @vstinner for the PR 🌮🎉.. I'm working now to backport this PR to: 3.12.
🐍🍒⛏🤖

[bedevere-bot]

GH-105952 is a backport of this pull request to the 3.12 branch.