Prefer using the module loader to get source in builder.

[gpshead]

Steps

• For new features or bug fixes, add a ChangeLog entry describing what your PR does.
• Write a good description on what the PR does.

Description

Prefer the module loader get_source() method in AstroidBuilder's module_build() when possible to avoid assumptions about source code being available on a filesystem. Otherwise the source cannot be found and application behavior changes when running within an embedded hermetic interpreter environment (pyoxidizer, etc.).

Without this change, pylint packaged up as such would fail to infer that functions like bz2.compress returned
something other than None, triggering assignment-from-no-return when it clearly should not.

At the pylint level, this works as an application integration regression test - when you run the test build within such a no-filesystem environment.

    def test_safe_infer_on_bz2_compress_return(self):
        src_node = astroid.extract_node('import bz2\nunused = bz2.compress(b"")')
        assignment = next(src_node.nodes_of_class(astroid.Assign))
        inferred = pylint.checkers.utils.safe_infer(assignment.value.func)
        self.assertIsInstance(inferred, astroid.FunctionDef, msg=str(inferred))
        try:
            next(inferred.nodes_of_class(astroid.Return))
        except StopIteration:
            self.fail(f'No return found in body of:\n{str(inferred)}.')

I did not try to come up with a smaller targeted astroid unittest faking the "The path in .__file__ does not exist" environment. That would probably be useful.

Type of Changes

	Type
✓	🐛 Bug fix

Related Issue

[Pierre-Sassoulas]

If I understand well the testing is easier on pylint ? Is there a simple way to also test that on astroid ?

[Pierre-Sassoulas]

Hey @gpshead, I'd like to reproduce a fail in the test I just added in pylint here, do you have an idea on how to do that ?

embedded hermetic interpreter environment (pyoxidizer, etc.).

Could we create such a testing environment in github actions ?

[gpshead]

Try this:

1. Create a venv.
2. create a littletest.py module with a function who's return value astroid can correctly infer given the source code.
3. use the py_compile module on that to produce a .pyc file for the module...
4. Manually install that .pyc file in sys.path somewhere ${your_venv}/lib/pythonX.Y/site-packages/littletest.pyc works, but using your own tempdir you temporarily add to sys.path for this test is likely a better idea.
5. import littletest in your test code, assert that it's littletest.__loader__.get_source("littletest") returns None (as it should be a SourcelessFileLoader).
6. monkey patch littletest.__loader__.get_source("littletest") (mock.patch, etc) to return the actual not-in-sys.path str source code to the module.
7. use test code similar to that PR or what is listed above that imports and uses a function from littletest. Without the fix, presumably it fails as described? With the fix it gets the source and does correct inference?

I didn't try this, but I believe it'd work and be self contained and not have additional dependencies.

[Pierre-Sassoulas]

Merging without tests as pylint's test suite is passing and the actual testing will be done in pylint-dev/pylint#5341

Congratulation on becoming an astroid contributor :)

[gpshead]

Thanks! I had stalled on responding while pondering how to construct a meaningful regression test case without being overcomplicated! :)

[Pierre-Sassoulas]

You're welcome to contribute to pylint-dev/pylint#5341 if you have new ideas :)