[New-bugs-announce] [issue42839] SourceFileLoader does not (fully) accept path-like objects

[favonia]

New submission from favonia <favonia at gmail.com>:

If one uses the loader created by importlib.machinery.SourceFileLoader(module_name, path) in a meta path finder, where path is not a str, then the following error would happen at the moment the module is being imported. Note that, the error would not occur if the corresponding bytecode cache (__pycache__/*.pyc) is present.

  File "/usr/lib/python3.9/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1030, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1007, in _find_and_load
  File "<frozen importlib._bootstrap>", line 972, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 228, in _call_with_frames_removed
  File "<frozen importlib._bootstrap>", line 1030, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1007, in _find_and_load
  File "<frozen importlib._bootstrap>", line 986, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 680, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 786, in exec_module
  File "<frozen importlib._bootstrap_external>", line 932, in get_code
  File "<frozen importlib._bootstrap_external>", line 604, in _code_to_timestamp_pyc
ValueError: unmarshallable object

Here is an example that could trigger the error. The package unipath is not very important except that unipath.Path is not marshallable. (Any library with a fancy, unmarshallable path-like object would work.) The choice of 'local' and the use of 'os.getcwd()' are also irrelevant. The actual production code is quite complicated, using different path-like objects and convoluted logic. I made up this example from scratch.

import os
import sys
import importlib
import importlib.abc
import importlib.util
from unipath import Path # just an example

class DummyFinder(importlib.abc.MetaPathFinder):
    def __init__(self):
        self.cwd = os.getcwd()
    def find_spec(self, fullname, path, target=None):
        if fullname == 'local':
            initpath = os.path.join(self.cwd, '__init__.py')
            if os.path.isfile(initpath):
                loader = importlib.machinery.SourceFileLoader(fullname, Path(initpath)) # some non-str Path-like object here
                return importlib.util.spec_from_file_location(fullname, initpath,
                        loader = loader, submodule_search_locations=[self.cwd])
        else:
            return None

sys.meta_path.append(DummyFinder())
importlib.import_module("local.test2")

If this is indeed a bug, there might be other classes and functions in importlib that share the same problem.

----------
components: Library (Lib)
messages: 384504
nosy: favonia
priority: normal
severity: normal
status: open
title: SourceFileLoader does not (fully) accept path-like objects
type: enhancement
versions: Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue42839>
_______________________________________