[Python-Dev] Should there be a way or API for retrieving from a code object a loader method and package file where the code comes from?

[Brett Cannon]

On Tue, Dec 23, 2008 at 08:00, Paul Moore <p.f.moore at gmail.com> wrote:
> 2008/12/23 Nick Coghlan <ncoghlan at gmail.com>:
>> Finding a loader given only a pseudo-filename and no module is actually
>> possible in the specific case of zipimport, but is still pretty obscure
>> at this point in time:
>>
>> 1. Scan sys.path looking for an entry that matches the start of the
>> pseudo-filename (remembering to use os.path.normpath).
>>
>> 2. Once such a path entry has been found, use PEP 302 to find the
>> associated importer object (the undocumented pkgutil.get_importer
>> function does exactly that - although, as with any undocumented feature,
>> the promises of API compatibility across major version changes aren't as
>> strong as they would be for an officially documented and supported
>> interface).
>>
>> 3. Hope that the importer is one like zipimport that allows get_data()
>> to be invoked directly on the importer object, rather than only
>> providing it on a separate loader object after the module has been
>> loaded. If it needs a real loader instead of just the importer, then
>> you're back to the original problem of needing a module or package name
>> (or globals dictionary) in addition to the pseudo filename.
>
> There were lots of proposals tossed around on python-dev at the time
> PEP 302 was being developed, which might have made all this easier.
> Most, if not all, were killed by backward compatibility requirements.
>
> I have some hopes that when Brett completes his "import in Python"
> work, that will add sufficient flexibility to allow people to
> experiment with all of this machinery, and ultimately maybe move
> forward with a more modular import mechanism.

I have actually made a good amount of progress as of late. It's a New
Years resolution to get importlib done, but I am actually aiming for
before January 1 (sans the damn compile() problem I am having).This
goal does ignore everything but a compatible __import__, though.

> But the timescales for
> Brett's changes won't be until at least Python 3.1, and it'll be a
> release or two after that before any significant change can be eased
> in in a compatible manner.

I suspect that any import work will be a Pending/DeprecationWarning
deal, so 3.3 would be the first version that could have any real
changes as the default.

> That's going to take a lot of energy on
> someone's part.

That would be me. =) After importlib is finished I have a couple of
PEPs planned plus properly documenting how the import machinery works
in the language spec. And I suspect this will lead to some discussions
about things, e.g. requirements of the format for __file__ and
__path__ in regards to when they point inside of an archive, etc.

-Brett