[Import-SIG] Rough PEP: A ModuleSpec Type for the Import System

[Eric Snow]

On Fri, Aug 9, 2013 at 2:28 AM, Antoine Pitrou <solipsis at pitrou.net> wrote:

> Le Fri, 9 Aug 2013 00:34:34 -0600,
> Eric Snow <ericsnowcurrently at gmail.com> a écrit :
> > Attributes:
> >
> > * ``name`` - the module's name (compare to ``__name__``).
> > * ``loader`` - the loader to use during loading and for module data
> >   (compare to ``__loader__``).
>
> Should it be the loader or just a factory to build it?
> I'm wondering if in some cases creating a loader is costly.
>

The finder is currently responsible for creating the loader and this PEP
does not propose changing that.  So any such loader already has to deal
with this.  I suppose some loader could be expensive to create, but none of
the existing loaders in the stdlib are that costly.  If some future loader
runs into this problem they can pretty easily write the loader in such a
way that it defers the costly operations.  I'll make a note in the PEP
about this.


> > * ``package`` - the name of the module's parent (compare to
> >   ``__package__``).
>
> Is it None if there is no parent?
>

As Brett noted, it is ''.  This is the same as the __package__ attribute of
modules.  The goal is to keep the same behavior, as much as possible, for
all the feature that are moved into ModuleSpec.  I'll make this objective
more clear in the PEP.


>
> > * ``is_package`` - whether or not the module is a package.
> > * ``origin`` - the location from which the module originates.
> > * ``filename`` - like origin, but limited to a path-based location
> >   (compare to ``__file__``).
>
> Can you explain the difference between origin and filename (or, better,
> give an example)?
>

Yeah, that wasn't too clear, was it?  filename maps directly to the
module's __file__ attribute, which is not set for all modules.  For
instance, built-in modules do not set it nor do namespace packages.  In
those cases it is still nice to be able to indicate where the module came
from.  For built-in modules origin will be set to 'built-in' and for
namespace packages 'namespace'.  For any module with a filename, origin is
set to the filename.

Having both origin and filename is meant to provide for different usage.
 filename is used to populate a module's __file__ attribute.  If set, it
indicates a path-based module (along with cached and path).  In contrast,
origin has a broader meaning and is used by the module_repr() method.

I suppose there could be a flag to indicate the module is path-based, but I
went with a separate spec attribute.  Likewise, I toyed with the idea of a
path-based subclass, perhaps PathModuleSpec, but wanted to stick with a
one-size-fits-all spec class since it is meant to be used almost
exclusively for state rather than functionality.  In some ways it's like
types.SimpleNamespace, but with a couple of import-related methods and some
dedicated state.

I'll make sure the PEP reflects this.


> > * ``load(module=None, *, is_reload=False)`` - calls the loader's
> >   ``exec_module()``, falling back to ``load_module()`` if necessary.
> >   This method performs the former responsibilities of loaders for
> >   managing modules before actually loading and for cleaning up.  The
> >   reload case is facilitated by the ``module`` and ``is_reload``
> >   parameters.
>
> So how about separate load() and reload() methods?
>

I thought about that too, but found it simpler to keep them together.
 Also, reload is a pretty specialized activity and I plan on leaving some
of the boilerplate of it to importlib.reload().  However, I'm not convinced
either way actually.  I'll think about that some more and update the PEP
regardless.  Do you have a case to make for making them separate?


>
> > However, ``ModuleSpec.is_package`` (an attribute) conflicts with
> > ``InspectLoader.is_package()`` (a method).  Working around this
> > requires a more complicated solution but is not a large obstacle.
>
> Or how about keeping the method API?
>

Because it is a static piece of data.  At the point that we can remove the
backward compatibility support, we would be stuck with a method when it
should be just a normal attribute.


>
> > Module Objects
> > --------------
> >
> > Module objects will now have a ``__spec__`` attribute to which the
> > module's spec will be bound.
>
> Nice!
>

Ironic that this PEP adds yet another import-related attribute to modules.
:)  Hopefully it's the last one.


>
> > Loaders will have a new method, ``exec_module(module)``.  Its only job
> > is to "exec" the module and consequently populate the module's
> > namespace.  It is not responsible for creating or preparing the module
> > object, nor for any cleanup afterward.  It has no return value.
>
> Does it work with extension modules as well? Generally, extension
> modules are populated when created (i.e. the two steps aren't separate
> at the C API level, IIRC).
>

Yeah, it works great.  We simply don't implement exec_module() on
ExtensionFileLoader and things just stay the same.  There is room to add an
exec_module() and update the C-API for extension modules to support it, but
I'll leaving that out of the PEP.  However, I will mention that in the PEP
because your question is quite relevant and not well answered there.

-eric


> Regards
>
> Antoine.
>
>
> _______________________________________________
> Import-SIG mailing list
> Import-SIG at python.org
> http://mail.python.org/mailman/listinfo/import-sig
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/import-sig/attachments/20130809/467e76b8/attachment-0001.html>