[Python-Dev] PEP 364, Transitioning to the Py3K standard library

[Brett Cannon]

On 3/6/07, Barry Warsaw <barry at python.org> wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> >> Supported Renamings
> >> ===================
> >>
> >> There are at least 4 use cases explicitly supported by this PEP:
> >>
> >> - - Simple top-level package name renamings, such as ``StringIO`` to
> >>    ``stringio``;
> >>
> >> - - Sub-package renamings where the package name may or may not be
> >>    renamed, such as ``email.MIMEText`` to ``email.mime.text``;
> >>
> >> - - Extension module renaming, such as ``cStringIO`` to
> >> ``cstringio``;
> >>
> >
> > This feels a little misleading.  The renaming is not restricted to
> > extension modules but also works with any top-level module.
>
> Possibly.  It's just a use case that illustrates the solution isn't
> tied to pure-Python modules.  But I'm not wedded to making this
> distinction in the PEP.
>
> >> Third party package renaming is also supported, via several public
> >> interfaces accessible by any Python module.
> >>
> >
> > I guess a .pth file could install the mappings for the third-party
> > modules.
>
> How would that work?  .pth files don't execute arbitrary Python code,

It does through imports, though.

> but it seems like that would be necessary to make this work.  OTOH,
> your suggestion does make me think that it might be possible to
> piggyback the module name mappings onto .pth file processing.  It
> already understand the 'import' command, so why not a 'rename' command?
>

Not needed.  Just put your mapping code in a module in your package
and have a .pth file import that module.

> >> ``.mv`` files can appear anywhere in the file system, and there is a
> >> programmatic interface provided to parse them, and register the
> >> remappings inside them.  By default, when Python starts up, all the
> >> ``.mv`` files in the ``oldlib`` package are read, and their
> >> remappings
> >> are automatically registered.  This is where all the module
> >> remappings
> >> should be specified for top-level Python 2.x standard library
> >> modules.
> >>
> >
> > I personally prefer to just have this info in a dict with a specified
> > format for the dict.  I think that should at least be an Open Issue or
> > address why a new file format was chosen over that solution (don't
> > remember what you said at PyCon  =).
>
> Think about what that big dictionary would look like.  It would
> essentially be like a text file with mappings one per line, but you'd
> have a lot of extra goo on each line (quotes, colons, and commas).  I
> thought it was important to decentralize the mappings and then to put
> them in the simplest format that would serve the purpose.
>

Right, but I am just not a big special-purpose format guy.  =)  Plus
mappings for third-parties can be kept to a single file for the module
they import by keeping the dict in that file.

> > Probably should mention that sys.path_importer_cache is then set for
> > the magic string.
>
> Yep.
>
> > There is no mention of what the sys.meta_path importer/loader is
> > used for.
>
> Good point.
>
> >> Programmatic Interface
> >> ======================
> >>
> >> Several methods are added to the ``sys.stdlib_remapper`` object,
> >> which
> >> third party packages can use to register their own remappings.
> >
> > Is this really necessary?  It might be helpful if this use of mapping
> > old to new names gets picked up and used heavily, but otherwise this
> > might be overkill.  You can easily get the object from
> > sys.path_importer_cache as long as you know the magic string used on
> > sys.path.
>
> So, I was thinking that third party packages could use the same
> mechanism to provide their own remappings to PEP 8 names.  Certainly
> if that's something that third party packages want to do, it would be
> better to use some common Python API to do that rather than have them
> bake their own (or worse, avoid PEP 8-ifying their module names
> because of backward compatibility concerns).
>
> If we agree that's a use case we should support, then either we
> support this programmatically or we don't expose the API and do
> something automatic when the package is imported.  I'm of two
> different minds on this.  Probably the way I'd rather see this work
> is for a package to include a package.mv file in its package
> directory, and that file would get automatically loaded when the
> package was first imported.
>

I guess I would want to minimize the API in case we decide to directly
integrate this into import itself so we don't have to have both a
meta_path and path_importer_cache object.  I think the solution works
for PEP 3108 and the mail package but I think I would a more
integrated solution if we went all out in terms of providing the
support outside of the stdlib (at least eventually).

> If people really don't like the separate data file, and really want
> to just use a Python dictionary, then the way to keep the
> decentralization would be to reserve a __*__ symbol in the package
> namespace that contained this remapping.  Something like __renames__
> = {...}.
>

I think the .pth file solution alleviates this need.

> > If you are going to have the object accessible from outside
> > sys.path_importer_cache I would also add an attribute that contains
> > the magic string used on sys.path.
>
> Of course, that magic string is also available via oldlib._magic.  I
> tend to think it won't be necessary, but if people really want it, I
> can see putting the magic string as an attribute on the
> OldStdlibLoader object, e.g. sys.stdlib_remapper.importer_cache_key.
>

Yeah, that was what I was thinking of.  When you manipulate sys.path
you will need to be aware of it and what it is named.

-Brett