[Python-Dev] PEP 364, Transitioning to the Py3K standard library

[Barry Warsaw]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

>> Supported Renamings
>> ===================
>>
>> There are at least 4 use cases explicitly supported by this PEP:
>>
>> - - Simple top-level package name renamings, such as ``StringIO`` to
>>    ``stringio``;
>>
>> - - Sub-package renamings where the package name may or may not be
>>    renamed, such as ``email.MIMEText`` to ``email.mime.text``;
>>
>> - - Extension module renaming, such as ``cStringIO`` to  
>> ``cstringio``;
>>
>
> This feels a little misleading.  The renaming is not restricted to
> extension modules but also works with any top-level module.

Possibly.  It's just a use case that illustrates the solution isn't  
tied to pure-Python modules.  But I'm not wedded to making this  
distinction in the PEP.

>> Third party package renaming is also supported, via several public
>> interfaces accessible by any Python module.
>>
>
> I guess a .pth file could install the mappings for the third-party  
> modules.

How would that work?  .pth files don't execute arbitrary Python code,  
but it seems like that would be necessary to make this work.  OTOH,  
your suggestion does make me think that it might be possible to  
piggyback the module name mappings onto .pth file processing.  It  
already understand the 'import' command, so why not a 'rename' command?

>> ``.mv`` files can appear anywhere in the file system, and there is a
>> programmatic interface provided to parse them, and register the
>> remappings inside them.  By default, when Python starts up, all the
>> ``.mv`` files in the ``oldlib`` package are read, and their  
>> remappings
>> are automatically registered.  This is where all the module  
>> remappings
>> should be specified for top-level Python 2.x standard library  
>> modules.
>>
>
> I personally prefer to just have this info in a dict with a specified
> format for the dict.  I think that should at least be an Open Issue or
> address why a new file format was chosen over that solution (don't
> remember what you said at PyCon  =).

Think about what that big dictionary would look like.  It would  
essentially be like a text file with mappings one per line, but you'd  
have a lot of extra goo on each line (quotes, colons, and commas).  I  
thought it was important to decentralize the mappings and then to put  
them in the simplest format that would serve the purpose.

> Probably should mention that sys.path_importer_cache is then set for
> the magic string.

Yep.

> There is no mention of what the sys.meta_path importer/loader is  
> used for.

Good point.

>> Programmatic Interface
>> ======================
>>
>> Several methods are added to the ``sys.stdlib_remapper`` object,  
>> which
>> third party packages can use to register their own remappings.
>
> Is this really necessary?  It might be helpful if this use of mapping
> old to new names gets picked up and used heavily, but otherwise this
> might be overkill.  You can easily get the object from
> sys.path_importer_cache as long as you know the magic string used on
> sys.path.

So, I was thinking that third party packages could use the same  
mechanism to provide their own remappings to PEP 8 names.  Certainly  
if that's something that third party packages want to do, it would be  
better to use some common Python API to do that rather than have them  
bake their own (or worse, avoid PEP 8-ifying their module names  
because of backward compatibility concerns).

If we agree that's a use case we should support, then either we  
support this programmatically or we don't expose the API and do  
something automatic when the package is imported.  I'm of two  
different minds on this.  Probably the way I'd rather see this work  
is for a package to include a package.mv file in its package  
directory, and that file would get automatically loaded when the  
package was first imported.

If people really don't like the separate data file, and really want  
to just use a Python dictionary, then the way to keep the  
decentralization would be to reserve a __*__ symbol in the package  
namespace that contained this remapping.  Something like __renames__  
= {...}.

> If you are going to have the object accessible from outside
> sys.path_importer_cache I would also add an attribute that contains
> the magic string used on sys.path.

Of course, that magic string is also available via oldlib._magic.  I  
tend to think it won't be necessary, but if people really want it, I  
can see putting the magic string as an attribute on the  
OldStdlibLoader object, e.g. sys.stdlib_remapper.importer_cache_key.

> Thanks for writing this up, Barry!  I am just glad our idea for this
> actually worked in the end.  =)

Indeed!  I'm glad we got a chance to work on this together!
- -Barry

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.5 (Darwin)

iQCVAwUBRe4qj3EjvBPtnXfVAQJUvQP/Qe+rEoeZqn4OrnPg69aiW+m+5aF5TYLJ
f6YXrvp+QGCcVbfPcwBmjIAlSBNB8MOobWDD9g5A3IIBVM+2+m4y/xKtTtr/c7mr
bpPu6NZbbp2KaYiMVA6brbv9TQvjOP+eLft/hAEveKXZd1MhcdkhaEOX3uWS1u2X
UeCEbEDj2AA=
=VEHl
-----END PGP SIGNATURE-----