[Python-Dev] PEP 364, Transitioning to the Py3K standard library

[Barry Warsaw]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Mar 7, 2007, at 4:36 PM, Brett Cannon wrote:

>> >> Third party package renaming is also supported, via several public
>> >> interfaces accessible by any Python module.
>> >>
>> >
>> > I guess a .pth file could install the mappings for the third-party
>> > modules.
>>
>> How would that work?  .pth files don't execute arbitrary Python code,
>
> It does through imports, though.

I think it's important to import on demand only though.  I should  
probably make that clear in the PEP <wink>.  IOW, "import email"  
should not by side-effect import all sub-modules just because there's  
a remapping involved.  This actually would be quite easy to do, but  
it's one of the reasons why the Python 2.5 email package hack goes  
through so much trouble.

>> but it seems like that would be necessary to make this work.  OTOH,
>> your suggestion does make me think that it might be possible to
>> piggyback the module name mappings onto .pth file processing.  It
>> already understand the 'import' command, so why not a 'rename'  
>> command?
>
> Not needed.  Just put your mapping code in a module in your package
> and have a .pth file import that module.

>> Think about what that big dictionary would look like.  It would
>> essentially be like a text file with mappings one per line, but you'd
>> have a lot of extra goo on each line (quotes, colons, and commas).  I
>> thought it was important to decentralize the mappings and then to put
>> them in the simplest format that would serve the purpose.
>
> Right, but I am just not a big special-purpose format guy.  =)  Plus
> mappings for third-parties can be kept to a single file for the module
> they import by keeping the dict in that file.

If you went with the dictionary approach, then you wouldn't need  
the .pth file.  I guess a question for either approach is what you do  
if the top-level module is renamed.  E.g. if we rename Carbon to  
carbon, where do we put the remapping for this?  You can't keep both  
directories because of case-insensitive file systems (this has  
already been a "fun" pep to develop on a Mac ;).  If we narrow the  
scope to only the official stdlib, then it's probably not an issue  
because we can stick it anywhere and arrange for it to get loaded.

> I guess I would want to minimize the API in case we decide to directly
> integrate this into import itself so we don't have to have both a
> meta_path and path_importer_cache object.  I think the solution works
> for PEP 3108 and the mail package but I think I would a more
> integrated solution if we went all out in terms of providing the
> support outside of the stdlib (at least eventually).

It depends on how explicit or magical you want the hooks to be.  More  
magic is more convenient to hook into, but perhaps harder to decipher.

>> If people really don't like the separate data file, and really want
>> to just use a Python dictionary, then the way to keep the
>> decentralization would be to reserve a __*__ symbol in the package
>> namespace that contained this remapping.  Something like __renames__
>> = {...}.
>
> I think the .pth file solution alleviates this need.

At the cost of immediately importing all sub-modules when the package  
is imported.  Do you agree that lazy loading is a requirement we  
should keep?

>> > If you are going to have the object accessible from outside
>> > sys.path_importer_cache I would also add an attribute that contains
>> > the magic string used on sys.path.
>>
>> Of course, that magic string is also available via oldlib._magic.  I
>> tend to think it won't be necessary, but if people really want it, I
>> can see putting the magic string as an attribute on the
>> OldStdlibLoader object, e.g. sys.stdlib_remapper.importer_cache_key.
>
> Yeah, that was what I was thinking of.  When you manipulate sys.path
> you will need to be aware of it and what it is named.

Good point.  Yes, I like putting the magic key on the remapper  
object.  That does give us a bit of a chicken and egg though because  
until you know the magic string, you can't know the  
path_importer_cache key, so I think putting the remapper in sys is  
useful.

- -Barry

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.5 (Darwin)

iQCVAwUBRe9OFHEjvBPtnXfVAQIVhwP+Oloyxi/ldlbQ7RM2ifmgN03jc/SYN5Ht
suxsXiK3xwiCU5B/J6uAKDSol/WtxKV/EI2yN42xJIUFV0B0C8DCLHDZHLDAI67V
dfsJMAV+wgn8RjPVSuEw9vOFaVaoyIVvdDn+XMKKfKX6YOBnjLpnLaZJtEVx2QMg
ZigImiWp3VI=
=EHOG
-----END PGP SIGNATURE-----