PEP 3147 - new .pyc format

[Steven D'Aprano]

On Mon, 01 Feb 2010 11:14:42 +0100, Daniel Fetchinson wrote:

> I also think the PEP is a great idea and proposes a solution to a real
> problem. But I also hear the 'directory clutter' argument and I'm really
> concerned too, having all these extra directories around (and quite a
> large number of them indeed!). 

Keep in mind that if you don't explicitly ask for the proposed feature, 
you won't see any change at all. You need to run Python with the -R 
switch, or set an environment variable. The average developer won't see 
any clutter at all unless she is explicitly supporting multiple versions.



> How about this scheme:
> 
> 1. install python source files to a shared (among python installations)
> location /this/is/shared
> 2. when python X.Y imports a source file from /this/is/shared it will
> create pyc files in its private area /usr/lib/pythonX.Y/site-packages/

$ touch /usr/lib/python2.5/site-packages/STEVEN
touch: cannot touch `/usr/lib/python2.5/site-packages/STEVEN': Permission 
denied

There's your first problem: most users don't have write-access to the 
private area. When you install a package, you normally do so as root, and 
it all works. When you import a module and it gets compiled as a .pyc 
file, you're generally running as a regular user.


> Time comparison would be between /this/is/shared/x.py and 
> /usr/lib/pythonX.Y/site-packages/x.pyc, for instance.

I don't quite understand what you mean by "time comparison".


[...]
> In /usr/lib/pythonX.Y/site-packages there would be only pyc files with
> magic number matching python X.Y.

Personally, I think it is a terribly idea to keep the source file and 
byte code file in such radically different places. They should be kept 
together. What you call "clutter" I call having the files that belong 
together kept together.



> So, basically nothing would change only the location of py and pyc files
> would be different from current behavior, but the same algorithm would
> be run to determine which one to load, when to create a pyc file, when
> to ignore the old one, etc.

What happens when there is a .pyc file in the same location as the .py 
file? Because it *will* happen. Does it get ignored, or does it take 
precedence over the site specific file?

Given:

    ./module.pyc
    /usr/lib/pythonX.Y/site-packages/module.pyc

and you execute "import module", which gets used? Note that in this 
situation, there may or may not be a module.py file.


> What would be wrong with this setup?

Consider:

    ./module.py
    ./package/module.py

Under your suggestion, both of these will compile to

    /usr/lib/pythonX.Y/site-packages/module.pyc



-- 
Steven