PEP 3147 - new .pyc format

[Steven D'Aprano]

On Tue, 02 Feb 2010 09:38:07 +0100, Daniel Fetchinson wrote:

>> I like seeing them in the same place as the source file, because when I
>> start developing a module, I often end up renaming it multiple times
>> before it settles on a final name. When I rename or move it, I delete
>> the .pyc file, and that ensures that if I miss changing an import, and
>> try to import the old name, it will fail.
>>
>> By hiding the .pyc file elsewhere, it is easy to miss deleting one, and
>> then the import won't fail, it will succeed, but use the old, obsolete
>> byte code.
> 
> 
> Okay, I see your point but I think your argument about importing shows
> that python is doing something suboptimal because I have to worry about
> .pyc files. Ideally, I only would need to worry about python source
> files.

That's no different from any language that is compiled: you have to worry 
about keeping the compiled code (byte code or machine language) in sync 
with the source code.

Python does most of that for you: it automatically recompiles the source 
whenever the source code's last modified date stamp is newer than that of 
the byte code. So to a first approximation you can forget all about 
the .pyc files and just care about the source.

But that's only a first approximation. You might care about the .pyc 
files if:

(1) you want to distribute your application in a non-human readable 
format;

(2) if you care about clutter in your file system;

(3) if you suspect a bug in the compiler;

(4) if you are working with byte-code hacks;

(5) if the clock on your PC is wonky;

(6) if you leave random .pyc files floating around earlier in the 
PYTHONPATH than your source files;

etc.




> There is now a chance to 'fix' (quotation marks because maybe
> there is nothing to fix, according to some) this issue and make all pyc
> files go away and having python magically doing the right thing.

Famous last words... 

The only ways I can see to have Python magically do the right thing in 
all cases would be:

(1) Forget about byte-code compiling, and just treat Python as a purely 
interpreted language. If you think Python is slow now... 

(2) Compile as we do now, but only keep the byte code in memory. This 
would avoid all worries about scattered .pyc files, but would slow Python 
down significantly *and* reduce functionality (e.g. losing the ability to 
distribute non-source files).

Neither of these are seriously an option.


> A
> central pyc repository would be something I was thinking about, but I
> admit it's a half baked or not even that, probably quarter baked idea.

A central .pyc repository doesn't eliminate the issues developers may 
have with byte code files, it just puts them somewhere else, out of 
sight, where they are more likely to bite.



-- 
Steven