PEP 3147 - new .pyc format

[Steven D'Aprano]

On Wed, 03 Feb 2010 11:55:57 +0100, Daniel Fetchinson wrote:

[...]
>> Python does most of that for you: it automatically recompiles the
>> source whenever the source code's last modified date stamp is newer
>> than that of the byte code. So to a first approximation you can forget
>> all about the .pyc files and just care about the source.
> 
> True, but the .pyc file is lying around and I always have to do 'ls -al
> | grep -v pyc' in my python source directory.


So alias a one-word name to that :)


[...]
> Here is an example: shared object files. If your code needs them, you
> can use them easily, you can access them easily if you want to, but they
> are not in the directory where you keep your C files. They are somewhere
> in /usr/lib for example, where they are conveniently collected, you can
> inspect them, look at them, distribute them, do basically whatever you
> want, but they are out of the way, and 99% of the time while you develop
> your code, you don't need them. In the 1% of the case you can easily get
> at them in the centralized location, /usr/lib in our example.
> 
> Of course the relationship between C source files and shared objects is
> not parallel to the relationship to python source files and the created
> pyc files, please don't nitpick on this point. The analogy is in the
> sense that your project inevitable needs for whatever reason some binary
> files which are rarely needed at hand, only the
> linker/compiler/interpreter/etc needs to know where they are. These
> files can be stored separately, but at a location where one can inspect
> them if needed (which rarely happens).

I'll try not to nit-pick :)

When an object file is in /usr/lib, you're dealing with it as a user. 
You, or likely someone else, have almost certainly compiled it in a 
different directory and then used make to drop it in place. It's now a 
library, you're a user of that library, and you don't care where the 
object file is so long as your app can find it (until you have a 
conflict, and then you do).

While you are actively developing the library, on the other hand, the 
compiler typically puts the object file in the same directory as the 
source file. (There may be an option to gcc to do otherwise, but surely 
most people don't use it often.) While the library is still being 
actively developed, the last thing you want is for the object file to be 
placed somewhere other than in your working directory. A potentially 
unstable or broken library could end up in /usr/lib and stomp all over a 
working version. Even if it doesn't, it means you have to be flipping 
backwards and forwards between two locations to get anything done.

Python development is much the same, the only(?) differences are that we 
have a lower threshold between "in production" and "in development", and 
that we typically install both the source and the binary instead of just 
the binary.

When you are *using* a library/script/module, you don't care whether 
import uses the .py file or the .pyc, and you don't care where they are, 
so long as they are in your PYTHONPATH (and there are no conflicts). But 
I would argue that while you are *developing* the module, it would more 
nuisance than help to have the .pyc file anywhere other than immediately 
next to the .py file (either in the same directory, or in a clearly named 
sub-directory).



-- 
Steven