[Python-Dev] Importing .pyc in -O mode and vice versa

[Armin Rigo]

Hi Martin,

On Sat, Nov 04, 2006 at 04:47:37PM +0100, "Martin v. L?wis" wrote:
> Patch #1346572 proposes to also search for .pyc when OptimizeFlag
> is set, and for .pyo when it is not set. The author argues this is
> for consistency, as the zipimporter already does that.

My strong opinion on the matter is that importing a .pyc file if the .py
file is not present is wrong in the first place.  It caused many
headaches in several projects I worked on.  Additionally trying to
importing .pyo files looks like a complete semantic non-sense, but I
can't really argue from experience, as I never run python -O.

Typical example: someone in the project removes a .py file, and checks
in this change; someone else does an 'svn up', which kills the .py in
his working copy, but not the .pyc.  These stale .pyc's cause pain, e.g.
by shadowing the real module (further down sys.path), or simply by
preventing the project's developers from realizing that they forgot to
fix some imports.  We regularly had obscure problems that went away as
soon as we deleted all .pyc files around, but I cannot comment more on
that because we never really investigated.  

I know it's a discussion that comes up and dies out regularly.  My two
cents is that it would be saner to have two separate concepts: cache
files used internally by the interpreter for speed reasons only, and
bytecode files that can be shipped and imported.  This could e.g. be
done with different file extensions (then you just rename the files if
you want to ship them as bytecode without source), or with a temporary
cache directory (from where you can fish bytecode files if you want to
ship them).  Experience suggests I should not be holding my breath until
something is decided about this, though.  If I were asked to come up
with a patch I'd simply propose one that removes importing of stale .pyc
files (I'm always running a version of Python with such a patch, to
avoid the above-mentioned troubles).


A bientot,

Armin