[Python-Dev] Parallel pyc construction

[Brett Cannon]

[Guido van Rossum]

> > On a 384 processor job we have once again encountered that old question of
> > corrupted .pyc files, sometimes resulting in an error, sometimes in a silent
> > wrong behavior later. I know this was allegedly fixed previously but it
> > looks like it doesn't really work. We lost a couple of weeks work this time.
> >
> > Didn't we talk about an option to not make pyc files? I can't seem to find
> > it. (We totally don't care about the cost of imports. The documentation
> > mentions "ihooks" but not the module itself. I know that importing has been
> > an area of create turmoil so I don't really know where to look.) I couldn't
> > even find the list of command-line options for Python itself except a
> > mention of -c in the tutorial. Any pointers would be appreciated.
>
> I don't think we have such an option, but it's a good idea.  If you
> submit a patch, we'll add it.
>

What about PEP 301 and an import hook?  Couldn't a custom import hook be
written up that didn't output a .py file?  I would think it could be as
simple as finding the file, opening it, and then compiling it as a module
and inserting it directly into ``sys.modules``.  Wouldn't that circumvent
any .py(c|o)  writing?

Of course this assumes Paul is using 2.3, but even if he isn't couldn't a
solution be used like that, helping to prevent needing to write a patch
for  Python (unless you want this in the 2.2 branch)?

-Brett