[Python-Dev] PEP 3147, __pycache__ directories and umask

[Antoine Pitrou]

Barry Warsaw <barry <at> python.org> writes:
> 
> I'm not sure what the right answer is.  Some possible choices:
> 
> * Tough luck
> * Force the umask so that the directory is writable, but then the question is,
>   by whom?  ugo+w or something less?
> * Copy the permissions from the parent directory and ignore umask
> * Raise an exception or refuse to create __pycache__ if it's not writable
>   (again, by whom?)

Welcome to a problem PHP has known for years.

The problem with solution 3 is that a non-root user can't change the owner of a
directory, therefore copying the permissions doesn't ensure the directory will
be writable by the right users.

Solution 2 (chmod 777) is obviously too laxist. Anyone could put fake bytecode
files in a root-owned application. Less laxist chmods have the same problem as
solution 3, and can still be insecure.

I think solution 4 would be the best compromise. In the face of ambiguity,
refuse to guess. But, rather than :
    refuse to create __pycache__ if it's not writable,
the test should be :
    refuse to create __pycache__ if I can't create it with the same ownership
and permissions as the parent directory (and raise an ImportWarning).


In light of this issue, I'm -0.5 on __pycache__ becoming the default caching
mechanism. The directory ownership/permissions issue is too much of a mess,
especially for Web applications (think __pycache__ files created by the Apache
user).

__pycache__ should only be created if explicitly activated (for example by
distutils when installing stuff). Otherwise, if not present, the "legacy"
mechanism (writing an untagged pyc file along the py file) should be used.

Actually, __pycache__ creation doesn't have to be part of the import mechanism.
It can be part of distutils instead (or whatever third-party tool such as
distribute, or distro-specific packaging scripts). This would relax complexity
of core Python a bit.

Regards

Antoine.