[Python-Dev] PEP 3147, __pycache__ directories and umask

[Barry Warsaw]

On Mar 22, 2010, at 02:30 PM, Antoine Pitrou wrote:

>Barry Warsaw <barry <at> python.org> writes:
>> 
>> * Tough luck
>> * Force the umask so that the directory is writable, but then the question is,
>>   by whom?  ugo+w or something less?
>> * Copy the permissions from the parent directory and ignore umask
>> * Raise an exception or refuse to create __pycache__ if it's not writable
>>   (again, by whom?)
>
>Welcome to a problem PHP has known for years.

Yay.

>The problem with solution 3 is that a non-root user can't change the owner of
>a directory, therefore copying the permissions doesn't ensure the directory
>will be writable by the right users.
>
>Solution 2 (chmod 777) is obviously too laxist. Anyone could put fake bytecode
>files in a root-owned application. Less laxist chmods have the same problem as
>solution 3, and can still be insecure.
>
>I think solution 4 would be the best compromise. In the face of ambiguity,
>refuse to guess. But, rather than :
>    refuse to create __pycache__ if it's not writable,
>the test should be :
>    refuse to create __pycache__ if I can't create it with the same ownership
>and permissions as the parent directory (and raise an ImportWarning).

I actually think "tough luck" might be the right answer and that it's not
really all that serious.  Certainly not serious enough not to enable this by
default.

When Python is being installed, either by a from-source 'make install' or by
the distro packager, then you'd expect the umask not to be insane.  In the
latter case, it's a bug and in the former case you screwed up so you should be
able to delete and reinstall, or at the very least execute the same `find`
command that Python's own Makefile calls (in my branch) for 'make clean'.

When you're installing packages, again, I would expect that the system
installer, or you via `easy_install` or whatever, would not have an insane
umask.  As with above, if you *did* have a bad umask, you could certainly also
have a bad umask whenever you ran the third party tool as when you ran
Python.

You could also have a bad umask when you're doing local development, but that
should be easy enough to fix, and besides, I think you'd have other problems
if that were the case.

One possibility would be to instrument compileall to remove __pycache__
directories with a flag.

>In light of this issue, I'm -0.5 on __pycache__ becoming the default caching
>mechanism. The directory ownership/permissions issue is too much of a mess,
>especially for Web applications (think __pycache__ files created by the Apache
>user).
>
>__pycache__ should only be created if explicitly activated (for example by
>distutils when installing stuff). Otherwise, if not present, the "legacy"
>mechanism (writing an untagged pyc file along the py file) should be used.

I don't particularly like this much, because most people won't get the benefit
of it in a local dev environment.  While not the primary use case, it's a
useful enough side benefit, and unless Guido changes his mind, we want to
enable this by default.

>Actually, __pycache__ creation doesn't have to be part of the import
>mechanism.  It can be part of distutils instead (or whatever third-party tool
>such as distribute, or distro-specific packaging scripts). This would relax
>complexity of core Python a bit.

It's not really that complicated. :)

-Barry
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 836 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/python-dev/attachments/20100322/739eff36/attachment.pgp>