[jerome: Re: real vs effective user id]
At 09:09 PM 6/11/2007 +0200, Jerome Alet wrote:
On Mon, Jun 11, 2007 at 01:57:38PM -0400, Phillip J. Eby wrote:
then setuptools tries to play with root's .python-eggs directory instead of ~effective_user/.python-eggs/, because of the way os.path.expanduser() works (line 1039 in pkg_resource.py)
In these types of situations, it's best to set the PYTHON_EGG_CACHE path explicitly, either via the environment variable or code.
This is effectively what I had to do to fix the problem for my client, however this was completely unexpected because I can't control how different people install third party modules like this one. Considering that some imports will be done before the seteuid() call and some after, setting the environment variable would have to be done twice (for example), otherwise you might end up with root owned directories into an otherwise user-owned directory, and this might introduce further permission problems on different apps run by the same unpriviledged user but requiring the same modules.
Wouldn't it be better to use the effective user id when expanding the user's home directory in setuptools or in Python ?
That wouldn't fix your problem. The egg cache location is determined exactly *once* for the life of the process, unless you explicitly create and use ResourceManagers -- and unfortunately the process of importing C code from a zipped egg uses only the default ResourceManager.
On Mon, Jun 11, 2007 at 04:00:34PM -0400, Phillip J. Eby wrote:
Wouldn't it be better to use the effective user id when expanding the user's home directory in setuptools or in Python ?
That wouldn't fix your problem. The egg cache location is determined exactly *once* for the life of the process, unless you explicitly create and use ResourceManagers -- and unfortunately the process of importing C code from a zipped egg uses only the default ResourceManager.
So you mean that if for example another (different) module installed with setuptools was imported *before* the seteuid() call, then the mysqldb module import (done *after* the seteuid() call) would succeed ? Not sure if it was clear in my previous message, but the problem is that the import statement fails, while the very same import done when the very same module was installed the 'classic' (not through setuptools) way succeeds, because a 'classic' install import doesn't try to write to an user's home directory. How do you want developers to plan for this sort of things ? Then maybe setuptools should continue to work without creating/writing any file in an user's .python-eggs directory if this is not allowed to do so because of permissions (or different effective user). Instead it fails hard. Any plan to improve this ? NB : I don't know if this is possible or not, I just ask. bye Jerome Alet
At 10:18 PM 6/11/2007 +0200, Jerome Alet wrote:
On Mon, Jun 11, 2007 at 04:00:34PM -0400, Phillip J. Eby wrote:
Wouldn't it be better to use the effective user id when expanding the user's home directory in setuptools or in Python ?
That wouldn't fix your problem. The egg cache location is determined exactly *once* for the life of the process, unless you explicitly create and use ResourceManagers -- and unfortunately the process of importing C code from a zipped egg uses only the default ResourceManager.
So you mean that if for example another (different) module installed with setuptools was imported *before* the seteuid() call, then the mysqldb module import (done *after* the seteuid() call) would succeed ?
No. I'm saying that the directory for the default ResourceManager's cache is determined exactly once, and can't be changed thereafter for the effective life of the process. Thus, you must choose a location that is readable and writable by every user id that the process will be executing as, whether real or effective.
Then maybe setuptools should continue to work without creating/writing any file in an user's .python-eggs directory if this is not allowed to do so because of permissions (or different effective user).
Instead it fails hard. Any plan to improve this ?
You have several options. First, setuptools can build RPMs which install files in the usual way. Second, you can install eggs containing C code as directories instead of files (using easy_install --always-unzip), which then avoids the need for the use of the egg cache. Third, you can select an appropriate cache directory, either with the environment variable, or programmatically.
On Mon, Jun 11, 2007 at 05:02:23PM -0400, Phillip J. Eby wrote:
So you mean that if for example another (different) module installed with setuptools was imported *before* the seteuid() call, then the mysqldb module import (done *after* the seteuid() call) would succeed ?
No. I'm saying that the directory for the default ResourceManager's cache is determined exactly once, and can't be changed thereafter for the effective life of the process. Thus, you must choose a location that is readable and writable by every user id that the process will be executing as, whether real or effective.
The problem is that the setuptools code doesn't fail because the expanduser() call fails, it fails (during my import of the mysqldb this is setuptools' code which fails) because it tries to read from or write to a protected directory. It's IMHO a design problem with setuptools, which should do its best when permissions prevent it to run as expected. For example "standard" Python .py files are only compiled onto the harddisk as .pyc if permissions permit. If the process is started by root, there's no way I can tell the users of my software to make ~root/.python-eggs writable (or even readable) by the user this app will do a seteuid call to. It's basic security stuff. (and yes there are valid reasons why this software needs to be started as root)
Then maybe setuptools should continue to work without creating/writing any file in an user's .python-eggs directory if this is not allowed to do so because of permissions (or different effective user).
Instead it fails hard. Any plan to improve this ?
You have several options. First, setuptools can build RPMs which install files in the usual way. Second, you can install eggs containing C code as directories instead of files (using easy_install --always-unzip), which then avoids the need for the use of the egg cache. Third, you can select an appropriate cache directory, either with the environment variable, or programmatically.
I think you misunderstand the problem : 1 - I don't use setuptools as part of the installation of my own software. 2 - My own software relies on third party modules which may, or may not, be installed through setuptools. I can't know if they are, or which options (in the ones you list above) will be chosen by the administrators who install these third party modules. I'm in no position to dictate what admins have to do with regard to software I didn't wrote (mysqldb for example) or that I don't even use (setuptools). 3 - My own software, which runs perfectly for hundreds of customers, failed miserably for one of them because he chose to install a database driver with setuptools, for the reasons explained previously. It could have been a different module, and happen at a different import statement in my code (the app is made of several command line tools). 4 - My code basically does : ... groupid = 25 userid = 33 os.setegid(groupid) os.seteuid(userid) ... and so can revert to being root again when needed by calling : os.seteuid(0) os.setegid(0) I know this is not entirely safe from a security point of view, but this is to prevent accidents, not crackers. Is this a bug in my code ? I don't think so, it's valid code. 5 - Why would I need to put ugly hacks (environment variable) in my own code ? As seen earlier this probably can't be done safely since the directory will be evaluated only once (I could always set it to /tmp/username/.python-eggs to mitigate the problem, but this sucks big time IMHO). 6 - Besides not having to rewrite the expanduser() method, is there a valid reason why setuptools checks with the real user's home directory instead of with the effective one ? If not, then I propose to write the patch. If yes, then which one ? Any comment ? TIA Jerome Alet
At 01:33 AM 6/12/2007 +0200, Jerome Alet wrote:
On Mon, Jun 11, 2007 at 05:02:23PM -0400, Phillip J. Eby wrote:
So you mean that if for example another (different) module installed with setuptools was imported *before* the seteuid() call, then the mysqldb module import (done *after* the seteuid() call) would succeed ?
No. I'm saying that the directory for the default ResourceManager's cache is determined exactly once, and can't be changed thereafter for the effective life of the process. Thus, you must choose a location that is readable and writable by every user id that the process will be executing as, whether real or effective.
The problem is that the setuptools code doesn't fail because the expanduser() call fails, it fails (during my import of the mysqldb this is setuptools' code which fails) because it tries to read from or write to a protected directory.
...which is why the cache location has to be readable and writable by every user the process will be executing as.
It's IMHO a design problem with setuptools, which should do its best when permissions prevent it to run as expected.
When it doesn't have permission to write to a cache directory, it's game over. Currently, there is no other way to execute a .so, .dll, .pyd, or other dynamic library from inside a zipfile, except for some GPL code that can do it on Windows (and which I can't mix with setuptools).
For example "standard" Python .py files are only compiled onto the harddisk as .pyc if permissions permit.
This isn't an analagous situation; without unpacking the .so file, the driver isn't going to be loadable.
6 - Besides not having to rewrite the expanduser() method, is there a valid reason why setuptools checks with the real user's home directory instead of with the effective one ? If not, then I propose to write the patch. If yes, then which one ?
A patch to replace expanduser would be fine; please make sure, however, that it falls back to use of expanduser in the event of an error.
Hello, On Mon, Jun 11, 2007 at 07:51:27PM -0400, Phillip J. Eby wrote:
6 - Besides not having to rewrite the expanduser() method, is there a valid reason why setuptools checks with the real user's home directory instead of with the effective one ? If not, then I propose to write the patch. If yes, then which one ?
A patch to replace expanduser would be fine; please make sure, however, that it falls back to use of expanduser in the event of an error.
Attached to this message you'll find the patch. It only uses the effective user id to check for the .python-eggs directory if the one returned by os.path.expanduser() is not writeable, so for most people the actual behavior remains, and the impact is limited to applications which change the effective user id. Here's what it does : ad-port53-2:/home/jerome/setuptools-0.6c6# python Python 2.4.4 (#2, Apr 26 2007, 00:02:45) [GCC 4.1.2 20061115 (prerelease) (Debian 4.1.1-21)] on linux2 Type "help", "copyright", "credits" or "license" for more information.
import os os.getuid(), os.geteuid() (0, 0) from pkg_resources import get_default_cache get_default_cache() '/root/.python-eggs' os.setegid(500) os.seteuid(500) get_default_cache() '/home/jerome/.python-eggs'
I understand that this patch doesn't always improve the situation : since you've said the directory is computed only once for a particular process, importing a setuptools-installed module as root before doing the seteuid call and then importing another setuptools-installed directory after the seteuid call, would probably still give the same import error as before, but at least it should help in *some* situations. And finally it can probably serve as a basis for discussion. hoping this helps anyway. bye Jerome Alet
participants (2)
-
Jerome Alet -
Phillip J. Eby