Mailman 3 June 2007 - Distutils-SIG

Re: [Distutils] requiring python 2.5
by Phillip J. Eby June 15, 2007

June 15, 2007

At 01:03 PM 6/5/2007 -0400, Ryan O'Neil wrote: >My apologies if this was already covered; I couldn't find appropriate >mention of the issue. > >What is the best way to require Python 2.5 in setuptools? In PyGEP >0.1.2 I did the following: > > setup( > # --- snip --- > install_requires = ['python>=2.5'], > # --- snip --- > ) > >This worked wonderfully on Linux using setuptools 0.6c5, but on >Windows (also with Python 2.5 and setuptools 0.6c5), it couldn't >recognize the installed version of Python and tried to download and >compile it (python) from source. Interesting; it sounds like there might be a problem with what files are being installed on Windows; Python 2.5 is supposed to include a Python.egg-info file that indicates Python 2.5 is installed. However, it's probably the case that EasyInstall should not try to install Python from source, regardless. :) >I was able to get things mostly working by getting rid of the >install_requires line and adding this code to my setup.py: > > import sys > > if sys.version_info[:2] < (2, 5): > raise Exception('PyGEP requires Python 2.5') I'd recommend using either a DistutilsError subclass or SystemExit; both will be handled more gracefully by easy_install. >However, easy_install will now download PyGEP for Python 2.4 and fail >during setup, which is less than ideal (and ends up with cheesecaking >slandering its installability ;). Am I just doing everything >completely wrong? No; at the moment the infrastructure for requiring a particular version of Python just isn't that great.

3 12

Setuptools bug with tag_svn_revision
by Ian Bicking June 13, 2007

June 13, 2007

If you try to do egg_info in a package that is in svn, but where any directory or subdirectory in the package has been added but not committed to svn, you'll get this error: Traceback (most recent call last): File "setup.py", line 25, in ? entry_points=""" File "distutils/core.py", line 149, in setup File "/usr/lib/python2.4/distutils/dist.py", line 946, in run_commands self.run_command(cmd) File "/usr/lib/python2.4/distutils/dist.py", line 965, in run_command cmd_obj.ensure_finalized() File "/usr/lib/python2.4/cmd.py", line 117, in ensure_finalized pass File "/home/ianb/env/webdev/lib/python2.4/setuptools-0.6c6-py2.4.egg/setuptools/command/egg_info.py", line 85, in finalize_options self.vtags = self.tags() File "/home/ianb/env/webdev/lib/python2.4/setuptools-0.6c6-py2.4.egg/setuptools/command/egg_info.py", line 179, in tags ): version += '-r%s' % self.get_svn_revision() File "/home/ianb/env/webdev/lib/python2.4/setuptools-0.6c6-py2.4.egg/setuptools/command/egg_info.py", line 227, in get_svn_revision localrev = max([int(m.group(1)) for m in revre.finditer(data)]) ValueError: max() arg is an empty sequence -- Ian Bicking | ianb(a)colorstudy.com | http://blog.ianbicking.org | Write code, do good | http://topp.openplans.org/careers

2 1

Conflict between egg and standard package with same namespace
by Andreas Jung June 13, 2007

June 13, 2007

In my setup I have a zopyx.slimp egg inside site-package and a package zopyx.txng3. With this setup I can not longer import files from the zopyx.txng3 package. ajung@galactica:~> ls /opt/python-2.4.3/lib/python2.4/site-packages/zopyx* /opt/python-2.4.3/lib/python2.4/site-packages/zopyx.slimp-0.2.0-py2.4.egg /opt/python-2.4.3/lib/python2.4/site-packages/zopyx: __init__.py __init__.pyc txng3 ajung@galactica:~> python2.4 Python 2.4.4 (#4, Oct 19 2006, 21:09:46) [GCC 4.1.0 (SUSE Linux)] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> import zopyx.txng3 Traceback (most recent call last): File "<stdin>", line 1, in ? ImportError: No module named txng3 >>> import zopyx.slimp >>> After removing the egg it is possible to import zopyx.txng3 again. Is this a bug or setuptools misconception? Andreas

2 1

[jerome: Re: real vs effective user id]
by Jerome Alet June 12, 2007

June 12, 2007

Sorry, I forgot to CC the list.

2 6

real vs effective user id
by Jerome Alet June 11, 2007

June 11, 2007

Hi there, I've got an application which runs as root This application calls seteuid to execute part of its code as a lower priviledged user temporarily. after the seteuid call, the MySQLdb module is imported, which was installed with setuptools 0.6c6. then setuptools tries to play with root's .python-eggs directory instead of ~effective_user/.python-eggs/, because of the way os.path.expanduser() works (line 1039 in pkg_resource.py) is this a bug in setuptools, in Python, or more probably in my own application ? shouldn't Python's os.path.expanduser() call use the effective user id instead of the real one ? thanks in advance for any hint. bye Jerome Alet

2 1

I load entry point ".so", then it only loads 60 / 63 ".so" files (3 fail?) :-(
by Mowry, Peter June 6, 2007

June 6, 2007

I am using ctypes to control a computer hardware simulation via a single shared library entry point, but having trouble getting it to fully load all the .so files on Linux :-( ctypes directly loads my entry point ".so" shared library: self.mainentrylib = ctypes.CDLL(strPathAbs, mode=ctypes.RTLD_GLOBAL) # load main entry point # I don't know why I have to use RTLD_GLOBAL, but when I do 60 / 63 work Then I call an exported function on self.mainentrylib, which causes the entry point shared library to load a bunch of model pieces (from the C++ code) (from a "libs" dir). On Windows, all 63 pieces (".dll" files) load without problem. On Linux, only 60 of the 63 pieces (".so" files) load, and 3 of them give me an error trying to load. I modified the C++ code (of the main entry point shared library) to print GetLastError(), which reported 2, which translates via strerror(2) to "No such file or directory". But this error message can't be true b/c the file does exist (and it's in the same dir as the other 60). I didn't think anything was special about these 3 shared libraries, but something must be. Anyone run into something similar or have any ideas? Any responses appreciated; thanks!

1 0

requiring python 2.5
by Ryan O'Neil June 5, 2007

June 5, 2007

My apologies if this was already covered; I couldn't find appropriate mention of the issue. What is the best way to require Python 2.5 in setuptools? In PyGEP 0.1.2 I did the following: setup( # --- snip --- install_requires = ['python>=2.5'], # --- snip --- ) This worked wonderfully on Linux using setuptools 0.6c5, but on Windows (also with Python 2.5 and setuptools 0.6c5), it couldn't recognize the installed version of Python and tried to download and compile it (python) from source. Granted this seems like the wrong thing to do. I was able to get things mostly working by getting rid of the install_requires line and adding this code to my setup.py: import sys if sys.version_info[:2] < (2, 5): raise Exception('PyGEP requires Python 2.5') However, easy_install will now download PyGEP for Python 2.4 and fail during setup, which is less than ideal (and ends up with cheesecaking slandering its installability ;). Am I just doing everything completely wrong? There is some more information about the error here: http://code.google.com/p/pygep/issues/detail?id=7 Thanks for your help and thanks for setuptools! -r

1 0

Re: [Distutils] Parsing setup()
by Phillip J. Eby June 5, 2007

June 5, 2007

At 05:07 PM 6/4/2007 -0700, Rob wrote: >Phillip J. Eby wrote: > > If you use "-e sometmpdir" as an option, you can have the .egg-info put > > in a temporary directory with a known location. You need this because > > some package layouts don't have the .egg-info in the package root, so > > the rest of your stuff won't work. > > > > To be honest, though, I think you're going to way too much trouble > > here. Just what exactly is the use case here? Couldn't you just > > easy_install the package to a temporary directory with --no-deps? Why > > do you think you need to inspect the setup.py in the first place? > > > >Sorry, I should have explained what I was attempting. > >The code is going to be used for g-pypi[1], a tool I wrote specifically for >creating ebuilds for Gentoo's portage package manager. It creates ebuilds >by using setuptools and querying The Cheese Shop. In that case, using "easy_install -mNd atempdir sourcedir" is probably your best recipe. After easy_install runs, "atempdir" will contain an egg built from "sourcedir", and you can then query it via the standard API. You don't even need to run it with system(); here's your recipe: from setuptools.command.easy_install import main as easy from pkg_resources import find_distributions easy(['-Nmxd', atempdir, sourcedir]) for dist in find_distributions(atempdir): requirements = dist.requires() # ...and whatever atempdir should be an empty directory prior to the easy-install invocation, of course. The -N means no dependencies, -m means multi-version (no .pth file), -x excludes scripts, and -d sets the target directory.

1 0

Re: [Distutils] Parsing setup()
by Rob June 5, 2007

June 5, 2007

Phillip J. Eby wrote: > If you use "-e sometmpdir" as an option, you can have the .egg-info put > in a temporary directory with a known location. You need this because > some package layouts don't have the .egg-info in the package root, so > the rest of your stuff won't work. > > To be honest, though, I think you're going to way too much trouble > here. Just what exactly is the use case here? Couldn't you just > easy_install the package to a temporary directory with --no-deps? Why > do you think you need to inspect the setup.py in the first place? > Sorry, I should have explained what I was attempting. The code is going to be used for g-pypi[1], a tool I wrote specifically for creating ebuilds for Gentoo's portage package manager. It creates ebuilds by using setuptools and querying The Cheese Shop. We still use setuptools/distutils normally, I'm just grabbing the dependency info to help create the ebuilds which still have to have all the dependency info in them (portage doesn't use easy_install). > (You might also want to consider using the > setuptools.sandbox.run_setup() function instead of os.system.) > That looks much better, I'll check that out. > In general, I would suggest that trying to work your way around > setuptools' APIs is a very bad idea. More often than not, you will miss > one of the nine jillion supported configurations and thus will end up > with a tool that only works on your system... *sometimes*. > > Case in point: your code above depends on the .egg-info being in the > setup directory, which is often not the case. Zope packages, for > example, usually have a src/ or lib/ subdirectory that will be where the > egg-info is. Other packages may have it in other places, perhaps even > more deeply nested. > > Pretty much *any* place where you are reading a file format directly or > looking at directory contents yourself, you're probably breaking > something that setuptools has encapsulated for a Very Good Reason. When > in doubt, ask. :) > I'll keep all that in mind. I was thinking of making the code more general in the future in case someone else wanted to use it for a different package manager and needed to find out all the dependencies before creating rpm's or debs etc. So far it's working good - it's better than downloading, unpacking and reading all the setup.py's by hand, which we'd been doing ;) Thanks, Phillip! [1] g-pypi: http://tools.assembla.com/g-pypi

1 0

Re: [Distutils] Parsing setup()
by Phillip J. Eby June 4, 2007

June 4, 2007

At 03:58 PM 6/4/2007 -0700, Rob wrote: >Ian Bicking wrote: > > > If you have a built egg (which you can do with python setup.py > egg_info, without actually installing > > anything) you can read the requires.txt file with pkg_resources. > >I had tried that but I was having problems with un-installed packages. > >I have this so far, which works with *nux, but I'll have to figure out >how to find the correct Python >executable in Windows (I think using different Python versions could >yield different requirements?). > > >pkg_name = "pycopia-net" >os.chdir(setup_dir) >os.system("/usr/bin/env python %s egg_info" % setup_file) If you use "-e sometmpdir" as an option, you can have the .egg-info put in a temporary directory with a known location. You need this because some package layouts don't have the .egg-info in the package root, so the rest of your stuff won't work. To be honest, though, I think you're going to way too much trouble here. Just what exactly is the use case here? Couldn't you just easy_install the package to a temporary directory with --no-deps? Why do you think you need to inspect the setup.py in the first place? By the way, your code here is broken even on *nux systems if the user runs your tool with a different Python than the system one. sys.executable gives you the full path to the currently-running version of Python, irrespective of platform. (You might also want to consider using the setuptools.sandbox.run_setup() function instead of os.system.) >ws = WorkingSet(setup_dir) >env = Environment() >dist = env.best_match(Requirement.parse(pkg_name), ws) >print dist.get_metadata("requires.txt").split() > >['pycopia-process>=0.9', 'pycopia-utils>=0.9'] Just use dist.requires() here, and then you won't be dependent on internal details of the format. In general, I would suggest that trying to work your way around setuptools' APIs is a very bad idea. More often than not, you will miss one of the nine jillion supported configurations and thus will end up with a tool that only works on your system... *sometimes*. Case in point: your code above depends on the .egg-info being in the setup directory, which is often not the case. Zope packages, for example, usually have a src/ or lib/ subdirectory that will be where the egg-info is. Other packages may have it in other places, perhaps even more deeply nested. Pretty much *any* place where you are reading a file format directly or looking at directory contents yourself, you're probably breaking something that setuptools has encapsulated for a Very Good Reason. When in doubt, ask. :)

1 0