Mailman 3 July 2006 - Distutils-SIG

Import error from Trac for Setuptools
by chtaylo3 20 Jul '06

20 Jul '06

Hello all, I'm running Tracd as a normal user. When I try and access the tracd server from a remote computer, I get the error mentioned at the end of this post. It looks like a new Workset object is being created with no parameters. So because entries==NONE then entires=sys.path in the constructor. It then calls add_entry which calls find_distributions(). Somehow (I'm not sure how), we get into find_on_path where os.listdir() is called, and this is where it blows up. >From the errors given, it looks like it's trying to do a listdir on the contents of /usr/bin and it get's into permissions trouble. However, /usr/bin is not part of $PYTHONPATH and therefore should not be in the results returned from sys.path. When I run python from the shell, and import sys and then do a sys.path I get the following output: ['', '/usr/lib/python2.4/site-packages/setuptools-0.6b4-py2.4.egg', '/usr/lib/python2.4', '/usr/local/lib/svn-python', '/usr/local/lib/svn-python/svn', '/usr/local/lib/svn-python/libsvn', '/usr/lib/python24.zip', '/usr/lib/python2.4/plat-linux2', '/usr/lib/python2.4/lib-tk', '/usr/lib/python2.4/lib-dynload', '/usr/lib/python2.4/site-packages'] So I have two questions 1) From a simple import command, how are we getting into find_on_path when it's find_distributions that is being called? 2) What could have interjected the /usr/bin dir into the $PYTHONPATH variable that reuslts in us calling listidr on /usr/bin? Thank you for your help, Christopher Below is the error: Exception happened during processing of request from ('10.11.1.142', 1543) Traceback (most recent call last): File "/usr/lib/python2.4/SocketServer.py", line 463, in process_request_thread self.finish_request(request, client_address) File "/usr/lib/python2.4/SocketServer.py", line 254, in finish_request self.RequestHandlerClass(request, client_address, self) File "/usr/lib/python2.4/SocketServer.py", line 521, in __init__ self.handle() File "/usr/lib/python2.4/BaseHTTPServer.py", line 314, in handle self.handle_one_request() File "/usr/lib/python2.4/BaseHTTPServer.py", line 308, in handle_one_request method() File "/usr/lib/python2.4/site-packages/trac/web/standalone.py", line 259, in do_GET self._do_trac_req() File "/usr/lib/python2.4/site-packages/trac/web/standalone.py", line 285, in _do_trac_req env = get_environment(req, opts) File "/usr/lib/python2.4/site-packages/trac/web/main.py", line 335, in get_environment return _open_environment(env_path, threaded) File "/usr/lib/python2.4/site-packages/trac/web/main.py", line 51, in _open_environment env_cache[env_path] = open_environment(env_path) File "/usr/lib/python2.4/site-packages/trac/env.py", line 374, in open_environment env = Environment(env_path) File "/usr/lib/python2.4/site-packages/trac/env.py", line 85, in __init__ from trac.loader import load_components File "/usr/lib/python2.4/site-packages/trac/loader.py", line 21, in ? import pkg_resources File "/usr/lib/python2.4/site-packages/setuptools-0.6b4-py2.4.egg/pkg_resources.py" , line 2470, in ? working_set = WorkingSet() File "/usr/lib/python2.4/site-packages/setuptools-0.6b4-py2.4.egg/pkg_resources.py" , line 343, in __init__ self.add_entry(entry) File "/usr/lib/python2.4/site-packages/setuptools-0.6b4-py2.4.egg/pkg_resources.py" , line 358, in add_entry for dist in find_distributions(entry, True): File "/usr/lib/python2.4/site-packages/setuptools-0.6b4-py2.4.egg/pkg_resources.py" , line 1577, in find_on_path for entry in os.listdir(path_item): OSError: [Errno 13] Permission denied: '/usr/bin'

2 1

setuptools 0.6 release candidate 1
by Phillip J. Eby 20 Jul '06

20 Jul '06

setuptools 0.6c1 has just been released. I'm shooting for a final release of 0.6 in about two weeks, so get those bug reports in pronto. :)

1 0

setuptools and extra_path
by Bob Ippolito 18 Jul '06

18 Jul '06

Is there any reason why setuptools totally ignores extra_path when using compatibility mode (e.g. calling .run() manually or --single- version-externally-managed)? It makes uninstallation much more painful than it would've been otherwise. -bob

2 11

Re: [Distutils] [PATCH] catch egg_base=./ foolishness ? was (distutils data_files and setuptools.pkg_resources are driving me crazy)
by Phillip J. Eby 17 Jul '06

17 Jul '06

At 10:06 PM 7/15/2006 +0100, Robin Bryce wrote: >On 15/07/06, Robin Bryce <robinbryce(a)gmail.com> wrote: >>egg_base=./ seemed to prevent the installation of console_scripts. > >Oh fiddle sticks. this is because './' is not equal to os.curdir on >*any* system. > >line 103: egg_info.py does the right thing: if self.egg_base != >os.curdir. This mirrors exactly how distutils.filelist.findall does >the same gaurd on files in the current directory. > >Observing that: >os.path.join('.' + os.sep, '') == os.path.join(os.curdir, '') and >os.path.join('.', '') == os.path.join(os.curdir, '') > >how about this patch:: I don't see the point of the patch. egg_base is supposed to be a directory name. './' is not a directory name, and I don't see a use case for setting egg_base to that. In fact, 99% or more of the time, there's no use case for setting --egg-base in the first place. The option was created mainly to support certain kinds of workaround scenarios in the early days of setuptools development. I would deprecate the option, except that it doesn't really do any harm, unless somebody messes around with it on the theory that it does more than what it actually does. :)

2 2

Keep downloaded files in case of an error
by Dmitry Vasiliev 17 Jul '06

17 Jul '06

Hi, Is there a way to keep downloaded files in case of an error? For example in my test setup.py I wrote: ... 'install_requires=["Twisted >= 2.4.0"]' ... Then I did: $ python setup.py develop ... Downloading ... Twisted-2.4.0.tar.bz2 And then few minutes later: Processing Twisted-2.4.0.tar.bz2 error: Not a recognized archive type: .../Twisted-2.4.0.tar.bz2 The downloaded archive disappears. I think it would be useful to keep (or move to some selectable location) downloaded files in case of an error. -- Dmitry Vasiliev (dima at hlabs.spb.ru) http://hlabs.spb.ru

2 2

Re: [Distutils] [PATCH] catch egg_base=./ foolishness ? was (distutils data_files and setuptools.pkg_resources are driving me crazy)
by Robin Bryce 15 Jul '06

15 Jul '06

On 15/07/06, Robin Bryce <robinbryce(a)gmail.com> wrote: > egg_base=./ seemed to prevent the installation of console_scripts. Oh fiddle sticks. this is because './' is not equal to os.curdir on *any* system. line 103: egg_info.py does the right thing: if self.egg_base != os.curdir. This mirrors exactly how distutils.filelist.findall does the same gaurd on files in the current directory. Observing that: os.path.join('.' + os.sep, '') == os.path.join(os.curdir, '') and os.path.join('.', '') == os.path.join(os.curdir, '') how about this patch:: Index: setuptools/command/egg_info.py =================================================================== --- setuptools/command/egg_info.py (revision 85) +++ setuptools/command/egg_info.py (working copy) @@ -100,7 +100,7 @@ self.ensure_dirname('egg_base') self.egg_info = to_filename(self.egg_name)+'.egg-info' - if self.egg_base != os.curdir: + if os.path.join(self.egg_base, '') != os.path.join(os.curdir,''): self.egg_info = os.path.join(self.egg_base, self.egg_info) if '-' in self.egg_name: self.check_broken_egg_info()

1 0

md5 checksums through PyPI XML-RPC interface
by Ronald Oussoren 15 Jul '06

15 Jul '06

Hi, I'm building a tool that interfaces with PyPI and noticed that while it is possible to fetch a lot of information through XML-RPC, the MD5 checksum of files isn't amongst that. Ronald

2 1

Re: [Distutils] distutils data_files and setuptools.pkg_resources are driving me crazy
by Phillip J. Eby 15 Jul '06

15 Jul '06

At 08:52 PM 7/13/2006 +0100, Robin Bryce wrote: >file:setup.cfg >[egg_info] >egg_base=./ # because I guessed (incorrectly) that this would help. This option controls where the .egg-info *build* directory will be, and *nothing* else. But I think you figured that out. :) Meanwhile, I think the solution to your overall problem will require either expanding the metadata API to allow file access, or else adding some kind of path facility to the resource API (e.g. '/'-prefixed paths as you suggested) to access a 'data/' subdirectory of the egg-info directory. I'll look into this option in setuptools 0.7, although I think it will also require some work on the install_data command. There's also a potentially significant limitation in that you won't be able to look up non-package resources using package names. That is, you're not going to be able to do resource_filename('foo.bar', '/baz'). Instead, you would have to do resource_filename(Requirement.parse('MyProject'), '/baz'). Again, this is an unfortunate quirk caused by the need to support legacy package layouts for system packaging tools such as RPM, Debian, etc.

2 1

Re: [Distutils] Mystery solved
by Phillip J. Eby 14 Jul '06

14 Jul '06

At 08:38 AM 6/24/2006 -0400, Jim Fulton wrote: >Suppose you have some distributions of interest in a directory >somewhere. You ask easy_install to update your packages. You use >find-links to tell it to look in the directory of distributions. By >default, it looks for an index at http://www.python.org/pypi. >Further assume that the distribution you are updating is not in >PyPI. You'll get the following messages: > > Couldn't find index page for 'jimtest' (maybe misspelled?) > Scanning index of all packages (this may take a while) > >And easy install will have made 2 connections, to: > http://www.python.org/pypi/jimtest/, and to http://www.python.org/ pypi/. > >You should be able to easily reproduce this. > >Now, suppose, instead, you provide an index (-i) option telling >easy_install to look somewhere else? Say, http://www.python.org. You >don't get the messages and only a single connection is made.Why? >What's the difference you ask? Well, PyPI (and my test server) >returns a content type of text/plain for 404 results, while most web >servers return text/html. I changed my test server to return HTML >Non-Found results, and the messages went away and I only get a single >connection. Ah, the irony. Your change, and the recent change of PyPI to do the same thing, are actually forcing *erroneous* behavior now, by suppressing the fallback search. :) >So what is the correct behavior? I dunno. The warnings are rather >annoying, so I'm glad I've found out how to make them go away. They'll come back when I release 0.6b4, since it's setuptools' behavior that's broken here. If a response was text/plain, it was doing the right thing, but it wasn't checking text/html responses to see if they were 404 status. >However, I'm pretty sure your intent was to search both / project_name/ >and /. When a server returns an HTML not-found error, >easy_install skips the / check. > >I think there are 2 bugs here: > >1. The scan of the all-package index is skipped if the content-type >of the > if the /project_name/ request is HTML. Yep, that's a bug, and I'm fixing it now. >2. The messages: > > Couldn't find index page for 'jimtest' (maybe misspelled?) > Scanning index of all packages (this may take a while) > > should be info messages, not warning messages. If they > remain warnings, and you fix the first problem, it will be > impossible to avoid the warnings without creating a PyPI > project or creating an index server, and I don't think it was > your intent to require either of these. I don't think a warning > should be issues for correct use of software. Here's the problem. Reducing everything to info messages means there's effectively no control over output detail. I generally use 'warn()' for things that *may* reflect an error in input parameters. So, my take on the above is that although the "Scanning" message could become an info(), the previous one shouldn't.

2 8

eggs and PYTHONPATH (cont'd from peak list...)
by Michael Bayer 14 Jul '06

14 Jul '06

Hi - I have some further thoughts on the PYTHONPATH topic Phillip and I were discussing on the PEAK list a few days ago. On Jun 28, 2006, at 4:09 PM, Phillip J. Eby wrote: > Eggs always override non-eggs, because eggs can be controlled in > ways that non-egg paths can't. The order is (roughly): > > eggs on PYTHONPATH > eggs in site-packages > non-eggs on PYTHONPATH > non-eggs in site-packages > I would propose the order be this: eggs on PYTHONPATH non-eggs on PYTHONPATH eggs in site-packages non-eggs in site-packages It is possible that a non-egg install might be named in PYTHONPATH for a package for which newer versions that are setuptools/egg- enabled are installed system-wide. However, I cant think of a possible scenario where a path would explicitly exist in PYTHONPATH, non-egg or egg, where the user would still like the system-wide installation to take precedence, regardless of versioning or anything else. Otherwise why put the path on PYTHONPATH ? (Though Im guessing there is such a scenario, id just like to see what it is). I know that eggs are more powerful than non-egg paths, allowing precedence to take place based on versioning, which is obviously extremely useful; but I think the idea of PYTHONPATH is that system- wide library paths can be explicitly overridden regardless of any versioning logic taking place either within site-packages or elsewhere. For example, if a particular package on a multi-user shell account is upgraded systemwide from its previously non-egg version to a setuptools/egg version, then any users who have applications running on a custom PYTHONPATH setup with dependencies on their locally- installed, non-egg versions will now have a broken install; theyd have to log in, read all the setuptools docs to figure out whats going on, and do the "virtual python" setup in order to fix things. It seems that setuptools redefines the intention of PYTHONPATH from being a "runtime path determination" variable to an "installation path determination" variable; why repurpose PYTHONPATH in this way, especially when many packages are going to exist in egg/non-egg formats? why not use another variable such as SETUPTOOLSPATH ? Im sure theres other reasons im missing for why its the way it is, so I apologize for harping on this. Also was wondering if anyone else had this same question. - mike

2 1