Mailman 3 June 2010 - Distutils-SIG

Re: [Distutils] Design rationale for the egg format ?
by David Cournapeau 15 Jun '10

15 Jun '10

On Tue, Jun 15, 2010 at 10:36 AM, P.J. Eby <pje(a)telecommunity.com> wrote: > As I said above, "it *also* scales better for performance" -- i.e., it's a > secondary concern. ok. > The #1 reason for separating metadata files is that it > makes the addition of new metadata much easier than maintaining a single > monolithic format. Do you still think it is true today ? I am asking because AFAIK, there aren't many packages besides setuptools which use those metadata ? I don't mean to criticize the design, just to see if you would do things differently today. cheers, David

1 0

Re: [Distutils] Design rationale for the egg format ?
by P.J. Eby 15 Jun '10

15 Jun '10

At 08:29 AM 6/15/2010 +0900, David Cournapeau wrote: >On Mon, Jun 14, 2010 at 11:28 PM, P.J. Eby <pje(a)telecommunity.com> wrote: > > At 03:59 PM 6/14/2010 +0900, David Cournapeau wrote: > >> Â - why are the metadata split into files instead of one single metadata > >> file ? > > > > Because that's simpler than trying to define a single universal file format > > that's forward and backward-compatible with every possible feature and use > > case. Â Each use case can have an optimized file format. > > > > It also scales better for performance when you have multiple things you > > might (or might not) be reading. Â For example, since entry points are > > separate from dependencies, you you don't need to read the > dependencies from > > an egg that doesn't have an entry point you're scanning for. > >What I am interested in is the exact situations where this happens >(there is the case where eggs are used as plugins, the case where eggs >are namespace packages, etc...). For example, I don't quite understand >why reading dependencies need to be fast (it does not matter at >install time, so I guess I am missing some usecases) ? As I said above, "it *also* scales better for performance" -- i.e., it's a secondary concern. The #1 reason for separating metadata files is that it makes the addition of new metadata much easier than maintaining a single monolithic format. That is, programs that don't understand new metadata don't have to read it. Plugins that write metadata don't need to co-operate with others - they can just write their own files. And so on. That is the original reason for making separate metadata files: i.e. simplicity. It just turned out to also provide a performance benefit in the case of cross-egg scanning for distinct types of metadata.

1 0

Re: [Distutils] Easy install problems
by P.J. Eby 15 Jun '10

15 Jun '10

At 12:54 PM 6/14/2010 -0700, SEIGAL Nick wrote: >I am having trouble using easy install on a Windows Server 2008 >server. It seems as thought I am getting blocked from downloading >the dependencies. > >At the command line I get this: > >C:\Users>easy_install camelot >Searching for camelot >Best match: camelot 09.12.07 >Processing camelot-09.12.07-py2.6.egg >camelot 09.12.07 is already the active version in easy-install.pth >Installing camelot_manage-script.py script to C:\Python26\Scripts >Installing camelot_manage.exe script to C:\Python26\Scripts >Installing camelot_manage.exe.manifest script to C:\Python26\Scripts >Installing camelot_admin-script.py script to C:\Python26\Scripts >Installing camelot_admin.exe script to C:\Python26\Scripts >Installing camelot_admin.exe.manifest script to C:\Python26\Scripts > >Using c:\python26\lib\site-packages\camelot-09.12.07-py2.6.egg >Processing dependencies for camelot >Searching for Babel>=0.9.4 >Reading ><http://pypi.python.org/simple/Babel/>http://pypi.python.org/simple/Babel/ >Download error: [Errno 10060] A connection attempt failed because >the connected >party did not properly respond after a period of time, or >established connection >failed because connected host has failed to respond -- Some packages >may not be >found! >Couldn't find index page for 'Babel' (maybe misspelled?) > >This process results in a long series of errors at this point. > >Any suggestions? Check your internet settings to see if they're going through a proxy; if so, you'll need to set the http_proxy environment variable, so Python knows how to access the 'net from there.

1 0

Design rationale for the egg format ?
by David Cournapeau 15 Jun '10

15 Jun '10

Hi, I have a few questions about the egg format implementation, and was hoping people who designed it could answer them: - why does the filename encode some metadata (which python version if the package contains extensions, platform specifier) ? - why are the metadata split into files instead of one single metadata file ? I tried to find a rationale for those, but could not find much info on distutils-sig or setuptools doc, thanks, David

4 4

Easy install problems
by SEIGAL Nick 15 Jun '10

15 Jun '10

I am having trouble using easy install on a Windows Server 2008 server. It seems as thought I am getting blocked from downloading the dependencies. At the command line I get this: C:\Users>easy_install camelot Searching for camelot Best match: camelot 09.12.07 Processing camelot-09.12.07-py2.6.egg camelot 09.12.07 is already the active version in easy-install.pth Installing camelot_manage-script.py script to C:\Python26\Scripts Installing camelot_manage.exe script to C:\Python26\Scripts Installing camelot_manage.exe.manifest script to C:\Python26\Scripts Installing camelot_admin-script.py script to C:\Python26\Scripts Installing camelot_admin.exe script to C:\Python26\Scripts Installing camelot_admin.exe.manifest script to C:\Python26\Scripts Using c:\python26\lib\site-packages\camelot-09.12.07-py2.6.egg Processing dependencies for camelot Searching for Babel>=0.9.4 Reading http://pypi.python.org/simple/Babel/ Download error: [Errno 10060] A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond -- Some packages may not be found! Couldn't find index page for 'Babel' (maybe misspelled?) This process results in a long series of errors at this point. Any suggestions? Nick Seigal GIS Analyst/Application Developer Lane Council of Governments - 859 Willamette Street, Suite 500 - Eugene, Oregon 97401-2910 (541) 682-6733 - (541) 682-4099 (fax) - www.lcog.org<http://www.lcog.org/>

1 0

Re: [Distutils] Design rationale for the egg format ?
by P.J. Eby 14 Jun '10

14 Jun '10

At 03:59 PM 6/14/2010 +0900, David Cournapeau wrote: >Hi, > >I have a few questions about the egg format implementation, and was >hoping people who designed it could answer them: > - why does the filename encode some metadata (which python version if >the package contains extensions, platform specifier) ? As Paul mentioned, it's so that discovery can happen without needing anything more than a listdir(). (A legacy of eggs' origin as a binary plugin format.) > - why are the metadata split into files instead of one single > metadata file ? Because that's simpler than trying to define a single universal file format that's forward and backward-compatible with every possible feature and use case. Each use case can have an optimized file format. It also scales better for performance when you have multiple things you might (or might not) be reading. For example, since entry points are separate from dependencies, you you don't need to read the dependencies from an egg that doesn't have an entry point you're scanning for.

1 0

Re: [Distutils] [issue1054967] bdist_deb - Debian packager
by Andrew Straw 14 Jun '10

14 Jun '10

I am moving the venue for this discussion on stdeb to the distutils-sig (from http://bugs.python.org/issue1054967 ). I think this is better discussed there. On 4/29/10 10:16 PM, Éric Araujo wrote: > Éric Araujo<merwok(a)netwok.org> added the comment: > > Andrew, I am uncomfortable with stdeb. (Trying to express respectful > constructive critique, not just picking on your project.) The version in > Debian testing, 0.5.1-1, still requires setuptools (actually only > pkg_resources, which is split into another package in Debian, for space > or politic reasons). I see, thanks for pointing this out. I thought pkg_resources was in the stdlib (or on that track). I will investigate. > The py2dsc script seems to be obsoleted by the > to-be “debian” command. Can you refer me to the source code and/or documentation for that command? I was not previously aware of it. > The pypi-install script seems dangerous to me, > because downloading unverified software and putting it into the system > seems a rather bad idead (a.k.a. “sudo ./setup.py install considered > harmful”). > Yes, some might consider it dangerous, although it does of course require root privileges before installing anything system-wide (and only builds the .deb file with user privileges). Nevertheless, it is a useful tool and safer than "sudo python ./setup.py install" because one can always do "sudo dpkg --purge python-some-package" to remove that package again. > python-debian provides useful support code for generating and reading > debian formats. A quick read show the code would benefit from some 2.6 > modernizing (i.e. gain in simplicity and readability by using modern > idioms and stdlib functions). Yes, I'm not particularly proud of the code in stdeb -- please feel free to hack away. Any and all improvements accepted, preferably in bite-sized commits to a fork on github. > Would using this package in stdeb / > distutils2.command.debian instead of shelling out bring anything? > (Ah, I gather this is the debian command you refer to above?) Not knowing that command, I'm not sure what would be gained. If the command depends on debhelper (by shelling out to dh_make), it would remove that dependency. It also brings all the options described in the README.rst file. -Andrew

2 1

stable and dev packages - how to always install stable?
by Aljoša Mohorović 13 Jun '10

13 Jun '10

if i have stable release mypkg with version="0.9" in setup.py and a dev release with version="1.0.dev14" in setup.py how can i make sure that whenever somebody runs "pip install mypkg" it will always install the stable package? Aljosa Mohorovic

1 0

need help extending distutils clean class
by Ram Bhamidipaty 13 Jun '10

13 Jun '10

Hi, I am trying to create a custom clean class to use with distutils. The custom class works fine - it does what I want when I run it. But I ran into another problem - if I use the "--help-commands" option I get an exception related to my new class: Traceback (most recent call last): File "./setup.py", line 130, in <module> cmdclass = {"clean" : clean} ) File "/tools/pytoken/python-2.6.2-pytoken/lib/python2.6/distutils/core.py", line 138, in setup ok = dist.parse_command_line() File "/tools/pytoken/python-2.6.2-pytoken/lib/python2.6/distutils/dist.py", line 456, in parse_command_line if self.handle_display_options(option_order): File "/tools/pytoken/python-2.6.2-pytoken/lib/python2.6/distutils/dist.py", line 681, in handle_display_options self.print_commands() File "/tools/pytoken/python-2.6.2-pytoken/lib/python2.6/distutils/dist.py", line 759, in print_commands max_length) File "/tools/pytoken/python-2.6.2-pytoken/lib/python2.6/distutils/dist.py", line 721, in print_command_list klass = self.get_command_class(cmd) File "/tools/pytoken/python-2.6.2-pytoken/lib/python2.6/distutils/dist.py", line 852, in get_command_class raise DistutilsModuleError("invalid command '%s'" % command) distutils.errors.DistutilsModuleError: invalid command 'register' My setup looks like this: setup(name = 'pytoken', version = '1.01', description = 'Generates scanners for python.', author = 'Ram Bhamidipaty', author_email = 'rambham(a)gmail.com', url = 'http://code.google.com/p/pytoken/', ext_modules = mlist, py_modules = ['pytoken', 'pytoken_ply_lex'], cmdclass = {"clean" : clean} ) my clean class derives from distutils.command.clean. Any suggestions on how to fix this? Thanks for any help. -Ram

1 0

Multiple index server support in setuptools/buildout
by Benjamin Liles 11 Jun '10

11 Jun '10

Is there any plan to add support for multiple index servers or for the find-links to act as index server links in addition to dumb links to an individual package? Thanks, Ben Liles

1 0