Mailman 3 May 2009 - Distutils-SIG

[issue70] Patch to remove call to /usr/bin/sw_vers on OSX
by Ronald Oussoren 21 May '09

21 May '09

New submission from Ronald Oussoren <ronaldoussoren(a)mac.com>: The attached patch for setuptools 0.6c9 replaces a shell callout to /usr/bin/sw_vers to using platform.mac_ver on OSX. The primary reason for writing the patch is that gdb tends to cause havoc with calls to os.popen (the debugged process gets an signal and debugging gets aborted). This patch makes it a lot more convenient to run "setup.py test" in a gdb session, without this patch I regularly have to re-run the tests several times until the callout to sw_vers succeeds. ---------- files: setuptools-no-sw_vers.patch messages: 289 nosy: ronaldoussoren priority: feature status: unread title: Patch to remove call to /usr/bin/sw_vers on OSX Added file: http://bugs.python.org/setuptools/file52/setuptools-no-sw_vers.patch _______________________________________________ Setuptools tracker <setuptools(a)bugs.python.org> <http://bugs.python.org/setuptools/issue70> _______________________________________________

1 0

PEP 376 APIs
by Tarek Ziadé 21 May '09

21 May '09

Hello As discussed in python-dev, I have started a prototype for pep 376 to define the APIs. I have changed the APIs with Phillip feedback (eg using holders) It's located here : http://bitbucket.org/tarek/pep376 The idea would be to refine it until everyone finds it good, then update PEP 376 accordingly, then move forward with it It's on bitbucket, so feel free to create a branch to demonstrate a change. Regards Tarek -- Tarek Ziadé | http://ziade.org

1 0

Specifying version information once
by Jean-Paul Calderone 20 May '09

20 May '09

Hello, For a long time, the idiom I've used to specify version information in my distutils-based packages has been this simple one: put version information into the Python package somewhere, structure the source layout so that the package can be imported directly from an unpacked source tarball (or vcs wc), and import the version into setup.py. This worked great. And I only had to define my version once. Increasingly, it seems that new distutils-related tools don't support this idiom and break in various ways. (eg http://divmod.org/trac/ticket/2629 http://divmod.org/trac/ticket/2831 ) What is the recommendation for specifying version information in a way which is compatible with all these new tools? Jean-Paul

7 11

buildout/setuptools setup_requires and test_requires
by Chris Withers 20 May '09

20 May '09

Wichert Akkerman wrote: > Previously Chris Withers wrote: >> Wichert Akkerman wrote: >>>> BUT, one thing I have noticed so far is that after running buildout, I >>>> end up with eggs for Paste, PasteDeploy and PasteScript in >>>> pylons_buildout/myproject, which is wrong. >>> Everything in setup_requires and test_requires is installed there, >> Why? >> >>> and >>> buildout can't prevent that. >> Again, why? > > setuptools internals do that, and buildout has no influence over that as > far as I know. If you want to discuss that distutils-sig is the right > list. Sorry, still not sure I follow you... Are you saying that any packages specified in setup_requires and test_requires will be installed in the path of the package that specified them rather than the same location as all other eggs? If so, that would seem to be a bug, no? Chris -- Simplistix - Content Management, Zope & Python Consulting - http://www.simplistix.co.uk

2 2

Re: [Distutils] Specifying version information once
by P.J. Eby 19 May '09

19 May '09

At 09:05 PM 5/19/2009 -0400, Jean-Paul Calderone wrote: >On Tue, 19 May 2009 20:49:37 -0400, "P.J. Eby" <pje(a)telecommunity.com> wrote: >>At 04:28 PM 5/19/2009 -0400, Jean-Paul Calderone wrote: >>>On Tue, 19 May 2009 16:23:16 -0400, "P.J. Eby" >>><pje(a)telecommunity.com> wrote: >>>>At 02:21 PM 5/19/2009 -0400, Jean-Paul Calderone wrote: >>>>>What is the recommendation for specifying version information in >>>>>a way which >>>>>is compatible with all these new tools? >>>> >>>>Don't import the file with your version info; execfile() it instead. >>>> >>>>That is, you can have a somepackage.version module for importing, >>>>but from within your setup.py, do >>>>execfile('somepackage/version.py') instead, so as to avoid >>>>actually importing your package (and anything it depends on). >>> >>>execfile() doesn't set __name__ properly; setting __name__ in a namespace >>>passed to execfile() produces a CPython SystemError. At the moment my >>>version code depends on __name__ being set. I'm sure I could work around >>>this, but it'd be better if I didn't have to worry about code in the >>>version module being executed in two unrelated contexts. Any other >>>suggestions? >> >>Well, if you're okay with depending on setuptools, you could have >>your version code do this: >> >> version = pkg_resources.require('ThisProject')[0].version >> >>And *only* define the version in setup.py, but don't import or run that code. > >I don't think I'm ready to make setuptools a mandatory dependency yet. > >>Of, if you can't depend on setuptools (no pun intended), the only >>other thing I can think of would be to put the version in a text >>file by itself, and have both the version code and setup.py read that file. > >That's an idea that's come up before. I'm curious to here if anyone has >tried it and what their experiences have been. Oh, now that I think of it, there is one other thing that I've done. For some of my packages (including setuptools itself), I use a script that automatically searches and replaces version numbers, and stores the current version number in a configuration file. Another configuration file specifies what files to edit and what format to insert the numbers in. So, I just do a quick command line like './version incr build' to increment the most-minor version number in the docs, the setup.py, and even to edit my release script. And I suppose one last idea that you could use would be to define your version string in the module in such a way that setup.py could read it with a regular expression match, so it wouldn't be actually executing the code.

1 0

Re: [Distutils] Specifying version information once
by P.J. Eby 19 May '09

19 May '09

At 04:28 PM 5/19/2009 -0400, Jean-Paul Calderone wrote: >On Tue, 19 May 2009 16:23:16 -0400, "P.J. Eby" <pje(a)telecommunity.com> wrote: >>At 02:21 PM 5/19/2009 -0400, Jean-Paul Calderone wrote: >>>What is the recommendation for specifying version information in a way which >>>is compatible with all these new tools? >> >>Don't import the file with your version info; execfile() it instead. >> >>That is, you can have a somepackage.version module for importing, >>but from within your setup.py, do >>execfile('somepackage/version.py') instead, so as to avoid actually >>importing your package (and anything it depends on). > >execfile() doesn't set __name__ properly; setting __name__ in a namespace >passed to execfile() produces a CPython SystemError. At the moment my >version code depends on __name__ being set. I'm sure I could work around >this, but it'd be better if I didn't have to worry about code in the >version module being executed in two unrelated contexts. Any other >suggestions? Well, if you're okay with depending on setuptools, you could have your version code do this: version = pkg_resources.require('ThisProject')[0].version And *only* define the version in setup.py, but don't import or run that code. Of, if you can't depend on setuptools (no pun intended), the only other thing I can think of would be to put the version in a text file by itself, and have both the version code and setup.py read that file.

2 1

Re: [Distutils] Specifying version information once
by P.J. Eby 19 May '09

19 May '09

At 02:21 PM 5/19/2009 -0400, Jean-Paul Calderone wrote: >Hello, > >For a long time, the idiom I've used to specify version information in my >distutils-based packages has been this simple one: put version information >into the Python package somewhere, structure the source layout so that the >package can be imported directly from an unpacked source tarball (or vcs wc), >and import the version into setup.py. This worked great. And I only had to >define my version once. > >Increasingly, it seems that new distutils-related tools don't support this >idiom and break in various ways. (eg > > http://divmod.org/trac/ticket/2629 > http://divmod.org/trac/ticket/2831 >) > >What is the recommendation for specifying version information in a way which >is compatible with all these new tools? Don't import the file with your version info; execfile() it instead. That is, you can have a somepackage.version module for importing, but from within your setup.py, do execfile('somepackage/version.py') instead, so as to avoid actually importing your package (and anything it depends on).

2 1

Changes in r72585
by Tim Golden 19 May '09

19 May '09

[coming from a thread on python-win32: http://mail.python.org/pipermail/python-win32/2009-May/009130.html ] In short: changes to build_ext.py in r72585 caused the pywin32 packages to stop building correctly. The changes in question are those which call get_ext_filename with the short form of the ext.name (without the package prefix). The pywin32 setup.py expects the fully-qualified name in the get_ext_filename in its subclass. Now I don't know whether this is an incautious change to build_ext or a false -- altho' historically valid -- assumption on the part of the pywin32 developers. You can see their code here at line 1119: http://pywin32.cvs.sourceforge.net/viewvc/pywin32/pywin32/setup.py?revision… I'm afraid I'm fairly ignorant of the design rationale behind distutils; I'm hoping someone here has a better idea :) Opinions? TJG

3 11

Re: [Distutils] PEP 376 - critique - why more sub-directories within python aren't needed...
by P.J. Eby 18 May '09

18 May '09

At 12:59 PM 5/19/2009 +1200, Noah Gift wrote: >On Tue, May 19, 2009 at 12:59 PM, P.J. Eby ><<mailto:pje@telecommunity.com>pje(a)telecommunity.com> wrote: >At 08:34 PM 5/18/2009 -0400, David Lyon wrote: >Why not keep the .EGG_INFO files in the site-packages directory? > > >That's where they go. Each installed project has its own .egg-info >subdirectory containing the listed files. See the EggFormats >documentation for details. PEP 376 is just adding stdlib support >for the .egg-info format defined by setuptools, and adding a new >RECORD file to it. > > >But if this implementation is the same as eggs, then each egg >directory is then scanned and imported into sys.path, unlike normal >packages which are called via simple namespaces. You are confused. Please refer to the EggFormats documentation for the layout difference between .egg and .egg-info: http://peak.telecommunity.com/DevCenter/EggFormats .egg-info eggs look just like "normal" distutils installs, except for the addition of a "project-version-pyversion.egg-info" subdirectory alongside the code. In Python 2.5 and up, distutils also installs packages this way, except the "project-version-pyversion.egg-info" is a file instead of a directory - as documented in the above documentation. PEP 376 is simply proposing stdlib support for querying the existing formats, and adding a new standard RECORD file to go in that directory.

1 0

Re: [Distutils] PEP 376 - site-directories and site.addsitedir
by Tarek Ziadé 18 May '09

18 May '09

On Thu, May 14, 2009 at 12:33 PM, Noah Gift <noah.gift(a)gmail.com> wrote: > One thing that isn't clear to me is whether this could then lead to a > potentially new third party packaging system, or library, that will just do > the same thing again, and create a linear scanning algorithm. Is there a > way to architect things in a way that will never increase the number of > stats to the filesystem or limit them? I'd like to make a difference between what is "importable" and what is "installed" in Python. what's "installed" for me is what is in a site-packages directory that contains egg-info directories. At startup, you have to visit at least once every site-packages directory, that's linear and that won't go away. But it's OK eventually, with a cache or a memory index that takes care of making the lookup code non-linear after Python has started *as long as sys.path is not involved* So, by restricting this API to site-packages (e.g. "installed"), rather than sys.path (eg "importable"), is just a way of limiting the scope to some specific places where you are supposed to install packages (and where egg-info files are located). When a program is launched, the site-packages places should be limited to: - the python site-packages (wether is central, wether it's local, using virtualenv) - the per user site-packages (PEP 370) - maybe a third place added manually using site.addsitedir() > One other thing that hopefully isn't off topic is the idea of scanning for > imports in general. Does it at all seem logical to have an option for > python to have zero scanning to load a module? After all if you really know > the path to a module, why do you need to scan anything first to find it? well if it's not loaded already, you have to find it don't you ? And you don't know where it is, you have to locate it (see imporlib and pep 302) But for me that's another story, than locating the metadata info of an installed package. Tarek

7 13