From kevin at bud.ca Thu Oct 1 00:07:11 2009 From: kevin at bud.ca (Kevin Teague) Date: Wed, 30 Sep 2009 15:07:11 -0700 (PDT) Subject: [Distutils] compiling psycopg2 with zc.recipe.egg: allowing custom build_ext options? Message-ID: Regarding Chris recent thread on doing some hair pulling while getting psycopg2 to install, I recently added this distribution to a buildout with the following configuration: [versions] pscopg2 = 2.0.8 [buildout] parts = app postgresql init-pgsql pgsql-symlinks psycopg2 find-links = http://initd.org/pub/software/psycopg/PSYCOPG-2-0/ versions = versions # app is a Grok-based application that requires pscopg2 [app] recipe = zc.recipe.egg eggs = ${psycopg2:egg} gsc.docsdb z3c.evalexception>=2.0 Paste PasteScript PasteDeploy interpreter = python-console # PostgreSQL parts [psycopg2] recipe = zc.recipe.egg:custom egg = psycopg2 include-dirs = ${postgresql:location}/include library-dirs = ${postgresql:location}/lib [postgresql] recipe = zc.recipe.cmmi url = http://wwwmaster.postgresql.org/redir/198/h/source/v8.3.7/postgresql-8.3.7.tar.gz extra_options = --with-readline [init-pgsql] recipe = iw.recipe.cmd on_install = true on_update = false cmds = ${postgresql:location}/bin/initdb -D ${postgresql:location}/var/data -E UNICODE [pgsql-symlinks] recipe = cns.recipe.symlink symlink_target = ${buildout:directory}/bin symlink_base = ${postgresql:location}/bin symlink = clusterdb createdb createlang createuser dropdb droplang dropuser ecpg initdb ipcclean pg_config pg_controldata pg_ctl pg_dump pg_dumpall pg_resetxlog pg_restore postgres postmaster psql reindexdb vacuumdb This builds a complete PostgreSQL install inside the project directory. Of course, if you've already got a preferred postgres install, you can simplify this quite a bit and just point the include- dirs and library-dirs to that location. I did find that I had to pin psycopg2 down to 2.0.8, since it didn't seem to work with 2.0.10. But looking a bit further into this issue, the preferred way to build psycopg2 is to supply the full path to pg_config as a build_ext. However, zc.recipe.egg:custom only supports a fixed list of build_ext options. Is this a feature or a bug? e.g. Is there any reason why it wouldn't be possible to support arbitrary build_ext options with this recipe? Or is there another egg-building recipe out there which supports this? By patching a local copy of zc.recipe.egg to allow the pg_config build_ext option, I was able to use this part to compile psycopg2: [psycopg2] recipe = zc.recipe.egg:custom egg = psycopg2 pg_config = ${buildout:bin-directory}/pg_config Which worked fine for the latest release, 2.0.10, of psycopg2. (also, as a bonus 'Arg!', note that it's necessary to supply a custom find-links URL to buildout, since the PyPI page for psycopg2 has bogus download links) From greno at verizon.net Thu Oct 1 01:40:16 2009 From: greno at verizon.net (Gerry Reno) Date: Wed, 30 Sep 2009 19:40:16 -0400 Subject: [Distutils] bdist_deb in stdeb In-Reply-To: <4AC2E93B.9020506@astraw.com> References: <4AB30240.8020807@verizon.net> <4ABBE693.6050300@astraw.com> <4AC0BD37.1010507@verizon.net> <4AC0CCEA.5080509@astraw.com> <4AC0CDE6.2010604@verizon.net> <4AC0D48F.90000@astraw.com> <4AC0DEE9.5060800@verizon.net> <4AC0E572.4010401@astraw.com> <4AC0ED94.5050405@verizon.net> <4AC0FF50.5020906@verizon.net> <4AC10622.5080807@astraw.com> <4AC109B8.7060807@verizon.net> <4AC115B9.6000307@astraw.com> <4AC12290.3040404@verizon.net> <4AC126F5.9040204@astraw.com> <4AC1329D.6060809@verizon.net> <4AC135B0.2060302@astraw.com> <4AC142BF.4000004@verizon.net> <4AC1A008.7000100@verizon.net> <4AC22B50.7040106@astraw.com> <4AC235B3.3080900@verizon.net> <4AC23D9C.7090605@astraw.com> <4AC255C9.70501@verizon.net> <4AC2C998.7020807@verizon.net> <4AC2E93B.9020506@astraw.com> Message-ID: <4AC3EC60.5060209@verizon.net> Andrew Straw wrote: > Gerry Reno wrote: > >> Andrew, how can I generate the .deb package without the leading >> 'python-' in the name? Is this configurable? >> > Yes. The 'Package' option. Described in "Customizing the produced Debian > source package (config options)" in the README.rst. > > Ok, in my stdeb.cfg I put this: Package: myapp And it generates the .deb without the 'python-' prefix. But, when I install this .deb it installs but no longer runs the postinst script. If I put back the 'python-' prefix then the postinst runs again. ???? Regards, Gerry From greno at verizon.net Thu Oct 1 01:46:58 2009 From: greno at verizon.net (Gerry Reno) Date: Wed, 30 Sep 2009 19:46:58 -0400 Subject: [Distutils] bdist_deb in stdeb In-Reply-To: <4AC3EC60.5060209@verizon.net> References: <4AB30240.8020807@verizon.net> <4ABBE693.6050300@astraw.com> <4AC0BD37.1010507@verizon.net> <4AC0CCEA.5080509@astraw.com> <4AC0CDE6.2010604@verizon.net> <4AC0D48F.90000@astraw.com> <4AC0DEE9.5060800@verizon.net> <4AC0E572.4010401@astraw.com> <4AC0ED94.5050405@verizon.net> <4AC0FF50.5020906@verizon.net> <4AC10622.5080807@astraw.com> <4AC109B8.7060807@verizon.net> <4AC115B9.6000307@astraw.com> <4AC12290.3040404@verizon.net> <4AC126F5.9040204@astraw.com> <4AC1329D.6060809@verizon.net> <4AC135B0.2060302@astraw.com> <4AC142BF.4000004@verizon.net> <4AC1A008.7000100@verizon.net> <4AC22B50.7040106@astraw.com> <4AC235B3.3080900@verizon.net> <4AC23D9C.7090605@astraw.com> <4AC255C9.70501@verizon.net> <4AC2C998.7020807@verizon.net> <4AC2E93B.9020506@astraw.com> <4AC3EC60.5060209@verizon.net> Message-ID: <4AC3EDF2.9050109@verizon.net> Gerry Reno wrote: > Andrew Straw wrote: >> Gerry Reno wrote: >>> Andrew, how can I generate the .deb package without the leading >>> 'python-' in the name? Is this configurable? >> Yes. The 'Package' option. Described in "Customizing the produced Debian >> source package (config options)" in the README.rst. >> > Ok, in my stdeb.cfg I put this: > Package: myapp > > And it generates the .deb without the 'python-' prefix. But, when I > install this .deb it installs but no longer runs the postinst script. > If I put back the 'python-' prefix then the postinst runs again. > ???? > Ok, I found it. In the debian/ control directory I had to rename the postinst script and remove the leading 'python-' from it as well. Regards, Gerry From cgalvan at mail.utexas.edu Thu Oct 1 01:49:24 2009 From: cgalvan at mail.utexas.edu (Chris Galvan) Date: Wed, 30 Sep 2009 18:49:24 -0500 Subject: [Distutils] something else to throw into the static metadata mix In-Reply-To: References: <4AC372A4.4080701@simplistix.co.uk> <94bdd2610909300927l2a605af5x7c325e9aa0151799@mail.gmail.com> <4AC38715.90909@simplistix.co.uk> <94bdd2610909300953h146d871cm3fa134ecd2ff4761@mail.gmail.com> <20090930142615.9940expm804w4ko8@webmail.utexas.edu> Message-ID: <4AC3EE84.2080505@mail.utexas.edu> Sridhar Ratnakumar wrote: > On Wed, 30 Sep 2009 13:58:56 -0700, Ian Bicking > wrote: > >> The other option could be to enable support for static linking so as >> to make the egg a reasonably self-sufficient binary (eg: lxml's >> --static flag to setup.py) >> Just to throw more in the mix -- this option is totally ad hoc in >> lxml, which means it's hard to handle in an installer like >> easy_install or pip. That is, there's no good way to do >> "easy_install --static lxml" (it's more possible in pip, but still >> quite awkward). > > Maybe we can standardize the ``--static`` flag for setup.py and > installers? The benefit of this is that builders and installers can > support a static build feature without having to hack build scripts > for specific packages (Currently PyPM's backend maintains custom code > for building such special packages). > > Examples of usage:: > > $ python setup.py build --static install > $ easy_install --static lxml > $ pip install --install-option="--static" pip I like your thoughts here, but there currently isn't a way to distinguish between an egg that contains binaries that were linked statically vs. dynamically. It seems like in that case we would need some distinguishing flag in an egg name to signify it was linked statically, and at that point it seems like we are trying to do too much. Thoughts? -Chris > > -srid > From greno at verizon.net Thu Oct 1 02:13:03 2009 From: greno at verizon.net (Gerry Reno) Date: Wed, 30 Sep 2009 20:13:03 -0400 Subject: [Distutils] bdist_deb in stdeb In-Reply-To: <4AC3EDF2.9050109@verizon.net> References: <4AB30240.8020807@verizon.net> <4ABBE693.6050300@astraw.com> <4AC0BD37.1010507@verizon.net> <4AC0CCEA.5080509@astraw.com> <4AC0CDE6.2010604@verizon.net> <4AC0D48F.90000@astraw.com> <4AC0DEE9.5060800@verizon.net> <4AC0E572.4010401@astraw.com> <4AC0ED94.5050405@verizon.net> <4AC0FF50.5020906@verizon.net> <4AC10622.5080807@astraw.com> <4AC109B8.7060807@verizon.net> <4AC115B9.6000307@astraw.com> <4AC12290.3040404@verizon.net> <4AC126F5.9040204@astraw.com> <4AC1329D.6060809@verizon.net> <4AC135B0.2060302@astraw.com> <4AC142BF.4000004@verizon.net> <4AC1A008.7000100@verizon.net> <4AC22B50.7040106@astraw.com> <4AC235B3.3080900@verizon.net> <4AC23D9C.7090605@astraw.com> <4AC255C9.70501@verizon.net> <4AC2C998.7020807@verizon.net> <4AC2E93B.9020506@astraw.com> <4AC3EC60.5060209@verizon.net> <4AC3EDF2.9050109@verizon.net> Message-ID: <4AC3F40F.8040505@verizon.net> 'bdist_deb' command now seems to be working completely. At least on Ubuntu 8.0.4 LTS Server (Hardy). Can someone please test it on a Jaunty install with the latest stdeb 'master' branch code? Regards, Gerry From thomas at thomas-lotze.de Thu Oct 1 07:38:09 2009 From: thomas at thomas-lotze.de (Thomas Lotze) Date: Thu, 01 Oct 2009 07:38:09 +0200 Subject: [Distutils] something else to throw into the static metadata mix References: <4AC372A4.4080701@simplistix.co.uk> <94bdd2610909300927l2a605af5x7c325e9aa0151799@mail.gmail.com> <4AC38715.90909@simplistix.co.uk> Message-ID: Chris Withers wrote: > I think you're missing my point: how would the package be named and/or how > would you express in static metadata, that a particular version of the > package is for postgres 8 rather than postgres 7, or 8.4 rather than 8.3? Just to add my 2 cents: It seems to me that there are two sets of metadata getting confused here. One is the existing set including the Python dependencies. This set is meant to describe what the package does and relies on conceptually. The other is a set of system dependencies that describe in what precise environment a binary egg happens to have been built. The latter are currently expressed only in the file name of the egg or not at all. The set of system dependencies can be taken to be empty for a source egg and includes the target platform and architecture, some of the options Python was built with (what flavour of unicode, which optional libraries compiled in), and a set of third-party system libraries with or without version restrictions. Of the system dependencies, only the platform and architecture are currently accessible (through the egg file name) when downloading an egg, which is one of the reasons why releasing binary eggs for Linux is so unpopular - you might just end up with an unforeseeable mismatch of unicode flavour between your Python and the binary egg. In a dream world, a package index would have an API that allows an installer to query what system dependencies are relevant to binary eggs of a given package and for which combinations of values binary eggs actually exist. The installer could then figure out which of them fits its actual environment best or whether to get the source egg and roll its own binary. -- Thomas From chris at simplistix.co.uk Thu Oct 1 09:09:39 2009 From: chris at simplistix.co.uk (Chris Withers) Date: Thu, 01 Oct 2009 08:09:39 +0100 Subject: [Distutils] something else to throw into the static metadata mix In-Reply-To: <4AC3EE84.2080505@mail.utexas.edu> References: <4AC372A4.4080701@simplistix.co.uk> <94bdd2610909300927l2a605af5x7c325e9aa0151799@mail.gmail.com> <4AC38715.90909@simplistix.co.uk> <94bdd2610909300953h146d871cm3fa134ecd2ff4761@mail.gmail.com> <20090930142615.9940expm804w4ko8@webmail.utexas.edu> <4AC3EE84.2080505@mail.utexas.edu> Message-ID: <4AC455B3.80900@simplistix.co.uk> Chris Galvan wrote: > I like your thoughts here, but there currently isn't a way to > distinguish between an egg that contains binaries that were linked > statically vs. dynamically. It seems like in that case we would need > some distinguishing flag in an egg name to signify it was linked > statically, and at that point it seems like we are trying to do too much. Interestingly, I read that as "we currently use the egg name as a source of static metadata, and it's getting 'too full' to be used for that" ;-) I'm guessing the static metadata that we've been vigorously discussing would be the place to keep this kind of information, and the filename should no longer matter? This would certainly help with things like PIL, which is constantly being repackaged because setuptools didn't like the name that the Effbot chose... cheers, Chris -- Simplistix - Content Management, Batch Processing & Python Consulting - http://www.simplistix.co.uk From kiorky at cryptelium.net Thu Oct 1 10:37:07 2009 From: kiorky at cryptelium.net (kiorky) Date: Thu, 01 Oct 2009 10:37:07 +0200 Subject: [Distutils] something else to throw into the static metadata mix In-Reply-To: References: <4AC372A4.4080701@simplistix.co.uk> <94bdd2610909300927l2a605af5x7c325e9aa0151799@mail.gmail.com> <4AC38715.90909@simplistix.co.uk> <94bdd2610909300953h146d871cm3fa134ecd2ff4761@mail.gmail.com> <20090930142615.9940expm804w4ko8@webmail.utexas.edu> Message-ID: <4AC46A33.6080408@cryptelium.net> Python installers/package managers are not there to install system dependencies. For me, it s totally out of scope and can lead to security holes. Eg, you include a lib "a" (dynamic or static), and it has a hole, to fix it, you must re-release you egg and the users must reinstall it on their side. But if you use the system provided "library", no matter that it was provided by the host system or an isolation software like minitage: If you choose to use dynamic linking when building the egg: * You must only rebuild the library, all is dynamic. If you choose to use static linking when building the egg: * You must rebuild the implied egg too and in the meantime you are exposed to the security hole. Another reasons not to include system dependencies in the python dependencies expressions is that the system dependencies are: - To use the system package manager, you must have great privileges - Installation is not portable, there is plenty of package managers outside - Even if you have support for the underlying package manager, the user may want not to use the system sofware but a custom version of the library built somewhere. And it will be a P?TA and for the metadata parser to have the knowlegde about it. static is not a solution, even not an option for me except in very rare particular cases like having the binutils (/bin/ls, /bin/mount ...) static for recover puposes... Sridhar Ratnakumar a ?crit : > On Wed, 30 Sep 2009 13:58:56 -0700, Ian Bicking > wrote: > >> The other option could be to enable support for static linking so as >> to make the egg a reasonably self-sufficient binary (eg: lxml's >> --static flag to setup.py) >> Just to throw more in the mix -- this option is totally ad hoc in >> lxml, which means it's hard to handle in an installer like >> easy_install or pip. That is, there's no good way to do "easy_install >> --static lxml" (it's more possible in pip, but still quite awkward). > > Maybe we can standardize the ``--static`` flag for setup.py and > installers? The benefit of this is that builders and installers can > support a static build feature without having to hack build scripts for > specific packages (Currently PyPM's backend maintains custom code for > building such special packages). > > Examples of usage:: > > $ python setup.py build --static install > $ easy_install --static lxml > $ pip install --install-option="--static" pip > > -srid > _______________________________________________ > Distutils-SIG maillist - Distutils-SIG at python.org > http://mail.python.org/mailman/listinfo/distutils-sig -- -- Cordialement, KiOrKY GPG Key FingerPrint: 0x1A1194B7681112AF -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 261 bytes Desc: OpenPGP digital signature URL: From kiorky at cryptelium.net Thu Oct 1 10:50:37 2009 From: kiorky at cryptelium.net (kiorky) Date: Thu, 01 Oct 2009 10:50:37 +0200 Subject: [Distutils] something else to throw into the static metadata mix In-Reply-To: <4AC46A33.6080408@cryptelium.net> References: <4AC372A4.4080701@simplistix.co.uk> <94bdd2610909300927l2a605af5x7c325e9aa0151799@mail.gmail.com> <4AC38715.90909@simplistix.co.uk> <94bdd2610909300953h146d871cm3fa134ecd2ff4761@mail.gmail.com> <20090930142615.9940expm804w4ko8@webmail.utexas.edu> <4AC46A33.6080408@cryptelium.net> Message-ID: <4AC46D5D.2000906@cryptelium.net> And another thing is that to support multi-installation with differents versions/packages for projects on the same machine. If you just have a "global" system installation as a base, it is a problem. You must have a way to hook to use your software and you must have means to make multi-installations side by side isolated from each other. kiorky a ?crit : > Python installers/package managers are not there to install system dependencies. > > For me, it s totally out of scope and can lead to security holes. > Eg, you include a lib "a" (dynamic or static), and it has a hole, to fix it, you > must re-release you egg and the users must reinstall it on their side. > > But if you use the system provided "library", no matter that it was provided by > the host system or an isolation software like minitage: > If you choose to use dynamic linking when building the egg: > * You must only rebuild the library, all is dynamic. > > If you choose to use static linking when building the egg: > * You must rebuild the implied egg too and in the meantime you are exposed to > the security hole. > > Another reasons not to include system dependencies in the python dependencies > expressions is that the system dependencies are: > - To use the system package manager, you must have great privileges > - Installation is not portable, there is plenty of package managers outside > - Even if you have support for the underlying package manager, the user may > want not to use the system sofware but a custom version of the library built > somewhere. And it will be a P?TA and for the metadata parser to have the > knowlegde about it. > > static is not a solution, even not an option for me except in very rare > particular cases like having the binutils (/bin/ls, /bin/mount ...) static for > recover puposes... > > Sridhar Ratnakumar a ?crit : >> On Wed, 30 Sep 2009 13:58:56 -0700, Ian Bicking >> wrote: >> >>> The other option could be to enable support for static linking so as >>> to make the egg a reasonably self-sufficient binary (eg: lxml's >>> --static flag to setup.py) >>> Just to throw more in the mix -- this option is totally ad hoc in >>> lxml, which means it's hard to handle in an installer like >>> easy_install or pip. That is, there's no good way to do "easy_install >>> --static lxml" (it's more possible in pip, but still quite awkward). >> Maybe we can standardize the ``--static`` flag for setup.py and >> installers? The benefit of this is that builders and installers can >> support a static build feature without having to hack build scripts for >> specific packages (Currently PyPM's backend maintains custom code for >> building such special packages). >> >> Examples of usage:: >> >> $ python setup.py build --static install >> $ easy_install --static lxml >> $ pip install --install-option="--static" pip >> >> -srid >> _______________________________________________ >> Distutils-SIG maillist - Distutils-SIG at python.org >> http://mail.python.org/mailman/listinfo/distutils-sig > > > ------------------------------------------------------------------------ > > _______________________________________________ > Distutils-SIG maillist - Distutils-SIG at python.org > http://mail.python.org/mailman/listinfo/distutils-sig -- -- Cordialement, KiOrKY GPG Key FingerPrint: 0x1A1194B7681112AF -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 261 bytes Desc: OpenPGP digital signature URL: From roger at erens-krekels.net Thu Oct 1 13:37:51 2009 From: roger at erens-krekels.net (Roger Erens) Date: Thu, 1 Oct 2009 13:37:51 +0200 Subject: [Distutils] Clarification asked wrt FAQ "How does Distribute interacts with virtualenv?" Message-ID: <58ab5feb0910010437t4aa84cccr479c80830c09932e@mail.gmail.com> Hi Tarek e.a., thanks for the nice documentation for Distribute on Pypi. There are however a few more things that are not so clear to me in the virtualenv-FAQ: "How does Distribute interacts with virtualenv?" Typo: interacts -> interact "Everytime you create a virtualenv it will install setuptools, so you need to re-install Distribute in it right after. The Distribute project will not attempt to patch virtualenv so it uses it when globally installed." There are a few too many "it"s in those sentences to my taste. Is this meant: "Everytime you run virtualenv, it will install setuptools in the created virtualenv directory, so you need to install Distribute in that directory right after. (First activating the virtualenv)."? And "The Distribute project will not attempt to patch virtualenv, so [the virtual environment?] uses [the setuptools installed within the virtual environment?] when [Distribute? is [only]] globally installed."? "Once installed, your virtualenv will use Distribute transparently." explicitly this is: "Once Distribute is installed in your virtualenv, your virtualenv will use Distribute transparently."? "Although, if you have Setuptools installed in your system-wide Python, and if the virtualenv you are in was generated without the --no-site-packages option, the Distribute installation will stop." stop -> stop installing? "You need in this case to build a virtualenv with the --no-site-packages option or to install Distribute globally." Maybe it might also be noted that there's a virtualenv-distribute (by Florian Schulze) hanging around somewhere on the web. Hope this helps to improve the documentation, Roger From roger at erens-krekels.net Thu Oct 1 13:37:57 2009 From: roger at erens-krekels.net (Roger Erens) Date: Thu, 1 Oct 2009 13:37:57 +0200 Subject: [Distutils] No announcement for 0.6.3? Message-ID: <58ab5feb0910010437k3da351c4m511511665dee6561@mail.gmail.com> Hi, distribute-0.6.3 was uploaded to Pypi on 09-27-09, but no announcement of its availability was made? Cheers, Roger From alex.gronholm at nextday.fi Thu Oct 1 13:39:42 2009 From: alex.gronholm at nextday.fi (=?ISO-8859-1?Q?Alex_Gr=F6nholm?=) Date: Thu, 01 Oct 2009 14:39:42 +0300 Subject: [Distutils] No announcement for 0.6.3? In-Reply-To: <58ab5feb0910010437k3da351c4m511511665dee6561@mail.gmail.com> References: <58ab5feb0910010437k3da351c4m511511665dee6561@mail.gmail.com> Message-ID: <4AC494FE.7060903@nextday.fi> Roger Erens kirjoitti: > Hi, > > distribute-0.6.3 was uploaded to Pypi on 09-27-09, but no announcement > of its availability was made? > > Tarek mentioned it in the 0.6.2 thread, but yeah, there was no separate announcement. > Cheers, > > Roger > _______________________________________________ > Distutils-SIG maillist - Distutils-SIG at python.org > http://mail.python.org/mailman/listinfo/distutils-sig > From kiorky at cryptelium.net Thu Oct 1 16:21:40 2009 From: kiorky at cryptelium.net (kiorky) Date: Thu, 01 Oct 2009 16:21:40 +0200 Subject: [Distutils] compiling psycopg2 with zc.recipe.egg: allowing custom build_ext options? In-Reply-To: References: Message-ID: <4AC4BAF4.2010706@cryptelium.net> Here is how i would integrate postgresql in my python based projects: My 2 cents ;) ========================================== minitage installation. ========================================== virtualenv --no-site-packages distutils source distutils/bin/activate cd distutils easy_install -U minitage.core && minimerge -s ========================================================= Installing psycopg2 without a dedicated minitage project ========================================================= Install postgresql --------------------- minimerge postgresql-8.4 mkdir pgsql pushd pgsql cat << EOF > buildout.cfg [versions] psycopg2 = 2.0.12 [buildout] versions=versions extensions=buildout.minitagificator parts = psycopg [minitage] dependencies = postgresql-8.4 [psycopg] recipe=minitage.recipe.scripts eggs=psycopg2 interpreter=zopepy # use python datetime! # if you are inside a virtualenv but the python from which you forked has # mx inside, but you dont want to have mx.datetime inside your local virtualenv # you will need to add this bits to avoid build problems psycopg2-bdistext-use_pydatetime=1 psycopg2-bdistext-mx_include_dir=NONEXISTINGDIR EOF buildout bootstrap bin/buildout Checks --------------- ./foo/myproject/bin/zopepy -c "import psycopg2;print psycopg2.__version__" ... 2.0.12 (dt dec ext pq3) ... grep psycopg bin/zopepy ... sys.path[0:0] = [ '/home/kiorky/tmp/distutils/pgsql/eggs/psycopg2-2.0.12-py2.6-linux-i686.egg' ] ... # replace by otool -L on mac ldd eggs/psycopg2-2*.egg/psycopg2/*so ... libpq.so.5 => /home/kiorky/tmp/distutils/dependencies/postgresql-8.4/parts/part/lib/libpq.so.5 (0xb7ecd000) ... libssl.so.0.9.8 => /home/kiorky/tmp/distutils/dependencies/postgresql-8.4/../openssl-0.9/parts/part/lib/libssl.so.0.9.8 (0xb7d02000) libcrypto.so.0.9.8 => /home/kiorky/tmp/distutils/dependencies/postgresql-8.4/../openssl-0.9/parts/part/lib/libcrypto.so.0.9.8 (0xb7bab000) ... popd ==================================================================================== Installing psycopg2 inside a minitagified project (something that have a minibuild) ==================================================================================== Create the minilay/minibuild ----------------------------------- mkdir -p minilays/myprojectminilay/ cat << EOF > minilays/myprojectminilay/myproject [minibuild] dependencies=postgresql-8.4 src_uri="http://foo" # note that if you archive that somewhere, it would we reusable src_type=static category=foo install_method=buildout EOF Create the project dir ------------------------ Note that it is only needed the first time and if you archived your project, you ll just have to checkout the minilay and do "minimerge myproject" mkdir -p foo/myproject/ cat << EOF > foo/myproject/buildout.cfg [versions] psycopg2 = 2.0.12 [buildout] versions=versions extensions=buildout.minitagificator eggs-directory=../../eggs/cache parts = psycopg [psycopg] recipe=minitage.recipe.scripts eggs=psycopg2 interpreter=zopepy # use python datetime! # if you are inside a virtualenv but the python from which you forked has # mx inside, but you dont want to have mx.datetime inside your local virtualenv # you will need to add this bits to avoid build problems psycopg2-bdistext-use_pydatetime=1 psycopg2-bdistext-mx_include_dir=NONEXISTINGDIR EOF Install it ------------ minimerge -NRv myproject Checks --------- grep psy foo/myproject/bin/zopepy ... sys.path[0:0] = [ '/home/kiorky/tmp/distutils/eggs/cache/psycopg2-2.0.12-py2.6-linux-i686.egg' ] ... ./foo/myproject/bin/zopepy -c "import psycopg2;print psycopg2.__version__" 2.0.12 (dt dec ext pq3) # Note that the difference there is that we have include all needed dependencies thanks to the minibuilds infos. ldd ../eggs/cache//psycopg2-2*.egg/psycopg2/*so ... linux-gate.so.1 => (0xffffe000) libpython2.6.so.1.0 => /home/kiorky/tmp/distutils/dependencies/python-2.6/parts/part/lib/libpython2.6.so.1.0 (0xb7e61000) libpq.so.5 => /home/kiorky/tmp/distutils/dependencies/postgresql-8.4/parts/part/lib/libpq.so.5 (0xb7e3d000) libpthread.so.0 => /lib/libpthread.so.0 (0xb7dfc000) libc.so.6 => /lib/libc.so.6 (0xb7cbf000) libz.so.1 => /home/kiorky/tmp/distutils/dependencies/zlib-1.2/parts/part/lib/libz.so.1 (0xb7ca9000) libcrypt.so.1 => /lib/libcrypt.so.1 (0xb7c77000) libbz2.so.1.0 => /home/kiorky/tmp/distutils/dependencies/bzip2-1.0/parts/part/lib/libbz2.so.1.0 (0xb7c65000) libpanel.so.5 => /home/kiorky/tmp/distutils/dependencies/ncurses-5.6/parts/part/lib/libpanel.so.5 (0xb7c5f000) libform.so.5 => /home/kiorky/tmp/distutils/dependencies/ncurses-5.6/parts/part/lib/libform.so.5 (0xb7c50000) libmenu.so.5 => /home/kiorky/tmp/distutils/dependencies/ncurses-5.6/parts/part/lib/libmenu.so.5 (0xb7c47000) libncurses.so.5 => /home/kiorky/tmp/distutils/dependencies/ncurses-5.6/parts/part/lib/libncurses.so.5 (0xb7bfa000) libhistory.so.5 => /home/kiorky/tmp/distutils/dependencies/readline-5.2/parts/part/lib/libhistory.so.5 (0xb7bef000) libreadline.so.5 => /home/kiorky/tmp/distutils/dependencies/readline-5.2/parts/part/lib/libreadline.so.5 (0xb7bb8000) libcrypto.so.0.9.8 => /home/kiorky/tmp/distutils/dependencies/openssl-0.9/parts/part/lib/libcrypto.so.0.9.8 (0xb7a62000) libssl.so.0.9.8 => /home/kiorky/tmp/distutils/dependencies/openssl-0.9/parts/part/lib/libssl.so.0.9.8 (0xb7a1b000) libexpat.so.1 => /home/kiorky/tmp/distutils/dependencies/expat-2.0/parts/part/lib/libexpat.so.1 (0xb79ef000) libdb-4.7.so => /home/kiorky/tmp/distutils/dependencies/db-4.7/parts/part/lib/libdb-4.7.so (0xb7841000) libdl.so.2 => /lib/libdl.so.2 (0xb783d000) libutil.so.1 => /lib/libutil.so.1 (0xb7839000) libm.so.6 => /lib/libm.so.6 (0xb7813000) /lib/ld-linux.so.2 (0xb8079000) ... Install the "env" instance to have symlinks to minitage binaries ------------------------------------------------------------------ easy_install -U minitage.paste paster create --no-interactive -t minitage.instances.env myproject Use it ------- source foo/myproject/sys/share/minitage/minitage.env which psql ... /home/kiorky/tmp/distutils/dependencies/postgresql-8.4/parts/part/bin/psql ... You can even create a postgresql instance for your project --------------------------------------------------------------- Be ware to shutdown your running pgsql instance or run in interative and change the port: paster create --no-interactive -t minitage.instances.postgresql myproject ... Installation is now finished. * You can look for wrappers to various postgresql scripts located in "/home/kiorky/tmp/distutils/foo/myproject/sys/bin/myproject.*" eg : myproject.psql. You must use them as they are configured to use some useful defaults to connect to your database. * A configuration file for your postgresql instance has been linked from /home/kiorky/tmp/distutils/foo/myproject/sys/var/data/postgresql/myproject/postgresql.conf to /home/kiorky/tmp/distutils/foo/myproject/sys/etc/postgresql/myproject_myproject.postgresql.conf. * A pg_hba file for your postgresql instance has been linked from /home/kiorky/tmp/distutils/foo/myproject/sys/var/data/postgresql/myproject/pg_hba.conf to /home/kiorky/tmp/distutils/foo/myproject/sys/etc/postgresql/myproject_myproject.pg_hba.conf. * A pg_ident file for your postgresql instance has been linked from /home/kiorky/tmp/distutils/foo/myproject/sys/var/data/postgresql/myproject/pg_ident.conf to /home/kiorky/tmp/distutils/foo/myproject/sys/etc/postgresql/myproject_myproject.pg_ident.conf. * A init script to start your server is available in /home/kiorky/tmp/distutils/foo/myproject/sys/etc/init.d/postgresql_myproject.myproject. * A logrotate configuration file to handle your logs can be linked in global scope, it is available in /home/kiorky/tmp/distutils/foo/myproject/sys/etc/logrotate.d/myproject_myproject.postgresql. * By default, the user who created the database (kiorky) is now also superuser on it, only via localhost connections or via socket. * By default, you can connect to your database with the user 'myproject' and the supplied password Please note that you are also trusted on localhost. * For security rezasons, PostGreSQL only listens on localhost, change it in the configuration file if you want it to listen to other adresses. * The datadir is located under /home/kiorky/tmp/distutils/foo/myproject/sys/var/data/postgresql/myproject. * You can use pypgoptimizator to Tune automaticly your postgresql: easy_install pypgoptimizator pypgoptimizator -i /home/kiorky/tmp/distutils/foo/myproject/sys/var/data/postgresql/myproject/postgresql.conf -o /home/kiorky/tmp/distutils/foo/myproject/sys/var/data/postgresql/myproject/postgresql.conf ... Launching the server ----------------------------- source foo/myproject/sys/share/minitage/minitage.env ... foo/myproject/sys/etc/init.d/postgresql_myproject.myproject restart * Restarting PostgreSQL ... * Stopping PostgreSQL ...pg_ctl: PID file "/home/kiorky/tmp/distutils/foo/myproject/sys/var/data/postgresql/myproject/postmaster.pid" does not exist Is server running? [ !! ] * ... * Starting PostgreSQL ... ... Connecting ----------------- myproject.psql ... myproject.psql psql (8.4.0) Type "help" for help. myproject=> \l List of databases Name | Owner | Encoding | Collation | Ctype | Access privileges -----------+-----------+----------+-----------+-------+---------------------------- myproject | myproject | UTF8 | C | C | =Tc/myproject : myproject=C*T*c*/myproject postgres | kiorky | UTF8 | C | C | template0 | kiorky | UTF8 | C | C | =c/kiorky : kiorky=CTc/kiorky template1 | kiorky | UTF8 | C | C | =c/kiorky : kiorky=CTc/kiorky (4 rows) myproject=> ... Kevin Teague a ?crit : > Regarding Chris recent thread on doing some hair pulling while getting > psycopg2 to install, I recently added this distribution to a buildout > with the following configuration: > > [versions] > pscopg2 = 2.0.8 > > [buildout] > parts = app postgresql init-pgsql pgsql-symlinks psycopg2 > find-links = http://initd.org/pub/software/psycopg/PSYCOPG-2-0/ > versions = versions > > # app is a Grok-based application that requires pscopg2 > [app] > recipe = zc.recipe.egg > eggs = ${psycopg2:egg} > gsc.docsdb > z3c.evalexception>=2.0 > Paste > PasteScript > PasteDeploy > interpreter = python-console > > # PostgreSQL parts > > [psycopg2] > recipe = zc.recipe.egg:custom > egg = psycopg2 > include-dirs = ${postgresql:location}/include > library-dirs = ${postgresql:location}/lib > > [postgresql] > recipe = zc.recipe.cmmi > url = http://wwwmaster.postgresql.org/redir/198/h/source/v8.3.7/postgresql-8.3.7.tar.gz > extra_options = > --with-readline > > [init-pgsql] > recipe = iw.recipe.cmd > on_install = true > on_update = false > cmds = > ${postgresql:location}/bin/initdb -D ${postgresql:location}/var/data > -E UNICODE > > [pgsql-symlinks] > recipe = cns.recipe.symlink > symlink_target = ${buildout:directory}/bin > symlink_base = ${postgresql:location}/bin > symlink = > clusterdb > createdb > createlang > createuser > dropdb > droplang > dropuser > ecpg > initdb > ipcclean > pg_config > pg_controldata > pg_ctl > pg_dump > pg_dumpall > pg_resetxlog > pg_restore > postgres > postmaster > psql > reindexdb > vacuumdb > > This builds a complete PostgreSQL install inside the project > directory. Of course, if you've already got a preferred postgres > install, you can simplify this quite a bit and just point the include- > dirs and library-dirs to that location. > > I did find that I had to pin psycopg2 down to 2.0.8, since it didn't > seem to work with 2.0.10. But looking a bit further into this issue, > the preferred way to build psycopg2 is to supply the full path to > pg_config as a build_ext. > > However, zc.recipe.egg:custom only supports a fixed list of build_ext > options. Is this a feature or a bug? e.g. Is there any reason why it > wouldn't be possible to support arbitrary build_ext options with this > recipe? Or is there another egg-building recipe out there which > supports this? > > By patching a local copy of zc.recipe.egg to allow the pg_config > build_ext option, I was able to use this part to compile psycopg2: > > [psycopg2] > recipe = zc.recipe.egg:custom > egg = psycopg2 > pg_config = ${buildout:bin-directory}/pg_config > > Which worked fine for the latest release, 2.0.10, of psycopg2. > > (also, as a bonus 'Arg!', note that it's necessary to supply a custom > find-links URL to buildout, since the PyPI page for psycopg2 has bogus > download links) > > _______________________________________________ > Distutils-SIG maillist - Distutils-SIG at python.org > http://mail.python.org/mailman/listinfo/distutils-sig -- -- Cordialement, KiOrKY GPG Key FingerPrint: 0x1A1194B7681112AF -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 261 bytes Desc: OpenPGP digital signature URL: From nad at acm.org Sat Oct 3 11:47:52 2009 From: nad at acm.org (Ned Deily) Date: Sat, 03 Oct 2009 02:47:52 -0700 Subject: [Distutils] Package install failures in 2.6.3 - setuptools vs Distribute Message-ID: I'm afraid there is going to be a small deluge of very confused users who will end up needing to install Distribute but only when they eventually figure out why some packages with C extensions mysteriously no longer install after they upgrade to python 2.6.3. For example, following the package instructions to use setuptools easy_install, an easy_install lxml fails with 2.6.3 with the very cryptic: File "build/bdist.macosx-10.3-fat/egg/setuptools/command/build_ext.py", line 85, in get_ext_filename KeyError: 'etree' http://stackoverflow.com/questions/1512530/cant-install-lxml-python-2-6-3 -osx-10-6-snow-leopard Appscript is another package that fails similarly. There must be others. I don't know what's to be done about this awkward incompatibility now that 2.6.3 is out the door other than perhaps making sure maintainers of affected packages are aware of what their users are running into and can update their documentation. There is one small thing that may help some users who find their way to http://pypi.python.org/pypi/distribute. OS X, for one, does not ship with wget however it does have curl. Suggest adding another example: $ curl -O http://nightly.ziade.org/distribute_setup.py $ python distribute_setup.py -- Ned Deily, nad at acm.org From regebro at gmail.com Sat Oct 3 14:15:37 2009 From: regebro at gmail.com (Lennart Regebro) Date: Sat, 3 Oct 2009 14:15:37 +0200 Subject: [Distutils] Package install failures in 2.6.3 - setuptools vs Distribute In-Reply-To: References: Message-ID: <319e029f0910030515u71325b88h89037b3ea85f65b5@mail.gmail.com> 2009/10/3 Ned Deily : > I'm afraid there is going to be a small deluge of very confused users > who will end up needing to install Distribute but only when they > eventually figure out why some packages with C extensions mysteriously > no longer install after they upgrade to python 2.6.3. ?For example, > following the package instructions to use setuptools easy_install, an > easy_install lxml fails with 2.6.3 with the very cryptic: > > File "build/bdist.macosx-10.3-fat/egg/setuptools/command/build_ext.py", > line 85, in get_ext_filename KeyError: 'etree' > > http://stackoverflow.com/questions/1512530/cant-install-lxml-python-2-6-3 > -osx-10-6-snow-leopard > > Appscript is another package that fails similarly. ?There must be others. I can confirm this. It seems to be packages that use c-extensions. zope.interface also breaks. > ?$ curl -O http://nightly.ziade.org/distribute_setup.py > ?$ python distribute_setup.py Distribute doesn't break, though, so "sudo easy_install Distribute" seems to be enough. :-) -- Lennart Regebro: Python, Zope, Plone, Grok http://regebro.wordpress.com/ +33 661 58 14 64 From ziade.tarek at gmail.com Sat Oct 3 15:49:30 2009 From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=) Date: Sat, 3 Oct 2009 15:49:30 +0200 Subject: [Distutils] Package install failures in 2.6.3 - setuptools vs Distribute In-Reply-To: <319e029f0910030515u71325b88h89037b3ea85f65b5@mail.gmail.com> References: <319e029f0910030515u71325b88h89037b3ea85f65b5@mail.gmail.com> Message-ID: <94bdd2610910030649r431a5638y7c8b5332934f5ae@mail.gmail.com> On Sat, Oct 3, 2009 at 2:15 PM, Lennart Regebro wrote: > 2009/10/3 Ned Deily : >> I'm afraid there is going to be a small deluge of very confused users >> who will end up needing to install Distribute but only when they >> eventually figure out why some packages with C extensions mysteriously >> no longer install after they upgrade to python 2.6.3. ?For example, >> following the package instructions to use setuptools easy_install, an >> easy_install lxml fails with 2.6.3 with the very cryptic: >> >> File "build/bdist.macosx-10.3-fat/egg/setuptools/command/build_ext.py", >> line 85, in get_ext_filename KeyError: 'etree' >> >> http://stackoverflow.com/questions/1512530/cant-install-lxml-python-2-6-3 >> -osx-10-6-snow-leopard >> >> Appscript is another package that fails similarly. ?There must be others. > > I can confirm this. It seems to be packages that use c-extensions. > zope.interface also breaks. Yes Setuptools makes some assumptions on how distutils APIs are called and in which order. Once bitbucket is up, you can look at the details on the fix I did, it's fairly simple. Notice that this has been fixed in Ubuntu already with a patched version of setuptools, but the next probable step is to use Distribute there and in debian. > >> ?$ curl -O http://nightly.ziade.org/distribute_setup.py >> ?$ python distribute_setup.py > > Distribute doesn't break, though, so "sudo easy_install Distribute" > seems to be enough. :-) > Yes that's the simplest way to fix the problem. Notice that setuptools patches distutils in other places, and it will not work with Python 2.7 and 3.2 where Distribute does. Cheers Tarek -- Tarek Ziad? | http://ziade.org |????????????! |???????????? From ktenney at gmail.com Sat Oct 3 13:54:24 2009 From: ktenney at gmail.com (Kent Tenney) Date: Sat, 3 Oct 2009 06:54:24 -0500 Subject: [Distutils] Problem using zc.recipe.cmmi to build Python Message-ID: Howdy, In pursuit of a buildout for my core environment, I try the following: [python-2.6.3] recipe = zc.recipe.cmmi url = http://www.python.org/ftp/python/2.6.3/Python-2.6.3rc1.tgz ... make fails with ImportError: No module named cStringIO because ./configure creates a Modules/Setup which says #cStringIO cStringIO.c if I uncomment that line the make is fine. I haven't found a way to tell $:./configure or $:make to do that, zc.recipe.cmmi doesn't seem to provide a hook between configure and make. If someone can tell me to fix this with options or environment I'm done. If not, I welcome advice on alternatives to zc.recipe.cmmi, (which I would consider the 'standard' solution.(darn you Shuttleworth for co-opting the word 'canonical' :-))) - comb through the many xx.xx.cmmi recipes (without a sense of associated quality or support level) - pick a xx.xx.command recipe and write a part which does what I want - write my own 'recipe.build-interp' Thanks, Kent From ziade.tarek at gmail.com Sat Oct 3 15:52:20 2009 From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=) Date: Sat, 3 Oct 2009 15:52:20 +0200 Subject: [Distutils] Clarification asked wrt FAQ "How does Distribute interacts with virtualenv?" In-Reply-To: <58ab5feb0910010437t4aa84cccr479c80830c09932e@mail.gmail.com> References: <58ab5feb0910010437t4aa84cccr479c80830c09932e@mail.gmail.com> Message-ID: <94bdd2610910030652q46318b3emffb47923debaa261@mail.gmail.com> On Thu, Oct 1, 2009 at 1:37 PM, Roger Erens wrote: > Hi Tarek e.a., Hi Roger, thanks a lot for the feedback, I'll push them as soon as bitbuket is up again, CHeers Tarek From kiorky at cryptelium.net Sat Oct 3 17:16:36 2009 From: kiorky at cryptelium.net (kiorky) Date: Sat, 03 Oct 2009 17:16:36 +0200 Subject: [Distutils] Problem using zc.recipe.cmmi to build Python In-Reply-To: References: Message-ID: <4AC76AD4.8000106@cryptelium.net> You can look at : http://git.minitage.org/git/minitage/buildouts/dependencies/python-2.6.3/tree/buildout.cfg This buildout build a python with buildout as a "minitage system dependency". Note that the url points to a recent SVN snapshot as there are many ox snowleopard related bugs (blockers) fixed in trunk. This buildout uses minitage.recipe.cmmi which is suported and well tested ... You have hooks available at each stage of the build sequence, and many more features that in zc.recipe.cmmi. Options documentation is shared between the ".cmmi" and the ".common" recipe. IMO, you must remove from that the minitage-flags.diff bits at it's a way for me o force compilation of all extensions module i know i will be able to compile inside a minitage environment for this buildout, you are DONE. For your setup with cStringIO commented, you must be on an unsupported environment, this module must be always be compiled (triggered via setup.py) look at your lib/python-2.6/lib-dynload if there is any "cStringIO.so". See [1]. [1] - http://code.python.org/hg/branches/release2.6-maint/annotate/8fcbbeb5eabb/setup.py#l311 [2] - http://pypi.python.org/pypi/minitage.recipe.cmmi [3] - http://pypi.python.org/pypi/minitage.recipe.common Kent Tenney a ?crit : > Howdy, > > In pursuit of a buildout for my core environment, I try the following: > > [python-2.6.3] > recipe = zc.recipe.cmmi > url = http://www.python.org/ftp/python/2.6.3/Python-2.6.3rc1.tgz > ... > > make fails with > ImportError: No module named cStringIO > because ./configure creates a Modules/Setup which says > #cStringIO cStringIO.c > > if I uncomment that line the make is fine. > > I haven't found a way to tell $:./configure or $:make to do that, > zc.recipe.cmmi doesn't seem to provide a hook between configure and make. > > If someone can tell me to fix this with options or environment I'm done. > > If not, I welcome advice on alternatives to zc.recipe.cmmi, > (which I would consider the 'standard' solution.(darn you Shuttleworth > for co-opting the word 'canonical' :-))) > > > - comb through the many xx.xx.cmmi recipes > (without a sense of associated quality or support level) > > - pick a xx.xx.command recipe and write a part which does what I want > > - write my own 'recipe.build-interp' > > Thanks, > Kent > _______________________________________________ > Distutils-SIG maillist - Distutils-SIG at python.org > http://mail.python.org/mailman/listinfo/distutils-sig -- -- Cordialement, KiOrKY GPG Key FingerPrint: 0x1A1194B7681112AF -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 261 bytes Desc: OpenPGP digital signature URL: From kiorky at cryptelium.net Sat Oct 3 17:30:13 2009 From: kiorky at cryptelium.net (kiorky) Date: Sat, 03 Oct 2009 17:30:13 +0200 Subject: [Distutils] Problem using zc.recipe.cmmi to build Python In-Reply-To: <4AC76AD4.8000106@cryptelium.net> References: <4AC76AD4.8000106@cryptelium.net> Message-ID: <4AC76E05.7020802@cryptelium.net> Or you can even use minitage to have it, see my previous post. As a short: virtualenv --no-site-packages f source f/bin/activate easy_install minitage.core minimerge -s minimerge -v python-2.6 kiorky a ?crit : > You can look at : > http://git.minitage.org/git/minitage/buildouts/dependencies/python-2.6.3/tree/buildout.cfg > > This buildout build a python with buildout as a "minitage system dependency". > > Note that the url points to a recent SVN snapshot as there are many ox > snowleopard related bugs (blockers) fixed in trunk. > > This buildout uses minitage.recipe.cmmi which is suported and well tested ... > You have hooks available at each stage of the build sequence, and many more > features that in zc.recipe.cmmi. Options documentation is shared between the > ".cmmi" and the ".common" recipe. > > IMO, you must remove from that the minitage-flags.diff bits at it's a way for me > o force compilation of all extensions module i know i will be able to compile > inside a minitage environment for this buildout, you are DONE. > > For your setup with cStringIO commented, you must be on an unsupported > environment, this module must be always be compiled (triggered via setup.py) > look at your lib/python-2.6/lib-dynload if there is any "cStringIO.so". > > See [1]. > > [1] - > http://code.python.org/hg/branches/release2.6-maint/annotate/8fcbbeb5eabb/setup.py#l311 > [2] - http://pypi.python.org/pypi/minitage.recipe.cmmi > [3] - http://pypi.python.org/pypi/minitage.recipe.common > > > Kent Tenney a ?crit : >> Howdy, >> >> In pursuit of a buildout for my core environment, I try the following: >> >> [python-2.6.3] >> recipe = zc.recipe.cmmi >> url = http://www.python.org/ftp/python/2.6.3/Python-2.6.3rc1.tgz >> ... >> >> make fails with >> ImportError: No module named cStringIO >> because ./configure creates a Modules/Setup which says >> #cStringIO cStringIO.c >> >> if I uncomment that line the make is fine. >> >> I haven't found a way to tell $:./configure or $:make to do that, >> zc.recipe.cmmi doesn't seem to provide a hook between configure and make. >> >> If someone can tell me to fix this with options or environment I'm done. >> >> If not, I welcome advice on alternatives to zc.recipe.cmmi, >> (which I would consider the 'standard' solution.(darn you Shuttleworth >> for co-opting the word 'canonical' :-))) >> >> >> - comb through the many xx.xx.cmmi recipes >> (without a sense of associated quality or support level) >> >> - pick a xx.xx.command recipe and write a part which does what I want >> >> - write my own 'recipe.build-interp' >> >> Thanks, >> Kent >> _______________________________________________ >> Distutils-SIG maillist - Distutils-SIG at python.org >> http://mail.python.org/mailman/listinfo/distutils-sig > > > ------------------------------------------------------------------------ > > _______________________________________________ > Distutils-SIG maillist - Distutils-SIG at python.org > http://mail.python.org/mailman/listinfo/distutils-sig -- -- Cordialement, KiOrKY GPG Key FingerPrint: 0x1A1194B7681112AF -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 261 bytes Desc: OpenPGP digital signature URL: From nad at acm.org Sat Oct 3 19:00:09 2009 From: nad at acm.org (Ned Deily) Date: Sat, 03 Oct 2009 10:00:09 -0700 Subject: [Distutils] Package install failures in 2.6.3 - setuptools vs Distribute References: <319e029f0910030515u71325b88h89037b3ea85f65b5@mail.gmail.com> <94bdd2610910030649r431a5638y7c8b5332934f5ae@mail.gmail.com> Message-ID: In article <94bdd2610910030649r431a5638y7c8b5332934f5ae at mail.gmail.com>, Tarek Ziad? wrote: > On Sat, Oct 3, 2009 at 2:15 PM, Lennart Regebro wrote: > > 2009/10/3 Ned Deily : > >> I'm afraid there is going to be a small deluge of very confused users > >> who will end up needing to install Distribute but only when they > >> eventually figure out why some packages with C extensions mysteriously > >> no longer install after they upgrade to python 2.6.3. ?For example, > >> following the package instructions to use setuptools easy_install, an > >> easy_install lxml fails with 2.6.3 with the very cryptic: > >> > >> File "build/bdist.macosx-10.3-fat/egg/setuptools/command/build_ext.py", > >> line 85, in get_ext_filename KeyError: 'etree' > >> > >> http://stackoverflow.com/questions/1512530/cant-install-lxml-python-2-6-3 > >> -osx-10-6-snow-leopard > >> > >> Appscript is another package that fails similarly. ?There must be others. > > > > I can confirm this. It seems to be packages that use c-extensions. > > zope.interface also breaks. > > Yes Setuptools makes some assumptions on how distutils APIs are called > and in which order. Once bitbucket is up, you can look at the details > on the fix > I did, it's fairly simple. > > Notice that this has been fixed in Ubuntu already with a patched version > of setuptools, but the next probable step is to use Distribute there > and in debian. That's fine but how does this help users on OS X, for example, when the python 2.6.3 they are using is supplied only by the python.org installers? And the maintainers of the packages that can't be installed? > >> ?$ curl -O http://nightly.ziade.org/distribute_setup.py > >> ?$ python distribute_setup.py > > > > Distribute doesn't break, though, so "sudo easy_install Distribute" > > seems to be enough. :-) > Yes that's the simplest way to fix the problem. But that's assuming they have a working easy_install installed for *that* version (2.6.3) in the first place. For instance, the OS X installer does not include setuptools or Distribute. I'm concerned about users who just want to install some third-party package for the first time. They try to follow the recipe for bootstrapping Distribute and fail right at the first step because they don't have wget on their system (for example, OS X) and they don't know what it is. Getting it right for beginners is one of the reasons why easy_install was developed in the first place. > Notice that setuptools patches distutils in other places, and it will > not work with Python 2.7 and 3.2 > where Distribute does. That's fine, too, and there are lessons to be learned here, I think, that should influence 2.7 and 3.2. But the big problem is distutils in 2.6.3. This is not trying to debate the Distribute fork; the target audience for easy_install doesn't care one way or the other. More importantly, they haven't a clue that there is something called Distribute and they don't know what setuptools is. Like it or not, easy_install / setuptools is a major customer of distutils and somehow it got broken for 2.6.3, a point release. From the end user and the 3rd-party packagers' points of view: the 3rd-party package didn't change, setuptools didn't change (obviously!), but now the package won't install and, worse, fails cryptically. So the problem must be Python - so they think and with some justification, it seems. Once they get Distribute installed, everything should be fine but how do they know they are supposed to install Distribute? This is not a good experience for users. Unless I'm missing something (and I hope I am), this issue really can't be hand-waved away. -- Ned Deily, nad at acm.org From ssteinerx at gmail.com Sat Oct 3 22:00:45 2009 From: ssteinerx at gmail.com (ssteinerX@gmail.com) Date: Sat, 3 Oct 2009 16:00:45 -0400 Subject: [Distutils] Package install failures in 2.6.3 - setuptools vs Distribute In-Reply-To: References: <319e029f0910030515u71325b88h89037b3ea85f65b5@mail.gmail.com> <94bdd2610910030649r431a5638y7c8b5332934f5ae@mail.gmail.com> Message-ID: On Oct 3, 2009, at 1:00 PM, Ned Deily wrote: > This is not a good experience for users. Unless I'm missing something > (and I hope I am), this issue really can't be hand-waved away. What would you suggest? S From regebro at gmail.com Sat Oct 3 22:08:30 2009 From: regebro at gmail.com (Lennart Regebro) Date: Sat, 3 Oct 2009 22:08:30 +0200 Subject: [Distutils] Package install failures in 2.6.3 - setuptools vs Distribute In-Reply-To: References: <319e029f0910030515u71325b88h89037b3ea85f65b5@mail.gmail.com> <94bdd2610910030649r431a5638y7c8b5332934f5ae@mail.gmail.com> Message-ID: <319e029f0910031308u4182dff2w89760afb4122cf2c@mail.gmail.com> 2009/10/3 Ned Deily : > This is not a good experience for users. ?Unless I'm missing something > (and I hope I am), this issue really can't be hand-waved away. It's unfortunate that this comes in a minor release. But at the same time we can hardly avoid fixing bugs just because setuptools isn't getting updated at the moment. It's a lose-lose situation. As I see it this will speed up adaptation of Distribute. Word will spread. It's not ideal, but then it's not a perfect world. -- Lennart Regebro: Python, Zope, Plone, Grok http://regebro.wordpress.com/ +33 661 58 14 64 From ziade.tarek at gmail.com Sat Oct 3 22:09:47 2009 From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=) Date: Sat, 3 Oct 2009 22:09:47 +0200 Subject: [Distutils] Package install failures in 2.6.3 - setuptools vs Distribute In-Reply-To: References: <319e029f0910030515u71325b88h89037b3ea85f65b5@mail.gmail.com> <94bdd2610910030649r431a5638y7c8b5332934f5ae@mail.gmail.com> Message-ID: <94bdd2610910031309w61d72dcdo8faab4964bf67551@mail.gmail.com> On Sat, Oct 3, 2009 at 7:00 PM, Ned Deily wrote: > > This is not a good experience for users. ?Unless I'm missing something > (and I hope I am), this issue really can't be hand-waved away. Make sure to understand that the way setuptools patches distutils makes it very sensible to any change made in distutils, even backward compatibles ones like in the 2.6 branch. But I won't freeze distutils work because of that, and distutils is not to be blamed if setuptools is broken, neither Python. Especially for a project that has not been maintained for over a year... So it's not a good experience for the users that's true, and we can try to enhance the Distribute documentation to help them as you suggested, and we are already helping because we've forked to try to provide some solutions. But there's nothing else I can think of to help people. My advice would be that the projects that use setuptools just switch asap on distribute, which is actively maintained. > -- > ?Ned Deily, > ?nad at acm.org > > _______________________________________________ > Distutils-SIG maillist ?- ?Distutils-SIG at python.org > http://mail.python.org/mailman/listinfo/distutils-sig > -- Tarek Ziad? | http://ziade.org |????????????! |???????????? From ziade.tarek at gmail.com Sat Oct 3 22:11:57 2009 From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=) Date: Sat, 3 Oct 2009 22:11:57 +0200 Subject: [Distutils] Package install failures in 2.6.3 - setuptools vs Distribute In-Reply-To: <94bdd2610910031309w61d72dcdo8faab4964bf67551@mail.gmail.com> References: <319e029f0910030515u71325b88h89037b3ea85f65b5@mail.gmail.com> <94bdd2610910030649r431a5638y7c8b5332934f5ae@mail.gmail.com> <94bdd2610910031309w61d72dcdo8faab4964bf67551@mail.gmail.com> Message-ID: <94bdd2610910031311u6a3bbb26qa8dc794587da70f@mail.gmail.com> On Sat, Oct 3, 2009 at 10:09 PM, Tarek Ziad? wrote: > ...makes it very sensible to any change made in distutils, even backward > compatibles ones like in the 2.6 branch.... s/backward compatibles/ bug fixes/ From ssteinerx at gmail.com Sat Oct 3 22:30:46 2009 From: ssteinerx at gmail.com (ssteinerX@gmail.com) Date: Sat, 3 Oct 2009 16:30:46 -0400 Subject: [Distutils] Package install failures in 2.6.3 - setuptools vs Distribute In-Reply-To: <319e029f0910031308u4182dff2w89760afb4122cf2c@mail.gmail.com> References: <319e029f0910030515u71325b88h89037b3ea85f65b5@mail.gmail.com> <94bdd2610910030649r431a5638y7c8b5332934f5ae@mail.gmail.com> <319e029f0910031308u4182dff2w89760afb4122cf2c@mail.gmail.com> Message-ID: <1A4062E9-3B2C-48F1-9B97-992A03CAD148@gmail.com> On Oct 3, 2009, at 4:08 PM, Lennart Regebro wrote: > 2009/10/3 Ned Deily : >> This is not a good experience for users. Unless I'm missing >> something >> (and I hope I am), this issue really can't be hand-waved away. How about some sort of an announcement/warning on the setuptools site itself? I know the code's not going to get updated but how about a simple warning and suggestion to move on to Distribute? S From nad at acm.org Sat Oct 3 23:17:50 2009 From: nad at acm.org (Ned Deily) Date: Sat, 03 Oct 2009 14:17:50 -0700 Subject: [Distutils] Package install failures in 2.6.3 - setuptools vs Distribute References: <319e029f0910030515u71325b88h89037b3ea85f65b5@mail.gmail.com> <94bdd2610910030649r431a5638y7c8b5332934f5ae@mail.gmail.com> <94bdd2610910031309w61d72dcdo8faab4964bf67551@mail.gmail.com> Message-ID: In article <94bdd2610910031309w61d72dcdo8faab4964bf67551 at mail.gmail.com>, Tarek Ziad? wrote: > On Sat, Oct 3, 2009 at 7:00 PM, Ned Deily wrote: > > This is not a good experience for users. ?Unless I'm missing something > > (and I hope I am), this issue really can't be hand-waved away. > Make sure to understand that the way setuptools patches distutils > makes it very sensible to any change made in distutils, even backward > compatibles ones like in the 2.6 branch. > > But I won't freeze distutils work because of that, and distutils > is not to be blamed if setuptools is broken, neither Python. I understand very well the issues that lead to the establishment of the Distribute project and I think it is great that you have taken on this - at times - pretty thankless effort. So thank you for doing so. I'm not trying to point fingers here; if I were, I'd start with me not recognizing this issue before 2.6.3 went out the door. I'm simply trying to put myself in the place of the "naive user", the target audience of easy_install. And users, naive or otherwise, don't know or care who is to blame, they'll just know that something about using Python 2.6.3 is broken. > So it's not a good experience for the users that's true, and we can > try to enhance > the Distribute documentation to help them as you suggested, and we are > already helping > because we've forked to try to provide some solutions. > > But there's nothing else I can think of to help people. My advice would be > that > the projects that use setuptools just switch asap on distribute, which > is actively > maintained. That's fine but they're not going to know about Distribute unless they stumble across discussions like this. And it's not their usage of setuptools that is the problem here. I think most of them would view this (rightly or wrongly) as: "Without warning, my package broke on a bug fix release of Python because of changes in Python." One solution *is* for the maintainers of affected packages to change their packaging and documentation to explicitly require Distribute. That's not going to happen overnight, of course. So is it possible to get a better idea of what the impact of this problem is going to be? I've already identified one real-world use case: (1) OS X users who install 2.6.3 from python.org and who need to install certain packages (lxml, appscript, zope-interfaces, others?). On what other platforms is this likely to be a problem? Windows *? Linuxes? If that can be identified, if necessary the distributors of Python installers can be informed so they can inform their users (note, that python.org is itself a distributor of Python installers). I don't have the expertise to make an assessment of that. Is it possible to get a better idea of what packages might be affected, i.e. what is it that these apps are using in their setup.py, or elsewhere, that causes the problem to appear? If so, it would be easier to warn the affected developers and users. Certainly some kind of warning could be added to the python.org 2.6.3 download pages and sent out of the various news lists. Perhaps a note about 2.6.3 and 3.x could be added to the original setuptools / easy_install PyPi page. I guess the bottom line question is: Is there anything more pro-active that should be done? -- Ned Deily, nad at acm.org From sridharr at activestate.com Sat Oct 3 23:28:21 2009 From: sridharr at activestate.com (Sridhar Ratnakumar) Date: Sat, 03 Oct 2009 14:28:21 -0700 Subject: [Distutils] Package install failures in 2.6.3 - setuptools vs Distribute In-Reply-To: References: <319e029f0910030515u71325b88h89037b3ea85f65b5@mail.gmail.com> <94bdd2610910030649r431a5638y7c8b5332934f5ae@mail.gmail.com> <94bdd2610910031309w61d72dcdo8faab4964bf67551@mail.gmail.com> Message-ID: On Sat, 03 Oct 2009 14:17:50 -0700, Ned Deily wrote: > On what other platforms is this likely to be a problem? Windows *? > Linuxes? If that can be identified, if necessary the distributors of > Python installers can be informed so they can inform their users (note, > that python.org is itself a distributor of Python installers). I don't > have the expertise to make an assessment of that. From what I've seen, it affects all platforms. > Is it possible to get a better idea of what packages might be affected, > i.e. what is it that these apps are using in their setup.py, or > elsewhere, that causes the problem to appear? I believe whenever the build_ext command is used, this problem would occur. One trivial package that fails is zope.interface. -srid From regebro at gmail.com Sat Oct 3 23:32:03 2009 From: regebro at gmail.com (Lennart Regebro) Date: Sat, 3 Oct 2009 23:32:03 +0200 Subject: [Distutils] Package install failures in 2.6.3 - setuptools vs Distribute In-Reply-To: References: <319e029f0910030515u71325b88h89037b3ea85f65b5@mail.gmail.com> <94bdd2610910030649r431a5638y7c8b5332934f5ae@mail.gmail.com> <94bdd2610910031309w61d72dcdo8faab4964bf67551@mail.gmail.com> Message-ID: <319e029f0910031432x549e075fld018b34f7f829c59@mail.gmail.com> 2009/10/3 Ned Deily : > That's fine but they're not going to know about Distribute unless they > stumble across discussions like this. They are going to ask around, and somebody will know. Most reasonably, they are going to ask the maker of the module they are trying to install, and say "Hey your module doesn't work with Python 2.6.3. And that module guy will ask the python list. And the python list will know. People are not isolated. Information will reach them. -- Lennart Regebro: Python, Zope, Plone, Grok http://regebro.wordpress.com/ +33 661 58 14 64 From greno at verizon.net Sun Oct 4 01:13:36 2009 From: greno at verizon.net (Gerry Reno) Date: Sat, 03 Oct 2009 19:13:36 -0400 Subject: [Distutils] bdist_deb in stdeb In-Reply-To: <4AC3F40F.8040505@verizon.net> References: <4AB30240.8020807@verizon.net> <4ABBE693.6050300@astraw.com> <4AC0BD37.1010507@verizon.net> <4AC0CCEA.5080509@astraw.com> <4AC0CDE6.2010604@verizon.net> <4AC0D48F.90000@astraw.com> <4AC0DEE9.5060800@verizon.net> <4AC0E572.4010401@astraw.com> <4AC0ED94.5050405@verizon.net> <4AC0FF50.5020906@verizon.net> <4AC10622.5080807@astraw.com> <4AC109B8.7060807@verizon.net> <4AC115B9.6000307@astraw.com> <4AC12290.3040404@verizon.net> <4AC126F5.9040204@astraw.com> <4AC1329D.6060809@verizon.net> <4AC135B0.2060302@astraw.com> <4AC142BF.4000004@verizon.net> <4AC1A008.7000100@verizon.net> <4AC22B50.7040106@astraw.com> <4AC235B3.3080900@verizon.net> <4AC23D9C.7090605@astraw.com> <4AC255C9.70501@verizon.net> <4AC2C998.7020807@verizon.net> <4AC2E93B.9020506@astraw.com> <4AC3EC60.5060209@verizon.net> <4AC3EDF2.9050109@verizon.net> <4AC3F40F.8040505@verizon.net> Message-ID: <4AC7DAA0.7010204@verizon.net> Andrew, I see in the latest postings where setuptools is breaking with Python 2.6.3. Can you switch 'stdeb' over to using Distribute, which is being actively maintained? Regards, Gerry From strawman at astraw.com Sun Oct 4 01:58:26 2009 From: strawman at astraw.com (Andrew Straw) Date: Sat, 03 Oct 2009 16:58:26 -0700 Subject: [Distutils] bdist_deb in stdeb In-Reply-To: <4AC7DAA0.7010204@verizon.net> References: <4AB30240.8020807@verizon.net> <4AC0BD37.1010507@verizon.net> <4AC0CCEA.5080509@astraw.com> <4AC0CDE6.2010604@verizon.net> <4AC0D48F.90000@astraw.com> <4AC0DEE9.5060800@verizon.net> <4AC0E572.4010401@astraw.com> <4AC0ED94.5050405@verizon.net> <4AC0FF50.5020906@verizon.net> <4AC10622.5080807@astraw.com> <4AC109B8.7060807@verizon.net> <4AC115B9.6000307@astraw.com> <4AC12290.3040404@verizon.net> <4AC126F5.9040204@astraw.com> <4AC1329D.6060809@verizon.net> <4AC135B0.2060302@astraw.com> <4AC142BF.4000004@verizon.net> <4AC1A008.7000100@verizon.net> <4AC22B50.7040106@astraw.com> <4AC235B3.3080900@verizon.net> <4AC23D9C.7090605@astraw.com> <4AC255C9.70501@verizon.net> <4AC2C998.7020807@verizon.net> <4AC2E93B.9020506@astraw.com> <4AC3EC60.5060209@verizon.net> <4AC3EDF2.9050109@verizon.net> <4AC3F40F.8040505@verizon.net> <4AC7DAA0.7010204@verizon.net> Message-ID: <4AC7E522.5000400@astraw.com> Gerry Reno wrote: > Andrew, > I see in the latest postings where setuptools is breaking with Python > 2.6.3. Can you switch 'stdeb' over to using Distribute, which is being > actively maintained? As I understand it, installing Distribute installs a package called "setuptools". Therefore, my understanding is that you can do that yourself by simply installing Distribute. I have not tried stdeb with Distribute (yet), and don't know what changes, if any, would be necessary to switch. If you want to submit a patch against the master branch, I will definitely evaluate it. As it is, I assume that when Debian and Ubuntu package Python 2.6.3, they will also patch setuptools to be compatible, so I don't see this as a particularly pressing issue. (More pressing issues include getting a new release out.) In fact, Karmic is already using 2.6.3 and I haven't noticed any trouble there with stdeb, setuptools, and modules requiring build_ext. -Andrew From mihamina at gulfsat.mg Sun Oct 4 11:19:24 2009 From: mihamina at gulfsat.mg (Rakotomandimby Mihamina) Date: Sun, 04 Oct 2009 12:19:24 +0300 Subject: [Distutils] distribution packaging from Distribute Message-ID: <4AC8689C.2010401@gulfsat.mg> Hi all, Would you know a tutorial on Debian packaging from buildout/distribute? I mean: - I setup the buildout.cfg - I run buildout with Z or X option - I get the package I dont mind if it's not entirely Debian compatible (python and some libs duplication). Any link? Thank you. -- Architecte Informatique chez Blueline/Gulfsat: Administration Systeme, Recherche & Developpement +261 34 29 155 34 From wentland at cl.uni-heidelberg.de Sun Oct 4 13:04:56 2009 From: wentland at cl.uni-heidelberg.de (Wolodja Wentland) Date: Sun, 4 Oct 2009 13:04:56 +0200 Subject: [Distutils] Get install prefix for module at runtime In-Reply-To: <20090917190516.GC9682@kinakuta.local> References: <20090915153857.GB17381@kinakuta.local> <20090916154031.GA16640@kinakuta.local> <94bdd2610909170355tf27b3b0y25db2ca1f6d9b938@mail.gmail.com> <94bdd2610909170951j3bab982dg253bf3f930dc2c58@mail.gmail.com> <20090917190516.GC9682@kinakuta.local> Message-ID: <20091004110456.GA18456@kinakuta.local> On Thu, Sep 17, 2009 at 21:05 +0200, Wolodja Wentland wrote: > On Thu, Sep 17, 2009 at 18:51 +0200, Tarek Ziad? wrote: > > you might be able to alter it on-the-fly by overriding the build_py command > > instead of the install command > > That worked perfectly! Thanks again for the help and pointers you gave > me. I really appreciate that. > > The solution i came up with is: > > --- snip --- > class build_py(_build_py): > """build_py command > > This specific build_py command will modify module 'foo.build_config' so that it > contains information on installation prefixes afterwards. > """ > def build_module (self, module, module_file, package): > if type(package) is StringType: > _package = string.split(package, '.') > elif type(package) not in (ListType, TupleType): > raise TypeError, \ > "'package' must be a string (dot-separated), list, or tuple" > > if ( module == 'build_info' and len(_package) == 1 and > package[0] == 'foo'): > iobj = self.distribution.command_obj['install'] > > with open(module_file, 'w') as module_fp: > module_fp.write('# -*- coding: UTF-8 -*-\n\n') > module_fp.write("DATA_DIR = '%s'\n"%( > os.path.join(iobj.install_data, 'share'))) > module_fp.write("LIB_DIR = '%s'\n"%(iobj.install_lib)) > module_fp.write("SCRIPT_DIR = '%s'\n"%(iobj.install_scripts)) > > _build_py.build_module(self, module, module_file, package) > --- snip --- > > I might change the 'detect my module' logic a little because i rely on > python short circuit evaluation a bit too much. > > I have some final questions: > > 1. Is the distutils API i am using here likely to change? > > 2. Is there a better way to access install_{lib,scripts,data} than going > through the install command object available from distribution? > > 3. Could you include something like this in distutils? My idea on how to > handle this would be to define an additional argument 'foo_bar' for > core.setup() which will take a user defined dotted module name in > which information like this will be saved. > > So if a user defines: > > core.setup( ... > foo_bar : 'foo.build_info', > ... > ) > > A file lib_prefix/foo/build_info.py will be injected into the > library or an already one would be altered according to user defined > string templates. > > Something like this would end module.__file__ hacks once and for all > and is IMHO a much cleaner way to advertise this information to > libraries/application than trying to take care of this externally. Bump. Is this approach a feasible one, or am I better served by using the Distribute API, which I am not familiar at this point? What about the distutils internal I am using here? TIA Wolodja -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 835 bytes Desc: Digital signature URL: From kiorky at cryptelium.net Sun Oct 4 15:31:02 2009 From: kiorky at cryptelium.net (kiorky) Date: Sun, 04 Oct 2009 15:31:02 +0200 Subject: [Distutils] Package install failures in 2.6.3 - setuptools vs Distribute In-Reply-To: References: Message-ID: <4AC8A396.8030306@cryptelium.net> Hi, for the folks using virtualenv-distribute, i forked it to make the last 0.6.3 install instead of 0.6.1. See : http://bitbucket.org/kiorky/virtualenv-distribute/ Install it: easy_install http://distfiles.minitage.org/public/externals/minitage/virtualenv-distribute-1.3.5dev-1.zip Ned Deily a ?crit : > I'm afraid there is going to be a small deluge of very confused users > who will end up needing to install Distribute but only when they > eventually figure out why some packages with C extensions mysteriously > no longer install after they upgrade to python 2.6.3. For example, > following the package instructions to use setuptools easy_install, an > easy_install lxml fails with 2.6.3 with the very cryptic: > > File "build/bdist.macosx-10.3-fat/egg/setuptools/command/build_ext.py", > line 85, in get_ext_filename KeyError: 'etree' > > http://stackoverflow.com/questions/1512530/cant-install-lxml-python-2-6-3 > -osx-10-6-snow-leopard > > Appscript is another package that fails similarly. There must be others. > > I don't know what's to be done about this awkward incompatibility now > that 2.6.3 is out the door other than perhaps making sure maintainers of > affected packages are aware of what their users are running into and can > update their documentation. > > There is one small thing that may help some users who find their way to > http://pypi.python.org/pypi/distribute. OS X, for one, does not ship > with wget however it does have curl. Suggest adding another example: > > $ curl -O http://nightly.ziade.org/distribute_setup.py > $ python distribute_setup.py > -- -- Cordialement, KiOrKY GPG Key FingerPrint: 0x1A1194B7681112AF -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 261 bytes Desc: OpenPGP digital signature URL: From akitada at macports.org Sun Oct 4 17:10:57 2009 From: akitada at macports.org (Akira Kitada) Date: Mon, 5 Oct 2009 00:10:57 +0900 Subject: [Distutils] Packaging Distribute Message-ID: <90bb445a0910040810g48f7833cia02536e50e5c19f9@mail.gmail.com> Hi, I'm planning to make a port for Distribute to make it more accessible to users. However, thinking about up coming 0.7 release, which will break the compatibility with setuptools, I started wondering it's not so easy to make a package for distribute. If I understand correctly, Distutils 0.7 is like Python 3.x in some ways. It breaks compatibility and 0.7 and 0.6 both will continue to be developed in parallel at least a year or so. That sounds great but we can't have two versions of a Python module at the same time. Take a Jinja and Jinja2 for example. It has to have different names to make both package available at the same systen at the same time. For Distribute, there would be a few options to work around this: 1) Make 0.6 and 0.7 different package, like Distribute and Distribute2 (Jinja like solution) 2) Make 0.7 fat package, which includes 0.6(setuptools compat) features as deprecated. *) YOUR IDEAS COME HERE. Is there anyone having any other solutions? And what actually Distribute developers are thinking about this problem? # I heard Gentoo provides Distribute as setuptools but I think it's rather rude Thanks in advance. From hanno at hannosch.eu Sun Oct 4 17:28:54 2009 From: hanno at hannosch.eu (Hanno Schlichting) Date: Sun, 4 Oct 2009 17:28:54 +0200 Subject: [Distutils] Packaging Distribute In-Reply-To: <90bb445a0910040810g48f7833cia02536e50e5c19f9@mail.gmail.com> References: <90bb445a0910040810g48f7833cia02536e50e5c19f9@mail.gmail.com> Message-ID: <5cae42b20910040828l6157c3fbw51d06a61212602ce@mail.gmail.com> On Sun, Oct 4, 2009 at 5:10 PM, Akira Kitada wrote: > I'm planning to make a port for Distribute to make it more accessible to users. That's very much appreciated :) > For Distribute, there would be a few options to work around this: > > 1) Make 0.6 and 0.7 different package, like Distribute and Distribute2 > (Jinja like solution) > 2) Make 0.7 fat package, which includes 0.6(setuptools compat) > features as deprecated. > *) YOUR IDEAS COME HERE. I would just release the current 0.6 code as a port called distribute for now. We don't know yet when the 0.7 version is actually going to arrive. How exactly it looks like might very well change during the next months. So let's figure out how to deal with 0.7 once it is actually anywhere near. In general I would assume each package management system to either have support for multi-version support or have a standard way to deal with new feature releases. It's pretty common for any code to give up backwards compatibility to some degree over time. Distribute is no different here. Hanno From pje at telecommunity.com Sun Oct 4 19:07:11 2009 From: pje at telecommunity.com (P.J. Eby) Date: Sun, 04 Oct 2009 13:07:11 -0400 Subject: [Distutils] Package install failures in 2.6.3 - setuptools vs Distribute In-Reply-To: <94bdd2610910030649r431a5638y7c8b5332934f5ae@mail.gmail.com > References: <319e029f0910030515u71325b88h89037b3ea85f65b5@mail.gmail.com> <94bdd2610910030649r431a5638y7c8b5332934f5ae@mail.gmail.com> Message-ID: <20091004170717.9C52C3A4748@sparrow.telecommunity.com> At 03:49 PM 10/3/2009 +0200, Tarek Ziad? wrote: >Notice that this has been fixed in Ubuntu already with a patched >version of setuptools Is the patch or an equivalent already in the setuptools tracker? And if not, can someone please post it there? Thanks. From setuptools at bugs.python.org Sun Oct 4 20:09:04 2009 From: setuptools at bugs.python.org (Ned Deily) Date: Sun, 04 Oct 2009 18:09:04 +0000 Subject: [Distutils] [issue85] setuptools builds of extension modules fail with Python 2.6.3 In-Reply-To: <1254679743.94.0.794526085525.issue85@psf.upfronthosting.co.za> Message-ID: <1254679743.94.0.794526085525.issue85@psf.upfronthosting.co.za> New submission from Ned Deily : A change in distutils distributed with Python 2.6.3 breaks setuptools. Builds of extension modules fail with n get_ext_filename ext = self.ext_map[fullname] KeyError: ''. An analysis and patch is documented in the Distribute tracker issue 41: http://bitbucket.org/tarek/distribute/issue/41/ ---------- messages: 357 nosy: nad priority: critical status: unread title: setuptools builds of extension modules fail with Python 2.6.3 _______________________________________________ Setuptools tracker _______________________________________________ From strawman at astraw.com Sun Oct 4 20:13:43 2009 From: strawman at astraw.com (Andrew Straw) Date: Sun, 04 Oct 2009 11:13:43 -0700 Subject: [Distutils] [ANN] stdeb 0.3.2 and 0.4.1 Message-ID: <4AC8E5D7.8010602@astraw.com> stdeb produces Debian source packages from Python packages via a new distutils command, sdist_dsc. Automatic defaults are provided for the Debian package, but many aspects of the resulting package can be customized via a configuration file. An additional command, bdist_deb, creates a Debian binary package, a .deb file. stdeb: http://github.com/astraw/stdeb This email announces releases 0.3.2 and 0.4.1. release 0.4.1 download: http://pypi.python.org/pypi/stdeb/0.4.1 release 0.3.2 download: http://pypi.python.org/pypi/stdeb/0.3.2 Release 0.4.1 adds new features, conveniences and bug fixes to the recently released, but unannounced 0.4 release. These two releases move stdeb to use debhelper 7 (and therefore require at least Ubuntu Intrepid, Debian Lenny, or the use of backports). This use of debhelper 7 allows the auto-generated debian/rules file to be greatly simplified. stdeb 0.4 also switches to python-support from pycentral. If you do not need to migrate packages from earlier versions of stdeb, it is recommended to use the "--pycentral-backwards-compatibility=false" command-line argument. See the changelog[1] and release notes[2] for more information. [1] http://github.com/astraw/stdeb/blob/release-0.4.1/CHANGELOG.txt [2] http://github.com/astraw/stdeb/blob/release-0.4.1/RELEASE_NOTES.txt Release 0.3.2 (and the unannounced 0.3.1) are maintenance releases for the old-stable branch, which works with older distributions such as Ubuntu Hardy. These releases added the bdist_deb command. See the changelog[3] for more information. [3] http://github.com/astraw/stdeb/blob/release-0.3.2/CHANGELOG.txt As always, the venue for discussion of stdeb is the distutils mailing list at http://mail.python.org/mailman/listinfo/distutils-sig Thanks to all who have helped stdeb for these and previous releases: Authors ------- * Andrew Straw * Pedro Algarvio, aka, s0undt3ch * Gerry Reno (initial bdist_deb implementation) Additional Credits ------------------ * Zooko O'Whielacronx for the autofind-depends patch * Brett (last name unknown) for the --ignore-install-requires patch * Ximin Luo for a bug fix * Alexander D. Sedov for bug fixes and suggestions * GitHub for hosting services. * WebFaction (aka python-hosting) for previous hosting services. -Andrew From ziade.tarek at gmail.com Sun Oct 4 22:23:21 2009 From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=) Date: Sun, 4 Oct 2009 22:23:21 +0200 Subject: [Distutils] distribution packaging from Distribute In-Reply-To: <4AC8689C.2010401@gulfsat.mg> References: <4AC8689C.2010401@gulfsat.mg> Message-ID: <94bdd2610910041323n7ae6e543g814e5f51eceac469@mail.gmail.com> On Sun, Oct 4, 2009 at 11:19 AM, Rakotomandimby Mihamina wrote: > Hi all, > Hi > Would you know a tutorial on Debian packaging from buildout/distribute? > I mean: > - I setup the buildout.cfg > - I run buildout with Z or X option > - I get the package What package exactly ? I am not sure to understand Tarek From ziade.tarek at gmail.com Sun Oct 4 22:41:06 2009 From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=) Date: Sun, 4 Oct 2009 22:41:06 +0200 Subject: [Distutils] Packaging Distribute In-Reply-To: <5cae42b20910040828l6157c3fbw51d06a61212602ce@mail.gmail.com> References: <90bb445a0910040810g48f7833cia02536e50e5c19f9@mail.gmail.com> <5cae42b20910040828l6157c3fbw51d06a61212602ce@mail.gmail.com> Message-ID: <94bdd2610910041341v2c2483b1jd6c8f4b5707d789a@mail.gmail.com> On Sun, Oct 4, 2009 at 5:28 PM, Hanno Schlichting wrote: >> For Distribute, there would be a few options to work around this: >> >> 1) Make 0.6 and 0.7 different package, like Distribute and Distribute2 >> (Jinja like solution) >> 2) Make 0.7 fat package, which includes 0.6(setuptools compat) >> features as deprecated. >> *) YOUR IDEAS COME HERE. > > I would just release the current 0.6 code as a port called distribute for now. > > We don't know yet when the 0.7 version is actually going to arrive. > How exactly it looks like might very well change during the next > months. So let's figure out how to deal with 0.7 once it is actually > anywhere near. > > In general I would assume each package management system to either > have support for multi-version support or have a standard way to deal > with new feature releases. It's pretty common for any code to give up > backwards compatibility to some degree over time. Distribute is no > different here. Well in any case, what is sure is that we need to maintain a working 0.6 distribution to allow projects like pip or virtualenv to switch to Distribute if they want. For 0.7, what is also required is to be able to install it together with 0.6 because the 0.7 version will provide new features/changes that a project might want to use while working with 0.6, and other projects requiring 0.6 would be broken in case 0.7 removes 0.6. For the long term I find the fat package solution we've discussed earlier on IRC nicer because it will probably allow us to fix some bugs in the 0.6 part by using some bits of 0.7 underneath. Now for shipping Distribute 0.6 in MacPorts, I don't know how the dependency system works there, but the best way would be to remove the setuptools distribution as a pre-request of Distribute installation (and vice-versa), The other way would be to use Distribute instead of Setuptools for what the packaging system is calling "setuptools". That's pretty much what is happening in Gentoo (arch) and UHU-Linux (dev), right now, and that's what we might do in Debian and Ubuntu (that's just a thaught at this time in there). In any case, the chosen solution will be far from perfect. Cheers Tarek -- Tarek Ziad? | http://ziade.org |????????????! |???????????? From sridharr at activestate.com Sun Oct 4 23:52:25 2009 From: sridharr at activestate.com (Sridhar Ratnakumar) Date: Sun, 04 Oct 2009 14:52:25 -0700 Subject: [Distutils] Packaging Distribute In-Reply-To: <94bdd2610910041341v2c2483b1jd6c8f4b5707d789a@mail.gmail.com> References: <90bb445a0910040810g48f7833cia02536e50e5c19f9@mail.gmail.com> <5cae42b20910040828l6157c3fbw51d06a61212602ce@mail.gmail.com> <94bdd2610910041341v2c2483b1jd6c8f4b5707d789a@mail.gmail.com> Message-ID: On Sun, 04 Oct 2009 13:41:06 -0700, Tarek Ziad? wrote: > The other way would be to use Distribute instead of Setuptools for > what the packaging system is calling "setuptools". That's pretty > much what is happening in Gentoo (arch) and UHU-Linux (dev), > right now Interesting. Gentoo uses distribute but retains the name 'setuptools'? DEPEND="!! References: <90bb445a0910040810g48f7833cia02536e50e5c19f9@mail.gmail.com> <5cae42b20910040828l6157c3fbw51d06a61212602ce@mail.gmail.com> <94bdd2610910041341v2c2483b1jd6c8f4b5707d789a@mail.gmail.com> Message-ID: <4AC9211B.5060600@cryptelium.net> Sridhar Ratnakumar a ?crit : > On Sun, 04 Oct 2009 13:41:06 -0700, Tarek Ziad? > wrote: > >> The other way would be to use Distribute instead of Setuptools for >> what the packaging system is calling "setuptools". That's pretty >> much what is happening in Gentoo (arch) and UHU-Linux (dev), >> right now > > Interesting. Gentoo uses distribute but retains the name 'setuptools'? > > DEPEND="!! > Ah. But what if PJE releases setuptools with the *same* version number > 0.6.3? What would the gentoo folks do in order to get the new setuptools The src_uri explicitly target to DISTRIBUTE. grep SRC /usr/portage/dev-python/setuptools/setuptools-0.6.1.ebuild SRC_URI="http://pypi.python.org/packages/source/d/distribute/distribute-${PV}.tar.gz" If they want to reswitch to bare metal setuptools, they will update this url. > release in their packaging system? Or did they make a decision of > totally dropping setuptools from their repository? > > -srid > > _______________________________________________ > Distutils-SIG maillist - Distutils-SIG at python.org > http://mail.python.org/mailman/listinfo/distutils-sig -- -- Cordialement, KiOrKY GPG Key FingerPrint: 0x1A1194B7681112AF -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 261 bytes Desc: OpenPGP digital signature URL: From mihamina at gulfsat.mg Mon Oct 5 05:13:42 2009 From: mihamina at gulfsat.mg (Rakotomandimby Mihamina) Date: Mon, 05 Oct 2009 06:13:42 +0300 Subject: [Distutils] distribution packaging from Distribute In-Reply-To: <94bdd2610910041323n7ae6e543g814e5f51eceac469@mail.gmail.com> References: <4AC8689C.2010401@gulfsat.mg> <94bdd2610910041323n7ae6e543g814e5f51eceac469@mail.gmail.com> Message-ID: <4AC96466.3020506@gulfsat.mg> 10/04/2009 11:23 PM, Tarek Ziad?: > What package exactly ? I am not sure to understand Everything installable with buildout. Example: Plone. I want a debian Plone package. I configure buildout properly, run it with some appropriate options and abracadabra the .deb is there. The same with grok, and so on... -- Architecte Informatique chez Blueline/Gulfsat: Administration Systeme, Recherche & Developpement +261 34 29 155 34 From jeff at taupro.com Mon Oct 5 11:40:05 2009 From: jeff at taupro.com (Jeff Rush) Date: Mon, 05 Oct 2009 04:40:05 -0500 Subject: [Distutils] Package install failures in 2.6.3 - setuptools vs Distribute In-Reply-To: <319e029f0910031308u4182dff2w89760afb4122cf2c@mail.gmail.com> References: <319e029f0910030515u71325b88h89037b3ea85f65b5@mail.gmail.com> <94bdd2610910030649r431a5638y7c8b5332934f5ae@mail.gmail.com> <319e029f0910031308u4182dff2w89760afb4122cf2c@mail.gmail.com> Message-ID: <4AC9BEF5.7080204@taupro.com> Lennart Regebro wrote: > 2009/10/3 Ned Deily : >> This is not a good experience for users. Unless I'm missing something >> (and I hope I am), this issue really can't be hand-waved away. > > It's unfortunate that this comes in a minor release. Very unfortunate, as in, it should NOT have happened. And *especially* without any announcement on python.org or mention on the python-committers list of something this major. > But at the same > time we can hardly avoid fixing bugs just because setuptools isn't > getting updated at the moment. It's a lose-lose situation. Setuptools is hardly a minor software tool. Considering that setuptools is a major focus and key technology of this group, those making changes to distutils should have tested against setuptools and devised a solution providing backward compatibility, along with deprecation warnings. Lennart and Tarek, I know you at least are familiar with this valuable practice in Zope for years. At the least, those making changes to Distutils should, after testing against Setuptools, widely announced this was coming. It does not reflect positively on the Distribute team and even hints of under-the-table intentionality in forcing the community to abandon use of setuptools. Distribute should win because of superior technology not forced migration. > As I see it this will speed up adaptation of Distribute. Word will > spread. It's not ideal, but then it's not a perfect world. Distribute is very new and there are many folk who will not be adopting it until it has been out for quite some time. It is a fact of the conservative nature of some development teams. If setuptools were an optional, little-used technology in Python it would not be a big deal but it isn't. It is not appropriate to -force- users to adopt a particular branch of a fork, except perhaps in the move from Python 2.x to 3.x where major changes are anticipated and there was no setuptools for 3.x anyway. Considering that 2.6.3 is messed up in other ways, like displaying the SVN rc1 tag and a broken logging module, and probably will be re-released at 2.6.4 very shortly, perhaps we can get this -at least- into the Python docs and maybe introduce backward compatible hooks into distutils to support setuptools. -Jeff From regebro at gmail.com Mon Oct 5 11:53:29 2009 From: regebro at gmail.com (Lennart Regebro) Date: Mon, 5 Oct 2009 11:53:29 +0200 Subject: [Distutils] Package install failures in 2.6.3 - setuptools vs Distribute In-Reply-To: <4AC9BEF5.7080204@taupro.com> References: <319e029f0910030515u71325b88h89037b3ea85f65b5@mail.gmail.com> <94bdd2610910030649r431a5638y7c8b5332934f5ae@mail.gmail.com> <319e029f0910031308u4182dff2w89760afb4122cf2c@mail.gmail.com> <4AC9BEF5.7080204@taupro.com> Message-ID: <319e029f0910050253x1ad1cc7eg60e7f11b3f9f72b8@mail.gmail.com> 2009/10/5 Jeff Rush : > Very unfortunate, as in, it should NOT have happened. ?And *especially* > without any announcement on python.org or mention on the > python-committers list of something this major. Well "this major"... It's a bug fix that breaks a setuptools monkey-patch. But yes, it was discovered before release, and maybe it should have been discussed, I'm not on python-dev anymore. > and even hints of > under-the-table intentionality in forcing the community to abandon use > of setuptools. There are no such hints anywhere. > ?Distribute should win because of superior technology not > forced migration. That makes no sense. You move to distribute because you have to, because setuptools is buggy and not updated. Until people encounter bugs in setuptools, or need Python 3 support, they are not likely to move to Distribute. There is no other reason than forced migration. Also, there is no "win" or "lose". This is not a competition. > Distribute is very new and there are many folk who will not be adopting > it until it has been out for quite some time. Nobody will adopt it until they are forced to. This unfortunate bug means people are forced to quicker than expected. I don't think that's an actual problem. >?It is a fact of the > conservative nature of some development teams. Conservative development teams are not likely to either use Subversion 1.6 or Python 2.6.3, so they are not affected by any of the major setuptools problems. I would have expected people starting to get forced to Distribute when major distros where shipping with subversion 1.6. Now it's going to be when they ship with Python 2.6.3 instead. I fail to see how this is a big disaster in any way. Yes, it's not perfect, and yeah, maybe there should have been big warning signs somewhere. But we can NOT leave bugs in Python just because setuptools isn't getting updated. Setuptools has already been a break on Python 3 development, are we gonna lets it be a break on Python 2 bugfixes too? -- Lennart Regebro: Python, Zope, Plone, Grok http://regebro.wordpress.com/ +33 661 58 14 64 From ziade.tarek at gmail.com Mon Oct 5 13:01:33 2009 From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=) Date: Mon, 5 Oct 2009 13:01:33 +0200 Subject: [Distutils] distribution packaging from Distribute In-Reply-To: <4AC96466.3020506@gulfsat.mg> References: <4AC8689C.2010401@gulfsat.mg> <94bdd2610910041323n7ae6e543g814e5f51eceac469@mail.gmail.com> <4AC96466.3020506@gulfsat.mg> Message-ID: <94bdd2610910050401s36c0bc03ud85e81ff67ae2c7b@mail.gmail.com> On Mon, Oct 5, 2009 at 5:13 AM, Rakotomandimby Mihamina wrote: > 10/04/2009 11:23 PM, Tarek Ziad?: >> >> What package exactly ? I am not sure to understand > > Everything installable with buildout. > > Example: Plone. I want a debian Plone package. > I configure buildout properly, run it with some appropriate options > and abracadabra the .deb is there. > The same with grok, and so on... Ok I see. I know Jim does this with RPMs using zc.resourcerelease. I have planned to do it on my side once for debian package using a recipe that was building the debian tree once the buildout was made, but this work was not finished. I can send you what I have if it can help you Regards Tarek -- Tarek Ziad? | http://ziade.org |????????????! |???????????? From mihamina at gulfsat.mg Mon Oct 5 13:08:31 2009 From: mihamina at gulfsat.mg (Rakotomandimby Mihamina) Date: Mon, 05 Oct 2009 14:08:31 +0300 Subject: [Distutils] distribution packaging from Distribute In-Reply-To: <94bdd2610910050401s36c0bc03ud85e81ff67ae2c7b@mail.gmail.com> References: <4AC8689C.2010401@gulfsat.mg> <94bdd2610910041323n7ae6e543g814e5f51eceac469@mail.gmail.com> <4AC96466.3020506@gulfsat.mg> <94bdd2610910050401s36c0bc03ud85e81ff67ae2c7b@mail.gmail.com> Message-ID: <4AC9D3AF.9070902@gulfsat.mg> 10/05/2009 02:01 PM, Tarek Ziad?: > I have planned to do it on my side once for debian package using a > recipe that was building the debian tree once the buildout was made, but this work > was not finished. I can send you what I have if it can help you Please, do. -- Architecte Informatique chez Blueline/Gulfsat: Administration Systeme, Recherche & Developpement +261 34 29 155 34 From ronaldoussoren at mac.com Mon Oct 5 13:30:52 2009 From: ronaldoussoren at mac.com (Ronald Oussoren) Date: Mon, 05 Oct 2009 13:30:52 +0200 Subject: [Distutils] Package install failures in 2.6.3 - setuptools vs Distribute In-Reply-To: <28320134263785163387302965410901866648-Webmail@me.com> References: <28320134263785163387302965410901866648-Webmail@me.com> Message-ID: <72523684218386049404529325276719797765-Webmail@me.com> On Monday, 05 October, 2009, at 11:53AM, "Lennart Regebro" wrote: >2009/10/5 Jeff Rush : >> Very unfortunate, as in, it should NOT have happened. ?And *especially* >> without any announcement on python.org or mention on the >> python-committers list of something this major. > >Well "this major"... It's a bug fix that breaks a setuptools monkey-patch. >But yes, it was discovered before release, and maybe it should have >been discussed, I'm not on python-dev anymore. I agree with Jeff that this shouldn't have happened, or should at the very least have been documented in the release notes for 2.6.3. I know of several users of Python that have been bitten by this (they installed 2.6.3 and now easy_install doesn't work anymore for them). For beginners this issue is a showstopper that they cannot resolve without help. > > >> Distribute is very new and there are many folk who will not be adopting >> it until it has been out for quite some time. > >Nobody will adopt it until they are forced to. This unfortunate bug >means people are forced to quicker than expected. I don't think that's >an actual problem. This is a problem, it means 2.6.3 is not a simple drop-in replacement for 2.6.2 but requires the replacement of another component as well. That can be a problem in organizations with strict configuration management where you cannot install new software without going to lots of red tape (and that might involve lawyers when you install a new package instead of upgrading an existing one). Ronald From nad at acm.org Mon Oct 5 13:33:08 2009 From: nad at acm.org (Ned Deily) Date: Mon, 05 Oct 2009 04:33:08 -0700 Subject: [Distutils] Package install failures in 2.6.3 - setuptools vs Distribute References: <319e029f0910030515u71325b88h89037b3ea85f65b5@mail.gmail.com> <94bdd2610910030649r431a5638y7c8b5332934f5ae@mail.gmail.com> <319e029f0910031308u4182dff2w89760afb4122cf2c@mail.gmail.com> <4AC9BEF5.7080204@taupro.com> Message-ID: In article <4AC9BEF5.7080204 at taupro.com>, Jeff Rush wrote: > Lennart Regebro wrote: > > 2009/10/3 Ned Deily : > >> This is not a good experience for users. Unless I'm missing something > >> (and I hope I am), this issue really can't be hand-waved away. > > > > It's unfortunate that this comes in a minor release. > Very unfortunate, as in, it should NOT have happened. And *especially* > without any announcement on python.org or mention on the > python-committers list of something this major. > > But at the same > > time we can hardly avoid fixing bugs just because setuptools isn't > > getting updated at the moment. It's a lose-lose situation. > Setuptools is hardly a minor software tool. Considering that setuptools > is a major focus and key technology of this group, those making changes > to distutils should have tested against setuptools and devised a > solution providing backward compatibility, along with deprecation > warnings. Lennart and Tarek, I know you at least are familiar with this > valuable practice in Zope for years. > > At the least, those making changes to Distutils should, after testing > against Setuptools, widely announced this was coming. It does not > reflect positively on the Distribute team and even hints of > under-the-table intentionality in forcing the community to abandon use > of setuptools. Distribute should win because of superior technology not > forced migration. > > > As I see it this will speed up adaptation of Distribute. Word will > > spread. It's not ideal, but then it's not a perfect world. > > Distribute is very new and there are many folk who will not be adopting > it until it has been out for quite some time. It is a fact of the > conservative nature of some development teams. If setuptools were an > optional, little-used technology in Python it would not be a big deal > but it isn't. It is not appropriate to -force- users to adopt a > particular branch of a fork, except perhaps in the move from Python 2.x > to 3.x where major changes are anticipated and there was no setuptools > for 3.x anyway. Just another data point: the MacPorts distrbution has just upgraded their python26 to 2.6.3 and now a number of their py26 packages can't be installed because they use setuptools (or depend on another package that uses setuptools) to build C extensions and those fail due to the change in distutils: among others, ipython, numpy, pyobjc2, twisted. Until the MacPorts folks recognize and push out a fix, using the setuptools easy_install to install distribute works - if you know the trick. > Considering that 2.6.3 is messed up in other ways, like displaying the > SVN rc1 tag and a broken logging module, and probably will be > re-released at 2.6.4 very shortly, perhaps we can get this -at least- > into the Python docs and maybe introduce backward compatible hooks into > distutils to support setuptools. I've opened an issue of the main Python issue tracker outlining the problem, primarily for the benefit of affected users who search the tracker: http://bugs.python.org/issue7064 -- Ned Deily, nad at acm.org From ziade.tarek at gmail.com Mon Oct 5 13:54:54 2009 From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=) Date: Mon, 5 Oct 2009 13:54:54 +0200 Subject: [Distutils] Package install failures in 2.6.3 - setuptools vs Distribute In-Reply-To: <72523684218386049404529325276719797765-Webmail@me.com> References: <28320134263785163387302965410901866648-Webmail@me.com> <72523684218386049404529325276719797765-Webmail@me.com> Message-ID: <94bdd2610910050454p41c6f86ex2d97502133f5562a@mail.gmail.com> On Mon, Oct 5, 2009 at 1:30 PM, Ronald Oussoren wrote: >>Nobody will adopt it until they are forced to. This unfortunate bug >>means people are forced to quicker than expected. I don't think that's >>an actual problem. > > This is a problem, it means 2.6.3 is not a simple drop-in replacement for 2.6.2 but requires the replacement of another component as well. ?That can be a problem in organizations with strict configuration management where you cannot install new software without going to lots of red tape (and that might involve lawyers when you install a new package instead of upgrading an existing one). What is your solution ? Setuptools is not part of the standard library and monkey-patch Distutils. Its development has been discontinued for over a year. Everytime Distutils is changed for anything, wether it's a bug fix or not, Setuptools can be broken. Now I realize that some folks wants Distutils to be aware of that and be backward-compatible with Setuptools monkey-patches ? If that so, its means that Distutils can't be fixed and its code can't be changed at all, if Setuptools is not changed in the meantime. We can't force the Setuptools maintainer to do so. Which package is supposely on the top of which one ? A drop in replacement from 2.6.2 to 2.6.3 probably breaks some other softwares out there that are monkey patching some internals of Python, and the way to go is to fix those package. At the end, some folks here sounds like it's Distutils fault if Setuptools is broken, and it can't be simply because Setuptools is not maintained anymore and monkey patches Distutils. Like for the svn 1.6 compat problem we had earlier this year, this problem is a few line changes in Setuptools, it's an 1 hour work. If your company upgrades to Python 2.6.3 it can also upgrade to an hypothetical Setuptools 0.6c..10 ? Tarek -- Tarek Ziad? | http://ziade.org |????????????! |???????????? From kiorky at cryptelium.net Mon Oct 5 14:35:51 2009 From: kiorky at cryptelium.net (kiorky) Date: Mon, 05 Oct 2009 14:35:51 +0200 Subject: [Distutils] distribution packaging from Distribute In-Reply-To: <94bdd2610910050401s36c0bc03ud85e81ff67ae2c7b@mail.gmail.com> References: <4AC8689C.2010401@gulfsat.mg> <94bdd2610910041323n7ae6e543g814e5f51eceac469@mail.gmail.com> <4AC96466.3020506@gulfsat.mg> <94bdd2610910050401s36c0bc03ud85e81ff67ae2c7b@mail.gmail.com> Message-ID: <4AC9E827.7000002@cryptelium.net> Even if it s true that it is on package mainteners to fix their code to run on current underlying python version, there are many chances they won't. Not to blame themselves, but: - How many packages are/had been in bad shape towards setuptools right now (and i totally dont want to blame those great softwares, dont flame !!!) (PIL, openerp, reportlab as examples,), this will not be a great surprise that the new Distutils/Distribute adoption will do the same as python2/python3? - How many maintainers / enterprises could not afford the time to port their softwares, their releasing and maintenance procedures and so on? - How many users will be lost in the simple question to how install their softwares. Even if setuptools is broken and of course it is in some way, i think that totally breaking the compatiblity will make lot of users to leave the boat. Maintening a compatibility layer, even broken in single cases must be better. Even if we have to fix that layer time to time. Note that it would for example be good that those ".recipe.egg" buildout recipes behave the same as they were before with equal or more things but not less. That is what i call a good improvement to distribute/distutils. Tarek Ziad? a ?crit : > On Mon, Oct 5, 2009 at 5:13 AM, Rakotomandimby Mihamina > wrote: >> 10/04/2009 11:23 PM, Tarek Ziad?: >>> What package exactly ? I am not sure to understand >> Everything installable with buildout. >> >> Example: Plone. I want a debian Plone package. >> I configure buildout properly, run it with some appropriate options >> and abracadabra the .deb is there. >> The same with grok, and so on... > > Ok I see. I know Jim does this with RPMs using zc.resourcerelease. > > I have planned to do it on my side once for debian package using a > recipe that was > building the debian tree once the buildout was made, but this work > was not finished. I can send you what I have if it can help you > > Regards > Tarek > -- -- Cordialement, KiOrKY GPG Key FingerPrint: 0x1A1194B7681112AF -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 261 bytes Desc: OpenPGP digital signature URL: From regebro at gmail.com Mon Oct 5 13:44:02 2009 From: regebro at gmail.com (Lennart Regebro) Date: Mon, 5 Oct 2009 13:44:02 +0200 Subject: [Distutils] Package install failures in 2.6.3 - setuptools vs Distribute In-Reply-To: <72523684218386049404529325276719797765-Webmail@me.com> References: <28320134263785163387302965410901866648-Webmail@me.com> <72523684218386049404529325276719797765-Webmail@me.com> Message-ID: <319e029f0910050444t25761d79gce49d7ef13d98191@mail.gmail.com> 2009/10/5 Ronald Oussoren : > This is a problem, it means 2.6.3 is not a simple drop-in replacement for 2.6.2 but requires the replacement of another component as well. ?That can be a problem in organizations with strict configuration management where you cannot install new software without going to lots of red tape (and that might involve lawyers when you install a new package instead of upgrading an existing one). Sure, but that would have happened sooner or later anyway. Is it really so bad that it happens now instead of say, in half a year? I don't see why it's such a big deal. Sorry, but I don't. -- Lennart Regebro: Python, Zope, Plone, Grok http://regebro.wordpress.com/ +33 661 58 14 64 From barry at python.org Mon Oct 5 15:38:06 2009 From: barry at python.org (Barry Warsaw) Date: Mon, 5 Oct 2009 09:38:06 -0400 Subject: [Distutils] Package install failures in 2.6.3 - setuptools vs Distribute In-Reply-To: <4AC9BEF5.7080204@taupro.com> References: <319e029f0910030515u71325b88h89037b3ea85f65b5@mail.gmail.com> <94bdd2610910030649r431a5638y7c8b5332934f5ae@mail.gmail.com> <319e029f0910031308u4182dff2w89760afb4122cf2c@mail.gmail.com> <4AC9BEF5.7080204@taupro.com> Message-ID: <0A60A4FF-D32B-4D35-B607-1A02C838F743@python.org> On Oct 5, 2009, at 5:40 AM, Jeff Rush wrote: > Very unfortunate, as in, it should NOT have happened. And > *especially* > without any announcement on python.org or mention on the > python-committers list of something this major. [...] > Considering that 2.6.3 is messed up in other ways, like displaying the > SVN rc1 tag and a broken logging module, and probably will be > re-released at 2.6.4 very shortly, perhaps we can get this -at least- > into the Python docs and maybe introduce backward compatible hooks > into > distutils to support setuptools. I apologize for my part in this, but moving forward I think that if it's possible to patch and release a setuptools that works with Python 2.6.3 /and/ earlier Python 2.6.x's, then that should happen asap. If that's not possible, then we might need to revert the distutils change in a quick Python 2.6.4. -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: PGP.sig Type: application/pgp-signature Size: 832 bytes Desc: This is a digitally signed message part URL: From zooko at zooko.com Mon Oct 5 16:02:49 2009 From: zooko at zooko.com (Zooko Wilcox-O'Hearn) Date: Mon, 5 Oct 2009 08:02:49 -0600 Subject: [Distutils] Package install failures in 2.6.3 - setuptools vs Distribute In-Reply-To: <0A60A4FF-D32B-4D35-B607-1A02C838F743@python.org> References: <319e029f0910030515u71325b88h89037b3ea85f65b5@mail.gmail.com> <94bdd2610910030649r431a5638y7c8b5332934f5ae@mail.gmail.com> <319e029f0910031308u4182dff2w89760afb4122cf2c@mail.gmail.com> <4AC9BEF5.7080204@taupro.com> <0A60A4FF-D32B-4D35-B607-1A02C838F743@python.org> Message-ID: <53902AAD-D8B3-490B-9228-56416DE5E6D3@zooko.com> On Monday,2009-10-05, at 7:38 , Barry Warsaw wrote: > I apologize for my part in this, but moving forward I think that if > it's possible to patch and release a setuptools that works with > Python 2.6.3 /and/ earlier Python 2.6.x's, then that should happen > asap. If that's not possible, then we might need to revert the > distutils change in a quick Python 2.6.4. Why not move ahead with both? Then, for example, people who do have Python 2.6.3 installed and get the latest setuptools will be okay, and also people who have setuptools-0.6c9 installed and get the latest Python. There are a lot of people who have constraints on what they they can deploy and when, so it wouldn't hurt to have both fixes available. (Also, one of the fixes might not get done in a timely way.) Thanks, Zooko From ziade.tarek at gmail.com Mon Oct 5 16:08:17 2009 From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=) Date: Mon, 5 Oct 2009 16:08:17 +0200 Subject: [Distutils] Package install failures in 2.6.3 - setuptools vs Distribute In-Reply-To: <0A60A4FF-D32B-4D35-B607-1A02C838F743@python.org> References: <319e029f0910030515u71325b88h89037b3ea85f65b5@mail.gmail.com> <94bdd2610910030649r431a5638y7c8b5332934f5ae@mail.gmail.com> <319e029f0910031308u4182dff2w89760afb4122cf2c@mail.gmail.com> <4AC9BEF5.7080204@taupro.com> <0A60A4FF-D32B-4D35-B607-1A02C838F743@python.org> Message-ID: <94bdd2610910050708w366bb7fbi67749a497fd5df82@mail.gmail.com> On Mon, Oct 5, 2009 at 3:38 PM, Barry Warsaw wrote: > On Oct 5, 2009, at 5:40 AM, Jeff Rush wrote: > >> Very unfortunate, as in, it should NOT have happened. ?And *especially* >> without any announcement on python.org or mention on the >> python-committers list of something this major. > > [...] > >> Considering that 2.6.3 is messed up in other ways, like displaying the >> SVN rc1 tag and a broken logging module, and probably will be >> re-released at 2.6.4 very shortly, perhaps we can get this -at least- >> into the Python docs and maybe introduce backward compatible hooks into >> distutils to support setuptools. > > I apologize for my part in this, but moving forward I think that if it's > possible to patch and release a setuptools that works with Python 2.6.3 > /and/ earlier Python 2.6.x's, then that should happen asap. ?If that's not > possible, then we might need to revert the distutils change in a quick > Python 2.6.4. It's technically possible to fix Setuptools. We did this fix on Distribute, and the patch is 2 lines long. But it's just a matter of having the maintainer doing it. A few months ago we couldn't make him fix and release the bug that made setuptools fail with svn 1.6, and the year before it took several months to get it fixed for svn 1.5 (a one line, not risky change !!!) That's why we have forked and created Distribute, to provide bug fixes. If PJE is not concerned anymore by the maintenance, imho he should let someone that is willing to do it take over the maintenance of his package to fix this (and the other problems). That is not a new problem. Beware that I don't want to run in any new vicious thread here: I had my share of those. So about taking over Setuptools maintenance : 1/ I am not saying it should be me, and I am not saying that I am offended that PJE didn't open the maintenance of setuptools to me. I think he should trust the community and let the maintenance of setuptools be done by all the people that are actively working on the topic. 2/ No, as someone told me in IRC, that's not an evil plan of mine to make people switch to Distribute. This is not in our interest, it's a loss-loss situation. Now I am strongly opposed to revert any bug fix change in Distutils just because it breaks Setuptools, which is unmaintained since a year. We have been struggling over a year with this issue. And we are still struggling because we have to work in a fork to try to provide solutions for the community, with a lot of bootstrapping issues. Now I am astonished that we are talking about reverting changes in Distutils that were done for bugfixes, for a third party package that does monkey patches on Distutils. If this choice wins here, it means that setuptools and the stdlib are tied together, and that the setuptools package should be integrated to the stdlib *immediatly*. Tarek -- Tarek Ziad? | http://ziade.org |????????????! |???????????? From ziade.tarek at gmail.com Mon Oct 5 16:11:26 2009 From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=) Date: Mon, 5 Oct 2009 16:11:26 +0200 Subject: [Distutils] Package install failures in 2.6.3 - setuptools vs Distribute In-Reply-To: <53902AAD-D8B3-490B-9228-56416DE5E6D3@zooko.com> References: <319e029f0910030515u71325b88h89037b3ea85f65b5@mail.gmail.com> <94bdd2610910030649r431a5638y7c8b5332934f5ae@mail.gmail.com> <319e029f0910031308u4182dff2w89760afb4122cf2c@mail.gmail.com> <4AC9BEF5.7080204@taupro.com> <0A60A4FF-D32B-4D35-B607-1A02C838F743@python.org> <53902AAD-D8B3-490B-9228-56416DE5E6D3@zooko.com> Message-ID: <94bdd2610910050711i5d429043he3288986fe470516@mail.gmail.com> On Mon, Oct 5, 2009 at 4:02 PM, Zooko Wilcox-O'Hearn wrote: > On Monday,2009-10-05, at 7:38 , Barry Warsaw wrote: > >> I apologize for my part in this, but moving forward I think that if it's >> possible to patch and release a setuptools that works with Python 2.6.3 >> /and/ earlier Python 2.6.x's, then that should happen asap. ?If that's not >> possible, then we might need to revert the distutils change in a quick >> Python 2.6.4. > > Why not move ahead with both? ?Then, for example, people who do have Python > 2.6.3 installed and get the latest setuptools will be okay, and also people > who have setuptools-0.6c9 installed and get the latest Python. ?There are a > lot of people who have constraints on what they they can deploy and when, so > it wouldn't hurt to have both fixes available. So are you saying that in an environment where you are allowed to install Python 2.6.3, you will not be allowed to install an hypothetical setuptools-0.6c10 (or a Distribute 0.6.3) ? Tarek From ssteinerx at gmail.com Mon Oct 5 16:21:19 2009 From: ssteinerx at gmail.com (ssteinerX@gmail.com) Date: Mon, 5 Oct 2009 10:21:19 -0400 Subject: [Distutils] Package install failures in 2.6.3 - setuptools vs Distribute In-Reply-To: <319e029f0910050444t25761d79gce49d7ef13d98191@mail.gmail.com> References: <28320134263785163387302965410901866648-Webmail@me.com> <72523684218386049404529325276719797765-Webmail@me.com> <319e029f0910050444t25761d79gce49d7ef13d98191@mail.gmail.com> Message-ID: <9EDFE750-4102-49A4-8B35-34E6E7AE205C@gmail.com> On Oct 5, 2009, at 7:44 AM, Lennart Regebro wrote: > 2009/10/5 Ronald Oussoren : >> This is a problem, it means 2.6.3 is not a simple drop-in >> replacement for 2.6.2 but requires the replacement of another >> component as well. That can be a problem in organizations with >> strict configuration management where you cannot install new >> software without going to lots of red tape (and that might involve >> lawyers when you install a new package instead of upgrading an >> existing one). > > Sure, but that would have happened sooner or later anyway. Is it > really so bad that it happens now instead of say, in half a year? I > don't see why it's such a big deal. Sorry, but I don't. It's not the happening, it's the happening without a deprecation, warnings, announcements etc. deserved by a change that affects so much of the Python eco-system. S From ziade.tarek at gmail.com Mon Oct 5 16:32:06 2009 From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=) Date: Mon, 5 Oct 2009 16:32:06 +0200 Subject: [Distutils] Package install failures in 2.6.3 - setuptools vs Distribute In-Reply-To: <9EDFE750-4102-49A4-8B35-34E6E7AE205C@gmail.com> References: <28320134263785163387302965410901866648-Webmail@me.com> <72523684218386049404529325276719797765-Webmail@me.com> <319e029f0910050444t25761d79gce49d7ef13d98191@mail.gmail.com> <9EDFE750-4102-49A4-8B35-34E6E7AE205C@gmail.com> Message-ID: <94bdd2610910050732v35984fdfqdd1302539f77e99b@mail.gmail.com> On Mon, Oct 5, 2009 at 4:21 PM, ssteinerX at gmail.com wrote: > > On Oct 5, 2009, at 7:44 AM, Lennart Regebro wrote: > >> 2009/10/5 Ronald Oussoren : >>> >>> This is a problem, it means 2.6.3 is not a simple drop-in replacement for >>> 2.6.2 but requires the replacement of another component as well. ?That can >>> be a problem in organizations with strict configuration management where you >>> cannot install new software without going to lots of red tape (and that >>> might involve lawyers when you install a new package instead of upgrading an >>> existing one). >> >> Sure, but that would have happened sooner or later anyway. Is it >> really so bad that it happens now instead of say, in half a year? I >> don't see why it's such a big deal. Sorry, but I don't. > > It's not the happening, it's the happening without a deprecation, warnings, > announcements etc. deserved by a change that affects so much of the Python > eco-system. I am not going to add deprecation warnings in Distutils when I see that Setuptools is being used... And unfortunately we are not responsible for the setuptools project, and we can't drive what is happening there. About a year and a half ago, I did hit another monkey-patch problem with setuptools, after a change in Distutils, I have told PJE about it, and he answered that Distutils should revert that change (concerns Python 2.7) to continue working with setuptools 0.6cx. But that change was made upon Distutils users requests at bugs.python.org and was perfectly fine and wanted. So it was applied, because I won't tell Distutils users : no, you can't have it because Setuptools would turn to be incompatible with it one day. The proper answer is : Setuptools is on the top of Distutils and has to evolve with it. And since it monkey patches it, it has to be changed when a Distutils release breaks it. Now for our problem today the proper answer is: Setuptools 0.6c10 should be released or Distribute used because Distribute is actively maintained > > S > > _______________________________________________ > Distutils-SIG maillist ?- ?Distutils-SIG at python.org > http://mail.python.org/mailman/listinfo/distutils-sig > -- Tarek Ziad? | http://ziade.org |????????????! |???????????? From rich at noir.com Mon Oct 5 16:25:44 2009 From: rich at noir.com (K. Richard Pixley) Date: Mon, 05 Oct 2009 07:25:44 -0700 Subject: [Distutils] Package install failures in 2.6.3 - setuptools vs Distribute In-Reply-To: <72523684218386049404529325276719797765-Webmail@me.com> References: <28320134263785163387302965410901866648-Webmail@me.com> <72523684218386049404529325276719797765-Webmail@me.com> Message-ID: <4ACA01E8.8020504@noir.com> Ronald Oussoren wrote: > For beginners this issue is a showstopper that they cannot resolve without help. > I'm a relative beginner to distutils/setuptools/distribute, but a long time configuration/build/packaging professional. You're mistaken if you think that any of these technologies are suitable for beginners. The state of python package distribution resembles the state of linux packages circa 1995, except that it isn't very well documented at all. Easy_install certainly isn't, and never has been. My first questions about easy_install, (how do I get a list of installed packages? How do I upgrade all installed packages? How do I delete a package using easy_install?) are all unanswerable. I' spent over 20 hours this weekend just reading through documentation trying to figure out what the state of the art is expected to be here and I still don't know much. That's far beyond the investment of a casual user. I'm puzzled that people wonder why I haven't used standard packaging yet when it's pretty clear that standard packaging and distribution doesn't really solve my problems. Python packaging and distribution right now is not for beginners or the faint of heart. --rich - (who came here recently looking for an opportunity to write something akin to stdeb.) From rich at noir.com Mon Oct 5 16:37:15 2009 From: rich at noir.com (K. Richard Pixley) Date: Mon, 05 Oct 2009 07:37:15 -0700 Subject: [Distutils] Package install failures in 2.6.3 - setuptools vs Distribute In-Reply-To: <72523684218386049404529325276719797765-Webmail@me.com> References: <28320134263785163387302965410901866648-Webmail@me.com> <72523684218386049404529325276719797765-Webmail@me.com> Message-ID: <4ACA049B.5010201@noir.com> Ronald Oussoren wrote: > This is a problem, it means 2.6.3 is not a simple drop-in replacement for 2.6.2 but requires the replacement of another component as well. That can be a problem in organizations with strict configuration management where you cannot install new software without going to lots of red tape (and that might involve lawyers when you install a new package instead of upgrading an existing one). This would be a problem if distribute were in general release. It's not. It's clearly a development branch which is intended to move quickly. People using distribute are taking development, pre-alpha kinds of risk and that has been made pretty clear already. --rich From barry at python.org Mon Oct 5 16:37:45 2009 From: barry at python.org (Barry Warsaw) Date: Mon, 5 Oct 2009 10:37:45 -0400 Subject: [Distutils] Package install failures in 2.6.3 - setuptools vs Distribute In-Reply-To: <4ACA01E8.8020504@noir.com> References: <28320134263785163387302965410901866648-Webmail@me.com> <72523684218386049404529325276719797765-Webmail@me.com> <4ACA01E8.8020504@noir.com> Message-ID: On Oct 5, 2009, at 10:25 AM, K. Richard Pixley wrote: > Python packaging and distribution right now is not for beginners or > the faint of heart. If we're honest with ourselves, it's not for experienced developers either. Do you really even want to have to /think/ about this stuff? -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: PGP.sig Type: application/pgp-signature Size: 832 bytes Desc: This is a digitally signed message part URL: From ssteinerx at gmail.com Mon Oct 5 16:37:51 2009 From: ssteinerx at gmail.com (ssteinerX@gmail.com) Date: Mon, 5 Oct 2009 10:37:51 -0400 Subject: [Distutils] Package install failures in 2.6.3 - setuptools vs Distribute In-Reply-To: <0A60A4FF-D32B-4D35-B607-1A02C838F743@python.org> References: <319e029f0910030515u71325b88h89037b3ea85f65b5@mail.gmail.com> <94bdd2610910030649r431a5638y7c8b5332934f5ae@mail.gmail.com> <319e029f0910031308u4182dff2w89760afb4122cf2c@mail.gmail.com> <4AC9BEF5.7080204@taupro.com> <0A60A4FF-D32B-4D35-B607-1A02C838F743@python.org> Message-ID: On Oct 5, 2009, at 9:38 AM, Barry Warsaw wrote: > > I apologize for my part in this, but moving forward I think that if > it's possible to patch and release a setuptools that works with > Python 2.6.3 /and/ earlier Python 2.6.x's, then that should happen > asap. If that's not possible, then we might need to revert the > distutils change in a quick Python 2.6.4. I thought the whole point of the Distribute project was that we couldn't _get_ a new setuptools release and, so, had to fork. Hopefully, with this motivation, we'll get a new setuptools "one more time!" and make the transition to Distribute a little more controlled. Fast, but controlled. S -------------- next part -------------- An HTML attachment was scrubbed... URL: From rich at noir.com Mon Oct 5 16:28:24 2009 From: rich at noir.com (K. Richard Pixley) Date: Mon, 05 Oct 2009 07:28:24 -0700 Subject: [Distutils] Packaging Distribute In-Reply-To: References: <90bb445a0910040810g48f7833cia02536e50e5c19f9@mail.gmail.com> <5cae42b20910040828l6157c3fbw51d06a61212602ce@mail.gmail.com> <94bdd2610910041341v2c2483b1jd6c8f4b5707d789a@mail.gmail.com> Message-ID: <4ACA0288.6070007@noir.com> Sridhar Ratnakumar wrote: > On Sun, 04 Oct 2009 13:41:06 -0700, Tarek Ziad? > wrote: > >> The other way would be to use Distribute instead of Setuptools for >> what the packaging system is calling "setuptools". That's pretty >> much what is happening in Gentoo (arch) and UHU-Linux (dev), >> right now > Interesting. Gentoo uses distribute but retains the name 'setuptools'? > > DEPEND="!! > Ah. But what if PJE releases setuptools with the *same* version number > 0.6.3? What would the gentoo folks do in order to get the new > setuptools release in their packaging system? Or did they make a > decision of totally dropping setuptools from their repository? Debian would probably treat one side or the other as a branch. Sounds like gentoo has decided that distribute is the main line, which would define a new release of setuptools as a branch. So it would probably be something like setuptools-0.6.3pje-r0 or the like. That way it's progress from 0.6.3, but would be overwritten by a 0.6.4. (Or, they'd just ignore the pje release). --rich From regebro at gmail.com Mon Oct 5 16:57:17 2009 From: regebro at gmail.com (Lennart Regebro) Date: Mon, 5 Oct 2009 16:57:17 +0200 Subject: [Distutils] Package install failures in 2.6.3 - setuptools vs Distribute In-Reply-To: <0A60A4FF-D32B-4D35-B607-1A02C838F743@python.org> References: <319e029f0910030515u71325b88h89037b3ea85f65b5@mail.gmail.com> <94bdd2610910030649r431a5638y7c8b5332934f5ae@mail.gmail.com> <319e029f0910031308u4182dff2w89760afb4122cf2c@mail.gmail.com> <4AC9BEF5.7080204@taupro.com> <0A60A4FF-D32B-4D35-B607-1A02C838F743@python.org> Message-ID: <319e029f0910050757q4fc0dde3s91e4d34f045784d@mail.gmail.com> 2009/10/5 Barry Warsaw : > I apologize for my part in this, but moving forward I think that if it's > possible to patch and release a setuptools that works with Python 2.6.3 > /and/ earlier Python 2.6.x's, then that should happen asap. PJE seems interested in this, as he asked about a patch, so maybe. >?If that's not possible, then we might need to revert the distutils > change in a quick Python 2.6.4. That would be a big mistake. -- Lennart Regebro: Python, Zope, Plone, Grok http://regebro.wordpress.com/ +33 661 58 14 64 From regebro at gmail.com Mon Oct 5 17:03:48 2009 From: regebro at gmail.com (Lennart Regebro) Date: Mon, 5 Oct 2009 17:03:48 +0200 Subject: [Distutils] Package install failures in 2.6.3 - setuptools vs Distribute In-Reply-To: <4ACA049B.5010201@noir.com> References: <28320134263785163387302965410901866648-Webmail@me.com> <72523684218386049404529325276719797765-Webmail@me.com> <4ACA049B.5010201@noir.com> Message-ID: <319e029f0910050803j37dedd3eg69f6265d8b22693e@mail.gmail.com> 2009/10/5 K. Richard Pixley : > This would be a problem if distribute were in general release. ?It's not. > ?It's clearly a development branch which is intended to move quickly. No, this is incorrect. The 0.6-branch is not intended to move quickly, it is in bugfix mode. It is moving quickly only because some major bugfixes has been made, but it's not a development branch, that's 0.7, which isn't released. -- Lennart Regebro: Python, Zope, Plone, Grok http://regebro.wordpress.com/ +33 661 58 14 64 From ziade.tarek at gmail.com Mon Oct 5 17:05:53 2009 From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=) Date: Mon, 5 Oct 2009 17:05:53 +0200 Subject: [Distutils] Package install failures in 2.6.3 - setuptools vs Distribute In-Reply-To: <319e029f0910050757q4fc0dde3s91e4d34f045784d@mail.gmail.com> References: <319e029f0910030515u71325b88h89037b3ea85f65b5@mail.gmail.com> <94bdd2610910030649r431a5638y7c8b5332934f5ae@mail.gmail.com> <319e029f0910031308u4182dff2w89760afb4122cf2c@mail.gmail.com> <4AC9BEF5.7080204@taupro.com> <0A60A4FF-D32B-4D35-B607-1A02C838F743@python.org> <319e029f0910050757q4fc0dde3s91e4d34f045784d@mail.gmail.com> Message-ID: <94bdd2610910050805l7aa3f1fcsbd0e57e815bb97f1@mail.gmail.com> On Mon, Oct 5, 2009 at 4:57 PM, Lennart Regebro wrote: > 2009/10/5 Barry Warsaw : >> I apologize for my part in this, but moving forward I think that if it's >> possible to patch and release a setuptools that works with Python 2.6.3 >> /and/ earlier Python 2.6.x's, then that should happen asap. > > PJE seems interested in this, as he asked about a patch, so maybe. It was pushed in setuptools bug tracker by Ned yesterday IIRC, with links to the patch that works. From rich at noir.com Mon Oct 5 17:06:43 2009 From: rich at noir.com (K. Richard Pixley) Date: Mon, 05 Oct 2009 08:06:43 -0700 Subject: [Distutils] Package install failures in 2.6.3 - setuptools vs Distribute In-Reply-To: References: <28320134263785163387302965410901866648-Webmail@me.com> <72523684218386049404529325276719797765-Webmail@me.com> <4ACA01E8.8020504@noir.com> Message-ID: <4ACA0B83.2060003@noir.com> Barry Warsaw wrote: > On Oct 5, 2009, at 10:25 AM, K. Richard Pixley wrote: >> Python packaging and distribution right now is not for beginners or >> the faint of heart. > If we're honest with ourselves, it's not for experienced developers > either. Do you really even want to have to /think/ about this stuff? Depends. It's what I do professionally in slightly different arenas, so I think about it anyway. But I take your point. The packaging and distribution system should be simple, powerful, expressive, reliable, and definitive to the point where it's existence fades into the background allowing one to spend one's time and focus of attention on less easily solved problems. --rich From rich at noir.com Mon Oct 5 17:21:19 2009 From: rich at noir.com (K. Richard Pixley) Date: Mon, 05 Oct 2009 08:21:19 -0700 Subject: [Distutils] Package install failures in 2.6.3 - setuptools vs Distribute In-Reply-To: <319e029f0910050803j37dedd3eg69f6265d8b22693e@mail.gmail.com> References: <28320134263785163387302965410901866648-Webmail@me.com> <72523684218386049404529325276719797765-Webmail@me.com> <4ACA049B.5010201@noir.com> <319e029f0910050803j37dedd3eg69f6265d8b22693e@mail.gmail.com> Message-ID: <4ACA0EEF.7010105@noir.com> Lennart Regebro wrote: > 2009/10/5 K. Richard Pixley : > >> This would be a problem if distribute were in general release. It's not. >> It's clearly a development branch which is intended to move quickly. >> > > No, this is incorrect. The 0.6-branch is not intended to move quickly, > it is in bugfix mode. > It is moving quickly only because some major bugfixes has been made, > but it's not a development branch, that's 0.7, which isn't released. > I'm recent to python packaging and distribution, so let me see if I've put this together right from my reading of the various web pages involved over the weekend. Distutils is currently part of the standard python library. As such, it's released with python, (the reference implementation, anyway). Distutils is currently capable of producing only source archives of packages. While it's capable of producing "built" archives, those archives are machine specific, nonrelocatable, untrackable, and have no standard method for distribution nor installation nor tracking. Setuptools was a third party addition to, (and partial replacement of), distutils because distutils wasn't suitably usable nor was it moving fast enough. However, since setuptools was initiated, many of the major features of setuptools have since been folded back into distutils, making setuptools partially redundant and partly colliding. Setuptools provides the ability to produce machine independent "built" archives and a standard method for installing them, (although not for tracking or removing them). And the setuptools approach to installation, easy_install, doesn't play nice with the native installers on systems that have them like rpm, debian, etc. However, setuptools has fallen into disrepair and so distribute has been created, as a friendly branch off a third party tool which in turn was a form of branch off of distutils. And within distribute, there are two lines of development, the 0.7 line, which is intended to replace... I'm confused. Does it replace setuptools or distutils? And then there's the 0.6 branch, which is a branch off 0.7 which is a branch of setuptools which is a branch of distutils which is under recent active development and yet it also expected to be stable, as much as such a term can be applied to a third party branch off a third party of a colliding replacement with a standard facility. Is that about right? If I'm anywhere near right, then I can't really imagine what state you intend for the 0.6 branch if not development. --rich -------------- next part -------------- An HTML attachment was scrubbed... URL: From regebro at gmail.com Mon Oct 5 17:29:29 2009 From: regebro at gmail.com (Lennart Regebro) Date: Mon, 5 Oct 2009 17:29:29 +0200 Subject: [Distutils] Package install failures in 2.6.3 - setuptools vs Distribute In-Reply-To: <4ACA0EEF.7010105@noir.com> References: <28320134263785163387302965410901866648-Webmail@me.com> <72523684218386049404529325276719797765-Webmail@me.com> <4ACA049B.5010201@noir.com> <319e029f0910050803j37dedd3eg69f6265d8b22693e@mail.gmail.com> <4ACA0EEF.7010105@noir.com> Message-ID: <319e029f0910050829k670dd06fm80aed4ba98a74788@mail.gmail.com> 2009/10/5 K. Richard Pixley : > Is that about right? Nope. 0.6 is a fork of setuptools, providing bugfixes (and also 3.1 support). It's completely backwards compatible with setuptools. 0.7 is a development branch, which aims to refactor setuptools into something or (rather several somethings) that is simpler and easier to maintain. It will not be backwards compatible. -- Lennart Regebro: Python, Zope, Plone, Grok http://regebro.wordpress.com/ +33 661 58 14 64 From jeremy at jeremysanders.net Mon Oct 5 17:32:15 2009 From: jeremy at jeremysanders.net (Jeremy Sanders) Date: Mon, 05 Oct 2009 16:32:15 +0100 Subject: [Distutils] Package install failures in 2.6.3 - setuptools vs Distribute References: <28320134263785163387302965410901866648-Webmail@me.com> <72523684218386049404529325276719797765-Webmail@me.com> <4ACA01E8.8020504@noir.com> Message-ID: K. Richard Pixley wrote: > Ronald Oussoren wrote: >> For beginners this issue is a showstopper that they cannot resolve >> without help. >> > I'm a relative beginner to distutils/setuptools/distribute, but a long > time configuration/build/packaging professional. You're mistaken if you > think that any of these technologies are suitable for beginners. The > state of python package distribution resembles the state of linux > packages circa 1995, except that it isn't very well documented at all. As a general question, is there any planned project to improve the state of distutils or replace it? It appears to be one of the weakest parts of the Python system and needs replacing with something much cleaner, better documented and more powerful. Even making something like cmake the standard would help a lot. Jeremy -- jeremy at jeremysanders.net http://www.jeremysanders.net/ From alex.gronholm at nextday.fi Mon Oct 5 18:10:13 2009 From: alex.gronholm at nextday.fi (=?ISO-8859-1?Q?Alex_Gr=F6nholm?=) Date: Mon, 05 Oct 2009 19:10:13 +0300 Subject: [Distutils] Package install failures in 2.6.3 - setuptools vs Distribute In-Reply-To: References: <28320134263785163387302965410901866648-Webmail@me.com> <72523684218386049404529325276719797765-Webmail@me.com> <4ACA01E8.8020504@noir.com> Message-ID: <4ACA1A65.702@nextday.fi> Jeremy Sanders kirjoitti: > K. Richard Pixley wrote: > > >> Ronald Oussoren wrote: >> >>> For beginners this issue is a showstopper that they cannot resolve >>> without help. >>> >>> >> I'm a relative beginner to distutils/setuptools/distribute, but a long >> time configuration/build/packaging professional. You're mistaken if you >> think that any of these technologies are suitable for beginners. The >> state of python package distribution resembles the state of linux >> packages circa 1995, except that it isn't very well documented at all. >> > > As a general question, is there any planned project to improve the state of > distutils or replace it? It appears to be one of the weakest parts of the > Python system and needs replacing with something much cleaner, better > documented and more powerful. > > Even making something like cmake the standard would help a lot. > > There is a lack of consensus regarding how exactly they should work. If we are having this much trouble deciding how a third party tool should work, it is certainly not going to be merged into distutils until those issues have been resolved. Distutils is what houses (or should) the parts we all agree on. That said, I think that plenty of setuptools/distribute functionality should be moved to distutils (after the code has been cleaned up and the proper unit tests introduced). > Jeremy > > From zooko at zooko.com Mon Oct 5 18:13:52 2009 From: zooko at zooko.com (Zooko Wilcox-O'Hearn) Date: Mon, 5 Oct 2009 10:13:52 -0600 Subject: [Distutils] Package install failures in 2.6.3 - setuptools vs Distribute In-Reply-To: <94bdd2610910050711i5d429043he3288986fe470516@mail.gmail.com> References: <319e029f0910030515u71325b88h89037b3ea85f65b5@mail.gmail.com> <94bdd2610910030649r431a5638y7c8b5332934f5ae@mail.gmail.com> <319e029f0910031308u4182dff2w89760afb4122cf2c@mail.gmail.com> <4AC9BEF5.7080204@taupro.com> <0A60A4FF-D32B-4D35-B607-1A02C838F743@python.org> <53902AAD-D8B3-490B-9228-56416DE5E6D3@zooko.com> <94bdd2610910050711i5d429043he3288986fe470516@mail.gmail.com> Message-ID: On Monday,2009-10-05, at 8:11 , Tarek Ziad? wrote: > So are you saying that in an environment where you are allowed to > install Python 2.6.3, you will not be allowed to install an > hypothetical setuptools-0.6c10 (or a Distribute 0.6.3) ? Yes, situations like that can come up. For example, I guess the packaging of my own Tahoe-LAFS project is a case in point. The current requirements for Tahoe-LAFS are Python >= 2.4.2 and < 3.0, and the source distribution of Tahoe-LAFS comes with its own bundled copy of setuptools. We haven't finished our qualification of Distribute so we're not ready to switch our build system to Distribute. Setuptools-0.6c10 is hypothetical at this point. What do we do? We could tighten the Python version to >= 2.4.2 and <= 2.6.2. We could develop and test a patch to our bundled copy of setuptools to work-around this problem. We could ask the open source volunteer who is testing Distribute for us to "hurry up". We could ask PJE to "hurry up" and release a new version of setuptools. Perhaps we will end up doing more than one of these things. Many of our users won't even be able to diagnose the problem if they upgrade from Python 2.6.2 to 2.6.3 and the install breaks. They won't know what is wrong or how to work around it, other than by writing to our mailing list asking for help. The faster this gets fixed and the more "lenient" Python 2.6.x is with respect to what versions of setuptools/Distribute it requires and the more "lenient" setuptools and Distribute are with respect to what version of Python they require, the better for everyone. Thank you kindly for your attention. Regards, Zooko From zooko at zooko.com Mon Oct 5 18:28:59 2009 From: zooko at zooko.com (Zooko Wilcox-O'Hearn) Date: Mon, 5 Oct 2009 10:28:59 -0600 Subject: [Distutils] Package install failures in 2.6.3 - setuptools vs Distribute In-Reply-To: References: <319e029f0910030515u71325b88h89037b3ea85f65b5@mail.gmail.com> <94bdd2610910030649r431a5638y7c8b5332934f5ae@mail.gmail.com> <319e029f0910031308u4182dff2w89760afb4122cf2c@mail.gmail.com> <4AC9BEF5.7080204@taupro.com> <0A60A4FF-D32B-4D35-B607-1A02C838F743@python.org> <53902AAD-D8B3-490B-9228-56416DE5E6D3@zooko.com> <94bdd2610910050711i5d429043he3288986fe470516@mail.gmail.com> Message-ID: <5FA0EFA6-E808-4835-9ACE-98B878193D01@zooko.com> I'm sorry to follow-up to my own post, but I realized that I didn't make something clear: the current Tahoe-LAFS source distribution comes with its own copy of setuptools, so even if PJE releases a new version of setuptools or if we patch that copy to work-around this problem, we're going to have to re-qualify the combination of the new version on our buildbot and make a new stable release of Tahoe-LAFS (something that typically only happens every 3 or 4 months) before end users will be able to install Tahoe-LAFS on Python 2.6.3. I don't want to point fingers, because that goes nowhere. Lots of people could have written their software or managed their processes differently in order to avoid this situation, including me. However, I do want to emphasize that this is a serious problem. Backwards compatibility on minor releases such as from 2.6.2 to 2.6.3 is a huge concern for a lot of people. If something like this breaks -- regardless of whose fault(s) it is -- then it reduces people's trust in the stability of stable updates of their infrastructural software. I'm sorry to say that this event has already made me more hesitant to jump from setuptools to Distribute, just because some of the maintainers of Distribute have posted saying that they don't think this kind of thing is such a big deal. I prefer to use packaging tools which are stable. To achieve that kind of stability sometimes requires basically "taking responsibility" for other people's decisions, such as saying "Well, setuptools-0.6c9 monkey-patches distutils. We think that's a bad idea and we wish it didn't do that. But, we know that a lot of other people out there are relying on the combination of setuptools-0.6c9 and Python 2.6.x, so we're going to go the extra mile to make sure that those people don't experience disruptions.". Please take this in the constructive spirit that is intended. We're on the same team. I've already contributed a few patches and a lot of bug reports to setuptools, Distribute, and Python, and I'd like to contribute more in the future. Having a policy of actively working to maintain stability across stable updates will help everyone. Regards, Zooko From rich at noir.com Mon Oct 5 18:36:24 2009 From: rich at noir.com (K. Richard Pixley) Date: Mon, 05 Oct 2009 09:36:24 -0700 Subject: [Distutils] Package install failures in 2.6.3 - setuptools vs Distribute In-Reply-To: <4ACA1A65.702@nextday.fi> References: <28320134263785163387302965410901866648-Webmail@me.com> <72523684218386049404529325276719797765-Webmail@me.com> <4ACA01E8.8020504@noir.com> <4ACA1A65.702@nextday.fi> Message-ID: <4ACA2088.2010104@noir.com> Alex Gr?nholm wrote: > There is a lack of consensus regarding how exactly they should work. > If we are having this much trouble deciding how a third party tool > should work, it is certainly not going to be merged into distutils > until those issues have been resolved. Distutils is what houses (or > should) the parts we all agree on. That said, I think that plenty of > setuptools/distribute functionality should be moved to distutils > (after the code has been cleaned up and the proper unit tests > introduced). I agree there's a lack of consensus. But I dont' believe that distutils is a strong basis for growth. Distutils may be a reasonable choice of a build tool, (I'm not sure yet), but it's packaging and distribution support is minimal to nonexistent. I think what's needed here isn't a consensus, (which we'll never get), but rather a generational solution based not on available technology, but rather on state of the art requirements. For example, distutils says nothing about host packaging systems nor distribution. The distutils doc doesn't really even broach the topic of impure python packages or distribution. I don't know, for instance, whether C extensions are expected to be contained in a collection of different tar.gz archives or if there's expected to be one "fat" one which contains binary C extensions for a collection of different operating systems. The new system needs to support mass updates of all installed packages, querying of installed packages, removal of installed packages, recursive instances of the packaging database, (perhaps with something like virtualenv), as well as all of the things that easy_install supports now, and most of the features that debian/rpm/etc support now including virtual packages, dependencies, suggestions, conflicts, and both command line and gui installers. It should also be capable of at least "playing nice" with the host packaging system, (eg, rpm, deb, ipk, etc), or perhaps of completely excusing itself from the distribution process, (other than supporting the native distribution process). If necessary, we can create extra repositories for those systems such that python packages can be distributed through the native packaging system, even though we're managing our own release processes. I know that debian/ubuntu/rpm/easy_install all allow for this as it's very common for enterprises to use such a solution for distributing their own packages in-house. The new system should also support "cross compilation" to the extent that such is needed for C extensions and to the extent that the native packaging system supports. (Eg, debian doesn't, really but bitbake/OE virtually requires it). Most of what I'm talking about here speaks to packaging formats, distribution processes, and installation processes. And this isn't new technology. Both debian, rpm, and several other unix technologies have fine systems in operation right now. Sure, they all have weaknesses, but they are much better than easy_install. --rich From greno at verizon.net Mon Oct 5 18:47:55 2009 From: greno at verizon.net (Gerry Reno) Date: Mon, 05 Oct 2009 12:47:55 -0400 Subject: [Distutils] [ANN] stdeb 0.3.2 and 0.4.1 In-Reply-To: <4AC8E5D7.8010602@astraw.com> References: <4AC8E5D7.8010602@astraw.com> Message-ID: <4ACA233B.8030107@verizon.net> Andrew, I installed stdeb 0.3.2 from PyPi on my Hardy server and ran a couple tests just to be sure. stdeb 0.3.2 on Hardy is working fine. I didn't see any problems. 'bdist_deb' worked as expected and generated both a .dsc and a .deb file for my project which installed perfectly. Regards, Gerry From regebro at gmail.com Mon Oct 5 18:53:24 2009 From: regebro at gmail.com (Lennart Regebro) Date: Mon, 5 Oct 2009 18:53:24 +0200 Subject: [Distutils] Package install failures in 2.6.3 - setuptools vs Distribute In-Reply-To: <5FA0EFA6-E808-4835-9ACE-98B878193D01@zooko.com> References: <94bdd2610910030649r431a5638y7c8b5332934f5ae@mail.gmail.com> <319e029f0910031308u4182dff2w89760afb4122cf2c@mail.gmail.com> <4AC9BEF5.7080204@taupro.com> <0A60A4FF-D32B-4D35-B607-1A02C838F743@python.org> <53902AAD-D8B3-490B-9228-56416DE5E6D3@zooko.com> <94bdd2610910050711i5d429043he3288986fe470516@mail.gmail.com> <5FA0EFA6-E808-4835-9ACE-98B878193D01@zooko.com> Message-ID: <319e029f0910050953s5246ee71x8906e96fd99e2719@mail.gmail.com> 2009/10/5 Zooko Wilcox-O'Hearn : > I'm sorry to say that this event has already made me more hesitant to jump > from setuptools to Distribute, just because some of the maintainers of > Distribute have posted saying that they don't think this kind of thing is > such a big deal. ?I prefer to use packaging tools which are stable. You completely misunderstand. It's not that it isn't a big deal that setuptools isn't maintained. It *is* a big deal. But that you are forced to upgrade *now* is not a big deal, because you would be forced to upgrade soon any way. What you seem to miss in this whole discussion is that setuptools is buggy, and unmaintained, and that you will have to move from setuptools sooner or later. That setuptools is broken *is* a big deal. That is an effect of the current situation, and as you know that is a situation only one person can do anything about. BUT: It is *not* a big deal that this breakage affects you today, instead of in six months. I understand that maybe for you, of there had been more warnings, you could have had time to move to Distribute, and thereby move to 2.6.3 already now, instead of having to wait a bit. But then again, you are not only in a very special and unusual situation, you already use a patched version of setuptools (presumably because of it's unmaintaned state) and you can therefore easily apply this patch to your copy, and therefore be up and running again. For everyone else, who does not have the strict rules you do, the fix is easy. All they need to do is "easy_install Distribut" and it will work again. That is *not* a big deal. That bugs in setuptools make you reluctant to migrate to Distribute is a position I must admit I don't understand. Possibly if you somehow think it's the Distribute teams fault that a bugfix in Python ended up breaking setuptools. If it would have been better not to fix that bug, then the blame reasonably goes to the Python core developers, not the Distribute team. Once again, to make things completely clear: The breakage *is* a big deal. That it happens to break today instead of tomorrow is not. -- Lennart Regebro: Python, Zope, Plone, Grok http://regebro.wordpress.com/ +33 661 58 14 64 From hanno at hannosch.eu Mon Oct 5 19:53:33 2009 From: hanno at hannosch.eu (Hanno Schlichting) Date: Mon, 5 Oct 2009 19:53:33 +0200 Subject: [Distutils] Package install failures in 2.6.3 - setuptools vs Distribute In-Reply-To: References: <319e029f0910030515u71325b88h89037b3ea85f65b5@mail.gmail.com> <94bdd2610910030649r431a5638y7c8b5332934f5ae@mail.gmail.com> <319e029f0910031308u4182dff2w89760afb4122cf2c@mail.gmail.com> <4AC9BEF5.7080204@taupro.com> Message-ID: <5cae42b20910051053i5b160f8ewcdbb3cc51cfd1370@mail.gmail.com> On Mon, Oct 5, 2009 at 1:33 PM, Ned Deily wrote: > I've opened an issue of the main Python issue tracker outlining the > problem, primarily for the benefit of affected users who search the > tracker: > > ? http://bugs.python.org/issue7064 If I understand the comments on this ticket correctly, Tarek has changed distutils in a way so the last setuptools release continues to work, correct? So based on the current state of Python 2.6.4 will work again with an unmodified setuptools 0.6c9? Hanno From pje at telecommunity.com Mon Oct 5 20:02:57 2009 From: pje at telecommunity.com (P.J. Eby) Date: Mon, 05 Oct 2009 14:02:57 -0400 Subject: [Distutils] Package install failures in 2.6.3 - setuptools vs Distribute In-Reply-To: <319e029f0910050253x1ad1cc7eg60e7f11b3f9f72b8@mail.gmail.co m> References: <319e029f0910030515u71325b88h89037b3ea85f65b5@mail.gmail.com> <94bdd2610910030649r431a5638y7c8b5332934f5ae@mail.gmail.com> <319e029f0910031308u4182dff2w89760afb4122cf2c@mail.gmail.com> <4AC9BEF5.7080204@taupro.com> <319e029f0910050253x1ad1cc7eg60e7f11b3f9f72b8@mail.gmail.com> Message-ID: <20091005180304.71CF63A4045@sparrow.telecommunity.com> At 11:53 AM 10/5/2009 +0200, Lennart Regebro wrote: >2009/10/5 Jeff Rush : > > Very unfortunate, as in, it should NOT have happened. And *especially* > > without any announcement on python.org or mention on the > > python-committers list of something this major. > >Well "this major"... It's a bug fix that breaks a setuptools monkey-patch. Subclassing distutils commands != monkeypatching. If, say, numpy's distutils extensions subclass build_ext and override that method, they could have had the same problem. Same for anybody else's distutils extensions. From pje at telecommunity.com Mon Oct 5 20:06:12 2009 From: pje at telecommunity.com (P.J. Eby) Date: Mon, 05 Oct 2009 14:06:12 -0400 Subject: [Distutils] Package install failures in 2.6.3 - setuptools vs Distribute In-Reply-To: <4ACA01E8.8020504@noir.com> References: <28320134263785163387302965410901866648-Webmail@me.com> <72523684218386049404529325276719797765-Webmail@me.com> <4ACA01E8.8020504@noir.com> Message-ID: <20091005180619.7D3183A4045@sparrow.telecommunity.com> At 07:25 AM 10/5/2009 -0700, K. Richard Pixley wrote: >How do I delete a package using easy_install? http://peak.telecommunity.com/DevCenter/EasyInstall#uninstalling-packages From pje at telecommunity.com Mon Oct 5 20:10:16 2009 From: pje at telecommunity.com (P.J. Eby) Date: Mon, 05 Oct 2009 14:10:16 -0400 Subject: [Distutils] Package install failures in 2.6.3 - setuptools vs Distribute In-Reply-To: <319e029f0910050757q4fc0dde3s91e4d34f045784d@mail.gmail.com > References: <319e029f0910030515u71325b88h89037b3ea85f65b5@mail.gmail.com> <94bdd2610910030649r431a5638y7c8b5332934f5ae@mail.gmail.com> <319e029f0910031308u4182dff2w89760afb4122cf2c@mail.gmail.com> <4AC9BEF5.7080204@taupro.com> <0A60A4FF-D32B-4D35-B607-1A02C838F743@python.org> <319e029f0910050757q4fc0dde3s91e4d34f045784d@mail.gmail.com> Message-ID: <20091005181023.749023A4045@sparrow.telecommunity.com> At 04:57 PM 10/5/2009 +0200, Lennart Regebro wrote: >2009/10/5 Barry Warsaw : > > I apologize for my part in this, but moving forward I think that if it's > > possible to patch and release a setuptools that works with Python 2.6.3 > > /and/ earlier Python 2.6.x's, then that should happen asap. > >PJE seems interested in this, as he asked about a patch, so maybe. I expect to have a small amount of time later this week to work on setuptools. No guarantees, but this is certainly one of the bugs on my short list for doing something with. > > If that's not possible, then we might need to revert the distutils > > change in a quick Python 2.6.4. > >That would be a big mistake. Actually, I think the mistake here is where non-bugfix code was ported from 3.x back to the 2.6.x maintenance branch. I'm particularly concerned about the .compiler / ._compiler change, as it seems to me to be responsible for breaking mingw32 compilation on Windows, but I haven't had time to investigate thoroughly. (It actually looks like a problem that might be in Python 3 as well.) From pje at telecommunity.com Mon Oct 5 20:21:28 2009 From: pje at telecommunity.com (P.J. Eby) Date: Mon, 05 Oct 2009 14:21:28 -0400 Subject: [Distutils] Package install failures in 2.6.3 - setuptools vs Distribute In-Reply-To: <319e029f0910050953s5246ee71x8906e96fd99e2719@mail.gmail.co m> References: <94bdd2610910030649r431a5638y7c8b5332934f5ae@mail.gmail.com> <319e029f0910031308u4182dff2w89760afb4122cf2c@mail.gmail.com> <4AC9BEF5.7080204@taupro.com> <0A60A4FF-D32B-4D35-B607-1A02C838F743@python.org> <53902AAD-D8B3-490B-9228-56416DE5E6D3@zooko.com> <94bdd2610910050711i5d429043he3288986fe470516@mail.gmail.com> <5FA0EFA6-E808-4835-9ACE-98B878193D01@zooko.com> <319e029f0910050953s5246ee71x8906e96fd99e2719@mail.gmail.com> Message-ID: <20091005182137.854793A4045@sparrow.telecommunity.com> At 06:53 PM 10/5/2009 +0200, Lennart Regebro wrote: >Possibly if you somehow >think it's the Distribute teams fault that a bugfix in Python ended up >breaking setuptools. If it would have been better not to fix that bug, >then the blame reasonably goes to the Python core developers, not the >Distribute team. In this case, though, the "Python core developer" is also the Distribute lead. (i.e., it was Tarek who made the changes to the distutils.) So it's a bit understandable that some people might wonder if there was a conflict of interest. I don't personally think that's the case; it's pretty much inevitable that the distutils making progress means other things will break. But it's easy to see how others might take the situation another way, or treat it as an example of Distribute policy towards backward compatibility, or of what kind of breakage is considered acceptable in a dot release. It would be good to bear in mind that extending the distutils (or setuptools) is *not* monkeypatching; both libraries provide explicit assurance that subclassing is in fact allowed. And there's nothing all that special about setuptools' subclassing of build_ext; in fact, if you look back in the archives here, other people have done equivalent subclassing to support dynamic library building. I haven't checked their code, but there is a strong possibility that it would also fail in the same way. This is not really about monkeypatching, or about special support for setuptools. From rich at noir.com Mon Oct 5 20:29:59 2009 From: rich at noir.com (K. Richard Pixley) Date: Mon, 05 Oct 2009 11:29:59 -0700 Subject: [Distutils] Package install failures in 2.6.3 - setuptools vs Distribute In-Reply-To: <20091005180619.7D3183A4045@sparrow.telecommunity.com> References: <28320134263785163387302965410901866648-Webmail@me.com> <72523684218386049404529325276719797765-Webmail@me.com> <4ACA01E8.8020504@noir.com> <20091005180619.7D3183A4045@sparrow.telecommunity.com> Message-ID: <4ACA3B27.2080404@noir.com> P.J. Eby wrote: > At 07:25 AM 10/5/2009 -0700, K. Richard Pixley wrote: >> How do I delete a package using easy_install? > http://peak.telecommunity.com/DevCenter/EasyInstall#uninstalling-packages That doesn't remove a package. It simply removes the package from the search path by one method in hopes that no further instances of it will be loaded. To actually remove it, you have to know the format of the library, the package formats, local library storage conventions, and you need to be an expert user of distutils, setuptools, buildout, etc, in order to determine the content of the package itself in order to remove it manually. And even then you'd have to manually search your entire system for any packages that might still depend on this package lest you break them too. That's far beyond the scope of expectation for a casual package maintainer. That's more akin to current macosx standards, (install only), than debian or rpm, (install, update, query, remove, etc). I suppose it's not such a bad problem if you reload your OS frequently. Or solely work with virtualenv or the like so that you can easily discard and rebuild an environment whenever you need one. Of course, that doesn't really help with the task of keeping multiple servers current or rolling internal software out to an enterprise. --rich From pje at telecommunity.com Mon Oct 5 20:38:19 2009 From: pje at telecommunity.com (P.J. Eby) Date: Mon, 05 Oct 2009 14:38:19 -0400 Subject: [Distutils] Package install failures in 2.6.3 - setuptools vs Distribute In-Reply-To: <5cae42b20910051053i5b160f8ewcdbb3cc51cfd1370@mail.gmail.co m> References: <319e029f0910030515u71325b88h89037b3ea85f65b5@mail.gmail.com> <94bdd2610910030649r431a5638y7c8b5332934f5ae@mail.gmail.com> <319e029f0910031308u4182dff2w89760afb4122cf2c@mail.gmail.com> <4AC9BEF5.7080204@taupro.com> <5cae42b20910051053i5b160f8ewcdbb3cc51cfd1370@mail.gmail.com> Message-ID: <20091005183826.AF9B93A4045@sparrow.telecommunity.com> At 07:53 PM 10/5/2009 +0200, Hanno Schlichting wrote: >If I understand the comments on this ticket correctly, Tarek has >changed distutils in a way so the last setuptools release continues >to work, correct? Yes. And a very nice fix, done quite quickly. Thank you Tarek. >So based on the current state of Python 2.6.4 will work again with >an unmodified setuptools 0.6c9? AFAICT, that is correct. From zooko at zooko.com Mon Oct 5 20:45:50 2009 From: zooko at zooko.com (Zooko Wilcox-O'Hearn) Date: Mon, 5 Oct 2009 12:45:50 -0600 Subject: [Distutils] Package install failures in 2.6.3 - setuptools vs Distribute In-Reply-To: <319e029f0910050953s5246ee71x8906e96fd99e2719@mail.gmail.com> References: <94bdd2610910030649r431a5638y7c8b5332934f5ae@mail.gmail.com> <319e029f0910031308u4182dff2w89760afb4122cf2c@mail.gmail.com> <4AC9BEF5.7080204@taupro.com> <0A60A4FF-D32B-4D35-B607-1A02C838F743@python.org> <53902AAD-D8B3-490B-9228-56416DE5E6D3@zooko.com> <94bdd2610910050711i5d429043he3288986fe470516@mail.gmail.com> <5FA0EFA6-E808-4835-9ACE-98B878193D01@zooko.com> <319e029f0910050953s5246ee71x8906e96fd99e2719@mail.gmail.com> Message-ID: I'm struggling to articulate something here. When the maintainer of the stable branch of a platform that I rely on says "The fact that upgrading to our recent stable release will break this critical functionality is so-and-so's fault, not ours." this reduces my confidence in that maintainer. Not because he's wrong! Maybe it *is* so-and-so's fault. But what I'm looking for in the maintainer of a stable platform is someone who says "Maybe this wasn't our fault, but here are the steps we're taking to get you back on your feet as soon as possible.". Regards, Zooko From hanno at hannosch.eu Mon Oct 5 20:52:06 2009 From: hanno at hannosch.eu (Hanno Schlichting) Date: Mon, 5 Oct 2009 20:52:06 +0200 Subject: [Distutils] Package install failures in 2.6.3 - setuptools vs Distribute In-Reply-To: <20091005183826.AF9B93A4045@sparrow.telecommunity.com> References: <319e029f0910030515u71325b88h89037b3ea85f65b5@mail.gmail.com> <94bdd2610910030649r431a5638y7c8b5332934f5ae@mail.gmail.com> <319e029f0910031308u4182dff2w89760afb4122cf2c@mail.gmail.com> <4AC9BEF5.7080204@taupro.com> <5cae42b20910051053i5b160f8ewcdbb3cc51cfd1370@mail.gmail.com> <20091005183826.AF9B93A4045@sparrow.telecommunity.com> Message-ID: <5cae42b20910051152w3e9e5755wf7df007d153c9c90@mail.gmail.com> On Mon, Oct 5, 2009 at 8:38 PM, P.J. Eby wrote: > At 07:53 PM 10/5/2009 +0200, Hanno Schlichting wrote: >> >> If I understand the comments on this ticket correctly, Tarek has changed >> distutils in a way so the last setuptools release continues to work, >> correct? > > Yes. ?And a very nice fix, done quite quickly. ?Thank you Tarek. Wonderful. This seems to be the right approach to the current problem for me. Thank you Tarek indeed! >> So based on the current state of Python 2.6.4 will work again with an >> unmodified setuptools 0.6c9? > > AFAICT, that is correct. Good. So we can hopefully end this thread and move on to something more productive. If anyone wants to step up and provide help in testing pre-releases of Python, so we can avoid similar situation in the future, that would be most welcome. Hanno From dickinsm at gmail.com Mon Oct 5 21:02:36 2009 From: dickinsm at gmail.com (Mark Dickinson) Date: Mon, 5 Oct 2009 20:02:36 +0100 Subject: [Distutils] distutils: packaging a generated configuration file Message-ID: <5c6f2a5d0910051202i2ff95312pb06402d3c3f27d7b@mail.gmail.com> I'm trying to use distutils for the first time to package up a project, and struggling a bit; I wonder whether some kind soul could point me in the right direction. I'm packaging a pure Python project that uses ctypes. The only complication is that I'd like the setup script to install a configuration file alongside the various .py files. From the documentation, this sounds like something that would usually be specified with 'package_data' or 'data_files'. The catch is that parts of the configuration file are generated at setup time: that is, the setup script gathers various pieces of system information (e.g., library locations) and uses those to transform a hard-coded 'config.in' file into the 'config' file that I want to install. I could just generate the 'config' file in the top-level package directory, but it seems dangerous to assume that that directory is writable; it seems better to use one of the build directories that distutils already knows about. What's the best way to manage generation and installation of the config file? I've got as far as subclassing the build command and making __run__ create the config file from the config.in file, putting it into the build_temp directory; I can't figure out how to tell distutils to install it in the right place. It seems like I want a sort of build_data/install_data pair of commands, that allows creation and installation of *generated* data files. Any hints? Mark From pje at telecommunity.com Mon Oct 5 21:05:21 2009 From: pje at telecommunity.com (P.J. Eby) Date: Mon, 05 Oct 2009 15:05:21 -0400 Subject: [Distutils] Package install failures in 2.6.3 - setuptools vs Distribute In-Reply-To: <4ACA3B27.2080404@noir.com> References: <28320134263785163387302965410901866648-Webmail@me.com> <72523684218386049404529325276719797765-Webmail@me.com> <4ACA01E8.8020504@noir.com> <20091005180619.7D3183A4045@sparrow.telecommunity.com> <4ACA3B27.2080404@noir.com> Message-ID: <20091005190528.4901D3A408F@sparrow.telecommunity.com> At 11:29 AM 10/5/2009 -0700, K. Richard Pixley wrote: >P.J. Eby wrote: >>At 07:25 AM 10/5/2009 -0700, K. Richard Pixley wrote: >>>How do I delete a package using easy_install? >>http://peak.telecommunity.com/DevCenter/EasyInstall#uninstalling-packages >That doesn't remove a package. It simply removes the package from >the search path by one method in hopes that no further instances of >it will be loaded. > >To actually remove it, you have to know the format of the library, >the package formats, local library storage conventions, and you need >to be an expert user of distutils, setuptools, buildout, etc, in >order to determine the content of the package itself in order to >remove it manually. As it says at the above link: "After you've done this, you can safely delete the .egg files or directories, along with any scripts you wish to remove." What it doesn't mention (but which should be apparent if you actually run the command) is that it will output the locations of the .egg files or directories and scripts in question, allowing you to copy and paste them to 'rm' or 'del' commands. From p.f.moore at gmail.com Mon Oct 5 17:34:35 2009 From: p.f.moore at gmail.com (Paul Moore) Date: Mon, 5 Oct 2009 16:34:35 +0100 Subject: [Distutils] Package install failures in 2.6.3 - setuptools vs Distribute In-Reply-To: <4ACA0EEF.7010105@noir.com> References: <28320134263785163387302965410901866648-Webmail@me.com> <72523684218386049404529325276719797765-Webmail@me.com> <4ACA049B.5010201@noir.com> <319e029f0910050803j37dedd3eg69f6265d8b22693e@mail.gmail.com> <4ACA0EEF.7010105@noir.com> Message-ID: <79990c6b0910050834s25e92a0q14567c141c7a2775@mail.gmail.com> 2009/10/5 K. Richard Pixley : > I'm recent to python packaging and distribution, so let me see if I've put > this together right from my reading of the various web pages involved over > the weekend. > > Distutils is currently part of the standard python library.? As such, it's > released with python, (the reference implementation, anyway).? Distutils is > currently capable of producing only source archives of packages.? While it's > capable of producing "built" archives, those archives are machine specific, > nonrelocatable, untrackable, and have no standard method for distribution > nor installation nor tracking. > > Setuptools was a third party addition to, (and partial replacement of), > distutils because distutils wasn't suitably usable nor was it moving fast > enough.? However, since setuptools was initiated, many of the major features > of setuptools have since been folded back into distutils, making setuptools > partially redundant and partly colliding.? Setuptools provides the ability > to produce machine independent "built" archives and a standard method for > installing them, (although not for tracking or removing them).? And the > setuptools approach to installation, easy_install, doesn't play nice with > the native installers on systems that have them like rpm, debian, etc. > > However, setuptools has fallen into disrepair and so distribute has been > created, as a friendly branch off a third party tool which in turn was a > form of branch off of distutils.? And within distribute, there are two lines > of development, the 0.7 line, which is intended to replace...? I'm > confused.? Does it replace setuptools or distutils?? And then there's the > 0.6 branch, which is a branch off 0.7 which is a branch of setuptools which > is a branch of distutils which is under recent active development and yet it > also expected to be stable, as much as such a term can be applied to a third > party branch off a third party of a colliding replacement with a standard > facility. > > Is that about right? > > If I'm anywhere near right, then I can't really imagine what state you > intend for the 0.6 branch if not development. I believe that the Distribute 0.6 branch is a stable continuation of the setuptools (0.6) stable branch. Distribute 0.6 is intended to include maintenance-only stable fixes (on the basis that setuptools no longer provides even that minimal level of progress - something on which I won't comment for fear of starting another long thread). Distribute 0.7 is the development branch, looking at new features such as Python 3 support. The only other point you missed is that there is not universal approval of setuptools as a solution to the packaging problem. Take-up of setuptools (and eggs, and easy install, and the various other aspects of the setuptools "ecosystem") is variable, with (as far as I see it) strong support from the web development community, and mixed reception elsewhere in the Python community. Paul. From sridharr at activestate.com Mon Oct 5 21:51:02 2009 From: sridharr at activestate.com (Sridhar Ratnakumar) Date: Mon, 05 Oct 2009 12:51:02 -0700 Subject: [Distutils] Package install failures in 2.6.3 - setuptools vs Distribute In-Reply-To: <20091005182137.854793A4045@sparrow.telecommunity.com> References: <94bdd2610910030649r431a5638y7c8b5332934f5ae@mail.gmail.com> <319e029f0910031308u4182dff2w89760afb4122cf2c@mail.gmail.com> <4AC9BEF5.7080204@taupro.com> <0A60A4FF-D32B-4D35-B607-1A02C838F743@python.org> <53902AAD-D8B3-490B-9228-56416DE5E6D3@zooko.com> <94bdd2610910050711i5d429043he3288986fe470516@mail.gmail.com> <5FA0EFA6-E808-4835-9ACE-98B878193D01@zooko.com> <319e029f0910050953s5246ee71x8906e96fd99e2719@mail.gmail.com> <20091005182137.854793A4045@sparrow.telecommunity.com> Message-ID: On Mon, 05 Oct 2009 11:21:28 -0700, P.J. Eby wrote: > And there's nothing all that special about setuptools' subclassing of > build_ext; in fact, if you look back in the archives here, other people > have done equivalent subclassing to support dynamic library building. I > haven't checked their code, but there is a strong possibility that it > would also fail in the same way. Correct. pywin32-212 broke in similar way - http://bugs.python.org/issue7020 - which issue was fixed in pywin32 trunk. And Tarek commented on the incompleteness of the `get_ext_filename` API: TAREK: "[...] But this API, even if its doctest doesn't make it clear (I will change it to make it clearer) is used for both namespaced names and non namespaced names by the community." Personally, I prefer that setuptools is fixed (in accordance with the clarified API behavior). Yet, 2.6.3 currently has a critical bug open with the logging module that may warrant a quick 2.6.4 release. -srid From wentland at cl.uni-heidelberg.de Mon Oct 5 23:26:44 2009 From: wentland at cl.uni-heidelberg.de (Wolodja Wentland) Date: Mon, 5 Oct 2009 23:26:44 +0200 Subject: [Distutils] distutils: packaging a generated configuration file In-Reply-To: <5c6f2a5d0910051202i2ff95312pb06402d3c3f27d7b@mail.gmail.com> References: <5c6f2a5d0910051202i2ff95312pb06402d3c3f27d7b@mail.gmail.com> Message-ID: <20091005212644.GA30860@kinakuta.local> On Mon, Oct 05, 2009 at 20:02 +0100, Mark Dickinson wrote: [...] > specified with 'package_data' or 'data_files'. The catch is that > parts of the configuration file are generated at setup time: that is, > the setup script gathers various pieces of system information (e.g., > library locations) and uses those to transform a hard-coded > 'config.in' file into the 'config' file that I want to install. [...] How do you get this data? (just curious) > What's the best way to manage generation and installation of the > config file? I've got as far as subclassing the build command and > making __run__ create the config file from the config.in file, putting > it into the build_temp directory; I can't figure out how to tell > distutils to install it in the right place. What do you consider right place? Where do you expect your config file to be installed and how do you find it from within your code? > It seems like I want a sort of build_data/install_data pair of > commands, that allows creation and installation of *generated* data > files. I have been in a similar situation some time ago and had one simple, but unsupported, need when packaging data files. I wanted to access them from within my code after they have been installed. I detailed the problems in [1]. The solution I came up with [2] was to subclass the build_py command in distutils, include a 'build_info.py' at the package root and modify that file within the subclassed build_py command. I am still not sure if that is a good approach and haven't got any feedback on it. If the config file you want to install is *not* a python module and you expect it to be installed in DATA_PREFIX/config you might have some success with subclassing the install_data command, modify the config file before it is copied and include the file with package_data or data_files. You could also try to modify the data_files attribute of that problem if you can't specify all generated data files beforehand. But that might just cause havoc and major breakage... Note that installing data files within the python package violates the FHS, so I would recommend using data_files, if you are interested to have your software included in any *nix or BSD distribution. You will still have a problem to find it later on though. I hope that helps and I am *very* interested in any other way to solve this. good luck Wolodja Wentland [1] http://mail.python.org/pipermail/distutils-sig/2009-September/013238.html [2] http://mail.python.org/pipermail/distutils-sig/2009-September/013284.html -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 835 bytes Desc: Digital signature URL: From janssen at parc.com Tue Oct 6 01:28:49 2009 From: janssen at parc.com (Bill Janssen) Date: Mon, 5 Oct 2009 16:28:49 PDT Subject: [Distutils] Package install failures in 2.6.3 - setuptools vs Distribute In-Reply-To: References: <94bdd2610910030649r431a5638y7c8b5332934f5ae@mail.gmail.com> <319e029f0910031308u4182dff2w89760afb4122cf2c@mail.gmail.com> <4AC9BEF5.7080204@taupro.com> <0A60A4FF-D32B-4D35-B607-1A02C838F743@python.org> <53902AAD-D8B3-490B-9228-56416DE5E6D3@zooko.com> <94bdd2610910050711i5d429043he3288986fe470516@mail.gmail.com> <5FA0EFA6-E808-4835-9ACE-98B878193D01@zooko.com> <319e029f0910050953s5246ee71x8906e96fd99e2719@mail.gmail.com> Message-ID: <2109.1254785329@parc.com> Zooko Wilcox-O'Hearn wrote: > I'm struggling to articulate something here. When the maintainer of > the stable branch of a platform that I rely on says "The fact that > upgrading to our recent stable release will break this critical > functionality is so-and-so's fault, not ours." this reduces my > confidence in that maintainer. Not because he's wrong! Maybe it > *is* so-and-so's fault. But what I'm looking for in the maintainer > of a stable platform is someone who says "Maybe this wasn't our > fault, but here are the steps we're taking to get you back on your > feet as soon as possible.". Zooko, I've struggled with this over the last year, in integrating a dozen fairly complicated third-party Python extensions for my system. I've come to the conclusion that the problem is setuptools, and I'm trying hard to remove it entirely from my system. I have no problem with the .egg format or the basic idea, or even the implementation, which I think is pretty nice. It's the structure of the setuptools project that gives me pause. There seems to be one developer, and he seems to be too busy to fix the well-known bugs (like having easy_install ruin the sys.path settings by putting stuff on it in the wrong place -- is that one fixed yet?). In addition, I think the mere existence of setuptools stifles progress on distutils, which is where all the clever tricks of setuptools should properly appear. This would let the whole community of Python developers work on the codebase. I'd like to see a flag on PyPI marking whether the package relies on setuptools, in which case I'll avoid it, or voluntarily entangles itself with setuptools if present (as the only known way to create eggs), or is setuptools-free (my preferred configuration). Frankly, I'd also recommend putting up a warning to developers on PyPI noting these problems with setuptools. Bill From janssen at parc.com Tue Oct 6 01:33:15 2009 From: janssen at parc.com (Bill Janssen) Date: Mon, 5 Oct 2009 16:33:15 PDT Subject: [Distutils] how to build location-independent bdist packages on OS X? Message-ID: <2261.1254785595@parc.com> I'm trying to bundle up a Python package with a C extension in it for Python 2.6 on an OS X 10.6 system. I don't want to install it in the normal /Library/Python/2.6/site-packages/ location. But when I try "python setup.py bdist", it builds a tar file with the prefix /Library/Python/2.6/site-packages as a prefix to each file. I then tried "python setup.py bdist_dumb" with the same result. I then tried "python setup.py bdist --prefix=.", but "--prefix" isn't recognized by bdist. What's the appropriate magic here? Oh, and I don't want to have to use setuptools. TIA, Bill From p.f.moore at gmail.com Mon Oct 5 17:38:39 2009 From: p.f.moore at gmail.com (Paul Moore) Date: Mon, 5 Oct 2009 16:38:39 +0100 Subject: [Distutils] Package install failures in 2.6.3 - setuptools vs Distribute In-Reply-To: References: <28320134263785163387302965410901866648-Webmail@me.com> <72523684218386049404529325276719797765-Webmail@me.com> <4ACA01E8.8020504@noir.com> Message-ID: <79990c6b0910050838j4cba0c2bt2378cf7f17e9df84@mail.gmail.com> 2009/10/5 Jeremy Sanders : > As a general question, is there any planned project to improve the state of > distutils or replace it? It appears to be one of the weakest parts of the > Python system and needs replacing with something much cleaner, better > documented and more powerful. Tarek's working on enhancements to distutils. The current breakage of setuptools demonstrates why this is a slow, painful, process :-( > Even making something like cmake the standard would help a lot. There's no plan for anything like that. I'll leave it to you to imagine the cries of horror from the community when such a radical change meant that easy_install no longer worked, and there was no prospect of fixing it :-) Paul. From alex.gronholm at nextday.fi Tue Oct 6 05:50:09 2009 From: alex.gronholm at nextday.fi (=?ISO-8859-1?Q?Alex_Gr=F6nholm?=) Date: Tue, 06 Oct 2009 06:50:09 +0300 Subject: [Distutils] Package install failures in 2.6.3 - setuptools vs Distribute In-Reply-To: <2109.1254785329@parc.com> References: <94bdd2610910030649r431a5638y7c8b5332934f5ae@mail.gmail.com> <319e029f0910031308u4182dff2w89760afb4122cf2c@mail.gmail.com> <4AC9BEF5.7080204@taupro.com> <0A60A4FF-D32B-4D35-B607-1A02C838F743@python.org> <53902AAD-D8B3-490B-9228-56416DE5E6D3@zooko.com> <94bdd2610910050711i5d429043he3288986fe470516@mail.gmail.com> <5FA0EFA6-E808-4835-9ACE-98B878193D01@zooko.com> <319e029f0910050953s5246ee71x8906e96fd99e2719@mail.gmail.com> <2109.1254785329@parc.com> Message-ID: <4ACABE71.60602@nextday.fi> Bill Janssen kirjoitti: > Zooko Wilcox-O'Hearn wrote: > > >> I'm struggling to articulate something here. When the maintainer of >> the stable branch of a platform that I rely on says "The fact that >> upgrading to our recent stable release will break this critical >> functionality is so-and-so's fault, not ours." this reduces my >> confidence in that maintainer. Not because he's wrong! Maybe it >> *is* so-and-so's fault. But what I'm looking for in the maintainer >> of a stable platform is someone who says "Maybe this wasn't our >> fault, but here are the steps we're taking to get you back on your >> feet as soon as possible.". >> > > Zooko, I've struggled with this over the last year, in integrating a > dozen fairly complicated third-party Python extensions for my system. > > I've come to the conclusion that the problem is setuptools, and I'm > trying hard to remove it entirely from my system. > > I have no problem with the .egg format or the basic idea, or even the > implementation, which I think is pretty nice. It's the structure of the > setuptools project that gives me pause. There seems to be one > developer, and he seems to be too busy to fix the well-known bugs (like > having easy_install ruin the sys.path settings by putting stuff on it in > the wrong place -- is that one fixed yet?). > > In addition, I think the mere existence of setuptools stifles progress > on distutils, which is where all the clever tricks of setuptools should > properly appear. This would let the whole community of Python developers > work on the codebase. > > I'd like to see a flag on PyPI marking whether the package relies on > setuptools, in which case I'll avoid it, or voluntarily entangles itself > with setuptools if present (as the only known way to create eggs), or is > setuptools-free (my preferred configuration). Frankly, I'd also recommend > putting up a warning to developers on PyPI noting these problems with > setuptools. > > Does your bug still exist in Distribute? If so, please report it at http://bitbucket.org/tarek/distribute/ (assuming that bitbucket is operational, which it currently isn't) > Bill > _______________________________________________ > Distutils-SIG maillist - Distutils-SIG at python.org > http://mail.python.org/mailman/listinfo/distutils-sig > From greno at verizon.net Tue Oct 6 05:51:59 2009 From: greno at verizon.net (Gerry Reno) Date: Mon, 05 Oct 2009 23:51:59 -0400 Subject: [Distutils] [ANN] stdeb 0.3.2 and 0.4.1 In-Reply-To: <4ACA233B.8030107@verizon.net> References: <4AC8E5D7.8010602@astraw.com> <4ACA233B.8030107@verizon.net> Message-ID: <4ACABEDF.7070706@verizon.net> Andrew, Have you already or are you planning on submitting 'python-stdeb' packages to the debian and ubuntu repositories? Regards, Gerry From janssen at parc.com Tue Oct 6 06:43:04 2009 From: janssen at parc.com (Bill Janssen) Date: Mon, 5 Oct 2009 21:43:04 PDT Subject: [Distutils] Package install failures in 2.6.3 - setuptools vs Distribute In-Reply-To: <4ACABE71.60602@nextday.fi> References: <94bdd2610910030649r431a5638y7c8b5332934f5ae@mail.gmail.com> <319e029f0910031308u4182dff2w89760afb4122cf2c@mail.gmail.com> <4AC9BEF5.7080204@taupro.com> <0A60A4FF-D32B-4D35-B607-1A02C838F743@python.org> <53902AAD-D8B3-490B-9228-56416DE5E6D3@zooko.com> <94bdd2610910050711i5d429043he3288986fe470516@mail.gmail.com> <5FA0EFA6-E808-4835-9ACE-98B878193D01@zooko.com> <319e029f0910050953s5246ee71x8906e96fd99e2719@mail.gmail.com> <2109.1254785329@parc.com> <4ACABE71.60602@nextday.fi> Message-ID: <4954.1254804184@parc.com> Alex Gr?nholm wrote: > Does your bug still exist in Distribute? If so, please report it at > http://bitbucket.org/tarek/distribute/ (assuming that bitbucket is > operational, which it currently isn't) Sorry, Alex, I don't know about Distribute, don't (particularly) care. If you care, test for it and report it if it's there. It's bug 53 in the setuptools bug reporter. Bill From alex.gronholm at nextday.fi Tue Oct 6 07:24:23 2009 From: alex.gronholm at nextday.fi (=?UTF-8?B?QWxleCBHcsO2bmhvbG0=?=) Date: Tue, 06 Oct 2009 08:24:23 +0300 Subject: [Distutils] Package install failures in 2.6.3 - setuptools vs Distribute In-Reply-To: <4954.1254804184@parc.com> References: <94bdd2610910030649r431a5638y7c8b5332934f5ae@mail.gmail.com> <319e029f0910031308u4182dff2w89760afb4122cf2c@mail.gmail.com> <4AC9BEF5.7080204@taupro.com> <0A60A4FF-D32B-4D35-B607-1A02C838F743@python.org> <53902AAD-D8B3-490B-9228-56416DE5E6D3@zooko.com> <94bdd2610910050711i5d429043he3288986fe470516@mail.gmail.com> <5FA0EFA6-E808-4835-9ACE-98B878193D01@zooko.com> <319e029f0910050953s5246ee71x8906e96fd99e2719@mail.gmail.com> <2109.1254785329@parc.com> <4ACABE71.60602@nextday.fi> <4954.1254804184@parc.com> Message-ID: <4ACAD487.9070403@nextday.fi> Bill Janssen kirjoitti: > Alex Gr?nholm wrote: > > >> Does your bug still exist in Distribute? If so, please report it at >> http://bitbucket.org/tarek/distribute/ (assuming that bitbucket is >> operational, which it currently isn't) >> > > Sorry, Alex, I don't know about Distribute, don't (particularly) care. > If you care, test for it and report it if it's there. It's bug 53 in > the setuptools bug reporter. > > Bill > If you are seriously expecting setuptools to be fixed, I can only assume you haven't been following the conversation on this list. From ronaldoussoren at mac.com Tue Oct 6 07:37:08 2009 From: ronaldoussoren at mac.com (Ronald Oussoren) Date: Tue, 06 Oct 2009 07:37:08 +0200 Subject: [Distutils] Package install failures in 2.6.3 - setuptools vs Distribute In-Reply-To: <4ACA01E8.8020504@noir.com> References: <28320134263785163387302965410901866648-Webmail@me.com> <72523684218386049404529325276719797765-Webmail@me.com> <4ACA01E8.8020504@noir.com> Message-ID: <15AF73B1-4C40-4D22-ABC6-07A794FAB913@mac.com> On 5 Oct, 2009, at 16:25, K. Richard Pixley wrote: > Ronald Oussoren wrote: >> For beginners this issue is a showstopper that they cannot resolve >> without help. >> > I'm a relative beginner to distutils/setuptools/distribute, but a > long time configuration/build/packaging professional. You're > mistaken if you think that any of these technologies are suitable > for beginners. The state of python package distribution resembles > the state of linux packages circa 1995, except that it isn't very > well documented at all. I didn't say that distutils is suitable for beginners, but beginners to use them and are confused when they stop working. > > Easy_install certainly isn't, and never has been. My first > questions about easy_install, (how do I get a list of installed > packages? How do I upgrade all installed packages? How do I delete > a package using easy_install?) are all unanswerable. All of these are not yet implemented. PJE has mentioned that he wants to work on this in for setuptools 0.7, but development of setuptools has completely stalled. The distribute package was created because of this, and development is moving forward again. Ronald From ronaldoussoren at mac.com Tue Oct 6 07:43:54 2009 From: ronaldoussoren at mac.com (Ronald Oussoren) Date: Tue, 06 Oct 2009 07:43:54 +0200 Subject: [Distutils] Package install failures in 2.6.3 - setuptools vs Distribute In-Reply-To: <4ACA049B.5010201@noir.com> References: <28320134263785163387302965410901866648-Webmail@me.com> <72523684218386049404529325276719797765-Webmail@me.com> <4ACA049B.5010201@noir.com> Message-ID: <10081E45-60AF-4621-BD12-981E8C5703C0@mac.com> On 5 Oct, 2009, at 16:37, K. Richard Pixley wrote: > Ronald Oussoren wrote: >> This is a problem, it means 2.6.3 is not a simple drop-in >> replacement for 2.6.2 but requires the replacement of another >> component as well. That can be a problem in organizations with >> strict configuration management where you cannot install new >> software without going to lots of red tape (and that might involve >> lawyers when you install a new package instead of upgrading an >> existing one). > This would be a problem if distribute were in general release. It's > not. It's clearly a development branch which is intended to move > quickly. People using distribute are taking development, pre-alpha > kinds of risk and that has been made pretty clear already. AFAIK distribute 0.6 is a stable release, basicly "setuptools 0.6c9 + bugfixes + py3k support". Installing distribute is therefore not problematic for most people, if they know that the project exists. The fact that distribute is a seperate project from setuptools can be a problem for people: installing a bugfix release for a software product that we're already using at work is significantly easier than introducing a new software product. Ronald From ronaldoussoren at mac.com Tue Oct 6 08:00:22 2009 From: ronaldoussoren at mac.com (Ronald Oussoren) Date: Tue, 06 Oct 2009 08:00:22 +0200 Subject: [Distutils] Package install failures in 2.6.3 - setuptools vs Distribute In-Reply-To: <94bdd2610910050454p41c6f86ex2d97502133f5562a@mail.gmail.com> References: <28320134263785163387302965410901866648-Webmail@me.com> <72523684218386049404529325276719797765-Webmail@me.com> <94bdd2610910050454p41c6f86ex2d97502133f5562a@mail.gmail.com> Message-ID: <29D80A87-E881-4139-8267-16735406DB12@mac.com> On 5 Oct, 2009, at 13:54, Tarek Ziad? wrote: > On Mon, Oct 5, 2009 at 1:30 PM, Ronald Oussoren > wrote: >>> Nobody will adopt it until they are forced to. This unfortunate bug >>> means people are forced to quicker than expected. I don't think >>> that's >>> an actual problem. >> >> This is a problem, it means 2.6.3 is not a simple drop-in >> replacement for 2.6.2 but requires the replacement of another >> component as well. That can be a problem in organizations with >> strict configuration management where you cannot install new >> software without going to lots of red tape (and that might involve >> lawyers when you install a new package instead of upgrading an >> existing one). > > What is your solution ? Setuptools is not part of the standard library > and monkey-patch Distutils. > Its development has been discontinued for over a year. I don't have a real solution, beyond documentation. IMHO the issue should have been documented in the 2.6.3 release notes, which assumes that the issue was know before the release. > > Everytime Distutils is changed for anything, wether it's a bug fix or > not, Setuptools can be broken. > > Now I realize that some folks wants Distutils to be aware of that and > be backward-compatible with > Setuptools monkey-patches ? I'd prefer if distutils didn't break setuptools in the 2.6 branch, breaking it in 2.7 is fine because setuptools hooks into distutils at a very low level and is therefore sensitive too changes in implementation details of distutils. And if breaking setuptools in the 2.6 branch is unavoidable this should be noted in Python's release notes. Whether we like it or not setuptools, and easy_install in particular, is used a lot and not only by power users. Anyway, the issue is less relevant at the moment the NEWS file in the 2.6 branch says that a fix for the setuptools breakage will be in 2.6.4 when that's released. > > Like for the svn 1.6 compat problem we had earlier this year, > this problem is a few line changes in Setuptools, it's an 1 hour work. > If your company upgrades to Python 2.6.3 it can also upgrade to an > hypothetical Setuptools 0.6c..10 ? Sure, but 0.6c10 is only hypothetical at the moment. BTW. This doesn't mean that I don't appreciate distribute, I'm switching to that in the near future because there is real progress in distribute and it has support for py3k. Ronald From david at ar.media.kyoto-u.ac.jp Tue Oct 6 07:47:38 2009 From: david at ar.media.kyoto-u.ac.jp (David Cournapeau) Date: Tue, 06 Oct 2009 14:47:38 +0900 Subject: [Distutils] Package install failures in 2.6.3 - setuptools vs Distribute In-Reply-To: <94bdd2610910050732v35984fdfqdd1302539f77e99b@mail.gmail.com> References: <28320134263785163387302965410901866648-Webmail@me.com> <72523684218386049404529325276719797765-Webmail@me.com> <319e029f0910050444t25761d79gce49d7ef13d98191@mail.gmail.com> <9EDFE750-4102-49A4-8B35-34E6E7AE205C@gmail.com> <94bdd2610910050732v35984fdfqdd1302539f77e99b@mail.gmail.com> Message-ID: <4ACAD9FA.1020907@ar.media.kyoto-u.ac.jp> Tarek Ziad? wrote: > The proper answer is : Setuptools is on the top of Distutils and has > to evolve with it. > And since it monkey patches it, it has to be changed when a Distutils > release breaks it. I want to note that the issue here is not monkey-patching, it is subclassing the command classes. If two tools like setuptools each subclass a distutils command, they cannot work together, it is impossible by the very 'design' of distutils, that's not setuptools' fault. numpy.distutils has the same problem (and I am sure tools like paver or pip as well): everytime distutils changes its implementation, some things are broken and we have to adapt. The only way to make this work is to test everything combination and making sure each extension does not walk on each other - that's one of the core reason why distutils is such a mess IMO. cheers, David From alex.gronholm at nextday.fi Tue Oct 6 08:20:48 2009 From: alex.gronholm at nextday.fi (=?ISO-8859-1?Q?Alex_Gr=F6nholm?=) Date: Tue, 06 Oct 2009 09:20:48 +0300 Subject: [Distutils] Package install failures in 2.6.3 - setuptools vs Distribute In-Reply-To: <20091005180304.71CF63A4045@sparrow.telecommunity.com> References: <319e029f0910030515u71325b88h89037b3ea85f65b5@mail.gmail.com> <94bdd2610910030649r431a5638y7c8b5332934f5ae@mail.gmail.com> <319e029f0910031308u4182dff2w89760afb4122cf2c@mail.gmail.com> <4AC9BEF5.7080204@taupro.com> <319e029f0910050253x1ad1cc7eg60e7f11b3f9f72b8@mail.gmail.com> <20091005180304.71CF63A4045@sparrow.telecommunity.com> Message-ID: <4ACAE1C0.4040403@nextday.fi> P.J. Eby kirjoitti: > At 11:53 AM 10/5/2009 +0200, Lennart Regebro wrote: >> 2009/10/5 Jeff Rush : >> > Very unfortunate, as in, it should NOT have happened. And >> *especially* >> > without any announcement on python.org or mention on the >> > python-committers list of something this major. >> >> Well "this major"... It's a bug fix that breaks a setuptools >> monkey-patch. > > Subclassing distutils commands != monkeypatching. If, say, numpy's > distutils extensions subclass build_ext and override that method, they > could have had the same problem. Same for anybody else's distutils > extensions. > Setuptools subclasses distutils commands and then replaces the original classes with its own. Example from setuptools/__init__.py: import distutils.core distutils.core.Command = Command Isn't this exactly what monkey patching means? > _______________________________________________ > Distutils-SIG maillist - Distutils-SIG at python.org > http://mail.python.org/mailman/listinfo/distutils-sig From david at ar.media.kyoto-u.ac.jp Tue Oct 6 08:00:10 2009 From: david at ar.media.kyoto-u.ac.jp (David Cournapeau) Date: Tue, 06 Oct 2009 15:00:10 +0900 Subject: [Distutils] Package install failures in 2.6.3 - setuptools vs Distribute In-Reply-To: <4ACA2088.2010104@noir.com> References: <28320134263785163387302965410901866648-Webmail@me.com> <72523684218386049404529325276719797765-Webmail@me.com> <4ACA01E8.8020504@noir.com> <4ACA1A65.702@nextday.fi> <4ACA2088.2010104@noir.com> Message-ID: <4ACADCEA.8020608@ar.media.kyoto-u.ac.jp> K. Richard Pixley wrote: > Alex Gr?nholm wrote: >> There is a lack of consensus regarding how exactly they should work. >> If we are having this much trouble deciding how a third party tool >> should work, it is certainly not going to be merged into distutils >> until those issues have been resolved. Distutils is what houses (or >> should) the parts we all agree on. That said, I think that plenty of >> setuptools/distribute functionality should be moved to distutils >> (after the code has been cleaned up and the proper unit tests >> introduced). > I agree there's a lack of consensus. But I dont' believe that > distutils is a strong basis for growth. Distutils may be a reasonable > choice of a build tool, (I'm not sure yet), but it's packaging and > distribution support is minimal to nonexistent. It is certainly not a good basis as a build tool - it does not handle dependencies for once, and the only thing it really brings is a poor cross-platform implementation to build dynamically loaded libraries, without even a coherent and documented API (you can't introspect something as trivial as compilation flags or where files are built in a cross-platform way, for example). > Most of what I'm talking about here speaks to packaging formats, > distribution processes, and installation processes. And this isn't > new technology. Both debian, rpm, and several other unix technologies > have fine systems in operation right now. Sure, they all have > weaknesses, but they are much better than easy_install. Not if you don't want/can't spend quite some time to know about each supported platform and the tool details. cheers, David From nad at acm.org Tue Oct 6 10:20:43 2009 From: nad at acm.org (Ned Deily) Date: Tue, 06 Oct 2009 01:20:43 -0700 Subject: [Distutils] Package install failures in 2.6.3 - setuptools vs Distribute References: <28320134263785163387302965410901866648-Webmail@me.com> <72523684218386049404529325276719797765-Webmail@me.com> <94bdd2610910050454p41c6f86ex2d97502133f5562a@mail.gmail.com> <29D80A87-E881-4139-8267-16735406DB12@mac.com> Message-ID: An update on the immediate issue: after discussion elsewhere, it was decided that there were enough other problems with 2.6.3 to warrant a quick release of 2.6.4. Tarek has checked in a change to distutils to unbreak the setuptools currently out in the field. If all goes well, 2.6.4 should be released in a couple of weeks. Thanks, Tarek. -- Ned Deily, nad at acm.org From chris at simplistix.co.uk Tue Oct 6 12:26:34 2009 From: chris at simplistix.co.uk (Chris Withers) Date: Tue, 06 Oct 2009 11:26:34 +0100 Subject: [Distutils] Package install failures in 2.6.3 - setuptools vs Distribute In-Reply-To: <4AC8A396.8030306@cryptelium.net> References: <4AC8A396.8030306@cryptelium.net> Message-ID: <4ACB1B5A.2090608@simplistix.co.uk> kiorky wrote: > Hi, > for the folks using virtualenv-distribute, i forked it to make the last 0.6.3 > install instead of 0.6.1. > > See : > http://bitbucket.org/kiorky/virtualenv-distribute/ > > Install it: > > easy_install > http://distfiles.minitage.org/public/externals/minitage/virtualenv-distribute-1.3.5dev-1.zip So this is a fork of a fork? Nice :-( Chris -- Simplistix - Content Management, Batch Processing & Python Consulting - http://www.simplistix.co.uk From regebro at gmail.com Tue Oct 6 13:02:19 2009 From: regebro at gmail.com (Lennart Regebro) Date: Tue, 6 Oct 2009 13:02:19 +0200 Subject: [Distutils] Package install failures in 2.6.3 - setuptools vs Distribute In-Reply-To: <4ACB1B5A.2090608@simplistix.co.uk> References: <4AC8A396.8030306@cryptelium.net> <4ACB1B5A.2090608@simplistix.co.uk> Message-ID: <319e029f0910060402n306a992as4b9d854f7f29d546@mail.gmail.com> 2009/10/6 Chris Withers : > kiorky wrote: >> >> Hi, >> for the folks using virtualenv-distribute, i forked it to make the last >> 0.6.3 >> install instead of 0.6.1. >> >> See : >> http://bitbucket.org/kiorky/virtualenv-distribute/ >> >> Install it: >> >> easy_install >> >> http://distfiles.minitage.org/public/externals/minitage/virtualenv-distribute-1.3.5dev-1.zip > > So this is a fork of a fork? Nice :-( I think it's a fork of Virtualenv, no? Which uses a fork of distribute. :) We should get virtualenv to support Python3, and I suspect that means we need to get it to support Distribute on Python 2 as well, which probably is a good thing anyway. I do fear trying to understand how it works though, but maybe we can bribe/trick Ian into doing all the work it himself. :-) -- Lennart Regebro: Python, Zope, Plone, Grok http://regebro.wordpress.com/ +33 661 58 14 64 From regebro at gmail.com Tue Oct 6 13:02:58 2009 From: regebro at gmail.com (Lennart Regebro) Date: Tue, 6 Oct 2009 13:02:58 +0200 Subject: [Distutils] Package install failures in 2.6.3 - setuptools vs Distribute In-Reply-To: <319e029f0910060402n306a992as4b9d854f7f29d546@mail.gmail.com> References: <4AC8A396.8030306@cryptelium.net> <4ACB1B5A.2090608@simplistix.co.uk> <319e029f0910060402n306a992as4b9d854f7f29d546@mail.gmail.com> Message-ID: <319e029f0910060402m28ab35acnc9a52aefcdc6f04a@mail.gmail.com> 2009/10/6 Lennart Regebro : > I think it's a fork of Virtualenv, no? Which uses a fork of distribute. :) I meant that it uses a fork of setuptools, obviously.... -- Lennart Regebro: Python, Zope, Plone, Grok http://regebro.wordpress.com/ +33 661 58 14 64 From kiorky at cryptelium.net Tue Oct 6 13:06:22 2009 From: kiorky at cryptelium.net (kiorky) Date: Tue, 06 Oct 2009 13:06:22 +0200 Subject: [Distutils] Package install failures in 2.6.3 - setuptools vs Distribute In-Reply-To: <319e029f0910060402m28ab35acnc9a52aefcdc6f04a@mail.gmail.com> References: <4AC8A396.8030306@cryptelium.net> <4ACB1B5A.2090608@simplistix.co.uk> <319e029f0910060402n306a992as4b9d854f7f29d546@mail.gmail.com> <319e029f0910060402m28ab35acnc9a52aefcdc6f04a@mail.gmail.com> Message-ID: <4ACB24AE.4000300@cryptelium.net> It's just a fork of virtualenv to use distribute. It does not use a fork of distribute but distribute itself ;) Lennart Regebro a ?crit : > 2009/10/6 Lennart Regebro : >> I think it's a fork of Virtualenv, no? Which uses a fork of distribute. :) > > I meant that it uses a fork of setuptools, obviously.... > -- -- Cordialement, KiOrKY GPG Key FingerPrint: 0x1A1194B7681112AF -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 261 bytes Desc: OpenPGP digital signature URL: From kiorky at cryptelium.net Tue Oct 6 13:08:23 2009 From: kiorky at cryptelium.net (kiorky) Date: Tue, 06 Oct 2009 13:08:23 +0200 Subject: [Distutils] Package install failures in 2.6.3 - setuptools vs Distribute In-Reply-To: <4ACB24AE.4000300@cryptelium.net> References: <4AC8A396.8030306@cryptelium.net> <4ACB1B5A.2090608@simplistix.co.uk> <319e029f0910060402n306a992as4b9d854f7f29d546@mail.gmail.com> <319e029f0910060402m28ab35acnc9a52aefcdc6f04a@mail.gmail.com> <4ACB24AE.4000300@cryptelium.net> Message-ID: <4ACB2527.7010802@cryptelium.net> Anyway, it's released now on pypi as virtualenv-distribute-1.3.4.2. The code is also merged in florian branch and it has been decided that's the main repository. kiorky a ?crit : > It's just a fork of virtualenv to use distribute. > It does not use a fork of distribute but distribute itself ;) > > Lennart Regebro a ?crit : >> 2009/10/6 Lennart Regebro : >>> I think it's a fork of Virtualenv, no? Which uses a fork of distribute. :) >> I meant that it uses a fork of setuptools, obviously.... >> > > > ------------------------------------------------------------------------ > > _______________________________________________ > Distutils-SIG maillist - Distutils-SIG at python.org > http://mail.python.org/mailman/listinfo/distutils-sig -- -- Cordialement, KiOrKY GPG Key FingerPrint: 0x1A1194B7681112AF -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 261 bytes Desc: OpenPGP digital signature URL: From regebro at gmail.com Tue Oct 6 14:22:38 2009 From: regebro at gmail.com (Lennart Regebro) Date: Tue, 6 Oct 2009 14:22:38 +0200 Subject: [Distutils] Package install failures in 2.6.3 - setuptools vs Distribute In-Reply-To: <4ACB2527.7010802@cryptelium.net> References: <4AC8A396.8030306@cryptelium.net> <4ACB1B5A.2090608@simplistix.co.uk> <319e029f0910060402n306a992as4b9d854f7f29d546@mail.gmail.com> <319e029f0910060402m28ab35acnc9a52aefcdc6f04a@mail.gmail.com> <4ACB24AE.4000300@cryptelium.net> <4ACB2527.7010802@cryptelium.net> Message-ID: <319e029f0910060522l29499f46y2805ed0082d975c3@mail.gmail.com> 2009/10/6 kiorky : > Anyway, it's released now on pypi as virtualenv-distribute-1.3.4.2. > > The code is also merged in florian branch and it has been decided that's the > main repository. What is the florian branch, and in general, could you provide some more info. Your emails are very cryptic. :) -- Lennart Regebro: Python, Zope, Plone, Grok http://regebro.wordpress.com/ +33 661 58 14 64 From ralf.gommers at googlemail.com Tue Oct 6 16:37:30 2009 From: ralf.gommers at googlemail.com (Ralf Gommers) Date: Tue, 6 Oct 2009 16:37:30 +0200 Subject: [Distutils] Distribute install problem on Snow Leopard Message-ID: Hi, I installed Distribute on a Snow Leopard box using the default Python that comes with OS X. I used the installation method recommended in the docs, the distribute_setup.py script. Now easy_install fails no matter what package I try to install (it does not get around to looking for the name of the requested package): $ easy_install blablabla Traceback (most recent call last): File "/usr/bin/easy_install-2.6", line 10, in load_entry_point('setuptools==0.6c9', 'console_scripts', 'easy_install')() File "/Library/Python/2.6/site-packages/distribute-0.6.3-py2.6.egg/pkg_resources.py", line 281, in load_entry_point return get_distribution(dist).load_entry_point(group, name) File "/Library/Python/2.6/site-packages/distribute-0.6.3-py2.6.egg/pkg_resources.py", line 2197, in load_entry_point raise ImportError("Entry point %r not found" % ((group,name),)) ImportError: Entry point ('console_scripts', 'easy_install') not found Did I forget something, or is this a bug? Any suggestions on how to fix it? Cheers, Ralf -------------- next part -------------- An HTML attachment was scrubbed... URL: From ziade.tarek at gmail.com Tue Oct 6 16:43:56 2009 From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=) Date: Tue, 6 Oct 2009 16:43:56 +0200 Subject: [Distutils] Distribute install problem on Snow Leopard In-Reply-To: References: Message-ID: <94bdd2610910060743v4c8b8808pdbfcadef06c2a472@mail.gmail.com> On Tue, Oct 6, 2009 at 4:37 PM, Ralf Gommers wrote: > Hi, > > I installed Distribute on a Snow Leopard box using the default Python that > comes with OS X. I used the installation method recommended in the docs, the > distribute_setup.py script. Now easy_install fails no matter what package I > try to install (it does not get around to looking for the name of the > requested package): > > $ easy_install blablabla > Traceback (most recent call last): > ? File "/usr/bin/easy_install-2.6", line 10, in > ??? load_entry_point('setuptools==0.6c9', 'console_scripts', > 'easy_install')() > ? File > "/Library/Python/2.6/site-packages/distribute-0.6.3-py2.6.egg/pkg_resources.py", > line 281, in load_entry_point > ??? return get_distribution(dist).load_entry_point(group, name) > ? File > "/Library/Python/2.6/site-packages/distribute-0.6.3-py2.6.egg/pkg_resources.py", > line 2197, in load_entry_point > ??? raise ImportError("Entry point %r not found" % ((group,name),)) > ImportError: Entry point ('console_scripts', 'easy_install') not found > > > Did I forget something, or is this a bug? Any suggestions on how to fix it? This is a bug. It seems that the installation did not upgrade the "easy_install-2.6" script located in your "/usr/bin". We are going to investigate. The simplest way to fix this is to change in that script the name of the distribution: > load_entry_point('setuptools==0.6c9', 'console_scripts', 'easy_install')() by > load_entry_point('distribute==0.6.3', 'console_scripts', 'easy_install')() Let us know how it goes Tarek -- Tarek Ziad? | http://ziade.org |????????????! |???????????? From ralf.gommers at googlemail.com Tue Oct 6 16:55:32 2009 From: ralf.gommers at googlemail.com (Ralf Gommers) Date: Tue, 6 Oct 2009 16:55:32 +0200 Subject: [Distutils] Distribute install problem on Snow Leopard In-Reply-To: <94bdd2610910060743v4c8b8808pdbfcadef06c2a472@mail.gmail.com> References: <94bdd2610910060743v4c8b8808pdbfcadef06c2a472@mail.gmail.com> Message-ID: On Tue, Oct 6, 2009 at 4:43 PM, Tarek Ziad? wrote: > On Tue, Oct 6, 2009 at 4:37 PM, Ralf Gommers > wrote: > > Hi, > > > > I installed Distribute on a Snow Leopard box using the default Python > that > > comes with OS X. I used the installation method recommended in the docs, > the > > distribute_setup.py script. Now easy_install fails no matter what package > I > > try to install (it does not get around to looking for the name of the > > requested package): > > > > $ easy_install blablabla > > Traceback (most recent call last): > > File "/usr/bin/easy_install-2.6", line 10, in > > load_entry_point('setuptools==0.6c9', 'console_scripts', > > 'easy_install')() > > File > > > "/Library/Python/2.6/site-packages/distribute-0.6.3-py2.6.egg/pkg_resources.py", > > line 281, in load_entry_point > > return get_distribution(dist).load_entry_point(group, name) > > File > > > "/Library/Python/2.6/site-packages/distribute-0.6.3-py2.6.egg/pkg_resources.py", > > line 2197, in load_entry_point > > raise ImportError("Entry point %r not found" % ((group,name),)) > > ImportError: Entry point ('console_scripts', 'easy_install') not found > > > > > > Did I forget something, or is this a bug? Any suggestions on how to fix > it? > > This is a bug. It seems that the installation did not upgrade the > "easy_install-2.6" script located > in your "/usr/bin". We are going to investigate. > > The simplest way to fix this is to change in that script the name of > the distribution: > > > load_entry_point('setuptools==0.6c9', 'console_scripts', > 'easy_install')() > > by > > > load_entry_point('distribute==0.6.3', 'console_scripts', > 'easy_install')() > > Let us know how it goes > Yes, that fixed it. Thanks! Ralf > > Tarek > > -- > Tarek Ziad? | http://ziade.org | ???????????! | ??????????? > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ziade.tarek at gmail.com Tue Oct 6 17:00:08 2009 From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=) Date: Tue, 6 Oct 2009 17:00:08 +0200 Subject: [Distutils] Distribute install problem on Snow Leopard In-Reply-To: References: <94bdd2610910060743v4c8b8808pdbfcadef06c2a472@mail.gmail.com> Message-ID: <94bdd2610910060800n7d981ea7lbf5443aad91d4fe@mail.gmail.com> On Tue, Oct 6, 2009 at 4:55 PM, Ralf Gommers wrote: > > Yes, that fixed it. Thanks! Great. The bug is still to be fixed though, would you mind adding an issue following this link: http://bitbucket.org/tarek/distribute/issues/new/ Giving the details. Thanks From ralf.gommers at googlemail.com Tue Oct 6 17:12:47 2009 From: ralf.gommers at googlemail.com (Ralf Gommers) Date: Tue, 6 Oct 2009 17:12:47 +0200 Subject: [Distutils] Distribute install problem on Snow Leopard In-Reply-To: <94bdd2610910060800n7d981ea7lbf5443aad91d4fe@mail.gmail.com> References: <94bdd2610910060743v4c8b8808pdbfcadef06c2a472@mail.gmail.com> <94bdd2610910060800n7d981ea7lbf5443aad91d4fe@mail.gmail.com> Message-ID: On Tue, Oct 6, 2009 at 5:00 PM, Tarek Ziad? wrote: > On Tue, Oct 6, 2009 at 4:55 PM, Ralf Gommers > wrote: > > > > Yes, that fixed it. Thanks! > > Great. The bug is still to be fixed though, would you mind adding an > issue following this link: > > http://bitbucket.org/tarek/distribute/issues/new/ > > Giving the details. > > Thanks > Reported as issue #59. -------------- next part -------------- An HTML attachment was scrubbed... URL: From ziade.tarek at gmail.com Tue Oct 6 17:13:32 2009 From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=) Date: Tue, 6 Oct 2009 17:13:32 +0200 Subject: [Distutils] Distribute install problem on Snow Leopard In-Reply-To: References: <94bdd2610910060743v4c8b8808pdbfcadef06c2a472@mail.gmail.com> <94bdd2610910060800n7d981ea7lbf5443aad91d4fe@mail.gmail.com> Message-ID: <94bdd2610910060813v71b2dc94o2301ea6faa96a512@mail.gmail.com> On Tue, Oct 6, 2009 at 5:12 PM, Ralf Gommers wrote: > > Reported as issue #59. > Thank you > -- Tarek Ziad? | http://ziade.org |????????????! |???????????? From janssen at parc.com Tue Oct 6 19:00:54 2009 From: janssen at parc.com (Bill Janssen) Date: Tue, 6 Oct 2009 10:00:54 PDT Subject: [Distutils] Package install failures in 2.6.3 - setuptools vs Distribute In-Reply-To: <10081E45-60AF-4621-BD12-981E8C5703C0@mac.com> References: <28320134263785163387302965410901866648-Webmail@me.com> <72523684218386049404529325276719797765-Webmail@me.com> <4ACA049B.5010201@noir.com> <10081E45-60AF-4621-BD12-981E8C5703C0@mac.com> Message-ID: <2380.1254848454@parc.com> Ronald Oussoren wrote: > Installing distribute is therefore not problematic for most people, if > they know that the project exists. The fact that distribute is a > seperate project from setuptools can be a problem for people: > installing a bugfix release for a software product that we're already > using at work is significantly easier than introducing a new software > product. For me, it's more a matter of "OS X 10.6 already comes with setuptools; how can I mitigate the impact of this buggy unmaintained package on the systems I'm building to deploy on OS X?". Adding distribute to the mix, however good it is, doesn't help; I'm going to use a pure distutils solution. And I can't really install a bugfix release for the Python frameworks in /System; only Apple can do that. Bill From kiorky at cryptelium.net Tue Oct 6 19:25:04 2009 From: kiorky at cryptelium.net (kiorky) Date: Tue, 06 Oct 2009 19:25:04 +0200 Subject: [Distutils] Package install failures in 2.6.3 - setuptools vs Distribute In-Reply-To: <319e029f0910060522l29499f46y2805ed0082d975c3@mail.gmail.com> References: <4AC8A396.8030306@cryptelium.net> <4ACB1B5A.2090608@simplistix.co.uk> <319e029f0910060402n306a992as4b9d854f7f29d546@mail.gmail.com> <319e029f0910060402m28ab35acnc9a52aefcdc6f04a@mail.gmail.com> <4ACB24AE.4000300@cryptelium.net> <4ACB2527.7010802@cryptelium.net> <319e029f0910060522l29499f46y2805ed0082d975c3@mail.gmail.com> Message-ID: <4ACB7D70.9060404@cryptelium.net> Hi Lennart, If i read 'virtualenv-distribute 1.3.4.2 on pypi' I can do some googling or even do some Pypi searching for 'virtualenv-distribute'. Thus, the first link found may be [1]. On this link, the second sentence is: """ The fork was started by Philip Jenvey at http://bitbucket.org/pjenvey/virtualenv-distribute/ and this version by Florian Schulze lives at http://bitbucket.org/fschulze/virtualenv-distribute/ """ [1] - http://pypi.python.org/pypi/virtualenv-distribute/ Lennart Regebro a ?crit : > 2009/10/6 kiorky : >> Anyway, it's released now on pypi as virtualenv-distribute-1.3.4.2. >> >> The code is also merged in florian branch and it has been decided that's the >> main repository. > > What is the florian branch, and in general, could you provide some > more info. Your emails are very cryptic. :) > -- -- Cordialement, KiOrKY GPG Key FingerPrint: 0x1A1194B7681112AF -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 261 bytes Desc: OpenPGP digital signature URL: From hanno at hannosch.eu Tue Oct 6 19:36:42 2009 From: hanno at hannosch.eu (Hanno Schlichting) Date: Tue, 6 Oct 2009 19:36:42 +0200 Subject: [Distutils] Package install failures in 2.6.3 - setuptools vs Distribute In-Reply-To: <2380.1254848454@parc.com> References: <28320134263785163387302965410901866648-Webmail@me.com> <72523684218386049404529325276719797765-Webmail@me.com> <4ACA049B.5010201@noir.com> <10081E45-60AF-4621-BD12-981E8C5703C0@mac.com> <2380.1254848454@parc.com> Message-ID: <5cae42b20910061036y3d69c415rd4549d739daf09d4@mail.gmail.com> On Tue, Oct 6, 2009 at 7:00 PM, Bill Janssen wrote: > For me, it's more a matter of "OS X 10.6 already comes with setuptools; > how can I mitigate the impact of this buggy unmaintained package on the > systems I'm building to deploy on OS X?". ?Adding distribute to the mix, > however good it is, doesn't help; I'm going to use a pure distutils > solution. ?And I can't really install a bugfix release for the Python > frameworks in /System; only Apple can do that. I fear the canonical answer to this problem is: Don't use the system Python on Mac OS. It's not particular satisfactory, but as you noticed nobody besides Apple can fix problems in this Python install or update it to newer versions. Apple started to include quite a number of projects, like dateutil, Twisted, NumPy, zope.interface and setuptools to name a few. They will probably update those the next time in Mac OS 10.7 in one or two years. Even today the packages they ship are already outdated and miss bug fixes. So if you want to deploy to Mac OS, I fear the answer is to encourage people to install a good pristine Python version instead. Be that the official GUI installer, Macports, Fink or building your own Python from source. The installers for the Plone application include a full Python build for that reason. Hanno From pje at telecommunity.com Tue Oct 6 19:52:41 2009 From: pje at telecommunity.com (P.J. Eby) Date: Tue, 06 Oct 2009 13:52:41 -0400 Subject: [Distutils] Package install failures in 2.6.3 - setuptools vs Distribute In-Reply-To: <4ACAE1C0.4040403@nextday.fi> References: <319e029f0910030515u71325b88h89037b3ea85f65b5@mail.gmail.com> <94bdd2610910030649r431a5638y7c8b5332934f5ae@mail.gmail.com> <319e029f0910031308u4182dff2w89760afb4122cf2c@mail.gmail.com> <4AC9BEF5.7080204@taupro.com> <319e029f0910050253x1ad1cc7eg60e7f11b3f9f72b8@mail.gmail.com> <20091005180304.71CF63A4045@sparrow.telecommunity.com> <4ACAE1C0.4040403@nextday.fi> Message-ID: <20091006175250.6AAA33A414B@sparrow.telecommunity.com> At 09:20 AM 10/6/2009 +0300, Alex Gr?nholm wrote: >P.J. Eby kirjoitti: >>At 11:53 AM 10/5/2009 +0200, Lennart Regebro wrote: >>>2009/10/5 Jeff Rush : >>> > Very unfortunate, as in, it should NOT have happened. And *especially* >>> > without any announcement on python.org or mention on the >>> > python-committers list of something this major. >>> >>>Well "this major"... It's a bug fix that breaks a setuptools monkey-patch. >> >>Subclassing distutils commands != monkeypatching. If, say, numpy's >>distutils extensions subclass build_ext and override that method, >>they could have had the same problem. Same for anybody else's >>distutils extensions. >Setuptools subclasses distutils commands and then replaces the >original classes with its own. >Example from setuptools/__init__.py: > >import distutils.core >distutils.core.Command = Command > >Isn't this exactly what monkey patching means? Yes, but that's got nothing to do with the bug that's been being discussed. The same change bit pywin32, and it doesn't use setuptools at all. From regebro at gmail.com Tue Oct 6 20:17:27 2009 From: regebro at gmail.com (Lennart Regebro) Date: Tue, 6 Oct 2009 20:17:27 +0200 Subject: [Distutils] Package install failures in 2.6.3 - setuptools vs Distribute In-Reply-To: <20091006175250.6AAA33A414B@sparrow.telecommunity.com> References: <319e029f0910030515u71325b88h89037b3ea85f65b5@mail.gmail.com> <94bdd2610910030649r431a5638y7c8b5332934f5ae@mail.gmail.com> <319e029f0910031308u4182dff2w89760afb4122cf2c@mail.gmail.com> <4AC9BEF5.7080204@taupro.com> <319e029f0910050253x1ad1cc7eg60e7f11b3f9f72b8@mail.gmail.com> <20091005180304.71CF63A4045@sparrow.telecommunity.com> <4ACAE1C0.4040403@nextday.fi> <20091006175250.6AAA33A414B@sparrow.telecommunity.com> Message-ID: <319e029f0910061117q45456511k56da6979da31e1da@mail.gmail.com> 2009/10/6 P.J. Eby : > Yes, but that's got nothing to do with the bug that's been being discussed. > ?The same change bit pywin32, and it doesn't use setuptools at all. True. The problem was a badly documented interface, which meant people used it in one way, but a bug fix assumed another, and *kapow*! The only fixes for that is better documentation and a buildbot, as noticed. -- Lennart Regebro: Python, Zope, Plone, Grok http://regebro.wordpress.com/ +33 661 58 14 64 From sridharr at activestate.com Wed Oct 7 01:08:18 2009 From: sridharr at activestate.com (Sridhar Ratnakumar) Date: Tue, 06 Oct 2009 16:08:18 -0700 Subject: [Distutils] Package install failures in 2.6.3 - setuptools vs Distribute In-Reply-To: <4ACB7D70.9060404@cryptelium.net> References: <4AC8A396.8030306@cryptelium.net> <4ACB1B5A.2090608@simplistix.co.uk> <319e029f0910060402n306a992as4b9d854f7f29d546@mail.gmail.com> <319e029f0910060402m28ab35acnc9a52aefcdc6f04a@mail.gmail.com> <4ACB24AE.4000300@cryptelium.net> <4ACB2527.7010802@cryptelium.net> <319e029f0910060522l29499f46y2805ed0082d975c3@mail.gmail.com> <4ACB7D70.9060404@cryptelium.net> Message-ID: On Tue, 06 Oct 2009 10:25:04 -0700, kiorky wrote: > > """ > The fork was started by Philip Jenvey at > http://bitbucket.org/pjenvey/virtualenv-distribute/ and this version by > Florian > Schulze lives at http://bitbucket.org/fschulze/virtualenv-distribute/ > """ > [1] - http://pypi.python.org/pypi/virtualenv-distribute/ Just a suggestion: could someone make that /fschulze/virtualenv-distribute site (the current 'home page' of the project) link to the launchpad bug tracker? Up until now, I was not able to track down an issue tracker for this fork .. until you posted the PyPI link. This way one - who stumbled upon the bitbucket site - does not have to pull the source tree and look in docs/index.rst in order to get the URL to the bug tracker. (I had a bug to report a couple of days). -srid From sridharr at activestate.com Wed Oct 7 02:16:07 2009 From: sridharr at activestate.com (Sridhar Ratnakumar) Date: Tue, 06 Oct 2009 17:16:07 -0700 Subject: [Distutils] ANN: ActivePython 2.6.3.7 (and PyPM) is now available Message-ID: I'm happy to announce that ActivePython 2.6.3.7 is now available for download from: http://www.activestate.com/activepython/ This is a patch-level release that updates ActivePython to core Python 2.6.3 along with the fixes for a couple of critical regressions that instigated the work on 2.6.4. See the release notes for full details: http://docs.activestate.com/activepython/2.6/relnotes.html Introducing PyPM ---------------- This release includes a new packaging tool by activestate called Python Package Manager (PyPM). PyPM - currently in beta - is the package management utility for ActivePython. It simplifies the task of locating, installing, upgrading and removing Python modules. For full details, see: http://docs.activestate.com/activepython/2.6/pypm.html Here's a sample command line output:: $ pypm install lxml Get: [pypm.activestate.com] :repository-index: Ready to perform these actions: The following packages will be installed: lxml-2.2.2 Get: [pypm.activestate.com] lxml 2.2.2-1 Installing lxml-2.2.2 $ python >>> import lxml.etree >>>^D $ pypm remove lxml Ready to perform these actions: The following packages will be removed: lxml-2.2.2 Removing lxml-2.2.2 $ pypm install pylons Ready to perform these actions: The following packages will be installed: pastescript-1.7.3 formencode-1.2.2 weberror-0.10.1 simplejson-2.0.9 routes-1.11 nose-0.11.1 mako-0.2.5 past edeploy-1.3.3 pylons-0.9.7 tempita-0.4 webtest-1.2 beaker-1.4.2 webhelpers-0.6.4 paste-1.7.2 pygments-1.1.1 decorator-3.1.2 webob-0.9.6.1 Get: [pypm.activestate.com] formencode 1.2.2-1 Get: [pypm.activestate.com] nose 0.11.1-1 [...] Get: [pypm.activestate.com] decorator 3.1.2-1 Get: [pypm.activestate.com] webob 0.9.6.1-1 Installing formencode-1.2.2 Installing weberror-0.10.1 [...] Installing pygments-1.1.1 Fixing script /home/sridharr/.local/bin/pygmentize Installing decorator-3.1.2 What is ActivePython? --------------------- ActivePython is ActiveState's binary distribution of Python. Builds for Windows, Mac OS X, Linux, HP-UX and AIX are made freely available. ActivePython includes the Python core and the many core extensions: zlib and bzip2 for data compression, the Berkeley DB (bsddb) and SQLite (sqlite3) database libraries, OpenSSL bindings for HTTPS support, the Tix GUI widgets for Tkinter, ElementTree for XML processing, ctypes (on supported platforms) for low-level library access, and others. The Windows distribution ships with PyWin32 -- a suite of Windows tools developed by Mark Hammond, including bindings to the Win32 API and Windows COM. See this page for full details: http://docs.activestate.com/activepython/2.6/whatsincluded.html As well, ActivePython ships with a wealth of documentation for both new and experienced Python programmers. In addition to the core Python docs, ActivePython includes the "What's New in Python" series, "Dive into Python", the Python FAQs & HOWTOs, and the Python Enhancement Proposals (PEPs). An online version of the docs can be found here: http://docs.activestate.com/activepython/2.6/ We would welcome any and all feedback to: ActivePython-feedback at activestate.com Please file bugs against ActivePython at: http://bugs.activestate.com/query.cgi?set_product=ActivePython On what platforms does ActivePython run? ---------------------------------------- ActivePython includes installers for the following platforms: - Windows/x86 - Windows/x64 (aka "AMD64") - Mac OS X - Linux/x86 - Linux/x86_64 (aka "AMD64") - Solaris/SPARC - Solaris/x86 - HP-UX/PA-RISC - AIX/PowerPC - AIX/PowerPC 64-bit Extra Bits ---------- ActivePython releases also include the following: - ActivePython26.chm: An MS compiled help collection of the full ActivePython documentation set. Linux users of applications such as xCHM might find this useful. This package is installed by default on Windows. Extra bits are available from: http://downloads.activestate.com/ActivePython/etc/ Thanks, and enjoy! The Python Team -- Sridhar Ratnakumar sridharr at activestate.com From kevin at bud.ca Wed Oct 7 04:16:08 2009 From: kevin at bud.ca (Kevin Teague) Date: Tue, 6 Oct 2009 19:16:08 -0700 (PDT) Subject: [Distutils] Package install failures in 2.6.3 - setuptools vs Distribute In-Reply-To: <5cae42b20910061036y3d69c415rd4549d739daf09d4@mail.gmail.com> References: <28320134263785163387302965410901866648-Webmail@me.com> <72523684218386049404529325276719797765-Webmail@me.com> <4ACA049B.5010201@noir.com> <10081E45-60AF-4621-BD12-981E8C5703C0@mac.com> <2380.1254848454@parc.com> <5cae42b20910061036y3d69c415rd4549d739daf09d4@mail.gmail.com> Message-ID: On Oct 6, 2009, at 10:36 AM, Hanno Schlichting wrote: On Tue, Oct 6, 2009 at 7:00 PM, Bill Janssen wrote: >> For me, it's more a matter of "OS X 10.6 already comes with setuptools; >> how can I mitigate the impact of this buggy unmaintained package on the >> systems I'm building to deploy on OS X?". Adding distribute to the mix, >> however good it is, doesn't help; I'm going to use a pure distutils >> solution. And I can't really install a bugfix release for the Python >> frameworks in /System; only Apple can do that. > > I fear the canonical answer to this problem is: Don't use the system > Python on Mac OS. > > It's not particular satisfactory, but as you noticed nobody besides > Apple can fix problems in this Python install or update it to newer > versions. Apple started to include quite a number of projects, like > dateutil, Twisted, NumPy, zope.interface and setuptools to name a few. > They will probably update those the next time in Mac OS 10.7 in one or > two years. Even today the packages they ship are already outdated and > miss bug fixes. > > So if you want to deploy to Mac OS, I fear the answer is to encourage > people to install a good pristine Python version instead. Be that the > official GUI installer, Macports, Fink or building your own Python > from source. The installers for the Plone application include a full > Python build for that reason. I believe (but not 100% sure) that all of those projects (setuptools, Twisted, zope.interface) got pulled into site-packages when Apple began shipping iCal stuff. The existance of /usr/bin/easy_install and setuptools is just a side-effect of Apple using an assortment of Python projects to develop iCal. Perhaps we should have got one of those projects that iCal depends upon to put some lightweight need-to- import-it-but-don't-actually-use-it dependencies on VirtualEnv and Buildout. Then those projects would have gotten pulled into Apple's Python and we'd have the tools for easily isolating ourselves from Apple's project's dependencies! Alternatively, someone could work with Apple to get them to deploy their Python apps in a self-contained manner, leaving an Apple Python with a clean site-packages directory, all ready for people to go into and start polluting in a non os-conflicting manner :P (well, technically this directory is clean, it's just overshadowed on sys.path by /System/Library/Frameworks/Python.framework/Versions/2.5/ Extras/lib/python/). They do ship their Ruby apps self-contained. Heck, setuptools provides the same method of working with self- contained library dependencies as Ruby Gems does, and it's not like Apple has demonstrated any of the idiosyncratic "setuptools-aversions" that exist elsewhere in the Python community. They do after all ship with setuptools and ironically provide a /usr/bin/easy_install even though setuptools also provides entry points as a way of cleanly separating script installation from project installation. C'est la vie! From tseaver at palladion.com Wed Oct 7 05:36:09 2009 From: tseaver at palladion.com (Tres Seaver) Date: Tue, 06 Oct 2009 23:36:09 -0400 Subject: [Distutils] Package install failures in 2.6.3 - setuptools vs Distribute In-Reply-To: <319e029f0910050253x1ad1cc7eg60e7f11b3f9f72b8@mail.gmail.com> References: <319e029f0910030515u71325b88h89037b3ea85f65b5@mail.gmail.com> <94bdd2610910030649r431a5638y7c8b5332934f5ae@mail.gmail.com> <319e029f0910031308u4182dff2w89760afb4122cf2c@mail.gmail.com> <4AC9BEF5.7080204@taupro.com> <319e029f0910050253x1ad1cc7eg60e7f11b3f9f72b8@mail.gmail.com> Message-ID: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Lennart Regebro wrote: > 2009/10/5 Jeff Rush : >> Very unfortunate, as in, it should NOT have happened. And *especially* >> without any announcement on python.org or mention on the >> python-committers list of something this major. > > Well "this major"... It's a bug fix that breaks a setuptools monkey-patch. > But yes, it was discovered before release, and maybe it should have > been discussed, I'm not on python-dev anymore. The fact that the bugfix (to distutils) was committed by folks who are *alos* pusing a fork to setuptools is what raises the eyebrows here. Note that I am *not* suggesting that anyone acted in bad faith; but the "appearance of impropriety" can be present where no actual impropriety exists. The only cure is for the "conflicted" parties to take extreme measures to avoid appearing to act improperly. >> and even hints of >> under-the-table intentionality in forcing the community to abandon use >> of setuptools. > > There are no such hints anywhere. There are those postig on this thread that the distutils breakage is a good thing for Distribute, because it *forces* folks to switch: I call that a solid hint, myself. >> Distribute should win because of superior technology not >> forced migration. > > That makes no sense. You move to distribute because you have to, > because setuptools is buggy and not updated. Until people encounter > bugs in setuptools, or need Python 3 support, they are not likely to > move to Distribute. There is no other reason than forced migration. But you shouldn't have to move to Distribute becuase the Distrubute developers broke setuptools by changing the Python stdlib (intentionally or not). > I fail to see how this is a big disaster in any way. Yes, it's not > perfect, and yeah, maybe there should have been big warning signs > somewhere. But we can NOT leave bugs in Python just because setuptools > isn't getting updated. Setuptools has already been a break on Python 3 > development, are we gonna lets it be a break on Python 2 bugfixes too? Bugfixes which break backward compatibility in "minor" relaseses are "major" fouls, period. As PJE points out, the particular bugfix in question *also* broke other packages (pywin32), which means that it can't be just that setuptools is "unmaintained" (I'm not supposed to have to re-relase my packages to cope with a patch-level release in Python, period). Tres. - -- =================================================================== Tres Seaver +1 540-429-0999 tseaver at palladion.com Palladion Software "Excellence by Design" http://palladion.com -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iEYEARECAAYFAkrMDJcACgkQ+gerLs4ltQ5KxQCg0yhegMhrML5bv7xniRfngG8j FFMAnjmPsBHXZW3sc4/M/qx3N+x1ura2 =2gq3 -----END PGP SIGNATURE----- From david at ar.media.kyoto-u.ac.jp Wed Oct 7 06:18:15 2009 From: david at ar.media.kyoto-u.ac.jp (David Cournapeau) Date: Wed, 07 Oct 2009 13:18:15 +0900 Subject: [Distutils] Package install failures in 2.6.3 - setuptools vs Distribute In-Reply-To: References: <319e029f0910030515u71325b88h89037b3ea85f65b5@mail.gmail.com> <94bdd2610910030649r431a5638y7c8b5332934f5ae@mail.gmail.com> <319e029f0910031308u4182dff2w89760afb4122cf2c@mail.gmail.com> <4AC9BEF5.7080204@taupro.com> <319e029f0910050253x1ad1cc7eg60e7f11b3f9f72b8@mail.gmail.com> Message-ID: <4ACC1687.8010708@ar.media.kyoto-u.ac.jp> Tres Seaver wrote: > > Bugfixes which break backward compatibility in "minor" relaseses are > "major" fouls, period. Sure, but what does backward compatibility even mean for distutils ? Not much, as any non trivial extension needs to use undocumented implementation details. > As PJE points out, the particular bugfix in > question *also* broke other packages (pywin32), which means that it > can't be just that setuptools is "unmaintained" No, it is a consequence of distutils design and implementation. I think those breakages are unavoidable if you touch distutils besides trivial bug fixes (and the list of trivial things you can do to distutils without breaking anything is tiny). I already had to adapt numpy.distutils numerous times when distutils changed, so I don't think it is fair to point this particular issue as a major foul. It should be accepted that people relying on distutils internals (that is almost anybody using distutils extensions, especially for compiled code) will have to constantly change their code IMHO. cheers, David From regebro at gmail.com Wed Oct 7 08:41:10 2009 From: regebro at gmail.com (Lennart Regebro) Date: Wed, 7 Oct 2009 08:41:10 +0200 Subject: [Distutils] Package install failures in 2.6.3 - setuptools vs Distribute In-Reply-To: References: <319e029f0910030515u71325b88h89037b3ea85f65b5@mail.gmail.com> <94bdd2610910030649r431a5638y7c8b5332934f5ae@mail.gmail.com> <319e029f0910031308u4182dff2w89760afb4122cf2c@mail.gmail.com> <4AC9BEF5.7080204@taupro.com> <319e029f0910050253x1ad1cc7eg60e7f11b3f9f72b8@mail.gmail.com> Message-ID: <319e029f0910062341k557734e9y155237964ff1ebf2@mail.gmail.com> 2009/10/7 Tres Seaver : > The fact that the bugfix (to distutils) was committed by folks who are > *alos* pusing a fork to setuptools is what raises the eyebrows here. Eh... why? Tarek has become the lead for Python packaging programs and is trying hard to fix the sad state of Python packaging. It's not particularily surprising that he (or otehrs involved in Python packaging) would be involved in more than one package. > Note that I am *not* suggesting that anyone acted in bad faith; ?but the > "appearance of impropriety" can be present where no actual impropriety > exists. ?The only cure is for the "conflicted" parties to take extreme > measures to avoid appearing to act improperly. ... > There are those postig on this thread that the distutils breakage is a > good thing for Distribute, because it *forces* folks to switch: I call > that a solid hint, myself. I'm sorry, but I read this as you implying that somebody could think Tarek, me and other Distribute people is involved in some sort of Python packaging conspiracy. I can't take that seriously, so I really hope that you mean something else, because that idea is just laughable. There is no appearance of impropriety. There is no "conflicted party". There is no conflict. You can't read anything as being good for Distribute, because there is no "good for Distribute". Distribute does not benefit from people using it. People benefit from using Distribute. I suspect that you see some conflict or competition between Distribute and setuptools when there is none. -- Lennart Regebro: Python, Zope, Plone, Grok http://regebro.wordpress.com/ +33 661 58 14 64 From ziade.tarek at gmail.com Wed Oct 7 10:37:55 2009 From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=) Date: Wed, 7 Oct 2009 10:37:55 +0200 Subject: [Distutils] Distutils 0.6.4 this week Message-ID: <94bdd2610910070137x5a2954c0tce78a7722e8e1a07@mail.gmail.com> Hello, We are going to ship Distribute 0.6.4 by the end of this week with a few more bugs fixed, whereas they were listed in Setuptools tracker (http://bitbucket.org/tarek/distribute/wiki/bug_listing) whereas they were added directly in our tracker. If you had any issue you want to see fixed in this release, please speak up. If you want to help, you can install and play with the nightly build of 0.6.4 by running: http://nightly.ziade.org/distribute_setup_dev.py Last, a new wiki page has been added where we are giving more details on the roadmap for 0.6.x and 0.7.x : http://bitbucket.org/tarek/distribute/wiki/roadmap Regards Tarek -- Tarek Ziad? | http://ziade.org |????????????! |???????????? From reinout at vanrees.org Wed Oct 7 12:50:49 2009 From: reinout at vanrees.org (Reinout van Rees) Date: Wed, 7 Oct 2009 10:50:49 +0000 (UTC) Subject: [Distutils] Distutils 0.6.4 this week References: <94bdd2610910070137x5a2954c0tce78a7722e8e1a07@mail.gmail.com> Message-ID: On 2009-10-07, Tarek Ziad? wrote: > > We are going to ship Distribute 0.6.4 by the end of this week with a > few more bugs fixed, whereas they were listed in Setuptools tracker > (http://bitbucket.org/tarek/distribute/wiki/bug_listing) whereas they > were added directly in our tracker. > > If you had any issue you want to see fixed in this release, please speak up. http://bitbucket.org/tarek/distribute/issue/23/warning-about-module-was-already-imported The "Module pkg_resources was already imported from"-like warnings when running something inside buildout are irritating. They also break a script I have that calls "python setup.py --name" on several packages to grab their name. Normally you get the name, now you get several lines of warnings, followed by the name. Otherwise I have to go back to a regex :-) Reinout -- Reinout van Rees - reinout at vanrees.org - http://reinout.vanrees.org Software developer at http://www.thehealthagency.com "Military engineers build missiles. Civil engineers build targets" From reinout at vanrees.org Wed Oct 7 13:23:11 2009 From: reinout at vanrees.org (Reinout van Rees) Date: Wed, 7 Oct 2009 11:23:11 +0000 (UTC) Subject: [Distutils] distribute and buildout: best practices? Message-ID: What's the current best way to use both distribute and buildout? Apart from the bootstrap + apparently needed change in buildout, I've got the following questions: - Do my libraries have to list a dependency on distribute or on setuptools? Everything zopish has a 'setuptools >= 0.6c9' in it. I tried putting distribute in there, but I seemed to get less problems with leaving it at setuptools. - Should just globally installing distribute be enough? Everything ought to keep working. It does lead to a bunch of runtime warnings in buildout, however. (But Tarek has targeted that problem for the upcoming 0.6.4 :-) - Do I need to change buildout's bootstrap already? Doing that tries to change my global setuptools, for which you need to sudo. And it also forces everyone else using that buildout to switch (which in itself is OK). - From my experiments (not exhaustive! no real debugging!) it seems that the only safe way is to first install distribute globally before touching a buildout that brings in distribute. Is this to be expected? Reinout -- Reinout van Rees - reinout at vanrees.org - http://reinout.vanrees.org Software developer at http://www.thehealthagency.com "Military engineers build missiles. Civil engineers build targets" From nad at acm.org Sun Oct 4 20:13:06 2009 From: nad at acm.org (Ned Deily) Date: Sun, 4 Oct 2009 11:13:06 -0700 Subject: [Distutils] Package install failures in 2.6.3 - setuptools vs Distribute In-Reply-To: <20091004170717.9C52C3A4748@sparrow.telecommunity.com> References: <319e029f0910030515u71325b88h89037b3ea85f65b5@mail.gmail.com> <94bdd2610910030649r431a5638y7c8b5332934f5ae@mail.gmail.com> <20091004170717.9C52C3A4748@sparrow.telecommunity.com> Message-ID: <79C29451-D6FD-4B73-BE32-BF79A1885B79@acm.org> On Oct 4, 2009, at 10:07 , P.J. Eby wrote: > At 03:49 PM 10/3/2009 +0200, Tarek Ziad? wrote: >> Notice that this has been fixed in Ubuntu already with a patched >> version of setuptools > Is the patch or an equivalent already in the setuptools tracker? > And if not, can someone please post it there? Thanks. I've opened an issue: http://bugs.python.org/setuptools/issue85 -- Ned Deily nad at acm.org -- [] From florian.schulze at gmx.net Wed Oct 7 09:18:51 2009 From: florian.schulze at gmx.net (Florian Schulze) Date: Wed, 7 Oct 2009 09:18:51 +0200 Subject: [Distutils] Package install failures in 2.6.3 - setuptools vs Distribute In-Reply-To: References: <4AC8A396.8030306@cryptelium.net> <4ACB1B5A.2090608@simplistix.co.uk> <319e029f0910060402n306a992as4b9d854f7f29d546@mail.gmail.com> <319e029f0910060402m28ab35acnc9a52aefcdc6f04a@mail.gmail.com> <4ACB24AE.4000300@cryptelium.net> <4ACB2527.7010802@cryptelium.net> <319e029f0910060522l29499f46y2805ed0082d975c3@mail.gmail.com> <4ACB7D70.9060404@cryptelium.net> Message-ID: <100714AC-82C3-4C7C-AA51-9B3A22AB59B7@gmx.net> On 07.10.2009, at 01:08, Sridhar Ratnakumar wrote: > On Tue, 06 Oct 2009 10:25:04 -0700, kiorky > wrote: > >> >> """ >> The fork was started by Philip Jenvey at >> http://bitbucket.org/pjenvey/virtualenv-distribute/ and this >> version by Florian >> Schulze lives at http://bitbucket.org/fschulze/virtualenv-distribute/ >> """ >> [1] - http://pypi.python.org/pypi/virtualenv-distribute/ > > Just a suggestion: could someone make that /fschulze/virtualenv- > distribute site (the current 'home page' of the project) link to the > launchpad bug tracker? Up until now, I was not able to track down an > issue tracker for this fork .. until you posted the PyPI link. > > This way one - who stumbled upon the bitbucket site - does not have > to pull the source tree and look in docs/index.rst in order to get > the URL to the bug tracker. (I had a bug to report a couple of days). The launchpad bugtracker is for virtualenv, not this fork. Sadly I just realized bitbucket.org doesn't have an integrated issue tracker like github :( Regards, Florian Schulze From florian.schulze at gmx.net Wed Oct 7 10:03:52 2009 From: florian.schulze at gmx.net (Florian Schulze) Date: Wed, 7 Oct 2009 10:03:52 +0200 Subject: [Distutils] Package install failures in 2.6.3 - setuptools vs Distribute In-Reply-To: References: <4AC8A396.8030306@cryptelium.net> <4ACB1B5A.2090608@simplistix.co.uk> <319e029f0910060402n306a992as4b9d854f7f29d546@mail.gmail.com> <319e029f0910060402m28ab35acnc9a52aefcdc6f04a@mail.gmail.com> <4ACB24AE.4000300@cryptelium.net> <4ACB2527.7010802@cryptelium.net> <319e029f0910060522l29499f46y2805ed0082d975c3@mail.gmail.com> <4ACB7D70.9060404@cryptelium.net> Message-ID: <7DD35CBF-4122-457B-A0A6-F7713553AB71@gmx.net> On 07.10.2009, at 01:08, Sridhar Ratnakumar wrote: > On Tue, 06 Oct 2009 10:25:04 -0700, kiorky > wrote: > >> >> """ >> The fork was started by Philip Jenvey at >> http://bitbucket.org/pjenvey/virtualenv-distribute/ and this >> version by Florian >> Schulze lives at http://bitbucket.org/fschulze/virtualenv-distribute/ >> """ >> [1] - http://pypi.python.org/pypi/virtualenv-distribute/ > > Just a suggestion: could someone make that /fschulze/virtualenv- > distribute site (the current 'home page' of the project) link to the > launchpad bug tracker? Up until now, I was not able to track down an > issue tracker for this fork .. until you posted the PyPI link. > > This way one - who stumbled upon the bitbucket site - does not have > to pull the source tree and look in docs/index.rst in order to get > the URL to the bug tracker. (I had a bug to report a couple of days). Ah, found the setting at bitbucket to enable the issue tracker, thanks kiorky to make me look deeper. http://bitbucket.org/fschulze/virtualenv-distribute/issues/?status=new&status=open Regards, Florian Schulze From ziade.tarek at gmail.com Wed Oct 7 13:53:50 2009 From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=) Date: Wed, 7 Oct 2009 13:53:50 +0200 Subject: [Distutils] distribute and buildout: best practices? In-Reply-To: References: Message-ID: <94bdd2610910070453t63e7e4c4h1c9bf6ca567fa48f@mail.gmail.com> On Wed, Oct 7, 2009 at 1:23 PM, Reinout van Rees wrote: > What's the current best way to use both distribute and buildout? ?Apart from > the bootstrap + apparently needed change in buildout, I've got the following > questions: > > - Do my libraries have to list a dependency on distribute or on setuptools? > ?Everything zopish has a 'setuptools >= 0.6c9' in it. ?I tried putting > ?distribute in there, but I seemed to get less problems with leaving it at > ?setuptools. > No. Once we've resolved the problem you've mentioned in the hardcoded setuptools dependency, you will have an empty Setuptools 0.6c9 egg in eggs/ so libs that ask for this dependency will think it's met. > - Should just globally installing distribute be enough? ?Everything ought to > ?keep working. ?It does lead to a bunch of runtime warnings in buildout, > ?however. ?(But Tarek has targeted that problem for the upcoming 0.6.4 :-) We still require a specific zc.buildout bootstrap.py, otherwise you will have a local setuptools egg that will override any global installation. > > - Do I need to change buildout's bootstrap already? Doing that tries to change > ?my global setuptools, for which you need to sudo. ?And it also forces > ?everyone else using that buildout to switch (which in itself is OK). It shouldn't touch anything outside your buildout, please try with the latest one at buildout-distribute at bitbucket. > > - From my experiments (not exhaustive! no real debugging!) it seems that the > ?only safe way is to first install distribute globally before touching a > ?buildout that brings in distribute. ?Is this to be expected? The Distribute installation will try to patch any setuptools found in the path so it doesn't interfer. In a buildout environment, if it changes anything globally, it's a bug of our patching code and not intended, unless your buildout environment, somehow, uses a global setuptools. But I don't think this can happen. Tarek -- Tarek Ziad? | http://ziade.org |????????????! |???????????? From reinout at vanrees.org Wed Oct 7 14:32:44 2009 From: reinout at vanrees.org (Reinout van Rees) Date: Wed, 7 Oct 2009 12:32:44 +0000 (UTC) Subject: [Distutils] distribute and buildout: best practices? References: <94bdd2610910070453t63e7e4c4h1c9bf6ca567fa48f@mail.gmail.com> Message-ID: On 2009-10-07, Tarek Ziad? wrote: > On Wed, Oct 7, 2009 at 1:23 PM, Reinout van Rees wrote: >> >> - Do my libraries have to list a dependency on distribute or on setuptools? >> ?Everything zopish has a 'setuptools >= 0.6c9' in it. ?I tried putting >> ?distribute in there, but I seemed to get less problems with leaving it at >> ?setuptools. >> > > No. Once we've resolved the problem you've mentioned in the hardcoded setuptools > dependency, you will have an empty Setuptools 0.6c9 egg in eggs/ > so libs that ask for this dependency will think it's met. But to actually use distribute you need to depend on it somewhere, obviously. Otherwise I only get a setuptools egg in my path. > It shouldn't touch anything outside your buildout, please try with the latest > one at buildout-distribute at bitbucket. The latest bootstrap behaves itself wonderfully. So: once we have new zc.buildout + bootstrap it ought to be safe to change all buildouts without inconveniencing anyone. I'm still not 100% sure whether it is safe to put "distribute" in the install_requires list of a package right now, however. Reinout -- Reinout van Rees - reinout at vanrees.org - http://reinout.vanrees.org Software developer at http://www.thehealthagency.com "Military engineers build missiles. Civil engineers build targets" From ziade.tarek at gmail.com Wed Oct 7 15:32:56 2009 From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=) Date: Wed, 7 Oct 2009 15:32:56 +0200 Subject: [Distutils] distribute and buildout: best practices? In-Reply-To: References: <94bdd2610910070453t63e7e4c4h1c9bf6ca567fa48f@mail.gmail.com> Message-ID: <94bdd2610910070632j3a03b4den7567e3b387f33c20@mail.gmail.com> On Wed, Oct 7, 2009 at 2:32 PM, Reinout van Rees wrote: > > I'm still not 100% sure whether it is safe to put "distribute" in the > install_requires list of a package right now, however. It depends on what you mean by safe. (beside any bootstraping bug we might find in the near future, even if this code seems stabilized now) Adding distribute in install_requires will have the effect of calling the Distribute setup.py and renaming an existing setuptools installation detected in the environment so it doesn't get on the way, and to add a setuptools*.egg-info in the environment so the Setuptools dependency is still met (to avoid Distribute to be overriden in turn) By "environment" I mean here the sys.path than gets scanned by pkg_resources when looking for Distributions. So when your package is installed, its impact will depend on the environment its setup.py was called in. The impact is : now the environment is using Distribute. This has to be done manually in Distribute's setup.py because the micro-dependency managment system provided by easy_install and pkg_resources doesn't know how to deal with mutual exclusive dependencies or interchangeable dependency. But this is not a new problem: if you add dependencies in your distribution, you are impacting the whole execution environment, for every other distribution. Now you can say it outloud in your distribution documentation, that your distribution requires Distribute because it has more bugfixes. Tarek -- Tarek Ziad? | http://ziade.org |????????????! |???????????? From reinout at vanrees.org Wed Oct 7 15:55:10 2009 From: reinout at vanrees.org (Reinout van Rees) Date: Wed, 7 Oct 2009 13:55:10 +0000 (UTC) Subject: [Distutils] distribute and buildout: best practices? References: <94bdd2610910070453t63e7e4c4h1c9bf6ca567fa48f@mail.gmail.com> <94bdd2610910070632j3a03b4den7567e3b387f33c20@mail.gmail.com> Message-ID: On 2009-10-07, Tarek Ziad? wrote: > On Wed, Oct 7, 2009 at 2:32 PM, Reinout van Rees wrote: >> >> I'm still not 100% sure whether it is safe to put "distribute" in the >> install_requires list of a package right now, however. > > It depends on what you mean by safe. (beside any bootstraping bug we might > find in the near future, even if this code seems stabilized now) > > Adding distribute in install_requires will have the effect of calling the > Distribute setup.py and renaming an existing setuptools installation > detected in the environment so it doesn't get on the way, and to add a > setuptools*.egg-info in the environment so the Setuptools dependency is > still met (to avoid Distribute to be overriden in turn) > > By "environment" I mean here the sys.path than gets scanned by pkg_resources > when looking for Distributions. > > So when your package is installed, its impact will depend on the environment > its setup.py was called in. > > The impact is : now the environment is using Distribute. Ok, I've experimented some more and came to some sort of conclusion in the bug report: http://tinyurl.com/yaa23yl The warning problem occurs if the environment (which you mention above) uses distribute. This environment could be a buildout egg cache. If a buildout, which uses this cache, only knows about setuptools (as there's no distribute depencency in that buildout), you'll get the extra "hey, I already loaded this" UserWarnings. But, and that's the mostly-good news, these warnings only occur if you have a global setuptools and haven't installed distribute globally yet. In that case, somehow there's a fallback to the global setuptools which omits those warnings. IF warnings THEN install it also globally, basically. And, from what I see, a global install works just fine with a buildout that doesn't use the new bootstrap. Provided the setuptools in the egg cache has been patch already by some other buildout :-) Reinout -- Reinout van Rees - reinout at vanrees.org - http://reinout.vanrees.org Software developer at http://www.thehealthagency.com "Military engineers build missiles. Civil engineers build targets" From ssteinerx at gmail.com Wed Oct 7 16:21:31 2009 From: ssteinerx at gmail.com (ssteinerX@gmail.com) Date: Wed, 7 Oct 2009 10:21:31 -0400 Subject: [Distutils] Package install failures in 2.6.3 - setuptools vs Distribute In-Reply-To: <100714AC-82C3-4C7C-AA51-9B3A22AB59B7@gmx.net> References: <4AC8A396.8030306@cryptelium.net> <4ACB1B5A.2090608@simplistix.co.uk> <319e029f0910060402n306a992as4b9d854f7f29d546@mail.gmail.com> <319e029f0910060402m28ab35acnc9a52aefcdc6f04a@mail.gmail.com> <4ACB24AE.4000300@cryptelium.net> <4ACB2527.7010802@cryptelium.net> <319e029f0910060522l29499f46y2805ed0082d975c3@mail.gmail.com> <4ACB7D70.9060404@cryptelium.net> <100714AC-82C3-4C7C-AA51-9B3A22AB59B7@gmx.net> Message-ID: On Oct 7, 2009, at 3:18 AM, Florian Schulze wrote: >> >> This way one - who stumbled upon the bitbucket site - does not have >> to pull the source tree and look in docs/index.rst in order to get >> the URL to the bug tracker. (I had a bug to report a couple of days). > > The launchpad bugtracker is for virtualenv, not this fork. Sadly I > just realized bitbucket.org doesn't have an integrated issue tracker > like github :( Uh...it's cleverly hidden under the "Issues" tab on the project page. S From florian.schulze at gmx.net Wed Oct 7 16:51:58 2009 From: florian.schulze at gmx.net (Florian Schulze) Date: Wed, 7 Oct 2009 16:51:58 +0200 Subject: [Distutils] Package install failures in 2.6.3 - setuptools vs Distribute In-Reply-To: References: <4AC8A396.8030306@cryptelium.net> <4ACB1B5A.2090608@simplistix.co.uk> <319e029f0910060402n306a992as4b9d854f7f29d546@mail.gmail.com> <319e029f0910060402m28ab35acnc9a52aefcdc6f04a@mail.gmail.com> <4ACB24AE.4000300@cryptelium.net> <4ACB2527.7010802@cryptelium.net> <319e029f0910060522l29499f46y2805ed0082d975c3@mail.gmail.com> <4ACB7D70.9060404@cryptelium.net> <100714AC-82C3-4C7C-AA51-9B3A22AB59B7@gmx.net> Message-ID: <6917D9DB-D43E-4F47-84CA-BAFB1A930476@gmx.net> On 07.10.2009, at 16:21, ssteinerX at gmail.com wrote: > > On Oct 7, 2009, at 3:18 AM, Florian Schulze wrote: >>> >>> This way one - who stumbled upon the bitbucket site - does not >>> have to pull the source tree and look in docs/index.rst in order >>> to get the URL to the bug tracker. (I had a bug to report a couple >>> of days). >> >> The launchpad bugtracker is for virtualenv, not this fork. Sadly I >> just realized bitbucket.org doesn't have an integrated issue >> tracker like github :( > > Uh...it's cleverly hidden under the "Issues" tab on the project page. I had to enable it in the cleverly hidden "Admin" tab first. Regards, Florian Schulze From ssteinerx at gmail.com Wed Oct 7 17:26:49 2009 From: ssteinerx at gmail.com (ssteinerX@gmail.com) Date: Wed, 7 Oct 2009 11:26:49 -0400 Subject: [Distutils] Package install failures in 2.6.3 - setuptools vs Distribute In-Reply-To: <6917D9DB-D43E-4F47-84CA-BAFB1A930476@gmx.net> References: <4AC8A396.8030306@cryptelium.net> <4ACB1B5A.2090608@simplistix.co.uk> <319e029f0910060402n306a992as4b9d854f7f29d546@mail.gmail.com> <319e029f0910060402m28ab35acnc9a52aefcdc6f04a@mail.gmail.com> <4ACB24AE.4000300@cryptelium.net> <4ACB2527.7010802@cryptelium.net> <319e029f0910060522l29499f46y2805ed0082d975c3@mail.gmail.com> <4ACB7D70.9060404@cryptelium.net> <100714AC-82C3-4C7C-AA51-9B3A22AB59B7@gmx.net> <6917D9DB-D43E-4F47-84CA-BAFB1A930476@gmx.net> Message-ID: <6BC49A9D-FCF1-4F30-81D3-7B9ABA46748D@gmail.com> On Oct 7, 2009, at 10:51 AM, Florian Schulze wrote: > > On 07.10.2009, at 16:21, ssteinerX at gmail.com wrote: > >> >> On Oct 7, 2009, at 3:18 AM, Florian Schulze wrote: >>>> >>>> This way one - who stumbled upon the bitbucket site - does not >>>> have to pull the source tree and look in docs/index.rst in order >>>> to get the URL to the bug tracker. (I had a bug to report a >>>> couple of days). >>> >>> The launchpad bugtracker is for virtualenv, not this fork. Sadly I >>> just realized bitbucket.org doesn't have an integrated issue >>> tracker like github :( >> >> Uh...it's cleverly hidden under the "Issues" tab on the project >> page. > > I had to enable it in the cleverly hidden "Admin" tab first. Told'ya it was cleverly hidden. ;-) I forgot about that; I think I had exactly the same "Where the heck's the issue tracker?" moment when I started my first repository. Why it's not enabled by default is one of the deep mysteries... Glad you found it! S From carl at dirtcircle.com Thu Oct 8 18:42:02 2009 From: carl at dirtcircle.com (Carl Meyer) Date: Thu, 08 Oct 2009 12:42:02 -0400 Subject: [Distutils] tracking requested vs dependency installs in PEP 376 metadata Message-ID: <4ACE165A.9000700@dirtcircle.com> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Hey all, I propose adding a bit to the PEP 376 metadata that indicates whether a package was installed by user request or as a dependency of another package. This would allow (un)installer tools to intelligently remove orphaned dependencies, if they so choose. There might be questions about the details of such an uninstaller feature, but I'm not intending to discuss that here. The metadata itself is simple enough to track, and seems like it ought to be included for its future usefulness. I propose adding a metadata file REQUIRED within the .egg-info directory. The presence of this file indicates that the user specifically required this distribution. The absence of the file indicates that the distribution was installed as a dependency. The contents of the file are not used. For the API, I propose adding a "required" property to the Distribution class, which would be True or False based on the presence or absence of the REQUIRED file. I've added a demo implementation to a fork of Tarek's pep376 repo on bitbucket: http://bitbucket.org/carljm/pep376/changeset/0c8002e65cb7/ Thoughts? Carl -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.6 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iD8DBQFKzhZa1j/fhc23WEARApy3AJ0bbzHBaS85dURlRDEcQOHlrgovXgCg022F OkpQ56+Re1yRnoNFfFNYm88= =nkMB -----END PGP SIGNATURE----- From carl at meyerloewen.net Thu Oct 8 18:39:33 2009 From: carl at meyerloewen.net (Carl Meyer) Date: Thu, 08 Oct 2009 12:39:33 -0400 Subject: [Distutils] tracking requested vs dependency installs in PEP 376 metadata Message-ID: <4ACE15C5.4030105@meyerloewen.net> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Hey all, I propose adding a bit to the PEP 376 metadata that indicates whether a package was installed by user request or as a dependency of another package. This would allow (un)installer tools to intelligently remove orphaned dependencies, if they so choose. There might be questions about the details of such an uninstaller feature, but I'm not intending to discuss that here. The metadata itself is simple enough to track, and seems like it ought to be included for its future usefulness. I propose adding a metadata file REQUIRED within the .egg-info directory. The presence of this file indicates that the user specifically required this distribution. The absence of the file indicates that the distribution was installed as a dependency. The contents of the file are not used. For the API, I propose adding a "required" property to the Distribution class, which would be True or False based on the presence or absence of the REQUIRED file. I've added a demo implementation to a fork of Tarek's pep376 repo on bitbucket: http://bitbucket.org/carljm/pep376/changeset/0c8002e65cb7/ Thoughts? Carl -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.6 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iD8DBQFKzhXFFRcxmeyPUXIRAvPkAJ4+bBPrt0fvYi2u/LhpLo2KS9ImfACfX9oq 5gab4TRuuWWUknOO0SK2E6I= =Dm+n -----END PGP SIGNATURE----- From ianb at colorstudy.com Thu Oct 8 20:50:45 2009 From: ianb at colorstudy.com (Ian Bicking) Date: Thu, 8 Oct 2009 13:50:45 -0500 Subject: [Distutils] tracking requested vs dependency installs in PEP 376 metadata In-Reply-To: <4ACE15C5.4030105@meyerloewen.net> References: <4ACE15C5.4030105@meyerloewen.net> Message-ID: On Thu, Oct 8, 2009 at 11:39 AM, Carl Meyer wrote: > I propose adding a metadata file REQUIRED within the .egg-info > directory. The presence of this file indicates that the user > specifically required this distribution. The absence of the file > indicates that the distribution was installed as a dependency. The > contents of the file are not used. > I can imagine adding a little information, basically a log of when and why and who installed the package. For instance: agent: pip 0.5 install-date: 2009-10-08T13:44:00UTC installed-for-user: False installed-for-package: OtherPackage==0.3 Potentially a package could have multiple records, because multiple installers may in some sense touch a package (e.g., if you install YetAnotherPackage that requires the same library as OtherPackage). You could use INI-style and maybe label each record with the date, like [2009-10-08T13:44:00UTC]. This information seems fairly easy to generate. Updating it after installation would be nice, but also means already-installed packages can be written to, which is not as nice IMHO. Being unable to write to this file should be a non-fatal error for an installer. Either way, a package could become REQUIRED (or user-requested) at any time after it is installed. E.g.: easy_install Markdown easy_install ElementTree # which is required by MarkDown Now ElementTree should not be considered orphaned if MarkDown is removed. -- Ian Bicking | http://blog.ianbicking.org | http://topplabs.org/civichacker -------------- next part -------------- An HTML attachment was scrubbed... URL: From menesis at pov.lt Thu Oct 8 21:00:44 2009 From: menesis at pov.lt (Gediminas Paulauskas) Date: Thu, 8 Oct 2009 22:00:44 +0300 Subject: [Distutils] tracking requested vs dependency installs in PEP 376 metadata In-Reply-To: <4ACE15C5.4030105@meyerloewen.net> References: <4ACE15C5.4030105@meyerloewen.net> Message-ID: <2f8bd9a60910081200w7aaa27deu665641d6ec5d12a9@mail.gmail.com> 2009/10/8 Carl Meyer : > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > Hey all, > > I propose adding a bit to the PEP 376 metadata that indicates whether a > package was installed by user request or as a dependency of another > package. This would allow (un)installer tools to intelligently remove > orphaned dependencies, if they so choose. There might be questions about > the details of such an uninstaller feature, but I'm not intending to > discuss that here. The metadata itself is simple enough to track, and > seems like it ought to be included for its future usefulness. > > I propose adding a metadata file REQUIRED within the .egg-info > directory. The presence of this file indicates that the user > specifically required this distribution. The absence of the file > indicates that the distribution was installed as a dependency. The > contents of the file are not used. Debian's Apt has this capability, see https://wiki.ubuntu.com/PackageDependencyManagement . It keeps a separate file to track the manually installed packages, and the flag is named "Auto-Installed". REQUIRED is an ambiguous name -- is it required by other packages, or wanted by user for any reason (and that is the case)? > > For the API, I propose adding a "required" property to the Distribution > class, which would be True or False based on the presence or absence of > the REQUIRED file. > > I've added a demo implementation to a fork of Tarek's pep376 repo on > bitbucket: http://bitbucket.org/carljm/pep376/changeset/0c8002e65cb7/ > > Thoughts? > > Carl From a.badger at gmail.com Thu Oct 8 21:15:26 2009 From: a.badger at gmail.com (Toshio Kuratomi) Date: Thu, 8 Oct 2009 12:15:26 -0700 Subject: [Distutils] tracking requested vs dependency installs in PEP 376 metadata In-Reply-To: <4ACE15C5.4030105@meyerloewen.net> References: <4ACE15C5.4030105@meyerloewen.net> Message-ID: <20091008191526.GD513@clingman.lan> On Thu, Oct 08, 2009 at 12:39:33PM -0400, Carl Meyer wrote: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > Hey all, > > I propose adding a bit to the PEP 376 metadata that indicates whether a > package was installed by user request or as a dependency of another > package. This would allow (un)installer tools to intelligently remove > orphaned dependencies, if they so choose. There might be questions about > the details of such an uninstaller feature, but I'm not intending to > discuss that here. The metadata itself is simple enough to track, and > seems like it ought to be included for its future usefulness. > > I propose adding a metadata file REQUIRED within the .egg-info > directory. The presence of this file indicates that the user > specifically required this distribution. The absence of the file > indicates that the distribution was installed as a dependency. The > contents of the file are not used. > > For the API, I propose adding a "required" property to the Distribution > class, which would be True or False based on the presence or absence of > the REQUIRED file. > > I've added a demo implementation to a fork of Tarek's pep376 repo on > bitbucket: http://bitbucket.org/carljm/pep376/changeset/0c8002e65cb7/ > > Thoughts? > Note that Linux distributions have discussed this for ages and it's not always as useful as a naive first thought would imply. For instance, there are often many scripts written by a system administrator (or a user) that might need to have a module installed. This is not to say that it's a bad idea to record this information -- there could be some installers for specific use cases might find it useful or that it could be useful with confirmation by the user. Also note that a package manager should be able to tell required status from what is currently installed. So it might make more semantic sense to record what was requested by the user to be installed instead of what was required by a package. (When something is both required by a package and requested by a user, the user request is what takes precedence.) -Toshio -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 197 bytes Desc: not available URL: From carl at meyerloewen.net Thu Oct 8 21:33:15 2009 From: carl at meyerloewen.net (Carl Meyer) Date: Thu, 08 Oct 2009 15:33:15 -0400 Subject: [Distutils] tracking requested vs dependency installs in PEP 376 metadata In-Reply-To: References: <4ACE15C5.4030105@meyerloewen.net> Message-ID: <4ACE3E7B.6040802@meyerloewen.net> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Ian Bicking wrote: > I can imagine adding a little information, basically a log of when and > why and who installed the package. For instance: > > agent: pip 0.5 > install-date: 2009-10-08T13:44:00UTC > installed-for-user: False > installed-for-package: OtherPackage==0.3 > > Potentially a package could have multiple records, because multiple > installers may in some sense touch a package (e.g., if you install > YetAnotherPackage that requires the same library as OtherPackage). You > could use INI-style and maybe label each record with the date, like > [2009-10-08T13:44:00UTC]. Sure; I considered this possibility too, but wanted to keep the initial proposal to "the simplest thing that could possibly work." I have no problem with a log-style approach that includes more information, and if it makes sense to others I can work on the implementation. > This information seems fairly easy to generate. Updating it after > installation would be nice, but also means already-installed packages > can be written to, which is not as nice IMHO. Being unable to write to > this file should be a non-fatal error for an installer. Agreed. In my simple proposal, the only time an already-installed package would need to be touched again is if you're explicitly-installing an already-installed dependency (as in your example below). In that case the REQUIRED file would be added, but in that case touching the installation should be both possible and uncontroversial, since you're specifically asking "again" to install it. > Either way, a package could become REQUIRED (or user-requested) at any > time after it is installed. E.g.: > > easy_install Markdown > easy_install ElementTree # which is required by MarkDown > > Now ElementTree should not be considered orphaned if MarkDown is removed. Exactly. Carl -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.6 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iD4DBQFKzj57FRcxmeyPUXIRAvtfAJ4wu1d+30FF1nmSu23qKGshLphPYACXfwqs 04ISfNfoulbSf0HGDZ3f6g== =vQ0O -----END PGP SIGNATURE----- From carl at meyerloewen.net Thu Oct 8 21:35:03 2009 From: carl at meyerloewen.net (Carl Meyer) Date: Thu, 08 Oct 2009 15:35:03 -0400 Subject: [Distutils] tracking requested vs dependency installs in PEP 376 metadata In-Reply-To: <2f8bd9a60910081200w7aaa27deu665641d6ec5d12a9@mail.gmail.com> References: <4ACE15C5.4030105@meyerloewen.net> <2f8bd9a60910081200w7aaa27deu665641d6ec5d12a9@mail.gmail.com> Message-ID: <4ACE3EE7.4020100@meyerloewen.net> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Gediminas Paulauskas wrote: > Debian's Apt has this capability, see > https://wiki.ubuntu.com/PackageDependencyManagement . It keeps a > separate file to track the manually installed packages, and the flag > is named "Auto-Installed". REQUIRED is an ambiguous name -- is it > required by other packages, or wanted by user for any reason (and that > is the case)? I agree that the name is not ideal. Perhaps naming the file AUTO_INSTALLED, and flipping the semantics (so the presence of the file allows it to be considered orphaned) would be better. Carl -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.6 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iD8DBQFKzj7nFRcxmeyPUXIRAg+UAJkB51TXE/gKEOcsIhC7ID6/EhZbZACfcNS2 ZT7zcYCyy6VLy2qyxr6o+Ys= =q2mH -----END PGP SIGNATURE----- From carl at meyerloewen.net Thu Oct 8 21:41:44 2009 From: carl at meyerloewen.net (Carl Meyer) Date: Thu, 08 Oct 2009 15:41:44 -0400 Subject: [Distutils] tracking requested vs dependency installs in PEP 376 metadata In-Reply-To: <20091008191526.GD513@clingman.lan> References: <4ACE15C5.4030105@meyerloewen.net> <20091008191526.GD513@clingman.lan> Message-ID: <4ACE4078.70504@meyerloewen.net> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Toshio Kuratomi wrote: > Note that Linux distributions have discussed this for ages and it's not > always as useful as a naive first thought would imply. For instance, there > are often many scripts written by a system administrator (or a user) that > might need to have a module installed. This is not to say that it's a bad > idea to record this information -- there could be some installers for > specific use cases might find it useful or that it could be useful with > confirmation by the user. Sure, I'm aware of these issues, and didn't want to get too deep into implementation choices of (un)install tools. But it seems clear that under some circumstances it could be useful to have a record of which packages were installed "intentionally." > Also note that a package manager should be able to tell required status from > what is currently installed. So it might make more semantic sense to record > what was requested by the user to be installed instead of what was required > by a package. (When something is both required by a package and requested > by a user, the user request is what takes precedence.) Clearly my terminology choice was poor. REQUIRED in my proposal meant "requested by name by a user" (which has to be recorded at install time), not "required as a dependency by other installed packages" (which, as you say, can be calculated at runtime). Would REQUESTED, or AUTO_INSTALLED (with the sense flipped) be better options? Carl -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.6 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iD8DBQFKzkB4FRcxmeyPUXIRArxxAJ9B2YcLng0H0fSbTtLztKc5ILmvWACfcF0y qztiKsrXS9usj14Iz37OE1g= =AlFA -----END PGP SIGNATURE----- From a.badger at gmail.com Thu Oct 8 21:52:27 2009 From: a.badger at gmail.com (Toshio Kuratomi) Date: Thu, 8 Oct 2009 12:52:27 -0700 Subject: [Distutils] tracking requested vs dependency installs in PEP 376 metadata In-Reply-To: <4ACE4078.70504@meyerloewen.net> References: <4ACE15C5.4030105@meyerloewen.net> <20091008191526.GD513@clingman.lan> <4ACE4078.70504@meyerloewen.net> Message-ID: <20091008195227.GA735@clingman.lan> On Thu, Oct 08, 2009 at 03:41:44PM -0400, Carl Meyer wrote: > > Also note that a package manager should be able to tell required status from > > what is currently installed. So it might make more semantic sense to record > > what was requested by the user to be installed instead of what was required > > by a package. (When something is both required by a package and requested > > by a user, the user request is what takes precedence.) > > Clearly my terminology choice was poor. REQUIRED in my proposal meant > "requested by name by a user" (which has to be recorded at install > time), not "required as a dependency by other installed packages" > (which, as you say, can be calculated at runtime). Would REQUESTED, or > AUTO_INSTALLED (with the sense flipped) be better options? > I would say REQUESTED due to my arguments for not recording installed-as-package-dependency. -Toshio -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 197 bytes Desc: not available URL: From arfrever.fta at gmail.com Thu Oct 8 23:07:13 2009 From: arfrever.fta at gmail.com (Arfrever Frehtes Taifersar Arahesis) Date: Thu, 8 Oct 2009 23:07:13 +0200 Subject: [Distutils] Packaging Distribute In-Reply-To: References: <90bb445a0910040810g48f7833cia02536e50e5c19f9@mail.gmail.com> <94bdd2610910041341v2c2483b1jd6c8f4b5707d789a@mail.gmail.com> Message-ID: <200910082307.13513.Arfrever.FTA@gmail.com> 2009-10-04 23:52:25 Sridhar Ratnakumar napisa?(a): > On Sun, 04 Oct 2009 13:41:06 -0700, Tarek Ziad? > wrote: > > > The other way would be to use Distribute instead of Setuptools for > > what the packaging system is calling "setuptools". That's pretty > > much what is happening in Gentoo (arch) and UHU-Linux (dev), > > right now > > Interesting. Gentoo uses distribute but retains the name 'setuptools'? It's because Distribute 0.6.* installs setuptools.* modules. Distribute 0.7.* will be under name dev-python/distribute. > Ah. But what if PJE releases setuptools with the *same* version number > 0.6.3? What would the gentoo folks do in order to get the new setuptools > release in their packaging system? Or did they make a decision of totally > dropping setuptools from their repository? We could switch to back to Setuptools only if Setuptools became more maintained than Distribute. -- Arfrever Frehtes Taifersar Arahesis -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 198 bytes Desc: This is a digitally signed message part. URL: From ianb at colorstudy.com Thu Oct 8 23:41:25 2009 From: ianb at colorstudy.com (Ian Bicking) Date: Thu, 8 Oct 2009 16:41:25 -0500 Subject: [Distutils] dev versions Message-ID: So after creating, say, version 0.3.1, I always mark a package as 0.3.2dev. But this is annoying, you might never create a version 0.3.2 (e.g., 0.4 might be the next level). So, it would be better to use something like 0.3.1~dev. What is considered best practice for this? Ideally something that works with both Setuptools and the upcoming Distribute version spec. -- Ian Bicking | http://blog.ianbicking.org | http://topplabs.org/civichacker -------------- next part -------------- An HTML attachment was scrubbed... URL: From jim at zope.com Fri Oct 9 00:06:05 2009 From: jim at zope.com (Jim Fulton) Date: Thu, 8 Oct 2009 18:06:05 -0400 Subject: [Distutils] dev versions In-Reply-To: References: Message-ID: <1099b90b0910081506o5a6bb1f8o5c165382a61fdb7@mail.gmail.com> On Thu, Oct 8, 2009 at 5:41 PM, Ian Bicking wrote: > So after creating, say, version 0.3.1, I always mark a package as 0.3.2dev. > ?But this is annoying, you might never create a version 0.3.2 (e.g., 0.4 > might be the next level). > So, it would be better to use something like 0.3.1~dev. ?What is considered > best practice for this? ?Ideally something that works with both Setuptools > and the upcoming Distribute version spec. I like using a version of "0" on my project trunks. I set the release version on release tags. I really wish there was a special version (or a version pattern) that indicated that something is a development version *only* and can't be released. I don't think best practices have been established. Jim -- Jim Fulton From a.badger at gmail.com Fri Oct 9 00:03:37 2009 From: a.badger at gmail.com (Toshio Kuratomi) Date: Thu, 8 Oct 2009 15:03:37 -0700 Subject: [Distutils] Packaging Distribute In-Reply-To: <200910082307.13513.Arfrever.FTA@gmail.com> References: <90bb445a0910040810g48f7833cia02536e50e5c19f9@mail.gmail.com> <94bdd2610910041341v2c2483b1jd6c8f4b5707d789a@mail.gmail.com> <200910082307.13513.Arfrever.FTA@gmail.com> Message-ID: <20091008220337.GC1240@clingman.lan> On Thu, Oct 08, 2009 at 11:07:13PM +0200, Arfrever Frehtes Taifersar Arahesis wrote: > 2009-10-04 23:52:25 Sridhar Ratnakumar napisa?(a): > > On Sun, 04 Oct 2009 13:41:06 -0700, Tarek Ziad? > > wrote: > > > > > The other way would be to use Distribute instead of Setuptools for > > > what the packaging system is calling "setuptools". That's pretty > > > much what is happening in Gentoo (arch) and UHU-Linux (dev), > > > right now > > > > Interesting. Gentoo uses distribute but retains the name 'setuptools'? > > It's because Distribute 0.6.* installs setuptools.* modules. > Distribute 0.7.* will be under name dev-python/distribute. > I started thinking about what it might take to do this for Fedora as well. There's a number of worries I have but it sounds attractive because of the increased maintainence support from distribute's upstream. An alternative I thought of would be for us to ship both distribute-0.6 and distribute-0.7 (when it arrives) and parallel install them. Then we can patch the setuptools using packages we maintain to check first for the 0.6 distribute and fall back on setuptools if it's not found. That would hopefully get a bunch of upstreams onto a better supported code base. My question is: will Distribute have a parallel installation plan? For instance, renaming the module provided by 0.7 distribute2? If so, this makes a lot of sense. If not, it's the ability of gentoo to reuse the setuptools name that makes parallel installation of distribute-0.6 and distribute-0.7 easier. -Toshio -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 197 bytes Desc: not available URL: From ziade.tarek at gmail.com Fri Oct 9 00:27:41 2009 From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=) Date: Fri, 9 Oct 2009 00:27:41 +0200 Subject: [Distutils] Packaging Distribute In-Reply-To: <20091008220337.GC1240@clingman.lan> References: <90bb445a0910040810g48f7833cia02536e50e5c19f9@mail.gmail.com> <94bdd2610910041341v2c2483b1jd6c8f4b5707d789a@mail.gmail.com> <200910082307.13513.Arfrever.FTA@gmail.com> <20091008220337.GC1240@clingman.lan> Message-ID: <94bdd2610910081527s70ed6dd7g4e8e2cd157939c96@mail.gmail.com> On Fri, Oct 9, 2009 at 12:03 AM, Toshio Kuratomi wrote: > On Thu, Oct 08, 2009 at 11:07:13PM +0200, Arfrever Frehtes Taifersar Arahesis wrote: >> 2009-10-04 23:52:25 Sridhar Ratnakumar napisa?(a): >> > On Sun, 04 Oct 2009 13:41:06 -0700, Tarek Ziad? >> > wrote: >> > >> > > The other way would be to use Distribute instead of Setuptools for >> > > what the packaging system is calling "setuptools". That's pretty >> > > much what is happening in Gentoo (arch) and UHU-Linux (dev), >> > > right now >> > >> > Interesting. Gentoo uses distribute but retains the name 'setuptools'? >> >> It's because Distribute 0.6.* installs setuptools.* modules. >> Distribute 0.7.* will be under name dev-python/distribute. >> > I started thinking about what it might take to do this for Fedora as well. > There's a number of worries I have but it sounds attractive because of the > increased maintainence support from distribute's upstream. > > An alternative I thought of would be for us to ship both distribute-0.6 and > distribute-0.7 (when it arrives) and parallel install them. ?Then we can > patch the setuptools using packages we maintain to check first for the 0.6 > distribute and fall back on setuptools if it's not found. ?That would > hopefully get a bunch of upstreams onto a better supported code base. > > My question is: will Distribute have a parallel installation plan? ?For > instance, renaming the module provided by 0.7 distribute2? ?If so, this > makes a lot of sense. ?If not, it's the ability of gentoo to reuse the > setuptools name that makes parallel installation of distribute-0.6 and > distribute-0.7 easier. We are not sure yet, but when 0.7 will be ready we have two choices: 1- we ship 0.7 under a new name - e.g. like "distribute2" 2- we ship a fat 0.7 with the old 0.6 packages and modules included. I think 1- is better, because it's cleaner in many ways. Plus, people that will start to use 0.7 will not mind using the "distribute2" name. Tarek From ziade.tarek at gmail.com Fri Oct 9 00:29:48 2009 From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=) Date: Fri, 9 Oct 2009 00:29:48 +0200 Subject: [Distutils] Packaging Distribute In-Reply-To: <94bdd2610910081527s70ed6dd7g4e8e2cd157939c96@mail.gmail.com> References: <90bb445a0910040810g48f7833cia02536e50e5c19f9@mail.gmail.com> <94bdd2610910041341v2c2483b1jd6c8f4b5707d789a@mail.gmail.com> <200910082307.13513.Arfrever.FTA@gmail.com> <20091008220337.GC1240@clingman.lan> <94bdd2610910081527s70ed6dd7g4e8e2cd157939c96@mail.gmail.com> Message-ID: <94bdd2610910081529v5c1b8a18re5ce90f7335b1486@mail.gmail.com> On Fri, Oct 9, 2009 at 12:27 AM, Tarek Ziad? wrote: > We are not sure yet, but when 0.7 will be ready we have two choices: > > 1- we ship 0.7 under a new name - e.g. like "distribute2" > 2- we ship a fat 0.7 with the old 0.6 packages and modules included. > > I think 1- is better, because it's cleaner in many ways. Plus, people that will > start to use 0.7 will not mind using the "distribute2" name. notice that I was in favor of 2- earlier but I changed my mind after some readings today ;) From ianb at colorstudy.com Fri Oct 9 00:40:10 2009 From: ianb at colorstudy.com (Ian Bicking) Date: Thu, 8 Oct 2009 17:40:10 -0500 Subject: [Distutils] dev versions In-Reply-To: <1099b90b0910081506o5a6bb1f8o5c165382a61fdb7@mail.gmail.com> References: <1099b90b0910081506o5a6bb1f8o5c165382a61fdb7@mail.gmail.com> Message-ID: On Thu, Oct 8, 2009 at 5:06 PM, Jim Fulton wrote: > On Thu, Oct 8, 2009 at 5:41 PM, Ian Bicking wrote: > > So after creating, say, version 0.3.1, I always mark a package as > 0.3.2dev. > > But this is annoying, you might never create a version 0.3.2 (e.g., 0.4 > > might be the next level). > > So, it would be better to use something like 0.3.1~dev. What is > considered > > best practice for this? Ideally something that works with both > Setuptools > > and the upcoming Distribute version spec. > > I like using a version of "0" on my project trunks. I set the release > version on release tags. > I really wish there was a special version (or a version pattern) that > indicated that something is a development version *only* and can't be > released. I don't think best practices have been established. > It would be nice if there was a sense of branches. E.g., if I fork a project, say setuptools-0.6c3, I could make setuptools-ianb-0.6c3 and someone could install, say, setuptools==ianb, getting whatever was the newest version of my branch. But ianb-0.6c3 wouldn't be comparable to any other version except versions on that branch. Though once installed it would satisfy a generic "setuptools" requirement. This could be used for a dev branch as well, which would satisfy a requirement but not be considered part of the same version series as the stable releases. -- Ian Bicking | http://blog.ianbicking.org | http://topplabs.org/civichacker -------------- next part -------------- An HTML attachment was scrubbed... URL: From zooko at zooko.com Fri Oct 9 00:40:46 2009 From: zooko at zooko.com (Zooko Wilcox-O'Hearn) Date: Thu, 8 Oct 2009 16:40:46 -0600 Subject: [Distutils] dev versions In-Reply-To: References: Message-ID: <68F76A34-16CE-4313-B1CF-7219EF9F9269@zooko.com> On Thursday,2009-10-08, at 15:41 , Ian Bicking wrote: > So after creating, say, version 0.3.1, I always mark a package as > 0.3.2dev. But this is annoying, you might never create a version > 0.3.2 (e.g., 0.4 might be the next level). > > So, it would be better to use something like 0.3.1~dev. What is > considered best practice for this? Ideally something that works > with both Setuptools and the upcoming Distribute version spec. What we do in the Tahoe-LAFS project is we don't count down to a future version, we only count up from a past version. This is also what Twisted does (no coincidence -- we probably got the idea from them). So the most recent release of Tahoe-LAFS was 1.5.0. The next time we committed a patch to our darcs repository, it automatically got the version number 1.5.0-r4040. (The automation to do this is our own implementation -- darcsver [1], but it produces the same kind of output as setuptools's builtin svn integration and Twisted's own svn integration does.) The current snapshot build of Tahoe-LAFS is numbered 1.5.0-r4073, and eventually we will have one numbered something like 1.5.0-r4123, and then the very next version will be 1.6.0. The next version after that will be 1.6.0-r4125. This means that we don't have version numbers with things like "beta" in their name. The most recent time that we had a beta was, I think, Tahoe-LAFS 1.4.1-r4021. We just announced on our mailing list "Hey everybody, we're ready to make a new release! Please try the current version (1.4.1-r4021) and tell us if it works for you!". Regards, Zooko [1] http://pypi.python.org/pypi/darcsver From ziade.tarek at gmail.com Fri Oct 9 01:13:28 2009 From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=) Date: Fri, 9 Oct 2009 01:13:28 +0200 Subject: [Distutils] tracking requested vs dependency installs in PEP 376 metadata In-Reply-To: <20091008195227.GA735@clingman.lan> References: <4ACE15C5.4030105@meyerloewen.net> <20091008191526.GD513@clingman.lan> <4ACE4078.70504@meyerloewen.net> <20091008195227.GA735@clingman.lan> Message-ID: <94bdd2610910081613q2e152711k55c695ff6c33db05@mail.gmail.com> Ian Bicking wrote: > I can imagine adding a little information, basically a log of when and > why and who installed the package. For instance: > > agent: pip 0.5 > install-date: 2009-10-08T13:44:00UTC > installed-for-user: False > installed-for-package: OtherPackage==0.3 > > Potentially a package could have multiple records, because multiple > installers may in some sense touch a package (e.g., if you install > YetAnotherPackage that requires the same library as OtherPackage). You > could use INI-style and maybe label each record with the date, like > [2009-10-08T13:44:00UTC]. Carl: > Sure; I considered this possibility too, but wanted to keep the initial > proposal to "the simplest thing that could possibly work." I have no > problem with a log-style approach that includes more information, and if > it makes sense to others I can work on the implementation. I'd go for the simplest approach at first. On Thu, Oct 8, 2009 at 9:52 PM, Toshio Kuratomi wrote: > I would say REQUESTED due to my arguments for not recording > installed-as-package-dependency. +1 Tarek From floris.bruynooghe at gmail.com Fri Oct 9 01:20:10 2009 From: floris.bruynooghe at gmail.com (Floris Bruynooghe) Date: Fri, 9 Oct 2009 00:20:10 +0100 Subject: [Distutils] dev versions In-Reply-To: References: Message-ID: <20091008232010.GA19792@laurie.devork> On Thu, Oct 08, 2009 at 04:41:25PM -0500, Ian Bicking wrote: > So after creating, say, version 0.3.1, I always mark a package as 0.3.2dev. > But this is annoying, you might never create a version 0.3.2 (e.g., 0.4 > might be the next level). > So, it would be better to use something like 0.3.1~dev. What is considered > best practice for this? Ideally something that works with both Setuptools > and the upcoming Distribute version spec. The verlib [0] aka PEP 386 way as currently proposed is to use 0.3.1.post123 to do count away from releases (i.e. "post-releases"). Regards Floris [0] http://www.python.org/dev/peps/pep-0386/ -- Debian GNU/Linux -- The Power of Freedom www.debian.org | www.gnu.org | www.kernel.org From regebro at gmail.com Fri Oct 9 07:32:17 2009 From: regebro at gmail.com (Lennart Regebro) Date: Fri, 9 Oct 2009 07:32:17 +0200 Subject: [Distutils] dev versions In-Reply-To: <1099b90b0910081506o5a6bb1f8o5c165382a61fdb7@mail.gmail.com> References: <1099b90b0910081506o5a6bb1f8o5c165382a61fdb7@mail.gmail.com> Message-ID: <319e029f0910082232l2e62f2c0q1c4bc841f7b51898@mail.gmail.com> 2009/10/9 Jim Fulton : > I like using a version of "0" on my project trunks. ?I set the release > version on release tags. > I really wish there was a special version (or a version pattern) that > indicated that something is a development version *only* and can't be > released. ?I don't think best practices have been established. Some of the projects I work on have a setup.cfg which says it's a development version. If you then build it you get the horrid package version with r34567654 in it, and you notice that you shouldn't release it. I then usually delete it in the tag. I guess that's the "official" version of setting the the version to 0. I have no idea if something is better or worse. :) -- Lennart Regebro: Python, Zope, Plone, Grok http://regebro.wordpress.com/ +33 661 58 14 64 From menesis at pov.lt Fri Oct 9 11:10:28 2009 From: menesis at pov.lt (Gediminas Paulauskas) Date: Fri, 9 Oct 2009 12:10:28 +0300 Subject: [Distutils] tracking requested vs dependency installs in PEP 376 metadata In-Reply-To: <4ACE3EE7.4020100@meyerloewen.net> References: <4ACE15C5.4030105@meyerloewen.net> <2f8bd9a60910081200w7aaa27deu665641d6ec5d12a9@mail.gmail.com> <4ACE3EE7.4020100@meyerloewen.net> Message-ID: <2f8bd9a60910090210s6dbead18k10abfaecf280dd8d@mail.gmail.com> 2009/10/8 Carl Meyer : > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > Gediminas Paulauskas wrote: >> Debian's Apt has this capability, see >> https://wiki.ubuntu.com/PackageDependencyManagement . It keeps a >> separate file to track the manually installed packages, and the flag >> is named "Auto-Installed". REQUIRED is an ambiguous name -- is it >> required by other packages, or wanted by user for any reason (and that >> is the case)? > > I agree that the name is not ideal. Perhaps naming the file > AUTO_INSTALLED, and flipping the semantics (so the presence of the file > allows it to be considered orphaned) would be better. For backwards compatibility already installed packages have to be treated as manually installed, because it is not known why they were installed. So the presence of a new file or flag has to signify that it was AUTO_INSTALLED, i.e. the other way around than originally suggested. Gediminas From chris at simplistix.co.uk Fri Oct 9 12:54:57 2009 From: chris at simplistix.co.uk (Chris Withers) Date: Fri, 09 Oct 2009 11:54:57 +0100 Subject: [Distutils] the virtualenv-distribute mess In-Reply-To: <4ACB7D70.9060404@cryptelium.net> References: <4AC8A396.8030306@cryptelium.net> <4ACB1B5A.2090608@simplistix.co.uk> <319e029f0910060402n306a992as4b9d854f7f29d546@mail.gmail.com> <319e029f0910060402m28ab35acnc9a52aefcdc6f04a@mail.gmail.com> <4ACB24AE.4000300@cryptelium.net> <4ACB2527.7010802@cryptelium.net> <319e029f0910060522l29499f46y2805ed0082d975c3@mail.gmail.com> <4ACB7D70.9060404@cryptelium.net> Message-ID: <4ACF1681.7050506@simplistix.co.uk> kiorky wrote: > Hi Lennart, > > If i read 'virtualenv-distribute 1.3.4.2 on pypi' > I can do some googling or even do some Pypi searching for 'virtualenv-distribute'. > > Thus, the first link found may be [1]. > > On this link, the second sentence is: > > """ > The fork was started by Philip Jenvey at > http://bitbucket.org/pjenvey/virtualenv-distribute/ and this version by Florian > Schulze lives at http://bitbucket.org/fschulze/virtualenv-distribute/ Add http://bitbucket.org/kiorky/virtualenv-distribute/ to the list and this, ladies and gentlemen, is why distributed source control sucks. Why are there effectively 3 forks on virtualenv now, just to get it to use distribute? Is it really so hard to work with Ian Bicking to be the real virtualenv using distribute instead of setuptools, especially in the way of the bdfl pronouncement? Chris From chris at simplistix.co.uk Fri Oct 9 13:08:09 2009 From: chris at simplistix.co.uk (Chris Withers) Date: Fri, 09 Oct 2009 12:08:09 +0100 Subject: [Distutils] distribute and buildout: best practices? In-Reply-To: References: <94bdd2610910070453t63e7e4c4h1c9bf6ca567fa48f@mail.gmail.com> Message-ID: <4ACF1999.4080001@simplistix.co.uk> Reinout van Rees wrote: > I'm still not 100% sure whether it is safe to put "distribute" in the > install_requires list of a package right now, however. As with setuptools, why do you think you need to? Chris From chris at simplistix.co.uk Fri Oct 9 13:07:02 2009 From: chris at simplistix.co.uk (Chris Withers) Date: Fri, 09 Oct 2009 12:07:02 +0100 Subject: [Distutils] why would you ever need to specify setuptools as a dependency? In-Reply-To: References: Message-ID: <4ACF1956.5070903@simplistix.co.uk> Reinout van Rees wrote: > - Do my libraries have to list a dependency on distribute or on setuptools? > Everything zopish has a 'setuptools >= 0.6c9' in it. They shouldn't, unless you only require setuptools after your package is installed and don't use it in your setup.py, which seems unlikely. If you need it in your setup.py, what's the point of specifying it? You would have had to use it by the time you specify the requirement! > - Do I need to change buildout's bootstrap already? Doing that tries to change > my global setuptools, It shouldn't. The normal buildout bootstrap.py doesn't... Chris From chris at simplistix.co.uk Fri Oct 9 13:08:09 2009 From: chris at simplistix.co.uk (Chris Withers) Date: Fri, 09 Oct 2009 12:08:09 +0100 Subject: [Distutils] distribute and buildout: best practices? In-Reply-To: References: <94bdd2610910070453t63e7e4c4h1c9bf6ca567fa48f@mail.gmail.com> Message-ID: <4ACF1999.4080001@simplistix.co.uk> Reinout van Rees wrote: > I'm still not 100% sure whether it is safe to put "distribute" in the > install_requires list of a package right now, however. As with setuptools, why do you think you need to? Chris From chris at simplistix.co.uk Fri Oct 9 13:14:59 2009 From: chris at simplistix.co.uk (Chris Withers) Date: Fri, 09 Oct 2009 12:14:59 +0100 Subject: [Distutils] tracking requested vs dependency installs in PEP 376 metadata In-Reply-To: References: <4ACE15C5.4030105@meyerloewen.net> Message-ID: <4ACF1B33.5070102@simplistix.co.uk> Ian Bicking wrote: > I can imagine adding a little information, basically a log of when and > why and who installed the package. For instance: > > agent: pip 0.5 > install-date: 2009-10-08T13:44:00UTC > installed-for-user: False > installed-for-package: OtherPackage==0.3 I think this is a great idea. Recording more information rather than less and making sure it's kept up to date is a good idea here. Otherwise, when tools come along later down the line that want to make use of this information, they'll have to hack something themselves, and that's what got setuptools into the mess it currently is in the first place! Chris From chris at simplistix.co.uk Fri Oct 9 13:14:59 2009 From: chris at simplistix.co.uk (Chris Withers) Date: Fri, 09 Oct 2009 12:14:59 +0100 Subject: [Distutils] tracking requested vs dependency installs in PEP 376 metadata In-Reply-To: References: <4ACE15C5.4030105@meyerloewen.net> Message-ID: <4ACF1B33.5070102@simplistix.co.uk> Ian Bicking wrote: > I can imagine adding a little information, basically a log of when and > why and who installed the package. For instance: > > agent: pip 0.5 > install-date: 2009-10-08T13:44:00UTC > installed-for-user: False > installed-for-package: OtherPackage==0.3 I think this is a great idea. Recording more information rather than less and making sure it's kept up to date is a good idea here. Otherwise, when tools come along later down the line that want to make use of this information, they'll have to hack something themselves, and that's what got setuptools into the mess it currently is in the first place! Chris From chris at simplistix.co.uk Fri Oct 9 13:17:39 2009 From: chris at simplistix.co.uk (Chris Withers) Date: Fri, 09 Oct 2009 12:17:39 +0100 Subject: [Distutils] Packaging Distribute In-Reply-To: <94bdd2610910081527s70ed6dd7g4e8e2cd157939c96@mail.gmail.com> References: <90bb445a0910040810g48f7833cia02536e50e5c19f9@mail.gmail.com> <94bdd2610910041341v2c2483b1jd6c8f4b5707d789a@mail.gmail.com> <200910082307.13513.Arfrever.FTA@gmail.com> <20091008220337.GC1240@clingman.lan> <94bdd2610910081527s70ed6dd7g4e8e2cd157939c96@mail.gmail.com> Message-ID: <4ACF1BD3.5010402@simplistix.co.uk> Tarek Ziad? wrote: > 1- we ship 0.7 under a new name - e.g. like "distribute2" You could even take the opportunity to rename it completely to something that makes sense ;-) To re-iterate my argument: Distribute doesn't distribute anything. If anything was to be called Distribute, it'd be the PyPI software. Calling it Installer or Pacman might make more sense... Chris From chris at simplistix.co.uk Fri Oct 9 13:17:39 2009 From: chris at simplistix.co.uk (Chris Withers) Date: Fri, 09 Oct 2009 12:17:39 +0100 Subject: [Distutils] Packaging Distribute In-Reply-To: <94bdd2610910081527s70ed6dd7g4e8e2cd157939c96@mail.gmail.com> References: <90bb445a0910040810g48f7833cia02536e50e5c19f9@mail.gmail.com> <94bdd2610910041341v2c2483b1jd6c8f4b5707d789a@mail.gmail.com> <200910082307.13513.Arfrever.FTA@gmail.com> <20091008220337.GC1240@clingman.lan> <94bdd2610910081527s70ed6dd7g4e8e2cd157939c96@mail.gmail.com> Message-ID: <4ACF1BD3.5010402@simplistix.co.uk> Tarek Ziad? wrote: > 1- we ship 0.7 under a new name - e.g. like "distribute2" You could even take the opportunity to rename it completely to something that makes sense ;-) To re-iterate my argument: Distribute doesn't distribute anything. If anything was to be called Distribute, it'd be the PyPI software. Calling it Installer or Pacman might make more sense... Chris From chris at simplistix.co.uk Fri Oct 9 13:03:52 2009 From: chris at simplistix.co.uk (Chris Withers) Date: Fri, 09 Oct 2009 12:03:52 +0100 Subject: [Distutils] PyPM In-Reply-To: References: Message-ID: <4ACF1898.5020902@simplistix.co.uk> Sridhar Ratnakumar wrote: > This release includes a new packaging tool by activestate called Python > Package Manager (PyPM). Is PyPM available separately? > Here's a sample command line output:: > > $ pypm install lxml Where does this get lxml from? How can I control that? Where does this put the package on disk? How can I have separate package requirements for different python projects on the same box? How can I specify the version of lxml required? How can I reproduce the environment required for a particular project? cheers, Chris From chris at simplistix.co.uk Fri Oct 9 13:03:52 2009 From: chris at simplistix.co.uk (Chris Withers) Date: Fri, 09 Oct 2009 12:03:52 +0100 Subject: [Distutils] PyPM In-Reply-To: References: Message-ID: <4ACF1898.5020902@simplistix.co.uk> Sridhar Ratnakumar wrote: > This release includes a new packaging tool by activestate called Python > Package Manager (PyPM). Is PyPM available separately? > Here's a sample command line output:: > > $ pypm install lxml Where does this get lxml from? How can I control that? Where does this put the package on disk? How can I have separate package requirements for different python projects on the same box? How can I specify the version of lxml required? How can I reproduce the environment required for a particular project? cheers, Chris From chris at simplistix.co.uk Fri Oct 9 12:54:57 2009 From: chris at simplistix.co.uk (Chris Withers) Date: Fri, 09 Oct 2009 11:54:57 +0100 Subject: [Distutils] the virtualenv-distribute mess In-Reply-To: <4ACB7D70.9060404@cryptelium.net> References: <4AC8A396.8030306@cryptelium.net> <4ACB1B5A.2090608@simplistix.co.uk> <319e029f0910060402n306a992as4b9d854f7f29d546@mail.gmail.com> <319e029f0910060402m28ab35acnc9a52aefcdc6f04a@mail.gmail.com> <4ACB24AE.4000300@cryptelium.net> <4ACB2527.7010802@cryptelium.net> <319e029f0910060522l29499f46y2805ed0082d975c3@mail.gmail.com> <4ACB7D70.9060404@cryptelium.net> Message-ID: <4ACF1681.7050506@simplistix.co.uk> kiorky wrote: > Hi Lennart, > > If i read 'virtualenv-distribute 1.3.4.2 on pypi' > I can do some googling or even do some Pypi searching for 'virtualenv-distribute'. > > Thus, the first link found may be [1]. > > On this link, the second sentence is: > > """ > The fork was started by Philip Jenvey at > http://bitbucket.org/pjenvey/virtualenv-distribute/ and this version by Florian > Schulze lives at http://bitbucket.org/fschulze/virtualenv-distribute/ Add http://bitbucket.org/kiorky/virtualenv-distribute/ to the list and this, ladies and gentlemen, is why distributed source control sucks. Why are there effectively 3 forks on virtualenv now, just to get it to use distribute? Is it really so hard to work with Ian Bicking to be the real virtualenv using distribute instead of setuptools, especially in the way of the bdfl pronouncement? Chris From chris at simplistix.co.uk Fri Oct 9 13:07:02 2009 From: chris at simplistix.co.uk (Chris Withers) Date: Fri, 09 Oct 2009 12:07:02 +0100 Subject: [Distutils] why would you ever need to specify setuptools as a dependency? In-Reply-To: References: Message-ID: <4ACF1956.5070903@simplistix.co.uk> Reinout van Rees wrote: > - Do my libraries have to list a dependency on distribute or on setuptools? > Everything zopish has a 'setuptools >= 0.6c9' in it. They shouldn't, unless you only require setuptools after your package is installed and don't use it in your setup.py, which seems unlikely. If you need it in your setup.py, what's the point of specifying it? You would have had to use it by the time you specify the requirement! > - Do I need to change buildout's bootstrap already? Doing that tries to change > my global setuptools, It shouldn't. The normal buildout bootstrap.py doesn't... Chris From ziade.tarek at gmail.com Fri Oct 9 13:52:55 2009 From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=) Date: Fri, 9 Oct 2009 13:52:55 +0200 Subject: [Distutils] Packaging Distribute In-Reply-To: <4ACF1BD3.5010402@simplistix.co.uk> References: <90bb445a0910040810g48f7833cia02536e50e5c19f9@mail.gmail.com> <94bdd2610910041341v2c2483b1jd6c8f4b5707d789a@mail.gmail.com> <200910082307.13513.Arfrever.FTA@gmail.com> <20091008220337.GC1240@clingman.lan> <94bdd2610910081527s70ed6dd7g4e8e2cd157939c96@mail.gmail.com> <4ACF1BD3.5010402@simplistix.co.uk> Message-ID: <94bdd2610910090452l52e0cad5p4c11ab9989fc8b4b@mail.gmail.com> On Fri, Oct 9, 2009 at 1:17 PM, Chris Withers wrote: > Tarek Ziad? wrote: >> >> 1- we ship 0.7 under a new name - e.g. like "distribute2" > > You could even take the opportunity to rename it completely to something > that makes sense ;-) > > To re-iterate my argument: Distribute doesn't distribute anything. If > anything was to be called Distribute, it'd be the PyPI software. Calling it > Installer or Pacman might make more sense... Sorry I won't run a new poll again, Distribute is the name. Besides, it's already on page #1 on google when I type 'python disribute' or 'distribute python' Tarek From regebro at gmail.com Fri Oct 9 13:53:59 2009 From: regebro at gmail.com (Lennart Regebro) Date: Fri, 9 Oct 2009 13:53:59 +0200 Subject: [Distutils] the virtualenv-distribute mess In-Reply-To: <4ACF1681.7050506@simplistix.co.uk> References: <4AC8A396.8030306@cryptelium.net> <4ACB1B5A.2090608@simplistix.co.uk> <319e029f0910060402n306a992as4b9d854f7f29d546@mail.gmail.com> <319e029f0910060402m28ab35acnc9a52aefcdc6f04a@mail.gmail.com> <4ACB24AE.4000300@cryptelium.net> <4ACB2527.7010802@cryptelium.net> <319e029f0910060522l29499f46y2805ed0082d975c3@mail.gmail.com> <4ACB7D70.9060404@cryptelium.net> <4ACF1681.7050506@simplistix.co.uk> Message-ID: <319e029f0910090453k47739bdew773e6ae5832a4679@mail.gmail.com> 2009/10/9 Chris Withers : > Why are there effectively 3 forks on virtualenv now, just to get it to use > distribute? Is it really so hard to work with Ian Bicking to be the real > virtualenv using distribute instead of setuptools Well I would expect Ian to want to take it a bit easy with the switch. But I think the time has come now. I sure would be happy to help after the PloneConf when I have some more time, also with getting it onto Python 3. -- Lennart Regebro: Python, Zope, Plone, Grok http://regebro.wordpress.com/ +33 661 58 14 64 From ziade.tarek at gmail.com Fri Oct 9 13:54:56 2009 From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=) Date: Fri, 9 Oct 2009 13:54:56 +0200 Subject: [Distutils] the virtualenv-distribute mess In-Reply-To: <4ACF1681.7050506@simplistix.co.uk> References: <4AC8A396.8030306@cryptelium.net> <4ACB1B5A.2090608@simplistix.co.uk> <319e029f0910060402n306a992as4b9d854f7f29d546@mail.gmail.com> <319e029f0910060402m28ab35acnc9a52aefcdc6f04a@mail.gmail.com> <4ACB24AE.4000300@cryptelium.net> <4ACB2527.7010802@cryptelium.net> <319e029f0910060522l29499f46y2805ed0082d975c3@mail.gmail.com> <4ACB7D70.9060404@cryptelium.net> <4ACF1681.7050506@simplistix.co.uk> Message-ID: <94bdd2610910090454y6dffc360q46ed05b4b759064f@mail.gmail.com> On Fri, Oct 9, 2009 at 12:54 PM, Chris Withers wrote: > kiorky wrote: >> >> Hi Lennart, >> >> If i read 'virtualenv-distribute 1.3.4.2 on pypi' >> I can ?do some googling or even do some Pypi searching for >> ?'virtualenv-distribute'. >> >> Thus, the first link found may be [1]. >> >> On this link, the second sentence is: >> >> """ >> The fork was started by Philip Jenvey at >> http://bitbucket.org/pjenvey/virtualenv-distribute/ and this version by >> Florian >> Schulze lives at http://bitbucket.org/fschulze/virtualenv-distribute/ > > Add http://bitbucket.org/kiorky/virtualenv-distribute/ to the list and this, > ladies and gentlemen, is why distributed source control sucks. > > Why are there effectively 3 forks on virtualenv now, just to get it to use > distribute? Is it really so hard to work with Ian Bicking to be the real > virtualenv using distribute instead of setuptools, especially in the way of > the bdfl pronouncement? > I think the word "fork" here, in DVCS principles, just means that you copy a repository to work with, and eventually ask for the main repo to merge the changes. Which is fine imho. Tarek From alex.gronholm at nextday.fi Fri Oct 9 13:55:41 2009 From: alex.gronholm at nextday.fi (=?UTF-8?B?QWxleCBHcsO2bmhvbG0=?=) Date: Fri, 09 Oct 2009 14:55:41 +0300 Subject: [Distutils] Packaging Distribute In-Reply-To: <4ACF1BD3.5010402@simplistix.co.uk> References: <90bb445a0910040810g48f7833cia02536e50e5c19f9@mail.gmail.com> <94bdd2610910041341v2c2483b1jd6c8f4b5707d789a@mail.gmail.com> <200910082307.13513.Arfrever.FTA@gmail.com> <20091008220337.GC1240@clingman.lan> <94bdd2610910081527s70ed6dd7g4e8e2cd157939c96@mail.gmail.com> <4ACF1BD3.5010402@simplistix.co.uk> Message-ID: <4ACF24BD.8050601@nextday.fi> Chris Withers kirjoitti: > Tarek Ziad? wrote: >> 1- we ship 0.7 under a new name - e.g. like "distribute2" > > You could even take the opportunity to rename it completely to > something that makes sense ;-) > While I agree that the name was badly chosen (for reasons you outline below), a name change at this point would introduce even more confusion. > To re-iterate my argument: Distribute doesn't distribute anything. If > anything was to be called Distribute, it'd be the PyPI software. > Calling it Installer or Pacman might make more sense... > > Chris > > > _______________________________________________ > Distutils-SIG maillist - Distutils-SIG at python.org > http://mail.python.org/mailman/listinfo/distutils-sig From hanno at hannosch.eu Fri Oct 9 13:57:56 2009 From: hanno at hannosch.eu (Hanno Schlichting) Date: Fri, 9 Oct 2009 13:57:56 +0200 Subject: [Distutils] why would you ever need to specify setuptools as a dependency? In-Reply-To: <4ACF1956.5070903@simplistix.co.uk> References: <4ACF1956.5070903@simplistix.co.uk> Message-ID: <5cae42b20910090457p3f6e4a1bp361b240f974f1617@mail.gmail.com> On Fri, Oct 9, 2009 at 1:07 PM, Chris Withers wrote: > Reinout van Rees wrote: >> >> - Do my libraries have to list a dependency on distribute or on >> setuptools? >> ?Everything zopish has a 'setuptools >= 0.6c9' in it. > > They shouldn't, unless you only require setuptools after your package is > installed and don't use it in your setup.py, which seems unlikely. > > If you need it in your setup.py, what's the point of specifying it? You > would have had to use it by the time you specify the requirement! I assume most packages Reinout uses (like all zope.* packages) use namespace packages. So they actually do depend during runtime on the pkg_resources module, which happens to be available from either the distribute or setuptools distribution. So one of them should be specified in install_requires. Hanno From kiorky at cryptelium.net Fri Oct 9 14:03:40 2009 From: kiorky at cryptelium.net (kiorky) Date: Fri, 09 Oct 2009 14:03:40 +0200 Subject: [Distutils] the virtualenv-distribute mess In-Reply-To: <4ACF1681.7050506@simplistix.co.uk> References: <4AC8A396.8030306@cryptelium.net> <4ACB1B5A.2090608@simplistix.co.uk> <319e029f0910060402n306a992as4b9d854f7f29d546@mail.gmail.com> <319e029f0910060402m28ab35acnc9a52aefcdc6f04a@mail.gmail.com> <4ACB24AE.4000300@cryptelium.net> <4ACB2527.7010802@cryptelium.net> <319e029f0910060522l29499f46y2805ed0082d975c3@mail.gmail.com> <4ACB7D70.9060404@cryptelium.net> <4ACF1681.7050506@simplistix.co.uk> Message-ID: <4ACF269C.4010400@cryptelium.net> Hi Chris, As far as i know, we just give links for power users/developers to know how to find the actual related repositories. It s *CLEARLY* specified *IN THE HOMEPAGE METADATA* that the official thing lives inside florian branch. Those are not 3 forks, but 3 branches or the same code. And the florian branch is the reference as Linux kernel branch. And no DVCS is not a mess, this is just a mean to have bugfixes available as Fast as possible and it's a great way for the original authors to get and merge those fixes. AND no, virtualenv must continue to provide setuptools, backward compatibility, you know? Chris Withers a ?crit : > kiorky wrote: >> Hi Lennart, >> >> If i read 'virtualenv-distribute 1.3.4.2 on pypi' >> I can do some googling or even do some Pypi searching for >> 'virtualenv-distribute'. >> >> Thus, the first link found may be [1]. >> >> On this link, the second sentence is: >> >> """ >> The fork was started by Philip Jenvey at >> http://bitbucket.org/pjenvey/virtualenv-distribute/ and this version >> by Florian >> Schulze lives at http://bitbucket.org/fschulze/virtualenv-distribute/ > > Add http://bitbucket.org/kiorky/virtualenv-distribute/ to the list and > this, ladies and gentlemen, is why distributed source control sucks. > > Why are there effectively 3 forks on virtualenv now, just to get it to > use distribute? Is it really so hard to work with Ian Bicking to be the > real virtualenv using distribute instead of setuptools, especially in > the way of the bdfl pronouncement? > > Chris > -- -- Cordialement, KiOrKY GPG Key FingerPrint: 0x1A1194B7681112AF -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 261 bytes Desc: OpenPGP digital signature URL: From ziade.tarek at gmail.com Fri Oct 9 14:08:29 2009 From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=) Date: Fri, 9 Oct 2009 14:08:29 +0200 Subject: [Distutils] the virtualenv-distribute mess In-Reply-To: <4ACF269C.4010400@cryptelium.net> References: <4ACB1B5A.2090608@simplistix.co.uk> <319e029f0910060402n306a992as4b9d854f7f29d546@mail.gmail.com> <319e029f0910060402m28ab35acnc9a52aefcdc6f04a@mail.gmail.com> <4ACB24AE.4000300@cryptelium.net> <4ACB2527.7010802@cryptelium.net> <319e029f0910060522l29499f46y2805ed0082d975c3@mail.gmail.com> <4ACB7D70.9060404@cryptelium.net> <4ACF1681.7050506@simplistix.co.uk> <4ACF269C.4010400@cryptelium.net> Message-ID: <94bdd2610910090508j6804628fy5b5d435cce0c7427@mail.gmail.com> On Fri, Oct 9, 2009 at 2:03 PM, kiorky wrote: > AND no, virtualenv must continue to provide setuptools, backward compatibility, > you know? The *whole* point of Distribute 0.6.x is to be backward compatible, meaning that if virtualenv switch to it, you will not even notice it. Tarek From ben+python at benfinney.id.au Fri Oct 9 14:05:11 2009 From: ben+python at benfinney.id.au (Ben Finney) Date: Fri, 09 Oct 2009 23:05:11 +1100 Subject: [Distutils] the virtualenv-distribute mess References: <4AC8A396.8030306@cryptelium.net> <4ACB1B5A.2090608@simplistix.co.uk> <319e029f0910060402n306a992as4b9d854f7f29d546@mail.gmail.com> <319e029f0910060402m28ab35acnc9a52aefcdc6f04a@mail.gmail.com> <4ACB24AE.4000300@cryptelium.net> <4ACB2527.7010802@cryptelium.net> <319e029f0910060522l29499f46y2805ed0082d975c3@mail.gmail.com> <4ACB7D70.9060404@cryptelium.net> <4ACF1681.7050506@simplistix.co.uk> Message-ID: <87ab00u614.fsf@benfinney.id.au> Chris Withers writes: > Why are there effectively 3 forks on virtualenv now, just to get it to > use distribute? Is it really so hard to work with Ian Bicking to be > the real virtualenv using distribute instead of setuptools, especially > in the way of the bdfl pronouncement? You do realise that, because those branches share ancestry with Ian's branch, they can be merged back in? That, by being able to put a branch online that the VCS tool knows how to merge, it's much *easier* to collaborate than with centralised VCS? That, without any explicit co-ordination between the parties, Ian Bicking can take any of those branches and merge whatever changes he finds appropriate back into his own branch, and vice versa? This is the strength of distributed VCS like Bazaar, Git, and Mercurial: you're seeing *more* collaboration because there is less administrative barrier required to put a branch online and still have it be VCS-compatible with the original. -- \ ?It is seldom that liberty of any kind is lost all at once.? | `\ ?David Hume | _o__) | Ben Finney From regebro at gmail.com Fri Oct 9 14:09:38 2009 From: regebro at gmail.com (Lennart Regebro) Date: Fri, 9 Oct 2009 14:09:38 +0200 Subject: [Distutils] the virtualenv-distribute mess In-Reply-To: <94bdd2610910090454y6dffc360q46ed05b4b759064f@mail.gmail.com> References: <4ACB1B5A.2090608@simplistix.co.uk> <319e029f0910060402n306a992as4b9d854f7f29d546@mail.gmail.com> <319e029f0910060402m28ab35acnc9a52aefcdc6f04a@mail.gmail.com> <4ACB24AE.4000300@cryptelium.net> <4ACB2527.7010802@cryptelium.net> <319e029f0910060522l29499f46y2805ed0082d975c3@mail.gmail.com> <4ACB7D70.9060404@cryptelium.net> <4ACF1681.7050506@simplistix.co.uk> <94bdd2610910090454y6dffc360q46ed05b4b759064f@mail.gmail.com> Message-ID: <319e029f0910090509i23c09868s568af2827a3fe74b@mail.gmail.com> 2009/10/9 Tarek Ziad? : > I think the word "fork" ?here, in DVCS principles, just means that you > copy a repository > to work with, and eventually ask for the main repo to merge the changes. Yeah, it's just a branch, basically. But it is hard to get an overview. -- Lennart Regebro: Python, Zope, Plone, Grok http://regebro.wordpress.com/ +33 661 58 14 64 From hanno at hannosch.eu Fri Oct 9 14:10:36 2009 From: hanno at hannosch.eu (Hanno Schlichting) Date: Fri, 9 Oct 2009 14:10:36 +0200 Subject: [Distutils] PyPM In-Reply-To: <4ACF1898.5020902@simplistix.co.uk> References: <4ACF1898.5020902@simplistix.co.uk> Message-ID: <5cae42b20910090510v74da6328s9644f52b1eee385e@mail.gmail.com> On Fri, Oct 9, 2009 at 1:03 PM, Chris Withers wrote: > Sridhar Ratnakumar wrote: >> >> This release includes a new packaging tool by activestate called Python >> Package Manager (PyPM). > > Is PyPM available separately? I had been asked for pre-release feedback on PyPM and suggested to work on the UCS2/UCS4 build problem in PyPM. As a result the following has been noted (http://workspace.activestate.com/sridharr/pypm/ticket/83): "Jan further commented that we should not be bothering to make PyPM work with custom builds of Python other than ActivePython (which is UCS2 on all platforms) as PyPM is legally supposed to be used only with ActivePython. I originally noted the same .. but later thought about users hosting custom repositories and using custom Python versions (/usr/bin/python on Ubuntu, for instance). But if they are not legally allowed to use anything other ActivePython, why bother?" Which should answer your question about "available separately". Hanno From benji at zope.com Fri Oct 9 14:25:25 2009 From: benji at zope.com (Benji York) Date: Fri, 9 Oct 2009 08:25:25 -0400 Subject: [Distutils] dev versions In-Reply-To: <68F76A34-16CE-4313-B1CF-7219EF9F9269@zooko.com> References: <68F76A34-16CE-4313-B1CF-7219EF9F9269@zooko.com> Message-ID: On Thu, Oct 8, 2009 at 6:40 PM, Zooko Wilcox-O'Hearn wrote: > > What we do in the Tahoe-LAFS project is we don't count down to a future > version, we only count up from a past version. ?This is also what Twisted > does (no coincidence -- we probably got the idea from them). To clarify: that means that a beta for 2.0.0 might have a version of 1.6.3-r4321? -- Benji York Senior Software Engineer Zope Corporation From regebro at gmail.com Fri Oct 9 14:36:11 2009 From: regebro at gmail.com (Lennart Regebro) Date: Fri, 9 Oct 2009 14:36:11 +0200 Subject: [Distutils] the virtualenv-distribute mess In-Reply-To: <94bdd2610910090508j6804628fy5b5d435cce0c7427@mail.gmail.com> References: <319e029f0910060402n306a992as4b9d854f7f29d546@mail.gmail.com> <319e029f0910060402m28ab35acnc9a52aefcdc6f04a@mail.gmail.com> <4ACB24AE.4000300@cryptelium.net> <4ACB2527.7010802@cryptelium.net> <319e029f0910060522l29499f46y2805ed0082d975c3@mail.gmail.com> <4ACB7D70.9060404@cryptelium.net> <4ACF1681.7050506@simplistix.co.uk> <4ACF269C.4010400@cryptelium.net> <94bdd2610910090508j6804628fy5b5d435cce0c7427@mail.gmail.com> Message-ID: <319e029f0910090536j480b2339t530a2f6a9267d451@mail.gmail.com> 2009/10/9 Tarek Ziad? : > The *whole* point of Distribute 0.6.x is to be backward compatible, meaning > that if virtualenv switch to it, you will not even notice it. I guess the point is that is should still work even if you don't want to switch to distribute? -- Lennart Regebro: Python, Zope, Plone, Grok http://regebro.wordpress.com/ +33 661 58 14 64 From ziade.tarek at gmail.com Fri Oct 9 14:40:12 2009 From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=) Date: Fri, 9 Oct 2009 14:40:12 +0200 Subject: [Distutils] the virtualenv-distribute mess In-Reply-To: <319e029f0910090536j480b2339t530a2f6a9267d451@mail.gmail.com> References: <319e029f0910060402m28ab35acnc9a52aefcdc6f04a@mail.gmail.com> <4ACB24AE.4000300@cryptelium.net> <4ACB2527.7010802@cryptelium.net> <319e029f0910060522l29499f46y2805ed0082d975c3@mail.gmail.com> <4ACB7D70.9060404@cryptelium.net> <4ACF1681.7050506@simplistix.co.uk> <4ACF269C.4010400@cryptelium.net> <94bdd2610910090508j6804628fy5b5d435cce0c7427@mail.gmail.com> <319e029f0910090536j480b2339t530a2f6a9267d451@mail.gmail.com> Message-ID: <94bdd2610910090540u72f3f943w8c7ca6833f9ce6fb@mail.gmail.com> On Fri, Oct 9, 2009 at 2:36 PM, Lennart Regebro wrote: > 2009/10/9 Tarek Ziad? : >> The *whole* point of Distribute 0.6.x is to be backward compatible, meaning >> that if virtualenv switch to it, you will not even notice it. > > I guess the point is that is should still work even if you don't want > to switch to distribute? > If "you", is a package developer, yes of course = virtualenv will still behave as before > -- > Lennart Regebro: Python, Zope, Plone, Grok > http://regebro.wordpress.com/ > +33 661 58 14 64 > -- Tarek Ziad? | http://ziade.org |????????????! |???????????? From kiorky at cryptelium.net Fri Oct 9 14:42:45 2009 From: kiorky at cryptelium.net (kiorky) Date: Fri, 09 Oct 2009 14:42:45 +0200 Subject: [Distutils] the virtualenv-distribute mess In-Reply-To: <319e029f0910090536j480b2339t530a2f6a9267d451@mail.gmail.com> References: <319e029f0910060402n306a992as4b9d854f7f29d546@mail.gmail.com> <319e029f0910060402m28ab35acnc9a52aefcdc6f04a@mail.gmail.com> <4ACB24AE.4000300@cryptelium.net> <4ACB2527.7010802@cryptelium.net> <319e029f0910060522l29499f46y2805ed0082d975c3@mail.gmail.com> <4ACB7D70.9060404@cryptelium.net> <4ACF1681.7050506@simplistix.co.uk> <4ACF269C.4010400@cryptelium.net> <94bdd2610910090508j6804628fy5b5d435cce0c7427@mail.gmail.com> <319e029f0910090536j480b2339t530a2f6a9267d451@mail.gmail.com> Message-ID: <4ACF2FC5.6090706@cryptelium.net> Lennart Regebro a ?crit : > 2009/10/9 Tarek Ziad? : >> The *whole* point of Distribute 0.6.x is to be backward compatible, meaning >> that if virtualenv switch to it, you will not even notice it. > > I guess the point is that is should still work even if you don't want > to switch to distribute? > Exact, i want that distribute usage must be explicit, moreover nowadays. -- -- Cordialement, KiOrKY GPG Key FingerPrint: 0x1A1194B7681112AF -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 261 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 261 bytes Desc: OpenPGP digital signature URL: From kiorky at cryptelium.net Fri Oct 9 14:43:02 2009 From: kiorky at cryptelium.net (kiorky) Date: Fri, 09 Oct 2009 14:43:02 +0200 Subject: [Distutils] the virtualenv-distribute mess In-Reply-To: <94bdd2610910090508j6804628fy5b5d435cce0c7427@mail.gmail.com> References: <4ACB1B5A.2090608@simplistix.co.uk> <319e029f0910060402n306a992as4b9d854f7f29d546@mail.gmail.com> <319e029f0910060402m28ab35acnc9a52aefcdc6f04a@mail.gmail.com> <4ACB24AE.4000300@cryptelium.net> <4ACB2527.7010802@cryptelium.net> <319e029f0910060522l29499f46y2805ed0082d975c3@mail.gmail.com> <4ACB7D70.9060404@cryptelium.net> <4ACF1681.7050506@simplistix.co.uk> <4ACF269C.4010400@cryptelium.net> <94bdd2610910090508j6804628fy5b5d435cce0c7427@mail.gmail.com> Message-ID: <4ACF2FD6.4010504@cryptelium.net> Hi tarek, Tarek Ziad? a ?crit : > The *whole* point of Distribute 0.6.x is to be backward compatible, meaning > that if virtualenv switch to it, you will not even notice it. Living in my 0.6.x snail sandbox is not a solution. As it seems that Distribute 0.7 won't for a long time. "setuptools based" packages will be able to be installed via the distribute 0.6 branch but not compatible with "distribute based" stuff. Note that new things will eventually be packaged with the "new good way todo, aka with 0.7". There is a great risk that they can't live together aside. NOGO 0.7 packages wont be compatibles with setuptools installation/namespaces, so it will be impossible to install a lot of "setuptools based" packages aside with new stuff in with this way too. NOGO too. Maybe that the active community, zope, plone, django, etc. folks will take the step to repackage without a word, but there are more chances that more that one package on pypi that is not anymore maintened activly but, with 0.6, you can just use them without problems (eg: things with C code), will be unusable with 0.7. The transiton time will be long, there are no doubts. People has other things to do to repackage / rewrite their code even with a portage helper (ala 2to3)... It has a cost, it's useless, it's painfull, it's risky. Even more if you have undertested packages. But, in the meantime, you won't be able to do flawless deployments, you will have to masterise all about the full python deployment knowledge to install your stuff. That's unacceptable for me. The only way i saw is a compatibility layer inside 0.7 but what i have understood is that it's impossible by setuptools's implementation. Good or bad, that's not the point, i just want stability. I appreciate what you folks are doing with the distribute sphere, i have not that much problems with it, but i do not support that it breaks very badly the retro compatibilty for all things already packaged today, today tomorrow or in one year. > > Tarek -- -- Cordialement, KiOrKY GPG Key FingerPrint: 0x1A1194B7681112AF -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 261 bytes Desc: OpenPGP digital signature URL: From ziade.tarek at gmail.com Fri Oct 9 14:45:57 2009 From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=) Date: Fri, 9 Oct 2009 14:45:57 +0200 Subject: [Distutils] the virtualenv-distribute mess In-Reply-To: <4ACF2FD6.4010504@cryptelium.net> References: <319e029f0910060402m28ab35acnc9a52aefcdc6f04a@mail.gmail.com> <4ACB24AE.4000300@cryptelium.net> <4ACB2527.7010802@cryptelium.net> <319e029f0910060522l29499f46y2805ed0082d975c3@mail.gmail.com> <4ACB7D70.9060404@cryptelium.net> <4ACF1681.7050506@simplistix.co.uk> <4ACF269C.4010400@cryptelium.net> <94bdd2610910090508j6804628fy5b5d435cce0c7427@mail.gmail.com> <4ACF2FD6.4010504@cryptelium.net> Message-ID: <94bdd2610910090545p65a3b172l971fe01f11768a3f@mail.gmail.com> On Fri, Oct 9, 2009 at 2:43 PM, kiorky wrote: > Hi tarek, > > Tarek Ziad? a ?crit : > >> The *whole* point of Distribute 0.6.x is to be backward compatible, meaning >> that if virtualenv switch to it, you will not even notice it. > > Living in my 0.6.x snail sandbox is not a solution. > As it seems that Distribute 0.7 won't for a long time. > > "setuptools based" packages will be able to be installed via the distribute 0.6 > branch but not compatible with "distribute based" stuff. Note that new things > will eventually be packaged with the "new good way todo, aka with 0.7". There is > a great risk that they can't live together aside. NOGO Why they can't ? > > 0.7 packages wont be compatibles with setuptools installation/namespaces, so it > will be impossible to install a lot of "setuptools based" packages aside with > new stuff in with this way too. NOGO too. Why will it be impossible ? [...] > I appreciate what you folks are doing with the distribute sphere, i have not > that much problems with it, but i do not support that it breaks very badly the > retro compatibilty for all things already packaged today, today tomorrow or in > one year. Again, you will be able to use 0.6 and 0.7 together. or 0.6 alone, or 0.7 alone. Nothing will be broken in a distribution that uses 0.6. 0.6 stays maintained. Tarek From ben+python at benfinney.id.au Fri Oct 9 14:46:29 2009 From: ben+python at benfinney.id.au (Ben Finney) Date: Fri, 09 Oct 2009 23:46:29 +1100 Subject: [Distutils] dev versions References: Message-ID: <8763aou44a.fsf@benfinney.id.au> Ian Bicking writes: > So after creating, say, version 0.3.1, I always mark a package as > 0.3.2dev. Why not just mark it 0.3.2 during development, and change the version string in a later revision if warranted? > But this is annoying, you might never create a version 0.3.2 (e.g., > 0.4 might be the next level). So what? If there are revisions in your VCS that have a particular version string 0.3.2, you can look to the release log to see whether this revision is later than version 0.3 and earlier than version 0.4. What does it matter tha 0.3.2 was never actually released? Zooko Wilcox-O'Hearn writes: > On Thursday,2009-10-08, at 15:41 , Ian Bicking wrote: > > What is considered best practice for this? Ideally something that > > works with both Setuptools and the upcoming Distribute version spec. > > What we do in the Tahoe-LAFS project is we don't count down to a > future version, we only count up from a past version. This is also > what Twisted does (no coincidence -- we probably got the idea from > them). That sounds eminently sensible and simple. > So the most recent release of Tahoe-LAFS was 1.5.0. The next time we > committed a patch to our darcs repository, it automatically got the > version number 1.5.0-r4040. Right. If someone later decides that the version string should be incremented during development, they can do that; just so long as it only ever increases (and, by corrollary, that no-one increments the version string lightly). > This means that we don't have version numbers with things like "beta" > in their name. The most recent time that we had a beta was, I think, > Tahoe-LAFS 1.4.1-r4021. We just announced on our mailing list "Hey > everybody, we're ready to make a new release! Please try the current > version (1.4.1-r4021) and tell us if it works for you!". I applaud this practice; I find it far more simple and comprehensible than special out-of-sequence keywords in version strings. -- \ ?The man who is denied the opportunity of taking decisions of | `\ importance begins to regard as important the decisions he is | _o__) allowed to take.? ?C. Northcote Parkinson | Ben Finney From jim at zope.com Fri Oct 9 15:14:00 2009 From: jim at zope.com (Jim Fulton) Date: Fri, 9 Oct 2009 09:14:00 -0400 Subject: [Distutils] why would you ever need to specify setuptools as a dependency? In-Reply-To: <5cae42b20910090457p3f6e4a1bp361b240f974f1617@mail.gmail.com> References: <4ACF1956.5070903@simplistix.co.uk> <5cae42b20910090457p3f6e4a1bp361b240f974f1617@mail.gmail.com> Message-ID: <1099b90b0910090614m5bf1717cp60b401839b62f352@mail.gmail.com> On Fri, Oct 9, 2009 at 7:57 AM, Hanno Schlichting wrote: > On Fri, Oct 9, 2009 at 1:07 PM, Chris Withers wrote: >> Reinout van Rees wrote: >>> >>> - Do my libraries have to list a dependency on distribute or on >>> setuptools? >>> ?Everything zopish has a 'setuptools >= 0.6c9' in it. >> >> They shouldn't, unless you only require setuptools after your package is >> installed and don't use it in your setup.py, which seems unlikely. >> >> If you need it in your setup.py, what's the point of specifying it? You >> would have had to use it by the time you specify the requirement! > > I assume most packages Reinout uses (like all zope.* packages) use > namespace packages. So they actually do depend during runtime on the > pkg_resources module, which happens to be available from either the > distribute ?or setuptools distribution. So one of them should be > specified in install_requires. This functionality should be factored out and merged with pkgutil in the standard library. Jim -- Jim Fulton From carl at meyerloewen.net Fri Oct 9 15:14:00 2009 From: carl at meyerloewen.net (Carl Meyer) Date: Fri, 09 Oct 2009 09:14:00 -0400 Subject: [Distutils] tracking requested vs dependency installs in PEP 376 metadata In-Reply-To: <2f8bd9a60910090210s6dbead18k10abfaecf280dd8d@mail.gmail.com> References: <4ACE15C5.4030105@meyerloewen.net> <2f8bd9a60910081200w7aaa27deu665641d6ec5d12a9@mail.gmail.com> <4ACE3EE7.4020100@meyerloewen.net> <2f8bd9a60910090210s6dbead18k10abfaecf280dd8d@mail.gmail.com> Message-ID: <4ACF3718.6060201@meyerloewen.net> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Gediminas Paulauskas wrote: > For backwards compatibility already installed packages have to be > treated as manually installed, because it is not known why they were > installed. So the presence of a new file or flag has to signify that > it was AUTO_INSTALLED, i.e. the other way around than originally > suggested. I don't believe this is actually an issue. The PEP 376 installation format is markedly different from any existing installation format, so any tool dealing with installed packages will already have to know whether it's dealing with a PEP 376 install or a legacy install. Only PEP 376 installs could be considered orphaned. I don't have a problem with AUTO_INSTALLED, but more people seem to have favored REQUESTED thus far. Carl -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.6 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iD8DBQFKzzcYFRcxmeyPUXIRApWuAKCHDSfd0rKwNWeZkQPy71wyz/fJFwCdFUw7 XPWhAA4c0r+SFMzuFmz2rKo= =tAZN -----END PGP SIGNATURE----- From carl at meyerloewen.net Fri Oct 9 15:16:02 2009 From: carl at meyerloewen.net (Carl Meyer) Date: Fri, 09 Oct 2009 09:16:02 -0400 Subject: [Distutils] tracking requested vs dependency installs in PEP 376 metadata In-Reply-To: <20091008195227.GA735@clingman.lan> References: <4ACE15C5.4030105@meyerloewen.net> <20091008191526.GD513@clingman.lan> <4ACE4078.70504@meyerloewen.net> <20091008195227.GA735@clingman.lan> Message-ID: <4ACF3792.9040904@meyerloewen.net> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Toshio Kuratomi wrote: > I would say REQUESTED due to my arguments for not recording > installed-as-package-dependency. REQUESTED is fine, but I don't understand how the arguments apply, given that I'm not proposing to record information like _which_ package it was a dependency of. The same single bit (literally) of information is tracked either way, it's just a question of whether the presence or absence of a file signifies that bit. Carl -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.6 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iD8DBQFKzzeSFRcxmeyPUXIRAo0iAJ9QdQzxVUhTBO+rEGgCKmFg9zH7EwCcCfPc lWgoZVc+Z+H2OcIw1V/wNLI= =k12Y -----END PGP SIGNATURE----- From carl at dirtcircle.com Fri Oct 9 15:21:29 2009 From: carl at dirtcircle.com (Carl Meyer) Date: Fri, 09 Oct 2009 09:21:29 -0400 Subject: [Distutils] tracking requested vs dependency installs in PEP 376 metadata In-Reply-To: <4ACF1B33.5070102@simplistix.co.uk> References: <4ACE15C5.4030105@meyerloewen.net> <4ACF1B33.5070102@simplistix.co.uk> Message-ID: <4ACF38D9.8090007@dirtcircle.com> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Chris Withers wrote: > Ian Bicking wrote: >> I can imagine adding a little information, basically a log of when and >> why and who installed the package. For instance: >> >> agent: pip 0.5 >> install-date: 2009-10-08T13:44:00UTC >> installed-for-user: False >> installed-for-package: OtherPackage==0.3 > > I think this is a great idea. Recording more information rather than > less and making sure it's kept up to date is a good idea here. > Otherwise, when tools come along later down the line that want to make > use of this information, they'll have to hack something themselves, and > that's what got setuptools into the mess it currently is in the first > place! The downside here is that it introduces one more wrinkle for installers to worry about handling correctly. There are strong use cases for the single bit "requested vs auto-installed"; nobody's yet presented use cases for the additional log info. The only thing that comes to my mind is UI niceties: being able to tell the user when, why, and by what agent a package was installed. I'm not aware of existing package managers that go that far; doesn't mean it's a bad idea. Carl -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.6 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iD8DBQFKzzjZ1j/fhc23WEARAvw+AKDInwCU+Zzr6NIxlSUeOX+KWB+O6wCg2BWU BVwtV1ndSSmJJcJmxKGu9kU= =XQ57 -----END PGP SIGNATURE----- From hanno at hannosch.eu Fri Oct 9 15:22:49 2009 From: hanno at hannosch.eu (Hanno Schlichting) Date: Fri, 9 Oct 2009 15:22:49 +0200 Subject: [Distutils] why would you ever need to specify setuptools as a dependency? In-Reply-To: <1099b90b0910090614m5bf1717cp60b401839b62f352@mail.gmail.com> References: <4ACF1956.5070903@simplistix.co.uk> <5cae42b20910090457p3f6e4a1bp361b240f974f1617@mail.gmail.com> <1099b90b0910090614m5bf1717cp60b401839b62f352@mail.gmail.com> Message-ID: <5cae42b20910090622r2adc51fdxbae66f7613b93495@mail.gmail.com> On Fri, Oct 9, 2009 at 3:14 PM, Jim Fulton wrote: > On Fri, Oct 9, 2009 at 7:57 AM, Hanno Schlichting wrote: >> On Fri, Oct 9, 2009 at 1:07 PM, Chris Withers wrote: >> I assume most packages Reinout uses (like all zope.* packages) use >> namespace packages. So they actually do depend during runtime on the >> pkg_resources module, which happens to be available from either the >> distribute ?or setuptools distribution. So one of them should be >> specified in install_requires. > > This functionality should be factored out and merged with pkgutil in > the standard library. Sure. That's what http://www.python.org/dev/peps/pep-0382 is about. But both Python 2.7 and 3.2 are a bit into the future, so it doesn't help with the current question. Hanno From alex.gronholm at nextday.fi Fri Oct 9 15:23:07 2009 From: alex.gronholm at nextday.fi (=?ISO-8859-1?Q?Alex_Gr=F6nholm?=) Date: Fri, 09 Oct 2009 16:23:07 +0300 Subject: [Distutils] tracking requested vs dependency installs in PEP 376 metadata In-Reply-To: <4ACF3792.9040904@meyerloewen.net> References: <4ACE15C5.4030105@meyerloewen.net> <20091008191526.GD513@clingman.lan> <4ACE4078.70504@meyerloewen.net> <20091008195227.GA735@clingman.lan> <4ACF3792.9040904@meyerloewen.net> Message-ID: <4ACF393B.80206@nextday.fi> Carl Meyer kirjoitti: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > Toshio Kuratomi wrote: > >> I would say REQUESTED due to my arguments for not recording >> installed-as-package-dependency. >> > > REQUESTED is fine, but I don't understand how the arguments apply, given > that I'm not proposing to record information like _which_ package it was > a dependency of. The same single bit (literally) of information is > tracked either way, it's just a question of whether the presence or > absence of a file signifies that bit. > And what if the package is a dependency for multiple packages? Let's say we have packages A and B which are installed separately, in that order, and both depend on C. C gets installed with information that it was required by A. Now if A is uninstalled, won't C also get uninstalled? > Carl > -----BEGIN PGP SIGNATURE----- > Version: GnuPG v1.4.6 (GNU/Linux) > Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ > > iD8DBQFKzzeSFRcxmeyPUXIRAo0iAJ9QdQzxVUhTBO+rEGgCKmFg9zH7EwCcCfPc > lWgoZVc+Z+H2OcIw1V/wNLI= > =k12Y > -----END PGP SIGNATURE----- > _______________________________________________ > Distutils-SIG maillist - Distutils-SIG at python.org > http://mail.python.org/mailman/listinfo/distutils-sig > From exarkun at twistedmatrix.com Fri Oct 9 15:24:50 2009 From: exarkun at twistedmatrix.com (exarkun at twistedmatrix.com) Date: Fri, 09 Oct 2009 13:24:50 -0000 Subject: [Distutils] dev versions In-Reply-To: References: <68F76A34-16CE-4313-B1CF-7219EF9F9269@zooko.com> Message-ID: <20091009132450.24460.211168824.divmod.xquotient.230@boson> On 12:25 pm, benji at zope.com wrote: >On Thu, Oct 8, 2009 at 6:40 PM, Zooko Wilcox-O'Hearn >wrote: >> >>What we do in the Tahoe-LAFS project is we don't count down to a >>future >>version, we only count up from a past version. ?This is also what >>Twisted >>does (no coincidence -- we probably got the idea from them). > >To clarify: that means that a beta for 2.0.0 might have a version of >1.6.3-r4321? I'm not sure how zooko does this for Tahoe, but with Twisted (with which we don't do "betas" but we do do "pre-releases") if we were to start getting ready for 2.0.0, then we would create a release branch and change the version in that release branch to 2.0.0pre1. This, of course, complicates the matter. :) I don't think anyone has considered how our pre-release version numbers sort compared to the rest of our version numbers. After 2.0.0 final is released, we merge the release branch back into trunk, changing the trunk version to 2.0.0-r54321. Jean-Paul From kiorky at cryptelium.net Fri Oct 9 15:32:55 2009 From: kiorky at cryptelium.net (kiorky) Date: Fri, 09 Oct 2009 15:32:55 +0200 Subject: [Distutils] the virtualenv-distribute mess In-Reply-To: <94bdd2610910090545p65a3b172l971fe01f11768a3f@mail.gmail.com> References: <319e029f0910060402m28ab35acnc9a52aefcdc6f04a@mail.gmail.com> <4ACB24AE.4000300@cryptelium.net> <4ACB2527.7010802@cryptelium.net> <319e029f0910060522l29499f46y2805ed0082d975c3@mail.gmail.com> <4ACB7D70.9060404@cryptelium.net> <4ACF1681.7050506@simplistix.co.uk> <4ACF269C.4010400@cryptelium.net> <94bdd2610910090508j6804628fy5b5d435cce0c7427@mail.gmail.com> <4ACF2FD6.4010504@cryptelium.net> <94bdd2610910090545p65a3b172l971fe01f11768a3f@mail.gmail.com> Message-ID: <4ACF3B87.1070702@cryptelium.net> Tarek Ziad? a ?crit : > On Fri, Oct 9, 2009 at 2:43 PM, kiorky wrote: >> Hi tarek, >> >> Tarek Ziad? a ?crit : >> >>> The *whole* point of Distribute 0.6.x is to be backward compatible, meaning >>> that if virtualenv switch to it, you will not even notice it. >> Living in my 0.6.x snail sandbox is not a solution. >> As it seems that Distribute 0.7 won't for a long time. >> >> "setuptools based" packages will be able to be installed via the distribute 0.6 >> branch but not compatible with "distribute based" stuff. Note that new things >> will eventually be packaged with the "new good way todo, aka with 0.7". There is >> a great risk that they can't live together aside. NOGO > > Why they can't ? As i understood all those readings, packages for 0.6 and 0.7 will be installable with the appropriate distribute version, thus side by side, but for me, they may be incompatibles together. > >> 0.7 packages wont be compatibles with setuptools installation/namespaces, so it >> will be impossible to install a lot of "setuptools based" packages aside with >> new stuff in with this way too. NOGO too. > > Why will it be impossible ? pep-0382 is not equivalent to setuptools's one for example. Can i have been certified i will not have breakages when trying to import a setuptools based namespace package from a 0.7 sharing the same namespace? > > [...] >> I appreciate what you folks are doing with the distribute sphere, i have not >> that much problems with it, but i do not support that it breaks very badly the >> retro compatibilty for all things already packaged today, today tomorrow or in >> one year. > > Again, you will be able to use 0.6 and 0.7 together. or 0.6 alone, or > 0.7 alone. > > Nothing will be broken in a distribution that uses 0.6. > > 0.6 stays maintained. As i said ealier, there will be incompatibilities at some point. And also, to use them together, what a hell. For package A i need 0.6 (hard requirement), for package B i need 0.7 (hard requirement), for C i need 0.6. C depend on A which depends on B. I also have no sort of control over the maintenance of those products, think that the authors are dead. So, i ll have to manually install B for A to fulfill its requirements then C will install. Deployments will be simple :) > > Tarek -- -- Cordialement, KiOrKY GPG Key FingerPrint: 0x1A1194B7681112AF -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 261 bytes Desc: OpenPGP digital signature URL: From alex.gronholm at nextday.fi Fri Oct 9 15:38:38 2009 From: alex.gronholm at nextday.fi (=?ISO-8859-1?Q?Alex_Gr=F6nholm?=) Date: Fri, 09 Oct 2009 16:38:38 +0300 Subject: [Distutils] the virtualenv-distribute mess In-Reply-To: <4ACF3B87.1070702@cryptelium.net> References: <319e029f0910060402m28ab35acnc9a52aefcdc6f04a@mail.gmail.com> <4ACB24AE.4000300@cryptelium.net> <4ACB2527.7010802@cryptelium.net> <319e029f0910060522l29499f46y2805ed0082d975c3@mail.gmail.com> <4ACB7D70.9060404@cryptelium.net> <4ACF1681.7050506@simplistix.co.uk> <4ACF269C.4010400@cryptelium.net> <94bdd2610910090508j6804628fy5b5d435cce0c7427@mail.gmail.com> <4ACF2FD6.4010504@cryptelium.net> <94bdd2610910090545p65a3b172l971fe01f11768a3f@mail.gmail.com> <4ACF3B87.1070702@cryptelium.net> Message-ID: <4ACF3CDE.8050604@nextday.fi> kiorky kirjoitti: > Tarek Ziad? a ?crit : > >> On Fri, Oct 9, 2009 at 2:43 PM, kiorky wrote: >> >>> Hi tarek, >>> >>> Tarek Ziad? a ?crit : >>> >>> >>>> The *whole* point of Distribute 0.6.x is to be backward compatible, meaning >>>> that if virtualenv switch to it, you will not even notice it. >>>> >>> Living in my 0.6.x snail sandbox is not a solution. >>> As it seems that Distribute 0.7 won't for a long time. >>> >>> "setuptools based" packages will be able to be installed via the distribute 0.6 >>> branch but not compatible with "distribute based" stuff. Note that new things >>> will eventually be packaged with the "new good way todo, aka with 0.7". There is >>> a great risk that they can't live together aside. NOGO >>> >> Why they can't ? >> > > As i understood all those readings, packages for 0.6 and 0.7 will be installable > with the appropriate distribute version, thus side by side, but for me, they may > be incompatibles together. > > >>> 0.7 packages wont be compatibles with setuptools installation/namespaces, so it >>> will be impossible to install a lot of "setuptools based" packages aside with >>> new stuff in with this way too. NOGO too. >>> >> Why will it be impossible ? >> > > pep-0382 is not equivalent to setuptools's one for example. Can i have been > certified i will not have breakages when trying to import a setuptools based > namespace package from a 0.7 sharing the same namespace? > > >> [...] >> >>> I appreciate what you folks are doing with the distribute sphere, i have not >>> that much problems with it, but i do not support that it breaks very badly the >>> retro compatibilty for all things already packaged today, today tomorrow or in >>> one year. >>> >> Again, you will be able to use 0.6 and 0.7 together. or 0.6 alone, or >> 0.7 alone. >> >> Nothing will be broken in a distribution that uses 0.6. >> >> 0.6 stays maintained. >> > > As i said ealier, there will be incompatibilities at some point. > And also, to use them together, what a hell. For package A i need 0.6 (hard > requirement), for package B i need 0.7 (hard requirement), for C i need 0.6. C > depend on A which depends on B. I also have no sort of control over the > maintenance of those products, think that the authors are dead. > So, i ll have to manually install B for A to fulfill its requirements then C > will install. Deployments will be simple :) > I fail to follow this logic. If you install C, then both 0.6 and 0.7 will be installed as auto-dependencies (in case we decide to rename it to distribute2 or something), or just 0.7 (if we decide to provide a fat package). > >> Tarek >> > > > ------------------------------------------------------------------------ > > _______________________________________________ > Distutils-SIG maillist - Distutils-SIG at python.org > http://mail.python.org/mailman/listinfo/distutils-sig > From ziade.tarek at gmail.com Fri Oct 9 15:43:31 2009 From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=) Date: Fri, 9 Oct 2009 15:43:31 +0200 Subject: [Distutils] the virtualenv-distribute mess In-Reply-To: <4ACF3B87.1070702@cryptelium.net> References: <4ACB2527.7010802@cryptelium.net> <319e029f0910060522l29499f46y2805ed0082d975c3@mail.gmail.com> <4ACB7D70.9060404@cryptelium.net> <4ACF1681.7050506@simplistix.co.uk> <4ACF269C.4010400@cryptelium.net> <94bdd2610910090508j6804628fy5b5d435cce0c7427@mail.gmail.com> <4ACF2FD6.4010504@cryptelium.net> <94bdd2610910090545p65a3b172l971fe01f11768a3f@mail.gmail.com> <4ACF3B87.1070702@cryptelium.net> Message-ID: <94bdd2610910090643t406390d0o59294906c4659e5d@mail.gmail.com> On Fri, Oct 9, 2009 at 3:32 PM, kiorky wrote: >> Why they can't ? > > As i understood all those readings, packages for 0.6 and 0.7 will be installable > with the appropriate distribute version, thus side by side, but for me, they may > be incompatibles together. They "may" be ? what do you mean ? do you have a specific example ? >> >> Why will it be impossible ? > > pep-0382 is not equivalent to setuptools's one for example. Can i have been > certified i will not have breakages when trying to import a setuptools based > namespace package from a 0.7 sharing the same namespace? I really doubt PEP 382 will break backward compatibility for that, you can't get rid of .pth file and the associated pgutil mechanism, from one python version to the other. If one day they are removed, this will happen at the minimum at 2.8 / 3.3, so in more than 2 years. >> Nothing will be broken in a distribution that uses 0.6. >> >> 0.6 stays maintained. > > As i said ealier, there will be incompatibilities at some point. No. > And also, to use them together, what a hell. For package A i need 0.6 (hard > requirement), for package B i need 0.7 (hard requirement), for C i need 0.6. C > depend on A which depends on B. I also have no sort of control over the > maintenance of those products, think that the authors are dead. > So, i ll have to manually install B for A to fulfill its requirements then C > will install. Deployments will be simple :) It seems that you make the false assumption that a system can't have 0.6 and 0.7 at the same time. The "setuptools" package does not exists in 0.7 for example they are mutualy distinct. Tarek From carl at dirtcircle.com Fri Oct 9 15:45:27 2009 From: carl at dirtcircle.com (Carl Meyer) Date: Fri, 09 Oct 2009 09:45:27 -0400 Subject: [Distutils] tracking requested vs dependency installs in PEP 376 metadata In-Reply-To: <20091008195227.GA735@clingman.lan> References: <4ACE15C5.4030105@meyerloewen.net> <20091008191526.GD513@clingman.lan> <4ACE4078.70504@meyerloewen.net> <20091008195227.GA735@clingman.lan> Message-ID: <4ACF3E77.8080806@dirtcircle.com> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Alex wrote: > REQUESTED is fine, but I don't understand how the arguments apply, given > > that I'm not proposing to record information like _which_ package it was > > a dependency of. The same single bit (literally) of information is > > tracked either way, it's just a question of whether the presence or > > absence of a file signifies that bit. > > > And what if the package is a dependency for multiple packages? > Let's say we have packages A and B which are installed separately, in > that order, and both depend on C. > C gets installed with information that it was required by A. Now if A is > uninstalled, won't C also get uninstalled? No. We are NOT talking about recording the full dependency graph in package metadata. It is, as has always been the case, up to an uninstall tool to calculate the depgraph based on actual installed packages at runtime. A package is "orphaned" iff it was not REQUESTED by the user AND it is no longer depended on by other installed packages. Carl -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.6 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iD8DBQFKzz531j/fhc23WEARAlPfAJ93YHsbpzp5kF7hR0f98aSMOae6MwCgrP52 RZs5FU3h0U0eUBN2vgqm2HA= =jyi8 -----END PGP SIGNATURE----- From exarkun at twistedmatrix.com Fri Oct 9 16:05:04 2009 From: exarkun at twistedmatrix.com (exarkun at twistedmatrix.com) Date: Fri, 09 Oct 2009 14:05:04 -0000 Subject: [Distutils] tracking requested vs dependency installs in PEP 376 metadata In-Reply-To: <4ACF3E77.8080806@dirtcircle.com> References: <4ACE15C5.4030105@meyerloewen.net> <20091008191526.GD513@clingman.lan> <4ACE4078.70504@meyerloewen.net> <20091008195227.GA735@clingman.lan> <4ACF3E77.8080806@dirtcircle.com> Message-ID: <20091009140504.24460.1012182661.divmod.xquotient.237@boson> On 01:45 pm, carl at dirtcircle.com wrote: >-----BEGIN PGP SIGNED MESSAGE----- >Hash: SHA1 > > >Alex wrote: >>REQUESTED is fine, but I don't understand how the arguments apply, >>given >> > that I'm not proposing to record information like _which_ package it >>was >> > a dependency of. The same single bit (literally) of information is >> > tracked either way, it's just a question of whether the presence or >> > absence of a file signifies that bit. >> > >>And what if the package is a dependency for multiple packages? >>Let's say we have packages A and B which are installed separately, in >>that order, and both depend on C. >>C gets installed with information that it was required by A. Now if A >>is >>uninstalled, won't C also get uninstalled? > >No. We are NOT talking about recording the full dependency graph in >package metadata. It is, as has always been the case, up to an >uninstall >tool to calculate the depgraph based on actual installed packages at >runtime. A package is "orphaned" iff it was not REQUESTED by the user >AND it is no longer depended on by other installed packages. By doing this, I think you're dooming any Python package uninstaller to be unpleasantly slow. Jean-Paul From carl at dirtcircle.com Fri Oct 9 16:13:13 2009 From: carl at dirtcircle.com (Carl Meyer) Date: Fri, 09 Oct 2009 10:13:13 -0400 Subject: [Distutils] tracking requested vs dependency installs in PEP 376 metadata In-Reply-To: <20091009140504.24460.1012182661.divmod.xquotient.237@boson> References: <4ACE15C5.4030105@meyerloewen.net> <20091008191526.GD513@clingman.lan> <4ACE4078.70504@meyerloewen.net> <20091008195227.GA735@clingman.lan> <4ACF3E77.8080806@dirtcircle.com> <20091009140504.24460.1012182661.divmod.xquotient.237@boson> Message-ID: <4ACF44F9.3040204@dirtcircle.com> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 exarkun at twistedmatrix.com wrote: > By doing this, I think you're dooming any Python package uninstaller to > be unpleasantly slow. The process of searching for orphaned packages may be relatively slow on a system with many installed packages. I'm not sure this is a serious problem. I don't assume that an uninstaller implementation would (or should) automatically search for orphaned packages any time it takes any action. I also don't assume that tools could not maintain their own persistent caches of depgraph information to speed up certain operations. I do think there is some sense in leaving these choices to tool creators and keeping the metadata standard simple. If you'd like to propose that package installers should update the metadata of every dependency of an installed or removed package, so that each package always contains a full and reliable record of what other packages on the system depend on it, that would be interesting. But it is not this proposal. Carl -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.6 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iD8DBQFKz0T51j/fhc23WEARAvyHAKCncp3D1BMBzDXKAmV8SIQqAdqkAgCgvIYT 3Du1wO9sftj5nAKHEgpnf3I= =gjq3 -----END PGP SIGNATURE----- From chris at simplistix.co.uk Fri Oct 9 16:18:52 2009 From: chris at simplistix.co.uk (Chris Withers) Date: Fri, 09 Oct 2009 15:18:52 +0100 Subject: [Distutils] PyPM - no longer interesting :-( In-Reply-To: <5cae42b20910090510v74da6328s9644f52b1eee385e@mail.gmail.com> References: <4ACF1898.5020902@simplistix.co.uk> <5cae42b20910090510v74da6328s9644f52b1eee385e@mail.gmail.com> Message-ID: <4ACF464C.1090406@simplistix.co.uk> Hanno Schlichting wrote: > work on the UCS2/UCS4 build problem in PyPM. As a result the following > has been noted (http://workspace.activestate.com/sridharr/pypm/ticket/83): > > "Jan further commented that we should not be bothering to make PyPM > work with custom builds of Python other than ActivePython (which is > UCS2 on all platforms) as PyPM is legally supposed to be used only > with ActivePython. > > I originally noted the same .. but later thought about users hosting > custom repositories and using custom Python versions (/usr/bin/python > on Ubuntu, for instance). But if they are not legally allowed to use > anything other ActivePython, why bother?" > > Which should answer your question about "available separately". Indeed. It also quite effectively terminates any interest I have in PyPM whatsoever. What a shame :-( Chris -- Simplistix - Content Management, Batch Processing & Python Consulting - http://www.simplistix.co.uk From chris at simplistix.co.uk Fri Oct 9 16:21:17 2009 From: chris at simplistix.co.uk (Chris Withers) Date: Fri, 09 Oct 2009 15:21:17 +0100 Subject: [Distutils] the virtualenv-distribute mess In-Reply-To: <94bdd2610910090454y6dffc360q46ed05b4b759064f@mail.gmail.com> References: <4AC8A396.8030306@cryptelium.net> <4ACB1B5A.2090608@simplistix.co.uk> <319e029f0910060402n306a992as4b9d854f7f29d546@mail.gmail.com> <319e029f0910060402m28ab35acnc9a52aefcdc6f04a@mail.gmail.com> <4ACB24AE.4000300@cryptelium.net> <4ACB2527.7010802@cryptelium.net> <319e029f0910060522l29499f46y2805ed0082d975c3@mail.gmail.com> <4ACB7D70.9060404@cryptelium.net> <4ACF1681.7050506@simplistix.co.uk> <94bdd2610910090454y6dffc360q46ed05b4b759064f@mail.gmail.com> Message-ID: <4ACF46DD.6040703@simplistix.co.uk> Tarek Ziad? wrote: > I think the word "fork" here, in DVCS principles, just means that you > copy a repository > to work with, and eventually ask for the main repo to merge the changes. So what's the main repo? What one of these three options should someone looking to use virtualenv-distribute take? Chris -- Simplistix - Content Management, Batch Processing & Python Consulting - http://www.simplistix.co.uk From chris at simplistix.co.uk Fri Oct 9 16:23:33 2009 From: chris at simplistix.co.uk (Chris Withers) Date: Fri, 09 Oct 2009 15:23:33 +0100 Subject: [Distutils] the virtualenv-distribute mess In-Reply-To: <4ACF2FC5.6090706@cryptelium.net> References: <319e029f0910060402n306a992as4b9d854f7f29d546@mail.gmail.com> <319e029f0910060402m28ab35acnc9a52aefcdc6f04a@mail.gmail.com> <4ACB24AE.4000300@cryptelium.net> <4ACB2527.7010802@cryptelium.net> <319e029f0910060522l29499f46y2805ed0082d975c3@mail.gmail.com> <4ACB7D70.9060404@cryptelium.net> <4ACF1681.7050506@simplistix.co.uk> <4ACF269C.4010400@cryptelium.net> <94bdd2610910090508j6804628fy5b5d435cce0c7427@mail.gmail.com> <319e029f0910090536j480b2339t530a2f6a9267d451@mail.gmail.com> <4ACF2FC5.6090706@cryptelium.net> Message-ID: <4ACF4765.8020404@simplistix.co.uk> kiorky wrote: > Lennart Regebro a ?crit : >> 2009/10/9 Tarek Ziad? : >>> The *whole* point of Distribute 0.6.x is to be backward compatible, meaning >>> that if virtualenv switch to it, you will not even notice it. >> I guess the point is that is should still work even if you don't want >> to switch to distribute? > > Exact, i want that distribute usage must be explicit, moreover nowadays. Huh? Why? I just don't buy this. We've effectively had a bdfl pronouncement that setuptools is dead. distribute 0.6 *is* now the "live supported setuptools" so any project that relies on the package called "setuptools" should now be using the "distribute" distribution... Chris -- Simplistix - Content Management, Batch Processing & Python Consulting - http://www.simplistix.co.uk From chris at simplistix.co.uk Fri Oct 9 16:28:57 2009 From: chris at simplistix.co.uk (Chris Withers) Date: Fri, 09 Oct 2009 15:28:57 +0100 Subject: [Distutils] why would you ever need to specify setuptools as a dependency? In-Reply-To: <5cae42b20910090457p3f6e4a1bp361b240f974f1617@mail.gmail.com> References: <4ACF1956.5070903@simplistix.co.uk> <5cae42b20910090457p3f6e4a1bp361b240f974f1617@mail.gmail.com> Message-ID: <4ACF48A9.8060403@simplistix.co.uk> Hanno Schlichting wrote: > I assume most packages Reinout uses (like all zope.* packages) use > namespace packages. So they actually do depend during runtime on the > pkg_resources module, which happens to be available from either the > distribute or setuptools distribution. So one of them should be > specified in install_requires. You're missing my point. If you're using buildout, you automatically have setuptools available. If you're using easy_install, you automatically have setuptools available. So, the only case we can even consider is when you're installing a setuptools-based package with: python setup.py install In this case, which I suspect is extremely rare anyway, you'll need to have setuptools installed already. So, in *any* of these cases, specifying setuptools as a requirement seems like a total waste of time... Now, what case have I missed? ;-) Chris -- Simplistix - Content Management, Batch Processing & Python Consulting - http://www.simplistix.co.uk From chris at simplistix.co.uk Fri Oct 9 16:29:54 2009 From: chris at simplistix.co.uk (Chris Withers) Date: Fri, 09 Oct 2009 15:29:54 +0100 Subject: [Distutils] tracking requested vs dependency installs in PEP 376 metadata In-Reply-To: <4ACF38D9.8090007@dirtcircle.com> References: <4ACE15C5.4030105@meyerloewen.net> <4ACF1B33.5070102@simplistix.co.uk> <4ACF38D9.8090007@dirtcircle.com> Message-ID: <4ACF48E2.6090909@simplistix.co.uk> Carl Meyer wrote: > The downside here is that it introduces one more wrinkle for installers > to worry about handling correctly. How so? Chris -- Simplistix - Content Management, Batch Processing & Python Consulting - http://www.simplistix.co.uk From chris at simplistix.co.uk Fri Oct 9 16:34:11 2009 From: chris at simplistix.co.uk (Chris Withers) Date: Fri, 09 Oct 2009 15:34:11 +0100 Subject: [Distutils] Packaging Distribute In-Reply-To: <94bdd2610910090452l52e0cad5p4c11ab9989fc8b4b@mail.gmail.com> References: <90bb445a0910040810g48f7833cia02536e50e5c19f9@mail.gmail.com> <94bdd2610910041341v2c2483b1jd6c8f4b5707d789a@mail.gmail.com> <200910082307.13513.Arfrever.FTA@gmail.com> <20091008220337.GC1240@clingman.lan> <94bdd2610910081527s70ed6dd7g4e8e2cd157939c96@mail.gmail.com> <4ACF1BD3.5010402@simplistix.co.uk> <94bdd2610910090452l52e0cad5p4c11ab9989fc8b4b@mail.gmail.com> Message-ID: <4ACF49E3.407@simplistix.co.uk> Tarek Ziad? wrote: > Sorry I won't run a new poll again, Distribute is the name. > > Besides, it's already on page #1 on google when I type 'python > disribute' or 'distribute python' *shrugs* I will state now that I will fight tooth and nail before anything called "distribute" gets into the stdlib that has nothing to do with distribution. That said, the sooner bits and pieces (version comparision, standard metadata, building of eggs, installation and removal of packages from a file system path) make it into the stdlib, the better. cheers, Chris -- Simplistix - Content Management, Batch Processing & Python Consulting - http://www.simplistix.co.uk From carl at dirtcircle.com Fri Oct 9 16:36:21 2009 From: carl at dirtcircle.com (Carl Meyer) Date: Fri, 09 Oct 2009 10:36:21 -0400 Subject: [Distutils] tracking requested vs dependency installs in PEP 376 metadata In-Reply-To: <4ACF48E2.6090909@simplistix.co.uk> References: <4ACE15C5.4030105@meyerloewen.net> <4ACF1B33.5070102@simplistix.co.uk> <4ACF38D9.8090007@dirtcircle.com> <4ACF48E2.6090909@simplistix.co.uk> Message-ID: <4ACF4A65.2080309@dirtcircle.com> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Chris Withers wrote: > Carl Meyer wrote: >> The downside here is that it introduces one more wrinkle for installers >> to worry about handling correctly. > > How so? Write some more metadata, figure out whether to write it to already-installed packages, etc. Not saying it's a hard problem, just one more thing to do. I already said I'm not opposed to it in theory, but (as I also said) haven't seen strong use cases for it. Mostly I don't want the (slightly) better to be the enemy of the good; the REQUESTED bit is the critical part we don't have now. Carl -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.6 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iD8DBQFKz0pl1j/fhc23WEARAoEXAKDUsgEMOILvr3dvFMdc6PiN6MNqGgCbBvyb ttaA1Z3zbj7nIsC8FqOFf6c= =4ynG -----END PGP SIGNATURE----- From chris at simplistix.co.uk Fri Oct 9 16:37:43 2009 From: chris at simplistix.co.uk (Chris Withers) Date: Fri, 09 Oct 2009 15:37:43 +0100 Subject: [Distutils] dev versions In-Reply-To: <20091009132450.24460.211168824.divmod.xquotient.230@boson> References: <68F76A34-16CE-4313-B1CF-7219EF9F9269@zooko.com> <20091009132450.24460.211168824.divmod.xquotient.230@boson> Message-ID: <4ACF4AB7.7060101@simplistix.co.uk> exarkun at twistedmatrix.com wrote: > I'm not sure how zooko does this for Tahoe, but with Twisted (with which > we don't do "betas" but we do do "pre-releases") if we were to start > getting ready for 2.0.0, then we would create a release branch and > change the version in that release branch to 2.0.0pre1. This, of > course, complicates the matter. :) I don't think anyone has considered > how our pre-release version numbers sort compared to the rest of our > version numbers. After 2.0.0 final is released, we merge the release > branch back into trunk, changing the trunk version to 2.0.0-r54321. I beg you to make sure that whatever ends up in the python core to do with sorting and comparing version numbers takes your use case into account. I like the way you do things :-) (rare I say that about Twisted ;-) ) I'm -1 on "0" or any other special-case singleton version number... I'm +1 on the branch having a version of 1.6.3~dev after 1.6.3 has been released, and I like 2.0.0pre1 too :-) Moreover, I wish we could just get a bdfl pronouncement on something that works for everyone and get on with it :-D Chris -- Simplistix - Content Management, Batch Processing & Python Consulting - http://www.simplistix.co.uk From ziade.tarek at gmail.com Fri Oct 9 16:37:32 2009 From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=) Date: Fri, 9 Oct 2009 16:37:32 +0200 Subject: [Distutils] Packaging Distribute In-Reply-To: <4ACF49E3.407@simplistix.co.uk> References: <90bb445a0910040810g48f7833cia02536e50e5c19f9@mail.gmail.com> <94bdd2610910041341v2c2483b1jd6c8f4b5707d789a@mail.gmail.com> <200910082307.13513.Arfrever.FTA@gmail.com> <20091008220337.GC1240@clingman.lan> <94bdd2610910081527s70ed6dd7g4e8e2cd157939c96@mail.gmail.com> <4ACF1BD3.5010402@simplistix.co.uk> <94bdd2610910090452l52e0cad5p4c11ab9989fc8b4b@mail.gmail.com> <4ACF49E3.407@simplistix.co.uk> Message-ID: <94bdd2610910090737r7c526f31ic4b101a2448d53ea@mail.gmail.com> On Fri, Oct 9, 2009 at 4:34 PM, Chris Withers wrote: > Tarek Ziad? wrote: >> >> Sorry I won't run a new poll again, Distribute is the name. >> >> Besides, it's already on page #1 on google when I type 'python >> disribute' or 'distribute python' > > *shrugs* > > I will state now that I will fight tooth and nail before anything called > "distribute" gets into the stdlib that has nothing to do with distribution. Don't worry then. "distribute" will never get into the stdlib. What is likely to happen is that some bits it contains will make it in Distutils or pkgutil, etc. Tarek From kiorky at cryptelium.net Fri Oct 9 16:38:48 2009 From: kiorky at cryptelium.net (kiorky) Date: Fri, 09 Oct 2009 16:38:48 +0200 Subject: [Distutils] the virtualenv-distribute mess In-Reply-To: <4ACF46DD.6040703@simplistix.co.uk> References: <4AC8A396.8030306@cryptelium.net> <4ACB1B5A.2090608@simplistix.co.uk> <319e029f0910060402n306a992as4b9d854f7f29d546@mail.gmail.com> <319e029f0910060402m28ab35acnc9a52aefcdc6f04a@mail.gmail.com> <4ACB24AE.4000300@cryptelium.net> <4ACB2527.7010802@cryptelium.net> <319e029f0910060522l29499f46y2805ed0082d975c3@mail.gmail.com> <4ACB7D70.9060404@cryptelium.net> <4ACF1681.7050506@simplistix.co.uk> <94bdd2610910090454y6dffc360q46ed05b4b759064f@mail.gmail.com> <4ACF46DD.6040703@simplistix.co.uk> Message-ID: <4ACF4AF8.3010406@cryptelium.net> Chris Withers a ?crit : > Tarek Ziad? wrote: >> I think the word "fork" here, in DVCS principles, just means that you >> copy a repository >> to work with, and eventually ask for the main repo to merge the changes. > > So what's the main repo? > > What one of these three options should someone looking to use Do i said something like *homepage* metadata? > virtualenv-distribute take? > > Chris > -- -- Cordialement, KiOrKY GPG Key FingerPrint: 0x1A1194B7681112AF -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 261 bytes Desc: OpenPGP digital signature URL: From a.badger at gmail.com Fri Oct 9 16:39:30 2009 From: a.badger at gmail.com (Toshio Kuratomi) Date: Fri, 9 Oct 2009 07:39:30 -0700 Subject: [Distutils] tracking requested vs dependency installs in PEP 376 metadata In-Reply-To: <4ACF38D9.8090007@dirtcircle.com> References: <4ACE15C5.4030105@meyerloewen.net> <4ACF1B33.5070102@simplistix.co.uk> <4ACF38D9.8090007@dirtcircle.com> Message-ID: <20091009143930.GD1240@clingman.lan> On Fri, Oct 09, 2009 at 09:21:29AM -0400, Carl Meyer wrote: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > Chris Withers wrote: > The downside here is that it introduces one more wrinkle for installers > to worry about handling correctly. There are strong use cases for the > single bit "requested vs auto-installed"; nobody's yet presented use > cases for the additional log info. The only thing that comes to my mind > is UI niceties: being able to tell the user when, why, and by what agent > a package was installed. I'm not aware of existing package managers that > go that far; doesn't mean it's a bad idea. > rpm (one of the Linux package managers) tracks when a package was installed and when it was built. -Toshio -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 197 bytes Desc: not available URL: From strawman at astraw.com Fri Oct 9 16:54:16 2009 From: strawman at astraw.com (Andrew Straw) Date: Fri, 09 Oct 2009 07:54:16 -0700 Subject: [Distutils] dev versions In-Reply-To: <4ACF4AB7.7060101@simplistix.co.uk> References: <68F76A34-16CE-4313-B1CF-7219EF9F9269@zooko.com> <20091009132450.24460.211168824.divmod.xquotient.230@boson> <4ACF4AB7.7060101@simplistix.co.uk> Message-ID: <4ACF4E98.6030303@astraw.com> Chris Withers wrote: > I'm +1 on the branch having a version of 1.6.3~dev after 1.6.3 has > been released, and I like 2.0.0pre1 too :-) I'm -1 on "~" meaning afterwards, because in Debian package versions it means the exact opposite. -Andrew From a.badger at gmail.com Fri Oct 9 16:56:17 2009 From: a.badger at gmail.com (Toshio Kuratomi) Date: Fri, 9 Oct 2009 07:56:17 -0700 Subject: [Distutils] why would you ever need to specify setuptools as a dependency? In-Reply-To: <4ACF48A9.8060403@simplistix.co.uk> References: <4ACF1956.5070903@simplistix.co.uk> <5cae42b20910090457p3f6e4a1bp361b240f974f1617@mail.gmail.com> <4ACF48A9.8060403@simplistix.co.uk> Message-ID: <20091009145617.GE1240@clingman.lan> On Fri, Oct 09, 2009 at 03:28:57PM +0100, Chris Withers wrote: > > In this case, which I suspect is extremely rare anyway, you'll need to > have setuptools installed already. > > So, in *any* of these cases, specifying setuptools as a requirement > seems like a total waste of time... > > Now, what case have I missed? ;-) > It's nice for people creating system packages when you specify all of the packages that your runtime depends on in setup.py. That allows system packagers to read setup.py and be able to create the complete list of runtime dependencies for their packaging metadata. Several times I've been asked for help debugging why a python package fails to work for a few people only to discover that the package used entry-points or another setuptools runtime feature but only required it for buildtime. Note, however, that overspeciying the *versions* you need has the opposite effect :-) -Toshio -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 197 bytes Desc: not available URL: From chris at simplistix.co.uk Fri Oct 9 17:02:30 2009 From: chris at simplistix.co.uk (Chris Withers) Date: Fri, 09 Oct 2009 16:02:30 +0100 Subject: [Distutils] dev versions In-Reply-To: <4ACF4E98.6030303@astraw.com> References: <68F76A34-16CE-4313-B1CF-7219EF9F9269@zooko.com> <20091009132450.24460.211168824.divmod.xquotient.230@boson> <4ACF4AB7.7060101@simplistix.co.uk> <4ACF4E98.6030303@astraw.com> Message-ID: <4ACF5086.4060804@simplistix.co.uk> Andrew Straw wrote: > Chris Withers wrote: >> I'm +1 on the branch having a version of 1.6.3~dev after 1.6.3 has >> been released, and I like 2.0.0pre1 too :-) > I'm -1 on "~" meaning afterwards, because in Debian package versions it > means the exact opposite. I'm neutral on the exact spelling, I just like the idea :-) Chris -- Simplistix - Content Management, Batch Processing & Python Consulting - http://www.simplistix.co.uk From kiorky at cryptelium.net Fri Oct 9 17:03:03 2009 From: kiorky at cryptelium.net (kiorky) Date: Fri, 09 Oct 2009 17:03:03 +0200 Subject: [Distutils] the virtualenv-distribute mess In-Reply-To: <94bdd2610910090643t406390d0o59294906c4659e5d@mail.gmail.com> References: <4ACB2527.7010802@cryptelium.net> <319e029f0910060522l29499f46y2805ed0082d975c3@mail.gmail.com> <4ACB7D70.9060404@cryptelium.net> <4ACF1681.7050506@simplistix.co.uk> <4ACF269C.4010400@cryptelium.net> <94bdd2610910090508j6804628fy5b5d435cce0c7427@mail.gmail.com> <4ACF2FD6.4010504@cryptelium.net> <94bdd2610910090545p65a3b172l971fe01f11768a3f@mail.gmail.com> <4ACF3B87.1070702@cryptelium.net> <94bdd2610910090643t406390d0o59294906c4659e5d@mail.gmail.com> Message-ID: <4ACF50A7.5010605@cryptelium.net> Tarek Ziad? a ?crit : > On Fri, Oct 9, 2009 at 3:32 PM, kiorky wrote: >> And also, to use them together, what a hell. For package A i need 0.6 (hard >> requirement), for package B i need 0.7 (hard requirement), for C i need 0.6. C >> depend on A which depends on B. I also have no sort of control over the >> maintenance of those products, think that the authors are dead. >> So, i ll have to manually install B for A to fulfill its requirements then C >> will install. Deployments will be simple :) > > It seems that you make the false assumption that a system can't have 0.6 and 0.7 No, this is not the case. > at the same time. The "setuptools" package does not exists in 0.7 for example > they are mutualy distinct. This may be quite current even if it's not a good habit to have circular dependencies between distributions. Imagine that. B(0.7) -> A(0.6). A(0.6) -> B(0.7). Can i have the same namespace "ns" shared between the twice distributions with both the setuptools namespaces implementation (A) and the pkg_util's one (B)? "Have" mean that i can import ns in both distributions. So, if: * I have old distributions with C code even not declaring they are relying on setuptools, installing with the 0.6 code automatically. * I have entry points and namespaces from 0.7 available to import in 0.6 and vice-versa. I will see no more objections. Another related thing, as i read the pep376 implementation, it may be good and easy to provide some wrappers to some setuptools very used objects like WorkingSet or Environment as similary code is already implemented to smoothly migrate existing code. > > Tarek -- -- Cordialement, KiOrKY GPG Key FingerPrint: 0x1A1194B7681112AF -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 261 bytes Desc: OpenPGP digital signature URL: From chris at simplistix.co.uk Fri Oct 9 17:04:06 2009 From: chris at simplistix.co.uk (Chris Withers) Date: Fri, 09 Oct 2009 16:04:06 +0100 Subject: [Distutils] why would you ever need to specify setuptools as a dependency? In-Reply-To: <20091009145617.GE1240@clingman.lan> References: <4ACF1956.5070903@simplistix.co.uk> <5cae42b20910090457p3f6e4a1bp361b240f974f1617@mail.gmail.com> <4ACF48A9.8060403@simplistix.co.uk> <20091009145617.GE1240@clingman.lan> Message-ID: <4ACF50E6.7030703@simplistix.co.uk> Toshio Kuratomi wrote: > On Fri, Oct 09, 2009 at 03:28:57PM +0100, Chris Withers wrote: >> In this case, which I suspect is extremely rare anyway, you'll need to >> have setuptools installed already. >> >> So, in *any* of these cases, specifying setuptools as a requirement >> seems like a total waste of time... >> >> Now, what case have I missed? ;-) >> > It's nice for people creating system packages when you specify all of the > packages that your runtime depends on in setup.py. ...except that it causes problems that are a bit more serious than "nice to have" because of the ridiculous situation we're in with setuptools and distribute... Chris -- Simplistix - Content Management, Batch Processing & Python Consulting - http://www.simplistix.co.uk From zooko at zooko.com Fri Oct 9 17:04:21 2009 From: zooko at zooko.com (Zooko Wilcox-O'Hearn) Date: Fri, 9 Oct 2009 09:04:21 -0600 Subject: [Distutils] dev versions In-Reply-To: References: <68F76A34-16CE-4313-B1CF-7219EF9F9269@zooko.com> Message-ID: <658474D7-E05F-4C3B-A73D-25F9E3AF6B18@zooko.com> On Friday,2009-10-09, at 6:25 , Benji York wrote: > On Thu, Oct 8, 2009 at 6:40 PM, Zooko Wilcox-O'Hearn > wrote: >> >> What we do in the Tahoe-LAFS project is we don't count down to a >> future >> version, we only count up from a past version. This is also what >> Twisted >> does (no coincidence -- we probably got the idea from them). > > To clarify: that means that a beta for 2.0.0 might have a version of > 1.6.3-r4321? Yes, that's right. We haven't done this yet, but I suppose we could announce "the beta for 2.0.0" and give people 1.6.3-r4321 saying "This is the beta for 2.0.0.". What we've done so far is a little less formal. We say "Hey there are no more tickets open for 2.0.0 (see the issue tracker), so please everyone test the current version (1.6.3-r4321), and in about a week if there are no important new bugs discovered then we'll bump the version number to 2.0.0 and release it!". This is much like a beta but the word "beta" doesn't appear in the announcement. Regards, Zooko From chris at simplistix.co.uk Fri Oct 9 17:08:49 2009 From: chris at simplistix.co.uk (Chris Withers) Date: Fri, 09 Oct 2009 16:08:49 +0100 Subject: [Distutils] [buildout] hard coded urls considered harmful Message-ID: <4ACF5201.4090302@simplistix.co.uk> Hi All, bootstrap.py contains the following hard coded url: exec urllib2.urlopen('http://peak.telecommunity.com/dist/ez_setup.py' ).read() in ez With hindsight, this seems like a bad idea. For example, with the ridiculous situation we currently have with setuptools and distribute. It also means that the above url can't ever change, which is a bit of a harsh requirement. My suggestion would be for bootstrap.py to include the code in ez_setup.py, but that seems a little heavyweight. I guess this will all be solved when the standard library includes modules to download packages from PyPI and install them. But what to do in the meantime? cheers, Chris -- Simplistix - Content Management, Batch Processing & Python Consulting - http://www.simplistix.co.uk From sridharr at activestate.com Fri Oct 9 17:27:16 2009 From: sridharr at activestate.com (Sridhar Ratnakumar) Date: Fri, 09 Oct 2009 08:27:16 -0700 Subject: [Distutils] PyPM In-Reply-To: <4ACF1898.5020902@simplistix.co.uk> References: <4ACF1898.5020902@simplistix.co.uk> Message-ID: On Fri, 09 Oct 2009 04:03:52 -0700, Chris Withers wrote: > Sridhar Ratnakumar wrote: >> This release includes a new packaging tool by activestate called Python >> Package Manager (PyPM). > > Is PyPM available separately? No, PyPM comes only with ActivePython (just like PPM does with ActivePerl). >> Here's a sample command line output:: >> $ pypm install lxml > > Where does this get lxml from? From the binary repository: $ pypm -vv install lxml [...] Downloading http://pypm.activestate.com/root/2.6/macosx/pool/l/lx/lxml-2.2.2_macosx_2.6_2.pypm > How can I control that? $site-packages/pypm/client/client.conf OR the -L option > Where does this put the package on disk? "PyPM follows the PEP 370 specification and installs packages by default in ~/.local on Unix and Mac and %APPDATA%\Python on Windows;" http://docs.activestate.com/activepython/2.6/pypm.html#installing-packages $ pypm files lxml [...] /Users/sridharr/.local/lib/python2.6/site-packages/lxml/objectify.so /Users/sridharr/.local/lib/python2.6/site-packages/lxml/pyclasslookup [...] > How can I have separate package requirements for different python > projects on the same box? Hmm, I don't understand this question. > How can I specify the version of lxml required? It is currently not possible (not exposed as command line argument), but should be implemented in subsequent releases. > How can I reproduce the environment required for a particular project? If by that you mean something like 'pip bundle', this is currently not available. Please open a ticket at http://workspace.activestate.com/sridharr/pypm/report/1 -srid From ziade.tarek at gmail.com Fri Oct 9 17:28:56 2009 From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=) Date: Fri, 9 Oct 2009 17:28:56 +0200 Subject: [Distutils] [buildout] hard coded urls considered harmful In-Reply-To: <4ACF5201.4090302@simplistix.co.uk> References: <4ACF5201.4090302@simplistix.co.uk> Message-ID: <94bdd2610910090828k70fef06eqae966f8839ecbf30@mail.gmail.com> On Fri, Oct 9, 2009 at 5:08 PM, Chris Withers wrote: > Hi All, > > bootstrap.py contains the following hard coded url: > > exec urllib2.urlopen('http://peak.telecommunity.com/dist/ez_setup.py' > ? ? ? ? ? ? ? ? ? ? ).read() in ez > > With hindsight, this seems like a bad idea. For example, with the ridiculous > situation we currently have with setuptools and distribute. It also means > that the above url can't ever change, which is a bit of a harsh requirement. > > My suggestion would be for bootstrap.py to include the code in ez_setup.py, > but that seems a little heavyweight. I guess this will all be solved when > the standard library includes modules to download packages from PyPI and > install them. But what to do in the meantime? zc.buildout uses setuptools. it's hardcoded and it's deeper than bootstrap.py. Changing this properly requires quite some work. We need to leave this alone and provide a distribute bootstrap.py that does some extra things to make zc.buildout work with it. A specific bootstrap.py script for distribute is possible, and I happen to have it finished now : $ wget http://ziade.org/bootstrap.py (this is working with the current zc.buildout trunk) Tarek From jim at zope.com Fri Oct 9 17:39:31 2009 From: jim at zope.com (Jim Fulton) Date: Fri, 9 Oct 2009 11:39:31 -0400 Subject: [Distutils] [buildout] hard coded urls considered harmful In-Reply-To: <4ACF5201.4090302@simplistix.co.uk> References: <4ACF5201.4090302@simplistix.co.uk> Message-ID: <1099b90b0910090839h753fe631saf5dfa85fae0e875@mail.gmail.com> On Fri, Oct 9, 2009 at 11:08 AM, Chris Withers wrote: > Hi All, > > bootstrap.py contains the following hard coded url: > > exec urllib2.urlopen('http://peak.telecommunity.com/dist/ez_setup.py' > ? ? ? ? ? ? ? ? ? ? ).read() in ez > > With hindsight, this seems like a bad idea. I don't see a good alternative. > For example, with the ridiculous > situation we currently have with setuptools and distribute. It also means > that the above url can't ever change, which is a bit of a harsh requirement. > > My suggestion would be for bootstrap.py to include the code in ez_setup.py, > but that seems a little heavyweight. ez_setup.py has a hard coded URL too. And using it's code wouldn't do anything for the situation with distribute. > I guess this will all be solved when > the standard library includes modules to download packages from PyPI and > install them. But what to do in the meantime? For myself, I'm going to stay out of Tarek's way and see what develops. :) Jim -- Jim Fulton From ianb at colorstudy.com Fri Oct 9 17:58:39 2009 From: ianb at colorstudy.com (Ian Bicking) Date: Fri, 9 Oct 2009 10:58:39 -0500 Subject: [Distutils] the virtualenv-distribute mess In-Reply-To: <4ACF1681.7050506@simplistix.co.uk> References: <4AC8A396.8030306@cryptelium.net> <4ACB1B5A.2090608@simplistix.co.uk> <319e029f0910060402n306a992as4b9d854f7f29d546@mail.gmail.com> <319e029f0910060402m28ab35acnc9a52aefcdc6f04a@mail.gmail.com> <4ACB24AE.4000300@cryptelium.net> <4ACB2527.7010802@cryptelium.net> <319e029f0910060522l29499f46y2805ed0082d975c3@mail.gmail.com> <4ACB7D70.9060404@cryptelium.net> <4ACF1681.7050506@simplistix.co.uk> Message-ID: On Fri, Oct 9, 2009 at 5:54 AM, Chris Withers wrote: > kiorky wrote: >> >> Hi Lennart, >> >> If i read 'virtualenv-distribute 1.3.4.2 on pypi' >> I can ?do some googling or even do some Pypi searching for >> ?'virtualenv-distribute'. >> >> Thus, the first link found may be [1]. >> >> On this link, the second sentence is: >> >> """ >> The fork was started by Philip Jenvey at >> http://bitbucket.org/pjenvey/virtualenv-distribute/ and this version by >> Florian >> Schulze lives at http://bitbucket.org/fschulze/virtualenv-distribute/ > > Add http://bitbucket.org/kiorky/virtualenv-distribute/ to the list and this, > ladies and gentlemen, is why distributed source control sucks. I think one (pjenvey) was an experiment, and another is actually a released virtualenv-distribute package (updating the name in setup.py, etc). And the third, I dunno. Anyway -- I'm reluctant to switch virtualenv to distribute right now, as I'm not confident it is ready for people to use distribute without even realizing it. Because if someone just upgrades virtualenv and starts using it, they'll get whatever virtualenv is distributed with. I'd be happy to include a --distribute option if someone wants to repackage one of these forks to include both setuptools and distribute support. -- Ian Bicking | http://blog.ianbicking.org | http://topplabs.org/civichacker From kiorky at cryptelium.net Fri Oct 9 18:04:29 2009 From: kiorky at cryptelium.net (kiorky) Date: Fri, 09 Oct 2009 18:04:29 +0200 Subject: [Distutils] the virtualenv-distribute mess In-Reply-To: References: <4AC8A396.8030306@cryptelium.net> <4ACB1B5A.2090608@simplistix.co.uk> <319e029f0910060402n306a992as4b9d854f7f29d546@mail.gmail.com> <319e029f0910060402m28ab35acnc9a52aefcdc6f04a@mail.gmail.com> <4ACB24AE.4000300@cryptelium.net> <4ACB2527.7010802@cryptelium.net> <319e029f0910060522l29499f46y2805ed0082d975c3@mail.gmail.com> <4ACB7D70.9060404@cryptelium.net> <4ACF1681.7050506@simplistix.co.uk> Message-ID: <4ACF5F0D.9030805@cryptelium.net> Ian Bicking a ?crit : > On Fri, Oct 9, 2009 at 5:54 AM, Chris Withers wrote: >> kiorky wrote: > I think one (pjenvey) was an experiment, and another is actually a > released virtualenv-distribute package (updating the name in setup.py, > etc). And the third, I dunno. The third (mine) was a bump to distribute-current and a merge from current virtualenv basecode. > > Anyway -- I'm reluctant to switch virtualenv to distribute right now, > as I'm not confident it is ready for people to use distribute without > even realizing it. Because if someone just upgrades virtualenv and > starts using it, they'll get whatever virtualenv is distributed with. > I'd be happy to include a --distribute option if someone wants to > repackage one of these forks to include both setuptools and distribute > support. > I may not have sufficient time right now, but this feature maybe be quite easy to do. -- -- Cordialement, KiOrKY GPG Key FingerPrint: 0x1A1194B7681112AF -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 261 bytes Desc: OpenPGP digital signature URL: From a.badger at gmail.com Fri Oct 9 18:04:56 2009 From: a.badger at gmail.com (Toshio Kuratomi) Date: Fri, 9 Oct 2009 09:04:56 -0700 Subject: [Distutils] why would you ever need to specify setuptools as a dependency? In-Reply-To: <4ACF50E6.7030703@simplistix.co.uk> References: <4ACF1956.5070903@simplistix.co.uk> <5cae42b20910090457p3f6e4a1bp361b240f974f1617@mail.gmail.com> <4ACF48A9.8060403@simplistix.co.uk> <20091009145617.GE1240@clingman.lan> <4ACF50E6.7030703@simplistix.co.uk> Message-ID: <20091009160456.GF1240@clingman.lan> On Fri, Oct 09, 2009 at 04:04:06PM +0100, Chris Withers wrote: > Toshio Kuratomi wrote: >> On Fri, Oct 09, 2009 at 03:28:57PM +0100, Chris Withers wrote: >>> In this case, which I suspect is extremely rare anyway, you'll need >>> to have setuptools installed already. >>> >>> So, in *any* of these cases, specifying setuptools as a requirement >>> seems like a total waste of time... >>> >>> Now, what case have I missed? ;-) >>> >> It's nice for people creating system packages when you specify all of the >> packages that your runtime depends on in setup.py. > > ...except that it causes problems that are a bit more serious than "nice > to have" because of the ridiculous situation we're in with setuptools > and distribute... > What's the issue precisely? Once distribute is on the system, setuptools is provided by distribute so there's no problem there, correct? Is it that the installers don't know that there's more than one package providing the setuptools API? That sounds like pypi and easy_install aren't powerful enough to recognize that an API can be provided by multiple modules. If you actually want to have a full-blown package manager you'll need to fix that but at the same time I'd warn that having a full-blown package manager means having to deal with a lot of corner cases like this. -Toshio -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 197 bytes Desc: not available URL: From chris at simplistix.co.uk Fri Oct 9 18:10:47 2009 From: chris at simplistix.co.uk (Chris Withers) Date: Fri, 09 Oct 2009 17:10:47 +0100 Subject: [Distutils] the virtualenv-distribute mess In-Reply-To: References: <4AC8A396.8030306@cryptelium.net> <4ACB1B5A.2090608@simplistix.co.uk> <319e029f0910060402n306a992as4b9d854f7f29d546@mail.gmail.com> <319e029f0910060402m28ab35acnc9a52aefcdc6f04a@mail.gmail.com> <4ACB24AE.4000300@cryptelium.net> <4ACB2527.7010802@cryptelium.net> <319e029f0910060522l29499f46y2805ed0082d975c3@mail.gmail.com> <4ACB7D70.9060404@cryptelium.net> <4ACF1681.7050506@simplistix.co.uk> Message-ID: <4ACF6087.8050507@simplistix.co.uk> Ian Bicking wrote: > I think one (pjenvey) was an experiment, and another is actually a > released virtualenv-distribute package (updating the name in setup.py, > etc). And the third, I dunno. > > Anyway -- I'm reluctant to switch virtualenv to distribute right now, > as I'm not confident it is ready for people to use distribute without > even realizing it. How so? distribute 0.6 = setuptools 0.6 + bugfixes. How would sticking with a bug-riddled piece of code be better than moving to the same code with a few of the bugs fixed? Chris -- Simplistix - Content Management, Batch Processing & Python Consulting - http://www.simplistix.co.uk From chris at simplistix.co.uk Fri Oct 9 18:13:16 2009 From: chris at simplistix.co.uk (Chris Withers) Date: Fri, 09 Oct 2009 17:13:16 +0100 Subject: [Distutils] why would you ever need to specify setuptools as a dependency? In-Reply-To: <20091009160456.GF1240@clingman.lan> References: <4ACF1956.5070903@simplistix.co.uk> <5cae42b20910090457p3f6e4a1bp361b240f974f1617@mail.gmail.com> <4ACF48A9.8060403@simplistix.co.uk> <20091009145617.GE1240@clingman.lan> <4ACF50E6.7030703@simplistix.co.uk> <20091009160456.GF1240@clingman.lan> Message-ID: <4ACF611C.4070901@simplistix.co.uk> Toshio Kuratomi wrote: > On Fri, Oct 09, 2009 at 04:04:06PM +0100, Chris Withers wrote: >> Toshio Kuratomi wrote: >>> On Fri, Oct 09, 2009 at 03:28:57PM +0100, Chris Withers wrote: >>>> In this case, which I suspect is extremely rare anyway, you'll need >>>> to have setuptools installed already. >>>> >>>> So, in *any* of these cases, specifying setuptools as a requirement >>>> seems like a total waste of time... >>>> >>>> Now, what case have I missed? ;-) >>>> >>> It's nice for people creating system packages when you specify all of the >>> packages that your runtime depends on in setup.py. >> ...except that it causes problems that are a bit more serious than "nice >> to have" because of the ridiculous situation we're in with setuptools >> and distribute... >> > What's the issue precisely? Once distribute is on the system, setuptools is > provided by distribute so there's no problem there, correct? The issue is that both the setuptools and distribute distributions provide a setuptools package. This apparently causes problems, rather unsurprisingly ;-) Chris -- Simplistix - Content Management, Batch Processing & Python Consulting - http://www.simplistix.co.uk From chris at simplistix.co.uk Fri Oct 9 18:19:51 2009 From: chris at simplistix.co.uk (Chris Withers) Date: Fri, 09 Oct 2009 17:19:51 +0100 Subject: [Distutils] [buildout] hard coded urls considered harmful In-Reply-To: <94bdd2610910090828k70fef06eqae966f8839ecbf30@mail.gmail.com> References: <4ACF5201.4090302@simplistix.co.uk> <94bdd2610910090828k70fef06eqae966f8839ecbf30@mail.gmail.com> Message-ID: <4ACF62A7.7060807@simplistix.co.uk> Tarek Ziad? wrote: >> My suggestion would be for bootstrap.py to include the code in ez_setup.py, >> but that seems a little heavyweight. I guess this will all be solved when >> the standard library includes modules to download packages from PyPI and >> install them. But what to do in the meantime? > > zc.buildout uses setuptools. it's hardcoded and it's deeper than bootstrap.py. > Changing this properly requires quite some work. Not that much, as you already know, as I've already done the majority of this on a branch. Hopefully I'll get to look at this again soon, and the bugs I hit might well be solved now that Distribute is more mature... > We need to leave this alone and provide a distribute bootstrap.py > that does some extra things to make zc.buildout work with it. Hacking around like this seems sub optimal to me. > A specific bootstrap.py script for distribute is possible, and I > happen to have it finished now : > > $ wget http://ziade.org/bootstrap.py I'll certainly try this out, but I'm not using buildout trunk anywhere. Does it work with the current released version of buildout? Chris -- Simplistix - Content Management, Batch Processing & Python Consulting - http://www.simplistix.co.uk From chris at simplistix.co.uk Fri Oct 9 18:22:46 2009 From: chris at simplistix.co.uk (Chris Withers) Date: Fri, 09 Oct 2009 17:22:46 +0100 Subject: [Distutils] [buildout] hard coded urls considered harmful In-Reply-To: <1099b90b0910090839h753fe631saf5dfa85fae0e875@mail.gmail.com> References: <4ACF5201.4090302@simplistix.co.uk> <1099b90b0910090839h753fe631saf5dfa85fae0e875@mail.gmail.com> Message-ID: <4ACF6356.4020705@simplistix.co.uk> Jim Fulton wrote: > On Fri, Oct 9, 2009 at 11:08 AM, Chris Withers wrote: >> Hi All, >> >> bootstrap.py contains the following hard coded url: >> >> exec urllib2.urlopen('http://peak.telecommunity.com/dist/ez_setup.py' >> ).read() in ez >> >> With hindsight, this seems like a bad idea. > > I don't see a good alternative. At least having it overrideable on the command line and/or by an environment variable would be the way forward... >> My suggestion would be for bootstrap.py to include the code in ez_setup.py, >> but that seems a little heavyweight. > > ez_setup.py has a hard coded URL too. Indeed, but that uses the advertised public APIs of PyPI, which I have a little more faith in. But yes, this whole business of hardcoding download urls is brittle and annoying. What's needed to make it no longer necessary? stdlib support for downloading a package from an index, with PyPI as the default? Chris -- Simplistix - Content Management, Batch Processing & Python Consulting - http://www.simplistix.co.uk From ziade.tarek at gmail.com Fri Oct 9 18:23:57 2009 From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=) Date: Fri, 9 Oct 2009 18:23:57 +0200 Subject: [Distutils] [buildout] hard coded urls considered harmful In-Reply-To: <4ACF62A7.7060807@simplistix.co.uk> References: <4ACF5201.4090302@simplistix.co.uk> <94bdd2610910090828k70fef06eqae966f8839ecbf30@mail.gmail.com> <4ACF62A7.7060807@simplistix.co.uk> Message-ID: <94bdd2610910090923w7b92cb51se9d3d0a14e16aa4@mail.gmail.com> On Fri, Oct 9, 2009 at 6:19 PM, Chris Withers wrote: >> A specific bootstrap.py script for distribute is possible, and I >> happen to have it finished now : >> >> $ wget http://ziade.org/bootstrap.py > > I'll certainly try this out, but I'm not using buildout trunk anywhere. Does > it work with the current released version of buildout? No I need to release zc.buildout 1.4.2 for this, I have asked the other maintainers if they are OK. I'll do it this week-end together with the Distribute 0.6.4 release if no one objects Tarek From janssen at parc.com Fri Oct 9 18:30:53 2009 From: janssen at parc.com (Bill Janssen) Date: Fri, 9 Oct 2009 09:30:53 PDT Subject: [Distutils] how to easily consume just the parts of eggs that are good for you In-Reply-To: <42456.207.188.248.157.1207836698.squirrel@www.cpcug.org> References: <42456.207.188.248.157.1207836698.squirrel@www.cpcug.org> Message-ID: <29717.1255105853@parc.com> Stanley A. Klein wrote: > Windows and Mac are fundamentally single user systems that have added > capabilities for multiple users and are intended to be used with > proprietary software. Those considerations lead to minimal dependencies > among packages (each proprietary provider needs to control its own > package, except for the OS), individual users serving as their own > sysadmins, and similar factors. Any dependencies in the proprietary > software are hidden from the user because the provider has compiled the > dependencies into the binary code they supply. The problem with this analysis is that the modern Mac is also a capable Unix system, and the "just put everything in a bundle" philosophy that served the Mac so well for so long is beginning to fail. While many Mac apps do still follow this, many more (perhaps the majority) are now Unix apps that have complex inter-package dependencies. Bill From floris.bruynooghe at gmail.com Fri Oct 9 18:33:52 2009 From: floris.bruynooghe at gmail.com (Floris Bruynooghe) Date: Fri, 9 Oct 2009 17:33:52 +0100 Subject: [Distutils] dev versions In-Reply-To: <20091009132450.24460.211168824.divmod.xquotient.230@boson> References: <68F76A34-16CE-4313-B1CF-7219EF9F9269@zooko.com> <20091009132450.24460.211168824.divmod.xquotient.230@boson> Message-ID: <20091009163352.GA7385@laurie.devork> On Fri, Oct 09, 2009 at 01:24:50PM -0000, exarkun at twistedmatrix.com wrote: > On 12:25 pm, benji at zope.com wrote: > >On Thu, Oct 8, 2009 at 6:40 PM, Zooko Wilcox-O'Hearn > > wrote: > >> > >>What we do in the Tahoe-LAFS project is we don't count down to a > >>future > >>version, we only count up from a past version. ?This is also > >>what Twisted > >>does (no coincidence -- we probably got the idea from them). > > > >To clarify: that means that a beta for 2.0.0 might have a version of > >1.6.3-r4321? > > I'm not sure how zooko does this for Tahoe, but with Twisted (with > which we don't do "betas" but we do do "pre-releases") if we were to > start getting ready for 2.0.0, then we would create a release branch > and change the version in that release branch to 2.0.0pre1. This, > of course, complicates the matter. :) I don't think anyone has > considered how our pre-release version numbers sort compared to the > rest of our version numbers. After 2.0.0 final is released, we > merge the release branch back into trunk, changing the trunk version > to 2.0.0-r54321. The pre-release would become 2.0.0.dev1 in the proposed PEP386 spec I guess, however the suggest_rational_version() function suggests otherwise: >>> verlib.suggest_rational_version('2.0.0pre1') '2.0.0c1' It interprets your version string as a "release candidate" version, which it actually might be given that you gave it it a number of "1" and not some longer revision-control based one. Regards Floris -- Debian GNU/Linux -- The Power of Freedom www.debian.org | www.gnu.org | www.kernel.org From chris at simplistix.co.uk Fri Oct 9 18:34:56 2009 From: chris at simplistix.co.uk (Chris Withers) Date: Fri, 09 Oct 2009 17:34:56 +0100 Subject: [Distutils] [Python-Dev] excluding site-packages with buildout/virtualenv In-Reply-To: References: <94bdd2610910080131j323b98d9i871bce43465f237a@mail.gmail.com> <94bdd2610910081555h13c8b48cg8810377912633dc5@mail.gmail.com> <4ACE718D.8090400@cryptelium.net> <94bdd2610910081621n502803e4l8efeabae3c5863ff@mail.gmail.com> <4ACEFA3F.2020706@cryptelium.net> Message-ID: <4ACF6630.9070702@simplistix.co.uk> [moved to disutils-sig] Ian Bicking wrote: > Well, if multi-versioned installs were deprecated, it would not be > necessary to use Setuptools' style of script generation. Instead you > could simply dereference the entry point, calling the underlying > function directly in the script. This detail is probably more of a > distutils-sig question, and I don't have a strong opinion. > > But I was thinking specifically of the egg activation buildout puts at > the top of scripts. Not sure what you mean by this last bit... > I believe that to fully insulate buildout you need still virtualenv > --no-site-packages. buildout has a --no-site-packages option (hopefully also specifiable in the [buildout] section) and some other cool things that I believe Gary Post worked on lurking on a branch waiting for Jim to review... Chris -- Simplistix - Content Management, Batch Processing & Python Consulting - http://www.simplistix.co.uk From kiorky at cryptelium.net Fri Oct 9 18:39:11 2009 From: kiorky at cryptelium.net (kiorky) Date: Fri, 09 Oct 2009 18:39:11 +0200 Subject: [Distutils] [Python-Dev] Distutils and Distribute roadmap (and some words on Virtualenv, Pip) In-Reply-To: References: <94bdd2610910080131j323b98d9i871bce43465f237a@mail.gmail.com> <94bdd2610910081555h13c8b48cg8810377912633dc5@mail.gmail.com> <4ACE718D.8090400@cryptelium.net> <94bdd2610910081621n502803e4l8efeabae3c5863ff@mail.gmail.com> <4ACEFA3F.2020706@cryptelium.net> Message-ID: <4ACF672F.9070701@cryptelium.net> I'm crossposting to continue on distutils. Ian Bicking a ?crit : > On Fri, Oct 9, 2009 at 3:54 AM, kiorky wrote: > Well, if multi-versioned installs were deprecated, it would not be > necessary to use Setuptools' style of script generation. Instead you > could simply dereference the entry point, calling the underlying > function directly in the script. This detail is probably more of a > distutils-sig question, and I don't have a strong opinion. > > But I was thinking specifically of the egg activation buildout puts at > the top of scripts. The egg activation is just the result of the constructed working set, which aggregates all the python packages that fulfill the options inputed by the user. The only needed packages and nothing else or more. >>> began development, probably things would have gone this way. I think >>> it would make the environment more pleasant for buildout users. Also >> * I don't think so, buildout is the only tool atm that permit to have really >> reproducible and isolated environments. Even, if you use the pip freezing >> machinery, it is not equivalent to buildout, Control! > > I believe that to fully insulate buildout you need still virtualenv > --no-site-packages. But I'm not arguing that virtualenv/pip makes Right, I do so, but i also drop nothing in that virtualenv site-packages. > buildout obsolete, only that they have overlapping functionality, and > I think buildout would benefit from making use of that overlap. recipes have more features and are another way to do, complementary or not. (patching, compiling options, pinning version). You have also more control on what is installed/used/shared with buildout than with the others. >> * Buildout can have single part to construct required eggs, at a specific >> version and let you control that. Pip will just search for this version, see >> that it's not available and fail. You have even recipes (like >> minitage.recipe.egg that permit to construct eggs with special version when you >> apply patches onto, thus, you can have the same egg in different flavors in the >> same eggs cache available for different projects. Those projects will just have >> to pin the right version to use, Control!. > > In my own work I use multiple virtualenv environments for this use > case, to similar effect. pip of course is not a generalized build > tool, but then minitage.recipe.egg is not the main egg installer > either. *.recipe.egg can use caches to avoid duplications among many projects environments. You setup seems to duplicate stuff around. >> * Another thing is the "funny script generation", you have not one global >> site-packages for your project, but one global cache. But from this global >> cache, your scripts will only have available the eggs you declared, see Control! >> * Moreover buildout is not only a python packages manager, it's some of its >> recipes that permit to use it as. Buildout is just a great deployment tool that >> allow to script and manage your project in a "funny" and "flexible" way, Control! > > Sure; I'm just advocating that buildout more explicitly use some of > the functionality of virtualenv/pip (which may require some more > features in those tools, but I'm open to that). But specific > discussion of this would probably be more appropriate on > distutils-sig. > -- -- Cordialement, KiOrKY GPG Key FingerPrint: 0x1A1194B7681112AF -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 261 bytes Desc: OpenPGP digital signature URL: From ianb at colorstudy.com Fri Oct 9 19:02:07 2009 From: ianb at colorstudy.com (Ian Bicking) Date: Fri, 9 Oct 2009 12:02:07 -0500 Subject: [Distutils] [Python-Dev] Distutils and Distribute roadmap (and some words on Virtualenv, Pip) In-Reply-To: <4ACF6993.5040704@voidspace.org.uk> References: <94bdd2610910080131j323b98d9i871bce43465f237a@mail.gmail.com> <94bdd2610910081555h13c8b48cg8810377912633dc5@mail.gmail.com> <4ACE718D.8090400@cryptelium.net> <94bdd2610910081621n502803e4l8efeabae3c5863ff@mail.gmail.com> <79990c6b0910090532y63169b60q8dd0a95f056c244d@mail.gmail.com> <4ACF6993.5040704@voidspace.org.uk> Message-ID: Probably all these discussions are better on distutils-sig (just copying python-dev to note the movement of the discussion) On Fri, Oct 9, 2009 at 11:49 AM, Michael Foord wrote: >> Outside of binaries on Windows, I'm still unsure if installing eggs >> serves a useful purpose. ?I'm not sure if eggs are any better than >> wininst binaries either...? > > Many Windows users would be quite happy if the standard mechanism for > installing non-source distributions on Windows was via the wininst binaries. > > I wonder if it is going to be possible to make this compatible with the > upcoming distutils package management 'stuff' (querying for installed > packages, uninstallation etc) since installation/uninstallation goes through > the Windows system package management feature. ?I guess it would be > eminently possible but require some reasonably high level Windows-fu to do. As far as pip works, it unpacks a package and runs "python setup.py install (and some options that aren't that interesting, but are provided specifically by setuptools)". Well, it's slightly more complicated, but more to the point it doesn't install in-process or dictate how setup.py works, except that it takes some specific options. Running a Windows installer in the same way would be fine, in that sense. Alternately pip could unpack the wininst zip file and install it directly; I'm not sure if that would be better or worse? If wininst uses the central package manager of the OS then certain features (like virtualenv, PYTHONHOME, --prefix, etc) would not be possible. For Distribute (or Setuptools or by association pip) to see that a package is installed, it must have the appropriate metadata. For Setuptools (and Distribute 0.6) this is a directory or file, on sys.path, "Package.egg-info" (or in Package-X.Y.egg/EGG-INFO). If a file, it should be a PKG-INFO file, if a directory it should contain a PKG-INFO file. So however the package gets installed, if that metadata is installed then it can be queried. I don't think querying the Windows system package management would be necessary or desirable. Nobody is trying that with deb/rpm either. -- Ian Bicking | http://blog.ianbicking.org | http://topplabs.org/civichacker From floris.bruynooghe at gmail.com Fri Oct 9 19:34:59 2009 From: floris.bruynooghe at gmail.com (Floris Bruynooghe) Date: Fri, 9 Oct 2009 18:34:59 +0100 Subject: [Distutils] [Python-Dev] Bits-of-Distribute naming In-Reply-To: <94bdd2610910090905t6fcdc6d9rfe946bbfbd36a7b9@mail.gmail.com> References: <94bdd2610910080131j323b98d9i871bce43465f237a@mail.gmail.com> <4ACF59CC.5050504@simplistix.co.uk> <94bdd2610910090905t6fcdc6d9rfe946bbfbd36a7b9@mail.gmail.com> Message-ID: <20091009173459.GA8588@laurie.devork> On Fri, Oct 09, 2009 at 06:05:50PM +0200, Tarek Ziad? wrote: > On Fri, Oct 9, 2009 at 5:42 PM, Chris Withers wrote: > > Tarek Ziad? wrote: > >> > >> - The code is splitted in many packages and might be distributed under > >> several distributions. > >> > >> ? - distribute.resources: that's the old pkg_resources... > > > Why not just call it pkg_resources and/or merge it with pkgutil to get it > > into the python stdlib? > > Because "pkg_resources" belongs to 0.6 and we want 0.7 to be > installable alongside 0.6 if needed. [...] > > Why not call in "pypi.client" and "pypi.server" or, better yet, get it into > > the stdlib as disutils.index.client distutils.index.server? > > We are keeping a "distribute." namespace for all bits. > > Now maybe one day it'll make it into Distutils, yes. But not now. If the final plan is (or might be) to move distribute.resources into the stdlib might it not ease transition to pick an import name that will be able to remain the same? Regards Floris PS: Moved discussion from python-dev to distutils-sig, hope that's fine. -- Debian GNU/Linux -- The Power of Freedom www.debian.org | www.gnu.org | www.kernel.org From fdrake at gmail.com Fri Oct 9 19:24:14 2009 From: fdrake at gmail.com (Fred Drake) Date: Fri, 9 Oct 2009 13:24:14 -0400 Subject: [Distutils] [Python-Dev] Bits-of-Distribute naming In-Reply-To: <4ACF6008.8000201@simplistix.co.uk> References: <94bdd2610910080131j323b98d9i871bce43465f237a@mail.gmail.com> <4ACF59CC.5050504@simplistix.co.uk> <94bdd2610910090905t6fcdc6d9rfe946bbfbd36a7b9@mail.gmail.com> <4ACF6008.8000201@simplistix.co.uk> Message-ID: <9cee7ab80910091024y5da2b26fs94ea2a45a9a6ce35@mail.gmail.com> On Fri, Oct 9, 2009 at 12:08 PM, Chris Withers wrote: > distutils.entrypoints would seem to be the sensible place. Why's that? On the whole, I don't think entry points are specific to building & bundling, which is what distutils is all about. Entry points are about packages being able to provide metadata about how parts of them can be used. I'd rather see a pkgutil.entrypoint module with APIs for this. -Fred -- Fred L. Drake, Jr. "Chaos is the score upon which reality is written." --Henry Miller From arfrever.fta at gmail.com Fri Oct 9 20:35:32 2009 From: arfrever.fta at gmail.com (Arfrever Frehtes Taifersar Arahesis) Date: Fri, 9 Oct 2009 20:35:32 +0200 Subject: [Distutils] Packaging Distribute In-Reply-To: <94bdd2610910081527s70ed6dd7g4e8e2cd157939c96@mail.gmail.com> References: <90bb445a0910040810g48f7833cia02536e50e5c19f9@mail.gmail.com> <20091008220337.GC1240@clingman.lan> <94bdd2610910081527s70ed6dd7g4e8e2cd157939c96@mail.gmail.com> Message-ID: <200910092036.09208.Arfrever.FTA@gmail.com> 2009-10-09 00:27:41 Tarek Ziad? napisa?(a): > On Fri, Oct 9, 2009 at 12:03 AM, Toshio Kuratomi wrote: > > On Thu, Oct 08, 2009 at 11:07:13PM +0200, Arfrever Frehtes Taifersar Arahesis wrote: > >> 2009-10-04 23:52:25 Sridhar Ratnakumar napisa?(a): > >> > On Sun, 04 Oct 2009 13:41:06 -0700, Tarek Ziad? > >> > wrote: > >> > > >> > > The other way would be to use Distribute instead of Setuptools for > >> > > what the packaging system is calling "setuptools". That's pretty > >> > > much what is happening in Gentoo (arch) and UHU-Linux (dev), > >> > > right now > >> > > >> > Interesting. Gentoo uses distribute but retains the name 'setuptools'? > >> > >> It's because Distribute 0.6.* installs setuptools.* modules. > >> Distribute 0.7.* will be under name dev-python/distribute. > >> > > I started thinking about what it might take to do this for Fedora as well. > > There's a number of worries I have but it sounds attractive because of the > > increased maintainence support from distribute's upstream. > > > > An alternative I thought of would be for us to ship both distribute-0.6 and > > distribute-0.7 (when it arrives) and parallel install them. Then we can > > patch the setuptools using packages we maintain to check first for the 0.6 > > distribute and fall back on setuptools if it's not found. That would > > hopefully get a bunch of upstreams onto a better supported code base. > > > > My question is: will Distribute have a parallel installation plan? For > > instance, renaming the module provided by 0.7 distribute2? If so, this > > makes a lot of sense. If not, it's the ability of gentoo to reuse the > > setuptools name that makes parallel installation of distribute-0.6 and > > distribute-0.7 easier. > > We are not sure yet, but when 0.7 will be ready we have two choices: > > 1- we ship 0.7 under a new name - e.g. like "distribute2" > 2- we ship a fat 0.7 with the old 0.6 packages and modules included. I suggest to ship 0.7 under name "distribute" and allow parallel installation of 0.6 and 0.7. E.g. Python 2.5 and 2.6 can be installed parallelly. -- Arfrever Frehtes Taifersar Arahesis -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 198 bytes Desc: This is a digitally signed message part. URL: From a.badger at gmail.com Fri Oct 9 20:31:17 2009 From: a.badger at gmail.com (Toshio Kuratomi) Date: Fri, 9 Oct 2009 11:31:17 -0700 Subject: [Distutils] why would you ever need to specify setuptools as a dependency? In-Reply-To: <4ACF611C.4070901@simplistix.co.uk> References: <4ACF1956.5070903@simplistix.co.uk> <5cae42b20910090457p3f6e4a1bp361b240f974f1617@mail.gmail.com> <4ACF48A9.8060403@simplistix.co.uk> <20091009145617.GE1240@clingman.lan> <4ACF50E6.7030703@simplistix.co.uk> <20091009160456.GF1240@clingman.lan> <4ACF611C.4070901@simplistix.co.uk> Message-ID: <20091009183117.GB5033@clingman.lan> On Fri, Oct 09, 2009 at 05:13:16PM +0100, Chris Withers wrote: > Toshio Kuratomi wrote: >> On Fri, Oct 09, 2009 at 04:04:06PM +0100, Chris Withers wrote: >>> Toshio Kuratomi wrote: >>>> On Fri, Oct 09, 2009 at 03:28:57PM +0100, Chris Withers wrote: >>>>> In this case, which I suspect is extremely rare anyway, you'll >>>>> need to have setuptools installed already. >>>>> >>>>> So, in *any* of these cases, specifying setuptools as a >>>>> requirement seems like a total waste of time... >>>>> >>>>> Now, what case have I missed? ;-) >>>>> >>>> It's nice for people creating system packages when you specify all of the >>>> packages that your runtime depends on in setup.py. >>> ...except that it causes problems that are a bit more serious than >>> "nice to have" because of the ridiculous situation we're in with >>> setuptools and distribute... >>> >> What's the issue precisely? Once distribute is on the system, setuptools is >> provided by distribute so there's no problem there, correct? > > The issue is that both the setuptools and distribute distributions > provide a setuptools package. This apparently causes problems, rather > unsurprisingly ;-) > True... but because of that people are able to specify setuptools in setup.py and it will work with either distribute or setuptools. Is what you're getting at that if people didn't specify setuptools in setup.py, distribute-0.6 could install without using the setuptools name? I don't think that works since you still need to take over the setuptools module directory so import works inside the code and the setuptools egg-info so things like plugin modules belonging to setuptools work. -Toshio -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 197 bytes Desc: not available URL: From jmg3000 at gmail.com Sat Oct 10 23:10:30 2009 From: jmg3000 at gmail.com (John Gabriele) Date: Sat, 10 Oct 2009 17:10:30 -0400 Subject: [Distutils] Packaging Distribute In-Reply-To: <4ACF49E3.407@simplistix.co.uk> References: <90bb445a0910040810g48f7833cia02536e50e5c19f9@mail.gmail.com> <94bdd2610910041341v2c2483b1jd6c8f4b5707d789a@mail.gmail.com> <200910082307.13513.Arfrever.FTA@gmail.com> <20091008220337.GC1240@clingman.lan> <94bdd2610910081527s70ed6dd7g4e8e2cd157939c96@mail.gmail.com> <4ACF1BD3.5010402@simplistix.co.uk> <94bdd2610910090452l52e0cad5p4c11ab9989fc8b4b@mail.gmail.com> <4ACF49E3.407@simplistix.co.uk> Message-ID: <65e0bb520910101410g390f11f5s28722264b534ac2@mail.gmail.com> On Fri, Oct 9, 2009 at 10:34 AM, Chris Withers wrote: > Tarek Ziad? wrote: >> >> Sorry I won't run a new poll again, Distribute is the name. >> >> Besides, it's already on page #1 on google when I type 'python >> disribute' or 'distribute python' > > *shrugs* > > I will state now that I will fight tooth and nail before anything called > "distribute" gets into the stdlib that has nothing to do with distribution. Not to slash turpentine on an already painted bikeshed, but since the project's aim is to be a module distribution manager for Python, I like the name "modisty" (*mod*ule *dist*ribution manager for P*y*thon). In cases where it refuses to install a package for some reason, it could claim that "modisty forbids" such an action [snicker]. :) By the way, regarding name changes, even big projects change names every now and then (Xorg and Wireshark spring to mind) and seem to get their Google ranking back pretty quickly. Google's very good about noticing when things change. ---John From setuptools at bugs.python.org Sat Oct 10 23:48:46 2009 From: setuptools at bugs.python.org (Phillip J. Eby) Date: Sat, 10 Oct 2009 21:48:46 +0000 Subject: [Distutils] [issue86] [PATCH] python setup.py test in python2.6 In-Reply-To: <1255211326.69.0.989130095254.issue86@psf.upfronthosting.co.za> Message-ID: <1255211326.69.0.989130095254.issue86@psf.upfronthosting.co.za> New submission from Phillip J. Eby : Copy of issue 56 ---------- messages: 374 nosy: lregebro, pje, pjenvey priority: bug status: chatting title: [PATCH] python setup.py test in python2.6 _______________________________________________ Setuptools tracker _______________________________________________ From setuptools at bugs.python.org Sun Oct 11 00:07:30 2009 From: setuptools at bugs.python.org (Phillip J. Eby) Date: Sat, 10 Oct 2009 22:07:30 +0000 Subject: [Distutils] [issue87] [PATCH] Python 3 support In-Reply-To: <1255212450.24.0.545239688957.issue87@psf.upfronthosting.co.za> Message-ID: <1255212450.24.0.545239688957.issue87@psf.upfronthosting.co.za> New submission from Phillip J. Eby : Copy of issue 56; see file 35 for patch ---------- messages: 378 nosy: lregebro, pje, pjenvey priority: wish status: chatting title: [PATCH] Python 3 support _______________________________________________ Setuptools tracker _______________________________________________ From ziade.tarek at gmail.com Sun Oct 11 00:20:51 2009 From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=) Date: Sun, 11 Oct 2009 00:20:51 +0200 Subject: [Distutils] Distribute 0.6.4 released ! Message-ID: <94bdd2610910101520x7f827111u62fd28cc36b71d88@mail.gmail.com> Hello On behalf of the Distribute team I am happy to announce the release of Distribute 0.6.4. - This release is now compatible with zc.buildout, as long as you use the special bootstrap.py file provided at nightly.ziade.org/bootstrap.py - A new upload_docs command to easily upload project documentation to PyPI's http://packages.python.org was added. As a matter of fact, Distribute documentation is now available at : http://packages.python.org/distribute - A bug was fixed in the bootstraping process (IOError: Could not build the egg) The release is available at PyPI, together with the installation instructions: http://pypi.python.org/pypi/distribute The next release in the 0.6.x will continue the work on fixing bugs (see http://bitbucket.org/tarek/distribute/issues) Regards Tarek -- Tarek Ziad? | http://ziade.org |????????????! |???????????? From regebro at gmail.com Sun Oct 11 08:14:10 2009 From: regebro at gmail.com (Lennart Regebro) Date: Sun, 11 Oct 2009 08:14:10 +0200 Subject: [Distutils] Packaging Distribute In-Reply-To: <65e0bb520910101410g390f11f5s28722264b534ac2@mail.gmail.com> References: <90bb445a0910040810g48f7833cia02536e50e5c19f9@mail.gmail.com> <94bdd2610910041341v2c2483b1jd6c8f4b5707d789a@mail.gmail.com> <200910082307.13513.Arfrever.FTA@gmail.com> <20091008220337.GC1240@clingman.lan> <94bdd2610910081527s70ed6dd7g4e8e2cd157939c96@mail.gmail.com> <4ACF1BD3.5010402@simplistix.co.uk> <94bdd2610910090452l52e0cad5p4c11ab9989fc8b4b@mail.gmail.com> <4ACF49E3.407@simplistix.co.uk> <65e0bb520910101410g390f11f5s28722264b534ac2@mail.gmail.com> Message-ID: <319e029f0910102314k5242ea93gc538d99584929250@mail.gmail.com> 2009/10/10 John Gabriele : > Not to slash turpentine on an already painted bikeshed, but since the > project's aim is to be a module distribution manager for Python, I > like the name "modisty" (*mod*ule *dist*ribution manager for > P*y*thon). Ick. But "mopy-" is actually a good prefix. :) mopydist... Ah well, too late now, and Chris wouldn't like that either it's still "dist". -- Lennart Regebro: Python, Zope, Plone, Grok http://regebro.wordpress.com/ +33 661 58 14 64 From ronaldoussoren at mac.com Sun Oct 11 16:03:04 2009 From: ronaldoussoren at mac.com (Ronald Oussoren) Date: Sun, 11 Oct 2009 16:03:04 +0200 Subject: [Distutils] tracking requested vs dependency installs in PEP 376 metadata In-Reply-To: <4ACF3E77.8080806@dirtcircle.com> References: <4ACE15C5.4030105@meyerloewen.net> <20091008191526.GD513@clingman.lan> <4ACE4078.70504@meyerloewen.net> <20091008195227.GA735@clingman.lan> <4ACF3E77.8080806@dirtcircle.com> Message-ID: On 9 Oct, 2009, at 15:45, Carl Meyer wrote: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > > Alex wrote: >> REQUESTED is fine, but I don't understand how the arguments apply, >> given >>> that I'm not proposing to record information like _which_ package >>> it was >>> a dependency of. The same single bit (literally) of information is >>> tracked either way, it's just a question of whether the presence or >>> absence of a file signifies that bit. >>> >> And what if the package is a dependency for multiple packages? >> Let's say we have packages A and B which are installed separately, in >> that order, and both depend on C. >> C gets installed with information that it was required by A. Now if >> A is >> uninstalled, won't C also get uninstalled? > > No. We are NOT talking about recording the full dependency graph in > package metadata. It is, as has always been the case, up to an > uninstall > tool to calculate the depgraph based on actual installed packages at > runtime. A package is "orphaned" iff it was not REQUESTED by the user > AND it is no longer depended on by other installed packages. What about packages that are installed as a dependency of some other package and then used in user scripts without an explict depency on them? That is, I install "SuperWebFramework==1.0" which happens to depend on peak-rules. I later start using peak-rules in my own simple scripts (without a setup.py or other explicit dependency information), and yet later decide to uninstall "SuperWebFramework". If I understand the proposal correctly the uninstallation of "SuperWebFrameWork" would break my scripts. I also wonder how this interacts with system package managers. The .egg-info directory-structure and PEP377 where explicitly structured to allow (for example) RPM to create the egg-info directory without having to know about Python's package repository (which would be needed if that repository were a single-file database instead of the collection of egg-info directories). Ronald > > Carl > -----BEGIN PGP SIGNATURE----- > Version: GnuPG v1.4.6 (GNU/Linux) > Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ > > iD8DBQFKzz531j/fhc23WEARAlPfAJ93YHsbpzp5kF7hR0f98aSMOae6MwCgrP52 > RZs5FU3h0U0eUBN2vgqm2HA= > =jyi8 > -----END PGP SIGNATURE----- > _______________________________________________ > Distutils-SIG maillist - Distutils-SIG at python.org > http://mail.python.org/mailman/listinfo/distutils-sig -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 3567 bytes Desc: not available URL: From regebro at gmail.com Sun Oct 11 16:27:19 2009 From: regebro at gmail.com (Lennart Regebro) Date: Sun, 11 Oct 2009 16:27:19 +0200 Subject: [Distutils] tracking requested vs dependency installs in PEP 376 metadata In-Reply-To: References: <4ACE15C5.4030105@meyerloewen.net> <20091008191526.GD513@clingman.lan> <4ACE4078.70504@meyerloewen.net> <20091008195227.GA735@clingman.lan> <4ACF3E77.8080806@dirtcircle.com> Message-ID: <319e029f0910110727l7948ffccu6acee5617ea795bf@mail.gmail.com> 2009/10/11 Ronald Oussoren : > What about packages that are installed as a dependency of some other package > and then used in user scripts without an explict depency on them? > > That is, I install "SuperWebFramework==1.0" which happens to depend on > peak-rules. I later start using peak-rules in my own simple scripts (without > a setup.py or other explicit dependency information), and yet later decide > to uninstall "SuperWebFramework". ?If I understand the proposal correctly > the uninstallation of "SuperWebFrameWork" would break my scripts. Yes of course. Was that the question? Otherwise I didn't understand what you are asking. :) -- Lennart Regebro: Python, Zope, Plone, Grok http://regebro.wordpress.com/ +33 661 58 14 64 From carl at dirtcircle.com Sun Oct 11 18:21:20 2009 From: carl at dirtcircle.com (Carl Meyer) Date: Sun, 11 Oct 2009 12:21:20 -0400 Subject: [Distutils] tracking requested vs dependency installs in PEP 376 metadata In-Reply-To: References: <4ACE15C5.4030105@meyerloewen.net> <20091008191526.GD513@clingman.lan> <4ACE4078.70504@meyerloewen.net> <20091008195227.GA735@clingman.lan> <4ACF3E77.8080806@dirtcircle.com> Message-ID: <4AD20600.2080806@dirtcircle.com> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Ronald Oussoren wrote: > That is, I install "SuperWebFramework==1.0" which happens to depend on > peak-rules. I later start using peak-rules in my own simple scripts > (without a setup.py or other explicit dependency information), and yet > later decide to uninstall "SuperWebFramework". If I understand the > proposal correctly the uninstallation of "SuperWebFrameWork" would break > my scripts. Once again, let's distinguish this proposal to record a bit of metadata from the behavior of a hypothetical tool using that metadata. This proposal does not mandate any such behavior from an uninstall tool, or suggest that it be the default behavior. Many implementation choices are possible: to name just one possibility, the removal of "orphan" packages could be a separate command, which might prompt before removing anything. This proposal is merely to make available an extra bit of relevant metadata that may be useful to some tools, under some use cases. > I also wonder how this interacts with system package managers. The > .egg-info directory-structure and PEP377 where explicitly structured to > allow (for example) RPM to create the egg-info directory without having > to know about Python's package repository (which would be needed if that > repository were a single-file database instead of the collection of > egg-info directories). This concern is a general one about uninstallation, and is unrelated to the proposal to add REQUESTED metadata. The REQUESTED proposal does not create a centralized Python installation database, and changes nothing about how Python package managers and system package managers interact. FWIW, PEP 376 already addresses this concern via the "installer" metadata, so tools can choose to only uninstall packages they installed. Carl -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.6 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iD8DBQFK0gYA1j/fhc23WEARAu2yAJ45DYsMiDy+a85VW8khFRoTOwttjACdEFXb mraA5/j/oZ3MjroQslnVwhM= =Cdzb -----END PGP SIGNATURE----- From p.f.moore at gmail.com Sun Oct 11 19:45:33 2009 From: p.f.moore at gmail.com (Paul Moore) Date: Sun, 11 Oct 2009 18:45:33 +0100 Subject: [Distutils] Eggs vs bdist_wininst (Was: Distutils and Distribute roadmap (and some words on Virtualenv, Pip)) Message-ID: <79990c6b0910111045i228e147dl97c740874f0d3d20@mail.gmail.com> 2009/10/9 Michael Foord : > Many Windows users would be quite happy if the standard mechanism for > installing non-source distributions on Windows was via the wininst binaries. Actually, my understanding is that easy_install is perfectly able to use (download, unpack and install) bdist_wininst packages. So *as a binary distribution format* bdist_wininst installers can do all that eggs can, plus extra (install themselves unassisted!) So why is there a need for an egg binary format on Windows? Paul. From p.f.moore at gmail.com Sun Oct 11 19:49:42 2009 From: p.f.moore at gmail.com (Paul Moore) Date: Sun, 11 Oct 2009 18:49:42 +0100 Subject: [Distutils] [Python-Dev] Bits-of-Distribute naming In-Reply-To: <9cee7ab80910091024y5da2b26fs94ea2a45a9a6ce35@mail.gmail.com> References: <94bdd2610910080131j323b98d9i871bce43465f237a@mail.gmail.com> <4ACF59CC.5050504@simplistix.co.uk> <94bdd2610910090905t6fcdc6d9rfe946bbfbd36a7b9@mail.gmail.com> <4ACF6008.8000201@simplistix.co.uk> <9cee7ab80910091024y5da2b26fs94ea2a45a9a6ce35@mail.gmail.com> Message-ID: <79990c6b0910111049r5bd7de2ctf2d6dd14a2766ad7@mail.gmail.com> 2009/10/9 Fred Drake : > On Fri, Oct 9, 2009 at 12:08 PM, Chris Withers wrote: >> distutils.entrypoints would seem to be the sensible place. > > Why's that? ?On the whole, I don't think entry points are specific to > building & bundling, which is what distutils is all about. > > Entry points are about packages being able to provide metadata about > how parts of them can be used. ?I'd rather see a pkgutil.entrypoint > module with APIs for this. +1. It seems to me that pkgutil is the right place in the core for most of the "general utility" aspects of setuptools. Paul. From ronaldoussoren at mac.com Sun Oct 11 19:54:53 2009 From: ronaldoussoren at mac.com (Ronald Oussoren) Date: Sun, 11 Oct 2009 19:54:53 +0200 Subject: [Distutils] tracking requested vs dependency installs in PEP 376 metadata In-Reply-To: <319e029f0910110727l7948ffccu6acee5617ea795bf@mail.gmail.com> References: <4ACE15C5.4030105@meyerloewen.net> <20091008191526.GD513@clingman.lan> <4ACE4078.70504@meyerloewen.net> <20091008195227.GA735@clingman.lan> <4ACF3E77.8080806@dirtcircle.com> <319e029f0910110727l7948ffccu6acee5617ea795bf@mail.gmail.com> Message-ID: <1E5BF8F0-B345-4EF5-9C57-79E0F7C369D8@mac.com> On 11 Oct, 2009, at 16:27, Lennart Regebro wrote: > 2009/10/11 Ronald Oussoren : >> What about packages that are installed as a dependency of some >> other package >> and then used in user scripts without an explict depency on them? >> >> That is, I install "SuperWebFramework==1.0" which happens to depend >> on >> peak-rules. I later start using peak-rules in my own simple scripts >> (without >> a setup.py or other explicit dependency information), and yet later >> decide >> to uninstall "SuperWebFramework". If I understand the proposal >> correctly >> the uninstallation of "SuperWebFrameWork" would break my scripts. > > Yes of course. IMHO that is a bad experience for the user, because it is very unintuitive that I have to explicitly install something that's already installed to ensure that it doesn't go away in the future. This might work in a serious development environment, but can end up to be very annoying and confusing for casual users. Ronald -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 3567 bytes Desc: not available URL: From alex.gronholm at nextday.fi Sun Oct 11 20:02:23 2009 From: alex.gronholm at nextday.fi (=?ISO-8859-1?Q?Alex_Gr=F6nholm?=) Date: Sun, 11 Oct 2009 21:02:23 +0300 Subject: [Distutils] tracking requested vs dependency installs in PEP 376 metadata In-Reply-To: <1E5BF8F0-B345-4EF5-9C57-79E0F7C369D8@mac.com> References: <4ACE15C5.4030105@meyerloewen.net> <20091008191526.GD513@clingman.lan> <4ACE4078.70504@meyerloewen.net> <20091008195227.GA735@clingman.lan> <4ACF3E77.8080806@dirtcircle.com> <319e029f0910110727l7948ffccu6acee5617ea795bf@mail.gmail.com> <1E5BF8F0-B345-4EF5-9C57-79E0F7C369D8@mac.com> Message-ID: <4AD21DAF.3070107@nextday.fi> Ronald Oussoren kirjoitti: > > On 11 Oct, 2009, at 16:27, Lennart Regebro wrote: > >> 2009/10/11 Ronald Oussoren : >>> What about packages that are installed as a dependency of some other >>> package >>> and then used in user scripts without an explict depency on them? >>> >>> That is, I install "SuperWebFramework==1.0" which happens to depend on >>> peak-rules. I later start using peak-rules in my own simple scripts >>> (without >>> a setup.py or other explicit dependency information), and yet later >>> decide >>> to uninstall "SuperWebFramework". If I understand the proposal >>> correctly >>> the uninstallation of "SuperWebFrameWork" would break my scripts. >> >> Yes of course. > > IMHO that is a bad experience for the user, because it is very > unintuitive that I have to explicitly install something that's already > installed to ensure that it doesn't go away in the future. This might > work in a serious development environment, but can end up to be very > annoying and confusing for casual users. > The dpkg package manager (Debian) does not uninstall orphaned packages straight away, but notifies the user and allows them to "clean up" any orphans. Perhaps a similar approach would work here? > Ronald > ------------------------------------------------------------------------ > > _______________________________________________ > Distutils-SIG maillist - Distutils-SIG at python.org > http://mail.python.org/mailman/listinfo/distutils-sig > From regebro at gmail.com Sun Oct 11 20:58:47 2009 From: regebro at gmail.com (Lennart Regebro) Date: Sun, 11 Oct 2009 20:58:47 +0200 Subject: [Distutils] tracking requested vs dependency installs in PEP 376 metadata In-Reply-To: <1E5BF8F0-B345-4EF5-9C57-79E0F7C369D8@mac.com> References: <4ACE15C5.4030105@meyerloewen.net> <20091008191526.GD513@clingman.lan> <4ACE4078.70504@meyerloewen.net> <20091008195227.GA735@clingman.lan> <4ACF3E77.8080806@dirtcircle.com> <319e029f0910110727l7948ffccu6acee5617ea795bf@mail.gmail.com> <1E5BF8F0-B345-4EF5-9C57-79E0F7C369D8@mac.com> Message-ID: <319e029f0910111158y68fb25fajb3cbaf713062bf11@mail.gmail.com> 2009/10/11 Ronald Oussoren : >>> That is, I install "SuperWebFramework==1.0" which happens to depend on >>> peak-rules. I later start using peak-rules in my own simple scripts >>> (without >>> a setup.py or other explicit dependency information), and yet later >>> decide >>> to uninstall "SuperWebFramework". ?If I understand the proposal correctly >>> the uninstallation of "SuperWebFrameWork" would break my scripts. >> >> Yes of course. > > IMHO that is a bad experience for the user, because it is very unintuitive > that I have to explicitly install something that's already installed to > ensure that it doesn't go away in the future. But this is how all installs/uninstalls work and MUST work. If you uninstall the software that you use, then you can't use it. It's a simple necessity of life. > This might work in a serious > development environment, but can end up to be very annoying and confusing > for casual users. Can you come up with something better? How will the packaging registry know that you want a particular package installed if you don't tell it? -- Lennart Regebro: Python, Zope, Plone, Grok http://regebro.wordpress.com/ +33 661 58 14 64 From carl at dirtcircle.com Sun Oct 11 21:17:27 2009 From: carl at dirtcircle.com (Carl Meyer) Date: Sun, 11 Oct 2009 15:17:27 -0400 Subject: [Distutils] PEP 376 unclarity about function of INSTALLER Message-ID: <4AD22F47.8030903@dirtcircle.com> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Hi all, PEP 376 contains this line, regarding the INSTALLER metadata: "This allows a third-party application to use the uninstall function and make sure it's the only program that can remove a distribution it has previously installed." This sentence seems misleading to me. "We're all consenting adults here" and whatnot; it's not really possible for an installer to "make sure it's the only program that can remove a distribution it has previously installed." At best it is providing strong guidance. Perhaps replace "make sure it's the only program that can remove" with "strongly suggest that no other program remove"? Carl -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.6 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iD8DBQFK0i9H1j/fhc23WEARAoHrAKCv0EBG5HrjiSxBxJkNga8zmKmJTQCgtp1m RrLsw0+7LA1MHBqfYN+Vk9c= =nPHV -----END PGP SIGNATURE----- From ronaldoussoren at mac.com Sun Oct 11 21:30:20 2009 From: ronaldoussoren at mac.com (Ronald Oussoren) Date: Sun, 11 Oct 2009 21:30:20 +0200 Subject: [Distutils] tracking requested vs dependency installs in PEP 376 metadata In-Reply-To: <319e029f0910111158y68fb25fajb3cbaf713062bf11@mail.gmail.com> References: <4ACE15C5.4030105@meyerloewen.net> <20091008191526.GD513@clingman.lan> <4ACE4078.70504@meyerloewen.net> <20091008195227.GA735@clingman.lan> <4ACF3E77.8080806@dirtcircle.com> <319e029f0910110727l7948ffccu6acee5617ea795bf@mail.gmail.com> <1E5BF8F0-B345-4EF5-9C57-79E0F7C369D8@mac.com> <319e029f0910111158y68fb25fajb3cbaf713062bf11@mail.gmail.com> Message-ID: On 11 Oct, 2009, at 20:58, Lennart Regebro wrote: > 2009/10/11 Ronald Oussoren : >>>> That is, I install "SuperWebFramework==1.0" which happens to >>>> depend on >>>> peak-rules. I later start using peak-rules in my own simple scripts >>>> (without >>>> a setup.py or other explicit dependency information), and yet later >>>> decide >>>> to uninstall "SuperWebFramework". If I understand the proposal >>>> correctly >>>> the uninstallation of "SuperWebFrameWork" would break my scripts. >>> >>> Yes of course. >> >> IMHO that is a bad experience for the user, because it is very >> unintuitive >> that I have to explicitly install something that's already >> installed to >> ensure that it doesn't go away in the future. > > But this is how all installs/uninstalls work and MUST work. If you > uninstall the software that you use, then you can't use it. It's a > simple necessity of life. Sigh. 1) Install TurboGears, this installs simplejson as one of its dependencies 2) Notice that simplejson gets installed and is a useful module, then use that in your own scripts 3) Uninstall TurboGears With proposal of tracking which packages are installed as a dependency of other packages and automaticly uninstalling them when the package that depends on them gets uninstalled step 3 would break my scripts unless I explictly tell the installer that I really want to use simplejson in step 2. That's what I meant by "I have to explictly install something that is already installed to ensure that it doesn't go away in the future", and this is not a useful behaviour. > >> This might work in a serious >> development environment, but can end up to be very annoying and >> confusing >> for casual users. > > Can you come up with something better? How will the packaging registry > know that you want a particular package installed if you don't tell > it? The current proposal without tracking if I explicitly installed a package works fine and matches how system package managers work. This does make it harder to uninstall something like TurboGears that installs lots of dependencies, but it should be possible to write a tool that makes that easier. One way to do this in a GUI-ish manner is to have a "show dependencies" checkbox when you uninstall a package, this would then show all dependencies and allow easy selection of them for deinstallation. This gets you simular functionality as tracking explictly requested installs without needing additional metadata. Ronald -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 3567 bytes Desc: not available URL: From ziade.tarek at gmail.com Sun Oct 11 21:47:34 2009 From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=) Date: Sun, 11 Oct 2009 21:47:34 +0200 Subject: [Distutils] PEP 376 unclarity about function of INSTALLER In-Reply-To: <4AD22F47.8030903@dirtcircle.com> References: <4AD22F47.8030903@dirtcircle.com> Message-ID: <94bdd2610910111247w1540ac17uc815a3b5d7561b77@mail.gmail.com> On Sun, Oct 11, 2009 at 9:17 PM, Carl Meyer wrote: > > Perhaps replace "make sure it's the only program that can remove" with > "strongly suggest that no other program remove"? Sure, makes sense. Fixed. Tarek From regebro at gmail.com Sun Oct 11 21:48:55 2009 From: regebro at gmail.com (Lennart Regebro) Date: Sun, 11 Oct 2009 21:48:55 +0200 Subject: [Distutils] tracking requested vs dependency installs in PEP 376 metadata In-Reply-To: References: <4ACE15C5.4030105@meyerloewen.net> <20091008191526.GD513@clingman.lan> <4ACE4078.70504@meyerloewen.net> <20091008195227.GA735@clingman.lan> <4ACF3E77.8080806@dirtcircle.com> <319e029f0910110727l7948ffccu6acee5617ea795bf@mail.gmail.com> <1E5BF8F0-B345-4EF5-9C57-79E0F7C369D8@mac.com> <319e029f0910111158y68fb25fajb3cbaf713062bf11@mail.gmail.com> Message-ID: <319e029f0910111248p5a3fad27n74f6104f2ae380c7@mail.gmail.com> As usual, these debates are based on a ot of assumptions that seem obvious to one part but not the other. I assumed that when you mentioned uninstalling the framework, you meant to go through the procedure of actually uninstalling all parts of it. Instead you seem to have meant the case that any command that uninstalled the framework also by necessity HAD to result in the uninstalling of all dependencies, which is obviously not a good idea. Also, here are some more things I hold for obvious, to bar further misunderstandings: 1. Obviously any uninstaller would explicitly say which packages will be uninstalled. 2. Equally obvious, it should provide an option or maybe even ask, if you want the dependencies uninstalled or not. Plone, as an example, currently consists of some 170 packages. I think the most suprising result for a newbie here is that if you say "easy_uninstall Plone" and only one of these actually gets uninstalled. I agree the best option is asking. But barring that it's pretty obvious that the default should be either to uninstall all, or warn that there now are 169 orphaned packages and tell you how to uninstall them. -- Lennart Regebro: Python, Zope, Plone, Grok http://regebro.wordpress.com/ +33 661 58 14 64 From carl at dirtcircle.com Sun Oct 11 21:50:09 2009 From: carl at dirtcircle.com (Carl Meyer) Date: Sun, 11 Oct 2009 15:50:09 -0400 Subject: [Distutils] tracking requested vs dependency installs in PEP 376 metadata In-Reply-To: References: <4ACE15C5.4030105@meyerloewen.net> <20091008191526.GD513@clingman.lan> <4ACE4078.70504@meyerloewen.net> <20091008195227.GA735@clingman.lan> <4ACF3E77.8080806@dirtcircle.com> <319e029f0910110727l7948ffccu6acee5617ea795bf@mail.gmail.com> <1E5BF8F0-B345-4EF5-9C57-79E0F7C369D8@mac.com> <319e029f0910111158y68fb25fajb3cbaf713062bf11@mail.gmail.com> Message-ID: <4AD236F1.2080300@dirtcircle.com> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Ronald Oussoren wrote: > With proposal of tracking which packages are installed as a dependency > of other packages and automaticly uninstalling them when the package > that depends on them gets uninstalled Allow me to note (again) that the current proposal is ONLY part A (tracking). Part B ("automatically uninstalling...") is left completely unspecified, and would depend on what uninstaller tools choose to do with this metadata. I am NOT proposing that the basic uninstall function built in to distutils would automatically remove orphans. Unless you would argue that there are NO reasonable use-cases for detection of orphan dependencies, I would hope you could at least support tracking the data. > This does make it harder to uninstall something like TurboGears that > installs lots of dependencies, but it should be possible to write a tool > that makes that easier. One way to do this in a GUI-ish manner is to > have a "show dependencies" checkbox when you uninstall a package, this > would then show all dependencies and allow easy selection of them for > deinstallation. > This gets you simular functionality as tracking explictly requested > installs without needing additional metadata. I don't agree that the functionality is comparable. I haven't yet seen an argument against simply tracking REQUESTED metadata; can you agree that at the very least the ability to _list_ orphaned packages could be useful, apart from any kind of automatic uninstall behavior? Carl -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.6 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iD8DBQFK0jbx1j/fhc23WEARAvUQAJ9xqAi/684TLDADsx6hYFuX3UBC+wCeJoVp VQw/F11l9nnBeFLCljTxQwg= =8PIr -----END PGP SIGNATURE----- From ziade.tarek at gmail.com Sun Oct 11 21:59:59 2009 From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=) Date: Sun, 11 Oct 2009 21:59:59 +0200 Subject: [Distutils] tracking requested vs dependency installs in PEP 376 metadata In-Reply-To: <4AD236F1.2080300@dirtcircle.com> References: <4ACE15C5.4030105@meyerloewen.net> <4ACE4078.70504@meyerloewen.net> <20091008195227.GA735@clingman.lan> <4ACF3E77.8080806@dirtcircle.com> <319e029f0910110727l7948ffccu6acee5617ea795bf@mail.gmail.com> <1E5BF8F0-B345-4EF5-9C57-79E0F7C369D8@mac.com> <319e029f0910111158y68fb25fajb3cbaf713062bf11@mail.gmail.com> <4AD236F1.2080300@dirtcircle.com> Message-ID: <94bdd2610910111259x116948b5k35efe0aff6014c7a@mail.gmail.com> On Sun, Oct 11, 2009 at 9:50 PM, Carl Meyer wrote: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > > > Ronald Oussoren wrote: >> With proposal of tracking which packages are installed as a dependency >> of other packages and automaticly uninstalling them when the package >> that depends on them gets uninstalled > > Allow me to note (again) that the current proposal is ONLY part A > (tracking). Part B ("automatically uninstalling...") is left completely > unspecified, and would depend on what uninstaller tools choose to do > with this metadata. I am NOT proposing that the basic uninstall function > built in to distutils would automatically remove orphans. PEP 376 doesn't want to go that far anyway. The uninstall API is just a reference API that can be used to remove *one* distribution. So if there's a tracking file, it should be the simplest thing possible. The goal of PEP 376 is to provide tools for installers/uninstallers to work with a site-package directory, not a full featured uninstaller. I am starting to think when reading back this whole thread, that there's a missing API: a way to read and write arbitrary files in the egg-info directory of an installed distribution. This would allow third-party tools to have their own extra metadata information regarding the distributions. That is, IIRC, what Jim proposed a while ago when he said that we should have a way to define arbitrary metadata for distributions. Tarek. From carl at dirtcircle.com Sun Oct 11 22:03:50 2009 From: carl at dirtcircle.com (Carl Meyer) Date: Sun, 11 Oct 2009 16:03:50 -0400 Subject: [Distutils] tracking requested vs dependency installs in PEP 376 metadata In-Reply-To: <94bdd2610910111259x116948b5k35efe0aff6014c7a@mail.gmail.com> References: <4ACE15C5.4030105@meyerloewen.net> <4ACE4078.70504@meyerloewen.net> <20091008195227.GA735@clingman.lan> <4ACF3E77.8080806@dirtcircle.com> <319e029f0910110727l7948ffccu6acee5617ea795bf@mail.gmail.com> <1E5BF8F0-B345-4EF5-9C57-79E0F7C369D8@mac.com> <319e029f0910111158y68fb25fajb3cbaf713062bf11@mail.gmail.com> <4AD236F1.2080300@dirtcircle.com> <94bdd2610910111259x116948b5k35efe0aff6014c7a@mail.gmail.com> Message-ID: <4AD23A26.8030308@dirtcircle.com> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Tarek Ziad? wrote: > I am starting to think when reading back this whole thread, that > there's a missing API: > a way to read and write arbitrary files in the egg-info directory of > an installed distribution. I thought of that as well when writing the demo REQUESTED functionality - - I was duplicating work that was already done for RECORD and INSTALLER. pkg_resources has this generic API and it seems to work well. Even if PEP 376 gains that API, IMO the REQUESTED metadata is useful enough to be standardized, rather than restricted to a particular tool. Carl -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.6 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iD8DBQFK0jom1j/fhc23WEARAnZ2AKCoKM/rmG+CBPlzSa67UW3DYlAAnACdFZ/4 VfMIanS9/PLUUAMH3dZHx9w= =ITZH -----END PGP SIGNATURE----- From ziade.tarek at gmail.com Sun Oct 11 22:09:10 2009 From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=) Date: Sun, 11 Oct 2009 22:09:10 +0200 Subject: [Distutils] [Python-Dev] Bits-of-Distribute naming In-Reply-To: <79990c6b0910111049r5bd7de2ctf2d6dd14a2766ad7@mail.gmail.com> References: <94bdd2610910080131j323b98d9i871bce43465f237a@mail.gmail.com> <4ACF59CC.5050504@simplistix.co.uk> <94bdd2610910090905t6fcdc6d9rfe946bbfbd36a7b9@mail.gmail.com> <4ACF6008.8000201@simplistix.co.uk> <9cee7ab80910091024y5da2b26fs94ea2a45a9a6ce35@mail.gmail.com> <79990c6b0910111049r5bd7de2ctf2d6dd14a2766ad7@mail.gmail.com> Message-ID: <94bdd2610910111309u2ed6323bi49e5b0952516d0f0@mail.gmail.com> On Sun, Oct 11, 2009 at 7:49 PM, Paul Moore wrote: > 2009/10/9 Fred Drake : >> On Fri, Oct 9, 2009 at 12:08 PM, Chris Withers wrote: >>> distutils.entrypoints would seem to be the sensible place. >> >> Why's that? ?On the whole, I don't think entry points are specific to >> building & bundling, which is what distutils is all about. >> >> Entry points are about packages being able to provide metadata about >> how parts of them can be used. ?I'd rather see a pkgutil.entrypoint >> module with APIs for this. > > +1. It seems to me that pkgutil is the right place in the core for > most of the "general utility" aspects of setuptools. In any case, adding entry points in the stdlib is premature. It's a plugin discovery system based on reading and writing an extra file located in the egg-info *directory* of an installled Distribution. And that's setuptools installation format, not Distutils' current (where egg-info is the PKG-INFO *file*) Once PEP 376 is finished and accepted, then we can think about it, but until then, this feature has to live on Distribute side. Regards Tarek From mwhapples at aim.com Sun Oct 11 22:19:45 2009 From: mwhapples at aim.com (Michael Whapples) Date: Sun, 11 Oct 2009 21:19:45 +0100 Subject: [Distutils] Problem with distribute and converting tests for python3 Message-ID: <4AD23DE1.2050903@aim.com> Hello, Firstly thanks to those working on distribute, I had been using setuptools but was getting annoyed with it slowly developing (eg. not having python3 support, etc), so distribute seems like it is what I am looking for. Any way I am using distribute for one of my packages now (I wanted to try and support python3) but have noticed something which doesn't work for me. I have my main package sources in one directory and then tests in another directory (it makes it easy to separate them out when creating a binary package). Now this works fine for python2.x as I can include the tests in a source distribution by using MANIFEST.in, but it doesn't work for python3 as distribute doesn't seem to run 2to3 on them (although they are run by the distribute/setuptools test command in the setup.py script). When I looked at the distribute documentation the example there has the tests in a subpackage of the main package. So my question is, am I structuring my source distribution wrongly, is there a better way to do this which would work with distribute? If I were to structure my source distribution like the distribute example then how do I ensure unit tests aren't included in a binary distribution? Is there anyway I can get distribute to pick up my tests as I currently have my source distribution structured? Michael Whapples From ziade.tarek at gmail.com Sun Oct 11 22:39:48 2009 From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=) Date: Sun, 11 Oct 2009 22:39:48 +0200 Subject: [Distutils] Eggs vs bdist_wininst (Was: Distutils and Distribute roadmap (and some words on Virtualenv, Pip)) In-Reply-To: <79990c6b0910111045i228e147dl97c740874f0d3d20@mail.gmail.com> References: <79990c6b0910111045i228e147dl97c740874f0d3d20@mail.gmail.com> Message-ID: <94bdd2610910111339y30236851u6837c50a8c8d76f4@mail.gmail.com> On Sun, Oct 11, 2009 at 7:45 PM, Paul Moore wrote: > 2009/10/9 Michael Foord : >> Many Windows users would be quite happy if the standard mechanism for >> installing non-source distributions on Windows was via the wininst binaries. > > Actually, my understanding is that easy_install is perfectly able to > use (download, unpack and install) bdist_wininst packages. So *as a > binary distribution format* bdist_wininst installers can do all that > eggs can, plus extra (install themselves unassisted!) Yes, the script is able to unpack and install those. So it could be the standard yes. > So why is there a need for an egg binary format on Windows? I think the only extra specific feature bdist_egg provides is the ability to use the self-contained egg (zipped or not) directly in sys.path without having to install it in a site-packages like directory. That's how zc.buildout works for example to isolate an execution environment : it collects eggs in a directory called 'eggs', then creates scripts with a modified sys.path that lists the paths of each egg located in 'eggs'. Regards Tarek From regebro at gmail.com Sun Oct 11 22:43:07 2009 From: regebro at gmail.com (Lennart Regebro) Date: Sun, 11 Oct 2009 22:43:07 +0200 Subject: [Distutils] Problem with distribute and converting tests for python3 In-Reply-To: <4AD23DE1.2050903@aim.com> References: <4AD23DE1.2050903@aim.com> Message-ID: <319e029f0910111343l164bdacaw7ab552f863be935c@mail.gmail.com> 2009/10/11 Michael Whapples : > Hello, > Firstly thanks to those working on distribute, I had been using setuptools > but was getting annoyed with it slowly developing (eg. not having python3 > support, etc), so distribute seems like it is what I am looking for. > > Any way I am using distribute for one of my packages now (I wanted to try > and support python3) but have noticed something which doesn't work for me. I > have my main package sources in one directory and then tests in another > directory (it makes it easy to separate them out when creating a binary > package). Now this works fine for python2.x as I can include the tests in a > source distribution by using MANIFEST.in, but it doesn't work for python3 as > distribute doesn't seem to run 2to3 on them (although they are run by the > distribute/setuptools test command in the setup.py script). You have to tell setup that these files are source files, somehow. By just including them in MANIFEST.in but not in any source package they'll get treated as resources. Perhaps if you included it as a apckage or py_module? It is true that Distribute picks up files in test/test*.py for inclusion in source distributions, which I didn't know until now. These will not be included in the binary distribution, and hence not converted with 2to3 either, so that will not work with 2to3 support. That needs to be documented or fixed somehow, although I don't really know how... > If I were to > structure my source distribution like the distribute example then how do I > ensure unit tests aren't included in a binary distribution? Well... what is the actual use case for doing that? There's no problem with distributing the tests (in fact, I would say it's a good idea). -- Lennart Regebro: Python, Zope, Plone, Grok http://regebro.wordpress.com/ +33 661 58 14 64 From ianb at colorstudy.com Sun Oct 11 22:45:48 2009 From: ianb at colorstudy.com (Ian Bicking) Date: Sun, 11 Oct 2009 15:45:48 -0500 Subject: [Distutils] Eggs vs bdist_wininst (Was: Distutils and Distribute roadmap (and some words on Virtualenv, Pip)) In-Reply-To: <94bdd2610910111339y30236851u6837c50a8c8d76f4@mail.gmail.com> References: <79990c6b0910111045i228e147dl97c740874f0d3d20@mail.gmail.com> <94bdd2610910111339y30236851u6837c50a8c8d76f4@mail.gmail.com> Message-ID: On Sun, Oct 11, 2009 at 3:39 PM, Tarek Ziad? wrote: >> So why is there a need for an egg binary format on Windows? > > I think the only extra specific feature bdist_egg provides is the > ability to use the self-contained > egg (zipped or not) directly in sys.path without having to install it > in a site-packages like directory. > > That's how zc.buildout works for example to isolate an execution > environment : it collects eggs in a directory called 'eggs', then > creates scripts with a modified sys.path that lists the paths of each > egg located in 'eggs'. That's a feature of the installed (multi-version) egg, not the distribution format. That feature can be kept (or discussed, or whatever) separately from the discussion of bdist_egg as a distribution format. -- Ian Bicking | http://blog.ianbicking.org | http://topplabs.org/civichacker From ziade.tarek at gmail.com Sun Oct 11 23:07:22 2009 From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=) Date: Sun, 11 Oct 2009 23:07:22 +0200 Subject: [Distutils] Eggs vs bdist_wininst (Was: Distutils and Distribute roadmap (and some words on Virtualenv, Pip)) In-Reply-To: References: <79990c6b0910111045i228e147dl97c740874f0d3d20@mail.gmail.com> <94bdd2610910111339y30236851u6837c50a8c8d76f4@mail.gmail.com> Message-ID: <94bdd2610910111407g48c065f3le70fca58c6714d96@mail.gmail.com> On Sun, Oct 11, 2009 at 10:45 PM, Ian Bicking wrote: > On Sun, Oct 11, 2009 at 3:39 PM, Tarek Ziad? wrote: >>> So why is there a need for an egg binary format on Windows? >> >> I think the only extra specific feature bdist_egg provides is the >> ability to use the self-contained >> egg (zipped or not) directly in sys.path without having to install it >> in a site-packages like directory. >> >> That's how zc.buildout works for example to isolate an execution >> environment : it collects eggs in a directory called 'eggs', then >> creates scripts with a modified sys.path that lists the paths of each >> egg located in 'eggs'. > > That's a feature of the installed (multi-version) egg, not the > distribution format. ?That feature can be kept (or discussed, or > whatever) separately from the discussion of bdist_egg as a > distribution format. The bdist_egg distribution format, an 'egg', can be used directly in a Python interpreter without having to install it at all : $ wget http://pypi.python.org/packages/2.6/W/Wally/Wally-0.6.4-py2.6.egg $ python >>> import sys >>> sys.path.insert(0, 'Wally-0.6.4-py2.6.egg') >>> import wally >>> wally Tarek From mwhapples at aim.com Sun Oct 11 23:34:43 2009 From: mwhapples at aim.com (Michael Whapples) Date: Sun, 11 Oct 2009 22:34:43 +0100 Subject: [Distutils] Problem with distribute and converting tests for python3 In-Reply-To: <319e029f0910111343l164bdacaw7ab552f863be935c@mail.gmail.com> References: <4AD23DE1.2050903@aim.com> <319e029f0910111343l164bdacaw7ab552f863be935c@mail.gmail.com> Message-ID: <4AD24F73.1000105@aim.com> My problem with having unit tests included in a binary distribution is that they take up space and have no way to be run, so can certainly be called a waste of space. OK, may be they could be run by a user importing the correct modules, etc, I meant no simple way (eg. as done in a source package with the command $ setup.py test ). Also anyway what would the reason be for a user to run tests on the binary module, I (or another developer in the cases of modified versions) should have run the tests before distributing the binary package. Source distributions are a different matter in my mind, someone can modify it and so having tests is useful. OK, to possible solutions: 1. Distribute offers the keyword argument "convert_2to3_doctests", may be it could have one to give test directory, and then .py files in that could be converted. 2. As setuptools automatically includes test/test*.py, why can't distribute actually compile these files as well (I question whether tests in a test directory should be prefixed with test, but I don't have a really strong view should people disagree). May be even both of those could be combined, so if a test directory is not specified then test/test*.py is the default (if such a directory exists). As a note: my views of structuring a source distribution and not including tests in binary distributions may be influenced by my java programming, hope that helps explain where I may be coming from. Michael Whapples On 11/10/09 21:43, Lennart Regebro wrote: > 2009/10/11 Michael Whapples: > >> Hello, >> Firstly thanks to those working on distribute, I had been using setuptools >> but was getting annoyed with it slowly developing (eg. not having python3 >> support, etc), so distribute seems like it is what I am looking for. >> >> Any way I am using distribute for one of my packages now (I wanted to try >> and support python3) but have noticed something which doesn't work for me. I >> have my main package sources in one directory and then tests in another >> directory (it makes it easy to separate them out when creating a binary >> package). Now this works fine for python2.x as I can include the tests in a >> source distribution by using MANIFEST.in, but it doesn't work for python3 as >> distribute doesn't seem to run 2to3 on them (although they are run by the >> distribute/setuptools test command in the setup.py script). >> > You have to tell setup that these files are source files, somehow. By > just including them in MANIFEST.in but not in any source package > they'll get treated as resources. Perhaps if you included it as a > apckage or py_module? > > It is true that Distribute picks up files in test/test*.py for > inclusion in source distributions, which I didn't know until now. > These will not be included in the binary distribution, and hence not > converted with 2to3 either, so that will not work with 2to3 support. > That needs to be documented or fixed somehow, although I don't really > know how... > > >> If I were to >> structure my source distribution like the distribute example then how do I >> ensure unit tests aren't included in a binary distribution? >> > Well... what is the actual use case for doing that? There's no problem > with distributing the tests (in fact, I would say it's a good idea). > > From p.f.moore at gmail.com Mon Oct 12 00:11:54 2009 From: p.f.moore at gmail.com (Paul Moore) Date: Sun, 11 Oct 2009 23:11:54 +0100 Subject: [Distutils] Eggs vs bdist_wininst (Was: Distutils and Distribute roadmap (and some words on Virtualenv, Pip)) In-Reply-To: <94bdd2610910111407g48c065f3le70fca58c6714d96@mail.gmail.com> References: <79990c6b0910111045i228e147dl97c740874f0d3d20@mail.gmail.com> <94bdd2610910111339y30236851u6837c50a8c8d76f4@mail.gmail.com> <94bdd2610910111407g48c065f3le70fca58c6714d96@mail.gmail.com> Message-ID: <79990c6b0910111511g4b521efco137b5aa2f8ad9eba@mail.gmail.com> 2009/10/11 Tarek Ziad? : > On Sun, Oct 11, 2009 at 10:45 PM, Ian Bicking wrote: >> On Sun, Oct 11, 2009 at 3:39 PM, Tarek Ziad? wrote: >>>> So why is there a need for an egg binary format on Windows? >>> >>> I think the only extra specific feature bdist_egg provides is the >>> ability to use the self-contained >>> egg (zipped or not) directly in sys.path without having to install it >>> in a site-packages like directory. >>> >>> That's how zc.buildout works for example to isolate an execution >>> environment : it collects eggs in a directory called 'eggs', then >>> creates scripts with a modified sys.path that lists the paths of each >>> egg located in 'eggs'. >> >> That's a feature of the installed (multi-version) egg, not the >> distribution format. ?That feature can be kept (or discussed, or >> whatever) separately from the discussion of bdist_egg as a >> distribution format. > > The bdist_egg distribution format, an 'egg', can be used directly in a > Python interpreter > without having to install it at all : > > $ wget http://pypi.python.org/packages/2.6/W/Wally/Wally-0.6.4-py2.6.egg > $ python >>>> import sys >>>> sys.path.insert(0, 'Wally-0.6.4-py2.6.egg') >>>> import wally >>>> wally > OK, that's a fair point. But how many people actually use this functionality? More to the point, how many *windows* users use this functionality (as bdist_wininst isn't an alternative for non-windows users anyway)? And to restrict it even further, how many windows users use it for *binary* distributions (ie, using C code) as packaging non-C code from source is trivial)? Unfortunately, it's a social rather than a technical issue, but it would be really good if some way could be found to reverse the current trend away from a single distribution format. Paul. From ziade.tarek at gmail.com Mon Oct 12 00:17:10 2009 From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=) Date: Mon, 12 Oct 2009 00:17:10 +0200 Subject: [Distutils] PEP 390 - new format from setup.cfg Message-ID: <94bdd2610910111517u2fd0b136p44958f7faa46e974@mail.gmail.com> Hey this is the PEP for setup.cfg, as requested : http://www.python.org/dev/peps/pep-0390 Please comment, Tarek -- Tarek Ziad? | http://ziade.org |????????????! |???????????? From eric at trueblade.com Mon Oct 12 02:06:05 2009 From: eric at trueblade.com (Eric Smith) Date: Sun, 11 Oct 2009 20:06:05 -0400 Subject: [Distutils] tracking requested vs dependency installs in PEP 376 metadata In-Reply-To: References: <4ACE15C5.4030105@meyerloewen.net> <20091008191526.GD513@clingman.lan> <4ACE4078.70504@meyerloewen.net> <20091008195227.GA735@clingman.lan> <4ACF3E77.8080806@dirtcircle.com> <319e029f0910110727l7948ffccu6acee5617ea795bf@mail.gmail.com> <1E5BF8F0-B345-4EF5-9C57-79E0F7C369D8@mac.com> <319e029f0910111158y68fb25fajb3cbaf713062bf11@mail.gmail.com> Message-ID: <4AD272ED.4040508@trueblade.com> Ronald Oussoren wrote: > > On 11 Oct, 2009, at 20:58, Lennart Regebro wrote: > >> 2009/10/11 Ronald Oussoren : >>>>> That is, I install "SuperWebFramework==1.0" which happens to depend on >>>>> peak-rules. I later start using peak-rules in my own simple scripts >>>>> (without >>>>> a setup.py or other explicit dependency information), and yet later >>>>> decide >>>>> to uninstall "SuperWebFramework". If I understand the proposal >>>>> correctly >>>>> the uninstallation of "SuperWebFrameWork" would break my scripts. >>>> >>>> Yes of course. >>> >>> IMHO that is a bad experience for the user, because it is very >>> unintuitive >>> that I have to explicitly install something that's already installed to >>> ensure that it doesn't go away in the future. >> >> But this is how all installs/uninstalls work and MUST work. If you >> uninstall the software that you use, then you can't use it. It's a >> simple necessity of life. > > Sigh. > > 1) Install TurboGears, this installs simplejson as one of its dependencies > > 2) Notice that simplejson gets installed and is a useful module, then > use that in your own scripts > > 3) Uninstall TurboGears > > With proposal of tracking which packages are installed as a dependency > of other packages and automaticly uninstalling them when the package > that depends on them gets uninstalled step 3 would break my scripts > unless I explictly tell the installer that I really want to use > simplejson in step 2. You really need to evaluate the dependency graph when uninstalling, a simple bit doesn't help. Image step 2B above: install Yapper, which uses simplejson. Even if step 3 wants to uninstall simplejson, it now needs to know that other installed "module distributions" (as the term is used by distutils) depend on it. Eric. From ianb at colorstudy.com Mon Oct 12 02:29:15 2009 From: ianb at colorstudy.com (Ian Bicking) Date: Sun, 11 Oct 2009 19:29:15 -0500 Subject: [Distutils] PEP 390 - new format from setup.cfg In-Reply-To: <94bdd2610910111517u2fd0b136p44958f7faa46e974@mail.gmail.com> References: <94bdd2610910111517u2fd0b136p44958f7faa46e974@mail.gmail.com> Message-ID: The grammar in "Context-dependant sections" indicates possible EXPR values. Because the "in" operator is supported, I would assume that tuples should also be allowed. One aspect of the current setup.cfg is that it supports multiple sections, for different setup.py commands or components. This gives a kind of namespacing. I assume, but it isn't that specified, that any section (not just "metadata") will be parsed the same way? Presumably setup.py will just contain an empty call to setup()? (I assume at least setup.py will be allowed for, for backward compatibility, even if it is not required.) I believe this does not include the concept of extra requirements. Possibly it could as a side-effect of some variable available to the language. Like: [metadata:'xmlrpc' in features] requires = lxml Sets and the & operator could be useful for this. The way variables are handled is unclear. Presumably all variables are cumulative. But given: requires = Foo requires = Bar What is the value of "requires"? A list, a value with newlines? Presumably you could also do: requires = Foo Bar Anyway, we're diverging from INI semantics at this point, so it needs to be specified how it works. Is there a way to eliminate values? Right now, for instance, if you have "[build] home = /foo" in a distutils.cfg, there's no way to unset that. I'd like to see this functionality included. Perhaps to delete a specific value, but the simpler case of deleting a variable is really all I want. On Sun, Oct 11, 2009 at 5:17 PM, Tarek Ziad? wrote: > Hey > > this is the PEP for setup.cfg, as requested : > http://www.python.org/dev/peps/pep-0390 > > Please comment, > > Tarek > > -- > Tarek Ziad? | http://ziade.org |????????????! |???????????? > _______________________________________________ > Distutils-SIG maillist ?- ?Distutils-SIG at python.org > http://mail.python.org/mailman/listinfo/distutils-sig > -- Ian Bicking | http://blog.ianbicking.org | http://topplabs.org/civichacker From carl at meyerloewen.net Mon Oct 12 05:03:11 2009 From: carl at meyerloewen.net (Carl Meyer) Date: Sun, 11 Oct 2009 23:03:11 -0400 Subject: [Distutils] tracking requested vs dependency installs in PEP 376 metadata In-Reply-To: <4AD272ED.4040508@trueblade.com> References: <4ACE15C5.4030105@meyerloewen.net> <20091008191526.GD513@clingman.lan> <4ACE4078.70504@meyerloewen.net> <20091008195227.GA735@clingman.lan> <4ACF3E77.8080806@dirtcircle.com> <319e029f0910110727l7948ffccu6acee5617ea795bf@mail.gmail.com> <1E5BF8F0-B345-4EF5-9C57-79E0F7C369D8@mac.com> <319e029f0910111158y68fb25fajb3cbaf713062bf11@mail.gmail.com> <4AD272ED.4040508@trueblade.com> Message-ID: <4AD29C6F.3060204@meyerloewen.net> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Eric Smith wrote: > You really need to evaluate the dependency graph when uninstalling, Of course. As I've stated quite clearly already in this thread, in response to exactly the same example you provided. > a simple bit doesn't help. It doesn't tell you everything you need to know, but of course it helps; it's the necessary minimum in order to provide any awareness of orphaned dependencies. Carl -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.6 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iD8DBQFK0pxvFRcxmeyPUXIRArsVAJ0akGC5/jt990mE4NW8+9rI34YJtQCfVhIB 4g3Cs3+Y5OwzCcV8sxWSaPc= =EztZ -----END PGP SIGNATURE----- From regebro at gmail.com Mon Oct 12 06:24:49 2009 From: regebro at gmail.com (Lennart Regebro) Date: Mon, 12 Oct 2009 06:24:49 +0200 Subject: [Distutils] Problem with distribute and converting tests for python3 In-Reply-To: <4AD24F73.1000105@aim.com> References: <4AD23DE1.2050903@aim.com> <319e029f0910111343l164bdacaw7ab552f863be935c@mail.gmail.com> <4AD24F73.1000105@aim.com> Message-ID: <319e029f0910112124k4f89fe29qf2c02b24b3c8eb0d@mail.gmail.com> 2009/10/11 Michael Whapples : > My problem with having unit tests included in a binary distribution is that > they take up space and have no way to be run, so can certainly be called a > waste of space. OK, may be they could be run by a user importing the correct > modules, etc, I meant no simple way (eg. as done in a source package with > the command > $ setup.py test True, you need some test runner like nose or zope.tetsing or so. But then it works. > ). Also anyway what would the reason be for a user to run tests on the > binary module, I (or another developer in the cases of modified versions) > should have run the tests before distributing the binary package. Yes, but if something breaks it can be good to run the tests locally to see if something in the environments breaks them. Also tests can be useful as a sort of documentation. > OK, to possible solutions: > 1. Distribute offers the keyword argument "convert_2to3_doctests", may be it > could have one to give test directory, and then .py files in that could be > converted. Well, the test directory is test/test*.py. > 2. As setuptools automatically includes test/test*.py, why can't distribute > actually compile these files as well (I question whether tests in a test > directory should be prefixed with test, but I don't have a really strong > view should people disagree). The problem is that these files doesn't end up in the binary distribution and hence 2to3 can't be run on them. -- Lennart Regebro: Python, Zope, Plone, Grok http://regebro.wordpress.com/ +33 661 58 14 64 From reinout at vanrees.org Mon Oct 12 08:53:23 2009 From: reinout at vanrees.org (Reinout van Rees) Date: Mon, 12 Oct 2009 06:53:23 +0000 (UTC) Subject: [Distutils] dev versions References: Message-ID: On 2009-10-08, Ian Bicking wrote: > > So after creating, say, version 0.3.1, I always mark a package as 0.3.2dev. > But this is annoying, you might never create a version 0.3.2 (e.g., 0.4 > might be the next level). So, it would be better to use something like > 0.3.1~dev. What is considered best practice for this? Ideally something > that works with both Setuptools and the upcoming Distribute version spec. a) Where's the annoyment exactly? It is easy to change and it's a release-time decision anyway. b) In a previous discussion on a zope mailinglist (about using '0' for this purpose, which was pretty much shot down for the zope toolkit because of the problems attached to it), someone mentioned adding '+svn' to the previous version number. So from 0.3.1 to 0.3.1+svn. Apparently that sorts it behind 0.3.1. You could try something like that. The poster mentioned it as a debian standard. Reinout -- Reinout van Rees - reinout at vanrees.org - http://reinout.vanrees.org Software developer at http://www.thehealthagency.com "Military engineers build missiles. Civil engineers build targets" From reinout at vanrees.org Mon Oct 12 09:10:54 2009 From: reinout at vanrees.org (Reinout van Rees) Date: Mon, 12 Oct 2009 07:10:54 +0000 (UTC) Subject: [Distutils] the virtualenv-distribute mess References: <4ACB1B5A.2090608@simplistix.co.uk> <319e029f0910060402n306a992as4b9d854f7f29d546@mail.gmail.com> <319e029f0910060402m28ab35acnc9a52aefcdc6f04a@mail.gmail.com> <4ACB24AE.4000300@cryptelium.net> <4ACB2527.7010802@cryptelium.net> <319e029f0910060522l29499f46y2805ed0082d975c3@mail.gmail.com> <4ACB7D70.9060404@cryptelium.net> <4ACF1681.7050506@simplistix.co.uk> <4ACF269C.4010400@cryptelium.net> <94bdd2610910090508j6804628fy5b5d435cce0c7427@mail.gmail.com> Message-ID: On 2009-10-09, Tarek Ziad? wrote: > On Fri, Oct 9, 2009 at 2:03 PM, kiorky wrote: >> AND no, virtualenv must continue to provide setuptools, backward >> compatibility, you know? > > The *whole* point of Distribute 0.6.x is to be backward compatible, meaning > that if virtualenv switch to it, you will not even notice it. For something that's supposed to be a drop-in replacement, it sure makes it self noticeable. Note that I'm not holding that against distribute! The mess is of setuptools' making. - When using buildout, I get lots of warnings. The 1.4.2 isn't out yet, but I also won't update all old projects' pinned zc.buildout version so I'm stuck with warnings for a time. - On every computer where I installed it, somehow (I haven't figured out how, yet), the system python's distutils-replaced setuptools egg was borked. I had to revert the change and re-easy_install distribute (which worked afterwards). The stuff I see when this happens: Downloading http://pypi.python.org/packages/2.5/s/setuptools/setuptools-0.6c9-py2.5.egg usage: -c [options] requirement_or_url ... or: -c --help error: invalid command 'easy_install' Traceback (most recent call last): File "bootstrap.py", line 49, in ) == 0 AssertionError - This weekend our buildbot (which I upgraded to use distribute) spit chunks all over me. Nothing builds anymore since I upgraded. Similar error as above. This morning I did the "replace egg and easy_install distribute" dance in the buildbot slave's python dir and now it barfs with: Creating directory '/srv/buildbot.thehealthagency.com/parts/slave1/doctree.schemas_slave1/build/bin'. Creating directory '/srv/buildbot.thehealthagency.com/parts/slave1/doctree.schemas_slave1/build/parts'. Creating directory '/srv/buildbot.thehealthagency.com/parts/slave1/doctree.schemas_slave1/build/develop-eggs'. While: Bootstraping. An internal error occured due to a bug in either zc.buildout or in a recipe being used: Traceback (most recent call last): File "/tmp/tmpTuZdd7/zc.buildout-1.4.1-py2.5.egg/zc/buildout/buildout.py", line 1659, in main getattr(buildout, command)(args) File "/tmp/tmpTuZdd7/zc.buildout-1.4.1-py2.5.egg/zc/buildout/buildout.py", line 363, in bootstrap ws.require('zc.buildout') File "build/bdist.linux-i686/egg/pkg_resources.py", line 621, in require needed = self.resolve(parse_requirements(requirements)) File "build/bdist.linux-i686/egg/pkg_resources.py", line 519, in resolve raise DistributionNotFound(req) # XXX put more info here DistributionNotFound: setuptools I haven't been strangled yet by my colleagues for switching everything over. I'm convinced we have to do it. But I'd better fix it soon... Any idea what these kinds of errors mean? I can't believe I'm the only one. It happened on my osx laptop and on the linux server and within a buildout-installed buildbot. Why oh why hasn't setuptools' pypi entry be transfered to distribute.... Reinout -- Reinout van Rees - reinout at vanrees.org - http://reinout.vanrees.org Software developer at http://www.thehealthagency.com "Military engineers build missiles. Civil engineers build targets" From reinout at vanrees.org Mon Oct 12 09:14:18 2009 From: reinout at vanrees.org (Reinout van Rees) Date: Mon, 12 Oct 2009 07:14:18 +0000 (UTC) Subject: [Distutils] the virtualenv-distribute mess References: <319e029f0910060402n306a992as4b9d854f7f29d546@mail.gmail.com> <319e029f0910060402m28ab35acnc9a52aefcdc6f04a@mail.gmail.com> <4ACB24AE.4000300@cryptelium.net> <4ACB2527.7010802@cryptelium.net> <319e029f0910060522l29499f46y2805ed0082d975c3@mail.gmail.com> <4ACB7D70.9060404@cryptelium.net> <4ACF1681.7050506@simplistix.co.uk> <4ACF269C.4010400@cryptelium.net> <94bdd2610910090508j6804628fy5b5d435cce0c7427@mail.gmail.com> <319e029f0910090536j480b2339t530a2f6a9267d451@mail.gmail.com> <4ACF2FC5.6090706@cryptelium.net> <4ACF4765.8020404@simplistix.co.uk> Message-ID: On 2009-10-09, Chris Withers wrote: > > Huh? Why? I just don't buy this. We've effectively had a bdfl pronouncement > that setuptools is dead. distribute 0.6 *is* now the "live supported > setuptools" so any project that relies on the package called "setuptools" > should now be using the "distribute" distribution... Time for the BDFL to flip some ownership bits for setuptools in pypi. That would be the number one way out of this mess. I'm losing many otherwise-productive hours this way. Reinout -- Reinout van Rees - reinout at vanrees.org - http://reinout.vanrees.org Software developer at http://www.thehealthagency.com "Military engineers build missiles. Civil engineers build targets" From reinout at vanrees.org Mon Oct 12 09:21:31 2009 From: reinout at vanrees.org (Reinout van Rees) Date: Mon, 12 Oct 2009 07:21:31 +0000 (UTC) Subject: [Distutils] distribute and buildout: best practices? References: <94bdd2610910070453t63e7e4c4h1c9bf6ca567fa48f@mail.gmail.com> <4ACF1999.4080001@simplistix.co.uk> Message-ID: On 2009-10-09, Chris Withers wrote: > Reinout van Rees wrote: >> I'm still not 100% sure whether it is safe to put "distribute" in the >> install_requires list of a package right now, however. > > As with setuptools, why do you think you need to? Namespace packages. You get warnings "hey, no setuptools dependency" once you use namespaces. Note that if I depend on distribute, I also get warnings about not having a setuptools depencency sometimes... :-) Reinout -- Reinout van Rees - reinout at vanrees.org - http://reinout.vanrees.org Software developer at http://www.thehealthagency.com "Military engineers build missiles. Civil engineers build targets" From ben+python at benfinney.id.au Mon Oct 12 09:32:12 2009 From: ben+python at benfinney.id.au (Ben Finney) Date: Mon, 12 Oct 2009 18:32:12 +1100 Subject: [Distutils] the virtualenv-distribute mess References: <319e029f0910060402n306a992as4b9d854f7f29d546@mail.gmail.com> <319e029f0910060402m28ab35acnc9a52aefcdc6f04a@mail.gmail.com> <4ACB24AE.4000300@cryptelium.net> <4ACB2527.7010802@cryptelium.net> <319e029f0910060522l29499f46y2805ed0082d975c3@mail.gmail.com> <4ACB7D70.9060404@cryptelium.net> <4ACF1681.7050506@simplistix.co.uk> <4ACF269C.4010400@cryptelium.net> <94bdd2610910090508j6804628fy5b5d435cce0c7427@mail.gmail.com> <319e029f0910090536j480b2339t530a2f6a9267d451@mail.gmail.com> <4ACF2FC5.6090706@cryptelium.net> <4ACF4765.8020404@simplistix.co.uk> Message-ID: <87tyy5qd8j.fsf@benfinney.id.au> Reinout van Rees writes: > Time for the BDFL to flip some ownership bits for setuptools in pypi. You appear to think the BDFL has powers outside the Python language and extending to third-party projects that are independent. Regardless of what I think of Setuptools or Distribute, I definitely am *very* glad the Python BDFL does not have the power to make a change to a third-party project as you're suggesting. -- \ ?I was once walking through the forest alone and a tree fell | `\ right in front of me, and I didn't hear it.? ?Steven Wright | _o__) | Ben Finney From reinout at vanrees.org Mon Oct 12 09:34:31 2009 From: reinout at vanrees.org (Reinout van Rees) Date: Mon, 12 Oct 2009 07:34:31 +0000 (UTC) Subject: [Distutils] distribute or Distribute Message-ID: I'm probably overlooking it somewhere in the documentation: should I specify a distribute dependency as "distribute" or "Distribute"? I'm used to lowercase so that's what I'm using till now, but I've spotted a Capital Letter somewhere... So which one is preferred? Reinout -- Reinout van Rees - reinout at vanrees.org - http://reinout.vanrees.org Software developer at http://www.thehealthagency.com "Military engineers build missiles. Civil engineers build targets" From regebro at gmail.com Mon Oct 12 09:35:12 2009 From: regebro at gmail.com (Lennart Regebro) Date: Mon, 12 Oct 2009 09:35:12 +0200 Subject: [Distutils] the virtualenv-distribute mess In-Reply-To: References: <319e029f0910060402m28ab35acnc9a52aefcdc6f04a@mail.gmail.com> <4ACB24AE.4000300@cryptelium.net> <4ACB2527.7010802@cryptelium.net> <319e029f0910060522l29499f46y2805ed0082d975c3@mail.gmail.com> <4ACB7D70.9060404@cryptelium.net> <4ACF1681.7050506@simplistix.co.uk> <4ACF269C.4010400@cryptelium.net> <94bdd2610910090508j6804628fy5b5d435cce0c7427@mail.gmail.com> Message-ID: <319e029f0910120035s1589b864vae51844339d4ecfb@mail.gmail.com> 2009/10/12 Reinout van Rees : > - When using buildout, I get lots of warnings. Me too, but those went a way when I re-ran the bootstrap... -- Lennart Regebro: Python, Zope, Plone, Grok http://regebro.wordpress.com/ +33 661 58 14 64 From regebro at gmail.com Mon Oct 12 09:47:48 2009 From: regebro at gmail.com (Lennart Regebro) Date: Mon, 12 Oct 2009 09:47:48 +0200 Subject: [Distutils] the virtualenv-distribute mess In-Reply-To: References: <319e029f0910060522l29499f46y2805ed0082d975c3@mail.gmail.com> <4ACB7D70.9060404@cryptelium.net> <4ACF1681.7050506@simplistix.co.uk> <4ACF269C.4010400@cryptelium.net> <94bdd2610910090508j6804628fy5b5d435cce0c7427@mail.gmail.com> <319e029f0910090536j480b2339t530a2f6a9267d451@mail.gmail.com> <4ACF2FC5.6090706@cryptelium.net> <4ACF4765.8020404@simplistix.co.uk> Message-ID: <319e029f0910120047j1c5202c0g41ec0ac6921d614@mail.gmail.com> 2009/10/12 Reinout van Rees : > Time for the BDFL to flip some ownership bits for setuptools in pypi. ?That > would be the number one way out of this mess. If there had been a reasonably possibility of the setuptools project actually being transferred to somebody, then this would have been done. -- Lennart Regebro: Python, Zope, Plone, Grok http://regebro.wordpress.com/ +33 661 58 14 64 From reinout at vanrees.org Mon Oct 12 10:09:32 2009 From: reinout at vanrees.org (Reinout van Rees) Date: Mon, 12 Oct 2009 08:09:32 +0000 (UTC) Subject: [Distutils] the virtualenv-distribute mess References: <319e029f0910060402n306a992as4b9d854f7f29d546@mail.gmail.com> <319e029f0910060402m28ab35acnc9a52aefcdc6f04a@mail.gmail.com> <4ACB24AE.4000300@cryptelium.net> <4ACB2527.7010802@cryptelium.net> <319e029f0910060522l29499f46y2805ed0082d975c3@mail.gmail.com> <4ACB7D70.9060404@cryptelium.net> <4ACF1681.7050506@simplistix.co.uk> <4ACF269C.4010400@cryptelium.net> <94bdd2610910090508j6804628fy5b5d435cce0c7427@mail.gmail.com> <319e029f0910090536j480b2339t530a2f6a9267d451@mail.gmail.com> <4ACF2FC5.6090706@cryptelium.net> <4ACF4765.8020404@simplistix.co.uk> <87tyy5qd8j.fsf@benfinney.id.au> Message-ID: On 2009-10-12, Ben Finney wrote: > Reinout van Rees writes: > >> Time for the BDFL to flip some ownership bits for setuptools in pypi. > > You appear to think the BDFL has powers outside the Python language and > extending to third-party projects that are independent. > > Regardless of what I think of Setuptools or Distribute, I definitely am > *very* glad the Python BDFL does not have the power to make a change to > a third-party project as you're suggesting. I just read the rest of the thread on python dev. Yes, I should probably be glad it did not happen. OTOH, grumbl ... horrible breakage ... essential piece of infrastructure ... allowed to persist.... I'm pretty grumpy right now. Reinout -- Reinout van Rees - reinout at vanrees.org - http://reinout.vanrees.org Software developer at http://www.thehealthagency.com "Military engineers build missiles. Civil engineers build targets" From hanno at hannosch.eu Mon Oct 12 10:22:03 2009 From: hanno at hannosch.eu (Hanno Schlichting) Date: Mon, 12 Oct 2009 10:22:03 +0200 Subject: [Distutils] distribute or Distribute In-Reply-To: References: Message-ID: <5cae42b20910120122h88c364aj23b7ee8274e887fd@mail.gmail.com> On Mon, Oct 12, 2009 at 9:34 AM, Reinout van Rees wrote: > I'm probably overlooking it somewhere in the documentation: should I specify a > distribute dependency as "distribute" or "Distribute"? > > I'm used to lowercase so that's what I'm using till now, but I've spotted a > Capital Letter somewhere... ?So which one is preferred? Please use all lowercase. I think we referred to the the distribution with a capital D in the beginning, but decided it's easier to make it all lowercase in the end. Hanno From hanno at hannosch.eu Mon Oct 12 10:29:56 2009 From: hanno at hannosch.eu (Hanno Schlichting) Date: Mon, 12 Oct 2009 10:29:56 +0200 Subject: [Distutils] the virtualenv-distribute mess In-Reply-To: References: <319e029f0910060402m28ab35acnc9a52aefcdc6f04a@mail.gmail.com> <4ACB24AE.4000300@cryptelium.net> <4ACB2527.7010802@cryptelium.net> <319e029f0910060522l29499f46y2805ed0082d975c3@mail.gmail.com> <4ACB7D70.9060404@cryptelium.net> <4ACF1681.7050506@simplistix.co.uk> <4ACF269C.4010400@cryptelium.net> <94bdd2610910090508j6804628fy5b5d435cce0c7427@mail.gmail.com> Message-ID: <5cae42b20910120129r3a56c7bcje772625f25a3c36c@mail.gmail.com> Hi. On Mon, Oct 12, 2009 at 9:10 AM, Reinout van Rees wrote: > - When using buildout, I get lots of warnings. ?The 1.4.2 isn't out yet, but I > ?also won't update all old projects' pinned zc.buildout version so I'm stuck > ?with warnings for a time. Note that buildout is only really supported with the distribute 0.6.4 release and needs the latest bootstrap from http://nightly.ziade.org/bootstrap.py. We fixed a variety of issues with buildout over the weekend. It seems to work with zc.buildout 1.4.1 now, though. Hanno From floris.bruynooghe at gmail.com Mon Oct 12 11:01:15 2009 From: floris.bruynooghe at gmail.com (Floris Bruynooghe) Date: Mon, 12 Oct 2009 10:01:15 +0100 Subject: [Distutils] PEP 390 - new format from setup.cfg In-Reply-To: <94bdd2610910111517u2fd0b136p44958f7faa46e974@mail.gmail.com> References: <94bdd2610910111517u2fd0b136p44958f7faa46e974@mail.gmail.com> Message-ID: <20091012090115.GA31472@laurie.devork> On Mon, Oct 12, 2009 at 12:17:10AM +0200, Tarek Ziad? wrote: > this is the PEP for setup.cfg, as requested : > http://www.python.org/dev/peps/pep-0390 > > Please comment, <, <=, >=, > and python_hexversion are not mentioned, is this intentional? (I'm not sure on what the result of feedback on this was) Secondly, where are spaces allowed in the section headers? I'd be tempted to write: [metadata: os_machine == 'i386'] But in all your examples you never have spaces after the ':'. Would it be worth being explicit about this? Regards Floris -- Debian GNU/Linux -- The Power of Freedom www.debian.org | www.gnu.org | www.kernel.org From mihamina at gulfsat.mg Mon Oct 12 11:00:22 2009 From: mihamina at gulfsat.mg (Rakotomandimby Mihamina) Date: Mon, 12 Oct 2009 12:00:22 +0300 Subject: [Distutils] distribution packaging from Distribute In-Reply-To: <4AC9D3AF.9070902@gulfsat.mg> References: <4AC8689C.2010401@gulfsat.mg> <94bdd2610910041323n7ae6e543g814e5f51eceac469@mail.gmail.com> <4AC96466.3020506@gulfsat.mg> <94bdd2610910050401s36c0bc03ud85e81ff67ae2c7b@mail.gmail.com> <4AC9D3AF.9070902@gulfsat.mg> Message-ID: <4AD2F026.7010707@gulfsat.mg> 10/05/2009 02:08 PM, Rakotomandimby Mihamina: >> I have planned to do it on my side once for debian package using a >> recipe that was building the debian tree once the buildout was made, >> but this work >> was not finished. I can send you what I have if it can help you > Please, do. > [Up] -- Architecte Informatique chez Blueline/Gulfsat: Administration Systeme, Recherche & Developpement +261 34 29 155 34 From ziade.tarek at gmail.com Mon Oct 12 11:27:34 2009 From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=) Date: Mon, 12 Oct 2009 11:27:34 +0200 Subject: [Distutils] the virtualenv-distribute mess In-Reply-To: References: <319e029f0910060402m28ab35acnc9a52aefcdc6f04a@mail.gmail.com> <4ACB24AE.4000300@cryptelium.net> <4ACB2527.7010802@cryptelium.net> <319e029f0910060522l29499f46y2805ed0082d975c3@mail.gmail.com> <4ACB7D70.9060404@cryptelium.net> <4ACF1681.7050506@simplistix.co.uk> <4ACF269C.4010400@cryptelium.net> <94bdd2610910090508j6804628fy5b5d435cce0c7427@mail.gmail.com> Message-ID: <94bdd2610910120227p16e3510bi53d0abda0b551db3@mail.gmail.com> On Mon, Oct 12, 2009 at 9:10 AM, Reinout van Rees wrote: > On 2009-10-09, Tarek Ziad? wrote: >> On Fri, Oct 9, 2009 at 2:03 PM, kiorky wrote: >>> AND no, virtualenv must continue to provide setuptools, backward >>> compatibility, you know? >> >> The *whole* point of Distribute 0.6.x is to be backward compatible, meaning >> that if virtualenv switch to it, you will not even notice it. > > For something that's supposed to be a drop-in replacement, it sure makes it > self noticeable. ?Note that I'm not holding that against distribute! ?The mess > is of setuptools' making. > > - When using buildout, I get lots of warnings. ?The 1.4.2 isn't out yet, but I > ?also won't update all old projects' pinned zc.buildout version so I'm stuck > ?with warnings for a time. Please try with your current zc.buildout, we've also fixed compatiblity with older zc.buildout versions yesterday so it should work. > > Any idea what these kinds of errors mean? ?I can't believe I'm the only one. > It happened on my osx laptop and on the linux server and within a > buildout-installed buildbot. Do you happen to have a shared eggs directory ? if it's the case, all buildouts that uses it need to be re-bootstrapped with the distribute boostrap Tarek From ziade.tarek at gmail.com Mon Oct 12 11:45:56 2009 From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=) Date: Mon, 12 Oct 2009 11:45:56 +0200 Subject: [Distutils] PEP 390 - new format from setup.cfg In-Reply-To: References: <94bdd2610910111517u2fd0b136p44958f7faa46e974@mail.gmail.com> Message-ID: <94bdd2610910120245i2075d2d4n6bf1b7a44ae9d219@mail.gmail.com> On Mon, Oct 12, 2009 at 2:29 AM, Ian Bicking wrote: > The grammar in "Context-dependant sections" indicates possible EXPR > values. ?Because the "in" operator is supported, I would assume that > tuples should also be allowed. "in" here is restricted to string. It was added so we could write things like: 'linux' in sys_platform (where sys_platform can be linux2) I'll add a note on that. > > One aspect of the current setup.cfg is that it supports multiple > sections, for different setup.py commands or components. ?This gives a > kind of namespacing. ?I assume, but it isn't that specified, that any > section (not just "metadata") will be parsed the same way? I guess yes, even if I don't see a use case yet for that. > > Presumably setup.py will just contain an empty call to setup()? ?(I > assume at least setup.py will be allowed for, for backward > compatibility, even if it is not required.) No because we might need to define extra options in setup() that are not part of the metadata, like what is required the for the sdist command (package_data, scripts, package, etc) > > I believe this does not include the concept of extra requirements. > Possibly it could as a side-effect of some variable available to the > language. ?Like: > > ?[metadata:'xmlrpc' in features] > ?requires = lxml > > Sets and the & operator could be useful for this. How would you define/provide "features" here ? > > The way variables are handled is unclear. ?Presumably all variables > are cumulative. ?But given: > > ?requires = Foo > ?requires = Bar > > What is the value of "requires"? ?A list, a value with newlines? > Presumably you could also do: > > ?requires = Foo > ? ? ?Bar > > Anyway, we're diverging from INI semantics at this point, so it needs > to be specified how it works. Right, this needs to be defined. I would be in favor of the latter, to stay ConfigParser compatible. > > Is there a way to eliminate values? ?Right now, for instance, if you > have "[build] home = /foo" in a distutils.cfg, there's no way to unset > that. ?I'd like to see this functionality included. ?Perhaps to delete > a specific value, but the simpler case of deleting a variable is > really all I want. Do you have a syntax in mind for this ? Tarek From ziade.tarek at gmail.com Mon Oct 12 11:49:27 2009 From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=) Date: Mon, 12 Oct 2009 11:49:27 +0200 Subject: [Distutils] PEP 390 - new format from setup.cfg In-Reply-To: <20091012090115.GA31472@laurie.devork> References: <94bdd2610910111517u2fd0b136p44958f7faa46e974@mail.gmail.com> <20091012090115.GA31472@laurie.devork> Message-ID: <94bdd2610910120249i40f06b2fpc775a5be22703269@mail.gmail.com> On Mon, Oct 12, 2009 at 11:01 AM, Floris Bruynooghe wrote: > On Mon, Oct 12, 2009 at 12:17:10AM +0200, Tarek Ziad? wrote: >> this is the PEP for setup.cfg, as requested : >> http://www.python.org/dev/peps/pep-0390 >> >> Please comment, > > <, <=, >=, > and python_hexversion are not mentioned, is this > intentional? ?(I'm not sure on what the result of feedback on this > was) That was the result of the feedback we had (see 'limitations' section) for clarity. The idea is to leave a string "python_version", and remove '>', '<' with no python_hexversion. Then, the day PEP 386 is accepted, we turn "python_version" into a Version() object and we introduce '>', '<' and al. > > Secondly, where are spaces allowed in the section headers? ?I'd be > tempted to write: > > ? ?[metadata: os_machine == 'i386'] > > But in all your examples you never have spaces after the ':'. ?Would > it be worth being explicit about this? Good point, I'll add this. Tarek From ziade.tarek at gmail.com Mon Oct 12 11:51:56 2009 From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=) Date: Mon, 12 Oct 2009 11:51:56 +0200 Subject: [Distutils] distribution packaging from Distribute In-Reply-To: <4AD2F026.7010707@gulfsat.mg> References: <4AC8689C.2010401@gulfsat.mg> <94bdd2610910041323n7ae6e543g814e5f51eceac469@mail.gmail.com> <4AC96466.3020506@gulfsat.mg> <94bdd2610910050401s36c0bc03ud85e81ff67ae2c7b@mail.gmail.com> <4AC9D3AF.9070902@gulfsat.mg> <4AD2F026.7010707@gulfsat.mg> Message-ID: <94bdd2610910120251n225da904u45b9278349ce1757@mail.gmail.com> On Mon, Oct 12, 2009 at 11:00 AM, Rakotomandimby Mihamina wrote: > 10/05/2009 02:08 PM, Rakotomandimby Mihamina: >>> >>> I have planned to do it on my side once for debian package using a >>> recipe that was building the debian tree once the buildout was made, >>> but this work >>> was not finished. I can send you what I have if it can help you >> >> Please, do. >> > > [Up] > Sorry: http://bitbucket.org/tarek/z3crecipedebian > -- > ? ? ?Architecte Informatique chez Blueline/Gulfsat: > ? Administration Systeme, Recherche & Developpement > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? +261 34 29 155 34 > _______________________________________________ > Distutils-SIG maillist ?- ?Distutils-SIG at python.org > http://mail.python.org/mailman/listinfo/distutils-sig > -- Tarek Ziad? | http://ziade.org |????????????! |???????????? From greno at verizon.net Mon Oct 12 17:01:15 2009 From: greno at verizon.net (Gerry Reno) Date: Mon, 12 Oct 2009 11:01:15 -0400 Subject: [Distutils] dev versions In-Reply-To: References: Message-ID: <4AD344BB.2050900@verizon.net> Reinout van Rees wrote: > On 2009-10-08, Ian Bicking wrote: >> So after creating, say, version 0.3.1, I always mark a package as 0.3.2dev. >> But this is annoying, you might never create a version 0.3.2 (e.g., 0.4 >> might be the next level). So, it would be better to use something like >> 0.3.1~dev. What is considered best practice for this? Ideally something >> that works with both Setuptools and the upcoming Distribute version spec. > > > a) Where's the annoyment exactly? It is easy to change and it's a release-time > decision anyway. > > b) In a previous discussion on a zope mailinglist (about using '0' for this > purpose, which was pretty much shot down for the zope toolkit because of > the problems attached to it), someone mentioned adding '+svn' to the > previous version number. So from 0.3.1 to 0.3.1+svn. Apparently that > sorts it behind 0.3.1. You could try something like that. The poster > mentioned it as a debian standard. > > > Reinout > > I just caught the tail end of this thread so please excuse if you may have covered this. With regard to package version and release strings here is something that I have been using successfully that generates proper lexical package ordering for tarballs, debian (.deb) and redhat (.rpm) packaging. The same patterns should work just as well for any other packaging system: We use the following for the various VERSION-RELEASE strings: examples use: bzr as version control system but you can substitute hg, git, svn, rcs, or any other type of vcs rpm as packaging system Development: version="5.0.0" release="0.00012345" (where 00012345 is the revision number from your vcs in ZERO-PADDED FIELD OF 8) tarball: foo-5.0.0-0.00012345.tar.gz rpm: foo-5.0.0-0.00012345.noarch.rpm Alpha: version="5.0.0" release="0.alpha1.00123456" tarball: foo-5.0.0-0.alpha1.00123456.tar.gz rpm: foo-5.0.0-0.alpha1.00123456.noarch.rpm Beta: version="5.0.0" release="0.beta1.00234567" tarball: foo-5.0.0-0.beta1.00234567.tar.gz rpm: foo-5.0.0-0.beta1.00234567.noarch.rpm Release Candidate: version="5.0.0" release="0.rc1" tarball: foo-5.0.0-0.rc1.tar.gz rpm: foo-5.0.0-0.rc1.noarch.rpm Release Candidate code fix: version="5.0.0" release="0.rc1.00345678" tarball: foo-5.0.0-0.rc1.00345678.tar.gz rpm: foo-5.0.0-0.rc1.00345678.noarch.rpm Final Release: version="5.0.0" release="1" tarball: foo-5.0.0-1.tar.gz rpm: foo-5.0.0-1.noarch.rpm Final Release code fix: version="5.0.0" release="2" tarball: foo-5.0.0-2.tar.gz rpm: foo-5.0.0-2.noarch.rpm Notice that lexical ordering is proper in all cases. Even where the alpha, beta, rc pre-releases may be followed by a bug fix revision. The key is that all pre-releases (alpha, beta, etc.) have a release string starting with "0." and the first production release has a release string of "1". This is how the lexical ordering is kept for all packaging types (tarballs, rpm, deb). Regards, Gerry From mwhapples at aim.com Mon Oct 12 17:55:01 2009 From: mwhapples at aim.com (Michael Whapples) Date: Mon, 12 Oct 2009 16:55:01 +0100 Subject: [Distutils] Problem with distribute and converting tests for python3 In-Reply-To: <319e029f0910112124k4f89fe29qf2c02b24b3c8eb0d@mail.gmail.com> References: <4AD23DE1.2050903@aim.com> <319e029f0910111343l164bdacaw7ab552f863be935c@mail.gmail.com> <4AD24F73.1000105@aim.com> <319e029f0910112124k4f89fe29qf2c02b24b3c8eb0d@mail.gmail.com> Message-ID: <4AD35155.1080301@aim.com> On 12/10/09 05:24, Lennart Regebro wrote: > 2009/10/11 Michael Whapples: > > [...] > >> ). Also anyway what would the reason be for a user to run tests on the >> binary module, I (or another developer in the cases of modified versions) >> should have run the tests before distributing the binary package. >> > Yes, but if something breaks it can be good to run the tests locally > to see if something in the environments breaks them. Also tests can be > useful as a sort of documentation. > In that case I would ask them to download the source distribution and run the tests there, they probably will want the source distribution as I may need to ask them to apply a patch and test it out (they have the environment I didn't foresee/have). > I question how unit tests in a binary package are like a sort of documentation as the user will not be able to see the test source code as it will all be in .pyc files. Doctests are a different matter, but I am on about unit tests. >> OK, to possible solutions: >> 1. Distribute offers the keyword argument "convert_2to3_doctests", may be it >> could have one to give test directory, and then .py files in that could be >> converted. >> > Well, the test directory is test/test*.py. > > >> 2. As setuptools automatically includes test/test*.py, why can't distribute >> actually compile these files as well (I question whether tests in a test >> directory should be prefixed with test, but I don't have a really strong >> view should people disagree). >> > The problem is that these files doesn't end up in the binary > distribution and hence 2to3 can't be run on them. > Bad wording on my part, I said compile when I meant convert, so point two should be "As distribute will include test/test*.py, why can't distribute run 2to3 on these files and so convert my tests and so then be able to run tests from the test command". This probably would need distribute to create a separate directory (eg. test_3k) and then run tests from the new location to achieve this. > Michael Whapples From sklein at cpcug.org Mon Oct 12 17:44:16 2009 From: sklein at cpcug.org (Stanley A. Klein) Date: Mon, 12 Oct 2009 11:44:16 -0400 (EDT) Subject: [Distutils] PEP 390 - new format from setup.cfg In-Reply-To: References: Message-ID: <44288.71.163.197.194.1255362256.squirrel@www.cpcug.org> On Mon, October 12, 2009 12:24 am, Tarek Ziad? wrote: ------------------------------ > > Message: 2 > Date: Mon, 12 Oct 2009 00:17:10 +0200 > From: Tarek Ziad? > To: distutils-sig at python.org > Subject: [Distutils] PEP 390 - new format from setup.cfg > Hey > > this is the PEP for setup.cfg, as requested : > http://www.python.org/dev/peps/pep-0390 > > Please comment, > > Tarek > It isn't clear in the new format how one would set up a bdist_rpm for an SE-Linux system (such as Fedora or Centos) that has required pyc and pyo files to be included in the package. In the past this has been done using entries in setup.cfg. Stan Klein From ianb at colorstudy.com Mon Oct 12 18:10:05 2009 From: ianb at colorstudy.com (Ian Bicking) Date: Mon, 12 Oct 2009 11:10:05 -0500 Subject: [Distutils] PEP 390 - new format from setup.cfg In-Reply-To: <94bdd2610910120245i2075d2d4n6bf1b7a44ae9d219@mail.gmail.com> References: <94bdd2610910111517u2fd0b136p44958f7faa46e974@mail.gmail.com> <94bdd2610910120245i2075d2d4n6bf1b7a44ae9d219@mail.gmail.com> Message-ID: On Mon, Oct 12, 2009 at 4:45 AM, Tarek Ziad? wrote: > On Mon, Oct 12, 2009 at 2:29 AM, Ian Bicking wrote: >> The grammar in "Context-dependant sections" indicates possible EXPR >> values. ?Because the "in" operator is supported, I would assume that >> tuples should also be allowed. > > "in" here is restricted to string. It was added so we could write things like: > > 'linux' in sys_platform ?(where sys_platform can be linux2) If you don't have tuples or <, >, etc, it seems like something like "Python version 2.6 or higher" is hard to express. You'd have to enumerate 2.6, 2.7, and speculate on 2.8 and 2.9. > I'll add a note on that. > >> >> One aspect of the current setup.cfg is that it supports multiple >> sections, for different setup.py commands or components. ?This gives a >> kind of namespacing. ?I assume, but it isn't that specified, that any >> section (not just "metadata") will be parsed the same way? > > I guess yes, even if I don't see a use case yet for that. One use case for the current setup.cfg is for extensions, like generating Sphynx documentation. Those extensions can sometimes take quite a lot of options, so they are best when partitioned into their own section. I'm also not sure whether [metadata] is intended to have extensible variables, or a fixed set of variables. >> Presumably setup.py will just contain an empty call to setup()? ?(I >> assume at least setup.py will be allowed for, for backward >> compatibility, even if it is not required.) > > No because we might need to define extra options in setup() > that are not part of the metadata, like what is required the for the > sdist command > (package_data, scripts, package, etc) OK, so setup.cfg is for generating PKG-INFO, but installing a package still involves running setup.py and some maybe-declarative code in there. >> I believe this does not include the concept of extra requirements. >> Possibly it could as a side-effect of some variable available to the >> language. ?Like: >> >> ?[metadata:'xmlrpc' in features] >> ?requires = lxml >> >> Sets and the & operator could be useful for this. > > How would you define/provide "features" here ? I'm not sure. With Setuptools the equivalent is "extras", like: setup(name='foo', extras_require={'xmlrpc': ['lxml']}) Then if you do "easy_install foo[xmlrpc]" (or somehow require foo[xmlrpc]) it will also install/require lxml. If I was to say why this is a problem, I think it's because the default is that there are no features for a package. So someone naively wants your package and installs it, but they don't get all the features they thought the package provided (and which the package actually *does* provide, but just doesn't have the prerequisite libraries for -- since the package ships with the xmlrpc code, it's just not working xmlrpc code). There's also not a good way of seeing what extras are provided, or what their purpose is. So library authors avoid the issue entirely and don't factor out requirements into specific extras/features. If there was a set of "default" extras maybe it would be more workable. I.e., "easy_install foo" installs foo with all its default extras, and "easy_install foo[]" installs foo without any extras (or you put the specific extras you want in []). Anyway, the way extra requirements are serialized to disk is requires.txt, with: default_req1 default_req2 [extra] extra_req1 ... So, since the result involves multiple sections it wouldn't naturally map to what you are proposing. >> The way variables are handled is unclear. ?Presumably all variables >> are cumulative. ?But given: >> >> ?requires = Foo >> ?requires = Bar >> >> What is the value of "requires"? ?A list, a value with newlines? >> Presumably you could also do: >> >> ?requires = Foo >> ? ? ?Bar >> >> Anyway, we're diverging from INI semantics at this point, so it needs >> to be specified how it works. > > Right, this needs to be defined. I would be in favor of the latter, to > stay ConfigParser > compatible. You still have to define how options are combined from multiple sections, and what the resulting value is from the API. That is, if you have: [metadata] requires = Foo Bar [metadata:python_version == '2.4' or python_version== '2.3' or python_version=='2.2'] requires = BackwardCompat Then what is the ultimate value of "requires"? Is it "Foo\nBar\nBackwardCompat" or ["Foo", "Bar", "BackwardCompat"], or [Requirement.parse("Foo"), Requirement.parse('Bar"), Requirement.parse("BackwardCompat")]. And given other variables (ones that perhaps distribute doesn't even know about) how are they combined? >> Is there a way to eliminate values? ?Right now, for instance, if you >> have "[build] home = /foo" in a distutils.cfg, there's no way to unset >> that. ?I'd like to see this functionality included. ?Perhaps to delete >> a specific value, but the simpler case of deleting a variable is >> really all I want. > > Do you have a syntax in mind for this ? Well, the way I added more meta-operations in Paste Deploy (which uses ConfigParser) was to mess with the variable names. So maybe: [metadata:python_version=="2.5"] del requires = Foo It might have a value to delete a specific value (or line) from that "requires" variable, or if empty it would clear that variable. Since ConfigParser doesn't order the variables, it would delete them from "earlier" sections, not the current section. This requires defining what makes a section "earlier". Or maybe it would delete "Foo" from whatever the value of requires ends up being, or if you do "del requires =" would make that section provide the only value for requires. Another thought: one thing distutils does but sometimes confuses me is that it has boolean options (options which take no values, sometimes with an option and an anti-option, e.g., --feature-foo and --no-feature-foo) which turn into a single configuration variable with a boolean value. How that is supposed to work seems apropos (it's not strictly syntax, but I don't feel like I can judge the syntax without considering the full semantics of the configuration file, because the semantics drive the actual use cases which should in turn be used to judge the syntax). -- Ian Bicking | http://blog.ianbicking.org | http://topplabs.org/civichacker From ziade.tarek at gmail.com Mon Oct 12 18:34:03 2009 From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=) Date: Mon, 12 Oct 2009 18:34:03 +0200 Subject: [Distutils] PEP 390 - new format from setup.cfg In-Reply-To: References: <94bdd2610910111517u2fd0b136p44958f7faa46e974@mail.gmail.com> <94bdd2610910120245i2075d2d4n6bf1b7a44ae9d219@mail.gmail.com> Message-ID: <94bdd2610910120934p672cb041xbb5cd36ab64e54b5@mail.gmail.com> On Mon, Oct 12, 2009 at 6:10 PM, Ian Bicking wrote: > If you don't have tuples or <, >, etc, it seems like something like > "Python version 2.6 or higher" is hard to express. ?You'd have to > enumerate 2.6, 2.7, and speculate on 2.8 and 2.9. python_version not in '2.3,2.4,2.5' (it's not optimal, but enough I guess, until PEP 386 is accepted) > One use case for the current setup.cfg is for extensions, like > generating Sphynx documentation. ?Those extensions can sometimes take > quite a lot of options, so they are best when partitioned into their > own section. ?I'm also not sure whether [metadata] is intended to have > extensible variables, or a fixed set of variables. The variables are fixed so it works with a vanilla python and doesn't require anything else to run. Having extensible variables would break that idea and would just make people move code they use to have in setup.py in another place imho. >> No because we might need to define extra options in setup() >> that are not part of the metadata, like what is required the for the >> sdist command >> (package_data, scripts, package, etc) > > OK, so setup.cfg is for generating PKG-INFO, but installing a package > still involves running setup.py and some maybe-declarative code in > there. > Yes, this is restricted to the Metadata fields. >> How would you define/provide "features" here ? > > I'm not sure. ?With Setuptools the equivalent is "extras", like: > > setup(name='foo', > ? ?extras_require={'xmlrpc': ['lxml']}) > > Then if you do "easy_install foo[xmlrpc]" (or somehow require > foo[xmlrpc]) it will also install/require lxml. > [...] > ?... > > So, since the result involves multiple sections it wouldn't naturally > map to what you are proposing. That could be done I think as long as the extras are not calculated by third-party code. Let's say, we define an 'extras' variable, which value can be provided when interpreting setup.cfg. If could be described like this : [metadata:'xmlrpc' in extras] requires = lxml The part that is unclear to me is how to list the extras a setup.cfg file has. >> Right, this needs to be defined. I would be in favor of the latter, to >> stay ConfigParser >> compatible. > > You still have to define how options are combined from multiple > sections, and what the resulting value is from the API. ?That is, if > you have: > > ? ?[metadata] > ? ?requires = Foo > ? ? ? ?Bar > > ? ?[metadata:python_version == '2.4' or python_version== '2.3' or > python_version=='2.2'] > ? ?requires = BackwardCompat > > Then what is the ultimate value of "requires"? ?Is it > "Foo\nBar\nBackwardCompat" or ["Foo", "Bar", "BackwardCompat"], or > [Requirement.parse("Foo"), Requirement.parse('Bar"), > Requirement.parse("BackwardCompat")]. > > And given other variables (ones that perhaps distribute doesn't even > know about) how are they combined? Right, this needs clarification. In any case, I think using multiline is a bad idea because it'll break RCF 232 compatibility for the long_description field. So i've updated the PEP with ',' separator. >>> Is there a way to eliminate values? ?Right now, for instance, if you >>> have "[build] home = /foo" in a distutils.cfg, there's no way to unset >>> that. ?I'd like to see this functionality included. ?Perhaps to delete >>> a specific value, but the simpler case of deleting a variable is >>> really all I want. >> >> Do you have a syntax in mind for this ? > > Well, the way I added more meta-operations in Paste Deploy (which uses > ConfigParser) was to mess with the variable names. ?So maybe: > > ? ?[metadata:python_version=="2.5"] > ? ?del requires = Foo > > It might have a value to delete a specific value (or line) from that > "requires" variable, or if empty it would clear that variable. ?Since > ConfigParser doesn't order the variables, it would delete them from > "earlier" sections, not the current section. ?This requires defining > what makes a section "earlier". ?Or maybe it would delete "Foo" from > whatever the value of requires ends up being, or if you do "del > requires =" would make that section provide the only value for > requires. That makes sense. It completes what we can do. Also, I've discussed with MAL about this PEP, and he came up with the idea that the conditions should be pushed in PKG-INFO as well. See the latest change in PEP 390. Tarek From regebro at gmail.com Mon Oct 12 19:16:29 2009 From: regebro at gmail.com (Lennart Regebro) Date: Mon, 12 Oct 2009 19:16:29 +0200 Subject: [Distutils] Problem with distribute and converting tests for python3 In-Reply-To: <4AD35155.1080301@aim.com> References: <4AD23DE1.2050903@aim.com> <319e029f0910111343l164bdacaw7ab552f863be935c@mail.gmail.com> <4AD24F73.1000105@aim.com> <319e029f0910112124k4f89fe29qf2c02b24b3c8eb0d@mail.gmail.com> <4AD35155.1080301@aim.com> Message-ID: <319e029f0910121016m65eafb7k99f18b2081536d8b@mail.gmail.com> 2009/10/12 Michael Whapples : > In that case I would ask them to download the source distribution and run > the tests there, they probably will want the source distribution as I may > need to ask them to apply a patch and test it out (they have the environment > I didn't foresee/have). Well, that's one step extra, but fine. > I question how unit tests in a binary package are like a sort of > documentation as the user will not be able to see the test source code as it > will all be in .pyc files. A binary distribution will include the .py files as well. "Binary" is different from "source" in that extensions are already compiled. You don't need to make binary distributions for anything else than Windows, and then only if you have C-extensions. Modules without c-extensions are best distributed as a source distribution. >> The problem is that these files doesn't end up in the binary >> distribution and hence 2to3 can't be run on them. > Bad wording on my part, I said compile when I meant convert, so point two > should be "As distribute will include test/test*.py, why can't distribute > run 2to3 on these files and so convert my tests and so then be able to run > tests from the test command". This probably would need distribute to create > a separate directory (eg. test_3k) and then run tests from the new location > to achieve this. No, that wasn't bad wording on your part, I understood perfectly, I think. The problem is that these files doesn't end up in the binary distribution and hence 2to3 isn't run on them as the 2to3 conversion is done in build_py. -- Lennart Regebro: Python, Zope, Plone, Grok http://regebro.wordpress.com/ +33 661 58 14 64 From mwhapples at aim.com Mon Oct 12 19:44:14 2009 From: mwhapples at aim.com (Michael Whapples) Date: Mon, 12 Oct 2009 18:44:14 +0100 Subject: [Distutils] Problem with distribute and converting tests for python3 In-Reply-To: <319e029f0910121016m65eafb7k99f18b2081536d8b@mail.gmail.com> References: <4AD23DE1.2050903@aim.com> <319e029f0910111343l164bdacaw7ab552f863be935c@mail.gmail.com> <4AD24F73.1000105@aim.com> <319e029f0910112124k4f89fe29qf2c02b24b3c8eb0d@mail.gmail.com> <4AD35155.1080301@aim.com> <319e029f0910121016m65eafb7k99f18b2081536d8b@mail.gmail.com> Message-ID: <4AD36AEE.8040603@aim.com> OK, I didn't realise the .py files are included in a binary distribution, I understood it to be that the .pyc file was enough and so had followed through that the .py file wouldn't be included in a binary distribution like it is typically done in java (java tends not to include the .java source files in a binary .jar file, only the .class files). I did warn you that I may be making some assumptions/hold views due to my java programming. This leads to some more general questions about python and distributing modules (eg. why do we need binary distributions (bdist_egg, bdist, etc) when a source distribution should be fine and do all needed, why does python distribute source and compiled files in a binary distribution (what does the source files add in such a case), etc). Anyway back to the actual topic. My main question of the possible solution is, why can't we have a way of specifying extra directories to run 2to3 on? Tests are an example (well at least the way I view tests) of when this could be useful, it may be useful in other cases (not thought of an example). Michael Whapples On 12/10/09 18:16, Lennart Regebro wrote: > 2009/10/12 Michael Whapples: > >> In that case I would ask them to download the source distribution and run >> the tests there, they probably will want the source distribution as I may >> need to ask them to apply a patch and test it out (they have the environment >> I didn't foresee/have). >> > Well, that's one step extra, but fine. > > >> I question how unit tests in a binary package are like a sort of >> documentation as the user will not be able to see the test source code as it >> will all be in .pyc files. >> > A binary distribution will include the .py files as well. "Binary" is > different from "source" in that extensions are already compiled. You > don't need to make binary distributions for anything else than > Windows, and then only if you have C-extensions. Modules without > c-extensions are best distributed as a source distribution. > > >>> The problem is that these files doesn't end up in the binary >>> distribution and hence 2to3 can't be run on them. >>> > >> Bad wording on my part, I said compile when I meant convert, so point two >> should be "As distribute will include test/test*.py, why can't distribute >> run 2to3 on these files and so convert my tests and so then be able to run >> tests from the test command". This probably would need distribute to create >> a separate directory (eg. test_3k) and then run tests from the new location >> to achieve this. >> > No, that wasn't bad wording on your part, I understood perfectly, I > think. The problem is that these files doesn't end up in the binary > distribution and hence 2to3 isn't run on them as the 2to3 conversion > is done in build_py. > > From greno at verizon.net Mon Oct 12 20:05:16 2009 From: greno at verizon.net (Gerry Reno) Date: Mon, 12 Oct 2009 14:05:16 -0400 Subject: [Distutils] Problem with distribute and converting tests for python3 In-Reply-To: <4AD36AEE.8040603@aim.com> References: <4AD23DE1.2050903@aim.com> <319e029f0910111343l164bdacaw7ab552f863be935c@mail.gmail.com> <4AD24F73.1000105@aim.com> <319e029f0910112124k4f89fe29qf2c02b24b3c8eb0d@mail.gmail.com> <4AD35155.1080301@aim.com> <319e029f0910121016m65eafb7k99f18b2081536d8b@mail.gmail.com> <4AD36AEE.8040603@aim.com> Message-ID: <4AD36FDC.20602@verizon.net> Michael Whapples wrote: > OK, I didn't realise the .py files are included in a binary > distribution, I understood it to be that the .pyc file was enough and > so had followed through that the .py file wouldn't be included in a > binary distribution like it is typically done in java (java tends not > to include the .java source files in a binary .jar file, only the > .class files). I did warn you that I may be making some > assumptions/hold views due to my java programming. This leads to some > more general questions about python and distributing modules (eg. why > do we need binary distributions (bdist_egg, bdist, etc) when a source > distribution should be fine and do all needed, why does python > distribute source and compiled files in a binary distribution (what > does the source files add in such a case), etc). > > Anyway back to the actual topic. My main question of the possible > solution is, why can't we have a way of specifying extra directories > to run 2to3 on? Tests are an example (well at least the way I view > tests) of when this could be useful, it may be useful in other cases > (not thought of an example). > > Michael Whapples > On 12/10/09 18:16, Lennart Regebro wrote: >> 2009/10/12 Michael Whapples: >>> In that case I would ask them to download the source distribution >>> and run >>> the tests there, they probably will want the source distribution as >>> I may >>> need to ask them to apply a patch and test it out (they have the >>> environment >>> I didn't foresee/have). >> Well, that's one step extra, but fine. >> >>> I question how unit tests in a binary package are like a sort of >>> documentation as the user will not be able to see the test source >>> code as it >>> will all be in .pyc files. >> A binary distribution will include the .py files as well. "Binary" is >> different from "source" in that extensions are already compiled. You >> don't need to make binary distributions for anything else than >> Windows, and then only if you have C-extensions. Modules without >> c-extensions are best distributed as a source distribution. >> >>>> The problem is that these files doesn't end up in the binary >>>> distribution and hence 2to3 can't be run on them. >>> Bad wording on my part, I said compile when I meant convert, so >>> point two >>> should be "As distribute will include test/test*.py, why can't >>> distribute >>> run 2to3 on these files and so convert my tests and so then be able >>> to run >>> tests from the test command". This probably would need distribute to >>> create >>> a separate directory (eg. test_3k) and then run tests from the new >>> location >>> to achieve this. >> No, that wasn't bad wording on your part, I understood perfectly, I >> think. The problem is that these files doesn't end up in the binary >> distribution and hence 2to3 isn't run on them as the 2to3 conversion >> is done in build_py. >> > Binary distribution is a packaging type such as an rpm or a deb file usually for a particular distro. Not about whether source or various compilation products are or are not included in these binary packages. You can write an install that requires no build and and no source and just moves precompiled files into position. Regards, Gerry From regebro at gmail.com Mon Oct 12 20:37:52 2009 From: regebro at gmail.com (Lennart Regebro) Date: Mon, 12 Oct 2009 20:37:52 +0200 Subject: [Distutils] Problem with distribute and converting tests for python3 In-Reply-To: <4AD36AEE.8040603@aim.com> References: <4AD23DE1.2050903@aim.com> <319e029f0910111343l164bdacaw7ab552f863be935c@mail.gmail.com> <4AD24F73.1000105@aim.com> <319e029f0910112124k4f89fe29qf2c02b24b3c8eb0d@mail.gmail.com> <4AD35155.1080301@aim.com> <319e029f0910121016m65eafb7k99f18b2081536d8b@mail.gmail.com> <4AD36AEE.8040603@aim.com> Message-ID: <319e029f0910121137i1d5ecddfg74fcecbc53f7dcbd@mail.gmail.com> 2009/10/12 Michael Whapples : > This > leads to some more general questions about python and distributing modules > (eg. why do we need binary distributions (bdist_egg, bdist, etc) when a > source distribution should be fine and do all needed You don't. As mentioned, binary distributions are generally not needed, unless you have C-extensions that need compiling, and then usually only on Windows, as Windows doesn't include a compiler. > why does python > distribute source and compiled files in a binary distribution (what does the > source files add in such a case), etc). Well, why not? Excluding them serves no purpose. And including them means you can read the source, debug, etc. > Anyway back to the actual topic. My main question of the possible solution > is, why can't we have a way of specifying extra directories to run 2to3 on? It's a pretty special case. I agree that for the test/test*.py case we should find some sort of solution for backwards compatibility reason, but otherwise I'm afraid I don't really see the point. There is already support for including several source directories, it seems to me that this is just that, another source directory, in this case with tests. I haven't tried that with 2to3, but if that doesn't work, that's definitely a bug. In your case, if you absolutely want to have your code like this, I'd guess subclassing the test command to run 2to3 on the tests first would be the best way. > Tests are an example (well at least the way I view tests) of when this could > be useful, it may be useful in other cases (not thought of an example). -- Lennart Regebro: Python, Zope, Plone, Grok http://regebro.wordpress.com/ +33 661 58 14 64 From pje at telecommunity.com Mon Oct 12 22:25:52 2009 From: pje at telecommunity.com (P.J. Eby) Date: Mon, 12 Oct 2009 16:25:52 -0400 Subject: [Distutils] Setuptools 0.6c10 release imminent; please test Message-ID: <20091012202600.5B4B13A4068@sparrow.telecommunity.com> I've just updated the setuptools trunk and branch with a large number of bug fixes, preparing to release 0.6c10. These fixes include fixes for all the setuptools bugs currently marked "in-progress" or "testing" on the setuptools bug tracker as of this moment, and I'd like your help in ensuring that the new version is ready for release. Major updates and fixes include: * Fix for Vista UAC errors running easy_install.exe or other "installer-looking" executables * Fix for errors launching 64-bit Windows Python * Fix for the Python 2.6.3 build_ext API change * Support for the most recent Sourceforge download link insanity * Support for SVN 1.6 * Stop crashing on certain types of HTTP error * Stop re-trying URLs that already failed retrieval once * Fixes for various dependency management problems such as looping builds, re-downloading packages already present on sys.path (but not in a registered "site" directory), and randomly preferring local -f packages over local installed packages * Prevent lots of spurious "already imported from another path" warnings (e.g. when pkg_resources is imported late) * Ensure C libraries (as opposed to extensions) are also built when doing bdist_egg * Fixed running the "test" command under Python 2.6+ (0.7a1 trunk only) Other changes: * Misc. documentation fixes * Improved Jython support * Fewer warnings under Python 2.6+ * Warn when 'packages' uses paths instead of package names (because it causes other problems, like spurious "already imported" warnings) * Stop using /usr/bin/sw_vers on Mac OS (replaced w/'platform' module calls) You can install the updated version using easy_install (or pip!), asking for setuptools==dev (for the 0.7a1 trunk) or setuptools==dev06 (for the 0.6c10 development branch). But before you do that, be sure to uninstall Distribute completely. In particular, please note that these changes are NOT a merge from Distribute; upon review, many of the tracker-submitted patches used as a basis for forking were incorrect, incomplete, introduced new bugs, or were not addressing the root causes. (E.g., one of the changes I made fixes three superficially unrelated issues in the setuptools bug tracker.) Even if some of the change descriptions are the same in Distribute, many of the fixes are implemented quite differently. From ziade.tarek at gmail.com Mon Oct 12 22:55:57 2009 From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=) Date: Mon, 12 Oct 2009 22:55:57 +0200 Subject: [Distutils] Setuptools 0.6c10 release imminent; please test In-Reply-To: <20091012202600.5B4B13A4068@sparrow.telecommunity.com> References: <20091012202600.5B4B13A4068@sparrow.telecommunity.com> Message-ID: <94bdd2610910121355k72376650nf7af5d5af26faeda@mail.gmail.com> On Mon, Oct 12, 2009 at 10:25 PM, P.J. Eby wrote: > But before you do that, be sure to uninstall Distribute completely. ?In > particular, please note that these changes are NOT a merge from Distribute; > upon review, many of the tracker-submitted patches used as a basis for > forking were incorrect, incomplete, introduced new bugs, or were not > addressing the root causes. ?(E.g., one of the changes I made fixes three > superficially unrelated issues in the setuptools bug tracker.) ?Even if some > of the change descriptions are the same in Distribute, many of the fixes are > implemented quite differently Mmm, I can't let you say that, that's not true. I've read all your commit diffs, and I've recognized similar fixes we've done in Distribute. So this assertion you are making is pretty vague. But thanks anyway for this release, we will look at your fixes and backport them if required in Distribute. Tarek From zooko at zooko.com Mon Oct 12 23:49:41 2009 From: zooko at zooko.com (Zooko Wilcox-O'Hearn) Date: Mon, 12 Oct 2009 15:49:41 -0600 Subject: [Distutils] Setuptools 0.6c10 release imminent; please test In-Reply-To: <20091012202600.5B4B13A4068@sparrow.telecommunity.com> References: <20091012202600.5B4B13A4068@sparrow.telecommunity.com> Message-ID: <100C9CFB-E0CF-4F05-B193-E2AE6165910B@zooko.com> On Monday,2009-10-12, at 14:25 , P.J. Eby wrote: > (E.g., one of the changes I made fixes three superficially > unrelated issues in the setuptools bug tracker.) That would be: http://bugs.python.org/setuptools/issue17 and http://bugs.python.org/setuptools/issue65 Right? What was the third? Thanks, Zooko From tseaver at palladion.com Tue Oct 13 00:14:00 2009 From: tseaver at palladion.com (Tres Seaver) Date: Mon, 12 Oct 2009 18:14:00 -0400 Subject: [Distutils] the virtualenv-distribute mess In-Reply-To: <4ACF4765.8020404@simplistix.co.uk> References: <319e029f0910060402n306a992as4b9d854f7f29d546@mail.gmail.com> <319e029f0910060402m28ab35acnc9a52aefcdc6f04a@mail.gmail.com> <4ACB24AE.4000300@cryptelium.net> <4ACB2527.7010802@cryptelium.net> <319e029f0910060522l29499f46y2805ed0082d975c3@mail.gmail.com> <4ACB7D70.9060404@cryptelium.net> <4ACF1681.7050506@simplistix.co.uk> <4ACF269C.4010400@cryptelium.net> <94bdd2610910090508j6804628fy5b5d435cce0c7427@mail.gmail.com> <319e029f0910090536j480b2339t530a2f6a9267d451@mail.gmail.com> <4ACF2FC5.6090706@cryptelium.net> <4ACF4765.8020404@simplistix.co.uk> Message-ID: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Chris Withers wrote: > Huh? Why? I just don't buy this. We've effectively had a bdfl > pronouncement that setuptools is dead. distribute 0.6 *is* now the "live > supported setuptools" so any project that relies on the package called > "setuptools" should now be using the "distribute" distribution... Are you being wilfully ignorant here, Chris? he BDFL doesn't "control" setuptools, because it is a *third party package*. Tres. - -- =================================================================== Tres Seaver +1 540-429-0999 tseaver at palladion.com Palladion Software "Excellence by Design" http://palladion.com -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iEYEARECAAYFAkrTqiQACgkQ+gerLs4ltQ5RqwCglAgztZ59zvpYYEz3KAZ08oYv bFUAn2wlowNTnuze6QQuJbXopCZ0bOdH =fgvv -----END PGP SIGNATURE----- From greg.ewing at canterbury.ac.nz Tue Oct 13 00:25:26 2009 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Tue, 13 Oct 2009 11:25:26 +1300 Subject: [Distutils] the virtualenv-distribute mess In-Reply-To: References: <319e029f0910060402n306a992as4b9d854f7f29d546@mail.gmail.com> <319e029f0910060402m28ab35acnc9a52aefcdc6f04a@mail.gmail.com> <4ACB24AE.4000300@cryptelium.net> <4ACB2527.7010802@cryptelium.net> <319e029f0910060522l29499f46y2805ed0082d975c3@mail.gmail.com> <4ACB7D70.9060404@cryptelium.net> <4ACF1681.7050506@simplistix.co.uk> <4ACF269C.4010400@cryptelium.net> <94bdd2610910090508j6804628fy5b5d435cce0c7427@mail.gmail.com> <319e029f0910090536j480b2339t530a2f6a9267d451@mail.gmail.com> <4ACF2FC5.6090706@cryptelium.net> <4ACF4765.8020404@simplistix.co.uk> <87tyy5qd8j.fsf@benfinney.id.au> Message-ID: <4AD3ACD6.6000906@canterbury.ac.nz> Reinout van Rees wrote: > OTOH, grumbl ... horrible breakage ... essential piece of infrastructure ... > allowed to persist.... I'm pretty grumpy right now. I'd just like to point out that if anyone finds setuptools to be "essential" to them, it's because they've chosen to make themselves dependent on it. It's always been known that it's a third-party package with no official support, and one that does some pretty screwy things to boot. It's always been on the cards that it would all fall down in a crashing heap one day. -- Greg From pje at telecommunity.com Tue Oct 13 00:35:19 2009 From: pje at telecommunity.com (P.J. Eby) Date: Mon, 12 Oct 2009 18:35:19 -0400 Subject: [Distutils] the virtualenv-distribute mess In-Reply-To: References: <319e029f0910060402n306a992as4b9d854f7f29d546@mail.gmail.com> <319e029f0910060402m28ab35acnc9a52aefcdc6f04a@mail.gmail.com> <4ACB24AE.4000300@cryptelium.net> <4ACB2527.7010802@cryptelium.net> <319e029f0910060522l29499f46y2805ed0082d975c3@mail.gmail.com> <4ACB7D70.9060404@cryptelium.net> <4ACF1681.7050506@simplistix.co.uk> <4ACF269C.4010400@cryptelium.net> <94bdd2610910090508j6804628fy5b5d435cce0c7427@mail.gmail.com> <319e029f0910090536j480b2339t530a2f6a9267d451@mail.gmail.com> <4ACF2FC5.6090706@cryptelium.net> <4ACF4765.8020404@simplistix.co.uk> <87tyy5qd8j.fsf@benfinney.id.au> Message-ID: <20091012223531.7FCF63A4068@sparrow.telecommunity.com> At 08:09 AM 10/12/2009 +0000, Reinout van Rees wrote: >OTOH, grumbl ... horrible breakage ... essential piece of infrastructure ... >allowed to persist.... I'm pretty grumpy right now. Relax, take a deep breath, and then easy_install setuptools==dev or setuptools==dev06. ;-) It turned out to be easier than I expected to clean up pretty much all the outstanding (i.e. in the tracker) bugs this weekend, although that was mainly due to avoiding overhead by reading all the bugs together and making the changes more-or-less simultaneously. In this case, it was prompted mainly by the unfortunate, policy-breaking change to Python 2.6.3, but I'm thinking that maybe a quarterly Bug Weekend and subsequent maintenance release for setuptools might not be a bad thing. From ziade.tarek at gmail.com Tue Oct 13 00:50:17 2009 From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=) Date: Tue, 13 Oct 2009 00:50:17 +0200 Subject: [Distutils] the virtualenv-distribute mess In-Reply-To: <20091012223531.7FCF63A4068@sparrow.telecommunity.com> References: <4ACF269C.4010400@cryptelium.net> <94bdd2610910090508j6804628fy5b5d435cce0c7427@mail.gmail.com> <319e029f0910090536j480b2339t530a2f6a9267d451@mail.gmail.com> <4ACF2FC5.6090706@cryptelium.net> <4ACF4765.8020404@simplistix.co.uk> <87tyy5qd8j.fsf@benfinney.id.au> <20091012223531.7FCF63A4068@sparrow.telecommunity.com> Message-ID: <94bdd2610910121550v2e73311dh144285e9ab835fe7@mail.gmail.com> On Tue, Oct 13, 2009 at 12:35 AM, P.J. Eby wrote: > At 08:09 AM 10/12/2009 +0000, Reinout van Rees wrote: >> >> OTOH, grumbl ... horrible breakage ... essential piece of infrastructure >> ... >> allowed to persist.... ?I'm pretty grumpy right now. > > Relax, take a deep breath, and then easy_install setuptools==dev or > setuptools==dev06. ?;-) Nah it's ok, we've worked with Reinout and cleared out most of his problems today. (btw, no human can hold his breath for one year long ;) ) We are probably going to ship Distribute 0.6.5 next week after we will finish the review of your single gigantic commit on the setuptools package, so we can integrate more things if we can. Tarek. From ssteinerx at gmail.com Tue Oct 13 01:28:18 2009 From: ssteinerx at gmail.com (ssteinerX@gmail.com) Date: Mon, 12 Oct 2009 19:28:18 -0400 Subject: [Distutils] Setuptools 0.6c10 release imminent; please test In-Reply-To: <20091012202600.5B4B13A4068@sparrow.telecommunity.com> References: <20091012202600.5B4B13A4068@sparrow.telecommunity.com> Message-ID: On Oct 12, 2009, at 4:25 PM, P.J. Eby wrote: > But before you do that, be sure to uninstall Distribute completely Damn if I understand this...such a long time waiting for all these bug fixes...so little action, so much angst... Then that all that effort going into the Distribute fork, so much effort expended carefully working around so that Distribute would be a bug-fixed, drop-in replacement for the orphaned setuptools. Then suddenly out of the blue, everything is fixed in setuptools 06c10 and we must "uninstall Distribute completely" get these fixes. If the changes you made really were superior in any way, I trust that Tarek and co. are smart, and egoless enough to see that and will make those changes part of Distribute. Too little, too late, no thanks, I'll just be sticking with Distribute from now on. S From pje at telecommunity.com Tue Oct 13 01:48:38 2009 From: pje at telecommunity.com (P.J. Eby) Date: Mon, 12 Oct 2009 19:48:38 -0400 Subject: [Distutils] Setuptools 0.6c10 release imminent; please test In-Reply-To: References: <20091012202600.5B4B13A4068@sparrow.telecommunity.com> Message-ID: <20091012234846.BA7DF3A4068@sparrow.telecommunity.com> At 07:28 PM 10/12/2009 -0400, ssteinerX at gmail.com wrote: >we must "uninstall Distribute completely" get these fixes. That's Distribute's doing, not mine. As I understand it, their package includes a 'setuptools' package, and if it's on your sys.path, then installing the new version of setuptools will be a no-op. If they hadn't done that, there'd be no problem. See the Distribute documentation. In any case, the update is not intended for people who are happy to have Distribute, but the people who are unhappy about having to switch, or deal with its workarounds... or just wish the whole discussion would go away. >Then suddenly out of the blue It may appear sudden to you, if you haven't been reading Python-Dev. There's been quite a bit of discussion about an urgent bug that Tarek introduced in Python 2.6.3. It's mainly because of that bug that I took the time to go ahead and get a bunch of other pending bugs cleaned up and checked in. From alex.gronholm at nextday.fi Tue Oct 13 01:52:54 2009 From: alex.gronholm at nextday.fi (=?ISO-8859-1?Q?Alex_Gr=F6nholm?=) Date: Tue, 13 Oct 2009 02:52:54 +0300 Subject: [Distutils] Setuptools 0.6c10 release imminent; please test In-Reply-To: References: <20091012202600.5B4B13A4068@sparrow.telecommunity.com> Message-ID: <4AD3C156.8090903@nextday.fi> ssteinerX at gmail.com kirjoitti: > > On Oct 12, 2009, at 4:25 PM, P.J. Eby wrote: > >> But before you do that, be sure to uninstall Distribute completely > > Damn if I understand this...such a long time waiting for all these bug > fixes...so little action, so much angst... > > Then that all that effort going into the Distribute fork, so much > effort expended carefully working around so that Distribute would be a > bug-fixed, drop-in replacement for the orphaned setuptools. > > Then suddenly out of the blue, everything is fixed in setuptools 06c10 > and we must "uninstall Distribute completely" get these fixes. Makes sense because you can't have two "setuptools" packages in sys.path. > > > If the changes you made really were superior in any way, I trust that > Tarek and co. are smart, and egoless enough to see that and will make > those changes part of Distribute. One such fix has already made it to distribute's repo. > > > Too little, too late, no thanks, I'll just be sticking with Distribute > from now on. Several developers and an open development process vs a lone coder with a closed codebase? That's not really a choice at all... > > S > > _______________________________________________ > Distutils-SIG maillist - Distutils-SIG at python.org > http://mail.python.org/mailman/listinfo/distutils-sig From ssteinerx at gmail.com Tue Oct 13 02:14:59 2009 From: ssteinerx at gmail.com (ssteinerX@gmail.com) Date: Mon, 12 Oct 2009 20:14:59 -0400 Subject: [Distutils] Setuptools 0.6c10 release imminent; please test In-Reply-To: <20091012234846.BA7DF3A4068@sparrow.telecommunity.com> References: <20091012202600.5B4B13A4068@sparrow.telecommunity.com> <20091012234846.BA7DF3A4068@sparrow.telecommunity.com> Message-ID: On Oct 12, 2009, at 7:48 PM, P.J. Eby wrote: > At 07:28 PM 10/12/2009 -0400, ssteinerX at gmail.com wrote: >> we must "uninstall Distribute completely" get these fixes. > > That's Distribute's doing, not mine. As I understand it, their > package includes a 'setuptools' package, and if it's on your > sys.path, then installing the new version of setuptools will be a no- > op. If they hadn't done that, there'd be no problem. See the > Distribute documentation. > > In any case, the update is not intended for people who are happy to > have Distribute, but the people who are unhappy about having to > switch, or deal with its workarounds... or just wish the whole > discussion would go away. > >> Then suddenly out of the blue > > It may appear sudden to you, if you haven't been reading Python- > Dev. There's been quite a bit of discussion about an urgent bug > that Tarek introduced in Python 2.6.3. It's mainly because of that > bug that I took the time to go ahead and get a bunch of other > pending bugs cleaned up and checked in. No, I've been reading Python-Dev right along. Yes, Python 2.6.3 included a change that broke a year-old, orphaned product, in severe need of bug fixes that lots of people just happen to depend on. The fall-down was in the testing done before the Python release and I'm sure more testing will be done in that area before the 2.6.4 bugfix release. I find it kind of comical, and a little pathetic, that you think you can just whip out a bug fix after a year of frustration and everone's just going to forget history and sign up for more of the same. As Alex said in response to my previous message: > Too little, too late, no thanks, I'll just be sticking with > Distribute from now on. Several developers and an open development process vs a lone coder with a closed codebase? That's not really a choice at all... Sorry, it doesn't look like anyone wants to play with you any more; you can just keep your ball. Maybe you could submit some patches into the open Distribute process. Now _that_ would be helpful. S From bradallen137 at gmail.com Tue Oct 13 03:59:03 2009 From: bradallen137 at gmail.com (Brad Allen) Date: Mon, 12 Oct 2009 20:59:03 -0500 Subject: [Distutils] Setuptools 0.6c10 release imminent; please test In-Reply-To: <20091012234846.BA7DF3A4068@sparrow.telecommunity.com> References: <20091012202600.5B4B13A4068@sparrow.telecommunity.com> <20091012234846.BA7DF3A4068@sparrow.telecommunity.com> Message-ID: <4957f1ef0910121859i45161ee2wdecebb1d53fc4782@mail.gmail.com> On Mon, Oct 12, 2009 at 6:48 PM, P.J. Eby wrote: > At 07:28 PM 10/12/2009 -0400, ssteinerX at gmail.com wrote: >> >> we must "uninstall Distribute completely" get these fixes. > > That's Distribute's doing, not mine. ?As I understand it, their package > includes a 'setuptools' package, and if it's on your sys.path, then > installing the new version of setuptools will be a no-op. ?If they hadn't > done that, there'd be no problem. ?See the Distribute documentation. > > In any case, the update is not intended for people who are happy to have > Distribute, but the people who are unhappy about having to switch, or deal > with its workarounds... ?or just wish the whole discussion would go away. That's me, along with the other Python users who need this distutils/setuptools/buildout to work in production and cannot afford the kind of disruption which 'distribute' will introduce. I'm glad to see the distribute project in progress with an open development process but would rather it be used for a new Python 3.x PyPi, where all the new libraries can include the appropriate support for distribute (with no need to clean up a vast library of existing PyPi packages). Sorry to jump into the middle of a heated debate which I've not followed closely, but I wanted to provide some feedback that I (and my employer) am glad to have fixes to setuptools because that provides critical infrastructure to us today, and if Python 2.6.3 or 2.6.4 contains important fixes, we don't want to get stranded in 2.6.2. From regebro at gmail.com Tue Oct 13 06:27:56 2009 From: regebro at gmail.com (Lennart Regebro) Date: Tue, 13 Oct 2009 06:27:56 +0200 Subject: [Distutils] Setuptools 0.6c10 release imminent; please test In-Reply-To: References: <20091012202600.5B4B13A4068@sparrow.telecommunity.com> Message-ID: <319e029f0910122127t3696ffefvd5470be7a1a78f7@mail.gmail.com> 2009/10/13 ssteinerX at gmail.com : > Then suddenly out of the blue, everything is fixed in setuptools 06c10 and > we must "uninstall Distribute completely" get these fixes. Well yes. We knew this would happen sooner or later. It's not a problem. We all know it would be better if PJE would let others help him, but he doesn't want to, and that is his right and completely up to him. Obviously any fixes he does is going to end up in Distribute 0.6.x as well. They might not be done in exactly the same ways, unless he feels like explaining why his way is better for each of the fixes. But they will be fixed in Distribute, and in fact have for the most part already been fixed. 2009/10/13 P.J. Eby : > That's Distribute's doing, not mine. True. On the other hand, Distribute is your doing, so you aren't completely blameless. :-) > In any case, the update is not intended for people who are happy to have > Distribute, but the people who are unhappy about having to switch, or deal > with its workarounds... Absolutely true. > or just wish the whole discussion would go away. Well, I sure everyone wants the discussion to go away, but that probably only happens the day you let others help you. 2009/10/13 Brad Allen : > That's me, along with the other Python users who need this > distutils/setuptools/buildout to work in production and cannot afford > the kind of disruption which 'distribute' will introduce. Disruption? > I'm glad to > see the distribute project in progress with an open development > process but would rather it be used for a new Python 3.x PyPi, where > all the new libraries can include the appropriate support for > distribute (with no need to clean up a vast library of existing PyPi > packages). If there isn't a migration path where you can support both from the same codebase, Python 3 is dead. So that is not an option. > Sorry to jump into the middle of a heated debate which I've not > followed closely, but I wanted to provide some feedback that I (and my > employer) am glad to have fixes to setuptools because that provides > critical infrastructure to us today, and if Python 2.6.3 or 2.6.4 > contains important fixes, we don't want to get stranded in 2.6.2. There we completely agree. That PJE releases a new version of setuptools is good. It's not a problem for anyone, and the more software the merrier. -- Lennart Regebro: Python, Zope, Plone, Grok http://regebro.wordpress.com/ +33 661 58 14 64 From david.lyon at preisshare.net Tue Oct 13 08:18:11 2009 From: david.lyon at preisshare.net (David Lyon) Date: Tue, 13 Oct 2009 02:18:11 -0400 Subject: [Distutils] Alternate static metadata PEP submission... In-Reply-To: <94bdd2610910111517u2fd0b136p44958f7faa46e974@mail.gmail.com> References: <94bdd2610910111517u2fd0b136p44958f7faa46e974@mail.gmail.com> Message-ID: On Mon, 12 Oct 2009 00:17:10 +0200, Tarek Ziad? wrote: > Hey > > this is the PEP for setup.cfg, as requested : > http://www.python.org/dev/peps/pep-0390 > > Please comment, > > Tarek Can I have assistance submitting an alternate PEP? Generally speaking you never answer any email from me but in this case I ask that you assist in respect to having a fair and due process. I have subscribed to peps at python.org but havent received an email back. I have posted it to peps at python.org. Here is my PEP... PEP: 0 Title: An improved python application/package setup system using setup.info Version: 1.0 Last-Modified: 13-10-2009 Author: David Lyon Status: Draft Type: Standards Track Content-Type: text/x-rst Created: 13-Oct-2009 Discussions-To: david.lyon at preisshare.net Preamble -------- This document describes an improved way for application and package developers to specify information for installation to different systems compared to traditional method that uses setup.py. Abstract -------- This PEP proposes a change to the way that applications and libraries are installed in python, by using a new file called setup.info rather than setup.py. To hold installation information. For example: [setup] name = artistflair version = 1.2 description = artistflair is a package written in spare time to show colour variations. [dependencies] packages = pyopengl objectrelationalmapper >= 1.6 [dependencies python<=2.4] packages = lxml >= 2.5 [dependencies windows] packages = win32com [dependencies linux] packages = pyodbc [Application] scripts = artisticflairgui.py [configuration] files = artisticflair.conf [datafiles] files = artisticdb.db directory = db [Documentation] directory = docs url = index.html [preinstall] script = checksystem.py [postinstall linux] script = builddocumentation.py [uninstall] script = uninstallcleanup.py Rationale --------- Traditionally, python has used a setup.py to perform package installation. Metadata has been hardcoded into setup.py which is then passed to distutils and the installation is performed. Time has revealed limitations with the setup.py system and this pep proposes a more flexible and powerful system for installation using setup information stored as parameters within a setup.info file. The advantages of this new approach are: - a clean way to handle variations due to different operating systems. - a clean way to specify package dependencies (including conditional dependencies based on operating system and python interpretor version) - a way to handle pre and post installation processing. - a way to handle package/application uninstallation A boilerplate setup.py will be provided to accompany setup.info file on platforms where distutils isn't patched. This would provide backward compatability with previous python versions. The setup.info file will be based on the the standard python ConfigParser file format format. Adantages for using setup.info ------------------------------ 1) It's easier to extend past the simplest use case 2) It's more developer friendly. 3) Provides an easier way to specify package dependencies 4) Provides a portable setup system across platforms 5) Easier to integrate within the development build process where handling version numbers can often be a challenge. Section Name Construction ------------------------- The setup.info file is built as a standard configparser format file. Section names are built from a combination of a specifier then an option operating system name and finally an optional python version specifier. The following shows a very powerful way to build conditional installation for different operating systems and platforms that starts from very simple and can be extended with effort into the very complex. Most developers will choose the simplest option. Sections are built in the following format specifier [operating_system] [pythonversion] A simple real world example might be: [dependencies] packages = objectmapper >= 2.5 [dependencies windows] packages = win32com [dependencies linux] packages = pyodbc What this means is that for all platforms, package objectmapper >= version 2.5 is required. However on windows, there's an additional depency that win32com is required. On Linux or Mac, win32com won't be required. Then, on linux the package pyodbc is required, but not on the mac or windows. Therefore the above list gives us the following dependency matrix: platform windows linux mac objectmapper yes yes yes win32com yes no no pyodbc no yes no Thus we end up with a way of conditionally being able to specify sections that reflect the differences in underlying platforms. The specifier isn't limited to just the dependency section and can be applied to provide conditional installation on any of the following sections: dependencies preinstall postinstall scripts application documentation uninstall datafiles. Operating system is one of the following values but also may contain a version number which is used to denote the operating system: windows[-('95','98','me','2000','xp','vista','7')] mac ('9.5','10.0') linux linux[-('suse','redhat','ubuntu')] symbion wince This would provide ways to handle challenges that arrise for particular operating system versions. [postinstall windows=2000] script=registrycorrect.py preinstall linux-redhat<=6] script=youneedtoupgrade.py PythonVersion is one of the following python ('2.3','2.4','2.5','2.6','2.7','3.0,'3.1') ironpython jython So an example might be: [dependencies python<=2.4] packages = lxml Package Metadata ---------------- Traditionally, package metadata has been stored within the setup.py program file as hardcoded variables. #!/usr/bin/env python from distutils.core import setup setup(name='artisticflair', version='1.0', description='Demonstration of color usage', author='David Lyon', author_email='dlyon at noemail.pls', url='http://www.python.org', This PEP proposes moving the metadata from setup.py into setup.info to the section known as setup. [setup] name = Artisticflair version = 1.0 description = Demonstration of color usage author = David Lyon author_email = dlyon at noemail.pls url = http://www.python.org Readers may question why this information is here in addition to being in the PKG-INFO file. The reason is that it is intended that the PKG-INFO file could be created with the metadata from this section by reading the [setup] section and writing the values sequentially. In a development environment, reading a version number and updating it when the file is in configparser format is much easier than manually editing the setup.py file. Therefore, many tasks relating to the production of documentation, help files and the like will benefit greatly from the introduction of a setup.info file. The existance of a correspending PKG-INFO file will be complement the setup.info file. It's expected that the PKG-INFO file will be created from the metadata information held inside the setup.info file. This will preserve backwards compatability. Dependency Handling [dependencies] ---------------------------------- To handle package/application dependencies, it is proposed that there be a section within setup.info called 'dependencies' with a multiline list 'packages'. ie: [dependencies] packages = numpy >= 1.1 biolife > 1.5 Each package consists of a single line entry, with an optional operator ("==",">=",">") and a version string. There are many real world examples where dependencies changed based on the destination platform. So the section name can be changed so that additional dependencies can be added for specific platforms. [depencies linux] packages = pyodbc [dependencies windows python<=2.4] packages = lxml >= 2.5 Pre Installation Processing [preinstall] ---------------------------------------- It's often neccessary when doing an install to run a script to probe the system to determine if an installation is at all possible. Here's the simplest use case, specifying a script that runs to check the system: [preinstall] script = checksystem.py In the following case, different scripts are run depending on the operating system: [preinstall] script = checksystem.py [preinstall linux] script = checkmylinux.sh [preinstall mac-os/x] script = checkmymac.py [preinstall windows] script = checkmypc.bat When calling the script, a command line parameter is supplied. That parameter is destination directory to which it's calculated that installation will go to. For example: checkmysystem.py /usr/local/python2.5/lib/site-packages/artistflair-1.2 or on windows: checkmysystem.py "c:\python2.5\lib\site-packages\artistflair-1.2" If scripts return a 0 return code, installation will proceeed. If a non-zero return code is encountered, installation won't proceed and the user will be notified. Post Install Processing [postinstall] ------------------------------------- After install, that is, after the program/package directories are created and all the files are copied, it's often neccessary to go off and do initialisation of some sort. Typically, it's likely to be something such as: - generate/compile documentation (man pages or sphinx files) - migrate a users old database - remove old versions of an application - register files with the operating system To accomplish this, postinstall processing support is required. [postinstall] script = builddocumentation.py [postinstall linux] script = buildmanpages.py [postinstall mac] script = makeitlookniceforme.py Application Scripts [Application] --------------------------------- Traditionally, setup.py hasn't provided ready support for the installation of applications. This PEP proposes that the task of registering python applications becomes much simpler with the addition of an application section specifically oriented to assist developers get their applications registered on the user desktop. In simple terms, what this section is about is registering the python scripts with the window manager. On Linux - that might be KDE or Gnome, on Windows - the program manager, and on the Mac - the desktop. [Application] scripts = artisticflairgui.py The above example would cause the application to be installed in an "artistflair" folder (derived from [setup]/name) created on the window manager. Configuration files [Configuration] ----------------------------------- Most real world applications carry with them a configuration file of some sort. It is certainly not mandatory that this section exists but if it does it will provide a way to carry configuration files to their appropriate end destination. Where configuration files are actually stored is different between operating platforms. On Linux and Mac, files go to a user directory, or possibly /etc. On windows they go to \Documents and Settings\user\application data\myapp. [configuration] files = artisticflair.conf By specifying the configuration file, the system will ensure that the configuration file goes to the correct place. Data Files [Datafiles] ---------------------- In any application, data files may be an install option. This section specifies what the datafiles are and the name of the subdirectory that they will be placed in. [datafiles] files = artisticdb.dbf artisticdb.mdx authors.dbf authors.mdx directory = db Documentation ------------- Most projects carry documentation of some sort. More importantly, a developer wants to make it very easy for the user to be able to access that documentation. By specifying the documentation files in the documentation section, shortcuts to the documentation can then be created in the window manager to enable the user to easily access them. Traditionally, documentation has been built offline and kept in with the package. With the Internet, there's been a move to online documentation. The mechanism described here handles both very simply. [Documentation] directory = docs url = index.html readme = readme.txt In the above snippet, all documentation is locally stored and has been placed in a directory called 'docs'. At install time, that entire directory and subdirectores will be copied to the local system. Within the artisticflair/docs subdirectory, the main documentation file is index.html. A readme file is also provided. Both these will have a shortcut created for them within the local window manager. For online documentation, simply point the documentation url to an online link. [Documentation] url = www.python.org/docs The last example shows how to do documentation for different platforms. For example windows users may be disoriented when given screen shots to mac software, and vice-versa. There is a difference in logic with operating system and python version handling in the section names for 'documentation'. The difference is that documentation is more specific and there is usually only one copy of documentation. For example, if we are on the mac, we only want the mac documentation shown. We don't want the linux documentation to be available (unlesss we decide to do that). Therefore, the documentation section mechanism allows us to clearly seperate different documentation for different platforms. [Documentation] url = index.html [Documentation mac] url = www.nourl.at/artisticflairmac.html [Documentation linux] url = www.nourl.at/artisticflairlin.html [Documentation linux-kde] url = www.nourl.at/artisticflairlinkde.html [Documentation windows] url = www.nourl.at/artisticflairwin.html [Documentation windows-vista] url = www.nourl.at/artisticflairwinvista.html The final accomodation is the language prefix. For example to specify multilingual documentation: [Documentation] url = index.html url-es = index_es.html url-fr = index_fr.html url-de = index_de.html Uninstall --------- This section is to provide a script to do a final cleanup before uninstall takes place. [uninstall] script = dropmytables.py It's expected that setup.py will perform the actual deinstallation. Processing Sequence ------------------- This proposal suggests that setup.py be enhanced to allow reading of the metadata from setup.info as package/application install time. A boilerplate setup.py will be made available to read data from the setup.info file, before passing control to distutils to perform the package installation. Therefore, the python tradition of 'python setup.py install' can be preserved. Acknowdgements -------------- All the people on the distutils mailing list who asked for features. Copyright --------- This document has been placed in the public domain. Acknowledgements From reinout at vanrees.org Tue Oct 13 09:15:31 2009 From: reinout at vanrees.org (Reinout van Rees) Date: Tue, 13 Oct 2009 07:15:31 +0000 (UTC) Subject: [Distutils] the virtualenv-distribute mess References: <319e029f0910060402n306a992as4b9d854f7f29d546@mail.gmail.com> <319e029f0910060402m28ab35acnc9a52aefcdc6f04a@mail.gmail.com> <4ACB24AE.4000300@cryptelium.net> <4ACB2527.7010802@cryptelium.net> <319e029f0910060522l29499f46y2805ed0082d975c3@mail.gmail.com> <4ACB7D70.9060404@cryptelium.net> <4ACF1681.7050506@simplistix.co.uk> <4ACF269C.4010400@cryptelium.net> <94bdd2610910090508j6804628fy5b5d435cce0c7427@mail.gmail.com> <319e029f0910090536j480b2339t530a2f6a9267d451@mail.gmail.com> <4ACF2FC5.6090706@cryptelium.net> <4ACF4765.8020404@simplistix.co.uk> <87tyy5qd8j.fsf@benfinney.id.au> <20091012223531.7FCF63A4068@sparrow.telecommunity.com> Message-ID: On 2009-10-12, P.J. Eby wrote: > At 08:09 AM 10/12/2009 +0000, Reinout van Rees wrote: >>OTOH, grumbl ... horrible breakage ... essential piece of infrastructure ... >>allowed to persist.... I'm pretty grumpy right now. > > Relax, take a deep breath, and then easy_install setuptools==dev or > setuptools==dev06. ;-) Relax, take a deep breath, go to #distutils in irc and be amazed at the speed by which Tarek is able to fix things. I'm helping Tarek debug stuff now so that I can switch to distribute :-) Reinout -- Reinout van Rees - reinout at vanrees.org - http://reinout.vanrees.org Software developer at http://www.thehealthagency.com "Military engineers build missiles. Civil engineers build targets" From reinout at vanrees.org Tue Oct 13 09:20:16 2009 From: reinout at vanrees.org (Reinout van Rees) Date: Tue, 13 Oct 2009 07:20:16 +0000 (UTC) Subject: [Distutils] the virtualenv-distribute mess References: <319e029f0910060402m28ab35acnc9a52aefcdc6f04a@mail.gmail.com> <4ACB24AE.4000300@cryptelium.net> <4ACB2527.7010802@cryptelium.net> <319e029f0910060522l29499f46y2805ed0082d975c3@mail.gmail.com> <4ACB7D70.9060404@cryptelium.net> <4ACF1681.7050506@simplistix.co.uk> <4ACF269C.4010400@cryptelium.net> <94bdd2610910090508j6804628fy5b5d435cce0c7427@mail.gmail.com> <5cae42b20910120129r3a56c7bcje772625f25a3c36c@mail.gmail.com> Message-ID: On 2009-10-12, Hanno Schlichting wrote: > Hi. > > On Mon, Oct 12, 2009 at 9:10 AM, Reinout van Rees wrote: >> - When using buildout, I get lots of warnings. ?The 1.4.2 isn't out yet, but I >> ?also won't update all old projects' pinned zc.buildout version so I'm stuck >> ?with warnings for a time. > > Note that buildout is only really supported with the distribute 0.6.4 > release and needs the latest bootstrap from > http://nightly.ziade.org/bootstrap.py. We fixed a variety of issues > with buildout over the weekend. It seems to work with zc.buildout > 1.4.1 now, though. The problems were with the stuff from last weekend. Well, after some additional small fixes yesterday, most of the problems are indeed gone. What needs doing iirc: - Installing distribute with buildout shouldn't result in distribute doing anything with the system python's setuptools (as there are problems that way). The latest bootstrap already allows setuptools and distribute to co-exist happily in a shared eggs dir! - Checking what happens in the corner case when buildOUT sets up a buildBOT which sets up a virtualenv for a build slave which runs a buildOUT :-) Reinout -- Reinout van Rees - reinout at vanrees.org - http://reinout.vanrees.org Software developer at http://www.thehealthagency.com "Military engineers build missiles. Civil engineers build targets" From jeff at taupro.com Tue Oct 13 09:32:23 2009 From: jeff at taupro.com (Jeff Rush) Date: Tue, 13 Oct 2009 02:32:23 -0500 Subject: [Distutils] Setuptools 0.6c10 release imminent; please test In-Reply-To: References: <20091012202600.5B4B13A4068@sparrow.telecommunity.com> <20091012234846.BA7DF3A4068@sparrow.telecommunity.com> Message-ID: <4AD42D07.2020702@taupro.com> ssteinerX at gmail.com wrote: > > On Oct 12, 2009, at 7:48 PM, P.J. Eby wrote: > >> At 07:28 PM 10/12/2009 -0400, ssteinerX at gmail.com wrote: >> >> In any case, the update is not intended for people who are happy to >> have Distribute, but the people who are unhappy about having to >> switch, or deal with its workarounds... or just wish the whole >> discussion would go away. Thanks P.J. very much for this release of setuptools. I'm one of those who'd prefer not to switch to distribute until it has matured and built up a reputation for responsible stewardship of the technology. When it does, great I'm there but until then setuptools does what I need without hassle. > The fall-down was in the testing done before the Python release and I'm > sure more testing will be done in that area before the 2.6.4 bugfix > release. Not really - according to the emails on this list, the incompatibility was found _before_ the 2.6.3 release and it was decided so what let things break. More testing would _not_ have prevented this problem. >> Too little, too late, no thanks, I'll just be sticking with Distribute >> from now on. > Several developers and an open development process vs a lone coder with > a closed codebase? That's not really a choice at all... > > Sorry, it doesn't look like anyone wants to play with you any more; you > can just keep your ball. Please don't speak for others - it's rude. Only speak for yourself. -Jeff From ziade.tarek at gmail.com Tue Oct 13 10:25:37 2009 From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=) Date: Tue, 13 Oct 2009 10:25:37 +0200 Subject: [Distutils] Setuptools 0.6c10 release imminent; please test In-Reply-To: <4AD42D07.2020702@taupro.com> References: <20091012202600.5B4B13A4068@sparrow.telecommunity.com> <20091012234846.BA7DF3A4068@sparrow.telecommunity.com> <4AD42D07.2020702@taupro.com> Message-ID: <94bdd2610910130125v13073ea7r93a06946dc071c88@mail.gmail.com> On Tue, Oct 13, 2009 at 9:32 AM, Jeff Rush wrote: >> The fall-down was in the testing done before the Python release and I'm >> sure more testing will be done in that area before the 2.6.4 bugfix >> release. > > Not really - according to the emails on this list, the incompatibility > was found _before_ the 2.6.3 release and it was decided so what let > things break. ?More testing would _not_ have prevented this problem. That mistake I have done was not done on purpose so now stop with this kind of accusation immediatly. >> >> Sorry, it doesn't look like anyone wants to play with you any more; you >> can just keep your ball. > > Please don't speak for others - it's rude. ?Only speak for yourself. And you, stop to think for the others, pointing fingers and throwing mud, you are the one who is rude right now. Tarek. From ziade.tarek at gmail.com Tue Oct 13 10:36:12 2009 From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=) Date: Tue, 13 Oct 2009 10:36:12 +0200 Subject: [Distutils] Alternate static metadata PEP submission... In-Reply-To: References: <94bdd2610910111517u2fd0b136p44958f7faa46e974@mail.gmail.com> Message-ID: <94bdd2610910130136g51b559ey9e3a04123309342e@mail.gmail.com> On Tue, Oct 13, 2009 at 8:18 AM, David Lyon wrote: > On Mon, 12 Oct 2009 00:17:10 +0200, Tarek Ziad? > wrote: >> Hey >> >> this is the PEP for setup.cfg, as requested : >> http://www.python.org/dev/peps/pep-0390 >> >> Please comment, >> >> Tarek > > Can I have assistance submitting an alternate PEP? > > Generally speaking you never answer any email from me but in > this case I ask that you assist in respect to having a fair and > due process. That is not true. If you look at the distutils-archive I've answered to most of your questions. > > I have subscribed to peps at python.org but havent received an > email back. I have posted it to peps at python.org. > > Here is my PEP... Why are you doing a similar PEP, what's the point ? You've already helped a bit. I suggest that you help us finishing in the existing PEP 390 instead, by proposing changes. Tarek From david at ar.media.kyoto-u.ac.jp Tue Oct 13 10:55:36 2009 From: david at ar.media.kyoto-u.ac.jp (David Cournapeau) Date: Tue, 13 Oct 2009 17:55:36 +0900 Subject: [Distutils] Alternate static metadata PEP submission... In-Reply-To: References: <94bdd2610910111517u2fd0b136p44958f7faa46e974@mail.gmail.com> Message-ID: <4AD44088.7030406@ar.media.kyoto-u.ac.jp> Hi David, David Lyon wrote: > The setup.info file will be based on the the standard > python ConfigParser file format format. You should develop on that, and compare against the current proposal (the one written by Tarek and folks on distutils ML, PEP 390). I am in favor of a richer 'static metadata' format, and don't really like where the current PEP 390 is going, but it should be considered as the "default" against which your proposal will be compared. In particular, what's the point of ConfigParser format for a new file ? It brings very few things compared to a format written from scratch, and a lot of trouble (lack of proper variable definition, conditional and nesting, for example). In PEP 390, the rationale is backward compatibility. I also think your proposal would be stronger if you compared with existing formats (e.g. rpm .spec, haskell .cabal). Python is behind those systems on that point (from time and features POV), so a look at prior art is a good idea: http://www.haskell.org/ghc/docs/latest/html/Cabal/authors.html http://www.rpm.org/max-rpm/ch-rpm-inside.html There are other systems obviously. > > Adantages for using setup.info > ------------------------------ > > 1) It's easier to extend past the simplest use case > > 2) It's more developer friendly. > > 3) Provides an easier way to specify package dependencies > > 4) Provides a portable setup system across platforms > > 5) Easier to integrate within the development build > process where handling version numbers can often > be a challenge. > How so ? You need to provide examples of what is missing compared to what's available in python now. > > Section Name Construction > ------------------------- > > The setup.info file is built as a standard configparser > format file. > > Section names are built from a combination of a specifier > then an option operating system name and finally an > optional python version specifier. > > The following shows a very powerful way to build conditional > installation for different operating systems and platforms > that starts from very simple and can be extended with effort > into the very complex. > > Most developers will choose the simplest option. > > Sections are built in the following format > > specifier [operating_system] [pythonversion] > > A simple real world example might be: > > [dependencies] > packages = objectmapper >= 2.5 > > [dependencies windows] > packages = win32com > > [dependencies linux] > packages = pyodbc Is there a rationale for limiting yourself to platform and python version ? You may need to condition on other things (debugging may come to mind, for example). > > > Pre Installation Processing [preinstall] > ---------------------------------------- > In general, I think the notions pre-install/etc... are sorely needed, but I am not sure they belong here. A precise definition and protocol for what exactly is pre-install, pre-build, etc... is needed. This brings the format extensibility capability - you should also consider versioning the file format. > [datafiles] > files = artisticdb.dbf > artisticdb.mdx > authors.dbf > authors.mdx > directory = db How would you handle the case where you have several data directories ? Concerning data files, the notion is too vague IMHO - I know that's the notion used by distutils, but on linux at least, you need to consider icons vs images vs doc (man vs html vs info vs ...). It should not be mandatory for people who don't care, though. cheers, David From david.lyon at preisshare.net Tue Oct 13 11:16:21 2009 From: david.lyon at preisshare.net (David Lyon) Date: Tue, 13 Oct 2009 05:16:21 -0400 Subject: [Distutils] Alternate static metadata PEP submission... In-Reply-To: <94bdd2610910130136g51b559ey9e3a04123309342e@mail.gmail.com> References: <94bdd2610910111517u2fd0b136p44958f7faa46e974@mail.gmail.com> <94bdd2610910130136g51b559ey9e3a04123309342e@mail.gmail.com> Message-ID: <1d8404d17e3ebee2b734f928e5af9297@preisshare.net> On Tue, 13 Oct 2009 10:36:12 +0200, Tarek Ziad? wrote: > Why are you doing a similar PEP, what's the point ? You've already helped a > bit. Tarek, I championed the use of static metadata on the distutils mailing list before you picked it up. Don't you remember? The reason that I am doing an alternate PEP is because it contains improvements that would be very useful on windows and linux. A fair and open development process should mean that people are able to freely contribute their ideas and have them evaluated on the merits of the idea. > I suggest that you help us finishing in the existing PEP 390 instead, > by proposing changes. If I see things I can help with, then I'm happy to do that... David From ziade.tarek at gmail.com Tue Oct 13 11:33:26 2009 From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=) Date: Tue, 13 Oct 2009 11:33:26 +0200 Subject: [Distutils] Alternate static metadata PEP submission... In-Reply-To: <1d8404d17e3ebee2b734f928e5af9297@preisshare.net> References: <94bdd2610910111517u2fd0b136p44958f7faa46e974@mail.gmail.com> <94bdd2610910130136g51b559ey9e3a04123309342e@mail.gmail.com> <1d8404d17e3ebee2b734f928e5af9297@preisshare.net> Message-ID: <94bdd2610910130233o1ee3578fu6c50a584d1e5d53a@mail.gmail.com> On Tue, Oct 13, 2009 at 11:16 AM, David Lyon wrote: > On Tue, 13 Oct 2009 10:36:12 +0200, Tarek Ziad? > wrote: >> Why are you doing a similar PEP, what's the point ? You've already helped > a >> bit. > > Tarek, > > I championed the use of static metadata on the distutils mailing list > before you picked it up. Don't you remember? Frankly, I don't. The first time we've mentioned static metadata was at Pycon last year. The first one who mentioned the idea was Eric IIRC, then Matthias and Tres worked on it. But who cares ? I am not claiming any ownership on ideas. I am not tracking either who threw which idea. But if someone cares, speak up : I'll detail who participated in the Acknowledgment section. I just happen to be the one maintaining PEP 390 and I am trying to collect ideas here because I am maintaining Distutils so I'll be the one who will apply the changes there. > > The reason that I am doing an alternate PEP is because it contains > improvements that would be very useful on windows and linux. > > A fair and open development process should mean that people are > able to freely contribute their ideas and have them evaluated > on the merits of the idea. That's exactly how PEP 390 was built. The process is far from being perfect though, maybe some ideas didn't make it through. But that's how it goes. > >> I suggest that you help us finishing in the existing PEP 390 instead, >> by proposing changes. > > If I see things I can help with, then I'm happy to do that... Great then, I am looking forward for its improvements. Tarek From floris.bruynooghe at gmail.com Tue Oct 13 12:30:21 2009 From: floris.bruynooghe at gmail.com (Floris Bruynooghe) Date: Tue, 13 Oct 2009 11:30:21 +0100 Subject: [Distutils] Alternate static metadata PEP submission... In-Reply-To: References: Message-ID: <20091013103021.GA21066@laurie.devork> On Tue, Oct 13, 2009 at 02:18:11AM -0400, David Lyon wrote: > This PEP proposes a change to the way that applications > and libraries are installed in python, by using a new > file called setup.info rather than setup.py. To hold > installation information. This seems to propose a whole build system, while PEP390 only aims for static metadata. Not restricting it to static metadata makes the domain a lot wider and complicated as shown by countless previous discussions on this list (several of these things have been outlined before). Since the problem domain is so huge it will take a very long time to come up with a stable version that addresses all issues adequately, something which this PEP sidesteps solving the problem it targets in a sensible and contained way. It certainly is a laudable goal to come up with such a build system but I don't think it can be seen as *instead of* the metatdata PEP. IMHO such a system should be using the static metadata from whatever PEP 390 will become. Regards Floris From zedobject at gmail.com Tue Oct 13 13:27:24 2009 From: zedobject at gmail.com (Milind Khadilkar) Date: Tue, 13 Oct 2009 16:57:24 +0530 Subject: [Distutils] Setuptools 0.6c10 release imminent; please test In-Reply-To: <94bdd2610910130125v13073ea7r93a06946dc071c88@mail.gmail.com> References: <20091012202600.5B4B13A4068@sparrow.telecommunity.com> <20091012234846.BA7DF3A4068@sparrow.telecommunity.com> <4AD42D07.2020702@taupro.com> <94bdd2610910130125v13073ea7r93a06946dc071c88@mail.gmail.com> Message-ID: <116b13090910130427v7f368835x1eed065de946930d@mail.gmail.com> Peace. *What would newcomers think?????* On Tue, Oct 13, 2009 at 1:55 PM, Tarek Ziad? wrote: > On Tue, Oct 13, 2009 at 9:32 AM, Jeff Rush wrote: > >> The fall-down was in the testing done before the Python release and I'm > >> sure more testing will be done in that area before the 2.6.4 bugfix > >> release. > > > > Not really - according to the emails on this list, the incompatibility > > was found _before_ the 2.6.3 release and it was decided so what let > > things break. More testing would _not_ have prevented this problem. > > That mistake I have done was not done on purpose so now stop > with this kind of accusation immediatly. > > >> > >> Sorry, it doesn't look like anyone wants to play with you any more; you > >> can just keep your ball. > > > > Please don't speak for others - it's rude. Only speak for yourself. > > And you, stop to think for the others, pointing fingers and throwing mud, > you are the one who is rude right now. > > Tarek. > _______________________________________________ > Distutils-SIG maillist - Distutils-SIG at python.org > http://mail.python.org/mailman/listinfo/distutils-sig > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mwhapples at aim.com Tue Oct 13 13:16:31 2009 From: mwhapples at aim.com (Michael Whapples) Date: Tue, 13 Oct 2009 12:16:31 +0100 Subject: [Distutils] Distribute and setuptools: what are the differences Message-ID: <4AD4618F.6070207@aim.com> Hello, I realise I may be starting something I would prefer not to (looking at some of the replies of the announcement of setuptools 0.6c10), so I would like to ask that this tries to stay as a factual thing rather than a "having a go at the other". As it seems that setuptools is not "dead" just possibly a bit slow at being updated, could I ask what are the aims of the various projects setuptools and distribute (please try and keep this information restricted to your project and what it aims to do)? It would be nice to have this information as at the moment its a bit unclear what either really are trying to offer over the other. I would like to be able to make a fully informed choice on why use one over the other, I feel at the moment this isn't possible. I have to say from my view (as a developer of my own packages only using setuptools/distribute to help package my packages) that it currently looks a horrible mess, distribute tries to replace setuptools and does it in such a way that any package which specifically wants setuptools has to use distribute if another package wants it, so not allowing the one which specifically wants setuptools to get the latest setuptools (eg. now or soon it will be setuptools 0.6c10). It is tempting for me to use distutils and create my own custom code to give the features I currently really use from setuptools/distribute until this "mess" has been cleaned up. I know this idea of going a custom route doesn't really help things, but my main concern is my package and if no solution currently can satisfy me it is suitable then unfortunately custom solutions may be the best option for me (I won't support a mess). Michael Whapples From reinout at vanrees.org Tue Oct 13 14:05:03 2009 From: reinout at vanrees.org (Reinout van Rees) Date: Tue, 13 Oct 2009 12:05:03 +0000 (UTC) Subject: [Distutils] Distribute and setuptools: what are the differences References: <4AD4618F.6070207@aim.com> Message-ID: On 2009-10-13, Michael Whapples wrote: > Hello, > I realise I may be starting something I would prefer not to (looking at > some of the replies of the announcement of setuptools 0.6c10), so I > would like to ask that this tries to stay as a factual thing rather than > a "having a go at the other". > > As it seems that setuptools is not "dead" just possibly a bit slow at > being updated, Setuptools has effectively been dead for a year. Critical patches were not being applied and there was no hope (till a few days ago) of ever getting them fixed. This goes way beyond "a bit slow at being updated". > could I ask what are the aims of the various projects > setuptools and distribute (please try and keep this information > restricted to your project and what it aims to do)? Setuptools being completely dead (till a few days ago), distribute's aim was basically just to have a working setuptools. So: no differences, except that patches have been applied (and iirc one useful non-intrusive addition of being able to upload sphinx docs to pypi). > It would be nice to > have this information as at the moment its a bit unclear what either > really are trying to offer over the other. I would like to be able to > make a fully informed choice on why use one over the other, I feel at > the moment this isn't possible. Technically both are the same (not looking at the bugfixes). The choice is between open source development and project management styles: PJE's setuptools and Tarek-and-a-lot-of-other-people's distribute. > I have to say from my view (as a developer of my own packages only using > setuptools/distribute to help package my packages) that it currently > looks a horrible mess, distribute tries to replace setuptools and does > it in such a way that any package which specifically wants setuptools > has to use distribute if another package wants it, so not allowing the > one which specifically wants setuptools to get the latest setuptools > (eg. now or soon it will be setuptools 0.6c10). For a globally installed setuptools or distribute: yes, that's a mess. When you use buildout (or pip, apparently) to manage a per-project isolated set of eggs, they will happily coexist (once the 0.6.5 is out with a small fix). So it also depends a bit on the way you set up your project. Reinout -- Reinout van Rees - reinout at vanrees.org - http://reinout.vanrees.org Software developer at http://www.thehealthagency.com "Military engineers build missiles. Civil engineers build targets" From regebro at gmail.com Tue Oct 13 14:14:37 2009 From: regebro at gmail.com (Lennart Regebro) Date: Tue, 13 Oct 2009 14:14:37 +0200 Subject: [Distutils] Distribute and setuptools: what are the differences In-Reply-To: <4AD4618F.6070207@aim.com> References: <4AD4618F.6070207@aim.com> Message-ID: <319e029f0910130514l7338d45cn4952f59e20a0a6af@mail.gmail.com> 2009/10/13 Michael Whapples : > As it seems that setuptools is not "dead" just possibly a bit slow at being > updated, could I ask what are the aims of the various projects setuptools > and distribute (please try and keep this information restricted to your > project and what it aims to do)? Distribute is created as a response to setuptools not getting the maintenance it deserves. It provides various bugfixes that has been known in setuptools for a long time but not solved. It also nowadays provides Python 3 support. > I have to say from my view (as a developer of my own packages only using > setuptools/distribute to help package my packages) that it currently looks a > horrible mess, distribute tries to replace setuptools and does it in such a > way that any package which specifically wants setuptools has to use > distribute if another package wants it Distribute is a fork and a complete replacement of setuptools. Hence you can only have one installed in each environment at once. Yes it's a mess. There really is not many reasons for any package to require distribute over setuptools. It's not a package choice, it's a user-choice. > create my own custom code to give the features I currently really use from > setuptools/distribute until this "mess" has been cleaned up. > I know this > idea of going a custom route doesn't really help things If it helps you, it helps. -- Lennart Regebro: Python, Zope, Plone, Grok http://regebro.wordpress.com/ +33 661 58 14 64 From ssteinerx at gmail.com Tue Oct 13 14:42:02 2009 From: ssteinerx at gmail.com (ssteinerX@gmail.com) Date: Tue, 13 Oct 2009 08:42:02 -0400 Subject: [Distutils] the virtualenv-distribute mess In-Reply-To: References: <319e029f0910060402n306a992as4b9d854f7f29d546@mail.gmail.com> <319e029f0910060402m28ab35acnc9a52aefcdc6f04a@mail.gmail.com> <4ACB24AE.4000300@cryptelium.net> <4ACB2527.7010802@cryptelium.net> <319e029f0910060522l29499f46y2805ed0082d975c3@mail.gmail.com> <4ACB7D70.9060404@cryptelium.net> <4ACF1681.7050506@simplistix.co.uk> <4ACF269C.4010400@cryptelium.net> <94bdd2610910090508j6804628fy5b5d435cce0c7427@mail.gmail.com> <319e029f0910090536j480b2339t530a2f6a9267d451@mail.gmail.com> <4ACF2FC5.6090706@cryptelium.net> <4ACF4765.8020404@simplistix.co.uk> <87tyy5qd8j.fsf@benfinney.id.au> <20091012223531.7FCF63A4068@sparrow.telecommunity.com> Message-ID: <31406EDA-8EAA-4291-B7F6-918CD2E5EFAA@gmail.com> On Oct 13, 2009, at 3:15 AM, Reinout van Rees wrote: > On 2009-10-12, P.J. Eby wrote: >> At 08:09 AM 10/12/2009 +0000, Reinout van Rees wrote: >>> OTOH, grumbl ... horrible breakage ... essential piece of >>> infrastructure ... >>> allowed to persist.... I'm pretty grumpy right now. >> >> Relax, take a deep breath, and then easy_install setuptools==dev or >> setuptools==dev06. ;-) > > Relax, take a deep breath, go to #distutils in irc and be amazed at > the speed > by which Tarek is able to fix things. > > I'm helping Tarek debug stuff now so that I can switch to > distribute :-) Yes, now that people are allowed to help, things should improve rapidly. S From mwhapples at aim.com Tue Oct 13 15:48:34 2009 From: mwhapples at aim.com (Michael Whapples) Date: Tue, 13 Oct 2009 14:48:34 +0100 Subject: [Distutils] Distribute and setuptools: what are the differences In-Reply-To: <319e029f0910130514l7338d45cn4952f59e20a0a6af@mail.gmail.com> References: <4AD4618F.6070207@aim.com> <319e029f0910130514l7338d45cn4952f59e20a0a6af@mail.gmail.com> Message-ID: <4AD48532.1010808@aim.com> On 13/10/09 13:14, Lennart Regebro wrote: > 2009/10/13 Michael Whapples: > >> As it seems that setuptools is not "dead" just possibly a bit slow at being >> updated, could I ask what are the aims of the various projects setuptools >> and distribute (please try and keep this information restricted to your >> project and what it aims to do)? >> > Distribute is created as a response to setuptools not getting the > maintenance it deserves. It provides various bugfixes that has been > known in setuptools for a long time but not solved. It also nowadays > provides Python 3 support. > Thanks for that, now it would be nice to hear from P.J. Eby on what setuptools aims to do and why distribute needed to be a fork rather than people added to the development team (what was it in the distribute's teams view that made them unacceptable for setuptools). > >> I have to say from my view (as a developer of my own packages only using >> setuptools/distribute to help package my packages) that it currently looks a >> horrible mess, distribute tries to replace setuptools and does it in such a >> way that any package which specifically wants setuptools has to use >> distribute if another package wants it >> > Distribute is a fork and a complete replacement of setuptools. Hence > you can only have one installed in each environment at once. Yes it's > a mess. > May be there's certain reasons why setuptools and distribute couldn't be designed to coexist on a system (eg. may be its something to do with how they manage the package environment and include packages in eggs), but it would be nice for them to coexist. I am looking at cython/pyrex here. As I understand it cython forked from pyrex and aims to add extra features (eg. cython gives python3 support, cython has some extra ways of specifying for loops, etc), but as I understand it there is no packaging conflict between cython and pyrex and so both may be installed at the same time and not interfere with the other. > > There really is not many reasons for any package to require distribute > over setuptools. It's not a package choice, it's a user-choice. > I sort of understand that but not fully. A package may need to specify distribute if it wants to be python3 compatible. The only way I can get what you say to work in my mind is not to use the automatic install of setuptools/distribute (IE. not to use distribute_setup.py or ez_setup.py in my setup.py script, I mean those lines: from ex_setup import use_setuptools use_setuptools() > ). Are you saying/would you advise not to do this automatic install of setuptools/distribute? > >> create my own custom code to give the features I currently really use from >> setuptools/distribute until this "mess" has been cleaned up. >> > >> I know this >> idea of going a custom route doesn't really help things >> > If it helps you, it helps. > I was more meaning it doesn't help the python community, it doesn't help solve this mess. Also in a way it doesn't help me as much as I might like as it will take some of my time, time which may be better spent on the package I am developing (the work I may have specific skills in). > Anyway I will have to just consider what direction would be most productive for my packages, having seen some of the recent messages I am quite strongly of the view let a new default (although setuptools has always been a third party package, it was a fairly obvious and fairly popular choice in the past, so as good as default) win out. One final thing, in a different thread http://mail.python.org/pipermail/distutils-sig/2009-October/013845.html, it was said "What would newcomers think", well I am not new to python but only recently joined this list to ask about certain things with distribute and I am very much now thinking to avoid setuptools/distribute. Michael Whapples From ssteinerx at gmail.com Tue Oct 13 16:16:27 2009 From: ssteinerx at gmail.com (ssteinerX@gmail.com) Date: Tue, 13 Oct 2009 10:16:27 -0400 Subject: [Distutils] Setuptools 0.6c10 release imminent; please test In-Reply-To: <4AD42D07.2020702@taupro.com> References: <20091012202600.5B4B13A4068@sparrow.telecommunity.com> <20091012234846.BA7DF3A4068@sparrow.telecommunity.com> <4AD42D07.2020702@taupro.com> Message-ID: On Oct 13, 2009, at 3:32 AM, Jeff Rush wrote: >>> Too little, too late, no thanks, I'll just be sticking with >>> Distribute >>> from now on. >> Several developers and an open development process vs a lone coder >> with >> a closed codebase? That's not really a choice at all... >> >> Sorry, it doesn't look like anyone wants to play with you any more; >> you >> can just keep your ball. > > Please don't speak for others - it's rude. Only speak for yourself. I apologize -- I didn't mean to speak for anyone who hasn't already spoken. I was specifically speaking to the "Several developers..." Thank you for telling me how to behave, which is not rude at all. S From floris.bruynooghe at gmail.com Tue Oct 13 16:16:57 2009 From: floris.bruynooghe at gmail.com (Floris Bruynooghe) Date: Tue, 13 Oct 2009 15:16:57 +0100 Subject: [Distutils] Distribute and setuptools: what are the differences In-Reply-To: <4AD48532.1010808@aim.com> References: <4AD4618F.6070207@aim.com> <319e029f0910130514l7338d45cn4952f59e20a0a6af@mail.gmail.com> <4AD48532.1010808@aim.com> Message-ID: <20091013141657.GA22672@laurie.devork> On Tue, Oct 13, 2009 at 02:48:34PM +0100, Michael Whapples wrote: [...] > and I am very much now thinking to avoid > setuptools/distribute. Nothing wrong with that, if you don't need any of the features provided by setuptools/distribute then using them is pointless extra dependency. I've always avoided using them in my packages since I see no convincing benefit compared to downsides, that way users have the choice to use plain distutils or easy_install. And I see not forcing setuptools/distribute on users as a feature. Regards Floris -- Debian GNU/Linux -- The Power of Freedom www.debian.org | www.gnu.org | www.kernel.org From regebro at gmail.com Tue Oct 13 16:26:46 2009 From: regebro at gmail.com (Lennart Regebro) Date: Tue, 13 Oct 2009 16:26:46 +0200 Subject: [Distutils] Distribute and setuptools: what are the differences In-Reply-To: <4AD48532.1010808@aim.com> References: <4AD4618F.6070207@aim.com> <319e029f0910130514l7338d45cn4952f59e20a0a6af@mail.gmail.com> <4AD48532.1010808@aim.com> Message-ID: <319e029f0910130726h36772917re45cceb8f5251a74@mail.gmail.com> 2009/10/13 Michael Whapples : >> Distribute is a fork and a complete replacement of setuptools. Hence >> you can only have one installed in each environment at once. Yes it's >> a mess. > May be there's certain reasons why setuptools and distribute couldn't be > designed to coexist on a system (eg. may be its something to do with how > they manage the package environment and include packages in eggs) Because setuptools is used not only when you create packages, but also when you install them, is the main reason. And that means we need to replace setuptools. Again, it's not a choice on the package level, but on the user level. It's like a car. It requires wheels. Which brand you use doesn't matter. But you can't have 8 sets of tires on a 4 wheel car. :) >> There really is not many reasons for any package to require distribute >> over setuptools. It's not a package choice, it's a user-choice. >> I sort of understand that but not fully. A package may need to specify >> distribute if it wants to be python3 compatible. Nope. It needs to *use* Distribute to install under Python 3. But that is not a problem, as setuptools doens't exist for Python 3, so there is no conflict there. But it would still specify and import setuptools. > The only way I can get what > you say to work in my mind is not to use the automatic install of > setuptools/distribute (IE. not to use distribute_setup.py or ez_setup.py in > my setup.py script, I mean those lines: > > from ex_setup import use_setuptools > use_setuptools() Oh, right, don't do that. Does any package do that? Crazy stuff. :-) >> If it helps you, it helps. > I was more meaning it doesn't help the python community, it doesn't help > solve this mess. I don't think this mess can be solved. We can only walk around it by building something new and better by refactoring setuptools into small pieces. I know some people of the Distribute team has started (tentatively called Distribute2) and PJE has also expressed interest in doing this. If some sort of cooperation can happen there that would be good, if not people will have a choice, and that's also good. In any case Distribute 0.6/setuptools will slowly go out of use. But this is a process that will take years. Unfortunately I don't think a short term solution is realistic. > Anyway I will have to just consider what direction would be most productive > for my packages Neither. You don't need a direction for your packages. If setuptools have a feature you need, use it. If setuptools turns out to have bugs, or you need Python 3 support, then install Distribute. Your package will not notice the difference. I'd like to repeat again, just to make sure that there are no misunderstandings, that this is not about winning, and that there is no competition between setuptools and distribute. Distribute is, as of today, only one thing: setuptools + bugfixes + python 3 support. There is no reason for you to use Distribute unless you need Python 3 support or encounter a setuptools bug (unless you feel like fixing bugs or betatesting Distribute of course). setuptools 0.6c10 will remove most or all of the bug difference, which means there is less reason for people to switch to Distribute. That's a good thing. Distribute doesn't need users. It's users who needs Distribute. That's why it exists. It's driven purely out of necessity, nothing else. -- Lennart Regebro: Python, Zope, Plone, Grok http://regebro.wordpress.com/ +33 661 58 14 64 From p.f.moore at gmail.com Tue Oct 13 16:28:42 2009 From: p.f.moore at gmail.com (Paul Moore) Date: Tue, 13 Oct 2009 15:28:42 +0100 Subject: [Distutils] Setuptools 0.6c10 release imminent; please test In-Reply-To: <116b13090910130427v7f368835x1eed065de946930d@mail.gmail.com> References: <20091012202600.5B4B13A4068@sparrow.telecommunity.com> <20091012234846.BA7DF3A4068@sparrow.telecommunity.com> <4AD42D07.2020702@taupro.com> <94bdd2610910130125v13073ea7r93a06946dc071c88@mail.gmail.com> <116b13090910130427v7f368835x1eed065de946930d@mail.gmail.com> Message-ID: <79990c6b0910130728s7947923dic0006e1df66ac7e@mail.gmail.com> 2009/10/13 Milind Khadilkar : > Peace. > What would newcomers think????? Sadly, they'd probably get precisely the correct impression :-( Paul. From da4089 at me.com Tue Oct 13 14:16:28 2009 From: da4089 at me.com (David Arnold) Date: Tue, 13 Oct 2009 08:16:28 -0400 Subject: [Distutils] Distribute and setuptools: what are the differences In-Reply-To: <4AD4618F.6070207@aim.com> References: <4AD4618F.6070207@aim.com> Message-ID: <14B5D6E6-1798-4022-B1EE-0D4EFAE43FE3@me.com> On 13/10/2009, at 7:16, Michael Whapples wrote: < lots of rational stuff > +10 d > Hello, > I realise I may be starting something I would prefer not to (looking > at some of the replies of the announcement of setuptools 0.6c10), so > I would like to ask that this tries to stay as a factual thing > rather than a "having a go at the other". > > As it seems that setuptools is not "dead" just possibly a bit slow > at being updated, could I ask what are the aims of the various > projects setuptools and distribute (please try and keep this > information restricted to your project and what it aims to do)? It > would be nice to have this information as at the moment its a bit > unclear what either really are trying to offer over the other. I > would like to be able to make a fully informed choice on why use one > over the other, I feel at the moment this isn't possible. > > I have to say from my view (as a developer of my own packages only > using setuptools/distribute to help package my packages) that it > currently looks a horrible mess, distribute tries to replace > setuptools and does it in such a way that any package which > specifically wants setuptools has to use distribute if another > package wants it, so not allowing the one which specifically wants > setuptools to get the latest setuptools (eg. now or soon it will be > setuptools 0.6c10). It is tempting for me to use distutils and > create my own custom code to give the features I currently really > use from setuptools/distribute until this "mess" has been cleaned > up. I know this idea of going a custom route doesn't really help > things, but my main concern is my package and if no solution > currently can satisfy me it is suitable then unfortunately custom > solutions may be the best option for me (I won't support a mess). > > Michael Whapples > _______________________________________________ > Distutils-SIG maillist - Distutils-SIG at python.org > http://mail.python.org/mailman/listinfo/distutils-sig From floris.bruynooghe at gmail.com Tue Oct 13 17:31:26 2009 From: floris.bruynooghe at gmail.com (Floris Bruynooghe) Date: Tue, 13 Oct 2009 16:31:26 +0100 Subject: [Distutils] Distribute and setuptools: what are the differences In-Reply-To: <319e029f0910130726h36772917re45cceb8f5251a74@mail.gmail.com> References: <4AD4618F.6070207@aim.com> <319e029f0910130514l7338d45cn4952f59e20a0a6af@mail.gmail.com> <4AD48532.1010808@aim.com> <319e029f0910130726h36772917re45cceb8f5251a74@mail.gmail.com> Message-ID: <20091013153126.GB23031@laurie.devork> On Tue, Oct 13, 2009 at 04:26:46PM +0200, Lennart Regebro wrote: > 2009/10/13 Michael Whapples : > > from ex_setup import use_setuptools > > use_setuptools() > > Oh, right, don't do that. Does any package do that? Crazy stuff. :-) I'm glad I wasn't the only one thinking that! :-) Floris From mwhapples at aim.com Tue Oct 13 18:11:03 2009 From: mwhapples at aim.com (Michael Whapples) Date: Tue, 13 Oct 2009 17:11:03 +0100 Subject: [Distutils] Distribute and setuptools: what are the differences In-Reply-To: <20091013141657.GA22672@laurie.devork> References: <4AD4618F.6070207@aim.com> <319e029f0910130514l7338d45cn4952f59e20a0a6af@mail.gmail.com> <4AD48532.1010808@aim.com> <20091013141657.GA22672@laurie.devork> Message-ID: <4AD4A697.5040708@aim.com> Floris, I think I have come to the same conclusion. There are a couple of features I use from setuptools but I feel it might just be better to produce my own custom code for that. Michael Whapples On -10/01/37 20:59, Floris Bruynooghe wrote: > On Tue, Oct 13, 2009 at 02:48:34PM +0100, Michael Whapples wrote: > [...] > >> and I am very much now thinking to avoid >> setuptools/distribute. >> > Nothing wrong with that, if you don't need any of the features > provided by setuptools/distribute then using them is pointless extra > dependency. I've always avoided using them in my packages since I see > no convincing benefit compared to downsides, that way users have the > choice to use plain distutils or easy_install. And I see not forcing > setuptools/distribute on users as a feature. > > Regards > Floris > > From mwhapples at aim.com Tue Oct 13 18:16:25 2009 From: mwhapples at aim.com (Michael Whapples) Date: Tue, 13 Oct 2009 17:16:25 +0100 Subject: [Distutils] Setuptools 0.6c10 release imminent; please test In-Reply-To: <79990c6b0910130728s7947923dic0006e1df66ac7e@mail.gmail.com> References: <20091012202600.5B4B13A4068@sparrow.telecommunity.com> <20091012234846.BA7DF3A4068@sparrow.telecommunity.com> <4AD42D07.2020702@taupro.com> <94bdd2610910130125v13073ea7r93a06946dc071c88@mail.gmail.com> <116b13090910130427v7f368835x1eed065de946930d@mail.gmail.com> <79990c6b0910130728s7947923dic0006e1df66ac7e@mail.gmail.com> Message-ID: <4AD4A7D9.9020806@aim.com> On -10/01/37 20:59, Paul Moore wrote: > 2009/10/13 Milind Khadilkar: > >> Peace. >> What would newcomers think????? >> > Sadly, they'd probably get precisely the correct impression :-( > Exactly, what I have seen on this list for the last couple of days has finally given me that bit of motivation to chuck my package's dependency on setuptools and I will leave people here to continue as they wish. > Michael Whapples > Paul. > > From ziade.tarek at gmail.com Tue Oct 13 19:01:30 2009 From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=) Date: Tue, 13 Oct 2009 19:01:30 +0200 Subject: [Distutils] Alternate static metadata PEP submission... In-Reply-To: <20091013103021.GA21066@laurie.devork> References: <20091013103021.GA21066@laurie.devork> Message-ID: <94bdd2610910131001v70711ffeia27433cd23b32fca@mail.gmail.com> On Tue, Oct 13, 2009 at 12:30 PM, Floris Bruynooghe wrote: > On Tue, Oct 13, 2009 at 02:18:11AM -0400, David Lyon wrote: >> This PEP proposes a change to the way that applications >> and libraries are installed in python, by using a new >> file called setup.info rather than setup.py. To hold >> installation information. > > This seems to propose a whole build system, while PEP390 only aims for > static metadata. ?Not restricting it to static metadata makes the > domain a lot wider and complicated as shown by countless previous > discussions on this list (several of these things have been outlined > before). ?Since the problem domain is so huge it will take a very long > time to come up with a stable version that addresses all issues > adequately, something which this PEP sidesteps solving the problem it > targets in a sensible and contained way. > > It certainly is a laudable goal to come up with such a build system > but I don't think it can be seen as *instead of* the metatdata PEP. > IMHO such a system should be using the static metadata from whatever > PEP 390 will become. If wanted, I can go ahead I create a new PEP for David's proposal. And David can work it out using PEP 390 references maybe let me know Tarek From ianb at colorstudy.com Tue Oct 13 19:08:36 2009 From: ianb at colorstudy.com (Ian Bicking) Date: Tue, 13 Oct 2009 12:08:36 -0500 Subject: [Distutils] PEP 390 - new format from setup.cfg In-Reply-To: <94bdd2610910120934p672cb041xbb5cd36ab64e54b5@mail.gmail.com> References: <94bdd2610910111517u2fd0b136p44958f7faa46e974@mail.gmail.com> <94bdd2610910120245i2075d2d4n6bf1b7a44ae9d219@mail.gmail.com> <94bdd2610910120934p672cb041xbb5cd36ab64e54b5@mail.gmail.com> Message-ID: On Mon, Oct 12, 2009 at 11:34 AM, Tarek Ziad? wrote: > On Mon, Oct 12, 2009 at 6:10 PM, Ian Bicking wrote: >> If you don't have tuples or <, >, etc, it seems like something like >> "Python version 2.6 or higher" is hard to express. ?You'd have to >> enumerate 2.6, 2.7, and speculate on 2.8 and 2.9. > > python_version not in '2.3,2.4,2.5' > > (it's not optimal, but enough I guess, until PEP 386 is accepted) Given the small number of values available, string comparison works fine in this case. Is there a situation when the data would be dependent on a package version other than Python's? >> One use case for the current setup.cfg is for extensions, like >> generating Sphynx documentation. ?Those extensions can sometimes take >> quite a lot of options, so they are best when partitioned into their >> own section. ?I'm also not sure whether [metadata] is intended to have >> extensible variables, or a fixed set of variables. > > The variables are fixed so it works with a vanilla python and doesn't > require anything else > to run. Having extensible variables would break that idea and would just > make people move code they use to have in setup.py in another place imho. It should at least be allowed to have other values with a warning, similar to keyword arguments to setup() that are unknown. This lets the format be forward compatible with new variables. E.g., if in Python 2.8 there's a new variable, people can just dump it in. >>> How would you define/provide "features" here ? >> >> I'm not sure. ?With Setuptools the equivalent is "extras", like: >> >> setup(name='foo', >> ? ?extras_require={'xmlrpc': ['lxml']}) >> >> Then if you do "easy_install foo[xmlrpc]" (or somehow require >> foo[xmlrpc]) it will also install/require lxml. >> > [...] >> ?... >> >> So, since the result involves multiple sections it wouldn't naturally >> map to what you are proposing. > > That could be done I think as long as the extras are not calculated by > third-party code. > > Let's say, we define an 'extras' variable, which value can be provided > when interpreting setup.cfg. > > If could be described like this : > > [metadata:'xmlrpc' in extras] > requires = lxml > > The part that is unclear to me is how to list the extras a setup.cfg file has. There needs to be some enumeration of extras, yes. And there really needs to be more metadata than Setuptools allows for -- extras should have at least a readable description. So wherever they are defined, there should be room for that documentation as well. >> And given other variables (ones that perhaps distribute doesn't even >> know about) how are they combined? > > Right, this needs clarification. In any case, I think using multiline > is a bad idea > because it'll break RCF 232 compatibility for the long_description field. Oh... long_description -- I forgot about that. How will that be dealt with? I now typically do something like this in setup.py: long_desc = open(os.path.join(os.path.dirname(__file__), 'docs', 'index.txt')).read() long_desc = long_desc.split('..contents::', 1)[1].strip() setup(..., long_description=long_desc) Keeping the long description in setup.py (or in setup.cfg) is not something I want to do. It would be even worse in setup.cfg than it is in setup.py. Also ConfigParser eats leading whitespace (I believe that's also a problem with the PKG-INFO format), which breaks ReST. (I assume the "register" command does something other than uploading PKG-INFO to get long_description in place?) (Come to think of it, I'm not actually sure what we're accomplishing with this declarative metadata; "setup.py --name" is too minimal, but some new "setup.py metadata" command that dumps everything isn't hard to imagine; what are the *real* advantages of this new proposal? If I have to use a build process to build my setup.cfg, then absolutely nothing will be accomplished. And is the complexity all just because sometimes people need to use version-specific requirements?) -- Ian Bicking | http://blog.ianbicking.org | http://topplabs.org/civichacker From tseaver at palladion.com Tue Oct 13 19:30:19 2009 From: tseaver at palladion.com (Tres Seaver) Date: Tue, 13 Oct 2009 13:30:19 -0400 Subject: [Distutils] distribute and buildout: best practices? In-Reply-To: References: <94bdd2610910070453t63e7e4c4h1c9bf6ca567fa48f@mail.gmail.com> <4ACF1999.4080001@simplistix.co.uk> Message-ID: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Reinout van Rees wrote: > On 2009-10-09, Chris Withers wrote: >> Reinout van Rees wrote: >>> I'm still not 100% sure whether it is safe to put "distribute" in the >>> install_requires list of a package right now, however. >> As with setuptools, why do you think you need to? > > Namespace packages. You get warnings "hey, no setuptools dependency" once you > use namespaces. That is a buildout bug: whining about the lack of depenedency on the only package which makes dependencies meaningful is silly. No package should *ever* need to declare that it depends on setuptools. > Note that if I depend on distribute, I also get warnings about not having a > setuptools depencency sometimes... :-) Likewise. Tres. - -- =================================================================== Tres Seaver +1 540-429-0999 tseaver at palladion.com Palladion Software "Excellence by Design" http://palladion.com -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iEYEARECAAYFAkrUuSgACgkQ+gerLs4ltQ7lewCZAS/MML7cmnzEnK1M6gbSSQHS lrQAn2KDOhsfT8Wtj83DW9G4WTOVXoS2 =VxBP -----END PGP SIGNATURE----- From tseaver at palladion.com Tue Oct 13 19:31:24 2009 From: tseaver at palladion.com (Tres Seaver) Date: Tue, 13 Oct 2009 13:31:24 -0400 Subject: [Distutils] why would you ever need to specify setuptools as a dependency? In-Reply-To: <5cae42b20910090457p3f6e4a1bp361b240f974f1617@mail.gmail.com> References: <4ACF1956.5070903@simplistix.co.uk> <5cae42b20910090457p3f6e4a1bp361b240f974f1617@mail.gmail.com> Message-ID: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Hanno Schlichting wrote: > On Fri, Oct 9, 2009 at 1:07 PM, Chris Withers wrote: >> Reinout van Rees wrote: >>> - Do my libraries have to list a dependency on distribute or on >>> setuptools? >>> Everything zopish has a 'setuptools >= 0.6c9' in it. >> They shouldn't, unless you only require setuptools after your package is >> installed and don't use it in your setup.py, which seems unlikely. >> >> If you need it in your setup.py, what's the point of specifying it? You >> would have had to use it by the time you specify the requirement! > > I assume most packages Reinout uses (like all zope.* packages) use > namespace packages. So they actually do depend during runtime on the > pkg_resources module, which happens to be available from either the > distribute or setuptools distribution. So one of them should be > specified in install_requires. Why? Nobody will check / enforce / understand what 'install_requires' even means except setuptools / distribute. - -- =================================================================== Tres Seaver +1 540-429-0999 tseaver at palladion.com Palladion Software "Excellence by Design" http://palladion.com -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iEYEARECAAYFAkrUuWwACgkQ+gerLs4ltQ5G8wCgjfmDazDum+ClDnxDtw+nxImD W4AAoKElE+Uelm3yL30Kdjj/6YFpL+4C =KIbC -----END PGP SIGNATURE----- From zooko at zooko.com Tue Oct 13 16:53:56 2009 From: zooko at zooko.com (Zooko Wilcox-O'Hearn) Date: Tue, 13 Oct 2009 08:53:56 -0600 Subject: [Distutils] Distribute and setuptools: what are the differences In-Reply-To: <4AD4618F.6070207@aim.com> References: <4AD4618F.6070207@aim.com> Message-ID: <1AB36AC9-27FC-4B05-A85A-59D4CA02804A@zooko.com> Michael: Either one will probably work fine. The only differences at this point (setuptools 0.6c10 prerelease and distribute 0.6.4) are which issues have been fixed and how they were fixed. Also they are drop- in replacements for one another, so you can try one and then switch to the other without having to relearn anything. Please report any bugs. Thanks! Regards, Zooko From tseaver at palladion.com Tue Oct 13 19:45:36 2009 From: tseaver at palladion.com (Tres Seaver) Date: Tue, 13 Oct 2009 13:45:36 -0400 Subject: [Distutils] dev versions In-Reply-To: References: Message-ID: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Reinout van Rees wrote: > On 2009-10-08, Ian Bicking wrote: >> So after creating, say, version 0.3.1, I always mark a package as 0.3.2dev. >> But this is annoying, you might never create a version 0.3.2 (e.g., 0.4 >> might be the next level). So, it would be better to use something like >> 0.3.1~dev. What is considered best practice for this? Ideally something >> that works with both Setuptools and the upcoming Distribute version spec. > > > a) Where's the annoyment exactly? It is easy to change and it's a release-time > decision anyway. - - It makes the package under development "look" like it is released, which is a Bad Thing for people who blindly want to depend on stuff. - - It creates lots of unnecessary commit message churn (go count how many messages on the zope-commits list are for "SVB" management). - - Furthermore, it (appears) to promise a release which may or may not ever be made - - It encourages people to forego thinking about what the correct release number should be (we've had a number of releases which should have been "feature" releses which got made using the "default next version" bugfix number). > b) In a previous discussion on a zope mailinglist (about using '0' for this > purpose, which was pretty much shot down for the zope toolkit because of > the problems attached to it), someone mentioned adding '+svn' to the > previous version number. So from 0.3.1 to 0.3.1+svn. Apparently that > sorts it behind 0.3.1. You could try something like that. The poster > mentioned it as a debian standard. The ZTK discussion was closed by fiat, not by achieving consensus. Neither Jim nor I agree with the proclamation, but will (grudgingly, in my case) abide by it within the context of the ZTK. Tres. - -- =================================================================== Tres Seaver +1 540-429-0999 tseaver at palladion.com Palladion Software "Excellence by Design" http://palladion.com -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iEYEARECAAYFAkrUvMAACgkQ+gerLs4ltQ5uiwCfeGfFO8VpkdKNszrDdKoEFLyT A20An0TkCQYjaxWdCRJmp/I68OvBTn2D =cRaR -----END PGP SIGNATURE----- From hanno at hannosch.eu Tue Oct 13 19:55:09 2009 From: hanno at hannosch.eu (Hanno Schlichting) Date: Tue, 13 Oct 2009 19:55:09 +0200 Subject: [Distutils] why would you ever need to specify setuptools as a dependency? In-Reply-To: References: <4ACF1956.5070903@simplistix.co.uk> <5cae42b20910090457p3f6e4a1bp361b240f974f1617@mail.gmail.com> Message-ID: <5cae42b20910131055s5e152d62r4f7d385b1de98c9f@mail.gmail.com> On Tue, Oct 13, 2009 at 7:31 PM, Tres Seaver wrote: > Why? ?Nobody will check / enforce / understand what 'install_requires' > even means except setuptools / distribute. To quote Toshio Kuratomi: > It's nice for people creating system packages when you specify all of the > packages that your runtime depends on in setup.py. So apparently there are others who read and value this information. Hanno From pje at telecommunity.com Tue Oct 13 20:01:41 2009 From: pje at telecommunity.com (P.J. Eby) Date: Tue, 13 Oct 2009 14:01:41 -0400 Subject: [Distutils] why would you ever need to specify setuptools as a dependency? In-Reply-To: References: <4ACF1956.5070903@simplistix.co.uk> <5cae42b20910090457p3f6e4a1bp361b240f974f1617@mail.gmail.com> Message-ID: <20091013180150.DED5E3A4068@sparrow.telecommunity.com> At 01:31 PM 10/13/2009 -0400, Tres Seaver wrote: >Why? Because the user might have, say, setuptools 0.6c8, and the package relies on a bugfix in 0.6c9. Also, at some point, there will be an 0.7a1, with new features that some people might actually want to use. (Some projects also actually use setuptools or easy_install at runtime, not just pkg_resources; e.g. buildout, pip, etc.) > Nobody will check / enforce / understand what 'install_requires' >even means except setuptools / distribute. Nobody except system packagers, who would otherwise have to troll the source for pkg_resources imports and such. (Even then, a project could declare entry points in its setup.py that will get called by some other project, without ever directly using a pkg_resources or setuptools API.) From tseaver at palladion.com Tue Oct 13 20:06:13 2009 From: tseaver at palladion.com (Tres Seaver) Date: Tue, 13 Oct 2009 14:06:13 -0400 Subject: [Distutils] PEP 390 - new format from setup.cfg In-Reply-To: References: <94bdd2610910111517u2fd0b136p44958f7faa46e974@mail.gmail.com> <94bdd2610910120245i2075d2d4n6bf1b7a44ae9d219@mail.gmail.com> Message-ID: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Ian Bicking wrote: > On Mon, Oct 12, 2009 at 4:45 AM, Tarek Ziad? wrote: >> On Mon, Oct 12, 2009 at 2:29 AM, Ian Bicking wrote: >>> The grammar in "Context-dependant sections" indicates possible EXPR >>> values. Because the "in" operator is supported, I would assume that >>> tuples should also be allowed. >> "in" here is restricted to string. It was added so we could write things like: >> >> 'linux' in sys_platform (where sys_platform can be linux2) > > If you don't have tuples or <, >, etc, it seems like something like > "Python version 2.6 or higher" is hard to express. You'd have to > enumerate 2.6, 2.7, and speculate on 2.8 and 2.9. > >> I'll add a note on that. >> >>> One aspect of the current setup.cfg is that it supports multiple >>> sections, for different setup.py commands or components. This gives a >>> kind of namespacing. I assume, but it isn't that specified, that any >>> section (not just "metadata") will be parsed the same way? >> I guess yes, even if I don't see a use case yet for that. > > One use case for the current setup.cfg is for extensions, like > generating Sphynx documentation. Those extensions can sometimes take > quite a lot of options, so they are best when partitioned into their > own section. I'm also not sure whether [metadata] is intended to have > extensible variables, or a fixed set of variables. > >>> Presumably setup.py will just contain an empty call to setup()? (I >>> assume at least setup.py will be allowed for, for backward >>> compatibility, even if it is not required.) >> No because we might need to define extra options in setup() >> that are not part of the metadata, like what is required the for the >> sdist command >> (package_data, scripts, package, etc) > > OK, so setup.cfg is for generating PKG-INFO, but installing a package > still involves running setup.py and some maybe-declarative code in > there. > >>> I believe this does not include the concept of extra requirements. >>> Possibly it could as a side-effect of some variable available to the >>> language. Like: >>> >>> [metadata:'xmlrpc' in features] >>> requires = lxml >>> >>> Sets and the & operator could be useful for this. >> How would you define/provide "features" here ? > > I'm not sure. With Setuptools the equivalent is "extras", like: > > setup(name='foo', > extras_require={'xmlrpc': ['lxml']}) > > Then if you do "easy_install foo[xmlrpc]" (or somehow require > foo[xmlrpc]) it will also install/require lxml. > > If I was to say why this is a problem, I think it's because the > default is that there are no features for a package. So someone > naively wants your package and installs it, but they don't get all the > features they thought the package provided (and which the package > actually *does* provide, but just doesn't have the prerequisite > libraries for -- since the package ships with the xmlrpc code, it's > just not working xmlrpc code). There's also not a good way of seeing > what extras are provided, or what their purpose is. So library > authors avoid the issue entirely and don't factor out requirements > into specific extras/features. If there was a set of "default" extras > maybe it would be more workable. I.e., "easy_install foo" installs > foo with all its default extras, and "easy_install foo[]" installs foo > without any extras (or you put the specific extras you want in []). > > Anyway, the way extra requirements are serialized to disk is requires.txt, with: > > default_req1 > default_req2 > > [extra] > extra_req1 > ... > > So, since the result involves multiple sections it wouldn't naturally > map to what you are proposing. > >>> The way variables are handled is unclear. Presumably all variables >>> are cumulative. But given: >>> >>> requires = Foo >>> requires = Bar >>> >>> What is the value of "requires"? A list, a value with newlines? >>> Presumably you could also do: >>> >>> requires = Foo >>> Bar >>> >>> Anyway, we're diverging from INI semantics at this point, so it needs >>> to be specified how it works. >> Right, this needs to be defined. I would be in favor of the latter, to >> stay ConfigParser >> compatible. > > You still have to define how options are combined from multiple > sections, and what the resulting value is from the API. That is, if > you have: > > [metadata] > requires = Foo > Bar > > [metadata:python_version == '2.4' or python_version== '2.3' or > python_version=='2.2'] > requires = BackwardCompat > > Then what is the ultimate value of "requires"? Is it > "Foo\nBar\nBackwardCompat" or ["Foo", "Bar", "BackwardCompat"], or > [Requirement.parse("Foo"), Requirement.parse('Bar"), > Requirement.parse("BackwardCompat")]. > > And given other variables (ones that perhaps distribute doesn't even > know about) how are they combined? Hmm, setuptools already has a concept of "Features" (see setuptools.dist.Feature). Is that sense relevant here? >>> Is there a way to eliminate values? Right now, for instance, if you >>> have "[build] home = /foo" in a distutils.cfg, there's no way to unset >>> that. I'd like to see this functionality included. Perhaps to delete >>> a specific value, but the simpler case of deleting a variable is >>> really all I want. >> Do you have a syntax in mind for this ? Excluding a Feature causes packages to be dropped (e.g., from sdist / bdist operations). > Well, the way I added more meta-operations in Paste Deploy (which uses > ConfigParser) was to mess with the variable names. So maybe: > > [metadata:python_version=="2.5"] > del requires = Foo > > It might have a value to delete a specific value (or line) from that > "requires" variable, or if empty it would clear that variable. Since > ConfigParser doesn't order the variables, it would delete them from > "earlier" sections, not the current section. This requires defining > what makes a section "earlier". Or maybe it would delete "Foo" from > whatever the value of requires ends up being, or if you do "del > requires =" would make that section provide the only value for > requires. > > > Another thought: one thing distutils does but sometimes confuses me is > that it has boolean options (options which take no values, sometimes > with an option and an anti-option, e.g., --feature-foo and > --no-feature-foo) which turn into a single configuration variable with > a boolean value. How that is supposed to work seems apropos (it's not > strictly syntax, but I don't feel like I can judge the syntax without > considering the full semantics of the configuration file, because the > semantics drive the actual use cases which should in turn be used to > judge the syntax). - -- =================================================================== Tres Seaver +1 540-429-0999 tseaver at palladion.com Palladion Software "Excellence by Design" http://palladion.com -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iEYEARECAAYFAkrUwZQACgkQ+gerLs4ltQ47EgCfddqCE/g7HT5Kmfay+/+KhJMp UHEAoLWAqxJkUDk4UsxBTbfgRafiigKu =iH89 -----END PGP SIGNATURE----- From pje at telecommunity.com Tue Oct 13 20:08:05 2009 From: pje at telecommunity.com (P.J. Eby) Date: Tue, 13 Oct 2009 14:08:05 -0400 Subject: [Distutils] dev versions In-Reply-To: References: Message-ID: <20091013180814.CD45E3A4068@sparrow.telecommunity.com> At 01:45 PM 10/13/2009 -0400, Tres Seaver wrote: >Reinout van Rees wrote: > > On 2009-10-08, Ian Bicking wrote: > >> So after creating, say, version 0.3.1, I always mark a package > as 0.3.2dev. > >> But this is annoying, you might never create a version 0.3.2 (e.g., 0.4 > >> might be the next level). So, it would be better to use something like > >> 0.3.1~dev. What is considered best practice for this? Ideally something > >> that works with both Setuptools and the upcoming Distribute version spec. "0.3.1-anything" will sort after "0.3.1" and before "0.3.2dev", so that's probably what you want. Typically, the "anything" would be a revision number or date tag (e.g r5143 or 20091005), but it could also be a manually-assigned sequential number. Technically, this is a "post-release" tag, and in the previous proposal for a restricted version format, you would designate it with 0.3.1.postNNNN. From jim at zope.com Tue Oct 13 20:51:41 2009 From: jim at zope.com (Jim Fulton) Date: Tue, 13 Oct 2009 14:51:41 -0400 Subject: [Distutils] distribute and buildout: best practices? In-Reply-To: References: <94bdd2610910070453t63e7e4c4h1c9bf6ca567fa48f@mail.gmail.com> <4ACF1999.4080001@simplistix.co.uk> Message-ID: <1099b90b0910131151q571cd352ta24b2b35d77422a8@mail.gmail.com> On Tue, Oct 13, 2009 at 1:30 PM, Tres Seaver wrote: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > Reinout van Rees wrote: >> On 2009-10-09, Chris Withers wrote: >>> Reinout van Rees wrote: >>>> I'm still not 100% sure whether it is safe to put "distribute" in the >>>> install_requires list of a package right now, however. >>> As with setuptools, why do you think you need to? >> >> Namespace packages. ?You get warnings "hey, no setuptools dependency" once you >> use namespaces. > > That is a buildout bug: ?whining about the lack of depenedency on the > only package which makes dependencies meaningful is silly. ?No package > should *ever* need to declare that it depends on setuptools. I disagree. Dependencies only matter for installation. Lots of packages that are installed with setuptools don't need setuptools in their path to run. If a package imports setuptools, it should declare it as a dependency. The distribute fork and masquerade of setuptools makes this more complicated than it should be. Really the run-time code needed to support namespace packages should be split out into a separate package and eventually added to the standard library. Jim -- Jim Fulton From jim at zope.com Tue Oct 13 21:02:22 2009 From: jim at zope.com (Jim Fulton) Date: Tue, 13 Oct 2009 15:02:22 -0400 Subject: [Distutils] [buildout] RFC: buildout support for distribute Message-ID: <1099b90b0910131202x3425042ft1ba3c6035c56c13c@mail.gmail.com> I plan to add explicit buildout support for distribute. Here's a sketch. Basically it boils down to satisfying requirements for setuptools with distribute whenever distribute is in buildout's working set. So, when installing a package that requires setuptools, it will convert that requirement to a requirement for distribute. There will be 2 separate bootstrap scripts, one that uses distribute and one that uses setuptools. There will also be a buildout configuration option that will allow a buildout configuration author to fix which is used. Probably: use-distribute = true This will only serve as a check that the right bootstrap was used. Comments? Jim -- Jim Fulton From hanno at hannosch.eu Tue Oct 13 21:17:40 2009 From: hanno at hannosch.eu (Hanno Schlichting) Date: Tue, 13 Oct 2009 21:17:40 +0200 Subject: [Distutils] [buildout] RFC: buildout support for distribute In-Reply-To: <1099b90b0910131202x3425042ft1ba3c6035c56c13c@mail.gmail.com> References: <1099b90b0910131202x3425042ft1ba3c6035c56c13c@mail.gmail.com> Message-ID: <5cae42b20910131217y5c1aa5nfb9fa86c6902f032@mail.gmail.com> On Tue, Oct 13, 2009 at 9:02 PM, Jim Fulton wrote: > I plan to add explicit buildout support for distribute. ?Here's a > sketch. Basically it boils down to satisfying requirements for > setuptools with distribute whenever distribute is in buildout's > working set. ?So, when installing a package that requires setuptools, > it will convert that requirement to a requirement for distribute. > There will be 2 separate bootstrap scripts, one that uses distribute > and one that uses setuptools. ?There will also be a buildout > configuration option that will allow a buildout configuration author > to fix which is used. ?Probably: > > use-distribute = true > > This will only serve as a check that the right bootstrap was used. Sounds good. Tarek already implemented the separate bootstrap file including the requirements conversion. This is currently done via monkey-patches of buildout, which could be avoided if this finds its way into buildout itself. Hanno From pje at telecommunity.com Tue Oct 13 21:22:34 2009 From: pje at telecommunity.com (P.J. Eby) Date: Tue, 13 Oct 2009 15:22:34 -0400 Subject: [Distutils] distribute and buildout: best practices? In-Reply-To: <1099b90b0910131151q571cd352ta24b2b35d77422a8@mail.gmail.co m> References: <94bdd2610910070453t63e7e4c4h1c9bf6ca567fa48f@mail.gmail.com> <4ACF1999.4080001@simplistix.co.uk> <1099b90b0910131151q571cd352ta24b2b35d77422a8@mail.gmail.com> Message-ID: <20091013192243.1B1A43A4068@sparrow.telecommunity.com> At 02:51 PM 10/13/2009 -0400, Jim Fulton wrote: >Really the run-time >code needed to support namespace packages should be split out into a >separate package and eventually added to the standard library. Are you volunteering? ;-) Seriously, MvL's namespace package PEP (#382) already takes care of most everything that's needed. With it in place, the only advantage of setuptools' approach is the ability to add more items to a package __path__ after it's already imported. (i.e., the fixup feature.) I heartily approve of that PEP, in case it's not obvious. I'd love to see it in Python 2.7, and will happily add support for it in setuptools 0.7. From ziade.tarek at gmail.com Tue Oct 13 21:25:03 2009 From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=) Date: Tue, 13 Oct 2009 21:25:03 +0200 Subject: [Distutils] [buildout] RFC: buildout support for distribute In-Reply-To: <1099b90b0910131202x3425042ft1ba3c6035c56c13c@mail.gmail.com> References: <1099b90b0910131202x3425042ft1ba3c6035c56c13c@mail.gmail.com> Message-ID: <94bdd2610910131225h4305ee38kc2c1d21ba33313b9@mail.gmail.com> On Tue, Oct 13, 2009 at 9:02 PM, Jim Fulton wrote: > I plan to add explicit buildout support for distribute. ?Here's a > sketch. Basically it boils down to satisfying requirements for > setuptools with distribute whenever distribute is in buildout's > working set. ?So, when installing a package that requires setuptools, > it will convert that requirement to a requirement for distribute. > There will be 2 separate bootstrap scripts, one that uses distribute > and one that uses setuptools. ?There will also be a buildout > configuration option that will allow a buildout configuration author > to fix which is used. ?Probably: > > use-distribute = true > > This will only serve as a check that the right bootstrap was used. > Notice that the current trunk of Distribute is now changing any 'setuptools' requirement that is tiggered using "Requirement.parse" and "resolve" APIs into a 'distribute' one. Meaning that changes on zc.buildout side shouldn't be important as we don't require any monkey-patch anymore. So basically it would consist of picking the right bootstrap file URL for the "use_setuptools" call. Our bootstrap works as expected (besides a few things we need to iron out with Reinout) with the current Distribute trunk : http://bitbucket.org/tarek/buildout-distribute/src/tip/bootstrap_dev.py You can get it and run it at: http://nighty.ziade.org/bootstrap_dev.py It'll get the latest distribute we are working on. So I would propose changing bootstrap.py itself to choose between peak's ez_setup.py or our distribute_setup.py file with the option you've mentioned, and through the shell: $ python bootstrap.py --use-distribute I can't think of any other changed required in zc.buildout itself besides this. Tarek From ziade.tarek at gmail.com Tue Oct 13 21:27:29 2009 From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=) Date: Tue, 13 Oct 2009 21:27:29 +0200 Subject: [Distutils] [buildout] RFC: buildout support for distribute In-Reply-To: <5cae42b20910131217y5c1aa5nfb9fa86c6902f032@mail.gmail.com> References: <1099b90b0910131202x3425042ft1ba3c6035c56c13c@mail.gmail.com> <5cae42b20910131217y5c1aa5nfb9fa86c6902f032@mail.gmail.com> Message-ID: <94bdd2610910131227i5d4b132ds10a8f9e045b518ce@mail.gmail.com> On Tue, Oct 13, 2009 at 9:17 PM, Hanno Schlichting wrote: > > Sounds good. Tarek already implemented the separate bootstrap file > including the requirements conversion. This is currently done via > monkey-patches of buildout, which could be avoided if this finds its > way into buildout itself. > Yes that has changed yesterday. No more patches are required, since Distribute takes care of the distribute/setuptools requirements calls. Our bootstrap now is pretty similar to the original, besides the url we are calling. Tarek From jim at zope.com Tue Oct 13 21:38:53 2009 From: jim at zope.com (Jim Fulton) Date: Tue, 13 Oct 2009 15:38:53 -0400 Subject: [Distutils] [buildout] RFC: buildout support for distribute In-Reply-To: <94bdd2610910131225h4305ee38kc2c1d21ba33313b9@mail.gmail.com> References: <1099b90b0910131202x3425042ft1ba3c6035c56c13c@mail.gmail.com> <94bdd2610910131225h4305ee38kc2c1d21ba33313b9@mail.gmail.com> Message-ID: <1099b90b0910131238u52b1b864v5010dba157e73c56@mail.gmail.com> On Tue, Oct 13, 2009 at 3:25 PM, Tarek Ziad? wrote: ... > Notice that the current trunk of Distribute is now changing any 'setuptools' > requirement that is tiggered using "Requirement.parse" and "resolve" APIs into > a 'distribute' one. Ah cool. ... > So I would propose changing bootstrap.py itself to choose between > peak's ez_setup.py or our distribute_setup.py file > with the option you've mentioned, and through the shell: > > $ python bootstrap.py --use-distribute > > I can't think of any other changed required in zc.buildout itself besides this. Very good. Thanks. Jim -- Jim Fulton From jim at zope.com Tue Oct 13 21:40:47 2009 From: jim at zope.com (Jim Fulton) Date: Tue, 13 Oct 2009 15:40:47 -0400 Subject: [Distutils] distribute and buildout: best practices? In-Reply-To: <20091013192243.1B1A43A4068@sparrow.telecommunity.com> References: <94bdd2610910070453t63e7e4c4h1c9bf6ca567fa48f@mail.gmail.com> <4ACF1999.4080001@simplistix.co.uk> <1099b90b0910131151q571cd352ta24b2b35d77422a8@mail.gmail.com> <20091013192243.1B1A43A4068@sparrow.telecommunity.com> Message-ID: <1099b90b0910131240p56cb3899j3c949984e31c9e46@mail.gmail.com> On Tue, Oct 13, 2009 at 3:22 PM, P.J. Eby wrote: > At 02:51 PM 10/13/2009 -0400, Jim Fulton wrote: >> >> Really the run-time >> code needed to support namespace packages should be split out into a >> separate package and eventually added to the standard library. > > Are you volunteering? ?;-) No. :) I'd love to, but I'm over committed as it is. > Seriously, MvL's namespace package PEP (#382) already takes care of most > everything that's needed. ?With it in place, the only advantage of > setuptools' approach is the ability to add more items to a package __path__ > after it's already imported. ?(i.e., the fixup feature.) > > I heartily approve of that PEP, in case it's not obvious. ?I'd love to see > it in Python 2.7, and will happily add support for it in setuptools 0.7. Awesome! Jim -- Jim Fulton From pje at telecommunity.com Tue Oct 13 21:46:28 2009 From: pje at telecommunity.com (P.J. Eby) Date: Tue, 13 Oct 2009 15:46:28 -0400 Subject: [Distutils] [buildout] RFC: buildout support for distribute In-Reply-To: <94bdd2610910131225h4305ee38kc2c1d21ba33313b9@mail.gmail.co m> References: <1099b90b0910131202x3425042ft1ba3c6035c56c13c@mail.gmail.com> <94bdd2610910131225h4305ee38kc2c1d21ba33313b9@mail.gmail.com> Message-ID: <20091013194636.BA8073A4068@sparrow.telecommunity.com> At 09:25 PM 10/13/2009 +0200, Tarek Ziad? wrote: >Notice that the current trunk of Distribute is now changing any 'setuptools' >requirement that is tiggered using "Requirement.parse" and "resolve" APIs into >a 'distribute' one. Please note that this change will cause problems for people in the future, as new features are introduced in the 0.7a1 line. The earlier version of this hack, which only ignored requests for older versions of setuptools, would not have caused such a problem. I would suggest that Distribute should only silently ignore requests for versions of setuptools that it can explicitly emulate; other requests should result in an error, so that a problem cannot pass silently. Otherwise, a package depending on new features in the 0.7 line will silently fail when those features are not present in its operating environment. From ziade.tarek at gmail.com Tue Oct 13 21:53:03 2009 From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=) Date: Tue, 13 Oct 2009 21:53:03 +0200 Subject: [Distutils] [buildout] RFC: buildout support for distribute In-Reply-To: <20091013194636.BA8073A4068@sparrow.telecommunity.com> References: <1099b90b0910131202x3425042ft1ba3c6035c56c13c@mail.gmail.com> <94bdd2610910131225h4305ee38kc2c1d21ba33313b9@mail.gmail.com> <20091013194636.BA8073A4068@sparrow.telecommunity.com> Message-ID: <94bdd2610910131253r6b5451e9j5bdd15bc4e78d67c@mail.gmail.com> 2009/10/13 P.J. Eby : > At 09:25 PM 10/13/2009 +0200, Tarek Ziad? wrote: >> >> Notice that the current trunk of Distribute is now changing any >> 'setuptools' >> requirement that is tiggered using "Requirement.parse" and "resolve" APIs >> into >> a 'distribute' one. > > Please note that this change will cause problems for people in the future, > as new features are introduced in the 0.7a1 line. ?The earlier version of > this hack, which only ignored requests for older versions of setuptools, > would not have caused such a problem. > > I would suggest that Distribute should only silently ignore requests for > versions of setuptools that it can explicitly emulate; other requests should > result in an error, so that a problem cannot pass silently. > > Otherwise, a package depending on new features in the 0.7 line will silently > fail when those features are not present in its operating environment. Ok sure, makes sense. We will return 'distribute' only if 'setuptools' is from the 0.6.x series and below. From pje at telecommunity.com Tue Oct 13 22:07:22 2009 From: pje at telecommunity.com (P.J. Eby) Date: Tue, 13 Oct 2009 16:07:22 -0400 Subject: [Distutils] [buildout] RFC: buildout support for distribute In-Reply-To: <94bdd2610910131253r6b5451e9j5bdd15bc4e78d67c@mail.gmail.co m> References: <1099b90b0910131202x3425042ft1ba3c6035c56c13c@mail.gmail.com> <94bdd2610910131225h4305ee38kc2c1d21ba33313b9@mail.gmail.com> <20091013194636.BA8073A4068@sparrow.telecommunity.com> <94bdd2610910131253r6b5451e9j5bdd15bc4e78d67c@mail.gmail.com> Message-ID: <20091013200732.517E93A4068@sparrow.telecommunity.com> At 09:53 PM 10/13/2009 +0200, Tarek Ziad? wrote: >2009/10/13 P.J. Eby : > > At 09:25 PM 10/13/2009 +0200, Tarek Ziad? wrote: > >> > >> Notice that the current trunk of Distribute is now changing any > >> 'setuptools' > >> requirement that is tiggered using "Requirement.parse" and "resolve" APIs > >> into > >> a 'distribute' one. > > > > Please note that this change will cause problems for people in the future, > > as new features are introduced in the 0.7a1 line. The earlier version of > > this hack, which only ignored requests for older versions of setuptools, > > would not have caused such a problem. > > > > I would suggest that Distribute should only silently ignore requests for > > versions of setuptools that it can explicitly emulate; other > requests should > > result in an error, so that a problem cannot pass silently. > > > > Otherwise, a package depending on new features in the 0.7 line > will silently > > fail when those features are not present in its operating environment. > >Ok sure, makes sense. We will return 'distribute' only if 'setuptools' >is from the 0.6.x series >and below. Great. I assume that means you plan to incorporate any further bug fixes that land in 0.6c10 (or, gods forbid, c11) in the release where this change is made. For versions you don't support emulating, I strongly suggest adding an error of some kind, since actually *installing* setuptools with Distribute still present on sys.path would cause a problem. From ziade.tarek at gmail.com Tue Oct 13 22:27:04 2009 From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=) Date: Tue, 13 Oct 2009 22:27:04 +0200 Subject: [Distutils] [buildout] RFC: buildout support for distribute In-Reply-To: <20091013200732.517E93A4068@sparrow.telecommunity.com> References: <1099b90b0910131202x3425042ft1ba3c6035c56c13c@mail.gmail.com> <94bdd2610910131225h4305ee38kc2c1d21ba33313b9@mail.gmail.com> <20091013194636.BA8073A4068@sparrow.telecommunity.com> <94bdd2610910131253r6b5451e9j5bdd15bc4e78d67c@mail.gmail.com> <20091013200732.517E93A4068@sparrow.telecommunity.com> Message-ID: <94bdd2610910131327o21d95217l1ff33e86f08ab434@mail.gmail.com> On Tue, Oct 13, 2009 at 10:07 PM, P.J. Eby wrote: >> Ok sure, makes sense. We will return 'distribute' only if 'setuptools' >> is from the 0.6.x series >> and below. > > Great. ?I assume that means you plan to incorporate any further bug fixes > that land in 0.6c10 (or, gods forbid, c11) in the release where this change > is made. Yes, as long as we are able to spot them in your commits / issue trackers changes. The latest commit you've made in setuptools package is still cryptic to us because it fixes many things at once. We were able to backport the sandbox violation but that's it at this point. > For versions you don't support emulating, I strongly suggest adding an error > of some kind, since actually *installing* setuptools with Distribute still > present on sys.path would cause a problem. It is still unclear to me where this exception has to be thrown since some APIs in WorkingSet can be used just to check things out. I am guessing WorkingSet.resolve and parse_requirements are enough, but that's just from the zc.buildout use case. From reinout at vanrees.org Tue Oct 13 22:28:31 2009 From: reinout at vanrees.org (Reinout van Rees) Date: Tue, 13 Oct 2009 20:28:31 +0000 (UTC) Subject: [Distutils] distribute and buildout: best practices? References: <94bdd2610910070453t63e7e4c4h1c9bf6ca567fa48f@mail.gmail.com> <4ACF1999.4080001@simplistix.co.uk> <1099b90b0910131151q571cd352ta24b2b35d77422a8@mail.gmail.com> Message-ID: On 2009-10-13, Jim Fulton wrote: [snip] As I reported a problem with the distribute/buildout combination earlier: The original problems I reported (and some others that I found) will all be solved in the upcoming 0.6.5 release! I worked with tarek today and yesterday to find (mainly me) and fix (mainly Tarek) small bugs. It works surprisingly elegant now. - You can include distribute safely in a dependency provided it is 0.6.5 or better. Also in buildouts that don't have a new bootstrap. - A shared eggs directory is no problem. The setuptools egg stays unmolested and coexists happily. - A new bootstrap is available that elegantly makes sure bin/buildout uses only distribute. This in turn ensures that no setuptools egg will show up anywhere else in any of the binaries. So... later tonight or tomorrow there can be rejoicing in buildout land. At least, that's what my automated buildbot tests say :-) Reinout -- Reinout van Rees - reinout at vanrees.org - http://reinout.vanrees.org Software developer at http://www.thehealthagency.com "Military engineers build missiles. Civil engineers build targets" From david.lyon at preisshare.net Tue Oct 13 23:20:02 2009 From: david.lyon at preisshare.net (David Lyon) Date: Tue, 13 Oct 2009 17:20:02 -0400 Subject: [Distutils] Alternate static metadata PEP submission... In-Reply-To: <94bdd2610910131001v70711ffeia27433cd23b32fca@mail.gmail.com> References: <20091013103021.GA21066@laurie.devork> <94bdd2610910131001v70711ffeia27433cd23b32fca@mail.gmail.com> Message-ID: <79a53e276e5dffe24be0433771801d72@preisshare.net> On Tue, 13 Oct 2009 19:01:30 +0200, Tarek Ziad? wrote: > If wanted, I can go ahead I create a new PEP for David's proposal. And > David can work it out > using PEP 390 references maybe > > let me know Thank you Tarek, I appreciate it. Good management decision :-) David From ziade.tarek at gmail.com Tue Oct 13 23:26:57 2009 From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=) Date: Tue, 13 Oct 2009 23:26:57 +0200 Subject: [Distutils] Alternate static metadata PEP submission... In-Reply-To: <79a53e276e5dffe24be0433771801d72@preisshare.net> References: <20091013103021.GA21066@laurie.devork> <94bdd2610910131001v70711ffeia27433cd23b32fca@mail.gmail.com> <79a53e276e5dffe24be0433771801d72@preisshare.net> Message-ID: <94bdd2610910131426k64f707d7pf4d398d04b8ecc32@mail.gmail.com> On Tue, Oct 13, 2009 at 11:20 PM, David Lyon wrote: > On Tue, 13 Oct 2009 19:01:30 +0200, Tarek Ziad? > wrote: > >> If wanted, I can go ahead I create a new PEP for David's proposal. And >> David can work it out >> using PEP 390 references maybe >> >> let me know > > Thank you Tarek, > > I appreciate it. So is that a yes ? Tarek. From david.lyon at preisshare.net Tue Oct 13 23:31:32 2009 From: david.lyon at preisshare.net (David Lyon) Date: Tue, 13 Oct 2009 17:31:32 -0400 Subject: [Distutils] Alternate static metadata PEP submission... In-Reply-To: <94bdd2610910131426k64f707d7pf4d398d04b8ecc32@mail.gmail.com> References: <20091013103021.GA21066@laurie.devork> <94bdd2610910131001v70711ffeia27433cd23b32fca@mail.gmail.com> <79a53e276e5dffe24be0433771801d72@preisshare.net> <94bdd2610910131426k64f707d7pf4d398d04b8ecc32@mail.gmail.com> Message-ID: <7282def7aa3d67f52ceba54674dd774f@preisshare.net> On Tue, 13 Oct 2009 23:26:57 +0200, Tarek Ziad? wrote: >>> If wanted, I can go ahead I create a new PEP for David's proposal. And >>> David can work it out >>> using PEP 390 references maybe >>> >>> let me know >> >> Thank you Tarek, >> >> I appreciate it. > > So is that a yes ? I will answer offlist.. Thanks Tarek From pje at telecommunity.com Wed Oct 14 01:18:21 2009 From: pje at telecommunity.com (P.J. Eby) Date: Tue, 13 Oct 2009 19:18:21 -0400 Subject: [Distutils] [buildout] RFC: buildout support for distribute In-Reply-To: <94bdd2610910131327o21d95217l1ff33e86f08ab434@mail.gmail.co m> References: <1099b90b0910131202x3425042ft1ba3c6035c56c13c@mail.gmail.com> <94bdd2610910131225h4305ee38kc2c1d21ba33313b9@mail.gmail.com> <20091013194636.BA8073A4068@sparrow.telecommunity.com> <94bdd2610910131253r6b5451e9j5bdd15bc4e78d67c@mail.gmail.com> <20091013200732.517E93A4068@sparrow.telecommunity.com> <94bdd2610910131327o21d95217l1ff33e86f08ab434@mail.gmail.com> Message-ID: <20091013231830.DEF6B3A4068@sparrow.telecommunity.com> At 10:27 PM 10/13/2009 +0200, Tarek Ziad? wrote: >On Tue, Oct 13, 2009 at 10:07 PM, P.J. Eby wrote: > >> Ok sure, makes sense. We will return 'distribute' only if 'setuptools' > >> is from the 0.6.x series > >> and below. > > > > Great. I assume that means you plan to incorporate any further bug fixes > > that land in 0.6c10 (or, gods forbid, c11) in the release where this change > > is made. > >Yes, as long as we are able to spot them in your commits / issue >trackers changes. > >The latest commit you've made in setuptools package is still cryptic >to us because >it fixes many things at once. Sorry about that, but it was pretty much the only way to get it done in a weekend, without the overheads of separate commit messages, doc changes, and backporting killing me. Those overheads were the main reason I wasn't making changes more often; I dreaded the amount of work involved. Also, as it happened, I was able to fix multiple problems with single changes this way. >We were able to backport the sandbox violation but that's it at this point. The __getstate__/__setstate__ stuff in pkg_resources is also sandbox support; it makes it possible to keep working_set changes by recursive builds from corrupting the parent build's working set. I also expect it to support a "reset pkg_resources" API in 0.7a1 -- a long-requested feature. > > For versions you don't support emulating, I strongly suggest > adding an error > > of some kind, since actually *installing* setuptools with Distribute still > > present on sys.path would cause a problem. > >It is still unclear to me where this exception has to be thrown >since some APIs >in WorkingSet can be used just to check things out. > >I am guessing WorkingSet.resolve and parse_requirements are enough, but that's >just from the zc.buildout use case. One thing you could possibly do is capture Distribution.insert_on(), which would allow you to detect that setuptools is about to be added to a working set. Still another is that you could install a distribute.pth that puts Distribute on the path first, since 'distribute.pth' is alphabetically before 'easy-install.pth' and 'setuptools.pth'. (I'm not suggesting these are complete or risk-free solutions, of course.) From floris.bruynooghe at gmail.com Wed Oct 14 01:36:54 2009 From: floris.bruynooghe at gmail.com (Floris Bruynooghe) Date: Wed, 14 Oct 2009 00:36:54 +0100 Subject: [Distutils] Alternate static metadata PEP submission... In-Reply-To: <94bdd2610910131426k64f707d7pf4d398d04b8ecc32@mail.gmail.com> References: <20091013103021.GA21066@laurie.devork> <94bdd2610910131001v70711ffeia27433cd23b32fca@mail.gmail.com> <79a53e276e5dffe24be0433771801d72@preisshare.net> <94bdd2610910131426k64f707d7pf4d398d04b8ecc32@mail.gmail.com> Message-ID: <20091013233654.GA27339@laurie.devork> On Tue, Oct 13, 2009 at 11:26:57PM +0200, Tarek Ziad? wrote: > On Tue, Oct 13, 2009 at 11:20 PM, David Lyon wrote: > > On Tue, 13 Oct 2009 19:01:30 +0200, Tarek Ziad? > > wrote: > > > >> If wanted, I can go ahead I create a new PEP for David's proposal. And > >> David can work it out > >> using PEP 390 references maybe > >> > >> let me know > > > > Thank you Tarek, > > > > I appreciate it. > > So is that a yes ? To me it would seem a little early to start a PEP like this, there's been virtually no discussion about this particular proposal nor any proof of concept code. And given the scope of wanting to change the entire way build-steps are declared some code that can give experience and show the good and bad points seems important to me. Floris -- Debian GNU/Linux -- The Power of Freedom www.debian.org | www.gnu.org | www.kernel.org From david.lyon at preisshare.net Wed Oct 14 01:53:40 2009 From: david.lyon at preisshare.net (David Lyon) Date: Tue, 13 Oct 2009 19:53:40 -0400 Subject: [Distutils] Alternate static metadata PEP submission... In-Reply-To: <20091013233654.GA27339@laurie.devork> References: <20091013103021.GA21066@laurie.devork> <94bdd2610910131001v70711ffeia27433cd23b32fca@mail.gmail.com> <79a53e276e5dffe24be0433771801d72@preisshare.net> <94bdd2610910131426k64f707d7pf4d398d04b8ecc32@mail.gmail.com> <20091013233654.GA27339@laurie.devork> Message-ID: <5fba01026f52bbf1d89cac4908ee17dc@preisshare.net> On Wed, 14 Oct 2009 00:36:54 +0100, Floris Bruynooghe wrote: > To me it would seem a little early to start a PEP like this, there's > been virtually no discussion about this particular proposal nor any > proof of concept code. And given the scope of wanting to change the > entire way build-steps are declared some code that can give experience > and show the good and bad points seems important to me. There's been much discussion on the distutils sig about static metadata this year. So I have to disagree with you there on that. However, you are correct about proof of concept code, and in some respects that's the easy part. I will start work on that immediately. Central to this proposal is providing a boilerplate setup.py that is able to do an install of a package/application entirely from the information contained in the setup.info file. What it means is that a developer won't need to write a setup.py at all. Just use the one provided and edit the data in the setup.info file. Thank you for your suggestion. David From david.lyon at preisshare.net Wed Oct 14 02:06:56 2009 From: david.lyon at preisshare.net (David Lyon) Date: Tue, 13 Oct 2009 20:06:56 -0400 Subject: [Distutils] Alternate static metadata PEP submission... In-Reply-To: <20091013103021.GA21066@laurie.devork> References: <20091013103021.GA21066@laurie.devork> Message-ID: <9962812fc49fb8f3ef68d624b7e8e151@preisshare.net> On Tue, 13 Oct 2009 11:30:21 +0100, Floris Bruynooghe wrote: > This seems to propose a whole build system, while PEP390 only aims for > static metadata. It's not a whole new build system, but potentially a whole new install system. There's a big difference. However, the whole new install system is likely to be contained within the boilerplate setup.py, where those facilities don't already exist within distutils now. For the developer, that's really easy.. Just take a copy of the boilerplate setup.py... > Not restricting it to static metadata makes the > domain a lot wider and complicated as shown by countless previous > discussions on this list (several of these things have been outlined > before). Yes, it is much further reaching and broader. It's actually less complicated than you might think. What I'm proposing is less complicated than say.... setuptools.. > Since the problem domain is so huge it will take a very long > time to come up with a stable version that addresses all issues > adequately, Define long time.. I believe only a few months.. > something which this PEP sidesteps solving the problem it > targets in a sensible and contained way. I disagree. I believe that it's a small programming project amounting to no more than a few thousand lines of code. To me, the email correspondence and mailing list activity will be more voluminous. > It certainly is a laudable goal to come up with such a build system > but I don't think it can be seen as *instead of* the metatdata PEP. > IMHO such a system should be using the static metadata from whatever > PEP 390 will become. The metadata is the build/install system. How metadata is specified in setup.py now is dated. It's hard to understand for developers and hard to cajole into something that works well on all platforms. My proposal is actually about taking the shortest and easiest route to cleaning up python package/application installation issues. The end result is a setup.py --install and a setup.info file. That doesn't seem so overly complicated to me. David From ianb at colorstudy.com Wed Oct 14 05:05:25 2009 From: ianb at colorstudy.com (Ian Bicking) Date: Tue, 13 Oct 2009 22:05:25 -0500 Subject: [Distutils] Alternate static metadata PEP submission... In-Reply-To: <20091013233654.GA27339@laurie.devork> References: <20091013103021.GA21066@laurie.devork> <94bdd2610910131001v70711ffeia27433cd23b32fca@mail.gmail.com> <79a53e276e5dffe24be0433771801d72@preisshare.net> <94bdd2610910131426k64f707d7pf4d398d04b8ecc32@mail.gmail.com> <20091013233654.GA27339@laurie.devork> Message-ID: On Tue, Oct 13, 2009 at 6:36 PM, Floris Bruynooghe wrote: > To me it would seem a little early to start a PEP like this, there's > been virtually no discussion about this particular proposal nor any > proof of concept code. ?And given the scope of wanting to change the > entire way build-steps are declared some code that can give experience > and show the good and bad points seems important to me. Could an example API be encapsulated in something like this in setup.py? from test_this_pep import setup_cfg setup(other args, **setup_cfg()) Then packages could be converted to test it out, without breaking the package. -- Ian Bicking | http://blog.ianbicking.org | http://topplabs.org/civichacker From david at ar.media.kyoto-u.ac.jp Wed Oct 14 05:10:48 2009 From: david at ar.media.kyoto-u.ac.jp (David Cournapeau) Date: Wed, 14 Oct 2009 12:10:48 +0900 Subject: [Distutils] Alternate static metadata PEP submission... In-Reply-To: References: <20091013103021.GA21066@laurie.devork> <94bdd2610910131001v70711ffeia27433cd23b32fca@mail.gmail.com> <79a53e276e5dffe24be0433771801d72@preisshare.net> <94bdd2610910131426k64f707d7pf4d398d04b8ecc32@mail.gmail.com> <20091013233654.GA27339@laurie.devork> Message-ID: <4AD54138.9010602@ar.media.kyoto-u.ac.jp> Ian Bicking wrote: > On Tue, Oct 13, 2009 at 6:36 PM, Floris Bruynooghe > wrote: > >> To me it would seem a little early to start a PEP like this, there's >> been virtually no discussion about this particular proposal nor any >> proof of concept code. And given the scope of wanting to change the >> entire way build-steps are declared some code that can give experience >> and show the good and bad points seems important to me. >> > > Could an example API be encapsulated in something like this in setup.py? > > from test_this_pep import setup_cfg > setup(other args, **setup_cfg()) > > Then packages could be converted to test it out, without breaking the package. > This would work for the parts which are concerned using static metadata for what exists in distutils, but it would not work for pre_install, etc... The only way I can think to make this work is to actually generate a setup.py (or in the other direction, to generate a setup.info from setup.py). I think it is much better long-term solution for a 'distutils v2' than trying to fix the existing code, but it is certainly a lot of work, David From david.lyon at preisshare.net Wed Oct 14 06:03:25 2009 From: david.lyon at preisshare.net (David Lyon) Date: Wed, 14 Oct 2009 00:03:25 -0400 Subject: [Distutils] Alternate static metadata PEP submission... In-Reply-To: References: <20091013103021.GA21066@laurie.devork> <94bdd2610910131001v70711ffeia27433cd23b32fca@mail.gmail.com> <79a53e276e5dffe24be0433771801d72@preisshare.net> <94bdd2610910131426k64f707d7pf4d398d04b8ecc32@mail.gmail.com> <20091013233654.GA27339@laurie.devork> Message-ID: On Tue, 13 Oct 2009 22:05:25 -0500, Ian Bicking wrote: > Could an example API be encapsulated in something like this in setup.py? > > from test_this_pep import setup_cfg > setup(other args, **setup_cfg()) > > Then packages could be converted to test it out, without breaking the > package. Actually, my proposal doesn't break any package. Distutils included. There's one fatal flaw with patching distutils. That is that any patches done to distutils won't automatically find their way onto all the old systems that are out there. What we actually need, and I am building, is a setup.py script that is going to be a 'trimmed' version of distutils that includes all neccessary package/application installation code to do the install from the setup.info file. In a radical departure from the old way, you *don't* edit the setup.py. Just edit the setup.info. Then attach the setup.py unmodified. I'm very sure that Tarek is going to have many good suggestions on the best way to write this *new* boiler-plate setup.py and it is likely that it will turn out a joint effort, because he writes nicer code than me. Maybe I will beg him to help me.. Still, I need to do a proof of concept. Thats what I'm starting now. ---setup.py--------------------------------------------- #---------------------------------------------------- # -- DO NOT EDIT THIS FILE # # This file is the standard setup.py for your # application and you don't need to edit it. # # It was provided from the new distutils setup system # and can be downloaded from: # # www.python.org/distutils/standardsetup/setup.py # # Instead, modify your setup.info file and # provide a pre-install or post-install script # to do your custom processing if that is required. #---------------------------------------------------- # .. .. -------------------------------------------------------- ---setup.info---- [setup] name = artistflair version = 1.2 description_file = artisticflair.rst .. .. ----------------- David yes - it's setup.py upside down... From david.lyon at preisshare.net Wed Oct 14 07:15:10 2009 From: david.lyon at preisshare.net (David Lyon) Date: Wed, 14 Oct 2009 01:15:10 -0400 Subject: [Distutils] Alternate static metadata PEP submission... In-Reply-To: <4AD54138.9010602@ar.media.kyoto-u.ac.jp> References: <20091013103021.GA21066@laurie.devork> <94bdd2610910131001v70711ffeia27433cd23b32fca@mail.gmail.com> <79a53e276e5dffe24be0433771801d72@preisshare.net> <94bdd2610910131426k64f707d7pf4d398d04b8ecc32@mail.gmail.com> <20091013233654.GA27339@laurie.devork> <4AD54138.9010602@ar.media.kyoto-u.ac.jp> Message-ID: <50a1116c3a8ca9bcbb28121d90ccc5d0@preisshare.net> On Wed, 14 Oct 2009 12:10:48 +0900, David Cournapeau wrote: > The only way I can think to make this work is to actually > generate a setup.py That's right... > (or in the other direction, to generate a setup.info > from setup.py). Yes, that will be required also.. > I think it is much better long-term solution for a > 'distutils v2' ... My proposal is to build the installation section of 'distutils v2' into a setup.py. And share it around. > than trying to fix the existing code, but it is > certainly a lot of work, What's a few thousand lines of code between friends.. But don't forget it's the one chance that we get to build a whole new system with completely fresh code. After so many years of the old code.. I think we need a chance to get some new code.. David From david.lyon at preisshare.net Wed Oct 14 09:06:01 2009 From: david.lyon at preisshare.net (David Lyon) Date: Wed, 14 Oct 2009 03:06:01 -0400 Subject: [Distutils] Alternate static metadata PEP submission... In-Reply-To: <94bdd2610910130233o1ee3578fu6c50a584d1e5d53a@mail.gmail.com> References: <94bdd2610910111517u2fd0b136p44958f7faa46e974@mail.gmail.com> <94bdd2610910130136g51b559ey9e3a04123309342e@mail.gmail.com> <1d8404d17e3ebee2b734f928e5af9297@preisshare.net> <94bdd2610910130233o1ee3578fu6c50a584d1e5d53a@mail.gmail.com> Message-ID: <6b5118e1a8ea97f23335679195ab91aa@preisshare.net> On Tue, 13 Oct 2009 11:33:26 +0200, Tarek Ziad? wrote: > The first one who mentioned the idea was Eric IIRC, then Matthias and > Tres worked on it. Fine. They've been awfully quiet on distutils-sig lately :-) > But who cares ? I am not claiming any ownership on ideas. I am not > tracking either who threw which idea. But if someone cares, speak > up : I'll detail who participated in the Acknowledgment section. People care. Otherwise there wouldnt be discussions about it. > I just happen to be the one maintaining PEP 390 and I am trying to collect > ideas here because I am maintaining Distutils so I'll be the one > will apply the changes there. Sure. Well "collect" is a word that means different things to different people. I'd suggest some caution with using that word internationally because in some places it really can mean just outright theft. I'm very sure that that isn't how you want others to interpret your working process. Please.. don't use that word like that.. Generally though, if somebody maintains a collection, then they're usually very careful to leave the names of who they got contributions from. People are happy to give to collections. They're less happy finding things that they thought they owned in somebody elses collection. I'm sure you know what I mean here... >>> I suggest that you help us finishing in the existing PEP 390 instead, >>> by proposing changes. >> >> If I see things I can help with, then I'm happy to do that... > > Great then, I am looking forward for its improvements. My biggest concern with your pep is how it can be implemented on older systems. My proposal involves a boilerplate setup.py that can read the metadata and perform the installation. No update to distutils is required in my proposal. How are you planning to make yours work? David From ziade.tarek at gmail.com Wed Oct 14 11:21:40 2009 From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=) Date: Wed, 14 Oct 2009 11:21:40 +0200 Subject: [Distutils] Alternate static metadata PEP submission... In-Reply-To: <5fba01026f52bbf1d89cac4908ee17dc@preisshare.net> References: <20091013103021.GA21066@laurie.devork> <94bdd2610910131001v70711ffeia27433cd23b32fca@mail.gmail.com> <79a53e276e5dffe24be0433771801d72@preisshare.net> <94bdd2610910131426k64f707d7pf4d398d04b8ecc32@mail.gmail.com> <20091013233654.GA27339@laurie.devork> <5fba01026f52bbf1d89cac4908ee17dc@preisshare.net> Message-ID: <94bdd2610910140221o7d62839dl1d208c6c69959cc3@mail.gmail.com> 2009/10/14 David Lyon : > > However, you are correct about proof of concept code, and in some > respects that's the easy part. I will start work on that immediately. My suggestion is that once you have a revised PEP that includes some proof of concept code with less overlapping with PEP 390 (they are different proposals obviously), you post it here for a round of feedback, then I'll push it in the PEPs for further processing Tarek From david at ar.media.kyoto-u.ac.jp Wed Oct 14 13:47:32 2009 From: david at ar.media.kyoto-u.ac.jp (David Cournapeau) Date: Wed, 14 Oct 2009 20:47:32 +0900 Subject: [Distutils] Alternate static metadata PEP submission... In-Reply-To: <50a1116c3a8ca9bcbb28121d90ccc5d0@preisshare.net> References: <20091013103021.GA21066@laurie.devork> <94bdd2610910131001v70711ffeia27433cd23b32fca@mail.gmail.com> <79a53e276e5dffe24be0433771801d72@preisshare.net> <94bdd2610910131426k64f707d7pf4d398d04b8ecc32@mail.gmail.com> <20091013233654.GA27339@laurie.devork> <4AD54138.9010602@ar.media.kyoto-u.ac.jp> <50a1116c3a8ca9bcbb28121d90ccc5d0@preisshare.net> Message-ID: <4AD5BA54.4000600@ar.media.kyoto-u.ac.jp> David Lyon wrote: > > What's a few thousand lines of code between friends.. I think that's some quite tough thousand lines, though. You have to know a lot about the different platforms, and in my experience at least, designing a good tool and API around this kind of problems is hard. The good news is that there are existing tools in other environments which do this in a pretty good way already, so we could steal their idea and just reimplement the thing for python. This problem space has seen a lot of new ideas since distutils was implemented ten years ago (and we can learn from distutils mistakes). I still like the idea of reusing the same format as .cabal files (see here: -http://www.haskell.org/cabal/release/cabal-latest/doc/users-guide/authors.html). It does everything you want to do with .cfg files, with the advantage of being proven for real softwares and more complete. The use of ini-like files with conditional in sections really feels like an ugly and inflexible hack for a new system, I am not sure I understand why you want to use them for a brand-new system where backward compatibility with setup.cfg isn't a concern ? David From eric at trueblade.com Wed Oct 14 14:46:26 2009 From: eric at trueblade.com (Eric Smith) Date: Wed, 14 Oct 2009 08:46:26 -0400 Subject: [Distutils] Alternate static metadata PEP submission... In-Reply-To: <6b5118e1a8ea97f23335679195ab91aa@preisshare.net> References: <94bdd2610910111517u2fd0b136p44958f7faa46e974@mail.gmail.com> <94bdd2610910130136g51b559ey9e3a04123309342e@mail.gmail.com> <1d8404d17e3ebee2b734f928e5af9297@preisshare.net> <94bdd2610910130233o1ee3578fu6c50a584d1e5d53a@mail.gmail.com> <6b5118e1a8ea97f23335679195ab91aa@preisshare.net> Message-ID: <4AD5C822.6060000@trueblade.com> David Lyon wrote: > On Tue, 13 Oct 2009 11:33:26 +0200, Tarek Ziad? > wrote: >> The first one who mentioned the idea was Eric IIRC, then Matthias and >> Tres worked on it. > > Fine. They've been awfully quiet on distutils-sig lately :-) Not so! >> I just happen to be the one maintaining PEP 390 and I am trying to > collect >> ideas here because I am maintaining Distutils so I'll be the one >> will apply the changes there. > > Sure. Well "collect" is a word that means different things to different > people. I'd suggest some caution with using that word internationally > because in some places it really can mean just outright theft. I'm > very sure that that isn't how you want others to interpret your > working process. Please.. don't use that word like that.. I'm sure no offense was intended. I don't see how we can all know all possible words that might be construed as offensive. But back to the matter at hand: I don't see what David is proposing as being radical or even different from what we've been discussing: a single static file that contains enough metadata to describe what's in a distribution. This file needs to be extensible. The Distutils-SIG approach has been incremental, adding small parts. It seems he wants to go "big bang". I think that's fine, but it will be more difficult to migrate to, I fear. He's proposing shipping an application in each distribution (his new setup.py) that processes that file to do installations. Sounds like a bootstrapper of sorts, and not so radical of an idea. This setup.py would use parts of existing distutils to do its work, where it can. I think a name other than setup.py would reduce confusion, but the idea seems practical. I'd be interested to hear how this would interact with PyPI and buildout. Eric. From carl at dirtcircle.com Wed Oct 14 15:27:05 2009 From: carl at dirtcircle.com (Carl Meyer) Date: Wed, 14 Oct 2009 09:27:05 -0400 Subject: [Distutils] Summary: tracking requested vs dependency installs in PEP 376 metadata In-Reply-To: <4ACE15C5.4030105@meyerloewen.net> References: <4ACE15C5.4030105@meyerloewen.net> Message-ID: <4AD5D1A9.8080504@dirtcircle.com> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Hi all, Here's a summary of my take on the discussion thus far: - - There's been some discussion of terminology, based on which I'm proposing the file be called REQUESTED, and the Distribution boolean attribute be .requested. I've updated the demo implementation at http://bitbucket.org/carljm/pep376/ accordingly. - - Several people are in favor of tracking this metadata, and also interested in logging additional data about which installers dealt with an installed package, why, and when. The additional logging isn't inherent to this proposal and can be proposed separately if there's strong interest (same with the brief mention of a generic metadata API). - - At least one person is concerned that this level of metadata is insufficient for good uninstaller performance. AFAICS this concern is not possible to address directly without significant changes to the nature and scope of PEP 376 (i.e. a centralized "installed packages" database with dependency graph data, or a distributed simulation of one), but there are ways to mitigate the issue in actual (un)installer implementations. - - At least one person does not want orphan detection in their uninstaller tool. Since it's clear that other people do, and it would be simple to implement or modify an uninstaller such that it would treat all installed packages as REQUESTED and never mention orphan packages, I don't see this as a blocking concern. At this point I don't see any issues raised that would prevent moving ahead with the proposal. Is there anything I've missed? Thanks, Carl -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.6 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iD8DBQFK1dGp1j/fhc23WEARAnXvAKDX/HAKhXZFl5u0YBh6B5S3bNmX9gCeLcBM Gz5YA2Bhc4/9uxKKqtElL5k= =mrgu -----END PGP SIGNATURE----- From kiorky at cryptelium.net Wed Oct 14 16:19:51 2009 From: kiorky at cryptelium.net (kiorky) Date: Wed, 14 Oct 2009 16:19:51 +0200 Subject: [Distutils] the virtualenv-distribute mess In-Reply-To: <4ACF50A7.5010605@cryptelium.net> References: <4ACB2527.7010802@cryptelium.net> <319e029f0910060522l29499f46y2805ed0082d975c3@mail.gmail.com> <4ACB7D70.9060404@cryptelium.net> <4ACF1681.7050506@simplistix.co.uk> <4ACF269C.4010400@cryptelium.net> <94bdd2610910090508j6804628fy5b5d435cce0c7427@mail.gmail.com> <4ACF2FD6.4010504@cryptelium.net> <94bdd2610910090545p65a3b172l971fe01f11768a3f@mail.gmail.com> <4ACF3B87.1070702@cryptelium.net> <94bdd2610910090643t406390d0o59294906c4659e5d@mail.gmail.com> <4ACF50A7.5010605@cryptelium.net> Message-ID: <4AD5DE07.9000309@cryptelium.net> kiorky a ?crit : [] > This may be quite current even if it's not a good habit to have circular > dependencies between distributions. > Imagine that. > > B(0.7) -> A(0.6). > A(0.6) -> B(0.7). > > Can i have the same namespace "ns" shared between the twice distributions with > both the setuptools namespaces implementation (A) and the pkg_util's one (B)? > "Have" mean that i can import ns in both distributions. > > So, if: > * I have old distributions with C code even not declaring they are relying on > setuptools, installing with the 0.6 code automatically. > * I have entry points and namespaces from 0.7 available to import in 0.6 and > vice-versa. > > I will see no more objections. > > Another related thing, as i read the pep376 implementation, it may be good and > easy to provide some wrappers to some setuptools very used objects like > WorkingSet or Environment as similary code is already implemented to smoothly > migrate existing code. Must i interprete no response as a global No ? > ------------------------------------------------------------------------ > > _______________________________________________ > Distutils-SIG maillist - Distutils-SIG at python.org > http://mail.python.org/mailman/listinfo/distutils-sig -- -- Cordialement, KiOrKY GPG Key FingerPrint: 0x1A1194B7681112AF -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 261 bytes Desc: OpenPGP digital signature URL: From ziade.tarek at gmail.com Wed Oct 14 16:25:27 2009 From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=) Date: Wed, 14 Oct 2009 16:25:27 +0200 Subject: [Distutils] the virtualenv-distribute mess In-Reply-To: <4AD5DE07.9000309@cryptelium.net> References: <4ACF1681.7050506@simplistix.co.uk> <4ACF269C.4010400@cryptelium.net> <94bdd2610910090508j6804628fy5b5d435cce0c7427@mail.gmail.com> <4ACF2FD6.4010504@cryptelium.net> <94bdd2610910090545p65a3b172l971fe01f11768a3f@mail.gmail.com> <4ACF3B87.1070702@cryptelium.net> <94bdd2610910090643t406390d0o59294906c4659e5d@mail.gmail.com> <4ACF50A7.5010605@cryptelium.net> <4AD5DE07.9000309@cryptelium.net> Message-ID: <94bdd2610910140725x50b68699ib67fe709ed961e6@mail.gmail.com> On Wed, Oct 14, 2009 at 4:19 PM, kiorky wrote: > > > kiorky a ?crit : > > [] > >> This may be quite current even if it's not a good habit to have circular >> dependencies between distributions. >> Imagine that. >> >> B(0.7) -> A(0.6). >> A(0.6) -> B(0.7). >> >> Can i have the same namespace "ns" shared between the twice distributions with >> both the setuptools namespaces implementation (A) and the pkg_util's one (B)? >> "Have" mean that i can import ns in both distributions. Not sur to follow this. Is that two different working sets ? >> >> So, if: >> ? ? ? * I have old distributions with C code even not declaring they are relying on >> setuptools, installing with the 0.6 code automatically. >> ? ? ? * I have entry points and namespaces from 0.7 available to import in 0.6 and >> vice-versa. >> >> I will see no more objections. >> >> Another related thing, as i read the pep376 implementation, it may be good and >> easy to provide some wrappers to some setuptools very used objects like >> WorkingSet or Environment as similary code is already implemented to smoothly >> migrate existing code. In Distribute 0.7.x we will provide new tools, and we won't try to provide backward compat. Projects that whish to use Distribute 0.7 features will have to switch to it I guess, so, depending on the project, it can use various strategy to migrate its code. Tarek From regebro at gmail.com Wed Oct 14 16:28:25 2009 From: regebro at gmail.com (Lennart Regebro) Date: Wed, 14 Oct 2009 16:28:25 +0200 Subject: [Distutils] Future namespace support Message-ID: <319e029f0910140728k609cf813nae71d5d779935d41@mail.gmail.com> 2009/10/14 kiorky : >> This may be quite current even if it's not a good habit to have circular >> dependencies between distributions. >> Imagine that. >> >> B(0.7) -> A(0.6). >> A(0.6) -> B(0.7). I don't think that's a problem, although I can't see why you would need it in this case. >> Can i have the same namespace "ns" shared between the twice distributions with >> both the setuptools namespaces implementation (A) and the pkg_util's one (B)? I don't see why not, they would work the same way. Possibly the global _namespace_packages dictionary needs to be shared by both setuptools and the new module, but that can easily be handled by importing it from setuptools if setuptools is installed. -- Lennart Regebro: Python, Zope, Plone, Grok http://regebro.wordpress.com/ +33 661 58 14 64 From kiorky at cryptelium.net Wed Oct 14 16:41:35 2009 From: kiorky at cryptelium.net (kiorky) Date: Wed, 14 Oct 2009 16:41:35 +0200 Subject: [Distutils] the virtualenv-distribute mess In-Reply-To: <94bdd2610910140725x50b68699ib67fe709ed961e6@mail.gmail.com> References: <4ACF1681.7050506@simplistix.co.uk> <4ACF269C.4010400@cryptelium.net> <94bdd2610910090508j6804628fy5b5d435cce0c7427@mail.gmail.com> <4ACF2FD6.4010504@cryptelium.net> <94bdd2610910090545p65a3b172l971fe01f11768a3f@mail.gmail.com> <4ACF3B87.1070702@cryptelium.net> <94bdd2610910090643t406390d0o59294906c4659e5d@mail.gmail.com> <4ACF50A7.5010605@cryptelium.net> <4AD5DE07.9000309@cryptelium.net> <94bdd2610910140725x50b68699ib67fe709ed961e6@mail.gmail.com> Message-ID: <4AD5E31F.3000401@cryptelium.net> Tarek Ziad? a ?crit : > On Wed, Oct 14, 2009 at 4:19 PM, kiorky wrote: >> >> kiorky a ?crit : >> >> [] >> >>> This may be quite current even if it's not a good habit to have circular >>> dependencies between distributions. >>> Imagine that. >>> >>> B(0.7) -> A(0.6). >>> A(0.6) -> B(0.7). >>> >>> Can i have the same namespace "ns" shared between the twice distributions with >>> both the setuptools namespaces implementation (A) and the pkg_util's one (B)? >>> "Have" mean that i can import ns in both distributions. > > Not sur to follow this. Is that two different working sets ? Off course, no. Only two package i would have on sys.path as a time. > >>> So, if: >>> * I have old distributions with C code even not declaring they are relying on >>> setuptools, installing with the 0.6 code automatically. >>> * I have entry points and namespaces from 0.7 available to import in 0.6 and >>> vice-versa. >>> >>> I will see no more objections. >>> >>> Another related thing, as i read the pep376 implementation, it may be good and >>> easy to provide some wrappers to some setuptools very used objects like >>> WorkingSet or Environment as similary code is already implemented to smoothly >>> migrate existing code. > > In Distribute 0.7.x we will provide new tools, and we won't try to > provide backward compat. Sad to hear as emulation is possible, at least with 0.6. > Projects that whish to use Distribute 0.7 features will have to switch > to it I guess, Hard switching will make people hesitate to switch or not and IMO more not. Providing backward compatibility which runs on populars case may give additional time to distribute newcomers to appreciate the new APIs and migrate their stuff. > so, depending on the project, it can use various strategy to migrate its code. If the package maintainers can afford to think to those strategies. In the view of a new setuptools release (0.7), there will be three ways to have python packages installed, 2 or 3 ways (0.6, distribute, 0.7?) to have namespaces and/or entry points and incompatibilities at each level. Even if each package family live on its own without busing the others flavors, integration of all those packages will be a PITA. Add on this the burden to know each deployment thing. > > Tarek -- -- Cordialement, KiOrKY GPG Key FingerPrint: 0x1A1194B7681112AF -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 261 bytes Desc: OpenPGP digital signature URL: From kiorky at cryptelium.net Wed Oct 14 16:45:40 2009 From: kiorky at cryptelium.net (kiorky) Date: Wed, 14 Oct 2009 16:45:40 +0200 Subject: [Distutils] Future namespace support In-Reply-To: <319e029f0910140728k609cf813nae71d5d779935d41@mail.gmail.com> References: <319e029f0910140728k609cf813nae71d5d779935d41@mail.gmail.com> Message-ID: <4AD5E414.8000906@cryptelium.net> Lennart Regebro a ?crit : > 2009/10/14 kiorky : >>> This may be quite current even if it's not a good habit to have circular >>> dependencies between distributions. >>> Imagine that. >>> >>> B(0.7) -> A(0.6). >>> A(0.6) -> B(0.7). > > I don't think that's a problem, although I can't see why you would > need it in this case. > >>> Can i have the same namespace "ns" shared between the twice distributions with >>> both the setuptools namespaces implementation (A) and the pkg_util's one (B)? > > I don't see why not, they would work the same way. Possibly the global That's not what said the PEP (IOW what i had understood of) but yes the "namespace" stuff must be shared among all namespaces implementations. I think it is a primal requirement for integration. And it must also be compatible with previous implementations more than every other points like entry points or installation/environment AP?s. > _namespace_packages dictionary needs to be shared by both setuptools > and the new module, but that can easily be handled by importing it > from setuptools if setuptools is installed. > -- -- Cordialement, KiOrKY GPG Key FingerPrint: 0x1A1194B7681112AF -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 261 bytes Desc: OpenPGP digital signature URL: From regebro at gmail.com Wed Oct 14 16:55:38 2009 From: regebro at gmail.com (Lennart Regebro) Date: Wed, 14 Oct 2009 16:55:38 +0200 Subject: [Distutils] Future namespace support In-Reply-To: <4AD5E414.8000906@cryptelium.net> References: <319e029f0910140728k609cf813nae71d5d779935d41@mail.gmail.com> <4AD5E414.8000906@cryptelium.net> Message-ID: <319e029f0910140755t1ec8903aufd392cca80dc35e7@mail.gmail.com> 2009/10/14 kiorky : > That's not what said the PEP (IOW what i had understood of) Oh, you don't *use* it the same way, no, but I assumed that the internal mechanisms would be similar. Maybe I was wrong. -- Lennart Regebro: Python, Zope, Plone, Grok http://regebro.wordpress.com/ +33 661 58 14 64 From kiorky at cryptelium.net Wed Oct 14 17:01:45 2009 From: kiorky at cryptelium.net (kiorky) Date: Wed, 14 Oct 2009 17:01:45 +0200 Subject: [Distutils] Future namespace support In-Reply-To: <319e029f0910140755t1ec8903aufd392cca80dc35e7@mail.gmail.com> References: <319e029f0910140728k609cf813nae71d5d779935d41@mail.gmail.com> <4AD5E414.8000906@cryptelium.net> <319e029f0910140755t1ec8903aufd392cca80dc35e7@mail.gmail.com> Message-ID: <4AD5E7D9.8000402@cryptelium.net> I want when i import ns.foo that it works in every module i have on sys.path no matter which namespace implementation both packages use. Lennart Regebro a ?crit : > 2009/10/14 kiorky : >> That's not what said the PEP (IOW what i had understood of) > > Oh, you don't *use* it the same way, no, but I assumed that the > internal mechanisms would be similar. Maybe I was wrong. > -- Cordialement, KiOrKY GPG Key FingerPrint: 0x1A1194B7681112AF -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 261 bytes Desc: OpenPGP digital signature URL: From regebro at gmail.com Wed Oct 14 17:02:22 2009 From: regebro at gmail.com (Lennart Regebro) Date: Wed, 14 Oct 2009 17:02:22 +0200 Subject: [Distutils] Future namespace support In-Reply-To: <4AD5E7D9.8000402@cryptelium.net> References: <319e029f0910140728k609cf813nae71d5d779935d41@mail.gmail.com> <4AD5E414.8000906@cryptelium.net> <319e029f0910140755t1ec8903aufd392cca80dc35e7@mail.gmail.com> <4AD5E7D9.8000402@cryptelium.net> Message-ID: <319e029f0910140802x7c277e28ma7e5fa2d9028658e@mail.gmail.com> 2009/10/14 kiorky : > I want when i import ns.foo that it works in every module i have on sys.path no > matter which namespace implementation both packages use. Yes, otherwise namespace packages it would rather pointless. :) -- Lennart Regebro: Python, Zope, Plone, Grok http://regebro.wordpress.com/ +33 661 58 14 64 From kiorky at cryptelium.net Wed Oct 14 17:34:14 2009 From: kiorky at cryptelium.net (kiorky) Date: Wed, 14 Oct 2009 17:34:14 +0200 Subject: [Distutils] Future namespace support In-Reply-To: <319e029f0910140802x7c277e28ma7e5fa2d9028658e@mail.gmail.com> References: <319e029f0910140728k609cf813nae71d5d779935d41@mail.gmail.com> <4AD5E414.8000906@cryptelium.net> <319e029f0910140755t1ec8903aufd392cca80dc35e7@mail.gmail.com> <4AD5E7D9.8000402@cryptelium.net> <319e029f0910140802x7c277e28ma7e5fa2d9028658e@mail.gmail.com> Message-ID: <4AD5EF76.4080304@cryptelium.net> Lennart Regebro a ?crit : > 2009/10/14 kiorky : >> I want when i import ns.foo that it works in every module i have on sys.path no >> matter which namespace implementation both packages use. > > Yes, otherwise namespace packages it would rather pointless. :) You say yes but it is not what we ll have at least for the moment. > -- Cordialement, KiOrKY GPG Key FingerPrint: 0x1A1194B7681112AF -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 261 bytes Desc: OpenPGP digital signature URL: From regebro at gmail.com Wed Oct 14 17:44:40 2009 From: regebro at gmail.com (Lennart Regebro) Date: Wed, 14 Oct 2009 17:44:40 +0200 Subject: [Distutils] Future namespace support In-Reply-To: <4AD5EF76.4080304@cryptelium.net> References: <319e029f0910140728k609cf813nae71d5d779935d41@mail.gmail.com> <4AD5E414.8000906@cryptelium.net> <319e029f0910140755t1ec8903aufd392cca80dc35e7@mail.gmail.com> <4AD5E7D9.8000402@cryptelium.net> <319e029f0910140802x7c277e28ma7e5fa2d9028658e@mail.gmail.com> <4AD5EF76.4080304@cryptelium.net> Message-ID: <319e029f0910140844q1d0554day94276f84d0805cce@mail.gmail.com> 2009/10/14 kiorky : > Lennart Regebro a ?crit : >> 2009/10/14 kiorky : >>> I want when i import ns.foo that it works in every module i have on sys.path no >>> matter which namespace implementation both packages use. >> >> Yes, otherwise namespace packages it would rather pointless. :) > > You say yes but it is not what we ll have at least for the moment. Why not? -- Lennart Regebro: Python, Zope, Plone, Grok http://regebro.wordpress.com/ +33 661 58 14 64 From kiorky at cryptelium.net Wed Oct 14 18:11:39 2009 From: kiorky at cryptelium.net (kiorky) Date: Wed, 14 Oct 2009 18:11:39 +0200 Subject: [Distutils] Future namespace support In-Reply-To: <319e029f0910140844q1d0554day94276f84d0805cce@mail.gmail.com> References: <319e029f0910140728k609cf813nae71d5d779935d41@mail.gmail.com> <4AD5E414.8000906@cryptelium.net> <319e029f0910140755t1ec8903aufd392cca80dc35e7@mail.gmail.com> <4AD5E7D9.8000402@cryptelium.net> <319e029f0910140802x7c277e28ma7e5fa2d9028658e@mail.gmail.com> <4AD5EF76.4080304@cryptelium.net> <319e029f0910140844q1d0554day94276f84d0805cce@mail.gmail.com> Message-ID: <4AD5F83B.8010409@cryptelium.net> from [1] """ Likewise, setuptools supports inspecting zip files, and supports adding portions to its _namespace_packages variable, whereas pkgutil doesn't """ [1] - http://www.python.org/dev/peps/pep-0382/ Lennart Regebro a ?crit : > 2009/10/14 kiorky : >> Lennart Regebro a ?crit : >>> 2009/10/14 kiorky : >>>> I want when i import ns.foo that it works in every module i have on sys.path no >>>> matter which namespace implementation both packages use. >>> Yes, otherwise namespace packages it would rather pointless. :) >> You say yes but it is not what we ll have at least for the moment. > > Why not? > -- Cordialement, KiOrKY GPG Key FingerPrint: 0x1A1194B7681112AF -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 261 bytes Desc: OpenPGP digital signature URL: From ziade.tarek at gmail.com Wed Oct 14 18:14:54 2009 From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=) Date: Wed, 14 Oct 2009 18:14:54 +0200 Subject: [Distutils] Alternate static metadata PEP submission... In-Reply-To: <4AD5C822.6060000@trueblade.com> References: <94bdd2610910111517u2fd0b136p44958f7faa46e974@mail.gmail.com> <94bdd2610910130136g51b559ey9e3a04123309342e@mail.gmail.com> <1d8404d17e3ebee2b734f928e5af9297@preisshare.net> <94bdd2610910130233o1ee3578fu6c50a584d1e5d53a@mail.gmail.com> <6b5118e1a8ea97f23335679195ab91aa@preisshare.net> <4AD5C822.6060000@trueblade.com> Message-ID: <94bdd2610910140914k56bca7ddj8d0696d575a1b9a4@mail.gmail.com> On Wed, Oct 14, 2009 at 2:46 PM, Eric Smith wrote: >>> >>> ideas here because I am maintaining Distutils so I'll be the one ?will >>> apply the changes there. >> >> Sure. Well "collect" is a word that means different things to different >> people. I'd suggest some caution with using that word internationally >> because in some places it really can mean just outright theft. I'm >> very sure that that isn't how you want others to interpret your >> working process. Please.. don't use that word like that.. David, Everything is fine in what I have said and the words I have picked, you are misinterpreting. Please be sure to read and agree on everything that is written here before you submit a revised PEP I'll push in the svn: http://www.python.org/dev/peps/pep-0001 In particular, the fact that PEP will belong to the public domain once in our svn. Tarek From regebro at gmail.com Wed Oct 14 18:17:10 2009 From: regebro at gmail.com (Lennart Regebro) Date: Wed, 14 Oct 2009 18:17:10 +0200 Subject: [Distutils] Future namespace support In-Reply-To: <4AD5F83B.8010409@cryptelium.net> References: <319e029f0910140728k609cf813nae71d5d779935d41@mail.gmail.com> <4AD5E414.8000906@cryptelium.net> <319e029f0910140755t1ec8903aufd392cca80dc35e7@mail.gmail.com> <4AD5E7D9.8000402@cryptelium.net> <319e029f0910140802x7c277e28ma7e5fa2d9028658e@mail.gmail.com> <4AD5EF76.4080304@cryptelium.net> <319e029f0910140844q1d0554day94276f84d0805cce@mail.gmail.com> <4AD5F83B.8010409@cryptelium.net> Message-ID: <319e029f0910140917hc0dff08x53b292476482808@mail.gmail.com> 2009/10/14 kiorky : > from [1] > """ > Likewise, setuptools supports inspecting zip files, and supports adding portions > to its _namespace_packages variable, whereas pkgutil doesn't > """ Yes? -- Lennart Regebro: Python, Zope, Plone, Grok http://regebro.wordpress.com/ +33 661 58 14 64 From kiorky at cryptelium.net Wed Oct 14 18:28:20 2009 From: kiorky at cryptelium.net (kiorky) Date: Wed, 14 Oct 2009 18:28:20 +0200 Subject: [Distutils] Future namespace support In-Reply-To: <319e029f0910140917hc0dff08x53b292476482808@mail.gmail.com> References: <319e029f0910140728k609cf813nae71d5d779935d41@mail.gmail.com> <4AD5E414.8000906@cryptelium.net> <319e029f0910140755t1ec8903aufd392cca80dc35e7@mail.gmail.com> <4AD5E7D9.8000402@cryptelium.net> <319e029f0910140802x7c277e28ma7e5fa2d9028658e@mail.gmail.com> <4AD5EF76.4080304@cryptelium.net> <319e029f0910140844q1d0554day94276f84d0805cce@mail.gmail.com> <4AD5F83B.8010409@cryptelium.net> <319e029f0910140917hc0dff08x53b292476482808@mail.gmail.com> Message-ID: <4AD5FC24.30500@cryptelium.net> Lennart Regebro a ?crit : > 2009/10/14 kiorky : >> from [1] >> """ >> Likewise, setuptools supports inspecting zip files, and supports adding portions >> to its _namespace_packages variable, whereas pkgutil doesn't >> """ > > Yes? > There is one obvious incompatibility maybe there will be others with the future implementations. -- Cordialement, KiOrKY GPG Key FingerPrint: 0x1A1194B7681112AF -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 261 bytes Desc: OpenPGP digital signature URL: From regebro at gmail.com Wed Oct 14 18:31:14 2009 From: regebro at gmail.com (Lennart Regebro) Date: Wed, 14 Oct 2009 18:31:14 +0200 Subject: [Distutils] Future namespace support In-Reply-To: <4AD5FC24.30500@cryptelium.net> References: <319e029f0910140728k609cf813nae71d5d779935d41@mail.gmail.com> <4AD5E414.8000906@cryptelium.net> <319e029f0910140755t1ec8903aufd392cca80dc35e7@mail.gmail.com> <4AD5E7D9.8000402@cryptelium.net> <319e029f0910140802x7c277e28ma7e5fa2d9028658e@mail.gmail.com> <4AD5EF76.4080304@cryptelium.net> <319e029f0910140844q1d0554day94276f84d0805cce@mail.gmail.com> <4AD5F83B.8010409@cryptelium.net> <319e029f0910140917hc0dff08x53b292476482808@mail.gmail.com> <4AD5FC24.30500@cryptelium.net> Message-ID: <319e029f0910140931w443d927na77eb98a68b8f1f4@mail.gmail.com> 2009/10/14 kiorky : > There is one obvious incompatibility maybe there will be others with the future > implementations. It's again an incompatibility in how your package makes something be a namespace package. Not an incompatibility in how it is used from other packages. Obviously you would need setuptools installed for this to work, but it would work even from packages who do not require or use setuptools just as today. -- Lennart Regebro: Python, Zope, Plone, Grok http://regebro.wordpress.com/ +33 661 58 14 64 From kiorky at cryptelium.net Wed Oct 14 18:46:57 2009 From: kiorky at cryptelium.net (kiorky) Date: Wed, 14 Oct 2009 18:46:57 +0200 Subject: [Distutils] Future namespace support In-Reply-To: <319e029f0910140931w443d927na77eb98a68b8f1f4@mail.gmail.com> References: <319e029f0910140728k609cf813nae71d5d779935d41@mail.gmail.com> <4AD5E414.8000906@cryptelium.net> <319e029f0910140755t1ec8903aufd392cca80dc35e7@mail.gmail.com> <4AD5E7D9.8000402@cryptelium.net> <319e029f0910140802x7c277e28ma7e5fa2d9028658e@mail.gmail.com> <4AD5EF76.4080304@cryptelium.net> <319e029f0910140844q1d0554day94276f84d0805cce@mail.gmail.com> <4AD5F83B.8010409@cryptelium.net> <319e029f0910140917hc0dff08x53b292476482808@mail.gmail.com> <4AD5FC24.30500@cryptelium.net> <319e029f0910140931w443d927na77eb98a68b8f1f4@mail.gmail.com> Message-ID: <4AD60081.6050000@cryptelium.net> Lennart Regebro a ?crit : > 2009/10/14 kiorky : >> There is one obvious incompatibility maybe there will be others with the future >> implementations. > > It's again an incompatibility in how your package makes something be a > namespace package. Not an incompatibility in how it is used from other As i understood, NS.foo from setuptools can't be imported in NS.bar from pkgutil and vice versa. > packages. Obviously you would need setuptools installed for this to > work, but it would work even from packages who do not require or use > setuptools just as today. I just hope my old packages will work as well as they worked. You can't ask everyone to repackage/adapt their existing stuff because of your whims. Nowodays, lot of packages use setuptools's namespaces, i want them to just work. At the moment, the only thing i see is that distribute break many things in many places without any sense of backward compatibility for 0.7. And the only answer we have is "go for the 0.6 branch/sandbox if you want compatibility, it will be maintained for years". That's not an answer, that's not viable. As a Python user, web developer, scientist, or whatever, you'll need stuff from both systems, you do not control the whole packages out there, and you wont surely have the time and the means to check that everything out there fit together. -- Cordialement, KiOrKY GPG Key FingerPrint: 0x1A1194B7681112AF -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 261 bytes Desc: OpenPGP digital signature URL: From regebro at gmail.com Wed Oct 14 19:31:27 2009 From: regebro at gmail.com (Lennart Regebro) Date: Wed, 14 Oct 2009 19:31:27 +0200 Subject: [Distutils] Future namespace support In-Reply-To: <4AD60081.6050000@cryptelium.net> References: <319e029f0910140728k609cf813nae71d5d779935d41@mail.gmail.com> <4AD5E7D9.8000402@cryptelium.net> <319e029f0910140802x7c277e28ma7e5fa2d9028658e@mail.gmail.com> <4AD5EF76.4080304@cryptelium.net> <319e029f0910140844q1d0554day94276f84d0805cce@mail.gmail.com> <4AD5F83B.8010409@cryptelium.net> <319e029f0910140917hc0dff08x53b292476482808@mail.gmail.com> <4AD5FC24.30500@cryptelium.net> <319e029f0910140931w443d927na77eb98a68b8f1f4@mail.gmail.com> <4AD60081.6050000@cryptelium.net> Message-ID: <319e029f0910141031n2ebef57eodf16b48ea8e239b4@mail.gmail.com> 2009/10/14 kiorky : > As i understood, NS.foo from setuptools can't be imported in NS.bar from pkgutil > and vice versa. Oh, in the same namespace? Yeah, that could break. Is that a problem? Also, pkgutil doesn't know about setuptools, so possibly that breaks, but its' definitely possible to import a pkgutil namespace from a setuptools namespace (at least different namespaces). The new function obviously knows about both pkgutil and setuptools and could therefore make sure it's compatible. If it wasn't, it would be completely useless. > You can't ask everyone to repackage/adapt their existing stuff because of your > whims. Obviously. > Nowodays, lot of packages use setuptools's namespaces, i want them to just work. I see no reason that they should not continue to work. > At the moment, the only thing i see is that distribute break many things in many > places without any sense of backward compatibility for 0.7. Don't worry about vapourware. > That's not an answer, that's not viable. Yes it is. Again: This is backwards compatibility for the package that sets up the namespace we are talking about here. Not other packages that doesn't know anything about Distutils. Nobody is proposing that we break all namespace packages in existance. Neither Tarek nor Martin is that stupid. Have some trust in people. :-) -- Lennart Regebro: Python, Zope, Plone, Grok http://regebro.wordpress.com/ +33 661 58 14 64 From kiorky at cryptelium.net Wed Oct 14 21:34:32 2009 From: kiorky at cryptelium.net (kiorky) Date: Wed, 14 Oct 2009 21:34:32 +0200 Subject: [Distutils] Future namespace support In-Reply-To: <319e029f0910141031n2ebef57eodf16b48ea8e239b4@mail.gmail.com> References: <319e029f0910140728k609cf813nae71d5d779935d41@mail.gmail.com> <4AD5E7D9.8000402@cryptelium.net> <319e029f0910140802x7c277e28ma7e5fa2d9028658e@mail.gmail.com> <4AD5EF76.4080304@cryptelium.net> <319e029f0910140844q1d0554day94276f84d0805cce@mail.gmail.com> <4AD5F83B.8010409@cryptelium.net> <319e029f0910140917hc0dff08x53b292476482808@mail.gmail.com> <4AD5FC24.30500@cryptelium.net> <319e029f0910140931w443d927na77eb98a68b8f1f4@mail.gmail.com> <4AD60081.6050000@cryptelium.net> <319e029f0910141031n2ebef57eodf16b48ea8e239b4@mail.gmail.com> Message-ID: <4AD627C8.6080507@cryptelium.net> Lennart Regebro a ?crit : > 2009/10/14 kiorky : >> As i understood, NS.foo from setuptools can't be imported in NS.bar from pkgutil >> and vice versa. > > Oh, in the same namespace? Yeah, that could break. Is that a problem? > Also, pkgutil doesn't know about setuptools, so possibly that breaks, > but its' definitely possible to import a pkgutil namespace from a > setuptools namespace (at least different namespaces). The new function > obviously knows about both pkgutil and setuptools and could therefore > make sure it's compatible. In the context of migration, for example. Take the plone "collective" namespace, all the modules won't be updated at the same time, we will have a painful cohabitation time. And yes, i have nothing toward any new implementation, i just want compatibility on basic things. > > If it wasn't, it would be completely useless. > >> You can't ask everyone to repackage/adapt their existing stuff because of your >> whims. > > Obviously. > >> Nowodays, lot of packages use setuptools's namespaces, i want them to just work. > > I see no reason that they should not continue to work. Choose to go on that new stuff that is not backward compatible. > >> At the moment, the only thing i see is that distribute break many things in many >> places without any sense of backward compatibility for 0.7. > > Don't worry about vapourware. I ve heard that before. Hay, Tarek, go out that body ! Yep, i know, but these are point that we must keep in mind before the new implementation prevent us from providing bits of backward compatibility. > >> That's not an answer, that's not viable. > > Yes it is. Again: This is backwards compatibility for the package that > sets up the namespace we are talking about here. Not other packages > that doesn't know anything about Distutils. Nobody is proposing that > we break all namespace packages in existance. Neither Tarek nor Martin > is that stupid. Have some trust in people. :-) I don't trust anyone, even not me. But, i am sure we are a smart people who will provide great stuff in near future. Nevertheless, i can only submit ideas as contributions for the moment as i have only limited extra-time :( -- Cordialement, KiOrKY GPG Key FingerPrint: 0x1A1194B7681112AF -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 261 bytes Desc: OpenPGP digital signature URL: From pje at telecommunity.com Wed Oct 14 22:03:17 2009 From: pje at telecommunity.com (P.J. Eby) Date: Wed, 14 Oct 2009 16:03:17 -0400 Subject: [Distutils] Future namespace support In-Reply-To: <319e029f0910140755t1ec8903aufd392cca80dc35e7@mail.gmail.co m> References: <319e029f0910140728k609cf813nae71d5d779935d41@mail.gmail.com> <4AD5E414.8000906@cryptelium.net> <319e029f0910140755t1ec8903aufd392cca80dc35e7@mail.gmail.com> Message-ID: <20091014200328.4D2C83A407A@sparrow.telecommunity.com> At 04:55 PM 10/14/2009 +0200, Lennart Regebro wrote: >2009/10/14 kiorky : > > That's not what said the PEP (IOW what i had understood of) > >Oh, you don't *use* it the same way, no, but I assumed that the >internal mechanisms would be similar. Maybe I was wrong. You can use as many namespace package mechanisms as you want, simultaneously, as long as they're not all being used for the same namespace package. Bear in mind that the true mechanism of nsp's is simply a module's __path__ object; pkgutil and pkg_resources simply provide utilities for setting the __path__ appropriately. With PEP 382, the utilities become unnecessary, except for backward compatibility, and if you want to do fixup of a __path__ when sys.path is changed. However, as long as each nsp is only registered with one such utility, there should be no conflicts. From pje at telecommunity.com Wed Oct 14 22:04:39 2009 From: pje at telecommunity.com (P.J. Eby) Date: Wed, 14 Oct 2009 16:04:39 -0400 Subject: [Distutils] Future namespace support In-Reply-To: <4AD60081.6050000@cryptelium.net> References: <319e029f0910140728k609cf813nae71d5d779935d41@mail.gmail.com> <4AD5E414.8000906@cryptelium.net> <319e029f0910140755t1ec8903aufd392cca80dc35e7@mail.gmail.com> <4AD5E7D9.8000402@cryptelium.net> <319e029f0910140802x7c277e28ma7e5fa2d9028658e@mail.gmail.com> <4AD5EF76.4080304@cryptelium.net> <319e029f0910140844q1d0554day94276f84d0805cce@mail.gmail.com> <4AD5F83B.8010409@cryptelium.net> <319e029f0910140917hc0dff08x53b292476482808@mail.gmail.com> <4AD5FC24.30500@cryptelium.net> <319e029f0910140931w443d927na77eb98a68b8f1f4@mail.gmail.com> <4AD60081.6050000@cryptelium.net> Message-ID: <20091014200450.3DE723A407A@sparrow.telecommunity.com> At 06:46 PM 10/14/2009 +0200, kiorky wrote: >As i understood, NS.foo from setuptools can't be imported in NS.bar >from pkgutil >and vice versa. Not so. The true implementation of nsp's is in __path__, which is provided by Python's import libraries, not setuptools or pkgutil. From kiorky at cryptelium.net Wed Oct 14 22:40:37 2009 From: kiorky at cryptelium.net (kiorky) Date: Wed, 14 Oct 2009 22:40:37 +0200 Subject: [Distutils] Future namespace support In-Reply-To: <20091014200450.3DE723A407A@sparrow.telecommunity.com> References: <319e029f0910140728k609cf813nae71d5d779935d41@mail.gmail.com> <4AD5E414.8000906@cryptelium.net> <319e029f0910140755t1ec8903aufd392cca80dc35e7@mail.gmail.com> <4AD5E7D9.8000402@cryptelium.net> <319e029f0910140802x7c277e28ma7e5fa2d9028658e@mail.gmail.com> <4AD5EF76.4080304@cryptelium.net> <319e029f0910140844q1d0554day94276f84d0805cce@mail.gmail.com> <4AD5F83B.8010409@cryptelium.net> <319e029f0910140917hc0dff08x53b292476482808@mail.gmail.com> <4AD5FC24.30500@cryptelium.net> <319e029f0910140931w443d927na77eb98a68b8f1f4@mail.gmail.com> <4AD60081.6050000@cryptelium.net> <20091014200450.3DE723A407A@sparrow.telecommunity.com> Message-ID: <4AD63745.8000504@cryptelium.net> P.J. Eby a ?crit : > At 06:46 PM 10/14/2009 +0200, kiorky wrote: >> As i understood, NS.foo from setuptools can't be imported in NS.bar >> from pkgutil >> and vice versa. > > Not so. The true implementation of nsp's is in __path__, which is > provided by Python's import libraries, not setuptools or pkgutil. Ok, fine. But, it's the two machineries make that __path__ feeded, isn't it? When mixing packages that use pkgutil and setuptools, in a package which is just using one of the twice mecanisms, i'll not have the __path__ with the 2 packages inside ? -- Cordialement, KiOrKY GPG Key FingerPrint: 0x1A1194B7681112AF -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 261 bytes Desc: OpenPGP digital signature URL: From kiorky at cryptelium.net Wed Oct 14 22:43:41 2009 From: kiorky at cryptelium.net (kiorky) Date: Wed, 14 Oct 2009 22:43:41 +0200 Subject: [Distutils] Future namespace support In-Reply-To: <20091014200450.3DE723A407A@sparrow.telecommunity.com> References: <319e029f0910140728k609cf813nae71d5d779935d41@mail.gmail.com> <4AD5E414.8000906@cryptelium.net> <319e029f0910140755t1ec8903aufd392cca80dc35e7@mail.gmail.com> <4AD5E7D9.8000402@cryptelium.net> <319e029f0910140802x7c277e28ma7e5fa2d9028658e@mail.gmail.com> <4AD5EF76.4080304@cryptelium.net> <319e029f0910140844q1d0554day94276f84d0805cce@mail.gmail.com> <4AD5F83B.8010409@cryptelium.net> <319e029f0910140917hc0dff08x53b292476482808@mail.gmail.com> <4AD5FC24.30500@cryptelium.net> <319e029f0910140931w443d927na77eb98a68b8f1f4@mail.gmail.com> <4AD60081.6050000@cryptelium.net> <20091014200450.3DE723A407A@sparrow.telecommunity.com> Message-ID: <4AD637FD.9040508@cryptelium.net> P.J. Eby a ?crit : > At 06:46 PM 10/14/2009 +0200, kiorky wrote: >> As i understood, NS.foo from setuptools can't be imported in NS.bar >> from pkgutil >> and vice versa. > > Not so. The true implementation of nsp's is in __path__, which is > provided by Python's import libraries, not setuptools or pkgutil. Ok, fine. But, it's the two machineries make that __path__ feeded, isn't it? When mixing packages that use pkgutil and setuptools, in a package which is just using one of the twice mecanisms, i'll not have the __path__ with the 2 packages inside ? -- Cordialement, KiOrKY GPG Key FingerPrint: 0x1A1194B7681112AF -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 261 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 261 bytes Desc: OpenPGP digital signature URL: From regebro at gmail.com Wed Oct 14 23:46:28 2009 From: regebro at gmail.com (Lennart Regebro) Date: Wed, 14 Oct 2009 23:46:28 +0200 Subject: [Distutils] Future namespace support In-Reply-To: <20091014200328.4D2C83A407A@sparrow.telecommunity.com> References: <319e029f0910140728k609cf813nae71d5d779935d41@mail.gmail.com> <4AD5E414.8000906@cryptelium.net> <319e029f0910140755t1ec8903aufd392cca80dc35e7@mail.gmail.com> <20091014200328.4D2C83A407A@sparrow.telecommunity.com> Message-ID: <319e029f0910141446p5e57482dw94e27e653b4d0be@mail.gmail.com> 2009/10/14 P.J. Eby : > You can use as many namespace package mechanisms as you want, > simultaneously, as long as they're not all being used for the same namespace > package. But the same namespace works. I couldn't believe kiorky's claim that it wouldn't work, and I have now tested, and it works just fine. I had one packages nstest.sub, using setuptools and one nstest.sob use pkutils, and in each of them I had a method that would import the other module. It works just fine. -- Lennart Regebro: Python, Zope, Plone, Grok http://regebro.wordpress.com/ +33 661 58 14 64 From david.lyon at preisshare.net Wed Oct 14 23:59:50 2009 From: david.lyon at preisshare.net (David Lyon) Date: Wed, 14 Oct 2009 17:59:50 -0400 Subject: [Distutils] Alternate static metadata PEP submission... In-Reply-To: <4AD5BA54.4000600@ar.media.kyoto-u.ac.jp> References: <20091013103021.GA21066@laurie.devork> <94bdd2610910131001v70711ffeia27433cd23b32fca@mail.gmail.com> <79a53e276e5dffe24be0433771801d72@preisshare.net> <94bdd2610910131426k64f707d7pf4d398d04b8ecc32@mail.gmail.com> <20091013233654.GA27339@laurie.devork> <4AD54138.9010602@ar.media.kyoto-u.ac.jp> <50a1116c3a8ca9bcbb28121d90ccc5d0@preisshare.net> <4AD5BA54.4000600@ar.media.kyoto-u.ac.jp> Message-ID: <4520a61448e00a641236151aee632aa8@preisshare.net> On Wed, 14 Oct 2009 20:47:32 +0900, David Cournapeau > The good news is that there are existing tools in other environments > which do this in a pretty good way already, so we could steal their idea > and just reimplement the thing for python. This problem space has seen a > lot of new ideas since distutils was implemented ten years ago (and we > can learn from distutils mistakes). Distutils still implements some of it's file copy operations well. We're not talking about too much more than a file copy and run script api. > I still like the idea of reusing the same format as .cabal files (see > here: > -http://www.haskell.org/cabal/release/cabal-latest/doc/users-guide/authors.html). > It does everything you want to do with .cfg files, with the advantage of > being proven for real softwares and more complete. The use of ini-like > files with conditional in sections really feels like an ugly and > inflexible hack for a new system, I am not sure I understand why you > want to use them for a brand-new system where backward compatibility > with setup.cfg isn't a concern ? I'm really not hung up on .ini style files and am agnostic about the format. It's only from experience I know that they can hold any nearly type of textual data, and work equally on linux and windows. ConfigParser is in every python version that I know of. Tell me where I find a built in API in python for this other format so we can look at it? No harm in looking at it. David From tseaver at palladion.com Thu Oct 15 00:10:54 2009 From: tseaver at palladion.com (Tres Seaver) Date: Wed, 14 Oct 2009 18:10:54 -0400 Subject: [Distutils] Future namespace support In-Reply-To: <319e029f0910141446p5e57482dw94e27e653b4d0be@mail.gmail.com> References: <319e029f0910140728k609cf813nae71d5d779935d41@mail.gmail.com> <4AD5E414.8000906@cryptelium.net> <319e029f0910140755t1ec8903aufd392cca80dc35e7@mail.gmail.com> <20091014200328.4D2C83A407A@sparrow.telecommunity.com> <319e029f0910141446p5e57482dw94e27e653b4d0be@mail.gmail.com> Message-ID: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Lennart Regebro wrote: > 2009/10/14 P.J. Eby : >> You can use as many namespace package mechanisms as you want, >> simultaneously, as long as they're not all being used for the same namespace >> package. > > But the same namespace works. I couldn't believe kiorky's claim that > it wouldn't work, and I have now tested, and it works just fine. I had > one packages nstest.sub, using setuptools and one nstest.sob use > pkutils, and in each of them I had a method that would import the > other module. It works just fine. Right, its just a race: whichever project gets its version of the namespace package imported "first" wins, sets the __path__, and everything works. No consolation prizes for the others, because they *never get imported* (their __init__.py is never even considered). Tres. - -- =================================================================== Tres Seaver +1 540-429-0999 tseaver at palladion.com Palladion Software "Excellence by Design" http://palladion.com -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iEYEARECAAYFAkrWTGkACgkQ+gerLs4ltQ5YTwCePlqxXOdhviexEeLZU8HSVQ1x lbIAnRFq4DhbNsTLUdcIY2uIlcX0UliM =wkCg -----END PGP SIGNATURE----- From fdrake at gmail.com Thu Oct 15 00:39:00 2009 From: fdrake at gmail.com (Fred Drake) Date: Wed, 14 Oct 2009 18:39:00 -0400 Subject: [Distutils] Alternate static metadata PEP submission... In-Reply-To: <4520a61448e00a641236151aee632aa8@preisshare.net> References: <94bdd2610910131001v70711ffeia27433cd23b32fca@mail.gmail.com> <79a53e276e5dffe24be0433771801d72@preisshare.net> <94bdd2610910131426k64f707d7pf4d398d04b8ecc32@mail.gmail.com> <20091013233654.GA27339@laurie.devork> <4AD54138.9010602@ar.media.kyoto-u.ac.jp> <50a1116c3a8ca9bcbb28121d90ccc5d0@preisshare.net> <4AD5BA54.4000600@ar.media.kyoto-u.ac.jp> <4520a61448e00a641236151aee632aa8@preisshare.net> Message-ID: <9cee7ab80910141539u5994e688idf291e0bf69afdcc@mail.gmail.com> On Wed, Oct 14, 2009 at 5:59 PM, David Lyon wrote: > ConfigParser is in every python version that I know of. ConfigParser was first documented in Python 1.5.2, released 30 April 1999. Some of us remember the releases that came before, but (thankfully) we're outnumbered. :-) -Fred -- Fred L. Drake, Jr. "Chaos is the score upon which reality is written." --Henry Miller From david.lyon at preisshare.net Thu Oct 15 01:25:40 2009 From: david.lyon at preisshare.net (David Lyon) Date: Wed, 14 Oct 2009 19:25:40 -0400 Subject: [Distutils] Alternate static metadata PEP submission... In-Reply-To: <4AD5C822.6060000@trueblade.com> References: <94bdd2610910111517u2fd0b136p44958f7faa46e974@mail.gmail.com> <94bdd2610910130136g51b559ey9e3a04123309342e@mail.gmail.com> <1d8404d17e3ebee2b734f928e5af9297@preisshare.net> <94bdd2610910130233o1ee3578fu6c50a584d1e5d53a@mail.gmail.com> <6b5118e1a8ea97f23335679195ab91aa@preisshare.net> <4AD5C822.6060000@trueblade.com> Message-ID: <426b9f35b7c874621bcd79ed626705ae@preisshare.net> Hi Eric.. On Wed, 14 Oct 2009 08:46:26 -0400, Eric Smith wrote: >>> The first one who mentioned the idea was Eric IIRC, then Matthias and >>> Tres worked on it. >> >> Fine. They've been awfully quiet on distutils-sig lately :-) > > Not so! :-) >> Sure. Well "collect" is a word .. > > I'm sure no offense was intended. I don't see how we can all know all > possible words that might be construed as offensive. Collect or collection isn't the slightest bit offensive as Python itself is merely a collection of a lot of peoples good work. > I don't see what David is proposing as being radical or even different > from what we've been discussing: a single static file that contains > enough metadata to describe what's in a distribution. This file needs to > be extensible. The Distutils-SIG approach has been incremental, adding > small parts. It seems he wants to go "big bang". I think that's fine, > but it will be more difficult to migrate to, I fear. Downloading a new setup.py from a website and writing a setup file is easier than somebody new than coding a tradional setup.py file. A tool should be provided to read an existing setup.py and create the corresponding static file. > He's proposing shipping an application in each distribution (his new > setup.py) that processes that file to do installations. Sounds like a > bootstrapper of sorts, and not so radical of an idea. It isn't so different technically from what exists in setuptools. > This setup.py > would use parts of existing distutils to do its work, where it can. I > think a name other than setup.py would reduce confusion, but the idea > seems practical. Speeking as an end user (system administrator), all the documentation says type 'setup.py install'. It's a habit. I think it's the least confusing thing for an end user.. it might confuse developers.. not so much I think. > I'd be interested to hear how this would interact with PyPI and buildout. There is no impact on pypi or buildout. The setup.py has all the same behavior as a traditional setup.py except when it is run at the end users system. What I mean is that all existing command line options get passed through and work the same way as before. David From david.lyon at preisshare.net Thu Oct 15 02:53:24 2009 From: david.lyon at preisshare.net (David Lyon) Date: Wed, 14 Oct 2009 20:53:24 -0400 Subject: [Distutils] Alternate static metadata PEP submission... In-Reply-To: <94bdd2610910140221o7d62839dl1d208c6c69959cc3@mail.gmail.com> References: <20091013103021.GA21066@laurie.devork> <94bdd2610910131001v70711ffeia27433cd23b32fca@mail.gmail.com> <79a53e276e5dffe24be0433771801d72@preisshare.net> <94bdd2610910131426k64f707d7pf4d398d04b8ecc32@mail.gmail.com> <20091013233654.GA27339@laurie.devork> <5fba01026f52bbf1d89cac4908ee17dc@preisshare.net> <94bdd2610910140221o7d62839dl1d208c6c69959cc3@mail.gmail.com> Message-ID: On Wed, 14 Oct 2009 11:21:40 +0200, Tarek Ziad? wrote: > My suggestion is that once you have a revised PEP that includes some > proof of concept > code with less overlapping with PEP 390 (they are different proposals > obviously), > you post it here for a round of feedback, then I'll push it in the > PEPs for further processing Thank you kindly Tarek. I'll look into seeing how I can try to reduce overlap if you think that it's an issue. The problem for me is that under windows it's considered not good programming practice to build logic implicitly into section headers within a configuration file. So things like: [metadata:python_version == '2.4' or python_version == '2.5'] requires = bar Doesn't look at all good to me. It looks like code in a configuration file. I'm really against it. But hey - I'm biased. So for me, the overlap is not strong. Because I see my alternative as a more technically correct way to approach using configuration files. The sample code you provided on the other hand is very nice. Perphaps when you see my code you'll be as horrified at my code as I am at your configuration files. :-) David From kevin at bud.ca Thu Oct 15 03:52:57 2009 From: kevin at bud.ca (Kevin Teague) Date: Wed, 14 Oct 2009 18:52:57 -0700 Subject: [Distutils] Alternate static metadata PEP submission... In-Reply-To: <9cee7ab80910141539u5994e688idf291e0bf69afdcc@mail.gmail.com> References: <94bdd2610910131001v70711ffeia27433cd23b32fca@mail.gmail.com> <79a53e276e5dffe24be0433771801d72@preisshare.net> <94bdd2610910131426k64f707d7pf4d398d04b8ecc32@mail.gmail.com> <20091013233654.GA27339@laurie.devork> <4AD54138.9010602@ar.media.kyoto-u.ac.jp> <50a1116c3a8ca9bcbb28121d90ccc5d0@preisshare.net> <4AD5BA54.4000600@ar.media.kyoto-u.ac.jp> <4520a61448e00a641236151aee632aa8@preisshare.net> <9cee7ab80910141539u5994e688idf291e0bf69afdcc@mail.gmail.com> Message-ID: <938EA196-A76F-4355-BAD7-7767A34BF280@bud.ca> On Oct 14, 2009, at 3:39 PM, Fred Drake wrote: > On Wed, Oct 14, 2009 at 5:59 PM, David Lyon > wrote: >> ConfigParser is in every python version that I know of. > > ConfigParser was first documented in Python 1.5.2, released 30 April > 1999. > > Some of us remember the releases that came before, but (thankfully) > we're outnumbered. :-) 1.5.1 was the first Python that I used, seems like I managed to squeak onto the outnumbered side. The Python Tutorial was really primitive back then (http://www.python.org/doc/1.5.1/tut/ ), it didn't even get it's distinctive blue headers and footers until 1.5.2 (so that it at least a little bit no longer felt like "some dudes pet project"). I like section 9.4, "Randon Remarks" (http://www.python.org/doc/1.5.1/tut/remarks.html ): "Data attributes override method attributes with the same name; to avoid accidental name conflicts, which may cause hard-to-find bugs in large programs, it is wise to use some kind of convention that minimizes the chance of conflicts, e.g., capitalize method names, prefix data attribute names with a small unique string (perhaps just an underscore), or use verbs for methods and nouns for data attributes." The use of a leading underscore for private names hadn't yet been formalized as a convention! :) From david at ar.media.kyoto-u.ac.jp Thu Oct 15 03:59:48 2009 From: david at ar.media.kyoto-u.ac.jp (David Cournapeau) Date: Thu, 15 Oct 2009 10:59:48 +0900 Subject: [Distutils] Alternate static metadata PEP submission... In-Reply-To: <4520a61448e00a641236151aee632aa8@preisshare.net> References: <20091013103021.GA21066@laurie.devork> <94bdd2610910131001v70711ffeia27433cd23b32fca@mail.gmail.com> <79a53e276e5dffe24be0433771801d72@preisshare.net> <94bdd2610910131426k64f707d7pf4d398d04b8ecc32@mail.gmail.com> <20091013233654.GA27339@laurie.devork> <4AD54138.9010602@ar.media.kyoto-u.ac.jp> <50a1116c3a8ca9bcbb28121d90ccc5d0@preisshare.net> <4AD5BA54.4000600@ar.media.kyoto-u.ac.jp> <4520a61448e00a641236151aee632aa8@preisshare.net> Message-ID: <4AD68214.70407@ar.media.kyoto-u.ac.jp> David Lyon wrote: > > We're not talking about too much more than a file copy and run script > api. Saying that a packaging tool is about copying files is not very useful - you could say that programming is just moving bytes around and you would be right as well :) > > I'm really not hung up on .ini style files and am agnostic about > the format. I am quite agnostic on the syntax as well - as PJE mentioned a couple of months ago when talking about a new distutils from scratch, even using XML for a first prototype would be acceptable. The .cabal file has a nice syntax, though, and quite pythonic in some ways (indentation instead of brace for example). What matters is what you can put in the file, and .ini files are too limited IMHO, for the following reasons: - they don't allow hierarchy - they don't allow conditional in arbitrary locations - the set of variables is hard-coded, and custom variables cannot be defined (with the current PEP 390 scheme, there is no way to detect windows 64 for example). > Tell me where I find a built in API in python for this other format > so we can look at it? No harm in looking at it. There is no python package to read this file that I know of - the parser needs to be written. That's why a more explicit, straw-man format could be used as a prototype first (I have no experience nor knowledge in parsing, so I am not really useful for that part). The interesting part is which capabilities must be enabled: possibility to control data tagging in a fine-grained manner to support the traditional unix file system hierarchy, handling installed documentation, etc... And how it can interact with external tools (a default, 'dumb' build tool ala distutils, or more advanced build tools like scons/waf for people who require the complexity). The 'dumb' tool would handle the post install, pre install, etc... you mentioned. cheers, David From david.lyon at preisshare.net Thu Oct 15 04:55:43 2009 From: david.lyon at preisshare.net (David Lyon) Date: Wed, 14 Oct 2009 22:55:43 -0400 Subject: [Distutils] Alternate static metadata PEP submission... In-Reply-To: <938EA196-A76F-4355-BAD7-7767A34BF280@bud.ca> References: <94bdd2610910131001v70711ffeia27433cd23b32fca@mail.gmail.com> <79a53e276e5dffe24be0433771801d72@preisshare.net> <94bdd2610910131426k64f707d7pf4d398d04b8ecc32@mail.gmail.com> <20091013233654.GA27339@laurie.devork> <4AD54138.9010602@ar.media.kyoto-u.ac.jp> <50a1116c3a8ca9bcbb28121d90ccc5d0@preisshare.net> <4AD5BA54.4000600@ar.media.kyoto-u.ac.jp> <4520a61448e00a641236151aee632aa8@preisshare.net> <9cee7ab80910141539u5994e688idf291e0bf69afdcc@mail.gmail.com> <938EA196-A76F-4355-BAD7-7767A34BF280@bud.ca> Message-ID: <339900a32576f8e659f3a0266bd3c210@preisshare.net> On Wed, 14 Oct 2009 18:52:57 -0700, Kevin Teague wrote: >> ConfigParser was first documented in Python 1.5.2, released 30 April >> 1999. >> >> Some of us remember the releases that came before, but (thankfully) >> we're outnumbered. :-) > > 1.5.1 was the first Python that I used, seems like I managed to squeak > onto the outnumbered side. Show's how behind the 8-ball I am.. I only noticed it in 2007.. :-) David From david.lyon at preisshare.net Thu Oct 15 05:13:13 2009 From: david.lyon at preisshare.net (David Lyon) Date: Wed, 14 Oct 2009 23:13:13 -0400 Subject: [Distutils] Alternate static metadata PEP submission... In-Reply-To: <4AD68214.70407@ar.media.kyoto-u.ac.jp> References: <20091013103021.GA21066@laurie.devork> <94bdd2610910131001v70711ffeia27433cd23b32fca@mail.gmail.com> <79a53e276e5dffe24be0433771801d72@preisshare.net> <94bdd2610910131426k64f707d7pf4d398d04b8ecc32@mail.gmail.com> <20091013233654.GA27339@laurie.devork> <4AD54138.9010602@ar.media.kyoto-u.ac.jp> <50a1116c3a8ca9bcbb28121d90ccc5d0@preisshare.net> <4AD5BA54.4000600@ar.media.kyoto-u.ac.jp> <4520a61448e00a641236151aee632aa8@preisshare.net> <4AD68214.70407@ar.media.kyoto-u.ac.jp> Message-ID: On Thu, 15 Oct 2009 10:59:48 +0900, David Cournapeau wrote: > Saying that a packaging tool is about copying files is not very useful - > you could say that programming is just moving bytes around and you would > be right as well :) Well it is about moving bytes around.. That *is* just how I see it.. > I am quite agnostic on the syntax as well - as PJE mentioned a couple of > months ago when talking about a new distutils from scratch, even using > XML for a first prototype would be acceptable. The .cabal file has a > nice syntax, though, and quite pythonic in some ways (indentation > instead of brace for example). It's just too hard to do and adds another layer of complexity to deal with. I'm on limited time like everybody else. ConfigParser for all it's annoying faults is still quire powerful. I don't like nesting in configuration files. I tend to side with the configparser approach of not allowing nesting. More importantly than that, I don't believe that this particular use case gets complicated enough to require nesting. You can achieve a type nesting with configparser by just making the section names ever long. It achieves the same thing imho. Some people disapproved strongly of xml. Most were lukewarm about configparser - and we can't find an implementation of haskel. So for me, I'll go with the lukewarmers.. Primary concern is using something that exists or will work on 'most' python versions. I'm not sure how many python 1.5 thrillseekers still use 1.5 solely anymore. But if there are some, configparser shouldn't offend them. David From david at ar.media.kyoto-u.ac.jp Thu Oct 15 05:16:48 2009 From: david at ar.media.kyoto-u.ac.jp (David Cournapeau) Date: Thu, 15 Oct 2009 12:16:48 +0900 Subject: [Distutils] Alternate static metadata PEP submission... In-Reply-To: References: <20091013103021.GA21066@laurie.devork> <94bdd2610910131001v70711ffeia27433cd23b32fca@mail.gmail.com> <79a53e276e5dffe24be0433771801d72@preisshare.net> <94bdd2610910131426k64f707d7pf4d398d04b8ecc32@mail.gmail.com> <20091013233654.GA27339@laurie.devork> <4AD54138.9010602@ar.media.kyoto-u.ac.jp> <50a1116c3a8ca9bcbb28121d90ccc5d0@preisshare.net> <4AD5BA54.4000600@ar.media.kyoto-u.ac.jp> <4520a61448e00a641236151aee632aa8@preisshare.net> <4AD68214.70407@ar.media.kyoto-u.ac.jp> Message-ID: <4AD69420.8050804@ar.media.kyoto-u.ac.jp> David Lyon wrote: > It's just too hard to do and adds another layer of complexity to > deal with. Hence something like xml for a prototype and discussion - I did not suggest to use xml for the end product package. I have as much love for xml as the next python guy. > I don't like nesting in configuration files. I tend to side with > the configparser approach of not allowing nesting. More importantly > than that, I don't believe that this particular use case gets > complicated enough to require nesting. > I think nested conditions for configurations is a must-have. Once you have more than 2-3 variables, doing it like PEP 390 is not really manageable. Particularly for C extensions, having many configurations is quite common in my experience. > Primary concern is using something that exists or will work > on 'most' python versions. I'm not sure how many python 1.5 > thrillseekers still use 1.5 solely anymore. But if there are > some, configparser shouldn't offend them. > I don't think python 1.5 should be considered - I happen to contribute to scons, which is a build tool in python, and supports all python versions from 1.5.2: the 1.5 requirement is a real downside. You can't use list comprehension, you need apply instead of f(*args, **kw), you don't have subprocess for reliable process execution, and I am not even talking about testing. David From fdrake at gmail.com Thu Oct 15 05:51:27 2009 From: fdrake at gmail.com (Fred Drake) Date: Wed, 14 Oct 2009 23:51:27 -0400 Subject: [Distutils] Alternate static metadata PEP submission... In-Reply-To: <4AD69420.8050804@ar.media.kyoto-u.ac.jp> References: <20091013233654.GA27339@laurie.devork> <4AD54138.9010602@ar.media.kyoto-u.ac.jp> <50a1116c3a8ca9bcbb28121d90ccc5d0@preisshare.net> <4AD5BA54.4000600@ar.media.kyoto-u.ac.jp> <4520a61448e00a641236151aee632aa8@preisshare.net> <4AD68214.70407@ar.media.kyoto-u.ac.jp> <4AD69420.8050804@ar.media.kyoto-u.ac.jp> Message-ID: <9cee7ab80910142051t324812c4o6e70d3b3e77cb5@mail.gmail.com> On Wed, Oct 14, 2009 at 11:16 PM, David Cournapeau wrote: > I don't think python 1.5 should be considered - I happen to contribute > to scons, which is a build tool in python, and supports all python > versions from 1.5.2: the 1.5 requirement is a real downside. I hope 1.x isn't a target for any of these projects. At this point, I'd consider support for Python 2.4 to be quite generous on the part of the developers. And yes, I'm still using Python 2.4. Thankfully I'm not developing or maintaining anything on Python 2.3. I would be surprised if there was widespread interest in adopting new tools in projects using such old versions; keep in mind that even if something stable is available in 6 months, it'll be a year or more before adoption becomes widespread anyway. -Fred -- Fred L. Drake, Jr. "Chaos is the score upon which reality is written." --Henry Miller From david at ar.media.kyoto-u.ac.jp Thu Oct 15 05:48:11 2009 From: david at ar.media.kyoto-u.ac.jp (David Cournapeau) Date: Thu, 15 Oct 2009 12:48:11 +0900 Subject: [Distutils] Alternate static metadata PEP submission... In-Reply-To: <9cee7ab80910142051t324812c4o6e70d3b3e77cb5@mail.gmail.com> References: <20091013233654.GA27339@laurie.devork> <4AD54138.9010602@ar.media.kyoto-u.ac.jp> <50a1116c3a8ca9bcbb28121d90ccc5d0@preisshare.net> <4AD5BA54.4000600@ar.media.kyoto-u.ac.jp> <4520a61448e00a641236151aee632aa8@preisshare.net> <4AD68214.70407@ar.media.kyoto-u.ac.jp> <4AD69420.8050804@ar.media.kyoto-u.ac.jp> <9cee7ab80910142051t324812c4o6e70d3b3e77cb5@mail.gmail.com> Message-ID: <4AD69B7B.4010401@ar.media.kyoto-u.ac.jp> Fred Drake wrote: > On Wed, Oct 14, 2009 at 11:16 PM, David Cournapeau > wrote: > >> I don't think python 1.5 should be considered - I happen to contribute >> to scons, which is a build tool in python, and supports all python >> versions from 1.5.2: the 1.5 requirement is a real downside. >> > > I hope 1.x isn't a target for any of these projects. At this point, > I'd consider support for Python 2.4 to be quite generous on the part > of the developers. > 2.4 is a good number because it is supported on many "enterprise" linux OS (RHEL 5 for example), and brings things which are quite useful for building/packaging. > keep in mind that even if something stable is available in 6 > months, it'll be a year or more before adoption becomes widespread > anyway. > It is just one data point, but when scons started deprecating python < 2.2 sometimes last year, I have not seen anyone complaining on the scons ML. David From david.lyon at preisshare.net Thu Oct 15 06:22:42 2009 From: david.lyon at preisshare.net (David Lyon) Date: Thu, 15 Oct 2009 00:22:42 -0400 Subject: [Distutils] Alternate static metadata PEP submission... In-Reply-To: <4AD69420.8050804@ar.media.kyoto-u.ac.jp> References: <20091013103021.GA21066@laurie.devork> <94bdd2610910131001v70711ffeia27433cd23b32fca@mail.gmail.com> <79a53e276e5dffe24be0433771801d72@preisshare.net> <94bdd2610910131426k64f707d7pf4d398d04b8ecc32@mail.gmail.com> <20091013233654.GA27339@laurie.devork> <4AD54138.9010602@ar.media.kyoto-u.ac.jp> <50a1116c3a8ca9bcbb28121d90ccc5d0@preisshare.net> <4AD5BA54.4000600@ar.media.kyoto-u.ac.jp> <4520a61448e00a641236151aee632aa8@preisshare.net> <4AD68214.70407@ar.media.kyoto-u.ac.jp> <4AD69420.8050804@ar.media.kyoto-u.ac.jp> Message-ID: <1e5ca572044e2d50395bed41f1a4a67c@preisshare.net> On Thu, 15 Oct 2009 12:16:48 +0900, David Cournapeau wrote: > I think nested conditions for configurations is a must-have. Once you > have more than 2-3 variables, doing it like PEP 390 is not really > manageable. Particularly for C extensions, having many configurations is > quite common in my experience. Like I said, one can achieve 'and' and 'or' operations with configparser headings. [dependencies] packages = pyopengl [dependencies python<=2.4] packages = lxml >= 2.5 [dependencies mac python<=2.4] packages = shinyxml Translates roughly into the following python code: dependencies = ["pyopengl"] if sys.version <= '2.4': dependencies.append('lxml') if sys.platform == 'darwin': dependencies.append('shinyxml') So to me, configparser does provide a way to achieve a sort of nested capability. It just needs careful choices of headings. Some people also use '-' to provide nesting. For example: [top] value = 1 [top - nested level 1] value = 2 [top - nested level 2] value = 3 These are accessed with 'top' + " - nested level %d" % levelno To me this is all nesting. I'm extremely interested to know of a practical example of something related to packaging that we can't script for. If you think of something, please post it. 'build' and 'test' were too other keywords that were suggested to work in also. I don't really want to do a build platform, only something for the install platform. But here's how they could work. [dependencies windows build] packages = py2exe [dependencies mac build] packages = py2app [dependencies test] packages = unittestplus David From david at ar.media.kyoto-u.ac.jp Thu Oct 15 06:39:32 2009 From: david at ar.media.kyoto-u.ac.jp (David Cournapeau) Date: Thu, 15 Oct 2009 13:39:32 +0900 Subject: [Distutils] Alternate static metadata PEP submission... In-Reply-To: <1e5ca572044e2d50395bed41f1a4a67c@preisshare.net> References: <20091013103021.GA21066@laurie.devork> <94bdd2610910131001v70711ffeia27433cd23b32fca@mail.gmail.com> <79a53e276e5dffe24be0433771801d72@preisshare.net> <94bdd2610910131426k64f707d7pf4d398d04b8ecc32@mail.gmail.com> <20091013233654.GA27339@laurie.devork> <4AD54138.9010602@ar.media.kyoto-u.ac.jp> <50a1116c3a8ca9bcbb28121d90ccc5d0@preisshare.net> <4AD5BA54.4000600@ar.media.kyoto-u.ac.jp> <4520a61448e00a641236151aee632aa8@preisshare.net> <4AD68214.70407@ar.media.kyoto-u.ac.jp> <4AD69420.8050804@ar.media.kyoto-u.ac.jp> <1e5ca572044e2d50395bed41f1a4a67c@preisshare.net> Message-ID: <4AD6A784.3050607@ar.media.kyoto-u.ac.jp> David Lyon wrote: > [dependencies] > packages = pyopengl > > [dependencies python<=2.4] > packages = lxml >= 2.5 > > [dependencies mac python<=2.4] > packages = shinyxml > > Translates roughly into the following python code: > > dependencies = ["pyopengl"] > if sys.version <= '2.4': > dependencies.append('lxml') > if sys.platform == 'darwin': > dependencies.append('shinyxml') > > So to me, configparser does provide a way to achieve a > sort of nested capability. > I understand this, but when you a lot of conditioning, "flattening" the conditions is not really readable. If you have say 10 variables instead of 2 to condition on, the sections would be several lines. More significantly though, how can you define new conditions variables with .ini files ? Would you have a special section for this ? David From david.lyon at preisshare.net Thu Oct 15 07:38:46 2009 From: david.lyon at preisshare.net (David Lyon) Date: Thu, 15 Oct 2009 01:38:46 -0400 Subject: [Distutils] Alternate static metadata PEP submission... In-Reply-To: <4AD6A784.3050607@ar.media.kyoto-u.ac.jp> References: <20091013103021.GA21066@laurie.devork> <94bdd2610910131001v70711ffeia27433cd23b32fca@mail.gmail.com> <79a53e276e5dffe24be0433771801d72@preisshare.net> <94bdd2610910131426k64f707d7pf4d398d04b8ecc32@mail.gmail.com> <20091013233654.GA27339@laurie.devork> <4AD54138.9010602@ar.media.kyoto-u.ac.jp> <50a1116c3a8ca9bcbb28121d90ccc5d0@preisshare.net> <4AD5BA54.4000600@ar.media.kyoto-u.ac.jp> <4520a61448e00a641236151aee632aa8@preisshare.net> <4AD68214.70407@ar.media.kyoto-u.ac.jp> <4AD69420.8050804@ar.media.kyoto-u.ac.jp> <1e5ca572044e2d50395bed41f1a4a67c@preisshare.net> <4AD6A784.3050607@ar.media.kyoto-u.ac.jp> Message-ID: On Thu, 15 Oct 2009 13:39:32 +0900, David Cournapeau wrote: > I understand this, but when you a lot of conditioning, "flattening" the > conditions is not really readable. If you have say 10 variables instead > of 2 to condition on, the sections would be several lines. I'm only allowing up to four. These are: - specifier ['dependencies','preinstall','postinstall','scripts', 'application', 'package', 'documentation', 'uninstall', 'datafiles'] - operating system ['windows','mac','linux' etc...] plus some twiddle-bits like 32, 64, gnome, kde and os-version. - python version - operation ['test','build' ..]. So never above 4. Typically two or three. > More significantly though, how can you define new conditions variables > with .ini files ? Would you have a special section for this ? I don't have a way of doing that right now - and am not aware of practical examples that might require it. Anything that can't be handled with the above will have to go into a pre-install or post-install script and be coded with traditional python coding. If you can find any common use cases that aren't handled, please advise. In the real world, there is a point where things might need to go to code. This capability is provided for. [preinstall] script = checksystem.py [preinstall linux-kde] script = checkmylinux.sh and: [postinstall windows-64-xp python-2.4] script = regedit.exe /s fixpy24win.txt [postinstall windows-64-xp python-2.5] script = regedit.exe /s fixpy25win.txt What that would do would be to make a system registry correction after loading on 64 bit windows only for python versions 2.4 or 2.5. Similarly: [postinstall linux-debian-64 python-2.5] script = fixpy25deb.py [postinstall linux-suse-kde python-2.5] script = fixpy25susekde.py [postinstall mac] script = popupkoolmachinechoice.py I'm really not trying to handle every possible case, just the most common ones. That is, trying to provide a hook for specific unique situations that exist on different platforms and operating systems. David > > David From regebro at gmail.com Thu Oct 15 07:56:34 2009 From: regebro at gmail.com (Lennart Regebro) Date: Thu, 15 Oct 2009 07:56:34 +0200 Subject: [Distutils] Future namespace support In-Reply-To: References: <319e029f0910140728k609cf813nae71d5d779935d41@mail.gmail.com> <4AD5E414.8000906@cryptelium.net> <319e029f0910140755t1ec8903aufd392cca80dc35e7@mail.gmail.com> <20091014200328.4D2C83A407A@sparrow.telecommunity.com> <319e029f0910141446p5e57482dw94e27e653b4d0be@mail.gmail.com> Message-ID: <319e029f0910142256m21b6f37foaeba8b4b009bbcf5@mail.gmail.com> 2009/10/15 Tres Seaver : > Right, its just a race: ?whichever project gets its version of the > namespace package imported "first" wins, sets the __path__, and > everything works. ?No consolation prizes for the others, because they > *never get imported* (their __init__.py is never even considered). Well, in my little test they both get imported. Both the pkgutil one and the setuptools one. Otherwise, how could they extend the __path__? -- Lennart Regebro: Python, Zope, Plone, Grok http://regebro.wordpress.com/ +33 661 58 14 64 From regebro at gmail.com Thu Oct 15 07:58:08 2009 From: regebro at gmail.com (Lennart Regebro) Date: Thu, 15 Oct 2009 07:58:08 +0200 Subject: [Distutils] Future namespace support In-Reply-To: <4AD627C8.6080507@cryptelium.net> References: <319e029f0910140728k609cf813nae71d5d779935d41@mail.gmail.com> <4AD5EF76.4080304@cryptelium.net> <319e029f0910140844q1d0554day94276f84d0805cce@mail.gmail.com> <4AD5F83B.8010409@cryptelium.net> <319e029f0910140917hc0dff08x53b292476482808@mail.gmail.com> <4AD5FC24.30500@cryptelium.net> <319e029f0910140931w443d927na77eb98a68b8f1f4@mail.gmail.com> <4AD60081.6050000@cryptelium.net> <319e029f0910141031n2ebef57eodf16b48ea8e239b4@mail.gmail.com> <4AD627C8.6080507@cryptelium.net> Message-ID: <319e029f0910142258p3d7097c6m3dbebe94a5f77a2e@mail.gmail.com> 2009/10/14 kiorky : > In the context of migration, for example. > Take the plone "collective" namespace, all the modules won't be updated at the > same time, we will have a painful cohabitation time. Well, making coordinated releases there is tricky, since there is different people having the rights on PyPI. But that is one of the very few namespaces this would be a problem at. But the current solutions do not have any problems in being mixed, so I still don't see why a new solution should have that problem. >> I see no reason that they should not continue to work. > Choose to go on that new stuff that is not backward compatible. That new stuff doesn't exist yet. How do you know it will not work? > I ve heard that before. Hay, Tarek, go out that body ! Yep, i know, but these > are point that we must keep in mind before the new implementation prevent us > from providing bits of backward compatibility. You have gotten the idea that just because the namespace PEP will not be backwards compatible with setuptools 0.6, that means that you can't mix all of them in the same system. If this is correct, I would like to hear *why* this is so. Because then we need to change that. But unless you can explain why it will be impossible to mix them, then I will trust my gut feeling that it is possible. -- Lennart Regebro: Python, Zope, Plone, Grok http://regebro.wordpress.com/ +33 661 58 14 64 From david at ar.media.kyoto-u.ac.jp Thu Oct 15 07:48:00 2009 From: david at ar.media.kyoto-u.ac.jp (David Cournapeau) Date: Thu, 15 Oct 2009 14:48:00 +0900 Subject: [Distutils] Alternate static metadata PEP submission... In-Reply-To: References: <20091013103021.GA21066@laurie.devork> <94bdd2610910131001v70711ffeia27433cd23b32fca@mail.gmail.com> <79a53e276e5dffe24be0433771801d72@preisshare.net> <94bdd2610910131426k64f707d7pf4d398d04b8ecc32@mail.gmail.com> <20091013233654.GA27339@laurie.devork> <4AD54138.9010602@ar.media.kyoto-u.ac.jp> <50a1116c3a8ca9bcbb28121d90ccc5d0@preisshare.net> <4AD5BA54.4000600@ar.media.kyoto-u.ac.jp> <4520a61448e00a641236151aee632aa8@preisshare.net> <4AD68214.70407@ar.media.kyoto-u.ac.jp> <4AD69420.8050804@ar.media.kyoto-u.ac.jp> <1e5ca572044e2d50395bed41f1a4a67c@preisshare.net> <4AD6A784.3050607@ar.media.kyoto-u.ac.jp> Message-ID: <4AD6B790.4040208@ar.media.kyoto-u.ac.jp> David Lyon wrote: > I don't have a way of doing that right now - and am not aware of > practical examples that might require it. > > Anything that can't be handled with the above will have to go into > a pre-install or post-install script and be coded with traditional > python coding. > Let's take a common case: using cython/pyrex to accelerate some code, but you still want people to be able to use your package without a C compiler (bzr, cython are examples of such packages). If we had a way to define variables as in .cabal files, this could be used to control the build from 'outside'. I am not sure I understand your goal of a new format if you only want to handle common cases which are already handled in distutils. For me, the goal of having static metadata should be easier interoperation between different tools, and as such the format should be rich enough to describe entirely common cases. You could then have a distutils2-debianize tool which would read this file and produce a .deb, another tool for eggs, windows installers, etc... or you could use the parsed file description to control the build from a real build tool system like waf or scons. That's the kind of things I want to be able to do and are so painful right now with distutils. David From ziade.tarek at gmail.com Thu Oct 15 09:50:18 2009 From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=) Date: Thu, 15 Oct 2009 09:50:18 +0200 Subject: [Distutils] Distribute 0.6.5 coming up Message-ID: <94bdd2610910150050l3d12077dj211e5116390fa8f9@mail.gmail.com> Hey, Reinout and I have worked on bugs related to zc.buildout support, and it seems that now everything works fine. Thanks for those who try it on their side ! So we will be shipping a 0.6.5 probably friday for that, and postpone all the other issues for 0.6.6, except a few ones that are easy to tackle now. This version will also contaon the code that lakes sure we don't step on any Setuptools 0.7.x toes. If you have one issue you want to see fixed in 0.6.5, speak up. buildouters, if you want to try the latest dev version you can grab : http://python-distribute.org/bootstrap_dev.py Once 0.6.6 is proven to be working fine with zc.buildout, I'll change on zc.buildout side the bootstrap.py script as suggested, by adding a "--distribute" option so people can choose the tool they want to use. Thanks Tarek -- Tarek Ziad? | http://ziade.org |????????????! |???????????? From reinout at vanrees.org Thu Oct 15 11:10:53 2009 From: reinout at vanrees.org (Reinout van Rees) Date: Thu, 15 Oct 2009 09:10:53 +0000 (UTC) Subject: [Distutils] Distribute 0.6.5 coming up References: <94bdd2610910150050l3d12077dj211e5116390fa8f9@mail.gmail.com> Message-ID: On 2009-10-15, Tarek Ziad? wrote: > > buildouters, if you want to try the latest dev version you can grab : > http://python-distribute.org/bootstrap_dev.py To make it explicit: grabbing this bootstrap and using it is safe for every project. And it is the only change you need to make! - No need for a specific patched buildout release or so. - No need to mention distribute anywhere in your dependencies. - No worries about other buildouts: distribute doesn't touch the setuptools egg in your (shared) buildout eggs directory. They'll continue working just fine. - It just transparently works like a charm :-) When you want the advantages (svn 1.6 support and other bugfixes) of distribute outside of buildout: do the normal "easy_install distribute". 0.6.4 should work fine there (but do of course update when 0.6.5 is out). Reinout -- Reinout van Rees - reinout at vanrees.org - http://reinout.vanrees.org Software developer at http://www.thehealthagency.com "Military engineers build missiles. Civil engineers build targets" From ziade.tarek at gmail.com Thu Oct 15 14:32:16 2009 From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=) Date: Thu, 15 Oct 2009 14:32:16 +0200 Subject: [Distutils] Proposal for Distribute 0.7 Message-ID: <94bdd2610910150532x430ba857hf66afa19f8e90c70@mail.gmail.com> Phillip, I have a proposal for Distribute 0.7 : let's join forces and all work together in that tool. "All" would be the commiters of Distribute, and PJE. These people would all be in the same team with the same commiter rights, and I propose that you become the leader of that tool and guide us in its development. (but with the usual +1/-1 stuff of course) That would make us happy, because we would be able to work and continue if you are not available sometimes without the feeling that we are locked, and that would give you the manpower you miss to develop some of your idea. That would be the best thing that could happen imho for python packaging. And if you agree, and want the project name, we can all find a neutral name that everybody would like. Cheers, Tarek From ziade.tarek at gmail.com Thu Oct 15 14:33:55 2009 From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=) Date: Thu, 15 Oct 2009 14:33:55 +0200 Subject: [Distutils] Proposal for Distribute 0.7 In-Reply-To: <94bdd2610910150532x430ba857hf66afa19f8e90c70@mail.gmail.com> References: <94bdd2610910150532x430ba857hf66afa19f8e90c70@mail.gmail.com> Message-ID: <94bdd2610910150533t581786f2i494761ae4ac474e8@mail.gmail.com> On Thu, Oct 15, 2009 at 2:32 PM, Tarek Ziad? wrote: > > And if you agree, and want the project name, ... to be changed... > we can all find a neutral name that everybody would like. From lists at informa.tiker.net Thu Oct 15 15:40:38 2009 From: lists at informa.tiker.net (Andreas =?iso-8859-1?q?Kl=F6ckner?=) Date: Thu, 15 Oct 2009 09:40:38 -0400 Subject: [Distutils] distribute equivalent of "setup.py develop"? Message-ID: <200910150940.39599.lists@informa.tiker.net> Hi there, I've recently tried distribute, and much to my dismay found that it doesn't appear to contain an equivalent of "setup.py develop". Is such a thing planned, or, even better, available now? Thanks, Andreas -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 190 bytes Desc: This is a digitally signed message part. URL: From david.lyon at preisshare.net Thu Oct 15 15:55:23 2009 From: david.lyon at preisshare.net (David Lyon) Date: Thu, 15 Oct 2009 09:55:23 -0400 Subject: [Distutils] Alternate static metadata PEP submission... In-Reply-To: <4AD6B790.4040208@ar.media.kyoto-u.ac.jp> References: <20091013103021.GA21066@laurie.devork> <94bdd2610910131001v70711ffeia27433cd23b32fca@mail.gmail.com> <79a53e276e5dffe24be0433771801d72@preisshare.net> <94bdd2610910131426k64f707d7pf4d398d04b8ecc32@mail.gmail.com> <20091013233654.GA27339@laurie.devork> <4AD54138.9010602@ar.media.kyoto-u.ac.jp> <50a1116c3a8ca9bcbb28121d90ccc5d0@preisshare.net> <4AD5BA54.4000600@ar.media.kyoto-u.ac.jp> <4520a61448e00a641236151aee632aa8@preisshare.net> <4AD68214.70407@ar.media.kyoto-u.ac.jp> <4AD69420.8050804@ar.media.kyoto-u.ac.jp> <1e5ca572044e2d50395bed41f1a4a67c@preisshare.net> <4AD6A784.3050607@ar.media.kyoto-u.ac.jp> <4AD6B790.4040208@ar.media.kyoto-u.ac.jp> Message-ID: <7601812317e8e6525679f787bdd3aad8@preisshare.net> On Thu, 15 Oct 2009 14:48:00 +0900, David Cournapeau wrote: > Let's take a common case: using cython/pyrex to accelerate some code, > but you still want people to be able to use your package without a C > compiler (bzr, cython are examples of such packages). I believe that's possible through the [postinstall] section.. > If we had a way to define variables as in .cabal files, this could be > used to control the build from 'outside'. Any key can be variable: [postinstall] variable_value = 10 Outside programs can 'read' and 'write' and pass values via this setup.info file during build time. That is the whole point. Simply change the setup.info file. > For me, the goal of having static > metadata should be easier interoperation between different tools, and as > such the format should be rich enough to describe entirely common cases. I totally agree. > You could then have a distutils2-debianize tool which would read this > file and produce a .deb, another tool for eggs, windows installers, > etc... or you could use the parsed file description to control the build > from a real build tool system like waf or scons. That's the kind of > things I want to be able to do and are so painful right now with distutils. I'm not really addressing the developer build side at the moment because my experience with it is less. What I feel that I have more knowledge with is the install side and any build/configuration that might take place there. Thanks for the discussion - very informative. David From alex.gronholm at nextday.fi Thu Oct 15 16:08:51 2009 From: alex.gronholm at nextday.fi (=?ISO-8859-1?Q?Alex_Gr=F6nholm?=) Date: Thu, 15 Oct 2009 17:08:51 +0300 Subject: [Distutils] distribute equivalent of "setup.py develop"? In-Reply-To: <200910150940.39599.lists@informa.tiker.net> References: <200910150940.39599.lists@informa.tiker.net> Message-ID: <4AD72CF3.7010001@nextday.fi> Andreas Kl?ckner kirjoitti: > Hi there, > > I've recently tried distribute, and much to my dismay found that it doesn't > appear to contain an equivalent of "setup.py develop". Is such a thing > planned, or, even better, available now? > > You must be mistaken. Distribute definitely has such a command. > Thanks, > Andreas > > ------------------------------------------------------------------------ > > _______________________________________________ > Distutils-SIG maillist - Distutils-SIG at python.org > http://mail.python.org/mailman/listinfo/distutils-sig > From lists at informa.tiker.net Thu Oct 15 16:05:57 2009 From: lists at informa.tiker.net (Andreas =?iso-8859-15?q?Kl=F6ckner?=) Date: Thu, 15 Oct 2009 10:05:57 -0400 Subject: [Distutils] distribute equivalent of "setup.py develop"? In-Reply-To: <200910150940.39599.lists@informa.tiker.net> References: <200910150940.39599.lists@informa.tiker.net> Message-ID: <200910151005.59531.lists@informa.tiker.net> On Donnerstag 15 Oktober 2009, Andreas Kl?ckner wrote: > Hi there, > > I've recently tried distribute, and much to my dismay found that it doesn't > appear to contain an equivalent of "setup.py develop". Is such a thing > planned, or, even better, available now? Ah, never mind. I had screwed up my install somehow--not sure. In any case, it seems all too easy to do that. As a plea to everyone on this list: please keep in mind that what you decide affects large numbers of package authors and an even larger number of users. Please give us the packaging stability we need. The well-being of the Python ecosystem is at your mercy. Be excellent to each other--and to your user community. Thanks for all your work, Andreas -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 190 bytes Desc: This is a digitally signed message part. URL: From alex.gronholm at nextday.fi Thu Oct 15 16:15:48 2009 From: alex.gronholm at nextday.fi (=?ISO-8859-15?Q?Alex_Gr=F6nholm?=) Date: Thu, 15 Oct 2009 17:15:48 +0300 Subject: [Distutils] distribute equivalent of "setup.py develop"? In-Reply-To: <200910151005.59531.lists@informa.tiker.net> References: <200910150940.39599.lists@informa.tiker.net> <200910151005.59531.lists@informa.tiker.net> Message-ID: <4AD72E94.9020103@nextday.fi> Andreas Kl?ckner kirjoitti: > On Donnerstag 15 Oktober 2009, Andreas Kl?ckner wrote: > >> Hi there, >> >> I've recently tried distribute, and much to my dismay found that it doesn't >> appear to contain an equivalent of "setup.py develop". Is such a thing >> planned, or, even better, available now? >> > > Ah, never mind. I had screwed up my install somehow--not sure. In any case, it > seems all too easy to do that. > > As a plea to everyone on this list: please keep in mind that what you decide > affects large numbers of package authors and an even larger number of users. > Please give us the packaging stability we need. The well-being of the Python > ecosystem is at your mercy. > > We are well aware of that, thank you :P > Be excellent to each other--and to your user community. > > Thanks for all your work, > Andreas > > ------------------------------------------------------------------------ > > _______________________________________________ > Distutils-SIG maillist - Distutils-SIG at python.org > http://mail.python.org/mailman/listinfo/distutils-sig > From pje at telecommunity.com Thu Oct 15 19:50:27 2009 From: pje at telecommunity.com (P.J. Eby) Date: Thu, 15 Oct 2009 13:50:27 -0400 Subject: [Distutils] Proposal for Distribute 0.7 In-Reply-To: <94bdd2610910150532x430ba857hf66afa19f8e90c70@mail.gmail.co m> References: <94bdd2610910150532x430ba857hf66afa19f8e90c70@mail.gmail.com> Message-ID: <20091015175042.16F173A4068@sparrow.telecommunity.com> At 02:32 PM 10/15/2009 +0200, Tarek Ziad? wrote: >That would make us happy, because we would be able to work and >continue if you are not available sometimes without the feeling >that we are locked, and that would give you the manpower you miss to >develop some of your idea. So... you've explained what part of this proposal will make you happy. Where's the part that will make me happy? Generally speaking, that's how a "win-win" proposal works. There's a second issue, too. You've repeatedly smeared setuptools on Python-Dev to cover your mistakes, and you have *yet* to retract or amend any of those statements or apologize for them in any of the places where you made them -- despite numerous requests that you do so. In each case where I've made a request, I know you've received the request, because you've replied to the email or comment in which I made it. But in none of your replies have you even *acknowledged* the issue, let alone done anything about it. Instead, you've either changed the venue, the subject, or just launched a new smear. This latest "proposal" of yours is of course just another such subject and venue change, in reply to one of my more recent requests (via Reddit comment) that you do something about this outstanding issue. But if you were genuinely serious about the things you claim to be interested in, you might have paid attention to such things as my mentions that there are other people on the Distribute team who I'd seriously consider as committers on setuptools or even as a chief maintainer of the setuptools 0.6 line (if not more). In my book, the mere fact that you haven't asked me who those team members are, or offered to step down in favor of them (either privately OR publicly), tells me everything I need to know about where your real interest in this matter lies. Now, don't get me wrong - I'm not blacklisting you or refusing to support you. You are no less a user of setuptools than any other, after all. To the extent that you are courteous in your requests regarding setuptools, you will get reasonably courteous responses from me, as has been the case on Python-Dev and here in the last few days. (And for that matter, all along!) But please don't bother floating any more proposals for some sort of direct collaboration or co-operation between you and I, not now, or ever again, until such time as you have shown as consistent a track record of behaving with integrity, as you've shown a *lack of* in your interactions with me. Because as of right now, I can't trust you enough to work with you. I can't trust you not to blame every mistake you make on me, and then say, "oh let's just forget about everything" when you're called on it. I can't trust you to admit when you don't know something (see our recent Catalog-SIG exchange), or to not throw my agreement to work with you back in my face the first time we disagree on something. And these things alone would be more than enough reason not to work on a software project with someone. But at this point, I can't even trust you to not immediately follow up my mere *agreement* to work with you, with a blog post like "PJE finally comes to his senses" or some such rubbish. Your political gaming behavior is a deal-breaker for me, and it's not something that any behavior you do *now* can fix. I've tried for months to give you the benefit of the doubt, the benefit of the language barrier, and maybe even the benefit of a complete lack of sensitivity or clue on your part. And, most recently, you've had no fewer than *five* extra chances from me to act with integrity, but you've chosen to showboat instead. Every. Single. Time. So, good luck with that, 'cause I'm done with it. To the rest of the Distribute team: I have absolutely no quarrel with any of you, as far as I know, and I remain open to an (off-list) discussion on any topics or win-win proposals you'd like to discuss. P.S. Tarek: if your next move is to issue a series of halfhearted, backhanded retractions so you can whine that you did what I asked and I still won't work with you, please don't bother. It's not really credible evidence of your integrity at this point, so I'm done with asking you for anything. IOW, this is *not* a sixth request for a retraction, it's the termination of my many, many attempts to deal with you in good faith. P.P.S. Everyone else: please don't clog the SIG mailing list with further discussion on this topic. If you feel the need to flame me for some reason, please show more consideration to the group than Tarek has, by sending it to me in private, or go rant on your blog or twitter if your need for attention is too great. In return, I will guarantee this to be my last mailing list post on this subject. (I guarantee that I'm at *least* as tired of it as you are.) Thanks in advance. From ziade.tarek at gmail.com Thu Oct 15 20:06:27 2009 From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=) Date: Thu, 15 Oct 2009 20:06:27 +0200 Subject: [Distutils] Proposal for Distribute 0.7 In-Reply-To: <20091015175042.16F173A4068@sparrow.telecommunity.com> References: <94bdd2610910150532x430ba857hf66afa19f8e90c70@mail.gmail.com> <20091015175042.16F173A4068@sparrow.telecommunity.com> Message-ID: <94bdd2610910151106r23c84d65k710ab3979ee17b8f@mail.gmail.com> On Thu, Oct 15, 2009 at 7:50 PM, P.J. Eby wrote: > > So, good luck with that, 'cause I'm done with it. I am sorry we were unable to understand each other. I proposed you to join because I felt it was the right thing to do. But that doesn't matter at this point. We'll just work on our side. Good luck for your projects too then. From ssteinerx at gmail.com Thu Oct 15 20:38:09 2009 From: ssteinerx at gmail.com (ssteinerX@gmail.com) Date: Thu, 15 Oct 2009 14:38:09 -0400 Subject: [Distutils] Proposal for Distribute 0.7 In-Reply-To: <20091015175042.16F173A4068@sparrow.telecommunity.com> References: <94bdd2610910150532x430ba857hf66afa19f8e90c70@mail.gmail.com> <20091015175042.16F173A4068@sparrow.telecommunity.com> Message-ID: <823FAD8D-F2FA-4D0E-9BF8-011AE4516ADC@gmail.com> > that there are other people on the Distribute team who I'd > seriously consider as committers on setuptools or even as a chief > maintainer of the setuptools 0.6 line (if not more). > ...asked me who those team members are I'm asking. Who are they and, if you are willing to give them access, have you offered it and they've declined, or you were waiting until...what? You had Tarek's acknowledgement or permission? We, regular Python users, have been asking for you to let someone help with setuptools for years since you obviously have other priorities and the various issues in setuptools have affected many of us in various ways. Are you really willing to let anyone help? Really? Do it, then, and let's just let this asinine infighting go down in history as one of the stupidest things a language community has ever done to its own credibility in public and move on. Not having an evolving, comprehensive packaging system, that is keeping up with the ever-changing requirements of our daily work is really hurting the language, the community, and us, as working Python programmers, every day. Thanks, S From jmg3000 at gmail.com Thu Oct 15 20:41:56 2009 From: jmg3000 at gmail.com (John Gabriele) Date: Thu, 15 Oct 2009 14:41:56 -0400 Subject: [Distutils] Distribute docs at packages.python.org? Message-ID: <65e0bb520910151141h1c3c5ec5o8e6e8ee544b9049@mail.gmail.com> Was just looking at the Distribute docs at http://packages.python.org/distribute/index.html (nicely styled, btw). Are there any other projects at http://packages.python.org/ (note, that page actually doesn't even load)? What is the purpose of packages.python.org , and how does one get their project listed there? Btw, regarding the Distribute docs there, thanks for the Roadmap! ---John From reinout at vanrees.org Thu Oct 15 21:03:22 2009 From: reinout at vanrees.org (Reinout van Rees) Date: Thu, 15 Oct 2009 19:03:22 +0000 (UTC) Subject: [Distutils] Distribute docs at packages.python.org? References: <65e0bb520910151141h1c3c5ec5o8e6e8ee544b9049@mail.gmail.com> Message-ID: On 2009-10-15, John Gabriele wrote: > Was just looking at the Distribute docs at > http://packages.python.org/distribute/index.html (nicely styled, btw). > Are there any other projects at http://packages.python.org/ (note, > that page actually doesn't even load)? What is the purpose of > packages.python.org , and how does one get their project listed there? There are quite a lot, according to our trusty google: http://www.google.com/search?q=site%3Ahttp%3A%2F%2Fpackages.python.org There's an "upload zipfile with plain html for packages.python.org" link somewhere on your pypi projects's page. And there's a library around that does it for you. If I remember correcty, this has been integrated into "distribute" already to loud cheers. Reinout -- Reinout van Rees - reinout at vanrees.org - http://reinout.vanrees.org Software developer at http://www.thehealthagency.com "Military engineers build missiles. Civil engineers build targets" From pje at telecommunity.com Thu Oct 15 21:55:35 2009 From: pje at telecommunity.com (P.J. Eby) Date: Thu, 15 Oct 2009 15:55:35 -0400 Subject: [Distutils] Proposal for Distribute 0.7 In-Reply-To: <823FAD8D-F2FA-4D0E-9BF8-011AE4516ADC@gmail.com> References: <94bdd2610910150532x430ba857hf66afa19f8e90c70@mail.gmail.com> <20091015175042.16F173A4068@sparrow.telecommunity.com> <823FAD8D-F2FA-4D0E-9BF8-011AE4516ADC@gmail.com> Message-ID: <20091015195548.DCF8F3A4119@sparrow.telecommunity.com> At 02:38 PM 10/15/2009 -0400, ssteinerX at gmail.com wrote: >> that there are other people on the Distribute team who I'd >>seriously consider as committers on setuptools or even as a chief >>maintainer of the setuptools 0.6 line (if not more). >>...asked me who those team members are > >I'm asking. Who are they and, if you are willing to give them access, >have you offered it and they've declined, or you were waiting >until...what? You had Tarek's acknowledgement or permission? I am not trying to poach anyone or stir up trouble, which is why I've taken a very passive public stance on the issue -- although it has included me naming names in certain venues, including here. However, I *am* in private contact with more than one member of the Distribute team, each of whom first contacted me. In order to avoid creating any further drama here or in the Distribute team, I will leave it up to them to make any public statements, when/if they choose to confer with their colleagues on the matter. (I would prefer, of course, a joint statement at the appropriate time.) I wish that such complications weren't necessary. If cooler heads had prevailed in July, this could have and likely would have been resolved back then. >We, regular Python users, have been asking for you to let someone help >with setuptools for years since you obviously have other priorities >and the various issues in setuptools have affected many of us in >various ways. The only reason I've done an 0.6c10 is because of a policy-breaking change to Python that breaks setuptools users, and that can't be worked around with configuration, command-line options, or other tweaking of the runtime environment -- as was able to be done with approximately 9 out of 10 of the setuptools bugs in the tracker. If it weren't for that, I'd have been more-or-less happy to let Distribute become the quasi-official replacement for setuptools 0.6, despite my annoyance at some of the public comments made by its promoter(s) prior to the 2.6.3 issue. Indeed, as previously stated, I even tried to arrange a handoff of 0.6, wherein the changes made in Distribute would've been released as 0.6c10 or 0.6final back in July... and public talks broke down due to certain persons' flaming and posturing. That's one reason I'm only doing off-list talks now -- less chance of random jerks inserting themselves into the middle of the discussion, making me think they're part of Distribute and that their comments reflect Distribute policy (as also happened back in July). Also, no need for persons on either side to put up any posture or spin, when there's no audience to play to. >Are you really willing to let anyone help? Really? Indeed. There's a list, and some of them obviously have time to work on Distribute, so it's not a matter of me only picking people who don't have any time, as Tarek and others have more than once accused me of. >Do it, then, Discussions are ongoing. Also, I've made it pretty plain for a long time that if Ian Bicking or Jim Fulton were ever willing to take it over, I'd hand it *all* over -- 0.7 as well as 0.6 -- and happily retire from the distribution tools business. I trust their vision as architects, and their track record of supporting their users IMO speaks for itself. Both are past contributors of non-trivial features to setuptools, as well as accomplished installation tool developers in their own right. It would be a joy and an honor to turn the keys over to either one of them, either as an individual or as the leader of a new team. (I would love to see some of pip and buildout's features integrated in setuptools 0.7, for example.) (Ian, btw, already has PyPI maintainer rights to setuptools, and Jim already has commit rights to setuptools SVN... giving further lie to the idea that I'm not willing to give people access or let them help.) That having been said, there are definitely other people I'd give varying degrees of access to, just not at the "here, take it, please, I want you to have it!" level I would grant to Ian or Jim. ;-) From ssteinerx at gmail.com Thu Oct 15 22:12:08 2009 From: ssteinerx at gmail.com (ssteinerX@gmail.com) Date: Thu, 15 Oct 2009 16:12:08 -0400 Subject: [Distutils] Proposal for Distribute 0.7 In-Reply-To: <20091015195548.DCF8F3A4119@sparrow.telecommunity.com> References: <94bdd2610910150532x430ba857hf66afa19f8e90c70@mail.gmail.com> <20091015175042.16F173A4068@sparrow.telecommunity.com> <823FAD8D-F2FA-4D0E-9BF8-011AE4516ADC@gmail.com> <20091015195548.DCF8F3A4119@sparrow.telecommunity.com> Message-ID: On Oct 15, 2009, at 3:55 PM, P.J. Eby wrote: [a whole bunch of stuff...] Understood and I've observed the public portions of all of this. Much of it has not been pretty. Personality conflicts aside, I wonder if it would be possible to move some portion of the base of both setuptools and Distribute to some sort of shared foundation. That would reduce the amount of duplicate work that I can see coming out of this divergence and, perhaps, provide a basis on top of which setuptools and Distribute can continue to evolve. This kind of "shared utility" project could, perhaps, be undertaken by someone that the two team leaders both know and trust and could become a shared resource instead of having the two teams competing for the extremely valuable (and severely limited) pool of people you would trust to work on this. The thing that has bothered me for years is that, while the code that's being installed is generally of higher quality in the Python world, the tools for doing so in the Ruby world have pulled ahead and now do a better job of installing code that's not as good. Unfortunately, that can give the Ruby solution a visible head start that often times can't be overcome in the minds of management, regardless of the relative quality of the resultant products. Thanks, S From michael at susens-schurter.com Thu Oct 15 22:23:56 2009 From: michael at susens-schurter.com (Michael Schurter) Date: Thu, 15 Oct 2009 13:23:56 -0700 Subject: [Distutils] Proposal for Distribute 0.7 In-Reply-To: References: <94bdd2610910150532x430ba857hf66afa19f8e90c70@mail.gmail.com> <20091015175042.16F173A4068@sparrow.telecommunity.com> <823FAD8D-F2FA-4D0E-9BF8-011AE4516ADC@gmail.com> <20091015195548.DCF8F3A4119@sparrow.telecommunity.com> Message-ID: <240b71640910151323m204e53a3n88b4174680b275c1@mail.gmail.com> On Thu, Oct 15, 2009 at 1:12 PM, ssteinerX at gmail.com wrote: > On Oct 15, 2009, at 3:55 PM, P.J. Eby wrote: > > [a whole bunch of stuff...] > > Understood and I've observed the public portions of all of this. ?Much of it > has not been pretty. > > Personality conflicts aside, I wonder if it would be possible to move some > portion of the base of both setuptools and Distribute to some sort of shared > foundation. distutils in stdlib seems like this foundation. I'm not sure it would be beneficial to inject another layer (and therefore dependency) between distutils and setuptools/distribute, but will defer that judgment to the maintainers involved in the 3 projects (distutils, setuptools, and distribute). I think everyone would prefer only setuptools or distribute to survive in the long term as the current situation is less than ideal from a technical and social standpoint. When one of these projects ultimately deprecates the other, any shared foundation they created would be reduced to needless complexity and cruft. But I'm just a user/observer like yourself, so someone with a bit more authority might want to chime in. :-) HTH, Michael Schurter From ziade.tarek at gmail.com Thu Oct 15 23:06:37 2009 From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=) Date: Thu, 15 Oct 2009 23:06:37 +0200 Subject: [Distutils] Proposal for Distribute 0.7 In-Reply-To: <240b71640910151323m204e53a3n88b4174680b275c1@mail.gmail.com> References: <94bdd2610910150532x430ba857hf66afa19f8e90c70@mail.gmail.com> <20091015175042.16F173A4068@sparrow.telecommunity.com> <823FAD8D-F2FA-4D0E-9BF8-011AE4516ADC@gmail.com> <20091015195548.DCF8F3A4119@sparrow.telecommunity.com> <240b71640910151323m204e53a3n88b4174680b275c1@mail.gmail.com> Message-ID: <94bdd2610910151406u26453c8bk7e416f5b4a04714a@mail.gmail.com> On Thu, Oct 15, 2009 at 10:23 PM, Michael Schurter wrote: > On Thu, Oct 15, 2009 at 1:12 PM, ssteinerX at gmail.com > wrote: >> On Oct 15, 2009, at 3:55 PM, P.J. Eby wrote: >> >> [a whole bunch of stuff...] >> >> Understood and I've observed the public portions of all of this. ?Much of it >> has not been pretty. >> >> Personality conflicts aside, I wonder if it would be possible to move some >> portion of the base of both setuptools and Distribute to some sort of shared >> foundation. > > distutils in stdlib seems like this foundation. ?I'm not sure it would > be beneficial to inject another layer (and therefore dependency) > between distutils and setuptools/distribute, but will defer that > judgment to the maintainers involved in the 3 projects (distutils, > setuptools, and distribute). I'll talk as the Distutils maintainer. I have been working for quite some in cleaning up Distutils code, and its test coverage to make it ready to evolve in 2.7/2.8, 3.2/3.3. This time is coming soon. The plan is to include bits by bits setuptools goodies into Distutils. But only through all the legit PEPs work we are doing (PEP 386/376/390) with Distribute *or* Setuptools being the incubator of those. But as ssteinerX said, this can ony happen if the incubator of Distutils is managed like an open-source community project, and that is currenly impossible in the way Setuptools is managed. Just because this kind of project needs more that one involved person. That's what we are trying to do in Distribute, where there are already quite a few python core commiters and buildout experts, and that's why I have proposed to join forces there. We worked lately on a daily basis and talk at #distutils on freenode about it. Sorry, no one is called "Ian" or "Jim" yet in there. Just 7 or so people that have been working for years with packaging and setuptools. That's pretty much the team which is required for such a project. Now, ssteinerX, what do you mean exactly by shared foundation ? Cheers Tarek From reinout at vanrees.org Fri Oct 16 00:28:52 2009 From: reinout at vanrees.org (Reinout van Rees) Date: Thu, 15 Oct 2009 22:28:52 +0000 (UTC) Subject: [Distutils] distribute 0.6.6 is out Message-ID: Tarek is off to bed, so I'll throw out a quick mail to bring you all the happy tiding that distribute 0.6.6 is out! So get ready for your "easy_install -U distribute". Buildout users can grab http://python-distribute.org/bootstrap.py and get their bin/buildout to use this stable 0.6.6. Works like a charm. (Note: tiny update-time bug in 0.6.5, so that???s why there???s a 0.6.6) Reinout -- Reinout van Rees - reinout at vanrees.org - http://reinout.vanrees.org Software developer at http://www.thehealthagency.com "Military engineers build missiles. Civil engineers build targets" From kiorky at cryptelium.net Fri Oct 16 00:42:25 2009 From: kiorky at cryptelium.net (kiorky) Date: Fri, 16 Oct 2009 00:42:25 +0200 Subject: [Distutils] distribute 0.6.6 is out In-Reply-To: References: Message-ID: <4AD7A551.1040701@cryptelium.net> And virtualenv-distribute was updated at the same time too. Reinout van Rees a ?crit : > Tarek is off to bed, so I'll throw out a quick mail to bring you all the happy > tiding that distribute 0.6.6 is out! So get ready for your > "easy_install -U distribute". > > Buildout users can grab http://python-distribute.org/bootstrap.py and get > their bin/buildout to use this stable 0.6.6. Works like a charm. > > (Note: tiny update-time bug in 0.6.5, so that???s why there???s a 0.6.6) > > > Reinout > -- Cordialement, KiOrKY GPG Key FingerPrint: 0x1A1194B7681112AF -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 261 bytes Desc: OpenPGP digital signature URL: From ssteinerx at gmail.com Fri Oct 16 03:39:01 2009 From: ssteinerx at gmail.com (ssteinerX@gmail.com) Date: Thu, 15 Oct 2009 21:39:01 -0400 Subject: [Distutils] Proposal for Distribute 0.7 In-Reply-To: <94bdd2610910151406u26453c8bk7e416f5b4a04714a@mail.gmail.com> References: <94bdd2610910150532x430ba857hf66afa19f8e90c70@mail.gmail.com> <20091015175042.16F173A4068@sparrow.telecommunity.com> <823FAD8D-F2FA-4D0E-9BF8-011AE4516ADC@gmail.com> <20091015195548.DCF8F3A4119@sparrow.telecommunity.com> <240b71640910151323m204e53a3n88b4174680b275c1@mail.gmail.com> <94bdd2610910151406u26453c8bk7e416f5b4a04714a@mail.gmail.com> Message-ID: <6D114E2B-C560-4264-A25C-980CCA340057@gmail.com> Warning: 3 part message. It's all related so I decided to bundle it rather than starting several new threads on the already over-burdened distutils list. On Oct 15, 2009, at 5:06 PM, Tarek Ziad? wrote: > Now, ssteinerX, what do you mean exactly by shared foundation ? Well, at the moment, setuptools 6.10rc and Distribute 0.6.6 share only one identical directory; tests. (tests in the root directory which only contains shlib_test which I'm not even sure is used by anyone) Many other files are only different by a few lines, some additions to Distribute are just for Python 3 compatibility. Some of the changes are only in the way message text is distributed over a couple of lines (the kind of clean-up people naturally do as they're assimilating a product they didn't write), some extra comments, sometimes the wording (Couldn't vs. Could not) of an error message. Some of the files (e.g. setuptools/archive_util.py) are only different by a bit of whitespace. In taking a closer look, the two products are still very close to each other but it would take a careful person a solid few days or a week to merge them back together into a single code-base, if that would even be desirable. I had thought going forward would be to take some of the utility type functionality that is unlikely to change (archive_util.py for example) and move them to a shared utility library. This is probably unrealistic given the dynamics of the current situation, but it was a nice idea ;-). Thanks, S P.S. While I was in there, I did a little actual work. Not much, but now I feel like I've made a contribution, however small... ;-). Does this make me a "contributor?" This is from the archive pulled down by: # wget http://pypi.python.org/packages/source/d/distribute/distribute-0.6.6.tar.gz While I was pawing around, I noticed that, in distribute-0.6.6/ setuptools/command/alias.py, line 12, the <> operator is used. Since that's not allowed in Python 3.x, the test suite must not cover this 'cause Python 3 would barf. So...I whipped out my command line and: find . -name "*.py" -exec grep "<>" {} \; -print I get: (~/src/distribute-0.6.6/setuptools)# find . -name "*.py" -exec grep "<>" {} \; -print if arg.split()<>[arg]: if self.remove and len(self.args)<>1: ./command/alias.py if safe is None or bool(safe)<>flag: ./command/bdist_egg.py if self.verbose<>self.distribution.verbose: if k<>'target_version': if len(parts)<>2 or not name.endswith('.pth'): ./command/easy_install.py str(version)<>"unknown" and version >= self.requested_version ./depends.py if p<>package and not p.startswith(pfx) if p<>package and not p.startswith(pfx) if p.name<>package and not p.name.startswith(pfx) ./dist.py if cs.hexdigest()<>info[4:]: ./package_index.py Those aren't going to fly in Python 3 and can be safely removed for 2.x as well. A patch would be silly, just go fix it ;-). Thanks, S P.S.S. Part of the problem we'd have with merging the two products to form a single codebase is that there are some differences between the ways the tickets have been closed in the two products. Distribute's fixes are committed against the bugs in its tracker so it's pretty easy to see what change fixed what. Not so much in setuptools where many things are in one big mega-commit. >> The latest commit you've made in setuptools package is still >> cryptic to us because >> it fixes many things at once. > > Sorry about that, but it was pretty much the only way to get it done > in a weekend, without the overheads of separate commit messages, doc > changes, and backporting killing me. Those overheads were the main > reason I wasn't making changes more often; I dreaded the amount of > work involved. > > Also, as it happened, I was able to fix multiple problems with > single changes this way. I don't mean to take another pot-shot a PJE, but the facts is the facts and this shows clearly why the product with the open development process should be the one supported by the community. > Also, I've made it pretty plain for a long time that if Ian Bicking > or Jim Fulton were ever willing to take it over, I'd hand it *all* > over -- 0.7 as well as 0.6 -- and happily retire from the > distribution tools business. I'd love to see this be the last release of setuptools so if you're the aforementioned "Ian" or "Jim", could you please, please, very temporarily, accept the burden of "setuptools maintainer" so it can die an elegant, compatible, controlled, peaceful death, being merged into and supplanted by Distribute after long and good service to the Python world? We really need a good, modern, open, constantly evolving packaging system to keep Python competitive. Thanks, S From david.lyon at preisshare.net Fri Oct 16 04:36:41 2009 From: david.lyon at preisshare.net (david.lyon at preisshare.net) Date: Fri, 16 Oct 2009 12:36:41 +1000 (EST) Subject: [Distutils] Proposal for Distribute 0.7 In-Reply-To: <20091015195548.DCF8F3A4119@sparrow.telecommunity.com> References: <94bdd2610910150532x430ba857hf66afa19f8e90c70@mail.gmail.com> <20091015175042.16F173A4068@sparrow.telecommunity.com> <823FAD8D-F2FA-4D0E-9BF8-011AE4516ADC@gmail.com> <20091015195548.DCF8F3A4119@sparrow.telecommunity.com> Message-ID: <35798.203.171.195.248.1255660601.squirrel@syd-srv02.ezyreg.com> I really love this stuff.. It's better than soap opera.. We have jerks, interjectors, behind the scenes plots, secret messages, moles and flashbacks.. I'm staying tuned in and hope tomorow's episode will be just as thrilling as todays.. > At 02:38 PM 10/15/2009 -0400, ssteinerX at gmail.com wrote: >>> that there are other people on the Distribute team who I'd >>>seriously consider as committers on setuptools or even as a chief >>>maintainer of the setuptools 0.6 line (if not more). >>>...asked me who those team members are >> >>I'm asking. Who are they and, if you are willing to give them access, >>have you offered it and they've declined, or you were waiting >>until...what? You had Tarek's acknowledgement or permission? > > I am not trying to poach anyone or stir up trouble, which is why I've > taken a very passive public stance on the issue -- although it has > included me naming names in certain venues, including here. > > However, I *am* in private contact with more than one member of the > Distribute team, each of whom first contacted me. > > In order to avoid creating any further drama here or in the > Distribute team, I will leave it up to them to make any public > statements, when/if they choose to confer with their colleagues on > the matter. (I would prefer, of course, a joint statement at the > appropriate time.) > > I wish that such complications weren't necessary. If cooler heads > had prevailed in July, this could have and likely would have been > resolved back then. > > >>We, regular Python users, have been asking for you to let someone help >>with setuptools for years since you obviously have other priorities >>and the various issues in setuptools have affected many of us in >>various ways. > > The only reason I've done an 0.6c10 is because of a policy-breaking > change to Python that breaks setuptools users, and that can't be > worked around with configuration, command-line options, or other > tweaking of the runtime environment -- as was able to be done with > approximately 9 out of 10 of the setuptools bugs in the tracker. > > If it weren't for that, I'd have been more-or-less happy to let > Distribute become the quasi-official replacement for setuptools 0.6, > despite my annoyance at some of the public comments made by its > promoter(s) prior to the 2.6.3 issue. > > Indeed, as previously stated, I even tried to arrange a handoff of > 0.6, wherein the changes made in Distribute would've been released as > 0.6c10 or 0.6final back in July... and public talks broke down due > to certain persons' flaming and posturing. > > That's one reason I'm only doing off-list talks now -- less chance of > random jerks inserting themselves into the middle of the discussion, > making me think they're part of Distribute and that their comments > reflect Distribute policy (as also happened back in July). Also, no > need for persons on either side to put up any posture or spin, when > there's no audience to play to. > > >>Are you really willing to let anyone help? Really? > > Indeed. There's a list, and some of them obviously have time to work > on Distribute, so it's not a matter of me only picking people who > don't have any time, as Tarek and others have more than once accused me > of. > > >>Do it, then, > > Discussions are ongoing. > > Also, I've made it pretty plain for a long time that if Ian Bicking > or Jim Fulton were ever willing to take it over, I'd hand it *all* > over -- 0.7 as well as 0.6 -- and happily retire from the > distribution tools business. I trust their vision as architects, and > their track record of supporting their users IMO speaks for > itself. Both are past contributors of non-trivial features to > setuptools, as well as accomplished installation tool developers in > their own right. It would be a joy and an honor to turn the keys > over to either one of them, either as an individual or as the leader > of a new team. (I would love to see some of pip and buildout's > features integrated in setuptools 0.7, for example.) > > (Ian, btw, already has PyPI maintainer rights to setuptools, and Jim > already has commit rights to setuptools SVN... giving further lie to > the idea that I'm not willing to give people access or let them help.) > > That having been said, there are definitely other people I'd give > varying degrees of access to, just not at the "here, take it, please, > I want you to have it!" level I would grant to Ian or Jim. ;-) > > _______________________________________________ > Distutils-SIG maillist - Distutils-SIG at python.org > http://mail.python.org/mailman/listinfo/distutils-sig > From david at ar.media.kyoto-u.ac.jp Fri Oct 16 05:22:20 2009 From: david at ar.media.kyoto-u.ac.jp (David Cournapeau) Date: Fri, 16 Oct 2009 12:22:20 +0900 Subject: [Distutils] Alternate static metadata PEP submission... In-Reply-To: <7601812317e8e6525679f787bdd3aad8@preisshare.net> References: <20091013103021.GA21066@laurie.devork> <94bdd2610910131001v70711ffeia27433cd23b32fca@mail.gmail.com> <79a53e276e5dffe24be0433771801d72@preisshare.net> <94bdd2610910131426k64f707d7pf4d398d04b8ecc32@mail.gmail.com> <20091013233654.GA27339@laurie.devork> <4AD54138.9010602@ar.media.kyoto-u.ac.jp> <50a1116c3a8ca9bcbb28121d90ccc5d0@preisshare.net> <4AD5BA54.4000600@ar.media.kyoto-u.ac.jp> <4520a61448e00a641236151aee632aa8@preisshare.net> <4AD68214.70407@ar.media.kyoto-u.ac.jp> <4AD69420.8050804@ar.media.kyoto-u.ac.jp> <1e5ca572044e2d50395bed41f1a4a67c@preisshare.net> <4AD6A784.3050607@ar.media.kyoto-u.ac.jp> <4AD6B790.4040208@ar.media.kyoto-u.ac.jp> <7601812317e8e6525679f787bdd3aad8@preisshare.net> Message-ID: <4AD7E6EC.6030908@ar.media.kyoto-u.ac.jp> David Lyon wrote: > On Thu, 15 Oct 2009 14:48:00 +0900, David Cournapeau > wrote: > >> Let's take a common case: using cython/pyrex to accelerate some code, >> but you still want people to be able to use your package without a C >> compiler (bzr, cython are examples of such packages). >> > > I believe that's possible through the [postinstall] section.. > The problem is a bit more complicated, because it cannot be handled in one single section. You need to detect cython (configuration stage), you need to modify the extensions sources depending on this, and you need to install different files depending on this: it touches at least config, build_ext and install commands to use the distutils terminology. This is similar to my main complain about distutils: you can't easily share data between commands. AFAIK, the only way to do it is to use the Distribute class instance, which is problematic as that's what most tools on top of distutils need to monkey-patch. David From david.lyon at preisshare.net Fri Oct 16 06:35:07 2009 From: david.lyon at preisshare.net (David Lyon) Date: Fri, 16 Oct 2009 00:35:07 -0400 Subject: [Distutils] Alternate static metadata PEP submission... In-Reply-To: <4AD7E6EC.6030908@ar.media.kyoto-u.ac.jp> References: <20091013103021.GA21066@laurie.devork> <94bdd2610910131001v70711ffeia27433cd23b32fca@mail.gmail.com> <79a53e276e5dffe24be0433771801d72@preisshare.net> <94bdd2610910131426k64f707d7pf4d398d04b8ecc32@mail.gmail.com> <20091013233654.GA27339@laurie.devork> <4AD54138.9010602@ar.media.kyoto-u.ac.jp> <50a1116c3a8ca9bcbb28121d90ccc5d0@preisshare.net> <4AD5BA54.4000600@ar.media.kyoto-u.ac.jp> <4520a61448e00a641236151aee632aa8@preisshare.net> <4AD68214.70407@ar.media.kyoto-u.ac.jp> <4AD69420.8050804@ar.media.kyoto-u.ac.jp> <1e5ca572044e2d50395bed41f1a4a67c@preisshare.net> <4AD6A784.3050607@ar.media.kyoto-u.ac.jp> <4AD6B790.4040208@ar.media.kyoto-u.ac.jp> <7601812317e8e6525679f787bdd3aad8@preisshare.net> <4AD7E6EC.6030908@ar.media.kyoto-u.ac.jp> Message-ID: On Fri, 16 Oct 2009 12:22:20 +0900, David Cournapeau wrote: > The problem is a bit more complicated, because it cannot be handled in > one single section. I really appreciate you sharing this challenge with me. I'm really convinced that I can "tweak" something to make it work. And now is a really good time for me to examine it because I'm coding now. Let me post how the code is looking: def main(): # -- Return Code rc = 0 # -- Check for our config file if not os.path.exists(setupconfigname): logger.error("No %s file found. Installation is unable to proceed" % setupconfigname) sys.exit(10) try: # -- See what our platform is build_platform_bit_map() # -- Work out where to copy files to build_local_directory_targets() # -- Load basic install information load_basic_install_requirements() # -- Process python package dependencies process_dependencies() # -- Build a list of all the files that need to be copied build_file_copy_list() # -- Check that all the files in the copy list exist and are ready for copying validate_file_list() # -- Run preinstall scripts run_preinstall_scripts() # -- Copy files to the local system copy_files() # -- Adjust the python configuration finalise_python() # -- Update platform links/shortcuts finalise_local_links() # -- Run postinstall scripts run_postinstall_scripts() except: rollback(): rc = 1 return rc So let's just take what you wrote step by step.. > You need to detect cython (configuration stage) cython is a type of python interpretor? like jython or ironpython Assuming yes, > you need to modify the extensions sources depending on this, and you need to > install different files depending on this: [Application] scripts = artisticflairgui.py [Application cython] scripts = artisticflairguicython.py That's a different file being installed if we detect we are using cython and not python. Since we are copying files "out", like we used to do, we can include all the files that we need in our package. Only the files needed for the local platform end up getting copied. > it touches at least config, > build_ext and install commands to use the distutils terminology. [postinstall] scripts = buildit.py [postinstall cython] scripts = builtitspecialforcython.py [postinstall jython] scripts = builtitspecialforjython.py [postinstall ironpython] scripts = killvistaupgradetowin7.py > This is similar to my main complain about distutils: you can't easily > share data between commands. AFAIK, the only way to do it is to use the > Distribute class instance, which is problematic as that's what most > tools on top of distutils need to monkey-patch. It seems to be a really unusual (for me) problem. But I don't dispute that its a real problem. I don't follow all the python spin offs but I agree that they need to be. Hence, they need bits (constants). I just hope I'm understanding the problem the right way. Maybe next week I might even have something that can be tried to see if it will work. I'm busy coding now, so you asked at a good time. David From david at ar.media.kyoto-u.ac.jp Fri Oct 16 06:58:46 2009 From: david at ar.media.kyoto-u.ac.jp (David Cournapeau) Date: Fri, 16 Oct 2009 13:58:46 +0900 Subject: [Distutils] Alternate static metadata PEP submission... In-Reply-To: References: <94bdd2610910131001v70711ffeia27433cd23b32fca@mail.gmail.com> <79a53e276e5dffe24be0433771801d72@preisshare.net> <94bdd2610910131426k64f707d7pf4d398d04b8ecc32@mail.gmail.com> <20091013233654.GA27339@laurie.devork> <4AD54138.9010602@ar.media.kyoto-u.ac.jp> <50a1116c3a8ca9bcbb28121d90ccc5d0@preisshare.net> <4AD5BA54.4000600@ar.media.kyoto-u.ac.jp> <4520a61448e00a641236151aee632aa8@preisshare.net> <4AD68214.70407@ar.media.kyoto-u.ac.jp> <4AD69420.8050804@ar.media.kyoto-u.ac.jp> <1e5ca572044e2d50395bed41f1a4a67c@preisshare.net> <4AD6A784.3050607@ar.media.kyoto-u.ac.jp> <4AD6B790.4040208@ar.media.kyoto-u.ac.jp> <7601812317e8e6525679f787bdd3aad8@preisshare.net> <4AD7E6EC.6030908@ar.media.kyoto-u.ac.jp> Message-ID: <4AD7FD86.3040509@ar.media.kyoto-u.ac.jp> David Lyon wrote: >> You need to detect cython (configuration stage) >> > > cython is a type of python interpretor? like jython or ironpython > Cython is a language which is more or less a subset of python, with optional typing, and cython is the name of the Cython -> C compiler. It is one of the tool of choice to speed up python code in the python scientific community. > [Application] > scripts = artisticflairgui.py > > [Application cython] > scripts = artisticflairguicython.py > > That's a different file being installed if we detect we are using cython > and not python. > The usual pattern is: code = ['artisticflairgui.py'] if has_cython: code.append(Extension('_foo', sources=['artisticflairgui.pyx'])) You need commonalities between 'subsections' (which one another problem of the current usage of .ini-like format). >> This is similar to my main complain about distutils: you can't easily >> share data between commands. AFAIK, the only way to do it is to use the >> Distribute class instance, which is problematic as that's what most >> tools on top of distutils need to monkey-patch. >> > > It seems to be a really unusual (for me) problem. > It is a unusual problem if you are a user of distutils - it is a major problem if you want to extend distutils. Most if not all implementation problems of setuptools/paver and the like are rooted to this. For example, to build eggs, setuptools needs some info between its different egg commands, and the way to do it is to create its own Distribution subclass. Now, when we in numpy extends the Distribution class by ourselves, we have to take into account whether setuptools is used or not, and track the changes. Distribute fork only makes matter worse. That's why when people complain about setuptools hacks, they should give its authors some slack: the problem is rooted in distutils, and there is simply no way to fix this, short of rewriting distutils from scratch or breaking it so badly that there is no point reusing most of its current code. I see the use of metadata files as we are discussing here as the way forward for backward compatibility: a distutils command would convert a setup.py-based to the metadata, and the new tool would takes this. The metadata are the compat layer, not the distutils implementation anymore. In that perspective, the current .ini-based proposal (PEP 390 and yours) seems too weak to support a significant subset of existing code. David From ziade.tarek at gmail.com Fri Oct 16 10:24:36 2009 From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=) Date: Fri, 16 Oct 2009 10:24:36 +0200 Subject: [Distutils] Proposal for Distribute 0.7 In-Reply-To: <6D114E2B-C560-4264-A25C-980CCA340057@gmail.com> References: <94bdd2610910150532x430ba857hf66afa19f8e90c70@mail.gmail.com> <20091015175042.16F173A4068@sparrow.telecommunity.com> <823FAD8D-F2FA-4D0E-9BF8-011AE4516ADC@gmail.com> <20091015195548.DCF8F3A4119@sparrow.telecommunity.com> <240b71640910151323m204e53a3n88b4174680b275c1@mail.gmail.com> <94bdd2610910151406u26453c8bk7e416f5b4a04714a@mail.gmail.com> <6D114E2B-C560-4264-A25C-980CCA340057@gmail.com> Message-ID: <94bdd2610910160124j28eb7703ma41273b10d2a3f5d@mail.gmail.com> On Fri, Oct 16, 2009 at 3:39 AM, ssteinerX at gmail.com wrote: > Some of the files (e.g. setuptools/archive_util.py) are only different by a > bit of whitespace. archive_util, in the mid term (2.7/3.2) will dissapear, in favor of changes in Distutils side. > > In taking a closer look, the two products are still very close to each other > but it would take a careful person a solid few days or a week to merge them > back together into a single code-base, if that would even be desirable. We had a "setuptools-compatible" branch for a while when PJE said he might use our work to release setuptools, but that didn't happened and we moved on. [please no flame here,it's just a fact : it didn't happen. period] At this point, the code base is backward compatible, but is slightly changing yes (we are working in bug fixes so..) > > P.S. > > While I was in there, I did a little actual work. ?Not much, but now I feel > like I've made a contribution, however small... ;-). ? Does this make me a > "contributor?" Yes absolutely. Then if you keep them coming and you will be able the have a contributor access. btw, anyone that has a history in managing a known project for years in the community can get a commit access instantly because we want this project to be open and to be run by the people that are the ones doing packages every day. They know what they are doing. As, for the quality of the code, the current quality is very low in many ways, so, for obvious bug patches, we trust anyone that is already running a project succesfully, and we are counting on the fact that many eyes are now watching the changesets being done. Remember that 0.6.x is bugfix only. The only hard part in setuptools is understanding what the code is doing because it's hard to read, overdesigned and overcomplex and with giant monolithic modules. > > This is from the archive pulled down by: > > ? ? ? ?# wget > http://pypi.python.org/packages/source/d/distribute/distribute-0.6.6.tar.gz > > While I was pawing around, I noticed that, in > distribute-0.6.6/setuptools/command/alias.py, line 12, the <> operator is > used. > > Since that's not allowed in Python 3.x, the test suite must not cover this > 'cause Python 3 would barf. > > So...I whipped out my command line and: > > ? ? ? ?find . -name "*.py" -exec grep "<>" {} \; -print > > I get: > > (~/src/distribute-0.6.6/setuptools)# find . -name "*.py" -exec grep "<>" {} > \; -print > ? ?if arg.split()<>[arg]: > ? ? ? ?if self.remove and len(self.args)<>1: > ./command/alias.py > ? ? ? ? ? ?if safe is None or bool(safe)<>flag: > ./command/bdist_egg.py > ? ? ? ?if self.verbose<>self.distribution.verbose: > ? ? ? ? ? ? ? ?if k<>'target_version': > ? ? ? ? ? ?if len(parts)<>2 or not name.endswith('.pth'): > ./command/easy_install.py > ? ? ? ? ? ?str(version)<>"unknown" and version >= self.requested_version > ./depends.py > ? ? ? ? ? ? ? ? ? ?if p<>package and not p.startswith(pfx) > ? ? ? ? ? ? ? ? ? ?if p<>package and not p.startswith(pfx) > ? ? ? ? ? ? ? ? ? ?if p.name<>package and not p.name.startswith(pfx) > ./dist.py > ? ? ? ? ? ?if cs.hexdigest()<>info[4:]: > ./package_index.py > > Those aren't going to fly in Python 3 and can be safely removed for 2.x as > well. ?A patch would be silly, just go fix it ;-). Thanks ! we will fix it. The code base is not well covered in the tests, so this problem was not tackled. Notice that the latest released, if run under Python 3, will convert in the fly the code, using 2to3, so make sure the tests are run after 2to3 (that's what is supposed to happen) If you find anything else, please come in #distutils (freenode) Cheers Tarek From ziade.tarek at gmail.com Fri Oct 16 11:53:31 2009 From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=) Date: Fri, 16 Oct 2009 11:53:31 +0200 Subject: [Distutils] Distribute 0.7.x first sprint Message-ID: <94bdd2610910160253k492cf739nc1e9d6dfc40e657e@mail.gmail.com> Hey While we are working at fixing bugs in 0.6.x, I would like to organize a first online coding sprint for the 0.7.x series, goals: - finalize the first splitting version - define a more detailed roadmap for each splitted package - build a better test environment (see if we can set up a buildbot) possible times: - Sunday October 18th - 6 pm. GMT+2 - Tuesday October 20th - 7 pm. GMT+2 - Wenesday October 21th - 7 pm. GMT+2 - Friday October 23th - 7 pm. GMT+2 If you want to join, add your name here: http://doodle.com/bfgv3yi3pi48buuv Thanks -- Tarek Ziad? | http://ziade.org |????????????! |???????????? From ndbecker2 at gmail.com Fri Oct 16 13:20:14 2009 From: ndbecker2 at gmail.com (Neal Becker) Date: Fri, 16 Oct 2009 07:20:14 -0400 Subject: [Distutils] small distribute improvement Message-ID: I think it is confusing that easy_install blah doesn't do anything, and doesn't provide any clue, if it was to do an update. You need to say easy_install -U blah There are a few choices here. One might be that easy_install blah could notify you that an update is available. Personally, my preference would be to just do the update. Or, at least offer to if run interactively. From regebro at gmail.com Fri Oct 16 14:12:13 2009 From: regebro at gmail.com (Lennart Regebro) Date: Fri, 16 Oct 2009 14:12:13 +0200 Subject: [Distutils] small distribute improvement In-Reply-To: References: Message-ID: <319e029f0910160512p65ac4dbta87aa2f125d14f09@mail.gmail.com> 2009/10/16 Neal Becker : > I think it is confusing that > easy_install blah > > doesn't do anything, and doesn't provide any clue, if it was to do an > update. "Was to do"? It doesn't know that there is an update available, unless you ask it to check. It does tell you that it's already installed. > There are a few choices here. ?One might be that > easy_install blah > could notify you that an update is available. That means that anytime you call easy_install it goes out on the net and asks, even if the product already is installed. It's a possibility, but I'm not convinced it's a good one. > Personally, my preference would be to just do the update. That, I'm convinced is a bad idea. There may be a good reason for why the last version is not installed, such as something else you have installed requiring that version. >?Or, at least offer to if run interactively. What I can buy is an option to check, and ask if and upgrade should be done. -U will install the latest without asking, that may not be what you want. -- Lennart Regebro: Python, Zope, Plone, Grok http://regebro.wordpress.com/ +33 661 58 14 64 From ziade.tarek at gmail.com Fri Oct 16 15:54:29 2009 From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=) Date: Fri, 16 Oct 2009 15:54:29 +0200 Subject: [Distutils] [RFC] Recentering the static metadata need : PKG-INFO Message-ID: <94bdd2610910160654n351d786ap4366b6e9e99539dc@mail.gmail.com> Hey, Marc-Andr? Lemburg gave me a key points about the static metadata discussions we have (wether its PEP 390 or around it) He said that the important thing was to have the context-dependant markers in PKG-INFO, and that having in described in setup.cfg or in setup.py by any way is not the important thing. And he is right ! So, PEP 390 proposes a new setup.cfg format, that's fine. And others are thinking about other ways to build metadata, that's great. But what really matters at the end is to provide a new Metadata format where these marker are present, so any consumer can play with them (and Distutils provide an API to play with them : the DistributionMetadata class) So please, have a look at the new section I've added here with the syntax I am proposing : http://www.python.org/dev/peps/pep-0390/#impact-on-pkg-info-generation-and-pep-314 That will probably re-center the debate in creating a Metadata 1.2 format that includes these markers. As someone asked, here is the principal uses cases of having static metadata that includes markers like this: - pip, or easy_install, or any client code, will be able to query PyPI, to get the metadata of a package. *and* will be able to have it depending on a particular environment. - it will be possible to generate and provide alongside the source code the PKG-INFO file, like how install_egg_info does at installation time, exactly like how MANIFEST is built. If that sounds like the right path (it does to me), we could start to work on a PEP 314 update, and add in the work what was started last year: - revision of "requires", "obsoletes", etc.. - link to PEP 386 Regards Tarek From ssteinerx at gmail.com Fri Oct 16 16:46:49 2009 From: ssteinerx at gmail.com (ssteinerX@gmail.com) Date: Fri, 16 Oct 2009 10:46:49 -0400 Subject: [Distutils] Proposal for Distribute 0.7 In-Reply-To: <94bdd2610910160124j28eb7703ma41273b10d2a3f5d@mail.gmail.com> References: <94bdd2610910150532x430ba857hf66afa19f8e90c70@mail.gmail.com> <20091015175042.16F173A4068@sparrow.telecommunity.com> <823FAD8D-F2FA-4D0E-9BF8-011AE4516ADC@gmail.com> <20091015195548.DCF8F3A4119@sparrow.telecommunity.com> <240b71640910151323m204e53a3n88b4174680b275c1@mail.gmail.com> <94bdd2610910151406u26453c8bk7e416f5b4a04714a@mail.gmail.com> <6D114E2B-C560-4264-A25C-980CCA340057@gmail.com> <94bdd2610910160124j28eb7703ma41273b10d2a3f5d@mail.gmail.com> Message-ID: <3E2051F7-E2CB-4B17-95FE-CAF64D02B8EF@gmail.com> On Oct 16, 2009, at 4:24 AM, Tarek Ziad? wrote: > On Fri, Oct 16, 2009 at 3:39 AM, ssteinerX at gmail.com > wrote: >> Some of the files (e.g. setuptools/archive_util.py) are only >> different by a >> bit of whitespace. > > archive_util, in the mid term (2.7/3.2) will dissapear, in favor of > changes in Distutils side. Ok, it was just a datapoint, not a suggestion that it be retained. >> In taking a closer look, the two products are still very close to >> each other >> but it would take a careful person a solid few days or a week to >> merge them >> back together into a single code-base, if that would even be >> desirable. > > We had a "setuptools-compatible" branch for a while when PJE said he > might use our work to release setuptools, but that didn't happened > and we moved on. > [please no flame here,it's just a fact : it didn't happen. period] > > At this point, the code base is backward compatible, but is slightly > changing yes (we are working in bug fixes so..) Ok, after reviewing the code, I can see that maintaining "common code" would not actually be a good idea because it's not worth retaining. >> While I was in there, I did a little actual work. Not much, but >> now I feel >> like I've made a contribution, however small... ;-). Does this >> make me a >> "contributor?" > > Yes absolutely. Then if you keep them coming and you will be able the > have a contributor access. Thanks. At the moment, I, frankly, wouldn't touch that code with a 10 foot pole. I had never actually sat down to do anything resembling a "code review" on the setuptools code and, after doing so, I really think it should just be scrapped as quickly as possible. Dragging any of it along, for whatever reason, is just going to slow progress. The test coverage is pretty much non-existent and, without that, trying to figure out the intent of this code is pointlessly difficult. It'd just be easier to write the tests, write the code, and move on. > As, for the quality of the code, the current quality is very low in > many ways, so, for obvious bug patches, we trust anyone that is > already running a project succesfully, and we are counting on the fact > that many eyes are now watching the changesets being done. > > Remember that 0.6.x is bugfix only. Understood. > The only hard part in setuptools is understanding what the code is > doing because it's hard to read, overdesigned and overcomplex and with > giant monolithic modules. I don't know that I'd call what I saw "designed." Complex, yes, but design implies a plan and known direction, communicated by design documents with tests that prove the code is working as expected. This is not designed by any objective standard I'd use. Maybe PJE is some sort of savant who can keep the myriad details of a piece of code like this in his head but "design" is not that. >> This is from the archive pulled down by: >> >> # wget >> http://pypi.python.org/packages/source/d/distribute/distribute-0.6.6.tar.gz >> >> While I was pawing around, I noticed that, in >> distribute-0.6.6/setuptools/command/alias.py, line 12, the <> >> operator is >> used. >> >> Those aren't going to fly in Python 3 and can be safely removed for >> 2.x as >> well. A patch would be silly, just go fix it ;-). > > Thanks ! we will fix it. The code base is not well covered in the > tests, so this problem was not tackled. Notice that the latest > released, if run under Python > 3, will convert in the fly the code, using 2to3, so make sure the > tests are run after 2to3 (that's > what is supposed to happen) Yes, I'm sure 2to3 can fix it, but the less work it has to do, the fewer chances for mistakes in the automatic conversion. Not that I'd expect this to break something but having the 2to3 log less cluttered with simple things like this that can be easily done by hand cuts down on the amount of stuff that has to be reviewed in case of a problem. > If you find anything else, please come in #distutils (freenode) Will do... S From tseaver at palladion.com Fri Oct 16 16:46:23 2009 From: tseaver at palladion.com (Tres Seaver) Date: Fri, 16 Oct 2009 10:46:23 -0400 Subject: [Distutils] [RFC] Recentering the static metadata need : PKG-INFO In-Reply-To: <94bdd2610910160654n351d786ap4366b6e9e99539dc@mail.gmail.com> References: <94bdd2610910160654n351d786ap4366b6e9e99539dc@mail.gmail.com> Message-ID: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Tarek Ziad? wrote: > Hey, > > Marc-Andr? Lemburg gave me a key points about the static metadata > discussions we have (wether its PEP 390 or around it) > > He said that the important thing was to have the context-dependant > markers in PKG-INFO, and that having in described in > setup.cfg or in setup.py by any way is not the important thing. And he > is right ! > > So, PEP 390 proposes a new setup.cfg format, that's fine. And others > are thinking about other ways to build metadata, that's great. > > But what really matters at the end is to provide a new Metadata format > where these marker are present, so any consumer > can play with them (and Distutils provide an API to play with them : > the DistributionMetadata class) > > So please, have a look at the new section I've added here with the > syntax I am proposing : > > http://www.python.org/dev/peps/pep-0390/#impact-on-pkg-info-generation-and-pep-314 > > That will probably re-center the debate in creating a Metadata 1.2 > format that includes these markers. > > As someone asked, here is the principal uses cases of having static > metadata that includes markers like this: > > - pip, or easy_install, or any client code, will be able to query > PyPI, to get the metadata of a package. *and* > will be able to have it depending on a particular environment. > > - it will be possible to generate and provide alongside the source > code the PKG-INFO file, like how install_egg_info does > at installation time, exactly like how MANIFEST is built. > > If that sounds like the right path (it does to me), we could start to > work on a PEP 314 update, and add in the > work what was started last year: > > - revision of "requires", "obsoletes", etc.. > - link to PEP 386 Shouldn't this point at PEP 345, which already defines a metadata version 1.2? Tres. - -- =================================================================== Tres Seaver +1 540-429-0999 tseaver at palladion.com Palladion Software "Excellence by Design" http://palladion.com -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iEYEARECAAYFAkrYhz8ACgkQ+gerLs4ltQ7HxACeP6BVcIIze+WKjjCe/1ED9ko8 oYIAnAu9KC/10+4u8rJ8S9bUr1V7MbLj =qsDv -----END PGP SIGNATURE----- From ziade.tarek at gmail.com Fri Oct 16 16:50:21 2009 From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=) Date: Fri, 16 Oct 2009 16:50:21 +0200 Subject: [Distutils] [RFC] Recentering the static metadata need : PKG-INFO In-Reply-To: References: <94bdd2610910160654n351d786ap4366b6e9e99539dc@mail.gmail.com> Message-ID: <94bdd2610910160750r6a025abmced07890c5eba7bd@mail.gmail.com> On Fri, Oct 16, 2009 at 4:46 PM, Tres Seaver wrote: > Shouldn't this point at PEP 345, which already defines a metadata > version 1.2? Right sorry, I messed up with the PEPs numbers. PEP 314 is the final 1.1, and PEP 345 is the draft for 1.2, where we should work to change PKG-INFO. Tarek From pje at telecommunity.com Fri Oct 16 17:41:45 2009 From: pje at telecommunity.com (P.J. Eby) Date: Fri, 16 Oct 2009 11:41:45 -0400 Subject: [Distutils] Proposal for Distribute 0.7 In-Reply-To: <3E2051F7-E2CB-4B17-95FE-CAF64D02B8EF@gmail.com> References: <94bdd2610910150532x430ba857hf66afa19f8e90c70@mail.gmail.com> <20091015175042.16F173A4068@sparrow.telecommunity.com> <823FAD8D-F2FA-4D0E-9BF8-011AE4516ADC@gmail.com> <20091015195548.DCF8F3A4119@sparrow.telecommunity.com> <240b71640910151323m204e53a3n88b4174680b275c1@mail.gmail.com> <94bdd2610910151406u26453c8bk7e416f5b4a04714a@mail.gmail.com> <6D114E2B-C560-4264-A25C-980CCA340057@gmail.com> <94bdd2610910160124j28eb7703ma41273b10d2a3f5d@mail.gmail.com> <3E2051F7-E2CB-4B17-95FE-CAF64D02B8EF@gmail.com> Message-ID: <20091016154159.142273A4068@sparrow.telecommunity.com> At 10:46 AM 10/16/2009 -0400, ssteinerX at gmail.com wrote: >I don't know that I'd call what I saw "designed." Complex, yes, but >design implies a plan and known direction, communicated by design >documents with tests that prove the code is working as expected. > >This is not designed by any objective standard I'd use. You are correct; setuptools itself was not particularly designed. Eggs are, entry points are, a few other odds and ends were designed, but setuptools itself (and *especially* easy_install) are a collection of hacks upon hacks, mostly done in a rush to get something out the door... which as soon as it became popular, locked into a cycle of not having enough time to do improvements, because maintenance was taking so much time. >Maybe PJE is some sort of savant who can keep the myriad details of a >piece of code like this in his head Nope. Why do you think I've not been thrilled about working on it? ;-) From ssteinerx at gmail.com Fri Oct 16 18:05:30 2009 From: ssteinerx at gmail.com (ssteinerX@gmail.com) Date: Fri, 16 Oct 2009 12:05:30 -0400 Subject: [Distutils] Proposal for Distribute 0.7 In-Reply-To: <20091016154159.142273A4068@sparrow.telecommunity.com> References: <94bdd2610910150532x430ba857hf66afa19f8e90c70@mail.gmail.com> <20091015175042.16F173A4068@sparrow.telecommunity.com> <823FAD8D-F2FA-4D0E-9BF8-011AE4516ADC@gmail.com> <20091015195548.DCF8F3A4119@sparrow.telecommunity.com> <240b71640910151323m204e53a3n88b4174680b275c1@mail.gmail.com> <94bdd2610910151406u26453c8bk7e416f5b4a04714a@mail.gmail.com> <6D114E2B-C560-4264-A25C-980CCA340057@gmail.com> <94bdd2610910160124j28eb7703ma41273b10d2a3f5d@mail.gmail.com> <3E2051F7-E2CB-4B17-95FE-CAF64D02B8EF@gmail.com> <20091016154159.142273A4068@sparrow.telecommunity.com> Message-ID: <2E894DD7-CB1D-4F08-9E20-17D044534E1E@gmail.com> On Oct 16, 2009, at 11:41 AM, P.J. Eby wrote: > At 10:46 AM 10/16/2009 -0400, ssteinerX at gmail.com wrote: >> I don't know that I'd call what I saw "designed." Complex, yes, but >> design implies a plan and known direction, communicated by design >> documents with tests that prove the code is working as expected. >> >> This is not designed by any objective standard I'd use. > > You are correct; setuptools itself was not particularly designed. > Eggs are, entry points are, a few other odds and ends were designed, > but setuptools itself (and *especially* easy_install) are a > collection of hacks upon hacks, mostly done in a rush to get > something out the door... which as soon as it became popular, > locked into a cycle of not having enough time to do improvements, > because maintenance was taking so much time. Yup. Understood. It's a vicious circle/cycle that I'm sure we've all been trapped in at one time or another; just not quite so publicly or with such a large impact on so many people. And to do it for free, besides! Betcha' you'll think thrice next time...and say nah, no thanks... >> Maybe PJE is some sort of savant who can keep the myriad details of a >> piece of code like this in his head > > Nope. Why do you think I've not been thrilled about working on > it? ;-) Welp...here's to hoping you're freed from that burden partially of your own making if not choosing... S From nad at acm.org Fri Oct 16 18:11:14 2009 From: nad at acm.org (Ned Deily) Date: Fri, 16 Oct 2009 09:11:14 -0700 Subject: [Distutils] small distribute improvement References: Message-ID: In article , Neal Becker wrote: > I think it is confusing that > easy_install blah > > doesn't do anything, and doesn't provide any clue, if it was to do an > update. You need to say > easy_install -U blah > > There are a few choices here. One might be that > easy_install blah > could notify you that an update is available. > > Personally, my preference would be to just do the update. Or, at least > offer to if run interactively. $ more ~/.pydistutils.cfg [easy_install] upgrade = 1 zip-ok = 0 -- Ned Deily, nad at acm.org From pje at telecommunity.com Fri Oct 16 18:27:33 2009 From: pje at telecommunity.com (P.J. Eby) Date: Fri, 16 Oct 2009 12:27:33 -0400 Subject: [Distutils] setuptools 0.6c10 delayed until Monday Message-ID: <20091016162748.7CA123A4068@sparrow.telecommunity.com> After giving it some thought, I've decided to hold off on pushing out the 0.6c10 release until Monday morning, since it's likely that some people have scripts or build processes that run off of ez_setup SVN or the latest PyPI version of setuptools... and who aren't aware the release is coming and might be surprised by it. It would therefore probably not be a good idea to push this out on a Friday. Also, this will give those who are using Distribute and do *not* wish to use this release, a chance to upgrade to the latest version of Distribute first. (Which, if I understand correctly, includes code to prevent accidental re-installation of setuptools.) From ssteinerx at gmail.com Fri Oct 16 18:43:58 2009 From: ssteinerx at gmail.com (ssteinerX@gmail.com) Date: Fri, 16 Oct 2009 12:43:58 -0400 Subject: [Distutils] [venv] Re: two questions about virtualenv In-Reply-To: <87iqefl30b.fsf@rhodesmill.org> References: <87ljjbmsij.fsf@rhodesmill.org> <73D01E3C-A349-462A-BF2C-465F5A572AE3@leidel.info> <87r5t3l4je.fsf@rhodesmill.org> <715CB4F3-F653-49C1-B936-46466BF3C2F5@leidel.info> <87iqefl30b.fsf@rhodesmill.org> Message-ID: <5DFFC162-0687-45FB-B219-9F05DF3AFB60@gmail.com> On Oct 16, 2009, at 12:25 PM, Brandon Craig Rhodes wrote: > Jannis Leidel writes: > >> That'd be very much appreciated, I'm sure the experience of the >> distribute people would be helpful, too. > > I'm not sure I quite approve of their approach, since it makes > installing Distribute under Python 3 take a very long time - which, > as a > result, currently makes creating a virtualenv under Python 3 take a > very > long time as well. Someone was complaining to me about that just > earlier this week. Doesn't that also mean that the Python 3 version of Distribute is generated live on the installation system? That doesn't seem right because that leaves the possibility that the code generated by 2to3 is different from the code that passed the (hypothetical, at this point) tests before the distribution was created. I'm going to cross-post this to the distutils list as well. If the idea is to stabilize Distribute, get some tests in place, etc. then this method of distribution (running 2to3 in situ, as it were) is not the correct way to do it from a "shipping the tested product" perspective. S From regebro at gmail.com Fri Oct 16 18:59:51 2009 From: regebro at gmail.com (Lennart Regebro) Date: Fri, 16 Oct 2009 18:59:51 +0200 Subject: [Distutils] [venv] Re: two questions about virtualenv In-Reply-To: <5DFFC162-0687-45FB-B219-9F05DF3AFB60@gmail.com> References: <87ljjbmsij.fsf@rhodesmill.org> <73D01E3C-A349-462A-BF2C-465F5A572AE3@leidel.info> <87r5t3l4je.fsf@rhodesmill.org> <715CB4F3-F653-49C1-B936-46466BF3C2F5@leidel.info> <87iqefl30b.fsf@rhodesmill.org> <5DFFC162-0687-45FB-B219-9F05DF3AFB60@gmail.com> Message-ID: <319e029f0910160959g25cef5cch276099df5c601e24@mail.gmail.com> 2009/10/16 ssteinerX at gmail.com : > Doesn't that also mean that the Python 3 version of Distribute is generated > live on the installation system? That is absolutely true. >?That doesn't seem right because that > leaves the possibility that the code generated by 2to3 is different from the > code that passed the (hypothetical, at this point) tests before the > distribution was created. The tests are not hypothetical. The code may be different if you use different versions of Python, since they may have different version of the 2to3 library. That is however not a bigger problem than any other incompatibilities between different python versions. > I'm going to cross-post this to the distutils list as well. ?If the idea is > to stabilize Distribute, get some tests in place, etc. then this method of > distribution ?(running 2to3 in situ, as it were) is not the correct way to > do it from a "shipping the tested product" perspective. I don't agree with that statement. -- Lennart Regebro: Python, Zope, Plone, Grok http://regebro.wordpress.com/ +33 661 58 14 64 From ianb at colorstudy.com Fri Oct 16 19:11:16 2009 From: ianb at colorstudy.com (Ian Bicking) Date: Fri, 16 Oct 2009 12:11:16 -0500 Subject: [Distutils] small distribute improvement In-Reply-To: References: Message-ID: On Fri, Oct 16, 2009 at 6:20 AM, Neal Becker wrote: > I think it is confusing that > easy_install blah > > doesn't do anything, and doesn't provide any clue, if it was to do an > update. ?You need to say > easy_install -U blah > > There are a few choices here. ?One might be that > easy_install blah > could notify you that an update is available. pip works somewhat similarly (it won't do anything in this case), but in the tip it will say "Requirement already satisfied (use --upgrade to upgrade): blah". It doesn't actually check if an upgrade is available though. It actually keeps track of why something is installed, so it can tell the difference between something installed from the command line and installed as a requirement of some other package (where you definitely want to quickly pass over satisfied requirements). On the other hand, I find it useful sometimes to test things manually knowing the behavior is the same on the command line as with install_requires. -- Ian Bicking | http://blog.ianbicking.org | http://topplabs.org/civichacker From ssteinerx at gmail.com Fri Oct 16 19:22:37 2009 From: ssteinerx at gmail.com (ssteinerX@gmail.com) Date: Fri, 16 Oct 2009 13:22:37 -0400 Subject: [Distutils] [venv] Re: two questions about virtualenv In-Reply-To: <319e029f0910160959g25cef5cch276099df5c601e24@mail.gmail.com> References: <87ljjbmsij.fsf@rhodesmill.org> <73D01E3C-A349-462A-BF2C-465F5A572AE3@leidel.info> <87r5t3l4je.fsf@rhodesmill.org> <715CB4F3-F653-49C1-B936-46466BF3C2F5@leidel.info> <87iqefl30b.fsf@rhodesmill.org> <5DFFC162-0687-45FB-B219-9F05DF3AFB60@gmail.com> <319e029f0910160959g25cef5cch276099df5c601e24@mail.gmail.com> Message-ID: On Oct 16, 2009, at 12:59 PM, Lennart Regebro wrote: >> I'm going to cross-post this to the distutils list as well. If the >> idea is >> to stabilize Distribute, get some tests in place, etc. then this >> method of >> distribution (running 2to3 in situ, as it were) is not the correct >> way to >> do it from a "shipping the tested product" perspective. > > I don't agree with that statement. This is not analogous to running a C compiler, it's analogous to running a pre-processor maybe, but the fact that the distribution doesn't run the tests on the exact code delivered in the package gives me agita. S From ianb at colorstudy.com Fri Oct 16 19:29:49 2009 From: ianb at colorstudy.com (Ian Bicking) Date: Fri, 16 Oct 2009 12:29:49 -0500 Subject: [Distutils] [RFC] Recentering the static metadata need : PKG-INFO In-Reply-To: <94bdd2610910160654n351d786ap4366b6e9e99539dc@mail.gmail.com> References: <94bdd2610910160654n351d786ap4366b6e9e99539dc@mail.gmail.com> Message-ID: On Fri, Oct 16, 2009 at 8:54 AM, Tarek Ziad? wrote: > Marc-Andr? Lemburg gave me a key points about the static metadata > discussions we have (wether its PEP 390 or around it) > > He said that the important thing was to have the context-dependant > markers in PKG-INFO, and that having in described in > setup.cfg or in setup.py by any way is not the important thing. And he > is right ! Excellent, this resolves my own concern about the discussion as well. Putting this in PKG-INFO is something I can concretely make use of, regardless of how it is generated. PKG-INFO is already somewhat flawed as a format for holding the data, in particularly maintaining indentation for Description. I think adding a general "...; " places another syntactic constraint, where no field can have ";" in it. Ideally I'd like to see both cases resolved. And certainly I don't want to end up in a place where weird bugs emerge if I put ";" in a field (especially since many are free text). -- Ian Bicking | http://blog.ianbicking.org | http://topplabs.org/civichacker From tseaver at palladion.com Fri Oct 16 20:06:35 2009 From: tseaver at palladion.com (Tres Seaver) Date: Fri, 16 Oct 2009 14:06:35 -0400 Subject: [Distutils] [RFC] Recentering the static metadata need : PKG-INFO In-Reply-To: References: <94bdd2610910160654n351d786ap4366b6e9e99539dc@mail.gmail.com> Message-ID: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Ian Bicking wrote: > On Fri, Oct 16, 2009 at 8:54 AM, Tarek Ziad? wrote: >> Marc-Andr? Lemburg gave me a key points about the static metadata >> discussions we have (wether its PEP 390 or around it) >> >> He said that the important thing was to have the context-dependant >> markers in PKG-INFO, and that having in described in >> setup.cfg or in setup.py by any way is not the important thing. And he >> is right ! > > Excellent, this resolves my own concern about the discussion as well. > Putting this in PKG-INFO is something I can concretely make use of, > regardless of how it is generated. > > PKG-INFO is already somewhat flawed as a format for holding the data, > in particularly maintaining indentation for Description. I think > adding a general "...; " places another syntactic > constraint, where no field can have ";" in it. Ideally I'd like to > see both cases resolved. And certainly I don't want to end up in a > place where weird bugs emerge if I put ";" in a field (especially > since many are free text). The only fields where I see this syntax making any sense are the "requirements" ones: - - The ones specified in PEP 314: Requires, Provides, Obsoletes - - The new ones proposed in PEP 345[1]: Requires-Dist, Provides-Dist, Obsoletes-Dist [1] As amended here: http://svn.python.org/view/peps/branches/jim-update-345/pep-0345.txt?revision=73235&view=markup Tres. - -- =================================================================== Tres Seaver +1 540-429-0999 tseaver at palladion.com Palladion Software "Excellence by Design" http://palladion.com -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iEYEARECAAYFAkrYtiYACgkQ+gerLs4ltQ5XhACeJ1XHIreqAU4IipKoa0UHH2dO ADQAnRddgnKzi6iWw5Nd9T9cCOnbFE5S =wsCj -----END PGP SIGNATURE----- From tseaver at palladion.com Fri Oct 16 20:09:08 2009 From: tseaver at palladion.com (Tres Seaver) Date: Fri, 16 Oct 2009 14:09:08 -0400 Subject: [Distutils] [venv] Re: two questions about virtualenv In-Reply-To: References: <87ljjbmsij.fsf@rhodesmill.org> <73D01E3C-A349-462A-BF2C-465F5A572AE3@leidel.info> <87r5t3l4je.fsf@rhodesmill.org> <715CB4F3-F653-49C1-B936-46466BF3C2F5@leidel.info> <87iqefl30b.fsf@rhodesmill.org> <5DFFC162-0687-45FB-B219-9F05DF3AFB60@gmail.com> <319e029f0910160959g25cef5cch276099df5c601e24@mail.gmail.com> Message-ID: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 ssteinerX at gmail.com wrote: > On Oct 16, 2009, at 12:59 PM, Lennart Regebro wrote: >>> I'm going to cross-post this to the distutils list as well. If the >>> idea is >>> to stabilize Distribute, get some tests in place, etc. then this >>> method of >>> distribution (running 2to3 in situ, as it were) is not the correct >>> way to >>> do it from a "shipping the tested product" perspective. >> I don't agree with that statement. > > This is not analogous to running a C compiler, it's analogous to > running a pre-processor maybe, but the fact that the distribution > doesn't run the tests on the exact code delivered in the package gives > me agita. You can't run tests on an sdist: at a minimus, you have to build it and run the tests with the build directory on your path. Tests run inside a "develop" egg (a VCS checkout) are always potentially influenced by the environment they are run in, which is why we need buildbots (or the equivalent) for each supported platform. Tres. - -- =================================================================== Tres Seaver +1 540-429-0999 tseaver at palladion.com Palladion Software "Excellence by Design" http://palladion.com -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iEYEARECAAYFAkrYtsQACgkQ+gerLs4ltQ51CQCfX6nwpS50Vwj1VVX96eOi7MeR SCEAoIFuefuVRZa9UbyWzU6Q/R2WYBt1 =tdbU -----END PGP SIGNATURE----- From ssteinerx at gmail.com Fri Oct 16 20:33:02 2009 From: ssteinerx at gmail.com (ssteinerX@gmail.com) Date: Fri, 16 Oct 2009 14:33:02 -0400 Subject: [Distutils] [venv] Re: two questions about virtualenv In-Reply-To: References: <87ljjbmsij.fsf@rhodesmill.org> <73D01E3C-A349-462A-BF2C-465F5A572AE3@leidel.info> <87r5t3l4je.fsf@rhodesmill.org> <715CB4F3-F653-49C1-B936-46466BF3C2F5@leidel.info> <87iqefl30b.fsf@rhodesmill.org> <5DFFC162-0687-45FB-B219-9F05DF3AFB60@gmail.com> <319e029f0910160959g25cef5cch276099df5c601e24@mail.gmail.com> Message-ID: <94F617E5-1E74-4F0D-8568-E3D1F4359B26@gmail.com> On Oct 16, 2009, at 2:09 PM, Tres Seaver wrote: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > ssteinerX at gmail.com wrote: >> On Oct 16, 2009, at 12:59 PM, Lennart Regebro wrote: >>>> I'm going to cross-post this to the distutils list as well. If the >>>> idea is to stabilize Distribute, get some tests in place, etc. >>>> then this >>>> method of distribution (running 2to3 in situ, as it were) is not >>>> the correct >>>> way to do it from a "shipping the tested product" perspective. >>> I don't agree with that statement. >> >> This is not analogous to running a C compiler, it's analogous to >> running a pre-processor maybe, but the fact that the distribution >> doesn't run the tests on the exact code delivered in the package >> gives >> me agita. > > You can't run tests on an sdist: at a minimus, you have to build it > and > run the tests with the build directory on your path. Tests run > inside a > "develop" egg (a VCS checkout) are always potentially influenced by > the > environment they are run in, which is why we need buildbots (or the > equivalent) for each supported platform. Buildbots seem to be in short supply though they should be easy enough to come by in this day and age of penny-an-hour cloud computing. I think I'll see if maybe I can just whip up something about this... S From exarkun at twistedmatrix.com Fri Oct 16 21:56:34 2009 From: exarkun at twistedmatrix.com (exarkun at twistedmatrix.com) Date: Fri, 16 Oct 2009 19:56:34 -0000 Subject: [Distutils] [venv] Re: two questions about virtualenv In-Reply-To: <94F617E5-1E74-4F0D-8568-E3D1F4359B26@gmail.com> References: <87ljjbmsij.fsf@rhodesmill.org> <73D01E3C-A349-462A-BF2C-465F5A572AE3@leidel.info> <87r5t3l4je.fsf@rhodesmill.org> <715CB4F3-F653-49C1-B936-46466BF3C2F5@leidel.info> <87iqefl30b.fsf@rhodesmill.org> <5DFFC162-0687-45FB-B219-9F05DF3AFB60@gmail.com> <319e029f0910160959g25cef5cch276099df5c601e24@mail.gmail.com> <94F617E5-1E74-4F0D-8568-E3D1F4359B26@gmail.com> Message-ID: <20091016195634.8004.1854530971.divmod.xquotient.145@localhost.localdomain> On 06:33 pm, ssteinerx at gmail.com wrote: > >On Oct 16, 2009, at 2:09 PM, Tres Seaver wrote: >>-----BEGIN PGP SIGNED MESSAGE----- >>Hash: SHA1 >> >>ssteinerX at gmail.com wrote: >>>On Oct 16, 2009, at 12:59 PM, Lennart Regebro wrote: >>>>>I'm going to cross-post this to the distutils list as well. If the >>>>>idea is to stabilize Distribute, get some tests in place, etc. >>>>>then this >>>>>method of distribution (running 2to3 in situ, as it were) is not >>>>>the correct >>>>>way to do it from a "shipping the tested product" perspective. >>>>I don't agree with that statement. >>> >>>This is not analogous to running a C compiler, it's analogous to >>>running a pre-processor maybe, but the fact that the distribution >>>doesn't run the tests on the exact code delivered in the package >>>gives >>>me agita. >> >>You can't run tests on an sdist: at a minimus, you have to build it >>and >>run the tests with the build directory on your path. Tests run >>inside a >>"develop" egg (a VCS checkout) are always potentially influenced by >>the >>environment they are run in, which is why we need buildbots (or the >>equivalent) for each supported platform. > >Buildbots seem to be in short supply though they should be easy >enough to come by in this day and age of penny-an-hour cloud computing. > >I think I'll see if maybe I can just whip up something about this... Not sure if you're aware, but the latest version of buildbot includes support for managing ec2-based slaves. That's an easy way to get all the slaves you can afford. Jean-Paul From ssteinerx at gmail.com Fri Oct 16 22:22:04 2009 From: ssteinerx at gmail.com (ssteinerX@gmail.com) Date: Fri, 16 Oct 2009 16:22:04 -0400 Subject: [Distutils] [venv] Re: two questions about virtualenv In-Reply-To: <20091016195634.8004.1854530971.divmod.xquotient.145@localhost.localdomain> References: <87ljjbmsij.fsf@rhodesmill.org> <73D01E3C-A349-462A-BF2C-465F5A572AE3@leidel.info> <87r5t3l4je.fsf@rhodesmill.org> <715CB4F3-F653-49C1-B936-46466BF3C2F5@leidel.info> <87iqefl30b.fsf@rhodesmill.org> <5DFFC162-0687-45FB-B219-9F05DF3AFB60@gmail.com> <319e029f0910160959g25cef5cch276099df5c601e24@mail.gmail.com> <94F617E5-1E74-4F0D-8568-E3D1F4359B26@gmail.com> <20091016195634.8004.1854530971.divmod.xquotient.145@localhost.localdomain> Message-ID: <2843D1AB-04EC-4150-8037-4E17F7EBE4D0@gmail.com> On Oct 16, 2009, at 3:56 PM, exarkun at twistedmatrix.com wrote: >> Buildbots seem to be in short supply though they should be easy >> enough to come by in this day and age of penny-an-hour cloud >> computing. >> >> I think I'll see if maybe I can just whip up something about this... > > Not sure if you're aware, but the latest version of buildbot > includes support for managing ec2-based slaves. That's an easy way > to get all the slaves you can afford. No, I did not know that, thanks! Thing is, with ec2's high prices, the number of servers I could afford to put up would be pretty limited. I was thinking more along the lines of options at Rackspace and others (@ Rackspace, for example, 1.5 cents/hr for 256MB 10 GB server). Maybe the best thing to do is just extend buildbot to handle other server types where the prices aren't quite so steep. Thanks for the heads up! S From david.lyon at preisshare.net Sat Oct 17 00:39:02 2009 From: david.lyon at preisshare.net (David Lyon) Date: Fri, 16 Oct 2009 18:39:02 -0400 Subject: [Distutils] [RFC] Recentering the static metadata need : PKG-INFO In-Reply-To: <94bdd2610910160654n351d786ap4366b6e9e99539dc@mail.gmail.com> References: <94bdd2610910160654n351d786ap4366b6e9e99539dc@mail.gmail.com> Message-ID: On Fri, 16 Oct 2009 15:54:29 +0200, Tarek Ziad? wrote: > But what really matters at the end is to provide a new Metadata format > where these marker are present, so any consumer > can play with them (and Distutils provide an API to play with them : > the DistributionMetadata class) I'm fine with this. Although I suggest that we could clean up the language a bit using some of the conditional compound ideas from what I'm working on. > So please, have a look at the new section I've added here with the > syntax I am proposing : > > http://www.python.org/dev/peps/pep-0390/#impact-on-pkg-info-generation-and-pep-314 > Metadata-Version: 1.2 Name: distribute Version: 0.6.4 ... Requires: pywin32, bar > 1.0; sys_platform == 'win32' Requires: foo; os_machine == 'i386' Requires: bar; python_version == '2.4' or python_version == '2.5' Requires: baz; 'linux' in sys_platform Obsoletes = pywin31; sys_platform == 'win32' ... Classifier: Development Status :: 5 - Production/Stable Classifier: Intended Audience :: Developers Classifier: License :: OSI Approved :: Python Software Foundation License something like this (as best I can read the other format): Metadata-Version: 1.2 Name: distribute Version: 0.6.4 ... Requires: (windows) pywin32, bar > 1.0 Requires: (linux-suse-kde-64) foo Requires: (python < 2.5) bar Requires: (linux) baz Obsoletes = (windows) pywin31 ... Classifier: Development Status :: 5 - Production/Stable Classifier: Intended Audience :: Developers Classifier: License :: OSI Approved :: Python Software Foundation License Regards David From ssteinerx at gmail.com Sat Oct 17 00:51:40 2009 From: ssteinerx at gmail.com (ssteinerX@gmail.com) Date: Fri, 16 Oct 2009 18:51:40 -0400 Subject: [Distutils] Buildbot throwing a gear after default setup? Message-ID: <2E581399-E286-4C77-A787-30861315764E@gmail.com> I've just downloaded and configured the Distribute buildbot and it appears that a dependency is missing somewhere. I get this (just the errors excerpted). Just wanted to capture this before I figure out what needs to be done... test_make_tarball (distutils.tests.test_archive_util.ArchiveUtilTestCase) ... skipped 'requires zlib' test_make_zipfile (distutils.tests.test_archive_util.ArchiveUtilTestCase) ... skipped 'Requires zlib' test_tarfile_root_owner (distutils.tests.test_archive_util.ArchiveUtilTestCase) ... skipped 'Requires zlib' test_tarfile_vs_tar (distutils.tests.test_archive_util.ArchiveUtilTestCase) ... skipped 'Requires zlib' ====================================================================== ERROR: test_make_archive_owner_group (distutils.tests.test_archive_util.ArchiveUtilTestCase) ---------------------------------------------------------------------- Traceback (most recent call last): File "/home/ssteiner/distutils-builbot/parts/trunk-checkout/python/ Lib/distutils/tests/test_archive_util.py", line 224, in test_make_archive_owner_group group=group) File "/home/ssteiner/distutils-builbot/parts/trunk-checkout/python/ Lib/distutils/archive_util.py", line 236, in make_archive filename = func(base_name, base_dir, **kwargs) File "/home/ssteiner/distutils-builbot/parts/trunk-checkout/python/ Lib/distutils/archive_util.py", line 163, in make_zipfile compression=zipfile.ZIP_DEFLATED) File "/home/ssteiner/distutils-builbot/parts/trunk-checkout/python/ Lib/zipfile.py", line 675, in __init__ "Compression requires the (missing) zlib module" RuntimeError: Compression requires the (missing) zlib module ---------------------------------------------------------------------- Ran 158 tests in 1.242s FAILED (errors=1, skipped=11) test test_distutils crashed -- : [Errno 2] No such file or directory Traceback (most recent call last): File "Lib/test/regrtest.py", line 767, in runtest_inner File "Lib/test/regrtest.py", line 723, in __exit__ File "Lib/test/regrtest.py", line 686, in get_cwd OSError: [Errno 2] No such file or directory 1 test failed: test_distutils From nathan at creativecommons.org Sat Oct 17 00:58:08 2009 From: nathan at creativecommons.org (Nathan Yergler) Date: Fri, 16 Oct 2009 15:58:08 -0700 Subject: [Distutils] Buildbot throwing a gear after default setup? In-Reply-To: <2E581399-E286-4C77-A787-30861315764E@gmail.com> References: <2E581399-E286-4C77-A787-30861315764E@gmail.com> Message-ID: Looks like you're using a build of Python that wasn't compiled with zlib support (--with-zlib, IIRC). 2009/10/16 ssteinerX at gmail.com : > I've just downloaded and configured the Distribute buildbot and it appears that a dependency is missing somewhere. ?I get this (just the errors excerpted). > > Just wanted to capture this before I figure out what needs to be done... > > test_make_tarball (distutils.tests.test_archive_util.ArchiveUtilTestCase) ... skipped 'requires zlib' > test_make_zipfile (distutils.tests.test_archive_util.ArchiveUtilTestCase) ... skipped 'Requires zlib' > test_tarfile_root_owner (distutils.tests.test_archive_util.ArchiveUtilTestCase) ... skipped 'Requires zlib' > test_tarfile_vs_tar (distutils.tests.test_archive_util.ArchiveUtilTestCase) ... skipped 'Requires zlib' > > ====================================================================== > ERROR: test_make_archive_owner_group (distutils.tests.test_archive_util.ArchiveUtilTestCase) > ---------------------------------------------------------------------- > Traceback (most recent call last): > ?File "/home/ssteiner/distutils-builbot/parts/trunk-checkout/python/Lib/distutils/tests/test_archive_util.py", line 224, in test_make_archive_owner_group > ? ?group=group) > ?File "/home/ssteiner/distutils-builbot/parts/trunk-checkout/python/Lib/distutils/archive_util.py", line 236, in make_archive > ? ?filename = func(base_name, base_dir, **kwargs) > ?File "/home/ssteiner/distutils-builbot/parts/trunk-checkout/python/Lib/distutils/archive_util.py", line 163, in make_zipfile > ? ?compression=zipfile.ZIP_DEFLATED) > ?File "/home/ssteiner/distutils-builbot/parts/trunk-checkout/python/Lib/zipfile.py", line 675, in __init__ > ? ?"Compression requires the (missing) zlib module" > RuntimeError: Compression requires the (missing) zlib module > > ---------------------------------------------------------------------- > Ran 158 tests in 1.242s > > FAILED (errors=1, skipped=11) > test test_distutils crashed -- : [Errno 2] No such file or directory > Traceback (most recent call last): > ?File "Lib/test/regrtest.py", line 767, in runtest_inner > ?File "Lib/test/regrtest.py", line 723, in __exit__ > ?File "Lib/test/regrtest.py", line 686, in get_cwd > OSError: [Errno 2] No such file or directory > 1 test failed: > ? ?test_distutils > > > _______________________________________________ > Distutils-SIG maillist ?- ?Distutils-SIG at python.org > http://mail.python.org/mailman/listinfo/distutils-sig > From ssteinerx at gmail.com Sat Oct 17 01:03:37 2009 From: ssteinerx at gmail.com (ssteinerX@gmail.com) Date: Fri, 16 Oct 2009 19:03:37 -0400 Subject: [Distutils] Buildbot throwing a gear after default setup? In-Reply-To: References: <2E581399-E286-4C77-A787-30861315764E@gmail.com> Message-ID: <29B72F5A-5FDA-4DA5-B110-B6A64E0EACC7@gmail.com> On Oct 16, 2009, at 6:58 PM, Nathan Yergler wrote: > Looks like you're using a build of Python that wasn't compiled with > zlib support (--with-zlib, IIRC). Right -- that's the python that the buildbot compiled from source, that's why it's a bug ;-). S From ziade.tarek at gmail.com Sat Oct 17 01:04:38 2009 From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=) Date: Sat, 17 Oct 2009 01:04:38 +0200 Subject: [Distutils] Buildbot throwing a gear after default setup? In-Reply-To: References: <2E581399-E286-4C77-A787-30861315764E@gmail.com> Message-ID: <94bdd2610910161604t10b3cd0ame3dc324cd12a644b@mail.gmail.com> On Sat, Oct 17, 2009 at 12:58 AM, Nathan Yergler wrote: > Looks like you're using a build of Python that wasn't compiled with > zlib support (--with-zlib, IIRC). > > 2009/10/16 ssteinerX at gmail.com : >> I've just downloaded and configured the Distribute buildbot and it appears that a dependency is missing somewhere. ?I get this (just the errors excerpted). Nathan is right, but this Distutils test was supposed to check if zlib is supported before its run, I have fixed it in r75450. So you should be OK now. Thanks for noticing ! Tarek From ziade.tarek at gmail.com Sat Oct 17 01:11:58 2009 From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=) Date: Sat, 17 Oct 2009 01:11:58 +0200 Subject: [Distutils] [RFC] Recentering the static metadata need : PKG-INFO In-Reply-To: References: <94bdd2610910160654n351d786ap4366b6e9e99539dc@mail.gmail.com> Message-ID: <94bdd2610910161611s3cdcecc8s8ba9858abdbf3076@mail.gmail.com> On Fri, Oct 16, 2009 at 8:06 PM, Tres Seaver wrote: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > Ian Bicking wrote: >> On Fri, Oct 16, 2009 at 8:54 AM, Tarek Ziad? wrote: >>> Marc-Andr? Lemburg gave me a key points about the static metadata >>> discussions we have (wether its PEP 390 or around it) >>> >>> He said that the important thing was to have the context-dependant >>> markers in PKG-INFO, and that having in described in >>> setup.cfg or in setup.py by any way is not the important thing. And he >>> is right ! >> >> Excellent, this resolves my own concern about the discussion as well. >> Putting this in PKG-INFO is something I can concretely make use of, >> regardless of how it is generated. >> >> PKG-INFO is already somewhat flawed as a format for holding the data, >> in particularly maintaining indentation for Description. ?I think >> adding a general "...; " places another syntactic >> constraint, where no field can have ";" in it. ?Ideally I'd like to >> see both cases resolved. ?And certainly I don't want to end up in a >> place where weird bugs emerge if I put ";" in a field (especially >> since many are free text). > > The only fields where I see this syntax making any sense are the > "requirements" ones: > > - - The ones specified in PEP 314: Requires, Provides, Obsoletes > - - The new ones proposed in PEP 345[1]: ?Requires-Dist, Provides-Dist, > ?Obsoletes-Dist Yes, that would have to be a subset of the fields, and ";" makes PKG-INFO still RFC 232 compatible. Tarek From ssteinerx at gmail.com Sat Oct 17 01:13:53 2009 From: ssteinerx at gmail.com (ssteinerX@gmail.com) Date: Fri, 16 Oct 2009 19:13:53 -0400 Subject: [Distutils] Buildbot throwing a gear after default setup? In-Reply-To: <94bdd2610910161604t10b3cd0ame3dc324cd12a644b@mail.gmail.com> References: <2E581399-E286-4C77-A787-30861315764E@gmail.com> <94bdd2610910161604t10b3cd0ame3dc324cd12a644b@mail.gmail.com> Message-ID: <31A858D3-D8B1-4B03-8D0D-906DA5DA46A2@gmail.com> On Oct 16, 2009, at 7:04 PM, Tarek Ziad? wrote: > On Sat, Oct 17, 2009 at 12:58 AM, Nathan Yergler > wrote: >> Looks like you're using a build of Python that wasn't compiled with >> zlib support (--with-zlib, IIRC). >> >> 2009/10/16 ssteinerX at gmail.com : >>> I've just downloaded and configured the Distribute buildbot and it >>> appears that a dependency is missing somewhere. I get this (just >>> the errors excerpted). > > Nathan is right, but this Distutils test was supposed to check if zlib > is supported before its run, > > I have fixed it in r75450. So you should be OK now. > > Thanks for noticing ! Cool, no problem. I also reported it in the buildbot project on BitBucket. Odd thing is: ssteiner at ubuntu:~/distutils-builbot$ cat bin/test #!/usr/bin/python # -*- coding: utf-8 -*- ... ssteiner at ubuntu:~/distutils-builbot$ /usr/bin/python Python 2.6.2 (release26-maint, Apr 19 2009, 01:58:18) [GCC 4.3.3] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> import zlib works fine, so I'm not sure why it thinks it's not able to get zlib. S From ziade.tarek at gmail.com Sat Oct 17 01:15:25 2009 From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=) Date: Sat, 17 Oct 2009 01:15:25 +0200 Subject: [Distutils] [RFC] Recentering the static metadata need : PKG-INFO In-Reply-To: References: <94bdd2610910160654n351d786ap4366b6e9e99539dc@mail.gmail.com> Message-ID: <94bdd2610910161615h10ac409k210f4fbe88359624@mail.gmail.com> On Sat, Oct 17, 2009 at 12:39 AM, David Lyon wrote: > ?Requires: (linux-suse-kde-64) foo How this expression would be verified on the target system ? Regards Tarek From tseaver at palladion.com Sat Oct 17 01:18:24 2009 From: tseaver at palladion.com (Tres Seaver) Date: Fri, 16 Oct 2009 19:18:24 -0400 Subject: [Distutils] Buildbot throwing a gear after default setup? In-Reply-To: <29B72F5A-5FDA-4DA5-B110-B6A64E0EACC7@gmail.com> References: <2E581399-E286-4C77-A787-30861315764E@gmail.com> <29B72F5A-5FDA-4DA5-B110-B6A64E0EACC7@gmail.com> Message-ID: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 ssteinerX at gmail.com wrote: > On Oct 16, 2009, at 6:58 PM, Nathan Yergler wrote: > >> Looks like you're using a build of Python that wasn't compiled with >> zlib support (--with-zlib, IIRC). > > Right -- that's the python that the buildbot compiled from source, > that's why it's a bug ;-). You need to install package contains the headers for zlib in your OS / distro: Python conditionally compiles in zlib support based on whether it can find the .h file. Tres. - -- =================================================================== Tres Seaver +1 540-429-0999 tseaver at palladion.com Palladion Software "Excellence by Design" http://palladion.com -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iEYEARECAAYFAkrY/zwACgkQ+gerLs4ltQ6NzgCfbEAgqorW/mAyCcJAinLr5RB0 ZhcAn0TF6n9SjPEfKGPYJlJ4vj0fGIhY =UIai -----END PGP SIGNATURE----- From ziade.tarek at gmail.com Sat Oct 17 01:21:01 2009 From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=) Date: Sat, 17 Oct 2009 01:21:01 +0200 Subject: [Distutils] Buildbot throwing a gear after default setup? In-Reply-To: <31A858D3-D8B1-4B03-8D0D-906DA5DA46A2@gmail.com> References: <2E581399-E286-4C77-A787-30861315764E@gmail.com> <94bdd2610910161604t10b3cd0ame3dc324cd12a644b@mail.gmail.com> <31A858D3-D8B1-4B03-8D0D-906DA5DA46A2@gmail.com> Message-ID: <94bdd2610910161621t4a57dd0bv6d27ef45a676caba@mail.gmail.com> On Sat, Oct 17, 2009 at 1:13 AM, ssteinerX at gmail.com wrote: > > On Oct 16, 2009, at 7:04 PM, Tarek Ziad? wrote: > >> On Sat, Oct 17, 2009 at 12:58 AM, Nathan Yergler >> wrote: >>> >>> Looks like you're using a build of Python that wasn't compiled with >>> zlib support (--with-zlib, IIRC). >>> >>> 2009/10/16 ssteinerX at gmail.com : >>>> >>>> I've just downloaded and configured the Distribute buildbot and it >>>> appears that a dependency is missing somewhere. ?I get this (just the errors >>>> excerpted). >> >> Nathan is right, but this Distutils test was supposed to check if zlib >> is supported before its run, >> >> I have fixed it in r75450. So you should be OK now. >> >> Thanks for noticing ! > > Cool, no problem. ?I also reported it in the buildbot project on BitBucket. > > Odd thing is: > > ssteiner at ubuntu:~/distutils-builbot$ cat bin/test > #!/usr/bin/python > # -*- coding: utf-8 -*- > ... > > ssteiner at ubuntu:~/distutils-builbot$ /usr/bin/python > Python 2.6.2 (release26-maint, Apr 19 2009, 01:58:18) > [GCC 4.3.3] on linux2 > Type "help", "copyright", "credits" or "license" for more information. >>>> import zlib > > works fine, so I'm not sure why it thinks it's not able to get zlib. Python gets build from scratch everytime here, and "test" just runs a subprocess to call the "other" python located in the buildbot. IOW the way python is built in the bbot, is now using your system's zlib support... I need to check why... Regards Tarek > > S > > > -- Tarek Ziad? | http://ziade.org |????????????! |???????????? From ziade.tarek at gmail.com Sat Oct 17 01:26:04 2009 From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=) Date: Sat, 17 Oct 2009 01:26:04 +0200 Subject: [Distutils] Buildbot throwing a gear after default setup? In-Reply-To: <94bdd2610910161621t4a57dd0bv6d27ef45a676caba@mail.gmail.com> References: <2E581399-E286-4C77-A787-30861315764E@gmail.com> <94bdd2610910161604t10b3cd0ame3dc324cd12a644b@mail.gmail.com> <31A858D3-D8B1-4B03-8D0D-906DA5DA46A2@gmail.com> <94bdd2610910161621t4a57dd0bv6d27ef45a676caba@mail.gmail.com> Message-ID: <94bdd2610910161626o64a1376an25af942da011f1d5@mail.gmail.com> On Sat, Oct 17, 2009 at 1:21 AM, Tarek Ziad? wrote: > > IOW the way python is built in the bbot, is now using your system's > zlib support... Arggg... I am the typo master... s/now/not. So as Tres said maybe Ubuntu ships with a Python with zlib support, but doesn't provide zlib headers. From ssteinerx at gmail.com Sat Oct 17 01:36:31 2009 From: ssteinerx at gmail.com (ssteinerX@gmail.com) Date: Fri, 16 Oct 2009 19:36:31 -0400 Subject: [Distutils] Buildbot throwing a gear after default setup? In-Reply-To: <94bdd2610910161621t4a57dd0bv6d27ef45a676caba@mail.gmail.com> References: <2E581399-E286-4C77-A787-30861315764E@gmail.com> <94bdd2610910161604t10b3cd0ame3dc324cd12a644b@mail.gmail.com> <31A858D3-D8B1-4B03-8D0D-906DA5DA46A2@gmail.com> <94bdd2610910161621t4a57dd0bv6d27ef45a676caba@mail.gmail.com> Message-ID: <66153AF8-3F47-4407-8E14-6EBE85A91756@gmail.com> On Oct 16, 2009, at 7:21 PM, Tarek Ziad? wrote: > On Sat, Oct 17, 2009 at 1:13 AM, ssteinerX at gmail.com > wrote: >> >> On Oct 16, 2009, at 7:04 PM, Tarek Ziad? wrote: >> >>> On Sat, Oct 17, 2009 at 12:58 AM, Nathan Yergler >>> wrote: >>>> >>>> Looks like you're using a build of Python that wasn't compiled with >>>> zlib support (--with-zlib, IIRC). >>>> >>>> 2009/10/16 ssteinerX at gmail.com : >>>>> >>>>> I've just downloaded and configured the Distribute buildbot and it >>>>> appears that a dependency is missing somewhere. I get this >>>>> (just the errors >>>>> excerpted). >>> >>> Nathan is right, but this Distutils test was supposed to check if >>> zlib >>> is supported before its run, >>> >>> I have fixed it in r75450. So you should be OK now. >>> >>> Thanks for noticing ! >> >> Cool, no problem. I also reported it in the buildbot project on >> BitBucket. >> >> Odd thing is: >> >> ssteiner at ubuntu:~/distutils-builbot$ cat bin/test >> #!/usr/bin/python >> # -*- coding: utf-8 -*- >> ... >> >> ssteiner at ubuntu:~/distutils-builbot$ /usr/bin/python >> Python 2.6.2 (release26-maint, Apr 19 2009, 01:58:18) >> [GCC 4.3.3] on linux2 >> Type "help", "copyright", "credits" or "license" for more >> information. >>>>> import zlib >> >> works fine, so I'm not sure why it thinks it's not able to get zlib. > > Python gets build from scratch everytime here, and "test" just runs > a subprocess > to call the "other" python located in the buildbot. > > IOW the way python is built in the bbot, is now using your system's > zlib support... No, the Python I called was the system python in /usr/bin, I see that test is grabbing the trunk checkout python in python = os.path.join (root, 'parts', 'trunk-checkout', 'python', 'python') which is NOT built with zlib. The error is in the build of the trunk checkout -- zlib is in my system python just fine; it's just not in the python built from the trunk checkout: ssteiner at ubuntu:~/distutils-builbot$ parts/trunk-checkout/python/python Python 2.7a0 (trunk:75449, Oct 16 2009, 16:08:27) [GCC 4.3.3] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> import zlib Traceback (most recent call last): File "", line 1, in ImportError: No module named zlib >>> S From ssteinerx at gmail.com Sat Oct 17 01:39:57 2009 From: ssteinerx at gmail.com (ssteinerX@gmail.com) Date: Fri, 16 Oct 2009 19:39:57 -0400 Subject: [Distutils] Buildbot throwing a gear after default setup? In-Reply-To: <94bdd2610910161626o64a1376an25af942da011f1d5@mail.gmail.com> References: <2E581399-E286-4C77-A787-30861315764E@gmail.com> <94bdd2610910161604t10b3cd0ame3dc324cd12a644b@mail.gmail.com> <31A858D3-D8B1-4B03-8D0D-906DA5DA46A2@gmail.com> <94bdd2610910161621t4a57dd0bv6d27ef45a676caba@mail.gmail.com> <94bdd2610910161626o64a1376an25af942da011f1d5@mail.gmail.com> Message-ID: <64017BB5-2954-4CC5-805C-12066B0127F1@gmail.com> On Oct 16, 2009, at 7:26 PM, Tarek Ziad? wrote: > On Sat, Oct 17, 2009 at 1:21 AM, Tarek Ziad? > wrote: >> >> IOW the way python is built in the bbot, is now using your system's >> zlib support... > > Arggg... I am the typo master... > > s/now/not. > > So as Tres said maybe Ubuntu ships with a Python with zlib support, > but doesn't provide zlib headers. Ok, so the python you build from the trunk checkout doesn't have zlib support. I couldn't find an apt-get'able one, it looks like you have to build from source, but the lack of zlib support should be picked up before the unit tests fail. S From david.lyon at preisshare.net Sat Oct 17 01:38:40 2009 From: david.lyon at preisshare.net (David Lyon) Date: Fri, 16 Oct 2009 19:38:40 -0400 Subject: [Distutils] [RFC] Recentering the static metadata need : PKG-INFO In-Reply-To: <94bdd2610910161615h10ac409k210f4fbe88359624@mail.gmail.com> References: <94bdd2610910160654n351d786ap4366b6e9e99539dc@mail.gmail.com> <94bdd2610910161615h10ac409k210f4fbe88359624@mail.gmail.com> Message-ID: <97929ae1e24fc80e032ddc1b4fc4ea33@preisshare.net> On Sat, 17 Oct 2009 01:15:25 +0200, Tarek Ziad? wrote: > On Sat, Oct 17, 2009 at 12:39 AM, David Lyon > wrote: >> ?Requires: (linux-suse-kde-64) foo > > How this expression would be verified on the target system ? I'm working on coding it. Basically the strings are a in a list. # -- These strings describe different platforms standard_platform_bits = ('windows','linux','mac', 'xp','vista', 'os/x', '32','64', 'kde','gnome', 'wx', 'ubuntu','debian','suse','redhat','gentoo', 'centos','symbian' ) Then they go into a dictionary: platform_bits = {'windows' = False, 'linux' = False, 'mac' = False, 'xp' = False, 'vista' = False, 'os/x' = False, '32' = False, '64' = False, 'kde' = False, 'gnome' = False, 'wx' = False, 'gtk' = False, 'ubuntu' = False, 'debian' = False, 'suse' = False, 'redhat' = False, 'gentoo' = False, 'centos' = False, 'symbian = False' } # -- See what our platform is def build_platform_bit_map(): if sys.platform == 'darwin': self.platform_bits['mac'] = True # -- Further sub-bit determination .. else if sys.platform == 'win32': self.platform_bits['windows'] = True # -- Further sub-bit determination .. else if sys.platform == 'linux2': # -- Further sub-bit determination .. I'm working on the rest. David From ziade.tarek at gmail.com Sat Oct 17 01:53:48 2009 From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=) Date: Sat, 17 Oct 2009 01:53:48 +0200 Subject: [Distutils] [RFC] Recentering the static metadata need : PKG-INFO In-Reply-To: <97929ae1e24fc80e032ddc1b4fc4ea33@preisshare.net> References: <94bdd2610910160654n351d786ap4366b6e9e99539dc@mail.gmail.com> <94bdd2610910161615h10ac409k210f4fbe88359624@mail.gmail.com> <97929ae1e24fc80e032ddc1b4fc4ea33@preisshare.net> Message-ID: <94bdd2610910161653g7818c5e6jbb9c34b323583c61@mail.gmail.com> On Sat, Oct 17, 2009 at 1:38 AM, David Lyon wrote: > ? # -- See what our platform is > ? def build_platform_bit_map(): > > ? ? ? if sys.platform == 'darwin': > ? ? ? ? ? self.platform_bits['mac'] = True > > ? ? ? ? ? # -- Further sub-bit determination > ? ? ? ? ? .. > > ? ? ? else if sys.platform == 'win32': > ? ? ? ? ? self.platform_bits['windows'] = True > > ? ? ? ? ? # -- Further sub-bit determination > ? ? ? ? ? .. > ? ? ? else if sys.platform == 'linux2': > > ? ? ? ? ? # -- Further sub-bit determination So far.. so good. It's basically what we proposed earlier in PEP 390, but for the distribution, the desktop (kde, etc)... That would require a new stdlib module that would be very hard to implement and maintain since it's a fast moving ground. From ziade.tarek at gmail.com Sat Oct 17 01:57:11 2009 From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=) Date: Sat, 17 Oct 2009 01:57:11 +0200 Subject: [Distutils] Buildbot throwing a gear after default setup? In-Reply-To: <64017BB5-2954-4CC5-805C-12066B0127F1@gmail.com> References: <2E581399-E286-4C77-A787-30861315764E@gmail.com> <94bdd2610910161604t10b3cd0ame3dc324cd12a644b@mail.gmail.com> <31A858D3-D8B1-4B03-8D0D-906DA5DA46A2@gmail.com> <94bdd2610910161621t4a57dd0bv6d27ef45a676caba@mail.gmail.com> <94bdd2610910161626o64a1376an25af942da011f1d5@mail.gmail.com> <64017BB5-2954-4CC5-805C-12066B0127F1@gmail.com> Message-ID: <94bdd2610910161657i48f825dwcb2a7bb0732e617@mail.gmail.com> On Sat, Oct 17, 2009 at 1:39 AM, ssteinerX at gmail.com wrote: > > On Oct 16, 2009, at 7:26 PM, Tarek Ziad? wrote: > >> On Sat, Oct 17, 2009 at 1:21 AM, Tarek Ziad? >> wrote: >>> >>> IOW the way python is built in the bbot, is now using your system's >>> zlib support... >> >> Arggg... I am the typo master... >> >> s/now/not. >> >> So as Tres said maybe Ubuntu ships with a Python with zlib support, >> but doesn't provide zlib headers. > > Ok, so the python you build from the trunk checkout doesn't have zlib > support. Yup. > > I couldn't find an apt-get'able one, it looks like you have to build from > source, but the lack of zlib support should be picked up before the unit > tests fail. Yes that's what I've fixed, so it should not fail anymore. I'll digg in zlib at ubuntu support in any case. From ssteinerx at gmail.com Sat Oct 17 02:06:50 2009 From: ssteinerx at gmail.com (ssteinerX@gmail.com) Date: Fri, 16 Oct 2009 20:06:50 -0400 Subject: [Distutils] Buildbot throwing a gear after default setup? In-Reply-To: <94bdd2610910161657i48f825dwcb2a7bb0732e617@mail.gmail.com> References: <2E581399-E286-4C77-A787-30861315764E@gmail.com> <94bdd2610910161604t10b3cd0ame3dc324cd12a644b@mail.gmail.com> <31A858D3-D8B1-4B03-8D0D-906DA5DA46A2@gmail.com> <94bdd2610910161621t4a57dd0bv6d27ef45a676caba@mail.gmail.com> <94bdd2610910161626o64a1376an25af942da011f1d5@mail.gmail.com> <64017BB5-2954-4CC5-805C-12066B0127F1@gmail.com> <94bdd2610910161657i48f825dwcb2a7bb0732e617@mail.gmail.com> Message-ID: <4F29E48E-6F9F-4435-B5DB-4AE9331416CE@gmail.com> On Oct 16, 2009, at 7:57 PM, Tarek Ziad? wrote: > On Sat, Oct 17, 2009 at 1:39 AM, ssteinerX at gmail.com > wrote: >> >> On Oct 16, 2009, at 7:26 PM, Tarek Ziad? wrote: >> >>> On Sat, Oct 17, 2009 at 1:21 AM, Tarek Ziad? >>> wrote: >>>> >>>> IOW the way python is built in the bbot, is now using your system's >>>> zlib support... >>> >>> Arggg... I am the typo master... >>> >>> s/now/not. >>> >>> So as Tres said maybe Ubuntu ships with a Python with zlib support, >>> but doesn't provide zlib headers. >> >> Ok, so the python you build from the trunk checkout doesn't have zlib >> support. > > Yup. > >> >> I couldn't find an apt-get'able one, it looks like you have to >> build from >> source, but the lack of zlib support should be picked up before the >> unit >> tests fail. > > Yes that's what I've fixed, so it should not fail anymore. Where can I get that? I pulled into distutils-buildbot and nothing came down. > I'll digg in zlib at ubuntu support in any case. ssteiner at ubuntu:~/distutils-builbot$ aptitude search zlib p zlib-bin - compression library - sample programs p zlib-gst - Zlib bindings for GNU Smalltalk i zlib1g - compression library - runtime p zlib1g-dbg - compression library - development p zlib1g-dev - compression library - development p zlibc - An on- fly auto-uncompressing C library # sudo aptitude install zlib-bin zlib-dev zlib-dbg zlibc Not sure if they're all needed... l'll rebuild as soon as I know where to get the changes. S From reinout at vanrees.org Sat Oct 17 02:10:07 2009 From: reinout at vanrees.org (Reinout van Rees) Date: Sat, 17 Oct 2009 00:10:07 +0000 (UTC) Subject: [Distutils] Proposal for Distribute 0.7 References: <94bdd2610910150532x430ba857hf66afa19f8e90c70@mail.gmail.com> <20091015175042.16F173A4068@sparrow.telecommunity.com> <823FAD8D-F2FA-4D0E-9BF8-011AE4516ADC@gmail.com> <20091015195548.DCF8F3A4119@sparrow.telecommunity.com> <240b71640910151323m204e53a3n88b4174680b275c1@mail.gmail.com> <94bdd2610910151406u26453c8bk7e416f5b4a04714a@mail.gmail.com> <6D114E2B-C560-4264-A25C-980CCA340057@gmail.com> <94bdd2610910160124j28eb7703ma41273b10d2a3f5d@mail.gmail.com> <3E2051F7-E2CB-4B17-95FE-CAF64D02B8EF@gmail.com> <20091016154159.142273A4068@sparrow.telecommunity.com> Message-ID: On 2009-10-16, P.J. Eby wrote: > At 10:46 AM 10/16/2009 -0400, ssteinerX at gmail.com wrote: >> >>This is not designed by any objective standard I'd use. > > You are correct; setuptools itself was not particularly > designed. Eggs are, entry points are, a few other odds and ends were > designed, but setuptools itself (and *especially* easy_install) are a > collection of hacks upon hacks, mostly done in a rush to get > something out the door... which as soon as it became popular, locked > into a cycle of not having enough time to do improvements, because > maintenance was taking so much time. Ah, the perils of success :-) And you've had a lot of success with it. I still remember the more-or-less painful days of having to extract and install everything by hand. And I remember being yealous at perl's "oh, just install x y and z" dependency handling experience. Hurray for having eggs, pypi and friends! Reinout -- Reinout van Rees - reinout at vanrees.org - http://reinout.vanrees.org Software developer at http://www.thehealthagency.com "Military engineers build missiles. Civil engineers build targets" From ssteinerx at gmail.com Sat Oct 17 02:33:25 2009 From: ssteinerx at gmail.com (ssteinerX@gmail.com) Date: Fri, 16 Oct 2009 20:33:25 -0400 Subject: [Distutils] Proposal for Distribute 0.7 In-Reply-To: References: <94bdd2610910150532x430ba857hf66afa19f8e90c70@mail.gmail.com> <20091015175042.16F173A4068@sparrow.telecommunity.com> <823FAD8D-F2FA-4D0E-9BF8-011AE4516ADC@gmail.com> <20091015195548.DCF8F3A4119@sparrow.telecommunity.com> <240b71640910151323m204e53a3n88b4174680b275c1@mail.gmail.com> <94bdd2610910151406u26453c8bk7e416f5b4a04714a@mail.gmail.com> <6D114E2B-C560-4264-A25C-980CCA340057@gmail.com> <94bdd2610910160124j28eb7703ma41273b10d2a3f5d@mail.gmail.com> <3E2051F7-E2CB-4B17-95FE-CAF64D02B8EF@gmail.com> <20091016154159.142273A4068@sparrow.telecommunity.com> Message-ID: On Oct 16, 2009, at 8:10 PM, Reinout van Rees wrote: > On 2009-10-16, P.J. Eby wrote: >> At 10:46 AM 10/16/2009 -0400, ssteinerX at gmail.com wrote: >>> >>> This is not designed by any objective standard I'd use. >> >> You are correct; setuptools itself was not particularly >> designed. Eggs are, entry points are, a few other odds and ends were >> designed, but setuptools itself (and *especially* easy_install) are a >> collection of hacks upon hacks, mostly done in a rush to get >> something out the door... which as soon as it became popular, locked >> into a cycle of not having enough time to do improvements, because >> maintenance was taking so much time. > > Ah, the perils of success :-) And you've had a lot of success with > it. I > still remember the more-or-less painful days of having to extract > and install > everything by hand. > > And I remember being yealous at perl's "oh, just install x y and z" > dependency > handling experience. > > Hurray for having eggs, pypi and friends! Here's to catching up with Perl and Ruby and maybe even passing them, finally. S From david.lyon at preisshare.net Sat Oct 17 03:03:55 2009 From: david.lyon at preisshare.net (David Lyon) Date: Fri, 16 Oct 2009 21:03:55 -0400 Subject: [Distutils] [RFC] Recentering the static metadata need : PKG-INFO In-Reply-To: <94bdd2610910161653g7818c5e6jbb9c34b323583c61@mail.gmail.com> References: <94bdd2610910160654n351d786ap4366b6e9e99539dc@mail.gmail.com> <94bdd2610910161615h10ac409k210f4fbe88359624@mail.gmail.com> <97929ae1e24fc80e032ddc1b4fc4ea33@preisshare.net> <94bdd2610910161653g7818c5e6jbb9c34b323583c61@mail.gmail.com> Message-ID: <8de86db5e4096f1bbafccfcfcaf81377@preisshare.net> On Sat, 17 Oct 2009 01:53:48 +0200, Tarek Ziad? wrote: > So far.. so good. It's basically what we proposed earlier in PEP 390, but > for > the distribution, the desktop (kde, etc)... .. > That would require a new > stdlib module that would be very hard to implement and maintain since > it's a fast moving ground. I didn't see your pep for a new hard to implement stdlib module.. What number PEP was that? :-) Jokes aside, these desktops are easy to check for with one line of python code. It's either an environment variable or a configuration file. For example: if os.environ['KDEDIR'] <> '': That's a pretty good check to see if KDE is installed and running and won't bomb out, even if you try running it on windows. Most of these environment variables haven't changed in the last 10 years or more. I'd assert that there are some very static environment variables in these linux window managers. Regards David From reinout at vanrees.org Sat Oct 17 04:03:10 2009 From: reinout at vanrees.org (Reinout van Rees) Date: Sat, 17 Oct 2009 02:03:10 +0000 (UTC) Subject: [Distutils] setuptools 0.6c10 delayed until Monday References: <20091016162748.7CA123A4068@sparrow.telecommunity.com> Message-ID: On 2009-10-16, P.J. Eby wrote: > > Also, this will give those who are using Distribute and do *not* wish > to use this release, a chance to upgrade to the latest version of > Distribute first. (Which, if I understand correctly, includes code > to prevent accidental re-installation of setuptools.) Correct. There's an "if" in the one or two places where a requirement is looked up. If the request is for "setuptools", distribute will reply with "here you go, here's distribute". So that ought to work. Related: I added the code that checks whether we're not requesting a 0.7-series setuptools. So if a 0.7a1 setuptools comes out and someone requires it, it will be installed: distribute won't fake a 0.7. And I added a similar check in the (from the top of my head) on_insert() method that raises an error if you try to install a 0.7 setuptools inside a distribute-handled working set (like you suggested, iirc). Reinout -- Reinout van Rees - reinout at vanrees.org - http://reinout.vanrees.org Software developer at http://www.thehealthagency.com "Military engineers build missiles. Civil engineers build targets" From tseaver at palladion.com Sat Oct 17 04:36:40 2009 From: tseaver at palladion.com (Tres Seaver) Date: Fri, 16 Oct 2009 22:36:40 -0400 Subject: [Distutils] Buildbot throwing a gear after default setup? In-Reply-To: <64017BB5-2954-4CC5-805C-12066B0127F1@gmail.com> References: <2E581399-E286-4C77-A787-30861315764E@gmail.com> <94bdd2610910161604t10b3cd0ame3dc324cd12a644b@mail.gmail.com> <31A858D3-D8B1-4B03-8D0D-906DA5DA46A2@gmail.com> <94bdd2610910161621t4a57dd0bv6d27ef45a676caba@mail.gmail.com> <94bdd2610910161626o64a1376an25af942da011f1d5@mail.gmail.com> <64017BB5-2954-4CC5-805C-12066B0127F1@gmail.com> Message-ID: <4AD92DB8.1080609@palladion.com> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 ssteinerX at gmail.com wrote: > On Oct 16, 2009, at 7:26 PM, Tarek Ziad? wrote: >> So as Tres said maybe Ubuntu ships with a Python with zlib support, >> but doesn't provide zlib headers. > > Ok, so the python you build from the trunk checkout doesn't have zlib > support. > > I couldn't find an apt-get'able one, it looks like you have to build > from source, but the lack of zlib support should be picked up before > the unit tests fail. $ cat /etc/lsb-release DISTRIB_ID=Ubuntu DISTRIB_RELEASE=9.04 DISTRIB_CODENAME=jaunty DISTRIB_DESCRIPTION="Ubuntu 9.04" $ apt-cache search zlib | grep ^zlib.*dev zlib1g-dbg - compression library - development zlib1g-dev - compression library - development Tres. - -- =================================================================== Tres Seaver +1 540-429-0999 tseaver at palladion.com Palladion Software "Excellence by Design" http://palladion.com -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iEYEARECAAYFAkrZLbQACgkQ+gerLs4ltQ6RTQCeIDYNZNJ/xp5owC4xva1wCtPP jqEAnAim9iziE6rXbHtoxvsy94ZUNis7 =MxaL -----END PGP SIGNATURE----- From ssteinerx at gmail.com Sat Oct 17 04:58:28 2009 From: ssteinerx at gmail.com (ssteinerX@gmail.com) Date: Fri, 16 Oct 2009 22:58:28 -0400 Subject: [Distutils] Buildbot throwing a gear after default setup? In-Reply-To: <4AD92DB8.1080609@palladion.com> References: <2E581399-E286-4C77-A787-30861315764E@gmail.com> <94bdd2610910161604t10b3cd0ame3dc324cd12a644b@mail.gmail.com> <31A858D3-D8B1-4B03-8D0D-906DA5DA46A2@gmail.com> <94bdd2610910161621t4a57dd0bv6d27ef45a676caba@mail.gmail.com> <94bdd2610910161626o64a1376an25af942da011f1d5@mail.gmail.com> <64017BB5-2954-4CC5-805C-12066B0127F1@gmail.com> <4AD92DB8.1080609@palladion.com> Message-ID: <48011B84-CB7C-4F4D-AE6D-2D95287D31E6@gmail.com> Yes, I found those and installed -- not sure if the message went to the list, too. I'm waiting on how to get updated code from Tarek to re-run the whole thing. Thanks, S On Oct 16, 2009, at 10:36 PM, Tres Seaver wrote: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > ssteinerX at gmail.com wrote: >> On Oct 16, 2009, at 7:26 PM, Tarek Ziad? wrote: > >>> So as Tres said maybe Ubuntu ships with a Python with zlib support, >>> but doesn't provide zlib headers. >> >> Ok, so the python you build from the trunk checkout doesn't have zlib >> support. >> >> I couldn't find an apt-get'able one, it looks like you have to build >> from source, but the lack of zlib support should be picked up before >> the unit tests fail. > > $ cat /etc/lsb-release > DISTRIB_ID=Ubuntu > DISTRIB_RELEASE=9.04 > DISTRIB_CODENAME=jaunty > DISTRIB_DESCRIPTION="Ubuntu 9.04" > $ apt-cache search zlib | grep ^zlib.*dev > zlib1g-dbg - compression library - development > zlib1g-dev - compression library - development > > > Tres. > - -- > =================================================================== > Tres Seaver +1 540-429-0999 tseaver at palladion.com > Palladion Software "Excellence by Design" http://palladion.com > -----BEGIN PGP SIGNATURE----- > Version: GnuPG v1.4.9 (GNU/Linux) > Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org > > iEYEARECAAYFAkrZLbQACgkQ+gerLs4ltQ6RTQCeIDYNZNJ/xp5owC4xva1wCtPP > jqEAnAim9iziE6rXbHtoxvsy94ZUNis7 > =MxaL > -----END PGP SIGNATURE----- From ziade.tarek at gmail.com Sat Oct 17 10:34:10 2009 From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=) Date: Sat, 17 Oct 2009 10:34:10 +0200 Subject: [Distutils] Buildbot throwing a gear after default setup? In-Reply-To: <48011B84-CB7C-4F4D-AE6D-2D95287D31E6@gmail.com> References: <2E581399-E286-4C77-A787-30861315764E@gmail.com> <94bdd2610910161604t10b3cd0ame3dc324cd12a644b@mail.gmail.com> <31A858D3-D8B1-4B03-8D0D-906DA5DA46A2@gmail.com> <94bdd2610910161621t4a57dd0bv6d27ef45a676caba@mail.gmail.com> <94bdd2610910161626o64a1376an25af942da011f1d5@mail.gmail.com> <64017BB5-2954-4CC5-805C-12066B0127F1@gmail.com> <4AD92DB8.1080609@palladion.com> <48011B84-CB7C-4F4D-AE6D-2D95287D31E6@gmail.com> Message-ID: <94bdd2610910170134m225d07d4y8bcdc31e8906f898@mail.gmail.com> On Sat, Oct 17, 2009 at 4:58 AM, ssteinerX at gmail.com wrote: > Yes, I found those and installed -- not sure if the message went to the > list, too. > > I'm waiting on how to get updated code from Tarek to re-run the whole thing. > Hey, My update was done on python trunk. It's a buildbot so once your buildout is created, (python bootstrap.py && bin/buildout) you have new binaries in bin/: - one to launch the buildbot server (bin/master) - one to launch the buildbot slave on your machine (bin/ziade.org) once they are launched (daemons), you can go to http://localhost:9080/. The slave will automatically run the tests by: - doing "svn up" on the python trunk (that's where you will get the change I've done on Python trunk) - building python (configure + make) - run the python tests via bin/test Let me know how it goes. From ziade.tarek at gmail.com Sat Oct 17 10:54:44 2009 From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=) Date: Sat, 17 Oct 2009 10:54:44 +0200 Subject: [Distutils] Buildbot throwing a gear after default setup? In-Reply-To: <94bdd2610910170134m225d07d4y8bcdc31e8906f898@mail.gmail.com> References: <2E581399-E286-4C77-A787-30861315764E@gmail.com> <94bdd2610910161604t10b3cd0ame3dc324cd12a644b@mail.gmail.com> <31A858D3-D8B1-4B03-8D0D-906DA5DA46A2@gmail.com> <94bdd2610910161621t4a57dd0bv6d27ef45a676caba@mail.gmail.com> <94bdd2610910161626o64a1376an25af942da011f1d5@mail.gmail.com> <64017BB5-2954-4CC5-805C-12066B0127F1@gmail.com> <4AD92DB8.1080609@palladion.com> <48011B84-CB7C-4F4D-AE6D-2D95287D31E6@gmail.com> <94bdd2610910170134m225d07d4y8bcdc31e8906f898@mail.gmail.com> Message-ID: <94bdd2610910170154p69e86ed2xd5a08f3c7b937c74@mail.gmail.com> On Sat, Oct 17, 2009 at 10:34 AM, Tarek Ziad? wrote: > > It's a buildbot so once your buildout is created, (python bootstrap.py > && bin/buildout) precisely: python bootstrap.py && bin/buildout -c buildbot.cfg From southpolepen at hotmail.com Sat Oct 17 13:45:39 2009 From: southpolepen at hotmail.com (james27) Date: Sat, 17 Oct 2009 04:45:39 -0700 (PDT) Subject: [Distutils] making windows exe file on linux Message-ID: <25937633.post@talk.nabble.com> hello im new to cx_freeze . and also new to python. i want to make windows exe executable file on linux. i was searched much of time.but i couldn't found solution. is it possible to make windows exe file by use cx_freeze on linux? or there is some other solution is exist? if anyone can help ,much appreciate. thanks in advance. paul. -- View this message in context: http://www.nabble.com/making-windows-exe-file-on-linux-tp25937633p25937633.html Sent from the Python - distutils-sig mailing list archive at Nabble.com. From ssteinerx at gmail.com Sat Oct 17 16:47:51 2009 From: ssteinerx at gmail.com (ssteinerX@gmail.com) Date: Sat, 17 Oct 2009 10:47:51 -0400 Subject: [Distutils] Buildbot throwing a gear after default setup? In-Reply-To: <94bdd2610910170134m225d07d4y8bcdc31e8906f898@mail.gmail.com> References: <2E581399-E286-4C77-A787-30861315764E@gmail.com> <94bdd2610910161604t10b3cd0ame3dc324cd12a644b@mail.gmail.com> <31A858D3-D8B1-4B03-8D0D-906DA5DA46A2@gmail.com> <94bdd2610910161621t4a57dd0bv6d27ef45a676caba@mail.gmail.com> <94bdd2610910161626o64a1376an25af942da011f1d5@mail.gmail.com> <64017BB5-2954-4CC5-805C-12066B0127F1@gmail.com> <4AD92DB8.1080609@palladion.com> <48011B84-CB7C-4F4D-AE6D-2D95287D31E6@gmail.com> <94bdd2610910170134m225d07d4y8bcdc31e8906f898@mail.gmail.com> Message-ID: <531A9A2F-1D8E-4704-B1E6-FD71AB07B006@gmail.com> On Oct 17, 2009, at 4:34 AM, Tarek Ziad? wrote: > On Sat, Oct 17, 2009 at 4:58 AM, ssteinerX at gmail.com > wrote: >> Yes, I found those and installed -- not sure if the message went to >> the >> list, too. >> >> I'm waiting on how to get updated code from Tarek to re-run the >> whole thing. >> > > Hey, > > My update was done on python trunk. Ah, wasn't sure where to get it; I wasn't sure whether you'd changed the buildbot code on bitbucket or something else. > It's a buildbot so once your buildout is created, (python bootstrap.py > && bin/buildout) > you have new binaries in bin/: > > - one to launch the buildbot server (bin/master) > - one to launch the buildbot slave on your machine (bin/ziade.org) > > once they are launched (daemons), you can go to http://localhost: > 9080/. > > The slave will automatically run the tests by: > > - doing "svn up" on the python trunk (that's where you will get the > change I've done on Python trunk) > - building python (configure + make) > - run the python tests via bin/test bin/test was what complained about missing zlib. I ran it by hand to see if it was working. > Let me know how it goes. I thought I had apt-get installed all the zlib components so I wasn't expecting problems but I guess not. It now throws a different gear in a different way, so I guess I got your changes :-) Creating tar archive Traceback (most recent call last): File "setup.py", line 15, in packages=['extensions', 'extensions.command'] File "/home/ssteiner/distutils-builbot/parts/trunk-checkout/python/ Lib/distutils/core.py", line 149, in setup dist.run_commands() File "/home/ssteiner/distutils-builbot/parts/trunk-checkout/python/ Lib/distutils/dist.py", line 926, in run_commands self.run_command(cmd) File "/home/ssteiner/distutils-builbot/parts/trunk-checkout/python/ Lib/distutils/dist.py", line 945, in run_command cmd_obj.run() File "/home/ssteiner/distutils-builbot/parts/trunk-checkout/python/ Lib/distutils/command/sdist.py", line 166, in run self.make_distribution() File "/home/ssteiner/distutils-builbot/parts/trunk-checkout/python/ Lib/distutils/command/sdist.py", line 465, in make_distribution owner=self.owner, group=self.group) File "/home/ssteiner/distutils-builbot/parts/trunk-checkout/python/ Lib/distutils/cmd.py", line 392, in make_archive owner=owner, group=group) File "/home/ssteiner/distutils-builbot/parts/trunk-checkout/python/ Lib/distutils/archive_util.py", line 236, in make_archive filename = func(base_name, base_dir, **kwargs) File "/home/ssteiner/distutils-builbot/parts/trunk-checkout/python/ Lib/distutils/archive_util.py", line 101, in make_tarball tar = tarfile.open(archive_name, 'w|%s' % tar_compression [compress]) File "/home/ssteiner/distutils-builbot/parts/trunk-checkout/python/ Lib/tarfile.py", line 1668, in open _Stream(name, filemode, comptype, fileobj, bufsize), File "/home/ssteiner/distutils-builbot/parts/trunk-checkout/python/ Lib/tarfile.py", line 417, in __init__ raise CompressionError("zlib module is not available") tarfile.CompressionError: zlib module is not available Exception AttributeError: "_Stream instance has no attribute 'cmp'" in > ignored Could not create distro with /home/ssteiner/distutils-builbot/parts/ trunk-checkout/python/python2.7 From ssteinerx at gmail.com Sat Oct 17 16:49:55 2009 From: ssteinerx at gmail.com (ssteinerX@gmail.com) Date: Sat, 17 Oct 2009 10:49:55 -0400 Subject: [Distutils] Buildbot throwing a gear after default setup? In-Reply-To: <94bdd2610910170154p69e86ed2xd5a08f3c7b937c74@mail.gmail.com> References: <2E581399-E286-4C77-A787-30861315764E@gmail.com> <94bdd2610910161604t10b3cd0ame3dc324cd12a644b@mail.gmail.com> <31A858D3-D8B1-4B03-8D0D-906DA5DA46A2@gmail.com> <94bdd2610910161621t4a57dd0bv6d27ef45a676caba@mail.gmail.com> <94bdd2610910161626o64a1376an25af942da011f1d5@mail.gmail.com> <64017BB5-2954-4CC5-805C-12066B0127F1@gmail.com> <4AD92DB8.1080609@palladion.com> <48011B84-CB7C-4F4D-AE6D-2D95287D31E6@gmail.com> <94bdd2610910170134m225d07d4y8bcdc31e8906f898@mail.gmail.com> <94bdd2610910170154p69e86ed2xd5a08f3c7b937c74@mail.gmail.com> Message-ID: <34B8FB50-604B-41FA-82CC-C08DEEC6FC42@gmail.com> On Oct 17, 2009, at 4:54 AM, Tarek Ziad? wrote: > On Sat, Oct 17, 2009 at 10:34 AM, Tarek Ziad? > wrote: >> >> It's a buildbot so once your buildout is created, (python >> bootstrap.py >> && bin/buildout) > > precisely: > python bootstrap.py && bin/buildout -c buildbot.cfg ssteiner at ubuntu:~/distutils-builbot$ python bootstrap.py && bin/ buildout -c buildbot.cfg Downloading http://pypi.python.org/packages/2.6/s/setuptools/setuptools-0.6c9-py2.6.egg Getting distribution for 'Twisted'. twisted/runner/portmap.c:10:20: error: Python.h: No such file or directory twisted/runner/portmap.c:14: error: expected ?=?, ?,?, ?;?, ?asm? or ?__attribute__? before ?*? token twisted/runner/portmap.c:31: error: expected ?=?, ?,?, ?;?, ?asm? or ?__attribute__? before ?*? token twisted/runner/portmap.c:45: error: expected ?=?, ?,?, ?;?, ?asm? or ?__attribute__? before ?PortmapMethods? twisted/runner/portmap.c: In function ?initportmap?: twisted/runner/portmap.c:55: warning: implicit declaration of function ?Py_InitModule? twisted/runner/portmap.c:55: error: ?PortmapMethods? undeclared (first use in this function) twisted/runner/portmap.c:55: error: (Each undeclared identifier is reported only once twisted/runner/portmap.c:55: error: for each function it appears in.) error: Setup script exited with error: command 'gcc' failed with exit status 1 An error occured when trying to install Twisted 8.2.0.Look above this message for any errors thatwere output by easy_install. While: Installing. Getting section master. Initializing section master. Installing recipe collective.buildbot. Getting distribution for 'Twisted'. Error: Couldn't install: Twisted 8.2.0 From ziade.tarek at gmail.com Sat Oct 17 18:19:44 2009 From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=) Date: Sat, 17 Oct 2009 18:19:44 +0200 Subject: [Distutils] [RFC] Recentering the static metadata need : PKG-INFO In-Reply-To: <8de86db5e4096f1bbafccfcfcaf81377@preisshare.net> References: <94bdd2610910160654n351d786ap4366b6e9e99539dc@mail.gmail.com> <94bdd2610910161615h10ac409k210f4fbe88359624@mail.gmail.com> <97929ae1e24fc80e032ddc1b4fc4ea33@preisshare.net> <94bdd2610910161653g7818c5e6jbb9c34b323583c61@mail.gmail.com> <8de86db5e4096f1bbafccfcfcaf81377@preisshare.net> Message-ID: <94bdd2610910170919j32eabf09rf3d15850fd1b64d3@mail.gmail.com> On Sat, Oct 17, 2009 at 3:03 AM, David Lyon wrote: > On Sat, 17 Oct 2009 01:53:48 +0200, Tarek Ziad? > wrote: > >> So far.. so good. It's basically what we proposed earlier in PEP 390, but >> for >> the distribution, the desktop (kde, etc)... > .. >> That would require a new >> stdlib module that would be very hard to implement and maintain since >> it's a fast moving ground. > > I didn't see your pep for a new hard to implement stdlib module.. > > What number PEP was that? :-) There's no PEP for that because I don't want to add such a module. The context dependant marker are just relying on os, platform and sys. > It's either an environment variable or a configuration file. > > For example: > > ?if os.environ['KDEDIR'] <> '': > > That's a pretty good check to see if KDE is installed and running > and won't bomb out, even if you try running it on windows. > > Most of these environment variables haven't changed in the last > 10 years or more. I'd assert that there are some very static > environment variables in these linux window managers. If you can find a way to express conditions based on os.environ, maybe os.environ could be added to the variables that are used when checking the markers on a given platform. e.g. like = 'KDEDIR' in os.environ (so meaning we introduce sequence comparisons/operators in those markers) But we can't possibly add in the PKG-INFO description, a hardcoded list of plaftorms like you have shown (kde, ubuntu, etc). Do you have an example for a platform that would depend on a configuration file ? Tarek. From ziade.tarek at gmail.com Sat Oct 17 18:23:00 2009 From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=) Date: Sat, 17 Oct 2009 18:23:00 +0200 Subject: [Distutils] Buildbot throwing a gear after default setup? In-Reply-To: <531A9A2F-1D8E-4704-B1E6-FD71AB07B006@gmail.com> References: <2E581399-E286-4C77-A787-30861315764E@gmail.com> <94bdd2610910161604t10b3cd0ame3dc324cd12a644b@mail.gmail.com> <31A858D3-D8B1-4B03-8D0D-906DA5DA46A2@gmail.com> <94bdd2610910161621t4a57dd0bv6d27ef45a676caba@mail.gmail.com> <94bdd2610910161626o64a1376an25af942da011f1d5@mail.gmail.com> <64017BB5-2954-4CC5-805C-12066B0127F1@gmail.com> <4AD92DB8.1080609@palladion.com> <48011B84-CB7C-4F4D-AE6D-2D95287D31E6@gmail.com> <94bdd2610910170134m225d07d4y8bcdc31e8906f898@mail.gmail.com> <531A9A2F-1D8E-4704-B1E6-FD71AB07B006@gmail.com> Message-ID: <94bdd2610910170923m7b0b9b97s375eb346148cda53@mail.gmail.com> On Sat, Oct 17, 2009 at 4:47 PM, ssteinerX at gmail.com wrote: > >> Let me know how it goes. > > I thought I had apt-get installed all the zlib components so I wasn't > expecting problems but I guess not. ?It now throws a different gear in a > different way, so I guess I got your changes :-) > > Creating tar archive > Traceback (most recent call last): > ?File "setup.py", line 15, in > ? ?packages=['extensions', 'extensions.command'] > ?File Yes, unfortunaltely this buildbot step requires zlib to build a tar.gz... What we can do is change this test here (on distutils-buildbot side) so all tests are made with another archive format. Tarek From ziade.tarek at gmail.com Sat Oct 17 18:24:49 2009 From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=) Date: Sat, 17 Oct 2009 18:24:49 +0200 Subject: [Distutils] Buildbot throwing a gear after default setup? In-Reply-To: <34B8FB50-604B-41FA-82CC-C08DEEC6FC42@gmail.com> References: <2E581399-E286-4C77-A787-30861315764E@gmail.com> <31A858D3-D8B1-4B03-8D0D-906DA5DA46A2@gmail.com> <94bdd2610910161621t4a57dd0bv6d27ef45a676caba@mail.gmail.com> <94bdd2610910161626o64a1376an25af942da011f1d5@mail.gmail.com> <64017BB5-2954-4CC5-805C-12066B0127F1@gmail.com> <4AD92DB8.1080609@palladion.com> <48011B84-CB7C-4F4D-AE6D-2D95287D31E6@gmail.com> <94bdd2610910170134m225d07d4y8bcdc31e8906f898@mail.gmail.com> <94bdd2610910170154p69e86ed2xd5a08f3c7b937c74@mail.gmail.com> <34B8FB50-604B-41FA-82CC-C08DEEC6FC42@gmail.com> Message-ID: <94bdd2610910170924y3cf50526k8e8a9f0d81079802@mail.gmail.com> On Sat, Oct 17, 2009 at 4:49 PM, ssteinerX at gmail.com wrote: >> >> precisely: >> python bootstrap.py && bin/buildout -c buildbot.cfg > > ssteiner at ubuntu:~/distutils-builbot$ python bootstrap.py && bin/buildout -c > buildbot.cfg > Downloading > http://pypi.python.org/packages/2.6/s/setuptools/setuptools-0.6c9-py2.6.egg > Getting distribution for 'Twisted'. > twisted/runner/portmap.c:10:20: error: Python.h: No such file or directory Buildbot needs Twisted, so it needs the 'python-dev' to compile on ubuntu. $ apt-get install python-dev $ apt-get install build-essentials (should be enough) Tarek. From pje at telecommunity.com Sat Oct 17 20:38:54 2009 From: pje at telecommunity.com (P.J. Eby) Date: Sat, 17 Oct 2009 14:38:54 -0400 Subject: [Distutils] setuptools 0.6c10 delayed until Monday In-Reply-To: References: <20091016162748.7CA123A4068@sparrow.telecommunity.com> Message-ID: <20091017183859.AACCA3A4068@sparrow.telecommunity.com> At 02:03 AM 10/17/2009 +0000, Reinout van Rees wrote: >I added the code that checks whether we're not requesting a 0.7-series >setuptools. So if a 0.7a1 setuptools comes out and someone requires it, it >will be installed: distribute won't fake a 0.7. And I added a similar check >in the (from the top of my head) on_insert() method that raises an error if >you try to install a 0.7 setuptools inside a distribute-handled working set >(like you suggested, iirc). Yep. Thanks! From ssteinerx at gmail.com Sat Oct 17 20:45:07 2009 From: ssteinerx at gmail.com (ssteinerX@gmail.com) Date: Sat, 17 Oct 2009 14:45:07 -0400 Subject: [Distutils] Buildbot throwing a gear after default setup? In-Reply-To: <94bdd2610910170924y3cf50526k8e8a9f0d81079802@mail.gmail.com> References: <2E581399-E286-4C77-A787-30861315764E@gmail.com> <31A858D3-D8B1-4B03-8D0D-906DA5DA46A2@gmail.com> <94bdd2610910161621t4a57dd0bv6d27ef45a676caba@mail.gmail.com> <94bdd2610910161626o64a1376an25af942da011f1d5@mail.gmail.com> <64017BB5-2954-4CC5-805C-12066B0127F1@gmail.com> <4AD92DB8.1080609@palladion.com> <48011B84-CB7C-4F4D-AE6D-2D95287D31E6@gmail.com> <94bdd2610910170134m225d07d4y8bcdc31e8906f898@mail.gmail.com> <94bdd2610910170154p69e86ed2xd5a08f3c7b937c74@mail.gmail.com> <34B8FB50-604B-41FA-82CC-C08DEEC6FC42@gmail.com> <94bdd2610910170924y3cf50526k8e8a9f0d81079802@mail.gmail.com> Message-ID: On Oct 17, 2009, at 12:24 PM, Tarek Ziad? wrote: > On Sat, Oct 17, 2009 at 4:49 PM, ssteinerX at gmail.com > wrote: >>> >>> precisely: >>> python bootstrap.py && bin/buildout -c buildbot.cfg >> >> ssteiner at ubuntu:~/distutils-builbot$ python bootstrap.py && bin/ >> buildout -c >> buildbot.cfg >> Downloading >> http://pypi.python.org/packages/2.6/s/setuptools/setuptools-0.6c9-py2.6.egg >> Getting distribution for 'Twisted'. >> twisted/runner/portmap.c:10:20: error: Python.h: No such file or >> directory > > Buildbot needs Twisted, so it needs the 'python-dev' to compile on > ubuntu. > > $ apt-get install python-dev > $ apt-get install build-essentials Is there an easy way of getting bootstrap.py to run these (and get Zlib) on Ubuntu since they're not installed on a default image? S From ssteinerx at gmail.com Sat Oct 17 20:48:28 2009 From: ssteinerx at gmail.com (ssteinerX@gmail.com) Date: Sat, 17 Oct 2009 14:48:28 -0400 Subject: [Distutils] Buildbot throwing a gear after default setup? In-Reply-To: <94bdd2610910170924y3cf50526k8e8a9f0d81079802@mail.gmail.com> References: <2E581399-E286-4C77-A787-30861315764E@gmail.com> <31A858D3-D8B1-4B03-8D0D-906DA5DA46A2@gmail.com> <94bdd2610910161621t4a57dd0bv6d27ef45a676caba@mail.gmail.com> <94bdd2610910161626o64a1376an25af942da011f1d5@mail.gmail.com> <64017BB5-2954-4CC5-805C-12066B0127F1@gmail.com> <4AD92DB8.1080609@palladion.com> <48011B84-CB7C-4F4D-AE6D-2D95287D31E6@gmail.com> <94bdd2610910170134m225d07d4y8bcdc31e8906f898@mail.gmail.com> <94bdd2610910170154p69e86ed2xd5a08f3c7b937c74@mail.gmail.com> <34B8FB50-604B-41FA-82CC-C08DEEC6FC42@gmail.com> <94bdd2610910170924y3cf50526k8e8a9f0d81079802@mail.gmail.com> Message-ID: On Oct 17, 2009, at 12:24 PM, Tarek Ziad? wrote: > On Sat, Oct 17, 2009 at 4:49 PM, ssteinerX at gmail.com > wrote: >>> >>> precisely: >>> python bootstrap.py && bin/buildout -c buildbot.cfg >> >> ssteiner at ubuntu:~/distutils-builbot$ python bootstrap.py && bin/ >> buildout -c >> buildbot.cfg >> Downloading >> http://pypi.python.org/packages/2.6/s/setuptools/setuptools-0.6c9-py2.6.egg >> Getting distribution for 'Twisted'. >> twisted/runner/portmap.c:10:20: error: Python.h: No such file or >> directory > > Buildbot needs Twisted, so it needs the 'python-dev' to compile on > ubuntu. > > $ apt-get install python-dev > $ apt-get install build-essentials > (should be enough) That got me further (through installing Twisted) but now: # ssteiner at ubuntu:~/distutils-builbot$ python bootstrap.py && bin/ buildout -c buildbot.cfg Downloading http://pypi.python.org/packages/2.6/s/setuptools/setuptools-0.6c9-py2.6.egg Getting distribution for 'Twisted'. [Twisted builds...] Got Twisted 8.2.0. Getting distribution for 'virtualenv'. Got virtualenv 1.3.4. Getting distribution for 'zc.recipe.egg'. Got zc.recipe.egg 1.2.2. Getting distribution for 'PasteDeploy'. warning: no files found matching 'docs/*.html' warning: no previously-included files found matching 'docs/rebuild' Got PasteDeploy 1.3.3. Getting distribution for 'Paste>=1.3'. Got Paste 1.7.2. While: Installing. Getting section master. Initializing section master. Loading zc.buildout recipe entry collective.buildbot:master. An internal error occured due to a bug in either zc.buildout or in a recipe being used: Traceback (most recent call last): File "/home/ssteiner/distutils-builbot/eggs/zc.buildout-1.4.1- py2.6.egg/zc/buildout/buildout.py", line 1659, in main getattr(buildout, command)(args) File "/home/ssteiner/distutils-builbot/eggs/zc.buildout-1.4.1- py2.6.egg/zc/buildout/buildout.py", line 416, in install [self[part]['recipe'] for part in install_parts] File "/home/ssteiner/distutils-builbot/eggs/zc.buildout-1.4.1- py2.6.egg/zc/buildout/buildout.py", line 963, in __getitem__ options._initialize() File "/home/ssteiner/distutils-builbot/eggs/zc.buildout-1.4.1- py2.6.egg/zc/buildout/buildout.py", line 1047, in _initialize recipe_class = _install_and_load(reqs, 'zc.buildout', entry, buildout) File "/home/ssteiner/distutils-builbot/eggs/zc.buildout-1.4.1- py2.6.egg/zc/buildout/buildout.py", line 1008, in _install_and_load req.project_name, group, entry) File "/home/ssteiner/distutils-builbot/eggs/setuptools-0.6c9- py2.6.egg/pkg_resources.py", line 277, in load_entry_point return get_distribution(dist).load_entry_point(group, name) File "/home/ssteiner/distutils-builbot/eggs/setuptools-0.6c9- py2.6.egg/pkg_resources.py", line 2180, in load_entry_point return ep.load() File "/home/ssteiner/distutils-builbot/eggs/setuptools-0.6c9- py2.6.egg/pkg_resources.py", line 1913, in load entry = __import__(self.module_name, globals(),globals(), ['__name__']) File "/home/ssteiner/distutils-builbot/eggs/ collective.buildbot-0.3.5-py2.6.egg/collective/buildbot/__init__.py", line 48, in from buildbot.steps.shell import WarningCountingShellCommand File "/usr/local/lib/python2.6/dist-packages/buildbot/steps/ shell.py", line 5, in from buildbot.process.buildstep import LoggingBuildStep, RemoteShellCommand File "/usr/local/lib/python2.6/dist-packages/buildbot/process/ buildstep.py", line 9, in from twisted.web.util import formatFailure ImportError: No module named web.util ssteiner at ubuntu:~/distutils-builbot$ I guess it's a good thing we're doing this! ;-O S From ssteinerx at gmail.com Sat Oct 17 20:53:31 2009 From: ssteinerx at gmail.com (ssteinerX@gmail.com) Date: Sat, 17 Oct 2009 14:53:31 -0400 Subject: [Distutils] Buildbot throwing a gear after default setup? In-Reply-To: <94bdd2610910170924y3cf50526k8e8a9f0d81079802@mail.gmail.com> References: <2E581399-E286-4C77-A787-30861315764E@gmail.com> <31A858D3-D8B1-4B03-8D0D-906DA5DA46A2@gmail.com> <94bdd2610910161621t4a57dd0bv6d27ef45a676caba@mail.gmail.com> <94bdd2610910161626o64a1376an25af942da011f1d5@mail.gmail.com> <64017BB5-2954-4CC5-805C-12066B0127F1@gmail.com> <4AD92DB8.1080609@palladion.com> <48011B84-CB7C-4F4D-AE6D-2D95287D31E6@gmail.com> <94bdd2610910170134m225d07d4y8bcdc31e8906f898@mail.gmail.com> <94bdd2610910170154p69e86ed2xd5a08f3c7b937c74@mail.gmail.com> <34B8FB50-604B-41FA-82CC-C08DEEC6FC42@gmail.com> <94bdd2610910170924y3cf50526k8e8a9f0d81079802@mail.gmail.com> Message-ID: On Oct 17, 2009, at 12:24 PM, Tarek Ziad? wrote: Just to be clear...I'm not just giving up when the build falls down, I just do want to make sure to report everything I do so that we can get the buildbot to configure whatever's not there automatically instead of having to do anything manually. We want to be able to fire these up, on a remote, freshly minted Cloud Server, and have it pull in everything it needs to run with no operator intervention, run tests, and self-destruct all within less than an hour (i.e. 1.5 cents worth of compute power). Right? S P.S. To get Twisted to build, I had to: # sudo apt-get install zlib1g-dev zlib1g-dev s From ziade.tarek at gmail.com Sat Oct 17 21:38:57 2009 From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=) Date: Sat, 17 Oct 2009 21:38:57 +0200 Subject: [Distutils] Buildbot throwing a gear after default setup? In-Reply-To: References: <2E581399-E286-4C77-A787-30861315764E@gmail.com> <94bdd2610910161626o64a1376an25af942da011f1d5@mail.gmail.com> <64017BB5-2954-4CC5-805C-12066B0127F1@gmail.com> <4AD92DB8.1080609@palladion.com> <48011B84-CB7C-4F4D-AE6D-2D95287D31E6@gmail.com> <94bdd2610910170134m225d07d4y8bcdc31e8906f898@mail.gmail.com> <94bdd2610910170154p69e86ed2xd5a08f3c7b937c74@mail.gmail.com> <34B8FB50-604B-41FA-82CC-C08DEEC6FC42@gmail.com> <94bdd2610910170924y3cf50526k8e8a9f0d81079802@mail.gmail.com> Message-ID: <94bdd2610910171238l5a966a02vcb21d8670746f785@mail.gmail.com> On Sat, Oct 17, 2009 at 8:53 PM, ssteinerX at gmail.com wrote: > > On Oct 17, 2009, at 12:24 PM, Tarek Ziad? wrote: > > Just to be clear...I'm not just giving up when the build falls down, I just > do want to make sure to report everything I do so that we can get the > buildbot to configure whatever's not there automatically instead of having > to do anything manually. > > We want to be able to fire these up, on a remote, freshly minted Cloud > Server, and have it pull in everything it needs to run with no operator > intervention, run tests, and self-destruct all within less than an hour > (i.e. 1.5 cents worth of compute power). > > Right? Right, but my buildbot made some assumptions on what's required on the platform (you noticed ;) ) Now I don't understand your last error from twisted.web.util import formatFailure > ImportError: No module named web.util Since Twisted seemed to be added correctly I think we should meet online at #distutils to tackle this. Tarek From exarkun at twistedmatrix.com Sat Oct 17 21:47:58 2009 From: exarkun at twistedmatrix.com (exarkun at twistedmatrix.com) Date: Sat, 17 Oct 2009 19:47:58 -0000 Subject: [Distutils] Buildbot throwing a gear after default setup? In-Reply-To: <94bdd2610910170924y3cf50526k8e8a9f0d81079802@mail.gmail.com> References: <2E581399-E286-4C77-A787-30861315764E@gmail.com> <31A858D3-D8B1-4B03-8D0D-906DA5DA46A2@gmail.com> <94bdd2610910161621t4a57dd0bv6d27ef45a676caba@mail.gmail.com> <94bdd2610910161626o64a1376an25af942da011f1d5@mail.gmail.com> <64017BB5-2954-4CC5-805C-12066B0127F1@gmail.com> <4AD92DB8.1080609@palladion.com> <48011B84-CB7C-4F4D-AE6D-2D95287D31E6@gmail.com> <94bdd2610910170134m225d07d4y8bcdc31e8906f898@mail.gmail.com> <94bdd2610910170154p69e86ed2xd5a08f3c7b937c74@mail.gmail.com> <34B8FB50-604B-41FA-82CC-C08DEEC6FC42@gmail.com> <94bdd2610910170924y3cf50526k8e8a9f0d81079802@mail.gmail.com> Message-ID: <20091017194758.8004.158279606.divmod.xquotient.277@localhost.localdomain> On 04:24 pm, ziade.tarek at gmail.com wrote: >On Sat, Oct 17, 2009 at 4:49 PM, ssteinerX at gmail.com > wrote: >>> >>>precisely: >>>python bootstrap.py && bin/buildout -c buildbot.cfg >> >>ssteiner at ubuntu:~/distutils-builbot$ python bootstrap.py && >>bin/buildout -c >>buildbot.cfg >>Downloading >>http://pypi.python.org/packages/2.6/s/setuptools/setuptools-0.6c9-py2.6.egg >>Getting distribution for 'Twisted'. >>twisted/runner/portmap.c:10:20: error: Python.h: No such file or >>directory > >Buildbot needs Twisted, so it needs the 'python-dev' to compile on >ubuntu. > >$ apt-get install python-dev >$ apt-get install build-essentials >(should be enough) Almost all of Twisted's extension modules are optional (and portmap is hardly useful, I doubt anyone would miss it if it were gone). This failure reminds me that it would be nice if distutils had a way to express this. Jean-Paul From david.lyon at preisshare.net Sun Oct 18 00:00:51 2009 From: david.lyon at preisshare.net (David Lyon) Date: Sat, 17 Oct 2009 18:00:51 -0400 Subject: [Distutils] [RFC] Recentering the static metadata need : PKG-INFO In-Reply-To: <94bdd2610910170919j32eabf09rf3d15850fd1b64d3@mail.gmail.com> References: <94bdd2610910160654n351d786ap4366b6e9e99539dc@mail.gmail.com> <94bdd2610910161615h10ac409k210f4fbe88359624@mail.gmail.com> <97929ae1e24fc80e032ddc1b4fc4ea33@preisshare.net> <94bdd2610910161653g7818c5e6jbb9c34b323583c61@mail.gmail.com> <8de86db5e4096f1bbafccfcfcaf81377@preisshare.net> <94bdd2610910170919j32eabf09rf3d15850fd1b64d3@mail.gmail.com> Message-ID: <6d0a7802f78e2b87baa57bd048066201@preisshare.net> On Sat, 17 Oct 2009 18:19:44 +0200, Tarek Ziad? wrote: > If you can find a way to express conditions based on os.environ, > maybe os.environ could be added to the variables that are used > when checking the markers on a given platform. I'm not sure that you should try to do that. It would make things too complicated for the developer. I'm not sure that they would like to guess environment variables at an end-users system. It's different if the api does it internally. The developer then doesn't see how the check for kde or gnome is actually performed. > e.g. like = 'KDEDIR' in os.environ (so meaning we introduce sequence > comparisons/operators > in those markers) I suggest that you just add a variable for 'desktop' or something to your pep and allow them to check for that. I've told you an easy way to check (environment variables) so there's no reason you couldn't do it the same way. > Do you have an example for a platform that would depend on a > configuration file ? What I meant there was that desktops such as gnome or kde or flux or others can also be detected internally by the presence of their configuration files on linux. http://www.ibiblio.org/oswg/oswg-nightly/oswg/en_US.ISO_8859-1/articles/gdm-reference/gdm-reference/x135.html For example, if there is a /etc/gdm/gdm.conf it is pretty safe to assume that gnome is installed on that system. So in your PEP, your could have code.. if sys.platform == 'linux2': gnome_installed = os.path.exists('/etc/gdm/gdm.conf') Then exposed that as a variable in your condition: 'if desktop in "gnome|windows"' etc Regards David From ssteinerx at gmail.com Sun Oct 18 01:26:27 2009 From: ssteinerx at gmail.com (ssteinerX@gmail.com) Date: Sat, 17 Oct 2009 19:26:27 -0400 Subject: [Distutils] Buildbot throwing a gear after default setup? In-Reply-To: <20091017194758.8004.158279606.divmod.xquotient.277@localhost.localdomain> References: <2E581399-E286-4C77-A787-30861315764E@gmail.com> <31A858D3-D8B1-4B03-8D0D-906DA5DA46A2@gmail.com> <94bdd2610910161621t4a57dd0bv6d27ef45a676caba@mail.gmail.com> <94bdd2610910161626o64a1376an25af942da011f1d5@mail.gmail.com> <64017BB5-2954-4CC5-805C-12066B0127F1@gmail.com> <4AD92DB8.1080609@palladion.com> <48011B84-CB7C-4F4D-AE6D-2D95287D31E6@gmail.com> <94bdd2610910170134m225d07d4y8bcdc31e8906f898@mail.gmail.com> <94bdd2610910170154p69e86ed2xd5a08f3c7b937c74@mail.gmail.com> <34B8FB50-604B-41FA-82CC-C08DEEC6FC42@gmail.com> <94bdd2610910170924y3cf50526k8e8a9f0d81079802@mail.gmail.com> <20091017194758.8004.158279606.divmod.xquotient.277@localhost.localdomain> Message-ID: <0AF15530-3B1F-4D4B-AA34-1029CC0F34EF@gmail.com> On Oct 17, 2009, at 3:47 PM, exarkun at twistedmatrix.com wrote: > On 04:24 pm, ziade.tarek at gmail.com wrote: >> On Sat, Oct 17, 2009 at 4:49 PM, ssteinerX at gmail.com >> wrote: >>>> >>>> precisely: >>>> python bootstrap.py && bin/buildout -c buildbot.cfg >>> >>> ssteiner at ubuntu:~/distutils-builbot$ python bootstrap.py && bin/ >>> buildout -c >>> buildbot.cfg >>> Downloading >>> http://pypi.python.org/packages/2.6/s/setuptools/setuptools-0.6c9-py2.6.egg >>> Getting distribution for 'Twisted'. >>> twisted/runner/portmap.c:10:20: error: Python.h: No such file or >>> directory >> >> Buildbot needs Twisted, so it needs the 'python-dev' to compile on >> ubuntu. >> >> $ apt-get install python-dev >> $ apt-get install build-essentials >> (should be enough) > > Almost all of Twisted's extension modules are optional (and portmap > is hardly useful, I doubt anyone would miss it if it were gone). > This failure reminds me that it would be nice if distutils had a way > to express this. So, is this something I have to install in addition to the way Twisted's being installed now? I'm working on setting up the buildbot automatically on a cloud server and all these errors are what happen if you start from a plain-vanilla Ubuntu 9.0.4 and try to just bootstrap the distutils buildbot. S From ziade.tarek at gmail.com Sun Oct 18 15:35:21 2009 From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=) Date: Sun, 18 Oct 2009 15:35:21 +0200 Subject: [Distutils] Distribute 0.7.x first sprint In-Reply-To: <94bdd2610910160253k492cf739nc1e9d6dfc40e657e@mail.gmail.com> References: <94bdd2610910160253k492cf739nc1e9d6dfc40e657e@mail.gmail.com> Message-ID: <94bdd2610910180635q474f79ebq34025005552da2a6@mail.gmail.com> On Fri, Oct 16, 2009 at 11:53 AM, Tarek Ziad? wrote: > Hey > > While we are working at fixing bugs in 0.6.x, I would like to organize > a first online coding sprint for the 0.7.x series, > > goals: > - finalize the first splitting version > - define a more detailed roadmap for each splitted package > - build a better test environment (see if we can set up a buildbot) > > possible times: > - Sunday October 18th - 6 pm. GMT+2 > - Tuesday October 20th - 7 pm. GMT+2 > - Wenesday October 21th - 7 pm. GMT+2 > - Friday October 23th - 7 pm. GMT+2 > > If you want to join, add your name here: http://doodle.com/bfgv3yi3pi48buuv And the winner is... Tuesday October 20th - 7 pm. GMT+2 (CEST, UTC +0200) See you Tuesday , all. Tarek From ssteinerx at gmail.com Sun Oct 18 18:06:03 2009 From: ssteinerx at gmail.com (ssteinerX@gmail.com) Date: Sun, 18 Oct 2009 12:06:03 -0400 Subject: [Distutils] Buildbot throwing a gear after default setup? In-Reply-To: <20091018152655.11571.151326284.divmod.xquotient.649@localhost.localdomain> References: <2E581399-E286-4C77-A787-30861315764E@gmail.com> <31A858D3-D8B1-4B03-8D0D-906DA5DA46A2@gmail.com> <94bdd2610910161621t4a57dd0bv6d27ef45a676caba@mail.gmail.com> <94bdd2610910161626o64a1376an25af942da011f1d5@mail.gmail.com> <64017BB5-2954-4CC5-805C-12066B0127F1@gmail.com> <4AD92DB8.1080609@palladion.com> <48011B84-CB7C-4F4D-AE6D-2D95287D31E6@gmail.com> <94bdd2610910170134m225d07d4y8bcdc31e8906f898@mail.gmail.com> <94bdd2610910170154p69e86ed2xd5a08f3c7b937c74@mail.gmail.com> <34B8FB50-604B-41FA-82CC-C08DEEC6FC42@gmail.com> <94bdd2610910170924y3cf50526k8e8a9f0d81079802@mail.gmail.com> <20091017194758.8004.158279606.divmod.xquotient.277@localhost.localdomain> <0AF15530-3B1F-4D4B-AA34-1029CC0F34EF@gmail.com> <20091018152655.11571.151326284.divmod.xquotient.649@localhost.localdomain> Message-ID: On Oct 18, 2009, at 11:26 AM, exarkun at twistedmatrix.com wrote: >> So, is this something I have to install in addition to the way >> Twisted's being installed now? > > portmap? No, it's an extension module that's part of Twisted. Once > you have python-dev installed it should be okay. Yah, I'm thinking that this is supposed to be building against the Python 2.7 that's built in the course of building the buildbot so I think the bug has less to do with the host environment and more to do with the buildbot not using its own self-contained stuff to build what it needs but I haven't looked too closely yet. S From exarkun at twistedmatrix.com Sun Oct 18 17:26:55 2009 From: exarkun at twistedmatrix.com (exarkun at twistedmatrix.com) Date: Sun, 18 Oct 2009 15:26:55 -0000 Subject: [Distutils] Buildbot throwing a gear after default setup? In-Reply-To: <0AF15530-3B1F-4D4B-AA34-1029CC0F34EF@gmail.com> References: <2E581399-E286-4C77-A787-30861315764E@gmail.com> <31A858D3-D8B1-4B03-8D0D-906DA5DA46A2@gmail.com> <94bdd2610910161621t4a57dd0bv6d27ef45a676caba@mail.gmail.com> <94bdd2610910161626o64a1376an25af942da011f1d5@mail.gmail.com> <64017BB5-2954-4CC5-805C-12066B0127F1@gmail.com> <4AD92DB8.1080609@palladion.com> <48011B84-CB7C-4F4D-AE6D-2D95287D31E6@gmail.com> <94bdd2610910170134m225d07d4y8bcdc31e8906f898@mail.gmail.com> <94bdd2610910170154p69e86ed2xd5a08f3c7b937c74@mail.gmail.com> <34B8FB50-604B-41FA-82CC-C08DEEC6FC42@gmail.com> <94bdd2610910170924y3cf50526k8e8a9f0d81079802@mail.gmail.com> <20091017194758.8004.158279606.divmod.xquotient.277@localhost.localdomain> <0AF15530-3B1F-4D4B-AA34-1029CC0F34EF@gmail.com> Message-ID: <20091018152655.11571.151326284.divmod.xquotient.649@localhost.localdomain> On 17 Oct, 11:26 pm, ssteinerx at gmail.com wrote: > >On Oct 17, 2009, at 3:47 PM, exarkun at twistedmatrix.com wrote: >>On 04:24 pm, ziade.tarek at gmail.com wrote: >>>On Sat, Oct 17, 2009 at 4:49 PM, ssteinerX at gmail.com >>> wrote: >>>>> >>>>>precisely: >>>>>python bootstrap.py && bin/buildout -c buildbot.cfg >>>> >>>>ssteiner at ubuntu:~/distutils-builbot$ python bootstrap.py && bin/ >>>>buildout -c >>>>buildbot.cfg >>>>Downloading >>>>http://pypi.python.org/packages/2.6/s/setuptools/setuptools-0.6c9-py2.6.egg >>>>Getting distribution for 'Twisted'. >>>>twisted/runner/portmap.c:10:20: error: Python.h: No such file or >>>>directory >>> >>>Buildbot needs Twisted, so it needs the 'python-dev' to compile on >>>ubuntu. >>> >>>$ apt-get install python-dev >>>$ apt-get install build-essentials >>>(should be enough) >> >>Almost all of Twisted's extension modules are optional (and portmap >>is hardly useful, I doubt anyone would miss it if it were gone). >>This failure reminds me that it would be nice if distutils had a way >>to express this. > >So, is this something I have to install in addition to the way >Twisted's being installed now? portmap? No, it's an extension module that's part of Twisted. Once you have python-dev installed it should be okay. Jean-Paul From ziade.tarek at gmail.com Sun Oct 18 21:13:12 2009 From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=) Date: Sun, 18 Oct 2009 21:13:12 +0200 Subject: [Distutils] Checking mailing list for Distribute Message-ID: <94bdd2610910181213h46a680b0vea3449a1e0d0567a@mail.gmail.com> Hey, if you want to follow the commits diffs and the issues for Distribute, there's now a distinct mailing list you can follow : http://groups.google.fr/group/distribute-checkins Regards Tarek From david.lyon at preisshare.net Mon Oct 19 02:05:44 2009 From: david.lyon at preisshare.net (David Lyon) Date: Sun, 18 Oct 2009 20:05:44 -0400 Subject: [Distutils] Alternate static metadata PEP submission... In-Reply-To: <4AD44088.7030406@ar.media.kyoto-u.ac.jp> References: <94bdd2610910111517u2fd0b136p44958f7faa46e974@mail.gmail.com> <4AD44088.7030406@ar.media.kyoto-u.ac.jp> Message-ID: <319dd522f4fb12a8122c56e0aad4e0f5@preisshare.net> On Tue, 13 Oct 2009 17:55:36 +0900, David Cournapeau >> [datafiles] >> files = artisticdb.dbf >> artisticdb.mdx >> authors.dbf >> authors.mdx >> directory = db > > How would you handle the case where you have several data directories ? [datafiles] directories = data/us_data data/de_data data/uk_data data/fr_data or: [datafiles] directories = us_data de_data uk_data fr_data > Concerning data files, the notion is too vague IMHO - I know that's the > notion used by distutils Vague as in what? Datafiles is an installation term term meaning that's mostly related to databases, but could also be any other type of resource down to icons and bitmaps. David From david at ar.media.kyoto-u.ac.jp Mon Oct 19 06:18:03 2009 From: david at ar.media.kyoto-u.ac.jp (David Cournapeau) Date: Mon, 19 Oct 2009 13:18:03 +0900 Subject: [Distutils] Alternate static metadata PEP submission... In-Reply-To: <319dd522f4fb12a8122c56e0aad4e0f5@preisshare.net> References: <94bdd2610910111517u2fd0b136p44958f7faa46e974@mail.gmail.com> <4AD44088.7030406@ar.media.kyoto-u.ac.jp> <319dd522f4fb12a8122c56e0aad4e0f5@preisshare.net> Message-ID: <4ADBE87B.6050603@ar.media.kyoto-u.ac.jp> David Lyon wrote: > [datafiles] > directories = data/us_data > data/de_data > data/uk_data > data/fr_data > > Ok - then I think it is confusing to have a directory entry in datafiles. > , but could also be any other type of resource > down to icons and bitmaps. > In current distutils usage, they have become the kitchen sink for documentation, icons, test data, etc... You need more fine-grained control when installing complex packages, you can find several threads on that topic (Linux packagers are the most vocal on this problem). An updated format should care about those user-cases IMO, David From ziade.tarek at gmail.com Mon Oct 19 09:34:58 2009 From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=) Date: Mon, 19 Oct 2009 09:34:58 +0200 Subject: [Distutils] Brand new, clean server, set up from scratch, buildbot still fails In-Reply-To: <8069FBAC-BE4A-40DE-8526-F81508E34CDA@mac.com> References: <8069FBAC-BE4A-40DE-8526-F81508E34CDA@mac.com> Message-ID: <94bdd2610910190034x45cd0a35j5f6e34077c931c46@mail.gmail.com> On Mon, Oct 19, 2009 at 4:35 AM, Steve Steiner wrote: > http://pastebin.com/m69796bc5 sounds like version problems. > > This is running on a bare system, after figuring out which development tools > were needed, all the version control systems necessary etc. ?I have all that > captured in my Fabric script so we're all set for being able to duplicate > it. > > I have NOT installed anything into the system Python at all. ?In fact, other > than the tools necessary to pull the products out of VCS, and the compilers > and such, I haven't installed (or done) anything to the system at all. > ?That's kind of the point. ?We want to be able to go from bare metal to > running buildbot completely automatically. I tried to build the buildbot again on debian and Mac OSX and it worked fine Didn't try on Ubuntu though. It seems that it's using the buildbot installed in your system, which might be the problem since the buildout doesn't pinpoint the versions. Here, I use Python 2.6 and I have: - buildbot-0.7.11p3 - Twisted-8.2.0 - collective.buildbot-0.3.5 I suggest that you install virtualenv and that your run this sequence in an Ubuntu that has all the development tools we have discussed (python-dev, zlib, etc) : $ cd /var/ $ mkdir distutils-bot $ cd distutils-bot $ virtualenv --no-site-packages . $ hg clone https://tarek at bitbucket.org/tarek/distutils-builbot . $ bin/python bootstrap.py $ bin/buildout -c buildbot.cfg This is supposed to compile. Last option: give me an ssh access to your ubuntu box if possible, I'll probaby be able to fix it. Tarek From ziade.tarek at gmail.com Mon Oct 19 09:36:47 2009 From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=) Date: Mon, 19 Oct 2009 09:36:47 +0200 Subject: [Distutils] Brand new, clean server, set up from scratch, buildbot still fails In-Reply-To: <94bdd2610910190034x45cd0a35j5f6e34077c931c46@mail.gmail.com> References: <8069FBAC-BE4A-40DE-8526-F81508E34CDA@mac.com> <94bdd2610910190034x45cd0a35j5f6e34077c931c46@mail.gmail.com> Message-ID: <94bdd2610910190036s77e29f53l8ef7f9d5160c94fd@mail.gmail.com> On Mon, Oct 19, 2009 at 9:34 AM, Tarek Ziad? wrote: > > I tried to build the buildbot again on debian and Mac OSX and it worked fine > Didn't try on Ubuntu though. > Here's this very buildbot running on debian: http://ziade.org:9080/waterfall From floris.bruynooghe at gmail.com Mon Oct 19 11:18:57 2009 From: floris.bruynooghe at gmail.com (Floris Bruynooghe) Date: Mon, 19 Oct 2009 10:18:57 +0100 Subject: [Distutils] [RFC] Recentering the static metadata need : PKG-INFO In-Reply-To: <6d0a7802f78e2b87baa57bd048066201@preisshare.net> References: <94bdd2610910160654n351d786ap4366b6e9e99539dc@mail.gmail.com> <94bdd2610910161615h10ac409k210f4fbe88359624@mail.gmail.com> <97929ae1e24fc80e032ddc1b4fc4ea33@preisshare.net> <94bdd2610910161653g7818c5e6jbb9c34b323583c61@mail.gmail.com> <8de86db5e4096f1bbafccfcfcaf81377@preisshare.net> <94bdd2610910170919j32eabf09rf3d15850fd1b64d3@mail.gmail.com> <6d0a7802f78e2b87baa57bd048066201@preisshare.net> Message-ID: <20091019091857.GA3775@laurie.devork> On Sat, Oct 17, 2009 at 06:00:51PM -0400, David Lyon wrote: > On Sat, 17 Oct 2009 18:19:44 +0200, Tarek Ziad? > wrote: > > e.g. like = 'KDEDIR' in os.environ (so meaning we introduce sequence > > comparisons/operators > > in those markers) [...] > For example, if there is a /etc/gdm/gdm.conf it is pretty safe > to assume that gnome is installed on that system. > > So in your PEP, your could have code.. > > if sys.platform == 'linux2': > gnome_installed = os.path.exists('/etc/gdm/gdm.conf') > > Then exposed that as a variable in your condition: > > 'if desktop in "gnome|windows"' etc All this seems like a bad idea to me. Thinking from a GNU/Linux distro point of view now you'd need to add a build-depends to e.g. GDM *and* KDM only to get the correct .desktop file or something like that. And even worse, as proposed it would probably get confused when both are installed. There must be better ways of saying if you want to have (e.g.) a .desktop file installed. Also the choice you give to a Python package author is suddenly massive: linux-gnome-kde-32-whatever. Just how is a Python package author supposed to get this right? Regards Floris -- Debian GNU/Linux -- The Power of Freedom www.debian.org | www.gnu.org | www.kernel.org From david.lyon at preisshare.net Mon Oct 19 13:22:16 2009 From: david.lyon at preisshare.net (David Lyon) Date: Mon, 19 Oct 2009 07:22:16 -0400 Subject: [Distutils] [RFC] Recentering the static metadata need : PKG-INFO In-Reply-To: <20091019091857.GA3775@laurie.devork> References: <94bdd2610910160654n351d786ap4366b6e9e99539dc@mail.gmail.com> <94bdd2610910161615h10ac409k210f4fbe88359624@mail.gmail.com> <97929ae1e24fc80e032ddc1b4fc4ea33@preisshare.net> <94bdd2610910161653g7818c5e6jbb9c34b323583c61@mail.gmail.com> <8de86db5e4096f1bbafccfcfcaf81377@preisshare.net> <94bdd2610910170919j32eabf09rf3d15850fd1b64d3@mail.gmail.com> <6d0a7802f78e2b87baa57bd048066201@preisshare.net> <20091019091857.GA3775@laurie.devork> Message-ID: <837e5f5455e973f4cb39f51c5dbe872b@preisshare.net> On Mon, 19 Oct 2009 10:18:57 +0100, Floris Bruynooghe wrote: > All this seems like a bad idea to me. Thinking from a GNU/Linux > distro point of view now you'd need to add a build-depends to e.g. GDM > *and* KDM only to get the correct .desktop file or something like > that. And even worse, as proposed it would probably get confused when > both are installed. I don't see why. An application/package might use KDE and might want to check for it. An application/package might want to use GNOME and want to check for it. Why can't developers check what the platform is? There's no forcing developers to use these checks. They're only there to assist in overcoming platform specific installation issues. > There must be better ways of saying if you want to have (e.g.) a > .desktop file installed. Ok.. so throw me some ideas.. > Also the choice you give to a Python package author is suddenly > massive: linux-gnome-kde-32-whatever. Just how is a Python package > author supposed to get this right? Typically, the don't specify anything. They just assume everything works on every platform until their test process tells them it isn't so. In that case, this mechanism allows them to provide workarounds for problem cases. I cannot see how that is a bad idea... David From ziade.tarek at gmail.com Mon Oct 19 13:46:11 2009 From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=) Date: Mon, 19 Oct 2009 13:46:11 +0200 Subject: [Distutils] [RFC] Recentering the static metadata need : PKG-INFO In-Reply-To: <6d0a7802f78e2b87baa57bd048066201@preisshare.net> References: <94bdd2610910160654n351d786ap4366b6e9e99539dc@mail.gmail.com> <94bdd2610910161615h10ac409k210f4fbe88359624@mail.gmail.com> <97929ae1e24fc80e032ddc1b4fc4ea33@preisshare.net> <94bdd2610910161653g7818c5e6jbb9c34b323583c61@mail.gmail.com> <8de86db5e4096f1bbafccfcfcaf81377@preisshare.net> <94bdd2610910170919j32eabf09rf3d15850fd1b64d3@mail.gmail.com> <6d0a7802f78e2b87baa57bd048066201@preisshare.net> Message-ID: <94bdd2610910190446s3af106fwf77bc0fb8cb00ec7@mail.gmail.com> On Sun, Oct 18, 2009 at 12:00 AM, David Lyon wrote: > So in your PEP, your could have code.. > > ?if sys.platform == 'linux2': > ? ? gnome_installed = os.path.exists('/etc/gdm/gdm.conf') > > Then exposed that as a variable in your condition: > > 'if desktop in "gnome|windows"' etc >From the stdlib PoV, exposing new variables like "gnome" isn't appropriate, because we don't want to keep track of all desktop systems in the stdlib. That's impossible. So we have to work at providing tools for the developers in the conditions, for them to express them. So exposing os.environ for your use case could help. I am not sure about file presence though. More generally, I feel uncomfurtable to give too much power in the PKG-INFO syntax, I think things like os.platform() are enough to describe conditional fields in the metadata. I can see a use case for building Extensions and package_data etc.. But what is the use case to detect gnome for the metadata fields ? Tarek From ziade.tarek at gmail.com Mon Oct 19 13:54:03 2009 From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=) Date: Mon, 19 Oct 2009 13:54:03 +0200 Subject: [Distutils] Brand new, clean server, set up from scratch, buildbot still fails In-Reply-To: <8094F1DF-2D6A-46DD-8542-76190D3F5A24@mac.com> References: <8069FBAC-BE4A-40DE-8526-F81508E34CDA@mac.com> <94bdd2610910190034x45cd0a35j5f6e34077c931c46@mail.gmail.com> <8094F1DF-2D6A-46DD-8542-76190D3F5A24@mac.com> Message-ID: <94bdd2610910190454r5fcc3c5cpc1e2e609904c36dc@mail.gmail.com> On Mon, Oct 19, 2009 at 1:44 PM, Steve Steiner wrote: > > The whole idea here is to make a setup script that sets the machine: > > ? ? ? ?Fire up a new cloud server > ? ? ? ?Install whatever software is necessary > ? ? ? ?Download the buildbot code > ? ? ? ?Build the buildbot > ? ? ? ?Start it running for as long as needed > ? ? ? ?Kill the server > > We pay for the server by the hour (1.5 cents/hr) and can fire up as many as > necessary. > > I will put my fabric script up on bitbucket in my clone and you can pull it > over and see what I mean. > > Right now, I"m just using a local VM, but the idea would be to put a fresh > one up on Rackspace as needed, then kill when done. Sounds good. > >> $ cd /var/ >> $ mkdir distutils-bot >> $ cd distutils-bot >> $ virtualenv --no-site-packages . >> $ hg clone https://tarek at bitbucket.org/tarek/distutils-builbot . >> $ bin/python bootstrap.py >> $ bin/buildout -c buildbot.cfg >> >> This is supposed to compile. >> >> Last option: give me an ssh access to your ubuntu box if possible, >> I'll probaby be able to fix it. > > I am not looking to hand-fix this, that would be pointless. ?I want an > automated build script that will fire up a brand new Ubuntu cloud server, > set up the entire environment, install the buildbot and run. By "fix it" I mean "debugging it in your environment to find the problem" of course. Anyways, let see what my sequence does in your VM (with python-dev installed first, and zlib) And if it fails, ls me the content of eggs/ as well Tarek From ssteiner at mac.com Mon Oct 19 04:35:53 2009 From: ssteiner at mac.com (Steve Steiner) Date: Sun, 18 Oct 2009 22:35:53 -0400 Subject: [Distutils] Brand new, clean server, set up from scratch, buildbot still fails Message-ID: <8069FBAC-BE4A-40DE-8526-F81508E34CDA@mac.com> http://pastebin.com/m69796bc5 This is running on a bare system, after figuring out which development tools were needed, all the version control systems necessary etc. I have all that captured in my Fabric script so we're all set for being able to duplicate it. I have NOT installed anything into the system Python at all. In fact, other than the tools necessary to pull the products out of VCS, and the compilers and such, I haven't installed (or done) anything to the system at all. That's kind of the point. We want to be able to go from bare metal to running buildbot completely automatically. I don't quite understand what this thing is supposed to be doing but I'm sure it's not doing everything it needs to do to get itself "bootstrapped"; on a fresh Ubuntu install, anyway. BTW, on my OS X development system, running the system Python 2.6.1, it doesn't have any trouble at all except for this little gem early in the process. /usr/bin/python "-c" "from setuptools.command.easy_install import main; main()" "-mUNxd" "/Users/ssteiner/Dropbox/work/distutils-builbot/ eggs/tmptMBdZk" "-q" "/Users/ssteiner/Dropbox/src/twisted_trunk" path=/Users/ssteiner/Dropbox/work/distutils-builbot/eggs/ setuptools-0.6c9-py2.6.egg /Library/Python/2.6/site-packages/Pyrex-0.9.8.5-py2.6.egg/Pyrex/ Compiler/Errors.py:17: DeprecationWarning: BaseException.message has been deprecated as of Python 2.6 self.message = message unrecognized .svn/entries format in conftest.c:1:23: error: sys/epoll.h: No such file or directory conftest.c:1:23: error: sys/epoll.h: No such file or directory conftest.c:1:23: error: sys/epoll.h: No such file or directory lipo: can't open input file: /var/folders/28/28Xggq0cGXqLpT1Uz5Gk4E++ +TI/-Tmp-//ccOmCair.out (No such file or directory) Got Twisted 8.2.0-r27376. Picked: Twisted = 8.2.0-r27376 Getting required 'zope.interface' required by collective.buildbot 0.3.5. required by Twisted 8.2.0-r27376. ... and then it carries on. However, when trying to run it with /bin/test, something tries to refer to "trunk-checkout" when the python trunk was actually checked out into python-trunk. (~/work/distutils-builbot)# bin/test Traceback (most recent call last): File "bin/test", line 12, in os.chdir(python_trunk) OSError: [Errno 2] No such file or directory: '/Users/ssteiner/work/ distutils-builbot/parts/trunk-checkout/python' All in all, I really can't get it to build or run anywhere at this point... S From ssteiner at mac.com Mon Oct 19 14:09:53 2009 From: ssteiner at mac.com (Steve Steiner) Date: Mon, 19 Oct 2009 08:09:53 -0400 Subject: [Distutils] Brand new, clean server, set up from scratch, buildbot still fails In-Reply-To: <94bdd2610910190036s77e29f53l8ef7f9d5160c94fd@mail.gmail.com> References: <8069FBAC-BE4A-40DE-8526-F81508E34CDA@mac.com> <94bdd2610910190034x45cd0a35j5f6e34077c931c46@mail.gmail.com> <94bdd2610910190036s77e29f53l8ef7f9d5160c94fd@mail.gmail.com> Message-ID: <5C54FF8E-0267-4CD8-80A1-26B9ECD0D9C3@mac.com> On Oct 19, 2009, at 3:36 AM, Tarek Ziad? wrote: > On Mon, Oct 19, 2009 at 9:34 AM, Tarek Ziad? > wrote: >> >> I tried to build the buildbot again on debian and Mac OSX and it >> worked fine >> Didn't try on Ubuntu though. >> > > Here's this very buildbot running on debian: > > http://ziade.org:9080/waterfall I'm not disputing that it runs, I'm telling you that it won't run on a clean Ubuntu without some additional setup. S From david.lyon at preisshare.net Mon Oct 19 14:28:20 2009 From: david.lyon at preisshare.net (David Lyon) Date: Mon, 19 Oct 2009 08:28:20 -0400 Subject: [Distutils] [RFC] Recentering the static metadata need : PKG-INFO In-Reply-To: <94bdd2610910190446s3af106fwf77bc0fb8cb00ec7@mail.gmail.com> References: <94bdd2610910160654n351d786ap4366b6e9e99539dc@mail.gmail.com> <94bdd2610910161615h10ac409k210f4fbe88359624@mail.gmail.com> <97929ae1e24fc80e032ddc1b4fc4ea33@preisshare.net> <94bdd2610910161653g7818c5e6jbb9c34b323583c61@mail.gmail.com> <8de86db5e4096f1bbafccfcfcaf81377@preisshare.net> <94bdd2610910170919j32eabf09rf3d15850fd1b64d3@mail.gmail.com> <6d0a7802f78e2b87baa57bd048066201@preisshare.net> <94bdd2610910190446s3af106fwf77bc0fb8cb00ec7@mail.gmail.com> Message-ID: <0e90f2e42ab83fd0a81b5a981c248abc@preisshare.net> On Mon, 19 Oct 2009 13:46:11 +0200, Tarek Ziad? wrote: > From the stdlib PoV, exposing new variables like "gnome" isn't > appropriate, because > we don't want to keep track of all desktop systems in the stdlib. > That's impossible. Well, the list isn't infinite. And under Linux there are only two major ones existing in any great quantity unless I am mistaken. They are Gnome and KDE. The rest.. aren't so important for detection I would suggest.. ok - somebody stick a stake in my heart... haha > So exposing os.environ for your use case could help. I am not sure > about file presence though. As I said before.. I'm not so keen on that. > More generally, I feel uncomfurtable to give too much power in the > PKG-INFO syntax, I think > things like os.platform() are enough to describe conditional fields in > the metadata. > But what is the use case to detect gnome for the metadata fields ? Perphaps somebody is writing something that only works in gnome. The true rational is slightly different. Under windows there are distinct window manager versions. Eg XP, Vista, Windows 7, win-ce. They *do* have a big impact on python programs and change configurations significantly. So my rationale is simply to have the same thing under linux where there are two major desktops, KDE and GNOME. Maybe on the Mac there are differences to the python developer and we shouldn't forget those also. I'm proposing detecting less than 10 different desktops in all. And detecting those with two lines of code each. So 20 lines of code.. About 20 lines of new code shouldn't be so hard... I would have thought. David From ssteiner at mac.com Mon Oct 19 13:44:56 2009 From: ssteiner at mac.com (Steve Steiner) Date: Mon, 19 Oct 2009 07:44:56 -0400 Subject: [Distutils] Brand new, clean server, set up from scratch, buildbot still fails In-Reply-To: <94bdd2610910190034x45cd0a35j5f6e34077c931c46@mail.gmail.com> References: <8069FBAC-BE4A-40DE-8526-F81508E34CDA@mac.com> <94bdd2610910190034x45cd0a35j5f6e34077c931c46@mail.gmail.com> Message-ID: <8094F1DF-2D6A-46DD-8542-76190D3F5A24@mac.com> On Oct 19, 2009, at 3:34 AM, Tarek Ziad? wrote: > On Mon, Oct 19, 2009 at 4:35 AM, Steve Steiner > wrote: >> http://pastebin.com/m69796bc5 > > sounds like version problems. > >> >> This is running on a bare system, after figuring out which >> development tools >> were needed, all the version control systems necessary etc. I have >> all that >> captured in my Fabric script so we're all set for being able to >> duplicate >> it. > I tried to build the buildbot again on debian and Mac OSX and it > worked fine > Didn't try on Ubuntu though. > > It seems that it's using the buildbot installed in your system, which > might be the problem > since the buildout doesn't pinpoint the versions. > > Here, I use Python 2.6 and I have: > - buildbot-0.7.11p3 > - Twisted-8.2.0 > - collective.buildbot-0.3.5 > > I suggest that you install virtualenv and that your run this sequence > in an Ubuntu that has all the development tools we have discussed > (python-dev, zlib, etc) : The whole idea here is to make a setup script that sets the machine: Fire up a new cloud server Install whatever software is necessary Download the buildbot code Build the buildbot Start it running for as long as needed Kill the server We pay for the server by the hour (1.5 cents/hr) and can fire up as many as necessary. I will put my fabric script up on bitbucket in my clone and you can pull it over and see what I mean. Right now, I"m just using a local VM, but the idea would be to put a fresh one up on Rackspace as needed, then kill when done. > $ cd /var/ > $ mkdir distutils-bot > $ cd distutils-bot > $ virtualenv --no-site-packages . > $ hg clone https://tarek at bitbucket.org/tarek/distutils-builbot . > $ bin/python bootstrap.py > $ bin/buildout -c buildbot.cfg > > This is supposed to compile. > > Last option: give me an ssh access to your ubuntu box if possible, > I'll probaby be able to fix it. I am not looking to hand-fix this, that would be pointless. I want an automated build script that will fire up a brand new Ubuntu cloud server, set up the entire environment, install the buildbot and run. S From david.lyon at preisshare.net Mon Oct 19 14:57:23 2009 From: david.lyon at preisshare.net (David Lyon) Date: Mon, 19 Oct 2009 08:57:23 -0400 Subject: [Distutils] =?utf-8?q?Brand_new=2C_clean_server=2C_set_up_from_sc?= =?utf-8?q?ratch=2C_buildbot_still=09fails?= In-Reply-To: <8094F1DF-2D6A-46DD-8542-76190D3F5A24@mac.com> References: <8069FBAC-BE4A-40DE-8526-F81508E34CDA@mac.com> <94bdd2610910190034x45cd0a35j5f6e34077c931c46@mail.gmail.com> <8094F1DF-2D6A-46DD-8542-76190D3F5A24@mac.com> Message-ID: <2c9bcc58e385d899cc3aa30a07dfe6bf@preisshare.net> On Mon, 19 Oct 2009 07:44:56 -0400, Steve Steiner wrote: > I am not looking to hand-fix this, that would be pointless. I want an > automated build script that will fire up a brand new Ubuntu cloud > server, set up the entire environment, install the buildbot and run. Can I help? It's actually a pretty interesting and typical problem. One of the big challenges is spanning the internal reach of python packaging to 'the outside'. I wish I could use a mac - where we don't have these problems. But under Linux, even on a good distro like debian or ubuntu, it can take a while to locate all the compilers and external tools that are required to build your given package. Doing 'fresh' installs on a fresh os surely is a typical developer use case... or should be going forward.. It should be pretty simple to specify external os-packages and pull them in. Somehow.. :-) (apt-get) There must be some way to put these into a coherent config file. David From ssteiner at mac.com Mon Oct 19 15:23:06 2009 From: ssteiner at mac.com (Steve Steiner) Date: Mon, 19 Oct 2009 09:23:06 -0400 Subject: [Distutils] Brand new, clean server, set up from scratch, buildbot still fails In-Reply-To: <2c9bcc58e385d899cc3aa30a07dfe6bf@preisshare.net> References: <8069FBAC-BE4A-40DE-8526-F81508E34CDA@mac.com> <94bdd2610910190034x45cd0a35j5f6e34077c931c46@mail.gmail.com> <8094F1DF-2D6A-46DD-8542-76190D3F5A24@mac.com> <2c9bcc58e385d899cc3aa30a07dfe6bf@preisshare.net> Message-ID: <627D8F81-22DD-42C6-98FD-842246D6B2DF@mac.com> On Oct 19, 2009, at 8:57 AM, David Lyon wrote: > On Mon, 19 Oct 2009 07:44:56 -0400, Steve Steiner > wrote: > >> I am not looking to hand-fix this, that would be pointless. I want >> an >> automated build script that will fire up a brand new Ubuntu cloud >> server, set up the entire environment, install the buildbot and run. > > Can I help? I've just put the fabfile up in the distutils-buildbot http://bitbucket.org/tarek/distutils-buildbot/overview/ Feel free to jump in! I've run that exact fabfile onto a pure fresh ubuntu 9.04 setup and it won't run yet. Any insight you can provide would be much appreciated. I'd love to make a "make me a buildbot" tool; I think it would be incredibly useful, generally. S From ziade.tarek at gmail.com Mon Oct 19 15:29:23 2009 From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=) Date: Mon, 19 Oct 2009 15:29:23 +0200 Subject: [Distutils] Brand new, clean server, set up from scratch, buildbot still fails In-Reply-To: <627D8F81-22DD-42C6-98FD-842246D6B2DF@mac.com> References: <8069FBAC-BE4A-40DE-8526-F81508E34CDA@mac.com> <94bdd2610910190034x45cd0a35j5f6e34077c931c46@mail.gmail.com> <8094F1DF-2D6A-46DD-8542-76190D3F5A24@mac.com> <2c9bcc58e385d899cc3aa30a07dfe6bf@preisshare.net> <627D8F81-22DD-42C6-98FD-842246D6B2DF@mac.com> Message-ID: <94bdd2610910190629j3ac7aac7ic94346abf7e62373@mail.gmail.com> On Mon, Oct 19, 2009 at 3:23 PM, Steve Steiner wrote: > > On Oct 19, 2009, at 8:57 AM, David Lyon wrote: > >> On Mon, 19 Oct 2009 07:44:56 -0400, Steve Steiner >> wrote: >> >>> I am not looking to hand-fix this, that would be pointless. ?I want an >>> automated build script that will fire up a brand new Ubuntu cloud >>> server, set up the entire environment, install the buildbot and run. >> >> Can I help? > > I've just put the fabfile up in the distutils-buildbot > http://bitbucket.org/tarek/distutils-buildbot/overview/ > > Feel free to jump in! > > I've run that exact fabfile onto a pure fresh ubuntu 9.04 setup and it won't > run yet. > > Any insight you can provide would be much appreciated. > > I'd love to make a "make me a buildbot" tool; I think it would be incredibly > useful, generally. Beware that for the server part of buildbot, it doesn't work on Windows, Maybe this has changed but that's the last status I know of. But I definitely want to run the slaves on Win32. This would require the buildout to make the distinction between a slave machine and a machine that has a salve and a master. It' not the case right now but I can change it, by adding a 'only_slave.cfg' file Tarek From ssteiner at mac.com Mon Oct 19 16:27:54 2009 From: ssteiner at mac.com (Steve Steiner) Date: Mon, 19 Oct 2009 10:27:54 -0400 Subject: [Distutils] Brand new, clean server, set up from scratch, buildbot still fails In-Reply-To: <94bdd2610910190629j3ac7aac7ic94346abf7e62373@mail.gmail.com> References: <8069FBAC-BE4A-40DE-8526-F81508E34CDA@mac.com> <94bdd2610910190034x45cd0a35j5f6e34077c931c46@mail.gmail.com> <8094F1DF-2D6A-46DD-8542-76190D3F5A24@mac.com> <2c9bcc58e385d899cc3aa30a07dfe6bf@preisshare.net> <627D8F81-22DD-42C6-98FD-842246D6B2DF@mac.com> <94bdd2610910190629j3ac7aac7ic94346abf7e62373@mail.gmail.com> Message-ID: <2907746A-6F1F-4A31-835D-3BC215FB0913@mac.com> On Oct 19, 2009, at 9:29 AM, Tarek Ziad? wrote: > But I definitely want to run the slaves on Win32. How about we get it running, first... ;-) S From jon at multani.info Mon Oct 19 17:11:16 2009 From: jon at multani.info (Jonathan Ballet) Date: Mon, 19 Oct 2009 10:11:16 -0500 Subject: [Distutils] [buildout] Fixed verbose mode when developing eggs Message-ID: <426972e258f4382f8928d7a633de1dc2@mail.webfaction.com> Hello, I had a small headache using Buildout this morning: I had a package which contained a C extension and which was developed in my buildout.cfg file. I wanted to see what was the gcc call used to build this extension, but using as much as -v as I could (I stopped at ten), I didn't see anything more. So, I dig in the code, and there was a typo in easy_install.py, which is fixed in fix-verbose-develop.patch (this needs a test to be updated, BTW). The fix was trivial (replacing "==" with "="), but I think the way args is build is now better. I propose also another patch (uniformize-args.patch), which refactors args construction using a list instead of a tuple, which looks smarter (why a tuple whereas it is updated several times?). This is not mandatory for the fix, feel free to integrate it or not. Cheers, Jonathan -------------- next part -------------- Index: src/zc/buildout/tests.py =================================================================== --- src/zc/buildout/tests.py (r?vision 105143) +++ src/zc/buildout/tests.py (copie de travail) @@ -90,7 +90,7 @@ Installing... Develop: '/sample-buildout/foo' in: '/sample-buildout/foo' - ... -q develop -mxN -d /sample-buildout/develop-eggs/... + ... -v develop -mxN -d /sample-buildout/develop-eggs/... """ Index: src/zc/buildout/easy_install.py =================================================================== --- src/zc/buildout/easy_install.py (r?vision 105143) +++ src/zc/buildout/easy_install.py (copie de travail) @@ -873,18 +873,16 @@ tmp3 = tempfile.mkdtemp('build', dir=dest) undo.append(lambda : shutil.rmtree(tmp3)) - args = [ - zc.buildout.easy_install._safe_arg(tsetup), - '-q', 'develop', '-mxN', - '-d', _safe_arg(tmp3), - ] - + args = [zc.buildout.easy_install._safe_arg(tsetup)] log_level = logger.getEffectiveLevel() - if log_level <= 0: - if log_level == 0: - del args[1] - else: - args[1] == '-v' + if log_level < 0: + args.append('-v') + elif log_level > 0: + args.append('-q') + + args.extend(['develop', '-mxN', + '-d', _safe_arg(tmp3)]) + if log_level < logging.DEBUG: logger.debug("in: %r\n%s", directory, ' '.join(args)) -------------- next part -------------- diff -u src/zc/buildout/easy_install.py src/zc/buildout/easy_install.py --- src/zc/buildout/easy_install.py (copie de travail) +++ src/zc/buildout/easy_install.py (copie de travail) @@ -301,16 +301,16 @@ ws, False, )[0].location - args = ('-c', _easy_install_cmd, '-mUNxd', _safe_arg(tmp)) + args = ['-c', _easy_install_cmd, '-mUNxd', _safe_arg(tmp)] if self._always_unzip: - args += ('-Z', ) + args.append('-Z') level = logger.getEffectiveLevel() if level > 0: - args += ('-q', ) + args.append('-q') elif level < 0: - args += ('-v', ) + args.append('-v') - args += (_safe_arg(spec), ) + args.append(_safe_arg(spec)) if level <= logging.DEBUG: logger.debug('Running easy_install:\n%s "%s"\npath=%s\n', @@ -319,13 +319,13 @@ if is_jython: extra_env = dict(os.environ, PYTHONPATH=path) else: - args += (dict(os.environ, PYTHONPATH=path), ) + args.append(dict(os.environ, PYTHONPATH=path)) sys.stdout.flush() # We want any pending output first if is_jython: exit_code = subprocess.Popen( - [_safe_arg(self._executable)] + list(args), + [_safe_arg(self._executable)] + args, env=extra_env).wait() else: exit_code = os.spawnle( From jim at zope.com Mon Oct 19 17:26:03 2009 From: jim at zope.com (Jim Fulton) Date: Mon, 19 Oct 2009 11:26:03 -0400 Subject: [Distutils] [buildout] Fixed verbose mode when developing eggs In-Reply-To: <426972e258f4382f8928d7a633de1dc2@mail.webfaction.com> References: <426972e258f4382f8928d7a633de1dc2@mail.webfaction.com> Message-ID: <1099b90b0910190826m3feedce1h2af73b775ce23f82@mail.gmail.com> On Mon, Oct 19, 2009 at 11:11 AM, Jonathan Ballet wrote: > Hello, > > I had a small headache using Buildout this morning: I had a package which > contained a C extension and which was developed in my buildout.cfg file. I > wanted to see what was the gcc call used to build this extension, but using > as much as -v as I could (I stopped at ten), I didn't see anything more. > > So, I dig in the code, and there was a typo in easy_install.py, which is > fixed in fix-verbose-develop.patch (this needs a test to be updated, BTW). > The fix was trivial (replacing "==" with "="), but I think the way args is > build is now better. > > > I propose also another patch (uniformize-args.patch), which refactors args > construction using a list instead of a tuple, which looks smarter (why a > tuple whereas it is updated several times?). > This is not mandatory for the fix, feel free to integrate it or not. Thanks. I'll apply this soon. :) Jim -- Jim Fulton From barry at python.org Mon Oct 19 21:33:35 2009 From: barry at python.org (Barry Warsaw) Date: Mon, 19 Oct 2009 15:33:35 -0400 Subject: [Distutils] buglet in setuptools 0.6c10 Message-ID: <20091019153335.388b415a@limelight.wooz.org> Sorry for the lame-ass bug report, but my stack is really deep and I didn't want this one to go unreported. I just tried 0.6c10 to see if it would fix a problem I'm having. How fortunate that it hit Cheeseshop at just the right time ;). Unfortunately, there's a bug in setuptools/command/sdist.py Line 184 references string.join, but 'string' is undefined. -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 835 bytes Desc: not available URL: From pje at telecommunity.com Mon Oct 19 21:42:45 2009 From: pje at telecommunity.com (P.J. Eby) Date: Mon, 19 Oct 2009 15:42:45 -0400 Subject: [Distutils] Setuptools 0.6c10 released Message-ID: <20091019194252.8DDA43A4125@sparrow.telecommunity.com> The long-overdue setuptools 0.6c10 update is now available on PyPI, at: http://pypi.python.org/setuptools/ Major updates and fixes include: * Support for SVN 1.6 and Python 2.6 * Fix for the Python 2.6.3 build_ext API change * Support for the most recent Sourceforge download link insanity * Fix for Vista UAC errors running easy_install.exe or other "installer-looking" executables * Fix for errors launching 64-bit Windows Python * Stop crashing on certain types of HTTP error * Stop re-trying URLs that already failed retrieval once * Fixes for various dependency management problems such as looping builds, re-downloading packages already present on sys.path (but not in a registered "site" directory), and randomly preferring local -f packages over local installed packages * Prevent lots of spurious "already imported from another path" warnings (e.g. when pkg_resources is imported late) * Ensure C libraries (as opposed to extensions) are also built when doing bdist_egg Other changes: * Misc. documentation fixes * Improved Jython support * Fewer warnings under Python 2.6+ * Warn when 'packages' uses paths instead of package names (because it causes other problems, like spurious "already imported" warnings) * Stop using /usr/bin/sw_vers on Mac OS (replaced w/'platform' module calls) You can install the updated version using easy_install (or pip!), asking for setuptools==0.6c10. (Note for users of Distribute: Distribute and setuptools use the same package name 'setuptools', and thus cannot both be present on the same sys.path (e.g. in the same virtualenv). If you wish to switch a given environment from Distribute to setuptools or vice versa, you will need to completely uninstall one before installing the other. If you currently have Distribute installed, please follow Distribute's uninstall instructions if you wish to reinstall setuptools.) Please report any problems with setuptools to the setuptools bug tracker at: http://bugs.python.org/setuptools/ For faster response to questions, please use the distutils-sig mailing list, rather than the tracker. Setuptools documentation can be found via links at http://pypi.python.org/pypi/setuptools#using-setuptools-and-easyinstall -- and the pages on the PEAK wiki now load much, much faster than they did a few months ago. (They're static cached pages now, rather than dynamically generated, unless you're actually logged into the wiki.) From pje at telecommunity.com Mon Oct 19 21:55:43 2009 From: pje at telecommunity.com (P.J. Eby) Date: Mon, 19 Oct 2009 15:55:43 -0400 Subject: [Distutils] buglet in setuptools 0.6c10 In-Reply-To: <20091019153335.388b415a@limelight.wooz.org> References: <20091019153335.388b415a@limelight.wooz.org> Message-ID: <20091019195550.5868A3A4750@sparrow.telecommunity.com> At 03:33 PM 10/19/2009 -0400, Barry Warsaw wrote: >Sorry for the lame-ass bug report, but my stack is really deep and I didn't >want this one to go unreported. I just tried 0.6c10 to see if it would fix a >problem I'm having. How fortunate that it hit Cheeseshop at just the right >time ;). Unfortunately, there's a bug in > >setuptools/command/sdist.py > >Line 184 references string.join, but 'string' is undefined. > >-Barry Thanks; it's fixed and pushed back to PyPI. From barry at python.org Mon Oct 19 22:00:23 2009 From: barry at python.org (Barry Warsaw) Date: Mon, 19 Oct 2009 16:00:23 -0400 Subject: [Distutils] buglet in setuptools 0.6c10 In-Reply-To: <20091019195550.5868A3A4750@sparrow.telecommunity.com> References: <20091019153335.388b415a@limelight.wooz.org> <20091019195550.5868A3A4750@sparrow.telecommunity.com> Message-ID: On Oct 19, 2009, at 3:55 PM, P.J. Eby wrote: > At 03:33 PM 10/19/2009 -0400, Barry Warsaw wrote: >> Sorry for the lame-ass bug report, but my stack is really deep and >> I didn't >> want this one to go unreported. I just tried 0.6c10 to see if it >> would fix a >> problem I'm having. How fortunate that it hit Cheeseshop at just >> the right >> time ;). Unfortunately, there's a bug in >> >> setuptools/command/sdist.py >> >> Line 184 references string.join, but 'string' is undefined. >> >> -Barry > > Thanks; it's fixed and pushed back to PyPI. Thanks. -B -------------- next part -------------- A non-text attachment was scrubbed... Name: PGP.sig Type: application/pgp-signature Size: 832 bytes Desc: This is a digitally signed message part URL: From alex.gronholm at nextday.fi Mon Oct 19 22:33:08 2009 From: alex.gronholm at nextday.fi (=?ISO-8859-1?Q?Alex_Gr=F6nholm?=) Date: Mon, 19 Oct 2009 23:33:08 +0300 Subject: [Distutils] Setuptools 0.6c10 released In-Reply-To: <20091019194252.8DDA43A4125@sparrow.telecommunity.com> References: <20091019194252.8DDA43A4125@sparrow.telecommunity.com> Message-ID: <4ADCCD04.90809@nextday.fi> P.J. Eby kirjoitti: > The long-overdue setuptools 0.6c10 update is now available on PyPI, at: > > http://pypi.python.org/setuptools/ > > Major updates and fixes include: > > * Support for SVN 1.6 and Python 2.6 > * Fix for the Python 2.6.3 build_ext API change > * Support for the most recent Sourceforge download link insanity > * Fix for Vista UAC errors running easy_install.exe or other > "installer-looking" executables > * Fix for errors launching 64-bit Windows Python According to the notice on the setuptools bug tracker, you did this by avoiding the use of LoadLibraryEx(). I don't see any changes in launcher.c in setuptools SVN yet, did you forget to commit? Or am I looking in the wrong place? I'd like to see your solution. > * Stop crashing on certain types of HTTP error > * Stop re-trying URLs that already failed retrieval once > * Fixes for various dependency management problems such as looping > builds, re-downloading packages already present on sys.path (but not > in a registered "site" directory), and randomly preferring local -f > packages over local installed packages > * Prevent lots of spurious "already imported from another path" warnings > (e.g. when pkg_resources is imported late) > * Ensure C libraries (as opposed to extensions) are also built when > doing bdist_egg > > Other changes: > > * Misc. documentation fixes > * Improved Jython support > * Fewer warnings under Python 2.6+ > * Warn when 'packages' uses paths instead of package names (because it > causes other problems, like spurious "already imported" warnings) > * Stop using /usr/bin/sw_vers on Mac OS (replaced w/'platform' module > calls) > > You can install the updated version using easy_install (or pip!), > asking for setuptools==0.6c10. > > (Note for users of Distribute: Distribute and setuptools use the same > package name 'setuptools', and thus cannot both be present on the same > sys.path (e.g. in the same virtualenv). If you wish to switch a given > environment from Distribute to setuptools or vice versa, you will need > to completely uninstall one before installing the other. If you > currently have Distribute installed, please follow Distribute's > uninstall instructions if you wish to reinstall setuptools.) > > Please report any problems with setuptools to the setuptools bug > tracker at: > > http://bugs.python.org/setuptools/ > > For faster response to questions, please use the distutils-sig mailing > list, rather than the tracker. Setuptools documentation can be found > via links at > http://pypi.python.org/pypi/setuptools#using-setuptools-and-easyinstall > -- and the pages on the PEAK wiki now load much, much faster than they > did a few months ago. (They're static cached pages now, rather than > dynamically generated, unless you're actually logged into the wiki.) > > _______________________________________________ > Distutils-SIG maillist - Distutils-SIG at python.org > http://mail.python.org/mailman/listinfo/distutils-sig From pje at telecommunity.com Mon Oct 19 23:05:40 2009 From: pje at telecommunity.com (P.J. Eby) Date: Mon, 19 Oct 2009 17:05:40 -0400 Subject: [Distutils] Setuptools 0.6c10 released In-Reply-To: <4ADCCD04.90809@nextday.fi> References: <20091019194252.8DDA43A4125@sparrow.telecommunity.com> <4ADCCD04.90809@nextday.fi> Message-ID: <20091019210547.DAF4F3A4750@sparrow.telecommunity.com> At 11:33 PM 10/19/2009 +0300, Alex Gr?nholm wrote: >P.J. Eby kirjoitti: >>The long-overdue setuptools 0.6c10 update is now available on PyPI, at: >> >> http://pypi.python.org/setuptools/ >> >>Major updates and fixes include: >> >>* Support for SVN 1.6 and Python 2.6 >>* Fix for the Python 2.6.3 build_ext API change >>* Support for the most recent Sourceforge download link insanity >>* Fix for Vista UAC errors running easy_install.exe or other >> "installer-looking" executables >>* Fix for errors launching 64-bit Windows Python >According to the notice on the setuptools bug tracker, you did this >by avoiding the use of LoadLibraryEx(). >I don't see any changes in launcher.c in setuptools SVN yet, did you >forget to commit? It looks like I managed to revert it in my working copy, too. Crap. (Which is weird, because I know I built the executables before and tested them on a 64-bit Vista box.) I've reapplied the changes and put it back in SVN. 'easy_install setuptools==dev06' will let you get it for now. > Or am I looking in the wrong place? >I'd like to see your solution. I changed the loadable_exe() function to be a no-op string copy, essentially. Looks like I'll need to go ahead and put out a c11 tomorrow. Sorry about that. Thanks for the prompt report. From mailinglists at patrice.ch Tue Oct 20 00:02:51 2009 From: mailinglists at patrice.ch (Patrice Neff) Date: Tue, 20 Oct 2009 00:02:51 +0200 Subject: [Distutils] Avoid dependency_links / find_links Message-ID: <8D3E9366-B839-429F-B2EB-28B39424F879@patrice.ch> Hi, We manage our projects with setuptools. For this we have a local PyPi mirror with just our required packages. This mirror is specified in our setuptools environments using the index_url directive. The basic idea is: we don't want to access the internet when installing packages. Today I had to realized that we haven't been reaching that goal as Pylons contains a dependency_links pointing to pylonshq.com. So the question would be: is there any way to avoid following any dependency_links and/or find_links paths and only use the index_url? I have quickly checked the setuptools source code and couldn't find any way to do this. Would a patch be accepted if I implemented that behavior? Are there other approaches to do what I'm trying to do? Thanks in advance Patrice From pje at telecommunity.com Tue Oct 20 00:30:05 2009 From: pje at telecommunity.com (P.J. Eby) Date: Mon, 19 Oct 2009 18:30:05 -0400 Subject: [Distutils] Avoid dependency_links / find_links In-Reply-To: <8D3E9366-B839-429F-B2EB-28B39424F879@patrice.ch> References: <8D3E9366-B839-429F-B2EB-28B39424F879@patrice.ch> Message-ID: <20091019223013.5CE033A4750@sparrow.telecommunity.com> At 12:02 AM 10/20/2009 +0200, Patrice Neff wrote: >So the question would be: is there any way to avoid following any >dependency_links and/or find_links paths and only use the index_url? Yes. See: http://peak.telecommunity.com/DevCenter/EasyInstall#restricting-downloads-with-allow-hosts which explains how to use the command line or configuration files to control this. Also see: http://peak.telecommunity.com/DevCenter/EasyInstall#command-line-options under the '--allow-hosts' option for more details on the syntax. From mailinglists at patrice.ch Tue Oct 20 00:46:00 2009 From: mailinglists at patrice.ch (Patrice Neff) Date: Tue, 20 Oct 2009 00:46:00 +0200 Subject: [Distutils] Avoid dependency_links / find_links In-Reply-To: <20091019223013.5CE033A4750@sparrow.telecommunity.com> References: <8D3E9366-B839-429F-B2EB-28B39424F879@patrice.ch> <20091019223013.5CE033A4750@sparrow.telecommunity.com> Message-ID: <8E57B14D-491E-4F6F-80A5-799C7FA1A7D2@patrice.ch> On Oct 20, 2009, at 12:30 AM, P.J. Eby wrote: > Yes. See: > > http://peak.telecommunity.com/DevCenter/EasyInstall#restricting-downloads-with-allow-hosts > > which explains how to use the command line or configuration files to > control this. Also see: > > http://peak.telecommunity.com/DevCenter/EasyInstall#command-line-options > > under the '--allow-hosts' option for more details on the syntax. Thank you, this works great for my purpose. Sorry I missed that in the documentation. Patrice From matthew at matthewwilkes.co.uk Tue Oct 20 03:05:18 2009 From: matthew at matthewwilkes.co.uk (Matthew Wilkes) Date: Tue, 20 Oct 2009 02:05:18 +0100 Subject: [Distutils] buglet in setuptools 0.6c10 In-Reply-To: <20091019195550.5868A3A4750@sparrow.telecommunity.com> References: <20091019153335.388b415a@limelight.wooz.org> <20091019195550.5868A3A4750@sparrow.telecommunity.com> Message-ID: On 2009-10-19, at 2055, P.J. Eby wrote: > Thanks; it's fixed and pushed back to PyPI. With the same version number. What about the people who have already installed your broke 0.6c10 that don't read this list? Matt From david at ar.media.kyoto-u.ac.jp Tue Oct 20 05:41:52 2009 From: david at ar.media.kyoto-u.ac.jp (David Cournapeau) Date: Tue, 20 Oct 2009 12:41:52 +0900 Subject: [Distutils] [RFC] Recentering the static metadata need : PKG-INFO In-Reply-To: <20091019091857.GA3775@laurie.devork> References: <94bdd2610910160654n351d786ap4366b6e9e99539dc@mail.gmail.com> <94bdd2610910161615h10ac409k210f4fbe88359624@mail.gmail.com> <97929ae1e24fc80e032ddc1b4fc4ea33@preisshare.net> <94bdd2610910161653g7818c5e6jbb9c34b323583c61@mail.gmail.com> <8de86db5e4096f1bbafccfcfcaf81377@preisshare.net> <94bdd2610910170919j32eabf09rf3d15850fd1b64d3@mail.gmail.com> <6d0a7802f78e2b87baa57bd048066201@preisshare.net> <20091019091857.GA3775@laurie.devork> Message-ID: <4ADD3180.2020309@ar.media.kyoto-u.ac.jp> Floris Bruynooghe wrote: >> For example, if there is a /etc/gdm/gdm.conf it is pretty safe >> to assume that gnome is installed on that system. >> >> So in your PEP, your could have code.. >> >> if sys.platform == 'linux2': >> gnome_installed = os.path.exists('/etc/gdm/gdm.conf') >> >> Then exposed that as a variable in your condition: >> >> 'if desktop in "gnome|windows"' etc Agreed, this is too fragile and complicated. Anyone who has done packaging knows how fragile those things are: some buggy packages do not remove some files, some corner cases you did not think about are always coming. At least for things like platform, arch and os, things are quite stable. For everything else, either there is a way to define our own variable whose values are handled by outside code (say setup.py or command line), or you don't make them available at all in the static file. I don't see other solution, and the packaging solutions I know do not allow this kind of things. For any new feature in distutils, it should almost be mandatory to look at what other people did to the same problem space. Good design only comes with experience in that space domain, IMO. David From pje at telecommunity.com Tue Oct 20 06:33:27 2009 From: pje at telecommunity.com (P.J. Eby) Date: Tue, 20 Oct 2009 00:33:27 -0400 Subject: [Distutils] buglet in setuptools 0.6c10 In-Reply-To: References: <20091019153335.388b415a@limelight.wooz.org> <20091019195550.5868A3A4750@sparrow.telecommunity.com> Message-ID: <20091020043335.DCD9A3A474A@sparrow.telecommunity.com> At 02:05 AM 10/20/2009 +0100, Matthew Wilkes wrote: >On 2009-10-19, at 2055, P.J. Eby wrote: > >>Thanks; it's fixed and pushed back to PyPI. > >With the same version number. What about the people who have already >installed your broke 0.6c10 that don't read this list? They'll see an announcement for 0.6c11 tomorrow. I didn't want to bump the version number right away, in case there were any other problems. (And there were.) From david.lyon at preisshare.net Tue Oct 20 06:39:47 2009 From: david.lyon at preisshare.net (David Lyon) Date: Tue, 20 Oct 2009 00:39:47 -0400 Subject: [Distutils] [RFC] Recentering the static metadata need : PKG-INFO In-Reply-To: <4ADD3180.2020309@ar.media.kyoto-u.ac.jp> References: <94bdd2610910160654n351d786ap4366b6e9e99539dc@mail.gmail.com> <94bdd2610910161615h10ac409k210f4fbe88359624@mail.gmail.com> <97929ae1e24fc80e032ddc1b4fc4ea33@preisshare.net> <94bdd2610910161653g7818c5e6jbb9c34b323583c61@mail.gmail.com> <8de86db5e4096f1bbafccfcfcaf81377@preisshare.net> <94bdd2610910170919j32eabf09rf3d15850fd1b64d3@mail.gmail.com> <6d0a7802f78e2b87baa57bd048066201@preisshare.net> <20091019091857.GA3775@laurie.devork> <4ADD3180.2020309@ar.media.kyoto-u.ac.jp> Message-ID: On Tue, 20 Oct 2009 12:41:52 +0900, David Cournapeau wrote: >>> 'if desktop in "gnome|windows"' etc > > Agreed, this is too fragile and complicated. I don't understand which part is fragile. Detecting if gnome or windows exists? or implementing this via Tareks proposal (sort of shown as best as I understand it above). My proposal has always been: [dependencies windows] packages = win32com If you are really suggesting that detecting windows is so fragile that it really shouldn't be supported I really don't have much further to say. Windows is fragile... linux is fragile.. python packaging is fragile.. > At least for things like platform, arch and os, things are quite stable. Ok, so you agree with everything up to defining the window manager. That's good. Detecting the actual window manager is a fairly trivial detail. In reality. > For everything else, either there is a way to define our own variable > whose values are handled by outside code (say setup.py or command line), > or you don't make them available at all in the static file. I don't see > other solution, and the packaging solutions I know do not allow this > kind of things. For any new feature in distutils, it should almost be > mandatory to look at what other people did to the same problem space. > Good design only comes with experience in that space domain, IMO. I do many desktop software installs. I have a good working knowledge of the issues. Ok - I'm agreeing. So lets make all the python packaging as good as it is in perl. Really, you're only objection comes down to the tags 'gnome' and 'kde' if I'm not mistaken. To me they aren't big issues. Ok, so we have to ommit 4 lines of code. I'm not bothered. David From ziade.tarek at gmail.com Tue Oct 20 10:39:51 2009 From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=) Date: Tue, 20 Oct 2009 10:39:51 +0200 Subject: [Distutils] [RFC] PEP 345 and PEP 386 updates Message-ID: <94bdd2610910200139w5db29003pab66d5febca7ee4e@mail.gmail.com> Hi, PEP 345 is the PEP for the new metadata format, replacing PEP 314. I have upgraded it, using Tres's work from last year, and the next step is to add the context marker presented in PEP 390. I have also updated PEP 386 to link it to PEP 345. = PEP 345 : Summary of Differences From PEP 314 = * Metadata-Version is now 1.2. * Added fields: - Maintainer - Maintainer-email - Requires-Python - Requires-External - Requires-Dist - Provides-Dist - Obsoletes-Dist * Deprecated fields: - Requires (in favor of Requires-Dist) - Provides (in favor of Provides-Dist) - Obsoletes (in favor of Obsoletes-Dist) Please comment! Tarek From chris at simplistix.co.uk Tue Oct 20 12:29:38 2009 From: chris at simplistix.co.uk (Chris Withers) Date: Tue, 20 Oct 2009 11:29:38 +0100 Subject: [Distutils] the virtualenv-distribute mess In-Reply-To: References: <319e029f0910060402n306a992as4b9d854f7f29d546@mail.gmail.com> <319e029f0910060402m28ab35acnc9a52aefcdc6f04a@mail.gmail.com> <4ACB24AE.4000300@cryptelium.net> <4ACB2527.7010802@cryptelium.net> <319e029f0910060522l29499f46y2805ed0082d975c3@mail.gmail.com> <4ACB7D70.9060404@cryptelium.net> <4ACF1681.7050506@simplistix.co.uk> <4ACF269C.4010400@cryptelium.net> <94bdd2610910090508j6804628fy5b5d435cce0c7427@mail.gmail.com> <319e029f0910090536j480b2339t530a2f6a9267d451@mail.gmail.com> <4ACF2FC5.6090706@cryptelium.net> <4ACF4765.8020404@simplistix.co.uk> Message-ID: <4ADD9112.2040508@simplistix.co.uk> Tres Seaver wrote: >> Huh? Why? I just don't buy this. We've effectively had a bdfl >> pronouncement that setuptools is dead. distribute 0.6 *is* now the "live >> supported setuptools" so any project that relies on the package called >> "setuptools" should now be using the "distribute" distribution... > > Are you being wilfully ignorant here, Chris? he BDFL doesn't "control" > setuptools, because it is a *third party package*. Why is why an area as fundamentally important as this should be in the stdlib and not at the whim of third party developers ;-) Chris -- Simplistix - Content Management, Batch Processing & Python Consulting - http://www.simplistix.co.uk From chris at simplistix.co.uk Tue Oct 20 12:36:09 2009 From: chris at simplistix.co.uk (Chris Withers) Date: Tue, 20 Oct 2009 11:36:09 +0100 Subject: [Distutils] why would you ever need to specify setuptools as a dependency? In-Reply-To: <20091009183117.GB5033@clingman.lan> References: <4ACF1956.5070903@simplistix.co.uk> <5cae42b20910090457p3f6e4a1bp361b240f974f1617@mail.gmail.com> <4ACF48A9.8060403@simplistix.co.uk> <20091009145617.GE1240@clingman.lan> <4ACF50E6.7030703@simplistix.co.uk> <20091009160456.GF1240@clingman.lan> <4ACF611C.4070901@simplistix.co.uk> <20091009183117.GB5033@clingman.lan> Message-ID: <4ADD9299.3010404@simplistix.co.uk> Toshio Kuratomi wrote: > True... but because of that people are able to specify setuptools in > setup.py and it will work with either distribute or setuptools. If you mean in the "install_requires" or "setup_requires" parameters to setuptools setup method, then I'm yet to see a use case where this is required... > you're getting at that if people didn't specify setuptools in setup.py, > distribute-0.6 could install without using the setuptools name? Without faking the setuptools distribution, yes. > I don't > think that works since you still need to take over the setuptools module package != module != distribution. setuptools and distribute are both distributions that provide a setuptools package. However, requirements are specified in terms of distributions, which is why people specifying setuptools as a requirement causes problems. As I've said several times now, I just can't see any point in doing that... will someone please enlighten me as to why people feel the need to specify "setuptools" in "install_requires" or "setup_requires" when the code that parses those requirements can't execute unless a package called setuptools is present already? Chris -- Simplistix - Content Management, Batch Processing & Python Consulting - http://www.simplistix.co.uk From chris at simplistix.co.uk Tue Oct 20 12:37:45 2009 From: chris at simplistix.co.uk (Chris Withers) Date: Tue, 20 Oct 2009 11:37:45 +0100 Subject: [Distutils] why would you ever need to specify setuptools as a dependency? In-Reply-To: <5cae42b20910131055s5e152d62r4f7d385b1de98c9f@mail.gmail.com> References: <4ACF1956.5070903@simplistix.co.uk> <5cae42b20910090457p3f6e4a1bp361b240f974f1617@mail.gmail.com> <5cae42b20910131055s5e152d62r4f7d385b1de98c9f@mail.gmail.com> Message-ID: <4ADD92F9.5050404@simplistix.co.uk> Hanno Schlichting wrote: > On Tue, Oct 13, 2009 at 7:31 PM, Tres Seaver wrote: >> Why? Nobody will check / enforce / understand what 'install_requires' >> even means except setuptools / distribute. > > To quote Toshio Kuratomi: >> It's nice for people creating system packages when you specify all of the >> packages that your runtime depends on in setup.py. > > So apparently there are others who read and value this information. I'd put money on there being no packaging system that parses requirements sections out of a setup.py other that setuptools/distribute... Chris -- Simplistix - Content Management, Batch Processing & Python Consulting - http://www.simplistix.co.uk From kiorky at cryptelium.net Tue Oct 20 13:05:32 2009 From: kiorky at cryptelium.net (kiorky) Date: Tue, 20 Oct 2009 13:05:32 +0200 Subject: [Distutils] why would you ever need to specify setuptools as a dependency? In-Reply-To: <4ADD92F9.5050404@simplistix.co.uk> References: <4ACF1956.5070903@simplistix.co.uk> <5cae42b20910090457p3f6e4a1bp361b240f974f1617@mail.gmail.com> <5cae42b20910131055s5e152d62r4f7d385b1de98c9f@mail.gmail.com> <4ADD92F9.5050404@simplistix.co.uk> Message-ID: <4ADD997C.3020808@cryptelium.net> Chris Withers a ?crit : > Hanno Schlichting wrote: >> On Tue, Oct 13, 2009 at 7:31 PM, Tres Seaver >> wrote: >>> Why? Nobody will check / enforce / understand what 'install_requires' >>> even means except setuptools / distribute. >> >> To quote Toshio Kuratomi: >>> It's nice for people creating system packages when you specify all of >>> the >>> packages that your runtime depends on in setup.py. >> >> So apparently there are others who read and value this information. > > I'd put money on there being no packaging system that parses > requirements sections out of a setup.py other that setuptools/distribute... buildout, zc.recipe.egg and others recipes flavors do. > > Chris > -- Cordialement, KiOrKY GPG Key FingerPrint: 0x1A1194B7681112AF -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 261 bytes Desc: OpenPGP digital signature URL: From regebro at gmail.com Tue Oct 20 13:29:02 2009 From: regebro at gmail.com (Lennart Regebro) Date: Tue, 20 Oct 2009 13:29:02 +0200 Subject: [Distutils] why would you ever need to specify setuptools as a dependency? In-Reply-To: <4ADD997C.3020808@cryptelium.net> References: <4ACF1956.5070903@simplistix.co.uk> <5cae42b20910090457p3f6e4a1bp361b240f974f1617@mail.gmail.com> <5cae42b20910131055s5e152d62r4f7d385b1de98c9f@mail.gmail.com> <4ADD92F9.5050404@simplistix.co.uk> <4ADD997C.3020808@cryptelium.net> Message-ID: <319e029f0910200429v7fef5174p487068efe9dbc846@mail.gmail.com> 2009/10/20 kiorky : > buildout, zc.recipe.egg and others recipes flavors do. But they use setuptools to do so, I hope? -- Lennart Regebro: Python, Zope, Plone, Grok http://regebro.wordpress.com/ +33 661 58 14 64 From fdrake at gmail.com Tue Oct 20 13:31:40 2009 From: fdrake at gmail.com (Fred Drake) Date: Tue, 20 Oct 2009 07:31:40 -0400 Subject: [Distutils] why would you ever need to specify setuptools as a dependency? In-Reply-To: <4ADD997C.3020808@cryptelium.net> References: <4ACF1956.5070903@simplistix.co.uk> <5cae42b20910090457p3f6e4a1bp361b240f974f1617@mail.gmail.com> <5cae42b20910131055s5e152d62r4f7d385b1de98c9f@mail.gmail.com> <4ADD92F9.5050404@simplistix.co.uk> <4ADD997C.3020808@cryptelium.net> Message-ID: <9cee7ab80910200431j579bbe4fke78aa93f8c28729d@mail.gmail.com> On Tue, Oct 20, 2009 at 7:05 AM, kiorky wrote: > buildout, zc.recipe.egg and others recipes flavors do. These tools don't parse setup.py; they execute setup.py in a constrained context (possibly a separate process; don't recall offhand) in order to collect the metadata. Parsing the setup.py would be insane. -Fred -- Fred L. Drake, Jr. "Chaos is the score upon which reality is written." --Henry Miller From mal at egenix.com Tue Oct 20 13:37:48 2009 From: mal at egenix.com (M.-A. Lemburg) Date: Tue, 20 Oct 2009 13:37:48 +0200 Subject: [Distutils] [RFC] PEP 345 and PEP 386 updates In-Reply-To: <94bdd2610910200139w5db29003pab66d5febca7ee4e@mail.gmail.com> References: <94bdd2610910200139w5db29003pab66d5febca7ee4e@mail.gmail.com> Message-ID: <4ADDA10C.5010804@egenix.com> Tarek Ziad? wrote: > Hi, > > PEP 345 is the PEP for the new metadata format, replacing PEP 314. I > have upgraded it, using Tres's work from last year, and the next step > is to add the context marker presented in PEP 390. > > I have also updated PEP 386 to link it to PEP 345. > > = PEP 345 : Summary of Differences From PEP 314 = > > * Metadata-Version is now 1.2. > > * Added fields: > > - Maintainer > - Maintainer-email > - Requires-Python > - Requires-External > - Requires-Dist > - Provides-Dist > - Obsoletes-Dist > > * Deprecated fields: > > - Requires (in favor of Requires-Dist) > - Provides (in favor of Provides-Dist) > - Obsoletes (in favor of Obsoletes-Dist) Where's the benefit of renaming the above three fields ? After all, the meta-data format is a distutils specific format, so the extra "-Dist" qualifier appears unnecessary. I'd also remove the "Requires-" from "Requires-Python" and simply use "Python" as field name. The intent should be clear, e.g. for Python 2 packages that need at least 2.6: Python: >2.6, <3.0 I'd also like to suggest the addition of conditional field values as already discussed in private mail. This would allow defining requirements based on the target platform and use the same micro-language as suggested in PEP 390. Here's a sketch (following Python's inline if syntax and the example from PEP 390): requires: pywin32; if sys_platform == 'win32' requires: bar (>1.0); if sys_platform == 'win32' requires: foo; if platform_machine == 'i386' requires: bar; if python_version == '2.4' or python_version == '2.5' requires: baz; if 'linux' in sys_platform Each of the listed requires field entries only applies if the condition matches the target platform. The micro-language should provide a limited number of variables to use on the conditions: python_version = sys.version sys_platform = sys.platform os_name = os.name platform_machine = platform.machine() platform_version = platform.version() and possibly more. The same notation could also be used in setup() keyword parameters for the meta-data fields, e.g. setup(..., requires=["pywin32; if sys_platform == 'win32'"], ...) -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Oct 20 2009) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try our new mxODBC.Connect Python Database Interface for free ! :::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/ From kiorky at cryptelium.net Tue Oct 20 13:52:29 2009 From: kiorky at cryptelium.net (kiorky) Date: Tue, 20 Oct 2009 13:52:29 +0200 Subject: [Distutils] why would you ever need to specify setuptools as a dependency? In-Reply-To: <319e029f0910200429v7fef5174p487068efe9dbc846@mail.gmail.com> References: <4ACF1956.5070903@simplistix.co.uk> <5cae42b20910090457p3f6e4a1bp361b240f974f1617@mail.gmail.com> <5cae42b20910131055s5e152d62r4f7d385b1de98c9f@mail.gmail.com> <4ADD92F9.5050404@simplistix.co.uk> <4ADD997C.3020808@cryptelium.net> <319e029f0910200429v7fef5174p487068efe9dbc846@mail.gmail.com> Message-ID: <4ADDA47D.3090400@cryptelium.net> For minitage.recipe.*, yes it uses setuptools API. They also redo on the fly the python sdist dance in some case and reload the distribution using also setuptools API. And in the "patch" use case [1], they also patch the PKG-INFO file with a generated version info and use setuptools API to reload the distribution. For what i know of others API, they don't mess directly with setup.py either. [1] - http://pypi.python.org/pypi/minitage.recipe.egg#patches Lennart Regebro a ?crit : > 2009/10/20 kiorky : >> buildout, zc.recipe.egg and others recipes flavors do. > > But they use setuptools to do so, I hope? > -- Cordialement, KiOrKY GPG Key FingerPrint: 0x1A1194B7681112AF -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 261 bytes Desc: OpenPGP digital signature URL: From chris at simplistix.co.uk Tue Oct 20 14:11:44 2009 From: chris at simplistix.co.uk (Chris Withers) Date: Tue, 20 Oct 2009 13:11:44 +0100 Subject: [Distutils] tracking requested vs dependency installs in PEP 376 metadata In-Reply-To: References: <4ACE15C5.4030105@meyerloewen.net> <20091008191526.GD513@clingman.lan> <4ACE4078.70504@meyerloewen.net> <20091008195227.GA735@clingman.lan> <4ACF3E77.8080806@dirtcircle.com> <319e029f0910110727l7948ffccu6acee5617ea795bf@mail.gmail.com> <1E5BF8F0-B345-4EF5-9C57-79E0F7C369D8@mac.com> <319e029f0910111158y68fb25fajb3cbaf713062bf11@mail.gmail.com> Message-ID: <4ADDA900.4090505@simplistix.co.uk> Ronald Oussoren wrote: > > 2) Notice that simplejson gets installed and is a useful module, then > use that in your own scripts At this point, you specify your scripts' dependencies on simplejson. Look at it another way: You follow you process above, but after step 2, your development machine suffers a disk failure so you start from scratch on a new machine. Are you surprised at this point that simplejson isn't installed when you set things up again, even though by this point you've decided not to use TurboGears? Code should always specify their dependencies, not doing so will cause you problems down the line... Chris -- Simplistix - Content Management, Batch Processing & Python Consulting - http://www.simplistix.co.uk From chris at simplistix.co.uk Tue Oct 20 14:28:56 2009 From: chris at simplistix.co.uk (Chris Withers) Date: Tue, 20 Oct 2009 13:28:56 +0100 Subject: [Distutils] PEP 390 - new format from setup.cfg In-Reply-To: References: <94bdd2610910111517u2fd0b136p44958f7faa46e974@mail.gmail.com> <94bdd2610910120245i2075d2d4n6bf1b7a44ae9d219@mail.gmail.com> <94bdd2610910120934p672cb041xbb5cd36ab64e54b5@mail.gmail.com> Message-ID: <4ADDAD08.1060600@simplistix.co.uk> Ian Bicking wrote: > long_desc = open(os.path.join(os.path.dirname(__file__), 'docs', > 'index.txt')).read() > long_desc = long_desc.split('..contents::', 1)[1].strip() > setup(..., long_description=long_desc) Yes, and yet everytime I ask for the ability specify long description as a file path relative to the pacakge, I get shouted at ;-) > (Come to think of it, I'm not actually sure what we're accomplishing > with this declarative metadata; "setup.py --name" is too minimal, but > some new "setup.py metadata" command that dumps everything isn't hard > to imagine; what are the *real* advantages of this new proposal? If I > have to use a build process to build my setup.cfg, then absolutely > nothing will be accomplished. And is the complexity all just because > sometimes people need to use version-specific requirements?) To wind right back, the original requirement for static metadata was so that it could be published on PyPI and package management tools could parse requirements without having to download the package and execute an untrusted .py file. Chris -- Simplistix - Content Management, Batch Processing & Python Consulting - http://www.simplistix.co.uk From chris at simplistix.co.uk Tue Oct 20 14:30:32 2009 From: chris at simplistix.co.uk (Chris Withers) Date: Tue, 20 Oct 2009 13:30:32 +0100 Subject: [Distutils] PEP 386 In-Reply-To: <94bdd2610910120249i40f06b2fpc775a5be22703269@mail.gmail.com> References: <94bdd2610910111517u2fd0b136p44958f7faa46e974@mail.gmail.com> <20091012090115.GA31472@laurie.devork> <94bdd2610910120249i40f06b2fpc775a5be22703269@mail.gmail.com> Message-ID: <4ADDAD68.30607@simplistix.co.uk> Tarek Ziad? wrote: > Then, the day PEP 386 is accepted, we turn "python_version" into a Version() > object and we introduce '>', '<' and al. What's stopping PEP 386 being accepted? Seems like it'd be a good idea to get it out of the way first... Chris -- Simplistix - Content Management, Batch Processing & Python Consulting - http://www.simplistix.co.uk From ziade.tarek at gmail.com Tue Oct 20 14:36:43 2009 From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=) Date: Tue, 20 Oct 2009 14:36:43 +0200 Subject: [Distutils] PEP 386 In-Reply-To: <4ADDAD68.30607@simplistix.co.uk> References: <94bdd2610910111517u2fd0b136p44958f7faa46e974@mail.gmail.com> <20091012090115.GA31472@laurie.devork> <94bdd2610910120249i40f06b2fpc775a5be22703269@mail.gmail.com> <4ADDAD68.30607@simplistix.co.uk> Message-ID: <94bdd2610910200536r6569cbaayd9d074d52cd53169@mail.gmail.com> On Tue, Oct 20, 2009 at 2:30 PM, Chris Withers wrote: > Tarek Ziad? wrote: >> >> Then, the day PEP 386 is accepted, we turn "python_version" into a >> Version() >> object and we introduce '>', '<' and al. > > What's stopping PEP 386 being accepted? > Seems like it'd be a good idea to get it out of the way first... The last round (last summer) was not in favor of having post/dev markers in the version scheme (these are required by some developers), so PEP 345 and PEP 386 where sleeping a bit. But I have added a note in PEP 386 saying that following this scheme was not mandatory (as PEP 345 mentions), and these markers don't hurt people that want lighter schemes, so.. So I've sent a new mail to RFC PEP 345 + PEP 386, and I am cc'ing Paul, who was IIRC, not in favor of the PEP as it stands now. Tarek From chris at simplistix.co.uk Tue Oct 20 14:53:31 2009 From: chris at simplistix.co.uk (Chris Withers) Date: Tue, 20 Oct 2009 13:53:31 +0100 Subject: [Distutils] Alternate static metadata PEP submission... In-Reply-To: References: <20091013103021.GA21066@laurie.devork> <94bdd2610910131001v70711ffeia27433cd23b32fca@mail.gmail.com> <79a53e276e5dffe24be0433771801d72@preisshare.net> <94bdd2610910131426k64f707d7pf4d398d04b8ecc32@mail.gmail.com> <20091013233654.GA27339@laurie.devork> Message-ID: <4ADDB2CB.1090101@simplistix.co.uk> David Lyon wrote: > What we actually need, and I am building, is a setup.py script > that is going to be a 'trimmed' version of distutils that includes > all neccessary package/application installation code to do the install > from the setup.info file. Be very aware of the problems of proliferating boilerplate code (buildout has already suffered this): - old versions will have to work forever - bug fixes are *very* hard to do I would like my setup.py to look like: from xxx import setup setup() ...and have all the code be in xxx. If xxx is a separately available package for python<2.7, then fine. I would prefer to see "hard logic" in setup.py where there is a good programming language to support it any everything else in setup.cfg (don't like .info, sorry ;-) ). My hunch is that for 95% of packages, setup.py will be nothing more than the above... Chris -- Simplistix - Content Management, Batch Processing & Python Consulting - http://www.simplistix.co.uk From chris at simplistix.co.uk Tue Oct 20 14:56:40 2009 From: chris at simplistix.co.uk (Chris Withers) Date: Tue, 20 Oct 2009 13:56:40 +0100 Subject: [Distutils] PEP 386 In-Reply-To: <94bdd2610910200536r6569cbaayd9d074d52cd53169@mail.gmail.com> References: <94bdd2610910111517u2fd0b136p44958f7faa46e974@mail.gmail.com> <20091012090115.GA31472@laurie.devork> <94bdd2610910120249i40f06b2fpc775a5be22703269@mail.gmail.com> <4ADDAD68.30607@simplistix.co.uk> <94bdd2610910200536r6569cbaayd9d074d52cd53169@mail.gmail.com> Message-ID: <4ADDB388.4080200@simplistix.co.uk> Tarek Ziad? wrote: > On Tue, Oct 20, 2009 at 2:30 PM, Chris Withers wrote: >> Tarek Ziad? wrote: >>> Then, the day PEP 386 is accepted, we turn "python_version" into a >>> Version() >>> object and we introduce '>', '<' and al. >> What's stopping PEP 386 being accepted? >> Seems like it'd be a good idea to get it out of the way first... > > The last round (last summer) was not in favor of having post/dev > markers in the version scheme > (these are required by some developers), so PEP 345 and PEP 386 where > sleeping a bit. I'm sure I can't be the only person suffering from PEP overload when it comes to packaging. Any chance we could at least get dev/post markers in PEP386 and get it done and out of the way? I have a feeling that PEP345 and PEP390 along with David's alternative proposal are all related in such a way that the best thing ot do is bottom out the latter two first, but they all seem to depend (whether or not they want to admit it ;-) ) on PEP 386... Chris -- Simplistix - Content Management, Batch Processing & Python Consulting - http://www.simplistix.co.uk From ziade.tarek at gmail.com Tue Oct 20 15:17:37 2009 From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=) Date: Tue, 20 Oct 2009 15:17:37 +0200 Subject: [Distutils] PEP 386 In-Reply-To: <4ADDB388.4080200@simplistix.co.uk> References: <94bdd2610910111517u2fd0b136p44958f7faa46e974@mail.gmail.com> <20091012090115.GA31472@laurie.devork> <94bdd2610910120249i40f06b2fpc775a5be22703269@mail.gmail.com> <4ADDAD68.30607@simplistix.co.uk> <94bdd2610910200536r6569cbaayd9d074d52cd53169@mail.gmail.com> <4ADDB388.4080200@simplistix.co.uk> Message-ID: <94bdd2610910200617h52c0feeftd6f7ee7831d34ea4@mail.gmail.com> On Tue, Oct 20, 2009 at 2:56 PM, Chris Withers wrote: > Tarek Ziad? wrote: >> >> On Tue, Oct 20, 2009 at 2:30 PM, Chris Withers >> wrote: >>> >>> Tarek Ziad? wrote: >>>> >>>> Then, the day PEP 386 is accepted, we turn "python_version" into a >>>> Version() >>>> object and we introduce '>', '<' and al. >>> >>> What's stopping PEP 386 being accepted? >>> Seems like it'd be a good idea to get it out of the way first... >> >> The last round (last summer) was not in favor of having post/dev >> markers in the version scheme >> (these are required by some developers), so PEP 345 and PEP 386 where >> sleeping a bit. > > I'm sure I can't be the only person suffering from PEP overload when it > comes to packaging. > Any chance we could at least get dev/post markers in > PEP386 and get it done and out of the way? > > I have a feeling that PEP345 and PEP390 along with David's alternative > proposal are all related in such a way that the best thing ot do is bottom > out the latter two first, but they all seem to depend (whether or not they > want to admit it ;-) ) on PEP 386... > Agreed. And every piece of puzzle is starting to emerge. As Marc-Andr? said, PEP 390 is less important and could be done in distutils even without PEP, as long as we add the markers in PEP 345, (meaning we accept 386). And while David's alternative proposal competes with PEP 390 I don't see that as a problem: PEP 390 proposes to build the metadata using options in setup.cfg, whereas David's proposal proposes another system, but at the end, they all end up in a PKG-INFO. Tarek From ssteinerx at gmail.com Tue Oct 20 15:23:49 2009 From: ssteinerx at gmail.com (ssteinerX@gmail.com) Date: Tue, 20 Oct 2009 09:23:49 -0400 Subject: [Distutils] PEP 386 In-Reply-To: <4ADDB388.4080200@simplistix.co.uk> References: <94bdd2610910111517u2fd0b136p44958f7faa46e974@mail.gmail.com> <20091012090115.GA31472@laurie.devork> <94bdd2610910120249i40f06b2fpc775a5be22703269@mail.gmail.com> <4ADDAD68.30607@simplistix.co.uk> <94bdd2610910200536r6569cbaayd9d074d52cd53169@mail.gmail.com> <4ADDB388.4080200@simplistix.co.uk> Message-ID: <8B073A32-010A-4B39-94A9-F7883F8CF684@gmail.com> On Oct 20, 2009, at 8:56 AM, Chris Withers wrote: > I'm sure I can't be the only person suffering from PEP overload when > it comes to packaging. +1 on PEP overload > Any chance we could at least get dev/post markers in PEP386 and get > it done and out of the way? +1 on getting the versioning number scheme nailed down. The reference implementation looks pretty good from a quick review and the tests, while not insanely exhaustive, seem to cover everything that would come up in practice. If anyone disagrees with that last assessment, how about providing a test case? > I have a feeling that PEP345 and PEP390 along with David's > alternative proposal are all related in such a way that the best > thing ot do is bottom out the latter two first I'm not sure anymore what "bottom out the latter two first" means, but getting 386 out of the way would certainly make everything else easier. S From fdrake at gmail.com Tue Oct 20 15:36:08 2009 From: fdrake at gmail.com (Fred Drake) Date: Tue, 20 Oct 2009 09:36:08 -0400 Subject: [Distutils] [RFC] PEP 345 and PEP 386 updates In-Reply-To: <4ADDA10C.5010804@egenix.com> References: <94bdd2610910200139w5db29003pab66d5febca7ee4e@mail.gmail.com> <4ADDA10C.5010804@egenix.com> Message-ID: <9cee7ab80910200636p2384615fy516a0ce924ec0567@mail.gmail.com> On Tue, Oct 20, 2009 at 7:37 AM, M.-A. Lemburg wrote: > Where's the benefit of renaming the above three fields ? I don't see any benefit here either; if we want to add fields for Python package or anything else, new (more explicit) field names can be added, without ever requiring these to be changed. As much as having more explicit names would be nice, it's not worth the disruption caused by deprecation cycles, or the accompanying confusion. These sorts of changes have very long uptake, while the clarity benefit is close to non-existent. > I'd also remove the "Requires-" from "Requires-Python" and > simply use "Python" as field name. +0 > The micro-language should provide a limited number of > variables to use on the conditions: > > python_version = sys.version > sys_platform = sys.platform If we adopt such a micro-language (I'm reserving judgment until I've had more time to read the relevant PEPs carefully), I'd rather see the names match what's in the Python runtime more closely, probably only avoiding the call syntax. This would make the examples: requires: pywin32; if sys.platform == 'win32' requires: bar (>1.0); if sys.platform == 'win32' requires: foo; if platform.machine == 'i386' requires: bar; if sys.version == '2.4' or sys.version == '2.5' requires: baz; if 'linux' in sys.platform > The same notation could also be used in setup() keyword > parameters for the meta-data fields, e.g. Whatever notation is adopted should be used consistently. -Fred -- Fred L. Drake, Jr. "Chaos is the score upon which reality is written." --Henry Miller From chris at simplistix.co.uk Tue Oct 20 15:39:00 2009 From: chris at simplistix.co.uk (Chris Withers) Date: Tue, 20 Oct 2009 14:39:00 +0100 Subject: [Distutils] why would you ever need to specify setuptools as a dependency? In-Reply-To: <9cee7ab80910200431j579bbe4fke78aa93f8c28729d@mail.gmail.com> References: <4ACF1956.5070903@simplistix.co.uk> <5cae42b20910090457p3f6e4a1bp361b240f974f1617@mail.gmail.com> <5cae42b20910131055s5e152d62r4f7d385b1de98c9f@mail.gmail.com> <4ADD92F9.5050404@simplistix.co.uk> <4ADD997C.3020808@cryptelium.net> <9cee7ab80910200431j579bbe4fke78aa93f8c28729d@mail.gmail.com> Message-ID: <4ADDBD74.3080004@simplistix.co.uk> Fred Drake wrote: > Parsing the setup.py would be insane. As is specifying the setuptools distribution as a requirement when you're already using it... Chris -- Simplistix - Content Management, Batch Processing & Python Consulting - http://www.simplistix.co.uk From fdrake at gmail.com Tue Oct 20 15:43:56 2009 From: fdrake at gmail.com (Fred Drake) Date: Tue, 20 Oct 2009 09:43:56 -0400 Subject: [Distutils] why would you ever need to specify setuptools as a dependency? In-Reply-To: <4ADDBD74.3080004@simplistix.co.uk> References: <4ACF1956.5070903@simplistix.co.uk> <5cae42b20910090457p3f6e4a1bp361b240f974f1617@mail.gmail.com> <5cae42b20910131055s5e152d62r4f7d385b1de98c9f@mail.gmail.com> <4ADD92F9.5050404@simplistix.co.uk> <4ADD997C.3020808@cryptelium.net> <9cee7ab80910200431j579bbe4fke78aa93f8c28729d@mail.gmail.com> <4ADDBD74.3080004@simplistix.co.uk> Message-ID: <9cee7ab80910200643q1617b460pb7fd98007d52275a@mail.gmail.com> On Tue, Oct 20, 2009 at 9:39 AM, Chris Withers wrote: > As is specifying the setuptools distribution as a requirement when you're > already using it... I don't use setuptools at runtime unless something requires it. Having it available at install time and run time are two different things, and should remain so. -Fred -- Fred L. Drake, Jr. "Chaos is the score upon which reality is written." --Henry Miller From chris at simplistix.co.uk Tue Oct 20 15:47:40 2009 From: chris at simplistix.co.uk (Chris Withers) Date: Tue, 20 Oct 2009 14:47:40 +0100 Subject: [Distutils] PEP 345 In-Reply-To: <94bdd2610910200617h52c0feeftd6f7ee7831d34ea4@mail.gmail.com> References: <94bdd2610910111517u2fd0b136p44958f7faa46e974@mail.gmail.com> <20091012090115.GA31472@laurie.devork> <94bdd2610910120249i40f06b2fpc775a5be22703269@mail.gmail.com> <4ADDAD68.30607@simplistix.co.uk> <94bdd2610910200536r6569cbaayd9d074d52cd53169@mail.gmail.com> <4ADDB388.4080200@simplistix.co.uk> <94bdd2610910200617h52c0feeftd6f7ee7831d34ea4@mail.gmail.com> Message-ID: <4ADDBF7C.7010905@simplistix.co.uk> Tarek Ziad? wrote: > Agreed. And every piece of puzzle is starting to emerge. > > As Marc-Andr? said, PEP 390 is less important and could be done in > distutils even without PEP, > as long as we add the markers in PEP 345, (meaning we accept 386). What markers are you referring to? A quick scan of http://www.python.org/dev/peps/pep-0345/ shows no sign of any of the contentious bits of PEP390. What am I missing? (other than that 386 should be accepted and implemented asap...) Chris -- Simplistix - Content Management, Batch Processing & Python Consulting - http://www.simplistix.co.uk From chris at simplistix.co.uk Tue Oct 20 15:48:58 2009 From: chris at simplistix.co.uk (Chris Withers) Date: Tue, 20 Oct 2009 14:48:58 +0100 Subject: [Distutils] why would you ever need to specify setuptools as a dependency? In-Reply-To: <9cee7ab80910200643q1617b460pb7fd98007d52275a@mail.gmail.com> References: <4ACF1956.5070903@simplistix.co.uk> <5cae42b20910090457p3f6e4a1bp361b240f974f1617@mail.gmail.com> <5cae42b20910131055s5e152d62r4f7d385b1de98c9f@mail.gmail.com> <4ADD92F9.5050404@simplistix.co.uk> <4ADD997C.3020808@cryptelium.net> <9cee7ab80910200431j579bbe4fke78aa93f8c28729d@mail.gmail.com> <4ADDBD74.3080004@simplistix.co.uk> <9cee7ab80910200643q1617b460pb7fd98007d52275a@mail.gmail.com> Message-ID: <4ADDBFCA.40902@simplistix.co.uk> Fred Drake wrote: > On Tue, Oct 20, 2009 at 9:39 AM, Chris Withers wrote: >> As is specifying the setuptools distribution as a requirement when you're >> already using it... > > I don't use setuptools at runtime unless something requires it. > > Having it available at install time and run time are two different > things, and should remain so. All I'm saying is that packages shouldn't express a dependency on setuptools if setuptools is required even before that expression can be parsed. I'm not talking about install or run time... Chris -- Simplistix - Content Management, Batch Processing & Python Consulting - http://www.simplistix.co.uk From ziade.tarek at gmail.com Tue Oct 20 15:53:23 2009 From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=) Date: Tue, 20 Oct 2009 15:53:23 +0200 Subject: [Distutils] PEP 345 In-Reply-To: <4ADDBF7C.7010905@simplistix.co.uk> References: <94bdd2610910111517u2fd0b136p44958f7faa46e974@mail.gmail.com> <20091012090115.GA31472@laurie.devork> <94bdd2610910120249i40f06b2fpc775a5be22703269@mail.gmail.com> <4ADDAD68.30607@simplistix.co.uk> <94bdd2610910200536r6569cbaayd9d074d52cd53169@mail.gmail.com> <4ADDB388.4080200@simplistix.co.uk> <94bdd2610910200617h52c0feeftd6f7ee7831d34ea4@mail.gmail.com> <4ADDBF7C.7010905@simplistix.co.uk> Message-ID: <94bdd2610910200653l38be53a0hfebfe8ccaf3fe5db@mail.gmail.com> On Tue, Oct 20, 2009 at 3:47 PM, Chris Withers wrote: > Tarek Ziad? wrote: >> >> Agreed. And every piece of puzzle is starting to emerge. >> >> As Marc-Andr? said, PEP 390 is less important and could be done in >> distutils even without PEP, >> as long as we add the markers in PEP 345, (meaning we accept 386). > > What markers are you referring to? > > A quick scan of http://www.python.org/dev/peps/pep-0345/ shows no sign of > any of the contentious bits of PEP390. What am I missing? That's the part I have not added yet. Markers are the "sys_platform == 'win32" things discussed in PEP 390. See the other thread (RFC: PEP 345...) > (other than that 386 should be accepted and implemented asap...) Its implemented, we just need a consensus no consensus would mean that if a minority that have enough supporters objects the PEP, and provide an alernative, we would have another round. From ssteinerx at gmail.com Tue Oct 20 16:01:52 2009 From: ssteinerx at gmail.com (ssteinerX@gmail.com) Date: Tue, 20 Oct 2009 10:01:52 -0400 Subject: [Distutils] PEP 345 In-Reply-To: <94bdd2610910200653l38be53a0hfebfe8ccaf3fe5db@mail.gmail.com> References: <94bdd2610910111517u2fd0b136p44958f7faa46e974@mail.gmail.com> <20091012090115.GA31472@laurie.devork> <94bdd2610910120249i40f06b2fpc775a5be22703269@mail.gmail.com> <4ADDAD68.30607@simplistix.co.uk> <94bdd2610910200536r6569cbaayd9d074d52cd53169@mail.gmail.com> <4ADDB388.4080200@simplistix.co.uk> <94bdd2610910200617h52c0feeftd6f7ee7831d34ea4@mail.gmail.com> <4ADDBF7C.7010905@simplistix.co.uk> <94bdd2610910200653l38be53a0hfebfe8ccaf3fe5db@mail.gmail.com> Message-ID: <27EBA79C-55ED-4914-8976-980591AAA3F7@gmail.com> On Oct 20, 2009, at 9:53 AM, Tarek Ziad? wrote: >> (other than that 386 should be accepted and implemented asap...) > > Its implemented, we just need a consensus +1, ship it! S From chris at simplistix.co.uk Tue Oct 20 16:12:34 2009 From: chris at simplistix.co.uk (Chris Withers) Date: Tue, 20 Oct 2009 15:12:34 +0100 Subject: [Distutils] supporting multiple versions of one package in one environment is insane In-Reply-To: <20091009192001.GC5033@clingman.lan> References: <94bdd2610910080131j323b98d9i871bce43465f237a@mail.gmail.com> <4ACF5BE4.70907@simplistix.co.uk> <20091009162226.GA5033@clingman.lan> <4ACF64E8.8010501@simplistix.co.uk> <20091009192001.GC5033@clingman.lan> Message-ID: <4ADDC552.4040704@simplistix.co.uk> [moving to disutils-sig where this belongs] Toshio Kuratomi wrote: >>> where type of problem == sandboxed environment, sure. How do you solve the >>> problem for system packagers? >> What's to stop a system packager either just running the buildout on >> install, or running the buildout at package build time and then just >> dropping the resulting environment wherever they want to install >> applications? Such a package would only be dependent on the right python >> version at runtime... >> > If buildout creates sandboxed environments like virtualenv then everything > here applies: > > https://fedoraproject.org/wiki/Packaging:No_Bundled_Libraries That's fine, buildout can use a shared eggs directory (it's how I run it by default, to minimise the amount of downloading that goes on). In that case, there would be no duplication of packages, the "installed app" would just pick from a folder full of eggs that you could put wherever your OS wants them... In short, have multiple versions of a python package on the python path at the same time is insane ;-) Chris -- Simplistix - Content Management, Batch Processing & Python Consulting - http://www.simplistix.co.uk From pje at telecommunity.com Tue Oct 20 16:14:55 2009 From: pje at telecommunity.com (P.J. Eby) Date: Tue, 20 Oct 2009 10:14:55 -0400 Subject: [Distutils] Setuptools 0.6c11 released Message-ID: <20091020141504.C321F3A4108@sparrow.telecommunity.com> Oops! ----- Due to a couple of last-minute issues with the 0.6c10 release, I've released an 0.6c11 update at: http://pypi.python.org/setuptools/ It fixes an error when running the "sdist" command on a package with no README, and includes the 64-bit Windows fix that was promised in 0.6c10 but wasn't actually checked in to SVN. You can install the updated version using easy_install (or pip!), by asking for setuptools==0.6c11. The Fine Print -------------- (Note for users of Distribute: Distribute and setuptools use the same package name 'setuptools', and thus cannot both be present on the same sys.path (e.g. in the same virtualenv). If you wish to switch a given environment from Distribute to setuptools or vice versa, you will need to completely uninstall one before installing the other. If you currently have Distribute installed, please follow Distribute's uninstall instructions if you wish to reinstall setuptools.) Please report any bugs to the setuptools bug tracker at: http://bugs.python.org/setuptools/ For faster response to questions, please use the distutils-sig mailing list, rather than the tracker. Setuptools documentation can be found via links at http://pypi.python.org/pypi/setuptools#using-setuptools-and-easyinstall -- and the pages on the PEAK wiki now load much, much faster than they did a few months ago. (They're static cached pages now, rather than dynamically generated, unless you're actually logged into the wiki.) From chris at simplistix.co.uk Tue Oct 20 16:21:53 2009 From: chris at simplistix.co.uk (Chris Withers) Date: Tue, 20 Oct 2009 15:21:53 +0100 Subject: [Distutils] [Python-Dev] Distutils and Distribute roadmap (and some words on Virtualenv, Pip) In-Reply-To: <4ADDC6CA.5000809@simplistix.co.uk> References: <94bdd2610910080131j323b98d9i871bce43465f237a@mail.gmail.com> <94bdd2610910081555h13c8b48cg8810377912633dc5@mail.gmail.com> <4ACE718D.8090400@cryptelium.net> <94bdd2610910081621n502803e4l8efeabae3c5863ff@mail.gmail.com> <79990c6b0910090532y63169b60q8dd0a95f056c244d@mail.gmail.com> <4ACF6993.5040704@voidspace.org.uk> <4ADDC6CA.5000809@simplistix.co.uk> Message-ID: <4ADDC781.7070404@simplistix.co.uk> Chris Withers wrote: > [following up on distutils-sig] > > Michael Foord wrote: >> Many Windows users would be quite happy if the standard mechanism for >> installing non-source distributions on Windows was via the wininst >> binaries. > > ...and many users may not be ;-) I know I'd be extremely unhappy if that > were the case as I routines use multiple versions of packages with > multiple versions of python on one machine... > >> I wonder if it is going to be possible to make this compatible with >> the upcoming distutils package management 'stuff' (querying for >> installed packages, uninstallation etc) since >> installation/uninstallation goes through the Windows system package >> management feature. I guess it would be eminently possible but >> require some reasonably high level Windows-fu to do. > > I wouldn't have a problem if integrating with the windows package > manager was an optional extra, but I think it's one of many types of > package management that need to be worried about, so might be easier to > get the others working and let anyone who wants anything beyond a > pure-python packaging system that works across platforms, regardless of > whether binary extensions are needed, do the work themselves... > > Chris > -- Simplistix - Content Management, Batch Processing & Python Consulting - http://www.simplistix.co.uk From floris.bruynooghe at gmail.com Tue Oct 20 16:33:04 2009 From: floris.bruynooghe at gmail.com (Floris Bruynooghe) Date: Tue, 20 Oct 2009 15:33:04 +0100 Subject: [Distutils] [RFC] PEP 345 and PEP 386 updates In-Reply-To: <4ADDA10C.5010804@egenix.com> References: <94bdd2610910200139w5db29003pab66d5febca7ee4e@mail.gmail.com> <4ADDA10C.5010804@egenix.com> Message-ID: <20091020143304.GA6003@laurie.devork> On Tue, Oct 20, 2009 at 01:37:48PM +0200, M.-A. Lemburg wrote: [...] > > * Added fields: [...] > > - Requires-Dist > > - Provides-Dist > > - Obsoletes-Dist > > > > * Deprecated fields: > > > > - Requires (in favor of Requires-Dist) > > - Provides (in favor of Provides-Dist) > > - Obsoletes (in favor of Obsoletes-Dist) > > Where's the benefit of renaming the above three fields ? > > After all, the meta-data format is a distutils specific format, > so the extra "-Dist" qualifier appears unnecessary. In PEP 314 the "Requires" field is meant to contain module or packages names. Now OTOH it seems people are more interested in the distribution names. Making this change explicit is a good thing I reckon. (There's a whole other discussion about how it would be easier to provide an alternative implementation if it was done on module/package name instead of distribution name, but I haven't got enough experience with that so am happy to go with what actually seems to be used now.) Regards Floris -- Debian GNU/Linux -- The Power of Freedom www.debian.org | www.gnu.org | www.kernel.org From chris at simplistix.co.uk Tue Oct 20 17:17:00 2009 From: chris at simplistix.co.uk (Chris Withers) Date: Tue, 20 Oct 2009 16:17:00 +0100 Subject: [Distutils] PEP 345 In-Reply-To: <94bdd2610910200653l38be53a0hfebfe8ccaf3fe5db@mail.gmail.com> References: <94bdd2610910111517u2fd0b136p44958f7faa46e974@mail.gmail.com> <20091012090115.GA31472@laurie.devork> <94bdd2610910120249i40f06b2fpc775a5be22703269@mail.gmail.com> <4ADDAD68.30607@simplistix.co.uk> <94bdd2610910200536r6569cbaayd9d074d52cd53169@mail.gmail.com> <4ADDB388.4080200@simplistix.co.uk> <94bdd2610910200617h52c0feeftd6f7ee7831d34ea4@mail.gmail.com> <4ADDBF7C.7010905@simplistix.co.uk> <94bdd2610910200653l38be53a0hfebfe8ccaf3fe5db@mail.gmail.com> Message-ID: <4ADDD46C.8080506@simplistix.co.uk> Tarek Ziad? wrote: >> A quick scan of http://www.python.org/dev/peps/pep-0345/ shows no sign of >> any of the contentious bits of PEP390. What am I missing? > > That's the part I have not added yet. Markers are the "sys_platform == > 'win32" things > discussed in PEP 390. But PEP345 is about the metadata that appears in the installed package, no? What use is it having conditionals in there at all? >> (other than that 386 should be accepted and implemented asap...) > > Its implemented, we just need a consensus > > > no consensus would mean that if a minority that have enough supporters > objects the PEP, > and provide an alernative, we would have another round. > What are the current sticking points? I do remember some discussions surrounding this but didn't see any active threads when I went through distutils-sig today... Chris -- Simplistix - Content Management, Batch Processing & Python Consulting - http://www.simplistix.co.uk From mal at egenix.com Tue Oct 20 17:20:50 2009 From: mal at egenix.com (M.-A. Lemburg) Date: Tue, 20 Oct 2009 17:20:50 +0200 Subject: [Distutils] [RFC] PEP 345 and PEP 386 updates In-Reply-To: <9cee7ab80910200636p2384615fy516a0ce924ec0567@mail.gmail.com> References: <94bdd2610910200139w5db29003pab66d5febca7ee4e@mail.gmail.com> <4ADDA10C.5010804@egenix.com> <9cee7ab80910200636p2384615fy516a0ce924ec0567@mail.gmail.com> Message-ID: <4ADDD552.5070502@egenix.com> Fred Drake wrote: > On Tue, Oct 20, 2009 at 7:37 AM, M.-A. Lemburg wrote: >> Where's the benefit of renaming the above three fields ? > > I don't see any benefit here either; if we want to add fields for > Python package or anything else, new (more explicit) field names can > be added, without ever requiring these to be changed. > > As much as having more explicit names would be nice, it's not worth > the disruption caused by deprecation cycles, or the accompanying > confusion. These sorts of changes have very long uptake, while the > clarity benefit is close to non-existent. > >> I'd also remove the "Requires-" from "Requires-Python" and >> simply use "Python" as field name. > > +0 > >> The micro-language should provide a limited number of >> variables to use on the conditions: >> >> python_version = sys.version >> sys_platform = sys.platform > > If we adopt such a micro-language (I'm reserving judgment until I've > had more time to read the relevant PEPs carefully), I'd rather see the > names match what's in the Python runtime more closely, probably only > avoiding the call syntax. We could do that as well, yes. > This would make the examples: > > requires: pywin32; if sys.platform == 'win32' > requires: bar (>1.0); if sys.platform == 'win32' > requires: foo; if platform.machine == 'i386' > requires: bar; if sys.version == '2.4' or sys.version == '2.5' > requires: baz; if 'linux' in sys.platform > >> The same notation could also be used in setup() keyword >> parameters for the meta-data fields, e.g. > > Whatever notation is adopted should be used consistently. Agreed. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Oct 20 2009) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try our new mxODBC.Connect Python Database Interface for free ! :::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/ From ziade.tarek at gmail.com Tue Oct 20 18:12:55 2009 From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=) Date: Tue, 20 Oct 2009 18:12:55 +0200 Subject: [Distutils] PEP 345 In-Reply-To: <4ADDD46C.8080506@simplistix.co.uk> References: <94bdd2610910111517u2fd0b136p44958f7faa46e974@mail.gmail.com> <20091012090115.GA31472@laurie.devork> <94bdd2610910120249i40f06b2fpc775a5be22703269@mail.gmail.com> <4ADDAD68.30607@simplistix.co.uk> <94bdd2610910200536r6569cbaayd9d074d52cd53169@mail.gmail.com> <4ADDB388.4080200@simplistix.co.uk> <94bdd2610910200617h52c0feeftd6f7ee7831d34ea4@mail.gmail.com> <4ADDBF7C.7010905@simplistix.co.uk> <94bdd2610910200653l38be53a0hfebfe8ccaf3fe5db@mail.gmail.com> <4ADDD46C.8080506@simplistix.co.uk> Message-ID: <94bdd2610910200912s52b1bbacia89c831202bb7780@mail.gmail.com> On Tue, Oct 20, 2009 at 5:17 PM, Chris Withers wrote: > Tarek Ziad? wrote: >>> >>> A quick scan of http://www.python.org/dev/peps/pep-0345/ shows no sign of >>> any of the contentious bits of PEP390. What am I missing? >> >> That's the part I have not added yet. Markers are the "sys_platform == >> 'win32" things >> discussed in PEP 390. > > But PEP345 is about the metadata that appears in the installed package, no? > > What use is it having conditionals in there at all? Because we want to add the PEP 390 context dependent markers in there, as explained in PEP 390 (I've updated it yesterday, have a look) It provides the markers to any third party app that wants to read the metadata, and make it avaiable directly at PyPI. >>> (other than that 386 should be accepted and implemented asap...) > > What are the current sticking points? I do remember some discussions > surrounding this but didn't see any active threads when I went through > distutils-sig today... That's from this summer: some folks found the scheme overcomplex and dont want to see it as the standard. I am working in finding back the details. Tarek From david.lyon at preisshare.net Tue Oct 20 21:33:32 2009 From: david.lyon at preisshare.net (David Lyon) Date: Tue, 20 Oct 2009 15:33:32 -0400 Subject: [Distutils] [RFC] PEP 345 and PEP 386 updates In-Reply-To: <9cee7ab80910200636p2384615fy516a0ce924ec0567@mail.gmail.com> References: <94bdd2610910200139w5db29003pab66d5febca7ee4e@mail.gmail.com> <4ADDA10C.5010804@egenix.com> <9cee7ab80910200636p2384615fy516a0ce924ec0567@mail.gmail.com> Message-ID: On Tue, 20 Oct 2009 09:36:08 -0400, Fred Drake wrote: > If we adopt such a micro-language (I'm reserving judgment until I've > had more time to read the relevant PEPs carefully), I'd rather see the > names match what's in the Python runtime more closely, probably only > avoiding the call syntax. > .. > requires: pywin32; if sys.platform == 'win32' > .. I foresee problems with using constants from the python runtime as they currently stand. Only for the simple reason that the existing 'constants' were good 5 years ago. But won't be good - next year. For example, 'win32' refers to a specific windows because windows can now be 64 bit. Does win32 mean win64? It does now - but it shouldn't. I propose windows and 32 and 64 as further qualifiers. So for example, 'windows' or 'windows-32' or 'windows-64'. Not to mention 'windows-xp-32' and 'windows-xp-64' and 'windows-ce'. Maybe windows-128 is around the corner - who knows.. I respect those sys python constants.. but they are too unspecific to be used here. And for linux, what is 'linux2' ? It should just be linux.. to make it less confusing. Anyway, I'm coding this all up now into an example. So far I have in my table.. standard_platform_bits = ('windows','linux','mac', 'xp','vista','7','ce' 'os/x', '32','64', 'kde','gnome', 'wx','gtk', 'ubuntu','debian','suse','redhat','gentoo', 'centos','symbian' ) I think we need new constants for package installation. It will be less confusing. Whichever way we go. David From david.lyon at preisshare.net Tue Oct 20 21:43:26 2009 From: david.lyon at preisshare.net (David Lyon) Date: Tue, 20 Oct 2009 15:43:26 -0400 Subject: [Distutils] Alternate static metadata PEP submission... In-Reply-To: <4ADDB2CB.1090101@simplistix.co.uk> References: <20091013103021.GA21066@laurie.devork> <94bdd2610910131001v70711ffeia27433cd23b32fca@mail.gmail.com> <79a53e276e5dffe24be0433771801d72@preisshare.net> <94bdd2610910131426k64f707d7pf4d398d04b8ecc32@mail.gmail.com> <20091013233654.GA27339@laurie.devork> <4ADDB2CB.1090101@simplistix.co.uk> Message-ID: <56bcb7f7893e86ec565d89b7ef40f418@preisshare.net> On Tue, 20 Oct 2009 13:53:31 +0100, Chris Withers wrote: > Be very aware of the problems of proliferating boilerplate code > (buildout has already suffered this): > > - old versions will have to work forever > > - bug fixes are *very* hard to do Well, you're probably right. So I will build an option in to check on the net for an updated version before starting. > > I would like my setup.py to look like: > > from xxx import setup > setup() > > ...and have all the code be in xxx. If xxx is a separately available > package for python<2.7, then fine. > > I would prefer to see "hard logic" in setup.py where there is a good > programming language to support it any everything else in setup.cfg Of course. Code in one place, data in another. > (don't like .info, sorry ;-) ). I'm softening. > My hunch is that for 95% of packages, > setup.py will be nothing more than the above... Hope so David From david.lyon at preisshare.net Tue Oct 20 21:48:38 2009 From: david.lyon at preisshare.net (David Lyon) Date: Tue, 20 Oct 2009 15:48:38 -0400 Subject: [Distutils] PEP 390 - new format from setup.cfg In-Reply-To: <4ADDAD08.1060600@simplistix.co.uk> References: <94bdd2610910111517u2fd0b136p44958f7faa46e974@mail.gmail.com> <94bdd2610910120245i2075d2d4n6bf1b7a44ae9d219@mail.gmail.com> <94bdd2610910120934p672cb041xbb5cd36ab64e54b5@mail.gmail.com> <4ADDAD08.1060600@simplistix.co.uk> Message-ID: <5a2a695a18afa2d6f99e26aeeca13561@preisshare.net> On Tue, 20 Oct 2009 13:28:56 +0100, Chris Withers wrote: > Yes, and yet everytime I ask for the ability specify long description as > a file path relative to the pacakge, I get shouted at ;-) [setup] name = artistflair version = 1.2 description_file = flairdescript.rst A soft voice is calling you from the other (alternate) side... David From ziade.tarek at gmail.com Tue Oct 20 22:05:30 2009 From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=) Date: Tue, 20 Oct 2009 22:05:30 +0200 Subject: [Distutils] [RFC] PEP 345 and PEP 386 updates In-Reply-To: References: <94bdd2610910200139w5db29003pab66d5febca7ee4e@mail.gmail.com> <4ADDA10C.5010804@egenix.com> <9cee7ab80910200636p2384615fy516a0ce924ec0567@mail.gmail.com> Message-ID: <94bdd2610910201305j12691b33pc57ed2741b64d057@mail.gmail.com> On Tue, Oct 20, 2009 at 9:33 PM, David Lyon wrote: > On Tue, 20 Oct 2009 09:36:08 -0400, Fred Drake wrote: > >> If we adopt such a micro-language (I'm reserving judgment until I've >> had more time to read the relevant PEPs carefully), I'd rather see the >> names match what's in the Python runtime more closely, probably only >> avoiding the call syntax. >> .. >> ? requires: pywin32; if sys.platform == 'win32' >> .. > > I foresee problems with using constants from the python runtime as > they currently stand. Only for the simple reason that the existing > 'constants' were good 5 years ago. But won't be good - next year. > > For example, 'win32' refers to a specific windows because windows > can now be 64 bit. > > Does win32 mean win64? It does now - but it shouldn't. > > I propose windows and 32 and 64 as further qualifiers. So for example, > 'windows' or 'windows-32' or 'windows-64'. Not to mention 'windows-xp-32' > and 'windows-xp-64' and 'windows-ce'. > > Maybe windows-128 is around the corner - who knows.. > > I respect those sys python constants.. but they are too unspecific > to be used here. No, the architecture is given by platform.machine() so you have it already, by combining a test with sys.platform. > > And for linux, what is 'linux2' ? It should just be linux.. to > make it less confusing. That's how the platform is called, that's not created by Python (it's what uname -s returns) see http://docs.python.org/library/sys.html Tarek. From ziade.tarek at gmail.com Tue Oct 20 22:11:03 2009 From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=) Date: Tue, 20 Oct 2009 22:11:03 +0200 Subject: [Distutils] [RFC] PEP 345 and PEP 386 updates In-Reply-To: <4ADDD552.5070502@egenix.com> References: <94bdd2610910200139w5db29003pab66d5febca7ee4e@mail.gmail.com> <4ADDA10C.5010804@egenix.com> <9cee7ab80910200636p2384615fy516a0ce924ec0567@mail.gmail.com> <4ADDD552.5070502@egenix.com> Message-ID: <94bdd2610910201311n3b1d33d0i8e9142d716d444f3@mail.gmail.com> On Tue, Oct 20, 2009 at 5:20 PM, M.-A. Lemburg wrote: >>> The micro-language should provide a limited number of >>> variables to use on the conditions: >>> >>> python_version = sys.version >>> sys_platform = sys.platform >> >> If we adopt such a micro-language (I'm reserving judgment until I've >> had more time to read the relevant PEPs carefully), I'd rather see the >> names match what's in the Python runtime more closely, probably only >> avoiding the call syntax. > > We could do that as well, yes. +1 on close names + no call syntax but, notice that "python_version" here is: "%s.%s" % sys.version_info[0], sys.version_info[1] thats why we came up with that name, to disinguish it from sys.version (that contains more stuff) Tarek From a.badger at gmail.com Tue Oct 20 22:11:28 2009 From: a.badger at gmail.com (Toshio Kuratomi) Date: Tue, 20 Oct 2009 13:11:28 -0700 Subject: [Distutils] why would you ever need to specify setuptools as a dependency? In-Reply-To: <4ADDBFCA.40902@simplistix.co.uk> References: <4ACF1956.5070903@simplistix.co.uk> <5cae42b20910090457p3f6e4a1bp361b240f974f1617@mail.gmail.com> <5cae42b20910131055s5e152d62r4f7d385b1de98c9f@mail.gmail.com> <4ADD92F9.5050404@simplistix.co.uk> <4ADD997C.3020808@cryptelium.net> <9cee7ab80910200431j579bbe4fke78aa93f8c28729d@mail.gmail.com> <4ADDBD74.3080004@simplistix.co.uk> <9cee7ab80910200643q1617b460pb7fd98007d52275a@mail.gmail.com> <4ADDBFCA.40902@simplistix.co.uk> Message-ID: <20091020201128.GA31378@clingman.lan> On Tue, Oct 20, 2009 at 02:48:58PM +0100, Chris Withers wrote: > Fred Drake wrote: >> On Tue, Oct 20, 2009 at 9:39 AM, Chris Withers wrote: >>> As is specifying the setuptools distribution as a requirement when you're >>> already using it... >> >> I don't use setuptools at runtime unless something requires it. >> >> Having it available at install time and run time are two different >> things, and should remain so. > > All I'm saying is that packages shouldn't express a dependency on > setuptools if setuptools is required even before that expression can be > parsed. > > I'm not talking about install or run time... > Are you then talking about build time? -Toshio -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 197 bytes Desc: not available URL: From eric at trueblade.com Tue Oct 20 22:20:25 2009 From: eric at trueblade.com (Eric Smith) Date: Tue, 20 Oct 2009 16:20:25 -0400 Subject: [Distutils] [RFC] PEP 345 and PEP 386 updates In-Reply-To: <94bdd2610910201311n3b1d33d0i8e9142d716d444f3@mail.gmail.com> References: <94bdd2610910200139w5db29003pab66d5febca7ee4e@mail.gmail.com> <4ADDA10C.5010804@egenix.com> <9cee7ab80910200636p2384615fy516a0ce924ec0567@mail.gmail.com> <4ADDD552.5070502@egenix.com> <94bdd2610910201311n3b1d33d0i8e9142d716d444f3@mail.gmail.com> Message-ID: <4ADE1B89.5020804@trueblade.com> Tarek Ziad? wrote: > but, notice that "python_version" here is: "%s.%s" % > sys.version_info[0], sys.version_info[1] I think you mean: "{0[0]}.{0[1]}".format(sys.version_info) or better: "{0.major}.{0.minor}".format(sys.version_info) :) On the odd occasion that .format is less verbose than %, I like pointing it out! Eric. From ziade.tarek at gmail.com Tue Oct 20 22:23:55 2009 From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=) Date: Tue, 20 Oct 2009 22:23:55 +0200 Subject: [Distutils] [RFC] PEP 345 and PEP 386 updates In-Reply-To: <4ADE1B89.5020804@trueblade.com> References: <94bdd2610910200139w5db29003pab66d5febca7ee4e@mail.gmail.com> <4ADDA10C.5010804@egenix.com> <9cee7ab80910200636p2384615fy516a0ce924ec0567@mail.gmail.com> <4ADDD552.5070502@egenix.com> <94bdd2610910201311n3b1d33d0i8e9142d716d444f3@mail.gmail.com> <4ADE1B89.5020804@trueblade.com> Message-ID: <94bdd2610910201323x25afa718mc488fdaced5e037e@mail.gmail.com> On Tue, Oct 20, 2009 at 10:20 PM, Eric Smith wrote: > Tarek Ziad? wrote: >> >> but, notice that "python_version" here is: ? "%s.%s" % >> sys.version_info[0], sys.version_info[1] > > I think you mean: > "{0[0]}.{0[1]}".format(sys.version_info) > or better: > "{0.major}.{0.minor}".format(sys.version_info) > > :) > On the odd occasion that .format is less verbose than %, I like pointing it > out! Ah yes, that's nicer :) From david at ar.media.kyoto-u.ac.jp Wed Oct 21 03:45:47 2009 From: david at ar.media.kyoto-u.ac.jp (David Cournapeau) Date: Wed, 21 Oct 2009 10:45:47 +0900 Subject: [Distutils] [RFC] PEP 345 and PEP 386 updates In-Reply-To: <94bdd2610910201305j12691b33pc57ed2741b64d057@mail.gmail.com> References: <94bdd2610910200139w5db29003pab66d5febca7ee4e@mail.gmail.com> <4ADDA10C.5010804@egenix.com> <9cee7ab80910200636p2384615fy516a0ce924ec0567@mail.gmail.com> <94bdd2610910201305j12691b33pc57ed2741b64d057@mail.gmail.com> Message-ID: <4ADE67CB.2080307@ar.media.kyoto-u.ac.jp> Tarek Ziad? wrote: > No, the architecture is given by platform.machine() so you have it already, > by combining a test with sys.platform. But platform.machine() returns a 32 bits machine if you run 32 bits python on 64 bits windows. AFAIK, the only reliable way is to use the win32 API: http://blogs.msdn.com/oldnewthing/archive/2005/02/01/364563.aspx In scons (where we limit ourself to pure python code ATM), we tried the registry, but this is not reliable (some braindead softwares install WoW64 registry keys even on 32 bits windows). OTOH, I guess not that many applications need this information: after all, the whole point of WoW64 is to virtualize 32 bits applications so that they believe to run on a conventional 32 bits python. David From david.lyon at preisshare.net Wed Oct 21 07:59:16 2009 From: david.lyon at preisshare.net (David Lyon) Date: Wed, 21 Oct 2009 01:59:16 -0400 Subject: [Distutils] Alternate static metadata PEP submission... In-Reply-To: <4ADDB2CB.1090101@simplistix.co.uk> References: <20091013103021.GA21066@laurie.devork> <94bdd2610910131001v70711ffeia27433cd23b32fca@mail.gmail.com> <79a53e276e5dffe24be0433771801d72@preisshare.net> <94bdd2610910131426k64f707d7pf4d398d04b8ecc32@mail.gmail.com> <20091013233654.GA27339@laurie.devork> <4ADDB2CB.1090101@simplistix.co.uk> Message-ID: <8e9d1c135f56a0b37f12afe144b30f1a@preisshare.net> Hi Chris, Well I put my code up here: https://bitbucket.org/djlyon/original-setup.py-with-metadata/ Plenty on the todo list.. David On Tue, 20 Oct 2009 13:53:31 +0100, Chris Withers wrote: > David Lyon wrote: >> What we actually need, and I am building, is a setup.py script >> that is going to be a 'trimmed' version of distutils that includes >> all neccessary package/application installation code to do the install >> from the setup.info file. > > Be very aware of the problems of proliferating boilerplate code > (buildout has already suffered this): > > - old versions will have to work forever > > - bug fixes are *very* hard to do > > I would like my setup.py to look like: > > from xxx import setup > setup() > > ...and have all the code be in xxx. If xxx is a separately available > package for python<2.7, then fine. > > I would prefer to see "hard logic" in setup.py where there is a good > programming language to support it any everything else in setup.cfg > (don't like .info, sorry ;-) ). My hunch is that for 95% of packages, > setup.py will be nothing more than the above... > > Chris From chris at simplistix.co.uk Wed Oct 21 09:24:06 2009 From: chris at simplistix.co.uk (Chris Withers) Date: Wed, 21 Oct 2009 08:24:06 +0100 Subject: [Distutils] Alternate static metadata PEP submission... In-Reply-To: <8e9d1c135f56a0b37f12afe144b30f1a@preisshare.net> References: <20091013103021.GA21066@laurie.devork> <94bdd2610910131001v70711ffeia27433cd23b32fca@mail.gmail.com> <79a53e276e5dffe24be0433771801d72@preisshare.net> <94bdd2610910131426k64f707d7pf4d398d04b8ecc32@mail.gmail.com> <20091013233654.GA27339@laurie.devork> <4ADDB2CB.1090101@simplistix.co.uk> <8e9d1c135f56a0b37f12afe144b30f1a@preisshare.net> Message-ID: <4ADEB716.5020605@simplistix.co.uk> David Lyon wrote: > Well I put my code up here: > > https://bitbucket.org/djlyon/original-setup.py-with-metadata/ The thing is, there's just no need for all this code to be in setup.py. For me, anything more than: from xxx import setup setup() ...in the case where all the information is in setup.cfg is totally unacceptable. Chris -- Simplistix - Content Management, Batch Processing & Python Consulting - http://www.simplistix.co.uk From chris at simplistix.co.uk Wed Oct 21 09:29:02 2009 From: chris at simplistix.co.uk (Chris Withers) Date: Wed, 21 Oct 2009 08:29:02 +0100 Subject: [Distutils] Alternate static metadata PEP submission... In-Reply-To: <56bcb7f7893e86ec565d89b7ef40f418@preisshare.net> References: <20091013103021.GA21066@laurie.devork> <94bdd2610910131001v70711ffeia27433cd23b32fca@mail.gmail.com> <79a53e276e5dffe24be0433771801d72@preisshare.net> <94bdd2610910131426k64f707d7pf4d398d04b8ecc32@mail.gmail.com> <20091013233654.GA27339@laurie.devork> <4ADDB2CB.1090101@simplistix.co.uk> <56bcb7f7893e86ec565d89b7ef40f418@preisshare.net> Message-ID: <4ADEB83E.8080804@simplistix.co.uk> David Lyon wrote: > So I will build an option in to check on the net for an updated > version before starting. That won't fly. A significant number of people will be installing in situations where setup.py will have no access to the internet. Why is it that you basically want to put an entire module's worth of code inside every single package's setup.py? Take something like errorhandler: http://pypi.python.org/pypi/errorhandler The actual code of the package is only about 30 lines long. Having to include a 500 line setup.py just so I can include metadata in the packages seem pretty insane ;-) Chris -- Simplistix - Content Management, Batch Processing & Python Consulting - http://www.simplistix.co.uk From david.lyon at preisshare.net Wed Oct 21 09:49:59 2009 From: david.lyon at preisshare.net (David Lyon) Date: Wed, 21 Oct 2009 03:49:59 -0400 Subject: [Distutils] Alternate static metadata PEP submission... In-Reply-To: <4ADEB83E.8080804@simplistix.co.uk> References: <20091013103021.GA21066@laurie.devork> <94bdd2610910131001v70711ffeia27433cd23b32fca@mail.gmail.com> <79a53e276e5dffe24be0433771801d72@preisshare.net> <94bdd2610910131426k64f707d7pf4d398d04b8ecc32@mail.gmail.com> <20091013233654.GA27339@laurie.devork> <4ADDB2CB.1090101@simplistix.co.uk> <56bcb7f7893e86ec565d89b7ef40f418@preisshare.net> <4ADEB83E.8080804@simplistix.co.uk> Message-ID: <404efdf0624b873e17727d1f5966944d@preisshare.net> On Wed, 21 Oct 2009 08:29:02 +0100, Chris Withers wrote: > David Lyon wrote: >> So I will build an option in to check on the net for an updated >> version before starting. > > That won't fly. but it will most definitely work... and it will be backwards compatable with older python installations. > A significant number of people will be installing in situations where > setup.py will have no access to the internet. Of course. So an update check won't make it through. > Why is it that you basically want to put an entire module's worth of > code inside every single package's setup.py? Because it is easy. Let me put it another way - how does a 'normal-user' update distutils in the stdlib? > Take something like errorhandler: > > http://pypi.python.org/pypi/errorhandler > > The actual code of the package is only about 30 lines long. > > Having to include a 500 line setup.py just so I can include metadata in > the packages seem pretty insane ;-) Well, 500 lines is just this week. Maybe by next week it will be even more. If the file is 100kb long, I don't think a user would care. I don't think users in the real world will look at the file.. and think "wow - that's 100k.. we can't possibly run that program on our new quad core 3ghz machine with 4GB ram and 1TB disk space. To me, I think it will be the other way around.. I think they'll think "here's a crunchy setup.py.. lets run that". Same as always. Business as usual. David From chris at simplistix.co.uk Wed Oct 21 10:02:40 2009 From: chris at simplistix.co.uk (Chris Withers) Date: Wed, 21 Oct 2009 09:02:40 +0100 Subject: [Distutils] Alternate static metadata PEP submission... In-Reply-To: <404efdf0624b873e17727d1f5966944d@preisshare.net> References: <20091013103021.GA21066@laurie.devork> <94bdd2610910131001v70711ffeia27433cd23b32fca@mail.gmail.com> <79a53e276e5dffe24be0433771801d72@preisshare.net> <94bdd2610910131426k64f707d7pf4d398d04b8ecc32@mail.gmail.com> <20091013233654.GA27339@laurie.devork> <4ADDB2CB.1090101@simplistix.co.uk> <56bcb7f7893e86ec565d89b7ef40f418@preisshare.net> <4ADEB83E.8080804@simplistix.co.uk> <404efdf0624b873e17727d1f5966944d@preisshare.net> Message-ID: <4ADEC020.1050806@simplistix.co.uk> David Lyon wrote: >> Why is it that you basically want to put an entire module's worth of >> code inside every single package's setup.py? > > Because it is easy. > > Let me put it another way - how does a 'normal-user' update distutils > in the stdlib? easy_install/pip/etc distutils ...with the new version on PyPI. If that's not preferred, then they'd wait for Python 2.7, and do: from distutils.core import setup setup() > Well, 500 lines is just this week. Maybe by next week it will be even more. > > If the file is 100kb long, I don't think a user would care. I'm "a user" in this case, and I care... The maintenance burden a file like the setup.py you're proposing places on me as a package author in making sure the packages I maintain all include the right setup.py, and needing to bump my package's versions when code that has nothing to do with my package has bugs fixed isn't acceptable. > I don't think users in the real world will look at the file.. What happens for the author of one of the small number of packages that actually needs to have logic in setup.py? Chris -- Simplistix - Content Management, Batch Processing & Python Consulting - http://www.simplistix.co.uk From david.lyon at preisshare.net Wed Oct 21 10:02:11 2009 From: david.lyon at preisshare.net (David Lyon) Date: Wed, 21 Oct 2009 04:02:11 -0400 Subject: [Distutils] Alternate static metadata PEP submission... In-Reply-To: <4ADEB716.5020605@simplistix.co.uk> References: <20091013103021.GA21066@laurie.devork> <94bdd2610910131001v70711ffeia27433cd23b32fca@mail.gmail.com> <79a53e276e5dffe24be0433771801d72@preisshare.net> <94bdd2610910131426k64f707d7pf4d398d04b8ecc32@mail.gmail.com> <20091013233654.GA27339@laurie.devork> <4ADDB2CB.1090101@simplistix.co.uk> <8e9d1c135f56a0b37f12afe144b30f1a@preisshare.net> <4ADEB716.5020605@simplistix.co.uk> Message-ID: On Wed, 21 Oct 2009 08:24:06 +0100, Chris Withers wrote: > David Lyon wrote: >> Well I put my code up here: >> >> https://bitbucket.org/djlyon/original-setup.py-with-metadata/ > > The thing is, there's just no need for all this code to be in setup.py. > > For me, anything more than: > > from xxx import setup > setup() > > ...in the case where all the information is in setup.cfg is totally > unacceptable. so you are a minimalist... me too... but I want my proposed features implemented.. and code is the only way to do it. David :-) "what's a few thousand lines of code between friends" :-) From ziade.tarek at gmail.com Wed Oct 21 10:08:34 2009 From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=) Date: Wed, 21 Oct 2009 10:08:34 +0200 Subject: [Distutils] [RFC] PEP 345 and PEP 386 updates In-Reply-To: <4ADE67CB.2080307@ar.media.kyoto-u.ac.jp> References: <94bdd2610910200139w5db29003pab66d5febca7ee4e@mail.gmail.com> <4ADDA10C.5010804@egenix.com> <9cee7ab80910200636p2384615fy516a0ce924ec0567@mail.gmail.com> <94bdd2610910201305j12691b33pc57ed2741b64d057@mail.gmail.com> <4ADE67CB.2080307@ar.media.kyoto-u.ac.jp> Message-ID: <94bdd2610910210108n1ea0f683sc9c0f46f12c0c815@mail.gmail.com> On Wed, Oct 21, 2009 at 3:45 AM, David Cournapeau wrote: > Tarek Ziad? wrote: >> No, the architecture is given by platform.machine() so you have it already, >> by combining a test with sys.platform. > > But platform.machine() returns a 32 bits machine if you run 32 bits > python on 64 bits windows. AFAIK, the only reliable way is to use the > win32 API: > > http://blogs.msdn.com/oldnewthing/archive/2005/02/01/364563.aspx > > In scons (where we limit ourself to pure python code ATM), we tried the > registry, but this is not reliable (some braindead softwares install > WoW64 registry keys even on 32 bits windows). OTOH, I guess not that > many applications need this information: after all, the whole point of > WoW64 is to virtualize 32 bits applications so that they believe to run > on a conventional 32 bits python. There's something unclear for me here : What is suppose to happen when you use Python 32bits on a 64bits machine, for Python distributions that get build or installed ? Do you end up in a mixed environment of 64bits distributions within a 32bits interpreter ? From david.lyon at preisshare.net Wed Oct 21 10:14:41 2009 From: david.lyon at preisshare.net (David Lyon) Date: Wed, 21 Oct 2009 04:14:41 -0400 Subject: [Distutils] Alternate static metadata PEP submission... In-Reply-To: <4ADEC020.1050806@simplistix.co.uk> References: <20091013103021.GA21066@laurie.devork> <94bdd2610910131001v70711ffeia27433cd23b32fca@mail.gmail.com> <79a53e276e5dffe24be0433771801d72@preisshare.net> <94bdd2610910131426k64f707d7pf4d398d04b8ecc32@mail.gmail.com> <20091013233654.GA27339@laurie.devork> <4ADDB2CB.1090101@simplistix.co.uk> <56bcb7f7893e86ec565d89b7ef40f418@preisshare.net> <4ADEB83E.8080804@simplistix.co.uk> <404efdf0624b873e17727d1f5966944d@preisshare.net> <4ADEC020.1050806@simplistix.co.uk> Message-ID: <54d034c30100528e63b573ec3d7b137a@preisshare.net> On Wed, 21 Oct 2009 09:02:40 +0100, Chris Withers wrote: >> If the file is 100kb long, I don't think a user would care. > > I'm "a user" in this case, and I care... The maintenance burden a file > like the setup.py you're proposing places on me as a package author in > making sure the packages I maintain all include the right setup.py, and > needing to bump my package's versions when code that has nothing to do > with my package has bugs fixed isn't acceptable. ok. But nobody I know in software can stop new patches and versions of software from being made. Versions advance.. For some it's even a day job. >> I don't think users in the real world will look at the file.. > > What happens for the author of one of the small number of packages that > actually needs to have logic in setup.py? That's another issue. I'm providing external pre-install and post-install script hooks. To answer the question - It's too early to tell. I'm flexible. We'll see. A lot of code to write still. David From david at ar.media.kyoto-u.ac.jp Wed Oct 21 10:20:33 2009 From: david at ar.media.kyoto-u.ac.jp (David Cournapeau) Date: Wed, 21 Oct 2009 17:20:33 +0900 Subject: [Distutils] [RFC] PEP 345 and PEP 386 updates In-Reply-To: <94bdd2610910210108n1ea0f683sc9c0f46f12c0c815@mail.gmail.com> References: <94bdd2610910200139w5db29003pab66d5febca7ee4e@mail.gmail.com> <4ADDA10C.5010804@egenix.com> <9cee7ab80910200636p2384615fy516a0ce924ec0567@mail.gmail.com> <94bdd2610910201305j12691b33pc57ed2741b64d057@mail.gmail.com> <4ADE67CB.2080307@ar.media.kyoto-u.ac.jp> <94bdd2610910210108n1ea0f683sc9c0f46f12c0c815@mail.gmail.com> Message-ID: <4ADEC451.4010209@ar.media.kyoto-u.ac.jp> Tarek Ziad? wrote: > There's something unclear for me here : > > What is suppose to happen when you use Python 32bits on a 64bits machine, > for Python distributions that get build or installed ? > > Do you end up in a mixed environment of 64bits distributions within a > 32bits interpreter ? > Hopefully not - you obviously cannot load 64 bits extensions in a 32 bits interpreter. I don't know the details, but WoW64 virtualizes everything to look like a 32 bits windows to 32 bits processes (the registry is different, etc...). But you could imagine that independently on whether the interpreter is 32 or 64 bits, you need to process things differently on windows 32 and 64 bits (e.g. a 32 bits python on xp and a 32 bits python on xp 64 may have different requirements). FWIW, the user-case in scons was detecting Visual Studio. David From mal at egenix.com Wed Oct 21 10:42:30 2009 From: mal at egenix.com (M.-A. Lemburg) Date: Wed, 21 Oct 2009 10:42:30 +0200 Subject: [Distutils] [RFC] PEP 345 and PEP 386 updates In-Reply-To: <94bdd2610910201311n3b1d33d0i8e9142d716d444f3@mail.gmail.com> References: <94bdd2610910200139w5db29003pab66d5febca7ee4e@mail.gmail.com> <4ADDA10C.5010804@egenix.com> <9cee7ab80910200636p2384615fy516a0ce924ec0567@mail.gmail.com> <4ADDD552.5070502@egenix.com> <94bdd2610910201311n3b1d33d0i8e9142d716d444f3@mail.gmail.com> Message-ID: <4ADEC976.6070701@egenix.com> Tarek Ziad? wrote: > On Tue, Oct 20, 2009 at 5:20 PM, M.-A. Lemburg wrote: >>>> The micro-language should provide a limited number of >>>> variables to use on the conditions: >>>> >>>> python_version = sys.version >>>> sys_platform = sys.platform >>> >>> If we adopt such a micro-language (I'm reserving judgment until I've >>> had more time to read the relevant PEPs carefully), I'd rather see the >>> names match what's in the Python runtime more closely, probably only >>> avoiding the call syntax. >> >> We could do that as well, yes. > > +1 on close names + no call syntax > > but, notice that "python_version" here is: "%s.%s" % > sys.version_info[0], sys.version_info[1] > > thats why we came up with that name, to disinguish it from sys.version > (that contains more stuff) I think we do need the full version here, so perhaps using sys.version.split()[0] would be better or simply the full version string and then have conditions use >=, <=, <, > only (ie. not =, since that would likely never match). Note that packages do sometimes need to know that a certain Python patch level release is installed (e.g. Zope often has had this need) and a proper dependency checker would have to detect a possible mismatch. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Oct 21 2009) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try our new mxODBC.Connect Python Database Interface for free ! :::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/ From ziade.tarek at gmail.com Wed Oct 21 10:48:00 2009 From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=) Date: Wed, 21 Oct 2009 10:48:00 +0200 Subject: [Distutils] [RFC] PEP 345 and PEP 386 updates In-Reply-To: <4ADEC451.4010209@ar.media.kyoto-u.ac.jp> References: <94bdd2610910200139w5db29003pab66d5febca7ee4e@mail.gmail.com> <4ADDA10C.5010804@egenix.com> <9cee7ab80910200636p2384615fy516a0ce924ec0567@mail.gmail.com> <94bdd2610910201305j12691b33pc57ed2741b64d057@mail.gmail.com> <4ADE67CB.2080307@ar.media.kyoto-u.ac.jp> <94bdd2610910210108n1ea0f683sc9c0f46f12c0c815@mail.gmail.com> <4ADEC451.4010209@ar.media.kyoto-u.ac.jp> Message-ID: <94bdd2610910210148x2d12b8deh734da02dfc052761@mail.gmail.com> On Wed, Oct 21, 2009 at 10:20 AM, David Cournapeau wrote: > Tarek Ziad? wrote: >> There's something unclear for me here : >> >> What is suppose to happen when you use Python 32bits on a 64bits machine, >> for Python distributions that get build or installed ? >> >> Do you end up in a mixed environment of 64bits distributions within a >> 32bits interpreter ? >> > > Hopefully not - you obviously cannot load 64 bits extensions in a 32 > bits interpreter. I don't know the details, but WoW64 virtualizes > everything to look like a 32 bits windows to 32 bits processes (the > registry is different, etc...). But you could imagine that independently > on whether the interpreter is 32 or 64 bits, you need to process things > differently on windows 32 and 64 bits (e.g. a 32 bits python on xp and a > 32 bits python on xp 64 may have different requirements). FWIW, the > user-case in scons was detecting Visual Studio. The micro-language, though, is about the metadata fields like "requires" So do you have a use case where a Python distribution needs to use "64 bit (the machine)" in order to provide a different requires ? Tarek From david at ar.media.kyoto-u.ac.jp Wed Oct 21 10:49:52 2009 From: david at ar.media.kyoto-u.ac.jp (David Cournapeau) Date: Wed, 21 Oct 2009 17:49:52 +0900 Subject: [Distutils] Alternate static metadata PEP submission... In-Reply-To: <4ADEB716.5020605@simplistix.co.uk> References: <20091013103021.GA21066@laurie.devork> <94bdd2610910131001v70711ffeia27433cd23b32fca@mail.gmail.com> <79a53e276e5dffe24be0433771801d72@preisshare.net> <94bdd2610910131426k64f707d7pf4d398d04b8ecc32@mail.gmail.com> <20091013233654.GA27339@laurie.devork> <4ADDB2CB.1090101@simplistix.co.uk> <8e9d1c135f56a0b37f12afe144b30f1a@preisshare.net> <4ADEB716.5020605@simplistix.co.uk> Message-ID: <4ADECB30.10605@ar.media.kyoto-u.ac.jp> Chris Withers wrote: > The thing is, there's just no need for all this code to be in setup.py. > > For me, anything more than: > > from xxx import setup > setup() > > ...in the case where all the information is in setup.cfg is totally > unacceptable. FWIW, I have played a bit with something in that direction - this is not a suggestion for anything at that point, just a toy prototype to play with some ideas I have concerning a new distribution tool in python: http://github.com/cournape/toydist The basis is a static file format, but richer that the proposed PEP 390: - the goal is to be able to describe fully a package from the static data: meta-data, packages, doc, extensions, etc... - converting from existing setup.py to a static file is possible through an API (one could have a distutils command to automatically generate such a file) - a two lines toysetup.py can build basic packages from an existing toysetup.info static file - for the majority of packages out there (the ones which do not extend distutils, basically), the passage setup.py -> toysetup.info -> toysetup.py should be "lossless". This is huge IMHO, because it provides a transition to a system which does not depend on distutils anymore. An example of the static file is: """ Name: hello Version: 1.0 Library: BuildRequires: paver, sphinx, numpy if os(windows) BuildRequires: pywin32 Packages: hello Extension: hello._bar sources: src/hellomodule.c if os(linux) Extension: hello._linux_backend sources: src/linbackend.c """ The corresponding setup.py is simply: """ from distutils.core import setup from toydist import parse_static info_dict = parse_static('toysetup.info').to_dict() setup(**info_dict) """ The format is basically a rip-of from cabal (haskell distribution format) at this point, cheers, David From david at ar.media.kyoto-u.ac.jp Wed Oct 21 10:58:07 2009 From: david at ar.media.kyoto-u.ac.jp (David Cournapeau) Date: Wed, 21 Oct 2009 17:58:07 +0900 Subject: [Distutils] [RFC] PEP 345 and PEP 386 updates In-Reply-To: <94bdd2610910210148x2d12b8deh734da02dfc052761@mail.gmail.com> References: <94bdd2610910200139w5db29003pab66d5febca7ee4e@mail.gmail.com> <4ADDA10C.5010804@egenix.com> <9cee7ab80910200636p2384615fy516a0ce924ec0567@mail.gmail.com> <94bdd2610910201305j12691b33pc57ed2741b64d057@mail.gmail.com> <4ADE67CB.2080307@ar.media.kyoto-u.ac.jp> <94bdd2610910210108n1ea0f683sc9c0f46f12c0c815@mail.gmail.com> <4ADEC451.4010209@ar.media.kyoto-u.ac.jp> <94bdd2610910210148x2d12b8deh734da02dfc052761@mail.gmail.com> Message-ID: <4ADECD1F.9040509@ar.media.kyoto-u.ac.jp> Tarek Ziad? wrote: > The micro-language, though, is about the metadata fields like "requires" > > So do you have a use case where a Python distribution needs to use "64 > bit (the machine)" > Given the difficulty, I think such a field should only be added once python itself has a method to reliably detect windows 64, to avoid any inconsistencies. David From ziade.tarek at gmail.com Wed Oct 21 11:27:05 2009 From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=) Date: Wed, 21 Oct 2009 11:27:05 +0200 Subject: [Distutils] [RFC] PEP 345 and PEP 386 updates In-Reply-To: <4ADECD1F.9040509@ar.media.kyoto-u.ac.jp> References: <94bdd2610910200139w5db29003pab66d5febca7ee4e@mail.gmail.com> <4ADDA10C.5010804@egenix.com> <9cee7ab80910200636p2384615fy516a0ce924ec0567@mail.gmail.com> <94bdd2610910201305j12691b33pc57ed2741b64d057@mail.gmail.com> <4ADE67CB.2080307@ar.media.kyoto-u.ac.jp> <94bdd2610910210108n1ea0f683sc9c0f46f12c0c815@mail.gmail.com> <4ADEC451.4010209@ar.media.kyoto-u.ac.jp> <94bdd2610910210148x2d12b8deh734da02dfc052761@mail.gmail.com> <4ADECD1F.9040509@ar.media.kyoto-u.ac.jp> Message-ID: <94bdd2610910210227l33691f17hb4129ec21cd57c7a@mail.gmail.com> On Wed, Oct 21, 2009 at 10:58 AM, David Cournapeau wrote: > Tarek Ziad? wrote: >> The micro-language, though, is about the metadata fields like "requires" >> >> So do you have a use case where a Python distribution needs to use "64 >> bit (the machine)" >> > > Given the difficulty, I think such a field should only be added once > python itself has a method to reliably detect windows 64, to avoid any > inconsistencies. Ok then. Notice that introducing new variables afterwards in the micro-language will be easy Tarek From ozarow at gmail.com Wed Oct 21 11:24:12 2009 From: ozarow at gmail.com (Piotr Ozarowski) Date: Wed, 21 Oct 2009 11:24:12 +0200 Subject: [Distutils] Alternate static metadata PEP submission... In-Reply-To: <4ADEC020.1050806@simplistix.co.uk> References: <79a53e276e5dffe24be0433771801d72@preisshare.net> <94bdd2610910131426k64f707d7pf4d398d04b8ecc32@mail.gmail.com> <20091013233654.GA27339@laurie.devork> <4ADDB2CB.1090101@simplistix.co.uk> <56bcb7f7893e86ec565d89b7ef40f418@preisshare.net> <4ADEB83E.8080804@simplistix.co.uk> <404efdf0624b873e17727d1f5966944d@preisshare.net> <4ADEC020.1050806@simplistix.co.uk> Message-ID: <20091021092412.GB32502@piotro.eu> Hi list, I'm a Debian developer who is interested in changes you want to introduce so I'll add my 2 cents from time to time if you don't mind. [Chris Withers, 2009-10-21] > David Lyon wrote: >>> Why is it that you basically want to put an entire module's worth of >>> code inside every single package's setup.py? >> >> Because it is easy. and when there's a bug in this module you want all packages to be reuploaded instead of uploading the one with a bug? We already have to patch setup.py files f.e. to remove ez_setup which hardcodes setuptools versions (so we cannot use newer one) or tries to download packages from internet (that are not really needed at build time). >> Well, 500 lines is just this week. Maybe by next week it will be even more. >> >> If the file is 100kb long, I don't think a user would care. oh, reviewing 100kb setup.py files to see why they don't work with Python 2.X (and worked fine with Python 2.Y) sounds like fun ;-P -- Piotr O?arowski Debian GNU/Linux Developer www.ozarowski.pl www.griffith.cc www.debian.org GPG Fingerprint: 1D2F A898 58DA AF62 1786 2DF7 AEF6 F1A2 A745 7645 From ziade.tarek at gmail.com Wed Oct 21 11:43:37 2009 From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=) Date: Wed, 21 Oct 2009 11:43:37 +0200 Subject: [Distutils] [RFC] PEP 345 and PEP 386 updates In-Reply-To: <4ADEC976.6070701@egenix.com> References: <94bdd2610910200139w5db29003pab66d5febca7ee4e@mail.gmail.com> <4ADDA10C.5010804@egenix.com> <9cee7ab80910200636p2384615fy516a0ce924ec0567@mail.gmail.com> <4ADDD552.5070502@egenix.com> <94bdd2610910201311n3b1d33d0i8e9142d716d444f3@mail.gmail.com> <4ADEC976.6070701@egenix.com> Message-ID: <94bdd2610910210243y7151ff59u2216c6889a7ac887@mail.gmail.com> On Wed, Oct 21, 2009 at 10:42 AM, M.-A. Lemburg wrote: > Tarek Ziad? wrote: >> On Tue, Oct 20, 2009 at 5:20 PM, M.-A. Lemburg wrote: >>>>> The micro-language should provide a limited number of >>>>> variables to use on the conditions: >>>>> >>>>> python_version = sys.version >>>>> sys_platform = sys.platform >>>> >>>> If we adopt such a micro-language (I'm reserving judgment until I've >>>> had more time to read the relevant PEPs carefully), I'd rather see the >>>> names match what's in the Python runtime more closely, probably only >>>> avoiding the call syntax. >>> >>> We could do that as well, yes. >> >> +1 on close names + no call syntax >> >> but, notice that "python_version" here is: ? "%s.%s" % >> sys.version_info[0], sys.version_info[1] >> >> thats why we came up with that name, to disinguish it from sys.version >> (that contains more stuff) > > I think we do need the full version here, so perhaps using > sys.version.split()[0] would be better or simply the full > version string and then have conditions use >=, <=, <, > > only (ie. not =, since that would likely never match). Notice that for the comparisons >, < to work, that requires more than string comparisons in case we provide the ability to work with rc versions, because: >>> '2.6.3rc4' < '2.6.3' False We said earlier that we could use sys.hexversion to handle this, but otherwise, at least for the Python version, we can use the StrictVersion() class from Distutils, *But* we ended up thinking that it would be better for the first version to keep just "==", "!=", "in" and "not in" operators and a MAJOR.MINOR string. These are enough just to compare strings, on a limited number of versions for Python. What use case you have in mind for a three numbers version ? > > Note that packages do sometimes need to know that a certain > Python patch level release is installed (e.g. Zope often has > had this need) and a proper dependency checker would have to > detect a possible mismatch. I see. I would rather have this patch info in another variable, and restrict the micro language capabilities as much as possible. Any hint were Zope does this ? Regards Tarek From mal at egenix.com Wed Oct 21 11:53:43 2009 From: mal at egenix.com (M.-A. Lemburg) Date: Wed, 21 Oct 2009 11:53:43 +0200 Subject: [Distutils] [RFC] PEP 345 and PEP 386 updates In-Reply-To: <94bdd2610910210243y7151ff59u2216c6889a7ac887@mail.gmail.com> References: <94bdd2610910200139w5db29003pab66d5febca7ee4e@mail.gmail.com> <4ADDA10C.5010804@egenix.com> <9cee7ab80910200636p2384615fy516a0ce924ec0567@mail.gmail.com> <4ADDD552.5070502@egenix.com> <94bdd2610910201311n3b1d33d0i8e9142d716d444f3@mail.gmail.com> <4ADEC976.6070701@egenix.com> <94bdd2610910210243y7151ff59u2216c6889a7ac887@mail.gmail.com> Message-ID: <4ADEDA27.1090505@egenix.com> Tarek Ziad? wrote: > On Wed, Oct 21, 2009 at 10:42 AM, M.-A. Lemburg wrote: >> Tarek Ziad? wrote: >>> On Tue, Oct 20, 2009 at 5:20 PM, M.-A. Lemburg wrote: >>>>>> The micro-language should provide a limited number of >>>>>> variables to use on the conditions: >>>>>> >>>>>> python_version = sys.version >>>>>> sys_platform = sys.platform >>>>> >>>>> If we adopt such a micro-language (I'm reserving judgment until I've >>>>> had more time to read the relevant PEPs carefully), I'd rather see the >>>>> names match what's in the Python runtime more closely, probably only >>>>> avoiding the call syntax. >>>> >>>> We could do that as well, yes. >>> >>> +1 on close names + no call syntax >>> >>> but, notice that "python_version" here is: "%s.%s" % >>> sys.version_info[0], sys.version_info[1] >>> >>> thats why we came up with that name, to disinguish it from sys.version >>> (that contains more stuff) >> >> I think we do need the full version here, so perhaps using >> sys.version.split()[0] would be better or simply the full >> version string and then have conditions use >=, <=, <, > >> only (ie. not =, since that would likely never match). > > Notice that for the comparisons >, < to work, that requires more than string > comparisons in case we provide the ability to work with rc versions, because: > >>>> '2.6.3rc4' < '2.6.3' > False > > We said earlier that we could use sys.hexversion to handle this, but otherwise, > at least for the Python version, we can use the StrictVersion() class > from Distutils, > > *But* we ended up thinking that it would be better for the first version to keep > just "==", "!=", "in" and "not in" operators and a MAJOR.MINOR string. > > These are enough just to compare strings, on a limited number of > versions for Python. Ok. > What use case you have in mind for a three numbers version ? This may occur when you for example rely on PyXML, elementtree or simplejson which are also available as external packages (with a different release cycle). E.g. say there's a bug in the JSON package included in Python versions 2.6.0, 2.6.1, 2.6.2 which is fixed in 2.6.3 and has also been fixed in the PyPI version of the package. Now say that a system ships with 2.6.1 as Python version. For Python versions with the bug, you'd add a requirement for the external package in order to work around the bug in the system installation of Python. For Python versions >= 2.6.3, you'd continue to use the package from the system provided Python installation. It's a rather rare use case, so just something to think about for a future revision of the proposal. >> Note that packages do sometimes need to know that a certain >> Python patch level release is installed (e.g. Zope often has >> had this need) and a proper dependency checker would have to >> detect a possible mismatch. > > I see. I would rather have this patch info in another variable, and restrict > the micro language capabilities as much as possible. > > Any hint were Zope does this ? Early Zope versions simply warned about the problem in the release documentation. IIRC, later versions actually tested the installed Python version and warned about issues during the installation process. I'm not sure how Zope 2.12 with all the tiny little packages handles this case. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Oct 21 2009) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try our new mxODBC.Connect Python Database Interface for free ! :::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/ From david at ar.media.kyoto-u.ac.jp Wed Oct 21 12:44:03 2009 From: david at ar.media.kyoto-u.ac.jp (David Cournapeau) Date: Wed, 21 Oct 2009 19:44:03 +0900 Subject: [Distutils] [RFC] PEP 345 and PEP 386 updates In-Reply-To: <94bdd2610910210243y7151ff59u2216c6889a7ac887@mail.gmail.com> References: <94bdd2610910200139w5db29003pab66d5febca7ee4e@mail.gmail.com> <4ADDA10C.5010804@egenix.com> <9cee7ab80910200636p2384615fy516a0ce924ec0567@mail.gmail.com> <4ADDD552.5070502@egenix.com> <94bdd2610910201311n3b1d33d0i8e9142d716d444f3@mail.gmail.com> <4ADEC976.6070701@egenix.com> <94bdd2610910210243y7151ff59u2216c6889a7ac887@mail.gmail.com> Message-ID: <4ADEE5F3.9020909@ar.media.kyoto-u.ac.jp> Tarek Ziad? wrote: > We said earlier that we could use sys.hexversion to handle this, but otherwise, > at least for the Python version, we can use the StrictVersion() class > from Distutils, > > *But* we ended up thinking that it would be better for the first version to keep > just "==", "!=", "in" and "not in" operators and a MAJOR.MINOR string. > > These are enough just to compare strings, on a limited number of > versions for Python. > > What use case you have in mind for a three numbers version ? Although it is best to avoid depending on micro number, one recent use-case I can think about is the added MSVC runtime information at python level (between python 2.6.1 and 2.6.2), which we use in numpy to make sure we link the exact same C runtime as python. David From david.lyon at preisshare.net Wed Oct 21 13:04:54 2009 From: david.lyon at preisshare.net (David Lyon) Date: Wed, 21 Oct 2009 07:04:54 -0400 Subject: [Distutils] Alternate static metadata PEP submission... In-Reply-To: <20091021092412.GB32502@piotro.eu> References: <79a53e276e5dffe24be0433771801d72@preisshare.net> <94bdd2610910131426k64f707d7pf4d398d04b8ecc32@mail.gmail.com> <20091013233654.GA27339@laurie.devork> <4ADDB2CB.1090101@simplistix.co.uk> <56bcb7f7893e86ec565d89b7ef40f418@preisshare.net> <4ADEB83E.8080804@simplistix.co.uk> <404efdf0624b873e17727d1f5966944d@preisshare.net> <4ADEC020.1050806@simplistix.co.uk> <20091021092412.GB32502@piotro.eu> Message-ID: <1de82ef8b2417b9a6ea6801c975c130c@preisshare.net> On Wed, 21 Oct 2009 11:24:12 +0200, Piotr Ozarowski wrote: > Hi list, > > I'm a Debian developer who is interested in changes you want to > introduce so I'll add my 2 cents from time to time if you don't mind. Hello. Yes, a virtual 2c is valuable. Certainly the best offer I've had all day.. > and when there's a bug in this module you want all packages to be > reuploaded instead of uploading the one with a bug? > > We already have to patch setup.py files f.e. to remove ez_setup which > hardcodes setuptools versions (so we cannot use newer one) or tries to > download packages from internet (that are not really needed at build > time). I'm not arguing with you. It's no more final a solution than any other piece of source code.. But what happens to code that isn't updated? it is considered as dead code. Then everybody hates that too. > oh, reviewing 100kb setup.py files to see why they don't work with > Python 2.X (and worked fine with Python 2.Y) sounds like fun ;-P Maybe some client hoped it would work on 2.X after seeing it working on 2.Y and then have to pay you to look through it... since you're the local python expert. I only hope they pay you a good rate and give you the cheque within 14 days... and you post the patch back... :-) Take care David From david.lyon at preisshare.net Wed Oct 21 13:18:41 2009 From: david.lyon at preisshare.net (David Lyon) Date: Wed, 21 Oct 2009 07:18:41 -0400 Subject: [Distutils] [RFC] PEP 345 and PEP 386 updates In-Reply-To: <94bdd2610910210148x2d12b8deh734da02dfc052761@mail.gmail.com> References: <94bdd2610910200139w5db29003pab66d5febca7ee4e@mail.gmail.com> <4ADDA10C.5010804@egenix.com> <9cee7ab80910200636p2384615fy516a0ce924ec0567@mail.gmail.com> <94bdd2610910201305j12691b33pc57ed2741b64d057@mail.gmail.com> <4ADE67CB.2080307@ar.media.kyoto-u.ac.jp> <94bdd2610910210108n1ea0f683sc9c0f46f12c0c815@mail.gmail.com> <4ADEC451.4010209@ar.media.kyoto-u.ac.jp> <94bdd2610910210148x2d12b8deh734da02dfc052761@mail.gmail.com> Message-ID: <6077cefd1d86388666446ade6ef045ac@preisshare.net> On Wed, 21 Oct 2009 10:48:00 +0200, Tarek Ziad? wrote: > So do you have a use case where a Python distribution needs to use "64 > bit (the machine)" > in order to provide a different requires ? I wouldn't be too worried about this '64' bit issue. Hypothetically, C library extensions can/could be different between 32 and 64 bit versions. Maybe issues might arise sometime. My original point was only that 'win32' constant from the stdlib sys module is an inaccurate constant to describe windows now. The stdlib platform module uses 'Windows' and that seems better. Regards David From eric at trueblade.com Wed Oct 21 13:38:17 2009 From: eric at trueblade.com (Eric Smith) Date: Wed, 21 Oct 2009 07:38:17 -0400 Subject: [Distutils] [RFC] PEP 345 and PEP 386 updates In-Reply-To: <94bdd2610910210243y7151ff59u2216c6889a7ac887@mail.gmail.com> References: <94bdd2610910200139w5db29003pab66d5febca7ee4e@mail.gmail.com> <4ADDA10C.5010804@egenix.com> <9cee7ab80910200636p2384615fy516a0ce924ec0567@mail.gmail.com> <4ADDD552.5070502@egenix.com> <94bdd2610910201311n3b1d33d0i8e9142d716d444f3@mail.gmail.com> <4ADEC976.6070701@egenix.com> <94bdd2610910210243y7151ff59u2216c6889a7ac887@mail.gmail.com> Message-ID: <4ADEF2A9.9020604@trueblade.com> Tarek Ziad? wrote: > On Wed, Oct 21, 2009 at 10:42 AM, M.-A. Lemburg wrote: >> Tarek Ziad? wrote: >>> On Tue, Oct 20, 2009 at 5:20 PM, M.-A. Lemburg wrote: >>>>>> The micro-language should provide a limited number of >>>>>> variables to use on the conditions: >>>>>> >>>>>> python_version = sys.version >>>>>> sys_platform = sys.platform >>>>> If we adopt such a micro-language (I'm reserving judgment until I've >>>>> had more time to read the relevant PEPs carefully), I'd rather see the >>>>> names match what's in the Python runtime more closely, probably only >>>>> avoiding the call syntax. >>>> We could do that as well, yes. >>> +1 on close names + no call syntax >>> >>> but, notice that "python_version" here is: "%s.%s" % >>> sys.version_info[0], sys.version_info[1] >>> >>> thats why we came up with that name, to disinguish it from sys.version >>> (that contains more stuff) >> I think we do need the full version here, so perhaps using >> sys.version.split()[0] would be better or simply the full >> version string and then have conditions use >=, <=, <, > >> only (ie. not =, since that would likely never match). > > Notice that for the comparisons >, < to work, that requires more than string > comparisons in case we provide the ability to work with rc versions, because: > >>>> '2.6.3rc4' < '2.6.3' > False > > We said earlier that we could use sys.hexversion to handle this, but otherwise, > at least for the Python version, we can use the StrictVersion() class > from Distutils, > > *But* we ended up thinking that it would be better for the first version to keep > just "==", "!=", "in" and "not in" operators and a MAJOR.MINOR string. > > These are enough just to compare strings, on a limited number of > versions for Python. I thought the plan was to stick with major.minor and "==", "!=", "in" and "not in" until PEP 386 was accepted, then switch over to having all versions (including Python's) be some sort of Version object, at which point we can have richer version comparisons, with all of the existing uses being compatible. Eric. From sridharr at activestate.com Wed Oct 21 13:44:05 2009 From: sridharr at activestate.com (Sridhar Ratnakumar) Date: Wed, 21 Oct 2009 17:14:05 +0530 Subject: [Distutils] [RFC] PEP 345 and PEP 386 updates In-Reply-To: <6077cefd1d86388666446ade6ef045ac@preisshare.net> References: <94bdd2610910200139w5db29003pab66d5febca7ee4e@mail.gmail.com> <4ADDA10C.5010804@egenix.com> <9cee7ab80910200636p2384615fy516a0ce924ec0567@mail.gmail.com> <94bdd2610910201305j12691b33pc57ed2741b64d057@mail.gmail.com> <4ADE67CB.2080307@ar.media.kyoto-u.ac.jp> <94bdd2610910210108n1ea0f683sc9c0f46f12c0c815@mail.gmail.com> <4ADEC451.4010209@ar.media.kyoto-u.ac.jp> <94bdd2610910210148x2d12b8deh734da02dfc052761@mail.gmail.com> <6077cefd1d86388666446ade6ef045ac@preisshare.net> Message-ID: On Wed, 21 Oct 2009 16:48:41 +0530, David Lyon wrote: > My original point was only that 'win32' constant from the stdlib sys > module > is an > inaccurate constant to describe windows now. The stdlib platform module > uses > 'Windows' and that seems better. +1 I suggest to first have a standard library module/api that would give such 'friendly' names. This is what `platinfo` is heading towards: http://code.google.com/p/platinfo/wiki/FutureWork .. to quote: I'm considering using "mac" instead of "macosx" for PlatInfo.os on Mac. Certainly not decided though. I guess my main motivation is that a sufficiently unambiguous platform name for a universal Mac package is "mac" ("macosx" seems a bit pointless). Will use "windows" for PlatInfo.os instead of "win32" or "win64". The old names were imperfect and misleading references to either what used to be called the "win32 API" (but is now the Windows API). -srid From ozarow at gmail.com Wed Oct 21 14:28:06 2009 From: ozarow at gmail.com (Piotr Ozarowski) Date: Wed, 21 Oct 2009 14:28:06 +0200 Subject: [Distutils] Alternate static metadata PEP submission... In-Reply-To: <1de82ef8b2417b9a6ea6801c975c130c@preisshare.net> References: <20091013233654.GA27339@laurie.devork> <4ADDB2CB.1090101@simplistix.co.uk> <56bcb7f7893e86ec565d89b7ef40f418@preisshare.net> <4ADEB83E.8080804@simplistix.co.uk> <404efdf0624b873e17727d1f5966944d@preisshare.net> <4ADEC020.1050806@simplistix.co.uk> <20091021092412.GB32502@piotro.eu> <1de82ef8b2417b9a6ea6801c975c130c@preisshare.net> Message-ID: <20091021122806.GA2966@piotro.eu> [David Lyon, 2009-10-21] > On Wed, 21 Oct 2009 11:24:12 +0200, Piotr Ozarowski > wrote: > > and when there's a bug in this module you want all packages to be > > reuploaded instead of uploading the one with a bug? > > > > We already have to patch setup.py files f.e. to remove ez_setup which > > hardcodes setuptools versions (so we cannot use newer one) or tries to > > download packages from internet (that are not really needed at build > > time). > > I'm not arguing with you. It's no more final a solution than any > other piece of source code.. > > But what happens to code that isn't updated? it is considered as > dead code. Then everybody hates that too. Last MySQLdb stable release was over 2 years ago (there's a release candidate available since March 2009) and to make it work with Python >= 2.6.3 you just have to fix/update setuptools. Now imagine every package would ship its own (old) version of setuptools (just like packages ship their own version of ez_setup now). Do you see a problem already? Sure, one could just: | try: | import setuptools | except ImportError: | import ez_setup | ez_setup.use_setuptools() and thus use installed version if it's available (replace ez_setup/setuptools with what you propose to ship in setup.py) but that's not what module authors do unfortunately. My point is: it's easier to fix a bug in one place and 100kb setup.py would contain a lot of potential bugs or incompatibilities with new Python versions. -- Piotr O?arowski Debian GNU/Linux Developer www.ozarowski.pl www.griffith.cc www.debian.org GPG Fingerprint: 1D2F A898 58DA AF62 1786 2DF7 AEF6 F1A2 A745 7645 From ziade.tarek at gmail.com Wed Oct 21 15:16:51 2009 From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=) Date: Wed, 21 Oct 2009 15:16:51 +0200 Subject: [Distutils] [RFC] PEP 345 and PEP 386 updates In-Reply-To: References: <94bdd2610910200139w5db29003pab66d5febca7ee4e@mail.gmail.com> <9cee7ab80910200636p2384615fy516a0ce924ec0567@mail.gmail.com> <94bdd2610910201305j12691b33pc57ed2741b64d057@mail.gmail.com> <4ADE67CB.2080307@ar.media.kyoto-u.ac.jp> <94bdd2610910210108n1ea0f683sc9c0f46f12c0c815@mail.gmail.com> <4ADEC451.4010209@ar.media.kyoto-u.ac.jp> <94bdd2610910210148x2d12b8deh734da02dfc052761@mail.gmail.com> <6077cefd1d86388666446ade6ef045ac@preisshare.net> Message-ID: <94bdd2610910210616y5815670qb88e0b92ab5a6ec6@mail.gmail.com> 2009/10/21 Sridhar Ratnakumar : [...] > I suggest to first have a standard library module/api that would give such > 'friendly' names. This is what `platinfo` is heading towards: > http://code.google.com/p/platinfo/wiki/FutureWork .. to quote: > [...] I think we should focus here on what Python's stdlib provides *today*, for PEP 345 (and 390) discussions. "platinfo" could be another proposal, and if it makes it to the stlib, then we can revise in the future 345 and 390 so they provide platinfo variables to work with. Tarek From ziade.tarek at gmail.com Wed Oct 21 15:18:16 2009 From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=) Date: Wed, 21 Oct 2009 15:18:16 +0200 Subject: [Distutils] [RFC] PEP 345 and PEP 386 updates In-Reply-To: <4ADEF2A9.9020604@trueblade.com> References: <94bdd2610910200139w5db29003pab66d5febca7ee4e@mail.gmail.com> <4ADDA10C.5010804@egenix.com> <9cee7ab80910200636p2384615fy516a0ce924ec0567@mail.gmail.com> <4ADDD552.5070502@egenix.com> <94bdd2610910201311n3b1d33d0i8e9142d716d444f3@mail.gmail.com> <4ADEC976.6070701@egenix.com> <94bdd2610910210243y7151ff59u2216c6889a7ac887@mail.gmail.com> <4ADEF2A9.9020604@trueblade.com> Message-ID: <94bdd2610910210618j58a493c5v922dd0c22c4c5f07@mail.gmail.com> On Wed, Oct 21, 2009 at 1:38 PM, Eric Smith wrote: > Tarek Ziad? wrote: >> >> On Wed, Oct 21, 2009 at 10:42 AM, M.-A. Lemburg wrote: >>> >>> Tarek Ziad? wrote: >>>> >>>> On Tue, Oct 20, 2009 at 5:20 PM, M.-A. Lemburg wrote: >>>>>>> >>>>>>> The micro-language should provide a limited number of >>>>>>> variables to use on the conditions: >>>>>>> >>>>>>> python_version = sys.version >>>>>>> sys_platform = sys.platform >>>>>> >>>>>> If we adopt such a micro-language (I'm reserving judgment until I've >>>>>> had more time to read the relevant PEPs carefully), I'd rather see the >>>>>> names match what's in the Python runtime more closely, probably only >>>>>> avoiding the call syntax. >>>>> >>>>> We could do that as well, yes. >>>> >>>> +1 on close names + no call syntax >>>> >>>> but, notice that "python_version" here is: ? "%s.%s" % >>>> sys.version_info[0], sys.version_info[1] >>>> >>>> thats why we came up with that name, to disinguish it from sys.version >>>> (that contains more stuff) >>> >>> I think we do need the full version here, so perhaps using >>> sys.version.split()[0] would be better or simply the full >>> version string and then have conditions use >=, <=, <, > >>> only (ie. not =, since that would likely never match). >> >> Notice that for the comparisons >, < to work, that requires more than >> string >> comparisons in case we provide the ability to work with rc versions, >> because: >> >>>>> '2.6.3rc4' < '2.6.3' >> >> False >> >> We said earlier that we could use sys.hexversion to handle this, but >> otherwise, >> at least for the Python version, we can use the StrictVersion() class >> from Distutils, >> >> *But* we ended up thinking that it would be better for the first version >> to keep >> just "==", "!=", "in" and "not in" operators and a MAJOR.MINOR string. >> >> These are enough just to compare strings, on a limited number of >> versions for Python. > > I thought the plan was to stick with major.minor and "==", "!=", "in" and > "not in" until PEP 386 was accepted, then switch over to having all versions > (including Python's) be some sort of Version object, at which point we can > have richer version comparisons, with all of the existing uses being > compatible. Yes that's the plan too. PEP 345 is not updated yet, but that is clearly stated in PEP 386. Tarek From tseaver at palladion.com Wed Oct 21 17:54:49 2009 From: tseaver at palladion.com (Tres Seaver) Date: Wed, 21 Oct 2009 11:54:49 -0400 Subject: [Distutils] [RFC] PEP 345 and PEP 386 updates In-Reply-To: <4ADEDA27.1090505@egenix.com> References: <94bdd2610910200139w5db29003pab66d5febca7ee4e@mail.gmail.com> <4ADDA10C.5010804@egenix.com> <9cee7ab80910200636p2384615fy516a0ce924ec0567@mail.gmail.com> <4ADDD552.5070502@egenix.com> <94bdd2610910201311n3b1d33d0i8e9142d716d444f3@mail.gmail.com> <4ADEC976.6070701@egenix.com> <94bdd2610910210243y7151ff59u2216c6889a7ac887@mail.gmail.com> <4ADEDA27.1090505@egenix.com> Message-ID: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 M.-A. Lemburg wrote: > Tarek Ziad? wrote: >> On Wed, Oct 21, 2009 at 10:42 AM, M.-A. Lemburg wrote: >>> Tarek Ziad? wrote: >>>> On Tue, Oct 20, 2009 at 5:20 PM, M.-A. Lemburg wrote: >>>>>>> The micro-language should provide a limited number of >>>>>>> variables to use on the conditions: >>>>>>> >>>>>>> python_version = sys.version >>>>>>> sys_platform = sys.platform >>>>>> If we adopt such a micro-language (I'm reserving judgment until I've >>>>>> had more time to read the relevant PEPs carefully), I'd rather see the >>>>>> names match what's in the Python runtime more closely, probably only >>>>>> avoiding the call syntax. >>>>> We could do that as well, yes. >>>> +1 on close names + no call syntax >>>> >>>> but, notice that "python_version" here is: "%s.%s" % >>>> sys.version_info[0], sys.version_info[1] >>>> >>>> thats why we came up with that name, to disinguish it from sys.version >>>> (that contains more stuff) >>> I think we do need the full version here, so perhaps using >>> sys.version.split()[0] would be better or simply the full >>> version string and then have conditions use >=, <=, <, > >>> only (ie. not =, since that would likely never match). >> Notice that for the comparisons >, < to work, that requires more than string >> comparisons in case we provide the ability to work with rc versions, because: >> >>>>> '2.6.3rc4' < '2.6.3' >> False >> >> We said earlier that we could use sys.hexversion to handle this, but otherwise, >> at least for the Python version, we can use the StrictVersion() class >> from Distutils, >> >> *But* we ended up thinking that it would be better for the first version to keep >> just "==", "!=", "in" and "not in" operators and a MAJOR.MINOR string. >> >> These are enough just to compare strings, on a limited number of >> versions for Python. > > Ok. > >> What use case you have in mind for a three numbers version ? > > This may occur when you for example rely on PyXML, elementtree > or simplejson which are also available as external packages > (with a different release cycle). > > E.g. say there's a bug in the JSON package included in Python > versions 2.6.0, 2.6.1, 2.6.2 which is fixed in 2.6.3 and has > also been fixed in the PyPI version of the package. > > Now say that a system ships with 2.6.1 as Python version. > > For Python versions with the bug, you'd add a requirement for > the external package in order to work around the bug in the > system installation of Python. For Python versions >= 2.6.3, > you'd continue to use the package from the system provided > Python installation. > > It's a rather rare use case, so just something to think > about for a future revision of the proposal. > >>> Note that packages do sometimes need to know that a certain >>> Python patch level release is installed (e.g. Zope often has >>> had this need) and a proper dependency checker would have to >>> detect a possible mismatch. >> I see. I would rather have this patch info in another variable, and restrict >> the micro language capabilities as much as possible. >> >> Any hint were Zope does this ? > > Early Zope versions simply warned about the problem in the > release documentation. IIRC, later versions actually tested > the installed Python version and warned about issues during > the installation process. > > I'm not sure how Zope 2.12 with all the tiny little packages > handles this case. The top-level Zope 2.12 distribution's doc/INSTALL.txt says: Supported [Python] versions include: * 2.5.x, (x >= 4) * 2.6.x I *think* that the requirement for 2.5.4 is bogus, or at least I can't figure out from SVN log etc. what specific bugfix would affect Zope. There is no longer a "monolithic" build script which tries to check this constraint, however: the documented way to build Zope is either: $ virtualenv --no-site-packages /path/to/zope $ /path/to/zope/bin/easy_install \ -i http://download.zope.org/Zope2/index/2.12 Zope2 or via zc.buildout: $ wget \ http://pypi.python.org/packages/source/Z/Zope2/Zope2-2.12.0.tar.gz $ tar xfvz Zope2-2.12.0.tar.gz $ cd Zope2-2.12.0 $ /path/to/your/python bootstrap/bootstrap.py $ bin/buildout Tres. - -- =================================================================== Tres Seaver +1 540-429-0999 tseaver at palladion.com Palladion Software "Excellence by Design" http://palladion.com -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iEUEARECAAYFAkrfLsQACgkQ+gerLs4ltQ4VPACdEiNlrPIgOFcM6Dzj6uZH7xH4 vHYAmMkPqqpzKagZVpIuHn2dV6j2G8Y= =4ajI -----END PGP SIGNATURE----- From david.lyon at preisshare.net Wed Oct 21 22:57:47 2009 From: david.lyon at preisshare.net (David Lyon) Date: Wed, 21 Oct 2009 16:57:47 -0400 Subject: [Distutils] Alternate static metadata PEP submission... In-Reply-To: <20091021122806.GA2966@piotro.eu> References: <20091013233654.GA27339@laurie.devork> <4ADDB2CB.1090101@simplistix.co.uk> <56bcb7f7893e86ec565d89b7ef40f418@preisshare.net> <4ADEB83E.8080804@simplistix.co.uk> <404efdf0624b873e17727d1f5966944d@preisshare.net> <4ADEC020.1050806@simplistix.co.uk> <20091021092412.GB32502@piotro.eu> <1de82ef8b2417b9a6ea6801c975c130c@preisshare.net> <20091021122806.GA2966@piotro.eu> Message-ID: <2d6aa2909e7eee05de23ca45b14cfba8@preisshare.net> On Wed, 21 Oct 2009 14:28:06 +0200, Piotr Ozarowski wrote: > Sure, one could just: > > | try: > | import setuptools > | except ImportError: > | import ez_setup > | ez_setup.use_setuptools() > > and thus use installed version if it's available (replace > ez_setup/setuptools with what you propose to ship in setup.py) but > that's not what module authors do unfortunately. I can see that I should change over to something like that. I had my head stuck in the implementation details. It's a small and sophisticated refinement that isn't hard to do. David From ziade.tarek at gmail.com Thu Oct 22 07:56:07 2009 From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=) Date: Thu, 22 Oct 2009 07:56:07 +0200 Subject: [Distutils] Preparing Distribute sprint #1 wrapup Message-ID: <94bdd2610910212256v6f131a90x7be64abeae8559d6@mail.gmail.com> Hey, Thanks to all sprinters, this was fun. Another sprint will be soon planned. I'd like to write a wrapup of the sprint we've done, so we can explain what's been done and what's next. If you were at that sprint and a "nominated" leader of one of the team, please add a summary of what has been done so far in your team, with links to the repositories when needed here : http://wiki.python.org/moin/Distutils/DistributeSprint (or a link to your blog if this has been done there) Thank you ! Tarek From ronaldoussoren at mac.com Thu Oct 22 16:51:31 2009 From: ronaldoussoren at mac.com (Ronald Oussoren) Date: Thu, 22 Oct 2009 16:51:31 +0200 Subject: [Distutils] tracking requested vs dependency installs in PEP 376 metadata In-Reply-To: <4ADDA900.4090505@simplistix.co.uk> References: <4ACE15C5.4030105@meyerloewen.net> <20091008191526.GD513@clingman.lan> <4ACE4078.70504@meyerloewen.net> <20091008195227.GA735@clingman.lan> <4ACF3E77.8080806@dirtcircle.com> <319e029f0910110727l7948ffccu6acee5617ea795bf@mail.gmail.com> <1E5BF8F0-B345-4EF5-9C57-79E0F7C369D8@mac.com> <319e029f0910111158y68fb25fajb3cbaf713062bf11@mail.gmail.com> <4ADDA900.4090505@simplistix.co.uk> Message-ID: <435FDA0D-8B91-492F-B0C2-FE55971BE077@mac.com> On 20 Oct, 2009, at 14:11, Chris Withers wrote: > Ronald Oussoren wrote: >> 2) Notice that simplejson gets installed and is a useful module, >> then use that in your own scripts > > At this point, you specify your scripts' dependencies on simplejson. > > Look at it another way: > > You follow you process above, but after step 2, your development > machine suffers a disk failure so you start from scratch on a new > machine. > > Are you surprised at this point that simplejson isn't installed when > you set things up again, even though by this point you've decided > not to use TurboGears? > > Code should always specify their dependencies, not doing so will > cause you problems down the line... Sure, and I do that for for any real code. But do you really expect that users are careful to specify dependencies for each and every script they write. That may be true for development shops, but I don't expect that casual users are so careful. But this discussion is moot anyway, the proposed metadata is meant to help packaging tools to track which packages seem to be orphaned and for which they can ask the user if those packages should be removed from the system, instead of always removing seemingly orphaned packages. Ronald -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 3567 bytes Desc: not available URL: From optilude at gmail.com Thu Oct 22 16:36:31 2009 From: optilude at gmail.com (Martin Aspeli) Date: Thu, 22 Oct 2009 14:36:31 +0000 (UTC) Subject: [Distutils] zc.buildout and site-packages Message-ID: Hi, Is there a way (apart from putting buildout in a virtualenv with --no-site- packages) to tell buildout *not* to put site-packages as the first line in the mangled sys.path when it generates scripts? We have people doing horrid things to their global python, and we need the buildout to be safe and isolated in these environments. Martin From jim at zope.com Thu Oct 22 17:01:47 2009 From: jim at zope.com (Jim Fulton) Date: Thu, 22 Oct 2009 11:01:47 -0400 Subject: [Distutils] zc.buildout and site-packages In-Reply-To: References: Message-ID: <1099b90b0910220801o64d1cae0m257ab4408a026e9b@mail.gmail.com> On Thu, Oct 22, 2009 at 10:36 AM, Martin Aspeli wrote: > Hi, > > Is there a way (apart from putting buildout in a virtualenv with --no-site- > packages) to tell buildout *not* to put site-packages as the first line in the > mangled sys.path when it generates scripts? Not now, but soon. Gary Poster has implemented this on a branch. I need to review and merge his changes. Jim -- Jim Fulton From kai.lautaportti at hexagonit.fi Thu Oct 22 18:36:31 2009 From: kai.lautaportti at hexagonit.fi (Kai Lautaportti) Date: Thu, 22 Oct 2009 19:36:31 +0300 Subject: [Distutils] [buildout] A few pending issues Message-ID: Hi, I've reported a few issues on the zc.buildout bug tracker, mainly https://bugs.launchpad.net/zc.buildout/+bug/435837 https://bugs.launchpad.net/zc.buildout/+bug/439387 with proposed patches that I would be interested in having included in the next release of zc.buildout. The #439387 issue is trivial, but I am not sure if the patch I proposed for #435837 is appropriate and would appreciate some additional reviewing and feedback. I would also like to get https://bugs.launchpad.net/zc.buildout/+bug/113085 fixed to make zc.buildout interact better with buildbot. Has anyone looked into this lately? In any case, I would be willing to spend time on these issues during the upcoming Plone conference sprint to make sure the fixes get merged into trunk. Is there anything special I can do to make this happen? Cheers, Kai -- Kai Lautaportti From jim at zope.com Thu Oct 22 19:36:28 2009 From: jim at zope.com (Jim Fulton) Date: Thu, 22 Oct 2009 13:36:28 -0400 Subject: [Distutils] [buildout] A few pending issues In-Reply-To: References: Message-ID: <1099b90b0910221036k66fc4317x8098564df3321c30@mail.gmail.com> On Thu, Oct 22, 2009 at 12:36 PM, Kai Lautaportti wrote: > Hi, > > I've reported a few issues on the zc.buildout bug tracker, mainly > > ?https://bugs.launchpad.net/zc.buildout/+bug/435837 > ?https://bugs.launchpad.net/zc.buildout/+bug/439387 > > with proposed patches that I would be interested in having included in > the next release of zc.buildout. > > The #439387 issue is trivial, but I am not sure if the patch I > proposed for #435837 is appropriate and would appreciate some > additional reviewing and feedback. > > I would also like to get > https://bugs.launchpad.net/zc.buildout/+bug/113085 fixed to make > zc.buildout interact better with buildbot. Has anyone looked into this > lately? > > In any case, I would be willing to spend time on these issues during > the upcoming Plone conference sprint to make sure the fixes get merged > into trunk. Is there anything special I can do to make this happen? You could get your changes checked in on branches for my review. I have a bunch of other changes I need to review and merge first though. Jim -- Jim Fulton From tseaver at palladion.com Thu Oct 22 19:43:08 2009 From: tseaver at palladion.com (Tres Seaver) Date: Thu, 22 Oct 2009 13:43:08 -0400 Subject: [Distutils] zc.buildout and site-packages In-Reply-To: References: Message-ID: <4AE099AC.6060604@palladion.com> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Martin Aspeli wrote: > Hi, > > Is there a way (apart from putting buildout in a virtualenv with --no-site- > packages) to tell buildout *not* to put site-packages as the first line in the > mangled sys.path when it generates scripts? > > We have people doing horrid things to their global python, and we need the > buildout to be safe and isolated in these environments. Using a --no-site-packages virtualenv to drive the buildout is a pretty lightweight solution, and easier than the old standby of compiling your own Python to get isolation from the global one, whichstill highly recommended: I build my own Python, and then use a separate virtualenv for each project. Tres. - -- =================================================================== Tres Seaver +1 540-429-0999 tseaver at palladion.com Palladion Software "Excellence by Design" http://palladion.com -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iEYEARECAAYFAkrgmawACgkQ+gerLs4ltQ5UWACg3RC0QZgyLZ4cBs7+SMhRar6N NvkAnimlAvY7nVfoWBnwxd9ZHIT+lgrO =V4IK -----END PGP SIGNATURE----- From kai.lautaportti at hexagonit.fi Thu Oct 22 19:59:25 2009 From: kai.lautaportti at hexagonit.fi (Kai Lautaportti) Date: Thu, 22 Oct 2009 20:59:25 +0300 Subject: [Distutils] [buildout] A few pending issues In-Reply-To: <1099b90b0910221036k66fc4317x8098564df3321c30@mail.gmail.com> References: <1099b90b0910221036k66fc4317x8098564df3321c30@mail.gmail.com> Message-ID: On Thu, Oct 22, 2009 at 8:36 PM, Jim Fulton wrote: > On Thu, Oct 22, 2009 at 12:36 PM, Kai Lautaportti > wrote: >> Hi, >> >> I've reported a few issues on the zc.buildout bug tracker, mainly >> >> ?https://bugs.launchpad.net/zc.buildout/+bug/435837 >> ?https://bugs.launchpad.net/zc.buildout/+bug/439387 >> >> with proposed patches that I would be interested in having included in >> the next release of zc.buildout. >> >> The #439387 issue is trivial, but I am not sure if the patch I >> proposed for #435837 is appropriate and would appreciate some >> additional reviewing and feedback. >> >> I would also like to get >> https://bugs.launchpad.net/zc.buildout/+bug/113085 fixed to make >> zc.buildout interact better with buildbot. Has anyone looked into this >> lately? >> >> In any case, I would be willing to spend time on these issues during >> the upcoming Plone conference sprint to make sure the fixes get merged >> into trunk. Is there anything special I can do to make this happen? > > You could get your changes checked in on branches for my review. ?I > have a bunch of other changes I need to review and merge first though. Thanks, I will try to get commit access to the Zope SVN sorted out before the conference so I can prepare the branches for your review and hopefully work on #113085 too at the sprint. Cheers, Kai -- Kai Lautaportti From kaelin at acm.org Fri Oct 23 02:47:14 2009 From: kaelin at acm.org (Kaelin Colclasure) Date: Thu, 22 Oct 2009 17:47:14 -0700 Subject: [Distutils] First attempt: close but no data files! Message-ID: Greetings, I am attempting my first Python contribution and have run into a speed bump getting a working setup.py using setuptools. Specifically, I cannot coax it to install my package data files into the site-packages directory. Here is a link to my project where my setup.py is available for browsing: Soooo close! TIA, -- Kaelin From ssteinerx at gmail.com Fri Oct 23 03:05:59 2009 From: ssteinerx at gmail.com (ssteinerX@gmail.com) Date: Thu, 22 Oct 2009 21:05:59 -0400 Subject: [Distutils] First attempt: close but no data files! In-Reply-To: References: Message-ID: On Oct 22, 2009, at 8:47 PM, Kaelin Colclasure wrote: > Greetings, > > I am attempting my first Python contribution and have run into a > speed bump getting a working setup.py using setuptools. > Specifically, I cannot coax it to install my package data files into > the site-packages directory. Here is a link to my project where my > setup.py is available for browsing: > > > > Soooo close! How about some sort of description of what you're trying to do... ? Reproduce steps: Clearly mention the steps to reproduce the bug. ? Expected result: How application should behave on above mentioned steps. ? Actual result: What is the actual result on running above steps i.e. the bug behavior. Thanks, S From pje at telecommunity.com Fri Oct 23 03:14:26 2009 From: pje at telecommunity.com (P.J. Eby) Date: Thu, 22 Oct 2009 21:14:26 -0400 Subject: [Distutils] First attempt: close but no data files! In-Reply-To: References: Message-ID: <20091023011438.3354C3A4100@sparrow.telecommunity.com> At 05:47 PM 10/22/2009 -0700, Kaelin Colclasure wrote: >Greetings, > >I am attempting my first Python contribution and have run into a speed >bump getting a working setup.py using setuptools. Specifically, I >cannot coax it to install my package data files into the site-packages >directory. Here is a link to my project where my setup.py is available >for browsing: > > There's no setup.py there. >Soooo close! > >TIA, > >-- Kaelin > >_______________________________________________ >Distutils-SIG maillist - Distutils-SIG at python.org >http://mail.python.org/mailman/listinfo/distutils-sig From kaelin at acm.org Fri Oct 23 03:25:32 2009 From: kaelin at acm.org (Kaelin Colclasure) Date: Thu, 22 Oct 2009 18:25:32 -0700 Subject: [Distutils] First attempt: close but no data files! In-Reply-To: References: Message-ID: <6B8C51CE-64B6-47F4-A289-F9367F7E8E52@acm.org> On Oct 22, 2009, at 6:05 PM, ssteinerX at gmail.com wrote: > > On Oct 22, 2009, at 8:47 PM, Kaelin Colclasure wrote: > >> Greetings, >> >> I am attempting my first Python contribution and have run into a >> speed bump getting a working setup.py using setuptools. >> Specifically, I cannot coax it to install my package data files >> into the site-packages directory. Here is a link to my project >> where my setup.py is available for browsing: >> >> >> >> Soooo close! > > How about some sort of description of what you're trying to do... > > ? Reproduce steps: Clearly mention the steps to reproduce the bug. > ? Expected result: How application should behave on above mentioned > steps. > ? Actual result: What is the actual result on running above steps > i.e. the bug behavior. > > Thanks, My project has a file (cuttlefish-config.plist) that needs to be installed next to the module (cuttlefish.py) in the site-packages directory. There is also a directory (static) containing some html files and png images that should go in the same place. I first tried using include_package_data=True, # Requires setuptools_hg after installing setuptools_hg to hook setuptools to understand Mercurial. This got all my data files added to the ...egg-info/ SOURCES.txt and I thought it was good. But then after building an egg and installing it with easy_setup, all the data files were missing. I then tried adding a MANIFEST.in file: ---8<--- (Bottle-python2.6)syn:cuttlefish kaelin$ cat MANIFEST.in include *.plist graft static --->8--- Same result: The files are listed in the "manifest" SOURCES.txt, but not installed. Here is SOURCES.txt: ---8<--- (Bottle-python2.6)syn:cuttlefish kaelin$ cat Cuttlefish.egg-info/ SOURCES.txt .hgignore MANIFEST.in README.txt cuttlefish-config.plist cuttlefish.py setup.py Cuttlefish.egg-info/PKG-INFO Cuttlefish.egg-info/SOURCES.txt Cuttlefish.egg-info/dependency_links.txt Cuttlefish.egg-info/entry_points.txt Cuttlefish.egg-info/not-zip-safe Cuttlefish.egg-info/requires.txt Cuttlefish.egg-info/top_level.txt static/LICENSE.html static/bottle-sig.png static/cuttlefish.png static/mako-sig.png --->8--- Here is my setup.py: ---8<--- #!/usr/bin/env python # -*- coding: utf-8 -*- """ Setup script for cuttlefish. """ from setuptools import setup, find_packages import cuttlefish setup( name="Cuttlefish", version=cuttlefish.__version__, py_modules=[ 'cuttlefish', ], include_package_data=True, # Requires setuptools_hg entry_points={ 'console_scripts': [ 'cuttlefish = cuttlefish:see_bottle_run', ], }, zip_safe=False, # Needs a user-editable cuttlefish-config.plist install_requires=[ 'Bottle >= 0.6.4', 'Mako >= 0.2.5', ], # PyPI metadata author='Kaelin Colclasure', author_email='kaelin at acm.org', description="Browser-based search tool for quickly `grep`ing source code.", long_description="Cuttlefish is an example project using the Bottle flyweight Web framework and the Mako template engine in Python. It's intended to run on your desktop, and provide a lightweight search engine (using grep) for your source code.", license='MIT', platforms=['darwin', 'unix'], url='http://bitbucket.org/kaelin/cuttlefish/', classifiers=[ 'Development Status :: 3 - Alpha', 'Environment :: Web Environment', #'Framework :: Bottle', 'Intended Audience :: Developers', 'License :: OSI Approved :: MIT License', 'Operating System :: MacOS :: MacOS X', 'Operating System :: Unix', 'Programming Language :: Python :: 2.6', 'Topic :: Software Development :: Documentation', 'Topic :: Text Processing :: Indexing', ], ) --->8--- The whole project is available from . TIA, -- Kaelin From kaelin at acm.org Fri Oct 23 03:26:50 2009 From: kaelin at acm.org (Kaelin Colclasure) Date: Thu, 22 Oct 2009 18:26:50 -0700 Subject: [Distutils] First attempt: close but no data files! In-Reply-To: <20091023011438.3354C3A4100@sparrow.telecommunity.com> References: <20091023011438.3354C3A4100@sparrow.telecommunity.com> Message-ID: <2FCBA2AD-BEE7-4634-83D6-2C7DF0295D0D@acm.org> On Oct 22, 2009, at 6:14 PM, P.J. Eby wrote: > At 05:47 PM 10/22/2009 -0700, Kaelin Colclasure wrote: >> Greetings, >> >> I am attempting my first Python contribution and have run into a >> speed >> bump getting a working setup.py using setuptools. Specifically, I >> cannot coax it to install my package data files into the site- >> packages >> directory. Here is a link to my project where my setup.py is >> available >> for browsing: >> >> > > There's no setup.py there. Oops, forgot to `hg push`... It's there now. :-) > > >> Soooo close! >> >> TIA, >> >> -- Kaelin >> >> _______________________________________________ >> Distutils-SIG maillist - Distutils-SIG at python.org >> http://mail.python.org/mailman/listinfo/distutils-sig > From pje at telecommunity.com Fri Oct 23 04:37:48 2009 From: pje at telecommunity.com (P.J. Eby) Date: Thu, 22 Oct 2009 22:37:48 -0400 Subject: [Distutils] First attempt: close but no data files! In-Reply-To: <2FCBA2AD-BEE7-4634-83D6-2C7DF0295D0D@acm.org> References: <20091023011438.3354C3A4100@sparrow.telecommunity.com> <2FCBA2AD-BEE7-4634-83D6-2C7DF0295D0D@acm.org> Message-ID: <20091023023801.3E32E3A4100@sparrow.telecommunity.com> At 06:26 PM 10/22/2009 -0700, Kaelin Colclasure wrote: >On Oct 22, 2009, at 6:14 PM, P.J. Eby wrote: > >>At 05:47 PM 10/22/2009 -0700, Kaelin Colclasure wrote: >>>Greetings, >>> >>>I am attempting my first Python contribution and have run into a >>>speed >>>bump getting a working setup.py using setuptools. Specifically, I >>>cannot coax it to install my package data files into the site- packages >>>directory. Here is a link to my project where my setup.py is >>>available >>>for browsing: >>> >>> >> >>There's no setup.py there. > > >Oops, forgot to `hg push`... It's there now. :-) Your "package data" isn't being included because you don't have any packages, only a module. Move your cuttlefish.py to cuttlefish/__init__.py, and put the .plist file in the same directory with it, and switch from py_modules to packages in your setup() arguments. That will make it get included with your installs and eggs. Finally, you may want to drop use of __file__ and use the pkg_resources API to access your data files instead, so that your library will work when it's installed as a zipfile: http://peak.telecommunity.com/DevCenter/PkgResources#basic-resource-access For example, rather than using __file__ to find the .plist file, you might use: contents = resource_string(__name__, 'cuttlefish-config.plist') to load the contents of the file as a string. (This step is optional; if you use __file__ in your code, setuptools will mark your project's eggs as requiring unzipping, and they will be installed unzipped. So, you don't *have to* do it, the __file__ use will still work.) From kevin at bud.ca Fri Oct 23 05:08:15 2009 From: kevin at bud.ca (Kevin Teague) Date: Thu, 22 Oct 2009 20:08:15 -0700 Subject: [Distutils] zc.buildout and site-packages In-Reply-To: <4AE099AC.6060604@palladion.com> References: <4AE099AC.6060604@palladion.com> Message-ID: <99C111D5-BFE2-4043-956E-47BCAEB8FAC4@bud.ca> On Oct 22, 2009, at 10:43 AM, Tres Seaver wrote: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > Martin Aspeli wrote: >> Hi, >> >> Is there a way (apart from putting buildout in a virtualenv with -- >> no-site- >> packages) to tell buildout *not* to put site-packages as the first >> line in the >> mangled sys.path when it generates scripts? >> >> We have people doing horrid things to their global python, and we >> need the >> buildout to be safe and isolated in these environments. > > Using a --no-site-packages virtualenv to drive the buildout is a > pretty > lightweight solution, and easier than the old standby of compiling > your > own Python to get isolation from the global one, whichstill highly > recommended: I build my own Python, and then use a separate > virtualenv > for each project. > The idea behind Gary's branch (http://svn.zope.org/zc.buildout/branches/gary-4-include-site-packages ) is that unlike the --no-site-packages option of virtualenv, which is all-or-nothing proposition, you would be able to include site-package locations in Buildout's script generation, but care would be taken that if distributions are selected from a site-package location to make sure that when site-package locations are included on sys.path, those locations don't overshadow any other paths pointing explicitly to already picked versions of distributions. e.g. If I was using Apple's System Python on Leopard (10.5), then site-packages includes zope.interface 3.3.0 and bdist_mpkg 0.4.3. If I wanted to pick 'zope.interface == 3.3.0' and 'bdist_mpkg == 0.4.4', then currently Buildout could generate a path modification that looks like: sys.path[0:0] = [ '/System/Library/Frameworks/Python.framework/Versions/2.5/Extras/ lib/python', '/Users/kteague/buildouts/shared/eggs/bdist_mpkg-0.4.4-py2.5.egg', ] Where that System path contains bdist_mpkg 0.4.3. The ordering of whether the site-package location is put before or after version- specific paths is currently dependant upon the ordering in the install_requires field (so you get the correct versions importable if those distributions which are picked from site-packages are listed after the non-site-package picked versions!) - obviously this is just a side-effect of the current path manipulation implementation. One would assume that making this change is fairly easy. Just do a diff between normal sys.path and the site-package free sys.path when Python is launched with the -S flag. Which Gary's code does, but the script generation in Gary's branch right now also accounts for the fact that *.pth files have been processed, and that you are allowed to have import statements executed when *.pth files are processed, so he is generating scripts which also clean sys.modules, and then re-add site-packages locations with site.addsitedir(location) so that .pth files are properly re-processed. Which is pretty fancy, and probably "Does the Right Thing (TM)", but also greatly clutters up the generated scripts. I quite like having script generation generate scripts which are still reasonably compact (I often open generated scripts to see what Buildout is doing, or sometimes edit them to hand-pick a different egg if I want to quickly try out a different working set) and I also wonder how much overhead this additional processing adds (I guess this depends upon how much you have in site-packages). So perhaps if there was some option to still generate scripts using the existing style of script generation - maybe a "i-keep-my-site-packages-clean=true" option ... i dunno, perhaps the other way 'work-around-site-package- madness-in-script-generation=true' ... or just merge Buildout and VirtualEnv into one monolithic project so that you don't need to install two tools just to be able to use Buildout with a dirty Python! (rawr!) Anyways, for those distributions which are tough to install, I think some people will find this branch quite handy in that they can apt-get the tough to install distributions, and then safely include those distributions in working sets composed by Buildout. From kaelin at acm.org Fri Oct 23 06:58:24 2009 From: kaelin at acm.org (Kaelin Colclasure) Date: Thu, 22 Oct 2009 21:58:24 -0700 Subject: [Distutils] First attempt: close but no data files! In-Reply-To: <20091023023801.3E32E3A4100@sparrow.telecommunity.com> References: <20091023011438.3354C3A4100@sparrow.telecommunity.com> <2FCBA2AD-BEE7-4634-83D6-2C7DF0295D0D@acm.org> <20091023023801.3E32E3A4100@sparrow.telecommunity.com> Message-ID: <9BF8424E-5329-422C-A2D3-E4E9DAE5BC3B@acm.org> On Oct 22, 2009, at 7:37 PM, P.J. Eby wrote: > At 06:26 PM 10/22/2009 -0700, Kaelin Colclasure wrote: > >> On Oct 22, 2009, at 6:14 PM, P.J. Eby wrote: >> >>> At 05:47 PM 10/22/2009 -0700, Kaelin Colclasure wrote: >>>> Greetings, >>>> >>>> I am attempting my first Python contribution and have run into a >>>> speed >>>> bump getting a working setup.py using setuptools. Specifically, I >>>> cannot coax it to install my package data files into the site- >>>> packages >>>> directory. Here is a link to my project where my setup.py is >>>> available >>>> for browsing: >>>> >>>> >>> >>> There's no setup.py there. >> >> >> Oops, forgot to `hg push`... It's there now. :-) > > Your "package data" isn't being included because you don't have any > packages, only a module. > > Move your cuttlefish.py to cuttlefish/__init__.py, and put > the .plist file in the same directory with it, and switch from > py_modules to packages in your setup() arguments. That will make it > get included with your installs and eggs. > > Finally, you may want to drop use of __file__ and use the > pkg_resources API to access your data files instead, so that your > library will work when it's installed as a zipfile: > > http://peak.telecommunity.com/DevCenter/PkgResources#basic-resource-access > > For example, rather than using __file__ to find the .plist file, you > might use: > > contents = resource_string(__name__, 'cuttlefish-config.plist') > > to load the contents of the file as a string. > > (This step is optional; if you use __file__ in your code, setuptools > will mark your project's eggs as requiring unzipping, and they will > be installed unzipped. So, you don't *have to* do it, the __file__ > use will still work.) > Ah, so? Thanks for the guidance! Restructuring as a package did indeed get things working as expected. It's somewhat unfortunate that this is a requirement, as it made for a lot of noise in my Mercurial repository and now most of my code is in a module with the unhelpful name __init__.py? But this too shall pass. :-) Thanks again! -- Kaelin From ziade.tarek at gmail.com Fri Oct 23 11:46:50 2009 From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=) Date: Fri, 23 Oct 2009 11:46:50 +0200 Subject: [Distutils] Distribute support added in zc.buildout Message-ID: <94bdd2610910230246y67667b1bo218901cfbbeee5b0@mail.gmail.com> Hey, I have added in zc.buildout trunk the Distribute support. The unified bootstrap now has a --distribute boolean option that allows you to switch if you want. its located here as usual : http://svn.zope.org/*checkout*/zc.buildout/trunk/bootstrap/bootstrap.py Please try it ! I will wait a few days to make sure everything works as expected then push a zc.buildout 1.4.2 release that includes this change Cheers Tarek From hanno at hannosch.eu Fri Oct 23 12:09:07 2009 From: hanno at hannosch.eu (Hanno Schlichting) Date: Fri, 23 Oct 2009 12:09:07 +0200 Subject: [Distutils] Distribute support added in zc.buildout In-Reply-To: <94bdd2610910230246y67667b1bo218901cfbbeee5b0@mail.gmail.com> References: <94bdd2610910230246y67667b1bo218901cfbbeee5b0@mail.gmail.com> Message-ID: <5cae42b20910230309q75857a56l29aa02996098acc1@mail.gmail.com> Hi Tarek, On Fri, Oct 23, 2009 at 11:46 AM, Tarek Ziad? wrote: > I have added in zc.buildout trunk the Distribute support. Cool! > The unified bootstrap now has a --distribute boolean option that > allows you to switch if you want. What is the easiest / recommended way to tell the bootstrap to use distribute by default? I could live with a clear DEFAULT='setuptools' line at the top of the module, so even if one changes that line, the diff to the original isn't too large. Otherwise some config file would be the next but more complicated option. Thanks, Hanno From ziade.tarek at gmail.com Fri Oct 23 12:33:49 2009 From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=) Date: Fri, 23 Oct 2009 12:33:49 +0200 Subject: [Distutils] Distribute support added in zc.buildout In-Reply-To: <5cae42b20910230309q75857a56l29aa02996098acc1@mail.gmail.com> References: <94bdd2610910230246y67667b1bo218901cfbbeee5b0@mail.gmail.com> <5cae42b20910230309q75857a56l29aa02996098acc1@mail.gmail.com> Message-ID: <94bdd2610910230333g6225e125g524a9e7b8c536b09@mail.gmail.com> On Fri, Oct 23, 2009 at 12:09 PM, Hanno Schlichting wrote: > Hi Tarek, > > On Fri, Oct 23, 2009 at 11:46 AM, Tarek Ziad? wrote: >> I have added in zc.buildout trunk the Distribute support. > > Cool! > >> The unified bootstrap now has a --distribute boolean option that >> allows you to switch if you want. > > What is the easiest / recommended way to tell the bootstrap to use > distribute by default? > > I could live with a clear DEFAULT='setuptools' line at the top of the > module, so even if one changes that line, the diff to the original > isn't too large. Otherwise some config file would be the next but more > complicated option. > What about a "use-distribute" option in the [buildout] section in the cfg file called with bootstrap ? I need to investigate but I think it could be feasible to read the .cfg file early in the bootstrap process to read that option. Altough, I'd like to have Jim's opinion about that Tarek From pje at telecommunity.com Fri Oct 23 17:16:07 2009 From: pje at telecommunity.com (P.J. Eby) Date: Fri, 23 Oct 2009 11:16:07 -0400 Subject: [Distutils] First attempt: close but no data files! In-Reply-To: <9BF8424E-5329-422C-A2D3-E4E9DAE5BC3B@acm.org> References: <20091023011438.3354C3A4100@sparrow.telecommunity.com> <2FCBA2AD-BEE7-4634-83D6-2C7DF0295D0D@acm.org> <20091023023801.3E32E3A4100@sparrow.telecommunity.com> <9BF8424E-5329-422C-A2D3-E4E9DAE5BC3B@acm.org> Message-ID: <20091023151621.3DD073A4116@sparrow.telecommunity.com> At 09:58 PM 10/22/2009 -0700, Kaelin Colclasure wrote: >Restructuring as a package did indeed get things working as expected. >It's somewhat unfortunate that this is a requirement, as it made for a >lot of noise in my Mercurial repository and now most of my code is in >a module with the unhelpful name __init__.py Given how short your data file is, the fact that it's entirely text, and the fact that your module script always needs it to be loaded, I wonder why you don't just make it a string constant in the .py file to start with, or better yet, simply directly create the data structure it represents. From kaelin at acm.org Fri Oct 23 17:53:10 2009 From: kaelin at acm.org (Kaelin Colclasure) Date: Fri, 23 Oct 2009 08:53:10 -0700 Subject: [Distutils] First attempt: close but no data files! In-Reply-To: <20091023151621.3DD073A4116@sparrow.telecommunity.com> References: <20091023011438.3354C3A4100@sparrow.telecommunity.com> <2FCBA2AD-BEE7-4634-83D6-2C7DF0295D0D@acm.org> <20091023023801.3E32E3A4100@sparrow.telecommunity.com> <9BF8424E-5329-422C-A2D3-E4E9DAE5BC3B@acm.org> <20091023151621.3DD073A4116@sparrow.telecommunity.com> Message-ID: On Oct 23, 2009, at 8:16 AM, P.J. Eby wrote: > At 09:58 PM 10/22/2009 -0700, Kaelin Colclasure wrote: >> Restructuring as a package did indeed get things working as expected. >> It's somewhat unfortunate that this is a requirement, as it made >> for a >> lot of noise in my Mercurial repository and now most of my code is in >> a module with the unhelpful name __init__.py? > > Given how short your data file is, the fact that it's entirely text, > and the fact that your module script always needs it to be loaded, I > wonder why you don't just make it a string constant in the .py file > to start with, or better yet, simply directly create the data > structure it represents. I think you're referring to cuttlefish-config.plist, which is intended to be edited after installation to refer to the particulars of the installed site's filesystem, etc. [And yes, I realize having such a config file live inside the package is sub-optimal. I'm just looking for a low-impedence solution that works with the PyPI infrastructure. That said, if there are "best practices" for such things established and supported by setuptools I'm all ears?] There is also a 'static/' directory with a bunch of webapp resources in it. And it is now installing with the package perfectly too. Thanks again for the helpful guidance! -- Kaelin From gary.poster at canonical.com Fri Oct 23 18:01:53 2009 From: gary.poster at canonical.com (Gary Poster) Date: Fri, 23 Oct 2009 12:01:53 -0400 Subject: [Distutils] zc.buildout and site-packages In-Reply-To: <99C111D5-BFE2-4043-956E-47BCAEB8FAC4@bud.ca> References: <4AE099AC.6060604@palladion.com> <99C111D5-BFE2-4043-956E-47BCAEB8FAC4@bud.ca> Message-ID: <79909212-86E0-4A39-A008-3AE9B9F85A9D@canonical.com> On Oct 22, 2009, at 11:08 PM, Kevin Teague wrote: > > On Oct 22, 2009, at 10:43 AM, Tres Seaver wrote: > >> -----BEGIN PGP SIGNED MESSAGE----- >> Hash: SHA1 >> >> Martin Aspeli wrote: >>> Hi, >>> >>> Is there a way (apart from putting buildout in a virtualenv with -- >>> no-site- >>> packages) to tell buildout *not* to put site-packages as the first >>> line in the >>> mangled sys.path when it generates scripts? >>> >>> We have people doing horrid things to their global python, and we >>> need the >>> buildout to be safe and isolated in these environments. >> >> Using a --no-site-packages virtualenv to drive the buildout is a >> pretty >> lightweight solution, and easier than the old standby of compiling >> your >> own Python to get isolation from the global one, whichstill highly >> recommended: I build my own Python, and then use a separate >> virtualenv >> for each project. >> > > The idea behind Gary's branch To be clear, *an* idea. You can also just make a "don't give me what is in site-packages" gesture. (When you do that, in the current branch, the generated scripts still have the complexities you describe below, though.) > (http://svn.zope.org/zc.buildout/branches/gary-4-include-site- > packages) is that unlike the --no-site-packages option of > virtualenv, which is all-or-nothing proposition, you would be able > to include site-package locations in Buildout's script generation, > but care would be taken that if distributions are selected from a > site-package location to make sure that when site-package locations > are included on sys.path, those locations don't overshadow any other > paths pointing explicitly to already picked versions of > distributions. e.g. If I was using Apple's System Python on Leopard > (10.5), then site-packages includes zope.interface 3.3.0 and > bdist_mpkg 0.4.3. If I wanted to pick 'zope.interface == 3.3.0' and > 'bdist_mpkg == 0.4.4', then currently Buildout could generate a path > modification that looks like: > > sys.path[0:0] = [ > '/System/Library/Frameworks/Python.framework/Versions/2.5/Extras/ > lib/python', > '/Users/kteague/buildouts/shared/eggs/bdist_mpkg-0.4.4-py2.5.egg', > ] > > Where that System path contains bdist_mpkg 0.4.3. The ordering of > whether the site-package location is put before or after version- > specific paths is currently dependant upon the ordering in the > install_requires field (so you get the correct versions importable > if those distributions which are picked from site-packages are > listed after the non-site-package picked versions!) - obviously this > is just a side-effect of the current path manipulation implementation. Not exactly. I was going to go for that, but it was too hard/insane. (Do I need to update some docs on the branch?) If you use this feature, then eggs from site-packages can be inserted cleanly along with other eggs. They can be chosen individually, without masking other eggs. Site-packages-like directories themselves--the directories that are not eggs, but collections of standard directory packages--always go at the end of the sys.path. Otherwise their contents might mask the eggs you chose. What we actually ended up using ourselves (Launchpad) is "don't use any eggs from site-packages, but let site-packages through at the end so we can get some of the non-egg things from it that our system is providing, like Postgres-Python bindings." > One would assume that making this change is fairly easy. Just do a > diff between normal sys.path and the site-package free sys.path when > Python is launched with the -S flag. Which Gary's code does, but the > script generation in Gary's branch right now also accounts for the > fact that *.pth files have been processed, and that you are allowed > to have import statements executed when *.pth files are processed, > so he is generating scripts which also clean sys.modules, and then > re-add site-packages locations with site.addsitedir(location) so > that .pth files are properly re-processed. Which is pretty fancy, > and probably "Does the Right Thing (TM)", but also greatly clutters > up the generated scripts. Mostly right, and granted that the scripts are bigger and more annoying than they are in trunk. FWIW, the "fancy" bits are not primarily because .pth files might import. It's more because the setuptools approach to creating namespace packages in site-packages--that is, the approach that OS distributions typically use--creates fake modules for the namespace packages. These mask any sys.path eggs in the same namespace packages, at least as of c9. We have to clean the fake modules out, set up the sys.path, import pkg_resources because that magically does the right thing for any eggs on the sys.path, and *then* process .pth files. (I hope that PEP 382 is accepted and helps.) > I quite like having script generation generate scripts which are > still reasonably compact (I often open generated scripts to see what > Buildout is doing, or sometimes edit them to hand-pick a different > egg if I want to quickly try out a different working set) Granted. > and I also wonder how much overhead this additional processing adds > (I guess this depends upon how much you have in site-packages). Any overhead is lost in the cost of importing pkg_resources. Launchpad has a whole bunch of dependencies (~170 eggs last I checked). It's trivial to generate both a ``PYTHONPATH= [...dependencies...] python`` and a faux Python interpreter generated by buildout that does the tricks that you describe. To make the PYTHONPATH approach work with the namespace package problem I described above, you have to hack site.py to import pkg_resources before it processes the .pth files. I compared the two approaches with Launchpad's ~170 dependencies using ``time ${INTERPRETER_CHOICE} -c ''``. They were equivalent in my tests. (FWIW, they were both about 20 times slower than ``time python -c ''``). > So perhaps if there was some option to still generate scripts using > the existing style of script generation - maybe a "i-keep-my-site- > packages-clean=true" option ... i dunno, perhaps the other way 'work- > around-site-package-madness-in-script-generation=true' ... or just > merge Buildout and VirtualEnv into one monolithic project so that > you don't need to install two tools just to be able to use Buildout > with a dirty Python! (rawr!) I can understand the desire to make it possible to have a simpler script if you want to promise that you are going to have a clean site- packages. I'm not super-excited to add this feature and the related tests, but if that made it possible for my work to not be consigned to a branch forever, I suppose I'd sign up. Jim will be the arbiter there. And, as usual and of course, there are other approaches possible than the one I chose. > Anyways, for those distributions which are tough to install, I think > some people will find this branch quite handy in that they can apt- > get the tough to install distributions, and then safely include > those distributions in working sets composed by Buildout. Seems to be working for us. Thanks for looking at the branch, and for writing about it. Gary From pje at telecommunity.com Fri Oct 23 18:08:59 2009 From: pje at telecommunity.com (P.J. Eby) Date: Fri, 23 Oct 2009 12:08:59 -0400 Subject: [Distutils] First attempt: close but no data files! In-Reply-To: References: <20091023011438.3354C3A4100@sparrow.telecommunity.com> <2FCBA2AD-BEE7-4634-83D6-2C7DF0295D0D@acm.org> <20091023023801.3E32E3A4100@sparrow.telecommunity.com> <9BF8424E-5329-422C-A2D3-E4E9DAE5BC3B@acm.org> <20091023151621.3DD073A4116@sparrow.telecommunity.com> Message-ID: <20091023160913.5E31B3A4116@sparrow.telecommunity.com> At 08:53 AM 10/23/2009 -0700, Kaelin Colclasure wrote: >On Oct 23, 2009, at 8:16 AM, P.J. Eby wrote: > >>At 09:58 PM 10/22/2009 -0700, Kaelin Colclasure wrote: >>>Restructuring as a package did indeed get things working as expected. >>>It's somewhat unfortunate that this is a requirement, as it made >>>for a >>>lot of noise in my Mercurial repository and now most of my code is in >>>a module with the unhelpful name __init__.py >> >>Given how short your data file is, the fact that it's entirely text, >>and the fact that your module script always needs it to be loaded, I >>wonder why you don't just make it a string constant in the .py file >>to start with, or better yet, simply directly create the data >>structure it represents. > > >I think you're referring to cuttlefish-config.plist, which is intended >to be edited after installation to refer to the particulars of the >installed site's filesystem, etc. Don't do that. Package data is for static, read-only data used by a library or application at runtime. > [And yes, I realize having such a >config file live inside the package is sub-optimal. I'm just looking >for a low-impedence solution that works with the PyPI infrastructure. >That said, if there are "best practices" for such things established >and supported by setuptools I'm all ears ] Put it with documentation, or use the standard distutils data_files option, rather than package data. User editable files should *not* be installed adjacent to the code; it's rightly frowned on by Linux distributions and system administrators everywhere. If the issue is that it's a "global" configuration file and you don't know where else to put it, base the location on the user's home directory (on *nix ) or an APP_DATA subdirectory (on Windows). If it's not a single per-user location, then use an environment variable or command-line option (for command-line tools) or as an argument (for Python APIs). Some tools also ship with a lot of package data as *templates* for user-configurable stuff, and include a short script to copy the template to a user-specified location, possibly filling in a few things specified on the command line, or via interactive prompts. This is a good way to provide a nice UI for your installation/setup, especially in the case where someone might need multiple copies of the configured data (e.g. a webapp you can install multiple instances of, like Trac). >There is also a 'static/' directory with a bunch of webapp resources >in it. If those are user-editable, it's probably a good idea to include a script that copies and/or customizes them and places them in a user-defined location, rather than having the code read them directly from the package (and thus requiring the user to edit them directly). Of course, if those webapp resources are *not* user-editable, then leaving them in your package data is fine. > And it is now installing with the package perfectly too. Thanks >again for the helpful guidance! You can thank me (and make a lot of sysadmins happier, or at least a little less grumpy) by not locating any user-editable files inside your package directory. ;-) From encolpe.degoute at free.fr Fri Oct 23 21:03:12 2009 From: encolpe.degoute at free.fr (Encolpe Degoute) Date: Fri, 23 Oct 2009 21:03:12 +0200 Subject: [Distutils] Distribute support added in zc.buildout In-Reply-To: <94bdd2610910230333g6225e125g524a9e7b8c536b09@mail.gmail.com> References: <94bdd2610910230246y67667b1bo218901cfbbeee5b0@mail.gmail.com> <5cae42b20910230309q75857a56l29aa02996098acc1@mail.gmail.com> <94bdd2610910230333g6225e125g524a9e7b8c536b09@mail.gmail.com> Message-ID: Tarek Ziad? a ?crit : > On Fri, Oct 23, 2009 at 12:09 PM, Hanno Schlichting wrote: >> Hi Tarek, >> >> On Fri, Oct 23, 2009 at 11:46 AM, Tarek Ziad? wrote: >>> I have added in zc.buildout trunk the Distribute support. >> Cool! >> >>> The unified bootstrap now has a --distribute boolean option that >>> allows you to switch if you want. >> What is the easiest / recommended way to tell the bootstrap to use >> distribute by default? >> >> I could live with a clear DEFAULT='setuptools' line at the top of the >> module, so even if one changes that line, the diff to the original >> isn't too large. Otherwise some config file would be the next but more >> complicated option. >> > > What about a "use-distribute" option in the [buildout] section in the > cfg file called with bootstrap ? Today we have setuptools and distribute. What would we have tomorrow? distribution-tool = nothing|setuptools|distribute|pipeau You can replace 'tool' by 'api', 'software' or want you want. Regards, -- Encolpe DEGOUTE http://encolpe.degoute.free.fr/ Logiciels libres, hockey sur glace et autres activit?s c?r?brales From kiorky at cryptelium.net Fri Oct 23 22:10:31 2009 From: kiorky at cryptelium.net (kiorky) Date: Fri, 23 Oct 2009 22:10:31 +0200 Subject: [Distutils] Distribute support added in zc.buildout In-Reply-To: <94bdd2610910230333g6225e125g524a9e7b8c536b09@mail.gmail.com> References: <94bdd2610910230246y67667b1bo218901cfbbeee5b0@mail.gmail.com> <5cae42b20910230309q75857a56l29aa02996098acc1@mail.gmail.com> <94bdd2610910230333g6225e125g524a9e7b8c536b09@mail.gmail.com> Message-ID: <4AE20DB7.1030200@cryptelium.net> Would be great, specially if we can read the ~/.buildout/default.cfg to set up buildout to use distribute once for all. Tarek Ziad? a ?crit : > On Fri, Oct 23, 2009 at 12:09 PM, Hanno Schlichting wrote: >> Hi Tarek, >> >> On Fri, Oct 23, 2009 at 11:46 AM, Tarek Ziad? wrote: >>> I have added in zc.buildout trunk the Distribute support. >> Cool! >> >>> The unified bootstrap now has a --distribute boolean option that >>> allows you to switch if you want. >> What is the easiest / recommended way to tell the bootstrap to use >> distribute by default? >> >> I could live with a clear DEFAULT='setuptools' line at the top of the >> module, so even if one changes that line, the diff to the original >> isn't too large. Otherwise some config file would be the next but more >> complicated option. >> > > What about a "use-distribute" option in the [buildout] section in the > cfg file called with bootstrap ? > > I need to investigate but I think it could be feasible to read the > .cfg file early in the bootstrap process > to read that option. > > Altough, I'd like to have Jim's opinion about that > > Tarek > _______________________________________________ > Distutils-SIG maillist - Distutils-SIG at python.org > http://mail.python.org/mailman/listinfo/distutils-sig -- Cordialement, KiOrKY GPG Key FingerPrint: 0x1A1194B7681112AF -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 261 bytes Desc: OpenPGP digital signature URL: From ziade.tarek at gmail.com Sat Oct 24 00:46:50 2009 From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=) Date: Sat, 24 Oct 2009 00:46:50 +0200 Subject: [Distutils] Distribute sprint #1 wrapup Message-ID: <94bdd2610910231546m2f185e0eqcb3fcd0850684101@mail.gmail.com> As promised, here's a quick wrapup of the work started during the first sprint, on Distribute 0.7 Thanks to all the people that have participated (we were 6 or 7 IIRC) If you have sprinted and see a mistake or something missing is the wrapup, please complete it; = The fate of 0.6 code = First of all, we've discussed what is happening to the 0.6 code and we've decided that it was better to keep it under the pillow as a reference code, and code everything from scratch. The only code that is going to be recycled is some parts of pkg_resources. = 0.7 philosophy = Distribute 0.7 will be a good Distutils citizen and wants to be its incubator. It will implement Distutils commands, without changing Distutils behavior as it is doing now. Another important point is that the Distribute development will be driven by the PEP work that is going on and drop most other concepts of Setuptools: - PEP 376 : a *single* installation standard - PEP 345 : Requires, Obsoletes, etc, fields - PEP 390 : platform-dependant markers for the metadata - PEP 382 : namespace packages - PEP 386 : a new version scheme Distribute will implement those PEPs by gathering all the prototypes that have been created in the past months and when possible, will try to push them in Distutils. The biggest drops we are making are: - the egg formats are removed in favor of a unique PEP 376 standard. - easy_install is removed, and Distribute will cooperate with the pip project What's unclear right now is what will be pushed in Distutils by the time Python 2.7 and 3.3 are out. = what was started during the sprint = We started to write Distutils commands: - develop command: * Created a skeleton command for develop * Waiting for work to complete on the install and build_egg_info commands before work can continue here. - build_egg_info command (in setuptools, it was called "egg_info", we've renamed it): * Rewrote this command using setuptools code as a model * Removed all SVN support code * Removed obsolete egg-info writers * Waiting on the manifest generation feature to finish this one * Will implement PEP 376 egg-info writers when possible - generate_manifest command. Not started yet, but will be used to generate the MANIFEST file, and provide a plugin system for VCS support. (not an implicit system like in Setuptools) - install command - building a PEP 376 compatible install command (by rewriting pkg_resources, using the PEP 376 prototype we have written some time ago) - distutils.extensions : lightweight entry points, based on the PEP 376 standard taken back from the "extensions" project - Currently working on testing strategy = next sprint = The next sprint will be next week-end, and will be announced here; Anyone can join ! Thanks Tarek -- Tarek Ziad? | http://ziade.org |????????????! |???????????? From eric at trueblade.com Sat Oct 24 00:50:53 2009 From: eric at trueblade.com (Eric Smith) Date: Fri, 23 Oct 2009 18:50:53 -0400 Subject: [Distutils] Distribute sprint #1 wrapup In-Reply-To: <94bdd2610910231546m2f185e0eqcb3fcd0850684101@mail.gmail.com> References: <94bdd2610910231546m2f185e0eqcb3fcd0850684101@mail.gmail.com> Message-ID: <4AE2334D.3000304@trueblade.com> Tarek Ziad? wrote: > What's unclear right now is what will be pushed in Distutils by the > time Python 2.7 and 3.3 are out. Why 3.3 and not 3.2? Eric. From ziade.tarek at gmail.com Sat Oct 24 00:53:10 2009 From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=) Date: Sat, 24 Oct 2009 00:53:10 +0200 Subject: [Distutils] Distribute sprint #1 wrapup In-Reply-To: <4AE2334D.3000304@trueblade.com> References: <94bdd2610910231546m2f185e0eqcb3fcd0850684101@mail.gmail.com> <4AE2334D.3000304@trueblade.com> Message-ID: <94bdd2610910231553j28b361em6ccd26c550a3dc36@mail.gmail.com> On Sat, Oct 24, 2009 at 12:50 AM, Eric Smith wrote: > Tarek Ziad? wrote: > > > >> What's unclear right now is what will be pushed in Distutils by the >> time Python 2.7 and 3.3 are out. > > Why 3.3 and not 3.2? Oops: that was 3.2 of course ! All releases that we should have next summer, Tarek From kiorky at cryptelium.net Sat Oct 24 07:43:26 2009 From: kiorky at cryptelium.net (kiorky) Date: Sat, 24 Oct 2009 07:43:26 +0200 Subject: [Distutils] Distribute support added in zc.buildout In-Reply-To: References: <94bdd2610910230246y67667b1bo218901cfbbeee5b0@mail.gmail.com> <5cae42b20910230309q75857a56l29aa02996098acc1@mail.gmail.com> <94bdd2610910230333g6225e125g524a9e7b8c536b09@mail.gmail.com> Message-ID: <4AE293FE.7000601@cryptelium.net> Encolpe Degoute a ?crit : > Tarek Ziad? a ?crit : > > distribution-tool = nothing|setuptools|distribute|pipeau Although i can maybe make my way with distribute even if it can break backward compatibility somehow, I strongly dislike pip too. Moreover, if it's a forced choice. And even if it is for "future years", in a "very long time", all that it says it that we will have breakages. How that sounds like regressions for me. How about extra_requirements? how about buildout? More over, when you see that the only recipe useful has some strange things like: env.update({ ... 'LDFLAGS': '-I%s' % include_dir, }) That sounds like pretty unused stuff. > > You can replace 'tool' by 'api', 'software' or want you want. > > Regards, -- Cordialement, KiOrKY GPG Key FingerPrint: 0x1A1194B7681112AF -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 261 bytes Desc: OpenPGP digital signature URL: From kiorky at cryptelium.net Sat Oct 24 07:57:01 2009 From: kiorky at cryptelium.net (kiorky) Date: Sat, 24 Oct 2009 07:57:01 +0200 Subject: [Distutils] Distribute sprint #1 wrapup In-Reply-To: <94bdd2610910231553j28b361em6ccd26c550a3dc36@mail.gmail.com> References: <94bdd2610910231546m2f185e0eqcb3fcd0850684101@mail.gmail.com> <4AE2334D.3000304@trueblade.com> <94bdd2610910231553j28b361em6ccd26c550a3dc36@mail.gmail.com> Message-ID: <4AE2972D.8080703@cryptelium.net> Note about pep376 :: .egg-info becomes a directory As explained earlier, the EggFormats standard from setuptools proposes two formats to install the metadata information of a distribution: * A self-contained directory that can be zipped or left unzipped and contains the distribution files and an .egg-info directory containing the metadata. * A distinct .egg-info directory located in the site-packages directory, with the metadata inside. This PEP proposes to keep just one format and make it the standard way to install the metadata of a distribution : a distinct .egg-info directory located in the site-packages directory, containing the metadata. There is something i don't understand there: Does it imply that having no access on site-packages prevent you from installing third party libraries as a limited user? For example, on shared hosting where you have limited privileges. Before, i could upload/build some libs and start from there using, for exemple, buildout to scan that directory. I make assumption that also the entry points are written along the metadata dir. That will say that even if i don't have access to the site-packages but i have played with sys.path to include my distribution, i won't have its entry points (as the egg-info is not there) ? Tarek Ziad? a ?crit : > On Sat, Oct 24, 2009 at 12:50 AM, Eric Smith wrote: >> Tarek Ziad? wrote: >> >> >> >>> What's unclear right now is what will be pushed in Distutils by the >>> time Python 2.7 and 3.3 are out. >> Why 3.3 and not 3.2? > > Oops: that was 3.2 of course ! All releases that we should have next summer, > > Tarek > _______________________________________________ > Distutils-SIG maillist - Distutils-SIG at python.org > http://mail.python.org/mailman/listinfo/distutils-sig -- Cordialement, KiOrKY GPG Key FingerPrint: 0x1A1194B7681112AF -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 261 bytes Desc: OpenPGP digital signature URL: From ziade.tarek at gmail.com Sat Oct 24 09:39:54 2009 From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=) Date: Sat, 24 Oct 2009 09:39:54 +0200 Subject: [Distutils] Distribute sprint #1 wrapup In-Reply-To: <4AE2972D.8080703@cryptelium.net> References: <94bdd2610910231546m2f185e0eqcb3fcd0850684101@mail.gmail.com> <4AE2334D.3000304@trueblade.com> <94bdd2610910231553j28b361em6ccd26c550a3dc36@mail.gmail.com> <4AE2972D.8080703@cryptelium.net> Message-ID: <94bdd2610910240039s2f4f4626m2f1fc131af4a7178@mail.gmail.com> On Sat, Oct 24, 2009 at 7:57 AM, kiorky wrote: > > This PEP proposes to keep just one format and make it the standard way to > install the metadata of a distribution : a distinct .egg-info directory located > in the site-packages directory, containing the metadata. > > > There is something i don't understand there: > Does it imply that having no access on site-packages prevent you from installing > third party libraries as a limited user? As it s today, site-packages is just the default place, that is added and scan when Python starts. And you need root privileges to install packages there. There's another place where you can use when you are not root: per-user site packages. But those two place are just the places for "python-the-package-manager". > For example, on shared hosting where > you have limited privileges. Before, i could upload/build some libs and start > from there using, for exemple, buildout to scan that directory. > I make assumption that also the entry points are written along the metadata dir. > That will say that even if i don't have access to the site-packages but i have > played with sys.path to include my distribution, i won't have its entry points > (as the egg-info is not there) ? PEP 376 comes with a set of tools that will allow you to install/uninstall distributions in an arbitrary site-packages folder, and play with them. So it basically makes almost no differences for tools like zc.buildout-the-package-manager that is tweaking sys.path in the generated scripts. I guess the simplest way will be to make the "eggs" directory a regular site-package like folder. It will even simplify the scripts because you will not have to add one entry per eggs there as it is today. What could be awesome is to see a branch of zc.buildout built against distribute 0.7 when it starts to be usable, to experiment this. zc.buildout is a package manager, so it makes it one of the target use case for PEP 376 and other changes we will provide in distribute. Tarek From ziade.tarek at gmail.com Sat Oct 24 09:47:35 2009 From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=) Date: Sat, 24 Oct 2009 09:47:35 +0200 Subject: [Distutils] Distribute support added in zc.buildout In-Reply-To: <4AE293FE.7000601@cryptelium.net> References: <94bdd2610910230246y67667b1bo218901cfbbeee5b0@mail.gmail.com> <5cae42b20910230309q75857a56l29aa02996098acc1@mail.gmail.com> <94bdd2610910230333g6225e125g524a9e7b8c536b09@mail.gmail.com> <4AE293FE.7000601@cryptelium.net> Message-ID: <94bdd2610910240047v1c1598a3qcfa21a78ddb96f17@mail.gmail.com> On Sat, Oct 24, 2009 at 7:43 AM, kiorky wrote: > > > Encolpe Degoute a ?crit : >> Tarek Ziad? a ?crit : >> >> distribution-tool = nothing|setuptools|distribute|pipeau > > Although i can maybe make my way with distribute even if it can break backward > compatibility somehow, I strongly dislike pip too. Moreover, if it's a forced > choice. Forced choice ? Not sure to understand that. The Packaging Ecosystem is currently evolving a lot because it's badly needed. Distribute will promote pip because it does want to focus on being that thin layer on the top of distutils. But it's an installer. you will be able to write your own once PEP 376 is settled (like PyPM might do) Please read this thread and all it answers for the motivations we have: http://mail.python.org/pipermail/python-dev/2009-October/092754.html Last, notice is written in stone, you both are very welcome to help us in the discussions, or in the sprints. Like the PEPs, the Distribute and Distutils development are a community process. Tarek From regebro at gmail.com Sat Oct 24 10:06:36 2009 From: regebro at gmail.com (Lennart Regebro) Date: Sat, 24 Oct 2009 10:06:36 +0200 Subject: [Distutils] Distribute support added in zc.buildout In-Reply-To: <4AE293FE.7000601@cryptelium.net> References: <94bdd2610910230246y67667b1bo218901cfbbeee5b0@mail.gmail.com> <5cae42b20910230309q75857a56l29aa02996098acc1@mail.gmail.com> <94bdd2610910230333g6225e125g524a9e7b8c536b09@mail.gmail.com> <4AE293FE.7000601@cryptelium.net> Message-ID: <319e029f0910240106m79b6778erfa9c2aa38de105b6@mail.gmail.com> 2009/10/24 kiorky : > Moreover, if it's a forced choice. Unless your boss is forcing you, that's not a forced choice. :-) > More over, when you see that the only recipe useful has some strange things like: The only recipe useful? Huh. I guess nobody uses zc.buildout then. That must be why it has so bad ratings on PyPI. ;) A friendly recommendation: You need to get less negative. Seriously. -- Lennart Regebro: Python, Zope, Plone, Grok http://regebro.wordpress.com/ +33 661 58 14 64 From kiorky at cryptelium.net Sat Oct 24 11:08:04 2009 From: kiorky at cryptelium.net (kiorky) Date: Sat, 24 Oct 2009 11:08:04 +0200 Subject: [Distutils] Distribute sprint #1 wrapup In-Reply-To: <94bdd2610910240039s2f4f4626m2f1fc131af4a7178@mail.gmail.com> References: <94bdd2610910231546m2f185e0eqcb3fcd0850684101@mail.gmail.com> <4AE2334D.3000304@trueblade.com> <94bdd2610910231553j28b361em6ccd26c550a3dc36@mail.gmail.com> <4AE2972D.8080703@cryptelium.net> <94bdd2610910240039s2f4f4626m2f1fc131af4a7178@mail.gmail.com> Message-ID: <4AE2C3F4.50906@cryptelium.net> Tarek Ziad? a ?crit : > As it s today, site-packages is just the default place, that is added and scan > when Python starts. And you need root privileges to install packages there. > > There's another place where you can use when you are not root: > per-user site packages. > > But those two place are just the places for "python-the-package-manager". > Correct me if i am wrong but what i have understood is that the egg-info is not writed along with the "python code" but in another well known place. From then, how can i do the following as i don't want to be defaulted. 1. register directoy(ies) where to find distributions. 2. Register where to install the "python code" (new egg format). 3. Register where to install "egg-infos" 4. Be sure that both egg-infos from default places (system, home) AND my thirdparty place(?) are gathered together if possible. Or maybe just one place is possible ? In this case, we may need to provide means to assemble the whole installation to reproduce how things behave today. Maybe, we can make some additional writing on the PEP to make things clear on that point, i think that just things need to be clear. On the same subject, I have another reguard, It's for eggs actually released as only eggs on pypi, see [1]. How can i install them now ? Can we make wrappers to install them? I have at least two hosting services that give me access to /var/foo, but the user has no right either on python installation or its $HOME. [1] - http://pypi.python.org/pypi/Plomino/1.4 > Tarek -- Cordialement, KiOrKY GPG Key FingerPrint: 0x1A1194B7681112AF -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 261 bytes Desc: OpenPGP digital signature URL: From kiorky at cryptelium.net Sat Oct 24 11:12:18 2009 From: kiorky at cryptelium.net (kiorky) Date: Sat, 24 Oct 2009 11:12:18 +0200 Subject: [Distutils] Distribute support added in zc.buildout In-Reply-To: <94bdd2610910240047v1c1598a3qcfa21a78ddb96f17@mail.gmail.com> References: <94bdd2610910230246y67667b1bo218901cfbbeee5b0@mail.gmail.com> <5cae42b20910230309q75857a56l29aa02996098acc1@mail.gmail.com> <94bdd2610910230333g6225e125g524a9e7b8c536b09@mail.gmail.com> <4AE293FE.7000601@cryptelium.net> <94bdd2610910240047v1c1598a3qcfa21a78ddb96f17@mail.gmail.com> Message-ID: <4AE2C4F2.4090803@cryptelium.net> Tarek Ziad? a ?crit : > On Sat, Oct 24, 2009 at 7:43 AM, kiorky wrote: >> >> Encolpe Degoute a ?crit : >>> Tarek Ziad? a ?crit : > Last, notice is written in stone, you both are very welcome to help us > in the discussions, or in the sprints. You know why, my time is somehow limited. One thing, i can do now without loosing much time is boring you there with silly remarks :) I cannot promise to be there, i may be there for some, though i hope, but i can't schedule. > Like the PEPs, the Distribute and Distutils development are a community process. That's why even if i dislike some points upon it right now, i prefer it over setuptools. > Tarek -- Cordialement, KiOrKY GPG Key FingerPrint: 0x1A1194B7681112AF -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 261 bytes Desc: OpenPGP digital signature URL: From kiorky at cryptelium.net Sat Oct 24 11:16:11 2009 From: kiorky at cryptelium.net (kiorky) Date: Sat, 24 Oct 2009 11:16:11 +0200 Subject: [Distutils] Distribute support added in zc.buildout In-Reply-To: <319e029f0910240106m79b6778erfa9c2aa38de105b6@mail.gmail.com> References: <94bdd2610910230246y67667b1bo218901cfbbeee5b0@mail.gmail.com> <5cae42b20910230309q75857a56l29aa02996098acc1@mail.gmail.com> <94bdd2610910230333g6225e125g524a9e7b8c536b09@mail.gmail.com> <4AE293FE.7000601@cryptelium.net> <319e029f0910240106m79b6778erfa9c2aa38de105b6@mail.gmail.com> Message-ID: <4AE2C5DB.3080606@cryptelium.net> Lennart Regebro a ?crit : > 2009/10/24 kiorky : > The only recipe useful? Huh. I guess nobody uses zc.buildout then. > That must be why it has so bad ratings on PyPI. ;) If tomorrow frameworks i use, rely on that for deployment, it would be hard to ignore it. > A friendly recommendation: You need to get less negative. Seriously. I'm not negative and i think things evolve quite well :) I just want to make hard points not forgotten. As a side note, on all the projects i manage, all is already switched to use distribute. -- Cordialement, KiOrKY GPG Key FingerPrint: 0x1A1194B7681112AF -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 261 bytes Desc: OpenPGP digital signature URL: From kiorky at cryptelium.net Sat Oct 24 11:20:41 2009 From: kiorky at cryptelium.net (kiorky) Date: Sat, 24 Oct 2009 11:20:41 +0200 Subject: [Distutils] Distribute support added in zc.buildout In-Reply-To: <4AE2C5DB.3080606@cryptelium.net> References: <94bdd2610910230246y67667b1bo218901cfbbeee5b0@mail.gmail.com> <5cae42b20910230309q75857a56l29aa02996098acc1@mail.gmail.com> <94bdd2610910230333g6225e125g524a9e7b8c536b09@mail.gmail.com> <4AE293FE.7000601@cryptelium.net> <319e029f0910240106m79b6778erfa9c2aa38de105b6@mail.gmail.com> <4AE2C5DB.3080606@cryptelium.net> Message-ID: <4AE2C6E9.5000001@cryptelium.net> kiorky a ?crit : > > Lennart Regebro a ?crit : >> 2009/10/24 kiorky : >> The only recipe useful? Huh. I guess nobody uses zc.buildout then. And why i dislike so much pip is that i think it cannot provide me enough isolation as zc.buildout can. I don't want to have one python(venv)/project and afaik, i's the only way to do the same as i could do with buildout and shared cache directories. -- Cordialement, KiOrKY GPG Key FingerPrint: 0x1A1194B7681112AF -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 261 bytes Desc: OpenPGP digital signature URL: From regebro at gmail.com Sat Oct 24 11:32:49 2009 From: regebro at gmail.com (Lennart Regebro) Date: Sat, 24 Oct 2009 11:32:49 +0200 Subject: [Distutils] Distribute support added in zc.buildout In-Reply-To: <4AE2C5DB.3080606@cryptelium.net> References: <94bdd2610910230246y67667b1bo218901cfbbeee5b0@mail.gmail.com> <5cae42b20910230309q75857a56l29aa02996098acc1@mail.gmail.com> <94bdd2610910230333g6225e125g524a9e7b8c536b09@mail.gmail.com> <4AE293FE.7000601@cryptelium.net> <319e029f0910240106m79b6778erfa9c2aa38de105b6@mail.gmail.com> <4AE2C5DB.3080606@cryptelium.net> Message-ID: <319e029f0910240232u74465975wba6d7c52bfc9c617@mail.gmail.com> 2009/10/24 kiorky : > If tomorrow frameworks i use, rely on that for deployment, it would be hard to > ignore it. Yes. But hard to ignore does not equal forced. And especially in open source, where you always have the ability to improve or rewrite. -- Lennart Regebro: Python, Zope, Plone, Grok http://regebro.wordpress.com/ +33 661 58 14 64 From kiorky at cryptelium.net Sat Oct 24 11:43:03 2009 From: kiorky at cryptelium.net (kiorky) Date: Sat, 24 Oct 2009 11:43:03 +0200 Subject: [Distutils] Distribute support added in zc.buildout In-Reply-To: <319e029f0910240232u74465975wba6d7c52bfc9c617@mail.gmail.com> References: <94bdd2610910230246y67667b1bo218901cfbbeee5b0@mail.gmail.com> <5cae42b20910230309q75857a56l29aa02996098acc1@mail.gmail.com> <94bdd2610910230333g6225e125g524a9e7b8c536b09@mail.gmail.com> <4AE293FE.7000601@cryptelium.net> <319e029f0910240106m79b6778erfa9c2aa38de105b6@mail.gmail.com> <4AE2C5DB.3080606@cryptelium.net> <319e029f0910240232u74465975wba6d7c52bfc9c617@mail.gmail.com> Message-ID: <4AE2CC27.8070701@cryptelium.net> Lennart Regebro a ?crit : > 2009/10/24 kiorky : >> If tomorrow frameworks i use, rely on that for deployment, it would be hard to >> ignore it. > > Yes. But hard to ignore does not equal forced. And especially in open > source, where you always have the ability to improve or rewrite. In french that is called "mauvaise foi". A nearly translation is "You're acting in bad faith". -- Cordialement, KiOrKY GPG Key FingerPrint: 0x1A1194B7681112AF -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 261 bytes Desc: OpenPGP digital signature URL: From ssteinerx at gmail.com Sat Oct 24 15:26:15 2009 From: ssteinerx at gmail.com (ssteinerX@gmail.com) Date: Sat, 24 Oct 2009 09:26:15 -0400 Subject: [Distutils] Distribute support added in zc.buildout In-Reply-To: <4AE2CC27.8070701@cryptelium.net> References: <94bdd2610910230246y67667b1bo218901cfbbeee5b0@mail.gmail.com> <5cae42b20910230309q75857a56l29aa02996098acc1@mail.gmail.com> <94bdd2610910230333g6225e125g524a9e7b8c536b09@mail.gmail.com> <4AE293FE.7000601@cryptelium.net> <319e029f0910240106m79b6778erfa9c2aa38de105b6@mail.gmail.com> <4AE2C5DB.3080606@cryptelium.net> <319e029f0910240232u74465975wba6d7c52bfc9c617@mail.gmail.com> <4AE2CC27.8070701@cryptelium.net> Message-ID: <4D982E59-1EBB-4984-ACA8-172C97AFA123@gmail.com> On Oct 24, 2009, at 5:43 AM, kiorky wrote: > > > Lennart Regebro a ?crit : >> 2009/10/24 kiorky : >>> If tomorrow frameworks i use, rely on that for deployment, it >>> would be hard to >>> ignore it. >> >> Yes. But hard to ignore does not equal forced. And especially in open >> source, where you always have the ability to improve or rewrite. > > In french that is called "mauvaise foi". > A nearly translation is "You're acting in bad faith". Ok, why don't y'all go run off and have an argument in French and come back when you're done...and ready to play nice. S From floris.bruynooghe at gmail.com Sat Oct 24 16:17:58 2009 From: floris.bruynooghe at gmail.com (Floris Bruynooghe) Date: Sat, 24 Oct 2009 15:17:58 +0100 Subject: [Distutils] [RFC] PEP 345 and PEP 386 updates In-Reply-To: <94bdd2610910200139w5db29003pab66d5febca7ee4e@mail.gmail.com> References: <94bdd2610910200139w5db29003pab66d5febca7ee4e@mail.gmail.com> Message-ID: <20091024141758.GA31901@laurie.devork> On Tue, Oct 20, 2009 at 10:39:51AM +0200, Tarek Ziad? wrote: > PEP 345 is the PEP for the new metadata format, replacing PEP 314. I > have upgraded it, using Tres's work from last year, and the next step > is to add the context marker presented in PEP 390. > > I have also updated PEP 386 to link it to PEP 345. [...] > Please comment! In PEP 345 the Provides-Dist field mentions that when you use a virtual distribution you automatically conflict with other distributions providing this virtual dist. The next field, Obsoletes-Dist, explicitly says it's to conflict with another dist. I guess you can obsolte a dist without providing it, but when would this be used? Basically I don't fully understand the intended manings and interactions of Provides-Dist and Obsoletes-Dist. Also a minor typo in the Requires-External field: the examples just use "Requires" as field name. Regards Floris From kiorky at cryptelium.net Sat Oct 24 17:56:53 2009 From: kiorky at cryptelium.net (kiorky) Date: Sat, 24 Oct 2009 17:56:53 +0200 Subject: [Distutils] Distribute support added in zc.buildout In-Reply-To: <4D982E59-1EBB-4984-ACA8-172C97AFA123@gmail.com> References: <94bdd2610910230246y67667b1bo218901cfbbeee5b0@mail.gmail.com> <5cae42b20910230309q75857a56l29aa02996098acc1@mail.gmail.com> <94bdd2610910230333g6225e125g524a9e7b8c536b09@mail.gmail.com> <4AE293FE.7000601@cryptelium.net> <319e029f0910240106m79b6778erfa9c2aa38de105b6@mail.gmail.com> <4AE2C5DB.3080606@cryptelium.net> <319e029f0910240232u74465975wba6d7c52bfc9c617@mail.gmail.com> <4AE2CC27.8070701@cryptelium.net> <4D982E59-1EBB-4984-ACA8-172C97AFA123@gmail.com> Message-ID: <4AE323C5.9020603@cryptelium.net> ssteinerX at gmail.com a ?crit : > > Ok, why don't y'all go run off and have an argument in French and come > back when you're done...and ready to play nice. Just read back in the mailing, i give such arguments. At leat today. > > S > -- Cordialement, KiOrKY GPG Key FingerPrint: 0x1A1194B7681112AF -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 261 bytes Desc: OpenPGP digital signature URL: From kiorky at cryptelium.net Sat Oct 24 19:13:06 2009 From: kiorky at cryptelium.net (kiorky) Date: Sat, 24 Oct 2009 19:13:06 +0200 Subject: [Distutils] Distribute sprint #1 wrapup In-Reply-To: <94bdd2610910240039s2f4f4626m2f1fc131af4a7178@mail.gmail.com> References: <94bdd2610910231546m2f185e0eqcb3fcd0850684101@mail.gmail.com> <4AE2334D.3000304@trueblade.com> <94bdd2610910231553j28b361em6ccd26c550a3dc36@mail.gmail.com> <4AE2972D.8080703@cryptelium.net> <94bdd2610910240039s2f4f4626m2f1fc131af4a7178@mail.gmail.com> Message-ID: <4AE335A2.9070107@cryptelium.net> Whow sorry, Ignore my precedent mail, i missed the end of that email., i do not know why. Blame my eyes or thunderbird :/ > PEP 376 comes with a set of tools that will allow you to > install/uninstall distributions > in an arbitrary site-packages folder, and play with them. So it > basically makes almost no > differences for tools like zc.buildout-the-package-manager that is > tweaking sys.path in > the generated scripts. Great (really) :) Entry points are working as well on those non standard places also, aren't they? > I guess the simplest way will be to make the "eggs" directory a > regular site-package > like folder. It will even simplify the scripts because you will not > have to add one entry per eggs > there as it is today. I don't think it is a really good idea because you will not have isolation at project level. If you have A, B, C in this alternative site-packages, it would mean that A, B, and C are importable. But when i install something with "eggs=A C", i want that my pythonpath contains A and C but no B.I think the buildout way to do that actually is not that bad :) > What could be awesome is to see a branch of zc.buildout built against > distribute 0.7 Indeed. I ll rework the minitage recipes also to use pip and prepare work for distribute on a branch as soon a possible, i hope next we. > when it starts to be usable, to experiment this. > > zc.buildout is a package manager, so it makes it one of the target use > case for PEP 376 > and other changes we will provide in distribute. > How about trying to make a mirror of pypi packages rebuilt in the distribute format. That will prevent the overhead for users to contact the maintainer in case or breakage, to the maintainer to have to repackage its already packaged things, and also for the user to have problems installing 'foo' even it is a very simple package that we could just provide a rebuilt version to him. I think about that mainly for distributions packaged only as eggs (in the actual format). We could maybe extract this list of packages with no sdist (not that much i think and the only one i know is plomino) and repackage them. That's very low priority, but it can be useful and one more other step which will make the transition as transparent as possible. So mainly because i did not find time yet to test, and i will, my only concern WHICH MAY BE UNFOUNDED is on namespaces conflicts between setuptools and distribute. For me, the first one loaded module wins. And, it's low priority any way i think as a patch may by easy to do. Just one thing to borrow from the precedent mail would be to make one sentence on the PEP to write in rock that we can make installation elsewhere that in standard place for both python code & metadata. -- Cordialement, KiOrKY GPG Key FingerPrint: 0x1A1194B7681112AF -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 261 bytes Desc: OpenPGP digital signature URL: From sdouche at gmail.com Sun Oct 25 01:32:06 2009 From: sdouche at gmail.com (Sebastien Douche) Date: Sun, 25 Oct 2009 01:32:06 +0200 Subject: [Distutils] [buildout] A few pending issues In-Reply-To: References: <1099b90b0910221036k66fc4317x8098564df3321c30@mail.gmail.com> Message-ID: <5e1183fa0910241632s49e993b4v1294beb842fe7b06@mail.gmail.com> On Thu, Oct 22, 2009 at 19:59, Kai Lautaportti wrote: >> You could get your changes checked in on branches for my review. ?I >> have a bunch of other changes I need to review and merge first though. > > Thanks, I will try to get commit access to the Zope SVN sorted out > before the conference so I can prepare the branches for your review > and hopefully work on #113085 too at the sprint. Hi Kai, another solution is to work wth DVCS and send a patch on the list. A Hg mirror is available: http://hg.zope.mirrors.securactive.org/zc.buildout/ Cheers. -- Sebastien Douche Twitter: http://bit.ly/afkrK (agile, python, open source) From kai.lautaportti at hexagonit.fi Sun Oct 25 16:42:22 2009 From: kai.lautaportti at hexagonit.fi (Kai Lautaportti) Date: Sun, 25 Oct 2009 18:42:22 +0300 Subject: [Distutils] [buildout] A few pending issues In-Reply-To: <5e1183fa0910241632s49e993b4v1294beb842fe7b06@mail.gmail.com> References: <1099b90b0910221036k66fc4317x8098564df3321c30@mail.gmail.com> <5e1183fa0910241632s49e993b4v1294beb842fe7b06@mail.gmail.com> Message-ID: On Sun, Oct 25, 2009 at 2:32 AM, Sebastien Douche wrote: > On Thu, Oct 22, 2009 at 19:59, Kai Lautaportti > wrote: > >>> You could get your changes checked in on branches for my review. ?I >>> have a bunch of other changes I need to review and merge first though. >> >> Thanks, I will try to get commit access to the Zope SVN sorted out >> before the conference so I can prepare the branches for your review >> and hopefully work on #113085 too at the sprint. > > Hi Kai, > another solution is to work wth DVCS and send a patch on the list. A > Hg mirror is available: > http://hg.zope.mirrors.securactive.org/zc.buildout/ Right, that's what I've basically done with the patches attached to the launchpad issues. I assumed that having the changes in branches on svn.zope.org is what Jim prefers for his review. - Kai -- Kai Lautaportti kai.lautaportti at hexagonit.fi From tseaver at palladion.com Mon Oct 26 03:11:00 2009 From: tseaver at palladion.com (Tres Seaver) Date: Sun, 25 Oct 2009 22:11:00 -0400 Subject: [Distutils] [RFC] PEP 345 and PEP 386 updates In-Reply-To: <20091024141758.GA31901@laurie.devork> References: <94bdd2610910200139w5db29003pab66d5febca7ee4e@mail.gmail.com> <20091024141758.GA31901@laurie.devork> Message-ID: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Floris Bruynooghe wrote: > On Tue, Oct 20, 2009 at 10:39:51AM +0200, Tarek Ziad? wrote: >> PEP 345 is the PEP for the new metadata format, replacing PEP 314. I >> have upgraded it, using Tres's work from last year, and the next step >> is to add the context marker presented in PEP 390. >> >> I have also updated PEP 386 to link it to PEP 345. > [...] >> Please comment! > > In PEP 345 the Provides-Dist field mentions that when you use a > virtual distribution you automatically conflict with other > distributions providing this virtual dist. The next field, > Obsoletes-Dist, explicitly says it's to conflict with another dist. I > guess you can obsolte a dist without providing it, but when would this > be used? Basically I don't fully understand the intended manings and > interactions of Provides-Dist and Obsoletes-Dist. Obsoletes is normally used (as with dpkg / rpm) to cope with changes in distribution (package) names. Tres. - -- =================================================================== Tres Seaver +1 540-429-0999 tseaver at palladion.com Palladion Software "Excellence by Design" http://palladion.com -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iEYEARECAAYFAkrlBSoACgkQ+gerLs4ltQ4tUACgltNdAHD8GV6tbVrHvPgO6EuI mw4AoLU4qhQ5qv4yczdyD7QEdJK5FtSG =70rV -----END PGP SIGNATURE----- From floris.bruynooghe at gmail.com Mon Oct 26 10:59:06 2009 From: floris.bruynooghe at gmail.com (Floris Bruynooghe) Date: Mon, 26 Oct 2009 09:59:06 +0000 Subject: [Distutils] [RFC] PEP 345 and PEP 386 updates In-Reply-To: References: <94bdd2610910200139w5db29003pab66d5febca7ee4e@mail.gmail.com> <20091024141758.GA31901@laurie.devork> Message-ID: <20091026095906.GA19540@laurie.devork> On Sun, Oct 25, 2009 at 10:11:00PM -0400, Tres Seaver wrote: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > Floris Bruynooghe wrote: > > On Tue, Oct 20, 2009 at 10:39:51AM +0200, Tarek Ziad? wrote: > >> PEP 345 is the PEP for the new metadata format, replacing PEP 314. I > >> have upgraded it, using Tres's work from last year, and the next step > >> is to add the context marker presented in PEP 390. > >> > >> I have also updated PEP 386 to link it to PEP 345. > > [...] > >> Please comment! > > > > In PEP 345 the Provides-Dist field mentions that when you use a > > virtual distribution you automatically conflict with other > > distributions providing this virtual dist. The next field, > > Obsoletes-Dist, explicitly says it's to conflict with another dist. I > > guess you can obsolte a dist without providing it, but when would this > > be used? Basically I don't fully understand the intended manings and > > interactions of Provides-Dist and Obsoletes-Dist. > > Obsoletes is normally used (as with dpkg / rpm) to cope with changes in > distribution (package) names. But in dpkg you can have multiple packages installed with the same Provides as long as they don't mention each other in the Conflicts header. Floris From ziade.tarek at gmail.com Tue Oct 27 09:53:02 2009 From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=) Date: Tue, 27 Oct 2009 09:53:02 +0100 Subject: [Distutils] Distribute 0.6.7 this week Message-ID: <94bdd2610910270153n542a5911n82efbb67bfb241ab@mail.gmail.com> Hello We are going to release 0.6.7 this week. Among some new bugs fixes, we want to provide a specific change that will allow virtualenv to be released with an option to use Distribute instead of Setuptools, like zc.buildout. $ virtualenv --distribute ENV If you want to see another bug fixed in this release, speak up ! (http://bitbucket.org/tarek/distribute/issues) Regards Tarek -- Tarek Ziad? | http://ziade.org |????????????! |???????????? From ssteinerx at gmail.com Tue Oct 27 15:12:36 2009 From: ssteinerx at gmail.com (ssteinerX@gmail.com) Date: Tue, 27 Oct 2009 10:12:36 -0400 Subject: [Distutils] Distribute 0.6.7 this week In-Reply-To: <94bdd2610910270153n542a5911n82efbb67bfb241ab@mail.gmail.com> References: <94bdd2610910270153n542a5911n82efbb67bfb241ab@mail.gmail.com> Message-ID: <5C20DB98-E621-4FE8-9087-372F7280305B@gmail.com> On Oct 27, 2009, at 4:53 AM, Tarek Ziad? wrote: > Hello > > We are going to release 0.6.7 this week. Among some new bugs fixes, we > want to provide a specific change that will > allow virtualenv to be released with an option to use Distribute > instead of Setuptools, like zc.buildout. > > $ virtualenv --distribute ENV > > If you want to see another bug fixed in this release, speak up ! > (http://bitbucket.org/tarek/distribute/issues) I don't necessarily have a request bug-wise, but maybe this would be a good time to coordinate the upgrading of the test suite with the release. Maybe something like a TEST_README documenting the tests that prove the that the closed ticket's bugs have a test that proves that the former bad behaviour was fixed. S From ziade.tarek at gmail.com Tue Oct 27 15:21:54 2009 From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=) Date: Tue, 27 Oct 2009 15:21:54 +0100 Subject: [Distutils] Distribute 0.6.7 this week In-Reply-To: <5C20DB98-E621-4FE8-9087-372F7280305B@gmail.com> References: <94bdd2610910270153n542a5911n82efbb67bfb241ab@mail.gmail.com> <5C20DB98-E621-4FE8-9087-372F7280305B@gmail.com> Message-ID: <94bdd2610910270721t3026eb7dn7ad54df053891c64@mail.gmail.com> On Tue, Oct 27, 2009 at 3:12 PM, ssteinerX at gmail.com wrote: > > On Oct 27, 2009, at 4:53 AM, Tarek Ziad? wrote: > >> Hello >> >> We are going to release 0.6.7 this week. Among some new bugs fixes, we >> want to provide a specific change that will >> allow virtualenv to be released with an option to use Distribute >> instead of Setuptools, like zc.buildout. >> >> $ virtualenv --distribute ENV >> >> If you want to see another bug fixed in this release, ?speak up ! >> (http://bitbucket.org/tarek/distribute/issues) > > I don't necessarily have a request bug-wise, but maybe this would be a good > time to coordinate the upgrading of the test suite with the release. > > Maybe something like a TEST_README documenting the tests that prove the that > the closed ticket's bugs have a test that proves that the former bad > behaviour was fixed. Maybe a simpler way would be to use Bitbucket issue tracker features ? For the last two bugs I've fixed, they have a corresponding test in the commit. And if you go to the issue, you have a link to the commit with a diff, so you can see it. That's done with the special "fixes #N" bit in the commit messages, where N is the issue number. Bitbuckets links the tyicket with the commits when its pushed there. So we could maybe add the link to the issue in CHANGES ? Although some parts are cleary undertested yet, so even if it's unpleasant to add tests in an undertested code base, I am +1 in making the tests mandatory when the code is not yet covered for now on Tarek From ssteinerx at gmail.com Tue Oct 27 15:26:05 2009 From: ssteinerx at gmail.com (ssteinerX@gmail.com) Date: Tue, 27 Oct 2009 10:26:05 -0400 Subject: [Distutils] Distribute 0.6.7 this week In-Reply-To: <94bdd2610910270721t3026eb7dn7ad54df053891c64@mail.gmail.com> References: <94bdd2610910270153n542a5911n82efbb67bfb241ab@mail.gmail.com> <5C20DB98-E621-4FE8-9087-372F7280305B@gmail.com> <94bdd2610910270721t3026eb7dn7ad54df053891c64@mail.gmail.com> Message-ID: On Oct 27, 2009, at 10:21 AM, Tarek Ziad? wrote: >> Maybe something like a TEST_README documenting the tests that prove >> the that >> the closed ticket's bugs have a test that proves that the former bad >> behaviour was fixed. > > Maybe a simpler way would be to use Bitbucket issue tracker features ? Are you suggesting maybe getting at that through the (undocumented) API, or something else? > For the last two bugs I've fixed, they have a corresponding test in > the commit. > And if you go to the issue, you have a link to the commit with a diff, > so you can see it. > > That's done with the special "fixes #N" bit in the commit messages, > where N is the issue number. Bitbuckets links the tyicket with the > commits when its pushed there. So we could maybe add the link to > the issue in CHANGES ? I'm not sure what mechanisms we have for this, do you know? > Although some parts are cleary undertested yet, so even if it's > unpleasant to add tests in an undertested code base, I am +1 in > making the tests mandatory when the code is not yet covered for now on That was what we discussed at the sprint and I'm doing what I feel is my job as QA PITA (pain in the ass) by insisting that A> we have tests for anything fixed and especially not covered and B> that we document the test related to the fix so that whomever reported the bug can verify that the test does indeed cover the use-case they reported. What is the next step? S From ziade.tarek at gmail.com Tue Oct 27 15:37:59 2009 From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=) Date: Tue, 27 Oct 2009 15:37:59 +0100 Subject: [Distutils] Distribute 0.6.7 this week In-Reply-To: References: <94bdd2610910270153n542a5911n82efbb67bfb241ab@mail.gmail.com> <5C20DB98-E621-4FE8-9087-372F7280305B@gmail.com> <94bdd2610910270721t3026eb7dn7ad54df053891c64@mail.gmail.com> Message-ID: <94bdd2610910270737j7460da43x9f5d093be7bec5d9@mail.gmail.com> On Tue, Oct 27, 2009 at 3:26 PM, ssteinerX at gmail.com wrote: >> That's done with the special "fixes #N" bit in the commit messages, >> where N is the issue number. ?Bitbuckets links the tyicket with the >> commits when its pushed there. ?So we could maybe add the link to the issue >> in CHANGES ? > > I'm not sure what mechanisms we have for this, do you know? Yes, I use it. I have added this feature in the admin panel, provided by Bitbucket: $ hg ci -m "blablablba fixes #12" -> add a link to the diff in the issue #12 and close it $ hg ci -m "blablablba refs #12" -> add a link to the diff in the issue #12, but don't close it. > >> Although some parts are cleary undertested yet, so even if it's >> unpleasant to add tests in an undertested code base, I am +1 in making the >> tests mandatory when the code is not yet covered for now on > > That was what we discussed at the sprint and I'm doing what I feel is my job > as QA PITA (pain in the ass) by insisting that A> we have tests for anything > fixed and especially not covered and B> that we document the test related to > the fix so that whomever reported the bug can verify that the test does > indeed cover the use-case they reported. > > What is the next step? That sounds very cool ! As the QA master I guess your next steps would be: - to set up a bbot for distribute (you can PV me so we can work on the details with Titus that has offered some slave hosting too) - to watch all commits going on and blame people that don't do tests, or make comments on the quality of the commited code. An extra pair of eyes is great for that. Granted that we could all do that, but QA needs a lead or it slowly dies imho. I think we would all very appreciate it if you could help on this! Tarek From chris at simplistix.co.uk Tue Oct 27 16:35:03 2009 From: chris at simplistix.co.uk (Chris Withers) Date: Tue, 27 Oct 2009 15:35:03 +0000 Subject: [Distutils] why would you ever need to specify setuptools as a dependency? In-Reply-To: <20091020201128.GA31378@clingman.lan> References: <4ACF1956.5070903@simplistix.co.uk> <5cae42b20910090457p3f6e4a1bp361b240f974f1617@mail.gmail.com> <5cae42b20910131055s5e152d62r4f7d385b1de98c9f@mail.gmail.com> <4ADD92F9.5050404@simplistix.co.uk> <4ADD997C.3020808@cryptelium.net> <9cee7ab80910200431j579bbe4fke78aa93f8c28729d@mail.gmail.com> <4ADDBD74.3080004@simplistix.co.uk> <9cee7ab80910200643q1617b460pb7fd98007d52275a@mail.gmail.com> <4ADDBFCA.40902@simplistix.co.uk> <20091020201128.GA31378@clingman.lan> Message-ID: <4AE71327.7090101@simplistix.co.uk> Toshio Kuratomi wrote: > On Tue, Oct 20, 2009 at 02:48:58PM +0100, Chris Withers wrote: >> Fred Drake wrote: >>> On Tue, Oct 20, 2009 at 9:39 AM, Chris Withers wrote: >>>> As is specifying the setuptools distribution as a requirement when you're >>>> already using it... >>> I don't use setuptools at runtime unless something requires it. >>> >>> Having it available at install time and run time are two different >>> things, and should remain so. >> All I'm saying is that packages shouldn't express a dependency on >> setuptools if setuptools is required even before that expression can be >> parsed. >> >> I'm not talking about install or run time... >> > Are you then talking about build time? I'm talking about *at all*. Chris -- Simplistix - Content Management, Batch Processing & Python Consulting - http://www.simplistix.co.uk From fdrake at gmail.com Tue Oct 27 16:51:43 2009 From: fdrake at gmail.com (Fred Drake) Date: Tue, 27 Oct 2009 11:51:43 -0400 Subject: [Distutils] why would you ever need to specify setuptools as a dependency? In-Reply-To: <4AE71327.7090101@simplistix.co.uk> References: <4ACF1956.5070903@simplistix.co.uk> <5cae42b20910131055s5e152d62r4f7d385b1de98c9f@mail.gmail.com> <4ADD92F9.5050404@simplistix.co.uk> <4ADD997C.3020808@cryptelium.net> <9cee7ab80910200431j579bbe4fke78aa93f8c28729d@mail.gmail.com> <4ADDBD74.3080004@simplistix.co.uk> <9cee7ab80910200643q1617b460pb7fd98007d52275a@mail.gmail.com> <4ADDBFCA.40902@simplistix.co.uk> <20091020201128.GA31378@clingman.lan> <4AE71327.7090101@simplistix.co.uk> Message-ID: <9cee7ab80910270851l5e953122v59651ce15a0039e8@mail.gmail.com> On Tue, Oct 27, 2009 at 11:35 AM, Chris Withers wrote: > I'm talking about *at all*. If I don't include setuptools in install_requires, then I've no business expecting it to be available at runtime. Since our current approach in the Zope community is to use pkg_resources to support namespace packages, and we get that via setuptools, it makes sense to specify that my packages require setuptools. It's just unfortunate that we're not able to get what we need from the standard library for all interesting versions of Python. -Fred -- Fred L. Drake, Jr. "Chaos is the score upon which reality is written." --Henry Miller From chris at simplistix.co.uk Tue Oct 27 17:26:32 2009 From: chris at simplistix.co.uk (Chris Withers) Date: Tue, 27 Oct 2009 16:26:32 +0000 Subject: [Distutils] why would you ever need to specify setuptools as a dependency? In-Reply-To: <9cee7ab80910270851l5e953122v59651ce15a0039e8@mail.gmail.com> References: <4ACF1956.5070903@simplistix.co.uk> <5cae42b20910131055s5e152d62r4f7d385b1de98c9f@mail.gmail.com> <4ADD92F9.5050404@simplistix.co.uk> <4ADD997C.3020808@cryptelium.net> <9cee7ab80910200431j579bbe4fke78aa93f8c28729d@mail.gmail.com> <4ADDBD74.3080004@simplistix.co.uk> <9cee7ab80910200643q1617b460pb7fd98007d52275a@mail.gmail.com> <4ADDBFCA.40902@simplistix.co.uk> <20091020201128.GA31378@clingman.lan> <4AE71327.7090101@simplistix.co.uk> <9cee7ab80910270851l5e953122v59651ce15a0039e8@mail.gmail.com> Message-ID: <4AE71F38.8020703@simplistix.co.uk> Fred Drake wrote: > On Tue, Oct 27, 2009 at 11:35 AM, Chris Withers wrote: >> I'm talking about *at all*. > > If I don't include setuptools in install_requires, then I've no > business expecting it to be available at runtime. You're ignoring the fact that for the information in install_requires to even be parsed, you need to have setuptools present. I'll say it again: it seems pointless specifying a requirement that has to be met before any requirement specifications can even be processed... Chris -- Simplistix - Content Management, Batch Processing & Python Consulting - http://www.simplistix.co.uk From chris at simplistix.co.uk Tue Oct 27 17:41:41 2009 From: chris at simplistix.co.uk (Chris Withers) Date: Tue, 27 Oct 2009 16:41:41 +0000 Subject: [Distutils] PEP 345 In-Reply-To: <94bdd2610910200912s52b1bbacia89c831202bb7780@mail.gmail.com> References: <94bdd2610910111517u2fd0b136p44958f7faa46e974@mail.gmail.com> <20091012090115.GA31472@laurie.devork> <94bdd2610910120249i40f06b2fpc775a5be22703269@mail.gmail.com> <4ADDAD68.30607@simplistix.co.uk> <94bdd2610910200536r6569cbaayd9d074d52cd53169@mail.gmail.com> <4ADDB388.4080200@simplistix.co.uk> <94bdd2610910200617h52c0feeftd6f7ee7831d34ea4@mail.gmail.com> <4ADDBF7C.7010905@simplistix.co.uk> <94bdd2610910200653l38be53a0hfebfe8ccaf3fe5db@mail.gmail.com> <4ADDD46C.8080506@simplistix.co.uk> <94bdd2610910200912s52b1bbacia89c831202bb7780@mail.gmail.com> Message-ID: <4AE722C5.9030900@simplistix.co.uk> Tarek Ziad? wrote: > On Tue, Oct 20, 2009 at 5:17 PM, Chris Withers wrote: >> Tarek Ziad? wrote: >>>> A quick scan of http://www.python.org/dev/peps/pep-0345/ shows no sign of >>>> any of the contentious bits of PEP390. What am I missing? >>> That's the part I have not added yet. Markers are the "sys_platform == >>> 'win32" things >>> discussed in PEP 390. >> But PEP345 is about the metadata that appears in the installed package, no? >> >> What use is it having conditionals in there at all? > > Because we want to add the PEP 390 context dependent markers in there, > as explained in PEP 390 (I've updated it yesterday, have a look) > > It provides the markers to any third party app that wants to read the metadata, > and make it avaiable directly at PyPI. I can understand the latter, but the former seems pointless. Comments on 345: - Requires/Provides seems like a bad idea. I'm pretty sure distributions are the only thing that should be involved in Requirements and the only things that should have versions. Why not just abandon Requires and Provides and have Requires-Dist and Provides-Dist instead? - Obsoletes/Obsoletes-Dist seems unlikely to be used. If it was called "Incompatible-With" it'd make more sense. eg: for Setuptools and Distribute - having Provides-Dist as a multiple use seems wrong. How can one distribution contain multiple distributions? Seems like a bad idea to me, even if it were possible... Comments on 390: - how come the section about metadata references PEP 314 and not 345? - as others have commented, having ; be a special character in fields which are already free text feels like it might be brittle. - you already know how thrilled I am about the proposed metadata format ;-) > That's from this summer: some folks found the scheme overcomplex > and dont want to see it as the standard. I am working in finding back > the details. Look forward to hearing about that, since it's the least contentious of all the proposed changes and it'd be great to get it into the next release(s) of python... Chris -- Simplistix - Content Management, Batch Processing & Python Consulting - http://www.simplistix.co.uk From chris at simplistix.co.uk Tue Oct 27 17:47:15 2009 From: chris at simplistix.co.uk (Chris Withers) Date: Tue, 27 Oct 2009 16:47:15 +0000 Subject: [Distutils] [RFC] PEP 345 and PEP 386 updates In-Reply-To: <94bdd2610910210618j58a493c5v922dd0c22c4c5f07@mail.gmail.com> References: <94bdd2610910200139w5db29003pab66d5febca7ee4e@mail.gmail.com> <4ADDA10C.5010804@egenix.com> <9cee7ab80910200636p2384615fy516a0ce924ec0567@mail.gmail.com> <4ADDD552.5070502@egenix.com> <94bdd2610910201311n3b1d33d0i8e9142d716d444f3@mail.gmail.com> <4ADEC976.6070701@egenix.com> <94bdd2610910210243y7151ff59u2216c6889a7ac887@mail.gmail.com> <4ADEF2A9.9020604@trueblade.com> <94bdd2610910210618j58a493c5v922dd0c22c4c5f07@mail.gmail.com> Message-ID: <4AE72413.5010809@simplistix.co.uk> Tarek Ziad? wrote: >> I thought the plan was to stick with major.minor and "==", "!=", "in" and >> "not in" until PEP 386 was accepted, then switch over to having all versions >> (including Python's) be some sort of Version object, at which point we can >> have richer version comparisons, with all of the existing uses being >> compatible. > > Yes that's the plan too. PEP 345 is not updated yet, but that is > clearly stated in PEP 386. So why don't we just pause for a while, get 386 done, implemented in and then come back to 345? I can't see why we'd want to not treat a Python version like any other package version, but if we need 386 to get that, then lets do that rather than finding convoluted ways to work around the problem... Chris -- Simplistix - Content Management, Batch Processing & Python Consulting - http://www.simplistix.co.uk From chris at simplistix.co.uk Tue Oct 27 17:52:36 2009 From: chris at simplistix.co.uk (Chris Withers) Date: Tue, 27 Oct 2009 16:52:36 +0000 Subject: [Distutils] Alternate static metadata PEP submission... In-Reply-To: <4ADECB30.10605@ar.media.kyoto-u.ac.jp> References: <20091013103021.GA21066@laurie.devork> <94bdd2610910131001v70711ffeia27433cd23b32fca@mail.gmail.com> <79a53e276e5dffe24be0433771801d72@preisshare.net> <94bdd2610910131426k64f707d7pf4d398d04b8ecc32@mail.gmail.com> <20091013233654.GA27339@laurie.devork> <4ADDB2CB.1090101@simplistix.co.uk> <8e9d1c135f56a0b37f12afe144b30f1a@preisshare.net> <4ADEB716.5020605@simplistix.co.uk> <4ADECB30.10605@ar.media.kyoto-u.ac.jp> Message-ID: <4AE72554.6060108@simplistix.co.uk> David Cournapeau wrote: > """ > from distutils.core import setup > from toydist import parse_static > > info_dict = parse_static('toysetup.info').to_dict() > setup(**info_dict) > """ Yep, this is the kind of thing I'm looking for. Except if we're talking about the core, it would be: from distutils.core import setup,parse_static setup(**parse_static('setup.cfg').to_dict()) ...and of course, because setup.cfg is the default name, we just end up with: from distutils.core import setup setup() ...unless you want to name setup.cfg something else, right? Chris -- Simplistix - Content Management, Batch Processing & Python Consulting - http://www.simplistix.co.uk From chris at simplistix.co.uk Tue Oct 27 17:53:06 2009 From: chris at simplistix.co.uk (Chris Withers) Date: Tue, 27 Oct 2009 16:53:06 +0000 Subject: [Distutils] Alternate static metadata PEP submission... In-Reply-To: References: <20091013103021.GA21066@laurie.devork> <94bdd2610910131001v70711ffeia27433cd23b32fca@mail.gmail.com> <79a53e276e5dffe24be0433771801d72@preisshare.net> <94bdd2610910131426k64f707d7pf4d398d04b8ecc32@mail.gmail.com> <20091013233654.GA27339@laurie.devork> <4ADDB2CB.1090101@simplistix.co.uk> <8e9d1c135f56a0b37f12afe144b30f1a@preisshare.net> <4ADEB716.5020605@simplistix.co.uk> Message-ID: <4AE72572.6080803@simplistix.co.uk> David Lyon wrote: > but I want my proposed features implemented.. and code is the only > way to do it. yes, but that code belongs in the standard library, not in each and every single package distribution... Chris -- Simplistix - Content Management, Batch Processing & Python Consulting - http://www.simplistix.co.uk From chris at simplistix.co.uk Tue Oct 27 17:55:34 2009 From: chris at simplistix.co.uk (Chris Withers) Date: Tue, 27 Oct 2009 16:55:34 +0000 Subject: [Distutils] Alternate static metadata PEP submission... In-Reply-To: <54d034c30100528e63b573ec3d7b137a@preisshare.net> References: <20091013103021.GA21066@laurie.devork> <94bdd2610910131001v70711ffeia27433cd23b32fca@mail.gmail.com> <79a53e276e5dffe24be0433771801d72@preisshare.net> <94bdd2610910131426k64f707d7pf4d398d04b8ecc32@mail.gmail.com> <20091013233654.GA27339@laurie.devork> <4ADDB2CB.1090101@simplistix.co.uk> <56bcb7f7893e86ec565d89b7ef40f418@preisshare.net> <4ADEB83E.8080804@simplistix.co.uk> <404efdf0624b873e17727d1f5966944d@preisshare.net> <4ADEC020.1050806@simplistix.co.uk> <54d034c30100528e63b573ec3d7b137a@preisshare.net> Message-ID: <4AE72606.60808@simplistix.co.uk> David Lyon wrote: > ok. But nobody I know in software can stop new patches and versions > of software from being made. Versions advance.. For some it's even a > day job. That's a poor argument for increasing the workload on all package maintainers. Backwards compatibility is the valid argument that you've made. But, for me, that can be solved by requiring installation of a new version of distutils (which I'm -0 on) or waiting for a new python release (which I'm +1 on). cheers, Chris -- Simplistix - Content Management, Batch Processing & Python Consulting - http://www.simplistix.co.uk From chris at simplistix.co.uk Tue Oct 27 17:59:02 2009 From: chris at simplistix.co.uk (Chris Withers) Date: Tue, 27 Oct 2009 16:59:02 +0000 Subject: [Distutils] tracking requested vs dependency installs in PEP 376 metadata In-Reply-To: <435FDA0D-8B91-492F-B0C2-FE55971BE077@mac.com> References: <4ACE15C5.4030105@meyerloewen.net> <20091008191526.GD513@clingman.lan> <4ACE4078.70504@meyerloewen.net> <20091008195227.GA735@clingman.lan> <4ACF3E77.8080806@dirtcircle.com> <319e029f0910110727l7948ffccu6acee5617ea795bf@mail.gmail.com> <1E5BF8F0-B345-4EF5-9C57-79E0F7C369D8@mac.com> <319e029f0910111158y68fb25fajb3cbaf713062bf11@mail.gmail.com> <4ADDA900.4090505@simplistix.co.uk> <435FDA0D-8B91-492F-B0C2-FE55971BE077@mac.com> Message-ID: <4AE726D6.4030901@simplistix.co.uk> Ronald Oussoren wrote: > >> Code should always specify their dependencies, not doing so will cause >> you problems down the line... > > Sure, and I do that for for any real code. But do you really expect that > users are careful to specify dependencies for each and every script they > write. Yes, because it is easy. > That may be true for development shops, but I don't expect that > casual users are so careful. Then they shouldn't be surprised when things go wrong for them. > But this discussion is moot anyway, the proposed metadata is meant to > help packaging tools to track which packages seem to be orphaned and for > which they can ask the user if those packages should be removed from the > system, instead of always removing seemingly orphaned packages. Indeed. Although I'd hope there's some --prune option or such which automatically removes all orphaned packages. cheers, Chris -- Simplistix - Content Management, Batch Processing & Python Consulting - http://www.simplistix.co.uk From chris at simplistix.co.uk Tue Oct 27 18:08:14 2009 From: chris at simplistix.co.uk (Chris Withers) Date: Tue, 27 Oct 2009 17:08:14 +0000 Subject: [Distutils] Distutils and Distribute roadmap (and some words on Virtualenv, Pip) In-Reply-To: <79990c6b0910201408ve7a10f5t76bb409cef57c955@mail.gmail.com> References: <94bdd2610910080131j323b98d9i871bce43465f237a@mail.gmail.com> <94bdd2610910081621n502803e4l8efeabae3c5863ff@mail.gmail.com> <79990c6b0910090532y63169b60q8dd0a95f056c244d@mail.gmail.com> <4ACF6993.5040704@voidspace.org.uk> <4ADDC6CA.5000809@simplistix.co.uk> <79990c6b0910201349w6f0537f2q1eb91bb4fd8d3c8d@mail.gmail.com> <79990c6b0910201408ve7a10f5t76bb409cef57c955@mail.gmail.com> Message-ID: <4AE728FE.6050000@simplistix.co.uk> [adding in disutils-sig and leaving this message intact as that's where it belongs] Paul Moore wrote: > 2009/10/20 Ian Bicking : >> FWIW, I don't think there's a real conflict here. My understanding is >> that wininst installers can be treated as installable packages that >> don't *have* to go through the system package manager, so they are >> both installable system-wide and installable much like an egg or >> tarball is (you can't drop the wininst installer into the path >> directly like you can an egg, but that doesn't seem to be a >> widely-held goal). > > The conflict lies in the fact that I want the features that > bdist_wininst installers give me (integration with add/remove programs > and uninstallation) [1]. If every project offers a bdist_wininst > installer, I'm happy. But Chris isn't unless they provide his > preferred format (I'm not sure if Chris has a concrete suggestion, or > he's just speculating at the moment) as well. Huh? I don't really care how packages are installed, provided they end up in a consistent format on disk. I'm not sure how a bdist_wininst installer is going to make sure a package's dependencies are met though... > I'm assuming (based on experience with eggs) that projects won't > routinely provide all possible formats. Hence my contention that a > single format is necessary. > > Having multiple package management systems that consume that format is > fine by me. But easy_install can consume bdist_wininst installers, so > the evidence is that there's more needed for a single format to emerge > than has been done so far (otherwise why isn't bdist_wininst that > single format right now?) Because not everyone runs on Windows? Chris -- Simplistix - Content Management, Batch Processing & Python Consulting - http://www.simplistix.co.uk From chris at simplistix.co.uk Tue Oct 27 18:10:55 2009 From: chris at simplistix.co.uk (Chris Withers) Date: Tue, 27 Oct 2009 17:10:55 +0000 Subject: [Distutils] Distutils and Distribute roadmap (and some words on Virtualenv, Pip) In-Reply-To: <79990c6b0910201502u745d4fb0ie3a24c140b2ba9e0@mail.gmail.com> References: <94bdd2610910080131j323b98d9i871bce43465f237a@mail.gmail.com> <79990c6b0910090532y63169b60q8dd0a95f056c244d@mail.gmail.com> <4ACF6993.5040704@voidspace.org.uk> <4ADDC6CA.5000809@simplistix.co.uk> <79990c6b0910201349w6f0537f2q1eb91bb4fd8d3c8d@mail.gmail.com> <79990c6b0910201408ve7a10f5t76bb409cef57c955@mail.gmail.com> <79990c6b0910201409s390cffc4n43cde28b5dc6843a@mail.gmail.com> <79990c6b0910201502u745d4fb0ie3a24c140b2ba9e0@mail.gmail.com> Message-ID: <4AE7299F.7030108@simplistix.co.uk> Paul Moore wrote: > 2009/10/20 Ian Bicking : >> Well, I wanted to move it off python-dev, and I didn't really feel >> like moving it onto distutils-sig because I felt the debate was not >> particularly important to introduce the confusion of a half-started >> thread. > > Sorry, I hadn't realised it was on python-dev rather than > distutils-sig. You're right in that case. > >> My point is the bdist_win installers work fine for both your use case, >> Paul, and Chris' use case, so there's no real conflict. > > OK, and yet people still seem to argue for eggs rather than > bdist_wininst, and projects provide eggs rather than bdist_wininst. > Something's not working here, even though logic says it should. Well, speaking only for myself, I never use bdist_wininst packages because they do no dependency management. If I release a package that has no external dependencies, I'll stick a bdist_msi or bdist_wininst if asked, but it's rarely asked for. From that I infer that most people use tools like buildout or pip to manage installation, even on Windows. Again, as a package consumer, I prefer one cross-platform package management method for Python packages, regardless of what operating system I happen to be working on at the time... Chris -- Simplistix - Content Management, Batch Processing & Python Consulting - http://www.simplistix.co.uk From p.f.moore at gmail.com Tue Oct 27 19:16:06 2009 From: p.f.moore at gmail.com (Paul Moore) Date: Tue, 27 Oct 2009 18:16:06 +0000 Subject: [Distutils] Distutils and Distribute roadmap (and some words on Virtualenv, Pip) In-Reply-To: <4AE7299F.7030108@simplistix.co.uk> References: <94bdd2610910080131j323b98d9i871bce43465f237a@mail.gmail.com> <4ACF6993.5040704@voidspace.org.uk> <4ADDC6CA.5000809@simplistix.co.uk> <79990c6b0910201349w6f0537f2q1eb91bb4fd8d3c8d@mail.gmail.com> <79990c6b0910201408ve7a10f5t76bb409cef57c955@mail.gmail.com> <79990c6b0910201409s390cffc4n43cde28b5dc6843a@mail.gmail.com> <79990c6b0910201502u745d4fb0ie3a24c140b2ba9e0@mail.gmail.com> <4AE7299F.7030108@simplistix.co.uk> Message-ID: <79990c6b0910271116i233387bft6624305b0524738f@mail.gmail.com> 2009/10/27 Chris Withers : >>> My point is the bdist_win installers work fine for both your use case, >>> Paul, and Chris' use case, so there's no real conflict. >> >> OK, and yet people still seem to argue for eggs rather than >> bdist_wininst, and projects provide eggs rather than bdist_wininst. >> Something's not working here, even though logic says it should. > > Well, speaking only for myself, I never use bdist_wininst packages because > they do no dependency management. Maybe someone has confused me. I understood that easy_install can consume bdist_wininst format .exe files just as easily as .egg format, and easy_install will do your dependency management for you. So, if you provide a bdist_wininst .exe format file: 1. People who want integration with add/remove just run the .exe and are happy. 2. People who want dependency management use easy_install which transparently uses the .exe. If you provide egg format files, only group (2) are satisfied. And if you provide both, you just make extra work for yourself. Obviously, this only applies to Windows. By all means provide eggs, rpms, debs, or whatever suits your users for other platforms. If this scenario is wrong, then I've seriously misunderstood the situation (for which I apologise, but I don't think it's just me). I can't recall where I got this understanding from, but I *think* it was from the easy_install documentation. No time right now to check this, sorry. > If I release a package that has no external dependencies, I'll stick a > bdist_msi or bdist_wininst if asked, but it's rarely asked for. My point is that you could stick up the bdist_wininst at no cost to your existing users, and some benefit to certain other users. I suspect, based on my own behaviour, that there's a proportion of people who won't ask for a bdist_wininst, but will just not bother with your package (depending on lots of other things, not least of which is the level of competition). > From that I infer that most people use tools like buildout or pip to manage > installation, even on Windows. Again, as a package consumer, I prefer one > cross-platform package management method for Python packages, regardless of > what operating system I happen to be working on at the time... This inference probably depends on your user base and the area you're working in. My impression is that buildout/pip are favoured among web developers, but not so much people writing console scripts or standalone applications. I don't think anyone has a clear picture of the overall situation. My focus is very, very narrow here, and I acknowledge that freely. All I care is that Windows users without the tools, time or expertise to build non-trivial C extensions have access to workable binary builds. Consider the following list of packages: wxPython, PyQT, PyGame, cx_Oracle, pywin32, PIL, lxml, numpy, pyCrypto I can't build these myself, and if the projects only provide eggs, I can't use them (unless I use easy_install or something similar, and accept the implications of that). I had to move one project away from mod_python because there's no binary installer for Python 2.6. That's where my interest lies. For pure-python packages or packages with C code with no external dependencies, I have no problem building my own bdist_wininst so I don't care, any more than someone who exclusively uses python setup.py install would care. It's when I rely on someone else building binaries for me that their choice of binary format impacts me. Paul. From fdrake at gmail.com Tue Oct 27 20:44:17 2009 From: fdrake at gmail.com (Fred Drake) Date: Tue, 27 Oct 2009 15:44:17 -0400 Subject: [Distutils] why would you ever need to specify setuptools as a dependency? In-Reply-To: <4AE71F38.8020703@simplistix.co.uk> References: <4ACF1956.5070903@simplistix.co.uk> <4ADD997C.3020808@cryptelium.net> <9cee7ab80910200431j579bbe4fke78aa93f8c28729d@mail.gmail.com> <4ADDBD74.3080004@simplistix.co.uk> <9cee7ab80910200643q1617b460pb7fd98007d52275a@mail.gmail.com> <4ADDBFCA.40902@simplistix.co.uk> <20091020201128.GA31378@clingman.lan> <4AE71327.7090101@simplistix.co.uk> <9cee7ab80910270851l5e953122v59651ce15a0039e8@mail.gmail.com> <4AE71F38.8020703@simplistix.co.uk> Message-ID: <9cee7ab80910271244i28e2f224k1ae2730a6178f1f9@mail.gmail.com> On Tue, Oct 27, 2009 at 12:26 PM, Chris Withers wrote: > You're ignoring the fact that for the information in install_requires to > even be parsed, you need to have setuptools present. install_requires is processed at install time. At that time, clearly, setuptools/distribute is present. That's not necessarily the case at run time. I see no reason to expect that having setuptools or distribute on sys.path at install time would cause it to necessarily be on the path at run time if my package doesn't provide information that it's needed at run time. > I'll say it again: it seems pointless specifying a requirement that has to > be met before any requirement specifications can even be processed... It doesn't sound to me like you've used buildout nearly as much as easy install. -Fred -- Fred L. Drake, Jr. "Chaos is the score upon which reality is written." --Henry Miller From ziade.tarek at gmail.com Tue Oct 27 20:51:10 2009 From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=) Date: Tue, 27 Oct 2009 20:51:10 +0100 Subject: [Distutils] Distutils and Distribute roadmap (and some words on Virtualenv, Pip) In-Reply-To: <79990c6b0910271116i233387bft6624305b0524738f@mail.gmail.com> References: <94bdd2610910080131j323b98d9i871bce43465f237a@mail.gmail.com> <4ADDC6CA.5000809@simplistix.co.uk> <79990c6b0910201349w6f0537f2q1eb91bb4fd8d3c8d@mail.gmail.com> <79990c6b0910201408ve7a10f5t76bb409cef57c955@mail.gmail.com> <79990c6b0910201409s390cffc4n43cde28b5dc6843a@mail.gmail.com> <79990c6b0910201502u745d4fb0ie3a24c140b2ba9e0@mail.gmail.com> <4AE7299F.7030108@simplistix.co.uk> <79990c6b0910271116i233387bft6624305b0524738f@mail.gmail.com> Message-ID: <94bdd2610910271251k29bac458q13223529cca64bbc@mail.gmail.com> On Tue, Oct 27, 2009 at 7:16 PM, Paul Moore wrote: > Maybe someone has confused me. I understood that easy_install can > consume bdist_wininst format .exe files just as easily as .egg format, > and easy_install will do your dependency management for you. That's exactly the case, you are not wrong. $ easy_install your_bdist_wininst_dist.exe will install it and process the dependencies from the install_requires option. And pip should be compatible soon too. That makes this format a perfect binary format for win32. Regards Tarek From david.lyon at preisshare.net Wed Oct 28 00:41:05 2009 From: david.lyon at preisshare.net (David Lyon) Date: Tue, 27 Oct 2009 19:41:05 -0400 Subject: [Distutils] Distutils and Distribute roadmap (and some words on Virtualenv, Pip) In-Reply-To: <94bdd2610910271251k29bac458q13223529cca64bbc@mail.gmail.com> References: <94bdd2610910080131j323b98d9i871bce43465f237a@mail.gmail.com> <4ADDC6CA.5000809@simplistix.co.uk> <79990c6b0910201349w6f0537f2q1eb91bb4fd8d3c8d@mail.gmail.com> <79990c6b0910201408ve7a10f5t76bb409cef57c955@mail.gmail.com> <79990c6b0910201409s390cffc4n43cde28b5dc6843a@mail.gmail.com> <79990c6b0910201502u745d4fb0ie3a24c140b2ba9e0@mail.gmail.com> <4AE7299F.7030108@simplistix.co.uk> <79990c6b0910271116i233387bft6624305b0524738f@mail.gmail.com> <94bdd2610910271251k29bac458q13223529cca64bbc@mail.gmail.com> Message-ID: <67266f4c439d9b80ce1144c2c08c3430@preisshare.net> On Tue, 27 Oct 2009 20:51:10 +0100, Tarek Ziad? wrote: > $ easy_install your_bdist_wininst_dist.exe > > will install it and process the dependencies from the install_requires > option. > > And pip should be compatible soon too. That makes this format a > perfect binary format for win32. I'm not sure about that Tarek.. An .exe installer as a perfect binary format for python packages? Are you serious? That is the biggest security threat I can think of, asking python users to run unverified, unsigned, un-trusted executable files on their systems. No Tarek, don't dictate that untrusted .exe files present no security threat to windows users. In this case, there *couldn't be* a bigger security threat than untrusted, unverified executables. David From robert.kern at gmail.com Wed Oct 28 00:51:07 2009 From: robert.kern at gmail.com (Robert Kern) Date: Tue, 27 Oct 2009 18:51:07 -0500 Subject: [Distutils] Distutils and Distribute roadmap (and some words on Virtualenv, Pip) In-Reply-To: <67266f4c439d9b80ce1144c2c08c3430@preisshare.net> References: <94bdd2610910080131j323b98d9i871bce43465f237a@mail.gmail.com> <4ADDC6CA.5000809@simplistix.co.uk> <79990c6b0910201349w6f0537f2q1eb91bb4fd8d3c8d@mail.gmail.com> <79990c6b0910201408ve7a10f5t76bb409cef57c955@mail.gmail.com> <79990c6b0910201409s390cffc4n43cde28b5dc6843a@mail.gmail.com> <79990c6b0910201502u745d4fb0ie3a24c140b2ba9e0@mail.gmail.com> <4AE7299F.7030108@simplistix.co.uk> <79990c6b0910271116i233387bft6624305b0524738f@mail.gmail.com> <94bdd2610910271251k29bac458q13223529cca64bbc@mail.gmail.com> <67266f4c439d9b80ce1144c2c08c3430@preisshare.net> Message-ID: On 2009-10-27 18:41 PM, David Lyon wrote: > On Tue, 27 Oct 2009 20:51:10 +0100, Tarek Ziad? > wrote: > >> $ easy_install your_bdist_wininst_dist.exe >> >> will install it and process the dependencies from the install_requires >> option. >> >> And pip should be compatible soon too. That makes this format a >> perfect binary format for win32. > > I'm not sure about that Tarek.. > > An .exe installer as a perfect binary format for python packages? > > Are you serious? > > That is the biggest security threat I can think of, asking python > users to run unverified, unsigned, un-trusted executable files on > their systems. easy_install does not execute the executable. bdist_wininst installers are zip files concatenated with an executable header. easy_install just unzips the file as if it were a zipped egg and ignores the executable part. -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco From glyph at twistedmatrix.com Wed Oct 28 01:06:11 2009 From: glyph at twistedmatrix.com (Glyph Lefkowitz) Date: Tue, 27 Oct 2009 20:06:11 -0400 Subject: [Distutils] Distutils and Distribute roadmap (and some words on Virtualenv, Pip) In-Reply-To: <67266f4c439d9b80ce1144c2c08c3430@preisshare.net> References: <94bdd2610910080131j323b98d9i871bce43465f237a@mail.gmail.com> <4ADDC6CA.5000809@simplistix.co.uk> <79990c6b0910201349w6f0537f2q1eb91bb4fd8d3c8d@mail.gmail.com> <79990c6b0910201408ve7a10f5t76bb409cef57c955@mail.gmail.com> <79990c6b0910201409s390cffc4n43cde28b5dc6843a@mail.gmail.com> <79990c6b0910201502u745d4fb0ie3a24c140b2ba9e0@mail.gmail.com> <4AE7299F.7030108@simplistix.co.uk> <79990c6b0910271116i233387bft6624305b0524738f@mail.gmail.com> <94bdd2610910271251k29bac458q13223529cca64bbc@mail.gmail.com> <67266f4c439d9b80ce1144c2c08c3430@preisshare.net> Message-ID: <555A99E7-3A6D-4624-BB8C-EAF89CA91A06@twistedmatrix.com> On Oct 27, 2009, at 7:41 PM, David Lyon wrote: > > I'm not sure about that Tarek.. > > An .exe installer as a perfect binary format for python packages? > > Are you serious? > > That is the biggest security threat I can think of, asking python > users to run unverified, unsigned, un-trusted executable files on > their systems. easy_install, pip, and indeed all of PyPI is basically a system for executing untrusted code, usually as a system administrator, straight off of what is effectively a wiki. If you're concerned about security and distutils, there is a _lot_ of work to do. There is no particular additional danger in executing a .exe rather than a setup.py. -------------- next part -------------- An HTML attachment was scrubbed... URL: From david.lyon at preisshare.net Wed Oct 28 01:12:29 2009 From: david.lyon at preisshare.net (David Lyon) Date: Tue, 27 Oct 2009 20:12:29 -0400 Subject: [Distutils] Distutils and Distribute roadmap (and some words on Virtualenv, Pip) In-Reply-To: References: <94bdd2610910080131j323b98d9i871bce43465f237a@mail.gmail.com> <4ADDC6CA.5000809@simplistix.co.uk> <79990c6b0910201349w6f0537f2q1eb91bb4fd8d3c8d@mail.gmail.com> <79990c6b0910201408ve7a10f5t76bb409cef57c955@mail.gmail.com> <79990c6b0910201409s390cffc4n43cde28b5dc6843a@mail.gmail.com> <79990c6b0910201502u745d4fb0ie3a24c140b2ba9e0@mail.gmail.com> <4AE7299F.7030108@simplistix.co.uk> <79990c6b0910271116i233387bft6624305b0524738f@mail.gmail.com> <94bdd2610910271251k29bac458q13223529cca64bbc@mail.gmail.com> <67266f4c439d9b80ce1144c2c08c3430@preisshare.net> Message-ID: <85bccf2b1417d8f7ec1eb72c3695b54e@preisshare.net> On Tue, 27 Oct 2009 18:51:07 -0500, Robert Kern wrote: > easy_install does not execute the executable. bdist_wininst installers are > zip > files concatenated with an executable header. easy_install just unzips the > file > as if it were a zipped egg and ignores the executable part. Ok - well thanks for the information and I learned something. A typical windows user, if they see an .exe installer, they would just install it by double-clicking on it. I don't know how many winodw users would automatically think to open a command shell and type the command line. In all seriousness, a bdist_wininst installer would be ok for installing a python application, but the support for applications in distutils for windows is incomplete. .egg files are perfect for windows. But the neccessary python support to make them work properly isn't complete either. What I am referring to here, is the shell extensions to the registry to make them load automatically. There isn't any inherent requirement in windows saying that setuptools need to be loaded for eggs to work. It would be so nice if we could double click on an egg file on pypi, download it, and install it automatically. I can't think of anything simpler. That would be proper native windows support. We would only have to add one script to distutils and a few registry settings for this could be accomplished. It could be progressively rolled out with new python versions. >From my perspective, PJE did a heck of a lot of good work. EGG files are a good idea. As long as they get unpacked and installed as any other python package on arrival. David From david.lyon at preisshare.net Wed Oct 28 01:14:50 2009 From: david.lyon at preisshare.net (David Lyon) Date: Tue, 27 Oct 2009 20:14:50 -0400 Subject: [Distutils] Distutils and Distribute roadmap (and some words on Virtualenv, Pip) In-Reply-To: <555A99E7-3A6D-4624-BB8C-EAF89CA91A06@twistedmatrix.com> References: <94bdd2610910080131j323b98d9i871bce43465f237a@mail.gmail.com> <4ADDC6CA.5000809@simplistix.co.uk> <79990c6b0910201349w6f0537f2q1eb91bb4fd8d3c8d@mail.gmail.com> <79990c6b0910201408ve7a10f5t76bb409cef57c955@mail.gmail.com> <79990c6b0910201409s390cffc4n43cde28b5dc6843a@mail.gmail.com> <79990c6b0910201502u745d4fb0ie3a24c140b2ba9e0@mail.gmail.com> <4AE7299F.7030108@simplistix.co.uk> <79990c6b0910271116i233387bft6624305b0524738f@mail.gmail.com> <94bdd2610910271251k29bac458q13223529cca64bbc@mail.gmail.com> <67266f4c439d9b80ce1144c2c08c3430@preisshare.net> <555A99E7-3A6D-4624-BB8C-EAF89CA91A06@twistedmatrix.com> Message-ID: On Tue, 27 Oct 2009 20:06:11 -0400, Glyph Lefkowitz > If you're concerned about security and distutils, there is a _lot_ of > work to do. There is no particular additional danger in executing > a .exe rather than a setup.py. Come to think of it.. you're totally right.. :-( David From david.lyon at preisshare.net Wed Oct 28 02:18:33 2009 From: david.lyon at preisshare.net (David Lyon) Date: Tue, 27 Oct 2009 21:18:33 -0400 Subject: [Distutils] Alternate static metadata PEP submission... In-Reply-To: <4AE72554.6060108@simplistix.co.uk> References: <20091013103021.GA21066@laurie.devork> <94bdd2610910131001v70711ffeia27433cd23b32fca@mail.gmail.com> <79a53e276e5dffe24be0433771801d72@preisshare.net> <94bdd2610910131426k64f707d7pf4d398d04b8ecc32@mail.gmail.com> <20091013233654.GA27339@laurie.devork> <4ADDB2CB.1090101@simplistix.co.uk> <8e9d1c135f56a0b37f12afe144b30f1a@preisshare.net> <4ADEB716.5020605@simplistix.co.uk> <4ADECB30.10605@ar.media.kyoto-u.ac.jp> <4AE72554.6060108@simplistix.co.uk> Message-ID: <9dcde5f5c75b1dcd6fbd361d5b259474@preisshare.net> On Tue, 27 Oct 2009 16:52:36 +0000, Chris Withers wrote: > from distutils.core import setup,parse_static > setup(**parse_static('setup.cfg').to_dict()) > > ...and of course, because setup.cfg is the default name, we just end up > with: > > from distutils.core import setup > setup() Nice.. But one still ends up with a user based setup.py After OPs comments about security I'm totally convinced that we need to ban setup.py, or at a very minimum move a standard setup.py into distutils itself. So one would say.. setup.py --install mypackage.egg or setup.py --install mypackage.tgz (from any directory) In simple cases, setup.py programs don't add any value. All the information required to do a package install is pretty much all to be found in the PKG_INFO file anyway. For typical packages, that is. So we could even wave the magic wand on: > from distutils.core import setup > setup() making it just '' (disappear) if we wanted.. We'd just have to add a script to distutils. I'm not sure how hard that is. David From david at ar.media.kyoto-u.ac.jp Wed Oct 28 04:26:19 2009 From: david at ar.media.kyoto-u.ac.jp (David Cournapeau) Date: Wed, 28 Oct 2009 12:26:19 +0900 Subject: [Distutils] Alternate static metadata PEP submission... In-Reply-To: <9dcde5f5c75b1dcd6fbd361d5b259474@preisshare.net> References: <20091013103021.GA21066@laurie.devork> <94bdd2610910131001v70711ffeia27433cd23b32fca@mail.gmail.com> <79a53e276e5dffe24be0433771801d72@preisshare.net> <94bdd2610910131426k64f707d7pf4d398d04b8ecc32@mail.gmail.com> <20091013233654.GA27339@laurie.devork> <4ADDB2CB.1090101@simplistix.co.uk> <8e9d1c135f56a0b37f12afe144b30f1a@preisshare.net> <4ADEB716.5020605@simplistix.co.uk> <4ADECB30.10605@ar.media.kyoto-u.ac.jp> <4AE72554.6060108@simplistix.co.uk> <9dcde5f5c75b1dcd6fbd361d5b259474@preisshare.net> Message-ID: <4AE7B9DB.8080100@ar.media.kyoto-u.ac.jp> David Lyon wrote: > On Tue, 27 Oct 2009 16:52:36 +0000, Chris Withers > wrote: > >> from distutils.core import setup,parse_static >> setup(**parse_static('setup.cfg').to_dict()) >> >> ...and of course, because setup.cfg is the default name, we just end up >> with: >> >> from distutils.core import setup >> setup() >> > > Nice.. > > But one still ends up with a user based setup.py > > After OPs comments about security I'm totally convinced that we need > to ban setup.py You can't ban setup.py: although a lot of packages can rely on a purely declarative package description, some package won't be able to do so (unless your declarative thing ends up being a full language). We should certainly encourage people to use declarative description, as this works for most packages out there, but it should be more flexible if desired. > or at a very minimum move a standard setup.py into > distutils itself. With toydist, there would be a new build system not based on distutils, and which would indeed works like this. Assuming you have a static file toydist.info, I would have something like: toydist configure --prefix=/home/david/local toydist build toydist install Or even, for a default build: toydist install (which would do configure and build automatically). But more importantly, note how people who want to use setup.py still can, how people who want to use another system can (through the python API around the static description). This gives a way forward to a system not tied to distutils legacy: this is very flexible, would keep all the feature given by setuptools/distribute, and would enable people like me who are convinced that distutils design is fundamentally flawed for heavy-weight build needs. David From david at ar.media.kyoto-u.ac.jp Wed Oct 28 04:52:47 2009 From: david at ar.media.kyoto-u.ac.jp (David Cournapeau) Date: Wed, 28 Oct 2009 12:52:47 +0900 Subject: [Distutils] Distutils and Distribute roadmap (and some words on Virtualenv, Pip) In-Reply-To: <4AE7299F.7030108@simplistix.co.uk> References: <94bdd2610910080131j323b98d9i871bce43465f237a@mail.gmail.com> <79990c6b0910090532y63169b60q8dd0a95f056c244d@mail.gmail.com> <4ACF6993.5040704@voidspace.org.uk> <4ADDC6CA.5000809@simplistix.co.uk> <79990c6b0910201349w6f0537f2q1eb91bb4fd8d3c8d@mail.gmail.com> <79990c6b0910201408ve7a10f5t76bb409cef57c955@mail.gmail.com> <79990c6b0910201409s390cffc4n43cde28b5dc6843a@mail.gmail.com> <79990c6b0910201502u745d4fb0ie3a24c140b2ba9e0@mail.gmail.com> <4AE7299F.7030108@simplistix.co.uk> Message-ID: <4AE7C00F.7020800@ar.media.kyoto-u.ac.jp> Chris Withers wrote: > > From that I infer that most people use tools like buildout or pip to > manage installation, even on Windows. I am skeptical about this claim because up to recently, virtualenv did not even work correctly for windows when you needed to build a C extension (bug which has been fixed since then), and setuptools from 2 weeks ago could not even produce workable entry points for scripts on 64 bits windows (fixed in last setuptools 0.6.9c11). In the corporate world, msi rules (because of group policy and all), and generally, wininst are much more similar to what people expect on windows. I think it is also important to keep in mind that not everybody use the same workflow - I myself build a lot of non trivial python softwares on many platforms, and the whole virtualenv/buildout/pip workflows is very foreign to me, and do not fit at all how I want to build and deploy things. Data-free generalization on how people do things should be avoided for those discussions, and we should focus on the requirements and use-cases. David From david.lyon at preisshare.net Wed Oct 28 05:20:21 2009 From: david.lyon at preisshare.net (David Lyon) Date: Wed, 28 Oct 2009 00:20:21 -0400 Subject: [Distutils] Alternate static metadata PEP submission... In-Reply-To: <4AE7B9DB.8080100@ar.media.kyoto-u.ac.jp> References: <20091013103021.GA21066@laurie.devork> <94bdd2610910131001v70711ffeia27433cd23b32fca@mail.gmail.com> <79a53e276e5dffe24be0433771801d72@preisshare.net> <94bdd2610910131426k64f707d7pf4d398d04b8ecc32@mail.gmail.com> <20091013233654.GA27339@laurie.devork> <4ADDB2CB.1090101@simplistix.co.uk> <8e9d1c135f56a0b37f12afe144b30f1a@preisshare.net> <4ADEB716.5020605@simplistix.co.uk> <4ADECB30.10605@ar.media.kyoto-u.ac.jp> <4AE72554.6060108@simplistix.co.uk> <9dcde5f5c75b1dcd6fbd361d5b259474@preisshare.net> <4AE7B9DB.8080100@ar.media.kyoto-u.ac.jp> Message-ID: <86349278b756f10760c291b71607a239@preisshare.net> On Wed, 28 Oct 2009 12:26:19 +0900, David Cournapeau > You can't ban setup.py: although ... haha - of course. ban setup.py as in ban alcohol or ban using a mobile phone when driving. > With toydist, there would be a new build system not based on distutils, > and which would indeed works like this. Assuming you have a static file > toydist.info, I would have something like: > > toydist configure --prefix=/home/david/local > toydist build > toydist install > > Or even, for a default build: > > toydist install (which would do configure and build automatically). I just want .EGG built into my OS as the registered file extension for a python package. Right-click install.. > and would enable people like me who are convinced that distutils > design is fundamentally flawed for heavy-weight build needs. What design? It's a collection of nice pieces of programming along with less nice pieces of programming along with nice pieces of programming that were nice once and now are only nice if viewed with a heavily artistic coloured glass and crystal operating system.. Nobody can bag the design of distutils. It's just a collection of 'stuff'. To me it needs a cleanout.. make things that nearly work.. work. Add some django interfaces or cherrypy interfaces or something so that we can build things 'in safety'. We only have to look at the postings by new users to see how hazardous it is. Once they've broken their legs... tore off their shoulders and blinded themselves in one eye, only then are they ready to get some work for themselves done.. Not saying it distutils doesn't work - only that going along with what you're saying in that it isn't a particularly good working environment. On top of that, we know we have all these dudes here which use these arty farty shiny glossy computers. So there shouldn't be any excuse to keep the old stuff going on forever without a revamp. There are things here that don't make sense.. David From fdrake at acm.org Wed Oct 28 13:38:49 2009 From: fdrake at acm.org (Fred Drake) Date: Wed, 28 Oct 2009 08:38:49 -0400 Subject: [Distutils] Alternate static metadata PEP submission... In-Reply-To: <4AE7B9DB.8080100@ar.media.kyoto-u.ac.jp> References: <4ADDB2CB.1090101@simplistix.co.uk> <8e9d1c135f56a0b37f12afe144b30f1a@preisshare.net> <4ADEB716.5020605@simplistix.co.uk> <4ADECB30.10605@ar.media.kyoto-u.ac.jp> <4AE72554.6060108@simplistix.co.uk> <9dcde5f5c75b1dcd6fbd361d5b259474@preisshare.net> <4AE7B9DB.8080100@ar.media.kyoto-u.ac.jp> Message-ID: <9cee7ab80910280538jc9d999dk97111787c48b1999@mail.gmail.com> On Tue, Oct 27, 2009 at 11:26 PM, David Cournapeau wrote: > You can't ban setup.py: although a lot of packages can rely on a purely > declarative package description, some package won't be able to do so > (unless your declarative thing ends up being a full language). Given the variety of special cases that have to be dealt with, we'll always have to a way to build packages that use Python code for what we're currently using setup.py for. For the many packages that don't need that level of flexibility, using a tool that consumes static metadata is a major win, so long as we keep in mind that tools need to be able to determine when that static metadata is sufficient. That allows the development of tools that can usefully get things done *and* refuse to run untrusted code themselves. -Fred -- Fred L. Drake, Jr. "Chaos is the score upon which reality is written." --Henry Miller From setuptools at bugs.python.org Wed Oct 28 14:37:15 2009 From: setuptools at bugs.python.org (=?utf-8?q?J=C3=BCrgen_Hermann?=) Date: Wed, 28 Oct 2009 13:37:15 +0000 Subject: [Distutils] [issue88] urllib2.splituser In-Reply-To: <1256737035.31.0.657216354648.issue88@psf.upfronthosting.co.za> Message-ID: <1256737035.31.0.657216354648.issue88@psf.upfronthosting.co.za> New submission from J?rgen Hermann : setuptools/package_index.py throws an AttributeError in Python 2.3 due to auth, host = urllib2.splituser(netloc) It either needs the following monkey patch, or simply use splituser from urllib. import urllib import urllib2 if not hasattr(urllib2, 'splituser'): # setuptools wants to import this from urllib2 but it's not # in there in Python 2.3.3, so we just alias it. urllib2.splituser = urllib.splituser ---------- messages: 441 nosy: jhe priority: bug status: unread title: urllib2.splituser _______________________________________________ Setuptools tracker _______________________________________________ From chris at simplistix.co.uk Wed Oct 28 15:02:06 2009 From: chris at simplistix.co.uk (Chris Withers) Date: Wed, 28 Oct 2009 14:02:06 +0000 Subject: [Distutils] Distutils and Distribute roadmap (and some words on Virtualenv, Pip) In-Reply-To: <79990c6b0910271116i233387bft6624305b0524738f@mail.gmail.com> References: <94bdd2610910080131j323b98d9i871bce43465f237a@mail.gmail.com> <4ACF6993.5040704@voidspace.org.uk> <4ADDC6CA.5000809@simplistix.co.uk> <79990c6b0910201349w6f0537f2q1eb91bb4fd8d3c8d@mail.gmail.com> <79990c6b0910201408ve7a10f5t76bb409cef57c955@mail.gmail.com> <79990c6b0910201409s390cffc4n43cde28b5dc6843a@mail.gmail.com> <79990c6b0910201502u745d4fb0ie3a24c140b2ba9e0@mail.gmail.com> <4AE7299F.7030108@simplistix.co.uk> <79990c6b0910271116i233387bft6624305b0524738f@mail.gmail.com> Message-ID: <4AE84EDE.3080808@simplistix.co.uk> Paul Moore wrote: > Maybe someone has confused me. I understood that easy_install can > consume bdist_wininst format .exe files just as easily as .egg format, > and easy_install will do your dependency management for you. > > So, if you provide a bdist_wininst .exe format file: > > 1. People who want integration with add/remove just run the .exe and are happy. But will people not be surprised that none of the dependencies are installed if they do this? > If you provide egg format files, only group (2) are satisfied. And if > you provide both, you just make extra work for yourself. It generally equates to typing a whole extra len('bdist_wininst') characters ;-) > suspect, based on my own behaviour, that there's a proportion of > people who won't ask for a bdist_wininst, but will just not bother > with your package (depending on lots of other things, not least of > which is the level of competition). They're welcome to go elsewhere if they can't be bothered to use what's available or ask for what they think they need. > wxPython, PyQT, PyGame, cx_Oracle, pywin32, PIL, lxml, numpy, pyCrypto > > I can't build these myself, and if the projects only provide eggs, I > can't use them (unless I use easy_install or something similar, and > accept the implications of that). ..or you just place the egg on your python path... Chris -- Simplistix - Content Management, Batch Processing & Python Consulting - http://www.simplistix.co.uk From chris at simplistix.co.uk Wed Oct 28 15:09:04 2009 From: chris at simplistix.co.uk (Chris Withers) Date: Wed, 28 Oct 2009 14:09:04 +0000 Subject: [Distutils] why would you ever need to specify setuptools as a dependency? In-Reply-To: <9cee7ab80910271244i28e2f224k1ae2730a6178f1f9@mail.gmail.com> References: <4ACF1956.5070903@simplistix.co.uk> <4ADD997C.3020808@cryptelium.net> <9cee7ab80910200431j579bbe4fke78aa93f8c28729d@mail.gmail.com> <4ADDBD74.3080004@simplistix.co.uk> <9cee7ab80910200643q1617b460pb7fd98007d52275a@mail.gmail.com> <4ADDBFCA.40902@simplistix.co.uk> <20091020201128.GA31378@clingman.lan> <4AE71327.7090101@simplistix.co.uk> <9cee7ab80910270851l5e953122v59651ce15a0039e8@mail.gmail.com> <4AE71F38.8020703@simplistix.co.uk> <9cee7ab80910271244i28e2f224k1ae2730a6178f1f9@mail.gmail.com> Message-ID: <4AE85080.3060603@simplistix.co.uk> Fred Drake wrote: > On Tue, Oct 27, 2009 at 12:26 PM, Chris Withers wrote: >> You're ignoring the fact that for the information in install_requires to >> even be parsed, you need to have setuptools present. > > install_requires is processed at install time. At that time, clearly, > setuptools/distribute is present. By "is", I'm sure you mean "needs to be". Specifying setuptools in install_requires is pointless chicken-and-egg'ing. But maybe I mean "setup_requires" thanks to setuptools "interesting" parameter naming ;-) > That's not necessarily the case at run time. I see no reason to > expect that having setuptools or distribute on sys.path at install > time would cause it to necessarily be on the path at run time if my > package doesn't provide information that it's needed at run time. Fair point. >> I'll say it again: it seems pointless specifying a requirement that has to >> be met before any requirement specifications can even be processed... > > It doesn't sound to me like you've used buildout nearly as much as easy install. Er? I *only* use buildout, however, I think I may have exempted myself by avoiding namespace packages until PEP 382 is done, dusted and in a python release. Chris -- Simplistix - Content Management, Batch Processing & Python Consulting - http://www.simplistix.co.uk From chris at simplistix.co.uk Wed Oct 28 15:18:12 2009 From: chris at simplistix.co.uk (Chris Withers) Date: Wed, 28 Oct 2009 14:18:12 +0000 Subject: [Distutils] Distutils and Distribute roadmap (and some words on Virtualenv, Pip) In-Reply-To: <4AE7C00F.7020800@ar.media.kyoto-u.ac.jp> References: <94bdd2610910080131j323b98d9i871bce43465f237a@mail.gmail.com> <79990c6b0910090532y63169b60q8dd0a95f056c244d@mail.gmail.com> <4ACF6993.5040704@voidspace.org.uk> <4ADDC6CA.5000809@simplistix.co.uk> <79990c6b0910201349w6f0537f2q1eb91bb4fd8d3c8d@mail.gmail.com> <79990c6b0910201408ve7a10f5t76bb409cef57c955@mail.gmail.com> <79990c6b0910201409s390cffc4n43cde28b5dc6843a@mail.gmail.com> <79990c6b0910201502u745d4fb0ie3a24c140b2ba9e0@mail.gmail.com> <4AE7299F.7030108@simplistix.co.uk> <4AE7C00F.7020800@ar.media.kyoto-u.ac.jp> Message-ID: <4AE852A4.2040708@simplistix.co.uk> David Cournapeau wrote: > Chris Withers wrote: >> From that I infer that most people use tools like buildout or pip to >> manage installation, even on Windows. > > I am skeptical about this claim because up to recently, virtualenv did > not even work correctly for windows when you needed to build a C > extension (bug which has been fixed since then), I don't use virtualenv and I didn't mention it for that reason. Buildout has behaved just fine in all my use of it... > and setuptools from 2 > weeks ago could not even produce workable entry points for scripts on 64 > bits windows (fixed in last setuptools 0.6.9c11). "even" seems to imply this is a basic requirement. I wonder how many people are rolling out python apps on Win64? I'm certainly not... > I think it is also important to keep in mind that not everybody use the > same workflow - I myself build a lot of non trivial python softwares on > many platforms, and the whole virtualenv/buildout/pip workflows is very > foreign to me, and do not fit at all how I want to build and deploy > things. How do you build and deploy things? Chris -- Simplistix - Content Management, Batch Processing & Python Consulting - http://www.simplistix.co.uk From chris at simplistix.co.uk Wed Oct 28 15:23:58 2009 From: chris at simplistix.co.uk (Chris Withers) Date: Wed, 28 Oct 2009 14:23:58 +0000 Subject: [Distutils] Alternate static metadata PEP submission... In-Reply-To: <9cee7ab80910280538jc9d999dk97111787c48b1999@mail.gmail.com> References: <4ADDB2CB.1090101@simplistix.co.uk> <8e9d1c135f56a0b37f12afe144b30f1a@preisshare.net> <4ADEB716.5020605@simplistix.co.uk> <4ADECB30.10605@ar.media.kyoto-u.ac.jp> <4AE72554.6060108@simplistix.co.uk> <9dcde5f5c75b1dcd6fbd361d5b259474@preisshare.net> <4AE7B9DB.8080100@ar.media.kyoto-u.ac.jp> <9cee7ab80910280538jc9d999dk97111787c48b1999@mail.gmail.com> Message-ID: <4AE853FE.5070602@simplistix.co.uk> Fred Drake wrote: > For the many packages that don't need that level of flexibility, using > a tool that consumes static metadata is a major win, so long as we > keep in mind that tools need to be able to determine when that static > metadata is sufficient. That allows the development of tools that can > usefully get things done *and* refuse to run untrusted code > themselves. I like the idea (I think I saw it in 390 or one of the other PEPs in this aread) that: setup() ...would get everyone from the static metadata file, while any keyword parameters to setup would override the appropriate setting from the static metadata file... cheers, Chris -- Simplistix - Content Management, Batch Processing & Python Consulting - http://www.simplistix.co.uk From p.f.moore at gmail.com Wed Oct 28 16:14:22 2009 From: p.f.moore at gmail.com (Paul Moore) Date: Wed, 28 Oct 2009 15:14:22 +0000 Subject: [Distutils] Distutils and Distribute roadmap (and some words on Virtualenv, Pip) In-Reply-To: <4AE84EDE.3080808@simplistix.co.uk> References: <94bdd2610910080131j323b98d9i871bce43465f237a@mail.gmail.com> <79990c6b0910201349w6f0537f2q1eb91bb4fd8d3c8d@mail.gmail.com> <79990c6b0910201408ve7a10f5t76bb409cef57c955@mail.gmail.com> <79990c6b0910201409s390cffc4n43cde28b5dc6843a@mail.gmail.com> <79990c6b0910201502u745d4fb0ie3a24c140b2ba9e0@mail.gmail.com> <4AE7299F.7030108@simplistix.co.uk> <79990c6b0910271116i233387bft6624305b0524738f@mail.gmail.com> <4AE84EDE.3080808@simplistix.co.uk> Message-ID: <79990c6b0910280814h9c299dfwc87e8473a3a1bd61@mail.gmail.com> 2009/10/28 Chris Withers : > Paul Moore wrote: >> >> Maybe someone has confused me. I understood that easy_install can >> consume bdist_wininst format .exe files just as easily as .egg format, >> and easy_install will do your dependency management for you. >> >> So, if you provide a bdist_wininst .exe format file: >> >> 1. People who want integration with add/remove just run the .exe and are >> happy. > > But will people not be surprised that none of the dependencies are installed > if they do this? No. That's the nature of Windows installers. I accept that your view is that automatic installation of dependencies is a key feature - please accept in turn that I prefer to manage dependencies manually. >> If you provide egg format files, only group (2) are satisfied. And if >> you provide both, you just make extra work for yourself. > > It generally equates to typing a whole extra len('bdist_wininst') characters > ;-) Hmm, OK, fair point :-) It's a pity more people don't just do it then... >> suspect, based on my own behaviour, that there's a proportion of >> people who won't ask for a bdist_wininst, but will just not bother >> with your package (depending on lots of other things, not least of >> which is the level of competition). > > They're welcome to go elsewhere if they can't be bothered to use what's > available or ask for what they think they need. True. Certainly in the case of "ask for what they think they need". I'm less convinced that "use what's available" is always appropriate (see below). >> wxPython, PyQT, PyGame, cx_Oracle, pywin32, PIL, lxml, numpy, pyCrypto >> >> I can't build these myself, and if the projects only provide eggs, I >> can't use them (unless I use easy_install or something similar, and >> accept the implications of that). > > ..or you just place the egg on your python path... My personal issue with this is that I lose the one listing of what I have installed from add/remove programs, and I lose the one-click uninstall of add/remove. This adds to my management overhead for my Python installation. Lots of this is personal preference. No argument there. Paul. From fdrake at acm.org Wed Oct 28 16:16:24 2009 From: fdrake at acm.org (Fred Drake) Date: Wed, 28 Oct 2009 11:16:24 -0400 Subject: [Distutils] Alternate static metadata PEP submission... In-Reply-To: <4AE853FE.5070602@simplistix.co.uk> References: <4ADDB2CB.1090101@simplistix.co.uk> <8e9d1c135f56a0b37f12afe144b30f1a@preisshare.net> <4ADEB716.5020605@simplistix.co.uk> <4ADECB30.10605@ar.media.kyoto-u.ac.jp> <4AE72554.6060108@simplistix.co.uk> <9dcde5f5c75b1dcd6fbd361d5b259474@preisshare.net> <4AE7B9DB.8080100@ar.media.kyoto-u.ac.jp> <9cee7ab80910280538jc9d999dk97111787c48b1999@mail.gmail.com> <4AE853FE.5070602@simplistix.co.uk> Message-ID: <9cee7ab80910280816y1b20ca14q8c03ea7ba886ee25@mail.gmail.com> On Wed, Oct 28, 2009 at 10:23 AM, Chris Withers wrote: > setup() > > ...would get everyone from the static metadata file, while any keyword > parameters to setup would override the appropriate setting from the static > metadata file... I'd rather see the static data in setup.cfg and no setup.py if there's no need for custom Python code. There are too many ways to write from distutils.core import setup setup() to make it useful as a way to say the static metadata is sufficient. That aside, being able to use an "empty" setup call like that is a good first step. -Fred -- Fred L. Drake, Jr. "Chaos is the score upon which reality is written." --Henry Miller From fdrake at gmail.com Wed Oct 28 19:19:28 2009 From: fdrake at gmail.com (Fred Drake) Date: Wed, 28 Oct 2009 14:19:28 -0400 Subject: [Distutils] why would you ever need to specify setuptools as a dependency? In-Reply-To: <4AE85080.3060603@simplistix.co.uk> References: <4ACF1956.5070903@simplistix.co.uk> <4ADDBD74.3080004@simplistix.co.uk> <9cee7ab80910200643q1617b460pb7fd98007d52275a@mail.gmail.com> <4ADDBFCA.40902@simplistix.co.uk> <20091020201128.GA31378@clingman.lan> <4AE71327.7090101@simplistix.co.uk> <9cee7ab80910270851l5e953122v59651ce15a0039e8@mail.gmail.com> <4AE71F38.8020703@simplistix.co.uk> <9cee7ab80910271244i28e2f224k1ae2730a6178f1f9@mail.gmail.com> <4AE85080.3060603@simplistix.co.uk> Message-ID: <9cee7ab80910281119v31315e36rc910a3b8156c03a8@mail.gmail.com> On Wed, Oct 28, 2009 at 10:09 AM, Chris Withers wrote: > By "is", I'm sure you mean "needs to be". Specifying setuptools in > install_requires is pointless chicken-and-egg'ing. But maybe I mean > "setup_requires" thanks to setuptools "interesting" parameter naming ;-) You just might. install_requires is for what's needed at run time, which is what most of us mean when we refer to install_requires by name. > Er? I *only* use buildout, however, I think I may have exempted myself by > avoiding namespace packages until PEP 382 is done, dusted and in a python > release. Your use of buildout is obvious in many contexts, but do realize why I (and likely others) wondered at your insistence that mentioning setuptools there was meaningless. Clarifying this hasn't been cheap (though valuable, IMO). -Fred -- Fred L. Drake, Jr. "Chaos is the score upon which reality is written." --Henry Miller From david.lyon at preisshare.net Thu Oct 29 01:25:01 2009 From: david.lyon at preisshare.net (David Lyon) Date: Wed, 28 Oct 2009 20:25:01 -0400 Subject: [Distutils] Alternate static metadata PEP submission... In-Reply-To: <4AE853FE.5070602@simplistix.co.uk> References: <4ADDB2CB.1090101@simplistix.co.uk> <8e9d1c135f56a0b37f12afe144b30f1a@preisshare.net> <4ADEB716.5020605@simplistix.co.uk> <4ADECB30.10605@ar.media.kyoto-u.ac.jp> <4AE72554.6060108@simplistix.co.uk> <9dcde5f5c75b1dcd6fbd361d5b259474@preisshare.net> <4AE7B9DB.8080100@ar.media.kyoto-u.ac.jp> <9cee7ab80910280538jc9d999dk97111787c48b1999@mail.gmail.com> <4AE853FE.5070602@simplistix.co.uk> Message-ID: <0740dcc56bb1c2e1f3d11399a85f9136@preisshare.net> On Wed, 28 Oct 2009 14:23:58 +0000, Chris Withers wrote: > ...would get everyone from the static metadata file, while any keyword > parameters to setup would override the appropriate setting from the > static metadata file... Hi Chris, David, Fred, Floris, Tarek, PJE, ... Everybody has been so helpful and I feel that I've now run out of things to discuss. I would like to invite everybody to make some sort of contribution to this, and in return we'll just keep your name as authors of the various bits/files. What I am trying to accomplish is a shell based install system for .EGG/Python packages, along with metadata driven installation. TSo basically getting all the good ideas that *everybody* has had into a cohesive (and real) collection. I'm calling for volounteers/co-owners to: - contribute utilities (say, convert existing setup.py files) - specify/design/code test cases (different platforms etc) - write sphinx documentation (rst) - create examples - critique The url is: https://bitbucket.org/djlyon/original-setup.py-with-metadata/ If we can complete it and have a working system by December, which I believe we can, we can find a new name and have a democratic vote on what that might be. Thank you everybody David From david at ar.media.kyoto-u.ac.jp Thu Oct 29 04:32:34 2009 From: david at ar.media.kyoto-u.ac.jp (David Cournapeau) Date: Thu, 29 Oct 2009 12:32:34 +0900 Subject: [Distutils] Alternate static metadata PEP submission... In-Reply-To: <9cee7ab80910280538jc9d999dk97111787c48b1999@mail.gmail.com> References: <4ADDB2CB.1090101@simplistix.co.uk> <8e9d1c135f56a0b37f12afe144b30f1a@preisshare.net> <4ADEB716.5020605@simplistix.co.uk> <4ADECB30.10605@ar.media.kyoto-u.ac.jp> <4AE72554.6060108@simplistix.co.uk> <9dcde5f5c75b1dcd6fbd361d5b259474@preisshare.net> <4AE7B9DB.8080100@ar.media.kyoto-u.ac.jp> <9cee7ab80910280538jc9d999dk97111787c48b1999@mail.gmail.com> Message-ID: <4AE90CD2.6010301@ar.media.kyoto-u.ac.jp> Fred Drake wrote: > On Tue, Oct 27, 2009 at 11:26 PM, David Cournapeau > wrote: > >> You can't ban setup.py: although a lot of packages can rely on a purely >> declarative package description, some package won't be able to do so >> (unless your declarative thing ends up being a full language). >> > > Given the variety of special cases that have to be dealt with, we'll > always have to a way to build packages that use Python code for what > we're currently using setup.py for. > > For the many packages that don't need that level of flexibility, using > a tool that consumes static metadata is a major win, so long as we > keep in mind that tools need to be able to determine when that static > metadata is sufficient. That allows the development of tools that can > usefully get things done *and* refuse to run untrusted code > themselves. > Maybe my English was lacking in my previous email, but that just described one of the main goal of something like toydist: fully describe in a declarative manner a large subset of python 'packages', and making it possible to interoperate/extend with other tools (for now, only distutils itself in toydist toy implementation, but I am more interested in interoperating with waf/scons in the short term, and other people would be more interested in buildout, paver, etc... depending on their needs). David From david at ar.media.kyoto-u.ac.jp Thu Oct 29 04:55:14 2009 From: david at ar.media.kyoto-u.ac.jp (David Cournapeau) Date: Thu, 29 Oct 2009 12:55:14 +0900 Subject: [Distutils] Distutils and Distribute roadmap (and some words on Virtualenv, Pip) In-Reply-To: <4AE852A4.2040708@simplistix.co.uk> References: <94bdd2610910080131j323b98d9i871bce43465f237a@mail.gmail.com> <79990c6b0910090532y63169b60q8dd0a95f056c244d@mail.gmail.com> <4ACF6993.5040704@voidspace.org.uk> <4ADDC6CA.5000809@simplistix.co.uk> <79990c6b0910201349w6f0537f2q1eb91bb4fd8d3c8d@mail.gmail.com> <79990c6b0910201408ve7a10f5t76bb409cef57c955@mail.gmail.com> <79990c6b0910201409s390cffc4n43cde28b5dc6843a@mail.gmail.com> <79990c6b0910201502u745d4fb0ie3a24c140b2ba9e0@mail.gmail.com> <4AE7299F.7030108@simplistix.co.uk> <4AE7C00F.7020800@ar.media.kyoto-u.ac.jp> <4AE852A4.2040708@simplistix.co.uk> Message-ID: <4AE91222.4060800@ar.media.kyoto-u.ac.jp> Chris Withers wrote: > "even" seems to imply this is a basic requirement. I wonder how many > people are rolling out python apps on Win64? I'm certainly not... Even was maybe a bit strong. Concerning win64, the first release of numpy we did for win64 has been downloaded 3000 times, even though it is experimental and not working. Obviously, this is linked to 'our' community: in scientific software, 64 bits arch has mattered for a long time, and a lot of people use windows, for better or worse. In numpy/scipy, even though the vast majority of dev are linux/mac os x users, windows installers are by far the most downloaded (the numbers are obviously skewed since many people can build from svn on linux/mac os x, whereas very few people will do so on windows). > >> I think it is also important to keep in mind that not everybody use the >> same workflow - I myself build a lot of non trivial python softwares on >> many platforms, and the whole virtualenv/buildout/pip workflows is very >> foreign to me, and do not fit at all how I want to build and deploy >> things. > > How do you build and deploy things? To answer your question: I use distutils, scons, and native OS package tools. Sandboxing at build time is done through chrooting (and some hacked wine setup for windows). I think the main issue is that we don't mean the same thing by "deploy things". For web developers, it seems that deploying mean repeat an exact environment with all its dependencies on a similar machine (or set of machines), often by the same person (or at least a technically competent person). In that scenario, if there is a problem (wrong pythonpath, double entry from an egg, etc...), you can easily fix it, and being able to use latest version of some libraries + developer convenience is the key. By deploying, I mean being able to give a single file which people can install by themselves, people who are often ignorant about PYTHONPATH, import mechanisms, what a .pth file is. In that scenario, altering the environment in a non workable state is a total failure. Developer convenience is a problem in many cases (concrete case: using the VCS to get which files to put in a distribution). This article is about ruby, but totally nails it IMHO: http://www.madstop.com/ruby/ruby_has_a_distribution_problem.html Given that web dev are the ones doing the work, it is natural and certainly legitimate that current distutils-related work is focused on this workflow. But this workflow is not always applicable to more 'traditional' deployments scenario, and it often does make life more difficult. It certainly did for me. David From a.cavallo at cavallinux.eu Thu Oct 29 08:44:03 2009 From: a.cavallo at cavallinux.eu (A. Cavallo) Date: Thu, 29 Oct 2009 07:44:03 +0000 Subject: [Distutils] Distutils and Distribute roadmap (and some words on Virtualenv, Pip) In-Reply-To: <85bccf2b1417d8f7ec1eb72c3695b54e@preisshare.net> References: <94bdd2610910080131j323b98d9i871bce43465f237a@mail.gmail.com> <85bccf2b1417d8f7ec1eb72c3695b54e@preisshare.net> Message-ID: <200910290744.04310.a.cavallo@cavallinux.eu> > A typical windows user, if they see an .exe installer, they would just > install it by double-clicking on it. I don't know how many winodw users > would automatically think to open a command shell and type the command > line. None in fact and it is a make or break requirement in so many companies that if anyone is thinking of dropping bdist_wininst should rethink of it. If it goes away I have to discourage use of python because in no case a language specific way to install things will ever replace the platform standardised way. I'll give you an anecdotal example: I worked in a company developing systems for an extremely larger one. Our software needn't any installer: just unzip the thing and start to using it. Simple? You bet it cannot be any easier than that..... One of the contractual requirements was about providing an "installer" for windows: no matter how easy it was to deploy the thing. This makes sense because their way to manage systems has been standardised in that way (the windows way, like it or hate it): no matter how clever our system was, it was extremely expensive to them doing it in "our" way. > In all seriousness, a bdist_wininst installer would be ok for installing > a python application, but the support for applications in distutils for > windows is incomplete. It is just enough to allow people to work: dependencies are handled manually and this goes in a project requirement document. An exception on a missing module it is just fine to give a hint about what is missing and trigger a user action. > .egg files are perfect for windows. But the neccessary python support > to make them work properly isn't complete either. > What I am referring to here, is the shell extensions to the registry > to make them load automatically. > It would be so nice if we could double click on an egg file on pypi, > download it, and install it automatically. > I can't think of anything simpler. That would be proper native windows > support. I disagree here and reinventing a package/dependencies manager to compete with cpan and rpm/dpkg/yum/zypper/synaptic/windows installers ... thanks no I don't feel any need for it. Ever heard of SuSE one click installer? It does exactly this for rpms. In windows clicking on a web link with an *.exe trigger the action you're just asking for already: no need to support another language-specific layer. > We would only have to add one script to distutils and a few registry > settings for this could be accomplished. It could be progressively > rolled out with new python versions. Sure and imagine a company stretching across three continents with hundred developers and many more users and slightly different settings/python versions: do you think anything like this will ever work in real life? How does anybody going to handle the chain of trust (yes sooner or later somebody will ask THIS question)? Again my solution will be easy: drop python off the list of language to develop with. > >From my perspective, PJE did a heck of a lot of good work. EGG files > > are a good idea. As long as they get unpacked and installed as any > other python package on arrival. Excellent last point aiming straight to the problem: I cannot agree more. Regards, Antonio From david.lyon at preisshare.net Fri Oct 30 00:15:23 2009 From: david.lyon at preisshare.net (David Lyon) Date: Thu, 29 Oct 2009 19:15:23 -0400 Subject: [Distutils] Distutils and Distribute roadmap (and some words on Virtualenv, Pip) In-Reply-To: <200910290744.04310.a.cavallo@cavallinux.eu> References: <94bdd2610910080131j323b98d9i871bce43465f237a@mail.gmail.com> <85bccf2b1417d8f7ec1eb72c3695b54e@preisshare.net> <200910290744.04310.a.cavallo@cavallinux.eu> Message-ID: <385dff1df301549ea1fda1f421098a7b@preisshare.net> Antonio, I completely get what you're saying. Anyway, we shouldn't be considering dropping bdist_wininst installers for windows unless we actually have something viable to replace them with. As much as I don't think they are the ideal solution, I'd hate to think about having to to package installation on windows without them. So from a pure practical point.. I totally agree with what you're saying.. David On Thu, 29 Oct 2009 07:44:03 +0000, "A. Cavallo" wrote: > None in fact and it is a make or break requirement in so many companies > that > if anyone is thinking of dropping bdist_wininst should rethink of it. > > If it goes away I have to discourage use of python because in no case a > language specific way to install things will ever replace the platform > standardised way. > > I'll give you an anecdotal example: I worked in a company developing > systems > for an extremely larger one. Our software needn't any installer: just unzip > > the thing and start to using it. Simple? You bet it cannot be any easier > than > that..... > One of the contractual requirements was about providing an "installer" for > windows: no matter how easy it was to deploy the thing. > > This makes sense because their way to manage systems has been standardised > in > that way (the windows way, like it or hate it): no matter how clever our > system was, it was extremely expensive to them doing it in "our" way. > > >> In all seriousness, a bdist_wininst installer would be ok for installing >> a python application, but the support for applications in distutils for >> windows is incomplete. > > It is just enough to allow people to work: dependencies are handled > manually > and this goes in a project requirement document. An exception on a missing > module it is just fine to give a hint about what is missing and trigger a > user > action. > > >> .egg files are perfect for windows. But the neccessary python support >> to make them work properly isn't complete either. >> What I am referring to here, is the shell extensions to the registry >> to make them load automatically. >> It would be so nice if we could double click on an egg file on pypi, >> download it, and install it automatically. >> I can't think of anything simpler. That would be proper native windows >> support. > > I disagree here and reinventing a package/dependencies manager to compete > with > cpan and rpm/dpkg/yum/zypper/synaptic/windows installers ... thanks no I > don't > feel any need for it. > > Ever heard of SuSE one click installer? It does exactly this for rpms. > In windows clicking on a web link with an *.exe trigger the action you're > just > asking for already: no need to support another language-specific layer. > > >> We would only have to add one script to distutils and a few registry >> settings for this could be accomplished. It could be progressively >> rolled out with new python versions. > > Sure and imagine a company stretching across three continents with hundred > developers and many more users and slightly different settings/python > versions: do you think anything like this will ever work in real life? How > does anybody going to handle the chain of trust (yes sooner or later > somebody > will ask THIS question)? > Again my solution will be easy: drop python off the list of language to > develop with. > > >> >From my perspective, PJE did a heck of a lot of good work. EGG files >> >> are a good idea. As long as they get unpacked and installed as any >> other python package on arrival. > > Excellent last point aiming straight to the problem: I cannot agree more. > > Regards, > Antonio > _______________________________________________ > Distutils-SIG maillist - Distutils-SIG at python.org > http://mail.python.org/mailman/listinfo/distutils-sig From ziade.tarek at gmail.com Fri Oct 30 13:41:09 2009 From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=) Date: Fri, 30 Oct 2009 13:41:09 +0100 Subject: [Distutils] Distribute sprint #2, next sunday Message-ID: <94bdd2610910300541w7f772962rcf620d88140df8d3@mail.gmail.com> Hey, We will have a sprint sunday afternoon, starting at 3 pm in Berlin/Germany. If you are interested you can join online in the irc (#distutils on Freenode) or in Berlin of course. This sprint will continue the work we've started on the 0.7 branch. If you have worked during the first sprint and you can't make it sunday, please make sure your worked was commited in the Distribute repository with notes so we can continue. Cheers Tarek -- Tarek Ziad? | http://ziade.org |????????????! |???????????? From chris at simplistix.co.uk Fri Oct 30 23:24:50 2009 From: chris at simplistix.co.uk (Chris Withers) Date: Fri, 30 Oct 2009 22:24:50 +0000 Subject: [Distutils] Alternate static metadata PEP submission... In-Reply-To: <9cee7ab80910280816y1b20ca14q8c03ea7ba886ee25@mail.gmail.com> References: <4ADDB2CB.1090101@simplistix.co.uk> <8e9d1c135f56a0b37f12afe144b30f1a@preisshare.net> <4ADEB716.5020605@simplistix.co.uk> <4ADECB30.10605@ar.media.kyoto-u.ac.jp> <4AE72554.6060108@simplistix.co.uk> <9dcde5f5c75b1dcd6fbd361d5b259474@preisshare.net> <4AE7B9DB.8080100@ar.media.kyoto-u.ac.jp> <9cee7ab80910280538jc9d999dk97111787c48b1999@mail.gmail.com> <4AE853FE.5070602@simplistix.co.uk> <9cee7ab80910280816y1b20ca14q8c03ea7ba886ee25@mail.gmail.com> Message-ID: <4AEB67B2.2020605@simplistix.co.uk> Fred Drake wrote: > On Wed, Oct 28, 2009 at 10:23 AM, Chris Withers wrote: >> setup() >> >> ...would get everyone from the static metadata file, while any keyword >> parameters to setup would override the appropriate setting from the static >> metadata file... > > I'd rather see the static data in setup.cfg and no setup.py if there's > no need for custom Python code. I think "python setup.py install" is so idiomatic that it seems silly to break it for the sake of two lines of python. There's also an element of "explicit is better than implicit" in the feel of actually running setup.py... > There are too many ways to write > > from distutils.core import setup > setup() > > to make it useful as a way to say the static metadata is sufficient. I genuinely don't know what you mean :-S Chris -- Simplistix - Content Management, Batch Processing & Python Consulting - http://www.simplistix.co.uk From chris at simplistix.co.uk Fri Oct 30 23:33:47 2009 From: chris at simplistix.co.uk (Chris Withers) Date: Fri, 30 Oct 2009 22:33:47 +0000 Subject: [Distutils] Distutils and Distribute roadmap (and some words on Virtualenv, Pip) In-Reply-To: <4AE91222.4060800@ar.media.kyoto-u.ac.jp> References: <94bdd2610910080131j323b98d9i871bce43465f237a@mail.gmail.com> <79990c6b0910090532y63169b60q8dd0a95f056c244d@mail.gmail.com> <4ACF6993.5040704@voidspace.org.uk> <4ADDC6CA.5000809@simplistix.co.uk> <79990c6b0910201349w6f0537f2q1eb91bb4fd8d3c8d@mail.gmail.com> <79990c6b0910201408ve7a10f5t76bb409cef57c955@mail.gmail.com> <79990c6b0910201409s390cffc4n43cde28b5dc6843a@mail.gmail.com> <79990c6b0910201502u745d4fb0ie3a24c140b2ba9e0@mail.gmail.com> <4AE7299F.7030108@simplistix.co.uk> <4AE7C00F.7020800@ar.media.kyoto-u.ac.jp> <4AE852A4.2040708@simplistix.co.uk> <4AE91222.4060800@ar.media.kyoto-u.ac.jp> Message-ID: <4AEB69CB.20700@simplistix.co.uk> David Cournapeau wrote: > Chris Withers wrote: >> "even" seems to imply this is a basic requirement. I wonder how many >> people are rolling out python apps on Win64? I'm certainly not... > > Even was maybe a bit strong. Concerning win64, the first release of > numpy we did for win64 has been downloaded 3000 times, By what? I'm always sceptical about the download figures on PyPI, and other places. I wonder how many bots are sucking down every .exe or .msi they can find to check for viruses or the like? >>> I think it is also important to keep in mind that not everybody use the >>> same workflow - I myself build a lot of non trivial python softwares on >>> many platforms, and the whole virtualenv/buildout/pip workflows is very >>> foreign to me, and do not fit at all how I want to build and deploy >>> things. >> How do you build and deploy things? > > To answer your question: I use distutils, scons, and native OS package > tools. Sandboxing at build time is done through chrooting (and some > hacked wine setup for windows). You should probably look at buildout, you may find your like getting a lot easier. Yes, the docs suck. I even promised to do something about that back at PyCon, sadly time has not been kind to my promises... > I think the main issue is that we don't mean the same thing by "deploy > things". For web developers, it seems that deploying mean repeat an > exact environment with all its dependencies on a similar machine (or set > of machines), often by the same person (or at least a technically > competent person). In that scenario, if there is a problem (wrong > pythonpath, double entry from an egg, etc...), you can easily fix it, > and being able to use latest version of some libraries + developer > convenience is the key. > > By deploying, I mean being able to give a single file which people can > install by themselves, people who are often ignorant about PYTHONPATH, > import mechanisms, what a .pth file is. Well, buildout.cfg is exacltly that file for me, for both cases ;-) Chris - for the record: .pth files are evil and should be punished... -- Simplistix - Content Management, Batch Processing & Python Consulting - http://www.simplistix.co.uk From eric at trueblade.com Fri Oct 30 23:43:29 2009 From: eric at trueblade.com (Eric Smith) Date: Fri, 30 Oct 2009 18:43:29 -0400 Subject: [Distutils] Alternate static metadata PEP submission... In-Reply-To: <4AEB67B2.2020605@simplistix.co.uk> References: <4ADDB2CB.1090101@simplistix.co.uk> <8e9d1c135f56a0b37f12afe144b30f1a@preisshare.net> <4ADEB716.5020605@simplistix.co.uk> <4ADECB30.10605@ar.media.kyoto-u.ac.jp> <4AE72554.6060108@simplistix.co.uk> <9dcde5f5c75b1dcd6fbd361d5b259474@preisshare.net> <4AE7B9DB.8080100@ar.media.kyoto-u.ac.jp> <9cee7ab80910280538jc9d999dk97111787c48b1999@mail.gmail.com> <4AE853FE.5070602@simplistix.co.uk> <9cee7ab80910280816y1b20ca14q8c03ea7ba886ee25@mail.gmail.com> <4AEB67B2.2020605@simplistix.co.uk> Message-ID: <4AEB6C11.9070008@trueblade.com> Chris Withers wrote: > Fred Drake wrote: >> I'd rather see the static data in setup.cfg and no setup.py if there's >> no need for custom Python code. Agreed. > I think "python setup.py install" is so idiomatic that it seems silly to > break it for the sake of two lines of python. There's also an element of > "explicit is better than implicit" in the feel of actually running > setup.py... Of course. But you don't know that there are 2 lines of Python, and you don't know what they do. The whole point of static metadata is that you can inspect a distribution without executing arbitrary, unknown code. Eric. From fdrake at acm.org Sat Oct 31 02:47:17 2009 From: fdrake at acm.org (Fred Drake) Date: Fri, 30 Oct 2009 21:47:17 -0400 Subject: [Distutils] Alternate static metadata PEP submission... In-Reply-To: <4AEB67B2.2020605@simplistix.co.uk> References: <4ADEB716.5020605@simplistix.co.uk> <4ADECB30.10605@ar.media.kyoto-u.ac.jp> <4AE72554.6060108@simplistix.co.uk> <9dcde5f5c75b1dcd6fbd361d5b259474@preisshare.net> <4AE7B9DB.8080100@ar.media.kyoto-u.ac.jp> <9cee7ab80910280538jc9d999dk97111787c48b1999@mail.gmail.com> <4AE853FE.5070602@simplistix.co.uk> <9cee7ab80910280816y1b20ca14q8c03ea7ba886ee25@mail.gmail.com> <4AEB67B2.2020605@simplistix.co.uk> Message-ID: <9cee7ab80910301847s749aa574r892d4281f6130330@mail.gmail.com> On Fri, Oct 30, 2009 at 6:24 PM, Chris Withers wrote: > I think "python setup.py install" is so idiomatic that it seems silly to > break it for the sake of two lines of python. Idiomatic, yes. But it's an idiom that developed as a side effect of the implementation. This may well have been a reasonable choice at the time it was made over a decade ago (and yes, I was there, discussing distutils with Greg Ward during the birthing, so you can probably even lay some of the blame at my feet). That would reflect the youth of Python packaging more than anything else. > There's also an element of "explicit is better than implicit" > in the feel of actually running setup.py... Running a known implementation isn't explicit? I worry more about having to review someone else's ad-hoc setup.py, usually under deadline pressure, especially given how many variations there are. Both are explicit; one is less risky. -Fred -- Fred L. Drake, Jr. "Chaos is the score upon which reality is written." --Henry Miller From david at ar.media.kyoto-u.ac.jp Sat Oct 31 07:04:28 2009 From: david at ar.media.kyoto-u.ac.jp (David Cournapeau) Date: Sat, 31 Oct 2009 15:04:28 +0900 Subject: [Distutils] Alternate static metadata PEP submission... In-Reply-To: <4AEB67B2.2020605@simplistix.co.uk> References: <4ADDB2CB.1090101@simplistix.co.uk> <8e9d1c135f56a0b37f12afe144b30f1a@preisshare.net> <4ADEB716.5020605@simplistix.co.uk> <4ADECB30.10605@ar.media.kyoto-u.ac.jp> <4AE72554.6060108@simplistix.co.uk> <9dcde5f5c75b1dcd6fbd361d5b259474@preisshare.net> <4AE7B9DB.8080100@ar.media.kyoto-u.ac.jp> <9cee7ab80910280538jc9d999dk97111787c48b1999@mail.gmail.com> <4AE853FE.5070602@simplistix.co.uk> <9cee7ab80910280816y1b20ca14q8c03ea7ba886ee25@mail.gmail.com> <4AEB67B2.2020605@simplistix.co.uk> Message-ID: <4AEBD36C.80604@ar.media.kyoto-u.ac.jp> Chris Withers wrote: > I think "python setup.py install" is so idiomatic that it seems silly > to break it for the sake of two lines of python. There's also an > element of "explicit is better than implicit" in the feel of actually > running setup.py... Besides the already given arguments, getting away of the current distutils commands/subcommands and their imperative nature is a big advantage by itself. David From david at ar.media.kyoto-u.ac.jp Sat Oct 31 07:18:03 2009 From: david at ar.media.kyoto-u.ac.jp (David Cournapeau) Date: Sat, 31 Oct 2009 15:18:03 +0900 Subject: [Distutils] Distutils and Distribute roadmap (and some words on Virtualenv, Pip) In-Reply-To: <4AEB69CB.20700@simplistix.co.uk> References: <94bdd2610910080131j323b98d9i871bce43465f237a@mail.gmail.com> <79990c6b0910090532y63169b60q8dd0a95f056c244d@mail.gmail.com> <4ACF6993.5040704@voidspace.org.uk> <4ADDC6CA.5000809@simplistix.co.uk> <79990c6b0910201349w6f0537f2q1eb91bb4fd8d3c8d@mail.gmail.com> <79990c6b0910201408ve7a10f5t76bb409cef57c955@mail.gmail.com> <79990c6b0910201409s390cffc4n43cde28b5dc6843a@mail.gmail.com> <79990c6b0910201502u745d4fb0ie3a24c140b2ba9e0@mail.gmail.com> <4AE7299F.7030108@simplistix.co.uk> <4AE7C00F.7020800@ar.media.kyoto-u.ac.jp> <4AE852A4.2040708@simplistix.co.uk> <4AE91222.4060800@ar.media.kyoto-u.ac.jp> <4AEB69CB.20700@simplistix.co.uk> Message-ID: <4AEBD69B.30605@ar.media.kyoto-u.ac.jp> Chris Withers wrote: > David Cournapeau wrote: >> Chris Withers wrote: >>> "even" seems to imply this is a basic requirement. I wonder how many >>> people are rolling out python apps on Win64? I'm certainly not... >> >> Even was maybe a bit strong. Concerning win64, the first release of >> numpy we did for win64 has been downloaded 3000 times, > > By what? I'm always sceptical about the download figures on PyPI, and > other places. I wonder how many bots are sucking down every .exe or > .msi they can find to check for viruses or the like? Sourceforge numbers. Besides, I can assure you that there is a lot of interest in windows 64 for scientific computing (in terms of people willing to pay). > You should probably look at buildout, you may find your like getting a > lot easier. Yes, the docs suck. I even promised to do something about > that back at PyCon, sadly time has not been kind to my promises... The docs are indeed less than ideal - everytime I tried to read something about buildout, I could not even understand what it was about :) But even though, I doubt buildout is what I am after: chroot-like sandboxing for example, is crucial (and for what it worths, it is a standard practice in the unix deployment world). What I use is kind of besides the point, though. What matters is to have a standard core on top of which people can build their tools: a core in the stdlib, and buildout-based, scons-based, whatever-based tools on top. If you have an API to retrieve where things are put, how they are built and so on, everyone can be happy with the tool he prefers. > Well, buildout.cfg is exacltly that file for me, for both cases ;-) I may not have been clear: by single file, I mean an installer, which needs to work offline, and is as integrated as possible in the user system. This mean msi, dmg, rpm, etc... A buildout.cfg can't possible be further from what I want in those cases. David From kiorky at cryptelium.net Sat Oct 31 12:42:55 2009 From: kiorky at cryptelium.net (kiorky) Date: Sat, 31 Oct 2009 12:42:55 +0100 Subject: [Distutils] Distutils and Distribute roadmap (and some words on Virtualenv, Pip) In-Reply-To: <4AEBD69B.30605@ar.media.kyoto-u.ac.jp> References: <94bdd2610910080131j323b98d9i871bce43465f237a@mail.gmail.com> <79990c6b0910090532y63169b60q8dd0a95f056c244d@mail.gmail.com> <4ACF6993.5040704@voidspace.org.uk> <4ADDC6CA.5000809@simplistix.co.uk> <79990c6b0910201349w6f0537f2q1eb91bb4fd8d3c8d@mail.gmail.com> <79990c6b0910201408ve7a10f5t76bb409cef57c955@mail.gmail.com> <79990c6b0910201409s390cffc4n43cde28b5dc6843a@mail.gmail.com> <79990c6b0910201502u745d4fb0ie3a24c140b2ba9e0@mail.gmail.com> <4AE7299F.7030108@simplistix.co.uk> <4AE7C00F.7020800@ar.media.kyoto-u.ac.jp> <4AE852A4.2040708@simplistix.co.uk> <4AE91222.4060800@ar.media.kyoto-u.ac.jp> <4AEB69CB.20700@simplistix.co.uk> <4AEBD69B.30605@ar.media.kyoto-u.ac.jp> Message-ID: <4AEC22BF.9090206@cryptelium.net> David Cournapeau a ?crit : > But even though, I doubt buildout is what I am after: chroot-like > sandboxing for example, is crucial (and for what it worths, it is a > standard practice in the unix deployment world). > > What I use is kind of besides the point, though. What matters is to have > a standard core on top of which people can build their tools: a core in > the stdlib, and buildout-based, scons-based, whatever-based tools on > top. If you have an API to retrieve where things are put, how they are > built and so on, everyone can be happy with the tool he prefers. For your chrooting/API purposes, you can have a look to minitage, it's all his goals. But even buildout can provide you a basic isolation layer. >> Well, buildout.cfg is exacltly that file for me, for both cases ;-) > > I may not have been clear: by single file, I mean an installer, which > needs to work offline, and is as integrated as possible in the user > system. This mean msi, dmg, rpm, etc... A buildout.cfg can't possible be > further from what I want in those cases. What about using both ? If it is for releasing final softwares for "non-developers", just archive the resulting buildout infra. from the targeted host and make a package from there. Many people there do in that way (rpm, or so). As a developer, you ll have the flexibility and reliability of buildout, and you ll assure to your users to use their lovely system package manager which install a locally tarball of you whole buildout based packaged software. No need to be online there. On the other hand, buildout can work totally offline if you have all your requirements fulfilled. -- Cordialement, KiOrKY GPG Key FingerPrint: 0x1A1194B7681112AF -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 261 bytes Desc: OpenPGP digital signature URL: