From regebro at gmail.com  Sat Jan  3 20:32:14 2009
From: regebro at gmail.com (Lennart Regebro)
Date: Sat, 3 Jan 2009 20:32:14 +0100
Subject: [Distutils] Some suggested patches
Message-ID: <319e029f0901031132l6ce152b9wd95d65e0512ad998@mail.gmail.com>

When trying to port setuptools, I found some things I think are bugs,
missing library reorganisation fixes more specifically.

Firstly, there are a lot of things missing from the urllib fixer, most
significantly all the split* methods. Secondly, htmlentitydefs has
moved, and this doesn't seem to be handled.

Am I correct that these are bugs, or should something else be done? A
suggested patch is included, although I haven't made any tests for it,
I wanted to check here first.

-- 
Lennart Regebro: Zope and Plone consulting.
http://www.colliberty.com/
+33 661 58 14 64
-------------- next part --------------
A non-text attachment was scrubbed...
Name: 2to3.diff
Type: text/x-patch
Size: 1834 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20090103/578fb8bf/attachment.bin>

From regebro at gmail.com  Sun Jan  4 12:21:08 2009
From: regebro at gmail.com (Lennart Regebro)
Date: Sun, 4 Jan 2009 12:21:08 +0100
Subject: [Distutils] Some suggested patches
In-Reply-To: <319e029f0901031132l6ce152b9wd95d65e0512ad998@mail.gmail.com>
References: <319e029f0901031132l6ce152b9wd95d65e0512ad998@mail.gmail.com>
Message-ID: <319e029f0901040321q6aa85827x2181494a3755e84e@mail.gmail.com>

I'm sorry, that was the wong list. Ignore.

On Sat, Jan 3, 2009 at 20:32, Lennart Regebro <regebro at gmail.com> wrote:
> When trying to port setuptools, I found some things I think are bugs,
> missing library reorganisation fixes more specifically.
>
> Firstly, there are a lot of things missing from the urllib fixer, most
> significantly all the split* methods. Secondly, htmlentitydefs has
> moved, and this doesn't seem to be handled.
>
> Am I correct that these are bugs, or should something else be done? A
> suggested patch is included, although I haven't made any tests for it,
> I wanted to check here first.
>
> --
> Lennart Regebro: Zope and Plone consulting.
> http://www.colliberty.com/
> +33 661 58 14 64
>



-- 
Lennart Regebro: Zope and Plone consulting.
http://www.colliberty.com/
+33 661 58 14 64

From regebro at gmail.com  Sun Jan  4 12:29:09 2009
From: regebro at gmail.com (Lennart Regebro)
Date: Sun, 4 Jan 2009 12:29:09 +0100
Subject: [Distutils] Setuptools for Python 3 patch.
Message-ID: <319e029f0901040329u7d7c814bn42daa768232dfd14@mail.gmail.com>

I have in issue 56 ( http://bugs.python.org/setuptools/issue56 ) added
a patch for setuptools, that enable it to be ported to Python 3. If
somebody would like to review the patches, I would be much obliged.
There are many choices I've done that could be done differently,
stylistically.

The porting itself is done with

/opt/python30/bin/2to3 -wn .
/opt/python30/bin/2to3 -dw /tmp/setuptools.LOCAL/setuptools/tests/api_tests.txt

but you currently need to patch 2to3 (with patches which I sent to
this list yesterday by mistake, sorry about that).

Also, I have not done patching to remove all 2to3 warnings. There are
many "foo = map(fun, bar)" calls in the code, and these get converted
by 2to3 to the ugly "foo = list(map(fun, bar))". Changing them all to
"foo = [fun(x) for in in bar]" would be the Python3-approved way of
doing it, but I only fixed the things that was needed to make the port
work.


I'm going to make a blogpost about the porting experience. I'd also
like to provide a source dist tgz for testing by others in that blog
post, of that's OK. If it's not OK, then tell me.

-- 
Lennart Regebro: Zope and Plone consulting.
http://www.colliberty.com/
+33 661 58 14 64

From ziade.tarek at gmail.com  Sun Jan  4 13:04:18 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Sun, 4 Jan 2009 13:04:18 +0100
Subject: [Distutils] [distutils] make the storage of the password optional
	in .pypirc
Message-ID: <94bdd2610901040404w6675999exfde5e81f49cbaf0d@mail.gmail.com>

Hello

I would like to make the storage of the password optional in .pypirc.
If unstored, the register command would prompt for it.

More details here:
http://bugs.python.org/issue4394

Any thaught or objection on this ?

Regards
Tarek

-- 
Tarek Ziad? | Association AfPy | www.afpy.org
Blog FR | http://programmation-python.org
Blog EN | http://tarekziade.wordpress.com/

From tseaver at palladion.com  Sun Jan  4 14:40:28 2009
From: tseaver at palladion.com (Tres Seaver)
Date: Sun, 04 Jan 2009 08:40:28 -0500
Subject: [Distutils] [distutils] make the storage of the password
	optional in .pypirc
In-Reply-To: <94bdd2610901040404w6675999exfde5e81f49cbaf0d@mail.gmail.com>
References: <94bdd2610901040404w6675999exfde5e81f49cbaf0d@mail.gmail.com>
Message-ID: <4960BC4C.7060207@palladion.com>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Tarek Ziad? wrote:

> I would like to make the storage of the password optional in .pypirc.
> If unstored, the register command would prompt for it.
> 
> More details here:
> http://bugs.python.org/issue4394
> 
> Any thaught or objection on this ?

+1.  Storing plaintext passwords on disk is just wrong.


Tres.
- --
===================================================================
Tres Seaver          +1 540-429-0999          tseaver at palladion.com
Palladion Software   "Excellence by Design"    http://palladion.com
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFJYLxM+gerLs4ltQ4RAgZeAKCN3cWY+fFNYOQL/AyEm7TTdpJ8rgCferqv
n53fAy+VTA5wDX0oD6QcKbU=
=O6RC
-----END PGP SIGNATURE-----


From paul at cravenfamily.com  Sun Jan  4 17:16:55 2009
From: paul at cravenfamily.com (Paul Vincent Craven)
Date: Sun, 4 Jan 2009 10:16:55 -0600
Subject: [Distutils] Easy installer on Windows Python 2.6
Message-ID: <5591393c0901040816j7b3ea363j998bd5fcd82e6946@mail.gmail.com>

There is no link here for a Windows installer for Python 2.6:
http://pypi.python.org/pypi/setuptools
Only 2.3, 2.4, and 2.5.

How does a person install 'easy_install' for Python 2.6 on windows?

-- 
Paul Vincent Craven
http://www.cravenfamily.com

From pje at telecommunity.com  Sun Jan  4 18:35:02 2009
From: pje at telecommunity.com (Phillip J. Eby)
Date: Sun, 04 Jan 2009 12:35:02 -0500
Subject: [Distutils] Easy installer on Windows Python 2.6
In-Reply-To: <5591393c0901040816j7b3ea363j998bd5fcd82e6946@mail.gmail.co
 m>
References: <5591393c0901040816j7b3ea363j998bd5fcd82e6946@mail.gmail.com>
Message-ID: <20090104173313.A54FF3A4077@sparrow.telecommunity.com>

At 10:16 AM 1/4/2009 -0600, Paul Vincent Craven wrote:
>There is no link here for a Windows installer for Python 2.6:
>http://pypi.python.org/pypi/setuptools
>Only 2.3, 2.4, and 2.5.
>
>How does a person install 'easy_install' for Python 2.6 on windows?

You might see if the 2.5 installer works.  If not, you can use the 
source and run a bdist_wininst to make a 2.6 installer.


From ziade.tarek at gmail.com  Mon Jan  5 06:00:34 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Mon, 5 Jan 2009 06:00:34 +0100
Subject: [Distutils] [distutils] make the storage of the password
	optional in .pypirc
In-Reply-To: <4960BC4C.7060207@palladion.com>
References: <94bdd2610901040404w6675999exfde5e81f49cbaf0d@mail.gmail.com>
	<4960BC4C.7060207@palladion.com>
Message-ID: <94bdd2610901042100g50901aabvd04c67afa67e5710@mail.gmail.com>

On Sun, Jan 4, 2009 at 2:40 PM, Tres Seaver <tseaver at palladion.com> wrote:
>>
>> Any thaught or objection on this ?
>
> +1.  Storing plaintext passwords on disk is just wrong.

Yes,

in a second phase, I was also wondering if a mechanism ala ssh-agent
could be set
to save the password in the user session.

Tarek

From duna.swlp at gmail.com  Mon Jan  5 20:29:03 2009
From: duna.swlp at gmail.com (dunia)
Date: Mon, 05 Jan 2009 14:29:03 -0500
Subject: [Distutils] buildout recipe options
Message-ID: <49625F7F.6070609@gmail.com>

Hi list

I'm a little bit confused with the buildout configuration files, and i 
can't find any useful information. I get that every part of the buildout 
has a recipe and this recipe takes some configuration options. My 
question is ...how can i know which are those options? I went through 
the code and found most of them, but not all.

For example, since i'm running on Debian i would like to use buildout + 
dzhandle, so i need plone.recipe.dzhandle. I found some configuration 
options used explicitly in the code (locationtype, location, zopetype, 
version, zope2-location, zopeuser, zcml, serviceuser and instancename), 
but i also found in an example buildout configuration file ?httpport? 
(obvious what it is used for) and ?threads? (not that obvious for me :) 
). Also i would like to know which other options i can use to set, for 
instance, dzhandle ?--addon-mode? that is mandatory when using dzhandle 
alone.

Maybe this is very simple, i recognize that my knowledge of python is 
not as good as it should be...yet ;). Anyways, any hint would be useful 
for me.

My english is not very good :(, my apologies.

Thanks & happy new year! :)

From jim at zope.com  Mon Jan  5 20:37:19 2009
From: jim at zope.com (Jim Fulton)
Date: Mon, 5 Jan 2009 14:37:19 -0500
Subject: [Distutils] buildout recipe options
In-Reply-To: <49625F7F.6070609@gmail.com>
References: <49625F7F.6070609@gmail.com>
Message-ID: <A00530A1-CF38-497C-8CEC-4B9E8E17E89A@zope.com>


On Jan 5, 2009, at 2:29 PM, dunia wrote:

> Hi list
>
> I'm a little bit confused with the buildout configuration files, and  
> i can't find any useful information. I get that every part of the  
> buildout has a recipe and this recipe takes some configuration  
> options. My question is ...how can i know which are those options? I  
> went through the code and found most of them, but not all.

Do you mean, "How do you know which options are available?"

I'll assume yes.

> Maybe this is very simple, i recognize that my knowledge of python  
> is not as good as it should be...yet ;). Anyways, any hint would be  
> useful for me.

You have to rely on the recipe documentation.  Recipes should provide  
documentation that describes how they're used, including what their  
options are.  Many recipes make this available as part of their PYPI  
pages (For example, see http://pypi.python.org/pypi/zc.recipe.deployment) 
. Buildout doesn't provide an API for a recipe to register or declare  
the options it provides.  Perhaps it should. My philosophy has been to  
try make as few demands on recipe authors as possible.  I could see  
some benefit in providing an optional registration mechanism that  
buildout could use if it's present.

Jim

--
Jim Fulton
Zope Corporation



From lists at zopyx.com  Mon Jan  5 20:30:12 2009
From: lists at zopyx.com (Andreas Jung)
Date: Mon, 05 Jan 2009 20:30:12 +0100
Subject: [Distutils] buildout recipe options
In-Reply-To: <49625F7F.6070609@gmail.com>
References: <49625F7F.6070609@gmail.com>
Message-ID: <49625FC4.2000805@zopyx.com>

A good recipe has a good documentation and/or doctests. You might check 
on PyPI if a recipe fulfills those requirements. Otherwise, you have to 
read the code or stick with a recipe with a similar functionality.


-aj


On 05.01.2009 20:29 Uhr, dunia wrote:
> Hi list
>
> I'm a little bit confused with the buildout configuration files, and i
> can't find any useful information. I get that every part of the buildout
> has a recipe and this recipe takes some configuration options. My
> question is ...how can i know which are those options? I went through
> the code and found most of them, but not all.
>
> For example, since i'm running on Debian i would like to use buildout +
> dzhandle, so i need plone.recipe.dzhandle. I found some configuration
> options used explicitly in the code (locationtype, location, zopetype,
> version, zope2-location, zopeuser, zcml, serviceuser and instancename),
> but i also found in an example buildout configuration file ?httpport?
> (obvious what it is used for) and ?threads? (not that obvious for me :)
> ). Also i would like to know which other options i can use to set, for
> instance, dzhandle ?--addon-mode? that is mandatory when using dzhandle
> alone.
>
> Maybe this is very simple, i recognize that my knowledge of python is
> not as good as it should be...yet ;). Anyways, any hint would be useful
> for me.
>
> My english is not very good :(, my apologies.
>
> Thanks & happy new year! :)
> _______________________________________________
> Distutils-SIG maillist - Distutils-SIG at python.org
> http://mail.python.org/mailman/listinfo/distutils-sig


-- 
ZOPYX Ltd. & Co. KG - Charlottenstr. 37/1 - 72070 T?bingen - Germany
Web: www.zopyx.com - Email: info at zopyx.com - Phone +49 - 7071 - 793376
Registergericht: Amtsgericht Stuttgart, Handelsregister A 381535
Gesch?ftsf?hrer/Gesellschafter: ZOPYX Limited, Birmingham, UK
------------------------------------------------------------------------
E-Publishing, Python, Zope & Plone development, Consulting

-------------- next part --------------
A non-text attachment was scrubbed...
Name: lists.vcf
Type: text/x-vcard
Size: 316 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20090105/e264acd6/attachment.vcf>

From barry at python.org  Mon Jan  5 21:02:24 2009
From: barry at python.org (Barry Warsaw)
Date: Mon, 5 Jan 2009 15:02:24 -0500
Subject: [Distutils] environment variables in buildout config files?
Message-ID: <F85047E2-CDF7-42E7-B5DB-7E1DA35EDE08@python.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

I just converted Mailman 3 to use zc.buildout.  Thanks to Gary Poster  
for providing some good examples and the impetus for my force vacation  
hacking. :)

I especially like being able to specify defaults in my ~/.buildout/ 
default.cfg file.  One thing that bugs me though is that I can't use  
environment variables like $HOME there.  This is a minor pain because  
I can't share my .buildout directory between Linux and OS X machines  
(say, via version control).  It appears as though I have to be  
explicit in the path, with /Users/barry on OS X and /home/barry on  
Linux.

Similarly, in my buildout.cfg file, if I include a develop field  
pointing to some source directories (other than .), I can't use $HOME/ 
projects/foo/bar here.

Any thoughts on supporting environment variables in .cfg files?  Or do  
they work and I'm just experiencing operator error? ;)

A couple of other minor questions.  In my ~/.buildout/default.cfg  
file, I have

[buildout]
eggs-directory=/Users/barry/.buildout/eggs
download-cache=/Users/barry/.buildout/download-cache

This is great because I can share that stuff between lots of  
development branches.  However buildout will create the eggs directory  
if needed, but complain that download-cache doesn't exist.  It seems  
like buildout could/should create download-cache on demand.

I'm also wondering, does it make sense to put develop-eggs and parts  
in say ~/.buildout?  I kind of dislike having artifacts like these  
live in my version controlled source directory (well, there's bin too,  
but I can kind of live with that).  What's the best way of doing that?

Thanks,
Barry

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (Darwin)

iQCVAwUBSWJnUHEjvBPtnXfVAQInXwP/VGi1JoXW2qMV7u1JhvVSIF2fm3WiCz01
z7M/OenoLzdpicidNY5+gfus8Y8b5qKosA/8XAhA5BM9WOmebjMuaWsR0EZiQEY5
LXwhHlEskoYJlhYRbnnCQyEfTZOIBB57kMGsgKNnRDETz4f7XWEqKvIhdBERWBhx
gWtQAufcE4c=
=81NH
-----END PGP SIGNATURE-----

From barry at python.org  Mon Jan  5 21:08:22 2009
From: barry at python.org (Barry Warsaw)
Date: Mon, 5 Jan 2009 15:08:22 -0500
Subject: [Distutils] zc.recipe.testrunner hook?
Message-ID: <694DE196-1AC5-4A11-89AE-FF28CD29BE86@python.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

I'm not sure this is the best place to ask about zc.recipe.testrunner,  
but I will anyway :).

Besides converting to zc.buildout, over my recent forced vacation, I  
also converted Mailman 3 to zc.recipe.testrunner.  This is another bit  
of awesome from Zope I'm glad to be using.

One small problem though: I need to hook into the option parser to add  
a few more options.  For example, I'm adding a -e/--stderr option  
which sets up logging propagation for some of my subprocess tests.

The way I've done it is to add this to my buildout.cfg file:

[test]
recipe = zc.recipe.testrunner
...
# Hack in extra arguments to zope.testrunner.
initialization = from mailman.testing.layers import ConfigLayer;
     ConfigLayer.hack_options_parser()

The hack_options_parser() method looks like this:

     def hack_options_parser(cls):
         """Hack our way into the zc.testing framework.

         Add our custom command line option parsing into  
zc.testing's.  We do
         the imports here so that if zc.testing isn't invoked, this  
stuff never
         gets in the way.  This is pretty fragile, depend on changes  
in the
         zc.testing package.  There should be a better way!
         """
         from zope.testing.testrunner.options import parser
         parser.add_option('-e', '--stderr',
                           action='callback',  
callback=cls.handle_stderr,
                           help=_('Propagate log errors to stderr.'))

So as the comment asks passive-aggressively: is there a better way?

Thanks,
Barry

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (Darwin)

iQCVAwUBSWJotnEjvBPtnXfVAQJvbwP+NkKyX9bYWLrZQ4L4b3wWbCfL9l72EyQi
RJh0EPQn1Efz6Oe7h+oTojbGxmpviHyJl+OnrO3OrwPxR5miXK3dWFmtdh7r/TUm
jRz3m4k58UwVe2/uWnlHGQ4c/4ooBP0kYZecwTms2cUtzZFkVF/IUoSPvJl/80R2
5q1ZBq9P5PE=
=pUiS
-----END PGP SIGNATURE-----

From marius at pov.lt  Thu Jan  8 19:24:23 2009
From: marius at pov.lt (Marius Gedminas)
Date: Thu, 8 Jan 2009 20:24:23 +0200
Subject: [Distutils] environment variables in buildout config files?
In-Reply-To: <F85047E2-CDF7-42E7-B5DB-7E1DA35EDE08@python.org>
References: <F85047E2-CDF7-42E7-B5DB-7E1DA35EDE08@python.org>
Message-ID: <20090108182423.GA406@fridge.pov.lt>

On Mon, Jan 05, 2009 at 03:02:24PM -0500, Barry Warsaw wrote:
> I just converted Mailman 3 to use zc.buildout.  Thanks to Gary Poster  
> for providing some good examples and the impetus for my force vacation  
> hacking. :)
> 
> I especially like being able to specify defaults in my ~/.buildout/ 
> default.cfg file.  One thing that bugs me though is that I can't use  
> environment variables like $HOME there.

Seconded: https://bugs.launchpad.net/zc.buildout/+bug/190260

> This is a minor pain because  
> I can't share my .buildout directory between Linux and OS X machines  
> (say, via version control).  It appears as though I have to be  
> explicit in the path, with /Users/barry on OS X and /home/barry on  
> Linux.
> 
> Similarly, in my buildout.cfg file, if I include a develop field  
> pointing to some source directories (other than .), I can't use $HOME/ 
> projects/foo/bar here.
> 
> Any thoughts on supporting environment variables in .cfg files?  Or do  
> they work and I'm just experiencing operator error? ;)

AFAIK, no.

> I'm also wondering, does it make sense to put develop-eggs and parts  
> in say ~/.buildout?

No, since those are specific to a particular project and shouldn't be
shared.

(Actually, I don't understand what 'develop-eggs' is for, other than
that removing it tends to fix weird problems after switching from a
develop-egg to a normal egg.)

> I kind of dislike having artifacts like these  
> live in my version controlled source directory (well, there's bin too,  
> but I can kind of live with that).  What's the best way of doing that?

echo 'bin parts develop-eggs' > .csvignore
svn propedit svn:ignore .
bzr ignore bin parts develop-eggs
or the equivalent for the VCS of your choice.

HTH,
Marius Gedminas
-- 
If something has not yet gone wrong then it would ultimately have been
beneficial for it to go wrong.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: Digital signature
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20090108/342ae945/attachment.pgp>

From jim at zope.com  Thu Jan  8 21:02:42 2009
From: jim at zope.com (Jim Fulton)
Date: Thu, 8 Jan 2009 15:02:42 -0500
Subject: [Distutils] environment variables in buildout config files?
In-Reply-To: <20090108182423.GA406@fridge.pov.lt>
References: <F85047E2-CDF7-42E7-B5DB-7E1DA35EDE08@python.org>
	<20090108182423.GA406@fridge.pov.lt>
Message-ID: <818E0066-B4C3-4457-B82D-2140BB405244@zope.com>


On Jan 8, 2009, at 1:24 PM, Marius Gedminas wrote:

Thanks for answering these questions. :)

> (Actually, I don't understand what 'develop-eggs' is for,

It is a place where develop eggs created by the buildout are placed so  
they can be found when looking for eggs to satisfy dependencies.

> other than
> that removing it tends to fix weird problems after switching from a
> develop-egg to a normal egg.)


I fixed a bug in managing develop egg links some time ago.  If you are  
still having problems, please submit a bug.

Jim

--
Jim Fulton
Zope Corporation



From barry at python.org  Thu Jan  8 22:12:11 2009
From: barry at python.org (Barry Warsaw)
Date: Thu, 8 Jan 2009 16:12:11 -0500
Subject: [Distutils] environment variables in buildout config files?
In-Reply-To: <20090108182423.GA406@fridge.pov.lt>
References: <F85047E2-CDF7-42E7-B5DB-7E1DA35EDE08@python.org>
	<20090108182423.GA406@fridge.pov.lt>
Message-ID: <0710438D-0BC2-4B83-8B05-09D47A965DF3@python.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Jan 8, 2009, at 1:24 PM, Marius Gedminas wrote:

> On Mon, Jan 05, 2009 at 03:02:24PM -0500, Barry Warsaw wrote:
>> I just converted Mailman 3 to use zc.buildout.  Thanks to Gary Poster
>> for providing some good examples and the impetus for my force  
>> vacation
>> hacking. :)
>>
>> I especially like being able to specify defaults in my ~/.buildout/
>> default.cfg file.  One thing that bugs me though is that I can't use
>> environment variables like $HOME there.
>
> Seconded: https://bugs.launchpad.net/zc.buildout/+bug/190260

Thanks.  I've just added a comment on that bug, and have subscribed to  
it.

>> This is a minor pain because
>> I can't share my .buildout directory between Linux and OS X machines
>> (say, via version control).  It appears as though I have to be
>> explicit in the path, with /Users/barry on OS X and /home/barry on
>> Linux.
>>
>> Similarly, in my buildout.cfg file, if I include a develop field
>> pointing to some source directories (other than .), I can't use  
>> $HOME/
>> projects/foo/bar here.
>>
>> Any thoughts on supporting environment variables in .cfg files?  Or  
>> do
>> they work and I'm just experiencing operator error? ;)
>
> AFAIK, no.
>
>> I'm also wondering, does it make sense to put develop-eggs and parts
>> in say ~/.buildout?
>
> No, since those are specific to a particular project and shouldn't be
> shared.

Ok.

> (Actually, I don't understand what 'develop-eggs' is for, other than
> that removing it tends to fix weird problems after switching from a
> develop-egg to a normal egg.)
>
>> I kind of dislike having artifacts like these
>> live in my version controlled source directory (well, there's bin  
>> too,
>> but I can kind of live with that).  What's the best way of doing  
>> that?
>
> echo 'bin parts develop-eggs' > .csvignore
> svn propedit svn:ignore .
> bzr ignore bin parts develop-eggs
> or the equivalent for the VCS of your choice.

Actually, it's not so much 'bzr stat' that I care about since that's  
easily ignored.  It's 'ls' that I care about since that's tougher to  
ignore.  I guess I could use dot-prefixes. ;)

Thanks
Barry


-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (Darwin)

iQCVAwUBSWZsK3EjvBPtnXfVAQI0tgP+IM7CeEQ2AxHBTXElbyLtwNjkuEAbK9ro
PUuGYasxoyedtqG2K2Mevso2Bsjngu0agdxyjQWzqKr90g8yKEgGBfxkVM93AHlo
NIb5An4MXzAlH2tJGQqv4JVlYmXwxer30n2ezT6ZVAD1S4twJMYTSIpA3q1QE2Hd
f5EYrF+JahE=
=KpdQ
-----END PGP SIGNATURE-----

From barry at python.org  Thu Jan  8 22:12:39 2009
From: barry at python.org (Barry Warsaw)
Date: Thu, 8 Jan 2009 16:12:39 -0500
Subject: [Distutils] environment variables in buildout config files?
In-Reply-To: <818E0066-B4C3-4457-B82D-2140BB405244@zope.com>
References: <F85047E2-CDF7-42E7-B5DB-7E1DA35EDE08@python.org>
	<20090108182423.GA406@fridge.pov.lt>
	<818E0066-B4C3-4457-B82D-2140BB405244@zope.com>
Message-ID: <991C8EB9-994F-4333-944F-CD3EA1D2BCF6@python.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Jan 8, 2009, at 3:02 PM, Jim Fulton wrote:

> Thanks for answering these questions. :)
>
>> (Actually, I don't understand what 'develop-eggs' is for,
>
> It is a place where develop eggs created by the buildout are placed  
> so they can be found when looking for eggs to satisfy dependencies.

Thanks for the answer!

Barry

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (Darwin)

iQCVAwUBSWZsR3EjvBPtnXfVAQK0wAP8C89AgIrrl9PZ8FM9Qq/ab+AC7eq5t1Hu
MJQ9DsN6z2KgC5+4JtcNyAXdiL00UXiMvvlYyynzCeT4k4N5hMU3Qh/D5s0K6aTX
GnOgYumxA+tJEL+f2dDBr0phSbd3SX6PNYoMMfb13nVfyID6XyFgdCD+dzOqHKZl
b7MFIBg2370=
=FeYw
-----END PGP SIGNATURE-----

From marius at pov.lt  Thu Jan  8 22:28:41 2009
From: marius at pov.lt (Marius Gedminas)
Date: Thu, 8 Jan 2009 23:28:41 +0200
Subject: [Distutils] environment variables in buildout config files?
In-Reply-To: <0710438D-0BC2-4B83-8B05-09D47A965DF3@python.org>
References: <F85047E2-CDF7-42E7-B5DB-7E1DA35EDE08@python.org>
	<20090108182423.GA406@fridge.pov.lt>
	<0710438D-0BC2-4B83-8B05-09D47A965DF3@python.org>
Message-ID: <20090108212840.GB15497@fridge.pov.lt>

On Thu, Jan 08, 2009 at 04:12:11PM -0500, Barry Warsaw wrote:
> On Jan 8, 2009, at 1:24 PM, Marius Gedminas wrote:
> > On Mon, Jan 05, 2009 at 03:02:24PM -0500, Barry Warsaw wrote:
> >> I'm also wondering, does it make sense to put develop-eggs and parts
> >> in say ~/.buildout?
> >
> > No, since those are specific to a particular project and shouldn't be
> > shared.
> 
> Ok.
> 
> > (Actually, I don't understand what 'develop-eggs' is for, other than
> > that removing it tends to fix weird problems after switching from a
> > develop-egg to a normal egg.)

After Jim's explanation (thanks!) I don't see a reason why develop-eggs
couldn't also be shared... no, wait, perhaps I see one.

Imagine project A using a develop-egg of a *branch* of zope.frobozz,
while project B uses a develop-egg of the *trunk* of zope.frobozz.  In
that case /path/to/project-A/develop-eggs/zope.frobozz.egg-link will
contain a link to ~/src/zope.frobozz/random-feature-branch, while
/path/to/project-B/develop-eggs/zope.frobozz.egg-link would containt a
link to ~/src/zope.frobozz/trunk.  Same filename, different content --
ergo, develop-eggs cannot be shared between projects.

I have a couple of source trees in my home directory that contain an
actual example of this.

> >> I kind of dislike having artifacts like these live in my version
> >> controlled source directory (well, there's bin  too, but I can kind
> >> of live with that).  What's the best way of doing  that?
> >
> > echo 'bin parts develop-eggs' > .csvignore
> > svn propedit svn:ignore .
> > bzr ignore bin parts develop-eggs
> > or the equivalent for the VCS of your choice.
> 
> Actually, it's not so much 'bzr stat' that I care about since that's  
> easily ignored.

(in which case your wording misled me ;)

> It's 'ls' that I care about since that's tougher to  
> ignore.  I guess I could use dot-prefixes. ;)

Personally, I'm more bothered by the '*.egg-info' directories created by
setuptools.  Also, but to a somewhat lesser extent, the 'build' and 'dist'
ones created by distutils.

'bin' and 'parts' are something I need to deal with rather often.  I've
grown to like bin/$randomscript, and while 'parts/$whatever/$somefile'
is a bit too deep and nested, I appreciate the namespacing and
explicitness ("this is a directory managed by buildout; it may be wiped
out at any moment; don't touch it").  'develop-eggs' is something that
I'm not personally interested in and wouldn't mind hiding with a
dot-prefix, but since I can't do that with 'build' and 'dist', I don't
see much point.  As long as it doesn't interfere with my tab-completion,
I'm fine.

Regards,
Marius Gedminas
-- 
BASIC:
        A programming language.  Related to certain social diseases
        in that those who have it will not admit it in polite company.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: Digital signature
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20090108/c46534d9/attachment.pgp>

From jim at zope.com  Thu Jan  8 22:35:46 2009
From: jim at zope.com (Jim Fulton)
Date: Thu, 8 Jan 2009 16:35:46 -0500
Subject: [Distutils] environment variables in buildout config files?
In-Reply-To: <20090108212840.GB15497@fridge.pov.lt>
References: <F85047E2-CDF7-42E7-B5DB-7E1DA35EDE08@python.org>
	<20090108182423.GA406@fridge.pov.lt>
	<0710438D-0BC2-4B83-8B05-09D47A965DF3@python.org>
	<20090108212840.GB15497@fridge.pov.lt>
Message-ID: <224D31B0-DB92-4782-96FD-DD4145DA57EE@zope.com>


On Jan 8, 2009, at 4:28 PM, Marius Gedminas wrote:

> On Thu, Jan 08, 2009 at 04:12:11PM -0500, Barry Warsaw wrote:
>> On Jan 8, 2009, at 1:24 PM, Marius Gedminas wrote:
>>> On Mon, Jan 05, 2009 at 03:02:24PM -0500, Barry Warsaw wrote:
>>>> I'm also wondering, does it make sense to put develop-eggs and  
>>>> parts
>>>> in say ~/.buildout?
>>>
>>> No, since those are specific to a particular project and shouldn't  
>>> be
>>> shared.
>>
>> Ok.
>>
>>> (Actually, I don't understand what 'develop-eggs' is for, other than
>>> that removing it tends to fix weird problems after switching from a
>>> develop-egg to a normal egg.)
>
> After Jim's explanation (thanks!) I don't see a reason why develop- 
> eggs
> couldn't also be shared... no, wait, perhaps I see one.
>
> Imagine project A using a develop-egg of a *branch* of zope.frobozz,
> while project B uses a develop-egg of the *trunk* of zope.frobozz.  In
> that case /path/to/project-A/develop-eggs/zope.frobozz.egg-link will
> contain a link to ~/src/zope.frobozz/random-feature-branch, while
> /path/to/project-B/develop-eggs/zope.frobozz.egg-link would containt a
> link to ~/src/zope.frobozz/trunk.  Same filename, different content --
> ergo, develop-eggs cannot be shared between projects.
>
> I have a couple of source trees in my home directory that contain an
> actual example of this.

Right. That's the main reason they shouldn't be shared.

My explanation was also incomplete.  If you use the  
zc.recipe.egg:custom recipe to do a custom build of a binary, then the  
resulting binary egg is also put in develop-eggs because the egg  
depends on the buildout configuration.

Jim

--
Jim Fulton
Zope Corporation



From barry at python.org  Thu Jan  8 22:37:05 2009
From: barry at python.org (Barry Warsaw)
Date: Thu, 8 Jan 2009 16:37:05 -0500
Subject: [Distutils] environment variables in buildout config files?
In-Reply-To: <20090108212840.GB15497@fridge.pov.lt>
References: <F85047E2-CDF7-42E7-B5DB-7E1DA35EDE08@python.org>
	<20090108182423.GA406@fridge.pov.lt>
	<0710438D-0BC2-4B83-8B05-09D47A965DF3@python.org>
	<20090108212840.GB15497@fridge.pov.lt>
Message-ID: <6AF7AC28-4351-4F06-8DE8-B9DE0C823A01@python.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Jan 8, 2009, at 4:28 PM, Marius Gedminas wrote:

> After Jim's explanation (thanks!) I don't see a reason why develop- 
> eggs
> couldn't also be shared... no, wait, perhaps I see one.
>
> Imagine project A using a develop-egg of a *branch* of zope.frobozz,
> while project B uses a develop-egg of the *trunk* of zope.frobozz.  In
> that case /path/to/project-A/develop-eggs/zope.frobozz.egg-link will
> contain a link to ~/src/zope.frobozz/random-feature-branch, while
> /path/to/project-B/develop-eggs/zope.frobozz.egg-link would containt a
> link to ~/src/zope.frobozz/trunk.  Same filename, different content --
> ergo, develop-eggs cannot be shared between projects.
>
> I have a couple of source trees in my home directory that contain an
> actual example of this.

That makes sense.  I don't do this, but it definitely makes sense.

>>>> I kind of dislike having artifacts like these live in my version
>>>> controlled source directory (well, there's bin  too, but I can kind
>>>> of live with that).  What's the best way of doing  that?
>>>
>>> echo 'bin parts develop-eggs' > .csvignore
>>> svn propedit svn:ignore .
>>> bzr ignore bin parts develop-eggs
>>> or the equivalent for the VCS of your choice.
>>
>> Actually, it's not so much 'bzr stat' that I care about since that's
>> easily ignored.
>
> (in which case your wording misled me ;)

Sorry ;)

>> It's 'ls' that I care about since that's tougher to
>> ignore.  I guess I could use dot-prefixes. ;)
>
> Personally, I'm more bothered by the '*.egg-info' directories  
> created by
> setuptools.  Also, but to a somewhat lesser extent, the 'build' and  
> 'dist'
> ones created by distutils.

Yeah, I hate all of those too!

> 'bin' and 'parts' are something I need to deal with rather often.   
> I've
> grown to like bin/$randomscript, and while 'parts/$whatever/$somefile'
> is a bit too deep and nested, I appreciate the namespacing and
> explicitness ("this is a directory managed by buildout; it may be  
> wiped
> out at any moment; don't touch it").  'develop-eggs' is something that
> I'm not personally interested in and wouldn't mind hiding with a
> dot-prefix, but since I can't do that with 'build' and 'dist', I don't
> see much point.  As long as it doesn't interfere with my tab- 
> completion,
> I'm fine.

bin is a tough one.  I love its convenience.  parts I've never used.

setup_requires dependencies also show up here (I always see the bzr  
egg because I setup-depend on setuptools_bzr).

One other thing that bugs me about these is that it's not obvious from  
an ls what you can actually delete because it's an artifact of  
setuptools/buildout/etc and what is part of your package.  I mean, you  
have to know that it's safe to delete bin, dist, and build  right?

setup.py has a 'clean' command but it's not enough.  Maybe if buildout  
added a --clean option or something that would be better.  What I  
really want is a simple way to revert a tree back to its pristine  
version-controlled layout, without all the build, develop, test  
artifacts.

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (Darwin)

iQCVAwUBSWZyAXEjvBPtnXfVAQKOYwP/Y3f/KD+/xXRzIP3TyRFUgtVywAfdnytU
+H58rajSCt/WrYhwkLlFxBga/+/FQ7HqtpBQYZ7WEidh6FM2qjMmVGkVgVtOE3kQ
umBgYc25lLytwj2nS05ZRANHDLYhWNbq8XMxrNnWLMPxxlT48ldrziBRv8MrKAX7
cBxHuKzJOAY=
=21nV
-----END PGP SIGNATURE-----

From marius at pov.lt  Thu Jan  8 22:46:38 2009
From: marius at pov.lt (Marius Gedminas)
Date: Thu, 8 Jan 2009 23:46:38 +0200
Subject: [Distutils] environment variables in buildout config files?
In-Reply-To: <6AF7AC28-4351-4F06-8DE8-B9DE0C823A01@python.org>
References: <F85047E2-CDF7-42E7-B5DB-7E1DA35EDE08@python.org>
	<20090108182423.GA406@fridge.pov.lt>
	<0710438D-0BC2-4B83-8B05-09D47A965DF3@python.org>
	<20090108212840.GB15497@fridge.pov.lt>
	<6AF7AC28-4351-4F06-8DE8-B9DE0C823A01@python.org>
Message-ID: <20090108214638.GC15497@fridge.pov.lt>

On Thu, Jan 08, 2009 at 04:37:05PM -0500, Barry Warsaw wrote:
> One other thing that bugs me about these is that it's not obvious from  
> an ls what you can actually delete because it's an artifact of  
> setuptools/buildout/etc and what is part of your package.  I mean, you  
> have to know that it's safe to delete bin, dist, and build  right?

Or you could see which files are ignored by your VCS.  E.g. with bzr
you'd do 'rm `bzr unknowns`', and I'm sure you could write a one-liner
to work with 'svn st -v'.

Or you could get a fresh checkout, or use svn export.

> setup.py has a 'clean' command but it's not enough.  Maybe if buildout  
> added a --clean option or something that would be better.  What I  
> really want is a simple way to revert a tree back to its pristine  
> version-controlled layout, without all the build, develop, test  
> artifacts.

I'd say this is up to your version-control system to implement this,
then.  It doesn't feel right to put this into buildout.  E.g. how would
it know that it's OK to erase ./eggs, but not OK to erase
~/.buildout/my-shared-egg-cache/, if the difference is just an option in
~/.buildout/default.cfg?  Sure, you could see which files are inside the
tree and which ones are outside, but it rapidly descends into guessing.
Which you should refuse the temptation to, according to Zen.

Also, there are files littering your source tree that aren't tracked in
a VCS, and weren't created by buildout (e.g. *.pyc).  I don't think
bin/buildout clean should remove files bin/buildout didn't create.

Marius Gedminas
-- 
Attempts to stick to simple on-or-off options lead to monsters like gcc, which
now has so many flags that programmers are using genetic algorithms to explore
them.
        -- http://www.third-bit.com/~gvwilson/xmlprog.html
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: Digital signature
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20090108/88df4c5d/attachment.pgp>

From greno at verizon.net  Fri Jan  9 02:12:19 2009
From: greno at verizon.net (Gerry Reno)
Date: Thu, 08 Jan 2009 20:12:19 -0500
Subject: [Distutils] pre-release versioning problems with sdist, bdist_rpm,
	bdist_debian
Message-ID: <4966A473.6090607@verizon.net>


When I create a set of distro packages for a specific distro, I need to 
distribute both a tarball and a distro-specific package such as an RPM 
or a deb archive.  Some people like installing things from tarballs, 
others like using the distro package manager.  So I let them have it 
both ways.  We can generate the tarball using 'sdist' and generate the 
distro-package with 'bdist_rpm', etc.

This all works great as long as we're talking final releases here: 
4.9.0, 5.0.0.   But, whenever you start to introduce pre-release 
candidate versioning, eg: 5.0.0_rc2, then things unravel.

What I have been attempting to do, and is apparently is like trying to 
find the holy grail, is to be able to generate both tarballs and 
distro-packages for both pre-release candidates and final releases, that 
exhibit the following behavior:

tarballs:                        foo-5.0.0_rc1.tar.gz       
(version=5.0.0_rc1); (extracts a foo-5.0.0_rc1 directory;  'sdist' 
requires version set like this so tarball name is unique and extract dir 
is unique)
distro-package (pre-release):    foo-5.0.0-0_rc1.noarch.rpm 
(version=5.0.0 and release=0_rc1)
distro-package (final release):  foo-5.0.0-1.noarch.rpm     
(version=5.0.0 and release=1);  'bdist_rpm' requires version and release 
set like this so final release will update the pre-release candidate package

The problem is that there appears to be no way to get a coordinated 
behavior between 'sdist' and 'bdist_rpm' as far as version and release 
strings are concerned that will satisfy both tarballs and bdist packages.
'sdist' knows nothing about 'release' string so you are forced to put 
the pre-release candidate information into the 'version' string' but as 
soon as you do that then you break 'bdist' packaging.  And to complicate 
matters, the world seems to have settled for making pre-release 
candidate version strings like '5.0.0_rc1' for many applications.

For example:

We have an application, foo-5.0.0, and we want to put out some 
pre-release candidates for testing, so we set the version to "5.0.0_rc1" 
in setup.py.

We create the source distribution with:
$ python setup.py sdist
which creates a source archive, foo-5.0.0_rc1.tar.gz.

We extract this archive and 'cd' into the foo-5.0.0_rc1 directory and 
create a built distribution with:
$ python setup.py bdist_rpm
which creates source and binary RPMS in the form:  
foo-5.0.0_rc1-1.noarch.rpm.

So we think everything is fine.  Everyone installs and tests using the 
pre-release candidate and subsequent candidates but when you eventually 
get to the final release, foo-5.0.0, and build your final release RPMS, 
foo-5.0.0-1.noarch.rpm, you find that it will not update your last 
pre-release candidate RPM, foo-5.0.0_rcX-1.noarch.rpm because it is not 
"rpm newer".

I'm hoping there is some distutils guru that can tell me how to solve 
this enigma or show me how to extend distutils to accomplish what is needed.


Regards,
Gerry

PS: In case anyone notices, yes, I posted a similar question yesterday 
on the python list without much response, but the question really 
belongs on this list.

From ziade.tarek at gmail.com  Fri Jan  9 09:32:39 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Fri, 9 Jan 2009 09:32:39 +0100
Subject: [Distutils] [distutils] make the storage of the password
	optional in .pypirc
In-Reply-To: <94bdd2610901042100g50901aabvd04c67afa67e5710@mail.gmail.com>
References: <94bdd2610901040404w6675999exfde5e81f49cbaf0d@mail.gmail.com>
	<4960BC4C.7060207@palladion.com>
	<94bdd2610901042100g50901aabvd04c67afa67e5710@mail.gmail.com>
Message-ID: <94bdd2610901090032o40116765j96b7f2a68df3791d@mail.gmail.com>

On Mon, Jan 5, 2009 at 6:00 AM, Tarek Ziad? <ziade.tarek at gmail.com> wrote:
> On Sun, Jan 4, 2009 at 2:40 PM, Tres Seaver <tseaver at palladion.com> wrote:
>>>
>>> Any thaught or objection on this ?
>>
>> +1.  Storing plaintext passwords on disk is just wrong.

This is done and available in trunk and 3.1,

I will also backport that feature in the collective.dist package,
which provides distutils new features for Python 2.4->2.6

Tarek

-- 
Tarek Ziad? | Association AfPy | www.afpy.org
Blog FR | http://programmation-python.org
Blog EN | http://tarekziade.wordpress.com/

From ziade.tarek at gmail.com  Fri Jan  9 09:36:36 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Fri, 9 Jan 2009 09:36:36 +0100
Subject: [Distutils] pre-release versioning problems with sdist,
	bdist_rpm, bdist_debian
In-Reply-To: <4966A473.6090607@verizon.net>
References: <4966A473.6090607@verizon.net>
Message-ID: <94bdd2610901090036v2c4ba13em55bb32ba6087ed35@mail.gmail.com>

On Fri, Jan 9, 2009 at 2:12 AM, Gerry Reno <greno at verizon.net> wrote:
> The problem is that there appears to be no way to get a coordinated behavior
> between 'sdist' and 'bdist_rpm' as far as version and release strings are
> concerned that will satisfy both tarballs and bdist packages.

If I understand your problem correclty, if sdist would simply
concatenate the version string and
the release string to use it as a "source version" when it starts to work,
you would be able to work things out ?

Regards
Tarek

-- 
Tarek Ziad? | Association AfPy | www.afpy.org
Blog FR | http://programmation-python.org
Blog EN | http://tarekziade.wordpress.com/

From greno at verizon.net  Fri Jan  9 15:51:39 2009
From: greno at verizon.net (Gerry Reno)
Date: Fri, 09 Jan 2009 09:51:39 -0500
Subject: [Distutils] pre-release versioning problems with sdist,
 bdist_rpm, bdist_debian
In-Reply-To: <94bdd2610901090036v2c4ba13em55bb32ba6087ed35@mail.gmail.com>
References: <4966A473.6090607@verizon.net>
	<94bdd2610901090036v2c4ba13em55bb32ba6087ed35@mail.gmail.com>
Message-ID: <4967647B.1010706@verizon.net>

Tarek Ziad? wrote:
> On Fri, Jan 9, 2009 at 2:12 AM, Gerry Reno <greno at verizon.net> wrote:
>   
>> The problem is that there appears to be no way to get a coordinated behavior
>> between 'sdist' and 'bdist_rpm' as far as version and release strings are
>> concerned that will satisfy both tarballs and bdist packages.
>>     
>
> If I understand your problem correclty, if sdist would simply
> concatenate the version string and
> the release string to use it as a "source version" when it starts to work,
> you would be able to work things out ?
>   
I don't know the internals of 'sdist' but I think if there were a way to 
extend 'sdist' to use 'release' as well as 'version' then that might 
work.  I would have to test that to see.

Regards.
Gerry
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20090109/a9c19844/attachment.htm>

From stephenemslie at gmail.com  Fri Jan  9 16:08:03 2009
From: stephenemslie at gmail.com (Stephen Emslie)
Date: Fri, 9 Jan 2009 15:08:03 +0000
Subject: [Distutils] [distutils] make the storage of the password
	optional in .pypirc
In-Reply-To: <94bdd2610901090032o40116765j96b7f2a68df3791d@mail.gmail.com>
References: <94bdd2610901040404w6675999exfde5e81f49cbaf0d@mail.gmail.com>
	<4960BC4C.7060207@palladion.com>
	<94bdd2610901042100g50901aabvd04c67afa67e5710@mail.gmail.com>
	<94bdd2610901090032o40116765j96b7f2a68df3791d@mail.gmail.com>
Message-ID: <51f97e530901090708w3105ecf3la220a32347ae126c@mail.gmail.com>

Brilliant.


A bit OT, but from your blog post on the subject:

>I'd like to go further and to think about a ssh-agent like system, so there's no need
>to enter the pasword everytime you work with PyPI in the same session.

Have you had any feedback on this yet?


On Fri, Jan 9, 2009 at 8:32 AM, Tarek Ziad? <ziade.tarek at gmail.com> wrote:
> On Mon, Jan 5, 2009 at 6:00 AM, Tarek Ziad? <ziade.tarek at gmail.com> wrote:
>> On Sun, Jan 4, 2009 at 2:40 PM, Tres Seaver <tseaver at palladion.com> wrote:
>>>>
>>>> Any thaught or objection on this ?
>>>
>>> +1.  Storing plaintext passwords on disk is just wrong.
>
> This is done and available in trunk and 3.1,
>
> I will also backport that feature in the collective.dist package,
> which provides distutils new features for Python 2.4->2.6
>
> Tarek
>
> --
> Tarek Ziad? | Association AfPy | www.afpy.org
> Blog FR | http://programmation-python.org
> Blog EN | http://tarekziade.wordpress.com/
> _______________________________________________
> Distutils-SIG maillist  -  Distutils-SIG at python.org
> http://mail.python.org/mailman/listinfo/distutils-sig
>

From ziade.tarek at gmail.com  Fri Jan  9 16:10:10 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Fri, 9 Jan 2009 16:10:10 +0100
Subject: [Distutils] pre-release versioning problems with sdist,
	bdist_rpm, bdist_debian
In-Reply-To: <4967647B.1010706@verizon.net>
References: <4966A473.6090607@verizon.net>
	<94bdd2610901090036v2c4ba13em55bb32ba6087ed35@mail.gmail.com>
	<4967647B.1010706@verizon.net>
Message-ID: <94bdd2610901090710kb7969eetf24786ae48ed3a0e@mail.gmail.com>

On Fri, Jan 9, 2009 at 3:51 PM, Gerry Reno <greno at verizon.net> wrote:
> Tarek Ziad? wrote:
>
> On Fri, Jan 9, 2009 at 2:12 AM, Gerry Reno <greno at verizon.net> wrote:
>
>
> The problem is that there appears to be no way to get a coordinated behavior
> between 'sdist' and 'bdist_rpm' as far as version and release strings are
> concerned that will satisfy both tarballs and bdist packages.
>
>
> If I understand your problem correclty, if sdist would simply
> concatenate the version string and
> the release string to use it as a "source version" when it starts to work,
> you would be able to work things out ?
>
>
> I don't know the internals of 'sdist' but I think if there were a way to
> extend 'sdist' to use 'release' as well as 'version' then that might work.
> I would have to test that to see.
>

Well, can you define how sdist should behave exactly ?

Based on that discussion I can make a prototype for you to try out, then we can
maybe propose in that mailing list a change to sdist

Regards
Tarek

-- 
Tarek Ziad? | Association AfPy | www.afpy.org
Blog FR | http://programmation-python.org
Blog EN | http://tarekziade.wordpress.com/

From ziade.tarek at gmail.com  Fri Jan  9 16:15:46 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Fri, 9 Jan 2009 16:15:46 +0100
Subject: [Distutils] [distutils] make the storage of the password
	optional in .pypirc
In-Reply-To: <51f97e530901090708w3105ecf3la220a32347ae126c@mail.gmail.com>
References: <94bdd2610901040404w6675999exfde5e81f49cbaf0d@mail.gmail.com>
	<4960BC4C.7060207@palladion.com>
	<94bdd2610901042100g50901aabvd04c67afa67e5710@mail.gmail.com>
	<94bdd2610901090032o40116765j96b7f2a68df3791d@mail.gmail.com>
	<51f97e530901090708w3105ecf3la220a32347ae126c@mail.gmail.com>
Message-ID: <94bdd2610901090715x35031bf6r626ec2a996624984@mail.gmail.com>

On Fri, Jan 9, 2009 at 4:08 PM, Stephen Emslie <stephenemslie at gmail.com> wrote:
> Brilliant.
>

Thanks :)

>
> A bit OT, but from your blog post on the subject:
>
>>I'd like to go further and to think about a ssh-agent like system, so there's no need
>>to enter the pasword everytime you work with PyPI in the same session.
>
> Have you had any feedback on this yet?

Not yet. I need to dig on how this could work properly, by studying ssh-agent

Regards
Tarek

From benji at zope.com  Fri Jan  9 16:17:50 2009
From: benji at zope.com (Benji York)
Date: Fri, 9 Jan 2009 10:17:50 -0500
Subject: [Distutils] [distutils] make the storage of the password
	optional in .pypirc
In-Reply-To: <51f97e530901090708w3105ecf3la220a32347ae126c@mail.gmail.com>
References: <94bdd2610901040404w6675999exfde5e81f49cbaf0d@mail.gmail.com>
	<4960BC4C.7060207@palladion.com>
	<94bdd2610901042100g50901aabvd04c67afa67e5710@mail.gmail.com>
	<94bdd2610901090032o40116765j96b7f2a68df3791d@mail.gmail.com>
	<51f97e530901090708w3105ecf3la220a32347ae126c@mail.gmail.com>
Message-ID: <e5fff6640901090717s74c9618ducf50a80177bd80c8@mail.gmail.com>

On Fri, Jan 9, 2009 at 10:08 AM, Stephen Emslie <stephenemslie at gmail.com> wrote:
> A bit OT, but from your blog post on the subject:
>
>>I'd like to go further and to think about a ssh-agent like system, so there's no need
>>to enter the pasword everytime you work with PyPI in the same session.
>
> Have you had any feedback on this yet?

Here's some:  how about instead of an ssh-like system, use ssh itself.  Front
PyPI with an ssh server that users connect to.  That way it is both secure and
the infrastructure (agent, etc.) is already in place.
-- 
Benji York
Senior Software Engineer
Zope Corporation

From greno at verizon.net  Fri Jan  9 16:28:46 2009
From: greno at verizon.net (Gerry Reno)
Date: Fri, 09 Jan 2009 10:28:46 -0500
Subject: [Distutils] pre-release versioning problems with sdist,
 bdist_rpm, bdist_debian
In-Reply-To: <94bdd2610901090710kb7969eetf24786ae48ed3a0e@mail.gmail.com>
References: <4966A473.6090607@verizon.net>
	<94bdd2610901090036v2c4ba13em55bb32ba6087ed35@mail.gmail.com>
	<4967647B.1010706@verizon.net>
	<94bdd2610901090710kb7969eetf24786ae48ed3a0e@mail.gmail.com>
Message-ID: <49676D2E.70001@verizon.net>

Tarek Ziad? wrote:
> On Fri, Jan 9, 2009 at 3:51 PM, Gerry Reno <greno at verizon.net> wrote:
>   
>> Tarek Ziad? wrote:
>>
>> On Fri, Jan 9, 2009 at 2:12 AM, Gerry Reno <greno at verizon.net> wrote:
>>
>>
>> The problem is that there appears to be no way to get a coordinated behavior
>> between 'sdist' and 'bdist_rpm' as far as version and release strings are
>> concerned that will satisfy both tarballs and bdist packages.
>>
>>
>> If I understand your problem correclty, if sdist would simply
>> concatenate the version string and
>> the release string to use it as a "source version" when it starts to work,
>> you would be able to work things out ?
>>
>>
>> I don't know the internals of 'sdist' but I think if there were a way to
>> extend 'sdist' to use 'release' as well as 'version' then that might work.
>> I would have to test that to see.
>>
>>     
>
> Well, can you define how sdist should behave exactly ?
>
> Based on that discussion I can make a prototype for you to try out, then we can
> maybe propose in that mailing list a change to sdist
>   
Thanks Tarek.  I think if it would do the same thing as bdist_rpm that 
it would be ok.  bdist_rpm looks like it does VERSION-RELEASE (hyphen 
separator).    So then doing this for 'sdist' I guess would produce a 
tarball name of foo-VERSION-RELEASE.tar.gz  and an extracted directory 
of foo-VERSION-RELEASE.   What this would allow then is for the 
'version' string to stay at '5.0.0' and then the 'release' string to 
contain any pre-release information such as '0_rc1' and then the final 
release would contain '1' which is lexically superior to the '0_rc1'.  
I'm not sure though what other targets in distutils also use 'version' 
so I don't know if this would affect anything else.    Also, I'm hoping 
this can be implemented as some kind of extension so that it can be made 
to work for existing installations as well.

Regards,
Gerry

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20090109/5a492b66/attachment-0001.htm>

From greno at verizon.net  Fri Jan  9 16:42:05 2009
From: greno at verizon.net (Gerry Reno)
Date: Fri, 09 Jan 2009 10:42:05 -0500
Subject: [Distutils] pre-release versioning problems with sdist,
 bdist_rpm, bdist_debian
In-Reply-To: <49676D2E.70001@verizon.net>
References: <4966A473.6090607@verizon.net>
	<94bdd2610901090036v2c4ba13em55bb32ba6087ed35@mail.gmail.com>
	<4967647B.1010706@verizon.net>
	<94bdd2610901090710kb7969eetf24786ae48ed3a0e@mail.gmail.com>
	<49676D2E.70001@verizon.net>
Message-ID: <4967704D.40005@verizon.net>

Gerry Reno wrote:
> Tarek Ziad? wrote:
>> On Fri, Jan 9, 2009 at 3:51 PM, Gerry Reno <greno at verizon.net> wrote:
>>   
>>> Tarek Ziad? wrote:
>>>
>>> On Fri, Jan 9, 2009 at 2:12 AM, Gerry Reno <greno at verizon.net> wrote:
>>>
>>>
>>> The problem is that there appears to be no way to get a coordinated behavior
>>> between 'sdist' and 'bdist_rpm' as far as version and release strings are
>>> concerned that will satisfy both tarballs and bdist packages.
>>>
>>>
>>> If I understand your problem correclty, if sdist would simply
>>> concatenate the version string and
>>> the release string to use it as a "source version" when it starts to work,
>>> you would be able to work things out ?
>>>
>>>
>>> I don't know the internals of 'sdist' but I think if there were a way to
>>> extend 'sdist' to use 'release' as well as 'version' then that might work.
>>> I would have to test that to see.
>>>
>>>     
>>
>> Well, can you define how sdist should behave exactly ?
>>
>> Based on that discussion I can make a prototype for you to try out, then we can
>> maybe propose in that mailing list a change to sdist
>>   
> Thanks Tarek.  I think if it would do the same thing as bdist_rpm that 
> it would be ok.  bdist_rpm looks like it does VERSION-RELEASE (hyphen 
> separator).    So then doing this for 'sdist' I guess would produce a 
> tarball name of foo-VERSION-RELEASE.tar.gz  and an extracted directory 
> of foo-VERSION-RELEASE.   What this would allow then is for the 
> 'version' string to stay at '5.0.0' and then the 'release' string to 
> contain any pre-release information such as '0_rc1' and then the final 
> release would contain '1' which is lexically superior to the '0_rc1'.  
> I'm not sure though what other targets in distutils also use 'version' 
> so I don't know if this would affect anything else.    
Updating my comment:   Yes, and all the 'bdist' targets would have to do 
the same type of thing as 'bdist_rpm'.  That is use the combination of 
VERSION-RELEASE.

> Also, I'm hoping this can be implemented as some kind of extension so 
> that it can be made to work for existing installations as well.
>
> Regards,
> Gerry
>
> ------------------------------------------------------------------------
>
> _______________________________________________
> Distutils-SIG maillist  -  Distutils-SIG at python.org
> http://mail.python.org/mailman/listinfo/distutils-sig
>   

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20090109/e82f7762/attachment.htm>

From marius at pov.lt  Fri Jan  9 16:45:04 2009
From: marius at pov.lt (Marius Gedminas)
Date: Fri, 9 Jan 2009 17:45:04 +0200
Subject: [Distutils] [distutils] make the storage of the password
	optional in .pypirc
In-Reply-To: <e5fff6640901090717s74c9618ducf50a80177bd80c8@mail.gmail.com>
References: <94bdd2610901040404w6675999exfde5e81f49cbaf0d@mail.gmail.com>
	<4960BC4C.7060207@palladion.com>
	<94bdd2610901042100g50901aabvd04c67afa67e5710@mail.gmail.com>
	<94bdd2610901090032o40116765j96b7f2a68df3791d@mail.gmail.com>
	<51f97e530901090708w3105ecf3la220a32347ae126c@mail.gmail.com>
	<e5fff6640901090717s74c9618ducf50a80177bd80c8@mail.gmail.com>
Message-ID: <20090109154504.GA25799@fridge.pov.lt>

On Fri, Jan 09, 2009 at 10:17:50AM -0500, Benji York wrote:
> On Fri, Jan 9, 2009 at 10:08 AM, Stephen Emslie <stephenemslie at gmail.com> wrote:
> > A bit OT, but from your blog post on the subject:
> >
> >>I'd like to go further and to think about a ssh-agent like system, so there's no need
> >>to enter the pasword everytime you work with PyPI in the same session.
> >
> > Have you had any feedback on this yet?
> 
> Here's some:  how about instead of an ssh-like system, use ssh itself.  Front
> PyPI with an ssh server that users connect to.  That way it is both secure and
> the infrastructure (agent, etc.) is already in place.

Yes please.  I'd rather have one agent running and reuse my SSH key for
authentication.

Marius Gedminas
-- 
The gates in my computer are AND, OR and NOT; they are not Bill.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: Digital signature
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20090109/013b8824/attachment.pgp>

From ziade.tarek at gmail.com  Fri Jan  9 17:13:01 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Fri, 9 Jan 2009 17:13:01 +0100
Subject: [Distutils] pre-release versioning problems with sdist,
	bdist_rpm, bdist_debian
In-Reply-To: <4967704D.40005@verizon.net>
References: <4966A473.6090607@verizon.net>
	<94bdd2610901090036v2c4ba13em55bb32ba6087ed35@mail.gmail.com>
	<4967647B.1010706@verizon.net>
	<94bdd2610901090710kb7969eetf24786ae48ed3a0e@mail.gmail.com>
	<49676D2E.70001@verizon.net> <4967704D.40005@verizon.net>
Message-ID: <94bdd2610901090813l4a62cc6u7ad11cbb2c6c8727@mail.gmail.com>

On Fri, Jan 9, 2009 at 4:42 PM, Gerry Reno <greno at verizon.net> wrote:
> Thanks Tarek.  I think if it would do the same thing as bdist_rpm that it
> would be ok.  bdist_rpm looks like it does VERSION-RELEASE (hyphen
> separator).    So then doing this for 'sdist' I guess would produce a
> tarball name of foo-VERSION-RELEASE.tar.gz  and an extracted directory of
> foo-VERSION-RELEASE.   What this would allow then is for the 'version'
> string to stay at '5.0.0' and then the 'release' string to contain any
> pre-release information such as '0_rc1' and then the final release would
> contain '1' which is lexically superior to the '0_rc1'.  I'm not sure though
> what other targets in distutils also use 'version' so I don't know if this
> would affect anything else.
>
> Updating my comment:   Yes, and all the 'bdist' targets would have to do the
> same type of thing as 'bdist_rpm'.  That is use the combination of
> VERSION-RELEASE.
>
> Also, I'm hoping this can be implemented as some kind of extension so that
> it can be made to work for existing installations as well.

In other words, introduce it globally. That is a big change,

I think this could stay compatible with the previous installations as long as :

- if the release string is not specified, then it is not used at all,
  (unlike version which becomes 0.0 when not specified)

- the "Package-Version-Release" string is OK with the tools out there
  (I have to double-check on how setuptools and zc.buildout works on
fragments to extract version numbers  for instance)

I can't think of other issues at the moment, but I am pretty sure there are more

Regards
Tarek

-- 
Tarek Ziad? | Association AfPy | www.afpy.org
Blog FR | http://programmation-python.org
Blog EN | http://tarekziade.wordpress.com/

From ziade.tarek at gmail.com  Fri Jan  9 17:24:33 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Fri, 9 Jan 2009 17:24:33 +0100
Subject: [Distutils] [distutils] make the storage of the password
	optional in .pypirc
In-Reply-To: <20090109154504.GA25799@fridge.pov.lt>
References: <94bdd2610901040404w6675999exfde5e81f49cbaf0d@mail.gmail.com>
	<4960BC4C.7060207@palladion.com>
	<94bdd2610901042100g50901aabvd04c67afa67e5710@mail.gmail.com>
	<94bdd2610901090032o40116765j96b7f2a68df3791d@mail.gmail.com>
	<51f97e530901090708w3105ecf3la220a32347ae126c@mail.gmail.com>
	<e5fff6640901090717s74c9618ducf50a80177bd80c8@mail.gmail.com>
	<20090109154504.GA25799@fridge.pov.lt>
Message-ID: <94bdd2610901090824r5f13e43sc446665eaea146f3@mail.gmail.com>

On Fri, Jan 9, 2009 at 4:45 PM, Marius Gedminas <marius at pov.lt> wrote:
> On Fri, Jan 09, 2009 at 10:17:50AM -0500, Benji York wrote:
>> On Fri, Jan 9, 2009 at 10:08 AM, Stephen Emslie <stephenemslie at gmail.com> wrote:
>> > A bit OT, but from your blog post on the subject:
>> >
>> >>I'd like to go further and to think about a ssh-agent like system, so there's no need
>> >>to enter the pasword everytime you work with PyPI in the same session.
>> >
>> > Have you had any feedback on this yet?
>>
>> Here's some:  how about instead of an ssh-like system, use ssh itself.  Front
>> PyPI with an ssh server that users connect to.  That way it is both secure and
>> the infrastructure (agent, etc.) is already in place.
>
> Yes please.  I'd rather have one agent running and reuse my SSH key for
> authentication.

That would be awesome indeed. But that would involve quite some
changes on server side,
I'll forward this mail to catalog-sig for Richard, Martin and others's feedback

Regards
Tarek

-- 
Tarek Ziad? | Association AfPy | www.afpy.org
Blog FR | http://programmation-python.org
Blog EN | http://tarekziade.wordpress.com/

From jim at zope.com  Fri Jan  9 17:46:25 2009
From: jim at zope.com (Jim Fulton)
Date: Fri, 9 Jan 2009 11:46:25 -0500
Subject: [Distutils] [distutils] make the storage of the password
	optional in .pypirc
In-Reply-To: <e5fff6640901090717s74c9618ducf50a80177bd80c8@mail.gmail.com>
References: <94bdd2610901040404w6675999exfde5e81f49cbaf0d@mail.gmail.com>
	<4960BC4C.7060207@palladion.com>
	<94bdd2610901042100g50901aabvd04c67afa67e5710@mail.gmail.com>
	<94bdd2610901090032o40116765j96b7f2a68df3791d@mail.gmail.com>
	<51f97e530901090708w3105ecf3la220a32347ae126c@mail.gmail.com>
	<e5fff6640901090717s74c9618ducf50a80177bd80c8@mail.gmail.com>
Message-ID: <3B353D3B-BEC1-48CD-BE27-5CEEA8A1A812@zope.com>


On Jan 9, 2009, at 10:17 AM, Benji York wrote:

> On Fri, Jan 9, 2009 at 10:08 AM, Stephen Emslie <stephenemslie at gmail.com 
> > wrote:
>> A bit OT, but from your blog post on the subject:
>>
>>> I'd like to go further and to think about a ssh-agent like system,  
>>> so there's no need
>>> to enter the pasword everytime you work with PyPI in the same  
>>> session.
>>
>> Have you had any feedback on this yet?
>
> Here's some:  how about instead of an ssh-like system, use ssh  
> itself.  Front
> PyPI with an ssh server that users connect to.  That way it is both  
> secure and
> the infrastructure (agent, etc.) is already in place.

+1  Note that paramiko or the Twisted ssh support should make this  
fairly straightforward to implement. (I have need to build a custom  
ssh server soon for a project, and I'll find out if this statement is  
correct. :)

Jim

--
Jim Fulton
Zope Corporation



From exarkun at divmod.com  Fri Jan  9 18:00:26 2009
From: exarkun at divmod.com (Jean-Paul Calderone)
Date: Fri, 9 Jan 2009 12:00:26 -0500
Subject: [Distutils] [distutils] make the storage of the password
 optional in .pypirc
In-Reply-To: <3B353D3B-BEC1-48CD-BE27-5CEEA8A1A812@zope.com>
Message-ID: <20090109170026.20272.892705278.divmod.quotient.33888@ohm>

On Fri, 9 Jan 2009 11:46:25 -0500, Jim Fulton <jim at zope.com> wrote:
>
>On Jan 9, 2009, at 10:17 AM, Benji York wrote:
>>On Fri, Jan 9, 2009 at 10:08 AM, Stephen Emslie <stephenemslie at gmail.com > 
>>wrote:
>>>A bit OT, but from your blog post on the subject:
>>>>I'd like to go further and to think about a ssh-agent like system,  so 
>>>>there's no need
>>>>to enter the pasword everytime you work with PyPI in the same  session.
>>>
>>>Have you had any feedback on this yet?
>>
>>Here's some:  how about instead of an ssh-like system, use ssh  itself. 
>>Front
>>PyPI with an ssh server that users connect to.  That way it is both  secure 
>>and
>>the infrastructure (agent, etc.) is already in place.
>
>+1  Note that paramiko or the Twisted ssh support should make this  fairly 
>straightforward to implement. (I have need to build a custom  ssh server 
>soon for a project, and I'll find out if this statement is  correct. :)

I'd like to hear more about the details of how distutils/pypi would be
using SSH, but this is potentially a project I'd be willing to volunteer
some time to implement.

Jean-Paul

From ziade.tarek at gmail.com  Fri Jan  9 18:28:41 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Fri, 9 Jan 2009 18:28:41 +0100
Subject: [Distutils] [distutils] make the storage of the password
	optional in .pypirc
In-Reply-To: <20090109170026.20272.892705278.divmod.quotient.33888@ohm>
References: <3B353D3B-BEC1-48CD-BE27-5CEEA8A1A812@zope.com>
	<20090109170026.20272.892705278.divmod.quotient.33888@ohm>
Message-ID: <94bdd2610901090928n23644efg7e3cb357db646c80@mail.gmail.com>

On Fri, Jan 9, 2009 at 6:00 PM, Jean-Paul Calderone <exarkun at divmod.com> wrote:
> On Fri, 9 Jan 2009 11:46:25 -0500, Jim Fulton <jim at zope.com> wrote:
>>
>> +1  Note that paramiko or the Twisted ssh support should make this  fairly
>> straightforward to implement. (I have need to build a custom  ssh server
>> soon for a project, and I'll find out if this statement is  correct. :)
>
> I'd like to hear more about the details of how distutils/pypi would be
> using SSH, but this is potentially a project I'd be willing to volunteer
> some time to implement.

Great !

I'd like to mention also, that there's a Python Language Summit right
before Pycon in Chicago
where such a topic can be presented (eg upcoming projects in distutils and pypi)

and that I have asked for a sprint slot for distutils and pypi matters,

Regards
Tarek

-- 
Tarek Ziad? | Association AfPy | www.afpy.org
Blog FR | http://programmation-python.org
Blog EN | http://tarekziade.wordpress.com/

From greno at verizon.net  Fri Jan  9 20:57:23 2009
From: greno at verizon.net (Gerry Reno)
Date: Fri, 09 Jan 2009 14:57:23 -0500
Subject: [Distutils] pre-release versioning problems with sdist,
 bdist_rpm, bdist_debian
In-Reply-To: <94bdd2610901090813l4a62cc6u7ad11cbb2c6c8727@mail.gmail.com>
References: <4966A473.6090607@verizon.net>
	<94bdd2610901090036v2c4ba13em55bb32ba6087ed35@mail.gmail.com>
	<4967647B.1010706@verizon.net>
	<94bdd2610901090710kb7969eetf24786ae48ed3a0e@mail.gmail.com>
	<49676D2E.70001@verizon.net> <4967704D.40005@verizon.net>
	<94bdd2610901090813l4a62cc6u7ad11cbb2c6c8727@mail.gmail.com>
Message-ID: <4967AC23.6070305@verizon.net>

Tarek Ziad? wrote:
> On Fri, Jan 9, 2009 at 4:42 PM, Gerry Reno <greno at verizon.net> wrote:
>   
>> Thanks Tarek.  I think if it would do the same thing as bdist_rpm that it
>> would be ok.  bdist_rpm looks like it does VERSION-RELEASE (hyphen
>> separator).    So then doing this for 'sdist' I guess would produce a
>> tarball name of foo-VERSION-RELEASE.tar.gz  and an extracted directory of
>> foo-VERSION-RELEASE.   What this would allow then is for the 'version'
>> string to stay at '5.0.0' and then the 'release' string to contain any
>> pre-release information such as '0_rc1' and then the final release would
>> contain '1' which is lexically superior to the '0_rc1'.  I'm not sure though
>> what other targets in distutils also use 'version' so I don't know if this
>> would affect anything else.
>>
>> Updating my comment:   Yes, and all the 'bdist' targets would have to do the
>> same type of thing as 'bdist_rpm'.  That is use the combination of
>> VERSION-RELEASE.
>>
>> Also, I'm hoping this can be implemented as some kind of extension so that
>> it can be made to work for existing installations as well.
>>     
>
> In other words, introduce it globally. That is a big change,
>   
Yes, but it solves a big problem.  I've seen this issue brought up on 
the fedora-devel mailing list several times.  And Toshio has commented 
about it.  The RPM-based distros appear to ignore 'bdist_rpm' and the 
distro packagers go about creating their own way of building python 
RPMS.  And when I inquired about this, the pre-release versioning issue 
was the first thing that was mentioned.  I think this was the 
showstopper that caused them to abandon 'bdist_rpm' because they 
couldn't find sanity between 'sdist' and 'bdist' command treatment of 
'version' and 'release' strings.  If this gets fixed now, I think they 
could be persuaded to consider using 'bdist_rpm' for building python 
packages which would bring a lot of consistency to the process between 
distros.  (yes, I know distros all want to store things in different 
areas, but that's just a config/preference thing).

> I think this could stay compatible with the previous installations as long as :
>
> - if the release string is not specified, then it is not used at all,
>   (unlike version which becomes 0.0 when not specified)
>
> - the "Package-Version-Release" string is OK with the tools out there
>   (I have to double-check on how setuptools and zc.buildout works on
> fragments to extract version numbers  for instance)
>   
That sounds like it will work.

I'm looking forward to any prototype you could generate for this and 
I'll be glad to help test it.


Regards,
Gerry

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20090109/111e59b2/attachment.htm>

From martin at v.loewis.de  Fri Jan  9 21:18:20 2009
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Fri, 09 Jan 2009 21:18:20 +0100
Subject: [Distutils] [Catalog-sig] [distutils] make the storage of the
 password optional in .pypirc
In-Reply-To: <94bdd2610901090824r5f13e43sc446665eaea146f3@mail.gmail.com>
References: <94bdd2610901040404w6675999exfde5e81f49cbaf0d@mail.gmail.com>	<4960BC4C.7060207@palladion.com>	<94bdd2610901042100g50901aabvd04c67afa67e5710@mail.gmail.com>	<94bdd2610901090032o40116765j96b7f2a68df3791d@mail.gmail.com>	<51f97e530901090708w3105ecf3la220a32347ae126c@mail.gmail.com>	<e5fff6640901090717s74c9618ducf50a80177bd80c8@mail.gmail.com>	<20090109154504.GA25799@fridge.pov.lt>
	<94bdd2610901090824r5f13e43sc446665eaea146f3@mail.gmail.com>
Message-ID: <4967B10C.6030904@v.loewis.de>

>>> Here's some:  how about instead of an ssh-like system, use ssh itself.  Front
>>> PyPI with an ssh server that users connect to.  That way it is both secure and
>>> the infrastructure (agent, etc.) is already in place.
>> Yes please.  I'd rather have one agent running and reuse my SSH key for
>> authentication.
> 
> That would be awesome indeed. But that would involve quite some
> changes on server side,
> I'll forward this mail to catalog-sig for Richard, Martin and others's feedback

I'm fairly skeptical. First, the infrastructure is *not* yet in place.
Nobody has uploaded SSH keys to PyPI, and in order to allow SSH access,
we probably would need to create a Unix account, which then runs a fixed
(Python) program on ssh login. That is much less secure than the current
setup, in the sense that this program can probably tricked much easier
than Apache can. So it opens a door for people hacking into the system;
all they have to do is to create a fake PyPI account and upload an SSH
key...

To improve password storage, I think it would be better to use the
platform's secure password storage services where available (e.g.
OSX Keychain, KDE KWallet, etc). Of course, such a library should be
developed independently of distutils. For Keychain, there is already

http://muffinresearch.co.uk/archives/2008/02/05/python-keychainpy-access-to-the-mac-osx-keychain/

Regards,
Martin


From jim at zope.com  Fri Jan  9 21:57:55 2009
From: jim at zope.com (Jim Fulton)
Date: Fri, 9 Jan 2009 15:57:55 -0500
Subject: [Distutils] [Catalog-sig] [distutils] make the storage of the
	password optional in .pypirc
In-Reply-To: <4967B10C.6030904@v.loewis.de>
References: <94bdd2610901040404w6675999exfde5e81f49cbaf0d@mail.gmail.com>	<4960BC4C.7060207@palladion.com>	<94bdd2610901042100g50901aabvd04c67afa67e5710@mail.gmail.com>	<94bdd2610901090032o40116765j96b7f2a68df3791d@mail.gmail.com>	<51f97e530901090708w3105ecf3la220a32347ae126c@mail.gmail.com>	<e5fff6640901090717s74c9618ducf50a80177bd80c8@mail.gmail.com>	<20090109154504.GA25799@fridge.pov.lt>
	<94bdd2610901090824r5f13e43sc446665eaea146f3@mail.gmail.com>
	<4967B10C.6030904@v.loewis.de>
Message-ID: <9A77A80A-133F-47F7-AD3B-3CBDB206DE7B@zope.com>


On Jan 9, 2009, at 3:18 PM, Martin v. L?wis wrote:

>>>> Here's some:  how about instead of an ssh-like system, use ssh  
>>>> itself.  Front
>>>> PyPI with an ssh server that users connect to.  That way it is  
>>>> both secure and
>>>> the infrastructure (agent, etc.) is already in place.
>>> Yes please.  I'd rather have one agent running and reuse my SSH  
>>> key for
>>> authentication.
>>
>> That would be awesome indeed. But that would involve quite some
>> changes on server side,
>> I'll forward this mail to catalog-sig for Richard, Martin and  
>> others's feedback
>
> I'm fairly skeptical. First, the infrastructure is *not* yet in place.
> Nobody has uploaded SSH keys to PyPI,

Right. PyPI would have to grow the ability to manage public keys for  
users.

> and in order to allow SSH access,
> we probably would need to create a Unix account,

No, you would not.

> which then runs a fixed
> (Python) program on ssh login. That is much less secure than the  
> current
> setup, in the sense that this program can probably tricked much easier
> than Apache can. So it opens a door for people hacking into the  
> system;
> all they have to do is to create a fake PyPI account and upload an SSH
> key...

No. You'd have a new server process, written in Python using Twisted  
or paramiko,  that would would provide a small number of specialized  
commands and that would read public keys from the pypi database for  
authentication and update the database in response to commands,

Jim

--
Jim Fulton
Zope Corporation



From jim at zope.com  Fri Jan  9 22:02:53 2009
From: jim at zope.com (Jim Fulton)
Date: Fri, 9 Jan 2009 16:02:53 -0500
Subject: [Distutils] [Catalog-sig] [distutils] make the storage of the
	password optional in .pypirc
In-Reply-To: <9A77A80A-133F-47F7-AD3B-3CBDB206DE7B@zope.com>
References: <94bdd2610901040404w6675999exfde5e81f49cbaf0d@mail.gmail.com>	<4960BC4C.7060207@palladion.com>	<94bdd2610901042100g50901aabvd04c67afa67e5710@mail.gmail.com>	<94bdd2610901090032o40116765j96b7f2a68df3791d@mail.gmail.com>	<51f97e530901090708w3105ecf3la220a32347ae126c@mail.gmail.com>	<e5fff6640901090717s74c9618ducf50a80177bd80c8@mail.gmail.com>	<20090109154504.GA25799@fridge.pov.lt>
	<94bdd2610901090824r5f13e43sc446665eaea146f3@mail.gmail.com>
	<4967B10C.6030904@v.loewis.de>
	<9A77A80A-133F-47F7-AD3B-3CBDB206DE7B@zope.com>
Message-ID: <EA391517-5EB4-426A-81C9-2FBF276FC512@zope.com>


On Jan 9, 2009, at 3:57 PM, Jim Fulton wrote:
> No. You'd have a new server process, written in Python using Twisted  
> or paramiko,  that would would provide a small number of specialized  
> commands

Or better yet, supported scp.  Then the upload/register process would  
be reduced to just scp-ing a distro to pypi.  The server could read  
meta-data from the distro, register the release, if necessary, and put  
the distro in the right place.

Jim

--
Jim Fulton
Zope Corporation



From rayjohnterrill at gmail.com  Fri Jan  9 22:03:08 2009
From: rayjohnterrill at gmail.com (ray terrill)
Date: Fri, 9 Jan 2009 15:03:08 -0600
Subject: [Distutils] Unable to import python script after installing using
	easy_install
Message-ID: <557a21f40901091303y65e75895r901dd79dccb808c8@mail.gmail.com>

I'm using python2.4 to try to package and deliver a custom python script.
I'm unable to import the package.

My setup.py looks like the following:
from setuptools import setup, find_packages
setup(
   name = "randomscript",
   version = "1.0",
   packages = find_packages(),
)

I'm building the package using the following:
python setup.py bdist_egg
running bdist_egg
running egg_info
writing randomscript.egg-info/PKG-INFO
writing top-level names to randomscript.egg-info/top_level.txt
writing dependency_links to randomscript.egg-info/dependency_links.txt
reading manifest file 'randomscript.egg-info/SOURCES.txt'
writing manifest file 'randomscript.egg-info/SOURCES.txt'
installing library code to build/bdist.linux-i686/egg
running install_lib
warning: install_lib: 'build/lib' does not exist -- no Python modules to
install
creating build/bdist.linux-i686/egg
creating build/bdist.linux-i686/egg/EGG-INFO
copying randomscript.egg-info/PKG-INFO ->
build/bdist.linux-i686/egg/EGG-INFO
copying randomscript.egg-info/SOURCES.txt ->
build/bdist.linux-i686/egg/EGG-INFO
copying randomscript.egg-info/dependency_links.txt ->
build/bdist.linux-i686/egg/EGG-INFO
copying randomscript.egg-info/top_level.txt ->
build/bdist.linux-i686/egg/EGG-INFO
zip_safe flag not set; analyzing archive contents...
creating 'dist/randomscript-1.0-py2.4.egg' and adding
'build/bdist.linux-i686/egg' to it
removing 'build/bdist.linux-i686/egg' (and everything under it)

I'm installing the package using the following:
easy_install randomscript-1.0-py2.4.egg
Processing randomscript-1.0-py2.4.egg
Copying randomscript-1.0-py2.4.egg to /usr/lib/python2.4/site-packages
Adding randomscript 1.0 to easy-install.pth file

Installed /usr/lib/python2.4/site-packages/randomscript-1.0-py2.4.egg
Processing dependencies for randomscript==1.0
Finished processing dependencies for randomscript==1.0

If I attempt to use the script, I get an import error, but I do see the
script listed in my sys.path:
ImportError: No module named randomscript

['/home/oracle/python/dist',
'/usr/lib/python2.4/site-packages/setuptools-0.6c9-py2.4.egg',
'/usr/lib/python2.4/site-packages/randomscript-1.0-py2.4.egg',
'/usr/lib/python24.zip', '/usr/lib/python2.4',
'/usr/lib/python2.4/plat-linux2', '/usr/lib/python2.4/lib-tk',
'/usr/lib/python2.4/lib-dynload', '/usr/lib/python2.4/site-packages']

Also, if I cd to /usr/lib/python2.4/site-packages/, I see that my egg is
there, but is not a directory (should it be)?
-rw-r--r--   1 root root  819 2009-01-09 20:06 randomscript-1.0-py2.4.egg

Any help would be appreciated.
-Ray
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20090109/8ab37721/attachment.htm>

From martin at v.loewis.de  Fri Jan  9 22:03:25 2009
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Fri, 09 Jan 2009 22:03:25 +0100
Subject: [Distutils] [Catalog-sig] [distutils] make the storage of the
 password optional in .pypirc
In-Reply-To: <9A77A80A-133F-47F7-AD3B-3CBDB206DE7B@zope.com>
References: <94bdd2610901040404w6675999exfde5e81f49cbaf0d@mail.gmail.com>	<4960BC4C.7060207@palladion.com>	<94bdd2610901042100g50901aabvd04c67afa67e5710@mail.gmail.com>	<94bdd2610901090032o40116765j96b7f2a68df3791d@mail.gmail.com>	<51f97e530901090708w3105ecf3la220a32347ae126c@mail.gmail.com>	<e5fff6640901090717s74c9618ducf50a80177bd80c8@mail.gmail.com>	<20090109154504.GA25799@fridge.pov.lt>
	<94bdd2610901090824r5f13e43sc446665eaea146f3@mail.gmail.com>
	<4967B10C.6030904@v.loewis.de>
	<9A77A80A-133F-47F7-AD3B-3CBDB206DE7B@zope.com>
Message-ID: <4967BB9D.6070307@v.loewis.de>

> No. You'd have a new server process, written in Python using Twisted or
> paramiko,  that would would provide a small number of specialized
> commands and that would read public keys from the pypi database for
> authentication and update the database in response to commands,

Ok. I guess "contributions are welcome".

Regards,
Martin

From martin at v.loewis.de  Fri Jan  9 22:07:36 2009
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Fri, 09 Jan 2009 22:07:36 +0100
Subject: [Distutils] [Catalog-sig] [distutils] make the storage of the
 password optional in .pypirc
In-Reply-To: <EA391517-5EB4-426A-81C9-2FBF276FC512@zope.com>
References: <94bdd2610901040404w6675999exfde5e81f49cbaf0d@mail.gmail.com>	<4960BC4C.7060207@palladion.com>	<94bdd2610901042100g50901aabvd04c67afa67e5710@mail.gmail.com>	<94bdd2610901090032o40116765j96b7f2a68df3791d@mail.gmail.com>	<51f97e530901090708w3105ecf3la220a32347ae126c@mail.gmail.com>	<e5fff6640901090717s74c9618ducf50a80177bd80c8@mail.gmail.com>	<20090109154504.GA25799@fridge.pov.lt>
	<94bdd2610901090824r5f13e43sc446665eaea146f3@mail.gmail.com>
	<4967B10C.6030904@v.loewis.de>
	<9A77A80A-133F-47F7-AD3B-3CBDB206DE7B@zope.com>
	<EA391517-5EB4-426A-81C9-2FBF276FC512@zope.com>
Message-ID: <4967BC98.8070508@v.loewis.de>

> Or better yet, supported scp.  Then the upload/register process would be
> reduced to just scp-ing a distro to pypi.  The server could read
> meta-data from the distro, register the release, if necessary, and put
> the distro in the right place.

That wouldn't fit too well with the existing "register" and "upload"
commands, I think.

Regards,
Martin

From rayjohnterrill at gmail.com  Fri Jan  9 23:08:06 2009
From: rayjohnterrill at gmail.com (ray terrill)
Date: Fri, 9 Jan 2009 16:08:06 -0600
Subject: [Distutils] Way to ask distutils what version of a package is
	installed?
Message-ID: <557a21f40901091408y9f1ca6x90ddd76b6c15ca7d@mail.gmail.com>

Is there a way to ask distutils what version of a package is currently
installed?  I provided the version number when creating my setup.py, but
when this is deployed over 800+ clients, I'd like to have a way to
interrogate the system to determine which version is installed.

Thanks,
-Ray
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20090109/d3962a3f/attachment.htm>

From ziade.tarek at gmail.com  Sat Jan 10 11:20:59 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Sat, 10 Jan 2009 11:20:59 +0100
Subject: [Distutils] Way to ask distutils what version of a package is
	installed?
In-Reply-To: <557a21f40901091408y9f1ca6x90ddd76b6c15ca7d@mail.gmail.com>
References: <557a21f40901091408y9f1ca6x90ddd76b6c15ca7d@mail.gmail.com>
Message-ID: <94bdd2610901100220j92a01f7rfec34955bab7a2a0@mail.gmail.com>

On Fri, Jan 9, 2009 at 11:08 PM, ray terrill <rayjohnterrill at gmail.com> wrote:
> Is there a way to ask distutils what version of a package is currently
> installed?  I provided the version number when creating my setup.py, but
> when this is deployed over 800+ clients, I'd like to have a way to
> interrogate the system to determine which version is installed.
>

Unfortunately not by code, or not in an universal way I know of..

Distutils creates an YOUR-PACKAGE-VERSION.egg-info file in your
package-includes folder, that contains the information.

If you use setuptools though, it might be located in
YOUR-PACKAGE-VERSION.egg/EGG-INFO/PKG-INFO

Imho, I think the best way to get the version of your packages would
be to maintain a __version__ string in your package
to be able to query it from the code.

That said, it would make sense to include in Distutils an API that
would let people browse by the Metadata of the *.egg-info
files for installed packages.

Regards
Tarek

> Thanks,
> -Ray
>
> _______________________________________________
> Distutils-SIG maillist  -  Distutils-SIG at python.org
> http://mail.python.org/mailman/listinfo/distutils-sig
>
>



-- 
Tarek Ziad? | Association AfPy | www.afpy.org
Blog FR | http://programmation-python.org
Blog EN | http://tarekziade.wordpress.com/

From ziade.tarek at gmail.com  Sat Jan 10 11:35:48 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Sat, 10 Jan 2009 11:35:48 +0100
Subject: [Distutils] [Catalog-sig] [distutils] make the storage of the
	password optional in .pypirc
In-Reply-To: <4967BC98.8070508@v.loewis.de>
References: <94bdd2610901040404w6675999exfde5e81f49cbaf0d@mail.gmail.com>
	<94bdd2610901090032o40116765j96b7f2a68df3791d@mail.gmail.com>
	<51f97e530901090708w3105ecf3la220a32347ae126c@mail.gmail.com>
	<e5fff6640901090717s74c9618ducf50a80177bd80c8@mail.gmail.com>
	<20090109154504.GA25799@fridge.pov.lt>
	<94bdd2610901090824r5f13e43sc446665eaea146f3@mail.gmail.com>
	<4967B10C.6030904@v.loewis.de>
	<9A77A80A-133F-47F7-AD3B-3CBDB206DE7B@zope.com>
	<EA391517-5EB4-426A-81C9-2FBF276FC512@zope.com>
	<4967BC98.8070508@v.loewis.de>
Message-ID: <94bdd2610901100235o6c1544b5u4e94a0fe6111304@mail.gmail.com>

On Fri, Jan 9, 2009 at 10:07 PM, "Martin v. L?wis" <martin at v.loewis.de> wrote:
>> Or better yet, supported scp.  Then the upload/register process would be
>> reduced to just scp-ing a distro to pypi.  The server could read
>> meta-data from the distro, register the release, if necessary, and put
>> the distro in the right place.
>
> That wouldn't fit too well with the existing "register" and "upload"
> commands, I think.

+1

and in any case those commands should stay compatible with the current
mechanism and
let people store the password in the pypirc file if they want to, and
use https authentication.

Imho a scp/ssh protocol should be implemented in a new set of commands,


Regards
Tarek

>
> Regards,
> Martin
>



-- 
Tarek Ziad? | Association AfPy | www.afpy.org
Blog FR | http://programmation-python.org
Blog EN | http://tarekziade.wordpress.com/

From marius at pov.lt  Sat Jan 10 15:26:19 2009
From: marius at pov.lt (Marius Gedminas)
Date: Sat, 10 Jan 2009 16:26:19 +0200
Subject: [Distutils] Unable to import python script after installing
	using easy_install
In-Reply-To: <557a21f40901091303y65e75895r901dd79dccb808c8@mail.gmail.com>
References: <557a21f40901091303y65e75895r901dd79dccb808c8@mail.gmail.com>
Message-ID: <20090110142619.GA21901@fridge.pov.lt>

On Fri, Jan 09, 2009 at 03:03:08PM -0600, ray terrill wrote:
> I'm using python2.4 to try to package and deliver a custom python script.
> I'm unable to import the package.
> 
> My setup.py looks like the following:
> from setuptools import setup, find_packages
> setup(
>    name = "randomscript",
>    version = "1.0",
>    packages = find_packages(),
> )

And where's the rest of the sources?

> I'm building the package using the following:
> python setup.py bdist_egg

Some people say bdist_egg is not a good idea, and suggest using sdists
only, unless you're building a package for Windows (or MacOS), and that
package contains C extension modules that end-users will have trouble
compiling.

I am one of those people.

> I'm installing the package using the following:
> easy_install randomscript-1.0-py2.4.egg
> Processing randomscript-1.0-py2.4.egg
> Copying randomscript-1.0-py2.4.egg to /usr/lib/python2.4/site-packages
> Adding randomscript 1.0 to easy-install.pth file
> 
> Installed /usr/lib/python2.4/site-packages/randomscript-1.0-py2.4.egg
> Processing dependencies for randomscript==1.0
> Finished processing dependencies for randomscript==1.0
> 
> If I attempt to use the script, I get an import error, but I do see the
> script listed in my sys.path:
> ImportError: No module named randomscript

I suspect the .egg is empty (i.e. contains no Python sources and modules).
You probably need to specify a path for find_packages().

> Also, if I cd to /usr/lib/python2.4/site-packages/, I see that my egg is
> there, but is not a directory (should it be)?
> -rw-r--r--   1 root root  819 2009-01-09 20:06 randomscript-1.0-py2.4.egg

It's a zip file, which is often inconvenient (and also slow, as a
bonus).  You can ask easy_install to unzip it while installing.

> Any help would be appreciated.

I'm generally very opposed to cargo cult programming, but when dealing
with distutils/setuptools, the only way I manage to get something done
is to start from a working example, then copy, paste and modify.

Here's a simple example of a package that has only one source file in
the same directory as setup.py:

  http://mg.pov.lt/eazysvn/svn/trunk/

The setup.py doesn't use find_packages, but specifies the name of the
module in py_modules, and also defines a few entry points to get
executable scripts in /usr/bin.

Here's a more complicated example of a package, that has a Python
package (the naming clash is unfortunate) under src/, with some resource
files (*.png, etc.):

  http://mg.pov.lt/gtimelog/svn/

setup.py specifies the sources via packages, package_dir, and
package_data.

Marius Gedminas
-- 
For vi emulation of emacs, just type ":sh emacs" (without the quotes).
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: Digital signature
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20090110/941d003d/attachment.pgp>

From pje at telecommunity.com  Sat Jan 10 18:06:36 2009
From: pje at telecommunity.com (Phillip J. Eby)
Date: Sat, 10 Jan 2009 12:06:36 -0500
Subject: [Distutils] Unable to import python script after installing
 using easy_install
In-Reply-To: <557a21f40901091303y65e75895r901dd79dccb808c8@mail.gmail.co
 m>
References: <557a21f40901091303y65e75895r901dd79dccb808c8@mail.gmail.com>
Message-ID: <20090110170444.C0A493A4077@sparrow.telecommunity.com>

At 03:03 PM 1/9/2009 -0600, ray terrill wrote:
>I'm using python2.4 to try to package and deliver a custom python 
>script.  I'm unable to import the package.
>
>My setup.py looks like the following:
>from setuptools import setup, find_packages
>setup(
>    name = "randomscript",
>    version = "1.0",
>    packages = find_packages(),
>)
>
>I'm building the package using the following:
>python setup.py bdist_egg
>running bdist_egg
>running egg_info
>writing randomscript.egg-info/PKG-INFO
>writing top-level names to randomscript.egg-info/top_level.txt
>writing dependency_links to randomscript.egg-info/dependency_links.txt
>reading manifest file 'randomscript.egg-info/SOURCES.txt'
>writing manifest file 'randomscript.egg-info/SOURCES.txt'
>installing library code to build/bdist.linux-i686/egg
>running install_lib
>warning: install_lib: 'build/lib' does not exist -- no Python 
>modules to install

The above is an indication that your source code isn't being 
found.  My guess is that since you're saying you want to install a 
"script" but can't import it, what you actually have is a standalone 
*module*, not a script or a package.  find_packages() cannot find 
such modules, you must list them individually, e.g.:

    py_modules = ['randomscript'],

rather than listing them in the 'packages' parameter.

By the way, if you plan to distribute your module via PyPI, it is 
recommended that you use 'sdist' to build a source distribution in 
addition to (or instead of) an .egg file.  .egg files are a very 
specialized distribution format intended for things like application 
plugins and site-local distribution (e.g., administrators making 
binary packages available across a campus or company network, 
etc.)  That is, they're more aimed at end-user distribution use 
cases, than at programmer or "open source" distribution use 
cases.  Or as I sometimes say, they're more of a "deployment" format 
than a "distribution" format.


From pje at telecommunity.com  Sat Jan 10 18:09:56 2009
From: pje at telecommunity.com (Phillip J. Eby)
Date: Sat, 10 Jan 2009 12:09:56 -0500
Subject: [Distutils] Way to ask distutils what version of a package is
 installed?
In-Reply-To: <557a21f40901091408y9f1ca6x90ddd76b6c15ca7d@mail.gmail.com>
References: <557a21f40901091408y9f1ca6x90ddd76b6c15ca7d@mail.gmail.com>
Message-ID: <20090110170804.D5FFC3A4077@sparrow.telecommunity.com>

At 04:08 PM 1/9/2009 -0600, ray terrill wrote:
>Is there a way to ask distutils what version of a package is 
>currently installed?  I provided the version number when creating my 
>setup.py, but when this is deployed over 800+ clients, I'd like to 
>have a way to interrogate the system to determine which version is installed.

Distutils has no way to do this, but setuptools does.  You can get 
the version number of "SomePackage" using:

     pkg_resources.require("SomePackage")[0].version

Note that "SomePackage" is the name of the *project* (i.e., what its 
PyPI name and distribution filename are based on), not an individual 
Python module or package or __version__ string within that package.

Also, for this API call to work, "SomePackage" must have been 
installed by setuptools, or by Python 2.5's distutils (which installs 
the metadata setuptools uses for version detection).

(You can consult the pkg_resources and setuptools documentation for 
more details on these matters.)


From ziade.tarek at gmail.com  Sat Jan 10 18:21:11 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Sat, 10 Jan 2009 18:21:11 +0100
Subject: [Distutils] Way to ask distutils what version of a package is
	installed?
In-Reply-To: <20090110170804.D5FFC3A4077@sparrow.telecommunity.com>
References: <557a21f40901091408y9f1ca6x90ddd76b6c15ca7d@mail.gmail.com>
	<20090110170804.D5FFC3A4077@sparrow.telecommunity.com>
Message-ID: <94bdd2610901100921t2fc70387u942d3fa962a6a68@mail.gmail.com>

On Sat, Jan 10, 2009 at 6:09 PM, Phillip J. Eby <pje at telecommunity.com> wrote:
> At 04:08 PM 1/9/2009 -0600, ray terrill wrote:
>>
>> Is there a way to ask distutils what version of a package is currently
>> installed?  I provided the version number when creating my setup.py, but
>> when this is deployed over 800+ clients, I'd like to have a way to
>> interrogate the system to determine which version is installed.
>
> Distutils has no way to do this, but setuptools does.  You can get the
> version number of "SomePackage" using:
>
>    pkg_resources.require("SomePackage")[0].version
>
> Note that "SomePackage" is the name of the *project* (i.e., what its PyPI
> name and distribution filename are based on), not an individual Python
> module or package or __version__ string within that package.
>
> Also, for this API call to work, "SomePackage" must have been installed by
> setuptools, or by Python 2.5's distutils (which installs the metadata
> setuptools uses for version detection).
>
> (You can consult the pkg_resources and setuptools documentation for more
> details on these matters.)
>

Why we wouldn't add in Distutils a simplified API for this since it is
already taking care of
creating the .egg-info ?

So people don't have to install an extra tool (setutptools) to get the
version of an package
installed by distutils.

A simpler API that pkg_resources's one, something like:

>>> from distutils import get_package_version
>>> get_package_version('SomePackage')


Regards
Tarek
-- 
Tarek Ziad? | Association AfPy | www.afpy.org
Blog FR | http://programmation-python.org
Blog EN | http://tarekziade.wordpress.com/

From ziade.tarek at gmail.com  Sat Jan 10 18:50:35 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Sat, 10 Jan 2009 18:50:35 +0100
Subject: [Distutils] Way to ask distutils what version of a package is
	installed?
In-Reply-To: <94bdd2610901100921t2fc70387u942d3fa962a6a68@mail.gmail.com>
References: <557a21f40901091408y9f1ca6x90ddd76b6c15ca7d@mail.gmail.com>
	<20090110170804.D5FFC3A4077@sparrow.telecommunity.com>
	<94bdd2610901100921t2fc70387u942d3fa962a6a68@mail.gmail.com>
Message-ID: <94bdd2610901100950u5a71118ekf6e3576684f28409@mail.gmail.com>

On Sat, Jan 10, 2009 at 6:21 PM, Tarek Ziad? <ziade.tarek at gmail.com> wrote:
>
> A simpler API that pkg_resources's one, something like:
>
>>>> from distutils import get_package_version
>>>> get_package_version('SomePackage')
>

A generic API could be ;

>>> from distutils import get_metadata
>>> get_metadata('SomePackage')
{'Name' : 'SomePackage', 'Version': 'xxx', ...}

Regards
Tarek

From rayjohnterrill at gmail.com  Sat Jan 10 19:21:49 2009
From: rayjohnterrill at gmail.com (rayterrill)
Date: Sat, 10 Jan 2009 10:21:49 -0800 (PST)
Subject: [Distutils] Unable to import python script after installing
 using easy_install
In-Reply-To: <20090110170444.C0A493A4077@sparrow.telecommunity.com>
References: <557a21f40901091303y65e75895r901dd79dccb808c8@mail.gmail.com>
	<20090110170444.C0A493A4077@sparrow.telecommunity.com>
Message-ID: <21390428.post@talk.nabble.com>


>The above is an indication that your source code isn't being 
>found.  My guess is that since you're saying you want to install a 
>"script" but can't import it, what you actually have is a standalone 
>*module*, not a script or a package.  find_packages() cannot find 
>such modules, you must list them individually, e.g.:

>    py_modules = ['randomscript'],

>rather than listing them in the 'packages' parameter.

I wasnt aware of this option.  This looks a lot more like what I need.

>By the way, if you plan to distribute your module via PyPI, it is 
>recommended that you use 'sdist' to build a source distribution in 
>addition to (or instead of) an .egg file.  .egg files are a very 
>specialized distribution format intended for things like application 
>plugins and site-local distribution (e.g., administrators making 
>binary packages available across a campus or company network, 
>etc.)  That is, they're more aimed at end-user distribution use 
>cases, than at programmer or "open source" distribution use 
>cases.  Or as I sometimes say, they're more of a "deployment" format 
>than a "distribution" format.

This is custom business-specific code, not intended for general use or being
uploaded to PyPI.  They're going to be distributed to over 800+ locations
internally, so I was looking for a repeatable structured mechanism to deploy
the code and verify that it was installed correctly, vs. just copying the
files into the python sys.path and hoping they're there when they need to be
run (how it currently is working).

-Ray
-- 
View this message in context: http://www.nabble.com/Unable-to-import-python-script-after-installing-using-easy_install-tp21380532p21390428.html
Sent from the Python - distutils-sig mailing list archive at Nabble.com.


From pje at telecommunity.com  Sat Jan 10 21:00:49 2009
From: pje at telecommunity.com (Phillip J. Eby)
Date: Sat, 10 Jan 2009 15:00:49 -0500
Subject: [Distutils] Unable to import python script after installing
 using easy_install
In-Reply-To: <21390428.post@talk.nabble.com>
References: <557a21f40901091303y65e75895r901dd79dccb808c8@mail.gmail.com>
	<20090110170444.C0A493A4077@sparrow.telecommunity.com>
	<21390428.post@talk.nabble.com>
Message-ID: <20090110195902.CB4623A40A0@sparrow.telecommunity.com>

At 10:21 AM 1/10/2009 -0800, rayterrill wrote:
>They're going to be distributed to over 800+ locations
>internally, so I was looking for a repeatable structured mechanism to deploy
>the code and verify that it was installed correctly, vs. just copying the
>files into the python sys.path and hoping they're there when they need to be
>run (how it currently is working).

In that case, an egg is a good choice.


From pje at telecommunity.com  Sat Jan 10 21:09:01 2009
From: pje at telecommunity.com (Phillip J. Eby)
Date: Sat, 10 Jan 2009 15:09:01 -0500
Subject: [Distutils] get_metadata in Distutils
In-Reply-To: <94bdd2610901100948t65cba1achc2764c97195f529d@mail.gmail.co
 m>
References: <94bdd2610901100948t65cba1achc2764c97195f529d@mail.gmail.com>
Message-ID: <20090110200709.D3B3B3A40A0@sparrow.telecommunity.com>

At 06:48 PM 1/10/2009 +0100, Tarek Ziad? wrote:
>Hello,
>
>I am going to work on a change in Distutils to include a get_metadata
>API, http://bugs.python.org/issue4908
>
>I don't see yet if this can be hard to do, and I will probably
>discover it while doing it,
>
>Since this is basically what has been done in setuptools, I thaught
>that you might wanted to help around for this change ?

I'd suggest looking at the pkg_resources code, particularly 
find_distributions and its related functions.  Note, for example, 
that egg layouts can be nested (even when zipped!), and that the 
metadata can have different filenames, depending on the installation 
format.  Version parsing also has certain peculiarities, which also 
means that people doing simple string comparisons on the version 
field is probably not going to suffice.


From rayjohnterrill at gmail.com  Sat Jan 10 22:59:20 2009
From: rayjohnterrill at gmail.com (rayterrill)
Date: Sat, 10 Jan 2009 13:59:20 -0800 (PST)
Subject: [Distutils] Unable to import python script after installing
 using easy_install
In-Reply-To: <21390428.post@talk.nabble.com>
References: <557a21f40901091303y65e75895r901dd79dccb808c8@mail.gmail.com>
	<20090110170444.C0A493A4077@sparrow.telecommunity.com>
	<21390428.post@talk.nabble.com>
Message-ID: <21393325.post@talk.nabble.com>


>The above is an indication that your source code isn't being 
>found.  My guess is that since you're saying you want to install a 
>"script" but can't import it, what you actually have is a standalone 
>*module*, not a script or a package.  find_packages() cannot find 
>such modules, you must list them individually, e.g.:

>    py_modules = ['randomscript'],

>rather than listing them in the 'packages' parameter.

I changed the setup.py to use the py_modules parameter vs. the
find_packages() parameter, and it does work correctly now.
-Ray
-- 
View this message in context: http://www.nabble.com/Unable-to-import-python-script-after-installing-using-easy_install-tp21380532p21393325.html
Sent from the Python - distutils-sig mailing list archive at Nabble.com.


From tseaver at palladion.com  Sun Jan 11 01:40:54 2009
From: tseaver at palladion.com (Tres Seaver)
Date: Sat, 10 Jan 2009 19:40:54 -0500
Subject: [Distutils] [Catalog-sig] [distutils] make the storage of the
 password optional in .pypirc
In-Reply-To: <4967B10C.6030904@v.loewis.de>
References: <94bdd2610901040404w6675999exfde5e81f49cbaf0d@mail.gmail.com>	<4960BC4C.7060207@palladion.com>	<94bdd2610901042100g50901aabvd04c67afa67e5710@mail.gmail.com>	<94bdd2610901090032o40116765j96b7f2a68df3791d@mail.gmail.com>	<51f97e530901090708w3105ecf3la220a32347ae126c@mail.gmail.com>	<e5fff6640901090717s74c9618ducf50a80177bd80c8@mail.gmail.com>	<20090109154504.GA25799@fridge.pov.lt>	<94bdd2610901090824r5f13e43sc446665eaea146f3@mail.gmail.com>
	<4967B10C.6030904@v.loewis.de>
Message-ID: <49694016.8080302@palladion.com>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Martin v. L?wis wrote:
>>>> Here's some:  how about instead of an ssh-like system, use ssh itself.  Front
>>>> PyPI with an ssh server that users connect to.  That way it is both secure and
>>>> the infrastructure (agent, etc.) is already in place.
>>> Yes please.  I'd rather have one agent running and reuse my SSH key for
>>> authentication.
>> That would be awesome indeed. But that would involve quite some
>> changes on server side,
>> I'll forward this mail to catalog-sig for Richard, Martin and others's feedback
> 
> I'm fairly skeptical. First, the infrastructure is *not* yet in place.
> Nobody has uploaded SSH keys to PyPI, and in order to allow SSH access,
> we probably would need to create a Unix account, which then runs a fixed
> (Python) program on ssh login.

Right, a single account with multiple keys (each with 'command='do_pypi
- -u <userid>').

> That is much less secure than the current
> setup, in the sense that this program can probably tricked much easier
> than Apache can. So it opens a door for people hacking into the system;
> all they have to do is to create a fake PyPI account and upload an SSH
> key...

Zope has been using the 'command=' bit to do SSH-protected CVS / SVN
access since 2000 with a lot of success;  370+ committers have keys on
the system.  The command being executed is actually a small shell
script, which barfs if the program being run is not one of 'svn', 'cvs',
or 'scp' (for uploading tarballs).

> To improve password storage, I think it would be better to use the
> platform's secure password storage services where available (e.g.
> OSX Keychain, KDE KWallet, etc). Of course, such a library should be
> developed independently of distutils. For Keychain, there is already
> 
> http://muffinresearch.co.uk/archives/2008/02/05/python-keychainpy-access-to-the-mac-osx-keychain/

Not only are PyPI passwords stored in the clear on user's hard drives,
they are sent in the clear on every authenticated request to the web
interface (basic auth over unencrypted HTTP):  it seems to me we ought
to worry about both those issues more.


Tres.
- --
===================================================================
Tres Seaver          +1 540-429-0999          tseaver at palladion.com
Palladion Software   "Excellence by Design"    http://palladion.com
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFJaUAW+gerLs4ltQ4RAhFXAJ47WOzMAe12m+YD5BNu22BzTU+QRQCeLTbX
DSaVk1I96K5mzaZro98HUTU=
=8sRs
-----END PGP SIGNATURE-----


From ziade.tarek at gmail.com  Sun Jan 11 20:22:31 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Sun, 11 Jan 2009 20:22:31 +0100
Subject: [Distutils] get_metadata in Distutils
In-Reply-To: <20090110200709.D3B3B3A40A0@sparrow.telecommunity.com>
References: <94bdd2610901100948t65cba1achc2764c97195f529d@mail.gmail.com>
	<20090110200709.D3B3B3A40A0@sparrow.telecommunity.com>
Message-ID: <94bdd2610901111122y156b691dh1db438a1b6fedc14@mail.gmail.com>

On Sat, Jan 10, 2009 at 9:09 PM, Phillip J. Eby <pje at telecommunity.com> wrote:
>>
>> Since this is basically what has been done in setuptools, I thaught
>> that you might wanted to help around for this change ?
>
> I'd suggest looking at the pkg_resources code, particularly
> find_distributions and its related functions.

Ok thanks,

> Note, for example, that egg
> layouts can be nested (even when zipped!), and that the metadata can have
> different filenames, depending on the installation format.

This is my understanding at this stage :
>From a Distutils point of view, it seems that this would suffice to
read the metadata from the egg-info files :

1/ add in distutils.dist.DistributionMetadata a new method to be able
to load an existing egg-info file
2/ add in the pkgutil module, the get_metadata function, that could
have this signature:

   get_metadata(package_name, path_item)

where path_item is a site-packages like directory,

this function would scan the path_item directory, like what
setuptools.pkg_resource code does,
and return the metadata (+fill a cache with all packages metadata found)

> Version parsing
> also has certain peculiarities, which also means that people doing simple
> string comparisons on the version field is probably not going to suffice.

Wouldn't distutils.versionpredicate be useful here ?

Regards
Tarek

>
>



-- 
Tarek Ziad? | Association AfPy | www.afpy.org
Blog FR | http://programmation-python.org
Blog EN | http://tarekziade.wordpress.com/

From pje at telecommunity.com  Mon Jan 12 04:01:22 2009
From: pje at telecommunity.com (Phillip J. Eby)
Date: Sun, 11 Jan 2009 22:01:22 -0500
Subject: [Distutils] get_metadata in Distutils
In-Reply-To: <94bdd2610901111122y156b691dh1db438a1b6fedc14@mail.gmail.co
 m>
References: <94bdd2610901100948t65cba1achc2764c97195f529d@mail.gmail.com>
	<20090110200709.D3B3B3A40A0@sparrow.telecommunity.com>
	<94bdd2610901111122y156b691dh1db438a1b6fedc14@mail.gmail.com>
Message-ID: <20090112025933.411B63A40A0@sparrow.telecommunity.com>

At 08:22 PM 1/11/2009 +0100, Tarek Ziad? wrote:
>On Sat, Jan 10, 2009 at 9:09 PM, Phillip J. Eby <pje at telecommunity.com> wrote:
> >>
> >> Since this is basically what has been done in setuptools, I thaught
> >> that you might wanted to help around for this change ?
> >
> > I'd suggest looking at the pkg_resources code, particularly
> > find_distributions and its related functions.
>
>Ok thanks,
>
> > Note, for example, that egg
> > layouts can be nested (even when zipped!), and that the metadata can have
> > different filenames, depending on the installation format.
>
>This is my understanding at this stage :
> From a Distutils point of view, it seems that this would suffice to
>read the metadata from the egg-info files :
>
>1/ add in distutils.dist.DistributionMetadata a new method to be able
>to load an existing egg-info file
>2/ add in the pkgutil module, the get_metadata function, that could
>have this signature:
>
>    get_metadata(package_name, path_item)
>
>where path_item is a site-packages like directory,
>
>this function would scan the path_item directory, like what
>setuptools.pkg_resource code does,
>and return the metadata (+fill a cache with all packages metadata found)

My point was that if you don't support .egg files or 
EGG-INFO/PKG-INFO, then your API is rather crippled; there is no 
reason for anyone using setuptools to use it in place of the 
corresponding pkg_resources API, as it simply will not work with 
packages installed by easy_install.


> > Version parsing
> > also has certain peculiarities, which also means that people doing simple
> > string comparisons on the version field is probably not going to suffice.
>
>Wouldn't distutils.versionpredicate be useful here ?

Only if the package in question follows distutils' rules, which 
aren't as flexible as those of setuptools...  and setuptools' version 
parsing system was designed to handle a significant number of 
versioning schemes that were in use on PyPI at the time -- schemes 
that the distutils mostly couldn't parse correctly.

I'm just trying to warn you that adding these features in a 
distutils-only way could easily end up like the PKG-INFO "requires" 
and "provides" fields -- that is, theoretically useful but of no 
practical value to anyone.  In fact, they could be "attractive 
nuisances" in that they would encourage people to use them, only to 
find out later that it was a complete dead end because of the lack of 
support for anything but pure-distutils packages.

Of course, they will then blame me and setuptools rather than you and 
the distutils, but I suppose I'm used to that.  ;-)


From jorge.vargas at gmail.com  Mon Jan 12 08:00:09 2009
From: jorge.vargas at gmail.com (Jorge Vargas)
Date: Mon, 12 Jan 2009 01:00:09 -0600
Subject: [Distutils] using 'python setup.py test' with an index?
Message-ID: <32822fe60901112300t2cc16f88p14bacd48efaebd0e@mail.gmail.com>

Hi,

I got the following situation. I have a big package which depends on
several others projects just for unit tests, the package is a maskup
of several others. Therefore I'm adding those dependencies to
tests_require, and everything is working great. Now I have stumble
upon a package that has no setup.py (they install with configure/make)
but someone "forked" it and releases a tarball/egg version on a
private index. Therefore my question, how can I add this test_require
dep, and tell it to fetch that one only from that index?

In case you were wondering what the project it, it's a (not released
yet) set of "helpers" to mount wsgi apps "with one liners" into a
bigger TurboGears2 application. The current (and temporal) hg
repository is here http://toscawidgets.org/hg/tgext.wsgiapps/, we
currently have WSGIControllers for trac, mercurial, TileCache and
Zine, and the project is less than 48hrs old!

From ziade.tarek at gmail.com  Mon Jan 12 14:11:09 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Mon, 12 Jan 2009 14:11:09 +0100
Subject: [Distutils] get_metadata in Distutils
In-Reply-To: <20090112025933.411B63A40A0@sparrow.telecommunity.com>
References: <94bdd2610901100948t65cba1achc2764c97195f529d@mail.gmail.com>
	<20090110200709.D3B3B3A40A0@sparrow.telecommunity.com>
	<94bdd2610901111122y156b691dh1db438a1b6fedc14@mail.gmail.com>
	<20090112025933.411B63A40A0@sparrow.telecommunity.com>
Message-ID: <94bdd2610901120511g28130ef5se249e0a87f4eb2ab@mail.gmail.com>

On Mon, Jan 12, 2009 at 4:01 AM, Phillip J. Eby <pje at telecommunity.com> wrote:
> At 08:22 PM 1/11/2009 +0100, Tarek Ziad? wrote:
>>
>> On Sat, Jan 10, 2009 at 9:09 PM, Phillip J. Eby <pje at telecommunity.com>
>> wrote:
>> >>
>> >> Since this is basically what has been done in setuptools, I thaught
>> >> that you might wanted to help around for this change ?
>> >
>> > I'd suggest looking at the pkg_resources code, particularly
>> > find_distributions and its related functions.
>>
>> Ok thanks,
>>
>> > Note, for example, that egg
>> > layouts can be nested (even when zipped!), and that the metadata can
>> > have
>> > different filenames, depending on the installation format.
>>
>> This is my understanding at this stage :
>> From a Distutils point of view, it seems that this would suffice to
>> read the metadata from the egg-info files :
>>
>> 1/ add in distutils.dist.DistributionMetadata a new method to be able
>> to load an existing egg-info file
>> 2/ add in the pkgutil module, the get_metadata function, that could
>> have this signature:
>>
>>   get_metadata(package_name, path_item)
>>
>> where path_item is a site-packages like directory,
>>
>> this function would scan the path_item directory, like what
>> setuptools.pkg_resource code does,
>> and return the metadata (+fill a cache with all packages metadata found)
>
> My point was that if you don't support .egg files or EGG-INFO/PKG-INFO, then
> your API is rather crippled; there is no reason for anyone using setuptools
> to use it in place of the corresponding pkg_resources API, as it simply will
> not work with packages installed by easy_install.

Ok, I will introduce in my patch the other formats. I am wondering though, how
far this would be from an integration of pkg_resources into Distutils/pkgutil,
with some API on the top and if it makes sense.

The only difference I can see is that I am working on an API that would return
DistributionMetadata instances

http://bugs.python.org/file12692/get_metadata.diff

>
>
>> > Version parsing
>> > also has certain peculiarities, which also means that people doing
>> > simple
>> > string comparisons on the version field is probably not going to
>> > suffice.
>>
>> Wouldn't distutils.versionpredicate be useful here ?
>
> Only if the package in question follows distutils' rules, which aren't as
> flexible as those of setuptools...  and setuptools' version parsing system
> was designed to handle a significant number of versioning schemes that were
> in use on PyPI at the time -- schemes that the distutils mostly couldn't
> parse correctly.

I'd say then : if setutpools version parsing system is superior to
distutils.versionpredicate,
wouldn't it make sense to merge it into Distutils ?

Maybe that was the initial plans ?

>
> I'm just trying to warn you that adding these features in a distutils-only
> way could easily end up like the PKG-INFO "requires" and "provides" fields
> -- that is, theoretically useful but of no practical value to anyone.  In
> fact, they could be "attractive nuisances" in that they would encourage
> people to use them, only to find out later that it was a complete dead end
> because of the lack of support for anything but pure-distutils packages.
>
> Of course, they will then blame me and setuptools rather than you and the
> distutils, but I suppose I'm used to that.  ;-)
>

Well, being able to query the installed version for a package seem to
me like a feature
that should live in Distutils or pkgutil

In the same way, being able to do version arithmetic is rather
important, and it would be better
ihmo to have one and only one way to deal with that in Python,

What are you plans for Setuptools development for the future ?

On Distutils side, I am trying to list what could be done and propose
a list here,

Regards,

Tarek

-- 
Tarek Ziad? | Association AfPy | www.afpy.org
Blog FR | http://programmation-python.org
Blog EN | http://tarekziade.wordpress.com/

From pje at telecommunity.com  Mon Jan 12 17:07:34 2009
From: pje at telecommunity.com (Phillip J. Eby)
Date: Mon, 12 Jan 2009 11:07:34 -0500
Subject: [Distutils] get_metadata in Distutils
In-Reply-To: <94bdd2610901120511g28130ef5se249e0a87f4eb2ab@mail.gmail.co
 m>
References: <94bdd2610901100948t65cba1achc2764c97195f529d@mail.gmail.com>
	<20090110200709.D3B3B3A40A0@sparrow.telecommunity.com>
	<94bdd2610901111122y156b691dh1db438a1b6fedc14@mail.gmail.com>
	<20090112025933.411B63A40A0@sparrow.telecommunity.com>
	<94bdd2610901120511g28130ef5se249e0a87f4eb2ab@mail.gmail.com>
Message-ID: <20090112160544.F25853A40A0@sparrow.telecommunity.com>

At 02:11 PM 1/12/2009 +0100, Tarek Ziad? wrote:
>Ok, I will introduce in my patch the other formats. I am wondering though, how
>far this would be from an integration of pkg_resources into Distutils/pkgutil,
>with some API on the top and if it makes sense.

It makes sense to me, I just wanted you to be aware how deep a job it 
is.  You're probably going to need a generic function by importer 
type, just like the others in pkgutil.


>The only difference I can see is that I am working on an API that would return
>DistributionMetadata instances
>
>http://bugs.python.org/file12692/get_metadata.diff

Yep...  which means that you're going to have to reinvent (and 
re-test) a fair number of wheels to get there.  pkg_resources just 
lets you ask for a distribution's PKG-INFO metadata, and get it back 
as a string, regardless of what egg format is in use (including 
distutils).  But to *just* support PKG-INFO, the irony is that you'll 
have to recreate a fair amount of stuff.


>I'd say then : if setutpools version parsing system is superior to
>distutils.versionpredicate,

I'm not actually familiar with "versionpredicate", so I couldn't 
say.  It's definitely superior to LooseVersion and StrictVersion, 
which were the only version tools available in the distutils when I 
started all this.


>wouldn't it make sense to merge it into Distutils ?
>
>Maybe that was the initial plans ?

Yes, and yes.  Setuptools version parsing (and name/version escaping) 
functions at least doesn't have any dependencies on anything 
else.  So that's an easy port.  If you want Requirement objects as 
well (specifying various version ranges), a bit more gets pulled 
in.  However, if all you want is to be able to turn versions into 
opaquely-comparable values, parse_version() and its dependencies are 
all you need.

(By the way, I previously proposed PEP 365 -- that pkg_resources 
simply be bundled in the stdlib -- to address these needs.)


>Well, being able to query the installed version for a package seem to
>me like a feature
>that should live in Distutils or pkgutil
>
>In the same way, being able to do version arithmetic is rather
>important, and it would be better
>ihmo to have one and only one way to deal with that in Python,
>
>What are you plans for Setuptools development for the future ?

I hope someday to have some time for it again.  :)


From bl8cki at gmail.com  Tue Jan 13 13:05:56 2009
From: bl8cki at gmail.com (bl8cki)
Date: Tue, 13 Jan 2009 14:05:56 +0200
Subject: [Distutils] Changing distribution versions runtime
Message-ID: <e9db7f440901130405n5eda91cbif257b78f553a5304@mail.gmail.com>

Hello all,

I would like at runtime to change the used version of installed egg ... (for
example based on some condition import specific version)

and the only solution i end up with is to remove from the 'working_set'
'by_key' hash the best matched version, added when importing pkg_resources,
and then 'resolve' the wished one and add it. otherwise i end up with
version conflict.

I do not want to add version in package naming (package_0_1) ... but that
would be the fastest solution :)

What's the best thing i can do?
Thanks!
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20090113/89b2bd9e/attachment.htm>

From amitsaha.in at gmail.com  Wed Jan 14 10:20:30 2009
From: amitsaha.in at gmail.com (Amit k. Saha)
Date: Wed, 14 Jan 2009 14:50:30 +0530
Subject: [Distutils] Building Python Eggs
Message-ID: <547db2260901140120l2ce45a0co37efd508fd44d8bd@mail.gmail.com>

Hi all,

I am just learning the basics of building Python eggs using
'setuptools'. What I have is the following code structure in a
top-down fashion:


- demo1.py
- __init.py__
- setup.py
-subpackage1
--__init.py/__


When I execute 'python setup.py bdist_egg' from the top-level
directory which has the 'setup.py' file:

__author__="amit"
__date__ ="$14 Jan, 2009 2:37:34 PM$"

from setuptools import setup,find_packages

setup (
  name = 'EggDemo',
  version = '0.1',
  Summary = 'Just another Python package for the cheese shop',
  author = 'amit',
  author_email = '',
  url = '',
  license = '',
  description= 'Long description of the package',
  platform= '',
  packages = find_packages(),
)

In the resulting, SOURCES.txt file, I have:

setup.py
EggDemo.egg-info/PKG-INFO
EggDemo.egg-info/SOURCES.txt
EggDemo.egg-info/dependency_links.txt
EggDemo.egg-info/top_level.txt
subpackage1/__init__.py


As you can see, the top-level 'demo1.py' is missing, even though I
have a '__init.py__' file there.

The manual at http://peak.telecommunity.com/DevCenter/setuptools#using-find-packages
makes me believe, this should work. Where am I going wrong?

Best,
Amit
-- 
Amit Kumar Saha
http://amitksaha.blogspot.com
http://amitsaha.in.googlepages.com/
*Bangalore Open Java Users Group*:http:www.bojug.in

From lists at zopyx.com  Wed Jan 14 10:23:10 2009
From: lists at zopyx.com (Andreas Jung)
Date: Wed, 14 Jan 2009 10:23:10 +0100
Subject: [Distutils] Building Python Eggs
In-Reply-To: <547db2260901140120l2ce45a0co37efd508fd44d8bd@mail.gmail.com>
References: <547db2260901140120l2ce45a0co37efd508fd44d8bd@mail.gmail.com>
Message-ID: <496DAEFE.3030602@zopyx.com>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 14.01.2009 10:20 Uhr, Amit k. Saha wrote:
> Hi all,
> 
> I am just learning the basics of building Python eggs using
> 'setuptools'. What I have is the following code structure in a
> top-down fashion:
> 
> 
> - demo1.py
> - __init.py__
> - setup.py
> -subpackage1
> --__init.py/__
> 
> 
> When I execute 'python setup.py bdist_egg' from the top-level
> directory which has the 'setup.py' file:

Using 'sdist' is usually good-enough unless you are working on binary
distributions. 'sdist' usually includes everything - especially stuff
under revision control (at least when using SVN).

- -aj
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (Darwin)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAkltrv4ACgkQCJIWIbr9KYymiQCfbXOaJKOKGnk8pCOouPAxt0Pi
RVUAoJyHMsSgdYKUp455EgdhPFmt4mpA
=tRaw
-----END PGP SIGNATURE-----
-------------- next part --------------
A non-text attachment was scrubbed...
Name: lists.vcf
Type: text/x-vcard
Size: 316 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20090114/f5e01910/attachment.vcf>

From amitsaha.in at gmail.com  Wed Jan 14 10:37:30 2009
From: amitsaha.in at gmail.com (Amit k. Saha)
Date: Wed, 14 Jan 2009 15:07:30 +0530
Subject: [Distutils] Building Python Eggs
In-Reply-To: <496DAEFE.3030602@zopyx.com>
References: <547db2260901140120l2ce45a0co37efd508fd44d8bd@mail.gmail.com>
	<496DAEFE.3030602@zopyx.com>
Message-ID: <547db2260901140137l76f850f5r46453f777485f300@mail.gmail.com>

Hello Andreas,

On Wed, Jan 14, 2009 at 2:53 PM, Andreas Jung <lists at zopyx.com> wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> On 14.01.2009 10:20 Uhr, Amit k. Saha wrote:
>> Hi all,
>>
>> I am just learning the basics of building Python eggs using
>> 'setuptools'. What I have is the following code structure in a
>> top-down fashion:
>>
>>
>> - demo1.py
>> - __init.py__
>> - setup.py
>> -subpackage1
>> --__init.py/__
>>
>>
>> When I execute 'python setup.py bdist_egg' from the top-level
>> directory which has the 'setup.py' file:
>
> Using 'sdist' is usually good-enough unless you are working on binary
> distributions. 'sdist' usually includes everything - especially stuff
> under revision control (at least when using SVN).

Doesn't help my cause. The result is the same.

PS: None of my sources is under version control.

-Amit


>
> - -aj
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v1.4.9 (Darwin)
> Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/
>
> iEYEARECAAYFAkltrv4ACgkQCJIWIbr9KYymiQCfbXOaJKOKGnk8pCOouPAxt0Pi
> RVUAoJyHMsSgdYKUp455EgdhPFmt4mpA
> =tRaw
> -----END PGP SIGNATURE-----
>



-- 
Amit Kumar Saha
http://amitksaha.blogspot.com
http://amitsaha.in.googlepages.com/
*Bangalore Open Java Users Group*:http:www.bojug.in

From lutz_p at gmx.net  Wed Jan 14 11:04:09 2009
From: lutz_p at gmx.net (Lutz Paelike)
Date: Wed, 14 Jan 2009 11:04:09 +0100
Subject: [Distutils] Building Python Eggs
In-Reply-To: <547db2260901140137l76f850f5r46453f777485f300@mail.gmail.com>
References: <547db2260901140120l2ce45a0co37efd508fd44d8bd@mail.gmail.com>
	<496DAEFE.3030602@zopyx.com>
	<547db2260901140137l76f850f5r46453f777485f300@mail.gmail.com>
Message-ID: <C62CD0B3-EAB0-47C2-BBDE-D7228FA746A5@gmx.net>

Hey Amit,

a __init__.py file needs to be contained in a folder to make the  
folder a package.
Check out the python documentation for more information.


And you should move your setup file on the same level as the top-level  
package
The reason why find_packages() does'nt find your mainpackage is  
because it is
searching for packages on the same level or below the setup.py.


You have to change your directory layout to this:

###################
setup.py

mainpackackage
    demo1.py
    __init.py__

    subpackage1
        __init.py/__
####################

Cheers,

Lutz



Am 14.01.2009 um 10:37 schrieb Amit k. Saha:

> Hello Andreas,
>
> On Wed, Jan 14, 2009 at 2:53 PM, Andreas Jung <lists at zopyx.com> wrote:
>> -----BEGIN PGP SIGNED MESSAGE-----
>> Hash: SHA1
>>
>> On 14.01.2009 10:20 Uhr, Amit k. Saha wrote:
>>> Hi all,
>>>
>>> I am just learning the basics of building Python eggs using
>>> 'setuptools'. What I have is the following code structure in a
>>> top-down fashion:
>>>
>>>
>>> - demo1.py
>>> - __init.py__
>>> - setup.py
>>> -subpackage1
>>> --__init.py/__
>>>
>>>
>>> When I execute 'python setup.py bdist_egg' from the top-level
>>> directory which has the 'setup.py' file:
>>
>> Using 'sdist' is usually good-enough unless you are working on binary
>> distributions. 'sdist' usually includes everything - especially stuff
>> under revision control (at least when using SVN).
>
> Doesn't help my cause. The result is the same.
>
> PS: None of my sources is under version control.
>
> -Amit
>
>
>>
>> - -aj
>> -----BEGIN PGP SIGNATURE-----
>> Version: GnuPG v1.4.9 (Darwin)
>> Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/
>>
>> iEYEARECAAYFAkltrv4ACgkQCJIWIbr9KYymiQCfbXOaJKOKGnk8pCOouPAxt0Pi
>> RVUAoJyHMsSgdYKUp455EgdhPFmt4mpA
>> =tRaw
>> -----END PGP SIGNATURE-----
>>
>
>
>
> -- 
> Amit Kumar Saha
> http://amitksaha.blogspot.com
> http://amitsaha.in.googlepages.com/
> *Bangalore Open Java Users Group*:http:www.bojug.in
> _______________________________________________
> Distutils-SIG maillist  -  Distutils-SIG at python.org
> http://mail.python.org/mailman/listinfo/distutils-sig
>


From amitsaha.in at gmail.com  Wed Jan 14 11:07:11 2009
From: amitsaha.in at gmail.com (Amit k. Saha)
Date: Wed, 14 Jan 2009 15:37:11 +0530
Subject: [Distutils] Building Python Eggs
In-Reply-To: <C62CD0B3-EAB0-47C2-BBDE-D7228FA746A5@gmx.net>
References: <547db2260901140120l2ce45a0co37efd508fd44d8bd@mail.gmail.com>
	<496DAEFE.3030602@zopyx.com>
	<547db2260901140137l76f850f5r46453f777485f300@mail.gmail.com>
	<C62CD0B3-EAB0-47C2-BBDE-D7228FA746A5@gmx.net>
Message-ID: <547db2260901140207p62e30240l4dac22ba5bcc18ff@mail.gmail.com>

On Wed, Jan 14, 2009 at 3:34 PM, Lutz Paelike <lutz_p at gmx.net> wrote:
> Hey Amit,
>
> a __init__.py file needs to be contained in a folder to make the folder a
> package.

I had missed the 'folder' thingy..


> Check out the python documentation for more information.
>
>
> And you should move your setup file on the same level as the top-level
> package
> The reason why find_packages() does'nt find your mainpackage is because it
> is
> searching for packages on the same level or below the setup.py.
>
>
> You have to change your directory layout to this:
>
> ###################
> setup.py
>
> mainpackackage
>   demo1.py
>   __init.py__
>
>   subpackage1
>       __init.py/__
> ####################
>

Done. Thanks a lot Lutz !

-Amit

> Cheers,
>
> Lutz
>
>
>
> Am 14.01.2009 um 10:37 schrieb Amit k. Saha:
>
>> Hello Andreas,
>>
>> On Wed, Jan 14, 2009 at 2:53 PM, Andreas Jung <lists at zopyx.com> wrote:
>>>
>>> -----BEGIN PGP SIGNED MESSAGE-----
>>> Hash: SHA1
>>>
>>> On 14.01.2009 10:20 Uhr, Amit k. Saha wrote:
>>>>
>>>> Hi all,
>>>>
>>>> I am just learning the basics of building Python eggs using
>>>> 'setuptools'. What I have is the following code structure in a
>>>> top-down fashion:
>>>>
>>>>
>>>> - demo1.py
>>>> - __init.py__
>>>> - setup.py
>>>> -subpackage1
>>>> --__init.py/__
>>>>
>>>>
>>>> When I execute 'python setup.py bdist_egg' from the top-level
>>>> directory which has the 'setup.py' file:
>>>
>>> Using 'sdist' is usually good-enough unless you are working on binary
>>> distributions. 'sdist' usually includes everything - especially stuff
>>> under revision control (at least when using SVN).
>>
>> Doesn't help my cause. The result is the same.
>>
>> PS: None of my sources is under version control.
>>
>> -Amit
>>
>>
>>>
>>> - -aj
>>> -----BEGIN PGP SIGNATURE-----
>>> Version: GnuPG v1.4.9 (Darwin)
>>> Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/
>>>
>>> iEYEARECAAYFAkltrv4ACgkQCJIWIbr9KYymiQCfbXOaJKOKGnk8pCOouPAxt0Pi
>>> RVUAoJyHMsSgdYKUp455EgdhPFmt4mpA
>>> =tRaw
>>> -----END PGP SIGNATURE-----
>>>
>>
>>
>>
>> --
>> Amit Kumar Saha
>> http://amitksaha.blogspot.com
>> http://amitsaha.in.googlepages.com/
>> *Bangalore Open Java Users Group*:http:www.bojug.in
>> _______________________________________________
>> Distutils-SIG maillist  -  Distutils-SIG at python.org
>> http://mail.python.org/mailman/listinfo/distutils-sig
>>
>
>



-- 
Amit Kumar Saha
http://amitksaha.blogspot.com
http://amitsaha.in.googlepages.com/
*Bangalore Open Java Users Group*:http:www.bojug.in

From bl8cki at gmail.com  Wed Jan 14 12:52:29 2009
From: bl8cki at gmail.com (bl8cki)
Date: Wed, 14 Jan 2009 13:52:29 +0200
Subject: [Distutils] Changing distribution versions runtime
In-Reply-To: <e9db7f440901130405n5eda91cbif257b78f553a5304@mail.gmail.com>
References: <e9db7f440901130405n5eda91cbif257b78f553a5304@mail.gmail.com>
Message-ID: <e9db7f440901140352t67f7ace1u71a0fa0e71cc3867@mail.gmail.com>

To avoid removing added things in working_set ... i remove the entry for
which i need to switch versions in easy-install.pth and then only
'resolve'-ing and 'add'-ing to the working_set.

I don't know if this (removing the lines in easy-install.pth) will reflect
on easy_install to work properly, but for now i'm happy with this solution
:)

Still will be very thankful for any comments and suggestions.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20090114/05366417/attachment.htm>

From pje at telecommunity.com  Wed Jan 14 19:01:41 2009
From: pje at telecommunity.com (Phillip J. Eby)
Date: Wed, 14 Jan 2009 13:01:41 -0500
Subject: [Distutils] Changing distribution versions runtime
In-Reply-To: <e9db7f440901140352t67f7ace1u71a0fa0e71cc3867@mail.gmail.co
 m>
References: <e9db7f440901130405n5eda91cbif257b78f553a5304@mail.gmail.com>
	<e9db7f440901140352t67f7ace1u71a0fa0e71cc3867@mail.gmail.com>
Message-ID: <20090114175948.9BB383A4088@sparrow.telecommunity.com>

At 01:52 PM 1/14/2009 +0200, bl8cki wrote:
>To avoid removing added things in working_set ... i remove the entry 
>for which i need to switch versions in easy-install.pth and then 
>only 'resolve'-ing and 'add'-ing to the working_set.
>
>I don't know if this (removing the lines in easy-install.pth) will 
>reflect on easy_install to work properly, but for now i'm happy with 
>this solution :)
>
>Still will be very thankful for any comments and suggestions.

"easy_install -m projectname" will remove the line for you.  For that 
matter, using -m when you install those eggs in the first place will 
also do the trick.


From ktenney at gmail.com  Thu Jan 15 05:00:41 2009
From: ktenney at gmail.com (Kent Tenney)
Date: Thu, 15 Jan 2009 22:00:41 +1800
Subject: [Distutils] buildout recipe options
In-Reply-To: <A00530A1-CF38-497C-8CEC-4B9E8E17E89A@zope.com>
References: <49625F7F.6070609@gmail.com>
	<A00530A1-CF38-497C-8CEC-4B9E8E17E89A@zope.com>
Message-ID: <ca816810901142000q28f668a0l38f2d7bbb6945749@mail.gmail.com>

On Tue, Jan 6, 2009 at 1:37 PM, Jim Fulton <jim at zope.com> wrote:
>
> On Jan 5, 2009, at 2:29 PM, dunia wrote:
>
>> Hi list
>>
>> I'm a little bit confused with the buildout configuration files, and i
>> can't find any useful information. I get that every part of the buildout has
>> a recipe and this recipe takes some configuration options. My question is
>> ...how can i know which are those options? I went through the code and found
>> most of them, but not all.
>
> Do you mean, "How do you know which options are available?"
>
> I'll assume yes.
>
>> Maybe this is very simple, i recognize that my knowledge of python is not
>> as good as it should be...yet ;). Anyways, any hint would be useful for me.
>
> You have to rely on the recipe documentation.  Recipes should provide
> documentation that describes how they're used, including what their options
> are.  Many recipes make this available as part of their PYPI pages (For
> example, see http://pypi.python.org/pypi/zc.recipe.deployment). Buildout
> doesn't provide an API for a recipe to register or declare the options it
> provides.  Perhaps it should.

+1

> My philosophy has been to try make as few
> demands on recipe authors as possible.

As a buildout newbie , I would recommend offering the
recipe author a standard way to briefly describe available options.

Thanks,
Kent

> I could see some benefit in
> providing an optional registration mechanism that buildout could use if it's
> present.
>
> Jim
>
> --
> Jim Fulton
> Zope Corporation
>
>
> _______________________________________________
> Distutils-SIG maillist  -  Distutils-SIG at python.org
> http://mail.python.org/mailman/listinfo/distutils-sig
>

From ziade.tarek at gmail.com  Thu Jan 15 10:51:10 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Thu, 15 Jan 2009 10:51:10 +0100
Subject: [Distutils] buildout recipe options
In-Reply-To: <A00530A1-CF38-497C-8CEC-4B9E8E17E89A@zope.com>
References: <49625F7F.6070609@gmail.com>
	<A00530A1-CF38-497C-8CEC-4B9E8E17E89A@zope.com>
Message-ID: <94bdd2610901150151u238484eex50e3c7aec51fb8f9@mail.gmail.com>

On Mon, Jan 5, 2009 at 8:37 PM, Jim Fulton <jim at zope.com> wrote:

> You have to rely on the recipe documentation.  Recipes should provide
> documentation that describes how they're used, including what their options
> are.  Many recipes make this available as part of their PYPI pages (For
> example, see http://pypi.python.org/pypi/zc.recipe.deployment). Buildout
> doesn't provide an API for a recipe to register or declare the options it
> provides.  Perhaps it should.

We did work on that in a sprint with Godefroid,  to be able to display
recipe options doc from the command line at the Snow sprint.

But I can't see the branch anymore..

Tarek

-- 
Tarek Ziad? | Association AfPy | www.afpy.org
Blog FR | http://programmation-python.org
Blog EN | http://tarekziade.wordpress.com/

From ziade.tarek at gmail.com  Thu Jan 15 10:54:47 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Thu, 15 Jan 2009 10:54:47 +0100
Subject: [Distutils] buildout recipe options
In-Reply-To: <ca816810901142000q28f668a0l38f2d7bbb6945749@mail.gmail.com>
References: <49625F7F.6070609@gmail.com>
	<A00530A1-CF38-497C-8CEC-4B9E8E17E89A@zope.com>
	<ca816810901142000q28f668a0l38f2d7bbb6945749@mail.gmail.com>
Message-ID: <94bdd2610901150154q97db900h69112d9eb39039b@mail.gmail.com>

On Thu, Jan 15, 2009 at 5:00 AM, Kent Tenney <ktenney at gmail.com> wrote:

>> My philosophy has been to try make as few
>> demands on recipe authors as possible.
>
> As a buildout newbie , I would recommend offering the
> recipe author a standard way to briefly describe available options.

ZopeSkel provides a template that creates your recipe layout with a
nice documentation ala Jim,
with comments on how you have to complete it.

It's massively used in the Plone community, and the recipe doc all look the same

Tarek

From ziade.tarek at gmail.com  Thu Jan 15 11:24:11 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Thu, 15 Jan 2009 11:24:11 +0100
Subject: [Distutils] get_metadata in Distutils
In-Reply-To: <20090112160544.F25853A40A0@sparrow.telecommunity.com>
References: <94bdd2610901100948t65cba1achc2764c97195f529d@mail.gmail.com>
	<20090110200709.D3B3B3A40A0@sparrow.telecommunity.com>
	<94bdd2610901111122y156b691dh1db438a1b6fedc14@mail.gmail.com>
	<20090112025933.411B63A40A0@sparrow.telecommunity.com>
	<94bdd2610901120511g28130ef5se249e0a87f4eb2ab@mail.gmail.com>
	<20090112160544.F25853A40A0@sparrow.telecommunity.com>
Message-ID: <94bdd2610901150224m53397703qde293cf9dc61ea92@mail.gmail.com>

On Mon, Jan 12, 2009 at 5:07 PM, Phillip J. Eby <pje at telecommunity.com> wrote:
> At 02:11 PM 1/12/2009 +0100, Tarek Ziad? wrote:
>>
>> Ok, I will introduce in my patch the other formats. I am wondering though,
>> how
>> far this would be from an integration of pkg_resources into
>> Distutils/pkgutil,
>> with some API on the top and if it makes sense.
>
> It makes sense to me, I just wanted you to be aware how deep a job it is.
>  You're probably going to need a generic function by importer type, just
> like the others in pkgutil.

I am making some progress, now the patch works with zipped eggs, unzipped eggs
and regular .egg-info files.

I am wondering about other cases though:

- it seems that pkg_resources works on .egg-info *directories*. When
does this case happen ?
- you said that its needs to work on nested packages, but I can't see
the difference when grabbing the metadata in such a package

There are also some mechanisms I am not sure at a 100% of, like the
right way to loop over sys.path entries, and the way to deal with
sys.meta_path, etc..

Regards
Tarek

-- 
Tarek Ziad? | Association AfPy | www.afpy.org
Blog FR | http://programmation-python.org
Blog EN | http://tarekziade.wordpress.com/

From gotcha at bubblenet.be  Thu Jan 15 12:31:25 2009
From: gotcha at bubblenet.be (Godefroid Chapelle)
Date: Thu, 15 Jan 2009 12:31:25 +0100
Subject: [Distutils] buildout recipe options
In-Reply-To: <94bdd2610901150151u238484eex50e3c7aec51fb8f9@mail.gmail.com>
References: <49625F7F.6070609@gmail.com>	<A00530A1-CF38-497C-8CEC-4B9E8E17E89A@zope.com>
	<94bdd2610901150151u238484eex50e3c7aec51fb8f9@mail.gmail.com>
Message-ID: <496F1E8D.3040209@bubblenet.be>

Tarek Ziad? wrote:
> On Mon, Jan 5, 2009 at 8:37 PM, Jim Fulton <jim at zope.com> wrote:
> 
>> You have to rely on the recipe documentation.  Recipes should provide
>> documentation that describes how they're used, including what their options
>> are.  Many recipes make this available as part of their PYPI pages (For
>> example, see http://pypi.python.org/pypi/zc.recipe.deployment). Buildout
>> doesn't provide an API for a recipe to register or declare the options it
>> provides.  Perhaps it should.
> 
> We did work on that in a sprint with Godefroid,  to be able to display
> recipe options doc from the command line at the Snow sprint.
> 
> But I can't see the branch anymore..
> 
> Tarek
> 

The branch was zc.buildout/branches/help-api.

-- 
Godefroid Chapelle (aka __gotcha) http://bubblenet.be

From eric.c2c at gmail.com  Thu Jan 15 14:44:22 2009
From: eric.c2c at gmail.com (Eric Lemoine)
Date: Thu, 15 Jan 2009 14:44:22 +0100
Subject: [Distutils] custom package index issue
Message-ID: <5ec103de0901150544o2a4bed7jcb12ee97adcec41b@mail.gmail.com>

Hi

For a project I'm involved in we created our own package index. The
package index includes our package (mapfish), as well as its direct
and indirect dependencies.

Then, the installation of the "mapfish" package and its dependencies
is done with this command:
easy_install -i http://dev.camptocamp.com/packages/mapfish/1.0/index
mapfish==1.0

The installation command works, but some indirect dependencies aren't
downloaded from our package index but from other sources. For
example,"simplejson" which "pylons" depends on (which "mapfish"
depends on), is downloaded from
"http://pylons.cachefly.net/download/0.9.6.2/simplejson-1.7.1-py2.5.egg"
as opposed to be downloaded from
"http://dev.camptocamp.com/packages/mapfish/1.0/simplejson-1.7.1.tar.gz".

We'd like every mapfish dependency to be downloaded from our package
index. Does anyone know how to achieve this?

Thanks a lot,

--
Eric

From ziade.tarek at gmail.com  Thu Jan 15 15:00:54 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Thu, 15 Jan 2009 15:00:54 +0100
Subject: [Distutils] custom package index issue
In-Reply-To: <5ec103de0901150544o2a4bed7jcb12ee97adcec41b@mail.gmail.com>
References: <5ec103de0901150544o2a4bed7jcb12ee97adcec41b@mail.gmail.com>
Message-ID: <94bdd2610901150600l1bda57a9uaf205bc840140923@mail.gmail.com>

On Thu, Jan 15, 2009 at 2:44 PM, Eric Lemoine <eric.c2c at gmail.com> wrote:
> Hi
>
> For a project I'm involved in we created our own package index. The
> package index includes our package (mapfish), as well as its direct
> and indirect dependencies.
>
> Then, the installation of the "mapfish" package and its dependencies
> is done with this command:
> easy_install -i http://dev.camptocamp.com/packages/mapfish/1.0/index
> mapfish==1.0
>
> The installation command works, but some indirect dependencies aren't
> downloaded from our package index but from other sources. For
> example,"simplejson" which "pylons" depends on (which "mapfish"
> depends on), is downloaded from
> "http://pylons.cachefly.net/download/0.9.6.2/simplejson-1.7.1-py2.5.egg"
> as opposed to be downloaded from
> "http://dev.camptocamp.com/packages/mapfish/1.0/simplejson-1.7.1.tar.gz".

Your index seems correct,

Maybe you have some dependency_links amongst your packages that forces
download links,

with easy_install you can filter the allowed host to prevent it, see:

http://peak.telecommunity.com/DevCenter/EasyInstall#restricting-downloads-with-allow-hosts

Try this:

$ easy_install -i http://dev.camptocamp.com/packages/mapfish/1.0/index
--allow-hosts=dev.camptocamp.com mapfish==1.0

You will see som ***BLOCKED*** lines and you will figure out what is happening

I was able to install your mapfish package this way FYI

>
> We'd like every mapfish dependency to be downloaded from our package
> index. Does anyone know how to achieve this?
>
> Thanks a lot,
>
> --
> Eric
> _______________________________________________
> Distutils-SIG maillist  -  Distutils-SIG at python.org
> http://mail.python.org/mailman/listinfo/distutils-sig
>



-- 
Tarek Ziad? | Association AfPy | www.afpy.org
Blog FR | http://programmation-python.org
Blog EN | http://tarekziade.wordpress.com/

From pje at telecommunity.com  Thu Jan 15 16:20:05 2009
From: pje at telecommunity.com (Phillip J. Eby)
Date: Thu, 15 Jan 2009 10:20:05 -0500
Subject: [Distutils] get_metadata in Distutils
In-Reply-To: <94bdd2610901150224m53397703qde293cf9dc61ea92@mail.gmail.co
 m>
References: <94bdd2610901100948t65cba1achc2764c97195f529d@mail.gmail.com>
	<20090110200709.D3B3B3A40A0@sparrow.telecommunity.com>
	<94bdd2610901111122y156b691dh1db438a1b6fedc14@mail.gmail.com>
	<20090112025933.411B63A40A0@sparrow.telecommunity.com>
	<94bdd2610901120511g28130ef5se249e0a87f4eb2ab@mail.gmail.com>
	<20090112160544.F25853A40A0@sparrow.telecommunity.com>
	<94bdd2610901150224m53397703qde293cf9dc61ea92@mail.gmail.com>
Message-ID: <20090115151816.119EE3A410E@sparrow.telecommunity.com>

At 11:24 AM 1/15/2009 +0100, Tarek Ziad? wrote:
>On Mon, Jan 12, 2009 at 5:07 PM, Phillip J. Eby <pje at telecommunity.com> wrote:
> > At 02:11 PM 1/12/2009 +0100, Tarek Ziad? wrote:
> >>
> >> Ok, I will introduce in my patch the other formats. I am wondering though,
> >> how
> >> far this would be from an integration of pkg_resources into
> >> Distutils/pkgutil,
> >> with some API on the top and if it makes sense.
> >
> > It makes sense to me, I just wanted you to be aware how deep a job it is.
> >  You're probably going to need a generic function by importer type, just
> > like the others in pkgutil.
>
>I am making some progress, now the patch works with zipped eggs, unzipped eggs
>and regular .egg-info files.
>
>I am wondering about other cases though:
>
>- it seems that pkg_resources works on .egg-info *directories*. When
>does this case happen ?

System packages (.rpm, .deb, etc.), or other install 
--root/--single-version-externally-managed installs of 
setuptools-based packages.

(Of course, there are also .egg directories with EGG-INFO 
subdirectories, installed by easy_install --always-unzip...)


>- you said that its needs to work on nested packages, but I can't see
>the difference when grabbing the metadata in such a package

More precisely: you can have nested .egg directories within a single 
zipfile.  This is to support bundling a bunch of eggs in a single 
zipfile for easy application distribution.


>There are also some mechanisms I am not sure at a 100% of, like the
>right way to loop over sys.path entries, and the way to deal with
>sys.meta_path, etc..

Well, setuptools ignores sys.meta_path, but I suppose technically it 
shouldn't.  A use case hasn't occurred yet, though, at least to my knowledge.


From pje at telecommunity.com  Thu Jan 15 16:24:00 2009
From: pje at telecommunity.com (Phillip J. Eby)
Date: Thu, 15 Jan 2009 10:24:00 -0500
Subject: [Distutils] custom package index issue
In-Reply-To: <5ec103de0901150544o2a4bed7jcb12ee97adcec41b@mail.gmail.com
 >
References: <5ec103de0901150544o2a4bed7jcb12ee97adcec41b@mail.gmail.com>
Message-ID: <20090115152206.E84CE3A410E@sparrow.telecommunity.com>

At 02:44 PM 1/15/2009 +0100, Eric Lemoine wrote:
>Hi
>
>For a project I'm involved in we created our own package index. The
>package index includes our package (mapfish), as well as its direct
>and indirect dependencies.
>
>Then, the installation of the "mapfish" package and its dependencies
>is done with this command:
>easy_install -i http://dev.camptocamp.com/packages/mapfish/1.0/index
>mapfish==1.0
>
>The installation command works, but some indirect dependencies aren't
>downloaded from our package index but from other sources. For
>example,"simplejson" which "pylons" depends on (which "mapfish"
>depends on), is downloaded from
>"http://pylons.cachefly.net/download/0.9.6.2/simplejson-1.7.1-py2.5.egg"
>as opposed to be downloaded from
>"http://dev.camptocamp.com/packages/mapfish/1.0/simplejson-1.7.1.tar.gz".

Binary packages are given precedence over source packages, and 
presumably Pylons is providing a direct link in its setup.py.

>We'd like every mapfish dependency to be downloaded from our package
>index. Does anyone know how to achieve this?

You'll need to either ensure that your index provides binaries, or 
use --allow-hosts to restrict what hosts easy_install will look for files on.


From tseaver at palladion.com  Thu Jan 15 16:34:49 2009
From: tseaver at palladion.com (Tres Seaver)
Date: Thu, 15 Jan 2009 10:34:49 -0500
Subject: [Distutils] custom package index issue
In-Reply-To: <20090115152206.E84CE3A410E@sparrow.telecommunity.com>
References: <5ec103de0901150544o2a4bed7jcb12ee97adcec41b@mail.gmail.com>
	<5ec103de0901150544o2a4bed7jcb12ee97adcec41b@mail.gmail.com >
	<20090115152206.E84CE3A410E@sparrow.telecommunity.com>
Message-ID: <496F5799.4020305@palladion.com>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Phillip J. Eby wrote:
> At 02:44 PM 1/15/2009 +0100, Eric Lemoine wrote:
>> Hi
>>
>> For a project I'm involved in we created our own package index. The
>> package index includes our package (mapfish), as well as its direct
>> and indirect dependencies.
>>
>> Then, the installation of the "mapfish" package and its dependencies
>> is done with this command:
>> easy_install -i http://dev.camptocamp.com/packages/mapfish/1.0/index
>> mapfish==1.0
>>
>> The installation command works, but some indirect dependencies aren't
>> downloaded from our package index but from other sources. For
>> example,"simplejson" which "pylons" depends on (which "mapfish"
>> depends on), is downloaded from
>> "http://pylons.cachefly.net/download/0.9.6.2/simplejson-1.7.1-py2.5.egg"
>> as opposed to be downloaded from
>> "http://dev.camptocamp.com/packages/mapfish/1.0/simplejson-1.7.1.tar.gz".
> 
> Binary packages are given precedence over source packages, and 
> presumably Pylons is providing a direct link in its setup.py.

I would certainly wish for a configuration knob which inverted this
policy:  even better would be one which refused to download binaries at all.

>> We'd like every mapfish dependency to be downloaded from our package
>> index. Does anyone know how to achieve this?
> 
> You'll need to either ensure that your index provides binaries, or 
> use --allow-hosts to restrict what hosts easy_install will look for files on.

While I'm at it ;):  I'd like a global config knob to disable use of
package-supplied 'dependency_links' (as opposed to those supplied on the
command line).  I guess the '--allow-hosts' option is probably a decent
workaround for that one:  can it be set in 'setup.cfg', perchance?


Tres.
- --
===================================================================
Tres Seaver          +1 540-429-0999          tseaver at palladion.com
Palladion Software   "Excellence by Design"    http://palladion.com
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFJb1eZ+gerLs4ltQ4RAii8AJ9kAXtl9jGNf6gMapJ1p5xRTEoM1gCeJgJJ
0+Dr5XJXJLnryEV78CTKGkU=
=BwDx
-----END PGP SIGNATURE-----


From ziade.tarek at gmail.com  Thu Jan 15 17:01:29 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Thu, 15 Jan 2009 17:01:29 +0100
Subject: [Distutils] custom package index issue
In-Reply-To: <496F5799.4020305@palladion.com>
References: <5ec103de0901150544o2a4bed7jcb12ee97adcec41b@mail.gmail.com>
	<20090115152206.E84CE3A410E@sparrow.telecommunity.com>
	<496F5799.4020305@palladion.com>
Message-ID: <94bdd2610901150801p19dcacf0u7ee8afbc712447c@mail.gmail.com>

On Thu, Jan 15, 2009 at 4:34 PM, Tres Seaver <tseaver at palladion.com> wrote:
>> You'll need to either ensure that your index provides binaries, or
>> use --allow-hosts to restrict what hosts easy_install will look for files on.
>
> While I'm at it ;):  I'd like a global config knob to disable use of
> package-supplied 'dependency_links' (as opposed to those supplied on the
> command line).  I guess the '--allow-hosts' option is probably a decent
> workaround for that one:  can it be set in 'setup.cfg', perchance?

If you use zc.buildout, I've added an allow-hosts option for that matter:

http://pypi.python.org/pypi/zc.buildout#allow-hosts

From eric.c2c at gmail.com  Thu Jan 15 17:11:04 2009
From: eric.c2c at gmail.com (Eric Lemoine)
Date: Thu, 15 Jan 2009 17:11:04 +0100
Subject: [Distutils] custom package index issue
In-Reply-To: <20090115152206.E84CE3A410E@sparrow.telecommunity.com>
References: <5ec103de0901150544o2a4bed7jcb12ee97adcec41b@mail.gmail.com>
	<20090115152206.E84CE3A410E@sparrow.telecommunity.com>
Message-ID: <5ec103de0901150811r26f7901dv13ecff042779ce18@mail.gmail.com>

On Thu, Jan 15, 2009 at 4:24 PM, Phillip J. Eby <pje at telecommunity.com> wrote:
> At 02:44 PM 1/15/2009 +0100, Eric Lemoine wrote:
>>
>> Hi
>>
>> For a project I'm involved in we created our own package index. The
>> package index includes our package (mapfish), as well as its direct
>> and indirect dependencies.
>>
>> Then, the installation of the "mapfish" package and its dependencies
>> is done with this command:
>> easy_install -i http://dev.camptocamp.com/packages/mapfish/1.0/index
>> mapfish==1.0
>>
>> The installation command works, but some indirect dependencies aren't
>> downloaded from our package index but from other sources. For
>> example,"simplejson" which "pylons" depends on (which "mapfish"
>> depends on), is downloaded from
>> "http://pylons.cachefly.net/download/0.9.6.2/simplejson-1.7.1-py2.5.egg"
>> as opposed to be downloaded from
>> "http://dev.camptocamp.com/packages/mapfish/1.0/simplejson-1.7.1.tar.gz".
>
> Binary packages are given precedence over source packages, and presumably
> Pylons is providing a direct link in its setup.py.
>
>> We'd like every mapfish dependency to be downloaded from our package
>> index. Does anyone know how to achieve this?
>
> You'll need to either ensure that your index provides binaries, or use
> --allow-hosts to restrict what hosts easy_install will look for files on.


Thanks you all, --allow-hosts does the trick!

--
Eric

From optilude+lists at gmail.com  Sat Jan 17 12:56:08 2009
From: optilude+lists at gmail.com (Martin Aspeli)
Date: Sat, 17 Jan 2009 11:56:08 +0000
Subject: [Distutils] zc.buildout 'extends' options
Message-ID: <4971C758.9070802@gmail.com>

Hi Jim & co,

Plone 3.2 currently distributes its packages as eggs, and people use an 
'extends' option with a remote URL to get a known good set for each 
version, e.g.:

  [buildout]
  extends = http://dist.plone.org/release/3.2/versions.cfg

This is similar to how Grok does its distributions, of course.

Now, as far as I can tell, this doesn't work offline. -o mode does not 
complain, but if you're totally offline (turn off your network 
interface), then urllib throws an error when buildout tries to evaluate 
the 'extends' option, and buildout crashes completely.

I'd like to have an optional download cache directory, e.g.

  [buildout]
  extends-cache = extends-cache-directory
  extends = http://dist.plone.org/release/3.2/versions.cfg

In offline mode, buildout would look for the file to extend (probably 
with some hash of the url used as filename) in the directory specified 
by extends-cache.

This would allow full offline mode (e.g. on a machine with no network 
card configured). In particular, it would allow Plone's buildout-based 
installers to ship with the versions.cfg file and not require people to 
be able to download them.

Is this desirable? Any plans for such a feature? Would you accept a 
patch, if we could make it work?

-- 
Author of `Professional Plone Development`, a book for developers who
want to work with Plone. See http://martinaspeli.net/plone-book


From akitada at gmail.com  Sun Jan 18 16:03:24 2009
From: akitada at gmail.com (Akira Kitada)
Date: Mon, 19 Jan 2009 00:03:24 +0900
Subject: [Distutils] The latest stable standalone Distutils?
Message-ID: <90bb445a0901180703h5981691g414f4ef0c4e70713@mail.gmail.com>

Hello,

I've just read distutils/README [1] which says there's "a separately packaged
standalone version of the Distutils" on Distutils web page [2].
It seems, however, this is just a unmaintained README and actually
all I could find there was old "Distutils-1.0.2".

Where can I find the latest standalone Distutils?
I looked at projects/distutils on svn.python.org but it didn't appear
to be maintained.

Thanks in advance,

[1] http://svn.python.org/view/python/trunk/Lib/distutils/README
[2] http://www.python.org/sigs/distutils-sig/

From ziade.tarek at gmail.com  Sun Jan 18 16:16:32 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Sun, 18 Jan 2009 16:16:32 +0100
Subject: [Distutils] The latest stable standalone Distutils?
In-Reply-To: <90bb445a0901180703h5981691g414f4ef0c4e70713@mail.gmail.com>
References: <90bb445a0901180703h5981691g414f4ef0c4e70713@mail.gmail.com>
Message-ID: <94bdd2610901180716o43d735f4sd1769562f4cdf6ff@mail.gmail.com>

Hi Akira,
this README is 6 years old, and unmaintained, (I am adding a ticket on
this right away and will take care of it)

there's no maintained standalone Distutils as well, we need to update the

http://www.python.org/community/sigs/current/distutils-sig/

page as well to refelct reality

Regards

On Sun, Jan 18, 2009 at 4:03 PM, Akira Kitada <akitada at gmail.com> wrote:
> Hello,
>
> I've just read distutils/README [1] which says there's "a separately packaged
> standalone version of the Distutils" on Distutils web page [2].
> It seems, however, this is just a unmaintained README and actually
> all I could find there was old "Distutils-1.0.2".
>
> Where can I find the latest standalone Distutils?
> I looked at projects/distutils on svn.python.org but it didn't appear
> to be maintained.
>
> Thanks in advance,
>
> [1] http://svn.python.org/view/python/trunk/Lib/distutils/README
> [2] http://www.python.org/sigs/distutils-sig/
> _______________________________________________
> Distutils-SIG maillist  -  Distutils-SIG at python.org
> http://mail.python.org/mailman/listinfo/distutils-sig
>



-- 
Tarek Ziad? | Association AfPy | www.afpy.org
Blog FR | http://programmation-python.org
Blog EN | http://tarekziade.wordpress.com/

From akitada at gmail.com  Sun Jan 18 17:32:28 2009
From: akitada at gmail.com (Akira Kitada)
Date: Mon, 19 Jan 2009 01:32:28 +0900
Subject: [Distutils] The latest stable standalone Distutils?
In-Reply-To: <94bdd2610901180716o43d735f4sd1769562f4cdf6ff@mail.gmail.com>
References: <90bb445a0901180703h5981691g414f4ef0c4e70713@mail.gmail.com>
	<94bdd2610901180716o43d735f4sd1769562f4cdf6ff@mail.gmail.com>
Message-ID: <90bb445a0901180832m7655a441w1ee0b0c11aa9a000@mail.gmail.com>

Hi Tarek,
Thanks for the response and openning a ticket for this.

(For the record, here's the link for the ticket:
http://bugs.python.org/issue4987)

So, the latest Distutils code would be one in svn trunk.
I was looking for that code for learning the implementation.

On Mon, Jan 19, 2009 at 12:16 AM, Tarek Ziad? <ziade.tarek at gmail.com> wrote:
> Hi Akira,
> this README is 6 years old, and unmaintained, (I am adding a ticket on
> this right away and will take care of it)
>
> there's no maintained standalone Distutils as well, we need to update the
>
> http://www.python.org/community/sigs/current/distutils-sig/
>
> page as well to refelct reality
>
> Regards
>
> On Sun, Jan 18, 2009 at 4:03 PM, Akira Kitada <akitada at gmail.com> wrote:
>> Hello,
>>
>> I've just read distutils/README [1] which says there's "a separately packaged
>> standalone version of the Distutils" on Distutils web page [2].
>> It seems, however, this is just a unmaintained README and actually
>> all I could find there was old "Distutils-1.0.2".
>>
>> Where can I find the latest standalone Distutils?
>> I looked at projects/distutils on svn.python.org but it didn't appear
>> to be maintained.
>>
>> Thanks in advance,
>>
>> [1] http://svn.python.org/view/python/trunk/Lib/distutils/README
>> [2] http://www.python.org/sigs/distutils-sig/
>> _______________________________________________
>> Distutils-SIG maillist  -  Distutils-SIG at python.org
>> http://mail.python.org/mailman/listinfo/distutils-sig
>>
>
>
>
> --
> Tarek Ziad? | Association AfPy | www.afpy.org
> Blog FR | http://programmation-python.org
> Blog EN | http://tarekziade.wordpress.com/
>

From ziade.tarek at gmail.com  Sun Jan 18 17:44:32 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Sun, 18 Jan 2009 17:44:32 +0100
Subject: [Distutils] The latest stable standalone Distutils?
In-Reply-To: <90bb445a0901180832m7655a441w1ee0b0c11aa9a000@mail.gmail.com>
References: <90bb445a0901180703h5981691g414f4ef0c4e70713@mail.gmail.com>
	<94bdd2610901180716o43d735f4sd1769562f4cdf6ff@mail.gmail.com>
	<90bb445a0901180832m7655a441w1ee0b0c11aa9a000@mail.gmail.com>
Message-ID: <94bdd2610901180844s1ae60cfr5b943d291d601248@mail.gmail.com>

On Sun, Jan 18, 2009 at 5:32 PM, Akira Kitada <akitada at gmail.com> wrote:
> Hi Tarek,
> Thanks for the response and openning a ticket for this.
>
> (For the record, here's the link for the ticket:
> http://bugs.python.org/issue4987)

Yes, I am currently on distutils maintenance, by reviewing and fixing
bugs in the tracker,

>
> So, the latest Distutils code would be one in svn trunk.
> I was looking for that code for learning the implementation.

Yes,

It is located here:

http://svn.python.org/projects/python/trunk/Lib/distutils/

Regards
Tarek

From akitada at gmail.com  Tue Jan 20 00:47:11 2009
From: akitada at gmail.com (Akira Kitada)
Date: Tue, 20 Jan 2009 08:47:11 +0900
Subject: [Distutils] Where to put a original distutils command?
Message-ID: <90bb445a0901191547w70feb547i39f1fd5c608162b1@mail.gmail.com>

Hi,

I was wondering where I'm supposed to put a new distutils command in.
The command is a kind of bdist_rpm-like thing for my own package system,
so it's just for local use so won't ever move in Python distribution.

I thought site-packages, but just for one command, it would be a bit overkill.

Any ideas?

From niklas at bivald.com  Tue Jan 20 08:25:44 2009
From: niklas at bivald.com (Niklas Bivald)
Date: Tue, 20 Jan 2009 08:25:44 +0100
Subject: [Distutils] Problems installing Easy Install/Setup Tools
	(segmentation fault)
Message-ID: <5A5D0D13-B9A9-4B0C-9466-CF1A0D682B24@bivald.com>

Dear peak,
I've been tearing my hair out on a installation issue I've been  
having. I've searched Google, I've searched the archives. I've been  
running setuptools on my OS X box for quite a while and it's been  
working great. However, moving on to my Debian (production) box has  
cause me considerable problems. It may be a Python issue, but I've  
tried Python 2.4, 2.5 and 2.6 (and 2.6.1) and I keep getting the same  
error. Hopefully I've just missed something easy.

I've tried to install setuptools using apt-get without luck (gives me  
a non-describt error about failed installation) and using eggs:

chat4:/home/orbited# sh setuptools-0.6c9-py2.6.egg
Segmentation fault

gives me segmentation fault. I've switched to using from source in  
order to catch the exact error, and I've boiled it down to:

chat4:/home/orbited/setuptools-0.6c9# 	python2.6 -dv easy_install.py  
install
[...] Quite a long output [...]
# /home/orbited/setuptools-0.6c9/setuptools/dist.pyc matches /home/ 
orbited/setuptools-0.6c9/setuptools/dist.py
import setuptools.dist # precompiled from /home/orbited/ 
setuptools-0.6c9/setuptools/dist.pyc
Segmentation fault

On Python2.6, 2.5 and 2.4. Verifying the problem on 2.5:

chat4:/home/orbited/setuptools-0.6c9# python
Python 2.5.2 (r252:60911, Jan  4 2009, 17:40:26)
[GCC 4.3.2] on linux2
Type "help", "copyright", "credits" or "license" for more information.
 >>> import setuptools.dist
Segmentation fault

On 2.6:

chat4:/home/orbited/setuptools-0.6c9# python2.6
Python 2.6.1 (r261:67515, Jan 20 2009, 07:59:19)
[GCC 4.3.1] on linux2
Type "help", "copyright", "credits" or "license" for more information.
 >>> import setuptools.dist
Segmentation fault

I  must be missing something vital, using more debug flags gives me:

import setuptools.dist # precompiled from /home/orbited/ 
setuptools-0.6c9/setuptools/dist.pyc
# trying /home/orbited/setuptools-0.6c9/setuptools/depends.so
# trying /home/orbited/setuptools-0.6c9/setuptools/dependsmodule.so
# trying /home/orbited/setuptools-0.6c9/setuptools/depends.py
Segmentation fault

Can anyone think of anything that would share some light on this issue?

Regards,
Niklas

From david at ar.media.kyoto-u.ac.jp  Tue Jan 20 09:05:28 2009
From: david at ar.media.kyoto-u.ac.jp (David Cournapeau)
Date: Tue, 20 Jan 2009 17:05:28 +0900
Subject: [Distutils] Problems installing Easy Install/Setup
 Tools	(segmentation fault)
In-Reply-To: <5A5D0D13-B9A9-4B0C-9466-CF1A0D682B24@bivald.com>
References: <5A5D0D13-B9A9-4B0C-9466-CF1A0D682B24@bivald.com>
Message-ID: <497585C8.5060304@ar.media.kyoto-u.ac.jp>

Niklas Bivald wrote:
>
>
> Can anyone think of anything that would share some light on this issue?

Obviously, your python installation is borked. It looks like you are
using your own custom-build python. Try using packaged python first - it
is more than likely that your problem is in the way you built python,

David

From ziade.tarek at gmail.com  Tue Jan 20 09:47:58 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Tue, 20 Jan 2009 09:47:58 +0100
Subject: [Distutils] Where to put a original distutils command?
In-Reply-To: <90bb445a0901191547w70feb547i39f1fd5c608162b1@mail.gmail.com>
References: <90bb445a0901191547w70feb547i39f1fd5c608162b1@mail.gmail.com>
Message-ID: <94bdd2610901200047s10353ebfq35ce24a3ae31b410@mail.gmail.com>

On Tue, Jan 20, 2009 at 12:47 AM, Akira Kitada <akitada at gmail.com> wrote:
> Hi,
>
> I was wondering where I'm supposed to put a new distutils command in.
> The command is a kind of bdist_rpm-like thing for my own package system,
> so it's just for local use so won't ever move in Python distribution.
>
> I thought site-packages, but just for one command, it would be a bit overkill.
>
> Any ideas?

You can create a new package that implements your command in two ways:

1/
using distutils extension mechanism (see
http://docs.python.org/distutils/extending.html#integrating-new-commands)

2/
using setuptools extension mechanism based on "entry points", see
http://peak.telecommunity.com/DevCenter/setuptools#adding-commands

for the latter you can take a look at collective.dist, that adds two
new commands using entry points -
http://pypi.python.org/pypi/collective.dist

Regards
Tarek

> _______________________________________________
> Distutils-SIG maillist  -  Distutils-SIG at python.org
> http://mail.python.org/mailman/listinfo/distutils-sig
>



-- 
Tarek Ziad? | Association AfPy | www.afpy.org
Blog FR | http://programmation-python.org
Blog EN | http://tarekziade.wordpress.com/

From r.ritz at biologie.hu-berlin.de  Tue Jan 20 11:29:26 2009
From: r.ritz at biologie.hu-berlin.de (Raphael Ritz)
Date: Tue, 20 Jan 2009 11:29:26 +0100
Subject: [Distutils] Building Python Eggs
In-Reply-To: <547db2260901140207p62e30240l4dac22ba5bcc18ff@mail.gmail.com>
References: <547db2260901140120l2ce45a0co37efd508fd44d8bd@mail.gmail.com>	<496DAEFE.3030602@zopyx.com>	<547db2260901140137l76f850f5r46453f777485f300@mail.gmail.com>	<C62CD0B3-EAB0-47C2-BBDE-D7228FA746A5@gmx.net>
	<547db2260901140207p62e30240l4dac22ba5bcc18ff@mail.gmail.com>
Message-ID: <gl4926$jl3$1@ger.gmane.org>

Amit k. Saha wrote:
> On Wed, Jan 14, 2009 at 3:34 PM, Lutz Paelike <lutz_p at gmx.net> wrote:
>> Hey Amit,
>>
>> a __init__.py file needs to be contained in a folder to make the folder a
>> package.
> 
> I had missed the 'folder' thingy..

Glad you've got a solution that works for you
but IIRC it should be possible to do as you
originally intended by explicitly adding

   py_modules = ['demo1.py']

in your setup call.

Raphael



> 
> 
>> Check out the python documentation for more information.
>>
>>
>> And you should move your setup file on the same level as the top-level
>> package
>> The reason why find_packages() does'nt find your mainpackage is because it
>> is
>> searching for packages on the same level or below the setup.py.
>>
>>
>> You have to change your directory layout to this:
>>
>> ###################
>> setup.py
>>
>> mainpackackage
>>   demo1.py
>>   __init.py__
>>
>>   subpackage1
>>       __init.py/__
>> ####################
>>
> 
> Done. Thanks a lot Lutz !
> 
> -Amit
> 
>> Cheers,
>>
>> Lutz
>>
>>
>>
>> Am 14.01.2009 um 10:37 schrieb Amit k. Saha:
>>
>>> Hello Andreas,
>>>
>>> On Wed, Jan 14, 2009 at 2:53 PM, Andreas Jung <lists at zopyx.com> wrote:
>>>> -----BEGIN PGP SIGNED MESSAGE-----
>>>> Hash: SHA1
>>>>
>>>> On 14.01.2009 10:20 Uhr, Amit k. Saha wrote:
>>>>> Hi all,
>>>>>
>>>>> I am just learning the basics of building Python eggs using
>>>>> 'setuptools'. What I have is the following code structure in a
>>>>> top-down fashion:
>>>>>
>>>>>
>>>>> - demo1.py
>>>>> - __init.py__
>>>>> - setup.py
>>>>> -subpackage1
>>>>> --__init.py/__
>>>>>
>>>>>
>>>>> When I execute 'python setup.py bdist_egg' from the top-level
>>>>> directory which has the 'setup.py' file:
>>>> Using 'sdist' is usually good-enough unless you are working on binary
>>>> distributions. 'sdist' usually includes everything - especially stuff
>>>> under revision control (at least when using SVN).
>>> Doesn't help my cause. The result is the same.
>>>
>>> PS: None of my sources is under version control.
>>>
>>> -Amit
>>>
>>>
>>>> - -aj
>>>> -----BEGIN PGP SIGNATURE-----
>>>> Version: GnuPG v1.4.9 (Darwin)
>>>> Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/
>>>>
>>>> iEYEARECAAYFAkltrv4ACgkQCJIWIbr9KYymiQCfbXOaJKOKGnk8pCOouPAxt0Pi
>>>> RVUAoJyHMsSgdYKUp455EgdhPFmt4mpA
>>>> =tRaw
>>>> -----END PGP SIGNATURE-----
>>>>
>>>
>>>
>>> --
>>> Amit Kumar Saha
>>> http://amitksaha.blogspot.com
>>> http://amitsaha.in.googlepages.com/
>>> *Bangalore Open Java Users Group*:http:www.bojug.in
>>> _______________________________________________
>>> Distutils-SIG maillist  -  Distutils-SIG at python.org
>>> http://mail.python.org/mailman/listinfo/distutils-sig
>>>
>>
> 
> 
> 


From amitsaha.in at gmail.com  Tue Jan 20 11:57:16 2009
From: amitsaha.in at gmail.com (Amit k. Saha)
Date: Tue, 20 Jan 2009 16:27:16 +0530
Subject: [Distutils] Building Python Eggs
In-Reply-To: <gl4926$jl3$1@ger.gmane.org>
References: <547db2260901140120l2ce45a0co37efd508fd44d8bd@mail.gmail.com>
	<496DAEFE.3030602@zopyx.com>
	<547db2260901140137l76f850f5r46453f777485f300@mail.gmail.com>
	<C62CD0B3-EAB0-47C2-BBDE-D7228FA746A5@gmx.net>
	<547db2260901140207p62e30240l4dac22ba5bcc18ff@mail.gmail.com>
	<gl4926$jl3$1@ger.gmane.org>
Message-ID: <547db2260901200257u54191e81t3d8b1877b4f5f896@mail.gmail.com>

On Tue, Jan 20, 2009 at 3:59 PM, Raphael Ritz
<r.ritz at biologie.hu-berlin.de> wrote:
> Amit k. Saha wrote:
>>
>> On Wed, Jan 14, 2009 at 3:34 PM, Lutz Paelike <lutz_p at gmx.net> wrote:
>>>
>>> Hey Amit,
>>>
>>> a __init__.py file needs to be contained in a folder to make the folder a
>>> package.
>>
>> I had missed the 'folder' thingy..
>
> Glad you've got a solution that works for you
> but IIRC it should be possible to do as you
> originally intended by explicitly adding
>
>  py_modules = ['demo1.py']
>
> in your setup call.

I have added this line to my 'setup.py' file, but the result remains the same.

Thanks for the suggestion!

-Amit

>
> Raphael
>
>
>
>>
>>
>>> Check out the python documentation for more information.
>>>
>>>
>>> And you should move your setup file on the same level as the top-level
>>> package
>>> The reason why find_packages() does'nt find your mainpackage is because
>>> it
>>> is
>>> searching for packages on the same level or below the setup.py.
>>>
>>>
>>> You have to change your directory layout to this:
>>>
>>> ###################
>>> setup.py
>>>
>>> mainpackackage
>>>  demo1.py
>>>  __init.py__
>>>
>>>  subpackage1
>>>      __init.py/__
>>> ####################
>>>
>>
>> Done. Thanks a lot Lutz !
>>
>> -Amit
>>
>>> Cheers,
>>>
>>> Lutz
>>>
>>>
>>>
>>> Am 14.01.2009 um 10:37 schrieb Amit k. Saha:
>>>
>>>> Hello Andreas,
>>>>
>>>> On Wed, Jan 14, 2009 at 2:53 PM, Andreas Jung <lists at zopyx.com> wrote:
>>>>>
>>>>> -----BEGIN PGP SIGNED MESSAGE-----
>>>>> Hash: SHA1
>>>>>
>>>>> On 14.01.2009 10:20 Uhr, Amit k. Saha wrote:
>>>>>>
>>>>>> Hi all,
>>>>>>
>>>>>> I am just learning the basics of building Python eggs using
>>>>>> 'setuptools'. What I have is the following code structure in a
>>>>>> top-down fashion:
>>>>>>
>>>>>>
>>>>>> - demo1.py
>>>>>> - __init.py__
>>>>>> - setup.py
>>>>>> -subpackage1
>>>>>> --__init.py/__
>>>>>>
>>>>>>
>>>>>> When I execute 'python setup.py bdist_egg' from the top-level
>>>>>> directory which has the 'setup.py' file:
>>>>>
>>>>> Using 'sdist' is usually good-enough unless you are working on binary
>>>>> distributions. 'sdist' usually includes everything - especially stuff
>>>>> under revision control (at least when using SVN).
>>>>
>>>> Doesn't help my cause. The result is the same.
>>>>
>>>> PS: None of my sources is under version control.
>>>>
>>>> -Amit
>>>>
>>>>
>>>>> - -aj
>>>>> -----BEGIN PGP SIGNATURE-----
>>>>> Version: GnuPG v1.4.9 (Darwin)
>>>>> Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/
>>>>>
>>>>> iEYEARECAAYFAkltrv4ACgkQCJIWIbr9KYymiQCfbXOaJKOKGnk8pCOouPAxt0Pi
>>>>> RVUAoJyHMsSgdYKUp455EgdhPFmt4mpA
>>>>> =tRaw
>>>>> -----END PGP SIGNATURE-----
>>>>>
>>>>
>>>>
>>>> --
>>>> Amit Kumar Saha
>>>> http://amitksaha.blogspot.com
>>>> http://amitsaha.in.googlepages.com/
>>>> *Bangalore Open Java Users Group*:http:www.bojug.in
>>>> _______________________________________________
>>>> Distutils-SIG maillist  -  Distutils-SIG at python.org
>>>> http://mail.python.org/mailman/listinfo/distutils-sig
>>>>
>>>
>>
>>
>>
>
> _______________________________________________
> Distutils-SIG maillist  -  Distutils-SIG at python.org
> http://mail.python.org/mailman/listinfo/distutils-sig
>



-- 
Amit Kumar Saha
http://amitksaha.blogspot.com
http://amitsaha.in.googlepages.com/
*Bangalore Open Java Users Group*:http:www.bojug.in

From pje at telecommunity.com  Tue Jan 20 16:50:12 2009
From: pje at telecommunity.com (Phillip J. Eby)
Date: Tue, 20 Jan 2009 10:50:12 -0500
Subject: [Distutils] Building Python Eggs
In-Reply-To: <gl4926$jl3$1@ger.gmane.org>
References: <547db2260901140120l2ce45a0co37efd508fd44d8bd@mail.gmail.com>
	<496DAEFE.3030602@zopyx.com>
	<547db2260901140137l76f850f5r46453f777485f300@mail.gmail.com>
	<C62CD0B3-EAB0-47C2-BBDE-D7228FA746A5@gmx.net>
	<547db2260901140207p62e30240l4dac22ba5bcc18ff@mail.gmail.com>
	<gl4926$jl3$1@ger.gmane.org>
Message-ID: <20090120155113.014203A411A@sparrow.telecommunity.com>

At 11:29 AM 1/20/2009 +0100, Raphael Ritz wrote:
>Amit k. Saha wrote:
>>On Wed, Jan 14, 2009 at 3:34 PM, Lutz Paelike <lutz_p at gmx.net> wrote:
>>>Hey Amit,
>>>
>>>a __init__.py file needs to be contained in a folder to make the folder a
>>>package.
>>I had missed the 'folder' thingy..
>
>Glad you've got a solution that works for you
>but IIRC it should be possible to do as you
>originally intended by explicitly adding
>
>   py_modules = ['demo1.py']
>
>in your setup call.

Omit the '.py' though -- that would mean you're installing the 'py' 
module of the 'demo1' package (i.e., 'demo1/py.py') -- which is 
probably not what you want.  ;-)


From stephen.pascoe at stfc.ac.uk  Tue Jan 20 17:37:49 2009
From: stephen.pascoe at stfc.ac.uk (Pascoe, S (Stephen))
Date: Tue, 20 Jan 2009 16:37:49 -0000
Subject: [Distutils] [zc.buildout] borrowing another part's eggs
Message-ID: <EB1E7CB92F5B35459E0B926D2A614DB6036EBF8E@EXCHANGE19.fed.cclrc.ac.uk>

Is there a way for a recipe to pick another part's dependencies so that
it can do the sys.path[0:0] = [...] magic that
zc.buildout.easy_install.scripts does.  I'm trying to add functionality
to a zc.recipe.egg buildout by writing a recipe but I'm finding I want
to use the working_set configured in the zc.recipe.egg part.  E.g.
 
"""
[buildout]
develop-eggs = recipes
parts = foo bar
 
[foo]
recipe = zc.recipe.egg
eggs = ...
 
[bar]
recipe = recipes:myrecipe
# I want to create scripts that include the eggs installed in foo
eggs_from = foo
...
"""
 
At the moment the only option I can see is:
 
 1. Grab config options from the [foo] section and re-parse all options
relating to egg configuration (eggs/dependency-links/etc.)
 2. Call zc.recipe.easy_install.install() to get a working set.
 3. Write sys.path[0:0] = [...] into my scripts based on the new working
set.
 
It seems a little redundant but the zc.buildout.easy_install.scripts API
doesn't quite do what I want.
 
Thanks,
Stephen.
 
---
Stephen Pascoe  +44 (0)1235 445980
British Atmospheric Data Centre
Rutherford Appleton Laboratory
 

-- 
Scanned by iCritical.

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20090120/fe6e7f40/attachment.htm>

From zooko at zooko.com  Tue Jan 20 18:38:39 2009
From: zooko at zooko.com (zooko)
Date: Tue, 20 Jan 2009 10:38:39 -0700
Subject: [Distutils] Problems installing Easy Install/Setup Tools
	(segmentation fault)
In-Reply-To: <5A5D0D13-B9A9-4B0C-9466-CF1A0D682B24@bivald.com>
References: <5A5D0D13-B9A9-4B0C-9466-CF1A0D682B24@bivald.com>
Message-ID: <194CEE60-95FF-49CB-9384-C18973A83647@zooko.com>

This seems unlikely to be setuptools-specific, but more likely a  
problem with your Python interpreter.  What python binary are you using?

Regards,

Zooko

From setuptools at bugs.python.org  Tue Jan 20 20:00:54 2009
From: setuptools at bugs.python.org (Zooko O'Whielacronx)
Date: Tue, 20 Jan 2009 19:00:54 +0000
Subject: [Distutils] [issue57] develop doesn't create .pth files and site.py
	if --multi-version
In-Reply-To: <1232478054.43.0.745098014311.issue57@psf.upfronthosting.co.za>
Message-ID: <1232478054.43.0.745098014311.issue57@psf.upfronthosting.co.za>


New submission from Zooko O'Whielacronx <zooko at zooko.com>:

The allmydata.org tahoe-lafs project uses "./setup.py develop" to build.  We
have started having problems with older versions of dependencies already being
installed, so we want to add "--multi-version" to the "./setup.py develop"
command, but when we do some parts of our build system break.  Investigation
shows that the only difference in the resulting filesystem between "setup.py
develop --multi-version --prefix=support", as shown here:

http://allmydata.org/buildbot/builders/cygwin/builds/1676/steps/install/logs/stdio

and "setup.py develop --prefix=support", as shown here:

http://allmydata.org/buildbot/builders/cygwin/builds/1685/steps/build_tahoe/logs/stdio

Is that in the former case, the "easy-install.pth", "setuptools.pth", and
"site.py" files are not created in the target directory "support".  Is this
intended?

Without those files, the subsequent attempt to use pkg_resources (which is
installed as an install_requirement) fails:

http://allmydata.org/buildbot/builders/cygwin/builds/1676/steps/tahoe-version/logs/stdio

----------
messages: 228
nosy: zooko
priority: bug
status: unread
title: develop doesn't create .pth files and site.py if --multi-version

_______________________________________________
Setuptools tracker <setuptools at bugs.python.org>
<http://bugs.python.org/setuptools/issue57>
_______________________________________________

From zooko at zooko.com  Tue Jan 20 21:56:54 2009
From: zooko at zooko.com (zooko)
Date: Tue, 20 Jan 2009 13:56:54 -0700
Subject: [Distutils] let's have a new release of stdeb!
Message-ID: <3156241E-A8F8-4561-B9F8-70A035948179@zooko.com>

Folks:

We've finally gotten the build/test/install process of allmydata.org  
tahoe working smoothly using setuptools.  (Thanks especially to Chris  
Galvan.)  See our buildbot for many details:

http://allmydata.org/buildbot/waterfall?show_events=true

Here is the list of dependencies which are automatically satisfied by  
setuptools.  (This list is taken from our Ubuntu Dapper buildslave.)

allmydata-tahoe: 1.2.0-r3452, foolscap: 0.3.2, pycryptopp: 0.5.12,  
zfec: 1.4.4, Twisted: 8.2.0, Nevow: 0.9.32, zope.interface: 3.5.0,  
simplejson: 2.0.7, argparse: 0.8.0, pyOpenSSL: 0.8, pyutil: 1.3.30,  
zbase32: 1.1.1, setuptools: 0.6c10dev

For reference, a year ago many of these couldn't be correctly  
installed by setuptools.

The next step that I would like to take is to automate the production  
of .deb's from setup.py's.  The stdeb tool works fairly well for  
this.  I would be especially pleased if Andrew Straw and Pedro  
Algarvio, the maintainers of stdeb, would release a new version  
including the patch that I submitted to automatically detect debian  
dependencies from setuptools dependencies, using apt-listfiles:

https://code.launchpad.net/~astraw/stdeb/autofind-depends

I just looked at the stdeb home page:

http://stdeb.python-hosting.com

and it said:

"""
Call for volunteers
I don't have a lot of time for this. This project stands a very real  
chance of being only a shadow of its potential self unless people  
step up and contribute. There are numerous ways in which people could  
help. In particular, I'd be interested in finding a co-maintainer or  
maintainer if the project generates any interest. Secondarily, I  
would appreciate advice from Debian developers or Ubuntu MOTUs about  
the arcane details of Python packaging.
Please address all questions to the distutils-SIG
"""

So, how about it?

Regards,

Zooko
---
Tahoe, the Least-Authority Filesystem -- http://allmydata.org
store your data: $10/month -- http://allmydata.com/?tracking=zsig

From zooko at zooko.com  Wed Jan 21 00:48:38 2009
From: zooko at zooko.com (zooko)
Date: Tue, 20 Jan 2009 16:48:38 -0700
Subject: [Distutils] patch: solving the two major things that people
	dislike about setuptools, part 1
In-Reply-To: <5AA53D0A-98D4-4AF5-843C-DEE1D37C2D24@zooko.com>
References: <5AA53D0A-98D4-4AF5-843C-DEE1D37C2D24@zooko.com>
Message-ID: <17519F49-7F21-40D0-BCF5-C0A77E0CFCCB@zooko.com>

Folks:

This is just to let you know that my programming partner Brian Warner  
is one of the many data points I have indicating that respecting the  
PYTHONPATH would make setuptools more palatable to more hackers.   
Brian, however, is the most important data point, to me, because I  
work closely with him every day and I greatly respect his software  
engineering practices.

A few days ago he wrote on the tahoe-dev mailing list:

"""
"import" should import whatever comes first in the PYTHONPATH, right?  
So it should be a simple (heh) matter of insuring that the source  
tree or the newly installed location appears earlier in PYTHONPATH?  
'trial' adds the current directory to sys.path before importing the  
code-under-test, so it's designed to either be run from a source tree  
(and test the code in that source tree), or from anywhere other than  
a source tree (and test the code that's installed on your system:  
whatever get used when you do 'import').

I must say, seeing how badly setuptools futzes around with sys.path,  
I can't say I feel confident that the ordering of PYTHONPATH matters  
anymore. I miss this loss of confidence and control.
"""

http://allmydata.org/pipermail/tahoe-dev/2009-January/001028.html

I replied, in part:

"""
PJE, the maintainer of setuptools rejected my patch, and some other  
people on the distutils-sig list replied to say that this was the  
last straw and they were going to fork setuptools.

Hopefully after Tahoe-1.3 release we can start using the fork and it  
will respect the PYTHONPATH.  Also, there's a possibility that we  
could contribute a patch to setuptools that PJE would accept that  
would also make it respect the PYTHONPATH, but I'm not precisely sure  
how it would work.  I intend to think about that at some point, but  
I've been awfully busy with Tahoe.  Please read the thread rooted at  
that URL above for full context.  See also, if you are curious the  
[[issue tickets]] tiddler on my klog which contains a (dizzyingly  
large) list of issue tickets that I'm trying to pay attention to.
"""

So, this is just to let everyone know that any setuptools variant  
(including a future release of PJE's setuptools) that respects the  
PYTHONPATH will get at least two users!

Regards,

Zooko

From pje at telecommunity.com  Wed Jan 21 01:30:47 2009
From: pje at telecommunity.com (Phillip J. Eby)
Date: Tue, 20 Jan 2009 19:30:47 -0500
Subject: [Distutils] patch: solving the two major things that people
 dislike about setuptools, part 1
In-Reply-To: <17519F49-7F21-40D0-BCF5-C0A77E0CFCCB@zooko.com>
References: <5AA53D0A-98D4-4AF5-843C-DEE1D37C2D24@zooko.com>
	<17519F49-7F21-40D0-BCF5-C0A77E0CFCCB@zooko.com>
Message-ID: <20090121002856.405BA3A4065@sparrow.telecommunity.com>

At 04:48 PM 1/20/2009 -0700, zooko wrote:
>So, this is just to let everyone know that any setuptools variant
>(including a future release of PJE's setuptools) that respects the
>PYTHONPATH will get at least two users!

As I've said before, if somebody creates a patch that addresses the 
use cases I pointed out, I'd be happy to accept it.

Please see:

http://mail.python.org/pipermail/distutils-sig/2008-November/010533.html

and the thread surrounding it for details.

Making it sound like I simply reject patches in this area with no 
reason, isn't at all helpful, and I'd appreciate it if you stop doing so.

Perhaps you could solicit volunteers to help develop an appropriate 
patch, propose possible approaches, etc., or ask for clarification if 
my previous explanations of what the patch needs to do are not clear.


From barry at python.org  Wed Jan 21 04:55:22 2009
From: barry at python.org (Barry Warsaw)
Date: Tue, 20 Jan 2009 22:55:22 -0500
Subject: [Distutils] Path problem with zope.testing and buildout
Message-ID: <3FB6B262-90BF-410E-BF4F-49272F180A8D@python.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

I'm not entirely sure this is the right place to report this problem,  
but I'll give it a shot anyway.  I'm sure this is pebkac but I haven't  
figure it out yet.

My current Mailman 3 code is buildout-based.  The branch is here:

https://code.launchpad.net/~mailman-coders/mailman/3.0

Here's my buildout.cfg file:

- -----snip snip-----
[buildout]
parts =
     interpreter
     tags
     test
unzip = true
develop = .

[interpreter]
recipe = zc.recipe.egg
interpreter = py
eggs =
     lazr.config
     lazr.delegates
     locknix
     mailman
     munepy
     storm
     zope.interface

[tags]
recipe = z3c.recipe.tag:tags
eggs = mailman

[test]
recipe = zc.recipe.testrunner
eggs =
     mailman
defaults = '--tests-pattern ^tests --exit-with-status'.split()
# Hack in extra arguments to zope.testrunner.
initialization = from mailman.testing.layers import ConfigLayer;
     ConfigLayer.hack_options_parser()
- -----snip snip-----

If I build this out and then run bin/test, my test suite passes, but I  
get ImportErrors for every dependent egg that gets downloaded.  It's  
as if the eggs directory is being search for tests even though  
eggs=mailman is in the [test] section.

However, if I put the following in my ~/.buildout/default.cfg, I don't  
those ImportErrors, presumably because the eggs are so far distant  
that Zope's testrunner doesn't find them.

- -----snip snip-----
[buildout]
eggs-directory=/Users/barry/.buildout/eggs
download-cache=/Users/barry/.buildout/download-cache
- -----snip snip-----

Here's an excerpt from the bin/test output (with extra blank lines  
suppressed):

Test-module import failures:
Module: eggs.lazr.config-1.1-py2.6.egg.lazr.config.tests
ImportError: No module named eggs.lazr.config-1.1- 
py2.6.egg.lazr.config.tests
Module: eggs.lazr.delegates-1.0-py2.6.egg.lazr.delegates.tests
ImportError: No module named eggs.lazr.delegates-1.0- 
py2.6.egg.lazr.delegates.tests
Module: eggs.zc.buildout-1.1.1-py2.6.egg.zc.buildout.tests
ImportError: No module named eggs.zc.buildout-1.1.1- 
py2.6.egg.zc.buildout.tests
Module: eggs.zc.buildout-1.1.1-py2.6.egg.zc.buildout.testselectingpython
ImportError: No module named eggs.zc.buildout-1.1.1- 
py2.6.egg.zc.buildout.testselectingpython
Module: eggs.zc.recipe.egg-1.1.0-py2.6.egg.zc.recipe.egg.tests
ImportError: No module named eggs.zc.recipe.egg-1.1.0- 
py2.6.egg.zc.recipe.egg.tests
Module: eggs.zc.recipe.testrunner-1.1.0- 
py2.6.egg.zc.recipe.testrunner.tests
ImportError: No module named eggs.zc.recipe.testrunner-1.1.0- 
py2.6.egg.zc.recipe.testrunner.tests
Module: eggs.zope.interface-3.5.0-py2.6-macosx-10.3- 
i386.egg.zope.interface.common.tests.test_idatetime
ImportError: No module named eggs.zope.interface-3.5.0-py2.6- 
macosx-10.3-i386.egg.zope.interface.common.tests.test_idatetime
Module: eggs.zope.interface-3.5.0-py2.6-macosx-10.3- 
i386.egg.zope.interface.common.tests.test_import_interfaces
ImportError: No module named eggs.zope.interface-3.5.0-py2.6- 
macosx-10.3-i386.egg.zope.interface.common.tests.test_import_interfaces
Module: eggs.zope.interface-3.5.0-py2.6-macosx-10.3- 
i386.egg.zope.interface.tests.test_adapter
ImportError: No module named eggs.zope.interface-3.5.0-py2.6- 
macosx-10.3-i386.egg.zope.interface.tests.test_adapter
Module: eggs.zope.interface-3.5.0-py2.6-macosx-10.3- 
i386.egg.zope.interface.tests.test_advice
ImportError: No module named eggs.zope.interface-3.5.0-py2.6- 
macosx-10.3-i386.egg.zope.interface.tests.test_advice
Module: eggs.zope.interface-3.5.0-py2.6-macosx-10.3- 
i386.egg.zope.interface.tests.test_declarations
ImportError: No module named eggs.zope.interface-3.5.0-py2.6- 
macosx-10.3-i386.egg.zope.interface.tests.test_declarations
Module: eggs.zope.interface-3.5.0-py2.6-macosx-10.3- 
i386.egg.zope.interface.tests.test_document
ImportError: No module named eggs.zope.interface-3.5.0-py2.6- 
macosx-10.3-i386.egg.zope.interface.tests.test_document
Module: eggs.zope.interface-3.5.0-py2.6-macosx-10.3- 
i386.egg.zope.interface.tests.test_element
ImportError: No module named eggs.zope.interface-3.5.0-py2.6- 
macosx-10.3-i386.egg.zope.interface.tests.test_element
Module: eggs.zope.interface-3.5.0-py2.6-macosx-10.3- 
i386.egg.zope.interface.tests.test_interface
ImportError: No module named eggs.zope.interface-3.5.0-py2.6- 
macosx-10.3-i386.egg.zope.interface.tests.test_interface
Module: eggs.zope.interface-3.5.0-py2.6-macosx-10.3- 
i386.egg.zope.interface.tests.test_odd_declarations
ImportError: No module named eggs.zope.interface-3.5.0-py2.6- 
macosx-10.3-i386.egg.zope.interface.tests.test_odd_declarations
Module: eggs.zope.interface-3.5.0-py2.6-macosx-10.3- 
i386.egg.zope.interface.tests.test_sorting
ImportError: No module named eggs.zope.interface-3.5.0-py2.6- 
macosx-10.3-i386.egg.zope.interface.tests.test_sorting
Module: eggs.zope.interface-3.5.0-py2.6-macosx-10.3- 
i386.egg.zope.interface.tests.test_verify
ImportError: No module named eggs.zope.interface-3.5.0-py2.6- 
macosx-10.3-i386.egg.zope.interface.tests.test_verify
Module: eggs.zope.testing-3.7.1-py2.6.egg.zope.testing.tests
ImportError: No module named eggs.zope.testing-3.7.1- 
py2.6.egg.zope.testing.tests
Module: eggs.zope.testing-3.7.1-py2.6.egg.zope.testing.testrunner.tests
ImportError: No module named eggs.zope.testing-3.7.1- 
py2.6.egg.zope.testing.testrunner.tests


The same behavior occurs on Linux.

Does anybody have any suggestions for making bin/test work without  
putting your eggs outside the source tree?  Or if it's not possible,  
I'll update my README.txt to explain that you have to have a  
~/.buildout/default.cfg.

Thanks,
Barry

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (Darwin)

iQCVAwUBSXacqnEjvBPtnXfVAQIfoQP+NmPMyOlky4rqSF9UWF/cOi4K/G++YYRO
DodibIMefqhoX+2SEjfavm9NN9nIVt0Wrvaty0acnZ1FHc7pPwN+szUV6is1QAao
Q7WQbZyQewPVk+3tucuBD7UtcMoBpHScq4afL4u4crSozf3DpVI0YwqhYf4hvvo8
XrFTINdDfCI=
=E5eP
-----END PGP SIGNATURE-----

From me at rpatterson.net  Wed Jan 21 07:34:12 2009
From: me at rpatterson.net (Ross Patterson)
Date: Tue, 20 Jan 2009 22:34:12 -0800
Subject: [Distutils] Path problem with zope.testing and buildout
References: <3FB6B262-90BF-410E-BF4F-49272F180A8D@python.org>
Message-ID: <87mydl5frv.fsf@transitory.lefae.org>

You might try the approach I used in the following:

http://rpatterson.net/blog/running-tests-in-egg-buildouts

Ross

Barry Warsaw <barry at python.org> writes:

> I'm not entirely sure this is the right place to report this problem,  
> but I'll give it a shot anyway.  I'm sure this is pebkac but I haven't  
> figure it out yet.
>
> My current Mailman 3 code is buildout-based.  The branch is here:
>
> https://code.launchpad.net/~mailman-coders/mailman/3.0
>
> Here's my buildout.cfg file:
>
> -----snip snip-----
> [buildout]
> parts =
>      interpreter
>      tags
>      test
> unzip = true
> develop = .
>
> [interpreter]
> recipe = zc.recipe.egg
> interpreter = py
> eggs =
>      lazr.config
>      lazr.delegates
>      locknix
>      mailman
>      munepy
>      storm
>      zope.interface
>
> [tags]
> recipe = z3c.recipe.tag:tags
> eggs = mailman
>
> [test]
> recipe = zc.recipe.testrunner
> eggs =
>      mailman
> defaults = '--tests-pattern ^tests --exit-with-status'.split()
> # Hack in extra arguments to zope.testrunner.
> initialization = from mailman.testing.layers import ConfigLayer;
>      ConfigLayer.hack_options_parser()
> -----snip snip-----
>
> If I build this out and then run bin/test, my test suite passes, but I  
> get ImportErrors for every dependent egg that gets downloaded.  It's  
> as if the eggs directory is being search for tests even though  
> eggs=mailman is in the [test] section.
>
> However, if I put the following in my ~/.buildout/default.cfg, I don't  
> those ImportErrors, presumably because the eggs are so far distant  
> that Zope's testrunner doesn't find them.
>
> -----snip snip-----
> [buildout]
> eggs-directory=/Users/barry/.buildout/eggs
> download-cache=/Users/barry/.buildout/download-cache
> -----snip snip-----
>
> Here's an excerpt from the bin/test output (with extra blank lines  
> suppressed):
>
> Test-module import failures:
> Module: eggs.lazr.config-1.1-py2.6.egg.lazr.config.tests
> ImportError: No module named eggs.lazr.config-1.1- 
> py2.6.egg.lazr.config.tests
> Module: eggs.lazr.delegates-1.0-py2.6.egg.lazr.delegates.tests
> ImportError: No module named eggs.lazr.delegates-1.0- 
> py2.6.egg.lazr.delegates.tests
> Module: eggs.zc.buildout-1.1.1-py2.6.egg.zc.buildout.tests
> ImportError: No module named eggs.zc.buildout-1.1.1- 
> py2.6.egg.zc.buildout.tests
> Module: eggs.zc.buildout-1.1.1-py2.6.egg.zc.buildout.testselectingpython
> ImportError: No module named eggs.zc.buildout-1.1.1- 
> py2.6.egg.zc.buildout.testselectingpython
> Module: eggs.zc.recipe.egg-1.1.0-py2.6.egg.zc.recipe.egg.tests
> ImportError: No module named eggs.zc.recipe.egg-1.1.0- 
> py2.6.egg.zc.recipe.egg.tests
> Module: eggs.zc.recipe.testrunner-1.1.0- 
> py2.6.egg.zc.recipe.testrunner.tests
> ImportError: No module named eggs.zc.recipe.testrunner-1.1.0- 
> py2.6.egg.zc.recipe.testrunner.tests
> Module: eggs.zope.interface-3.5.0-py2.6-macosx-10.3- 
> i386.egg.zope.interface.common.tests.test_idatetime
> ImportError: No module named eggs.zope.interface-3.5.0-py2.6- 
> macosx-10.3-i386.egg.zope.interface.common.tests.test_idatetime
> Module: eggs.zope.interface-3.5.0-py2.6-macosx-10.3- 
> i386.egg.zope.interface.common.tests.test_import_interfaces
> ImportError: No module named eggs.zope.interface-3.5.0-py2.6- 
> macosx-10.3-i386.egg.zope.interface.common.tests.test_import_interfaces
> Module: eggs.zope.interface-3.5.0-py2.6-macosx-10.3- 
> i386.egg.zope.interface.tests.test_adapter
> ImportError: No module named eggs.zope.interface-3.5.0-py2.6- 
> macosx-10.3-i386.egg.zope.interface.tests.test_adapter
> Module: eggs.zope.interface-3.5.0-py2.6-macosx-10.3- 
> i386.egg.zope.interface.tests.test_advice
> ImportError: No module named eggs.zope.interface-3.5.0-py2.6- 
> macosx-10.3-i386.egg.zope.interface.tests.test_advice
> Module: eggs.zope.interface-3.5.0-py2.6-macosx-10.3- 
> i386.egg.zope.interface.tests.test_declarations
> ImportError: No module named eggs.zope.interface-3.5.0-py2.6- 
> macosx-10.3-i386.egg.zope.interface.tests.test_declarations
> Module: eggs.zope.interface-3.5.0-py2.6-macosx-10.3- 
> i386.egg.zope.interface.tests.test_document
> ImportError: No module named eggs.zope.interface-3.5.0-py2.6- 
> macosx-10.3-i386.egg.zope.interface.tests.test_document
> Module: eggs.zope.interface-3.5.0-py2.6-macosx-10.3- 
> i386.egg.zope.interface.tests.test_element
> ImportError: No module named eggs.zope.interface-3.5.0-py2.6- 
> macosx-10.3-i386.egg.zope.interface.tests.test_element
> Module: eggs.zope.interface-3.5.0-py2.6-macosx-10.3- 
> i386.egg.zope.interface.tests.test_interface
> ImportError: No module named eggs.zope.interface-3.5.0-py2.6- 
> macosx-10.3-i386.egg.zope.interface.tests.test_interface
> Module: eggs.zope.interface-3.5.0-py2.6-macosx-10.3- 
> i386.egg.zope.interface.tests.test_odd_declarations
> ImportError: No module named eggs.zope.interface-3.5.0-py2.6- 
> macosx-10.3-i386.egg.zope.interface.tests.test_odd_declarations
> Module: eggs.zope.interface-3.5.0-py2.6-macosx-10.3- 
> i386.egg.zope.interface.tests.test_sorting
> ImportError: No module named eggs.zope.interface-3.5.0-py2.6- 
> macosx-10.3-i386.egg.zope.interface.tests.test_sorting
> Module: eggs.zope.interface-3.5.0-py2.6-macosx-10.3- 
> i386.egg.zope.interface.tests.test_verify
> ImportError: No module named eggs.zope.interface-3.5.0-py2.6- 
> macosx-10.3-i386.egg.zope.interface.tests.test_verify
> Module: eggs.zope.testing-3.7.1-py2.6.egg.zope.testing.tests
> ImportError: No module named eggs.zope.testing-3.7.1- 
> py2.6.egg.zope.testing.tests
> Module: eggs.zope.testing-3.7.1-py2.6.egg.zope.testing.testrunner.tests
> ImportError: No module named eggs.zope.testing-3.7.1- 
> py2.6.egg.zope.testing.testrunner.tests
>
>
> The same behavior occurs on Linux.
>
> Does anybody have any suggestions for making bin/test work without  
> putting your eggs outside the source tree?  Or if it's not possible,  
> I'll update my README.txt to explain that you have to have a  
> ~/.buildout/default.cfg.
>
> Thanks,
> Barry
>
> _______________________________________________
> Distutils-SIG maillist  -  Distutils-SIG at python.org
> http://mail.python.org/mailman/listinfo/distutils-sig


From jim at zope.com  Wed Jan 21 16:18:59 2009
From: jim at zope.com (Jim Fulton)
Date: Wed, 21 Jan 2009 10:18:59 -0500
Subject: [Distutils] zc.buildout 'extends' options
In-Reply-To: <4971C758.9070802@gmail.com>
References: <4971C758.9070802@gmail.com>
Message-ID: <124B5751-3521-42CB-B058-F3E3A5269019@zope.com>


On Jan 17, 2009, at 6:56 AM, Martin Aspeli wrote:

> Hi Jim & co,
>
> Plone 3.2 currently distributes its packages as eggs, and people use  
> an 'extends' option with a remote URL to get a known good set for  
> each version, e.g.:
>
> [buildout]
> extends = http://dist.plone.org/release/3.2/versions.cfg
>
> This is similar to how Grok does its distributions, of course.
>
> Now, as far as I can tell, this doesn't work offline. -o mode does  
> not complain, but if you're totally offline (turn off your network  
> interface), then urllib throws an error when buildout tries to  
> evaluate the 'extends' option, and buildout crashes completely.
>
> I'd like to have an optional download cache directory, e.g.
>
> [buildout]
> extends-cache = extends-cache-directory
> extends = http://dist.plone.org/release/3.2/versions.cfg
>
> In offline mode, buildout would look for the file to extend  
> (probably with some hash of the url used as filename) in the  
> directory specified by extends-cache.
>
> This would allow full offline mode (e.g. on a machine with no  
> network card configured). In particular, it would allow Plone's  
> buildout-based installers to ship with the versions.cfg file and not  
> require people to be able to download them.
>
> Is this desirable?

I think so.  I have a nagging concern that this might be better  
handled at a lower level, but having buildout aware of the caching  
means that zc.sourcerelease would also be able to take advantage of  
it, so, for example, you could build a source release from a buildout  
that used remote configuration.

> Any plans for such a feature?

Not yet. :)

> Would you accept a patch, if we could make it work?


Sure, assuming that it included tests.  Also, rather than a patch, I'd  
prefer a development branch if you have access to the zope.org  
repository.

Jim

--
Jim Fulton
Zope Corporation



From jim at zope.com  Wed Jan 21 18:43:09 2009
From: jim at zope.com (Jim Fulton)
Date: Wed, 21 Jan 2009 12:43:09 -0500
Subject: [Distutils] [zc.buildout] borrowing another part's eggs
In-Reply-To: <EB1E7CB92F5B35459E0B926D2A614DB6036EBF8E@EXCHANGE19.fed.cclrc.ac.uk>
References: <EB1E7CB92F5B35459E0B926D2A614DB6036EBF8E@EXCHANGE19.fed.cclrc.ac.uk>
Message-ID: <0DCFB903-A079-422B-8AE8-5AD5E57AFFF1@zope.com>


On Jan 20, 2009, at 11:37 AM, Pascoe, S (Stephen) wrote:

> Is there a way for a recipe to pick another part's dependencies so  
> that it can do the sys.path[0:0] = [...] magic that  
> zc.buildout.easy_install.scripts does.  I'm trying to add  
> functionality to a zc.recipe.egg buildout by writing a recipe but  
> I'm finding I want to use the working_set configured in the  
> zc.recipe.egg part.  E.g.
>
> """
> [buildout]
> develop-eggs = recipes
> parts = foo bar
>
> [foo]
> recipe = zc.recipe.egg
> eggs = ...
>
> [bar]
> recipe = recipes:myrecipe
> # I want to create scripts that include the eggs installed in foo
> eggs_from = foo
> ...
> """
>
> At the moment the only option I can see is:
>
>  1. Grab config options from the [foo] section and re-parse all  
> options relating to egg configuration (eggs/dependency-links/etc.)
>  2. Call zc.recipe.easy_install.install() to get a working set.
>  3. Write sys.path[0:0] = [...] into my scripts based on the new  
> working set.
>
> It seems a little redundant but the zc.buildout.easy_install.scripts  
> API doesn't quite do what I want.

You can have your recipe extend the zc.recipe.egg:scripts recipe.  See

   http://pypi.python.org/pypi/zc.recipe.egg#egg-recipe-api-for-other-recipes

zc.recipe.testrunner is one of several examples of of doing this.

Then, you'd have:

   [bar]
   recipe = recipes:myrecipe
   eggs = ${foo:eggs}
   ...

Jim

--
Jim Fulton
Zope Corporation



From ziade.tarek at gmail.com  Wed Jan 21 23:29:31 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Wed, 21 Jan 2009 23:29:31 +0100
Subject: [Distutils] get_metadata in Distutils
In-Reply-To: <20090115151816.119EE3A410E@sparrow.telecommunity.com>
References: <94bdd2610901100948t65cba1achc2764c97195f529d@mail.gmail.com>
	<20090110200709.D3B3B3A40A0@sparrow.telecommunity.com>
	<94bdd2610901111122y156b691dh1db438a1b6fedc14@mail.gmail.com>
	<20090112025933.411B63A40A0@sparrow.telecommunity.com>
	<94bdd2610901120511g28130ef5se249e0a87f4eb2ab@mail.gmail.com>
	<20090112160544.F25853A40A0@sparrow.telecommunity.com>
	<94bdd2610901150224m53397703qde293cf9dc61ea92@mail.gmail.com>
	<20090115151816.119EE3A410E@sparrow.telecommunity.com>
Message-ID: <94bdd2610901211429n1a366863n6f1b857a2bb239ea@mail.gmail.com>

On Thu, Jan 15, 2009 at 4:20 PM, Phillip J. Eby <pje at telecommunity.com> wrote:
>> I am wondering about other cases though:
>>
>> - it seems that pkg_resources works on .egg-info *directories*. When
>> does this case happen ?
>
> System packages (.rpm, .deb, etc.), or other install
> --root/--single-version-externally-managed installs of setuptools-based
> packages.

Ok, so in that case, the .egg-info directory contains a PKG-INFO file
right inside ?

>
>
>> - you said that its needs to work on nested packages, but I can't see
>> the difference when grabbing the metadata in such a package
>
> More precisely: you can have nested .egg directories within a single
> zipfile.  This is to support bundling a bunch of eggs in a single zipfile
> for easy application distribution.

I see. Does the global zip file have a specific extension ? (like .egg) or
it can be any zip file as long as it contains .egg directories inside.

I am sorry to ask those question, but I got lost in the code for those
two particular cases,

Regards
Tarek

-- 
Tarek Ziad? | Association AfPy | www.afpy.org
Blog FR | http://programmation-python.org
Blog EN | http://tarekziade.wordpress.com/

From akitada at gmail.com  Wed Jan 21 23:42:37 2009
From: akitada at gmail.com (Akira Kitada)
Date: Thu, 22 Jan 2009 07:42:37 +0900
Subject: [Distutils] Extending distutils with a package in site-packages?
Message-ID: <90bb445a0901211442o622a1771n428598ccf0c0beca@mail.gmail.com>

Hi,

I'm trying to write a module 'mydistutils', an extension to original distutils.
I would like to make it distutils-compatible so that user of the
module use mydistutils interchangeably.
The only difference between them would be the addition of
'MyDistribution' and 'bdist_mycmd'.
Because I don't want to mess up my python installation, It want to put
all of those extension parts in site-packages/.
But if I do that way, I cannot do "from mydistutils import core" or
the like because mydistutils is in different package...
Is there any pythonic tips for dealing this problem?
I'd like to keep it simple, so setuptools-free solution would be preferable

Thanks,

From ziade.tarek at gmail.com  Wed Jan 21 23:54:52 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Wed, 21 Jan 2009 23:54:52 +0100
Subject: [Distutils] Extending distutils with a package in site-packages?
In-Reply-To: <90bb445a0901211442o622a1771n428598ccf0c0beca@mail.gmail.com>
References: <90bb445a0901211442o622a1771n428598ccf0c0beca@mail.gmail.com>
Message-ID: <94bdd2610901211454i2b465897mcf236394b9a4033a@mail.gmail.com>

On Wed, Jan 21, 2009 at 11:42 PM, Akira Kitada <akitada at gmail.com> wrote:
> Hi,
>
> I'm trying to write a module 'mydistutils', an extension to original distutils.
> I would like to make it distutils-compatible so that user of the
> module use mydistutils interchangeably.
> The only difference between them would be the addition of
> 'MyDistribution' and 'bdist_mycmd'.
> Because I don't want to mess up my python installation, It want to put
> all of those extension parts in site-packages/.
> But if I do that way, I cannot do "from mydistutils import core" or
> the like because mydistutils is in different package...
> Is there any pythonic tips for dealing this problem?
> I'd like to keep it simple, so setuptools-free solution would be preferable

Do you mean, how to distribute your distutils extension ?

if so, you have to create a package with its own setup.py, and use
full paths in your import directives.


Regards
Tarek

>
> Thanks,
> _______________________________________________
> Distutils-SIG maillist  -  Distutils-SIG at python.org
> http://mail.python.org/mailman/listinfo/distutils-sig
>



-- 
Tarek Ziad? | Association AfPy | www.afpy.org
Blog FR | http://programmation-python.org
Blog EN | http://tarekziade.wordpress.com/

From akitada at gmail.com  Thu Jan 22 00:35:40 2009
From: akitada at gmail.com (Akira Kitada)
Date: Thu, 22 Jan 2009 08:35:40 +0900
Subject: [Distutils] Extending distutils with a package in site-packages?
In-Reply-To: <94bdd2610901211454i2b465897mcf236394b9a4033a@mail.gmail.com>
References: <90bb445a0901211442o622a1771n428598ccf0c0beca@mail.gmail.com>
	<94bdd2610901211454i2b465897mcf236394b9a4033a@mail.gmail.com>
Message-ID: <90bb445a0901211535p59b62459ibfc405161f527596@mail.gmail.com>

Hi Tarek,

I asked that how can I write 'mydistutils' package pythonic way.
The 'mydistutils' is distutils with only two additional things:
- MyDistribution (Based on Distribution
- bdist_mycmd (Based on Command)

If I copied whole distutils/ package into site-packages/mydistutils,
it should work, but I'm sure this is ugly and a serious kludge.
importing everything of distutils in mydistutils __init__.py might also work,
but I think this is not the way to go.

At the other place, I was suggested to use __path__ to workaround this.
It seems to me a much better than other solutions,
but I would like to know there're any other recommendations before
diving into __path__ solution...


> Do you mean, how to distribute your distutils extension ?
>
> if so, you have to create a package with its own setup.py, and use
> full paths in your import directives.

From optilude+lists at gmail.com  Thu Jan 22 00:38:39 2009
From: optilude+lists at gmail.com (Martin Aspeli)
Date: Wed, 21 Jan 2009 23:38:39 +0000
Subject: [Distutils] zc.buildout 'extends' options
In-Reply-To: <124B5751-3521-42CB-B058-F3E3A5269019@zope.com>
References: <4971C758.9070802@gmail.com>
	<124B5751-3521-42CB-B058-F3E3A5269019@zope.com>
Message-ID: <gl8blv$20p$1@ger.gmane.org>

Hi Jim,

>> Is this desirable?
> 
> I think so.  I have a nagging concern that this might be better  
> handled at a lower level, but having buildout aware of the caching  
> means that zc.sourcerelease would also be able to take advantage of  
> it, so, for example, you could build a source release from a buildout  
> that used remote configuration.

Cool.

>> Any plans for such a feature?
> 
> Not yet. :)
> 
>> Would you accept a patch, if we could make it work?
> 
> 
> Sure, assuming that it included tests.  Also, rather than a patch, I'd  
> prefer a development branch if you have access to the zope.org  
> repository.

Sure. Branch off trunk?

I'm seeing some test failures on trunk, by the way (OS X, Python 2.4). 
Are these expected?

$ ./bin/test
Running zope.testing.testrunner.layer.UnitTests tests:
   Set up zope.testing.testrunner.layer.UnitTests in 0.000 seconds.
zip_safe flag not set; analyzing archive contents...
zip_safe flag not set; analyzing archive contents...
zip_safe flag not set; analyzing archive contents...
zip_safe flag not set; analyzing archive contents...
zip_safe flag not set; analyzing archive contents...
zip_safe flag not set; analyzing archive contents...


Failure in test test_bootstrap_py (zc.buildout.tests)
Failed doctest test for zc.buildout.tests.test_bootstrap_py
   File 
"/Users/optilude/Development/Plone/Code/Build/buildout/zc.buildout/src/zc/buildout/tests.py", 
line 581, in test_bootstrap_py

----------------------------------------------------------------------
File 
"/Users/optilude/Development/Plone/Code/Build/buildout/zc.buildout/src/zc/buildout/tests.py", 
line 592, in zc.buildout.tests.test_bootstrap_py
Failed example:
     print system(zc.buildout.easy_install._safe_arg(sys.executable)+' '+
                  'bootstrap.py'), # doctest: +ELLIPSIS
Expected:
     Downloading ...
     Generated script '/sample/bin/buildout'.
Got:
     Creating directory '/sample/bin'.
     Creating directory '/sample/parts'.
     Creating directory '/sample/eggs'.
     Creating directory '/sample/develop-eggs'.
     Generated script '/sample/bin/buildout'.
----------------------------------------------------------------------
File 
"/Users/optilude/Development/Plone/Code/Build/buildout/zc.buildout/src/zc/buildout/tests.py", 
line 609, in zc.buildout.tests.test_bootstrap_py
Failed example:
     ls(sample_buildout, 'eggs')
Expected:
     -  setuptools.eggpyN.N.egg
     d  zc.buildout.eggpyN.N.egg
Got:
     d  setuptools.eggpyN.N.egg
     d  zc.buildout.eggpyN.N.egg

zip_safe flag not set; analyzing archive contents...


Failure in test 
/Users/optilude/Development/Plone/Code/Build/buildout/zc.buildout/zc.recipe.egg_/src/zc/recipe/egg/api.txt
Failed doctest test for api.txt
   File 
"/Users/optilude/Development/Plone/Code/Build/buildout/zc.buildout/zc.recipe.egg_/src/zc/recipe/egg/api.txt", 
line 0

----------------------------------------------------------------------
File 
"/Users/optilude/Development/Plone/Code/Build/buildout/zc.buildout/zc.recipe.egg_/src/zc/recipe/egg/api.txt", 
line 97, in api.txt
Failed example:
     cat(sample_buildout, '.installed.cfg')
Expected:
     [buildout]
     installed_develop_eggs = /sample-buildout/develop-eggs/sample.egg-link
     parts = sample-part
     <BLANKLINE>
     [sample-part]
     __buildout_installed__ =
     __buildout_signature__ = sample- zc.recipe.egg-_b = 
/sample-buildout/bin
     _d = /sample-buildout/develop-eggs
     _e = /sample-buildout/eggs
     bin-directory = /sample-buildout/bin
     develop-eggs-directory = /sample-buildout/develop-eggs
     eggs = demo<0.3
     eggs-directory = /sample-buildout/eggs
     executable = python
     extras = other
     find-links = http://localhost:8080/
     index = http://localhost:8080/index
     recipe = sample
Got:
     [buildout]
     installed_develop_eggs = /sample-buildout/develop-eggs/sample.egg-link
     parts = sample-part
     <BLANKLINE>
     [sample-part]
     __buildout_installed__ =
     __buildout_signature__ = sample- zc.recipe.egg-_b = 
/sample-buildout/bin
     _d = /sample-buildout/develop-eggs
     _e = /sample-buildout/eggs
     bin-directory = /sample-buildout/bin
     develop-eggs-directory = /sample-buildout/develop-eggs
     eggs = demo<0.3
     eggs-directory = /sample-buildout/eggs
     executable = 
/opt/local/Library/Frameworks/Python.framework/Versions/2.4/Resources/Python.app/Contents/MacOS/Python
     extras = other
     find-links = http://localhost:8080/
     index = http://localhost:8080/index
     recipe = sample

   Ran 360 tests with 2 failures and 0 errors in 5 minutes 0.629 seconds.
Tearing down left over layers:
   Tear down zope.testing.testrunner.layer.UnitTests in 0.000 seconds.

Martin


-- 
Author of `Professional Plone Development`, a book for developers who
want to work with Plone. See http://martinaspeli.net/plone-book


From barry at python.org  Thu Jan 22 05:50:56 2009
From: barry at python.org (Barry Warsaw)
Date: Wed, 21 Jan 2009 23:50:56 -0500
Subject: [Distutils] Path problem with zope.testing and buildout
In-Reply-To: <87mydl5frv.fsf@transitory.lefae.org>
References: <3FB6B262-90BF-410E-BF4F-49272F180A8D@python.org>
	<87mydl5frv.fsf@transitory.lefae.org>
Message-ID: <E22BFEE2-7225-41AE-AABB-6496293B46E7@python.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Jan 21, 2009, at 1:34 AM, Ross Patterson wrote:

> You might try the approach I used in the following:
>
> http://rpatterson.net/blog/running-tests-in-egg-buildouts

Indeed, sticking my top-level package under 'src' fixes the problem.   
Perhaps it's the same as this bug:

https://bugs.edge.launchpad.net/zope.testing/+bug/251759

- -Barry

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (Darwin)

iQCVAwUBSXf7MXEjvBPtnXfVAQJivQQArCEiHYI4X3TXt5TYyWgrlyfrC4OLWhzZ
d2hbez6I8pox+sn93KZhcOKyFvnOT3HKLwQPRRUFnX8R45j6c7z+5kS+7oTKaja3
PXBTGjwA4owDTzRbr/marFFIRx7KTpJZLlTT+Hx0peqyBWxuQ6Zmidl49bIN5aHs
Pxuss9S2e3M=
=3Q7M
-----END PGP SIGNATURE-----

From jim at zope.com  Thu Jan 22 13:24:59 2009
From: jim at zope.com (Jim Fulton)
Date: Thu, 22 Jan 2009 07:24:59 -0500
Subject: [Distutils] zc.buildout 'extends' options
In-Reply-To: <gl8blv$20p$1@ger.gmane.org>
References: <4971C758.9070802@gmail.com>
	<124B5751-3521-42CB-B058-F3E3A5269019@zope.com>
	<gl8blv$20p$1@ger.gmane.org>
Message-ID: <C14B3EB5-6961-47D7-B988-8D87B41F2433@zope.com>


On Jan 21, 2009, at 6:38 PM, Martin Aspeli wrote:
...
>>> Would you accept a patch, if we could make it work?
>> Sure, assuming that it included tests.  Also, rather than a patch,  
>> I'd  prefer a development branch if you have access to the  
>> zope.org  repository.
>
> Sure. Branch off trunk?

Yup.

> I'm seeing some test failures on trunk, by the way (OS X, Python  
> 2.4). Are these expected?


No, but I haven't run the tests myself in a while. Other folks have  
been making changes.

Jim

--
Jim Fulton
Zope Corporation



From sienkiew at stsci.edu  Thu Jan 22 21:00:48 2009
From: sienkiew at stsci.edu (Mark Sienkiewicz)
Date: Thu, 22 Jan 2009 15:00:48 -0500
Subject: [Distutils] patch: solving the two major things that people
 dislike about setuptools, part 1
In-Reply-To: <20090121002856.405BA3A4065@sparrow.telecommunity.com>
References: <5AA53D0A-98D4-4AF5-843C-DEE1D37C2D24@zooko.com>	<17519F49-7F21-40D0-BCF5-C0A77E0CFCCB@zooko.com>
	<20090121002856.405BA3A4065@sparrow.telecommunity.com>
Message-ID: <4978D070.9010709@stsci.edu>

Phillip J. Eby wrote:
> At 04:48 PM 1/20/2009 -0700, zooko wrote:
>> So, this is just to let everyone know that any setuptools variant
>> (including a future release of PJE's setuptools) that respects the
>> PYTHONPATH will get at least two users!
>
> As I've said before, if somebody creates a patch that addresses the 
> use cases I pointed out, I'd be happy to accept it.

As I understand it, you require that a package installed by setuptools 
will be discovered before a package installed by distutils >in the same 
directory<. 

You do not require that a package installed by setuptools be discovered 
before a package that is earlier in sys.path.

In that case, the .egg file should go on sys.path immediately before the 
directory where it is stored.  This ensures that a setuptools .egg file 
will always be discovered before a distutils-installed package/module in 
the same directory, but it will not override packages/modules that occur 
earlier in sys.path.

Do you have any other requirements?

In 
http://mail.python.org/pipermail/distutils-sig/2008-November/010534.html 
, I described a change to zooko's patch that implements this behaviour.

In 
http://mail.python.org/pipermail/distutils-sig/2008-November/010540.html 
, I address concerns about case-insensitive filesystems. 

So, taken together, does my suggestion meet your criteria for an 
acceptable patch?

If so,

- I will assemble it into a patch.
- I will test it on platforms that I have access to, in ways that I know 
how to use setuptools.  (For the record, that is "python setup.py 
install --home=/dir"; I never tried the automatic downloads.)

If not,

- what further requirements do you have?

Mark S.


From ben+python at benfinney.id.au  Thu Jan 22 23:04:44 2009
From: ben+python at benfinney.id.au (Ben Finney)
Date: Fri, 23 Jan 2009 09:04:44 +1100
Subject: [Distutils] patch: solving the two major things that people
	dislike about setuptools, part 1
References: <5AA53D0A-98D4-4AF5-843C-DEE1D37C2D24@zooko.com>
	<17519F49-7F21-40D0-BCF5-C0A77E0CFCCB@zooko.com>
	<20090121002856.405BA3A4065@sparrow.telecommunity.com>
	<4978D070.9010709@stsci.edu>
Message-ID: <87iqo72e0z.fsf@benfinney.id.au>

Mark Sienkiewicz <sienkiew at stsci.edu> writes:

> In that case, the .egg file should go on sys.path immediately before
> the directory where it is stored. This ensures that a setuptools
> .egg file will always be discovered before a distutils-installed
> package/module in the same directory, but it will not override
> packages/modules that occur earlier in sys.path.

What of distributions installed via setuptools that are *not*
installed as eggs?

-- 
 \           ?I do not believe in forgiveness as it is preached by the |
  `\        church. We do not need the forgiveness of God, but of each |
_o__)                    other and of ourselves.? ?Robert G. Ingersoll |
Ben Finney


From sienkiew at stsci.edu  Fri Jan 23 23:43:43 2009
From: sienkiew at stsci.edu (Mark Sienkiewicz)
Date: Fri, 23 Jan 2009 17:43:43 -0500
Subject: [Distutils] patch: solving the two major things that
 people	dislike about setuptools, part 1
In-Reply-To: <87iqo72e0z.fsf@benfinney.id.au>
References: <5AA53D0A-98D4-4AF5-843C-DEE1D37C2D24@zooko.com>	<17519F49-7F21-40D0-BCF5-C0A77E0CFCCB@zooko.com>	<20090121002856.405BA3A4065@sparrow.telecommunity.com>	<4978D070.9010709@stsci.edu>
	<87iqo72e0z.fsf@benfinney.id.au>
Message-ID: <497A481F.4000703@stsci.edu>

Ben Finney wrote:
> Mark Sienkiewicz <sienkiew at stsci.edu> writes:
>
>   
>> In that case, the .egg file should go on sys.path immediately before
>> the directory where it is stored. This ensures that a setuptools
>> .egg file will always be discovered before a distutils-installed
>> package/module in the same directory, but it will not override
>> packages/modules that occur earlier in sys.path.
>>     
>
> What of distributions installed via setuptools that are *not*
> installed as eggs?
>
>   

I don't know.  Can you tell me more about that?  I'm not particularly 
familiar with setuptools -- I'm just offering to write the patch because 
I don't see it getting fixed any other way.

Every thing that I have ever installed with setuptools has landed in a 
file or directory named like "*.egg".  For example:
site-packages/nose-0.10.4-py2.5.egg
site-packages/Sphinx-0.5.1-py2.5.egg
site-packages/Jinja-1.2-py2.5-macosx-10.3-i386.egg

The installation creates a file "easy-install.pth" that looks like this:

% cat easy-install.pth
import sys; sys.__plen = len(sys.path)
./setuptools-0.6c9-py2.5.egg
./nose-0.10.4-py2.5.egg
./Sphinx-0.5.1-py2.5.egg
./Jinja-1.2-py2.5-macosx-10.3-i386.egg
./Pygments-1.0-py2.5.egg
./docutils-0.4-py2.5.egg
import sys; new=sys.path[sys.__plen:]; del sys.path[sys.__plen:]; 
p=getattr(sys,'__egginsert',0); sys.path[p:p]=new; sys.__egginsert = 
p+len(new)
%

So, where I said ".egg files", you can substitute the name of the things 
on line 2 through N-1 in the easy-install.pth file. 

That last line of easy-install.pth guarantees that the .egg files (in 
this example, they are all eggs) will be placed in the wrong place on 
sys.path, and that is all I am proposing to fix.  If there is another 
case that does not involve the easy-install.pth file, that is outside 
the scope of my patch.

So, about distributions that are not installed as eggs: Can you tell me 
a little about it?  Can you give me an example of how I can see it 
happen?  Or does it turn out to be irrelevant to my patch?

Mark S.


From fredbasset1000 at gmail.com  Fri Jan 23 23:50:21 2009
From: fredbasset1000 at gmail.com (fred basset)
Date: Fri, 23 Jan 2009 14:50:21 -0800
Subject: [Distutils] easy_install with python 2.4 and 2.5 installed
Message-ID: <53f6cdc90901231450s41cea456ied4e63c62d861a25@mail.gmail.com>

Hi All,

My Fedora machine has python 2.5 installed, but I needed to install Python
2.4 on it for TurboGears.  TurboGears needs some extra libraries such as
pytz, sqlalchem.
How can I use easy_install to install those libraries for python 2.4 only?
Easy_install is already setup on my machine for python 2.5

Thanks,
Fred
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20090123/73b7f221/attachment.htm>

From ben+python at benfinney.id.au  Sat Jan 24 02:00:11 2009
From: ben+python at benfinney.id.au (Ben Finney)
Date: Sat, 24 Jan 2009 12:00:11 +1100
Subject: [Distutils] patch: solving the two major things that
	people	dislike about setuptools, part 1
References: <5AA53D0A-98D4-4AF5-843C-DEE1D37C2D24@zooko.com>
	<17519F49-7F21-40D0-BCF5-C0A77E0CFCCB@zooko.com>
	<20090121002856.405BA3A4065@sparrow.telecommunity.com>
	<4978D070.9010709@stsci.edu> <87iqo72e0z.fsf@benfinney.id.au>
	<497A481F.4000703@stsci.edu>
Message-ID: <87ab9h34dg.fsf@benfinney.id.au>

Mark Sienkiewicz <sienkiew at stsci.edu> writes:

> Ben Finney wrote:
> > What of distributions installed via setuptools that are *not*
> > installed as eggs?
> 
> I don't know.  Can you tell me more about that?

It's rather unfortunate that the term ?egg? has been used to mean
two rather different things in setuptools.

AIUI, an ?egg? is one possible *format* for a distribution: a zipped
collection of the distribution files and metadata files
<URL:http://peak.telecommunity.com/DevCenter/PythonEggs>. It's
supposed to be equivalent to a Java ?jar?, in that it can be copied
around easily and imported from directly via setuptools's
infrastructure. This is offset by the relative opacity of it: it's
more difficult to integrate with a proper OS-level package manager, so
many developers deliberately *avoid* building or distributing eggs.

When building a distribution via setuptools, an egg is *optional*; the
distribution may be in an egg, or it may be in a regular tarball or
whatever. When installed, it may or may not be installed as an egg
<URL:http://peak.telecommunity.com/DevCenter/setuptools#install-run-easy-install-or-old-style-installation>.

The confusion is compounded by the fact that setuptools creates its
metadata in a directory named ?foo.egg-info/?, even if the
distribution is not built to an egg and never will be.

Any of the above could be wrong: I'm not a setuptools expert either,
and have only rubbed up against its rough edges trying to get things
working.

-- 
 \        ?I installed a skylight in my apartment. The people who live |
  `\                             above me are furious!? ?Steven Wright |
_o__)                                                                  |
Ben Finney


From iElectric at gmail.com  Sat Jan 24 02:40:50 2009
From: iElectric at gmail.com (=?windows-1252?Q?Domen_Ko=9Ear?=)
Date: Fri, 23 Jan 2009 17:40:50 -0800 (PST)
Subject: [Distutils] including an empty directory
Message-ID: <116b0f6c-31aa-4a10-9d1c-ca57a808d19e@i20g2000prf.googlegroups.com>

I'm having issues to include an empty directory to setuptools package.

      package_data={'data':['templates/*tmpl']}

this includes all the files, but there is data/templates/Test
directory that is empty and there is no way of globing it.

From pje at telecommunity.com  Sat Jan 24 05:07:07 2009
From: pje at telecommunity.com (Phillip J. Eby)
Date: Fri, 23 Jan 2009 23:07:07 -0500
Subject: [Distutils] easy_install with python 2.4 and 2.5 installed
In-Reply-To: <53f6cdc90901231450s41cea456ied4e63c62d861a25@mail.gmail.co
 m>
References: <53f6cdc90901231450s41cea456ied4e63c62d861a25@mail.gmail.com>
Message-ID: <7.0.1.0.0.20090123230638.027a7678@telecommunity.com>

At 05:50 PM 1/23/2009, fred basset wrote:
>Hi All,
>
>My Fedora machine has python 2.5 installed, but I needed to install 
>Python 2.4 on it for TurboGears.  TurboGears needs some extra 
>libraries such as pytz, sqlalchem.
>How can I use easy_install to install those libraries for python 2.4 
>only?  Easy_install is already setup on my machine for python 2.5


python2.4 -m easy_install

or:

easy_install-2.4


From pje at telecommunity.com  Sat Jan 24 05:06:29 2009
From: pje at telecommunity.com (Phillip J. Eby)
Date: Fri, 23 Jan 2009 23:06:29 -0500
Subject: [Distutils] patch: solving the two major things that
 people	dislike about setuptools, part 1
In-Reply-To: <497A481F.4000703@stsci.edu>
References: <5AA53D0A-98D4-4AF5-843C-DEE1D37C2D24@zooko.com>
	<17519F49-7F21-40D0-BCF5-C0A77E0CFCCB@zooko.com>
	<20090121002856.405BA3A4065@sparrow.telecommunity.com>
	<4978D070.9010709@stsci.edu> <87iqo72e0z.fsf@benfinney.id.au>
	<497A481F.4000703@stsci.edu>
Message-ID: <7.0.1.0.0.20090123230523.0278fec8@telecommunity.com>

Packages that aren't installed in eggs don't need to be "before their 
own directory".  The only reason that .egg files or directories do, 
is because if there's a non-egg package in the same directory, it 
could conflict with or override the .egg version otherwise.

At 05:43 PM 1/23/2009, Mark Sienkiewicz wrote:
>Ben Finney wrote:
>>Mark Sienkiewicz <sienkiew at stsci.edu> writes:
>>
>>
>>>In that case, the .egg file should go on sys.path immediately before
>>>the directory where it is stored. This ensures that a setuptools
>>>.egg file will always be discovered before a distutils-installed
>>>package/module in the same directory, but it will not override
>>>packages/modules that occur earlier in sys.path.
>>>
>>
>>What of distributions installed via setuptools that are *not*
>>installed as eggs?
>>
>>
>
>I don't know.  Can you tell me more about that?  I'm not 
>particularly familiar with setuptools -- I'm just offering to write 
>the patch because I don't see it getting fixed any other way.
>
>Every thing that I have ever installed with setuptools has landed in 
>a file or directory named like "*.egg".  For example:
>site-packages/nose-0.10.4-py2.5.egg
>site-packages/Sphinx-0.5.1-py2.5.egg
>site-packages/Jinja-1.2-py2.5-macosx-10.3-i386.egg
>
>The installation creates a file "easy-install.pth" that looks like this:
>
>% cat easy-install.pth
>import sys; sys.__plen = len(sys.path)
>./setuptools-0.6c9-py2.5.egg
>./nose-0.10.4-py2.5.egg
>./Sphinx-0.5.1-py2.5.egg
>./Jinja-1.2-py2.5-macosx-10.3-i386.egg
>./Pygments-1.0-py2.5.egg
>./docutils-0.4-py2.5.egg
>import sys; new=sys.path[sys.__plen:]; del sys.path[sys.__plen:]; 
>p=getattr(sys,'__egginsert',0); sys.path[p:p]=new; sys.__egginsert = p+len(new)
>%
>
>So, where I said ".egg files", you can substitute the name of the 
>things on line 2 through N-1 in the easy-install.pth file.
>That last line of easy-install.pth guarantees that the .egg files 
>(in this example, they are all eggs) will be placed in the wrong 
>place on sys.path, and that is all I am proposing to fix.  If there 
>is another case that does not involve the easy-install.pth file, 
>that is outside the scope of my patch.
>
>So, about distributions that are not installed as eggs: Can you tell 
>me a little about it?  Can you give me an example of how I can see 
>it happen?  Or does it turn out to be irrelevant to my patch?
>
>Mark S.
>
>_______________________________________________
>Distutils-SIG maillist  -  Distutils-SIG at python.org
>http://mail.python.org/mailman/listinfo/distutils-sig


From iElectric at gmail.com  Sat Jan 24 13:19:23 2009
From: iElectric at gmail.com (=?windows-1252?Q?Domen_Ko=9Ear?=)
Date: Sat, 24 Jan 2009 04:19:23 -0800 (PST)
Subject: [Distutils] including an empty directory
In-Reply-To: <116b0f6c-31aa-4a10-9d1c-ca57a808d19e@i20g2000prf.googlegroups.com>
References: <116b0f6c-31aa-4a10-9d1c-ca57a808d19e@i20g2000prf.googlegroups.com>
Message-ID: <f4c9d2d9-f9fb-4347-89b3-2ac6bd379770@y1g2000pra.googlegroups.com>

according to http://svn.python.org/projects/sandbox/trunk/distutils_refactor/distutils/command/install_data.py
and http://docs.python.org/distutils/setupscript.html#installing-additional-files,
you should use the following parameter:

      data_files=[('data', [])],

That *should* make data directory in install dir, but it does not.
Suggestions?

From ziade.tarek at gmail.com  Sun Jan 25 00:18:05 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Sun, 25 Jan 2009 00:18:05 +0100
Subject: [Distutils] including an empty directory
In-Reply-To: <116b0f6c-31aa-4a10-9d1c-ca57a808d19e@i20g2000prf.googlegroups.com>
References: <116b0f6c-31aa-4a10-9d1c-ca57a808d19e@i20g2000prf.googlegroups.com>
Message-ID: <94bdd2610901241518r7e408da8we22a2d198bfc1a1a@mail.gmail.com>

I don't know on setuptools side, but in distutils, I don't think
there's any way to include an empty directory
like this, even with a graft in the MANIFEST.in file.

The commands are looking for files to include, then create the
necessary directories in the distribution, to include those files

Can't you add a README.txt file in that directory ?

What is the use case to include an empty directory in a release ?

Regards
Tarek

On Sat, Jan 24, 2009 at 2:40 AM, Domen Ko?ar <iElectric at gmail.com> wrote:
> I'm having issues to include an empty directory to setuptools package.
>
>      package_data={'data':['templates/*tmpl']}
>
> this includes all the files, but there is data/templates/Test
> directory that is empty and there is no way of globing it.
> _______________________________________________
> Distutils-SIG maillist  -  Distutils-SIG at python.org
> http://mail.python.org/mailman/listinfo/distutils-sig
>



-- 
Tarek Ziad? | Association AfPy | www.afpy.org
Blog FR | http://programmation-python.org
Blog EN | http://tarekziade.wordpress.com/

From iElectric at gmail.com  Sun Jan 25 00:39:39 2009
From: iElectric at gmail.com (=?windows-1252?Q?Domen_Ko=9Ear?=)
Date: Sat, 24 Jan 2009 15:39:39 -0800 (PST)
Subject: [Distutils] including an empty directory
In-Reply-To: <94bdd2610901241518r7e408da8we22a2d198bfc1a1a@mail.gmail.com>
References: <116b0f6c-31aa-4a10-9d1c-ca57a808d19e@i20g2000prf.googlegroups.com>
	<94bdd2610901241518r7e408da8we22a2d198bfc1a1a@mail.gmail.com>
Message-ID: <ac5188cd-8b2c-402b-a7a8-1da71325d418@o4g2000pra.googlegroups.com>

Usage is to dispatch Paste templates http://pythonpaste.org/script/developer.html#templates.

And the dir must be empty, for some reasons.

So this distutils syntax does not work (interesting though, as it does
create the dir in the egg, it's just not copied)

On Jan 25, 12:18?am, Tarek Ziad? <ziade.ta... at gmail.com> wrote:
> I don't know on setuptools side, but in distutils, I don't think
> there's any way to include an empty directory
> like this, even with a graft in the MANIFEST.in file.
>
> The commands are looking for files to include, then create the
> necessary directories in the distribution, to include those files
>
> Can't you add a README.txt file in that directory ?
>
> What is the use case to include an empty directory in a release ?
>
> Regards
> Tarek
>
> On Sat, Jan 24, 2009 at 2:40 AM, Domen Ko?ar <iElect... at gmail.com> wrote:
> > I'm having issues to include an empty directory to setuptools package.
>
> > ? ? ?package_data={'data':['templates/*tmpl']}
>
> > this includes all the files, but there is data/templates/Test
> > directory that is empty and there is no way of globing it.
> > _______________________________________________
> > Distutils-SIG maillist ?- ?Distutils-... at python.org
> >http://mail.python.org/mailman/listinfo/distutils-sig
>
> --
> Tarek Ziad? | Association AfPy |www.afpy.org
> Blog FR |http://programmation-python.org
> Blog EN |http://tarekziade.wordpress.com/
> _______________________________________________
> Distutils-SIG maillist ?- ?Distutils-... at python.orghttp://mail.python.org/mailman/listinfo/distutils-sig

From jorge.vargas at gmail.com  Sun Jan 25 10:07:12 2009
From: jorge.vargas at gmail.com (Jorge Vargas)
Date: Sun, 25 Jan 2009 05:07:12 -0400
Subject: [Distutils] including an empty directory
In-Reply-To: <ac5188cd-8b2c-402b-a7a8-1da71325d418@o4g2000pra.googlegroups.com>
References: <116b0f6c-31aa-4a10-9d1c-ca57a808d19e@i20g2000prf.googlegroups.com>
	<94bdd2610901241518r7e408da8we22a2d198bfc1a1a@mail.gmail.com>
	<ac5188cd-8b2c-402b-a7a8-1da71325d418@o4g2000pra.googlegroups.com>
Message-ID: <32822fe60901250107v148ca0e1ge11bf31e6a102a89@mail.gmail.com>

FYI, this is a know bug/limitation I never bother to look much into it
as we simply add a placeholder file.

On Sat, Jan 24, 2009 at 7:39 PM, Domen Ko?ar <iElectric at gmail.com> wrote:
> Usage is to dispatch Paste templates http://pythonpaste.org/script/developer.html#templates.
>
> And the dir must be empty, for some reasons.
>
> So this distutils syntax does not work (interesting though, as it does
> create the dir in the egg, it's just not copied)
>
> On Jan 25, 12:18 am, Tarek Ziad? <ziade.ta... at gmail.com> wrote:
>> I don't know on setuptools side, but in distutils, I don't think
>> there's any way to include an empty directory
>> like this, even with a graft in the MANIFEST.in file.
>>
>> The commands are looking for files to include, then create the
>> necessary directories in the distribution, to include those files
>>
>> Can't you add a README.txt file in that directory ?
>>
>> What is the use case to include an empty directory in a release ?
>>
>> Regards
>> Tarek
>>
>> On Sat, Jan 24, 2009 at 2:40 AM, Domen Ko?ar <iElect... at gmail.com> wrote:
>> > I'm having issues to include an empty directory to setuptools package.
>>
>> >      package_data={'data':['templates/*tmpl']}
>>
>> > this includes all the files, but there is data/templates/Test
>> > directory that is empty and there is no way of globing it.
>> > _______________________________________________
>> > Distutils-SIG maillist  -  Distutils-... at python.org
>> >http://mail.python.org/mailman/listinfo/distutils-sig
>>
>> --
>> Tarek Ziad? | Association AfPy |www.afpy.org
>> Blog FR |http://programmation-python.org
>> Blog EN |http://tarekziade.wordpress.com/
>> _______________________________________________
>> Distutils-SIG maillist  -  Distutils-... at python.orghttp://mail.python.org/mailman/listinfo/distutils-sig
> _______________________________________________
> Distutils-SIG maillist  -  Distutils-SIG at python.org
> http://mail.python.org/mailman/listinfo/distutils-sig
>

From ziade.tarek at gmail.com  Sun Jan 25 11:05:48 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Sun, 25 Jan 2009 11:05:48 +0100
Subject: [Distutils] including an empty directory
In-Reply-To: <32822fe60901250107v148ca0e1ge11bf31e6a102a89@mail.gmail.com>
References: <116b0f6c-31aa-4a10-9d1c-ca57a808d19e@i20g2000prf.googlegroups.com>
	<94bdd2610901241518r7e408da8we22a2d198bfc1a1a@mail.gmail.com>
	<ac5188cd-8b2c-402b-a7a8-1da71325d418@o4g2000pra.googlegroups.com>
	<32822fe60901250107v148ca0e1ge11bf31e6a102a89@mail.gmail.com>
Message-ID: <94bdd2610901250205u7e60b68ahcf826f496d533502@mail.gmail.com>

On Sun, Jan 25, 2009 at 10:07 AM, Jorge Vargas <jorge.vargas at gmail.com> wrote:
> FYI, this is a know bug/limitation I never bother to look much into it
> as we simply add a placeholder file.

Well if that's a recurrent need maybe we can add this feature.

What about a new option called "include_empty_folders" (False by default)


Tarek

-- 
Tarek Ziad? | Association AfPy | www.afpy.org
Blog FR | http://programmation-python.org
Blog EN | http://tarekziade.wordpress.com/

From tarek.ziade at ingeniweb.com  Sun Jan 25 11:52:02 2009
From: tarek.ziade at ingeniweb.com (Tarek Ziade)
Date: Sun, 25 Jan 2009 11:52:02 +0100
Subject: [Distutils] including an empty directory
In-Reply-To: <94bdd2610901250205u7e60b68ahcf826f496d533502@mail.gmail.com>
References: <116b0f6c-31aa-4a10-9d1c-ca57a808d19e@i20g2000prf.googlegroups.com>
	<94bdd2610901241518r7e408da8we22a2d198bfc1a1a@mail.gmail.com>
	<ac5188cd-8b2c-402b-a7a8-1da71325d418@o4g2000pra.googlegroups.com>
	<32822fe60901250107v148ca0e1ge11bf31e6a102a89@mail.gmail.com>
	<94bdd2610901250205u7e60b68ahcf826f496d533502@mail.gmail.com>
Message-ID: <a26746990901250252o4d4536c0t2a6e1bed2ed709ce@mail.gmail.com>

2009/1/25 Tarek Ziad? <ziade.tarek at gmail.com>:
> On Sun, Jan 25, 2009 at 10:07 AM, Jorge Vargas <jorge.vargas at gmail.com> wrote:
>> FYI, this is a know bug/limitation I never bother to look much into it
>> as we simply add a placeholder file.
>
> Well if that's a recurrent need maybe we can add this feature.
>
> What about a new option called "include_empty_folders" (False by default)

rather : include_empty_directory

>
>
> Tarek
>
> --
> Tarek Ziad? | Association AfPy | www.afpy.org
> Blog FR | http://programmation-python.org
> Blog EN | http://tarekziade.wordpress.com/
> _______________________________________________
> Distutils-SIG maillist  -  Distutils-SIG at python.org
> http://mail.python.org/mailman/listinfo/distutils-sig
>



-- 
Tarek Ziad? - Directeur Technique
INGENIWEB (TM) - SAS 50000 Euros - RC B 438 725 632
Bureaux de la Colline - 1 rue Royale - B?timent D - 9?me ?tage
92210 Saint Cloud - France
Phone : 01.78.15.24.00 / Fax : 01 46 02 44 04
http://www.ingeniweb.com - une soci?t? du groupe Alter Way

From ziade.tarek at gmail.com  Sun Jan 25 12:06:43 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Sun, 25 Jan 2009 12:06:43 +0100
Subject: [Distutils] [Python Language Summit] Distutils / Packaging survey
Message-ID: <94bdd2610901250306n1e27a928sa0b276d91f80862a@mail.gmail.com>

Hello

A Language Summit will be held in Chicago the day before Pycon.
http://us.pycon.org/2009/about/summits/language/

I am preparing a survey to get some feedback from the community before
the summit, where I will try to summarize
the status, and the possible paths we can take to make things better.

The desired output for the summit, will be to have a roadmap for
distutils, on actions that meets people' consensus

By "community" I mean all the people that have to use Distutils at
some point, wether they are installing, developing
or packaging for a specific OS, a piece of Python code.

I am working on the survey here : http://wiki.python.org/moin/Packaging%20Survey

And I need help ! :)

If you see weird things, or missing points, please feel free to fix
the wiki page and/or discuss it here

Thanks

Tarek

-- 
Tarek Ziad? | Association AfPy | www.afpy.org
Blog FR | http://programmation-python.org
Blog EN | http://tarekziade.wordpress.com/

From iElectric at gmail.com  Sun Jan 25 13:06:46 2009
From: iElectric at gmail.com (=?UTF-8?B?RG9tZW4gS2/FvmFy?=)
Date: Sun, 25 Jan 2009 04:06:46 -0800 (PST)
Subject: [Distutils] including an empty directory
In-Reply-To: <a26746990901250252o4d4536c0t2a6e1bed2ed709ce@mail.gmail.com>
References: <116b0f6c-31aa-4a10-9d1c-ca57a808d19e@i20g2000prf.googlegroups.com>
	<94bdd2610901241518r7e408da8we22a2d198bfc1a1a@mail.gmail.com> 
	<ac5188cd-8b2c-402b-a7a8-1da71325d418@o4g2000pra.googlegroups.com> 
	<32822fe60901250107v148ca0e1ge11bf31e6a102a89@mail.gmail.com> 
	<94bdd2610901250205u7e60b68ahcf826f496d533502@mail.gmail.com> 
	<a26746990901250252o4d4536c0t2a6e1bed2ed709ce@mail.gmail.com>
Message-ID: <cedeea16-d12f-4a53-b9b7-5e75a4fffd4a@t39g2000prh.googlegroups.com>

that's one of the options yes, or package_include parameter to
setuptools that would take a list of paths that should be included
recursivly.

On Jan 25, 11:52?am, Tarek Ziade <tarek.zi... at ingeniweb.com> wrote:
> 2009/1/25 Tarek Ziad? <ziade.ta... at gmail.com>:
>
> > On Sun, Jan 25, 2009 at 10:07 AM, Jorge Vargas <jorge.var... at gmail.com> wrote:
> >> FYI, this is a know bug/limitation I never bother to look much into it
> >> as we simply add a placeholder file.
>
> > Well if that's a recurrent need maybe we can add this feature.
>
> > What about a new option called "include_empty_folders" (False by default)
>
> rather : include_empty_directory
>
>
>
> > Tarek
>
> > --
> > Tarek Ziad? | Association AfPy |www.afpy.org
> > Blog FR |http://programmation-python.org
> > Blog EN |http://tarekziade.wordpress.com/
> > _______________________________________________
> > Distutils-SIG maillist ?- ?Distutils-... at python.org
> >http://mail.python.org/mailman/listinfo/distutils-sig
>
> --
> Tarek Ziad? - Directeur Technique
> INGENIWEB (TM) - SAS 50000 Euros - RC B 438 725 632
> Bureaux de la Colline - 1 rue Royale - B?timent D - 9?me ?tage
> 92210 Saint Cloud - France
> Phone : 01.78.15.24.00 / Fax : 01 46 02 44 04http://www.ingeniweb.com- une soci?t? du groupe Alter Way
> _______________________________________________
> Distutils-SIG maillist ?- ?Distutils-... at python.orghttp://mail.python.org/mailman/listinfo/distutils-sig

From ziade.tarek at gmail.com  Sun Jan 25 13:30:40 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Sun, 25 Jan 2009 13:30:40 +0100
Subject: [Distutils] including an empty directory
In-Reply-To: <cedeea16-d12f-4a53-b9b7-5e75a4fffd4a@t39g2000prh.googlegroups.com>
References: <116b0f6c-31aa-4a10-9d1c-ca57a808d19e@i20g2000prf.googlegroups.com>
	<94bdd2610901241518r7e408da8we22a2d198bfc1a1a@mail.gmail.com>
	<ac5188cd-8b2c-402b-a7a8-1da71325d418@o4g2000pra.googlegroups.com>
	<32822fe60901250107v148ca0e1ge11bf31e6a102a89@mail.gmail.com>
	<94bdd2610901250205u7e60b68ahcf826f496d533502@mail.gmail.com>
	<a26746990901250252o4d4536c0t2a6e1bed2ed709ce@mail.gmail.com>
	<cedeea16-d12f-4a53-b9b7-5e75a4fffd4a@t39g2000prh.googlegroups.com>
Message-ID: <94bdd2610901250430i329fdad5mba0cccc8d5570b39@mail.gmail.com>

On Sun, Jan 25, 2009 at 1:06 PM, Domen Ko?ar <iElectric at gmail.com> wrote:
> that's one of the options yes, or package_include parameter to
> setuptools that would take a list of paths that should be included
> recursivly.

Beware that I am talking about Distutils code here rather than Setuptools' one,
while this won't probably change anything (the feature would then be
usable from Setuptools transparently),
it's important to make the distinction because this feature will only
be available in python 2.7

Also, maybe there's something manageable at Setuptools level already
for this, Phillip ?

Tarek

-- 
Tarek Ziad? | Association AfPy | www.afpy.org
Blog FR | http://programmation-python.org
Blog EN | http://tarekziade.wordpress.com/

From zooko at zooko.com  Sun Jan 25 16:54:45 2009
From: zooko at zooko.com (zooko)
Date: Sun, 25 Jan 2009 08:54:45 -0700
Subject: [Distutils] including an empty directory
In-Reply-To: <94bdd2610901250205u7e60b68ahcf826f496d533502@mail.gmail.com>
References: <116b0f6c-31aa-4a10-9d1c-ca57a808d19e@i20g2000prf.googlegroups.com>
	<94bdd2610901241518r7e408da8we22a2d198bfc1a1a@mail.gmail.com>
	<ac5188cd-8b2c-402b-a7a8-1da71325d418@o4g2000pra.googlegroups.com>
	<32822fe60901250107v148ca0e1ge11bf31e6a102a89@mail.gmail.com>
	<94bdd2610901250205u7e60b68ahcf826f496d533502@mail.gmail.com>
Message-ID: <91BCAC8A-4A4B-458B-8DDA-736C4BB7AE9D@zooko.com>

On Jan 25, 2009, at 3:05 AM, Tarek Ziad? wrote:

> Well if that's a recurrent need maybe we can add this feature.
>
> What about a new option called "include_empty_folders" (False by  
> default)


In a related story, I recently found a problem with the packaging of  
nevow.  We had earlier added "twisted.plugins" to "packages", not  
because nevow includes a package named "twisted.plugins", but because  
that's the only way we could figure out how to make sure that  
distutils and/or setuptools would create the empty "twisted/plugins"  
directory on installation which nevow requires:

http://divmod.org/trac/changeset/16896

However, this leads to this warning message at import time:

"/usr/lib/python2.5/site-packages/zope/__init__.py:19: UserWarning:  
Module twisted was already imported from /usr/lib/python2.5/site- 
packages/twisted/__init__.pyc, but /home/buildslave/slave-gutsy-tahoe/ 
gutsy/build/support/lib/python2.5/site-packages/Nevow-0.9.32- 
py2.5.egg is being added to sys.path"

http://bugs.python.org/setuptools/issue36

So I've just submitted a patch to nevow to use data_files instead of  
packages to indicate that this directory should be created, and to do  
that we have to include a file in the directory so that distutils  
doesn't prune it out for being an empty directory:

http://divmod.org/trac/ticket/2830

One thing that I don't understand about all this is why setuptools's  
"include_package_data=True" didn't cause the directory to be  
installed.  I'm not sure if calling that "twisted/plugins" directory  
a "data file" or an "included package data" makes more sense, but the  
former currently works.

Regards,

Zooko
---
Tahoe, the Least-Authority Filesystem -- http://allmydata.org
store your data: $10/month -- http://allmydata.com/?tracking=zsig

From akitada at gmail.com  Sun Jan 25 17:40:03 2009
From: akitada at gmail.com (Akira Kitada)
Date: Mon, 26 Jan 2009 01:40:03 +0900
Subject: [Distutils] Anyone interested in updating/reoriganizing distutils
	pages on wiki.python.org?
Message-ID: <90bb445a0901250840j2d1e0954s2f959d8c1826eb9f@mail.gmail.com>

Hi,

I'm planning to reorganize distutils pages on wiki.python.org.
Before I begin, I would like to know if there are anyone interested in
working on this.

My goals are:
- Collect distutils information into one place - One top page and sub pages
- Update contents of distutils page.
- Have a link to distutils top page on the wiki.
- etc, etc.

If there is anyone interested in this, please reply this mail.

From ziade.tarek at gmail.com  Sun Jan 25 17:58:41 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Sun, 25 Jan 2009 17:58:41 +0100
Subject: [Distutils] Anyone interested in updating/reoriganizing
	distutils pages on wiki.python.org?
In-Reply-To: <90bb445a0901250840j2d1e0954s2f959d8c1826eb9f@mail.gmail.com>
References: <90bb445a0901250840j2d1e0954s2f959d8c1826eb9f@mail.gmail.com>
Message-ID: <94bdd2610901250858v8fabaebneabaad2a182b1895@mail.gmail.com>

On Sun, Jan 25, 2009 at 5:40 PM, Akira Kitada <akitada at gmail.com> wrote:
> Hi,
>
> I'm planning to reorganize distutils pages on wiki.python.org.
> Before I begin, I would like to know if there are anyone interested in
> working on this.

Great !

I am

>
> My goals are:
> - Collect distutils information into one place - One top page and sub pages
> - Update contents of distutils page.
> - Have a link to distutils top page on the wiki.
> - etc, etc.
>
> If there is anyone interested in this, please reply this mail.

There's one page we have started during the D.C. sprint that might
be interested to take back and update, in your reorg plan.

http://wiki.python.org/moin/PythonPackagingTerminology

This is something quite handy for people to discuss using the
same terminology


> _______________________________________________
> Distutils-SIG maillist  -  Distutils-SIG at python.org
> http://mail.python.org/mailman/listinfo/distutils-sig
>



-- 
Tarek Ziad? | Association AfPy | www.afpy.org
Blog FR | http://programmation-python.org
Blog EN | http://tarekziade.wordpress.com/

From akitada at gmail.com  Sun Jan 25 19:23:19 2009
From: akitada at gmail.com (Akira Kitada)
Date: Mon, 26 Jan 2009 03:23:19 +0900
Subject: [Distutils] Anyone interested in updating/reoriganizing
	distutils pages on wiki.python.org?
In-Reply-To: <94bdd2610901250858v8fabaebneabaad2a182b1895@mail.gmail.com>
References: <90bb445a0901250840j2d1e0954s2f959d8c1826eb9f@mail.gmail.com>
	<94bdd2610901250858v8fabaebneabaad2a182b1895@mail.gmail.com>
Message-ID: <90bb445a0901251023o16b645d1qfd79aec3dd4f0414@mail.gmail.com>

Hi Tarek,

Thanks for your interest in this.
http://wiki.python.org/moin/Distutils would be the start page of
Distutils wiki.
I modified its format and added FAQ page.

On Mon, Jan 26, 2009 at 1:58 AM, Tarek Ziad? <ziade.tarek at gmail.com> wrote:
> On Sun, Jan 25, 2009 at 5:40 PM, Akira Kitada <akitada at gmail.com> wrote:
>> Hi,
>>
>> I'm planning to reorganize distutils pages on wiki.python.org.
>> Before I begin, I would like to know if there are anyone interested in
>> working on this.
>
> Great !
>
> I am
>
>>
>> My goals are:
>> - Collect distutils information into one place - One top page and sub pages
>> - Update contents of distutils page.
>> - Have a link to distutils top page on the wiki.
>> - etc, etc.
>>
>> If there is anyone interested in this, please reply this mail.
>
> There's one page we have started during the D.C. sprint that might
> be interested to take back and update, in your reorg plan.
>
> http://wiki.python.org/moin/PythonPackagingTerminology
>
> This is something quite handy for people to discuss using the
> same terminology
>
>
>> _______________________________________________
>> Distutils-SIG maillist  -  Distutils-SIG at python.org
>> http://mail.python.org/mailman/listinfo/distutils-sig
>>
>
>
>
> --
> Tarek Ziad? | Association AfPy | www.afpy.org
> Blog FR | http://programmation-python.org
> Blog EN | http://tarekziade.wordpress.com/
>

From sienkiew at stsci.edu  Mon Jan 26 22:53:57 2009
From: sienkiew at stsci.edu (Mark Sienkiewicz)
Date: Mon, 26 Jan 2009 16:53:57 -0500
Subject: [Distutils] patch: solving the two major things that people
 dislike about setuptools, part 1
In-Reply-To: <7.0.1.0.0.20090123230523.0278fec8@telecommunity.com>
References: <5AA53D0A-98D4-4AF5-843C-DEE1D37C2D24@zooko.com>
	<17519F49-7F21-40D0-BCF5-C0A77E0CFCCB@zooko.com>
	<20090121002856.405BA3A4065@sparrow.telecommunity.com>
	<4978D070.9010709@stsci.edu> <87iqo72e0z.fsf@benfinney.id.au>
	<497A481F.4000703@stsci.edu>
	<7.0.1.0.0.20090123230523.0278fec8@telecommunity.com>
Message-ID: <497E30F5.1040906@stsci.edu>

Phillip J. Eby wrote:
> Packages that aren't installed in eggs don't need to be "before their 
> own directory".  The only reason that .egg files or directories do, is 
> because if there's a non-egg package in the same directory, it could 
> conflict with or override the .egg version otherwise.

In that case, do you find my proposed patch acceptable?

Mark S.


From dpeterson at enthought.com  Tue Jan 27 01:48:06 2009
From: dpeterson at enthought.com (Dave Peterson)
Date: Mon, 26 Jan 2009 18:48:06 -0600
Subject: [Distutils] Solaris and distutils: Need to pass LIBDIR explicitly
 with -L when building extensions?
Message-ID: <497E59C6.4020802@enthought.com>

I am trying to build a number of projects that use Python extensions on 
Solaris 10 and I've discovered that nothing with extensions will link 
unless I explicitly pass in a '-L/path/to/python/lib/dir' because 
libpython2.5.so is not otherwise found when the '-lpython2.5' argument 
is specified during linking.  This doesn't seem right as I don't have to 
specify this explicitly on Linux or Windows.

Admittedly I'm using a custom built Python, so perhaps I missed some 
configure argument?   Here's my Python build process:

./configure --prefix=/home/dpeterson/py/build --enable-shared
make
make test
make install

It is built with the gcc provided by Sun in a default install of Solaris 
10, which means the gcc compiler and the sun linker.  "gcc -v" gives:

Reading specs from /usr/sfw/lib/gcc/i386-pc-solaris2.10/3.4.3/specs
Configured with: /builds/sfw10-gate/usr/src/cmd/gcc/gcc-3.4.3/configure 
--prefix=/usr/sfw --with-as=/usr/sfw/bin/gas --with-gnu-as 
--with-ld=/usr/ccs/bin/ld --without-gnu-ld --enable-languages=c,c++ 
--enable-shared
Thread model: posix
gcc version 3.4.3 (csl-sol210-3_4-branch+sol_rpath)


If that all seems correct, then it appears the issue is the 
finalize_options() source in lib/distutils/commands/build_ext.py.  There 
are a number of "if" blocks that explicitly append the value of 
distutils.sysconfig.get_config_vars('LIBDIR') to the list of 
library_dirs used to link built extensions with.  However, there doesn't 
seem to be one of these for Solaris / sunos.   Is this just an 
oversite?  I've verified that adding a simple block with a 'sunos' value 
gate solves the problem and I no longer need to be explicit about 
providing '-L'.   Am I missing something or should this patch be 
submitted as a bugfix on the issue tracker?


-- Dave


From akitada at gmail.com  Tue Jan 27 17:29:36 2009
From: akitada at gmail.com (Akira Kitada)
Date: Wed, 28 Jan 2009 01:29:36 +0900
Subject: [Distutils] [Python Language Summit] Distutils / Packaging
	survey
In-Reply-To: <94bdd2610901250306n1e27a928sa0b276d91f80862a@mail.gmail.com>
References: <94bdd2610901250306n1e27a928sa0b276d91f80862a@mail.gmail.com>
Message-ID: <90bb445a0901270829y7019eca3p72164c259c4d35d4@mail.gmail.com>

Hello Tarek,

I think "apt, yum, etc" would be also used for packaging/distributing apps.


On Sun, Jan 25, 2009 at 8:06 PM, Tarek Ziad? <ziade.tarek at gmail.com> wrote:
> Hello
>
> A Language Summit will be held in Chicago the day before Pycon.
> http://us.pycon.org/2009/about/summits/language/
>
> I am preparing a survey to get some feedback from the community before
> the summit, where I will try to summarize
> the status, and the possible paths we can take to make things better.
>
> The desired output for the summit, will be to have a roadmap for
> distutils, on actions that meets people' consensus
>
> By "community" I mean all the people that have to use Distutils at
> some point, wether they are installing, developing
> or packaging for a specific OS, a piece of Python code.
>
> I am working on the survey here : http://wiki.python.org/moin/Packaging%20Survey
>
> And I need help ! :)
>
> If you see weird things, or missing points, please feel free to fix
> the wiki page and/or discuss it here
>
> Thanks
>
> Tarek
>
> --
> Tarek Ziad? | Association AfPy | www.afpy.org
> Blog FR | http://programmation-python.org
> Blog EN | http://tarekziade.wordpress.com/
> _______________________________________________
> Distutils-SIG maillist  -  Distutils-SIG at python.org
> http://mail.python.org/mailman/listinfo/distutils-sig
>

From sienkiew at stsci.edu  Tue Jan 27 18:56:58 2009
From: sienkiew at stsci.edu (Mark Sienkiewicz)
Date: Tue, 27 Jan 2009 12:56:58 -0500
Subject: [Distutils] [Python Language Summit] Distutils / Packaging
	survey
In-Reply-To: <94bdd2610901250306n1e27a928sa0b276d91f80862a@mail.gmail.com>
References: <94bdd2610901250306n1e27a928sa0b276d91f80862a@mail.gmail.com>
Message-ID: <497F4AEA.9070805@stsci.edu>

Tarek Ziad? wrote:
> I am working on the survey here : http://wiki.python.org/moin/Packaging%20Survey
>
>   

One of the survey questions asks:

> Would you like to see Python's Distutils package provide an uninstall 
> command that removes just the files installed, and cleanup the .pth, 
> even if it does undesirable things ? (one answer)

Why would you ask that?  Is there some reason that we can't have an 
uninstall that works correctly?

Mark S.


From ziade.tarek at gmail.com  Tue Jan 27 23:00:14 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Tue, 27 Jan 2009 23:00:14 +0100
Subject: [Distutils] [Python Language Summit] Distutils / Packaging
	survey
In-Reply-To: <90bb445a0901270829y7019eca3p72164c259c4d35d4@mail.gmail.com>
References: <94bdd2610901250306n1e27a928sa0b276d91f80862a@mail.gmail.com>
	<90bb445a0901270829y7019eca3p72164c259c4d35d4@mail.gmail.com>
Message-ID: <94bdd2610901271400k32ff74edmea94fcbdfce88f53@mail.gmail.com>

On Tue, Jan 27, 2009 at 5:29 PM, Akira Kitada <akitada at gmail.com> wrote:
> Hello Tarek,
>
> I think "apt, yum, etc" would be also used for packaging/distributing apps.
>

There is already a command that let you create a rpm package
(bdist_rpm) out of a python package,

There were also a bdist_deb project but it never made it to distutils,
also for Debian there's a policy on how to work with python packages :
http://www.debian.org/doc/packaging-manuals/python-policy/

Last, this mailing list had a lot of threads about the fact that
there's no standard way in Python to work with resources that
could be installed in the system, using a LSB-compliant approach.

So I don't have (I think no one does at this point) any clear view of
what could be done in this area.

Tarek
-- 
Tarek Ziad? | Association AfPy | www.afpy.org
Blog FR | http://programmation-python.org
Blog EN | http://tarekziade.wordpress.com/

From ziade.tarek at gmail.com  Tue Jan 27 23:10:44 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Tue, 27 Jan 2009 23:10:44 +0100
Subject: [Distutils] [Python Language Summit] Distutils / Packaging
	survey
In-Reply-To: <497F4AEA.9070805@stsci.edu>
References: <94bdd2610901250306n1e27a928sa0b276d91f80862a@mail.gmail.com>
	<497F4AEA.9070805@stsci.edu>
Message-ID: <94bdd2610901271410t67240ec3t8bca36ffd32fcb07@mail.gmail.com>

On Tue, Jan 27, 2009 at 6:56 PM, Mark Sienkiewicz <sienkiew at stsci.edu> wrote:
> Tarek Ziad? wrote:
>>
>> I am working on the survey here :
>> http://wiki.python.org/moin/Packaging%20Survey
>>
>>
>
> One of the survey questions asks:
>
>> Would you like to see Python's Distutils package provide an uninstall
>> command that removes just the files installed, and cleanup the .pth, even if
>> it does undesirable things ? (one answer)
>
> Why would you ask that?  Is there some reason that we can't have an
> uninstall that works correctly?

There are possible side-effects: what would happen if one of the file
installed on your
system is used by other elements in the system ? (Python is not the almighty OS)

On the other hand, it's OK most of the time.

(I'll add some links to previous discussion from this ML on the topic)

Regards
Tarek

-- 
Tarek Ziad? | Association AfPy | www.afpy.org
Blog FR | http://programmation-python.org
Blog EN | http://tarekziade.wordpress.com/

From zooko at zooko.com  Tue Jan 27 23:18:46 2009
From: zooko at zooko.com (zooko)
Date: Tue, 27 Jan 2009 15:18:46 -0700
Subject: [Distutils] [Python Language Summit] Distutils / Packaging
	survey
In-Reply-To: <94bdd2610901271400k32ff74edmea94fcbdfce88f53@mail.gmail.com>
References: <94bdd2610901250306n1e27a928sa0b276d91f80862a@mail.gmail.com>
	<90bb445a0901270829y7019eca3p72164c259c4d35d4@mail.gmail.com>
	<94bdd2610901271400k32ff74edmea94fcbdfce88f53@mail.gmail.com>
Message-ID: <515B9271-0E7D-4B4E-B63F-F717604A5FEA@zooko.com>

On Jan 27, 2009, at 15:00 PM, Tarek Ziad? wrote:

> There were also a bdist_deb project but it never made it to  
> distutils, also for Debian there's a policy on how to work with  
> python packages :
> http://www.debian.org/doc/packaging-manuals/python-policy/
>
> Last, this mailing list had a lot of threads about the fact that  
> there's no standard way in Python to work with resources that could  
> be installed in the system, using a LSB-compliant approach.
>
> So I don't have (I think no one does at this point) any clear view  
> of what could be done in this area.

I don't understand what are the potential problems, but so far I've  
been happy using stdeb to produce .deb's from my Python sdists.

Regards,

Zooko
---
Tahoe, the Least-Authority Filesystem -- http://allmydata.org
store your data: $10/month -- http://allmydata.com/?tracking=zsig

From ziade.tarek at gmail.com  Wed Jan 28 10:27:28 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Wed, 28 Jan 2009 10:27:28 +0100
Subject: [Distutils] [Python Language Summit] Distutils / Packaging
	survey
In-Reply-To: <515B9271-0E7D-4B4E-B63F-F717604A5FEA@zooko.com>
References: <94bdd2610901250306n1e27a928sa0b276d91f80862a@mail.gmail.com>
	<90bb445a0901270829y7019eca3p72164c259c4d35d4@mail.gmail.com>
	<94bdd2610901271400k32ff74edmea94fcbdfce88f53@mail.gmail.com>
	<515B9271-0E7D-4B4E-B63F-F717604A5FEA@zooko.com>
Message-ID: <94bdd2610901280127r597f811fmc2a333510aca2fad@mail.gmail.com>

On Tue, Jan 27, 2009 at 11:18 PM, zooko <zooko at zooko.com> wrote:
>> So I don't have (I think no one does at this point) any clear view of what
>> could be done in this area.
>
> I don't understand what are the potential problems, but so far I've been
> happy using stdeb to produce .deb's from my Python sdists.

I guess it is perfectly fine if you are your own debian packager.

The problem I see is when people that builds packages and are not
their own debian packagers.

For example, if my application has a log file, it is better under
Debian to have it in /var/log/xxx

In the meantime, for the same application, I don't want to bother
under win32 about that, the log can leave inside the package
for instance.

So how can I describe in my package, that I will have a log file, and
how can the Debian packager
can tell to my package that it has to be in /var/log/... ?

In other words, what would be the new metadata we could put in the
setup.py in the package to minimize
the work to be done on stdeb side ?


Regards
Tarek


-- 
Tarek Ziad? | Association AfPy | www.afpy.org
Blog FR | http://programmation-python.org
Blog EN | http://tarekziade.wordpress.com/

From david at ar.media.kyoto-u.ac.jp  Wed Jan 28 11:45:38 2009
From: david at ar.media.kyoto-u.ac.jp (David Cournapeau)
Date: Wed, 28 Jan 2009 19:45:38 +0900
Subject: [Distutils] [Python Language Summit] Distutils /
	Packaging	survey
In-Reply-To: <515B9271-0E7D-4B4E-B63F-F717604A5FEA@zooko.com>
References: <94bdd2610901250306n1e27a928sa0b276d91f80862a@mail.gmail.com>	<90bb445a0901270829y7019eca3p72164c259c4d35d4@mail.gmail.com>	<94bdd2610901271400k32ff74edmea94fcbdfce88f53@mail.gmail.com>
	<515B9271-0E7D-4B4E-B63F-F717604A5FEA@zooko.com>
Message-ID: <49803752.9040704@ar.media.kyoto-u.ac.jp>

zooko wrote:
> On Jan 27, 2009, at 15:00 PM, Tarek Ziad? wrote:
>
>> There were also a bdist_deb project but it never made it to
>> distutils, also for Debian there's a policy on how to work with
>> python packages :
>> http://www.debian.org/doc/packaging-manuals/python-policy/
>>
>> Last, this mailing list had a lot of threads about the fact that
>> there's no standard way in Python to work with resources that could
>> be installed in the system, using a LSB-compliant approach.
>>
>> So I don't have (I think no one does at this point) any clear view of
>> what could be done in this area.
>
> I don't understand what are the potential problems, but so far I've
> been happy using stdeb to produce .deb's from my Python sdists.

This is not the right solution for distributions maintainers: it is a
good tool for individual (it gives you uninstallation, etec...) but .deb
packages produced by stddeb are not debian-compatible, and cannot be
included in debian proper. This is not a critic of stddeb, I think it is
a very good tool and useful tool.

The *only* right solution for packaging python modules on Linux
distribution is to make it as "easy" for packagers as it is for autoconf
packages. Meaning having clear differences between installation, binary,
libraries, etc... (what's called resources by setuptools, IIUC), so that
maintainers can set it up how they want. This way, python developers do
not have to care about debian, and distributions maintainers do not have
to care about python (well, not more than now).

It is a solved problem: autoconf does it well, and has all the required
features,

David



From ziade.tarek at gmail.com  Wed Jan 28 12:07:36 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Wed, 28 Jan 2009 12:07:36 +0100
Subject: [Distutils] [Python Language Summit] Distutils / Packaging
	survey
In-Reply-To: <49803752.9040704@ar.media.kyoto-u.ac.jp>
References: <94bdd2610901250306n1e27a928sa0b276d91f80862a@mail.gmail.com>
	<90bb445a0901270829y7019eca3p72164c259c4d35d4@mail.gmail.com>
	<94bdd2610901271400k32ff74edmea94fcbdfce88f53@mail.gmail.com>
	<515B9271-0E7D-4B4E-B63F-F717604A5FEA@zooko.com>
	<49803752.9040704@ar.media.kyoto-u.ac.jp>
Message-ID: <94bdd2610901280307h42b05abcr5599d64189948201@mail.gmail.com>

On Wed, Jan 28, 2009 at 11:45 AM, David Cournapeau
<david at ar.media.kyoto-u.ac.jp> wrote:
> It is a solved problem: autoconf does it well, and has all the required
> features,

So does it mean that having Distutils generate some kind of
"configure.in" template that could
be used by autoconf could be the right approach ?

From david at ar.media.kyoto-u.ac.jp  Wed Jan 28 12:05:03 2009
From: david at ar.media.kyoto-u.ac.jp (David Cournapeau)
Date: Wed, 28 Jan 2009 20:05:03 +0900
Subject: [Distutils] [Python Language Summit] Distutils / Packaging
	survey
In-Reply-To: <94bdd2610901280307h42b05abcr5599d64189948201@mail.gmail.com>
References: <94bdd2610901250306n1e27a928sa0b276d91f80862a@mail.gmail.com>	
	<90bb445a0901270829y7019eca3p72164c259c4d35d4@mail.gmail.com>	
	<94bdd2610901271400k32ff74edmea94fcbdfce88f53@mail.gmail.com>	
	<515B9271-0E7D-4B4E-B63F-F717604A5FEA@zooko.com>	
	<49803752.9040704@ar.media.kyoto-u.ac.jp>
	<94bdd2610901280307h42b05abcr5599d64189948201@mail.gmail.com>
Message-ID: <49803BDF.4060204@ar.media.kyoto-u.ac.jp>

Tarek Ziad? wrote:
> On Wed, Jan 28, 2009 at 11:45 AM, David Cournapeau
> <david at ar.media.kyoto-u.ac.jp> wrote:
>   
>> It is a solved problem: autoconf does it well, and has all the required
>> features,
>>     
>
> So does it mean that having Distutils generate some kind of
> "configure.in" template that could
> be used by autoconf could be the right approach ?
>   

I meant that instead of installing almost everything indistinctly like
we do now with distutils/setuptools, we should have something like:

python setup.py install --bindir=foodir --sbindir=bardir --mandir=mandir
....

E.g. just copying the categories from autoconf (the ones which make
sense for python packages). So making a FHS-compliant or something like
currently done is the responsibility of the packagers -  assuming the
directories are correctly set by the developer in the setup.py. The main
problem is how to retrieve the different files when it is needed in the
package - I would guess pkg_resources would need to be modified as well
for that purpose,

cheers,

David


From ziade.tarek at gmail.com  Wed Jan 28 12:30:43 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Wed, 28 Jan 2009 12:30:43 +0100
Subject: [Distutils] [Python Language Summit] Distutils / Packaging
	survey
In-Reply-To: <49803BDF.4060204@ar.media.kyoto-u.ac.jp>
References: <94bdd2610901250306n1e27a928sa0b276d91f80862a@mail.gmail.com>
	<90bb445a0901270829y7019eca3p72164c259c4d35d4@mail.gmail.com>
	<94bdd2610901271400k32ff74edmea94fcbdfce88f53@mail.gmail.com>
	<515B9271-0E7D-4B4E-B63F-F717604A5FEA@zooko.com>
	<49803752.9040704@ar.media.kyoto-u.ac.jp>
	<94bdd2610901280307h42b05abcr5599d64189948201@mail.gmail.com>
	<49803BDF.4060204@ar.media.kyoto-u.ac.jp>
Message-ID: <94bdd2610901280330s9977025uad52ba8f52ae3b6f@mail.gmail.com>

On Wed, Jan 28, 2009 at 12:05 PM, David Cournapeau
<david at ar.media.kyoto-u.ac.jp> wrote:
> Tarek Ziad? wrote:
>> On Wed, Jan 28, 2009 at 11:45 AM, David Cournapeau
>> <david at ar.media.kyoto-u.ac.jp> wrote:
>>
>>> It is a solved problem: autoconf does it well, and has all the required
>>> features,
>>>
>>
>> So does it mean that having Distutils generate some kind of
>> "configure.in" template that could
>> be used by autoconf could be the right approach ?
>>
>
> I meant that instead of installing almost everything indistinctly like
> we do now with distutils/setuptools, we should have something like:
>
> python setup.py install --bindir=foodir --sbindir=bardir --mandir=mandir
> ....

Sounds like new metadata to me, but besides the executable scripts location,
how would you deal with man file from the Python developer point of view ?

Man files seem rather specific to Linux, unless we have some kind of script
that would know how to transform a reStructuredText file into a man file ?

Would you like to write down a detailed description of these elements ?


>
> E.g. just copying the categories from autoconf (the ones which make
> sense for python packages). So making a FHS-compliant or something like
> currently done is the responsibility of the packagers -  assuming the
> directories are correctly set by the developer in the setup.py. The main
> problem is how to retrieve the different files when it is needed in the
> package - I would guess pkg_resources would need to be modified as well
> for that purpose,

Yes that was my example (the log file), and the feature needeed to have it
should be thaught and implemented on its own, because setuptools's
pkg_resources
does myriad of things.

Would you like to write down a detail description of that feature as well ?

We could add this feature as a possible enhancement in Distutils, in
the Survey I am building.

Regards
Tarek

From david at ar.media.kyoto-u.ac.jp  Wed Jan 28 12:35:03 2009
From: david at ar.media.kyoto-u.ac.jp (David Cournapeau)
Date: Wed, 28 Jan 2009 20:35:03 +0900
Subject: [Distutils] [Python Language Summit] Distutils / Packaging
	survey
In-Reply-To: <94bdd2610901280330s9977025uad52ba8f52ae3b6f@mail.gmail.com>
References: <94bdd2610901250306n1e27a928sa0b276d91f80862a@mail.gmail.com>	
	<90bb445a0901270829y7019eca3p72164c259c4d35d4@mail.gmail.com>	
	<94bdd2610901271400k32ff74edmea94fcbdfce88f53@mail.gmail.com>	
	<515B9271-0E7D-4B4E-B63F-F717604A5FEA@zooko.com>	
	<49803752.9040704@ar.media.kyoto-u.ac.jp>	
	<94bdd2610901280307h42b05abcr5599d64189948201@mail.gmail.com>	
	<49803BDF.4060204@ar.media.kyoto-u.ac.jp>
	<94bdd2610901280330s9977025uad52ba8f52ae3b6f@mail.gmail.com>
Message-ID: <498042E7.7040907@ar.media.kyoto-u.ac.jp>

Tarek Ziad? wrote:
> On Wed, Jan 28, 2009 at 12:05 PM, David Cournapeau
> <david at ar.media.kyoto-u.ac.jp> wrote:
>   
>> Tarek Ziad? wrote:
>>     
>>> On Wed, Jan 28, 2009 at 11:45 AM, David Cournapeau
>>> <david at ar.media.kyoto-u.ac.jp> wrote:
>>>
>>>       
>>>> It is a solved problem: autoconf does it well, and has all the required
>>>> features,
>>>>
>>>>         
>>> So does it mean that having Distutils generate some kind of
>>> "configure.in" template that could
>>> be used by autoconf could be the right approach ?
>>>
>>>       
>> I meant that instead of installing almost everything indistinctly like
>> we do now with distutils/setuptools, we should have something like:
>>
>> python setup.py install --bindir=foodir --sbindir=bardir --mandir=mandir
>> ....
>>     
>
> Sounds like new metadata to me

Yes - sorry if I don't use the right terms. Although I am quite familiar
with distutils for the build parts (compilation and co) from my work on
numpy/scipy, I am not so familiar with the installation parts and its
vocabulary.

>  but besides the executable scripts location,
> how would you deal with man file from the Python developer point of view ?
>   

What's the problem with man files for python developers ? I am not
saying that any developer is required to add a manpage, only that if
she/he does though, that should be marked as man.

That's quite simple, really: the python developer should not - does not
-  have to care about FHS, windows, mac os X, whatever, only about
marking correctly the different files: mark the pdf as pdf, html doc as
html, etc... so that the *user* (the one executing setup.py) can choose
whatever he wants, like for autoconf. You *can* do ./configure
--mandir=bla --bindir=foo, but most people are fine with ./configure
--prefix.

It also means if the developer made a mistake in the setup.py, the
package maintainer can fix the problem in the setup.py, and sent a patch
upstream (so any platform is fixed). IMHO, that's really a part of
distutils where explicitly avoiding any policy (where to install what)
makes sense, and should be relatively easy to implement.


> Yes that was my example (the log file), and the feature needeed to have it
> should be thaught and implemented on its own, because setuptools's
> pkg_resources
> does myriad of things.
>   

Maybe, I don't claim to be familiar with pkg_resources - I am not a user
of setuptools, except when I am forced to.

> Would you like to write down a detail description of that feature as well ?

I am not sure I will have time to do it - I may underestimate the
difficult of some things because I am not familiar with distutils
implementation for install-related commands. Doing a description without
any design proposal would certainly be possible, though.

cheers,

David


From ziade.tarek at gmail.com  Wed Jan 28 12:59:35 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Wed, 28 Jan 2009 12:59:35 +0100
Subject: [Distutils] [Python Language Summit] Distutils / Packaging
	survey
In-Reply-To: <498042E7.7040907@ar.media.kyoto-u.ac.jp>
References: <94bdd2610901250306n1e27a928sa0b276d91f80862a@mail.gmail.com>
	<90bb445a0901270829y7019eca3p72164c259c4d35d4@mail.gmail.com>
	<94bdd2610901271400k32ff74edmea94fcbdfce88f53@mail.gmail.com>
	<515B9271-0E7D-4B4E-B63F-F717604A5FEA@zooko.com>
	<49803752.9040704@ar.media.kyoto-u.ac.jp>
	<94bdd2610901280307h42b05abcr5599d64189948201@mail.gmail.com>
	<49803BDF.4060204@ar.media.kyoto-u.ac.jp>
	<94bdd2610901280330s9977025uad52ba8f52ae3b6f@mail.gmail.com>
	<498042E7.7040907@ar.media.kyoto-u.ac.jp>
Message-ID: <94bdd2610901280359m5ad835d6y9623d11e6c4b47db@mail.gmail.com>

On Wed, Jan 28, 2009 at 12:35 PM, David Cournapeau
<david at ar.media.kyoto-u.ac.jp> wrote:
> What's the problem with man files for python developers ? I am not
> saying that any developer is required to add a manpage, only that if
> she/he does though, that should be marked as man.

well, I do write some documentation for the commands I have in my
Python application,
but not as man files. Simple text files using the reStructuredText
format because it
is the "common" format in the Python world I guess.

I never bothered to write a man file. Maybe I should, but maybe the
system should
be smart enough and take my text file to make it a man file when it's
packaged in Debian or equivalent,

>
> That's quite simple, really: the python developer should not - does not
> -  have to care about FHS, windows, mac os X, whatever, only about
> marking correctly the different files: mark the pdf as pdf, html doc as
> html, etc... so that the *user* (the one executing setup.py) can choose
> whatever he wants, like for autoconf. You *can* do ./configure
> --mandir=bla --bindir=foo, but most people are fine with ./configure
> --prefix.
>
> It also means if the developer made a mistake in the setup.py, the
> package maintainer can fix the problem in the setup.py, and sent a patch
> upstream (so any platform is fixed). IMHO, that's really a part of
> distutils where explicitly avoiding any policy (where to install what)
> makes sense, and should be relatively easy to implement.
>

Right. From a Python developer point of view, that could maybe be
done in setup.py as long as we have the mechanism to find back all pieces
to work in the code.

>> Would you like to write down a detail description of that feature as well ?
>
> I am not sure I will have time to do it - I may underestimate the
> difficult of some things because I am not familiar with distutils
> implementation for install-related commands. Doing a description without
> any design proposal would certainly be possible, though.
>

A description would be great as well for sure :)

Cheers
Tarek


-- 
Tarek Ziad? | Association AfPy | www.afpy.org
Blog FR | http://programmation-python.org
Blog EN | http://tarekziade.wordpress.com/

From ben+python at benfinney.id.au  Wed Jan 28 13:35:29 2009
From: ben+python at benfinney.id.au (Ben Finney)
Date: Wed, 28 Jan 2009 23:35:29 +1100
Subject: [Distutils] [Python Language Summit] Distutils / Packaging
	survey
References: <94bdd2610901250306n1e27a928sa0b276d91f80862a@mail.gmail.com>
	<90bb445a0901270829y7019eca3p72164c259c4d35d4@mail.gmail.com>
	<94bdd2610901271400k32ff74edmea94fcbdfce88f53@mail.gmail.com>
	<515B9271-0E7D-4B4E-B63F-F717604A5FEA@zooko.com>
	<49803752.9040704@ar.media.kyoto-u.ac.jp>
	<94bdd2610901280307h42b05abcr5599d64189948201@mail.gmail.com>
	<49803BDF.4060204@ar.media.kyoto-u.ac.jp>
	<94bdd2610901280330s9977025uad52ba8f52ae3b6f@mail.gmail.com>
Message-ID: <87bptry5f2.fsf@benfinney.id.au>

Tarek Ziad? <ziade.tarek at gmail.com> writes:

> Man files seem rather specific to Linux

Hardly. Man pages have been part of every Unix-like operating system
(which is to say, all major operating systems except one) since before
Linus even discovered a C compiler.

> unless we have some kind of script that would know how to transform
> a reStructuredText file into a man file ?

There is an experimental ?manpage? writer for docutils, that is
packaged for (at least) Debian [0]. It has a corresponding program,
?rst2man? [1], for exactly the task you describe.


[0]: <URL:http://packages.debian.org/sid/docutils-writer-manpage>
[1]: <URL:http://packages.debian.org/sid/rst2man>

-- 
 \              ?When cryptography is outlawed, bayl bhgynjf jvyy unir |
  `\                                              cevinpl.? ?Anonymous |
_o__)                                                                  |
Ben Finney


From ben+python at benfinney.id.au  Wed Jan 28 13:38:42 2009
From: ben+python at benfinney.id.au (Ben Finney)
Date: Wed, 28 Jan 2009 23:38:42 +1100
Subject: [Distutils] [Python Language Summit] Distutils / Packaging
	survey
References: <94bdd2610901250306n1e27a928sa0b276d91f80862a@mail.gmail.com>
	<90bb445a0901270829y7019eca3p72164c259c4d35d4@mail.gmail.com>
	<94bdd2610901271400k32ff74edmea94fcbdfce88f53@mail.gmail.com>
	<515B9271-0E7D-4B4E-B63F-F717604A5FEA@zooko.com>
	<49803752.9040704@ar.media.kyoto-u.ac.jp>
	<94bdd2610901280307h42b05abcr5599d64189948201@mail.gmail.com>
	<49803BDF.4060204@ar.media.kyoto-u.ac.jp>
	<94bdd2610901280330s9977025uad52ba8f52ae3b6f@mail.gmail.com>
Message-ID: <874ozjy59p.fsf@benfinney.id.au>

Tarek Ziad? <ziade.tarek at gmail.com> writes:

> >>> It is a solved problem: autoconf does it well, and has all the
> >>> required features,
> 
> Would you like to write down a detailed description of these
> elements ?

I second this. David has several times invoked ?setuptools should do
it like autoconf?, but that isn't much use without a detailed
specification of *exactly what* setuptools should do, written by
people who know what is being asked for (like David) and targetted to
people who don't know anything about autoconf.

It's a lot of work to come up with such a specification, and will
require input from experts and much discussion, but until it's done
such requests for change carry little weight.

-- 
 \      ?I wish a robot would get elected president. That way, when he |
  `\    came to town, we could all take a shot at him and not feel too |
_o__)                                               bad.? ?Jack Handey |
Ben Finney


From barry at python.org  Wed Jan 28 14:07:14 2009
From: barry at python.org (Barry Warsaw)
Date: Wed, 28 Jan 2009 08:07:14 -0500
Subject: [Distutils] [Python Language Summit] Distutils /
	Packaging	survey
In-Reply-To: <49803752.9040704@ar.media.kyoto-u.ac.jp>
References: <94bdd2610901250306n1e27a928sa0b276d91f80862a@mail.gmail.com>	<90bb445a0901270829y7019eca3p72164c259c4d35d4@mail.gmail.com>	<94bdd2610901271400k32ff74edmea94fcbdfce88f53@mail.gmail.com>
	<515B9271-0E7D-4B4E-B63F-F717604A5FEA@zooko.com>
	<49803752.9040704@ar.media.kyoto-u.ac.jp>
Message-ID: <B0B60DA3-D275-4DF9-B88D-7903A0DB31F2@python.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Jan 28, 2009, at 5:45 AM, David Cournapeau wrote:

> This is not the right solution for distributions maintainers: it is a
> good tool for individual (it gives you uninstallation, etec...)  
> but .deb
> packages produced by stddeb are not debian-compatible, and cannot be
> included in debian proper. This is not a critic of stddeb, I think  
> it is
> a very good tool and useful tool.
>
> The *only* right solution for packaging python modules on Linux
> distribution is to make it as "easy" for packagers as it is for  
> autoconf
> packages. Meaning having clear differences between installation,  
> binary,
> libraries, etc... (what's called resources by setuptools, IIUC), so  
> that
> maintainers can set it up how they want. This way, python developers  
> do
> not have to care about debian, and distributions maintainers do not  
> have
> to care about python (well, not more than now).
>
> It is a solved problem: autoconf does it well, and has all the  
> required
> features,

I'd like to make a radical suggestion: upstream authors should never  
have to worry about building distribution blobs.

In my ideal world, I would make a release by tagging my release in  
whatever vcs I'm using, then I would tell cheeseshop, "hey I just  
released version 3.1 and it is here" where "here" means whatever  
native vcs syntax points to the revision I just released.  Then PyPI  
would do the "coagulation" into distribution formats and distribute it  
amongst its worldwide mirrors, all automatically.

I as the Python developer don't want to know about eggs, tarballs,  
debs, rpms, whatevers, I just want to write some software.  I'm happy  
to add a bit of metadata to my setup.py to play ball, but otherwise I  
really just want one command to release my code and then magically  
have it appear available to everybody.

If that's not feasible <wink> then the next best thing would be to  
just spin the source tarball and upload that.  Tarballs I can handle.  
<wink>

On the other end, when I zc.buildout, or paver, or easy_install, or  
aptitude install, or emerge, or port install, or whatever, it would go  
out to the Worldwide Python Distribution Network and pull down the  
proper blobby thing to install based on what I'm trying to do, e.g. an  
egg if I'm developing, a deb if I'm installing into my system Python,  
etc.  Again, I don't really care and shouldn't have to know.

Barry

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (Darwin)

iQCVAwUBSYBYgnEjvBPtnXfVAQKM/wQAnoINOJlvFhFB7yHemIujIveFKsZv4cm7
eekwAwEnaIp8fUP2xAuvMZrnGmz51cjRGh7ih24DcZtqPG3pwjrha8+DR7vY9/08
Oa0LKyTvWXDoqC/eCeyTxSvGHe733eDMSp+G283o/XLbgtNUtENfr17+DruRi+9E
pYIJZSYIffc=
=DZic
-----END PGP SIGNATURE-----

From regebro at gmail.com  Wed Jan 28 14:15:05 2009
From: regebro at gmail.com (Lennart Regebro)
Date: Wed, 28 Jan 2009 14:15:05 +0100
Subject: [Distutils] [Python Language Summit] Distutils / Packaging
	survey
In-Reply-To: <B0B60DA3-D275-4DF9-B88D-7903A0DB31F2@python.org>
References: <94bdd2610901250306n1e27a928sa0b276d91f80862a@mail.gmail.com>
	<90bb445a0901270829y7019eca3p72164c259c4d35d4@mail.gmail.com>
	<94bdd2610901271400k32ff74edmea94fcbdfce88f53@mail.gmail.com>
	<515B9271-0E7D-4B4E-B63F-F717604A5FEA@zooko.com>
	<49803752.9040704@ar.media.kyoto-u.ac.jp>
	<B0B60DA3-D275-4DF9-B88D-7903A0DB31F2@python.org>
Message-ID: <319e029f0901280515o30d3ca1fmcae8e69209444a90@mail.gmail.com>

On Wed, Jan 28, 2009 at 14:07, Barry Warsaw <barry at python.org> wrote:
> In my ideal world, I would make a release by tagging my release in whatever
> vcs I'm using, then I would tell cheeseshop, "hey I just released version
> 3.1 and it is here" where "here" means whatever native vcs syntax points to
> the revision I just released.  Then PyPI would do the "coagulation" into
> distribution formats and distribute it amongst its worldwide mirrors, all
> automatically.
>
> I as the Python developer don't want to know about eggs, tarballs, debs,
> rpms, whatevers, I just want to write some software.  I'm happy to add a bit
> of metadata to my setup.py to play ball, but otherwise I really just want
> one command to release my code and then magically have it appear available
> to everybody.

That would without a doubt result in an immense mass of broken debs, etc...

> On the other end, when I zc.buildout, or paver, or easy_install, or aptitude
> install, or emerge, or port install, or whatever, it would go out to the
> Worldwide Python Distribution Network and pull down the proper blobby thing
> to install based on what I'm trying to do, e.g. an egg if I'm developing, a
> deb if I'm installing into my system Python, etc.

That sounds very reasonable. That way if you install the foo python
library, and it requires libfoo to be installed, it could get the
foo.deb, which then would require and install the libfood.deb
automatically. That would be very nice.

-- 
Lennart Regebro: Zope and Plone consulting.
http://www.colliberty.com/
+33 661 58 14 64

From zooko at zooko.com  Wed Jan 28 15:30:50 2009
From: zooko at zooko.com (zooko)
Date: Wed, 28 Jan 2009 07:30:50 -0700
Subject: [Distutils] [Python Language Summit] Distutils / Packaging
	survey
In-Reply-To: <94bdd2610901280127r597f811fmc2a333510aca2fad@mail.gmail.com>
References: <94bdd2610901250306n1e27a928sa0b276d91f80862a@mail.gmail.com>
	<90bb445a0901270829y7019eca3p72164c259c4d35d4@mail.gmail.com>
	<94bdd2610901271400k32ff74edmea94fcbdfce88f53@mail.gmail.com>
	<515B9271-0E7D-4B4E-B63F-F717604A5FEA@zooko.com>
	<94bdd2610901280127r597f811fmc2a333510aca2fad@mail.gmail.com>
Message-ID: <A7E23D21-6885-4249-BC91-5C88A2A1038A@zooko.com>

On Jan 28, 2009, at 2:27 AM, Tarek Ziad? wrote:

> For example, if my application has a log file, it is better under  
> Debian to have it in /var/log/xxx
>
> In the meantime, for the same application, I don't want to bother  
> under win32 about that, the log can leave inside the package for  
> instance.

Yes, I see.  None of my packages have log files, so I haven't had  
that particular problem, but I do have a problem with doc files.   
Under Debian, they ought to be in /usr/share/doc/$PKG/, and so I do  
this in my setup.py:

doc_loc = "share/doc/python-" + PKG
data_files = [(doc_loc, data_fnames)]

http://allmydata.org/trac/zfec/browser/zfec/setup.py?rev=285#L119

This means that stdeb produces a .deb which puts the doc files into  
the right directory, but it also means that if you install with  
easy_install then there will be a "share/doc/python-$PKG" directory  
inside the .egg directory, which isn't ideal.

It would be better if there were a way to inform distutils/setuptools  
that certain files were doc files.  Distutils/setuptools wouldn't  
need to know what to *do* with doc files.  That would be up to a  
plugin like stdeb to say "Oh, doc files!  I'll put them in share/doc/ 
python-$PKG".

> In other words, what would be the new metadata we could put in the  
> setup.py in the package to minimize the work to be done on stdeb  
> side ?

Yes, exactly!

Regards,

Zooko
---
Tahoe, the Least-Authority Filesystem -- http://allmydata.org
store your data: $10/month -- http://allmydata.com/?tracking=zsig

From zooko at zooko.com  Wed Jan 28 15:44:04 2009
From: zooko at zooko.com (zooko)
Date: Wed, 28 Jan 2009 07:44:04 -0700
Subject: [Distutils] [Python Language Summit] Distutils /
	Packaging	survey
In-Reply-To: <49803752.9040704@ar.media.kyoto-u.ac.jp>
References: <94bdd2610901250306n1e27a928sa0b276d91f80862a@mail.gmail.com>	<90bb445a0901270829y7019eca3p72164c259c4d35d4@mail.gmail.com>	<94bdd2610901271400k32ff74edmea94fcbdfce88f53@mail.gmail.com>
	<515B9271-0E7D-4B4E-B63F-F717604A5FEA@zooko.com>
	<49803752.9040704@ar.media.kyoto-u.ac.jp>
Message-ID: <C34F2C72-B619-4A46-B4FA-F4FDB41A5EB2@zooko.com>

On Jan 28, 2009, at 3:45 AM, David Cournapeau wrote:

>> I don't understand what are the potential problems, but so far  
>> I've been happy using stdeb to produce .deb's from my Python sdists.
>
> This is not the right solution for distributions maintainers: it is  
> a good tool for individual (it gives you uninstallation, etec...)  
> but .deb packages produced by stddeb are not debian-compatible, and  
> cannot be included in debian proper. This is not a critic of  
> stddeb, I think it is a very good tool and useful tool.

I've heard things like this from Debian developers before, and I  
don't understand.  Please provide me with more explanation.   I don't  
intend to put words in your mouth, but I will offer a few guesses as  
to why you say stddeb can't be used for Debian proper:

1.  You want the production of .deb's from Python packages to be done  
by a human instead of automatedly, therefore stdeb can't do it.

2.  You want the production of .deb's from Python packages to be done  
by a Debian developer instead of by the upstream developer of the  
Python package.

3.  It would be okay for this process to be automated (or semi- 
automated), but there's some flaw in the design of stdeb which means  
it will never be able to do it right unless stdeb is rewritten with a  
new design.

4.  It would be okay, and the general design of stdeb is okay, but  
there are some bugs in stdeb which currently cause it to produce the  
wrong results.

Thanks!

Regards,

Zooko
---
Tahoe, the Least-Authority Filesystem -- http://allmydata.org
store your data: $10/month -- http://allmydata.com/?tracking=zsig

From iElectric at gmail.com  Wed Jan 28 16:23:12 2009
From: iElectric at gmail.com (=?windows-1252?Q?Domen_Ko=9Ear?=)
Date: Wed, 28 Jan 2009 07:23:12 -0800 (PST)
Subject: [Distutils] including an empty directory
In-Reply-To: <91BCAC8A-4A4B-458B-8DDA-736C4BB7AE9D@zooko.com>
References: <116b0f6c-31aa-4a10-9d1c-ca57a808d19e@i20g2000prf.googlegroups.com>
	<94bdd2610901241518r7e408da8we22a2d198bfc1a1a@mail.gmail.com> 
	<ac5188cd-8b2c-402b-a7a8-1da71325d418@o4g2000pra.googlegroups.com> 
	<32822fe60901250107v148ca0e1ge11bf31e6a102a89@mail.gmail.com> 
	<94bdd2610901250205u7e60b68ahcf826f496d533502@mail.gmail.com> 
	<91BCAC8A-4A4B-458B-8DDA-736C4BB7AE9D@zooko.com>
Message-ID: <20755e5b-2b05-4b11-81dd-f259380a5108@z27g2000prd.googlegroups.com>

nice post zooko, thanks.

On Jan 25, 4:54?pm, zooko <zo... at zooko.com> wrote:
> On Jan 25, 2009, at 3:05 AM, Tarek Ziad? wrote:
>
> > Well if that's a recurrent need maybe we can add this feature.
>
> > What about a new option called "include_empty_folders" (False by ?
> > default)
>
> In a related story, I recently found a problem with the packaging of ?
> nevow. ?We had earlier added "twisted.plugins" to "packages", not ?
> because nevow includes a package named "twisted.plugins", but because ?
> that's the only way we could figure out how to make sure that ?
> distutils and/or setuptools would create the empty "twisted/plugins" ?
> directory on installation which nevow requires:
>
> http://divmod.org/trac/changeset/16896
>
> However, this leads to this warning message at import time:
>
> "/usr/lib/python2.5/site-packages/zope/__init__.py:19: UserWarning: ?
> Module twisted was already imported from /usr/lib/python2.5/site-
> packages/twisted/__init__.pyc, but /home/buildslave/slave-gutsy-tahoe/
> gutsy/build/support/lib/python2.5/site-packages/Nevow-0.9.32-
> py2.5.egg is being added to sys.path"
>
> http://bugs.python.org/setuptools/issue36
>
> So I've just submitted a patch to nevow to use data_files instead of ?
> packages to indicate that this directory should be created, and to do ?
> that we have to include a file in the directory so that distutils ?
> doesn't prune it out for being an empty directory:
>
> http://divmod.org/trac/ticket/2830
>
> One thing that I don't understand about all this is why setuptools's ?
> "include_package_data=True" didn't cause the directory to be ?
> installed. ?I'm not sure if calling that "twisted/plugins" directory ?
> a "data file" or an "included package data" makes more sense, but the ?
> former currently works.
>
> Regards,
>
> Zooko
> ---
> Tahoe, the Least-Authority Filesystem --http://allmydata.org
> store your data: $10/month --http://allmydata.com/?tracking=zsig
> _______________________________________________
> Distutils-SIG maillist ?- ?Distutils-... at python.orghttp://mail.python.org/mailman/listinfo/distutils-sig

From akitada at gmail.com  Wed Jan 28 16:46:03 2009
From: akitada at gmail.com (Akira Kitada)
Date: Thu, 29 Jan 2009 00:46:03 +0900
Subject: [Distutils] [Python Language Summit] Distutils / Packaging
	survey
In-Reply-To: <94bdd2610901271400k32ff74edmea94fcbdfce88f53@mail.gmail.com>
References: <94bdd2610901250306n1e27a928sa0b276d91f80862a@mail.gmail.com>
	<90bb445a0901270829y7019eca3p72164c259c4d35d4@mail.gmail.com>
	<94bdd2610901271400k32ff74edmea94fcbdfce88f53@mail.gmail.com>
Message-ID: <90bb445a0901280746s54929e23pc13c9b8d70676441@mail.gmail.com>

In my opinion, bdist_rpm and the like are "nice hacks" at best
and nothing more.
Peoplo who love rigorous distribution or control freaks would probably
prefer to
bother packaging themselves and that will leads them to use apt, yum...

On Wed, Jan 28, 2009 at 7:00 AM, Tarek Ziad? <ziade.tarek at gmail.com> wrote:
> On Tue, Jan 27, 2009 at 5:29 PM, Akira Kitada <akitada at gmail.com> wrote:
>> Hello Tarek,
>>
>> I think "apt, yum, etc" would be also used for packaging/distributing apps.
>>
>
> There is already a command that let you create a rpm package
> (bdist_rpm) out of a python package,
>
> There were also a bdist_deb project but it never made it to distutils,
> also for Debian there's a policy on how to work with python packages :
> http://www.debian.org/doc/packaging-manuals/python-policy/
>
> Last, this mailing list had a lot of threads about the fact that
> there's no standard way in Python to work with resources that
> could be installed in the system, using a LSB-compliant approach.
>
> So I don't have (I think no one does at this point) any clear view of
> what could be done in this area.
>
> Tarek
> --
> Tarek Ziad? | Association AfPy | www.afpy.org
> Blog FR | http://programmation-python.org
> Blog EN | http://tarekziade.wordpress.com/
>

From tarek.ziade at ingeniweb.com  Wed Jan 28 16:50:46 2009
From: tarek.ziade at ingeniweb.com (Tarek Ziade)
Date: Wed, 28 Jan 2009 16:50:46 +0100
Subject: [Distutils] [Python Language Summit] Distutils / Packaging
	survey
In-Reply-To: <87bptry5f2.fsf@benfinney.id.au>
References: <94bdd2610901250306n1e27a928sa0b276d91f80862a@mail.gmail.com>
	<90bb445a0901270829y7019eca3p72164c259c4d35d4@mail.gmail.com>
	<94bdd2610901271400k32ff74edmea94fcbdfce88f53@mail.gmail.com>
	<515B9271-0E7D-4B4E-B63F-F717604A5FEA@zooko.com>
	<49803752.9040704@ar.media.kyoto-u.ac.jp>
	<94bdd2610901280307h42b05abcr5599d64189948201@mail.gmail.com>
	<49803BDF.4060204@ar.media.kyoto-u.ac.jp>
	<94bdd2610901280330s9977025uad52ba8f52ae3b6f@mail.gmail.com>
	<87bptry5f2.fsf@benfinney.id.au>
Message-ID: <a26746990901280750o4c5d4b31x385137e7e5a659e3@mail.gmail.com>

2009/1/28 Ben Finney <ben+python at benfinney.id.au>:
>> unless we have some kind of script that would know how to transform
>> a reStructuredText file into a man file ?
>
> There is an experimental 'manpage' writer for docutils, that is
> packaged for (at least) Debian [0]. It has a corresponding program,
> 'rst2man' [1], for exactly the task you describe.
>

Sounds like the right tool to keep in mind if something is done on this side,

>
> [0]: <URL:http://packages.debian.org/sid/docutils-writer-manpage>
> [1]: <URL:http://packages.debian.org/sid/rst2man>
>
> --
>  \              "When cryptography is outlawed, bayl bhgynjf jvyy unir |
>  `\                                              cevinpl." ?Anonymous |
> _o__)                                                                  |
> Ben Finney
>
> _______________________________________________
> Distutils-SIG maillist  -  Distutils-SIG at python.org
> http://mail.python.org/mailman/listinfo/distutils-sig
>



-- 
Tarek Ziad? - Directeur Technique
INGENIWEB (TM) - SAS 50000 Euros - RC B 438 725 632
Bureaux de la Colline - 1 rue Royale - B?timent D - 9?me ?tage
92210 Saint Cloud - France
Phone : 01.78.15.24.00 / Fax : 01 46 02 44 04
http://www.ingeniweb.com - une soci?t? du groupe Alter Way

From ziade.tarek at gmail.com  Wed Jan 28 16:56:10 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Wed, 28 Jan 2009 16:56:10 +0100
Subject: [Distutils] [Python Language Summit] Distutils / Packaging
	survey
In-Reply-To: <874ozjy59p.fsf@benfinney.id.au>
References: <94bdd2610901250306n1e27a928sa0b276d91f80862a@mail.gmail.com>
	<90bb445a0901270829y7019eca3p72164c259c4d35d4@mail.gmail.com>
	<94bdd2610901271400k32ff74edmea94fcbdfce88f53@mail.gmail.com>
	<515B9271-0E7D-4B4E-B63F-F717604A5FEA@zooko.com>
	<49803752.9040704@ar.media.kyoto-u.ac.jp>
	<94bdd2610901280307h42b05abcr5599d64189948201@mail.gmail.com>
	<49803BDF.4060204@ar.media.kyoto-u.ac.jp>
	<94bdd2610901280330s9977025uad52ba8f52ae3b6f@mail.gmail.com>
	<874ozjy59p.fsf@benfinney.id.au>
Message-ID: <94bdd2610901280756v5d214704s9ee54906ad8fa765@mail.gmail.com>

On Wed, Jan 28, 2009 at 1:38 PM, Ben Finney <ben+python at benfinney.id.au> wrote:
> Tarek Ziad? <ziade.tarek at gmail.com> writes:
>
>> >>> It is a solved problem: autoconf does it well, and has all the
>> >>> required features,
>>
>> Would you like to write down a detailed description of these
>> elements ?
>
> I second this. David has several times invoked "setuptools should do
> it like autoconf", but that isn't much use without a detailed
> specification of *exactly what* setuptools should do, written by
> people who know what is being asked for (like David) and targetted to
> people who don't know anything about autoconf.
>
> It's a lot of work to come up with such a specification, and will
> require input from experts and much discussion, but until it's done
> such requests for change carry little weight.

Right, maybe the first step would be for David to write down the desired
change in the wiki, to be discussed in this list.

The threads that we had in the past in these topics did not produce any
output unfortunately, because it was hard for everyone to read +100 messages

If a wiki page is started with simple requirements, it can be used to
start to work things out

++
Tarek

-- 
Tarek Ziad? | Association AfPy | www.afpy.org
Blog FR | http://programmation-python.org
Blog EN | http://tarekziade.wordpress.com/

From ziade.tarek at gmail.com  Wed Jan 28 17:07:34 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Wed, 28 Jan 2009 17:07:34 +0100
Subject: [Distutils] [Python Language Summit] Distutils / Packaging
	survey
In-Reply-To: <90bb445a0901280746s54929e23pc13c9b8d70676441@mail.gmail.com>
References: <94bdd2610901250306n1e27a928sa0b276d91f80862a@mail.gmail.com>
	<90bb445a0901270829y7019eca3p72164c259c4d35d4@mail.gmail.com>
	<94bdd2610901271400k32ff74edmea94fcbdfce88f53@mail.gmail.com>
	<90bb445a0901280746s54929e23pc13c9b8d70676441@mail.gmail.com>
Message-ID: <94bdd2610901280807q76cc96f1o8d1785e02bfba4be@mail.gmail.com>

On Wed, Jan 28, 2009 at 4:46 PM, Akira Kitada <akitada at gmail.com> wrote:
> In my opinion, bdist_rpm and the like are "nice hacks" at best
> and nothing more.
> Peoplo who love rigorous distribution or control freaks would probably
> prefer to
> bother packaging themselves and that will leads them to use apt, yum...

I think there's a slight misunderstanding here on what bdist_rpm does :
it creates a rmp file you can then use with yum, it doesn't install anything.

It does it by taking the metadata out of your setup.py file and make a rmp file
with RPM own metadata structure.

see http://docs.python.org/dev/distutils/builtdist.html

>
> On Wed, Jan 28, 2009 at 7:00 AM, Tarek Ziad? <ziade.tarek at gmail.com> wrote:
>> On Tue, Jan 27, 2009 at 5:29 PM, Akira Kitada <akitada at gmail.com> wrote:
>>> Hello Tarek,
>>>
>>> I think "apt, yum, etc" would be also used for packaging/distributing apps.
>>>
>>
>> There is already a command that let you create a rpm package
>> (bdist_rpm) out of a python package,
>>
>> There were also a bdist_deb project but it never made it to distutils,
>> also for Debian there's a policy on how to work with python packages :
>> http://www.debian.org/doc/packaging-manuals/python-policy/
>>
>> Last, this mailing list had a lot of threads about the fact that
>> there's no standard way in Python to work with resources that
>> could be installed in the system, using a LSB-compliant approach.
>>
>> So I don't have (I think no one does at this point) any clear view of
>> what could be done in this area.
>>
>> Tarek
>> --
>> Tarek Ziad? | Association AfPy | www.afpy.org
>> Blog FR | http://programmation-python.org
>> Blog EN | http://tarekziade.wordpress.com/
>>
>



-- 
Tarek Ziad? | Association AfPy | www.afpy.org
Blog FR | http://programmation-python.org
Blog EN | http://tarekziade.wordpress.com/

From regebro at gmail.com  Wed Jan 28 17:09:11 2009
From: regebro at gmail.com (Lennart Regebro)
Date: Wed, 28 Jan 2009 17:09:11 +0100
Subject: [Distutils] [Python Language Summit] Distutils / Packaging
	survey
In-Reply-To: <49803BDF.4060204@ar.media.kyoto-u.ac.jp>
References: <94bdd2610901250306n1e27a928sa0b276d91f80862a@mail.gmail.com>
	<90bb445a0901270829y7019eca3p72164c259c4d35d4@mail.gmail.com>
	<94bdd2610901271400k32ff74edmea94fcbdfce88f53@mail.gmail.com>
	<515B9271-0E7D-4B4E-B63F-F717604A5FEA@zooko.com>
	<49803752.9040704@ar.media.kyoto-u.ac.jp>
	<94bdd2610901280307h42b05abcr5599d64189948201@mail.gmail.com>
	<49803BDF.4060204@ar.media.kyoto-u.ac.jp>
Message-ID: <319e029f0901280809o3d7047d1u6270385b7391b3c2@mail.gmail.com>

On Wed, Jan 28, 2009 at 12:05, David Cournapeau
<david at ar.media.kyoto-u.ac.jp> wrote:
> I meant that instead of installing almost everything indistinctly like
> we do now with distutils/setuptools, we should have something like:
>
> python setup.py install --bindir=foodir --sbindir=bardir --mandir=mandir
> ....

I think this shows how different the goals are for linux packaging and
python packaging. Python packaging is mostly about installing python
libraries, linux packaging is mostly about distributing applications
(or indeed a whole OS). But maybe these options should be there, so
that we can get disutils to nicely distribute applications on Linux as
well?

It would mean that we from a python setup.py command would be able to
create both Windows distros and Linux distros and maybe even OS X
distros once you set up setup.py correctly. Which would be *some* kind
of awesome, although I'm not entirely sure which kind. :)


I'm entirely sure that the Debian people would not just run a
automated debian packager from a setup.py, at least not the first
couple of years until they are sure it works. But that doesn't mean
it's not a good thing to have. I regularily install WingIDE from the
Ubuntu deb-files, even though they are not distributed in the Ubuntu
repository. If you make a python library that has a Ubuntu/debian/red
hat/something distro, you might want to release new versions in a way
that is compatible with this package, even though somebody else made
the package. Or would that make the Linux people very unhappy?

-- 
Lennart Regebro: Zope and Plone consulting.
http://www.colliberty.com/
+33 661 58 14 64

From eric at trueblade.com  Wed Jan 28 16:56:54 2009
From: eric at trueblade.com (Eric Smith)
Date: Wed, 28 Jan 2009 10:56:54 -0500
Subject: [Distutils] [Python Language Summit] Distutils /
	Packaging	survey
In-Reply-To: <90bb445a0901280746s54929e23pc13c9b8d70676441@mail.gmail.com>
References: <94bdd2610901250306n1e27a928sa0b276d91f80862a@mail.gmail.com>	<90bb445a0901270829y7019eca3p72164c259c4d35d4@mail.gmail.com>	<94bdd2610901271400k32ff74edmea94fcbdfce88f53@mail.gmail.com>
	<90bb445a0901280746s54929e23pc13c9b8d70676441@mail.gmail.com>
Message-ID: <49808046.6080208@trueblade.com>

Akira Kitada wrote:
> In my opinion, bdist_rpm and the like are "nice hacks" at best
> and nothing more.
> Peoplo who love rigorous distribution or control freaks would probably
> prefer to
> bother packaging themselves and that will leads them to use apt, yum...

I am both a control freak and love rigorous distribution. I use 
bdist_rpm all the time. There's no reason it can't be as good as 
creating a .spec file by hand, as long as setup.py has all of the 
required metadata. And I think that's the point here, to identify all of 
the metadata.

> 
> On Wed, Jan 28, 2009 at 7:00 AM, Tarek Ziad? <ziade.tarek at gmail.com> wrote:
>> On Tue, Jan 27, 2009 at 5:29 PM, Akira Kitada <akitada at gmail.com> wrote:
>>> Hello Tarek,
>>>
>>> I think "apt, yum, etc" would be also used for packaging/distributing apps.
>>>
>> There is already a command that let you create a rpm package
>> (bdist_rpm) out of a python package,
>>
>> There were also a bdist_deb project but it never made it to distutils,
>> also for Debian there's a policy on how to work with python packages :
>> http://www.debian.org/doc/packaging-manuals/python-policy/
>>
>> Last, this mailing list had a lot of threads about the fact that
>> there's no standard way in Python to work with resources that
>> could be installed in the system, using a LSB-compliant approach.
>>
>> So I don't have (I think no one does at this point) any clear view of
>> what could be done in this area.
>>
>> Tarek
>> --
>> Tarek Ziad? | Association AfPy | www.afpy.org
>> Blog FR | http://programmation-python.org
>> Blog EN | http://tarekziade.wordpress.com/
>>
> _______________________________________________
> Distutils-SIG maillist  -  Distutils-SIG at python.org
> http://mail.python.org/mailman/listinfo/distutils-sig
> 


From zooko at zooko.com  Wed Jan 28 18:03:42 2009
From: zooko at zooko.com (zooko)
Date: Wed, 28 Jan 2009 10:03:42 -0700
Subject: [Distutils] [Python Language Summit] Distutils / Packaging
	survey
In-Reply-To: <87bptry5f2.fsf@benfinney.id.au>
References: <94bdd2610901250306n1e27a928sa0b276d91f80862a@mail.gmail.com>
	<90bb445a0901270829y7019eca3p72164c259c4d35d4@mail.gmail.com>
	<94bdd2610901271400k32ff74edmea94fcbdfce88f53@mail.gmail.com>
	<515B9271-0E7D-4B4E-B63F-F717604A5FEA@zooko.com>
	<49803752.9040704@ar.media.kyoto-u.ac.jp>
	<94bdd2610901280307h42b05abcr5599d64189948201@mail.gmail.com>
	<49803BDF.4060204@ar.media.kyoto-u.ac.jp>
	<94bdd2610901280330s9977025uad52ba8f52ae3b6f@mail.gmail.com>
	<87bptry5f2.fsf@benfinney.id.au>
Message-ID: <C2F4CF48-DD7E-41D9-8EE2-55658B50D035@zooko.com>

Yesterday I spoke to a couple of hackers who use my allmydata-tahoe  
project, which is packaged with setuptools.  They complained about  
setuptools getting in their way, and I asked them each to name their  
top two complaints.

The first hacker, David, said:

David 1.  He can't easily install eggs into an arbitrary directory.   
Setuptools yells at him loudly when he tries.  .pth files work only  
in magical directories.  easy_install.pth makes it impossible to use  
GNU stow.  site.py should just just look for .eggs in all directories  
on sys.path.

David 2.  entry_points doesn't work when the svn checkout is just in  
PYTHONPATH or ".", and fails if there is a non-existent directory on  
his PYTHONPATH.

The second hacker, Nathan, said:

Nathan 1: You cannot easily install in a non-standard location like / 
usr/local/stow/$pkg_name.

Nathan 2: You cannot uninstall.

Both David's and Nathan's first desire would be somewhat improved by  
my patch http://bugs.python.org/setuptools/issue54 (be more like  
distutils with regard to --prefix=).  Both of them would be *more*  
improved by my proposed extension to the standard Python import  
mechanism: "how to easily consume just the parts of eggs that are  
good for you" [1] (which is exactly what David suggested, except he  
suggested site.py do that instead of python importer doing it).

David's second problem I don't understand.  Perhaps it is a bug in  
setuptools?

Nathan's second problem -- you can't uninstall -- is widely known.   
Note that using GNU stow is one excellent solution to this problem,  
so perhaps if the first problem were solved then we would have  
another tool against the second.  Also, most users of setuptools  
don't seem to realize that they can uninstall almost everything  
(everything except scripts) simply by removing the .egg.

I see that PJE has replied to http://bugs.python.org/setuptools/ 
issue54 .  I'll follow-up on that ticket.

Regards,

Zooko

[1] http://mail.python.org/pipermail/python-dev/2008-March/078243.html
---
Tahoe, the Least-Authority Filesystem -- http://allmydata.org
store your data: $10/month -- http://allmydata.com/?tracking=zsig

From tarek.ziade at ingeniweb.com  Wed Jan 28 18:11:37 2009
From: tarek.ziade at ingeniweb.com (Tarek Ziade)
Date: Wed, 28 Jan 2009 18:11:37 +0100
Subject: [Distutils] [Python Language Summit] Distutils / Packaging
	survey
In-Reply-To: <C2F4CF48-DD7E-41D9-8EE2-55658B50D035@zooko.com>
References: <94bdd2610901250306n1e27a928sa0b276d91f80862a@mail.gmail.com>
	<90bb445a0901270829y7019eca3p72164c259c4d35d4@mail.gmail.com>
	<94bdd2610901271400k32ff74edmea94fcbdfce88f53@mail.gmail.com>
	<515B9271-0E7D-4B4E-B63F-F717604A5FEA@zooko.com>
	<49803752.9040704@ar.media.kyoto-u.ac.jp>
	<94bdd2610901280307h42b05abcr5599d64189948201@mail.gmail.com>
	<49803BDF.4060204@ar.media.kyoto-u.ac.jp>
	<94bdd2610901280330s9977025uad52ba8f52ae3b6f@mail.gmail.com>
	<87bptry5f2.fsf@benfinney.id.au>
	<C2F4CF48-DD7E-41D9-8EE2-55658B50D035@zooko.com>
Message-ID: <a26746990901280911h566e0b94u948422c42ae9e589@mail.gmail.com>

2009/1/28 zooko <zooko at zooko.com>:
> Yesterday I spoke to a couple of hackers who use my allmydata-tahoe project,
> which is packaged with setuptools.  They complained about setuptools getting
> in their way, and I asked them each to name their top two complaints.

Please be aware that the Language Summit, and the Survey we are
building for it is mostly about Distutils and PyPI, not setuptools.

>From my point of view, there's no intent to contribute in setuptools,
but rather to see what are the good bits in setuptools that
could be put in Distutils, because this project adds gerat features.

That said, the uninstall feature is a requirement on Distutils level
as well, and that could be used in setuptools.

>
> The first hacker, David, said:
>
> David 1.  He can't easily install eggs into an arbitrary directory.
>  Setuptools yells at him loudly when he tries.  .pth files work only in
> magical directories.  easy_install.pth makes it impossible to use GNU stow.
>  site.py should just just look for .eggs in all directories on sys.path.
>
> David 2.  entry_points doesn't work when the svn checkout is just in
> PYTHONPATH or ".", and fails if there is a non-existent directory on his
> PYTHONPATH.
>
> The second hacker, Nathan, said:
>
> Nathan 1: You cannot easily install in a non-standard location like
> /usr/local/stow/$pkg_name.
>
> Nathan 2: You cannot uninstall.
>
> Both David's and Nathan's first desire would be somewhat improved by my
> patch http://bugs.python.org/setuptools/issue54 (be more like distutils with
> regard to --prefix=).  Both of them would be *more* improved by my proposed
> extension to the standard Python import mechanism: "how to easily consume
> just the parts of eggs that are good for you" [1] (which is exactly what
> David suggested, except he suggested site.py do that instead of python
> importer doing it).
>
> David's second problem I don't understand.  Perhaps it is a bug in
> setuptools?
>
> Nathan's second problem -- you can't uninstall -- is widely known.  Note
> that using GNU stow is one excellent solution to this problem, so perhaps if
> the first problem were solved then we would have another tool against the
> second.  Also, most users of setuptools don't seem to realize that they can
> uninstall almost everything (everything except scripts) simply by removing
> the .egg.
>
> I see that PJE has replied to http://bugs.python.org/setuptools/issue54 .
>  I'll follow-up on that ticket.
>
> Regards,
>
> Zooko
>
> [1] http://mail.python.org/pipermail/python-dev/2008-March/078243.html
> ---
> Tahoe, the Least-Authority Filesystem -- http://allmydata.org
> store your data: $10/month -- http://allmydata.com/?tracking=zsig
> _______________________________________________
> Distutils-SIG maillist  -  Distutils-SIG at python.org
> http://mail.python.org/mailman/listinfo/distutils-sig
>



-- 
Tarek Ziad? - Directeur Technique
INGENIWEB (TM) - SAS 50000 Euros - RC B 438 725 632
Bureaux de la Colline - 1 rue Royale - B?timent D - 9?me ?tage
92210 Saint Cloud - France
Phone : 01.78.15.24.00 / Fax : 01 46 02 44 04
http://www.ingeniweb.com - une soci?t? du groupe Alter Way

From ianb at colorstudy.com  Wed Jan 28 20:02:43 2009
From: ianb at colorstudy.com (Ian Bicking)
Date: Wed, 28 Jan 2009 13:02:43 -0600
Subject: [Distutils] [Python Language Summit] Distutils / Packaging
	survey
In-Reply-To: <C2F4CF48-DD7E-41D9-8EE2-55658B50D035@zooko.com>
References: <94bdd2610901250306n1e27a928sa0b276d91f80862a@mail.gmail.com>
	<90bb445a0901270829y7019eca3p72164c259c4d35d4@mail.gmail.com>
	<94bdd2610901271400k32ff74edmea94fcbdfce88f53@mail.gmail.com>
	<515B9271-0E7D-4B4E-B63F-F717604A5FEA@zooko.com>
	<49803752.9040704@ar.media.kyoto-u.ac.jp>
	<94bdd2610901280307h42b05abcr5599d64189948201@mail.gmail.com>
	<49803BDF.4060204@ar.media.kyoto-u.ac.jp>
	<94bdd2610901280330s9977025uad52ba8f52ae3b6f@mail.gmail.com>
	<87bptry5f2.fsf@benfinney.id.au>
	<C2F4CF48-DD7E-41D9-8EE2-55658B50D035@zooko.com>
Message-ID: <b654cd2e0901281102j21d62decpc12163942352da25@mail.gmail.com>

On Wed, Jan 28, 2009 at 11:03 AM, zooko <zooko at zooko.com> wrote:

> David 2.  entry_points doesn't work when the svn checkout is just in
> PYTHONPATH or ".", and fails if there is a non-existent directory on his
> PYTHONPATH.
>

Probably this is because the package isn't activated, and if it's not
activated you can't see its entry point.  When a .pth file is on PYTHONPATH,
Python won't load it up (it only loads .pth files in some specific
locations).  So while easy-install.pth would normally activate a package (by
adding it to sys.path), with PYTHONPATH that doesn't work.

I think the site.py that Setuptools will sometimes create is intended to
address this, but it might not always work.

Or there might be some entirely different problem I'm unaware of.

-- 
Ian Bicking  |  http://blog.ianbicking.org
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20090128/beb48c32/attachment.htm>

From ianb at colorstudy.com  Wed Jan 28 20:16:50 2009
From: ianb at colorstudy.com (Ian Bicking)
Date: Wed, 28 Jan 2009 13:16:50 -0600
Subject: [Distutils] [Python Language Summit] Distutils / Packaging
	survey
In-Reply-To: <49803BDF.4060204@ar.media.kyoto-u.ac.jp>
References: <94bdd2610901250306n1e27a928sa0b276d91f80862a@mail.gmail.com>
	<90bb445a0901270829y7019eca3p72164c259c4d35d4@mail.gmail.com>
	<94bdd2610901271400k32ff74edmea94fcbdfce88f53@mail.gmail.com>
	<515B9271-0E7D-4B4E-B63F-F717604A5FEA@zooko.com>
	<49803752.9040704@ar.media.kyoto-u.ac.jp>
	<94bdd2610901280307h42b05abcr5599d64189948201@mail.gmail.com>
	<49803BDF.4060204@ar.media.kyoto-u.ac.jp>
Message-ID: <b654cd2e0901281116u57f8967dl785ffc09a0edf291@mail.gmail.com>

On Wed, Jan 28, 2009 at 5:05 AM, David Cournapeau <
david at ar.media.kyoto-u.ac.jp> wrote:

> I meant that instead of installing almost everything indistinctly like
> we do now with distutils/setuptools, we should have something like:
>
> python setup.py install --bindir=foodir --sbindir=bardir --mandir=mandir
> ....
>
> E.g. just copying the categories from autoconf (the ones which make
> sense for python packages). So making a FHS-compliant or something like
> currently done is the responsibility of the packagers -  assuming the
> directories are correctly set by the developer in the setup.py. The main
> problem is how to retrieve the different files when it is needed in the
> package - I would guess pkg_resources would need to be modified as well
> for that purpose,
>

In the previous discussion (somewhere, I'm too lazy to look it up) people
started getting interested in the idea of an improved sdist format.  I think
the basic idea was to tag files by type.  Then (I guess) you'd just do:

  python setup.py build

to get the platform-specific parts of the library built, and then move the
files into place based on the descriptions of the files.  The actual tool to
do this would be external to setup.py and distutils; e.g., you'd have a
python-sdist-to-rpm tool, developed entirely separately from distutils or
setuptools.  Potentially this would make it easier to provide your own file
tags, so you could adapt a library without requiring immediate upstream
support or patches.

As you mention, there would have to be some extension to pkg_resources (or
an equivalent library) to handle finding these files at runtime.  Getting a
runtime in place is probably the harder thing, as it is more intrusive for
the upstream developers.

-- 
Ian Bicking  |  http://blog.ianbicking.org
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20090128/361f4b12/attachment-0001.htm>

From zooko at zooko.com  Wed Jan 28 23:03:50 2009
From: zooko at zooko.com (zooko)
Date: Wed, 28 Jan 2009 15:03:50 -0700
Subject: [Distutils] announcing setuptools_trial -- a setuptools plugin to
	run unit tests with trial
Message-ID: <7302EA94-C6B6-4DBD-B970-50231424C33A@zooko.com>

Hey folks, if you use setuptools to build your project and you want  
to use Twisted trial to run the unit tests, just install  
setuptools_trial and run "python ./setup.py trial":

http://pypi.python.org/pypi/setuptools_trial

Thanks to Chris Galvan and the folks from the Elisa project.

Regards,

Zooko

From floris.bruynooghe at gmail.com  Wed Jan 28 23:54:02 2009
From: floris.bruynooghe at gmail.com (Floris Bruynooghe)
Date: Wed, 28 Jan 2009 22:54:02 +0000
Subject: [Distutils] [Python Language Summit] Distutils /
	Packaging	survey
In-Reply-To: <C34F2C72-B619-4A46-B4FA-F4FDB41A5EB2@zooko.com>
References: <94bdd2610901250306n1e27a928sa0b276d91f80862a@mail.gmail.com>
	<90bb445a0901270829y7019eca3p72164c259c4d35d4@mail.gmail.com>
	<94bdd2610901271400k32ff74edmea94fcbdfce88f53@mail.gmail.com>
	<515B9271-0E7D-4B4E-B63F-F717604A5FEA@zooko.com>
	<49803752.9040704@ar.media.kyoto-u.ac.jp>
	<C34F2C72-B619-4A46-B4FA-F4FDB41A5EB2@zooko.com>
Message-ID: <20090128225402.GB4101@laurie.devork>

Hello Zooko

On Wed, Jan 28, 2009 at 07:44:04AM -0700, zooko wrote:
> On Jan 28, 2009, at 3:45 AM, David Cournapeau wrote:
>
>>> I don't understand what are the potential problems, but so far I've 
>>> been happy using stdeb to produce .deb's from my Python sdists.
>>
>> This is not the right solution for distributions maintainers: it is a 
>> good tool for individual (it gives you uninstallation, etec...) but 
>> .deb packages produced by stddeb are not debian-compatible, and cannot 
>> be included in debian proper. This is not a critic of stddeb, I think 
>> it is a very good tool and useful tool.
>
> I've heard things like this from Debian developers before, and I don't 
> understand.  Please provide me with more explanation.

The answer (or the part that comes to my mind at this moment at least)
is: Policy, Policy and Policy.  Debian is so good in no small part
because each package has to follow the Policy (and all applicable
sub-policies).

* I don't trust a random python developer who might never even have
  heard of Debian to pick the correct name of their python project
  that will ensure the resulting package name will be correct and not
  conflict.

* I know for a fact that distutils capabilities are not good enough to
  fully automatically follow the FHS (discussed many a times on this
  list already).

* setup.py allows for arbitrary code to be executed, if you don't know
  the proper distutils way to do something you can make it do it in
  many different ways.  These will break assumptions automated tools
  will make.

* <more variations on the same themes here>

> I don't intend to 
> put words in your mouth, but I will offer a few guesses as to why you say 
> stddeb can't be used for Debian proper:

This is pretty much flamebait but I'll answer anyway, again it boils
down to the above points though.

> 1.  You want the production of .deb's from Python packages to be done by 
> a human instead of automatedly, therefore stdeb can't do it.

Yes, currently distutils is just not good enough.  Maybe if the tools
(including distutils) improve this could be wittled down to just
checking setup.py (or equivalent) declares everything properly before
letting the automated tools loose.

> 2.  You want the production of .deb's from Python packages to be done by 
> a Debian developer instead of by the upstream developer of the Python 
> package.

Yes, someone who cares about the Debian Policy.

> 3.  It would be okay for this process to be automated (or semi- 
> automated), but there's some flaw in the design of stdeb which means it 
> will never be able to do it right unless stdeb is rewritten with a new 
> design.

I have no idea about stdeb as I've never used it, but distutils just
doesn't provide enough information as I mentioned earlier.  So yes.

> 4.  It would be okay, and the general design of stdeb is okay, but there 
> are some bugs in stdeb which currently cause it to produce the wrong 
> results.

Ugh, really repeating things by now.  But for a change I'll go for
"no" as this is simply not applicable.


There are probably many more reasons that I'm not thinking off right
now.  Most of those can probably be found sprayed around in various
long threads on this list.  IIRC Josselin Mouette did once summarise
Debian's issues quite decently on this list.


Regards
Floris

-- 
Debian GNU/Linux -- The Power of Freedom
www.debian.org | www.gnu.org | www.kernel.org

From david at ar.media.kyoto-u.ac.jp  Thu Jan 29 02:47:22 2009
From: david at ar.media.kyoto-u.ac.jp (David Cournapeau)
Date: Thu, 29 Jan 2009 10:47:22 +0900
Subject: [Distutils] [Python Language Summit] Distutils /
	Packaging	survey
In-Reply-To: <B0B60DA3-D275-4DF9-B88D-7903A0DB31F2@python.org>
References: <94bdd2610901250306n1e27a928sa0b276d91f80862a@mail.gmail.com>	<90bb445a0901270829y7019eca3p72164c259c4d35d4@mail.gmail.com>	<94bdd2610901271400k32ff74edmea94fcbdfce88f53@mail.gmail.com>
	<515B9271-0E7D-4B4E-B63F-F717604A5FEA@zooko.com>
	<49803752.9040704@ar.media.kyoto-u.ac.jp>
	<B0B60DA3-D275-4DF9-B88D-7903A0DB31F2@python.org>
Message-ID: <49810AAA.3050300@ar.media.kyoto-u.ac.jp>

Barry Warsaw wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> On Jan 28, 2009, at 5:45 AM, David Cournapeau wrote:
>
>> This is not the right solution for distributions maintainers: it is a
>> good tool for individual (it gives you uninstallation, etec...) but .deb
>> packages produced by stddeb are not debian-compatible, and cannot be
>> included in debian proper. This is not a critic of stddeb, I think it is
>> a very good tool and useful tool.
>>
>> The *only* right solution for packaging python modules on Linux
>> distribution is to make it as "easy" for packagers as it is for autoconf
>> packages. Meaning having clear differences between installation, binary,
>> libraries, etc... (what's called resources by setuptools, IIUC), so that
>> maintainers can set it up how they want. This way, python developers do
>> not have to care about debian, and distributions maintainers do not have
>> to care about python (well, not more than now).
>>
>> It is a solved problem: autoconf does it well, and has all the required
>> features,
>
> I'd like to make a radical suggestion: upstream authors should never
> have to worry about building distribution blobs.

that you think it is radical is quite saying about the state of affairs
in python IMHO :) For me, it is obvious that the upstream author should
not have to worry about debian if he is not producing .deb. The problem
is just that today, we make it much harder for packagers than it needs
to be.

cheers,

David

From david.lyon at preisshare.net  Thu Jan 29 02:11:19 2009
From: david.lyon at preisshare.net (david.lyon at preisshare.net)
Date: Wed, 28 Jan 2009 20:11:19 -0500
Subject: [Distutils] GUI for Python package Management
Message-ID: <4db42df7554839e3151045936e5b0ee4@preisshare.net>

Hi All,

I am a developer from other languages and find myself on python now.

I'm wondering if anybody has objections to doing a package management gui
for python.

My design suggestions would be to:

 - use WXWidgets (cross compatability for most pythonic platforms)
 - provide a GUI wrapper for EasyInstall
 - source packages from http://pypi.python.org/pypi

It seems we have most of the componentry in place. The problem seems to be
that it doesn't come together in a very user friendly way at the moment
that is cohesive.

Can anybody/everybody flame/tell me why we shouldn't work to simplify this
area of python ?






From david at ar.media.kyoto-u.ac.jp  Thu Jan 29 03:00:40 2009
From: david at ar.media.kyoto-u.ac.jp (David Cournapeau)
Date: Thu, 29 Jan 2009 11:00:40 +0900
Subject: [Distutils] [Python Language Summit] Distutils /
	Packaging	survey
In-Reply-To: <C34F2C72-B619-4A46-B4FA-F4FDB41A5EB2@zooko.com>
References: <94bdd2610901250306n1e27a928sa0b276d91f80862a@mail.gmail.com>	<90bb445a0901270829y7019eca3p72164c259c4d35d4@mail.gmail.com>	<94bdd2610901271400k32ff74edmea94fcbdfce88f53@mail.gmail.com>
	<515B9271-0E7D-4B4E-B63F-F717604A5FEA@zooko.com>
	<49803752.9040704@ar.media.kyoto-u.ac.jp>
	<C34F2C72-B619-4A46-B4FA-F4FDB41A5EB2@zooko.com>
Message-ID: <49810DC8.40705@ar.media.kyoto-u.ac.jp>

zooko wrote:
> On Jan 28, 2009, at 3:45 AM, David Cournapeau wrote:
>
>>> I don't understand what are the potential problems, but so far I've
>>> been happy using stdeb to produce .deb's from my Python sdists.
>>
>> This is not the right solution for distributions maintainers: it is a
>> good tool for individual (it gives you uninstallation, etec...) but
>> .deb packages produced by stddeb are not debian-compatible, and
>> cannot be included in debian proper. This is not a critic of stddeb,
>> I think it is a very good tool and useful tool.
>
> I've heard things like this from Debian developers before, and I don't
> understand.  Please provide me with more explanation.   I don't intend
> to put words in your mouth, but I will offer a few guesses as to why
> you say stddeb can't be used for Debian proper:
>
> 1.  You want the production of .deb's from Python packages to be done
> by a human instead of automatedly, therefore stdeb can't do it.

I don't *want* human production (I think some Debian developers want to
- but that's not something that we need to care about I think). But for
non trivial packages, human intervention will always be needed: packages
which use autoconf cannot be automated either, because you may need
post/pre install scripts, you need to split doc/non doc parts, devel and
non devel parts, debug/release, etc... All of this is seen as a good
thing and some even required by Debian policy.

>
> 2.  You want the production of .deb's from Python packages to be done
> by a Debian developer instead of by the upstream developer of the
> Python package.

That's mandatory, indeed. An *official* debian package can only be done
by a debian developer, almost by definition - only official debian
developers can upload .deb to official debian repositories. This has no
consequences for the python developer, though.

>
> 3.  It would be okay for this process to be automated (or
> semi-automated), but there's some flaw in the design of stdeb which
> means it will never be able to do it right unless stdeb is rewritten
> with a new design.

There are some fundamental issues in *distutils* which make it
impossible to do it correctly at the moment, mainly the lack of metadata
about the installed files, but it looks like this point is already
understood and agreed on. I will work on the requirements,

cheers,

David

From david.lyon at preisshare.net  Thu Jan 29 03:18:58 2009
From: david.lyon at preisshare.net (david.lyon at preisshare.net)
Date: Wed, 28 Jan 2009 21:18:58 -0500
Subject: [Distutils]
 =?utf-8?q?=5BPython_Language_Summit=5D_Distutils_/_Pa?=
 =?utf-8?q?ckaging=09survey?=
In-Reply-To: <49810AAA.3050300@ar.media.kyoto-u.ac.jp>
References: <94bdd2610901250306n1e27a928sa0b276d91f80862a@mail.gmail.com>	<90bb445a0901270829y7019eca3p72164c259c4d35d4@mail.gmail.com>	<94bdd2610901271400k32ff74edmea94fcbdfce88f53@mail.gmail.com>
	<515B9271-0E7D-4B4E-B63F-F717604A5FEA@zooko.com>
	<49803752.9040704@ar.media.kyoto-u.ac.jp>
	<B0B60DA3-D275-4DF9-B88D-7903A0DB31F2@python.org>
	<49810AAA.3050300@ar.media.kyoto-u.ac.jp>
Message-ID: <222cb5ff86e3aa612a59db8389f4d087@preisshare.net>


>> I'd like to make a radical suggestion: upstream authors should never
>> have to worry about building distribution blobs.
> 
> that you think it is radical is quite saying about the state of affairs
> in python IMHO :) For me, it is obvious that the upstream author should
> not have to worry about debian if he is not producing .deb. The problem
> is just that today, we make it much harder for packagers than it needs
> to be.

Can I add my 2c...

Sometimes it is easier under linux to get an operating system python
package (from debian for example) installed than it is to get a pure python
package...

That seems very wrong to me.....

So I am agreeing...

What can we do to fix this?


From david.lyon at preisshare.net  Thu Jan 29 03:20:22 2009
From: david.lyon at preisshare.net (david.lyon at preisshare.net)
Date: Wed, 28 Jan 2009 21:20:22 -0500
Subject: [Distutils]
 =?utf-8?q?=5BPython_Language_Summit=5D_Distutils_/_Pa?=
 =?utf-8?q?ckaging=09survey?=
In-Reply-To: <49810DC8.40705@ar.media.kyoto-u.ac.jp>
References: <94bdd2610901250306n1e27a928sa0b276d91f80862a@mail.gmail.com>	<90bb445a0901270829y7019eca3p72164c259c4d35d4@mail.gmail.com>	<94bdd2610901271400k32ff74edmea94fcbdfce88f53@mail.gmail.com>
	<515B9271-0E7D-4B4E-B63F-F717604A5FEA@zooko.com>
	<49803752.9040704@ar.media.kyoto-u.ac.jp>
	<C34F2C72-B619-4A46-B4FA-F4FDB41A5EB2@zooko.com>
	<49810DC8.40705@ar.media.kyoto-u.ac.jp>
Message-ID: <f0d59551b346fa5c6eb1ce00c61d4635@preisshare.net>


Forgive me if I am new... but I totally agree....

On Thu, 29 Jan 2009 11:00:40 +0900, David Cournapeau
<david at ar.media.kyoto-u.ac.jp> wrote:
> zooko wrote:
>> On Jan 28, 2009, at 3:45 AM, David Cournapeau wrote:
>>
>>>> I don't understand what are the potential problems, but so far I've
>>>> been happy using stdeb to produce .deb's from my Python sdists.
>>>
>>> This is not the right solution for distributions maintainers: it is a
>>> good tool for individual (it gives you uninstallation, etec...) but
>>> .deb packages produced by stddeb are not debian-compatible, and
>>> cannot be included in debian proper. This is not a critic of stddeb,
>>> I think it is a very good tool and useful tool.
>>
>> I've heard things like this from Debian developers before, and I don't
>> understand.  Please provide me with more explanation.   I don't intend
>> to put words in your mouth, but I will offer a few guesses as to why
>> you say stddeb can't be used for Debian proper:
>>
>> 1.  You want the production of .deb's from Python packages to be done
>> by a human instead of automatedly, therefore stdeb can't do it.
> 
> I don't *want* human production (I think some Debian developers want to
> - but that's not something that we need to care about I think). But for
> non trivial packages, human intervention will always be needed: packages
> which use autoconf cannot be automated either, because you may need
> post/pre install scripts, you need to split doc/non doc parts, devel and
> non devel parts, debug/release, etc... All of this is seen as a good
> thing and some even required by Debian policy.
> 
>>
>> 2.  You want the production of .deb's from Python packages to be done
>> by a Debian developer instead of by the upstream developer of the
>> Python package.
> 
> That's mandatory, indeed. An *official* debian package can only be done
> by a debian developer, almost by definition - only official debian
> developers can upload .deb to official debian repositories. This has no
> consequences for the python developer, though.
> 
>>
>> 3.  It would be okay for this process to be automated (or
>> semi-automated), but there's some flaw in the design of stdeb which
>> means it will never be able to do it right unless stdeb is rewritten
>> with a new design.
> 
> There are some fundamental issues in *distutils* which make it
> impossible to do it correctly at the moment, mainly the lack of metadata
> about the installed files, but it looks like this point is already
> understood and agreed on. I will work on the requirements,
> 
> cheers,
> 
> David
> _______________________________________________
> Distutils-SIG maillist  -  Distutils-SIG at python.org
> http://mail.python.org/mailman/listinfo/distutils-sig


From santagada at gmail.com  Thu Jan 29 03:49:33 2009
From: santagada at gmail.com (Leonardo Santagada)
Date: Thu, 29 Jan 2009 00:49:33 -0200
Subject: [Distutils] GUI for Python package Management
In-Reply-To: <4db42df7554839e3151045936e5b0ee4@preisshare.net>
References: <4db42df7554839e3151045936e5b0ee4@preisshare.net>
Message-ID: <7CCBA902-15CE-441A-BEE6-DD0972026530@gmail.com>


On Jan 28, 2009, at 11:11 PM, <david.lyon at preisshare.net> <david.lyon at preisshare.net 
 > wrote:

> Hi All,
>
> I am a developer from other languages and find myself on python now.
>
> I'm wondering if anybody has objections to doing a package  
> management gui
> for python.
>
> My design suggestions would be to:
>
> - use WXWidgets (cross compatability for most pythonic platforms)
> - provide a GUI wrapper for EasyInstall
> - source packages from http://pypi.python.org/pypi
>
> It seems we have most of the componentry in place. The problem seems  
> to be
> that it doesn't come together in a very user friendly way at the  
> moment
> that is cohesive.
>
> Can anybody/everybody flame/tell me why we shouldn't work to  
> simplify this
> area of python ?


I don't think that the missing of a gui is what is the problem today  
for python. Package uninstall is something that bothers some (maybe a  
lot) of users. Another point is to have something like webstart for  
java. If I understand how it works, running an application goes like  
this: If you open an app and it depends on another package not  
installed on your system java goes around and download the missing  
modules for you (and stores them in a relative place to the package or  
the user package dir) so the application can run. This would be cool  
for python, the problem being that there is no security to guarantee  
that those modules are not malicious in any way...

That is it, I would not opose to have a gui to install python modules,  
but I couldn't care less for one. There are a lot more pressing  
matters in the distutils/setuptools part of the python env than a gui.

--
Leonardo Santagada
santagada at gmail.com




From david.lyon at preisshare.net  Thu Jan 29 04:07:00 2009
From: david.lyon at preisshare.net (david.lyon at preisshare.net)
Date: Wed, 28 Jan 2009 22:07:00 -0500
Subject: [Distutils] GUI for Python package Management
In-Reply-To: <7CCBA902-15CE-441A-BEE6-DD0972026530@gmail.com>
References: <4db42df7554839e3151045936e5b0ee4@preisshare.net>
	<7CCBA902-15CE-441A-BEE6-DD0972026530@gmail.com>
Message-ID: <7edd5adaf7ff6650b908cfc76362a64a@preisshare.net>


Hi Leonardo,

> I don't think that the missing of a gui is what is the problem today
> for python. Package uninstall is something that bothers some (maybe a
> lot) of users. 

Yeah. Me too. Possibly made easier by having a GUI.

> Another point is to have something like webstart for
> java. If I understand how it works, running an application goes like
> this: If you open an app and it depends on another package not
> installed on your system java goes around and download the missing
> modules for you (and stores them in a relative place to the package or
> the user package dir) so the application can run. This would be cool
> for python, the problem being that there is no security to guarantee
> that those modules are not malicious in any way...

I agree with that strongly.

In my travels, I have just come across PyPI. Which seems like a sort of
CPAN. It seems to have a good base of packages. I am not sure if that is
competitive to distutils or not, but would like to know.

Following is some code I could incorporate into the GUI:

    PyPI's XML-RPC methods

    Example usage:

    >>> import xmlrpclib
    >>> server = xmlrpclib.Server('http://pypi.python.org/pypi')
    >>> server.package_releases('roundup')
    ['1.1.2']
    >>> server.search('roundup', '1.1.2')

> That is it, I would not opose to have a gui to install python modules,
> but I couldn't care less for one. There are a lot more pressing
> matters in the distutils/setuptools part of the python env than a gui.

hmm... I am kindof the reverse. :-) Not keen to work on the code behind the
scenes only keen to have some gui. 

Hey, thanks. Very informative. Hope I can do something.

David 





From amitsaha.in at gmail.com  Thu Jan 29 04:46:39 2009
From: amitsaha.in at gmail.com (Amit k. Saha)
Date: Thu, 29 Jan 2009 09:16:39 +0530
Subject: [Distutils] GUI for Python package Management
In-Reply-To: <4db42df7554839e3151045936e5b0ee4@preisshare.net>
References: <4db42df7554839e3151045936e5b0ee4@preisshare.net>
Message-ID: <547db2260901281946m23df891r4a0458939ee0cd64@mail.gmail.com>

Hello!

On Thu, Jan 29, 2009 at 6:41 AM,  <david.lyon at preisshare.net> wrote:
> Hi All,
>
> I am a developer from other languages and find myself on python now.
>
> I'm wondering if anybody has objections to doing a package management gui
> for python.
>
> My design suggestions would be to:
>
>  - use WXWidgets (cross compatability for most pythonic platforms)
>  - provide a GUI wrapper for EasyInstall
>  - source packages from http://pypi.python.org/pypi
>
> It seems we have most of the componentry in place. The problem seems to be
> that it doesn't come together in a very user friendly way at the moment
> that is cohesive.
>
> Can anybody/everybody flame/tell me why we shouldn't work to simplify this
> area of python ?

I am one of developers working on the Python support in NetBeans
project: http://wiki.netbeans.org/Python. I am working on the design
of a "Egg Manager" in NetBeans, which will enable the Python developer
to manage the "eggs" from NetBeans IDE itself. The manager will use
'easy_install' for the time being, via its XML-RPC interface, and will
have support for user created 'virtualenv' platforms.

This I am sure will be a definite value addition to the Python
developers who are working with NetBeans. Coming to your question,( I
am a newbie to this area), but seems like 'PyPI' is perhaps the  best
way to go with if anyone wants to design something related Package
manager for Python and 'easy_install' will be the background working
'thread'.

Just my 2-cents..

-Amit

>
>
>
>
>
> _______________________________________________
> Distutils-SIG maillist  -  Distutils-SIG at python.org
> http://mail.python.org/mailman/listinfo/distutils-sig
>



-- 
Amit Kumar Saha
http://amitksaha.blogspot.com
http://amitsaha.in.googlepages.com/
*Bangalore Open Java Users Group*:http:www.bojug.in

From strawman at astraw.com  Thu Jan 29 10:04:00 2009
From: strawman at astraw.com (Andrew Straw)
Date: Thu, 29 Jan 2009 01:04:00 -0800
Subject: [Distutils] [ANN] stdeb 0.2.2 released
Message-ID: <49817100.7080508@astraw.com>

stdeb ("setuptools debian") produces Debian source packages from Python
packages via a new distutils command, sdist_dsc. Automatic defaults are
provided for the Debian package, but many aspects of the resulting
package can be customized via a configuration file.

This new release, 0.2.2, adds optional automatic dependency finding from
Zooko O'Whielacronx. Additionally, I have moved development to github.

The homepage: http://github.com/astraw/stdeb
Download: http://pypi.python.org/pypi/stdeb/0.2.2

Thanks and enjoy,
Andrew

From interstar at gmail.com  Thu Jan 29 13:17:16 2009
From: interstar at gmail.com (phil jones)
Date: Thu, 29 Jan 2009 12:17:16 +0000
Subject: [Distutils] GUI for Python package Management
In-Reply-To: <4db42df7554839e3151045936e5b0ee4@preisshare.net>
References: <4db42df7554839e3151045936e5b0ee4@preisshare.net>
Message-ID: <dc2a299f0901290417v4256e8bfh5b86df5134abd68b@mail.gmail.com>

I'd be delighted to see someone working on this. However, in my ideal
world, this would be something that integrated with IDLE. I know tk is
not a patch on wx, but it is an included battery.


On Wed, Jan 28, 2009 at 10:11 PM,  <david.lyon at preisshare.net> wrote:
> Hi All,
>
> I am a developer from other languages and find myself on python now.
>
> I'm wondering if anybody has objections to doing a package management gui
> for python.
>
> My design suggestions would be to:
>
>  - use WXWidgets (cross compatability for most pythonic platforms)
>  - provide a GUI wrapper for EasyInstall
>  - source packages from http://pypi.python.org/pypi
>
> It seems we have most of the componentry in place. The problem seems to be
> that it doesn't come together in a very user friendly way at the moment
> that is cohesive.
>
> Can anybody/everybody flame/tell me why we shouldn't work to simplify this
> area of python ?
>
>
>
>
>
> _______________________________________________
> Distutils-SIG maillist  -  Distutils-SIG at python.org
> http://mail.python.org/mailman/listinfo/distutils-sig
>

From arve.knudsen at gmail.com  Thu Jan 29 14:36:51 2009
From: arve.knudsen at gmail.com (Arve Knudsen)
Date: Thu, 29 Jan 2009 14:36:51 +0100
Subject: [Distutils] Zip-safe way of detecting plugins?
Message-ID: <a0d6258d0901290536w13b56a84gf96c1b7656969c61@mail.gmail.com>

What's the safe way of detecting plugins in a package that is potentially
zipped?

Thanks,
Arve
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20090129/e3cb69d4/attachment.htm>

From joss at debian.org  Thu Jan 29 15:10:19 2009
From: joss at debian.org (Josselin Mouette)
Date: Thu, 29 Jan 2009 15:10:19 +0100
Subject: [Distutils] [Python Language Summit] Distutils
	/	Packaging	survey
In-Reply-To: <C34F2C72-B619-4A46-B4FA-F4FDB41A5EB2@zooko.com>
References: <94bdd2610901250306n1e27a928sa0b276d91f80862a@mail.gmail.com>
	<90bb445a0901270829y7019eca3p72164c259c4d35d4@mail.gmail.com>
	<94bdd2610901271400k32ff74edmea94fcbdfce88f53@mail.gmail.com>
	<515B9271-0E7D-4B4E-B63F-F717604A5FEA@zooko.com>
	<49803752.9040704@ar.media.kyoto-u.ac.jp>
	<C34F2C72-B619-4A46-B4FA-F4FDB41A5EB2@zooko.com>
Message-ID: <1233238219.8068.25.camel@tomoyo>

Le mercredi 28 janvier 2009 ? 07:44 -0700, zooko a ?crit :
> 3.  It would be okay for this process to be automated (or semi- 
> automated), but there's some flaw in the design of stdeb which means  
> it will never be able to do it right unless stdeb is rewritten with a  
> new design.

This is the one. BTW, I don?t consider this a flaw in stdeb, it?s just
that stdeb was not designed with the goal to produce packages suitable
for Debian itself.

What we need is the equivalent of dh_make_perl [0]. That is, a script
that will generate the debian/ structure in a semi-automated fashion,
leading to a package ready to be installed after minor tweaks and a
human?s review. Bonus points would go for providing a script suggesting
changes in the description and/or dependencies when updating the package
for a new upstream release.

As I have already explained [1], I should be the one to write it since I
know all the tools and how to make clean packages, but I don?t have the
time these days. If anyone starts to work on such a project, I?d be of
course willing to help.

 [0] http://packages.debian.org/sid/dh-make-perl
 [1] http://np237.livejournal.com/21033.html
-- 
 .''`.
: :' :      We are debian.org. Lower your prices, surrender your code.
`. `'       We will add your hardware and software distinctiveness to
  `-        our own. Resistance is futile.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: Ceci est une partie de message num?riquement sign?e
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20090129/5b2ed123/attachment.pgp>

From joss at debian.org  Thu Jan 29 14:53:04 2009
From: joss at debian.org (Josselin Mouette)
Date: Thu, 29 Jan 2009 14:53:04 +0100
Subject: [Distutils] [Python Language Summit] Distutils /
	Packaging	survey
In-Reply-To: <b654cd2e0901281116u57f8967dl785ffc09a0edf291@mail.gmail.com>
References: <94bdd2610901250306n1e27a928sa0b276d91f80862a@mail.gmail.com>
	<90bb445a0901270829y7019eca3p72164c259c4d35d4@mail.gmail.com>
	<94bdd2610901271400k32ff74edmea94fcbdfce88f53@mail.gmail.com>
	<515B9271-0E7D-4B4E-B63F-F717604A5FEA@zooko.com>
	<49803752.9040704@ar.media.kyoto-u.ac.jp>
	<94bdd2610901280307h42b05abcr5599d64189948201@mail.gmail.com>
	<49803BDF.4060204@ar.media.kyoto-u.ac.jp>
	<b654cd2e0901281116u57f8967dl785ffc09a0edf291@mail.gmail.com>
Message-ID: <1233237184.8068.9.camel@tomoyo>

Le mercredi 28 janvier 2009 ? 13:16 -0600, Ian Bicking a ?crit :
> As you mention, there would have to be some extension to pkg_resources
> (or an equivalent library) to handle finding these files at runtime.
> Getting a runtime in place is probably the harder thing, as it is more
> intrusive for the upstream developers.

As I have already explained in the previous discussion, this could
easily be solved just like autoconf does, with an automatically
generated config.py file that would hold all variables set at build
time.

Cheers,
-- 
 .''`.
: :' :      We are debian.org. Lower your prices, surrender your code.
`. `'       We will add your hardware and software distinctiveness to
  `-        our own. Resistance is futile.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: Ceci est une partie de message num?riquement sign?e
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20090129/54992105/attachment.pgp>

From joss at debian.org  Thu Jan 29 14:57:59 2009
From: joss at debian.org (Josselin Mouette)
Date: Thu, 29 Jan 2009 14:57:59 +0100
Subject: [Distutils] [Python Language Summit] Distutils
	/	Packaging	survey
In-Reply-To: <B0B60DA3-D275-4DF9-B88D-7903A0DB31F2@python.org>
References: <94bdd2610901250306n1e27a928sa0b276d91f80862a@mail.gmail.com>
	<90bb445a0901270829y7019eca3p72164c259c4d35d4@mail.gmail.com>
	<94bdd2610901271400k32ff74edmea94fcbdfce88f53@mail.gmail.com>
	<515B9271-0E7D-4B4E-B63F-F717604A5FEA@zooko.com>
	<49803752.9040704@ar.media.kyoto-u.ac.jp>
	<B0B60DA3-D275-4DF9-B88D-7903A0DB31F2@python.org>
Message-ID: <1233237479.8068.14.camel@tomoyo>

Le mercredi 28 janvier 2009 ? 08:07 -0500, Barry Warsaw a ?crit :
> I'd like to make a radical suggestion: upstream authors should never  
> have to worry about building distribution blobs.

This is just silly. You don?t have to worry about the distribution?s
internals, and what is specific to each package format, but the very
idea of developing without any kind of knowledge about how the software
will integrate on a system is a guaranteed recipe for a development
disaster.

-- 
 .''`.
: :' :      We are debian.org. Lower your prices, surrender your code.
`. `'       We will add your hardware and software distinctiveness to
  `-        our own. Resistance is futile.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: Ceci est une partie de message num?riquement sign?e
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20090129/92d8f2b5/attachment.pgp>

From joss at debian.org  Thu Jan 29 14:56:17 2009
From: joss at debian.org (Josselin Mouette)
Date: Thu, 29 Jan 2009 14:56:17 +0100
Subject: [Distutils] [Python Language Summit] Distutils
	/	Packaging	survey
In-Reply-To: <222cb5ff86e3aa612a59db8389f4d087@preisshare.net>
References: <94bdd2610901250306n1e27a928sa0b276d91f80862a@mail.gmail.com>
	<90bb445a0901270829y7019eca3p72164c259c4d35d4@mail.gmail.com>
	<94bdd2610901271400k32ff74edmea94fcbdfce88f53@mail.gmail.com>
	<515B9271-0E7D-4B4E-B63F-F717604A5FEA@zooko.com>
	<49803752.9040704@ar.media.kyoto-u.ac.jp>
	<B0B60DA3-D275-4DF9-B88D-7903A0DB31F2@python.org>
	<49810AAA.3050300@ar.media.kyoto-u.ac.jp>
	<222cb5ff86e3aa612a59db8389f4d087@preisshare.net>
Message-ID: <1233237377.8068.11.camel@tomoyo>

Le mercredi 28 janvier 2009 ? 21:18 -0500, david.lyon at preisshare.net a
?crit :
> Sometimes it is easier under linux to get an operating system python
> package (from debian for example) installed than it is to get a pure python
> package...
> 
> That seems very wrong to me.....
> 
> So I am agreeing...
> 
> What can we do to fix this?

As the whole point of packaging is to ease the installation of software,
it is a pretty normal situation to find packages are easier to install.

If you know cases where they are not, that would definitely be something
to fix.

-- 
 .''`.
: :' :      We are debian.org. Lower your prices, surrender your code.
`. `'       We will add your hardware and software distinctiveness to
  `-        our own. Resistance is futile.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: Ceci est une partie de message num?riquement sign?e
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20090129/37d30883/attachment.pgp>

From barry at python.org  Thu Jan 29 15:35:18 2009
From: barry at python.org (Barry Warsaw)
Date: Thu, 29 Jan 2009 09:35:18 -0500
Subject: [Distutils] [Python Language Summit] Distutils
	/	Packaging	survey
In-Reply-To: <1233237479.8068.14.camel@tomoyo>
References: <94bdd2610901250306n1e27a928sa0b276d91f80862a@mail.gmail.com>
	<90bb445a0901270829y7019eca3p72164c259c4d35d4@mail.gmail.com>
	<94bdd2610901271400k32ff74edmea94fcbdfce88f53@mail.gmail.com>
	<515B9271-0E7D-4B4E-B63F-F717604A5FEA@zooko.com>
	<49803752.9040704@ar.media.kyoto-u.ac.jp>
	<B0B60DA3-D275-4DF9-B88D-7903A0DB31F2@python.org>
	<1233237479.8068.14.camel@tomoyo>
Message-ID: <EE06B43C-20AB-46F5-82F0-53F375AF2D74@python.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Jan 29, 2009, at 8:57 AM, Josselin Mouette wrote:

> Le mercredi 28 janvier 2009 ? 08:07 -0500, Barry Warsaw a ?crit :
>> I'd like to make a radical suggestion: upstream authors should never
>> have to worry about building distribution blobs.
>
> This is just silly. You don?t have to worry about the distribution?s
> internals, and what is specific to each package format, but the very
> idea of developing without any kind of knowledge about how the  
> software
> will integrate on a system is a guaranteed recipe for a development
> disaster.

That's not what I'm saying.  What I'm really saying is that I don't  
want to have to run 5 different setup.py commands every time I do a  
release in order to upload all the possible distribution formats that  
my users may want.

Barry

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (Darwin)

iQCVAwUBSYG+pnEjvBPtnXfVAQIc2wP/Q3Fks9V6KX7LtUBEw2S4FVcjIOmb2iII
H3fnb8D5hft7BiGbRQ+jLakzVSff9cV9F1TIsFLepXI+buBAupJ7XmiKIyVcfDRT
Y6KIQQfO150Tlw5K/L9XdnJjPuEmXji+f0LcY8DjI2SqYZkQZBxV7+XX35CdqbYK
dQgCgAgtaRU=
=zSqr
-----END PGP SIGNATURE-----

From regebro at gmail.com  Thu Jan 29 15:51:03 2009
From: regebro at gmail.com (Lennart Regebro)
Date: Thu, 29 Jan 2009 15:51:03 +0100
Subject: [Distutils] [Python Language Summit] Distutils / Packaging
	survey
In-Reply-To: <EE06B43C-20AB-46F5-82F0-53F375AF2D74@python.org>
References: <94bdd2610901250306n1e27a928sa0b276d91f80862a@mail.gmail.com>
	<90bb445a0901270829y7019eca3p72164c259c4d35d4@mail.gmail.com>
	<94bdd2610901271400k32ff74edmea94fcbdfce88f53@mail.gmail.com>
	<515B9271-0E7D-4B4E-B63F-F717604A5FEA@zooko.com>
	<49803752.9040704@ar.media.kyoto-u.ac.jp>
	<B0B60DA3-D275-4DF9-B88D-7903A0DB31F2@python.org>
	<1233237479.8068.14.camel@tomoyo>
	<EE06B43C-20AB-46F5-82F0-53F375AF2D74@python.org>
Message-ID: <319e029f0901290651h3645abffwd0648eba5879e59a@mail.gmail.com>

On Thu, Jan 29, 2009 at 15:35, Barry Warsaw <barry at python.org> wrote:
> That's not what I'm saying.  What I'm really saying is that I don't want to
> have to run 5 different setup.py commands every time I do a release in order
> to upload all the possible distribution formats that my users may want.

Well, you are gonna have to, because you also need to test that they
work anyway. We can't distribute autogenerated distribution packages
that nobody ever tested that they actually work.

-- 
Lennart Regebro: Zope and Plone consulting.
http://www.colliberty.com/
+33 661 58 14 64

From eric at trueblade.com  Thu Jan 29 15:55:29 2009
From: eric at trueblade.com (Eric Smith)
Date: Thu, 29 Jan 2009 09:55:29 -0500
Subject: [Distutils] [Python Language Summit] Distutils /
	Packaging	survey
In-Reply-To: <319e029f0901290651h3645abffwd0648eba5879e59a@mail.gmail.com>
References: <94bdd2610901250306n1e27a928sa0b276d91f80862a@mail.gmail.com>	<90bb445a0901270829y7019eca3p72164c259c4d35d4@mail.gmail.com>	<94bdd2610901271400k32ff74edmea94fcbdfce88f53@mail.gmail.com>	<515B9271-0E7D-4B4E-B63F-F717604A5FEA@zooko.com>	<49803752.9040704@ar.media.kyoto-u.ac.jp>	<B0B60DA3-D275-4DF9-B88D-7903A0DB31F2@python.org>	<1233237479.8068.14.camel@tomoyo>	<EE06B43C-20AB-46F5-82F0-53F375AF2D74@python.org>
	<319e029f0901290651h3645abffwd0648eba5879e59a@mail.gmail.com>
Message-ID: <4981C361.7020101@trueblade.com>

Lennart Regebro wrote:
> On Thu, Jan 29, 2009 at 15:35, Barry Warsaw <barry at python.org> wrote:
>> That's not what I'm saying.  What I'm really saying is that I don't want to
>> have to run 5 different setup.py commands every time I do a release in order
>> to upload all the possible distribution formats that my users may want.
> 
> Well, you are gonna have to, because you also need to test that they
> work anyway. We can't distribute autogenerated distribution packages
> that nobody ever tested that they actually work.
> 

Someone should, true. But it need not be the original author, who might 
not have the ability to test (or even produce) every desirable 
distribution format.

From exarkun at divmod.com  Thu Jan 29 16:03:08 2009
From: exarkun at divmod.com (Jean-Paul Calderone)
Date: Thu, 29 Jan 2009 10:03:08 -0500
Subject: [Distutils] [Python Language Summit] Distutils
 /	Packaging	survey
In-Reply-To: <EE06B43C-20AB-46F5-82F0-53F375AF2D74@python.org>
Message-ID: <20090129150308.24460.259439371.divmod.quotient.3763@henry.divmod.com>

On Thu, 29 Jan 2009 09:35:18 -0500, Barry Warsaw <barry at python.org> wrote:
>-----BEGIN PGP SIGNED MESSAGE-----
>Hash: SHA1
>
>On Jan 29, 2009, at 8:57 AM, Josselin Mouette wrote:
>>Le mercredi 28 janvier 2009 ? 08:07 -0500, Barry Warsaw a ?crit :
>>>I'd like to make a radical suggestion: upstream authors should never
>>>have to worry about building distribution blobs.
>>
>>This is just silly. You don?t have to worry about the distribution?s
>>internals, and what is specific to each package format, but the very
>>idea of developing without any kind of knowledge about how the  software
>>will integrate on a system is a guaranteed recipe for a development
>>disaster.
>
>That's not what I'm saying.  What I'm really saying is that I don't  want to 
>have to run 5 different setup.py commands every time I do a  release in 
>order to upload all the possible distribution formats that  my users may 
>want.

Barry, just have your buildbot do it for you - and not just when you do a
release, but every time you commit.  As a bonus, it can then install the
packages and run your test suite and tell you if you've got installation
problems (or any other bugs) on any of the platforms. :)

Jean-Paul

From regebro at gmail.com  Thu Jan 29 16:11:22 2009
From: regebro at gmail.com (Lennart Regebro)
Date: Thu, 29 Jan 2009 16:11:22 +0100
Subject: [Distutils] [Python Language Summit] Distutils / Packaging
	survey
In-Reply-To: <4981C361.7020101@trueblade.com>
References: <94bdd2610901250306n1e27a928sa0b276d91f80862a@mail.gmail.com>
	<90bb445a0901270829y7019eca3p72164c259c4d35d4@mail.gmail.com>
	<94bdd2610901271400k32ff74edmea94fcbdfce88f53@mail.gmail.com>
	<515B9271-0E7D-4B4E-B63F-F717604A5FEA@zooko.com>
	<49803752.9040704@ar.media.kyoto-u.ac.jp>
	<B0B60DA3-D275-4DF9-B88D-7903A0DB31F2@python.org>
	<1233237479.8068.14.camel@tomoyo>
	<EE06B43C-20AB-46F5-82F0-53F375AF2D74@python.org>
	<319e029f0901290651h3645abffwd0648eba5879e59a@mail.gmail.com>
	<4981C361.7020101@trueblade.com>
Message-ID: <319e029f0901290711x643a28d2u75f669cd3bbed84b@mail.gmail.com>

On Thu, Jan 29, 2009 at 15:55, Eric Smith <eric at trueblade.com> wrote:
> Someone should, true. But it need not be the original author, who might not
> have the ability to test (or even produce) every desirable distribution
> format.

Absolutely. But this is already the case, you don't have to make and
upload any distribution format except source code. Others can do that
for you.

-- 
Lennart Regebro: Zope and Plone consulting.
http://www.colliberty.com/
+33 661 58 14 64

From barry at python.org  Thu Jan 29 16:24:08 2009
From: barry at python.org (Barry Warsaw)
Date: Thu, 29 Jan 2009 10:24:08 -0500
Subject: [Distutils] [Python Language Summit] Distutils
	/	Packaging	survey
In-Reply-To: <20090129150308.24460.259439371.divmod.quotient.3763@henry.divmod.com>
References: <20090129150308.24460.259439371.divmod.quotient.3763@henry.divmod.com>
Message-ID: <FADBAC15-2AA6-4899-B9BC-00AD6B869949@python.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Jan 29, 2009, at 10:03 AM, Jean-Paul Calderone wrote:

> Barry, just have your buildbot do it for you - and not just when you  
> do a
> release, but every time you commit.  As a bonus, it can then install  
> the
> packages and run your test suite and tell you if you've got  
> installation
> problems (or any other bugs) on any of the platforms. :)

I feel like I need to be snake bitten. :)

Barry

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (Darwin)

iQCVAwUBSYHKGHEjvBPtnXfVAQJB+gQAllkJ7g+OzVu6PpXLsyuuHNyOkEhSmcb9
7+aHtoDuW1Brj6u+aiUrZOt8K5tRkTWTmkyOqJMrGa1/JnssU25qsGb8SODUbfO3
xZ9jcO9P6VKx/kRDBLUt7kNebR5aYUS7Z5MhjdM/at66CWhcVHaTlMuIkHyvZwuG
8/hP301QAE0=
=Dx6o
-----END PGP SIGNATURE-----

From barry at python.org  Thu Jan 29 16:49:06 2009
From: barry at python.org (Barry Warsaw)
Date: Thu, 29 Jan 2009 10:49:06 -0500
Subject: [Distutils] [Python Language Summit] Distutils / Packaging
	survey
In-Reply-To: <319e029f0901290651h3645abffwd0648eba5879e59a@mail.gmail.com>
References: <94bdd2610901250306n1e27a928sa0b276d91f80862a@mail.gmail.com>
	<90bb445a0901270829y7019eca3p72164c259c4d35d4@mail.gmail.com>
	<94bdd2610901271400k32ff74edmea94fcbdfce88f53@mail.gmail.com>
	<515B9271-0E7D-4B4E-B63F-F717604A5FEA@zooko.com>
	<49803752.9040704@ar.media.kyoto-u.ac.jp>
	<B0B60DA3-D275-4DF9-B88D-7903A0DB31F2@python.org>
	<1233237479.8068.14.camel@tomoyo>
	<EE06B43C-20AB-46F5-82F0-53F375AF2D74@python.org>
	<319e029f0901290651h3645abffwd0648eba5879e59a@mail.gmail.com>
Message-ID: <D50E1651-7066-42F1-8145-24FC0C7EB5D1@python.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Jan 29, 2009, at 9:51 AM, Lennart Regebro wrote:

> On Thu, Jan 29, 2009 at 15:35, Barry Warsaw <barry at python.org> wrote:
>> That's not what I'm saying.  What I'm really saying is that I don't  
>> want to
>> have to run 5 different setup.py commands every time I do a release  
>> in order
>> to upload all the possible distribution formats that my users may  
>> want.
>
> Well, you are gonna have to, because you also need to test that they
> work anyway. We can't distribute autogenerated distribution packages
> that nobody ever tested that they actually work.

That's what users, continuous integration, and and bug reports are for!

Barry

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (Darwin)

iQCVAwUBSYHP8nEjvBPtnXfVAQJKbwP9G6f2QjKvf7lv0frBuNTJPzryOG6Riu/8
g3Kh/NZ1fkrNerS8Qvzcn7Az9pbwWGkIYjgBTXOjgd0WHlyTsEUG7vMqnTNB3jM4
m3UUH+WbndZKaaQHim9g1xikKKk+2b90atrQ6Ss5mVOYRLHuH6ftILetsfMz1IUL
NG5HUGtRz2Y=
=DrTe
-----END PGP SIGNATURE-----

From r.ritz at biologie.hu-berlin.de  Thu Jan 29 16:56:23 2009
From: r.ritz at biologie.hu-berlin.de (Raphael Ritz)
Date: Thu, 29 Jan 2009 16:56:23 +0100
Subject: [Distutils] [Python Language Summit] Distutils / Packaging
	survey
In-Reply-To: <FADBAC15-2AA6-4899-B9BC-00AD6B869949@python.org>
References: <20090129150308.24460.259439371.divmod.quotient.3763@henry.divmod.com>
	<FADBAC15-2AA6-4899-B9BC-00AD6B869949@python.org>
Message-ID: <glsjj6$32r$2@ger.gmane.org>

Barry Warsaw wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
> 
> On Jan 29, 2009, at 10:03 AM, Jean-Paul Calderone wrote:
> 
>> Barry, just have your buildbot do it for you - and not just when you do a
>> release, but every time you commit.  As a bonus, it can then install the
>> packages and run your test suite and tell you if you've got installation
>> problems (or any other bugs) on any of the platforms. :)
> 
> I feel like I need to be snake bitten. :)

http://www.snakebite.org/

and

http://mail.python.org/pipermail/python-committers/2009-January/000331.html

for those who didn't get it ;-)

Seriously, I think that's the way forward.

Raphael


> 
> Barry
> 
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v1.4.9 (Darwin)
> 
> iQCVAwUBSYHKGHEjvBPtnXfVAQJB+gQAllkJ7g+OzVu6PpXLsyuuHNyOkEhSmcb9
> 7+aHtoDuW1Brj6u+aiUrZOt8K5tRkTWTmkyOqJMrGa1/JnssU25qsGb8SODUbfO3
> xZ9jcO9P6VKx/kRDBLUt7kNebR5aYUS7Z5MhjdM/at66CWhcVHaTlMuIkHyvZwuG
> 8/hP301QAE0=
> =Dx6o
> -----END PGP SIGNATURE-----
> _______________________________________________
> Distutils-SIG maillist  -  Distutils-SIG at python.org
> http://mail.python.org/mailman/listinfo/distutils-sig
> 


From tseaver at palladion.com  Thu Jan 29 18:17:13 2009
From: tseaver at palladion.com (Tres Seaver)
Date: Thu, 29 Jan 2009 12:17:13 -0500
Subject: [Distutils] [Python Language Summit] Distutils / Packaging
	survey
In-Reply-To: <EE06B43C-20AB-46F5-82F0-53F375AF2D74@python.org>
References: <94bdd2610901250306n1e27a928sa0b276d91f80862a@mail.gmail.com>	<90bb445a0901270829y7019eca3p72164c259c4d35d4@mail.gmail.com>	<94bdd2610901271400k32ff74edmea94fcbdfce88f53@mail.gmail.com>	<515B9271-0E7D-4B4E-B63F-F717604A5FEA@zooko.com>	<49803752.9040704@ar.media.kyoto-u.ac.jp>	<B0B60DA3-D275-4DF9-B88D-7903A0DB31F2@python.org>	<1233237479.8068.14.camel@tomoyo>
	<EE06B43C-20AB-46F5-82F0-53F375AF2D74@python.org>
Message-ID: <4981E499.20402@palladion.com>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Barry Warsaw wrote:
> On Jan 29, 2009, at 8:57 AM, Josselin Mouette wrote:
> 
>> Le mercredi 28 janvier 2009 ? 08:07 -0500, Barry Warsaw a ?crit :
>>> I'd like to make a radical suggestion: upstream authors should never
>>> have to worry about building distribution blobs.
>> This is just silly. You dont have to worry about the distributions
>> internals, and what is specific to each package format, but the very
>> idea of developing without any kind of knowledge about how the  
>> software
>> will integrate on a system is a guaranteed recipe for a development
>> disaster.
> 
> That's not what I'm saying.  What I'm really saying is that I don't  
> want to have to run 5 different setup.py commands every time I do a  
> release in order to upload all the possible distribution formats that  
> my users may want.

I would argue that the upstream developer should (almost) never be
uploading anything but a metadata-rich sdist:  except for packages with
C extensions, nobody really needs anything else for "library" packages,
and it's really only the Windows folks who can't build those binaries
for themselves.

People distributing applications might want to provide installers, I
guess, for the command-line challenged.

Otherwise, if we provided enough metadata in the sdist, packagers can
build the other formats (.deb, .rpm, etc.) for us, assuming that we can
solve the "resource file" problem.


Tres.
- --
===================================================================
Tres Seaver          +1 540-429-0999          tseaver at palladion.com
Palladion Software   "Excellence by Design"    http://palladion.com
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFJgeSZ+gerLs4ltQ4RAhmkAJ0Z1fdFA3v6ccVNnur2gR16or9diQCgmA9C
u4em+Ec62cNvvV7UCD/2+Dw=
=X/O5
-----END PGP SIGNATURE-----


From barry at python.org  Thu Jan 29 18:20:22 2009
From: barry at python.org (Barry Warsaw)
Date: Thu, 29 Jan 2009 12:20:22 -0500
Subject: [Distutils] [Python Language Summit] Distutils / Packaging
	survey
In-Reply-To: <4981E499.20402@palladion.com>
References: <94bdd2610901250306n1e27a928sa0b276d91f80862a@mail.gmail.com>	<90bb445a0901270829y7019eca3p72164c259c4d35d4@mail.gmail.com>	<94bdd2610901271400k32ff74edmea94fcbdfce88f53@mail.gmail.com>	<515B9271-0E7D-4B4E-B63F-F717604A5FEA@zooko.com>	<49803752.9040704@ar.media.kyoto-u.ac.jp>	<B0B60DA3-D275-4DF9-B88D-7903A0DB31F2@python.org>	<1233237479.8068.14.camel@tomoyo>
	<EE06B43C-20AB-46F5-82F0-53F375AF2D74@python.org>
	<4981E499.20402@palladion.com>
Message-ID: <502030B7-2B41-437F-9FED-DFECCEBA208D@python.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Jan 29, 2009, at 12:17 PM, Tres Seaver wrote:

> Barry Warsaw wrote:
>> On Jan 29, 2009, at 8:57 AM, Josselin Mouette wrote:
>>
>>> Le mercredi 28 janvier 2009 ? 08:07 -0500, Barry Warsaw a ?crit :
>>>> I'd like to make a radical suggestion: upstream authors should  
>>>> never
>>>> have to worry about building distribution blobs.
>>> This is just silly. You dont have to worry about the distributions
>>> internals, and what is specific to each package format, but the very
>>> idea of developing without any kind of knowledge about how the
>>> software
>>> will integrate on a system is a guaranteed recipe for a development
>>> disaster.
>>
>> That's not what I'm saying.  What I'm really saying is that I don't
>> want to have to run 5 different setup.py commands every time I do a
>> release in order to upload all the possible distribution formats that
>> my users may want.
>
> I would argue that the upstream developer should (almost) never be
> uploading anything but a metadata-rich sdist:  except for packages  
> with
> C extensions, nobody really needs anything else for "library"  
> packages,
> and it's really only the Windows folks who can't build those binaries
> for themselves.
>
> People distributing applications might want to provide installers, I
> guess, for the command-line challenged.
>
> Otherwise, if we provided enough metadata in the sdist, packagers can
> build the other formats (.deb, .rpm, etc.) for us, assuming that we  
> can
> solve the "resource file" problem.

+1-enthusiastical-ly y'rs,
Barry

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (Darwin)

iQCVAwUBSYHlVnEjvBPtnXfVAQL18QP/TtIRsN2f93yAs1hNBl/Rb31q1Dpe6NnT
R6vIZU9yGw0XyFaEEb9a/f8TC/EUKUDoaq2/g+RnCoZCABWBo3tiFrbmUxZs5C9n
1sbP1dFYSVGkOThtPz6gBa5hqYMwiA/l/UuCJAXVJdeRdVmiWMMJDRBftHIbMITB
Mp53xK/NEmQ=
=zeh5
-----END PGP SIGNATURE-----

From regebro at gmail.com  Thu Jan 29 18:21:14 2009
From: regebro at gmail.com (Lennart Regebro)
Date: Thu, 29 Jan 2009 18:21:14 +0100
Subject: [Distutils] [Python Language Summit] Distutils / Packaging
	survey
In-Reply-To: <D50E1651-7066-42F1-8145-24FC0C7EB5D1@python.org>
References: <94bdd2610901250306n1e27a928sa0b276d91f80862a@mail.gmail.com>
	<90bb445a0901270829y7019eca3p72164c259c4d35d4@mail.gmail.com>
	<94bdd2610901271400k32ff74edmea94fcbdfce88f53@mail.gmail.com>
	<515B9271-0E7D-4B4E-B63F-F717604A5FEA@zooko.com>
	<49803752.9040704@ar.media.kyoto-u.ac.jp>
	<B0B60DA3-D275-4DF9-B88D-7903A0DB31F2@python.org>
	<1233237479.8068.14.camel@tomoyo>
	<EE06B43C-20AB-46F5-82F0-53F375AF2D74@python.org>
	<319e029f0901290651h3645abffwd0648eba5879e59a@mail.gmail.com>
	<D50E1651-7066-42F1-8145-24FC0C7EB5D1@python.org>
Message-ID: <319e029f0901290921u1f554f97p95fd1deb4c3aa10@mail.gmail.com>

On Thu, Jan 29, 2009 at 16:49, Barry Warsaw <barry at python.org> wrote:
> That's what users, continuous integration, and and bug reports are for!

Yes, and if you have that, then you also have automatically generated
packages, which you can upload to pypi.

-- 
Lennart Regebro: Zope and Plone consulting.
http://www.colliberty.com/
+33 661 58 14 64

From ianb at colorstudy.com  Thu Jan 29 18:51:05 2009
From: ianb at colorstudy.com (Ian Bicking)
Date: Thu, 29 Jan 2009 11:51:05 -0600
Subject: [Distutils] [Python Language Summit] Distutils / Packaging
	survey
In-Reply-To: <1233237184.8068.9.camel@tomoyo>
References: <94bdd2610901250306n1e27a928sa0b276d91f80862a@mail.gmail.com>
	<90bb445a0901270829y7019eca3p72164c259c4d35d4@mail.gmail.com>
	<94bdd2610901271400k32ff74edmea94fcbdfce88f53@mail.gmail.com>
	<515B9271-0E7D-4B4E-B63F-F717604A5FEA@zooko.com>
	<49803752.9040704@ar.media.kyoto-u.ac.jp>
	<94bdd2610901280307h42b05abcr5599d64189948201@mail.gmail.com>
	<49803BDF.4060204@ar.media.kyoto-u.ac.jp>
	<b654cd2e0901281116u57f8967dl785ffc09a0edf291@mail.gmail.com>
	<1233237184.8068.9.camel@tomoyo>
Message-ID: <b654cd2e0901290951n4c11e29cw637fc37642addd9@mail.gmail.com>

On Thu, Jan 29, 2009 at 7:53 AM, Josselin Mouette <joss at debian.org> wrote:

> Le mercredi 28 janvier 2009 ? 13:16 -0600, Ian Bicking a ?crit :
> > As you mention, there would have to be some extension to pkg_resources
> > (or an equivalent library) to handle finding these files at runtime.
> > Getting a runtime in place is probably the harder thing, as it is more
> > intrusive for the upstream developers.
>
> As I have already explained in the previous discussion, this could
> easily be solved just like autoconf does, with an automatically
> generated config.py file that would hold all variables set at build
> time.


"Easily solved" is a misstatement -- you can lead a horse to water, but you
can't make him drink.  In this case, you can provide infrastructure for
library authors, but you can't make them use it.  The solution has to be
sufficiently simple that people who don't care at all about FHS or Linux
packages won't find it onerous to use, because if they do find it onerous
then they won't use it.

But anyway, where would this config.py go and what would it look like?


-- 
Ian Bicking  |  http://blog.ianbicking.org
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20090129/d536894f/attachment.htm>

From floris.bruynooghe at gmail.com  Thu Jan 29 20:07:57 2009
From: floris.bruynooghe at gmail.com (Floris Bruynooghe)
Date: Thu, 29 Jan 2009 19:07:57 +0000
Subject: [Distutils] [Python Language Summit] Distutils /
	Packaging	survey
In-Reply-To: <4981C361.7020101@trueblade.com>
References: <94bdd2610901250306n1e27a928sa0b276d91f80862a@mail.gmail.com>
	<90bb445a0901270829y7019eca3p72164c259c4d35d4@mail.gmail.com>
	<94bdd2610901271400k32ff74edmea94fcbdfce88f53@mail.gmail.com>
	<515B9271-0E7D-4B4E-B63F-F717604A5FEA@zooko.com>
	<49803752.9040704@ar.media.kyoto-u.ac.jp>
	<B0B60DA3-D275-4DF9-B88D-7903A0DB31F2@python.org>
	<1233237479.8068.14.camel@tomoyo>
	<EE06B43C-20AB-46F5-82F0-53F375AF2D74@python.org>
	<319e029f0901290651h3645abffwd0648eba5879e59a@mail.gmail.com>
	<4981C361.7020101@trueblade.com>
Message-ID: <20090129190757.GA8189@laurie.devork>

On Thu, Jan 29, 2009 at 09:55:29AM -0500, Eric Smith wrote:
> Lennart Regebro wrote:
>> On Thu, Jan 29, 2009 at 15:35, Barry Warsaw <barry at python.org> wrote:
>>> That's not what I'm saying.  What I'm really saying is that I don't want to
>>> have to run 5 different setup.py commands every time I do a release in order
>>> to upload all the possible distribution formats that my users may want.
>>
>> Well, you are gonna have to, because you also need to test that they
>> work anyway. We can't distribute autogenerated distribution packages
>> that nobody ever tested that they actually work.
>>
>
> Someone should, true. But it need not be the original author, who might  
> not have the ability to test (or even produce) every desirable  
> distribution format.

In my ideal world python package developers would only ever have to
worry about uploading an sdist.  It's up to whoever wants to consume
these to re-pack them in the appropriate format if that's what they
want.

If I am correct the only reason people started to go away from this is
because Windows users don't tend to have compilers installed.


Regards
Floris

-- 
Debian GNU/Linux -- The Power of Freedom
www.debian.org | www.gnu.org | www.kernel.org

From floris.bruynooghe at gmail.com  Thu Jan 29 20:10:13 2009
From: floris.bruynooghe at gmail.com (Floris Bruynooghe)
Date: Thu, 29 Jan 2009 19:10:13 +0000
Subject: [Distutils] [Python Language Summit] Distutils /
	Packaging	survey
In-Reply-To: <502030B7-2B41-437F-9FED-DFECCEBA208D@python.org>
References: <94bdd2610901250306n1e27a928sa0b276d91f80862a@mail.gmail.com>
	<90bb445a0901270829y7019eca3p72164c259c4d35d4@mail.gmail.com>
	<94bdd2610901271400k32ff74edmea94fcbdfce88f53@mail.gmail.com>
	<515B9271-0E7D-4B4E-B63F-F717604A5FEA@zooko.com>
	<49803752.9040704@ar.media.kyoto-u.ac.jp>
	<B0B60DA3-D275-4DF9-B88D-7903A0DB31F2@python.org>
	<1233237479.8068.14.camel@tomoyo>
	<EE06B43C-20AB-46F5-82F0-53F375AF2D74@python.org>
	<4981E499.20402@palladion.com>
	<502030B7-2B41-437F-9FED-DFECCEBA208D@python.org>
Message-ID: <20090129191013.GB8189@laurie.devork>

On Thu, Jan 29, 2009 at 12:20:22PM -0500, Barry Warsaw wrote:
> On Jan 29, 2009, at 12:17 PM, Tres Seaver wrote:
>
>> Barry Warsaw wrote:
>>> On Jan 29, 2009, at 8:57 AM, Josselin Mouette wrote:
>>>
>>>> Le mercredi 28 janvier 2009 ? 08:07 -0500, Barry Warsaw a ?crit :
>>>>> I'd like to make a radical suggestion: upstream authors should  
>>>>> never
>>>>> have to worry about building distribution blobs.
>>>> This is just silly. You dont have to worry about the distributions
>>>> internals, and what is specific to each package format, but the very
>>>> idea of developing without any kind of knowledge about how the
>>>> software
>>>> will integrate on a system is a guaranteed recipe for a development
>>>> disaster.
>>>
>>> That's not what I'm saying.  What I'm really saying is that I don't
>>> want to have to run 5 different setup.py commands every time I do a
>>> release in order to upload all the possible distribution formats that
>>> my users may want.
>>
>> I would argue that the upstream developer should (almost) never be
>> uploading anything but a metadata-rich sdist:  except for packages  
>> with
>> C extensions, nobody really needs anything else for "library"  
>> packages,
>> and it's really only the Windows folks who can't build those binaries
>> for themselves.
>>
>> People distributing applications might want to provide installers, I
>> guess, for the command-line challenged.
>>
>> Otherwise, if we provided enough metadata in the sdist, packagers can
>> build the other formats (.deb, .rpm, etc.) for us, assuming that we  
>> can
>> solve the "resource file" problem.
>
> +1-enthusiastical-ly y'rs,

+1 too :-)

-- 
Debian GNU/Linux -- The Power of Freedom
www.debian.org | www.gnu.org | www.kernel.org

From sienkiew at stsci.edu  Thu Jan 29 22:01:06 2009
From: sienkiew at stsci.edu (Mark Sienkiewicz)
Date: Thu, 29 Jan 2009 16:01:06 -0500
Subject: [Distutils] GUI for Python package Management
In-Reply-To: <7edd5adaf7ff6650b908cfc76362a64a@preisshare.net>
References: <4db42df7554839e3151045936e5b0ee4@preisshare.net>	<7CCBA902-15CE-441A-BEE6-DD0972026530@gmail.com>
	<7edd5adaf7ff6650b908cfc76362a64a@preisshare.net>
Message-ID: <49821912.7050806@stsci.edu>

david.lyon at preisshare.net wrote:
>> I don't think that the missing of a gui is what is the problem today
>> for python. Package uninstall is something that bothers some (maybe a
>> lot) of users. 
>>     
>
> Yeah. Me too. Possibly made easier by having a GUI.
>   

The biggest problem is that the installer does not save the information 
necessary to perform the uninstall.  A secondary problem is accounting 
for dependencies between packages.

A GUI does not help either of these problems.  If you can implement an 
uninstall in the GUI, you can implement it from the command line.

>> Another point is to have something like webstart for
>> java. If I understand how it works, running an application goes like
>> this: If you open an app and it depends on another package not
>> installed on your system java goes around and download the missing
>> modules for you (and stores them in a relative place to the package or
>> the user package dir) so the application can run. This would be cool
>> for python, the problem being that there is no security to guarantee
>> that those modules are not malicious in any way...
>>     
>
> I agree with that strongly.
>   

I very much dislike things that automatically download and install 
software.  An automatic installer may find a different version of a 
supporting package every time I install software on another machine.

I keep careful track of what is installed on all my machines.  If the 
tool automatically installs any version other than the one I specified, 
then the tool is working _against_ me.  I don't need that.

Ideally, there would be a flag that says "if you can't find something, 
give me an error -- do not attempt to download/install anything".  But 
it would be helpful if it can tell me "Package xyzzy is missing, but you 
can get it from here:..."

Mark S.


From ben+python at benfinney.id.au  Thu Jan 29 22:35:55 2009
From: ben+python at benfinney.id.au (Ben Finney)
Date: Fri, 30 Jan 2009 08:35:55 +1100
Subject: [Distutils] [Python Language Summit] Distutils /
	Packaging	survey
References: <94bdd2610901250306n1e27a928sa0b276d91f80862a@mail.gmail.com>
	<90bb445a0901270829y7019eca3p72164c259c4d35d4@mail.gmail.com>
	<94bdd2610901271400k32ff74edmea94fcbdfce88f53@mail.gmail.com>
	<515B9271-0E7D-4B4E-B63F-F717604A5FEA@zooko.com>
	<49803752.9040704@ar.media.kyoto-u.ac.jp>
	<B0B60DA3-D275-4DF9-B88D-7903A0DB31F2@python.org>
	<1233237479.8068.14.camel@tomoyo>
	<EE06B43C-20AB-46F5-82F0-53F375AF2D74@python.org>
	<319e029f0901290651h3645abffwd0648eba5879e59a@mail.gmail.com>
	<4981C361.7020101@trueblade.com>
	<20090129190757.GA8189@laurie.devork>
Message-ID: <87bptpvlqc.fsf@benfinney.id.au>

Floris Bruynooghe <floris.bruynooghe at gmail.com> writes:

> In my ideal world python package developers would only ever have to
> worry about uploading an sdist.

+1

> If I am correct the only reason people started to go away from this
> is because Windows users don't tend to have compilers installed.

I think that may, indeed, be correct.

-- 
 \         ?Apologize, v. To lay the foundation for a future offense.? |
  `\                   ?Ambrose Bierce, _The Devil's Dictionary_, 1906 |
_o__)                                                                  |
Ben Finney


From santagada at gmail.com  Thu Jan 29 22:38:21 2009
From: santagada at gmail.com (Leonardo Santagada)
Date: Thu, 29 Jan 2009 19:38:21 -0200
Subject: [Distutils] GUI for Python package Management
In-Reply-To: <49821912.7050806@stsci.edu>
References: <4db42df7554839e3151045936e5b0ee4@preisshare.net>	<7CCBA902-15CE-441A-BEE6-DD0972026530@gmail.com>
	<7edd5adaf7ff6650b908cfc76362a64a@preisshare.net>
	<49821912.7050806@stsci.edu>
Message-ID: <80297AE9-6A91-4A0D-96C3-E9FD7317BA09@gmail.com>


On Jan 29, 2009, at 7:01 PM, Mark Sienkiewicz wrote:

> david.lyon at preisshare.net wrote:
>>> I don't think that the missing of a gui is what is the problem today
>>> for python. Package uninstall is something that bothers some  
>>> (maybe a
>>> lot) of users.
>>
>> Yeah. Me too. Possibly made easier by having a GUI.
>>
>
> The biggest problem is that the installer does not save the  
> information necessary to perform the uninstall.  A secondary problem  
> is accounting for dependencies between packages.
>
> A GUI does not help either of these problems.  If you can implement  
> an uninstall in the GUI, you can implement it from the command line.
>
>>> Another point is to have something like webstart for
>>> java. If I understand how it works, running an application goes like
>>> this: If you open an app and it depends on another package not
>>> installed on your system java goes around and download the missing
>>> modules for you (and stores them in a relative place to the  
>>> package or
>>> the user package dir) so the application can run. This would be cool
>>> for python, the problem being that there is no security to guarantee
>>> that those modules are not malicious in any way...
>>>
>>
>> I agree with that strongly.
>>
>
> I very much dislike things that automatically download and install  
> software.  An automatic installer may find a different version of a  
> supporting package every time I install software on another machine.

if the application asks for the different version then yes it should  
download the version that was asked for and installed for that  
application. It will only find a different version each time if the  
application asks for it.

> I keep careful track of what is installed on all my machines.  If  
> the tool automatically installs any version other than the one I  
> specified, then the tool is working _against_ me.  I don't need that.

No it is working like the application writer specified it.

> Ideally, there would be a flag that says "if you can't find  
> something, give me an error -- do not attempt to download/install  
> anything".  But it would be helpful if it can tell me "Package xyzzy  
> is missing, but you can get it from here:..."

This I agree, it should have a way to ignore some requests or even all  
requests of the application that is launched, maybe even have a  
configuration file somewhere (or a registry key, or a plist file  
depending on the os) that override the default, which I think should  
be give the running application the version that it specifies  
(standard packaging version rules apply, if it asks for package >=1.0  
then any version newer than 1.0 is sufficient)

But this is just a proposition that I think will never be able to work  
with python without security or only signed and pre approved packages  
on pypi.

> Mark S.
>

--
Leonardo Santagada
santagada at gmail.com




From ben+python at benfinney.id.au  Thu Jan 29 22:51:41 2009
From: ben+python at benfinney.id.au (Ben Finney)
Date: Fri, 30 Jan 2009 08:51:41 +1100
Subject: [Distutils] [Python Language Summit] Distutils /
	Packaging	survey
References: <94bdd2610901250306n1e27a928sa0b276d91f80862a@mail.gmail.com>
	<90bb445a0901270829y7019eca3p72164c259c4d35d4@mail.gmail.com>
	<94bdd2610901271400k32ff74edmea94fcbdfce88f53@mail.gmail.com>
	<515B9271-0E7D-4B4E-B63F-F717604A5FEA@zooko.com>
	<49803752.9040704@ar.media.kyoto-u.ac.jp>
	<94bdd2610901280307h42b05abcr5599d64189948201@mail.gmail.com>
	<49803BDF.4060204@ar.media.kyoto-u.ac.jp>
	<b654cd2e0901281116u57f8967dl785ffc09a0edf291@mail.gmail.com>
	<1233237184.8068.9.camel@tomoyo>
Message-ID: <877i4dvl02.fsf@benfinney.id.au>

Josselin Mouette <joss at debian.org> writes:

> Le mercredi 28 janvier 2009 ? 13:16 -0600, Ian Bicking a ?crit :
> > As you mention, there would have to be some extension to
> > pkg_resources (or an equivalent library) to handle finding these
> > files at runtime. Getting a runtime in place is probably the
> > harder thing, as it is more intrusive for the upstream developers.
> 
> As I have already explained in the previous discussion, this could
> easily be solved just like autoconf does, with an automatically
> generated config.py file that would hold all variables set at build
> time.

I would recommend choosing (earlier rather than later) a different
name for that file. ?config.py? suggests rather the run-time
configuration for the program.

Perhaps ?setup_config.py?? Something that makes it clear that the
configuration is intended for the setup and installation, *not* the
running Python package.

-- 
 \       ?My classmates would copulate with anything that moved, but I |
  `\               never saw any reason to limit myself.? ?Emo Philips |
_o__)                                                                  |
Ben Finney


From ziade.tarek at gmail.com  Thu Jan 29 23:19:17 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Thu, 29 Jan 2009 23:19:17 +0100
Subject: [Distutils] [Python Language Summit] Distutils / Packaging
	survey
In-Reply-To: <877i4dvl02.fsf@benfinney.id.au>
References: <94bdd2610901250306n1e27a928sa0b276d91f80862a@mail.gmail.com>
	<90bb445a0901270829y7019eca3p72164c259c4d35d4@mail.gmail.com>
	<94bdd2610901271400k32ff74edmea94fcbdfce88f53@mail.gmail.com>
	<515B9271-0E7D-4B4E-B63F-F717604A5FEA@zooko.com>
	<49803752.9040704@ar.media.kyoto-u.ac.jp>
	<94bdd2610901280307h42b05abcr5599d64189948201@mail.gmail.com>
	<49803BDF.4060204@ar.media.kyoto-u.ac.jp>
	<b654cd2e0901281116u57f8967dl785ffc09a0edf291@mail.gmail.com>
	<1233237184.8068.9.camel@tomoyo> <877i4dvl02.fsf@benfinney.id.au>
Message-ID: <94bdd2610901291419g3d7d348cpfd2756d039ee160c@mail.gmail.com>

On Thu, Jan 29, 2009 at 10:51 PM, Ben Finney <ben+python at benfinney.id.au> wrote:
> Josselin Mouette <joss at debian.org> writes:
>
>> Le mercredi 28 janvier 2009 ? 13:16 -0600, Ian Bicking a ?crit :
>> > As you mention, there would have to be some extension to
>> > pkg_resources (or an equivalent library) to handle finding these
>> > files at runtime. Getting a runtime in place is probably the
>> > harder thing, as it is more intrusive for the upstream developers.
>>
>> As I have already explained in the previous discussion, this could
>> easily be solved just like autoconf does, with an automatically
>> generated config.py file that would hold all variables set at build
>> time.
>
> I would recommend choosing (earlier rather than later) a different
> name for that file. 'config.py' suggests rather the run-time
> configuration for the program.
>
> Perhaps 'setup_config.py'? Something that makes it clear that the
> configuration is intended for the setup and installation, *not* the
> running Python package.

what about "metadata.py"  (that can be used by setup.py as well)

++
Tarek

From sienkiew at stsci.edu  Thu Jan 29 23:29:28 2009
From: sienkiew at stsci.edu (Mark Sienkiewicz)
Date: Thu, 29 Jan 2009 17:29:28 -0500
Subject: [Distutils] GUI for Python package Management
In-Reply-To: <80297AE9-6A91-4A0D-96C3-E9FD7317BA09@gmail.com>
References: <4db42df7554839e3151045936e5b0ee4@preisshare.net>	<7CCBA902-15CE-441A-BEE6-DD0972026530@gmail.com>
	<7edd5adaf7ff6650b908cfc76362a64a@preisshare.net>
	<49821912.7050806@stsci.edu>
	<80297AE9-6A91-4A0D-96C3-E9FD7317BA09@gmail.com>
Message-ID: <49822DC8.7090901@stsci.edu>


>>
>> I very much dislike things that automatically download and install 
>> software.  An automatic installer may find a different version of a 
>> supporting package every time I install software on another machine.
>
> if the application asks for the different version then yes it should 
> download the version that was asked for and installed for that 
> application. It will only find a different version each time if the 
> application asks for it.

Here is an example of the scenario I am trying to avoid:

Suppose the package foobar asks for "xyzzy > 2.3".

On machine Fred, I install foobar on Tuesday.  I do not even know that 
foobar needs xyzzy, so unless I watch the install closely, Fred may have 
xyzzy 2.4 installed. 

On machine Barney, I install foobar on Wednesday.  I do not know there 
was a new release of xyzzy overnight, but Barney now has xyzzy 2.5 
installed.

Six months from now, my user says "YOUR program is broken - it doesn't 
do the same thing on Fred and Barney".

I figure out that it is the version of xyzzy, then decide that the 
correct answers come when I use 2.4.  It was automatically installed, so 
I don't have a copy of it in my archive of source code.  I try to 
download it, but I can only find 2.5, 2.6, and 2.7 online.

What do I do?

Compare with:

On machine Fred, I install foobar on Tuesday, but I use --no-automatic.  
It says "You must have xyzzy > 2.3".

I download xyzzy 2.4 to my collection of source code, install it on 
Fred, then install foobar.  Everything works.

On Wednesday, I install, on Barney, xyzzy 2.4 from my collection of 
source code and my own package foobar.

Six months from now, my user sees that my program does the same thing on 
Fred and Barney.

>
>> I keep careful track of what is installed on all my machines.  If the 
>> tool automatically installs any version other than the one I 
>> specified, then the tool is working _against_ me.  I don't need that.
>
> No it is working like the application writer specified it.

I agree, except for the part where you said "No". :)

I think the correct description is  "Yes!  It is working like the 
application writer specified it, but it is making proper configuration 
control difficult."

>> Ideally, there would be a flag that says "if you can't find 
>> something, give me an error -- do not attempt to download/install 
>> anything".  But it would be helpful if it can tell me "Package xyzzy 
>> is missing, but you can get it from here:..."
>
> This I agree, it should have a way to ignore some requests or even all 
> requests of the application that is launched, maybe even have a 
> configuration file somewhere (or a registry key, or a plist file 
> depending on the os) that override the default, which I think should 
> be give the running application the version that it specifies 
> (standard packaging version rules apply, if it asks for package >=1.0 
> then any version newer than 1.0 is sufficient)

Good point -- we need two options here:

1. do not download/install anything, just raise an error

2. use the version module XXX that I have, even though the package says 
it is not suitable


>
> But this is just a proposition that I think will never be able to work 
> with python without security or only signed and pre approved packages 
> on pypi.

Is it really different from what setuptools already does?

Do we know how CPAN handles security?

Mark S.


From santagada at gmail.com  Thu Jan 29 23:42:31 2009
From: santagada at gmail.com (Leonardo Santagada)
Date: Thu, 29 Jan 2009 20:42:31 -0200
Subject: [Distutils] GUI for Python package Management
In-Reply-To: <49822DC8.7090901@stsci.edu>
References: <4db42df7554839e3151045936e5b0ee4@preisshare.net>	<7CCBA902-15CE-441A-BEE6-DD0972026530@gmail.com>
	<7edd5adaf7ff6650b908cfc76362a64a@preisshare.net>
	<49821912.7050806@stsci.edu>
	<80297AE9-6A91-4A0D-96C3-E9FD7317BA09@gmail.com>
	<49822DC8.7090901@stsci.edu>
Message-ID: <B2256D8B-425C-4642-98F3-AD5505D5ED39@gmail.com>


On Jan 29, 2009, at 8:29 PM, Mark Sienkiewicz wrote:

>
>>>
>>> I very much dislike things that automatically download and install  
>>> software.  An automatic installer may find a different version of  
>>> a supporting package every time I install software on another  
>>> machine.
>>
>> if the application asks for the different version then yes it  
>> should download the version that was asked for and installed for  
>> that application. It will only find a different version each time  
>> if the application asks for it.
>
> Here is an example of the scenario I am trying to avoid:
>
> Suppose the package foobar asks for "xyzzy > 2.3".
>
> On machine Fred, I install foobar on Tuesday.  I do not even know  
> that foobar needs xyzzy, so unless I watch the install closely, Fred  
> may have xyzzy 2.4 installed.
> On machine Barney, I install foobar on Wednesday.  I do not know  
> there was a new release of xyzzy overnight, but Barney now has xyzzy  
> 2.5 installed.
>
> Six months from now, my user says "YOUR program is broken - it  
> doesn't do the same thing on Fred and Barney".
>
> I figure out that it is the version of xyzzy, then decide that the  
> correct answers come when I use 2.4.  It was automatically  
> installed, so I don't have a copy of it in my archive of source  
> code.  I try to download it, but I can only find 2.5, 2.6, and 2.7  
> online.
>
> What do I do?
>
> Compare with:
>
> On machine Fred, I install foobar on Tuesday, but I use --no- 
> automatic.  It says "You must have xyzzy > 2.3".
>
> I download xyzzy 2.4 to my collection of source code, install it on  
> Fred, then install foobar.  Everything works.
>
> On Wednesday, I install, on Barney, xyzzy 2.4 from my collection of  
> source code and my own package foobar.
>
> Six months from now, my user sees that my program does the same  
> thing on Fred and Barney.

This is just one of the example of why packages should not use ">=" in  
vain :)

But yep, it gets complex but I think there is ways around this,  
specially if this behavior became accepted.

>>> I keep careful track of what is installed on all my machines.  If  
>>> the tool automatically installs any version other than the one I  
>>> specified, then the tool is working _against_ me.  I don't need  
>>> that.
>>
>> No it is working like the application writer specified it.
>
> I agree, except for the part where you said "No". :)
>
> I think the correct description is  "Yes!  It is working like the  
> application writer specified it, but it is making proper  
> configuration control difficult."

Who does configuration control? Not end users right? So I think this  
autoinstall method should be used the same way that webstart is used,  
for end users only. It will not take the place of buildout or manual  
installation. This is a feature to make python programs easier to run  
for the end user. So we can distribute python packages without having  
to use py2exe or anything like that.


>
>>> Ideally, there would be a flag that says "if you can't find  
>>> something, give me an error -- do not attempt to download/install  
>>> anything".  But it would be helpful if it can tell me "Package  
>>> xyzzy is missing, but you can get it from here:..."
>>
>> This I agree, it should have a way to ignore some requests or even  
>> all requests of the application that is launched, maybe even have a  
>> configuration file somewhere (or a registry key, or a plist file  
>> depending on the os) that override the default, which I think  
>> should be give the running application the version that it  
>> specifies (standard packaging version rules apply, if it asks for  
>> package >=1.0 then any version newer than 1.0 is sufficient)
>
> Good point -- we need two options here:
>
> 1. do not download/install anything, just raise an error
>
> 2. use the version module XXX that I have, even though the package  
> says it is not suitable

>> But this is just a proposition that I think will never be able to  
>> work with python without security or only signed and pre approved  
>> packages on pypi.
>
> Is it really different from what setuptools already does?
>
> Do we know how CPAN handles security?


No it is not. the difference is that you have to today easy_install an  
application for it to download its dependencies, it is like saying "I  
believe in this package dev team and so I believe they are not bring  
harmfull modules to my system". But well... as python doesn't have a  
sandbox everytime you run an application you are trusting the  
developer... so if this system build a virtual env for each app it  
runs all will be ok (as a package that the program brings will not  
affect the system configuration).


--
Leonardo Santagada
santagada at gmail.com




From tseaver at palladion.com  Thu Jan 29 23:50:08 2009
From: tseaver at palladion.com (Tres Seaver)
Date: Thu, 29 Jan 2009 17:50:08 -0500
Subject: [Distutils] [Python Language Summit] Distutils / Packaging
	survey
In-Reply-To: <877i4dvl02.fsf@benfinney.id.au>
References: <94bdd2610901250306n1e27a928sa0b276d91f80862a@mail.gmail.com>	<90bb445a0901270829y7019eca3p72164c259c4d35d4@mail.gmail.com>	<94bdd2610901271400k32ff74edmea94fcbdfce88f53@mail.gmail.com>	<515B9271-0E7D-4B4E-B63F-F717604A5FEA@zooko.com>	<49803752.9040704@ar.media.kyoto-u.ac.jp>	<94bdd2610901280307h42b05abcr5599d64189948201@mail.gmail.com>	<49803BDF.4060204@ar.media.kyoto-u.ac.jp>	<b654cd2e0901281116u57f8967dl785ffc09a0edf291@mail.gmail.com>	<1233237184.8068.9.camel@tomoyo>
	<877i4dvl02.fsf@benfinney.id.au>
Message-ID: <498232A0.8060306@palladion.com>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Ben Finney wrote:
> Josselin Mouette <joss at debian.org> writes:
> 
>> Le mercredi 28 janvier 2009 ? 13:16 -0600, Ian Bicking a ?crit :
>>> As you mention, there would have to be some extension to
>>> pkg_resources (or an equivalent library) to handle finding these
>>> files at runtime. Getting a runtime in place is probably the
>>> harder thing, as it is more intrusive for the upstream developers.
>> As I have already explained in the previous discussion, this could
>> easily be solved just like autoconf does, with an automatically
>> generated config.py file that would hold all variables set at build
>> time.
> 
> I would recommend choosing (earlier rather than later) a different
> name for that file. ?config.py? suggests rather the run-time
> configuration for the program.
> 
> Perhaps ?setup_config.py?? Something that makes it clear that the
> configuration is intended for the setup and installation, *not* the
> running Python package.

I would argue against making it a .py file at all: make it an INI file,
or something.


Tres.
- --
===================================================================
Tres Seaver          +1 540-429-0999          tseaver at palladion.com
Palladion Software   "Excellence by Design"    http://palladion.com
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFJgjKg+gerLs4ltQ4RAicLAKC0/IJnCA7dQqXrE58mQpLHNyJE9wCfQ3LO
nW98d8KkYqszyQNUAGOnsik=
=JHM5
-----END PGP SIGNATURE-----

From tseaver at palladion.com  Thu Jan 29 23:50:08 2009
From: tseaver at palladion.com (Tres Seaver)
Date: Thu, 29 Jan 2009 17:50:08 -0500
Subject: [Distutils] [Python Language Summit] Distutils / Packaging
	survey
In-Reply-To: <877i4dvl02.fsf@benfinney.id.au>
References: <94bdd2610901250306n1e27a928sa0b276d91f80862a@mail.gmail.com>	<90bb445a0901270829y7019eca3p72164c259c4d35d4@mail.gmail.com>	<94bdd2610901271400k32ff74edmea94fcbdfce88f53@mail.gmail.com>	<515B9271-0E7D-4B4E-B63F-F717604A5FEA@zooko.com>	<49803752.9040704@ar.media.kyoto-u.ac.jp>	<94bdd2610901280307h42b05abcr5599d64189948201@mail.gmail.com>	<49803BDF.4060204@ar.media.kyoto-u.ac.jp>	<b654cd2e0901281116u57f8967dl785ffc09a0edf291@mail.gmail.com>	<1233237184.8068.9.camel@tomoyo>
	<877i4dvl02.fsf@benfinney.id.au>
Message-ID: <498232A0.8060306@palladion.com>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Ben Finney wrote:
> Josselin Mouette <joss at debian.org> writes:
> 
>> Le mercredi 28 janvier 2009 ? 13:16 -0600, Ian Bicking a ?crit :
>>> As you mention, there would have to be some extension to
>>> pkg_resources (or an equivalent library) to handle finding these
>>> files at runtime. Getting a runtime in place is probably the
>>> harder thing, as it is more intrusive for the upstream developers.
>> As I have already explained in the previous discussion, this could
>> easily be solved just like autoconf does, with an automatically
>> generated config.py file that would hold all variables set at build
>> time.
> 
> I would recommend choosing (earlier rather than later) a different
> name for that file. ?config.py? suggests rather the run-time
> configuration for the program.
> 
> Perhaps ?setup_config.py?? Something that makes it clear that the
> configuration is intended for the setup and installation, *not* the
> running Python package.

I would argue against making it a .py file at all: make it an INI file,
or something.


Tres.
- --
===================================================================
Tres Seaver          +1 540-429-0999          tseaver at palladion.com
Palladion Software   "Excellence by Design"    http://palladion.com
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFJgjKg+gerLs4ltQ4RAicLAKC0/IJnCA7dQqXrE58mQpLHNyJE9wCfQ3LO
nW98d8KkYqszyQNUAGOnsik=
=JHM5
-----END PGP SIGNATURE-----


From ziade.tarek at gmail.com  Thu Jan 29 23:54:38 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Thu, 29 Jan 2009 23:54:38 +0100
Subject: [Distutils] [Python Language Summit] Distutils / Packaging
	survey
In-Reply-To: <498232A0.8060306@palladion.com>
References: <94bdd2610901250306n1e27a928sa0b276d91f80862a@mail.gmail.com>
	<94bdd2610901271400k32ff74edmea94fcbdfce88f53@mail.gmail.com>
	<515B9271-0E7D-4B4E-B63F-F717604A5FEA@zooko.com>
	<49803752.9040704@ar.media.kyoto-u.ac.jp>
	<94bdd2610901280307h42b05abcr5599d64189948201@mail.gmail.com>
	<49803BDF.4060204@ar.media.kyoto-u.ac.jp>
	<b654cd2e0901281116u57f8967dl785ffc09a0edf291@mail.gmail.com>
	<1233237184.8068.9.camel@tomoyo> <877i4dvl02.fsf@benfinney.id.au>
	<498232A0.8060306@palladion.com>
Message-ID: <94bdd2610901291454s1eee0d0fvfbcb8d21d871e9ee@mail.gmail.com>

On Thu, Jan 29, 2009 at 11:50 PM, Tres Seaver <tseaver at palladion.com> wrote:
>> Perhaps 'setup_config.py'? Something that makes it clear that the
>> configuration is intended for the setup and installation, *not* the
>> running Python package.
>
> I would argue against making it a .py file at all: make it an INI file,
> or something.

setup.cfg could be used in that case.

But having a .py file would allow having some values calculated dynamically,

++
Tarek

-- 
Tarek Ziad? | Association AfPy | www.afpy.org
Blog FR | http://programmation-python.org
Blog EN | http://tarekziade.wordpress.com/

From ben+python at benfinney.id.au  Thu Jan 29 23:55:50 2009
From: ben+python at benfinney.id.au (Ben Finney)
Date: Fri, 30 Jan 2009 09:55:50 +1100
Subject: [Distutils] [Python Language Summit] Distutils / Packaging
	survey
References: <94bdd2610901250306n1e27a928sa0b276d91f80862a@mail.gmail.com>
	<90bb445a0901270829y7019eca3p72164c259c4d35d4@mail.gmail.com>
	<94bdd2610901271400k32ff74edmea94fcbdfce88f53@mail.gmail.com>
	<515B9271-0E7D-4B4E-B63F-F717604A5FEA@zooko.com>
	<49803752.9040704@ar.media.kyoto-u.ac.jp>
	<94bdd2610901280307h42b05abcr5599d64189948201@mail.gmail.com>
	<49803BDF.4060204@ar.media.kyoto-u.ac.jp>
	<b654cd2e0901281116u57f8967dl785ffc09a0edf291@mail.gmail.com>
	<1233237184.8068.9.camel@tomoyo> <877i4dvl02.fsf@benfinney.id.au>
	<94bdd2610901291419g3d7d348cpfd2756d039ee160c@mail.gmail.com>
Message-ID: <87y6wtu3gp.fsf@benfinney.id.au>

Tarek Ziad? <ziade.tarek at gmail.com> writes:

> On Thu, Jan 29, 2009 at 10:51 PM, Ben Finney <ben+python at benfinney.id.au> wrote:
> > Perhaps 'setup_config.py'? Something that makes it clear that the
> > configuration is intended for the setup and installation, *not*
> > the running Python package.
> 
> what about "metadata.py"  (that can be used by setup.py as well)

No complaints from me, but I'll leave it to more level heads than mine
to consider it.

-- 
 \       ?The surest way to corrupt a youth is to instruct him to hold |
  `\       in higher esteem those who think alike than those who think |
_o__)                               differently.? ?Friedrich Nietzsche |
Ben Finney


From floris.bruynooghe at gmail.com  Fri Jan 30 00:34:28 2009
From: floris.bruynooghe at gmail.com (Floris Bruynooghe)
Date: Thu, 29 Jan 2009 23:34:28 +0000
Subject: [Distutils] [Python Language Summit] Distutils /
	Packaging	survey
In-Reply-To: <94bdd2610901291454s1eee0d0fvfbcb8d21d871e9ee@mail.gmail.com>
References: <94bdd2610901271400k32ff74edmea94fcbdfce88f53@mail.gmail.com>
	<515B9271-0E7D-4B4E-B63F-F717604A5FEA@zooko.com>
	<49803752.9040704@ar.media.kyoto-u.ac.jp>
	<94bdd2610901280307h42b05abcr5599d64189948201@mail.gmail.com>
	<49803BDF.4060204@ar.media.kyoto-u.ac.jp>
	<b654cd2e0901281116u57f8967dl785ffc09a0edf291@mail.gmail.com>
	<1233237184.8068.9.camel@tomoyo> <877i4dvl02.fsf@benfinney.id.au>
	<498232A0.8060306@palladion.com>
	<94bdd2610901291454s1eee0d0fvfbcb8d21d871e9ee@mail.gmail.com>
Message-ID: <20090129233428.GA10050@laurie.devork>

On Thu, Jan 29, 2009 at 11:54:38PM +0100, Tarek Ziad? wrote:
> On Thu, Jan 29, 2009 at 11:50 PM, Tres Seaver <tseaver at palladion.com> wrote:
> >> Perhaps 'setup_config.py'? Something that makes it clear that the
> >> configuration is intended for the setup and installation, *not* the
> >> running Python package.
> >
> > I would argue against making it a .py file at all: make it an INI file,
> > or something.
> 
> setup.cfg could be used in that case.

setup.cfg is a configuration file to be edited by developers.  Going
to modify that at build time is guaranteed to result in unhappy people
at some stage I reckon.

> But having a .py file would allow having some values calculated
> dynamically,

But those could be calculated at the time setup.py runs too.

If it's not directly a .py file there needs to be an stdlib module to
access it easily, manually using ConfigParser is not an option IMHO,
I'd rather do `import foo; foo.prefix'.

An argument against a generated .py file is that this won't work for
single-module python distributions.  But having a .cfg file or
something next to the module/package might defeat the point of trying
to help the FHS in violating it already.  OTOH .egg-info does that too
and seems to be accepted currently.


Regards
Floris

-- 
Debian GNU/Linux -- The Power of Freedom
www.debian.org | www.gnu.org | www.kernel.org

From ziade.tarek at gmail.com  Fri Jan 30 01:21:12 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Fri, 30 Jan 2009 01:21:12 +0100
Subject: [Distutils] [Python Language Summit] Distutils / Packaging
	survey
In-Reply-To: <20090129233428.GA10050@laurie.devork>
References: <94bdd2610901271400k32ff74edmea94fcbdfce88f53@mail.gmail.com>
	<49803752.9040704@ar.media.kyoto-u.ac.jp>
	<94bdd2610901280307h42b05abcr5599d64189948201@mail.gmail.com>
	<49803BDF.4060204@ar.media.kyoto-u.ac.jp>
	<b654cd2e0901281116u57f8967dl785ffc09a0edf291@mail.gmail.com>
	<1233237184.8068.9.camel@tomoyo> <877i4dvl02.fsf@benfinney.id.au>
	<498232A0.8060306@palladion.com>
	<94bdd2610901291454s1eee0d0fvfbcb8d21d871e9ee@mail.gmail.com>
	<20090129233428.GA10050@laurie.devork>
Message-ID: <94bdd2610901291621m10056e9bvf702474e866265cf@mail.gmail.com>

On Fri, Jan 30, 2009 at 12:34 AM, Floris Bruynooghe
<floris.bruynooghe at gmail.com> wrote:
>> setup.cfg could be used in that case.
>
> setup.cfg is a configuration file to be edited by developers.  Going
> to modify that at build time is guaranteed to result in unhappy people
> at some stage I reckon.
>

right

>> But having a .py file would allow having some values calculated
>> dynamically,
>
> But those could be calculated at the time setup.py runs too.
>
> If it's not directly a .py file there needs to be an stdlib module to
> access it easily, manually using ConfigParser is not an option IMHO,
> I'd rather do `import foo; foo.prefix'.
>
> An argument against a generated .py file is that this won't work for
> single-module python distributions.  But having a .cfg file or
> something next to the module/package might defeat the point of trying
> to help the FHS in violating it already.  OTOH .egg-info does that too
> and seems to be accepted currently.

Well, the egg-info file is the static version of setup.py metadata in some way,
I would find a new file redundant.

I am really curious though, to see what the file would contain..

from my current understanding, and having in mind to introduce such a
file in a way things
can gently evolve:

What if an egg-info-like file was present in the package from the very
beginning,
describing the package metadata, and used by setup.py, *and* at build time
by third party tools. (with setup.py knowing that it can be changed at run time)

egg.info file:
=========
Metadata-Version: 1.0
Name: foo
Version: 1.0
etc..
*new metadata modified at build time*
=========

setup.py file:
==========
from disutils.core import setup, get_metadata

static_metadata = get_metadata()

static_metadata['more_things'] = 'xxx'

setup(**static_metadata)
==========

Regards
Tarek
-- 
Tarek Ziad? | Association AfPy | www.afpy.org
Blog FR | http://programmation-python.org
Blog EN | http://tarekziade.wordpress.com/

From david.lyon at preisshare.net  Fri Jan 30 01:30:06 2009
From: david.lyon at preisshare.net (david.lyon at preisshare.net)
Date: Thu, 29 Jan 2009 19:30:06 -0500
Subject: [Distutils] GUI for Python package Management
In-Reply-To: <dc2a299f0901290417v4256e8bfh5b86df5134abd68b@mail.gmail.com>
References: <4db42df7554839e3151045936e5b0ee4@preisshare.net>
	<dc2a299f0901290417v4256e8bfh5b86df5134abd68b@mail.gmail.com>
Message-ID: <8dba8567e72c0c0d687b16507e25d557@preisshare.net>


On Thu, 29 Jan 2009 12:17:16 +0000, phil jones <interstar at gmail.com> wrote:
> I'd be delighted to see someone working on this. However, in my ideal
> world, this would be something that integrated with IDLE. I know tk is
> not a patch on wx, but it is an included battery.

Thanks Phil.

My skills don't extend to tk although I can assure you that I have tried
:-) wx is easier for me.

What I'm hoping to do is a standalone application that could be fired off
just like an os package manager or ppm in perl.

As good as IDLE is, not everybody uses it. I am thinking of doing something
that could sit next to idle in the python program group.

Take care

David


From david.lyon at preisshare.net  Fri Jan 30 01:41:37 2009
From: david.lyon at preisshare.net (david.lyon at preisshare.net)
Date: Thu, 29 Jan 2009 19:41:37 -0500
Subject: [Distutils] GUI for Python package Management
In-Reply-To: <49822DC8.7090901@stsci.edu>
References: <4db42df7554839e3151045936e5b0ee4@preisshare.net>	<7CCBA902-15CE-441A-BEE6-DD0972026530@gmail.com>
	<7edd5adaf7ff6650b908cfc76362a64a@preisshare.net>
	<49821912.7050806@stsci.edu>
	<80297AE9-6A91-4A0D-96C3-E9FD7317BA09@gmail.com>
	<49822DC8.7090901@stsci.edu>
Message-ID: <dabf7bf09ede5ae956e60236de9d8c9f@preisshare.net>



On Thu, 29 Jan 2009 17:29:28 -0500, Mark Sienkiewicz <sienkiew at stsci.edu>
wrote:
> 
>>>
>>> I very much dislike things that automatically download and install
>>> software.  An automatic installer may find a different version of a
>>> supporting package every time I install software on another machine.
..
..
> Here is an example of the scenario I am trying to avoid:
> Suppose the package foobar asks for "xyzzy > 2.3".
> On machine Fred, I install foobar on Tuesday.  I do not even know that
> foobar needs xyzzy, so unless I watch the install closely, Fred may have
> xyzzy 2.4 installed.
> On machine Barney, I install foobar on Wednesday.  I do not know there
> was a new release of xyzzy overnight, but Barney now has xyzzy 2.5
> installed.
> Six months from now, my user says "YOUR program is broken - it doesn't
> do the same thing on Fred and Barney".
> ....

I so totally agree...

I work in an IT department with dozens of machines. I am always upgrading
and changing machines. It wastes so much time going and searching for
dependant packages, downloading and installing them.

Asking users on site to upgrade a dependent package seems to be one
of the worst things to ever do from my experience. They invariably get
it wrong - and blame the developer for their mistake. Maybe it is just me.

What I need for myself, is a package manifest, tracking all the packages
that I have installed. Then when I go set up on a new machine, I can load
the manifest, and the tool will go get me all my packages.

So I totally agree with your comments... that is why I am thinking
that a GUI could streamline this area.

David









From ziade.tarek at gmail.com  Fri Jan 30 01:49:58 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Fri, 30 Jan 2009 01:49:58 +0100
Subject: [Distutils] Uninstall command, the return
Message-ID: <94bdd2610901291649kbd31d86mbb5c552ba01f85d0@mail.gmail.com>

Hello

Many people are asking for an uninstall command. While there are
possible side-effects when removing all installed files,
I think it worths it...

I would like to introduce an uninstall command in distutils, using
Marc-Andr? Lemburg's mxSetup tool
(see http://www.egenix.com/products/python/mxBase/) and turning it
into a uninstall command.

It's quite straighforward since it uses the install command in dry-run
mode to get the files to remove.
It requires of course to keep the source.

Next, (in a second step) I was wondering if a uninstall registery
could not be a good thing to have,
to store a record of the installed files so there's no need to keep
the source for uninstallation.
This would required a new command, (and a detailed specification of course)

There's an open ticket here about that (#4673)

Any thoughts ?

Regards
Tarek

-- 
Tarek Ziad? | Association AfPy | www.afpy.org
Blog FR | http://programmation-python.org
Blog EN | http://tarekziade.wordpress.com/

From ben+python at benfinney.id.au  Fri Jan 30 03:04:30 2009
From: ben+python at benfinney.id.au (Ben Finney)
Date: Fri, 30 Jan 2009 13:04:30 +1100
Subject: [Distutils] GUI for Python package Management
References: <4db42df7554839e3151045936e5b0ee4@preisshare.net>
	<7CCBA902-15CE-441A-BEE6-DD0972026530@gmail.com>
	<7edd5adaf7ff6650b908cfc76362a64a@preisshare.net>
	<49821912.7050806@stsci.edu>
	<80297AE9-6A91-4A0D-96C3-E9FD7317BA09@gmail.com>
	<49822DC8.7090901@stsci.edu>
	<dabf7bf09ede5ae956e60236de9d8c9f@preisshare.net>
Message-ID: <87ljsttuq9.fsf@benfinney.id.au>

<david.lyon at preisshare.net> writes:

> What I need for myself, is a package manifest, tracking all the
> packages that I have installed. Then when I go set up on a new
> machine, I can load the manifest, and the tool will go get me all my
> packages.
> 
> So I totally agree with your comments... that is why I am thinking
> that a GUI could streamline this area.

An interface is premature if the manifest doesn't yet exist. Let's
solve that problem before thinking of a GUI for it.

-- 
 \                ?I got fired from my job the other day. They said my |
  `\          personality was weird. ? That's okay, I have four more.? |
_o__)                                       ?Bug-Eyed Earl, _Red Meat_ |
Ben Finney


From david.lyon at preisshare.net  Fri Jan 30 03:35:16 2009
From: david.lyon at preisshare.net (david.lyon at preisshare.net)
Date: Thu, 29 Jan 2009 21:35:16 -0500
Subject: [Distutils] GUI for Python package Management
In-Reply-To: <87ljsttuq9.fsf@benfinney.id.au>
References: <4db42df7554839e3151045936e5b0ee4@preisshare.net>
	<7CCBA902-15CE-441A-BEE6-DD0972026530@gmail.com>
	<7edd5adaf7ff6650b908cfc76362a64a@preisshare.net>
	<49821912.7050806@stsci.edu>
	<80297AE9-6A91-4A0D-96C3-E9FD7317BA09@gmail.com>
	<49822DC8.7090901@stsci.edu>
	<dabf7bf09ede5ae956e60236de9d8c9f@preisshare.net>
	<87ljsttuq9.fsf@benfinney.id.au>
Message-ID: <f6fdd83f5c5c1c6c81402f98666c8c6a@preisshare.net>

Hi Ben,

On Fri, 30 Jan 2009 13:04:30 +1100, Ben Finney <ben+python at benfinney.id.au>
wrote:

> An interface is premature if the manifest doesn't yet exist. Let's
> solve that problem before thinking of a GUI for it.

I can see your point there..

>From the other perspective, it is a fairly trivial task for the gui to
track what packages it has installed - through it's own configuration file
and generate a manifest from that.

That would save a lot of time waiting for the back-end guys to do all
the hard slog behind the scenes.

That is just how I see it anyway...

Best Regards

David


From cgalvan at mail.utexas.edu  Fri Jan 30 04:22:05 2009
From: cgalvan at mail.utexas.edu (Chris Galvan)
Date: Thu, 29 Jan 2009 21:22:05 -0600
Subject: [Distutils] Uninstall command, the return
In-Reply-To: <94bdd2610901291649kbd31d86mbb5c552ba01f85d0@mail.gmail.com>
References: <94bdd2610901291649kbd31d86mbb5c552ba01f85d0@mail.gmail.com>
Message-ID: <4982725D.9040704@mail.utexas.edu>

Enthought has a package called Enstaller which includes a modified 
version of setuptools + an enstaller package that adds some different 
functionality.  One of the modifications to setuptools adds an uninstall 
command which works well with our .egg's.  Dealing with eggs allows us 
to do some dependency checking to warn the user if the package they are 
trying to remove is required by another package.

Another feature we added is similar to the uninstall registry you 
mentioned.  Whenever someone invokes easy_install and a change is 
made(i.e. a package(s) was removed, added, or modified), we create an 
entry in a rollback cache file that stores the package-version of 
everything installed in site-packages at that time.  There is a rollback 
menu that displays the changes between the different known rollback 
points and you can restore your state to a previous working state in 
this way.

A lot of this stuff is specific to setuptools/eggs but the fundamentals 
of it could be helpful when determining how to implement these kinds of 
things into distutils itself, if that is the desired path.  Here is the 
trunk for Enstaller:

https://svn.enthought.com/svn/enthought/Enstaller/trunk/

-- Chris Galvan

Tarek Ziad? wrote:
> Hello
>
> Many people are asking for an uninstall command. While there are
> possible side-effects when removing all installed files,
> I think it worths it...
>
> I would like to introduce an uninstall command in distutils, using
> Marc-Andr? Lemburg's mxSetup tool
> (see http://www.egenix.com/products/python/mxBase/) and turning it
> into a uninstall command.
>
> It's quite straighforward since it uses the install command in dry-run
> mode to get the files to remove.
> It requires of course to keep the source.
>
> Next, (in a second step) I was wondering if a uninstall registery
> could not be a good thing to have,
> to store a record of the installed files so there's no need to keep
> the source for uninstallation.
> This would required a new command, (and a detailed specification of course)
>
> There's an open ticket here about that (#4673)
>
> Any thoughts ?
>
> Regards
> Tarek
>
>   


From ben+python at benfinney.id.au  Fri Jan 30 04:47:16 2009
From: ben+python at benfinney.id.au (Ben Finney)
Date: Fri, 30 Jan 2009 14:47:16 +1100
Subject: [Distutils] GUI for Python package Management
References: <4db42df7554839e3151045936e5b0ee4@preisshare.net>
	<7CCBA902-15CE-441A-BEE6-DD0972026530@gmail.com>
	<7edd5adaf7ff6650b908cfc76362a64a@preisshare.net>
	<49821912.7050806@stsci.edu>
	<80297AE9-6A91-4A0D-96C3-E9FD7317BA09@gmail.com>
	<49822DC8.7090901@stsci.edu>
	<dabf7bf09ede5ae956e60236de9d8c9f@preisshare.net>
	<87ljsttuq9.fsf@benfinney.id.au>
	<f6fdd83f5c5c1c6c81402f98666c8c6a@preisshare.net>
Message-ID: <87zlh9sbej.fsf@benfinney.id.au>

(Could you please set your name in your From field? Thanks.)

<david.lyon at preisshare.net> writes:

> >From the other perspective, it is a fairly trivial task for the gui to
> track what packages it has installed - through it's own configuration file
> and generate a manifest from that.

You mean, have packages tracked separately depending on which program
was used to install them? That seems like a recipe for madness.

-- 
 \        ?Hanging one scoundrel, it appears, does not deter the next. |
  `\         Well, what of it? The first one is at least disposed of.? |
_o__)                                                ?Henry L. Mencken |
Ben Finney


From ianb at colorstudy.com  Fri Jan 30 06:24:00 2009
From: ianb at colorstudy.com (Ian Bicking)
Date: Thu, 29 Jan 2009 23:24:00 -0600
Subject: [Distutils] Uninstall command, the return
In-Reply-To: <94bdd2610901291649kbd31d86mbb5c552ba01f85d0@mail.gmail.com>
References: <94bdd2610901291649kbd31d86mbb5c552ba01f85d0@mail.gmail.com>
Message-ID: <b654cd2e0901292124r6eb372cesc20083931cf319ef@mail.gmail.com>

On Thu, Jan 29, 2009 at 6:49 PM, Tarek Ziad? <ziade.tarek at gmail.com> wrote:

> Next, (in a second step) I was wondering if a uninstall registery
> could not be a good thing to have,
> to store a record of the installed files so there's no need to keep
> the source for uninstallation.
> This would required a new command, (and a detailed specification of course)
>

pip writes an installation record in Package.egg-info/installed-files.txt
(based on the setuptools --record option, with filenames made relative).
So... that's similar to it.  Of course, to be accurate you have to make sure
you don't install over those files.  So pip should really be uninstalling
before installing something new, and probably be fancy about the whole thing
(maybe like Enstaller is doing).

But if tools do respect the integrity of those files, it's a reasonably
simple record.  Well, that and they should be careful about one package
overwriting another packages file (which I haven't really seen happen, but
of course it *could* happen).


-- 
Ian Bicking  |  http://blog.ianbicking.org
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20090129/f5096599/attachment-0001.htm>

From theller at ctypes.org  Fri Jan 30 08:42:51 2009
From: theller at ctypes.org (Thomas Heller)
Date: Fri, 30 Jan 2009 08:42:51 +0100
Subject: [Distutils] Uninstall command, the return
In-Reply-To: <b654cd2e0901292124r6eb372cesc20083931cf319ef@mail.gmail.com>
References: <94bdd2610901291649kbd31d86mbb5c552ba01f85d0@mail.gmail.com>
	<b654cd2e0901292124r6eb372cesc20083931cf319ef@mail.gmail.com>
Message-ID: <glub1o$8gp$1@ger.gmane.org>

Ian Bicking schrieb:
> On Thu, Jan 29, 2009 at 6:49 PM, Tarek Ziad? <ziade.tarek at gmail.com> wrote:
> 
>> Next, (in a second step) I was wondering if a uninstall registery
>> could not be a good thing to have,
>> to store a record of the installed files so there's no need to keep
>> the source for uninstallation.
>> This would required a new command, (and a detailed specification of course)
>>
> 
> pip writes an installation record in Package.egg-info/installed-files.txt
> (based on the setuptools --record option, with filenames made relative).
> So... that's similar to it.  Of course, to be accurate you have to make sure
> you don't install over those files.  So pip should really be uninstalling
> before installing something new, and probably be fancy about the whole thing
> (maybe like Enstaller is doing).
> 
> But if tools do respect the integrity of those files, it's a reasonably
> simple record.  Well, that and they should be careful about one package
> overwriting another packages file (which I haven't really seen happen, but
> of course it *could* happen).

You could also take a look into the log-file that bdist_wininst installers
create (for deinstallation).  It contains information about directories and
registry entries created, files copied, and some more information about what
the installer has done.  Reinstalling a package simply appends to the log file.

-- 
Thanks,
Thomas


From ziade.tarek at gmail.com  Fri Jan 30 10:07:18 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Fri, 30 Jan 2009 10:07:18 +0100
Subject: [Distutils] Uninstall command, the return
In-Reply-To: <b654cd2e0901292124r6eb372cesc20083931cf319ef@mail.gmail.com>
References: <94bdd2610901291649kbd31d86mbb5c552ba01f85d0@mail.gmail.com>
	<b654cd2e0901292124r6eb372cesc20083931cf319ef@mail.gmail.com>
Message-ID: <94bdd2610901300107l6bd6476pd67f478cdaceb63c@mail.gmail.com>

Reading your answer and Chris's, I am now wondering, if we have a
global uninstall command,
why we wouldn't have a global install command....

How hard would it be, from your projects, to have the
install/uninstall feature, on the top
of today's distutils ? (since the record feature is in distutils)

In other words, is the current "record" feature of distutils would be
sufficient ?


On Fri, Jan 30, 2009 at 6:24 AM, Ian Bicking <ianb at colorstudy.com> wrote:
> On Thu, Jan 29, 2009 at 6:49 PM, Tarek Ziad? <ziade.tarek at gmail.com> wrote:
>>
>> Next, (in a second step) I was wondering if a uninstall registery
>> could not be a good thing to have,
>> to store a record of the installed files so there's no need to keep
>> the source for uninstallation.
>> This would required a new command, (and a detailed specification of
>> course)
>
> pip writes an installation record in Package.egg-info/installed-files.txt
> (based on the setuptools --record option, with filenames made relative).
> So... that's similar to it.  Of course, to be accurate you have to make sure
> you don't install over those files.  So pip should really be uninstalling
> before installing something new, and probably be fancy about the whole thing
> (maybe like Enstaller is doing).
>
> But if tools do respect the integrity of those files, it's a reasonably
> simple record.  Well, that and they should be careful about one package
> overwriting another packages file (which I haven't really seen happen, but
> of course it *could* happen).
>
> --
> Ian Bicking  |  http://blog.ianbicking.org
>



-- 
Tarek Ziad? | Association AfPy | www.afpy.org
Blog FR | http://programmation-python.org
Blog EN | http://tarekziade.wordpress.com/

From david at ar.media.kyoto-u.ac.jp  Fri Jan 30 10:28:56 2009
From: david at ar.media.kyoto-u.ac.jp (David Cournapeau)
Date: Fri, 30 Jan 2009 18:28:56 +0900
Subject: [Distutils] Uninstall command, the return
In-Reply-To: <94bdd2610901300107l6bd6476pd67f478cdaceb63c@mail.gmail.com>
References: <94bdd2610901291649kbd31d86mbb5c552ba01f85d0@mail.gmail.com>	<b654cd2e0901292124r6eb372cesc20083931cf319ef@mail.gmail.com>
	<94bdd2610901300107l6bd6476pd67f478cdaceb63c@mail.gmail.com>
Message-ID: <4982C858.1020202@ar.media.kyoto-u.ac.jp>

Tarek Ziad? wrote:
> Reading your answer and Chris's, I am now wondering, if we have a
> global uninstall command,
> why we wouldn't have a global install command....
>
> How hard would it be, from your projects, to have the
> install/uninstall feature, on the top
> of today's distutils ? (since the record feature is in distutils)
>
> In other words, is the current "record" feature of distutils would be
> sufficient ?
>   

I can see several things which have a big impact on an uninstall feature
design:
    - how to handle multiple versions of one package ?
    - should it handle dependencies ?
    - should it work for packages installed anywhere, or only for a few
reserved places ?

Only the last point is worth considering at the distutils level - the
two others are specific to setuptools (or more exactly any system on top
of distutils). Once those decisions are made, we can make decision on
how to deal with uninstall.

For example, if uninstalling only packages installed in some blessed
locations is acceptable, the problem of a global registry of some kind
becomes much easier to deal with. If it is not, I don't see much choice
but to put the packages-specific uninstall data in the package itself.
Some people do not like setuptools stubborness about installing things
in some places - I am one of them. But not being able to uninstall
automatically would not bother me too much (actually, I install things
manually so that I can use stow, which provides me an uninstall feature).

I also wonder whether we should add uninstall feature to distutils
itself from a UI POV. Of course, distutils should have some support of
some kind, but the high level stuff could be done by another program, no
? After all, uninstall is NOT the opposite of install - many platforms
let you install things on a per program basis, but then uninstalling is
done in a centralized manner. I noticed that the ruby gems system, which
I don't claim to know, use this system:

http://www.rubygems.org/read/chapter/10#page38

Having a separate program would enable asking questions and co - which
is not something I would be pleased to see in the python setup.py dance,

cheers,

David

From ziade.tarek at gmail.com  Fri Jan 30 12:17:59 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Fri, 30 Jan 2009 12:17:59 +0100
Subject: [Distutils] Uninstall command, the return
In-Reply-To: <4982C858.1020202@ar.media.kyoto-u.ac.jp>
References: <94bdd2610901291649kbd31d86mbb5c552ba01f85d0@mail.gmail.com>
	<b654cd2e0901292124r6eb372cesc20083931cf319ef@mail.gmail.com>
	<94bdd2610901300107l6bd6476pd67f478cdaceb63c@mail.gmail.com>
	<4982C858.1020202@ar.media.kyoto-u.ac.jp>
Message-ID: <94bdd2610901300317i439b4454r714d53d2d921fab8@mail.gmail.com>

On Fri, Jan 30, 2009 at 10:28 AM, David Cournapeau
<david at ar.media.kyoto-u.ac.jp> wrote:
>
> I can see several things which have a big impact on an uninstall feature
> design:
>    - how to handle multiple versions of one package ?
>    - should it handle dependencies ?

from a distutils point of view, none of these features exist. My opinion
is that  the multiple version and the dependencies matters should be
treated separately,
because the atomic part is the package.

And if you know how to uninstall a package, it's the basis of a
meta-uninstaller.

>    - should it work for packages installed anywhere, or only for a few
> reserved places ?

It should work for any package installed. Wherever it's installed. If
it's in the path and usable in Python,
it should be removable.

>
> Only the last point is worth considering at the distutils level - the
> two others are specific to setuptools (or more exactly any system on top
> of distutils). Once those decisions are made, we can make decision on
> how to deal with uninstall.
>
> For example, if uninstalling only packages installed in some blessed
> locations is acceptable, the problem of a global registry of some kind
> becomes much easier to deal with.

the "blessed locations"  are already known. For instance, if you take a look
at the patch I am working on here (http://bugs.python.org/issue4908)
we are able to find the packages and their egg-link file/folders (so
any record of installation if it's kept in the egg-info)

this code is still incomplete, I am waiting for PJE feedback to add
the support for zip files that contains
several .egg folders or zip files. But the current version is able to
find a package and its metadata.

notice that there are probably some edge cases we need to address I
believe (setuptools too)
like sys.meta_path and sys.path_hooks support.

> I also wonder whether we should add uninstall feature to distutils
> itself from a UI POV.
> [cut]
> Having a separate program would enable asking questions and co - which
> is not something I would be pleased to see in the python setup.py dance,

My opinion is that distutils and pkgutil should provide a set of apis to:

- locate a package (see http://bugs.python.org/issue4908)
- install / uninstall a package

Then two simple basics commands could be offered in Python executable scripts.

>From there any third party app could have fun adding a better UI

But my point is that the logic behind install/uninstall should be
unique and clearly defined
and in one and only one place (e.g. Python) to avoid having many
different ways to deal with
those problems.

I think this is why we have several kinds of tools to answer to these
problems slighlty differenlty
at the moment. But if we can reach a consensus on the
install/uninstall part and have something for it in
Python, it's a good step forward imho.

I have added a question in the survey about uninstallation. and this
thread could
be a good starting point for providing more infos to the people to
answer that question
(eg what could be possible.?)

Cheers
Tarek


-- 
Tarek Ziad? | Association AfPy | www.afpy.org
Blog FR | http://programmation-python.org
Blog EN | http://tarekziade.wordpress.com/

From david at ar.media.kyoto-u.ac.jp  Fri Jan 30 12:21:56 2009
From: david at ar.media.kyoto-u.ac.jp (David Cournapeau)
Date: Fri, 30 Jan 2009 20:21:56 +0900
Subject: [Distutils] Uninstall command, the return
In-Reply-To: <94bdd2610901300317i439b4454r714d53d2d921fab8@mail.gmail.com>
References: <94bdd2610901291649kbd31d86mbb5c552ba01f85d0@mail.gmail.com>	
	<b654cd2e0901292124r6eb372cesc20083931cf319ef@mail.gmail.com>	
	<94bdd2610901300107l6bd6476pd67f478cdaceb63c@mail.gmail.com>	
	<4982C858.1020202@ar.media.kyoto-u.ac.jp>
	<94bdd2610901300317i439b4454r714d53d2d921fab8@mail.gmail.com>
Message-ID: <4982E2D4.2090309@ar.media.kyoto-u.ac.jp>

Tarek Ziad? wrote:
> On Fri, Jan 30, 2009 at 10:28 AM, David Cournapeau
> <david at ar.media.kyoto-u.ac.jp> wrote:
>   
>> I can see several things which have a big impact on an uninstall feature
>> design:
>>    - how to handle multiple versions of one package ?
>>    - should it handle dependencies ?
>>     
>
> from a distutils point of view, none of these features exist. My opinion
> is that  the multiple version and the dependencies matters should be
> treated separately,
> because the atomic part is the package.
>   

agreed

> And if you know how to uninstall a package, it's the basis of a
> meta-uninstaller.
>   

Not necessarily: many package systems simply do not deal at all with
multiple versions. This really complicates the matter a lot right away
whether that's something which needs to be supported, even if it is not
supported by the tool itself. IOW, ignoring the problem in distutils may
well render any multiple version handling impossible for tools build on
top of distutils.

>>    - should it work for packages installed anywhere, or only for a few
>> reserved places ?
>>     
>
> It should work for any package installed. Wherever it's installed. If
> it's in the path and usable in Python,
> it should be removable.
>   

Again, that's a major complication - so if it is required, it should
really be weight against the implementation and reliability cost.

For example, I am somewhat familiar with the following systems used for
deployment: autoconf-based, .deb based, .net based, stow-based, windows
Side by side, globac assembly cache (GAC, the .net stuff). AFAIK, not a
single one supports both multiple versions deployment and arbitrary
installation location. That should at least be an indication this is not
a trivial matter.

>
> My opinion is that distutils and pkgutil should provide a set of apis to:
>
> - locate a package (see http://bugs.python.org/issue4908)
> - install / uninstall a package
>
> Then two simple basics commands could be offered in Python executable scripts.
>
> >From there any third party app could have fun adding a better UI
>
> But my point is that the logic behind install/uninstall should be
> unique and clearly defined
> and in one and only one place (e.g. Python) to avoid having many
> different ways to deal with
> those problems.
>   

100 % agreed. My remark was just that uninstalling may require asking
some questions to the user depending on the system state, so a distutils
uninstall command at the UI level may not be advisable . Of course, the
actual implementation for uninstalling should be done in distutils
itself, with a *documented* API so that other tools can be built on top
of it.

David



From david.lyon at preisshare.net  Fri Jan 30 13:57:27 2009
From: david.lyon at preisshare.net (David Lyon)
Date: Fri, 30 Jan 2009 07:57:27 -0500
Subject: [Distutils]
 =?utf-8?q?GUI_for_Python_package_Management_-_Right_w?=
 =?utf-8?q?ay_to_install_packages=3F?=
In-Reply-To: <49822DC8.7090901@stsci.edu>
References: <4db42df7554839e3151045936e5b0ee4@preisshare.net>	<7CCBA902-15CE-441A-BEE6-DD0972026530@gmail.com>
	<7edd5adaf7ff6650b908cfc76362a64a@preisshare.net>
	<49821912.7050806@stsci.edu>
	<80297AE9-6A91-4A0D-96C3-E9FD7317BA09@gmail.com>
	<49822DC8.7090901@stsci.edu>
Message-ID: <9c6ce8c86e6ef4f0a93ebb2a501a2c41@preisshare.net>

Hi All,

I seem to have a prototype gui of some description running based
around the PyPiXmlRpc

Using that code I can locate packages, but not install or download
them.... (strange - incomplete api i guess)

Anyway, now I need a foolproof/pre-existing way to download and
install modules.

This following script seems to pop up as being a good one to
use:

http://peak.telecommunity.com/dist/ez_setup.py

One good (?) thing is that it seems to resolve package dependencies.

but... my question is... is it "right" ? as far as distutils
and the general python programming community is concerned?

Instead of this, is there even a "right-way" or "better way"?

David


From jeff at taupro.com  Fri Jan 30 14:43:45 2009
From: jeff at taupro.com (Jeff Rush)
Date: Fri, 30 Jan 2009 07:43:45 -0600
Subject: [Distutils] GUI for Python package Management
In-Reply-To: <87zlh9sbej.fsf@benfinney.id.au>
References: <4db42df7554839e3151045936e5b0ee4@preisshare.net>	<7CCBA902-15CE-441A-BEE6-DD0972026530@gmail.com>	<7edd5adaf7ff6650b908cfc76362a64a@preisshare.net>	<49821912.7050806@stsci.edu>	<80297AE9-6A91-4A0D-96C3-E9FD7317BA09@gmail.com>	<49822DC8.7090901@stsci.edu>	<dabf7bf09ede5ae956e60236de9d8c9f@preisshare.net>	<87ljsttuq9.fsf@benfinney.id.au>	<f6fdd83f5c5c1c6c81402f98666c8c6a@preisshare.net>
	<87zlh9sbej.fsf@benfinney.id.au>
Message-ID: <49830411.2050104@taupro.com>

Ben Finney wrote:
> (Could you please set your name in your From field? Thanks.)
> 
> <david.lyon at preisshare.net> writes:
> 
>> >From the other perspective, it is a fairly trivial task for the gui to
>> track what packages it has installed - through it's own configuration file
>> and generate a manifest from that.
> 
> You mean, have packages tracked separately depending on which program
> was used to install them? That seems like a recipe for madness.

Agreed, manifest generation -has- to be done in the backend.

Also any such tracking of what is installed has to address the use case of
sandboxes into which different things are installed concurrently.  And it
needs to handle that some of those sandboxes are connected via wormholes to
the system Python site-packages.  I'm talking about the 'virtualenv' command
and its command-line option whether to let the system site-packages be
available as a last resort when searching for packages.

And having experience with Red Hat RPMs, you also should have a tool to
rebuild the registry of packages, say from .egg metadata, should the registry
be corrupted.

Not an easy problem at all to get right.

-Jeff

From pje at telecommunity.com  Fri Jan 30 16:41:41 2009
From: pje at telecommunity.com (P.J. Eby)
Date: Fri, 30 Jan 2009 10:41:41 -0500
Subject: [Distutils] Uninstall command, the return
In-Reply-To: <94bdd2610901300317i439b4454r714d53d2d921fab8@mail.gmail.co
 m>
References: <94bdd2610901291649kbd31d86mbb5c552ba01f85d0@mail.gmail.com>
	<b654cd2e0901292124r6eb372cesc20083931cf319ef@mail.gmail.com>
	<94bdd2610901300107l6bd6476pd67f478cdaceb63c@mail.gmail.com>
	<4982C858.1020202@ar.media.kyoto-u.ac.jp>
	<94bdd2610901300317i439b4454r714d53d2d921fab8@mail.gmail.com>
Message-ID: <20090130153939.955D83A4116@sparrow.telecommunity.com>

At 12:17 PM 1/30/2009 +0100, Tarek Ziad? wrote:
>this code is still incomplete, I am waiting for PJE feedback to add
>the support for zip files that contains
>several .egg folders or zip files. But the current version is able to
>find a package and its metadata.

What feedback are you waiting for?  I thought I already pointed you 
to the file format details page, and the code in pkg_resources.

FWIW, I think the uninstall manifest should be done as a file within 
an .egg-info directory, as it's the shortest route to 
setuptools/distutils compatibility in this area.  Distutils' 
install_egg_info would just make a directory with PKG-INFO and the 
manifest, and easy_install would record the manifest.

At that point, easy_install would also be able (in principle) to stop:

1. using .pth files,
2. moving eggs to the front of PYTHONPATH, and
3. patching site.py

...which would probably make a lot of people happier.

(It could stop doing these things because it could just extract 
active versions directly to their target directories, and move 
inactive versions to .egg files or directories.)

I'm not saying the change will be *easy*, and uninstall will still 
have to be careful about namespace packages (you'd need to scan all 
the manifests in the target directory to see who else is using those 
files), but the raw material would be there to get the job done.

The manifest should contain size/date/checksum information, and 
should use /-separated paths, and make them relative paths based on 
the sys.path directory, for any file that is a child of the sys.path 
directory.  (i.e., if installed to site-packages, then any file in 
that package that is under site-packages should be listed relative to 
site-packages.)  In this way, moving an entire directory (e.g., 
moving one of your PYTHONPATH directories) will not break the manifest.

The /-separation part is also important: I've seen environments where 
the same libraries are in use on a network with Linux, Mac, *and* 
Windows machines - /-separation is the only way such an install 
manifest would be readable across all 3.


From pje at telecommunity.com  Fri Jan 30 17:15:04 2009
From: pje at telecommunity.com (P.J. Eby)
Date: Fri, 30 Jan 2009 11:15:04 -0500
Subject: [Distutils] patch: solving the two major things that people
 dislike about setuptools, part 1
In-Reply-To: <4978D070.9010709@stsci.edu>
References: <5AA53D0A-98D4-4AF5-843C-DEE1D37C2D24@zooko.com>
	<17519F49-7F21-40D0-BCF5-C0A77E0CFCCB@zooko.com>
	<20090121002856.405BA3A4065@sparrow.telecommunity.com>
	<4978D070.9010709@stsci.edu>
Message-ID: <20090130161302.C57BE3A4116@sparrow.telecommunity.com>

At 03:00 PM 1/22/2009 -0500, Mark Sienkiewicz wrote:
>Phillip J. Eby wrote:
>>At 04:48 PM 1/20/2009 -0700, zooko wrote:
>>>So, this is just to let everyone know that any setuptools variant
>>>(including a future release of PJE's setuptools) that respects the
>>>PYTHONPATH will get at least two users!
>>
>>As I've said before, if somebody creates a patch that addresses the 
>>use cases I pointed out, I'd be happy to accept it.
>
>As I understand it, you require that a package installed by 
>setuptools will be discovered before a package installed by 
>distutils >in the same directory<.
>You do not require that a package installed by setuptools be 
>discovered before a package that is earlier in sys.path.
>
>In that case, the .egg file should go on sys.path immediately before 
>the directory where it is stored.  This ensures that a setuptools 
>.egg file will always be discovered before a distutils-installed 
>package/module in the same directory, but it will not override 
>packages/modules that occur earlier in sys.path.
>
>Do you have any other requirements?
>
>In 
>http://mail.python.org/pipermail/distutils-sig/2008-November/010534.html 
>, I described a change to zooko's patch that implements this behaviour.
>
>In 
>http://mail.python.org/pipermail/distutils-sig/2008-November/010540.html 
>, I address concerns about case-insensitive filesystems.
>So, taken together, does my suggestion meet your criteria for an 
>acceptable patch?
>
>If so,
>
>- I will assemble it into a patch.
>- I will test it on platforms that I have access to, in ways that I 
>know how to use setuptools.  (For the record, that is "python 
>setup.py install --home=/dir"; I never tried the automatic downloads.)
>
>If not,
>
>- what further requirements do you have?

In principle, this sounds pretty good.  The only piece left that I'm 
worried about is that there needs to be a fallback for the .index() 
call in the .pth file.  Otherwise, if for some reason there is *not* 
a match, there will be an error and .pth loading will fail.

On the other hand, maybe it's better for there to be an error in that 
case, I'm just a bit worried about the new and uncontrolled possible 
way for things to break, vs. the known and predictable way they break now.  ;-)

As for the two "big loops" in site-patch.py, the first one is there 
to load the existing site module, without altering sys.path in any 
way.  And it has to use both the path importer cache and 'imp', in 
order to be compatible w/e.g. systems where the stdlib is a zipfile.

The second "big loop" is responsible for moving any paths added by 
.pth files on PYTHONPATH so that they are ahead of the stdlib and 
site-packages.  (Otherwise, if you install something like PIL to 
PYTHONPATH, it can't override a version installed in site-packages.)

It's not clear to me at this moment whether this loop would need to 
change with the patch.  I don't think it will, though.  The 
site-packages eggs would get inserted before site-packages, and would 
be considered "known paths", and thus have their position left 
unchanged.  Whereas PYTHONPATH eggs would be before the stdlib, and 
also left alone.

The last concern I have is whether you can partially upgrade an 
installation, i.e., what happens if you have two PYTHONPATH 
directories, one where this patch has been used, and the other where 
it has not.  The one where it has been used will put its eggs in the 
"right" place, while the other will blindly insert to the 
beginning.  Hm.  Yes, it seems like that would be okay, because the 
"right" place would never be before sys.__egginsert.  Thus, the old 
code could never stick an egg in a place that would mess something 
up.  And both approaches result in PYTHONPATH eggs being before the 
stdlib, and thus left alone by the second loop.

Okay, yes, I think this approach can work... *except* for perhaps the 
case where an old site-patch is in effect, due to being earlier on 
PYTHONPATH, with newer .pth files in a second PYTHONPATH directory.

In that case, the addsitedir() calls in site-patch.py will *not* be normalized.

So, for this to work, site-patch.py should *not* be 
changed.  Instead, the .pth-embedded code needs to do the makepath() 
call before the .index() lookup.  Otherwise, people with multiple 
PYTHONPATH directories would need to specifically update the site.py 
in them, to not have inexplicable breakages.

I'll review an updated patch along these lines.


From pje at telecommunity.com  Fri Jan 30 17:21:29 2009
From: pje at telecommunity.com (P.J. Eby)
Date: Fri, 30 Jan 2009 11:21:29 -0500
Subject: [Distutils] GUI for Python package Management - Right w ay to
 install packages?
In-Reply-To: <9c6ce8c86e6ef4f0a93ebb2a501a2c41@preisshare.net>
References: <4db42df7554839e3151045936e5b0ee4@preisshare.net>
	<7CCBA902-15CE-441A-BEE6-DD0972026530@gmail.com>
	<7edd5adaf7ff6650b908cfc76362a64a@preisshare.net>
	<49821912.7050806@stsci.edu>
	<80297AE9-6A91-4A0D-96C3-E9FD7317BA09@gmail.com>
	<49822DC8.7090901@stsci.edu>
	<9c6ce8c86e6ef4f0a93ebb2a501a2c41@preisshare.net>
Message-ID: <20090130161928.8E9A03A4116@sparrow.telecommunity.com>

At 07:57 AM 1/30/2009 -0500, David Lyon wrote:
>I seem to have a prototype gui of some description running based
>around the PyPiXmlRpc
>
>Using that code I can locate packages, but not install or download
>them.... (strange - incomplete api i guess)
>
>Anyway, now I need a foolproof/pre-existing way to download and
>install modules.
>
>This following script seems to pop up as being a good one to
>use:
>
>http://peak.telecommunity.com/dist/ez_setup.py
>
>One good (?) thing is that it seems to resolve package dependencies.
>
>but... my question is... is it "right" ? as far as distutils
>and the general python programming community is concerned?

Apparently not.  ;-)

However, the primary complaints have to do with how the resulting 
packages are installed, not how they're found or obtained.  And it's 
quite possible to use easy_install as a base for locating and 
retrieving packages from PyPI, based on version requirements, 
*without* using it as the actual installation method.

See, e.g., Ian Bicking's install tool (whose name escapes me at the 
moment), that does this.  zc.buildout wraps easy_install in a similar 
way, to implement still other installation policies.

Using e.g. 'easy_install -e -b somedir SomePackage', will get you a 
somedir/somepackage directory (with 'somepackage' always in lower 
case) with the package's complete source code.  If you then wish to 
install it with distutils or something else, you can then do 
so.  (This is more or less what Ian's tool does.)


>Instead of this, is there even a "right-way" or "better way"?

Not really -- hence the current huge discussions.  ;-)  (Which, by 
the way, I'm glad to see.  I tried to get the same basic discussion 
rolling a few months back, but it didn't really go anywhere back then.)


From chris at simplistix.co.uk  Fri Jan 30 13:18:09 2009
From: chris at simplistix.co.uk (Chris Withers)
Date: Fri, 30 Jan 2009 12:18:09 +0000
Subject: [Distutils] [Python Language Summit] Distutils
	/	Packaging	survey
In-Reply-To: <B0B60DA3-D275-4DF9-B88D-7903A0DB31F2@python.org>
References: <94bdd2610901250306n1e27a928sa0b276d91f80862a@mail.gmail.com>	<90bb445a0901270829y7019eca3p72164c259c4d35d4@mail.gmail.com>	<94bdd2610901271400k32ff74edmea94fcbdfce88f53@mail.gmail.com>	<515B9271-0E7D-4B4E-B63F-F717604A5FEA@zooko.com>	<49803752.9040704@ar.media.kyoto-u.ac.jp>
	<B0B60DA3-D275-4DF9-B88D-7903A0DB31F2@python.org>
Message-ID: <4982F001.2060507@simplistix.co.uk>

Barry Warsaw wrote:
> I'd like to make a radical suggestion: upstream authors should never 
> have to worry about building distribution blobs.
> 
> In my ideal world, I would make a release by tagging my release in 
> whatever vcs I'm using, then I would tell cheeseshop, "hey I just 
> released version 3.1 and it is here" where "here" means whatever native 
> vcs syntax points to the revision I just released.  Then PyPI would do 
> the "coagulation" into distribution formats and distribute it amongst 
> its worldwide mirrors, all automatically.
> 
> I as the Python developer don't want to know about eggs, tarballs, debs, 
> rpms, whatevers, I just want to write some software.  I'm happy to add a 
> bit of metadata to my setup.py to play ball, but otherwise I really just 
> want one command to release my code and then magically have it appear 
> available to everybody.
> 
> If that's not feasible <wink> then the next best thing would be to just 
> spin the source tarball and upload that.  Tarballs I can handle. <wink>
> 
> On the other end, when I zc.buildout, or paver, or easy_install, or 
> aptitude install, or emerge, or port install, or whatever, it would go 
> out to the Worldwide Python Distribution Network and pull down the 
> proper blobby thing to install based on what I'm trying to do, e.g. an 
> egg if I'm developing, a deb if I'm installing into my system Python, 
> etc.  Again, I don't really care and shouldn't have to know.

+sys.maxint to all of the above.

...and I'm damned sure the same is true for anyone using a package.
Of course, here is where the distributions (debian, I'm looking at you!) 
start causing problems by only having ancient versions of packages, 
arbitarily stripping out dev headers and (in the case of xlwt) even 
removing actual source code from the package because they so chose to!

...and they then wonder why people who write python apps and libraries 
tell people not to bother using debian packages and the like? ;-)

</rant>

Chris

-- 
Simplistix - Content Management, Zope & Python Consulting
            - http://www.simplistix.co.uk


From ziade.tarek at gmail.com  Fri Jan 30 17:27:55 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Fri, 30 Jan 2009 17:27:55 +0100
Subject: [Distutils] Uninstall command, the return
In-Reply-To: <20090130153939.955D83A4116@sparrow.telecommunity.com>
References: <94bdd2610901291649kbd31d86mbb5c552ba01f85d0@mail.gmail.com>
	<b654cd2e0901292124r6eb372cesc20083931cf319ef@mail.gmail.com>
	<94bdd2610901300107l6bd6476pd67f478cdaceb63c@mail.gmail.com>
	<4982C858.1020202@ar.media.kyoto-u.ac.jp>
	<94bdd2610901300317i439b4454r714d53d2d921fab8@mail.gmail.com>
	<20090130153939.955D83A4116@sparrow.telecommunity.com>
Message-ID: <94bdd2610901300827m6963ab71n596247ba71426b1@mail.gmail.com>

On Fri, Jan 30, 2009 at 4:41 PM, P.J. Eby <pje at telecommunity.com> wrote:
> At 12:17 PM 1/30/2009 +0100, Tarek Ziad? wrote:
>>
>> this code is still incomplete, I am waiting for PJE feedback to add
>> the support for zip files that contains
>> several .egg folders or zip files. But the current version is able to
>> find a package and its metadata.
>
> What feedback are you waiting for?  I thought I already pointed you to the
> file format details page, and the code in pkg_resources.

I got lost, and I was waiting for feedback on this last mail:

http://www.mail-archive.com/distutils-sig at python.org/msg06367.html

>
> FWIW, I think the uninstall manifest should be done as a file within an
> .egg-info directory, as it's the shortest route to setuptools/distutils
> compatibility in this area.  Distutils' install_egg_info would just make a
> directory with PKG-INFO and the manifest, and easy_install would record the
> manifest.
>
> At that point, easy_install would also be able (in principle) to stop:
>
> 1. using .pth files,
> 2. moving eggs to the front of PYTHONPATH, and
> 3. patching site.py
>

sounds good

> ...which would probably make a lot of people happier.
>
> (It could stop doing these things because it could just extract active
> versions directly to their target directories, and move inactive versions to
> .egg files or directories.)
>
> I'm not saying the change will be *easy*, and uninstall will still have to
> be careful about namespace packages (you'd need to scan all the manifests in
> the target directory to see who else is using those files), but the raw
> material would be there to get the job done.
>
> The manifest should contain size/date/checksum information, and should use
> /-separated paths, and make them relative paths based on the sys.path
> directory, for any file that is a child of the sys.path directory.  (i.e.,
> if installed to site-packages, then any file in that package that is under
> site-packages should be listed relative to site-packages.)  In this way,
> moving an entire directory (e.g., moving one of your PYTHONPATH directories)
> will not break the manifest.
>
> The /-separation part is also important: I've seen environments where the
> same libraries are in use on a network with Linux, Mac, *and* Windows
> machines - /-separation is the only way such an install manifest would be
> readable across all 3.
>

ok thanks for the tips, I guess there are enough infos from everyone to start
to write down a proposal


-- 
Tarek Ziad? | Association AfPy | www.afpy.org
Blog FR | http://programmation-python.org
Blog EN | http://tarekziade.wordpress.com/

From floris.bruynooghe at gmail.com  Fri Jan 30 19:39:51 2009
From: floris.bruynooghe at gmail.com (Floris Bruynooghe)
Date: Fri, 30 Jan 2009 18:39:51 +0000
Subject: [Distutils] [Python Language Summit] Distutils /
	Packaging	survey
In-Reply-To: <94bdd2610901291621m10056e9bvf702474e866265cf@mail.gmail.com>
References: <49803752.9040704@ar.media.kyoto-u.ac.jp>
	<94bdd2610901280307h42b05abcr5599d64189948201@mail.gmail.com>
	<49803BDF.4060204@ar.media.kyoto-u.ac.jp>
	<b654cd2e0901281116u57f8967dl785ffc09a0edf291@mail.gmail.com>
	<1233237184.8068.9.camel@tomoyo> <877i4dvl02.fsf@benfinney.id.au>
	<498232A0.8060306@palladion.com>
	<94bdd2610901291454s1eee0d0fvfbcb8d21d871e9ee@mail.gmail.com>
	<20090129233428.GA10050@laurie.devork>
	<94bdd2610901291621m10056e9bvf702474e866265cf@mail.gmail.com>
Message-ID: <20090130183951.GA5119@laurie.devork>

On Fri, Jan 30, 2009 at 01:21:12AM +0100, Tarek Ziad? wrote:
> On Fri, Jan 30, 2009 at 12:34 AM, Floris Bruynooghe
> <floris.bruynooghe at gmail.com> wrote:
> > An argument against a generated .py file is that this won't work for
> > single-module python distributions.  But having a .cfg file or
> > something next to the module/package might defeat the point of trying
> > to help the FHS in violating it already.  OTOH .egg-info does that too
> > and seems to be accepted currently.
> 
> Well, the egg-info file is the static version of setup.py metadata in some way,
> I would find a new file redundant.
> 
> I am really curious though, to see what the file would contain..

I imagine things like libdir, prefix, datadir, docdir and other things
copied from autoconf.  Where the defaults would be something like:

prefix = sys.prefix
libdir = sys.prefix/lib/pythonX.Y/site-packages/pkgname
datadir = sys.prefix/share/mypackage
docdir = sys.prefix/share/doc/mypackage


> from my current understanding, and having in mind to introduce such a
> file in a way things
> can gently evolve:
> 
> What if an egg-info-like file was present in the package from the very
> beginning,
> describing the package metadata, and used by setup.py, *and* at build time
> by third party tools. (with setup.py knowing that it can be changed at run time)

I don't see why moving all the metadata to egg-info would improve
things.  You could easily(?) deprecate the `package_data' and
`data_files' keywords to setup() and replace them with `doc_files',
`data_files', `man_files', `config_files', etc.  And depending on
which --sysconfdir, --datadir, etc options where used to setup.py
install_egg_info would write the correct values for datadir etc.  The
runtime would then use your pkginfo.get_metadata() to find the files.

It might even be possible to do all of this inside distutils without
breaking backwards compatibility, just deprecating some keywords once
people start liking and using it.


Regards
Floris


-- 
Debian GNU/Linux -- The Power of Freedom
www.debian.org | www.gnu.org | www.kernel.org

From pje at telecommunity.com  Fri Jan 30 20:36:43 2009
From: pje at telecommunity.com (P.J. Eby)
Date: Fri, 30 Jan 2009 14:36:43 -0500
Subject: [Distutils] [Python Language Summit] Distutils /
 Packaging	survey
In-Reply-To: <20090130183951.GA5119@laurie.devork>
References: <49803752.9040704@ar.media.kyoto-u.ac.jp>
	<94bdd2610901280307h42b05abcr5599d64189948201@mail.gmail.com>
	<49803BDF.4060204@ar.media.kyoto-u.ac.jp>
	<b654cd2e0901281116u57f8967dl785ffc09a0edf291@mail.gmail.com>
	<1233237184.8068.9.camel@tomoyo> <877i4dvl02.fsf@benfinney.id.au>
	<498232A0.8060306@palladion.com>
	<94bdd2610901291454s1eee0d0fvfbcb8d21d871e9ee@mail.gmail.com>
	<20090129233428.GA10050@laurie.devork>
	<94bdd2610901291621m10056e9bvf702474e866265cf@mail.gmail.com>
	<20090130183951.GA5119@laurie.devork>
Message-ID: <20090130193447.89D7E3A40A0@sparrow.telecommunity.com>

At 06:39 PM 1/30/2009 +0000, Floris Bruynooghe wrote:
>On Fri, Jan 30, 2009 at 01:21:12AM +0100, Tarek Ziad? wrote:
> > On Fri, Jan 30, 2009 at 12:34 AM, Floris Bruynooghe
> > <floris.bruynooghe at gmail.com> wrote:
> > > An argument against a generated .py file is that this won't work for
> > > single-module python distributions.  But having a .cfg file or
> > > something next to the module/package might defeat the point of trying
> > > to help the FHS in violating it already.  OTOH .egg-info does that too
> > > and seems to be accepted currently.
> >
> > Well, the egg-info file is the static version of setup.py 
> metadata in some way,
> > I would find a new file redundant.
> >
> > I am really curious though, to see what the file would contain..
>
>I imagine things like libdir, prefix, datadir, docdir and other things
>copied from autoconf.  Where the defaults would be something like:
>
>prefix = sys.prefix
>libdir = sys.prefix/lib/pythonX.Y/site-packages/pkgname
>datadir = sys.prefix/share/mypackage
>docdir = sys.prefix/share/doc/mypackage

I'm confused by the above lines: do you mean the *project* name, or 
the name of some package within the project?  What if the project 
contains no packages, only modules?  What is libdir for?


> > from my current understanding, and having in mind to introduce such a
> > file in a way things
> > can gently evolve:
> >
> > What if an egg-info-like file was present in the package from the very
> > beginning,
> > describing the package metadata, and used by setup.py, *and* at build time
> > by third party tools. (with setup.py knowing that it can be 
> changed at run time)
>
>I don't see why moving all the metadata to egg-info would improve
>things.

Moving from code to data means better tool 
interoperability.  setup.py sucks as a format for obtaining data, 
especially since many distutils newbs hardcode all sorts of rubbish 
in their setup.py files, like writing to files without paying 
attention to the command line.


>   You could easily(?) deprecate the `package_data' and
>`data_files' keywords to setup() and replace them with `doc_files',
>`data_files', `man_files', `config_files', etc.  And depending on
>which --sysconfdir, --datadir, etc options where used to setup.py
>install_egg_info would write the correct values for datadir etc.  The
>runtime would then use your pkginfo.get_metadata() to find the files.

...in which case, why not just put all the files in the .egg-info to 
begin with?

Meanwhile, getting rid of package_data or *requiring* a runtime API 
to access files is going to be a major barrier to adoption in the short run.


From ziade.tarek at gmail.com  Fri Jan 30 20:35:07 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Fri, 30 Jan 2009 20:35:07 +0100
Subject: [Distutils] [Python Language Summit] Distutils / Packaging
	survey
In-Reply-To: <20090130183951.GA5119@laurie.devork>
References: <49803752.9040704@ar.media.kyoto-u.ac.jp>
	<49803BDF.4060204@ar.media.kyoto-u.ac.jp>
	<b654cd2e0901281116u57f8967dl785ffc09a0edf291@mail.gmail.com>
	<1233237184.8068.9.camel@tomoyo> <877i4dvl02.fsf@benfinney.id.au>
	<498232A0.8060306@palladion.com>
	<94bdd2610901291454s1eee0d0fvfbcb8d21d871e9ee@mail.gmail.com>
	<20090129233428.GA10050@laurie.devork>
	<94bdd2610901291621m10056e9bvf702474e866265cf@mail.gmail.com>
	<20090130183951.GA5119@laurie.devork>
Message-ID: <94bdd2610901301135v1e88cadehc25fdb9472329912@mail.gmail.com>

On Fri, Jan 30, 2009 at 7:39 PM, Floris Bruynooghe
<floris.bruynooghe at gmail.com> wrote:
> On Fri, Jan 30, 2009 at 01:21:12AM +0100, Tarek Ziad? wrote:
>> On Fri, Jan 30, 2009 at 12:34 AM, Floris Bruynooghe
>> <floris.bruynooghe at gmail.com> wrote:
>> > An argument against a generated .py file is that this won't work for
>> > single-module python distributions.  But having a .cfg file or
>> > something next to the module/package might defeat the point of trying
>> > to help the FHS in violating it already.  OTOH .egg-info does that too
>> > and seems to be accepted currently.
>>
>> Well, the egg-info file is the static version of setup.py metadata in some way,
>> I would find a new file redundant.
>>
>> I am really curious though, to see what the file would contain..
>
> I imagine things like libdir, prefix, datadir, docdir and other things
> copied from autoconf.  Where the defaults would be something like:
>
> prefix = sys.prefix
> libdir = sys.prefix/lib/pythonX.Y/site-packages/pkgname
> datadir = sys.prefix/share/mypackage
> docdir = sys.prefix/share/doc/mypackage
>

David Cournapeau has sent me a first draft of a document, you might
want to look at.

http://wiki.python.org/moin/distutils-autoconf

Might be a good basis for a common understanding



-- 
Tarek Ziad? | Association AfPy | www.afpy.org
Blog FR | http://programmation-python.org
Blog EN | http://tarekziade.wordpress.com/

From ianb at colorstudy.com  Fri Jan 30 22:19:05 2009
From: ianb at colorstudy.com (Ian Bicking)
Date: Fri, 30 Jan 2009 15:19:05 -0600
Subject: [Distutils] [Python Language Summit] Distutils / Packaging
	survey
In-Reply-To: <20090130183951.GA5119@laurie.devork>
References: <49803752.9040704@ar.media.kyoto-u.ac.jp>
	<49803BDF.4060204@ar.media.kyoto-u.ac.jp>
	<b654cd2e0901281116u57f8967dl785ffc09a0edf291@mail.gmail.com>
	<1233237184.8068.9.camel@tomoyo> <877i4dvl02.fsf@benfinney.id.au>
	<498232A0.8060306@palladion.com>
	<94bdd2610901291454s1eee0d0fvfbcb8d21d871e9ee@mail.gmail.com>
	<20090129233428.GA10050@laurie.devork>
	<94bdd2610901291621m10056e9bvf702474e866265cf@mail.gmail.com>
	<20090130183951.GA5119@laurie.devork>
Message-ID: <b654cd2e0901301319k13974239oe3ac3cb874c02fe0@mail.gmail.com>

On Fri, Jan 30, 2009 at 12:39 PM, Floris Bruynooghe <
floris.bruynooghe at gmail.com> wrote:

> I imagine things like libdir, prefix, datadir, docdir and other things
> copied from autoconf.  Where the defaults would be something like:
>
> prefix = sys.prefix
> libdir = sys.prefix/lib/pythonX.Y/site-packages/pkgname
> datadir = sys.prefix/share/mypackage
> docdir = sys.prefix/share/doc/mypackage
>

I wouldn't want to use those.  What goes in libdir, what goes in datadir?  I
don't know, and frankly the distinctions start getting really arbitrary.

I would rather see something like pkg_resources existing API, where there is
some file that maps out how the local names of files (where they'd be in a
checkout) map to their installed location, then the pkg_resources code could
finds the real location of the file.


-- 
Ian Bicking  |  http://blog.ianbicking.org
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20090130/1f30a5b0/attachment.htm>

From floris.bruynooghe at gmail.com  Fri Jan 30 22:54:39 2009
From: floris.bruynooghe at gmail.com (Floris Bruynooghe)
Date: Fri, 30 Jan 2009 21:54:39 +0000
Subject: [Distutils] [Python Language Summit] Distutils /
	Packaging	survey
In-Reply-To: <94bdd2610901301135v1e88cadehc25fdb9472329912@mail.gmail.com>
References: <49803BDF.4060204@ar.media.kyoto-u.ac.jp>
	<b654cd2e0901281116u57f8967dl785ffc09a0edf291@mail.gmail.com>
	<1233237184.8068.9.camel@tomoyo> <877i4dvl02.fsf@benfinney.id.au>
	<498232A0.8060306@palladion.com>
	<94bdd2610901291454s1eee0d0fvfbcb8d21d871e9ee@mail.gmail.com>
	<20090129233428.GA10050@laurie.devork>
	<94bdd2610901291621m10056e9bvf702474e866265cf@mail.gmail.com>
	<20090130183951.GA5119@laurie.devork>
	<94bdd2610901301135v1e88cadehc25fdb9472329912@mail.gmail.com>
Message-ID: <20090130215439.GB5606@laurie.devork>

On Fri, Jan 30, 2009 at 08:35:07PM +0100, Tarek Ziad? wrote:
> David Cournapeau has sent me a first draft of a document, you might
> want to look at.
> 
> http://wiki.python.org/moin/distutils-autoconf
> 
> Might be a good basis for a common understanding

Looks useful!  Not sure where to comment on this so doing it here...

Under Categories I think "configuration files (sysconfdir)" and
"statedir" are missing.  I also don't know in how far it's useful to
provide a separate "logdir", autotools doesn't seem to do this it
rather let's you construct one using statedir.  As for "html doc" and
"pdf doc" I'm not sure why they are separate, surely they can both
just be constructed from "docdir"?  The trick is to make it fine
grained enough so packagers will be able to customise it enough and
leave it coarse enough so that developers will want to classify their
files properly.


Regards
Floris

-- 
Debian GNU/Linux -- The Power of Freedom
www.debian.org | www.gnu.org | www.kernel.org

From david.lyon at preisshare.net  Fri Jan 30 22:57:08 2009
From: david.lyon at preisshare.net (David Lyon)
Date: Fri, 30 Jan 2009 16:57:08 -0500
Subject: [Distutils]
 =?utf-8?q?GUI_for_Python_package_Management_-_Right_w?=
 =?utf-8?q?_ay_to_install_packages=3F?=
In-Reply-To: <20090130161928.8E9A03A4116@sparrow.telecommunity.com>
References: <4db42df7554839e3151045936e5b0ee4@preisshare.net>
	<7CCBA902-15CE-441A-BEE6-DD0972026530@gmail.com>
	<7edd5adaf7ff6650b908cfc76362a64a@preisshare.net>
	<49821912.7050806@stsci.edu>
	<80297AE9-6A91-4A0D-96C3-E9FD7317BA09@gmail.com>
	<49822DC8.7090901@stsci.edu>
	<9c6ce8c86e6ef4f0a93ebb2a501a2c41@preisshare.net>
	<20090130161928.8E9A03A4116@sparrow.telecommunity.com>
Message-ID: <aa1e82671c0d68b3b93036f7a7ff8a4b@preisshare.net>



On Fri, 30 Jan 2009 11:21:29 -0500, "P.J. Eby" <pje at telecommunity.com>
wrote:

>>but... my question is... is it "right" ? as far as distutils
>>and the general python programming community is concerned?
> 
> Apparently not.  ;-)

So the "right" way is to download it, unpack it in a directory and
then just run "python setup.py install" as we would do from the
command line?

Another 20 lines of code or so.... 


From david.lyon at preisshare.net  Fri Jan 30 23:15:23 2009
From: david.lyon at preisshare.net (David Lyon)
Date: Fri, 30 Jan 2009 17:15:23 -0500
Subject: [Distutils] GUI for Python package Management
In-Reply-To: <49830411.2050104@taupro.com>
References: <4db42df7554839e3151045936e5b0ee4@preisshare.net>	<7CCBA902-15CE-441A-BEE6-DD0972026530@gmail.com>	<7edd5adaf7ff6650b908cfc76362a64a@preisshare.net>	<49821912.7050806@stsci.edu>	<80297AE9-6A91-4A0D-96C3-E9FD7317BA09@gmail.com>	<49822DC8.7090901@stsci.edu>	<dabf7bf09ede5ae956e60236de9d8c9f@preisshare.net>	<87ljsttuq9.fsf@benfinney.id.au>	<f6fdd83f5c5c1c6c81402f98666c8c6a@preisshare.net>
	<87zlh9sbej.fsf@benfinney.id.au> <49830411.2050104@taupro.com>
Message-ID: <84748f63954c9b9d2f59c76506efa54e@preisshare.net>



On Fri, 30 Jan 2009 07:43:45 -0600, Jeff Rush <jeff at taupro.com> wrote:
>> You mean, have packages tracked separately depending on which program
>> was used to install them? That seems like a recipe for madness.
> 
> Agreed, manifest generation -has- to be done in the backend.

Isn't this already been done in the backend? :-)

Actually, when you install enough packages, this is where it gets crazy
and it drives the average mad - from a user perspective. Some packages are 
installed using a windows installer (or rpm), some using 'setup.py
install', 
some using easy_install.

In other languages, you just use the one tool..

Users already know madness... :-) and have had it for some time not
through anybodies fault. 

The discussion you guys are having about all the fixes that you
can/could do is very interesting. I think you are all on the right
track.

Regards

David








From floris.bruynooghe at gmail.com  Fri Jan 30 23:42:54 2009
From: floris.bruynooghe at gmail.com (Floris Bruynooghe)
Date: Fri, 30 Jan 2009 22:42:54 +0000
Subject: [Distutils] [Python Language Summit] Distutils /
	Packaging	survey
In-Reply-To: <20090130193447.89D7E3A40A0@sparrow.telecommunity.com>
References: <49803BDF.4060204@ar.media.kyoto-u.ac.jp>
	<b654cd2e0901281116u57f8967dl785ffc09a0edf291@mail.gmail.com>
	<1233237184.8068.9.camel@tomoyo> <877i4dvl02.fsf@benfinney.id.au>
	<498232A0.8060306@palladion.com>
	<94bdd2610901291454s1eee0d0fvfbcb8d21d871e9ee@mail.gmail.com>
	<20090129233428.GA10050@laurie.devork>
	<94bdd2610901291621m10056e9bvf702474e866265cf@mail.gmail.com>
	<20090130183951.GA5119@laurie.devork>
	<20090130193447.89D7E3A40A0@sparrow.telecommunity.com>
Message-ID: <20090130224254.GC5606@laurie.devork>

On Fri, Jan 30, 2009 at 02:36:43PM -0500, P.J. Eby wrote:
> At 06:39 PM 1/30/2009 +0000, Floris Bruynooghe wrote:
>> I imagine things like libdir, prefix, datadir, docdir and other things
>> copied from autoconf.  Where the defaults would be something like:
>>
>> prefix = sys.prefix
>> libdir = sys.prefix/lib/pythonX.Y/site-packages/pkgname
>> datadir = sys.prefix/share/mypackage
>> docdir = sys.prefix/share/doc/mypackage
>
> I'm confused by the above lines: do you mean the *project* name, or the 
> name of some package within the project?  What if the project contains no 
> packages, only modules?  What is libdir for?

Oops, sorry.  That's even inconsistent in a few lines!

libdir = sys.prefix/lib/pythonX.Y/site-packages
datadir = sys.prefix/share/<project_name>
docdir = sys.prefix/share/doc/<project_name>

libdir's purpose is no more then saying where the modules and packages
go.  I can't think of any reason (right now) why a module/application
would want to know this, but the installation tool needs to know this.
http://wiki.python.org/moin/distutils-autoconf tries to split this up
into arch-independent and arch-dependent files (.py/.pyc/.pyo and
.so/.pyd) but I'm not sure if that will work since that creates the
namespaces problem of having an extension module inside a package with
normal modules.  It might be better to keep treating pure python
modules as arch-dependent, as is done now.


>> > from my current understanding, and having in mind to introduce such a
>> > file in a way things
>> > can gently evolve:
>> >
>> > What if an egg-info-like file was present in the package from the very
>> > beginning,
>> > describing the package metadata, and used by setup.py, *and* at build time
>> > by third party tools. (with setup.py knowing that it can be changed 
>> at run time)
>>
>> I don't see why moving all the metadata to egg-info would improve
>> things.
>
> Moving from code to data means better tool interoperability.  setup.py 
> sucks as a format for obtaining data, especially since many distutils 
> newbs hardcode all sorts of rubbish in their setup.py files, like writing 
> to files without paying attention to the command line.

Yes that's true, I agree.  I forgot about this argument and was just
thinking of what would be the smallest step for setup.py writers
(developers).

>>   You could easily(?) deprecate the `package_data' and
>> `data_files' keywords to setup() and replace them with `doc_files',
>> `data_files', `man_files', `config_files', etc.  And depending on
>> which --sysconfdir, --datadir, etc options where used to setup.py
>> install_egg_info would write the correct values for datadir etc.  The
>> runtime would then use your pkginfo.get_metadata() to find the files.
>
> ...in which case, why not just put all the files in the .egg-info to  
> begin with?

You're right, it may need more changes to distutils and especially
since both the old and new way will need to be supported at the same
time, but it will be better in the end.

> Meanwhile, getting rid of package_data or *requiring* a runtime API to 
> access files is going to be a major barrier to adoption in the short
> run.

It should not be *required*.  If someone keeps using the
`package_data' keyword it should just end up in $libdir like it used
too (and you will be able to find it using os.path and __file__), thus
not forcing anyone to change (until a packager gets fed up and submits
a patch ;-)).  It's just if someone *wants* to provide the extra
metadata and use an API to find the data at runtime then it's there in
a nice and sensible way.


Regards
Floris


-- 
Debian GNU/Linux -- The Power of Freedom
www.debian.org | www.gnu.org | www.kernel.org

From floris.bruynooghe at gmail.com  Sat Jan 31 00:02:36 2009
From: floris.bruynooghe at gmail.com (Floris Bruynooghe)
Date: Fri, 30 Jan 2009 23:02:36 +0000
Subject: [Distutils] [Python Language Summit] Distutils /
	Packaging	survey
In-Reply-To: <b654cd2e0901301319k13974239oe3ac3cb874c02fe0@mail.gmail.com>
References: <49803BDF.4060204@ar.media.kyoto-u.ac.jp>
	<b654cd2e0901281116u57f8967dl785ffc09a0edf291@mail.gmail.com>
	<1233237184.8068.9.camel@tomoyo> <877i4dvl02.fsf@benfinney.id.au>
	<498232A0.8060306@palladion.com>
	<94bdd2610901291454s1eee0d0fvfbcb8d21d871e9ee@mail.gmail.com>
	<20090129233428.GA10050@laurie.devork>
	<94bdd2610901291621m10056e9bvf702474e866265cf@mail.gmail.com>
	<20090130183951.GA5119@laurie.devork>
	<b654cd2e0901301319k13974239oe3ac3cb874c02fe0@mail.gmail.com>
Message-ID: <20090130230236.GD5606@laurie.devork>

On Fri, Jan 30, 2009 at 03:19:05PM -0600, Ian Bicking wrote:
> On Fri, Jan 30, 2009 at 12:39 PM, Floris Bruynooghe <
> floris.bruynooghe at gmail.com> wrote:
> 
> > I imagine things like libdir, prefix, datadir, docdir and other things
> > copied from autoconf.  Where the defaults would be something like:
> >
> > prefix = sys.prefix
> > libdir = sys.prefix/lib/pythonX.Y/site-packages/pkgname
> > datadir = sys.prefix/share/mypackage
> > docdir = sys.prefix/share/doc/mypackage

Ok, that was supposed to read:

prefix = sys.prefix
libdir = sys.prefix/lib/pythonX.Y/site-packages/pkgname
datadir = sys.prefix/share/mypackage
docdir = sys.prefix/share/doc/mypackage

> I wouldn't want to use those.  What goes in libdir, what goes in datadir?  I
> don't know, and frankly the distinctions start getting really
> arbitrary.

You won't have to know or care.  What you define with `package_data',
`py_modules' and `package_dir' would just be translated to it (they
all go to libdir), it's a distutils (or other tool that's consuming
the metadata) implementation detail.  If you'd opt to tag a file as
going into "datadir" for example instead of using `data_files' then
you'd have to use a pkg_resources-like API to retrieve that (with one
more argument to say that you want a "datadir" resource).


Regards
Floris

-- 
Debian GNU/Linux -- The Power of Freedom
www.debian.org | www.gnu.org | www.kernel.org

From ben+python at benfinney.id.au  Sat Jan 31 00:03:50 2009
From: ben+python at benfinney.id.au (Ben Finney)
Date: Sat, 31 Jan 2009 10:03:50 +1100
Subject: [Distutils] GUI for Python package Management
References: <4db42df7554839e3151045936e5b0ee4@preisshare.net>
	<7CCBA902-15CE-441A-BEE6-DD0972026530@gmail.com>
	<7edd5adaf7ff6650b908cfc76362a64a@preisshare.net>
	<49821912.7050806@stsci.edu>
	<80297AE9-6A91-4A0D-96C3-E9FD7317BA09@gmail.com>
	<49822DC8.7090901@stsci.edu>
	<dabf7bf09ede5ae956e60236de9d8c9f@preisshare.net>
	<87ljsttuq9.fsf@benfinney.id.au>
	<f6fdd83f5c5c1c6c81402f98666c8c6a@preisshare.net>
	<87zlh9sbej.fsf@benfinney.id.au> <49830411.2050104@taupro.com>
	<84748f63954c9b9d2f59c76506efa54e@preisshare.net>
Message-ID: <8763jw1jmx.fsf@benfinney.id.au>

David Lyon <david.lyon at preisshare.net> writes:

> In other languages, you just use the one tool..

That's misleading. In well-designed systems, you use any of a number
of different *tools*, all of which use the same back-end *library
interface* (which might effectively mean ?the same library? or
?mutliple libraries coded to the same interface?).

If the only workable back-end is a single tool, then that tool
effectively becomes the back-end library and people bolt on multiple
tools as front-ends to it. But that seems inferior to a true reusable
library that is intended for use in implementing front-end tools.

-- 
 \        ?Odious ideas are not entitled to hide from criticism behind |
  `\          the human shield of their believers' feelings.? ?Richard |
_o__)                                                         Stallman |
Ben Finney


From floris.bruynooghe at gmail.com  Sat Jan 31 00:08:49 2009
From: floris.bruynooghe at gmail.com (Floris Bruynooghe)
Date: Fri, 30 Jan 2009 23:08:49 +0000
Subject: [Distutils] Solaris and distutils: Need to pass
	LIBDIR	explicitly with -L when building extensions?
In-Reply-To: <497E59C6.4020802@enthought.com>
References: <497E59C6.4020802@enthought.com>
Message-ID: <20090130230849.GE5606@laurie.devork>

Hi Dave

On Mon, Jan 26, 2009 at 06:48:06PM -0600, Dave Peterson wrote:
> I am trying to build a number of projects that use Python extensions on  
> Solaris 10 and I've discovered that nothing with extensions will link  
> unless I explicitly pass in a '-L/path/to/python/lib/dir' because  
> libpython2.5.so is not otherwise found when the '-lpython2.5' argument  
> is specified during linking.
[...]
> If that all seems correct, then it appears the issue is the  
> finalize_options() source in lib/distutils/commands/build_ext.py.  There  
> are a number of "if" blocks that explicitly append the value of  
> distutils.sysconfig.get_config_vars('LIBDIR') to the list of  
> library_dirs used to link built extensions with.  However, there doesn't  
> seem to be one of these for Solaris / sunos.

Could you point to one of the projects you're having trouble with?  I
haven't had any such problems with Python 2.5 on Solaris 10, distutils
always finds the right things when I'm building extension modules.


Regards
Floris

-- 
Debian GNU/Linux -- The Power of Freedom
www.debian.org | www.gnu.org | www.kernel.org

From ben+python at benfinney.id.au  Sat Jan 31 00:09:28 2009
From: ben+python at benfinney.id.au (Ben Finney)
Date: Sat, 31 Jan 2009 10:09:28 +1100
Subject: [Distutils] [Python Language Summit] Distutils /
	Packaging	survey
References: <49803752.9040704@ar.media.kyoto-u.ac.jp>
	<94bdd2610901280307h42b05abcr5599d64189948201@mail.gmail.com>
	<49803BDF.4060204@ar.media.kyoto-u.ac.jp>
	<b654cd2e0901281116u57f8967dl785ffc09a0edf291@mail.gmail.com>
	<1233237184.8068.9.camel@tomoyo> <877i4dvl02.fsf@benfinney.id.au>
	<498232A0.8060306@palladion.com>
	<94bdd2610901291454s1eee0d0fvfbcb8d21d871e9ee@mail.gmail.com>
	<20090129233428.GA10050@laurie.devork>
	<94bdd2610901291621m10056e9bvf702474e866265cf@mail.gmail.com>
	<20090130183951.GA5119@laurie.devork>
	<20090130193447.89D7E3A40A0@sparrow.telecommunity.com>
Message-ID: <871vuk1jdj.fsf@benfinney.id.au>

"P.J. Eby" <pje at telecommunity.com> writes:

> At 06:39 PM 1/30/2009 +0000, Floris Bruynooghe wrote:
> >I imagine things like libdir, prefix, datadir, docdir and other
> >things copied from autoconf. Where the defaults would be something
> >like:
> >
> >prefix = sys.prefix
> >libdir = sys.prefix/lib/pythonX.Y/site-packages/pkgname
> >datadir = sys.prefix/share/mypackage
> >docdir = sys.prefix/share/doc/mypackage
> 
> I'm confused by the above lines: do you mean the *project* name, or
> the name of some package within the project? What if the project
> contains no packages, only modules? What is libdir for?

To clarify: Here again we come up against the unfortunate choice of
the term ?package? in Python to mean something confisingly different
from what it means to most people dealing with software.

I imagine Floris meant what most hackers mean by ?package?, which is
what Python perversely calls a ?distribution?. That may or may not
be what setuptools calls a ?project?, I've never been clear on that
:-/

-- 
 \                             ?Everything is futile.? ?Marvin of Borg |
  `\                                                                   |
_o__)                                                                  |
Ben Finney


From robert.kern at gmail.com  Sat Jan 31 00:12:19 2009
From: robert.kern at gmail.com (Robert Kern)
Date: Fri, 30 Jan 2009 17:12:19 -0600
Subject: [Distutils] Solaris and distutils: Need to pass LIBDIR
 explicitly with -L when building extensions?
In-Reply-To: <20090130230849.GE5606@laurie.devork>
References: <497E59C6.4020802@enthought.com>
	<20090130230849.GE5606@laurie.devork>
Message-ID: <gm01gj$2h9$1@ger.gmane.org>

On 2009-01-30 17:08, Floris Bruynooghe wrote:
> Hi Dave
>
> On Mon, Jan 26, 2009 at 06:48:06PM -0600, Dave Peterson wrote:
>> I am trying to build a number of projects that use Python extensions on
>> Solaris 10 and I've discovered that nothing with extensions will link
>> unless I explicitly pass in a '-L/path/to/python/lib/dir' because
>> libpython2.5.so is not otherwise found when the '-lpython2.5' argument
>> is specified during linking.
> [...]
>> If that all seems correct, then it appears the issue is the
>> finalize_options() source in lib/distutils/commands/build_ext.py.  There
>> are a number of "if" blocks that explicitly append the value of
>> distutils.sysconfig.get_config_vars('LIBDIR') to the list of
>> library_dirs used to link built extensions with.  However, there doesn't
>> seem to be one of these for Solaris / sunos.
>
> Could you point to one of the projects you're having trouble with?  I
> haven't had any such problems with Python 2.5 on Solaris 10, distutils
> always finds the right things when I'm building extension modules.

It's not specific to any one project. Mostly, it seems to be tied to using a 
custom --prefix where $prefix/lib is not on the default library search path. Are 
you using such a custom --prefix?

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
  that is made terrible by our own mad attempt to interpret it as though it had
  an underlying truth."
   -- Umberto Eco


From ziade.tarek at gmail.com  Sat Jan 31 00:15:56 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Sat, 31 Jan 2009 00:15:56 +0100
Subject: [Distutils] [Python Language Summit] Distutils / Packaging
	survey
In-Reply-To: <871vuk1jdj.fsf@benfinney.id.au>
References: <49803752.9040704@ar.media.kyoto-u.ac.jp>
	<1233237184.8068.9.camel@tomoyo> <877i4dvl02.fsf@benfinney.id.au>
	<498232A0.8060306@palladion.com>
	<94bdd2610901291454s1eee0d0fvfbcb8d21d871e9ee@mail.gmail.com>
	<20090129233428.GA10050@laurie.devork>
	<94bdd2610901291621m10056e9bvf702474e866265cf@mail.gmail.com>
	<20090130183951.GA5119@laurie.devork>
	<20090130193447.89D7E3A40A0@sparrow.telecommunity.com>
	<871vuk1jdj.fsf@benfinney.id.au>
Message-ID: <94bdd2610901301515x7e79bd3ek2ac048cce8d0a6e4@mail.gmail.com>

On Sat, Jan 31, 2009 at 12:09 AM, Ben Finney <ben+python at benfinney.id.au> wrote:
> "P.J. Eby" <pje at telecommunity.com> writes:
>
>> At 06:39 PM 1/30/2009 +0000, Floris Bruynooghe wrote:
>> >I imagine things like libdir, prefix, datadir, docdir and other
>> >things copied from autoconf. Where the defaults would be something
>> >like:
>> >
>> >prefix = sys.prefix
>> >libdir = sys.prefix/lib/pythonX.Y/site-packages/pkgname
>> >datadir = sys.prefix/share/mypackage
>> >docdir = sys.prefix/share/doc/mypackage
>>
>> I'm confused by the above lines: do you mean the *project* name, or
>> the name of some package within the project? What if the project
>> contains no packages, only modules? What is libdir for?
>
> To clarify: Here again we come up against the unfortunate choice of
> the term "package" in Python to mean something confisingly different
> from what it means to most people dealing with software.
>
> I imagine Floris meant what most hackers mean by "package", which is
> what Python perversely calls a "distribution". That may or may not
> be what setuptools calls a "project", I've never been clear on that

let's all use this then maybe ?

http://wiki.python.org/moin/PythonPackagingTerminology



> :-/
>
> --
>  \                             "Everything is futile." ?Marvin of Borg |
>  `\                                                                   |
> _o__)                                                                  |
> Ben Finney
>
> _______________________________________________
> Distutils-SIG maillist  -  Distutils-SIG at python.org
> http://mail.python.org/mailman/listinfo/distutils-sig
>



-- 
Tarek Ziad? | Association AfPy | www.afpy.org
Blog FR | http://programmation-python.org
Blog EN | http://tarekziade.wordpress.com/

From ben+python at benfinney.id.au  Sat Jan 31 00:16:51 2009
From: ben+python at benfinney.id.au (Ben Finney)
Date: Sat, 31 Jan 2009 10:16:51 +1100
Subject: [Distutils] [Python Language Summit] Distutils / Packaging
	survey
References: <49803752.9040704@ar.media.kyoto-u.ac.jp>
	<49803BDF.4060204@ar.media.kyoto-u.ac.jp>
	<b654cd2e0901281116u57f8967dl785ffc09a0edf291@mail.gmail.com>
	<1233237184.8068.9.camel@tomoyo> <877i4dvl02.fsf@benfinney.id.au>
	<498232A0.8060306@palladion.com>
	<94bdd2610901291454s1eee0d0fvfbcb8d21d871e9ee@mail.gmail.com>
	<20090129233428.GA10050@laurie.devork>
	<94bdd2610901291621m10056e9bvf702474e866265cf@mail.gmail.com>
	<20090130183951.GA5119@laurie.devork>
	<b654cd2e0901301319k13974239oe3ac3cb874c02fe0@mail.gmail.com>
Message-ID: <87wsccz8nw.fsf@benfinney.id.au>

Ian Bicking <ianb at colorstudy.com> writes:

> On Fri, Jan 30, 2009 at 12:39 PM, Floris Bruynooghe <
> floris.bruynooghe at gmail.com> wrote:
> 
> > I imagine things like libdir, prefix, datadir, docdir and other
> > things copied from autoconf. Where the defaults would be something
> > like:
> >
> > prefix = sys.prefix
> > libdir = sys.prefix/lib/pythonX.Y/site-packages/pkgname
> > datadir = sys.prefix/share/mypackage
> > docdir = sys.prefix/share/doc/mypackage
> 
> I wouldn't want to use those. What goes in libdir, what goes in
> datadir?

Again, I expect Floris is using these terms in their traditional
meanings.

?library? would be a collection of executable code intended for use
as a unit; what Pythoncalls a ?package? (or the degerate case of a
?module?).

?data? would be non-executable files used by the package that don't
fit any other (e.g. ?documentation?) classification.

> I don't know, and frankly the distinctions start getting really
> arbitrary.

Hopefully that clarifies.

> I would rather see something like pkg_resources existing API, where
> there is some file that maps out how the local names of files (where
> they'd be in a checkout) map to their installed location, then the
> pkg_resources code could finds the real location of the file.

This loses the indirection that is so sorely needed of tagging
resources by *type*, so that their install location can be decided on
that basis.

Deciding on a file-by-file basis where files get installed is
something that distributors and packagers already have to do, and it's
a massive pain.

Allowing the developer to tag resources by type is a sensible division
of responsibility: the developer knows the *purpose* (and therefore
type) of the resource, and the packager knows the appropriate
*location* on the filesystem for resources of a particular type.

-- 
 \         ?If life deals you lemons, why not go kill someone with the |
  `\     lemons (maybe by shoving them down his throat).? ?Jack Handey |
_o__)                                                                  |
Ben Finney


From david.lyon at preisshare.net  Sat Jan 31 00:20:26 2009
From: david.lyon at preisshare.net (David Lyon)
Date: Fri, 30 Jan 2009 18:20:26 -0500
Subject: [Distutils] GUI for Python package Management
In-Reply-To: <8763jw1jmx.fsf@benfinney.id.au>
References: <4db42df7554839e3151045936e5b0ee4@preisshare.net>
	<7CCBA902-15CE-441A-BEE6-DD0972026530@gmail.com>
	<7edd5adaf7ff6650b908cfc76362a64a@preisshare.net>
	<49821912.7050806@stsci.edu>
	<80297AE9-6A91-4A0D-96C3-E9FD7317BA09@gmail.com>
	<49822DC8.7090901@stsci.edu>
	<dabf7bf09ede5ae956e60236de9d8c9f@preisshare.net>
	<87ljsttuq9.fsf@benfinney.id.au>
	<f6fdd83f5c5c1c6c81402f98666c8c6a@preisshare.net>
	<87zlh9sbej.fsf@benfinney.id.au> <49830411.2050104@taupro.com>
	<84748f63954c9b9d2f59c76506efa54e@preisshare.net>
	<8763jw1jmx.fsf@benfinney.id.au>
Message-ID: <d10f52aae1238bdc002a58ba21839e5a@preisshare.net>


Hi Ben,

Well I could argue with you all day... but my wife seems to have taken that
job..

Here is a screenshot of what I have done so far...

On Sat, 31 Jan 2009 10:03:50 +1100, Ben Finney <ben+python at benfinney.id.au>
wrote:
> David Lyon <david.lyon at preisshare.net> writes:
> 
>> In other languages, you just use the one tool..
> 
> That's misleading. In well-designed systems, you use any of a number
> of different *tools*, all of which use the same back-end *library
> interface* (which might effectively mean ?the same library? or
> ?mutliple libraries coded to the same interface?).
> 
> If the only workable back-end is a single tool, then that tool
> effectively becomes the back-end library and people bolt on multiple
> tools as front-ends to it. But that seems inferior to a true reusable
> library that is intended for use in implementing front-end tools.
> 
>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: PythonPackageManager.PNG
Type: image/png
Size: 20152 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20090130/ce5c5ac4/attachment-0001.png>

From ben+python at benfinney.id.au  Sat Jan 31 00:32:40 2009
From: ben+python at benfinney.id.au (Ben Finney)
Date: Sat, 31 Jan 2009 10:32:40 +1100
Subject: [Distutils] [Python Language Summit] Distutils / Packaging
	survey
References: <49803752.9040704@ar.media.kyoto-u.ac.jp>
	<1233237184.8068.9.camel@tomoyo> <877i4dvl02.fsf@benfinney.id.au>
	<498232A0.8060306@palladion.com>
	<94bdd2610901291454s1eee0d0fvfbcb8d21d871e9ee@mail.gmail.com>
	<20090129233428.GA10050@laurie.devork>
	<94bdd2610901291621m10056e9bvf702474e866265cf@mail.gmail.com>
	<20090130183951.GA5119@laurie.devork>
	<20090130193447.89D7E3A40A0@sparrow.telecommunity.com>
	<871vuk1jdj.fsf@benfinney.id.au>
	<94bdd2610901301515x7e79bd3ek2ac048cce8d0a6e4@mail.gmail.com>
Message-ID: <87skn0z7xj.fsf@benfinney.id.au>

Tarek Ziad? <ziade.tarek at gmail.com> writes:

> On Sat, Jan 31, 2009 at 12:09 AM, Ben Finney <ben+python at benfinney.id.au> wrote:
> > I imagine Floris meant what most hackers mean by "package", which
> > is what Python perversely calls a "distribution". That may or may
> > not be what setuptools calls a "project", I've never been clear on
> > that
> 
> let's all use this then maybe ?
> 
> http://wiki.python.org/moin/PythonPackagingTerminology

Who is ?us?? You and I know about those terms and their non-standard
meanings in Python, but how is anyone supposed to know who comes to
Python expecting things to mean what they mean in the majority of
other software projects?

Given the inertia on both sides of this terminological schism, and the
fact that these concepts are not exactly things that an outsider would
even expect to have to learn new terms for, I don't think it matters
how prominent such a ?Python packaging terminology? web page is. No
newcomer is going to be looking for it, and IMO rightly so.

The damage is done and, short of educating everyone the first time
they learn about software, we in the Python community have the burden
of teaching these non-standard meanings indefinitely every time the
conflicting meanings raise themselves.

-- 
 \       ?Pinky, are you pondering what I'm pondering?? ?Well, I think |
  `\   so (hiccup), but Kevin Costner with an English accent?? ?_Pinky |
_o__)                                                   and The Brain_ |
Ben Finney


From dpeterson at enthought.com  Sat Jan 31 00:45:26 2009
From: dpeterson at enthought.com (Dave Peterson)
Date: Fri, 30 Jan 2009 17:45:26 -0600
Subject: [Distutils] Solaris and distutils: Need to pass
 LIBDIR	explicitly with -L when building extensions?
In-Reply-To: <20090130230849.GE5606@laurie.devork>
References: <497E59C6.4020802@enthought.com>
	<20090130230849.GE5606@laurie.devork>
Message-ID: <49839116.2050801@enthought.com>

Floris Bruynooghe wrote:
> Hi Dave
>
> On Mon, Jan 26, 2009 at 06:48:06PM -0600, Dave Peterson wrote:
>   
>> I am trying to build a number of projects that use Python extensions on  
>> Solaris 10 and I've discovered that nothing with extensions will link  
>> unless I explicitly pass in a '-L/path/to/python/lib/dir' because  
>> libpython2.5.so is not otherwise found when the '-lpython2.5' argument  
>> is specified during linking.
>>     
> [...]
>   
>> If that all seems correct, then it appears the issue is the  
>> finalize_options() source in lib/distutils/commands/build_ext.py.  There  
>> are a number of "if" blocks that explicitly append the value of  
>> distutils.sysconfig.get_config_vars('LIBDIR') to the list of  
>> library_dirs used to link built extensions with.  However, there doesn't  
>> seem to be one of these for Solaris / sunos.
>>     
>
> Could you point to one of the projects you're having trouble with?  I
> haven't had any such problems with Python 2.5 on Solaris 10, distutils
> always finds the right things when I'm building extension modules.
>   

Pretty much everything I've tried that uses an extension has this 
problem.  Cython, Numpy, Traits, etc.  As Robert Kern pointed out, I'm 
using a custom built Python (Python 2.5.4) built and installed into a 
custom location via '--prefix'.   What Python are you using?

Just for grins, I've tried building Python with a number of other 
compilers, but all with a --prefix setting, and they've all exhibited 
this problem.  I'm leaning more and more toward this is actually a bug 
with the distutils source on Solaris.


-- Dave

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20090130/a75a6304/attachment.htm>

From ben+python at benfinney.id.au  Sat Jan 31 01:16:27 2009
From: ben+python at benfinney.id.au (Ben Finney)
Date: Sat, 31 Jan 2009 11:16:27 +1100
Subject: [Distutils] GUI for Python package Management
References: <4db42df7554839e3151045936e5b0ee4@preisshare.net>
	<7CCBA902-15CE-441A-BEE6-DD0972026530@gmail.com>
	<7edd5adaf7ff6650b908cfc76362a64a@preisshare.net>
	<49821912.7050806@stsci.edu>
	<80297AE9-6A91-4A0D-96C3-E9FD7317BA09@gmail.com>
	<49822DC8.7090901@stsci.edu>
	<dabf7bf09ede5ae956e60236de9d8c9f@preisshare.net>
	<87ljsttuq9.fsf@benfinney.id.au>
	<f6fdd83f5c5c1c6c81402f98666c8c6a@preisshare.net>
	<87zlh9sbej.fsf@benfinney.id.au> <49830411.2050104@taupro.com>
	<84748f63954c9b9d2f59c76506efa54e@preisshare.net>
	<8763jw1jmx.fsf@benfinney.id.au>
	<d10f52aae1238bdc002a58ba21839e5a@preisshare.net>
Message-ID: <87ocxoz5wk.fsf@benfinney.id.au>

David Lyon <david.lyon at preisshare.net> writes:

> Here is a screenshot of what I have done so far...

Please *do not* send binary attachments, or *any* large attachment,
unsolicited to a mailing list.

Especially not if you're sending *antoher* copy to me personally.

Instead, pop it on a free hosting site and give us a URL; then we can
decide whether and when the file arrives at our computer.

-- 
 \          ?All good things are cheap; all bad are very dear.? ?Henry |
  `\                                                     David Thoreau |
_o__)                                                                  |
Ben Finney


From akitada at gmail.com  Sat Jan 31 07:41:11 2009
From: akitada at gmail.com (Akira Kitada)
Date: Sat, 31 Jan 2009 15:41:11 +0900
Subject: [Distutils] [Python Language Summit] Distutils / Packaging
	survey
In-Reply-To: <94bdd2610901301135v1e88cadehc25fdb9472329912@mail.gmail.com>
References: <49803752.9040704@ar.media.kyoto-u.ac.jp>
	<b654cd2e0901281116u57f8967dl785ffc09a0edf291@mail.gmail.com>
	<1233237184.8068.9.camel@tomoyo> <877i4dvl02.fsf@benfinney.id.au>
	<498232A0.8060306@palladion.com>
	<94bdd2610901291454s1eee0d0fvfbcb8d21d871e9ee@mail.gmail.com>
	<20090129233428.GA10050@laurie.devork>
	<94bdd2610901291621m10056e9bvf702474e866265cf@mail.gmail.com>
	<20090130183951.GA5119@laurie.devork>
	<94bdd2610901301135v1e88cadehc25fdb9472329912@mail.gmail.com>
Message-ID: <90bb445a0901302241m492a3680t6bb933961741577c@mail.gmail.com>

Added to proposals page.
http://wiki.python.org/moin/Distutils/Proposals

On Sat, Jan 31, 2009 at 4:35 AM, Tarek Ziad? <ziade.tarek at gmail.com> wrote:
> On Fri, Jan 30, 2009 at 7:39 PM, Floris Bruynooghe
> <floris.bruynooghe at gmail.com> wrote:
>> On Fri, Jan 30, 2009 at 01:21:12AM +0100, Tarek Ziad? wrote:
>>> On Fri, Jan 30, 2009 at 12:34 AM, Floris Bruynooghe
>>> <floris.bruynooghe at gmail.com> wrote:
>>> > An argument against a generated .py file is that this won't work for
>>> > single-module python distributions.  But having a .cfg file or
>>> > something next to the module/package might defeat the point of trying
>>> > to help the FHS in violating it already.  OTOH .egg-info does that too
>>> > and seems to be accepted currently.
>>>
>>> Well, the egg-info file is the static version of setup.py metadata in some way,
>>> I would find a new file redundant.
>>>
>>> I am really curious though, to see what the file would contain..
>>
>> I imagine things like libdir, prefix, datadir, docdir and other things
>> copied from autoconf.  Where the defaults would be something like:
>>
>> prefix = sys.prefix
>> libdir = sys.prefix/lib/pythonX.Y/site-packages/pkgname
>> datadir = sys.prefix/share/mypackage
>> docdir = sys.prefix/share/doc/mypackage
>>
>
> David Cournapeau has sent me a first draft of a document, you might
> want to look at.
>
> http://wiki.python.org/moin/distutils-autoconf
>
> Might be a good basis for a common understanding
>
>
>
> --
> Tarek Ziad? | Association AfPy | www.afpy.org
> Blog FR | http://programmation-python.org
> Blog EN | http://tarekziade.wordpress.com/
> _______________________________________________
> Distutils-SIG maillist  -  Distutils-SIG at python.org
> http://mail.python.org/mailman/listinfo/distutils-sig
>

From david at ar.media.kyoto-u.ac.jp  Sat Jan 31 09:40:44 2009
From: david at ar.media.kyoto-u.ac.jp (David Cournapeau)
Date: Sat, 31 Jan 2009 17:40:44 +0900
Subject: [Distutils] [Python Language Summit] Distutils /
	Packaging	survey
In-Reply-To: <b654cd2e0901301319k13974239oe3ac3cb874c02fe0@mail.gmail.com>
References: <49803752.9040704@ar.media.kyoto-u.ac.jp>	<49803BDF.4060204@ar.media.kyoto-u.ac.jp>	<b654cd2e0901281116u57f8967dl785ffc09a0edf291@mail.gmail.com>	<1233237184.8068.9.camel@tomoyo>
	<877i4dvl02.fsf@benfinney.id.au>	<498232A0.8060306@palladion.com>	<94bdd2610901291454s1eee0d0fvfbcb8d21d871e9ee@mail.gmail.com>	<20090129233428.GA10050@laurie.devork>	<94bdd2610901291621m10056e9bvf702474e866265cf@mail.gmail.com>	<20090130183951.GA5119@laurie.devork>
	<b654cd2e0901301319k13974239oe3ac3cb874c02fe0@mail.gmail.com>
Message-ID: <49840E8C.1090302@ar.media.kyoto-u.ac.jp>

Ian Bicking wrote:
> On Fri, Jan 30, 2009 at 12:39 PM, Floris Bruynooghe
> <floris.bruynooghe at gmail.com <mailto:floris.bruynooghe at gmail.com>> wrote:
>
>     I imagine things like libdir, prefix, datadir, docdir and other things
>     copied from autoconf.  Where the defaults would be something like:
>
>     prefix = sys.prefix
>     libdir = sys.prefix/lib/pythonX.Y/site-packages/pkgname
>     datadir = sys.prefix/share/mypackage
>     docdir = sys.prefix/share/doc/mypackage
>
>
> I wouldn't want to use those.  What goes in libdir, what goes in
> datadir?  I don't know, and frankly the distinctions start getting
> really arbitrary.

They are not arbitrary - they come from standard usage and have a
rationale, at least on Unix (datadir for arch independent, and libdir
for arch dependent, to simplify).

But you mostly do not need to care, as a developer: .py files would be
considered as data files, extensions as arch-dependent, etc... The main
category which needs special care is documentation, and I think I am not
the only one thinking that's one thing missing in distutils ATM.
 
>
> I would rather see something like pkg_resources existing API, where
> there is some file that maps out how the local names of files (where
> they'd be in a checkout) map to their installed location, then the
> pkg_resources code could finds the real location of the file.

I am not sure I understand how this would help OS packagers - this does
not sound as the same problem at all.

David

From tseaver at palladion.com  Sat Jan 31 10:00:08 2009
From: tseaver at palladion.com (Tres Seaver)
Date: Sat, 31 Jan 2009 04:00:08 -0500
Subject: [Distutils] [Python Language Summit] Distutils / Packaging
	survey
In-Reply-To: <87wsccz8nw.fsf@benfinney.id.au>
References: <49803752.9040704@ar.media.kyoto-u.ac.jp>	<49803BDF.4060204@ar.media.kyoto-u.ac.jp>	<b654cd2e0901281116u57f8967dl785ffc09a0edf291@mail.gmail.com>	<1233237184.8068.9.camel@tomoyo>
	<877i4dvl02.fsf@benfinney.id.au>	<498232A0.8060306@palladion.com>	<94bdd2610901291454s1eee0d0fvfbcb8d21d871e9ee@mail.gmail.com>	<20090129233428.GA10050@laurie.devork>	<94bdd2610901291621m10056e9bvf702474e866265cf@mail.gmail.com>	<20090130183951.GA5119@laurie.devork>	<b654cd2e0901301319k13974239oe3ac3cb874c02fe0@mail.gmail.com>
	<87wsccz8nw.fsf@benfinney.id.au>
Message-ID: <49841318.6000800@palladion.com>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Ben Finney wrote:
> Ian Bicking <ianb at colorstudy.com> writes:
> 
>> On Fri, Jan 30, 2009 at 12:39 PM, Floris Bruynooghe <
>> floris.bruynooghe at gmail.com> wrote:
>>
>>> I imagine things like libdir, prefix, datadir, docdir and other
>>> things copied from autoconf. Where the defaults would be something
>>> like:
>>>
>>> prefix = sys.prefix
>>> libdir = sys.prefix/lib/pythonX.Y/site-packages/pkgname
>>> datadir = sys.prefix/share/mypackage
>>> docdir = sys.prefix/share/doc/mypackage
>> I wouldn't want to use those. What goes in libdir, what goes in
>> datadir?
> 
> Again, I expect Floris is using these terms in their traditional
> meanings.
> 
> ?library? would be a collection of executable code intended for use
> as a unit; what Pythoncalls a ?package? (or the degerate case of a
> ?module?).
> 
> ?data? would be non-executable files used by the package that don't
> fit any other (e.g. ?documentation?) classification.
> 
>> I don't know, and frankly the distinctions start getting really
>> arbitrary.
> 
> Hopefully that clarifies.
> 
>> I would rather see something like pkg_resources existing API, where
>> there is some file that maps out how the local names of files (where
>> they'd be in a checkout) map to their installed location, then the
>> pkg_resources code could finds the real location of the file.
> 
> This loses the indirection that is so sorely needed of tagging
> resources by *type*, so that their install location can be decided on
> that basis.
> 
> Deciding on a file-by-file basis where files get installed is
> something that distributors and packagers already have to do, and it's
> a massive pain.
>
> Allowing the developer to tag resources by type is a sensible division
> of responsibility: the developer knows the *purpose* (and therefore
> type) of the resource, and the packager knows the appropriate
> *location* on the filesystem for resources of a particular type.

The need for those distinctions derives from a particular distributor's
/ packager's "religion":  e.g., the insistance that "template" files
(software, but not "executable" by some criterion) mut not be kept in
the same directory with the other software.  That rule is important for
some packagers, and completely mystifying / irrelevant to the original
develpoers or to packagers with a different model.

Right now, making the 'pkg_resources' jump can be motivated by
cross-platform concerns, e.g. a desire to allow the "package_data" files
to be loaded from a zipfile / egg.  Anything more complicated at
*runtime* (as opposed to metadata added to the 'sdist') is harder to
motivate.

The "uptake" problem here is both to motivate the non-believer to help
with the distinctions, and to make the burden of doing so as minimal as
possible.  This discussion has broken down before at exactly this point,
IIRC.

As a packaging-agnostic developer, I'm +0 on the idea of making it
easier to package my software:

- - I do appreciate having folks able to install the software easily;

- - However, complexifying the software itself to make some packagers
  happy is a burden, and a potential source of bugs;

- - Sometimes, packagesrs *break* my software in ways I can't support.

If we can drive the first concern down, I can live better with the
second (and perhaps it will become less likely).


Tres.
- --
===================================================================
Tres Seaver          +1 540-429-0999          tseaver at palladion.com
Palladion Software   "Excellence by Design"    http://palladion.com
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFJhBMY+gerLs4ltQ4RAl5CAKCHP1Y3ja65uQZGRowdu8C+R2lcJwCgmNbV
JjXgoibfZumIRJjYmlsr9XE=
=//L/
-----END PGP SIGNATURE-----


From david at ar.media.kyoto-u.ac.jp  Sat Jan 31 09:58:26 2009
From: david at ar.media.kyoto-u.ac.jp (David Cournapeau)
Date: Sat, 31 Jan 2009 17:58:26 +0900
Subject: [Distutils] [Python Language Summit] Distutils /
	Packaging	survey
In-Reply-To: <49841318.6000800@palladion.com>
References: <49803752.9040704@ar.media.kyoto-u.ac.jp>	<49803BDF.4060204@ar.media.kyoto-u.ac.jp>	<b654cd2e0901281116u57f8967dl785ffc09a0edf291@mail.gmail.com>	<1233237184.8068.9.camel@tomoyo>	<877i4dvl02.fsf@benfinney.id.au>	<498232A0.8060306@palladion.com>	<94bdd2610901291454s1eee0d0fvfbcb8d21d871e9ee@mail.gmail.com>	<20090129233428.GA10050@laurie.devork>	<94bdd2610901291621m10056e9bvf702474e866265cf@mail.gmail.com>	<20090130183951.GA5119@laurie.devork>	<b654cd2e0901301319k13974239oe3ac3cb874c02fe0@mail.gmail.com>	<87wsccz8nw.fsf@benfinney.id.au>
	<49841318.6000800@palladion.com>
Message-ID: <498412B2.6060703@ar.media.kyoto-u.ac.jp>

Tres Seaver wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> Ben Finney wrote:
>   
>> Ian Bicking <ianb at colorstudy.com> writes:
>>
>>     
>>> On Fri, Jan 30, 2009 at 12:39 PM, Floris Bruynooghe <
>>> floris.bruynooghe at gmail.com> wrote:
>>>
>>>       
>>>> I imagine things like libdir, prefix, datadir, docdir and other
>>>> things copied from autoconf. Where the defaults would be something
>>>> like:
>>>>
>>>> prefix = sys.prefix
>>>> libdir = sys.prefix/lib/pythonX.Y/site-packages/pkgname
>>>> datadir = sys.prefix/share/mypackage
>>>> docdir = sys.prefix/share/doc/mypackage
>>>>         
>>> I wouldn't want to use those. What goes in libdir, what goes in
>>> datadir?
>>>       
>> Again, I expect Floris is using these terms in their traditional
>> meanings.
>>
>> ?library? would be a collection of executable code intended for use
>> as a unit; what Pythoncalls a ?package? (or the degerate case of a
>> ?module?).
>>
>> ?data? would be non-executable files used by the package that don't
>> fit any other (e.g. ?documentation?) classification.
>>
>>     
>>> I don't know, and frankly the distinctions start getting really
>>> arbitrary.
>>>       
>> Hopefully that clarifies.
>>
>>     
>>> I would rather see something like pkg_resources existing API, where
>>> there is some file that maps out how the local names of files (where
>>> they'd be in a checkout) map to their installed location, then the
>>> pkg_resources code could finds the real location of the file.
>>>       
>> This loses the indirection that is so sorely needed of tagging
>> resources by *type*, so that their install location can be decided on
>> that basis.
>>
>> Deciding on a file-by-file basis where files get installed is
>> something that distributors and packagers already have to do, and it's
>> a massive pain.
>>
>> Allowing the developer to tag resources by type is a sensible division
>> of responsibility: the developer knows the *purpose* (and therefore
>> type) of the resource, and the packager knows the appropriate
>> *location* on the filesystem for resources of a particular type.
>>     
>
> The need for those distinctions derives from a particular distributor's
> / packager's "religion":  e.g., the insistance that "template" files
> (software, but not "executable" by some criterion) mut not be kept in
> the same directory with the other software.  That rule is important for
> some packagers, and completely mystifying / irrelevant to the original
> develpoers or to packagers with a different model.
>   

That's not a religion: it has rationales. You may not agree with them,
both those splits are not arbitrary. Each platform has its own way of
dealing with this, and doing against it is both futile and
counter-productive.

Of course some of the distributions aspects are irrelevant to you - but
so are some of your interests to packagers. At the risk of sounding
obvious: developers and packagers do not have the same goals, and they
sometimes conflict. But you cannot expect "not to care" and "my software
will be available everywhere" at the same time. That's why those
discussion are tiresome and difficult: there has to be some compromise.
There is not right solution,

David

From ziade.tarek at gmail.com  Sat Jan 31 11:15:48 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Sat, 31 Jan 2009 11:15:48 +0100
Subject: [Distutils] [Python Language Summit] Distutils / Packaging
	survey
In-Reply-To: <87skn0z7xj.fsf@benfinney.id.au>
References: <49803752.9040704@ar.media.kyoto-u.ac.jp>
	<498232A0.8060306@palladion.com>
	<94bdd2610901291454s1eee0d0fvfbcb8d21d871e9ee@mail.gmail.com>
	<20090129233428.GA10050@laurie.devork>
	<94bdd2610901291621m10056e9bvf702474e866265cf@mail.gmail.com>
	<20090130183951.GA5119@laurie.devork>
	<20090130193447.89D7E3A40A0@sparrow.telecommunity.com>
	<871vuk1jdj.fsf@benfinney.id.au>
	<94bdd2610901301515x7e79bd3ek2ac048cce8d0a6e4@mail.gmail.com>
	<87skn0z7xj.fsf@benfinney.id.au>
Message-ID: <94bdd2610901310215n418daaefjdb78f09bee8ddb40@mail.gmail.com>

On Sat, Jan 31, 2009 at 12:32 AM, Ben Finney <ben+python at benfinney.id.au> wrote:
> Tarek Ziad? <ziade.tarek at gmail.com> writes:
>
>> On Sat, Jan 31, 2009 at 12:09 AM, Ben Finney <ben+python at benfinney.id.au> wrote:
>> > I imagine Floris meant what most hackers mean by "package", which
>> > is what Python perversely calls a "distribution". That may or may
>> > not be what setuptools calls a "project", I've never been clear on
>> > that
>>
>> let's all use this then maybe ?
>>
>> http://wiki.python.org/moin/PythonPackagingTerminology
>
> Who is "us"?

All the people in this thread and all the people that are concerned
by packaging matters. Also the problem is, even inside the community,
people don't use
the same terms or don't understand them

ask a python developer what is an "egg" for example.

> You and I know about those terms and their non-standard
> meanings in Python, but how is anyone supposed to know who comes to
> Python expecting things to mean what they mean in the majority of
> other software projects?
>
> Given the inertia on both sides of this terminological schism, and the
> fact that these concepts are not exactly things that an outsider would
> even expect to have to learn new terms for, I don't think it matters
> how prominent such a "Python packaging terminology" web page is. No
> newcomer is going to be looking for it, and IMO rightly so.

For the newcomers:

Well, between reading some thread where there is +100 mails
and reading 4 lines of a definition for each word  in a FAQ or Taxonomy...

For the non-newcomers:

Plus, last time we had a lot of discussions on packaging matters here, nothing
came out really. And I can already read things in this thread that were
said many times by the same persons in the past.

>
> The damage is done and, short of educating everyone the first time
> they learn about software, we in the Python community have the burden
> of teaching these non-standard meanings indefinitely every time the
> conflicting meanings raise themselves.

Maybe so,

Cheers,
Tarek

From ben+python at benfinney.id.au  Sat Jan 31 12:33:41 2009
From: ben+python at benfinney.id.au (Ben Finney)
Date: Sat, 31 Jan 2009 22:33:41 +1100
Subject: [Distutils] [Python Language Summit] Distutils / Packaging
	survey
References: <49803752.9040704@ar.media.kyoto-u.ac.jp>
	<498232A0.8060306@palladion.com>
	<94bdd2610901291454s1eee0d0fvfbcb8d21d871e9ee@mail.gmail.com>
	<20090129233428.GA10050@laurie.devork>
	<94bdd2610901291621m10056e9bvf702474e866265cf@mail.gmail.com>
	<20090130183951.GA5119@laurie.devork>
	<20090130193447.89D7E3A40A0@sparrow.telecommunity.com>
	<871vuk1jdj.fsf@benfinney.id.au>
	<94bdd2610901301515x7e79bd3ek2ac048cce8d0a6e4@mail.gmail.com>
	<87skn0z7xj.fsf@benfinney.id.au>
	<94bdd2610901310215n418daaefjdb78f09bee8ddb40@mail.gmail.com>
Message-ID: <87iqnv676y.fsf@benfinney.id.au>

Tarek Ziad? <ziade.tarek at gmail.com> writes:

> On Sat, Jan 31, 2009 at 12:32 AM, Ben Finney <ben+python at benfinney.id.au> wrote:
> > Given the inertia on both sides of this terminological schism, and
> > the fact that these concepts are not exactly things that an
> > outsider would even expect to have to learn new terms for, I don't
> > think it matters how prominent such a "Python packaging
> > terminology" web page is. No newcomer is going to be looking for
> > it, and IMO rightly so.
> 
> For the newcomers:
> 
> Well, between reading some thread where there is +100 mails and
> reading 4 lines of a definition for each word in a FAQ or
> Taxonomy...

I'm not sure what your point is with that paragraph; I presume you're
saying that a short online definition is quicker to read than a long
discussion thread.

If so, I've already addressed that: The online definition document
won't *be* read by the representative newcomer until it's already
caused them confusion, most likely in a long discussion thread. That's
because it won't even occur to most of them that Python would be
perverse enough to use these terms in a contradictory meaning to the
majority of the software world.

And, in my opinion, they're right to have that expectation. To think
otherwise is to blame the victim. From that, it follows that we can't
expect the newcomer to have any awareness that this terminological
issue even *exists* until it bites them.

-- 
 \     ?I went to a fancy French restaurant called ?D?j? Vu?. The head |
  `\                  waiter said, ?Don't I know you??? ?Steven Wright |
_o__)                                                                  |
Ben Finney


From floris.bruynooghe at gmail.com  Sat Jan 31 15:01:43 2009
From: floris.bruynooghe at gmail.com (Floris Bruynooghe)
Date: Sat, 31 Jan 2009 14:01:43 +0000
Subject: [Distutils] Solaris and distutils: Need to pass
	LIBDIR	explicitly with -L when building extensions?
In-Reply-To: <49839116.2050801@enthought.com>
References: <497E59C6.4020802@enthought.com>
	<20090130230849.GE5606@laurie.devork>
	<49839116.2050801@enthought.com>
Message-ID: <20090131140143.GA7382@laurie.devork>

On Fri, Jan 30, 2009 at 05:45:26PM -0600, Dave Peterson wrote:
> Floris Bruynooghe wrote:
>> On Mon, Jan 26, 2009 at 06:48:06PM -0600, Dave Peterson wrote:
>>   
>>> I am trying to build a number of projects that use Python extensions 
>>> on  Solaris 10 and I've discovered that nothing with extensions will 
>>> link  unless I explicitly pass in a '-L/path/to/python/lib/dir' 
>>> because  libpython2.5.so is not otherwise found when the 
>>> '-lpython2.5' argument  is specified during linking.
>>>     
>> [...]
>>   
>>> If that all seems correct, then it appears the issue is the   
>>> finalize_options() source in lib/distutils/commands/build_ext.py.  
>>> There  are a number of "if" blocks that explicitly append the value 
>>> of  distutils.sysconfig.get_config_vars('LIBDIR') to the list of   
>>> library_dirs used to link built extensions with.  However, there 
>>> doesn't  seem to be one of these for Solaris / sunos.
>>>     
>>
>> Could you point to one of the projects you're having trouble with?  I
>> haven't had any such problems with Python 2.5 on Solaris 10, distutils
>> always finds the right things when I'm building extension modules.
>>   
>
> Pretty much everything I've tried that uses an extension has this  
> problem.  Cython, Numpy, Traits, etc.  As Robert Kern pointed out, I'm  
> using a custom built Python (Python 2.5.4) built and installed into a  
> custom location via '--prefix'.   What Python are you using?

Ah, I get it now.  I did cut out the important part: you build with
--enable-shared.  I'm also using a custom build Python with --prefix
but don't use --enable-shared so don't get this problem.

> I'm leaning more and more toward this is actually a bug  
> with the distutils source on Solaris.

Yes, I agree now that it is a bug in distutils and I agree with your
fix.  The if statement should check both that it is SunOS and that it
is using a shared python, just like the linux one.  If fact the linux
one could just be modified:

if (sys.paltform.startswith('linux') \
    or sys.platform.startswith('gnu') \
    or sys.platform.startswith('sunos')) \
        and sysconfig.get_config_var('Py_ENABLE_SHARED'):
    ...

I'll leave the honours of reporting the bug to you if you agree with
this.

Lastly out of curiosity: why --enable-shared?  Do you embed python?
Or are there other reasons to use it?

Regards
Floris

-- 
Debian GNU/Linux -- The Power of Freedom
www.debian.org | www.gnu.org | www.kernel.org

From floris.bruynooghe at gmail.com  Sat Jan 31 15:10:28 2009
From: floris.bruynooghe at gmail.com (Floris Bruynooghe)
Date: Sat, 31 Jan 2009 14:10:28 +0000
Subject: [Distutils] [Python Language Summit] Distutils /
	Packaging	survey
In-Reply-To: <20090130230236.GD5606@laurie.devork>
References: <b654cd2e0901281116u57f8967dl785ffc09a0edf291@mail.gmail.com>
	<1233237184.8068.9.camel@tomoyo> <877i4dvl02.fsf@benfinney.id.au>
	<498232A0.8060306@palladion.com>
	<94bdd2610901291454s1eee0d0fvfbcb8d21d871e9ee@mail.gmail.com>
	<20090129233428.GA10050@laurie.devork>
	<94bdd2610901291621m10056e9bvf702474e866265cf@mail.gmail.com>
	<20090130183951.GA5119@laurie.devork>
	<b654cd2e0901301319k13974239oe3ac3cb874c02fe0@mail.gmail.com>
	<20090130230236.GD5606@laurie.devork>
Message-ID: <20090131141028.GA8616@laurie.devork>

On Fri, Jan 30, 2009 at 11:02:36PM +0000, Floris Bruynooghe wrote:
> Ok, that was supposed to read:
> 
> prefix = sys.prefix
> libdir = sys.prefix/lib/pythonX.Y/site-packages/pkgname
> datadir = sys.prefix/share/mypackage
> docdir = sys.prefix/share/doc/mypackage

Just how many time will I get this wrong??

libdir = sys.prefix/lib/pythonX.Y/site-packages
datadir = sys.prefix/share/<project_name>
docdir = sys.prefix/share/doc/<project_name>


sorry
Floris

From pje at telecommunity.com  Sat Jan 31 16:37:56 2009
From: pje at telecommunity.com (P.J. Eby)
Date: Sat, 31 Jan 2009 10:37:56 -0500
Subject: [Distutils] [Python Language Summit] Distutils /
 Packaging	survey
In-Reply-To: <49840E8C.1090302@ar.media.kyoto-u.ac.jp>
References: <49803752.9040704@ar.media.kyoto-u.ac.jp>
	<49803BDF.4060204@ar.media.kyoto-u.ac.jp>
	<b654cd2e0901281116u57f8967dl785ffc09a0edf291@mail.gmail.com>
	<1233237184.8068.9.camel@tomoyo> <877i4dvl02.fsf@benfinney.id.au>
	<498232A0.8060306@palladion.com>
	<94bdd2610901291454s1eee0d0fvfbcb8d21d871e9ee@mail.gmail.com>
	<20090129233428.GA10050@laurie.devork>
	<94bdd2610901291621m10056e9bvf702474e866265cf@mail.gmail.com>
	<20090130183951.GA5119@laurie.devork>
	<b654cd2e0901301319k13974239oe3ac3cb874c02fe0@mail.gmail.com>
	<49840E8C.1090302@ar.media.kyoto-u.ac.jp>
Message-ID: <20090131153556.407903A4114@sparrow.telecommunity.com>

At 05:40 PM 1/31/2009 +0900, David Cournapeau wrote:
>Ian Bicking wrote:
> > On Fri, Jan 30, 2009 at 12:39 PM, Floris Bruynooghe
> > I wouldn't want to use those.  What goes in libdir, what goes in
> > datadir?  I don't know, and frankly the distinctions start getting
> > really arbitrary.
>
>They are not arbitrary - they come from standard usage and have a
>rationale, at least on Unix (datadir for arch independent, and libdir
>for arch dependent, to simplify).
>
>But you mostly do not need to care, as a developer: .py files would be
>considered as data files, extensions as arch-dependent, etc...

If this is true, then there's no need to distinguish between .py 
files and any other data files - they both belong in /share to begin 
with, not in /lib.  Or else they ALL belong in /lib.  The entire "FHS 
demands they be split" concept is wrong from the get-go, under that 
interpretation.


From zooko at zooko.com  Sat Jan 31 16:44:05 2009
From: zooko at zooko.com (zooko)
Date: Sat, 31 Jan 2009 08:44:05 -0700
Subject: [Distutils] [Python Language Summit] Distutils
	/	Packaging	survey
In-Reply-To: <1233238219.8068.25.camel@tomoyo>
References: <94bdd2610901250306n1e27a928sa0b276d91f80862a@mail.gmail.com>
	<90bb445a0901270829y7019eca3p72164c259c4d35d4@mail.gmail.com>
	<94bdd2610901271400k32ff74edmea94fcbdfce88f53@mail.gmail.com>
	<515B9271-0E7D-4B4E-B63F-F717604A5FEA@zooko.com>
	<49803752.9040704@ar.media.kyoto-u.ac.jp>
	<C34F2C72-B619-4A46-B4FA-F4FDB41A5EB2@zooko.com>
	<1233238219.8068.25.camel@tomoyo>
Message-ID: <F6592B9D-ADD2-4709-8759-1D2B755BB5A2@zooko.com>

Thanks for the information, Josselin:

On Jan 29, 2009, at 7:10 AM, Josselin Mouette wrote:

> Le mercredi 28 janvier 2009 ? 07:44 -0700, zooko a ?crit :
>> 3.  It would be okay for this process to be automated (or semi- 
>> automated), but there's some flaw in the design of stdeb which  
>> means it will never be able to do it right unless stdeb is  
>> rewritten with a new design.
>
> This is the one. BTW, I don?t consider this a flaw in stdeb, it?s  
> just that stdeb was not designed with the goal to produce packages  
> suitable for Debian itself.
>
> What we need is the equivalent of dh_make_perl [0]. That is, a  
> script that will generate the debian/ structure in a semi-automated  
> fashion, leading to a package ready to be installed after minor  
> tweaks and a human?s review. Bonus points would go for providing a  
> script suggesting changes in the description and/or dependencies  
> when updating the package for a new upstream release.

A-ha!  I think I understand the disagreement now!  It hinges on the  
subtle distinction between "manual", "automated", and "semi-automated".

Unless I'm misunderstanding something (which is quite possible),  
stdeb already does exactly what you just suggested.  It produces a  
debian/ subdirectory and a .dsc file, which you can them feed into  
"dpkg-buildpackage" to produce a .deb.  You can, of course, inspect  
and modify those files after stdeb produced then and before dpkg- 
buildpackage consumes them.

http://github.com/astraw/stdeb/tree/master

Perhaps there is some confusion on this point because I like to  
*talk* about stdeb as though it sucks in Python source trees and  
spits out .deb's.  That's how I like to use it -- I try not to look  
at or change the .dsc or debian/ files.  However, there's no  
fundamental reason that I am aware of that it couldn't be used by a  
real Debian developer to ease his task of producing completely Policy- 
Compliant, high-quality Debian packages.

Apparently the Perl and Haskell Debian developers have already  
started using semi-automation this way to maintain large numbers of  
Policy-Compliant Perl and Haskell .deb's produced from Perl and  
Haskell source trees.

If you, or anyone, tries this, please post to this list and also Cc:  
me (the list is just too high-volume for me to keep up :-() and I'll  
try to help.

Regards,

Zooko
---
Tahoe, the Least-Authority Filesystem -- http://allmydata.org
store your data: $10/month -- http://allmydata.com/?tracking=zsig


From akitada at gmail.com  Sat Jan 31 17:31:59 2009
From: akitada at gmail.com (Akira Kitada)
Date: Sun, 1 Feb 2009 01:31:59 +0900
Subject: [Distutils] [ANN] Reorganized Distutils Wiki, Cookbook,
	Proposal Index, FAQ and 	more
Message-ID: <90bb445a0901310831j58d7f509s528019d0fde50a8d@mail.gmail.com>

Hi,

I reorganized Distutils page on wiki.python.org.
Here are some of the contents.

    * http://wiki.python.org/moin/Distutils
    * http://wiki.python.org/moin/Distutils/Cookbook
    * http://wiki.python.org/moin/Distutils/Proposals
    * http://wiki.python.org/moin/Distutils/FAQ

From floris.bruynooghe at gmail.com  Sat Jan 31 17:58:47 2009
From: floris.bruynooghe at gmail.com (Floris Bruynooghe)
Date: Sat, 31 Jan 2009 16:58:47 +0000
Subject: [Distutils] [Python Language Summit] Distutils /
	Packaging	survey
In-Reply-To: <20090131153556.407903A4114@sparrow.telecommunity.com>
References: <1233237184.8068.9.camel@tomoyo> <877i4dvl02.fsf@benfinney.id.au>
	<498232A0.8060306@palladion.com>
	<94bdd2610901291454s1eee0d0fvfbcb8d21d871e9ee@mail.gmail.com>
	<20090129233428.GA10050@laurie.devork>
	<94bdd2610901291621m10056e9bvf702474e866265cf@mail.gmail.com>
	<20090130183951.GA5119@laurie.devork>
	<b654cd2e0901301319k13974239oe3ac3cb874c02fe0@mail.gmail.com>
	<49840E8C.1090302@ar.media.kyoto-u.ac.jp>
	<20090131153556.407903A4114@sparrow.telecommunity.com>
Message-ID: <20090131165847.GA13200@laurie.devork>

On Sat, Jan 31, 2009 at 10:37:56AM -0500, P.J. Eby wrote:
> At 05:40 PM 1/31/2009 +0900, David Cournapeau wrote:
>> But you mostly do not need to care, as a developer: .py files would be
>> considered as data files, extensions as arch-dependent, etc...
>
> If this is true, then there's no need to distinguish between .py files 
> and any other data files - they both belong in /share to begin with, not 
> in /lib.  Or else they ALL belong in /lib.  The entire "FHS demands they 
> be split" concept is wrong from the get-go, under that interpretation.

I don't think trying to split off .py and .so files is a good idea.
Python will just break because of namespace package issues etc.  I
recon all .py/.pyc/.pyo/.so/.pyd files should be treated as arch
dependent, and other data files like templates and INI files or
anything else you have should be tagged as data files.  Then there's
of course tagging things as binaries, documentation, configuration
and state files etc.

Regards
Floris

-- 
Debian GNU/Linux -- The Power of Freedom
www.debian.org | www.gnu.org | www.kernel.org

From zookog at gmail.com  Sat Jan 31 18:22:03 2009
From: zookog at gmail.com (Zooko O'Whielacronx)
Date: Sat, 31 Jan 2009 10:22:03 -0700
Subject: [Distutils] (no subject)
Message-ID: <cd6401a0901310922x1fd55d69u7e12ab546c676315@mail.gmail.com>

 Floris Bruynooghe <floris.bruynooghe at gmail.com> writes:
>
> In my ideal world python package developers would only ever have to
> worry about uploading an sdist.
...
> If I am correct the only reason people started to go away from this is
> because Windows users don't tend to have compilers installed.

I recently made this transition.  For Tahoe-1.2.0 I tried to make sdists
the canonical, preferred format (not, of course, omitting other formats
such as .deb's).

It turns out that using sdists as the primary, preferred format, failed
in a lot of cases.  A huge number of cases is that there is no C
compiler or no Python.h -- not only on Windows but increasingly on
Linux.  My sons's OLPCs, for example, come with Python as the standard,
integrated programming language, but not with a C compiler -- they can't
afford the storage space!

Many Python packages -- including three or four of the ones used by
Tahoe -- have optional C/C++ extension modules which are only for CPU
optimization or which are only for specific platforms or uses.  This
ticket is to change distutils so that these modules can be marked as
optional and building the package can finish even when the compilation
of the extension fails:

http://bugs.python.org/issue4706

Another large number of cases was that some Python packages require
special environmental customization, such as pyOpenSSL, which requires
that OpenSSL binaries have been installed in just the right way before
pyOpenSSL's build will work.  (You would be surprised at just how much
work it is to get OpenSSL installed in just the right way so that
pyOpenSSL can build -- the pyOpenSSL maintainers still haven't
accomplished it on Windows.)

Then there is the problem that compiling certain packages takes way too
long, such as my own pycryptopp package, which takes minutes to build
(compiling a lot of C++ files from Wei Dai's Crypto++ library).

Finally, there is the problem that a large number of users refused to
*try* the sdist install because they (more or less correctly) assumed
that it would require a lot of manual administration of build tools on
their part and would fail on their particular platform.

So, for the imminent Tahoe-1.3.0 release, I'm trying to make bdist_eggs
be the default, preferred format for any packages which include C/C++
extensions, and sdist for pure-Python packages (not, of course,
excluding other formats such as .deb's).  Here is the decentralized,
secure directory where I'm accumulating these eggs:

http://testgrid.allmydata.org:3567/uri/URI:DIR2-RO:snrfwfxatrci35zdgjnzxxx2ke:unarxv347edtku3xzmefy4mcdmfngxzeb72iyqcadbjzjpczjx5a/index.html

Now if pypi is unreachable (which seems to happen a lot recently), the
Tahoe build process will get the packages from that directory.  Also,
that URL includes the secure hash of a public key, and only authorized
uploaders have the private key, and every entry in the directory is
signed by that private key, so we can be more confident that these
packages are the ones uploaded by those authorized people.

Regards,

Zooko
---
Tahoe, the Least-Authority Filesystem -- http://allmydata.org
store your data: $10/month -- http://allmydata.com/?tracking=zsig

From tseaver at palladion.com  Sat Jan 31 21:17:43 2009
From: tseaver at palladion.com (Tres Seaver)
Date: Sat, 31 Jan 2009 15:17:43 -0500
Subject: [Distutils] [Python Language Summit] Distutils / Packaging
	survey
In-Reply-To: <498412B2.6060703@ar.media.kyoto-u.ac.jp>
References: <49803752.9040704@ar.media.kyoto-u.ac.jp>	<49803BDF.4060204@ar.media.kyoto-u.ac.jp>	<b654cd2e0901281116u57f8967dl785ffc09a0edf291@mail.gmail.com>	<1233237184.8068.9.camel@tomoyo>	<877i4dvl02.fsf@benfinney.id.au>	<498232A0.8060306@palladion.com>	<94bdd2610901291454s1eee0d0fvfbcb8d21d871e9ee@mail.gmail.com>	<20090129233428.GA10050@laurie.devork>	<94bdd2610901291621m10056e9bvf702474e866265cf@mail.gmail.com>	<20090130183951.GA5119@laurie.devork>	<b654cd2e0901301319k13974239oe3ac3cb874c02fe0@mail.gmail.com>	<87wsccz8nw.fsf@benfinney.id.au>	<49841318.6000800@palladion.com>
	<498412B2.6060703@ar.media.kyoto-u.ac.jp>
Message-ID: <4984B1E7.6060101@palladion.com>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

David Cournapeau wrote:
> Tres Seaver wrote:
>> -----BEGIN PGP SIGNED MESSAGE-----
>> Hash: SHA1
>>
>> Ben Finney wrote:
>>   
>>> Ian Bicking <ianb at colorstudy.com> writes:
>>>
>>>     
>>>> On Fri, Jan 30, 2009 at 12:39 PM, Floris Bruynooghe <
>>>> floris.bruynooghe at gmail.com> wrote:
>>>>
>>>>       
>>>>> I imagine things like libdir, prefix, datadir, docdir and other
>>>>> things copied from autoconf. Where the defaults would be something
>>>>> like:
>>>>>
>>>>> prefix = sys.prefix
>>>>> libdir = sys.prefix/lib/pythonX.Y/site-packages/pkgname
>>>>> datadir = sys.prefix/share/mypackage
>>>>> docdir = sys.prefix/share/doc/mypackage
>>>>>         
>>>> I wouldn't want to use those. What goes in libdir, what goes in
>>>> datadir?
>>>>       
>>> Again, I expect Floris is using these terms in their traditional
>>> meanings.
>>>
>>> ?library? would be a collection of executable code intended for use
>>> as a unit; what Pythoncalls a ?package? (or the degerate case of a
>>> ?module?).
>>>
>>> ?data? would be non-executable files used by the package that don't
>>> fit any other (e.g. ?documentation?) classification.
>>>
>>>     
>>>> I don't know, and frankly the distinctions start getting really
>>>> arbitrary.
>>>>       
>>> Hopefully that clarifies.
>>>
>>>     
>>>> I would rather see something like pkg_resources existing API, where
>>>> there is some file that maps out how the local names of files (where
>>>> they'd be in a checkout) map to their installed location, then the
>>>> pkg_resources code could finds the real location of the file.
>>>>       
>>> This loses the indirection that is so sorely needed of tagging
>>> resources by *type*, so that their install location can be decided on
>>> that basis.
>>>
>>> Deciding on a file-by-file basis where files get installed is
>>> something that distributors and packagers already have to do, and it's
>>> a massive pain.
>>>
>>> Allowing the developer to tag resources by type is a sensible division
>>> of responsibility: the developer knows the *purpose* (and therefore
>>> type) of the resource, and the packager knows the appropriate
>>> *location* on the filesystem for resources of a particular type.
>>>     
>> The need for those distinctions derives from a particular distributor's
>> / packager's "religion":  e.g., the insistance that "template" files
>> (software, but not "executable" by some criterion) mut not be kept in
>> the same directory with the other software.  That rule is important for
>> some packagers, and completely mystifying / irrelevant to the original
>> develpoers or to packagers with a different model.
>>   
> 
> That's not a religion: it has rationales. You may not agree with them,
> both those splits are not arbitrary.

I don't equate "religious" with "irrational":  I mean it in the sense
that people have different assumptions ("postulate sheaths", if you like
a more mathematical approach) which are taken as "given."

For isntance, the whole idea of dividing architecure-specific bits from
architecture-independent bits is legacy (from my POV, of course) of a
day when disk space was expensive, and when sysadmins wanted / needed to
share installed software across multiple, heterogenous machines.  In
today's world, I bet that is a percent-of-a-percent minority of all
installed Unix-like boxes.

Calling .py files "data" is another choice which seems (to me)
arbitrary:  in my view, .py files, as well as others, are "software",
not data, and should be kept together with the other software (e.g.,
templates used for rendering HTML) that they are distributed with.

> Each platform has its own way of
> dealing with this, and doing against it is both futile and
> counter-productive.

I'm arguing that making it easier for platform-focused people to apply
their policies is a good thing, but is in tension with keeping the
software itself simple and supportable across multiple platforms.  For
instance, adding standardized metadata to setup.py to help packagers is
a reasonable thing to ask of Python developers.  It is a harder thing to
ask them to use an API to load support files (like the templates I
mentioned), but that can be motivated by cross-platform concerns.  On
the other hand, changing my software so that it uses some Debian- (or
BSD-, or Windows-specific) policy at runtime to load those files is
Right Out (TM).

> Of course some of the distributions aspects are irrelevant to you - but
> so are some of your interests to packagers. At the risk of sounding
> obvious: developers and packagers do not have the same goals, and they
> sometimes conflict. But you cannot expect "not to care" and "my software
> will be available everywhere" at the same time. That's why those
> discussion are tiresome and difficult: there has to be some compromise.
> There is not right solution,

My original post higlighted the tension in those goals, and suggested
some of the compromises (repeated above) which packagers might
reasonably ask developers to make.


Tres.
- --
===================================================================
Tres Seaver          +1 540-429-0999          tseaver at palladion.com
Palladion Software   "Excellence by Design"    http://palladion.com
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFJhLHn+gerLs4ltQ4RAhduAKDQw0iJK40uR/CdoPAweMsX0cXjjQCeMQKb
nnINTY3DHvN5EWbzUTqdyPY=
=fsQ6
-----END PGP SIGNATURE-----

From tseaver at palladion.com  Sat Jan 31 21:17:43 2009
From: tseaver at palladion.com (Tres Seaver)
Date: Sat, 31 Jan 2009 15:17:43 -0500
Subject: [Distutils] [Python Language Summit] Distutils / Packaging
	survey
In-Reply-To: <498412B2.6060703@ar.media.kyoto-u.ac.jp>
References: <49803752.9040704@ar.media.kyoto-u.ac.jp>	<49803BDF.4060204@ar.media.kyoto-u.ac.jp>	<b654cd2e0901281116u57f8967dl785ffc09a0edf291@mail.gmail.com>	<1233237184.8068.9.camel@tomoyo>	<877i4dvl02.fsf@benfinney.id.au>	<498232A0.8060306@palladion.com>	<94bdd2610901291454s1eee0d0fvfbcb8d21d871e9ee@mail.gmail.com>	<20090129233428.GA10050@laurie.devork>	<94bdd2610901291621m10056e9bvf702474e866265cf@mail.gmail.com>	<20090130183951.GA5119@laurie.devork>	<b654cd2e0901301319k13974239oe3ac3cb874c02fe0@mail.gmail.com>	<87wsccz8nw.fsf@benfinney.id.au>	<49841318.6000800@palladion.com>
	<498412B2.6060703@ar.media.kyoto-u.ac.jp>
Message-ID: <4984B1E7.6060101@palladion.com>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

David Cournapeau wrote:
> Tres Seaver wrote:
>> -----BEGIN PGP SIGNED MESSAGE-----
>> Hash: SHA1
>>
>> Ben Finney wrote:
>>   
>>> Ian Bicking <ianb at colorstudy.com> writes:
>>>
>>>     
>>>> On Fri, Jan 30, 2009 at 12:39 PM, Floris Bruynooghe <
>>>> floris.bruynooghe at gmail.com> wrote:
>>>>
>>>>       
>>>>> I imagine things like libdir, prefix, datadir, docdir and other
>>>>> things copied from autoconf. Where the defaults would be something
>>>>> like:
>>>>>
>>>>> prefix = sys.prefix
>>>>> libdir = sys.prefix/lib/pythonX.Y/site-packages/pkgname
>>>>> datadir = sys.prefix/share/mypackage
>>>>> docdir = sys.prefix/share/doc/mypackage
>>>>>         
>>>> I wouldn't want to use those. What goes in libdir, what goes in
>>>> datadir?
>>>>       
>>> Again, I expect Floris is using these terms in their traditional
>>> meanings.
>>>
>>> ?library? would be a collection of executable code intended for use
>>> as a unit; what Pythoncalls a ?package? (or the degerate case of a
>>> ?module?).
>>>
>>> ?data? would be non-executable files used by the package that don't
>>> fit any other (e.g. ?documentation?) classification.
>>>
>>>     
>>>> I don't know, and frankly the distinctions start getting really
>>>> arbitrary.
>>>>       
>>> Hopefully that clarifies.
>>>
>>>     
>>>> I would rather see something like pkg_resources existing API, where
>>>> there is some file that maps out how the local names of files (where
>>>> they'd be in a checkout) map to their installed location, then the
>>>> pkg_resources code could finds the real location of the file.
>>>>       
>>> This loses the indirection that is so sorely needed of tagging
>>> resources by *type*, so that their install location can be decided on
>>> that basis.
>>>
>>> Deciding on a file-by-file basis where files get installed is
>>> something that distributors and packagers already have to do, and it's
>>> a massive pain.
>>>
>>> Allowing the developer to tag resources by type is a sensible division
>>> of responsibility: the developer knows the *purpose* (and therefore
>>> type) of the resource, and the packager knows the appropriate
>>> *location* on the filesystem for resources of a particular type.
>>>     
>> The need for those distinctions derives from a particular distributor's
>> / packager's "religion":  e.g., the insistance that "template" files
>> (software, but not "executable" by some criterion) mut not be kept in
>> the same directory with the other software.  That rule is important for
>> some packagers, and completely mystifying / irrelevant to the original
>> develpoers or to packagers with a different model.
>>   
> 
> That's not a religion: it has rationales. You may not agree with them,
> both those splits are not arbitrary.

I don't equate "religious" with "irrational":  I mean it in the sense
that people have different assumptions ("postulate sheaths", if you like
a more mathematical approach) which are taken as "given."

For isntance, the whole idea of dividing architecure-specific bits from
architecture-independent bits is legacy (from my POV, of course) of a
day when disk space was expensive, and when sysadmins wanted / needed to
share installed software across multiple, heterogenous machines.  In
today's world, I bet that is a percent-of-a-percent minority of all
installed Unix-like boxes.

Calling .py files "data" is another choice which seems (to me)
arbitrary:  in my view, .py files, as well as others, are "software",
not data, and should be kept together with the other software (e.g.,
templates used for rendering HTML) that they are distributed with.

> Each platform has its own way of
> dealing with this, and doing against it is both futile and
> counter-productive.

I'm arguing that making it easier for platform-focused people to apply
their policies is a good thing, but is in tension with keeping the
software itself simple and supportable across multiple platforms.  For
instance, adding standardized metadata to setup.py to help packagers is
a reasonable thing to ask of Python developers.  It is a harder thing to
ask them to use an API to load support files (like the templates I
mentioned), but that can be motivated by cross-platform concerns.  On
the other hand, changing my software so that it uses some Debian- (or
BSD-, or Windows-specific) policy at runtime to load those files is
Right Out (TM).

> Of course some of the distributions aspects are irrelevant to you - but
> so are some of your interests to packagers. At the risk of sounding
> obvious: developers and packagers do not have the same goals, and they
> sometimes conflict. But you cannot expect "not to care" and "my software
> will be available everywhere" at the same time. That's why those
> discussion are tiresome and difficult: there has to be some compromise.
> There is not right solution,

My original post higlighted the tension in those goals, and suggested
some of the compromises (repeated above) which packagers might
reasonably ask developers to make.


Tres.
- --
===================================================================
Tres Seaver          +1 540-429-0999          tseaver at palladion.com
Palladion Software   "Excellence by Design"    http://palladion.com
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFJhLHn+gerLs4ltQ4RAhduAKDQw0iJK40uR/CdoPAweMsX0cXjjQCeMQKb
nnINTY3DHvN5EWbzUTqdyPY=
=fsQ6
-----END PGP SIGNATURE-----


From zooko at zooko.com  Sat Jan 31 23:21:07 2009
From: zooko at zooko.com (zooko)
Date: Sat, 31 Jan 2009 15:21:07 -0700
Subject: [Distutils] Uninstall command, the return
In-Reply-To: <94bdd2610901291649kbd31d86mbb5c552ba01f85d0@mail.gmail.com>
References: <94bdd2610901291649kbd31d86mbb5c552ba01f85d0@mail.gmail.com>
Message-ID: <E64C99AC-4E00-4385-83D4-D634F01AB8DB@zooko.com>

On Jan 29, 2009, at 17:49 PM, Tarek Ziad? wrote:

> Many people are asking for an uninstall command.

This is true.

Many other people, however, are unwilling to let the Python packaging  
tool nor the packages that it is installing write into their system  
directories.  For these people, the system directories are never  
allowed to be written into by any tool other than their operating  
system packaging tool e.g. apt.  For these people, offering a new  
feature so that, in addition to the Python packaging tool scribbling  
into their system directory at install time, it will *also* later  
scribble into it again attempting to undo the changes that it earlier  
did is only compounding the problem.  :-)

For those people, there are several potential solutions.  One is the  
path from Python package through Python packaging tool (e.g. stdeb)  
to operating system package (e.g. .deb) to operating system tool  
(e.g. apt) to system directories.  Another is GNU stow.

For me, and for some hackers that I've talked to, making setuptools  
compatible with GNU stow would satisfy their need to have a way to  
cleanly uninstall.

So, please consider this a reminder in the distutils development  
process that you are undergoing that GNU stow compatibility has many  
benefits.

Regards,

Zooko
---
Tahoe, the Least-Authority Filesystem -- http://allmydata.org
store your data: $10/month -- http://allmydata.com/?tracking=zsig

From mailinglists at patrice.ch  Fri Jan 23 09:00:36 2009
From: mailinglists at patrice.ch (Patrice Neff)
Date: Fri, 23 Jan 2009 08:00:36 -0000
Subject: [Distutils] Properly handle username/password for subversion
Message-ID: <4B910F69-6508-4D95-A5FF-9FD25422A0CC@patrice.ch>

Hi,

In the online documentations of EasyInstall I read about the username/ 
password syntax: <http://peak.telecommunity.com/DevCenter/EasyInstall#password-protected-sites 
 >. I was then surprised to find that this does not work for links to  
Subversion repositories. That's important to us as we deploy all our  
development snapshots from Subversion directly.

For the time being I have worked around the problem by deploying the  
~/.subversion/auth directory to our servers. But I was still  
interested in a proper fix. May I suggest the patch which I uploaded  
on <http://paste.pocoo.org/show/100934/>?

Patrice Neff