From skip at pobox.com  Thu Dec  1 00:49:57 2005
From: skip at pobox.com (skip@pobox.com)
Date: Wed, 30 Nov 2005 17:49:57 -0600
Subject: [Distutils] Trouble installing Myghty with --prefix
In-Reply-To: <5.1.1.6.0.20051130134300.01fa4458@mail.telecommunity.com>
References: <5.1.1.6.0.20051130134300.01fa4458@mail.telecommunity.com>
Message-ID: <17294.15013.32330.287850@montanaro.dyndns.org>


    pje> 1.  Change your .pth file to read:

    pje>      import site; site.addsitedir('/usr/local/mojam/lib/python2.3/site-packages')

Did that:

    $ cat /usr/lib/python2.3/site-packages/mojam.pth
    # This file should be the only intrusion on the system-installed version of
    # python...
    import site; site.addsitedir("/usr/local/mojam/lib/python2.3/site-packages")

    pje> 2. Create a distutils.cfg in /usr/lib/python2.3/distutils/ with the 
    pje> contents I mentioned before.

Again with my desire to minimize cruft in /usr/lib/python2.3, I saw that
setuptools is supposed to honor ~/.pydistutils.cfg and created:

    $ cat ~/.pydistutils.cfg
    [install]
    prefix = /usr/local/mojam

    [easy_install]
    site_dirs = /usr/local/mojam/lib/python2.3/site-packages

Then, after removing all the previously installed .egg directories and the
build directory in my Myghty source tree, I tried

    python setup.py install --prefix=/usr/local/mojam

again.  Same result.  So I suffered with creating
/usr/lib/python2.3/distutils/distutils.cfg.  Lather.  Rinse.  Repeat.  Same
result:

    $ python
    Python 2.3.4 (#1, Feb 22 2005, 04:09:37) 
    [GCC 3.4.3 20041212 (Red Hat 3.4.3-9.EL4)] on linux2
    Type "help", "copyright", "credits" or "license" for more information.
    >>> import pkg_resources
    Traceback (most recent call last):
      File "<stdin>", line 1, in ?
    ImportError: No module named pkg_resources

What to try next?

Skip

From skip at pobox.com  Thu Dec  1 01:34:13 2005
From: skip at pobox.com (skip@pobox.com)
Date: Wed, 30 Nov 2005 18:34:13 -0600
Subject: [Distutils] Trouble installing Myghty with --prefix
In-Reply-To: <5.1.1.6.0.20051130134308.020634c0@mail.telecommunity.com>
References: <5.1.1.6.0.20051130115201.01fcb5c8@mail.telecommunity.com>
	<17293.51781.473397.360361@montanaro.dyndns.org>
	<5.1.1.6.0.20051130134308.020634c0@mail.telecommunity.com>
Message-ID: <17294.17669.620604.466044@montanaro.dyndns.org>


    >> I'll take a look.  At first blush I'm a bit put off by the notion it
    >> seems to promote that somehow using .pth files is wrong.

    pje> On the contrary, setuptools deals with them just fine, and uses
    pje> them just fine.  The problem is that your *Python* won't process
    pje> the .pth files installed by easy_install unless you take additional
    pje> steps.  This is literally a Python problem, in that Bob Ippolito
    pje> has previously filed bugs and patches requesting that .pth
    pje> processing be transitive.  That is, that if you list a directory in
    pje> a .pth, any .pth files in *that* directory should be processed.
    pje> Guido has vetoed this at least once already, though.  The
    pje> workaround is to use an 'import' hack in your .pth file and call
    pje> site.addsitedir().  It's a pain to set up, but it works.

Thanks.  Perhaps it would be worth changing the wording of the text in the
URL you referenced to make sure people understand that it's the nonrecursion
pth file processing that causes problems, not the mere existence of pth
files.

Skip


From pje at telecommunity.com  Thu Dec  1 01:03:53 2005
From: pje at telecommunity.com (Phillip J. Eby)
Date: Wed, 30 Nov 2005 19:03:53 -0500
Subject: [Distutils] Trouble installing Myghty with --prefix
In-Reply-To: <17294.15013.32330.287850@montanaro.dyndns.org>
References: <5.1.1.6.0.20051130134300.01fa4458@mail.telecommunity.com>
	<5.1.1.6.0.20051130134300.01fa4458@mail.telecommunity.com>
Message-ID: <5.1.1.6.0.20051130185333.02115898@mail.telecommunity.com>

At 05:49 PM 11/30/2005 -0600, skip at pobox.com wrote:
>Then, after removing all the previously installed .egg directories and the
>build directory in my Myghty source tree, I tried
>
>     python setup.py install --prefix=/usr/local/mojam
>
>again.  Same result.

Argh.  Sorry, I just realized I was ignoring the part where you are using 
"setup.py install" rather than easy_install.  "setup.py install" doesn't 
honor easy_install's options, because it's assumed that only people who 
don't have/use easy_install will run "setup.py install".  So, it's sort of 
a chicken and egg thing here, as I had you edit options that somebody using 
"setup.py install" wouldn't ordinarily be working with.  This is arguably a 
bug in setuptools, so I'll work on fixing it.

In the meantime, you should be able to just do:

     python setup.py easy_install .

in order to do the installation, whether you use distutils.cfg or 
~/.pydistutils.cfg.

Again, sorry about that, it's a problem with the setuptools "install" 
command, which is just a wrapper around easy_install, but apparently isn't 
loading its config file settings.  I'll fix that as soon as I can.

I should probably also be recommending that authors using setuptools 
recommend that their users refer to the EasyInstall documentation if they 
are installing to custom locations.



From skip at pobox.com  Thu Dec  1 02:39:20 2005
From: skip at pobox.com (skip@pobox.com)
Date: Wed, 30 Nov 2005 19:39:20 -0600
Subject: [Distutils] Trouble installing Myghty with --prefix
In-Reply-To: <5.1.1.6.0.20051130185333.02115898@mail.telecommunity.com>
References: <5.1.1.6.0.20051130134300.01fa4458@mail.telecommunity.com>
	<5.1.1.6.0.20051130185333.02115898@mail.telecommunity.com>
Message-ID: <17294.21576.945830.875127@montanaro.dyndns.org>

    >> python setup.py install --prefix=/usr/local/mojam
    >> 
    >> again.  Same result.

    ...

    pje> In the meantime, you should be able to just do:

    pje>      python setup.py easy_install .

    pje> in order to do the installation, whether you use distutils.cfg or
    pje> ~/.pydistutils.cfg.

That seems to have worked.  I can now "import myghty".  Much thanks for your
patient help.

Skip

From pje at telecommunity.com  Thu Dec  1 03:10:19 2005
From: pje at telecommunity.com (Phillip J. Eby)
Date: Wed, 30 Nov 2005 21:10:19 -0500
Subject: [Distutils] Trouble installing Myghty with --prefix
In-Reply-To: <17294.21576.945830.875127@montanaro.dyndns.org>
References: <5.1.1.6.0.20051130185333.02115898@mail.telecommunity.com>
	<5.1.1.6.0.20051130134300.01fa4458@mail.telecommunity.com>
	<5.1.1.6.0.20051130185333.02115898@mail.telecommunity.com>
Message-ID: <5.1.1.6.0.20051130210739.01fc5068@mail.telecommunity.com>

At 07:39 PM 11/30/2005 -0600, skip at pobox.com wrote:
>     >> python setup.py install --prefix=/usr/local/mojam
>     >>
>     >> again.  Same result.
>
>     ...
>
>     pje> In the meantime, you should be able to just do:
>
>     pje>      python setup.py easy_install .
>
>     pje> in order to do the installation, whether you use distutils.cfg or
>     pje> ~/.pydistutils.cfg.
>
>That seems to have worked.  I can now "import myghty".  Much thanks for your
>patient help.

And thanks for your bug report.  The SVN version of setuptools now fixes 
this problem, so that "setup.py install" honors any configuration file 
settings for the "easy_install" command.

I'll also be refactoring the "non-root installation" docs to be a more 
general section on "Customizing Installation Locations", with a sub-heading 
on non-root installations, and add in more stuff about the site hack, .pth 
transitivity, etc.  I kind of hate to make the docs more complex, but oh 
well.  I'll also be adding a "What Your Users Should Know" section to the 
setuptools manual that will cross-reference to all that, so that authors of 
setuptools-based packages can direct people to a single link.


From titus at caltech.edu  Sat Dec  3 08:02:27 2005
From: titus at caltech.edu (Titus Brown)
Date: Fri, 2 Dec 2005 23:02:27 -0800
Subject: [Distutils] easy_install problem.
Message-ID: <20051203070227.GD13002@caltech.edu>

Hi folks,

it appears easy_install has trouble understanding package names with
periods in them, e.g. "zope.testbrowser".

Also, 'python ez_setup.py --help-commands' doesn't work; "unrecognized
command".

(I'm not sure where to send reports of these problems, but google found
some discussion on this list, and it doesn't seem terribly
inappropriate... sorry if it is!)

cheers,
--titus

From jjl at pobox.com  Sat Dec  3 19:07:38 2005
From: jjl at pobox.com (John J Lee)
Date: Sat, 3 Dec 2005 18:07:38 +0000 (UTC)
Subject: [Distutils] easy_install problem.
In-Reply-To: <20051203070227.GD13002@caltech.edu>
References: <20051203070227.GD13002@caltech.edu>
Message-ID: <Pine.LNX.4.58.0512031805200.6217@alice>

Hi Titus

On Fri, 2 Dec 2005, Titus Brown wrote:

> it appears easy_install has trouble understanding package names with
> periods in them, e.g. "zope.testbrowser".
> 
> Also, 'python ez_setup.py --help-commands' doesn't work; "unrecognized
> command".

1). ez_setup.py is just the bootstrap script.  Run it once, throw it away.

2). The argument to easy_install is the PyPI (aka Python Cheese Shop)  
name, not a Python module name.  Try, for example:

easy_install -d. -m zopetestbrowser


> (I'm not sure where to send reports of these problems, but google found
> some discussion on this list, and it doesn't seem terribly
> inappropriate... sorry if it is!)

This is the right place.


John 

From pje at telecommunity.com  Sat Dec  3 20:46:29 2005
From: pje at telecommunity.com (Phillip J. Eby)
Date: Sat, 03 Dec 2005 14:46:29 -0500
Subject: [Distutils] easy_install problem.
In-Reply-To: <20051203070227.GD13002@caltech.edu>
Message-ID: <5.1.1.6.0.20051203144431.021d3f88@mail.telecommunity.com>

At 11:02 PM 12/2/2005 -0800, Titus Brown wrote:
>Hi folks,
>
>it appears easy_install has trouble understanding package names with
>periods in them, e.g. "zope.testbrowser".

Use 'zope-testbrowser' or 'zope_testbrowser' instead to refer to such a 
package for now; '.' is not currently allowed in package names.


>Also, 'python ez_setup.py --help-commands' doesn't work; "unrecognized
>command".

Just use --help.  ez_setup.py isn't a setup script.


>(I'm not sure where to send reports of these problems, but google found
>some discussion on this list, and it doesn't seem terribly
>inappropriate... sorry if it is!)

This is currently the "official" place.


From pje at telecommunity.com  Sat Dec  3 20:48:36 2005
From: pje at telecommunity.com (Phillip J. Eby)
Date: Sat, 03 Dec 2005 14:48:36 -0500
Subject: [Distutils] easy_install problem.
In-Reply-To: <Pine.LNX.4.58.0512031805200.6217@alice>
References: <20051203070227.GD13002@caltech.edu>
	<20051203070227.GD13002@caltech.edu>
Message-ID: <5.1.1.6.0.20051203144745.01fa4008@mail.telecommunity.com>

At 06:07 PM 12/3/2005 +0000, John J Lee wrote:
>1). ez_setup.py is just the bootstrap script.  Run it once, throw it away.

Except on Windows, where you need to use it to update setuptools.  If you 
use easy_install.exe to upgrade itself, Windows will complain about trying 
to overwrite the running executable.  I have a workaround planned for that 
eventually, but it's not very high priority at the moment.


From titus at caltech.edu  Sat Dec  3 20:54:40 2005
From: titus at caltech.edu (Titus Brown)
Date: Sat, 3 Dec 2005 11:54:40 -0800
Subject: [Distutils] easy_install problem.
In-Reply-To: <5.1.1.6.0.20051203144431.021d3f88@mail.telecommunity.com>
References: <20051203070227.GD13002@caltech.edu>
	<5.1.1.6.0.20051203144431.021d3f88@mail.telecommunity.com>
Message-ID: <20051203195440.GA19718@caltech.edu>

-> At 11:02 PM 12/2/2005 -0800, Titus Brown wrote:
-> >Hi folks,
-> >
-> >it appears easy_install has trouble understanding package names with
-> >periods in them, e.g. "zope.testbrowser".
-> 
-> Use 'zope-testbrowser' or 'zope_testbrowser' instead to refer to such a 
-> package for now; '.' is not currently allowed in package names.

ahh, thanks.

-> >Also, 'python ez_setup.py --help-commands' doesn't work; "unrecognized
-> >command".
-> 
-> Just use --help.  ez_setup.py isn't a setup script.

The help documentation says it's ok:

% python ez_setup.py -h

...

usage: ez_setup.py [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...]
   or: ez_setup.py --help [cmd1 cmd2 ...]
   or: ez_setup.py --help-commands
   or: ez_setup.py cmd --help

--titus

From titus at caltech.edu  Sat Dec  3 20:59:37 2005
From: titus at caltech.edu (Titus Brown)
Date: Sat, 3 Dec 2005 11:59:37 -0800
Subject: [Distutils] easy_install problem.
In-Reply-To: <Pine.LNX.4.58.0512031805200.6217@alice>
References: <20051203070227.GD13002@caltech.edu>
	<Pine.LNX.4.58.0512031805200.6217@alice>
Message-ID: <20051203195937.GB19718@caltech.edu>

-> Hi Titus
-> 
-> On Fri, 2 Dec 2005, Titus Brown wrote:
-> 
-> > it appears easy_install has trouble understanding package names with
-> > periods in them, e.g. "zope.testbrowser".
-> > 
-> > Also, 'python ez_setup.py --help-commands' doesn't work; "unrecognized
-> > command".
-> 
-> 1). ez_setup.py is just the bootstrap script.  Run it once, throw it away.

Humm, I remember reading somewhere that it needed to be included.  Can't
find that any more, so it was probably a misunderstanding -- good to
know!

-> 2). The argument to easy_install is the PyPI (aka Python Cheese Shop)  
-> name, not a Python module name.  Try, for example:
-> 
-> easy_install -d. -m zopetestbrowser

the package name is, in fact, "zope.testbrowser" ;).  So that *is* the
PyPi name:

	http://www.python.org/pypi/zope.testbrowser/

Perhaps PyPi should reject '.' in names, or is this something that will
be changed in the future in easy_install?

cheers,
--titus

From jjl at pobox.com  Sat Dec  3 21:21:49 2005
From: jjl at pobox.com (John J Lee)
Date: Sat, 3 Dec 2005 20:21:49 +0000 (UTC)
Subject: [Distutils] easy_install problem.
In-Reply-To: <20051203195937.GB19718@caltech.edu>
References: <20051203070227.GD13002@caltech.edu>
	<Pine.LNX.4.58.0512031805200.6217@alice>
	<20051203195937.GB19718@caltech.edu>
Message-ID: <Pine.LNX.4.58.0512032015270.6217@alice>

On Sat, 3 Dec 2005, Titus Brown wrote:
[...]
> -> 1). ez_setup.py is just the bootstrap script.  Run it once, throw it away.
> 
> Humm, I remember reading somewhere that it needed to be included.  Can't
> find that any more, so it was probably a misunderstanding -- good to
> know!

It should be included by module distributors in their module source as a
bootstrap, yes.

But if you're just installing something, you can throw it away (unless,
ATM at least, you're on Windows).

[...]
> the package name is, in fact, "zope.testbrowser" ;).  So that *is* the
> PyPi name:
> 
> 	http://www.python.org/pypi/zope.testbrowser/
> 
> Perhaps PyPi should reject '.' in names, or is this something that will
> be changed in the future in easy_install?

Oops, right you are.

ZopeTestbrowser is, it seems, the standalone version of zope.testbrowser,
which is why my command worked.


John

From pje at telecommunity.com  Sat Dec  3 21:21:20 2005
From: pje at telecommunity.com (Phillip J. Eby)
Date: Sat, 03 Dec 2005 15:21:20 -0500
Subject: [Distutils] easy_install problem.
In-Reply-To: <Pine.LNX.4.58.0512032015270.6217@alice>
References: <20051203195937.GB19718@caltech.edu>
	<20051203070227.GD13002@caltech.edu>
	<Pine.LNX.4.58.0512031805200.6217@alice>
	<20051203195937.GB19718@caltech.edu>
Message-ID: <5.1.1.6.0.20051203151939.01f9fe60@mail.telecommunity.com>

At 08:21 PM 12/3/2005 +0000, John J Lee wrote:
>On Sat, 3 Dec 2005, Titus Brown wrote:
>[...]
> > -> 1). ez_setup.py is just the bootstrap script.  Run it once, throw it 
> away.
> >
> > Humm, I remember reading somewhere that it needed to be included.  Can't
> > find that any more, so it was probably a misunderstanding -- good to
> > know!
>
>It should be included by module distributors in their module source as a
>bootstrap, yes.

Specifically, if they're doing the "from ez_setup import use_setuptools" 
bootstrap.  Also, they don't have to distribute ez_setup.py for that; they 
can use the ez_setup svn:externals property described in the docs, so that 
their checkouts always track the current latest "stable" release of setuptools.

(I put "stable" in quotes because of course everything's still alpha.  But 
more stable than whatever's the latest trunk version in the sandbox.)


From jjl at pobox.com  Sat Dec  3 21:53:10 2005
From: jjl at pobox.com (John J Lee)
Date: Sat, 3 Dec 2005 20:53:10 +0000 (UTC)
Subject: [Distutils] Choice of dependencies?
Message-ID: <Pine.LNX.4.58.0512031808190.6217@alice>

Is it possible for a package to depend on one of several packages, with
the user having the option to pick?  For example, my package P might
depend on package A, plus either package B or package C.

I know I could (ab)use extras_require, but in my case this is a real
dependency rather than an optional extra.  The other option I have is to
just pick one of the dependencies (B or C in my example above), and
thereby force people to install it even though it's not strictly required.
Neither option seems particularly attractive.


John

From pje at telecommunity.com  Sat Dec  3 23:00:23 2005
From: pje at telecommunity.com (Phillip J. Eby)
Date: Sat, 03 Dec 2005 17:00:23 -0500
Subject: [Distutils] Choice of dependencies?
In-Reply-To: <Pine.LNX.4.58.0512031808190.6217@alice>
Message-ID: <5.1.1.6.0.20051203165908.03193450@mail.telecommunity.com>

At 08:53 PM 12/3/2005 +0000, John J Lee wrote:
>Is it possible for a package to depend on one of several packages, with
>the user having the option to pick?  For example, my package P might
>depend on package A, plus either package B or package C.

Interesting; I'd be curious to know what P, B and C translate to in real 
packages.  :)


>I know I could (ab)use extras_require, but in my case this is a real
>dependency rather than an optional extra.  The other option I have is to
>just pick one of the dependencies (B or C in my example above), and
>thereby force people to install it even though it's not strictly required.
>Neither option seems particularly attractive.

I'm surprised that such a condition exists; it seems like one of the two 
would be redundant.


From robert.kern at gmail.com  Sat Dec  3 23:09:48 2005
From: robert.kern at gmail.com (Robert Kern)
Date: Sat, 03 Dec 2005 14:09:48 -0800
Subject: [Distutils] Choice of dependencies?
In-Reply-To: <5.1.1.6.0.20051203165908.03193450@mail.telecommunity.com>
References: <Pine.LNX.4.58.0512031808190.6217@alice>
	<5.1.1.6.0.20051203165908.03193450@mail.telecommunity.com>
Message-ID: <dmt53c$bbu$1@sea.gmane.org>

Phillip J. Eby wrote:
> At 08:53 PM 12/3/2005 +0000, John J Lee wrote:
> 
>>Is it possible for a package to depend on one of several packages, with
>>the user having the option to pick?  For example, my package P might
>>depend on package A, plus either package B or package C.
> 
> Interesting; I'd be curious to know what P, B and C translate to in real 
> packages.  :)

P = matplotlib
B = Numeric
C = numarray
D = scipy_core

Until everyone ports their stuff to scipy_core, there are still people
who need to use numarray or Numeric to be compatible with other packages.

-- 
Robert Kern
robert.kern at gmail.com

"In the fields of hell where the grass grows high
 Are the graves of dreams allowed to die."
  -- Richard Harter


From titus at caltech.edu  Sun Dec  4 00:09:03 2005
From: titus at caltech.edu (Titus Brown)
Date: Sat, 3 Dec 2005 15:09:03 -0800
Subject: [Distutils] easy_install problem.
In-Reply-To: <20051203195937.GB19718@caltech.edu>
References: <20051203070227.GD13002@caltech.edu>
	<Pine.LNX.4.58.0512031805200.6217@alice>
	<20051203195937.GB19718@caltech.edu>
Message-ID: <20051203230903.GA22597@caltech.edu>

-> -> > Also, 'python ez_setup.py --help-commands' doesn't work; "unrecognized
-> -> > command".
-> -> 
-> -> 1). ez_setup.py is just the bootstrap script.  Run it once, throw it away.
-> 
-> Humm, I remember reading somewhere that it needed to be included.  Can't
-> find that any more, so it was probably a misunderstanding -- good to
-> know!

Found it!  A presentation of Ian Bicking's,

	http://ianbicking.org/docs/setuptools-presentation/

slide 21.

Ian's recommended usage is to put the following into setup.py:

	from ez_setup import use_setuptools
	use_setuptools()

	from setuptools import setup

	...

So you need to include ez_setup if you want to use the nifty features of
setuptools while still allowing Python naive users to use your package.

...or am I missing something?

--titus

From pje at telecommunity.com  Sun Dec  4 02:47:03 2005
From: pje at telecommunity.com (Phillip J. Eby)
Date: Sat, 03 Dec 2005 20:47:03 -0500
Subject: [Distutils] easy_install problem.
In-Reply-To: <20051203230903.GA22597@caltech.edu>
References: <20051203195937.GB19718@caltech.edu>
	<20051203070227.GD13002@caltech.edu>
	<Pine.LNX.4.58.0512031805200.6217@alice>
	<20051203195937.GB19718@caltech.edu>
Message-ID: <5.1.1.6.0.20051203204439.01f7ba78@mail.telecommunity.com>

At 03:09 PM 12/3/2005 -0800, Titus Brown wrote:

>So you need to include ez_setup if you want to use the nifty features of
>setuptools while still allowing Python naive users to use your package.
>
>...or am I missing something?

Only that if you're using Subversion, you don't have to manually copy and 
update ez_setup.py whenever a new version is released.  You can use an 
svn:externals property to share an ez_setup/__init__.py, as described at:

http://peak.telecommunity.com/DevCenter/setuptools#managing-multiple-projects



From pje at telecommunity.com  Sun Dec  4 02:50:27 2005
From: pje at telecommunity.com (Phillip J. Eby)
Date: Sat, 03 Dec 2005 20:50:27 -0500
Subject: [Distutils] Choice of dependencies?
In-Reply-To: <dmt53c$bbu$1@sea.gmane.org>
References: <5.1.1.6.0.20051203165908.03193450@mail.telecommunity.com>
	<Pine.LNX.4.58.0512031808190.6217@alice>
	<5.1.1.6.0.20051203165908.03193450@mail.telecommunity.com>
Message-ID: <5.1.1.6.0.20051203204745.033377b0@mail.telecommunity.com>

At 02:09 PM 12/3/2005 -0800, Robert Kern wrote:
>Phillip J. Eby wrote:
> > At 08:53 PM 12/3/2005 +0000, John J Lee wrote:
> >
> >>Is it possible for a package to depend on one of several packages, with
> >>the user having the option to pick?  For example, my package P might
> >>depend on package A, plus either package B or package C.
> >
> > Interesting; I'd be curious to know what P, B and C translate to in real
> > packages.  :)
>
>P = matplotlib
>B = Numeric
>C = numarray
>D = scipy_core
>
>Until everyone ports their stuff to scipy_core, there are still people
>who need to use numarray or Numeric to be compatible with other packages.

Well, there isn't really a way to do 1-of-N dependencies, since you would 
need a way for the user to pick one to be satisfied by easy_install.  It 
seems like the straightforward thing to do is treat it as an optional 
("extras") dependency, and have the user install matplotlib[Numeric] or 
matplotlib[numarray] accordingly.


From jjl at pobox.com  Sun Dec  4 21:45:04 2005
From: jjl at pobox.com (John J Lee)
Date: Sun, 4 Dec 2005 20:45:04 +0000 (UTC)
Subject: [Distutils] Choice of dependencies?
In-Reply-To: <5.1.1.6.0.20051203204745.033377b0@mail.telecommunity.com>
References: <5.1.1.6.0.20051203165908.03193450@mail.telecommunity.com>
	<Pine.LNX.4.58.0512031808190.6217@alice>
	<5.1.1.6.0.20051203165908.03193450@mail.telecommunity.com>
	<5.1.1.6.0.20051203204745.033377b0@mail.telecommunity.com>
Message-ID: <Pine.LNX.4.58.0512040241460.6217@alice>

On Sat, 3 Dec 2005, Phillip J. Eby wrote:
[...]
> Well, there isn't really a way to do 1-of-N dependencies, 

OK.


> since you would need a way for the user to pick one to be satisfied by
> easy_install.

That's not a "since", that's an "and" :-)

I know where you're coming from though, of course: no need to explain
further.


> It seems like the straightforward thing to do is treat it as an optional
> ("extras") dependency, and have the user install matplotlib[Numeric] or
> matplotlib[numarray] accordingly.

How about setup.py authors having some means of defining a default
dependency, where there is an choice of dependencies?  In my case, and I
guess for Scipy too (presumably they want people to use scipy_core, all
else being equal), there's an obvious 'recommended choice'.  That
recommended choice may vary with time, but ISTM the users often split
fairly neatly into two groups:

1. People who just want to say 'give me the standard setup, whatever is
currently recommended'

2. People who know just what they want


It also seems to me that #1 is most often the bigger group.  Those people
then need to read install instructions specific to the particular package
they're installing.  It may be that the choice of dependencies isn't
terribly significant for most users, but they're forced to know about it
regardless in the current state of affairs.

FWIW, the particular P, B, and C I had in mind are (C is not a current
dependency of P, but may or may not an alternative to B be in future):

P: mechanize
B: pullparser
C: BeautifulSoup


John

From mcantor at stanford.edu  Tue Dec  6 00:49:44 2005
From: mcantor at stanford.edu (mike cantor)
Date: Mon, 05 Dec 2005 15:49:44 -0800
Subject: [Distutils] trouble building C extensions with mingw
Message-ID: <6.0.1.1.2.20051205153335.01e9ea88@mcantor.pobox.stanford.edu>

I am having difficulty getting disutils to build C extensions using the 
mingw compiler.



After installing mingw (file: MinGW-4.1.0.exe) I followed the instructions 
described in http://sebsauvage.net/python/mingw.html, using pexports and 
dlltool as instructed and placing libpython23.a the python installation 
directory.  As was noted in this page, disutils did not have to be tweaked.

I then copied example.c and example.i and setup.py to the same folder and 
from within that directory in a dos prompt ran "python setup.py build 
-cmingw32 "

The result was the following compiler errors:

E:\RESEARCH\Modeling\PythonFiles\SWIG test>python setup.py build -c mingw32
running build
running build_ext
building 'example' extension
swigging example.i to example_wrap.c
E:\swigwin-1.3.27\swig.exe -python -o example_wrap.c example.i
C:\MinGW\bin\gcc.exe -mno-cygwin -mdll -O -Wall -IE:\Python23\include 
-IE:\Pytho
n23\PC -c example_wrap.c -o build\temp.win32-2.3\Release\example_wrap.o
example_wrap.c: In function `_wrap_My_variable_set':
example_wrap.c:1621: error: `My_variable' undeclared (first use in this 
function
)
example_wrap.c:1621: error: (Each undeclared identifier is reported only once
example_wrap.c:1621: error: for each function it appears in.)
example_wrap.c: In function `_wrap_My_variable_get':
example_wrap.c:1631: error: `My_variable' undeclared (first use in this 
function
)
example_wrap.c: In function `_wrap_fact':
example_wrap.c:1648: warning: implicit declaration of function `fact'
example_wrap.c: In function `_wrap_my_mod':
example_wrap.c:1676: warning: implicit declaration of function `my_mod'
example_wrap.c: In function `_wrap_get_time':
example_wrap.c:1692: warning: implicit declaration of function `get_time'
error: command 'gcc' failed with exit status 1

Can anyone tell me where to go from here?  I have been trying to get this 
to work for three days now...

Thanks,
-mike



From pje at telecommunity.com  Tue Dec  6 02:17:39 2005
From: pje at telecommunity.com (Phillip J. Eby)
Date: Mon, 05 Dec 2005 20:17:39 -0500
Subject: [Distutils] Preliminary support for .egg-info files in Subversion
Message-ID: <5.1.1.6.0.20051205200607.01dd7510@mail.telecommunity.com>

I've just added experimental preliminary support for the .egg-info file 
convention in Subversion.  It makes the assumption that an .egg-info file 
will contain the standard distutils "PKG-INFO" data, that the distutils 
generate for source distributions and that existing egg formats include as 
an EGG-INFO/PKG-INFO or .egg-info/PKG-INFO file.  (It still recognizes 
.egg-info directories, of course.)

I haven't yet added support for installing an egg in this format; I've been 
under other deadline pressure and this is the first I've gotten to work on 
this in a couple of weeks.  I wanted to go ahead and get this basic 
capability out there so that people porting e.g. TurboGears could go ahead 
and make some progress by manually creating .egg-info files.  Please let me 
know (via the distutils-sig list) if you have any issues.  Issues with the 
*implementation*, that is, not the idea of having .egg-info files.  I'm 
already aware that it's somewhat controversial to some.  ;)

The setuptools patch is at:

    http://svn.python.org/view?view=rev&rev=41615


And you can fetch the latest source into a 'setuptools' directory, ready 
for patching, by using:

    easy_install -eb. setuptools==dev

(The -e means "fetch an editable source version and unpack it", the -b. 
means to do so in the current directory.)

Or you can just use a direct Subversion checkout.  (I don't build binary 
eggs for interim releases.)

Note that if you build a setuptools egg from this version, its version 
number (as expressed in its filename or .egg-info filename) should be 
'0.6a9.dev-r41615'.  This will be handled automatically if you build the 
egg with setuptools, but if you're creating an .egg-info directory in order 
to flatten it, you'll want to make sure you keep the same version 
information as the original.

I'll post again when the --single-version-externally-managed option to 
'install' and 'easy_install' is also available.


From pje at telecommunity.com  Tue Dec  6 03:22:26 2005
From: pje at telecommunity.com (Phillip J. Eby)
Date: Mon, 05 Dec 2005 21:22:26 -0500
Subject: [Distutils] Choice of dependencies?
In-Reply-To: <Pine.LNX.4.58.0512040241460.6217@alice>
References: <5.1.1.6.0.20051203204745.033377b0@mail.telecommunity.com>
	<5.1.1.6.0.20051203165908.03193450@mail.telecommunity.com>
	<Pine.LNX.4.58.0512031808190.6217@alice>
	<5.1.1.6.0.20051203165908.03193450@mail.telecommunity.com>
	<5.1.1.6.0.20051203204745.033377b0@mail.telecommunity.com>
Message-ID: <5.1.1.6.0.20051205211538.037bc998@mail.telecommunity.com>

At 08:45 PM 12/4/2005 +0000, John J Lee wrote:
> > It seems like the straightforward thing to do is treat it as an optional
> > ("extras") dependency, and have the user install matplotlib[Numeric] or
> > matplotlib[numarray] accordingly.
>
>How about setup.py authors having some means of defining a default
>dependency, where there is an choice of dependencies?

After giving this some thought, it seems to me that there might be a better 
way to conceptualize this.  Instead of "choose 1 of N" dependencies, why 
not allow fallbacks from a requirement?  In other words, if you have the 
preferred package, use that.  If you don't have the preferred package, but 
have the fallback package, use the fallback package.  If you have neither, 
act as though the requirement is the original specified requirement.

This seems like something that I could more likely fit into the current 
architecture, since there are already various fallback stages.  Even so, 
it's not trivial to implement and likely wouldn't be possible before 0.7 or 
0.8.



>  In my case, and I
>guess for Scipy too (presumably they want people to use scipy_core, all
>else being equal), there's an obvious 'recommended choice'.  That
>recommended choice may vary with time, but ISTM the users often split
>fairly neatly into two groups:
>
>1. People who just want to say 'give me the standard setup, whatever is
>currently recommended'
>
>2. People who know just what they want
>
>It also seems to me that #1 is most often the bigger group.  Those people
>then need to read install instructions specific to the particular package
>they're installing.  It may be that the choice of dependencies isn't
>terribly significant for most users, but they're forced to know about it
>regardless in the current state of affairs.

I'm not sure why; in this particular situation as described it seems to me 
the way to go would be to just require scipy_core, and treat support for 
alternative packages as optional.  That would inconvenience group #2 only 
if they have no intention of ever moving to scipy_core, or if for some 
reason they're unable to build or get it for their platform.


From jjl at pobox.com  Tue Dec  6 18:45:02 2005
From: jjl at pobox.com (John J Lee)
Date: Tue, 6 Dec 2005 17:45:02 +0000 (GMT Standard Time)
Subject: [Distutils] Choice of dependencies?
In-Reply-To: <5.1.1.6.0.20051205211538.037bc998@mail.telecommunity.com>
References: <5.1.1.6.0.20051203204745.033377b0@mail.telecommunity.com>
	<5.1.1.6.0.20051203165908.03193450@mail.telecommunity.com>
	<Pine.LNX.4.58.0512031808190.6217@alice>
	<5.1.1.6.0.20051203165908.03193450@mail.telecommunity.com>
	<5.1.1.6.0.20051203204745.033377b0@mail.telecommunity.com>
	<5.1.1.6.0.20051205211538.037bc998@mail.telecommunity.com>
Message-ID: <Pine.WNT.4.64.0512061741450.3292@shaolin>

On Mon, 5 Dec 2005, Phillip J. Eby wrote:

> At 08:45 PM 12/4/2005 +0000, John J Lee wrote:
[...]
> After giving this some thought, it seems to me that there might be a better 
> way to conceptualize this.  Instead of "choose 1 of N" dependencies, why not 
> allow fallbacks from a requirement?  In other words, if you have the 
[...]

Makes sense to me.

[...]
>> terribly significant for most users, but they're forced to know about it
>> regardless in the current state of affairs.
>
> I'm not sure why; in this particular situation as described it seems to me 
> the way to go would be to just require scipy_core, and treat support for 
> alternative packages as optional.  That would inconvenience group #2 only if 
> they have no intention of ever moving to scipy_core, or if for some reason 
> they're unable to build or get it for their platform.

Yes, you're right -- I just didn't like requiring something not really 
required, but I guess it's no big deal for pure-Python packages (which 
Scipy isn't, of course).


John

From pje at telecommunity.com  Tue Dec  6 18:52:32 2005
From: pje at telecommunity.com (Phillip J. Eby)
Date: Tue, 06 Dec 2005 12:52:32 -0500
Subject: [Distutils] Setuptools version parsing changed in SVN
Message-ID: <5.1.1.6.0.20051206124347.01fce7b8@mail.telecommunity.com>

FYI, I've changed the setuptools version parsing scheme.  The release of a 
'Subway 0.2-rc1' today made it clear that the existing scheme was 
unintuitive in how it handled '-' characters, because people expect the '-' 
not to override the 'rc'-ness.

In all previous versions of setuptools, a '-' always meant "this is a 
post-release patch of the thing before the dash".  As of 0.6a9-r41630, 
however, the '-' is ignored when it comes before a prerelease tag like 'rc' 
or 'b' or 'dev', so that the intuitively-expected behavior holds.

This should also fix the existing issues with CherryPy 2.1.0-rc2 and 
similar releases out there.

I've also updated the setuptools doc page with a section on specifying your 
project's version number:

http://peak.telecommunity.com/DevCenter/setuptools#specifying%2Dyour%2Dproject%2Ds%2Dversion

(The advice given there will work for all versions of setuptools, as it is 
based on the less-forgiving algorithm that's already out there.)


From mal at egenix.com  Wed Dec  7 10:38:39 2005
From: mal at egenix.com (M.-A. Lemburg)
Date: Wed, 07 Dec 2005 10:38:39 +0100
Subject: [Distutils] eggs on distutils-sig
Message-ID: <4396AD9F.1050009@egenix.com>

Hi everybody,

as you may have noticed, the distutils-sig list has recently
gotten flooded with setuptools (egg generator) related postings.

While setuptools is a nice add-on for distutils, the discussions
around it do not necessarily bring the distutils package
itself forward.

I'm wondering whether we should ask the eggs fans to start
a new mailing list for egg support and development.

What do you think ?

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Dec 07 2005)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! ::::

From mailinglist.account at gmail.com  Wed Dec  7 11:07:46 2005
From: mailinglist.account at gmail.com (Anthony Tarlano)
Date: Wed, 7 Dec 2005 11:07:46 +0100
Subject: [Distutils] setuptools site-package pollution
Message-ID: <f2b573190512070207w5e67cfd8rb952967a7506c9b7@mail.gmail.com>

Hi all..

Firstly, I would like to say thanks for the setuptools package, which
I was introduced to after reading about the RuleDispatch package on
the IBM developerworks charming python series. Oh btw. RuleDispatch is
the most useful python package that I have seen in the last year in
the python world, so thanks for that also ;)

Now that the congrats and hugs are out of the way, I would like to ask
a question. How can I tell setuptools not to put packages, such as
dispatch, protocols, and setuptools, in a subdirectory named the same
as the egg and to just put the package name.

For example from pydoc I currently get:

c:\python\lib\site-packages\ruledispatch-0.5a0.dev_r2100-py2.4-win32.egg

dispatch (package)

 c:\python\lib\site-packages\pyprotocols-1.0a0dev_r2082-py2.4-win32.egg

dispatch
 protocols (package)

  c:\python\lib\site-packages\setuptools-0.6a8-py2.4.egg

easy_install
 pkg_resources
  setuptools (package)
 site

what I would like to see is just:

..\site-packages
dispatch (package)

..\site-packages
protocols (package)

..\site-packages
setuptools (package)

Maybe it is just a pet-peeve, but I like to keep a nice tidy
site-packages directory, and these long directory names just seem to
me to be pollution of my site-packages directory in my command shell.

If there is an option to have just the packages, I would really
appreciate someone telling me what it is. If not maybe it could be
considered, and to put whatever meta-data the directory names are
providing somewhere else..

Thanks,

Anthony

From pje at telecommunity.com  Wed Dec  7 13:53:35 2005
From: pje at telecommunity.com (Phillip J. Eby)
Date: Wed, 07 Dec 2005 07:53:35 -0500
Subject: [Distutils] setuptools site-package pollution
In-Reply-To: <f2b573190512070207w5e67cfd8rb952967a7506c9b7@mail.gmail.co
 m>
Message-ID: <5.1.1.6.0.20051207074256.027a69c0@mail.telecommunity.com>

At 11:07 AM 12/7/2005 +0100, Anthony Tarlano wrote:
>Maybe it is just a pet-peeve, but I like to keep a nice tidy
>site-packages directory, and these long directory names just seem to
>me to be pollution of my site-packages directory in my command shell.

The purpose is to allow easy removal or upgrade of packages.  The normal 
distutils installation process has no provision for uninstallation or the 
deletion of obsolete modules when a package is upgraded.  Putting each 
package in its own versioned subdirectory allows comparatively trivial 
uninstallation and upgrades.  This is particularly important if you are 
using packages that are not provided by a system package manager (e.g. an 
RPM or Debian, etc.)

For example, during the development of PyProtocols, there have been times 
where I've deleted or renamed a module.  Using the standard distutils 
installation, this would sometimes result in people's upgrades still having 
the old modules present.  In short, the standard distutils installation 
process is not manageable without wrapping in some kind of package 
management tool like bdist_rpm or bdist_wininst.


>If there is an option to have just the packages, I would really
>appreciate someone telling me what it is. If not maybe it could be
>considered, and to put whatever meta-data the directory names are
>providing somewhere else..

There is an --old-and-unmanageable option to the "setup.py install" command 
which will install setuptools-based packages the old and, yes, unmanageable 
way.  It does not work with packages that require the additional metadata - 
such as setuptools itself.  It should be able to be used with RuleDispatch 
and PyProtocols, at least for the moment.  It will *not* work if you try to 
install packages that declare dependencies on those packages.  For example, 
if you install RuleDispatch using --old-and-unmanageable, then TurboGears 
will try to reinstall it and warn you about conflicting packages installed 
the old and unmanageable way in site-packages.

There is another option I'm working on, 
--single-version-externally-managed, which is designed for system packagers 
to create old-style packages, but with the metadata included as a separate 
.egg-info directory.  This won't help you, though, because it will result 
in having even *more* directories in site-packages - the package 
directories plus an .egg-info directory.  Anyway, this approach will be 
used for future setuptools integration with bdist_rpm, bdist_wininst, etc., 
since these package management tools are able to handle safe upgrades and 
uninstallation in a way that the unadorned distutils install process cannot.


From wl at flexis.de  Wed Dec  7 12:49:18 2005
From: wl at flexis.de (Wolfgang Langner)
Date: Wed, 07 Dec 2005 12:49:18 +0100
Subject: [Distutils] eggs on distutils-sig
In-Reply-To: <4396AD9F.1050009@egenix.com>
References: <4396AD9F.1050009@egenix.com>
Message-ID: <dn6i7u$tu8$1@sea.gmane.org>

Hello,

M.-A. Lemburg wrote:

> I'm wondering whether we should ask the eggs fans to start
> a new mailing list for egg support and development.
> 
> What do you think ?

I hope setuptools will become part of distutils in future.

This should happen as fast as possible. Python 2.5 ?

For me
-1
for extra mailing list.


bye by Wolfgang


From dangoor at gmail.com  Wed Dec  7 14:16:50 2005
From: dangoor at gmail.com (Kevin Dangoor)
Date: Wed, 7 Dec 2005 08:16:50 -0500
Subject: [Distutils] eggs on distutils-sig
In-Reply-To: <4396AD9F.1050009@egenix.com>
References: <4396AD9F.1050009@egenix.com>
Message-ID: <3f085ecd0512070516h34290103gaaa37a44e952dfb7@mail.gmail.com>

-1

In the months that I've been subscribed to this list, I haven't really
seen any *other* discussion that seems geared towards improving
distribution of Python modules and add-ons.

Kevin

On 12/7/05, M.-A. Lemburg <mal at egenix.com> wrote:
> While setuptools is a nice add-on for distutils, the discussions
> around it do not necessarily bring the distutils package
> itself forward.

From smurf at smurf.noris.de  Wed Dec  7 14:57:10 2005
From: smurf at smurf.noris.de (Matthias Urlichs)
Date: Wed, 7 Dec 2005 14:57:10 +0100
Subject: [Distutils] setuptools site-package pollution
In-Reply-To: <5.1.1.6.0.20051207074256.027a69c0@mail.telecommunity.com>
References: <f2b573190512070207w5e67cfd8rb952967a7506c9b7@mail.gmail.com>
	<5.1.1.6.0.20051207074256.027a69c0@mail.telecommunity.com>
Message-ID: <20051207135710.GE11533@kiste.smurf.noris.de>

Hi,

Phillip J. Eby:
> There is another option I'm working on, 
> --single-version-externally-managed, which is designed for system packagers 
> to create old-style packages, but with the metadata included as a separate 
> .egg-info directory.  This won't help you, though, because it will result 
> in having even *more* directories in site-packages - the package 
> directories plus an .egg-info directory.

The package directory is not a problem, one directory needs to be
present anyway -- whether it's called "foo" or "foo-VERSION.egg" doesn't
matter much.

Maybe it would be a good idea to hide all the .egg-info directories
under one common subdirectory, i.e. instead of "foo.egg-info" (or, as you
plan IIRC, "foo-VERSION.egg-info"), the metadata would be in something
like .../EGG-INFO/foo-VERSION/*.

Advantage: no clutter, less work for every tools because you have to
scan through fewer directory items.

I'd go as far as disallowing .egg files in directories which have an
EGG-INFO subdirectory, which would reduce scan times even more.
Rationale: Either a directory is externally managed, or it is not.
(/usr vs. /usr/local.) If it is, shipping an .egg makes no sense --
you don't gain anything by doing that. If it is not, on the other hand
installing externally-managed versions makes no sense.

-- 
Matthias Urlichs   |   {M:U} IT Design @ m-u-it.de   |  smurf at smurf.noris.de
Disclaimer: The quote was selected randomly. Really. | http://smurf.noris.de
 - -
The minute you start to analyze why sex feels so good, it stops feeling good
and starts feeling silly.
					-- Solomon Short

From mal at egenix.com  Wed Dec  7 15:31:09 2005
From: mal at egenix.com (M.-A. Lemburg)
Date: Wed, 07 Dec 2005 15:31:09 +0100
Subject: [Distutils] trouble building C extensions with mingw
In-Reply-To: <6.0.1.1.2.20051205153335.01e9ea88@mcantor.pobox.stanford.edu>
References: <6.0.1.1.2.20051205153335.01e9ea88@mcantor.pobox.stanford.edu>
Message-ID: <4396F22D.6090609@egenix.com>

mike cantor wrote:
> I am having difficulty getting disutils to build C extensions using the 
> mingw compiler.
> 
> 
> 
> After installing mingw (file: MinGW-4.1.0.exe) I followed the instructions 
> described in http://sebsauvage.net/python/mingw.html, using pexports and 
> dlltool as instructed and placing libpython23.a the python installation 
> directory.  As was noted in this page, disutils did not have to be tweaked.
> 
> I then copied example.c and example.i and setup.py to the same folder and 
> from within that directory in a dos prompt ran "python setup.py build 
> -cmingw32 "
> 
> The result was the following compiler errors:
> 
> E:\RESEARCH\Modeling\PythonFiles\SWIG test>python setup.py build -c mingw32
> running build
> running build_ext
> building 'example' extension
> swigging example.i to example_wrap.c
> E:\swigwin-1.3.27\swig.exe -python -o example_wrap.c example.i
> C:\MinGW\bin\gcc.exe -mno-cygwin -mdll -O -Wall -IE:\Python23\include 
> -IE:\Pytho
> n23\PC -c example_wrap.c -o build\temp.win32-2.3\Release\example_wrap.o
> example_wrap.c: In function `_wrap_My_variable_set':
> example_wrap.c:1621: error: `My_variable' undeclared (first use in this 
> function
> )
> example_wrap.c:1621: error: (Each undeclared identifier is reported only once
> example_wrap.c:1621: error: for each function it appears in.)
> example_wrap.c: In function `_wrap_My_variable_get':
> example_wrap.c:1631: error: `My_variable' undeclared (first use in this 
> function
> )
> example_wrap.c: In function `_wrap_fact':
> example_wrap.c:1648: warning: implicit declaration of function `fact'
> example_wrap.c: In function `_wrap_my_mod':
> example_wrap.c:1676: warning: implicit declaration of function `my_mod'
> example_wrap.c: In function `_wrap_get_time':
> example_wrap.c:1692: warning: implicit declaration of function `get_time'
> error: command 'gcc' failed with exit status 1
> 
> Can anyone tell me where to go from here?  I have been trying to get this 
> to work for three days now...

You should have a look at the generated example_wrap.c file.
It appears as if you have a missing library somewhere in
the chain, e.g. the lib that you are trying to wrap.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Dec 07 2005)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! ::::

From ianb at colorstudy.com  Wed Dec  7 18:10:26 2005
From: ianb at colorstudy.com (Ian Bicking)
Date: Wed, 07 Dec 2005 11:10:26 -0600
Subject: [Distutils] eggs on distutils-sig
In-Reply-To: <3f085ecd0512070516h34290103gaaa37a44e952dfb7@mail.gmail.com>
References: <4396AD9F.1050009@egenix.com>
	<3f085ecd0512070516h34290103gaaa37a44e952dfb7@mail.gmail.com>
Message-ID: <43971782.2070801@colorstudy.com>

Kevin Dangoor wrote:
> -1
> 
> In the months that I've been subscribed to this list, I haven't really
> seen any *other* discussion that seems geared towards improving
> distribution of Python modules and add-ons.

Ditto.  Setuptools is the most active distutils-related development 
right now, no surprise there's lots of messages on it.  The volume is a 
*good* thing ;)

-- 
Ian Bicking  /  ianb at colorstudy.com  /  http://blog.ianbicking.org

From stephen.langer at nist.gov  Wed Dec  7 20:22:03 2005
From: stephen.langer at nist.gov (Stephen Langer)
Date: Wed, 7 Dec 2005 14:22:03 -0500
Subject: [Distutils] eggs on distutils-sig
In-Reply-To: <43971782.2070801@colorstudy.com>
References: <4396AD9F.1050009@egenix.com>
	<3f085ecd0512070516h34290103gaaa37a44e952dfb7@mail.gmail.com>
	<43971782.2070801@colorstudy.com>
Message-ID: <C2A49395-EF6A-4344-B086-1B63063AD76D@nist.gov>


On Dec 7, 2005, at 12:10 PM, Ian Bicking wrote:

> Kevin Dangoor wrote:
>> -1
>>
>> In the months that I've been subscribed to this list, I haven't  
>> really
>> seen any *other* discussion that seems geared towards improving
>> distribution of Python modules and add-ons.
>
> Ditto.  Setuptools is the most active distutils-related development
> right now, no surprise there's lots of messages on it.  The volume  
> is a
> *good* thing ;)

I guess the concern is that the dominance of setuptools message is  
scaring away people with old-fashioned distutils questions.  There  
haven't been many of those lately, and some of them seem to have  
received no response.  I don't think that splitting the list in two  
is a solution, though.  There's not enough traffic to warrant that  
(except for a brief spurt a week or so ago...)

  -- Steve

--
-- EMail: stephen.langer at nist.gov                    Phone: (301)  
975-5423 --
-- WWW:  http://math.nist.gov/mcsd/Staff/SLanger/    Fax:   (301)  
990-4127 --
-- Mail: NIST; 100 Bureau Drive -- Stop 8910; Gaithersburg, Md   
20899-8910 --

-- "I don't think this will work.  That's why it's  
science."               --
--                                      Naomi Langer,  17 Feb  
2003         --



From sharkypt at gmail.com  Wed Dec  7 22:17:30 2005
From: sharkypt at gmail.com (Sharky On PTNet)
Date: Wed, 7 Dec 2005 21:17:30 +0000
Subject: [Distutils] easy_install trac
Message-ID: <40b686290512071317p7c271902y@mail.gmail.com>

Hi,

  I need some help to install trac (http://www.python.org/pypi/trac/0.9)
with easy_install. My problem is that trac have modules to instal in
python/site-packages/, have scripts to install in /usr/bin/ and have files
to install in /usr/share/trac.

  Any one can tell me how to do this with easy_install?


Many Thanks,
Helio Pereira
Sharky @ PTNet
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/distutils-sig/attachments/20051207/92f8bb7f/attachment.htm

From pje at telecommunity.com  Wed Dec  7 23:01:01 2005
From: pje at telecommunity.com (Phillip J. Eby)
Date: Wed, 07 Dec 2005 17:01:01 -0500
Subject: [Distutils] easy_install trac
In-Reply-To: <40b686290512071317p7c271902y@mail.gmail.com>
Message-ID: <5.1.1.6.0.20051207165806.0361b1e8@mail.telecommunity.com>

At 09:17 PM 12/7/2005 +0000, Sharky On PTNet wrote:
>Hi,
>
>   I need some help to install trac 
> (<http://www.python.org/pypi/trac/0.9>http://www.python.org/pypi/trac/0.9) 
>   with easy_install. My problem is that trac have modules to instal in 
> python/site-packages/, have scripts to install in /usr/bin/ and have 
> files to install in /usr/share/trac.
>
>   Any one can tell me how to do this with easy_install?

The --script-dir option indicates where to install scripts.  Data is 
installed with the package, not to /usr/share.  Installing data alongside a 
package is more useful when the package needs to access that data, as there 
is no cross-platform standard location for the distutils to install data, 
and thus no easy way for a package to find its data otherwise.


From skip at pobox.com  Thu Dec  8 05:16:31 2005
From: skip at pobox.com (skip@pobox.com)
Date: Wed, 7 Dec 2005 22:16:31 -0600
Subject: [Distutils] Installing stuff using setuptools w/ Python from CVS
	HEAD?
Message-ID: <17303.45983.602184.300426@montanaro.dyndns.org>


On my Mac Powerbook (OSX 10.3.9) I routinely use Python built from CVS HEAD
as my day-to-day Python interpreter.  Today I tried installing Myghty 0.99a
using that interpreter:

    % python setup.py easy_install .

    ---------------------------------------------------------------------------
    This script requires setuptools version 0.6a5 to run (even to display
    help).  I will attempt to download it for you (from
    http://cheeseshop.python.org/packages/2.5/s/setuptools/), but
    you may need to enable firewall access for this script first.
    I will start the download in 15 seconds.
    ---------------------------------------------------------------------------
    Downloading http://cheeseshop.python.org/packages/2.5/s/setuptools/setuptools-0.6a5-py2.5.egg
    Traceback (most recent call last):
      File "setup.py", line 2, in <module>
        use_setuptools()
      File "/Users/skip/src/Myghty-0.99a/ez_setup.py", line 74, in use_setuptools
        egg = download_setuptools(version, download_base, to_dir, download_delay)
      File "/Users/skip/src/Myghty-0.99a/ez_setup.py", line 122, in download_setuptools
        src = urllib2.urlopen(url)
      File "/Users/skip/local/lib/python2.5/urllib2.py", line 130, in urlopen
        return _opener.open(url, data)
      File "/Users/skip/local/lib/python2.5/urllib2.py", line 367, in open
        response = meth(req, response)
      File "/Users/skip/local/lib/python2.5/urllib2.py", line 475, in http_response
        'http', request, response, code, msg, hdrs)
      File "/Users/skip/local/lib/python2.5/urllib2.py", line 405, in error
        return self._call_chain(*args)
      File "/Users/skip/local/lib/python2.5/urllib2.py", line 340, in _call_chain
        result = func(*args)
      File "/Users/skip/local/lib/python2.5/urllib2.py", line 483, in http_error_default
        raise HTTPError(req.get_full_url(), code, msg, hdrs, fp)
    urllib2.HTTPError: HTTP Error 404: Not Found

Even "python setup.py --help" doesn't work.  I can understand that there's
no packages/2.5 directory on cheeseshop yet.  Is there a way to work around
this problem?

Thx,

-- 
Skip Montanaro
Katrina Benefit Concerts: http://www.musi-cal.com/katrina
skip at pobox.com

From t.greenwoodgeer at gmail.com  Thu Dec  8 07:25:35 2005
From: t.greenwoodgeer at gmail.com (tgreenwood)
Date: Wed, 07 Dec 2005 22:25:35 -0800
Subject: [Distutils] local development scenarios
Message-ID: <4397D1DF.40907@gmail.com>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

I'm new to distutils/and the egg system. I have not researched the
archives as much as I should, so please excuse me if this is familiar
territory...

Q: Is it possible for dependency checks to work for non pipy deployed eggs?
 - A.egg
 - B.egg (setup_requires and install_requires A.egg)
 - email A and B eggs to friend
 - tell friend to : easy_install B
 - but B.egg does not install A...in fact, in my test case, it fails b/c
it cannot find A in pipy
 - in this example, what happens if friend does not have net connection
on target machine? how can they install from the eggs alone and get the
dep checking to work?

Q: How does my friend run the unittests on the egg that they have just
installed? As I am developing, I frequently run the unittests and after
every bdist_egg build, I also run the unit tests that way, too. How can
the consumer do the same?

Q: One of my apps is a command line app, say foo.py is a command line
app that takes arg1, arg2,... How can the user run this app when it is
encased in an egg?

Q: How do I define my own pipy? While I like the idea of deploying there
for the overall community, I don't want to send my half baked code into
the wild. But I'd like to try the general functionality of having a
deploy site for my beta customers. How can I set this up?

Thanks in advance.

- -Todd
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.1 (GNU/Linux)

iD8DBQFDl9Hfz6uXX4lQc/URAnY4AJ47DFHL/cpEltmqoUyGqzhsK5FjRACgt01v
akc3Wh3IOXVv2jiS8TszhuE=
=Rv/x
-----END PGP SIGNATURE-----

From no228 at cam.ac.uk  Thu Dec  8 14:58:00 2005
From: no228 at cam.ac.uk (Noel O'Boyle)
Date: Thu, 08 Dec 2005 13:58:00 +0000
Subject: [Distutils] A distutils question
Message-ID: <1134050280.12223.18.camel@sandwi.ch.cam.ac.uk>

It's true (see thread below) - I have been scared away from asking
questions about distutils. But here I go...

Can someone help me. I think this is quite a common
problem but there doesn't appear to be any obvious answer:

Can someone suggest how I can write a setup.py for a pure-Python
application that should install on both Linux and Windows?

The application includes a startup script, a package, and some data
files (or package data) referred to by some modules in the package.

The Linux version will be distributed as a source distribution. The
Windows version will be distributed as a bdist_wininst, and should
create a shortcut in the Start Menu folder.

An answer will not only receive my heartfelt thanks, but be
posted on the web (http://www.redbrick.dcu.ie/~noel/distutils), so that
others will not have the same problem. A method that will work for both
Python 2.3 and 2.4 would be the most appreciated.


---------contents of development directory
setup.py
README
MANIFEST.in
license.txt
src/
    startupscript.py
    mypkg/
        __init__.py
        module_a.py
        help.html
        images/
            shamrock.png

Regards,
 Noel


On Wed, 2005-12-07 at 14:22 -0500, Stephen Langer wrote:
> On Dec 7, 2005, at 12:10 PM, Ian Bicking wrote:
> 
> > Kevin Dangoor wrote:
> >> -1
> >>
> >> In the months that I've been subscribed to this list, I haven't  
> >> really
> >> seen any *other* discussion that seems geared towards improving
> >> distribution of Python modules and add-ons.
> >
> > Ditto.  Setuptools is the most active distutils-related development
> > right now, no surprise there's lots of messages on it.  The volume  
> > is a
> > *good* thing ;)
> 
> I guess the concern is that the dominance of setuptools message is  
> scaring away people with old-fashioned distutils questions.  There  
> haven't been many of those lately, and some of them seem to have  
> received no response.  I don't think that splitting the list in two  
> is a solution, though.  There's not enough traffic to warrant that  
> (except for a brief spurt a week or so ago...)
> 
>   -- Steve

> 


From theller at python.net  Thu Dec  8 16:53:06 2005
From: theller at python.net (Thomas Heller)
Date: Thu, 08 Dec 2005 16:53:06 +0100
Subject: [Distutils] A distutils question
References: <1134050280.12223.18.camel@sandwi.ch.cam.ac.uk>
Message-ID: <4q5j7fh9.fsf@python.net>

"Noel O'Boyle" <no228 at cam.ac.uk> writes:

> It's true (see thread below) - I have been scared away from asking
> questions about distutils. But here I go...
>
> Can someone help me. I think this is quite a common
> problem but there doesn't appear to be any obvious answer:
>
> Can someone suggest how I can write a setup.py for a pure-Python
> application that should install on both Linux and Windows?
>
> The application includes a startup script, a package, and some data
> files (or package data) referred to by some modules in the package.
>
> The Linux version will be distributed as a source distribution. The
> Windows version will be distributed as a bdist_wininst, and should
> create a shortcut in the Start Menu folder.

In bdist_wininst, you can create shortcuts (and do other fancy stuff) in
the postinstall-script.  It is badly documented, maybe even fragile, but
looking into other setup scripts should get you started.

Here are some pointers:

PythonCard postinstall script:

http://cvs.sourceforge.net/viewcvs.py/pythoncard/PythonCardPrototype/install-pythoncard.py?rev=1.4&view=auto

Pywin32 postinstall script:

http://cvs.sourceforge.net/viewcvs.py/pywin32/pywin32/pywin32_postinstall.py?rev=1.18&view=auto

> An answer will not only receive my heartfelt thanks, but be
> posted on the web (http://www.redbrick.dcu.ie/~noel/distutils), so that
> others will not have the same problem. A method that will work for both
> Python 2.3 and 2.4 would be the most appreciated.

You should use the newest Python version (2.4.2) to build the installer.

Thomas


From pje at telecommunity.com  Thu Dec  8 17:47:21 2005
From: pje at telecommunity.com (Phillip J. Eby)
Date: Thu, 08 Dec 2005 11:47:21 -0500
Subject: [Distutils] Installing stuff using setuptools w/ Python from
 CVS HEAD?
In-Reply-To: <17303.45983.602184.300426@montanaro.dyndns.org>
Message-ID: <5.1.1.6.0.20051208114616.01f6ee30@mail.telecommunity.com>

At 10:16 PM 12/7/2005 -0600, skip at pobox.com wrote:
>Even "python setup.py --help" doesn't work.  I can understand that there's
>no packages/2.5 directory on cheeseshop yet.  Is there a way to work around
>this problem?

You'll have to install setuptools manually; check it out of Subversion and 
run its "setup.py install".  Then you'll have a locally-installed 
setuptools egg for Python 2.5.


From pje at telecommunity.com  Thu Dec  8 17:54:24 2005
From: pje at telecommunity.com (Phillip J. Eby)
Date: Thu, 08 Dec 2005 11:54:24 -0500
Subject: [Distutils] local development scenarios
In-Reply-To: <4397D1DF.40907@gmail.com>
Message-ID: <5.1.1.6.0.20051208114741.03eb3680@mail.telecommunity.com>

At 10:25 PM 12/7/2005 -0800, tgreenwood wrote:
>-----BEGIN PGP SIGNED MESSAGE-----
>Hash: SHA1
>
>I'm new to distutils/and the egg system. I have not researched the
>archives as much as I should, so please excuse me if this is familiar
>territory...
>
>Q: Is it possible for dependency checks to work for non pipy deployed eggs?
>  - A.egg
>  - B.egg (setup_requires and install_requires A.egg)
>  - email A and B eggs to friend
>  - tell friend to : easy_install B
>  - but B.egg does not install A...in fact, in my test case, it fails b/c
>it cannot find A in pipy

This will work if the eggs are both in the current directory:

     easy_install -f. B

-f (aka --find-links) specifies URLs or directories that will be scanned 
*before* going to PyPI to meet dependencies.


>  - in this example, what happens if friend does not have net connection
>on target machine? how can they install from the eggs alone and get the
>dep checking to work?

No net connection is necessary if all dependencies can be found using -f 
directories.


>Q: How does my friend run the unittests on the egg that they have just
>installed? As I am developing, I frequently run the unittests and after
>every bdist_egg build, I also run the unit tests that way, too. How can
>the consumer do the same?

Well, if it's a unit test suite that's contained within the egg, then they 
could use the unittest utility just as they normally would.


>Q: One of my apps is a command line app, say foo.py is a command line
>app that takes arg1, arg2,... How can the user run this app when it is
>encased in an egg?

EasyInstall generates wrapper scripts during installation that ensure the 
needed eggs are on sys.path and run the scripts.  This works for both 
standard distutils scripts (specified using setup(scripts=[...])) and for 
entry point scripts (see the setuptools doc).

The --script-dir argument to easy_install controls where scripts are 
installed; it defaults to the same as the installation directory if an 
installation directory is manually specified; otherwise it defaults to 
wherever the distutils are configured to install scripts.


>Q: How do I define my own pipy? While I like the idea of deploying there
>for the overall community, I don't want to send my half baked code into
>the wild. But I'd like to try the general functionality of having a
>deploy site for my beta customers. How can I set this up?

Using -f with the URL of a web page with download links to your eggs or 
source distributions.  See http://turbogears.org/download/ for one 
example.  In the simplest case, a simple Apache directory listing will do 
the trick; see http://peak.telecommunity.com/snapshots/ for an example of 
that approach.  Simply having someone use a URL like that with -f ensures 
that any download links on the page will be given precedence over PyPI 
listings, if it's possible to meet all requirements that way.

Only if a dependency can't be satisfied by links on a -f page, will PyPI be 
consulted.


From pje at telecommunity.com  Thu Dec  8 17:58:47 2005
From: pje at telecommunity.com (Phillip J. Eby)
Date: Thu, 08 Dec 2005 11:58:47 -0500
Subject: [Distutils] A distutils question
In-Reply-To: <1134050280.12223.18.camel@sandwi.ch.cam.ac.uk>
Message-ID: <5.1.1.6.0.20051208115436.03eefaa8@mail.telecommunity.com>

At 01:58 PM 12/8/2005 +0000, Noel O'Boyle wrote:
>Can someone suggest how I can write a setup.py for a pure-Python
>application that should install on both Linux and Windows?
>
>The application includes a startup script, a package, and some data
>files (or package data) referred to by some modules in the package.
>
>The Linux version will be distributed as a source distribution. The
>Windows version will be distributed as a bdist_wininst, and should
>create a shortcut in the Start Menu folder.
>
>An answer will not only receive my heartfelt thanks, but be
>posted on the web (http://www.redbrick.dcu.ie/~noel/distutils), so that
>others will not have the same problem. A method that will work for both
>Python 2.3 and 2.4 would be the most appreciated.
>
>
>---------contents of development directory
>setup.py
>README
>MANIFEST.in
>license.txt
>src/
>     startupscript.py
>     mypkg/
>         __init__.py
>         module_a.py
>         help.html
>         images/
>             shamrock.png
>
>Regards,
>  Noel

If your question is how to include the data files, you can do it by adding 
something like:

    package_data = {'mypkg': ['*.html', 'images/*.png']},

to your setup() call.  See:

     http://python.org/doc/2.4.1/dist/node11.html

for documentation.

If you want to do this with Python 2.3, however, you'll need to either use 
setuptools, or write a custom extension to the distutils install_data 
command, as the Python 2.3 distutils don't support the package_data option 
(which was originally developed in setuptools and then added to the 2.4 
distutils).


From bkc at murkworks.com  Thu Dec  8 19:27:57 2005
From: bkc at murkworks.com (Brad Clements)
Date: Thu, 08 Dec 2005 13:27:57 -0500
Subject: [Distutils] local development scenarios
In-Reply-To: <5.1.1.6.0.20051208114741.03eb3680@mail.telecommunity.com>
References: <4397D1DF.40907@gmail.com>
Message-ID: <439834DD.26452.EA7E4BD@bkc.murkworks.com>

On 8 Dec 2005 at 11:54, Phillip J. Eby wrote:

> 
> >Q: How do I define my own pipy? While I like the idea of deploying there
> >for the overall community, I don't want to send my half baked code into
> >the wild. But I'd like to try the general functionality of having a
> >deploy site for my beta customers. How can I set this up?
> 
> Using -f with the URL of a web page with download links to your eggs or 
> source distributions.  See http://turbogears.org/download/ for one 


Awsome, I need the same thing too.

What if the url  passed to -f requires authentication? 



-- 
Brad Clements,                bkc at murkworks.com    (315)268-1000
http://www.murkworks.com                          
AOL-IM or SKYPE: BKClements



From pje at telecommunity.com  Thu Dec  8 21:12:25 2005
From: pje at telecommunity.com (Phillip J. Eby)
Date: Thu, 08 Dec 2005 15:12:25 -0500
Subject: [Distutils] local development scenarios
In-Reply-To: <439834DD.26452.EA7E4BD@bkc.murkworks.com>
References: <5.1.1.6.0.20051208114741.03eb3680@mail.telecommunity.com>
	<4397D1DF.40907@gmail.com>
Message-ID: <5.1.1.6.0.20051208151024.01f3f808@mail.telecommunity.com>

At 01:27 PM 12/8/2005 -0500, Brad Clements wrote:
>On 8 Dec 2005 at 11:54, Phillip J. Eby wrote:
>
> >
> > >Q: How do I define my own pipy? While I like the idea of deploying there
> > >for the overall community, I don't want to send my half baked code into
> > >the wild. But I'd like to try the general functionality of having a
> > >deploy site for my beta customers. How can I set this up?
> >
> > Using -f with the URL of a web page with download links to your eggs or
> > source distributions.  See http://turbogears.org/download/ for one
>
>Awsome, I need the same thing too.
>
>What if the url  passed to -f requires authentication?

Then it won't work.  :)  Urllib doesn't provide login prompts by 
default.  I imagine the feature could be added, though, and I wouldn't be 
averse to including support, someone wants to implement it.


From niclane at oakcom.co.nz  Thu Dec  8 21:11:13 2005
From: niclane at oakcom.co.nz (nic)
Date: Thu, 08 Dec 2005 15:11:13 -0500
Subject: [Distutils] python port for openwrt
Message-ID: <20051208151113.4ap7op320wwskk4w@horde.cs.dartmouth.edu>

Hi Tim,

I'm trying to get a cut down version of the python interpreter running under
openwrt. I saw you message on the python mailing list from 2004 and was
wondering
if you could let me know more about how you compiled it.

Could you send me the make file you used or some cut and pastes of the commands
you ran. I'm new to cross compilation and a bit green.

I'm also trying to get the size down of the interpreter. I only need to run a
few scripts which don't need that many modules, in the scripts they just import
serial, sys, time, string, cStringIO, and struct. Did you ever just make a
interpreter that included a sub set of the modules?

I've been following the basic idea of compiling then renaming two exe python and
pgen and so you have the two exes that will run on the native machine which is
the tricky part from what people say.

I've tried following the description that you link to in your post:

http://www.ailis.de/~k/docs/crosscompiling/python.php

but when I try and do the step of the following:

 # CXX=$mips_linux_path\g++ \
CC=$mips_linux_path\gcc \
AR=$mips_linux_path\ar \
RANLIB=$mips_linux_path\ranlib \
./configure mips_linux --target=mips_linux --prefix=/usr

I get errors from the ./configure step saying it that g++ can not create
executables. I guess it's trying to build an executable on my desktop with the
cross compiler g++ and then seeing that it doesn't work on the desktop and then
complaining, how did you get around this?

Any help would be really appreciated.

Thanks a bunch,


Nic

From hawk78_it at yahoo.it  Fri Dec  9 01:14:07 2005
From: hawk78_it at yahoo.it (Vincenzo Di Massa)
Date: Fri, 9 Dec 2005 01:14:07 +0100
Subject: [Distutils] eggs on distutils-sig
In-Reply-To: <43971782.2070801@colorstudy.com>
References: <4396AD9F.1050009@egenix.com>
	<3f085ecd0512070516h34290103gaaa37a44e952dfb7@mail.gmail.com>
	<43971782.2070801@colorstudy.com>
Message-ID: <200512090114.07480.hawk78_it@yahoo.it>

-1 me too

It is better to keep distutils users/developers close to setuptools 
users/developers IMHO.
Setuptools and distutils are really related and should be kept toghether.

Vincenzo

Alle 18:10, mercoled? 07 dicembre 2005, Ian Bicking ha scritto:
> Kevin Dangoor wrote:
> > -1
> >
> > In the months that I've been subscribed to this list, I haven't really
> > seen any *other* discussion that seems geared towards improving
> > distribution of Python modules and add-ons.
>
> Ditto.  Setuptools is the most active distutils-related development
> right now, no surprise there's lots of messages on it.  The volume is a
> *good* thing ;)

	

	
		
___________________________________ 
Yahoo! Mail: gratis 1GB per i messaggi e allegati da 10MB 
http://mail.yahoo.it

From bob at redivi.com  Fri Dec  9 01:27:59 2005
From: bob at redivi.com (Bob Ippolito)
Date: Thu, 8 Dec 2005 16:27:59 -0800
Subject: [Distutils] eggs on distutils-sig
In-Reply-To: <200512090114.07480.hawk78_it@yahoo.it>
References: <4396AD9F.1050009@egenix.com>
	<3f085ecd0512070516h34290103gaaa37a44e952dfb7@mail.gmail.com>
	<43971782.2070801@colorstudy.com>
	<200512090114.07480.hawk78_it@yahoo.it>
Message-ID: <8CD90AF4-4037-4E95-B911-6F19A815E281@redivi.com>

On Dec 8, 2005, at 4:14 PM, Vincenzo Di Massa wrote:

> Alle 18:10, mercoled? 07 dicembre 2005, Ian Bicking ha scritto:
>> Kevin Dangoor wrote:
>>> -1
>>>
>>> In the months that I've been subscribed to this list, I haven't  
>>> really
>>> seen any *other* discussion that seems geared towards improving
>>> distribution of Python modules and add-ons.
>>
>> Ditto.  Setuptools is the most active distutils-related development
>> right now, no surprise there's lots of messages on it.  The volume  
>> is a
>> *good* thing ;)
>
> -1 me too
>
> It is better to keep distutils users/developers close to setuptools
> users/developers IMHO.
> Setuptools and distutils are really related and should be kept  
> toghether.

-1 also.  As far I'm concerned, setuptools is distutils.

-bob


From mcantor at stanford.edu  Fri Dec  9 01:43:49 2005
From: mcantor at stanford.edu (mike cantor)
Date: Thu, 08 Dec 2005 16:43:49 -0800
Subject: [Distutils] Using printf from within a C extension
Message-ID: <6.0.1.1.2.20051208164308.01e3df28@mcantor.pobox.stanford.edu>

Hi,

I am extending python with C and trying to debug with printf.   The code 
below succssfully returns the string "hello" when compiled and called, but 
the "can print from in here phrase" does not reach python stdout.  Is there 
something screwy with my environment or is there some trick to this that I 
don't know.  Any help would be greatly appreciated!

-mike

#include <Python.h>

static PyObject*
helloworld(PyObject* self)
{
     printf("can print from in here?");
     return Py_BuildValue("s", "hello");
}

static char hw_docstr[] = "hello docstr";

static PyMethodDef hw_funcs[] = {
     {"helloworld", (PyCFunction)helloworld, METH_NOARGS, hw_docstr},
     {NULL}
};

void
inithelloworld(void)
{
     Py_InitModule3("helloworld",hw_funcs, "simple module");
} 



From bob at redivi.com  Fri Dec  9 03:09:09 2005
From: bob at redivi.com (Bob Ippolito)
Date: Thu, 8 Dec 2005 18:09:09 -0800
Subject: [Distutils] Using printf from within a C extension
In-Reply-To: <6.0.1.1.2.20051208164308.01e3df28@mcantor.pobox.stanford.edu>
References: <6.0.1.1.2.20051208164308.01e3df28@mcantor.pobox.stanford.edu>
Message-ID: <CA08B09A-3B07-4847-A82E-7DF8E95363FF@redivi.com>


On Dec 8, 2005, at 4:43 PM, mike cantor wrote:

> I am extending python with C and trying to debug with printf.   The  
> code
> below succssfully returns the string "hello" when compiled and  
> called, but
> the "can print from in here phrase" does not reach python stdout.   
> Is there
> something screwy with my environment or is there some trick to this  
> that I
> don't know.  Any help would be greatly appreciated!
>
> -mike
>
> #include <Python.h>
>
> static PyObject*
> helloworld(PyObject* self)
> {
>      printf("can print from in here?");
>      return Py_BuildValue("s", "hello");
> }

It'll probably work if you add a line break there.  It's likely just  
sitting in a buffer waiting to be flushed.

-bob


From pje at telecommunity.com  Fri Dec  9 06:07:56 2005
From: pje at telecommunity.com (Phillip J. Eby)
Date: Fri, 09 Dec 2005 00:07:56 -0500
Subject: [Distutils] local development scenarios
In-Reply-To: <439907CE.7070503@yahoo.com>
References: <5.1.1.6.0.20051208114741.03eb3680@mail.telecommunity.com>
	<5.1.1.6.0.20051208114741.03eb3680@mail.telecommunity.com>
Message-ID: <5.1.1.6.0.20051208234423.02d580e8@mail.telecommunity.com>

At 08:27 PM 12/8/2005 -0800, Todd Greenwood-Geer wrote:
small_unittest.py::

>         import small
>         import unittest
>         import doctest
>
>         def getTestSuite():
>                 suite = unittest.TestSuite()
>                 for mod in small,:
>                         suite.addTest(doctest.DocTestSuite(mod))
>                 return suite
>
>         runner = unittest.TextTestRunner()
>         runner.run(getTestSuite())

If 'small.small_unittest' is the actual name of the module, then doing this:

   python -c "from unittest import main; main(None)" 
small.small_unittest.getTestSuite

will run the test, even if it's inside an egg.  No unzipping is 
necessary.  If the egg wasn't installed with a .pth file (i.e. 
--multi-version or to a non-standard location), you'll need to make sure 
it's on PYTHONPATH, along with its dependencies.

>Here is what the src tree for the simple example looks like:
>
>dir listing::
>
>         small/src/small/test/test.txt
>         small/src/small/small2.py
>         small/src/small/small.py
>         small/src/small/small_unittest.pyc
>         small/src/small/__init__.py
>         small/src/small/small_unittest.py
>         small/src/small_unittest.py

This looks like a duplicate.  I'm also not sure why you have a small2, or 
why you bother having a separate test subdirectory.  Indeed, for this 
"small" of an example, a src/ directory is probably overkill.  For clarity, 
I'd also suggest calling the topmost directory something like SmallExample 
to help distinguish your project name (SmallExample) from your package name 
(small).

>         small/setup_mult.py

What's this file for?


>         small/ez_setup.py

If you're using Subversion, btw, you can use the svn:externals trick so 
that you don't have to manually maintain this file; see the setuptools 
manual under:

http://peak.telecommunity.com/DevCenter/setuptools#using-setuptools-without-bundling-it



>The thing to note here is how small.small and small.small2 are referenced::
>
>         from small import small as s1
>         from small import small2 as s2
>
>For some reason, this gave me plenty of problems.

It's generally not a good idea to name a module the same as a package, and 
definitely not the same as a class within the module, or else it gets 
unclear in code which one you're talking about.  In a language like Java, 
there's no such ambiguity because syntactically you can't refer to a class 
where a package should be or vice versa, but in Python there are only 
objects, so you should not name them all the same thing or you (and/or the 
interpreter) will be confused.  :)

Similarly, I suspect that your example has way too many files perhaps 
because you have a Java background?  It really isn't necessary to split 
Python projects up into so many small pieces; it just makes more work for 
you and doesn't get you any benefit until the files get too large to 
conveniently work with.


>Complex Case
>------------
>
>The complex case splits out the test modules into a nested package 
>hierarchy like this:
...
>Package Hierarchy
>+++++++++++++++++
>
>package hierarchy::
>
>         small/src/
>         small/src/size/
>         small/src/size/small
>         small/src/size/large

Ouch.  I thought the previous example was the complex one.  ;)  Seriously, 
your simple example is way more complex than it needs to be.  This bigger 
one makes my head hurt, so I'm not going to try to comment on it further, 
except to suggest that its top-level directory should be named 
"small-mult-test" since that's your project name.  Also, I'm guessing you 
mean your package hiearchy is size, size.small, and size.large.  The 
small/src stuff isn't part of the package hierarchy, just the directory 
hierarchy.

By the way, the only reason to have a 'src' directory under your main 
project directory is if you have a bunch of stuff that's *not* source code 
there.  Such as documentation, documentation directories, test directories, 
examples, etc.  Your projects here have none of those things, only a setup 
script, so there's no reason to have a 'src' subdirectory; you could just 
put the 'small' or 'size' package directly under the project directory, and 
in fact this is the most common layout for published Python packages.  The 
'src' or 'lib' subdirectory approach is mainly used by large projects with 
a lot of other things in their project directory, and by people who got 
used to that format by working on large projects.  :)


>         from size.small import small
>         from size.large import large

I said I wasn't going to comment further, but this is technically a 
repetition of my earlier comment: please don't name packages and modules 
the same thing, you will confuse yourself and everyone else who will never 
be sure if 'small' means 'size.small' or 'size.small.small' or worse, 
'size.small.small.small'.  Eeek!


>  * url : need to see if that works for downloading dependencies (next 
> tutorial, not this one)

Yes, it does, *if* you register your package with PyPI.  The URL given will 
be scanned for download links when and if easy_install looks for your 
package on PyPI.


From tgreenwoodgeer at yahoo.com  Fri Dec  9 06:57:28 2005
From: tgreenwoodgeer at yahoo.com (Todd Greenwood-Geer)
Date: Thu, 08 Dec 2005 21:57:28 -0800
Subject: [Distutils] local development scenarios
In-Reply-To: <5.1.1.6.0.20051208234423.02d580e8@mail.telecommunity.com>
References: <5.1.1.6.0.20051208114741.03eb3680@mail.telecommunity.com>
	<5.1.1.6.0.20051208114741.03eb3680@mail.telecommunity.com>
	<5.1.1.6.0.20051208234423.02d580e8@mail.telecommunity.com>
Message-ID: <43991CC8.4050005@yahoo.com>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Phillip -

This is great feedback. I wrote this tutorial for myself because, in the
course of writing a more involved application, I realized that I did not
really understand how to use the egg/setuptools functionality. So, the
example is definitely contrived.

So, if I may summarize your guidelines:

1. Add complexity as needed
~ a) keep source in the root dir unless there is a need to differentiate
~ disparate types of files, such as source/docs/tests/etc
~ b) use the svn copy of ez_setup.py rather than checking this in to the
repository

2. Clear use of names
~ a) use different names for packages and modules
~ b) again, nest only as necessary

And, above all else, never admit to having a java background ;o).

Thanks again. I'll re-factor as suggested and throw this at you one more
time.

- -Todd

Phillip J. Eby wrote:
| At 08:27 PM 12/8/2005 -0800, Todd Greenwood-Geer wrote:
| small_unittest.py::
|
|>         import small
|>         import unittest
|>         import doctest
|>
|>         def getTestSuite():
|>                 suite = unittest.TestSuite()
|>                 for mod in small,:
|>                         suite.addTest(doctest.DocTestSuite(mod))
|>                 return suite
|>
|>         runner = unittest.TextTestRunner()
|>         runner.run(getTestSuite())
|
| If 'small.small_unittest' is the actual name of the module, then doing
this:
|
|    python -c "from unittest import main; main(None)"
| small.small_unittest.getTestSuite
|
| will run the test, even if it's inside an egg.  No unzipping is
| necessary.  If the egg wasn't installed with a .pth file (i.e.
| --multi-version or to a non-standard location), you'll need to make sure
| it's on PYTHONPATH, along with its dependencies.
|
|> Here is what the src tree for the simple example looks like:
|>
|> dir listing::
|>
|>         small/src/small/test/test.txt
|>         small/src/small/small2.py
|>         small/src/small/small.py
|>         small/src/small/small_unittest.pyc
|>         small/src/small/__init__.py
|>         small/src/small/small_unittest.py
|>         small/src/small_unittest.py
|
| This looks like a duplicate.  I'm also not sure why you have a small2, or
| why you bother having a separate test subdirectory.  Indeed, for this
| "small" of an example, a src/ directory is probably overkill.  For
clarity,
| I'd also suggest calling the topmost directory something like
SmallExample
| to help distinguish your project name (SmallExample) from your package
name
| (small).
|
|>         small/setup_mult.py
|

| What's this file for?
|
|
|>         small/ez_setup.py
|
| If you're using Subversion, btw, you can use the svn:externals trick so
| that you don't have to manually maintain this file; see the setuptools
| manual under:
|
|
http://peak.telecommunity.com/DevCenter/setuptools#using-setuptools-without-bundling-it
|
|
|
|> The thing to note here is how small.small and small.small2 are
referenced::
|>
|>         from small import small as s1
|>         from small import small2 as s2
|>
|> For some reason, this gave me plenty of problems.
|
| It's generally not a good idea to name a module the same as a package,
and
| definitely not the same as a class within the module, or else it gets
| unclear in code which one you're talking about.  In a language like Java,
| there's no such ambiguity because syntactically you can't refer to a
class
| where a package should be or vice versa, but in Python there are only
| objects, so you should not name them all the same thing or you (and/or
the
| interpreter) will be confused.  :)
|
| Similarly, I suspect that your example has way too many files perhaps
| because you have a Java background?  It really isn't necessary to split
| Python projects up into so many small pieces; it just makes more work for
| you and doesn't get you any benefit until the files get too large to
| conveniently work with.
|
|
|> Complex Case
|> ------------
|>
|> The complex case splits out the test modules into a nested package
|> hierarchy like this:
| ...
|> Package Hierarchy
|> +++++++++++++++++
|>
|> package hierarchy::
|>
|>         small/src/
|>         small/src/size/
|>         small/src/size/small
|>         small/src/size/large
|
| Ouch.  I thought the previous example was the complex one.  ;)
Seriously,
| your simple example is way more complex than it needs to be.  This bigger
| one makes my head hurt, so I'm not going to try to comment on it further,
| except to suggest that its top-level directory should be named
| "small-mult-test" since that's your project name.  Also, I'm guessing you
| mean your package hiearchy is size, size.small, and size.large.  The
| small/src stuff isn't part of the package hierarchy, just the directory
| hierarchy.
|
| By the way, the only reason to have a 'src' directory under your main
| project directory is if you have a bunch of stuff that's *not* source
code
| there.  Such as documentation, documentation directories, test
directories,
| examples, etc.  Your projects here have none of those things, only a
setup
| script, so there's no reason to have a 'src' subdirectory; you could just
| put the 'small' or 'size' package directly under the project
directory, and
| in fact this is the most common layout for published Python packages.
  The
| 'src' or 'lib' subdirectory approach is mainly used by large projects
with
| a lot of other things in their project directory, and by people who got
| used to that format by working on large projects.  :)
|
|
|>         from size.small import small
|>         from size.large import large
|
| I said I wasn't going to comment further, but this is technically a
| repetition of my earlier comment: please don't name packages and modules
| the same thing, you will confuse yourself and everyone else who will
never
| be sure if 'small' means 'size.small' or 'size.small.small' or worse,
| 'size.small.small.small'.  Eeek!
|
|
|>  * url : need to see if that works for downloading dependencies (next
|> tutorial, not this one)
|
| Yes, it does, *if* you register your package with PyPI.  The URL given
will
| be scanned for download links when and if easy_install looks for your
| package on PyPI.
|
|
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.1 (GNU/Linux)

iD8DBQFDmRzIz6uXX4lQc/URAsaWAKCXNcWjf7o59m2MgM2B8Nmx1CfLewCgpqOW
wA2wfi08C4RPW151NngP7BU=
=N5e6
-----END PGP SIGNATURE-----

From tgreenwoodgeer at yahoo.com  Fri Dec  9 07:42:00 2005
From: tgreenwoodgeer at yahoo.com (Todd Greenwood-Geer)
Date: Thu, 08 Dec 2005 22:42:00 -0800
Subject: [Distutils] local development scenarios
In-Reply-To: <43991CC8.4050005@yahoo.com>
References: <5.1.1.6.0.20051208114741.03eb3680@mail.telecommunity.com>
	<5.1.1.6.0.20051208114741.03eb3680@mail.telecommunity.com>
	<5.1.1.6.0.20051208234423.02d580e8@mail.telecommunity.com>
	<43991CC8.4050005@yahoo.com>
Message-ID: <43992738.1090101@yahoo.com>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Perhaps the docs could flesh out a bit more on the svn/ez_setup trick.

As an example of a full set of instructions:

- - mkdir testApp
- - touch testApp/foo.py
- - svn import testApp file:///....path to my repos
- - cd to some working area
- - svn co file:///path-to-repos/testApp testApp
- - cd testApp
- - svn propedit svn:externals .
- - enter this in to the editor, save and exit >>>
~    ez_setup svn://svn.eby-sarna.com/svnroot/ez_setup
- - modify setup.py like so:
"""
from ez_setup import use_setuptools
use_setuptools()
from setuptools import setup, find_packages

setup(name = "testApp",
~    version = "0.1",
~    description = "test",
~    packages = find_packages(exclude=['ez_setup'], ''),
~    package_dir = {'':''},
~    classifiers = [
~        "Development Status :: 1 - Alpha",
~        "Intended Audience :: Developers",
~        "License :: OSI Approved :: GNU Library or Lesser General Public
License (LGPL)",
~        "Programming Language :: Python",
~        "Topic :: Database :: Front-Ends",
~    ]
~    )
"""
- - check to see your mods to the externals : svn propget svn:externals
- - get the ez_setup files : svn update
- - do a dir listing, you should see:
~  ./ez_setup/
~  ./ez_setup/__init__.py
~  ./ez_setup/README.txt


- - Todd


Todd Greenwood-Geer wrote:
| Phillip -
|
| This is great feedback. I wrote this tutorial for myself because, in the
| course of writing a more involved application, I realized that I did not
| really understand how to use the egg/setuptools functionality. So, the
| example is definitely contrived.
|
| So, if I may summarize your guidelines:
|
| 1. Add complexity as needed
| ~ a) keep source in the root dir unless there is a need to differentiate
| ~ disparate types of files, such as source/docs/tests/etc
| ~ b) use the svn copy of ez_setup.py rather than checking this in to the
| repository
|
| 2. Clear use of names
| ~ a) use different names for packages and modules
| ~ b) again, nest only as necessary
|
| And, above all else, never admit to having a java background ;o).
|
| Thanks again. I'll re-factor as suggested and throw this at you one more
| time.
|
| -Todd
|
| Phillip J. Eby wrote:
| | At 08:27 PM 12/8/2005 -0800, Todd Greenwood-Geer wrote:
| | small_unittest.py::
| |
| |>         import small
| |>         import unittest
| |>         import doctest
| |>
| |>         def getTestSuite():
| |>                 suite = unittest.TestSuite()
| |>                 for mod in small,:
| |>                         suite.addTest(doctest.DocTestSuite(mod))
| |>                 return suite
| |>
| |>         runner = unittest.TextTestRunner()
| |>         runner.run(getTestSuite())
| |
| | If 'small.small_unittest' is the actual name of the module, then doing
| this:
| |
| |    python -c "from unittest import main; main(None)"
| | small.small_unittest.getTestSuite
| |
| | will run the test, even if it's inside an egg.  No unzipping is
| | necessary.  If the egg wasn't installed with a .pth file (i.e.
| | --multi-version or to a non-standard location), you'll need to make sure
| | it's on PYTHONPATH, along with its dependencies.
| |
| |> Here is what the src tree for the simple example looks like:
| |>
| |> dir listing::
| |>
| |>         small/src/small/test/test.txt
| |>         small/src/small/small2.py
| |>         small/src/small/small.py
| |>         small/src/small/small_unittest.pyc
| |>         small/src/small/__init__.py
| |>         small/src/small/small_unittest.py
| |>         small/src/small_unittest.py
| |
| | This looks like a duplicate.  I'm also not sure why you have a
small2, or
| | why you bother having a separate test subdirectory.  Indeed, for this
| | "small" of an example, a src/ directory is probably overkill.  For
| clarity,
| | I'd also suggest calling the topmost directory something like
| SmallExample
| | to help distinguish your project name (SmallExample) from your package
| name
| | (small).
| |
| |>         small/setup_mult.py
| |
|
| | What's this file for?
| |
| |
| |>         small/ez_setup.py
| |
| | If you're using Subversion, btw, you can use the svn:externals trick so
| | that you don't have to manually maintain this file; see the setuptools
| | manual under:
| |
| |
|
http://peak.telecommunity.com/DevCenter/setuptools#using-setuptools-without-bundling-it
| |
| |
| |
| |> The thing to note here is how small.small and small.small2 are
| referenced::
| |>
| |>         from small import small as s1
| |>         from small import small2 as s2
| |>
| |> For some reason, this gave me plenty of problems.
| |
| | It's generally not a good idea to name a module the same as a package,
| and
| | definitely not the same as a class within the module, or else it gets
| | unclear in code which one you're talking about.  In a language like
Java,
| | there's no such ambiguity because syntactically you can't refer to a
| class
| | where a package should be or vice versa, but in Python there are only
| | objects, so you should not name them all the same thing or you (and/or
| the
| | interpreter) will be confused.  :)
| |
| | Similarly, I suspect that your example has way too many files perhaps
| | because you have a Java background?  It really isn't necessary to split
| | Python projects up into so many small pieces; it just makes more
work for
| | you and doesn't get you any benefit until the files get too large to
| | conveniently work with.
| |
| |
| |> Complex Case
| |> ------------
| |>
| |> The complex case splits out the test modules into a nested package
| |> hierarchy like this:
| | ...
| |> Package Hierarchy
| |> +++++++++++++++++
| |>
| |> package hierarchy::
| |>
| |>         small/src/
| |>         small/src/size/
| |>         small/src/size/small
| |>         small/src/size/large
| |
| | Ouch.  I thought the previous example was the complex one.  ;)
| Seriously,
| | your simple example is way more complex than it needs to be.  This
bigger
| | one makes my head hurt, so I'm not going to try to comment on it
further,
| | except to suggest that its top-level directory should be named
| | "small-mult-test" since that's your project name.  Also, I'm
guessing you
| | mean your package hiearchy is size, size.small, and size.large.  The
| | small/src stuff isn't part of the package hierarchy, just the directory
| | hierarchy.
| |
| | By the way, the only reason to have a 'src' directory under your main
| | project directory is if you have a bunch of stuff that's *not* source
| code
| | there.  Such as documentation, documentation directories, test
| directories,
| | examples, etc.  Your projects here have none of those things, only a
| setup
| | script, so there's no reason to have a 'src' subdirectory; you could
just
| | put the 'small' or 'size' package directly under the project
| directory, and
| | in fact this is the most common layout for published Python packages.
|   The
| | 'src' or 'lib' subdirectory approach is mainly used by large projects
| with
| | a lot of other things in their project directory, and by people who got
| | used to that format by working on large projects.  :)
| |
| |
| |>         from size.small import small
| |>         from size.large import large
| |
| | I said I wasn't going to comment further, but this is technically a
| | repetition of my earlier comment: please don't name packages and modules
| | the same thing, you will confuse yourself and everyone else who will
| never
| | be sure if 'small' means 'size.small' or 'size.small.small' or worse,
| | 'size.small.small.small'.  Eeek!
| |
| |
| |>  * url : need to see if that works for downloading dependencies (next
| |> tutorial, not this one)
| |
| | Yes, it does, *if* you register your package with PyPI.  The URL given
| will
| | be scanned for download links when and if easy_install looks for your
| | package on PyPI.
| |
| |
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.1 (GNU/Linux)

iD8DBQFDmSc4z6uXX4lQc/URAoVlAKCzwzH1wr3G9x7nzPOFeWdn97oU/QCgjrqO
q9gT1aNL+ZagzWeoxqnVQIk=
=L/oc
-----END PGP SIGNATURE-----

From tgreenwoodgeer at yahoo.com  Fri Dec  9 05:27:58 2005
From: tgreenwoodgeer at yahoo.com (Todd Greenwood-Geer)
Date: Thu, 08 Dec 2005 20:27:58 -0800
Subject: [Distutils] local development scenarios
In-Reply-To: <5.1.1.6.0.20051208114741.03eb3680@mail.telecommunity.com>
References: <5.1.1.6.0.20051208114741.03eb3680@mail.telecommunity.com>
Message-ID: <439907CE.7070503@yahoo.com>

Great, thanks Phillip. Now that you mention the part about the scripts
generation, I remember reading that in the docs. Thanks for clearing
things up.

Regarding the unit tests...let me add a bit more to the question.
Typically, as I'm developing a module, I embed doctests in the module.
This lets me maintain a known good state throughout my dev cycle. Then,
as I start to wrap things up, I create a standard unit test to wrap the
doctests. Finally, I reference a method in the unit test that gives back
the test suite (code sample attached). I reference this test suite in
the setup.py for the given egg. So the question is, given that my
friend/client is receiving eggs, how can they perform the equivalent of
'$python setup.py test'? Currently, I have suggested that they unzip the
egg and run the unittests dirctly from the unzipped files...something
like this:

- unzip foo.egg
- cd foo
- python src/foo/unittest.py

I'm attaching rest and html formatted copies of my blog tutorial entry
on using setup tools...feel free to comment on that as well.

small.py::

        #example method with doctest compatible docstring
        def removeme():
                """
        >>> import small as m
        >>> m.removeme()
        'remove me'
                """
                return 'remove me'


        def _test():
                import doctest
                return doctest.testmod()

        if __name__ == "__main__":
                _test()


small_unittest.py::

        import small
        import unittest
        import doctest

        def getTestSuite():
                suite = unittest.TestSuite()
                for mod in small,:
                        suite.addTest(doctest.DocTestSuite(mod))
                return suite

        runner = unittest.TextTestRunner()
        runner.run(getTestSuite())

setup.py::

        from ez_setup import use_setuptools
        use_setuptools()
        from setuptools import setup, find_packages

        setup(name = "small",
                version = "0.1",
                description = "test",
                author = "Todd Greenwood",
                author_email = "t.greenwoodgeer at gmail.com",
                url = "http://www.angelfire.com/planet/tango",
                packages = find_packages('src'),
                package_dir = {'':'src'},
                package_data = {'small':['test/*.txt']},
                test_suite = 'small.small_unittest.getTestSuite',
                license = "GNU Lesser General Public License",
                classifiers = [
                "Development Status :: 1 - Alpha",
                "Intended Audience :: Developers",
                "License :: OSI Approved :: GNU Library or Lesser
General Public License (LGPL)",
                "Programming Language :: Python",
                "Topic :: Database :: Front-Ends",
                ]
                )


-Todd



Phillip J. Eby wrote:
> At 10:25 PM 12/7/2005 -0800, tgreenwood wrote:
>> -----BEGIN PGP SIGNED MESSAGE-----
>> Hash: SHA1
>>
>> I'm new to distutils/and the egg system. I have not researched the
>> archives as much as I should, so please excuse me if this is familiar
>> territory...
>>
>> Q: Is it possible for dependency checks to work for non pipy deployed
>> eggs?
>>  - A.egg
>>  - B.egg (setup_requires and install_requires A.egg)
>>  - email A and B eggs to friend
>>  - tell friend to : easy_install B
>>  - but B.egg does not install A...in fact, in my test case, it fails b/c
>> it cannot find A in pipy
> 
> This will work if the eggs are both in the current directory:
> 
>     easy_install -f. B
> 
> -f (aka --find-links) specifies URLs or directories that will be scanned
> *before* going to PyPI to meet dependencies.
> 
> 
>>  - in this example, what happens if friend does not have net connection
>> on target machine? how can they install from the eggs alone and get the
>> dep checking to work?
> 
> No net connection is necessary if all dependencies can be found using -f
> directories.
> 
> 
>> Q: How does my friend run the unittests on the egg that they have just
>> installed? As I am developing, I frequently run the unittests and after
>> every bdist_egg build, I also run the unit tests that way, too. How can
>> the consumer do the same?
> 
> Well, if it's a unit test suite that's contained within the egg, then
> they could use the unittest utility just as they normally would.
> 
> 
>> Q: One of my apps is a command line app, say foo.py is a command line
>> app that takes arg1, arg2,... How can the user run this app when it is
>> encased in an egg?
> 
> EasyInstall generates wrapper scripts during installation that ensure
> the needed eggs are on sys.path and run the scripts.  This works for
> both standard distutils scripts (specified using setup(scripts=[...]))
> and for entry point scripts (see the setuptools doc).
> 
> The --script-dir argument to easy_install controls where scripts are
> installed; it defaults to the same as the installation directory if an
> installation directory is manually specified; otherwise it defaults to
> wherever the distutils are configured to install scripts.
> 
> 
>> Q: How do I define my own pipy? While I like the idea of deploying there
>> for the overall community, I don't want to send my half baked code into
>> the wild. But I'd like to try the general functionality of having a
>> deploy site for my beta customers. How can I set this up?
> 
> Using -f with the URL of a web page with download links to your eggs or
> source distributions.  See http://turbogears.org/download/ for one
> example.  In the simplest case, a simple Apache directory listing will
> do the trick; see http://peak.telecommunity.com/snapshots/ for an
> example of that approach.  Simply having someone use a URL like that
> with -f ensures that any download links on the page will be given
> precedence over PyPI listings, if it's possible to meet all requirements
> that way.
> 
> Only if a dependency can't be satisfied by links on a -f page, will PyPI
> be consulted.
> 
> 
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: small_tutorial.rest
Url: http://mail.python.org/pipermail/distutils-sig/attachments/20051208/d322bcca/small_tutorial-0001.asc
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/distutils-sig/attachments/20051208/d322bcca/small_tutorial-0001.html
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 189 bytes
Desc: OpenPGP digital signature
Url : http://mail.python.org/pipermail/distutils-sig/attachments/20051208/d322bcca/signature-0001.pgp

From jjl at pobox.com  Fri Dec  9 15:27:53 2005
From: jjl at pobox.com (John J Lee)
Date: Fri, 9 Dec 2005 14:27:53 +0000 (GMT Standard Time)
Subject: [Distutils] local development scenarios
In-Reply-To: <439907CE.7070503@yahoo.com>
References: <5.1.1.6.0.20051208114741.03eb3680@mail.telecommunity.com>
	<439907CE.7070503@yahoo.com>
Message-ID: <Pine.WNT.4.64.0512091421360.3480@shaolin>

On Thu, 8 Dec 2005, Todd Greenwood-Geer wrote:
[...]
> the setup.py for the given egg. So the question is, given that my
> friend/client is receiving eggs, how can they perform the equivalent of
> '$python setup.py test'? Currently, I have suggested that they unzip the
> egg and run the unittests dirctly from the unzipped files...something
> like this:
>
> - unzip foo.egg
> - cd foo
> - python src/foo/unittest.py
[...]

1. To avoid confusion, rename your module unittest so as not to have the 
same name as Python stdlib module unittest (don't call it 'test' either, 
for the same reason -- module test contains *Python's* unittests; 
personally I find it unfortunate that standard Python comes with a module 
named 'test', but there it is).  Let's say you call it 'tests'.

2. Make module tests importable from your own package, so one can do:

from mypkg import tests


3. Use setuptools to provide a script that imports and runs module tests 
direct from the egg.


John

From bob at redivi.com  Fri Dec  9 21:39:18 2005
From: bob at redivi.com (Bob Ippolito)
Date: Fri, 9 Dec 2005 12:39:18 -0800
Subject: [Distutils] Determining if a module is in an egg or not
Message-ID: <127CE84B-7E9C-4271-815E-403B6BF19F50@redivi.com>

I started refactoring py2app, modulegraph, etc. for setuptools and  
eggs.  After doing dependency resolution via modulegraph, py2app  
needs to know which of those dependencies are in eggs and where those  
eggs are so that it can just throw the egg (and its dependencies)  
into the application bundle instead of doing the normal py2app thing.

Is there an API to map __file__ paths to eggs?  It seems possible for  
zip and directory eggs by walking up os.path.dirname(__file__) and  
using find_distributions, but that doesn't seem to work for "setup.py  
develop" eggs.

-bob


From pje at telecommunity.com  Fri Dec  9 22:44:33 2005
From: pje at telecommunity.com (Phillip J. Eby)
Date: Fri, 09 Dec 2005 16:44:33 -0500
Subject: [Distutils] Determining if a module is in an egg or not
In-Reply-To: <127CE84B-7E9C-4271-815E-403B6BF19F50@redivi.com>
Message-ID: <5.1.1.6.0.20051209163702.020b3120@mail.telecommunity.com>

At 12:39 PM 12/9/2005 -0800, Bob Ippolito wrote:
>I started refactoring py2app, modulegraph, etc. for setuptools and
>eggs.  After doing dependency resolution via modulegraph, py2app
>needs to know which of those dependencies are in eggs and where those
>eggs are so that it can just throw the egg (and its dependencies)
>into the application bundle instead of doing the normal py2app thing.
>
>Is there an API to map __file__ paths to eggs?  It seems possible for
>zip and directory eggs by walking up os.path.dirname(__file__) and
>using find_distributions, but that doesn't seem to work for "setup.py
>develop" eggs.

You would probably have an easier time if you go the other way, by walking 
a WorkingSet and querying the eggs to see if they contain the modules 
you're looking for, rather than the other way around.  You could use either 
the default working_set (based on current sys.path), or you could create a 
new WorkingSet instance based on a hypothetical sys.path, whatever makes 
sense for your tool.

System packages (which put multiple eggs in the same directory) are a 
tougher problem, though, unless they grow an additional manifest file to 
list all their modules, not just top-level ones.  (Though that really only 
becomes an issue with namespace packages, and of course system packages of 
eggs are still just a possibility rather than a reality.)


From skip at pobox.com  Sun Dec 11 00:23:18 2005
From: skip at pobox.com (skip@pobox.com)
Date: Sat, 10 Dec 2005 17:23:18 -0600
Subject: [Distutils] Installing stuff using setuptools w/ Python from
 CVS HEAD?
In-Reply-To: <5.1.1.6.0.20051208114616.01f6ee30@mail.telecommunity.com>
References: <17303.45983.602184.300426@montanaro.dyndns.org>
	<5.1.1.6.0.20051208114616.01f6ee30@mail.telecommunity.com>
Message-ID: <17307.25446.870065.90928@montanaro.dyndns.org>


    pje> You'll have to install setuptools manually; check it out of
    pje> Subversion and run its "setup.py install".  Then you'll have a
    pje> locally-installed setuptools egg for Python 2.5.

Tried that.  With a fresh svn up of both Python and setuptools and a
from-scratch install of said Python, I get a bus error installing
setuptools.  Looks like a Python ast problem.  I'll report the bug and try
again after a fix is applied...

Skip




From pje at telecommunity.com  Sun Dec 11 03:32:39 2005
From: pje at telecommunity.com (Phillip J. Eby)
Date: Sat, 10 Dec 2005 21:32:39 -0500
Subject: [Distutils] local development scenarios
In-Reply-To: <439B8D5C.5090406@gmail.com>
References: <5.1.1.6.0.20051208234423.02d580e8@mail.telecommunity.com>
	<5.1.1.6.0.20051208114741.03eb3680@mail.telecommunity.com>
	<5.1.1.6.0.20051208114741.03eb3680@mail.telecommunity.com>
	<5.1.1.6.0.20051208234423.02d580e8@mail.telecommunity.com>
Message-ID: <5.1.1.6.0.20051210212653.01f2e2d0@mail.telecommunity.com>

At 06:22 PM 12/10/2005 -0800, tgreenwood wrote:
>Phillip -
>Perhaps this is another obvious rtfm question, but here goes. How can I
>add custom actions to the build? I'm used to writing ant scripts to
>create build targets and chain dependencies, etc.
>
>In the python/egg world, I'd like to do the same thing. For instance,
>I'd like to define a target that would:
>~  - generate the egg ala bdist_egg
>~  - generate my restructured text documentation (vim.vst)...not sure how
>to call this from vim yet, but I'm sure it's possible
>~  - run my unittest suite
>~  - deploy resulting egg to my ftp site


The 'alias' command lets you define new "setup.py" commands as a 
combination of existing commands:

http://peak.telecommunity.com/DevCenter/setuptools#alias-define-shortcuts-for-commonly-used-commands

The 'upload' command sends eggs and/or source distributions to PyPI:

http://peak.telecommunity.com/DevCenter/setuptools#upload-upload-source-and-or-egg-distributions-to-pypi


And you can extend setuptools with new commands via entry points, although 
usually it's best to package these in a separate egg from the project you 
will use them on:

http://peak.telecommunity.com/DevCenter/setuptools#creating-distutils-extensions
http://peak.telecommunity.com/DevCenter/setuptools#adding-commands



From bob at redivi.com  Sun Dec 11 22:37:37 2005
From: bob at redivi.com (Bob Ippolito)
Date: Sun, 11 Dec 2005 13:37:37 -0800
Subject: [Distutils] Splitting large packages into multiple eggs
Message-ID: <8417B72C-0F3F-4404-91AB-8E1BE71CA4F0@redivi.com>

I'm currently refactoring PyObjC and py2app to be setuptools- 
friendly, but one issue I'm coming across is that PyObjC is a large  
package, and normally people wouldn't want to include all of it.  The  
way to do that would be to split it up into lots of eggs, so py2app  
can include the subset of eggs that are necessary.

The issue at hand is how to structure the setup.py to support  
creation of multiple eggs, with an egg for installation purposes that  
depends on everything.  PyObjC can be broken up into about 30 eggs,  
one for each package, one for the Xcode support (which depends on  
py2app and altgraph), one for all of the tests (or maybe separate  
eggs for each test suite).  Obviously I'm not looking to create 30+  
setup.py files, so what do I do?

-bob


From pje at telecommunity.com  Sun Dec 11 23:11:42 2005
From: pje at telecommunity.com (Phillip J. Eby)
Date: Sun, 11 Dec 2005 17:11:42 -0500
Subject: [Distutils] Splitting large packages into multiple eggs
In-Reply-To: <8417B72C-0F3F-4404-91AB-8E1BE71CA4F0@redivi.com>
Message-ID: <5.1.1.6.0.20051211170340.03340a30@mail.telecommunity.com>

At 01:37 PM 12/11/2005 -0800, Bob Ippolito wrote:
>I'm currently refactoring PyObjC and py2app to be setuptools-
>friendly, but one issue I'm coming across is that PyObjC is a large
>package, and normally people wouldn't want to include all of it.  The
>way to do that would be to split it up into lots of eggs, so py2app
>can include the subset of eggs that are necessary.
>
>The issue at hand is how to structure the setup.py to support
>creation of multiple eggs, with an egg for installation purposes that
>depends on everything.  PyObjC can be broken up into about 30 eggs,
>one for each package, one for the Xcode support (which depends on
>py2app and altgraph), one for all of the tests (or maybe separate
>eggs for each test suite).  Obviously I'm not looking to create 30+
>setup.py files, so what do I do?

Um, create a script to generate the setup.py files for you?  ;)

If you don't mind building everything together, you could just create a 
data structure and then call setup() in a "for" loop that loops through all 
the projects...  no, scratch that, it won't work unless you clean out the 
build directory every single time.

Seriously, distutils isn't made for this.  I originally intended to make 
setuptools do something like it for PEAK, and then gave up because it 
really just doesn't work.  Too many of the "install" commands depend on 
just copying everything that's in the build directories, which means you'll 
get inter-project crosstalk.

The One Obvious Way (30+ setup.py files) is in fact the only practical way 
without some pretty major work on the distutils or scaffolding to work 
around them.  I'd suggest, however, that maybe one egg per package is too 
fine-grained, and you break it into just a handful of eggs instead.


From bob at redivi.com  Sun Dec 11 23:49:08 2005
From: bob at redivi.com (Bob Ippolito)
Date: Sun, 11 Dec 2005 14:49:08 -0800
Subject: [Distutils] Splitting large packages into multiple eggs
In-Reply-To: <5.1.1.6.0.20051211170340.03340a30@mail.telecommunity.com>
References: <5.1.1.6.0.20051211170340.03340a30@mail.telecommunity.com>
Message-ID: <28E84919-5544-46EA-A0D9-E50413053543@redivi.com>


On Dec 11, 2005, at 2:11 PM, Phillip J. Eby wrote:

> At 01:37 PM 12/11/2005 -0800, Bob Ippolito wrote:
>> I'm currently refactoring PyObjC and py2app to be setuptools-
>> friendly, but one issue I'm coming across is that PyObjC is a large
>> package, and normally people wouldn't want to include all of it.  The
>> way to do that would be to split it up into lots of eggs, so py2app
>> can include the subset of eggs that are necessary.
>>
>> The issue at hand is how to structure the setup.py to support
>> creation of multiple eggs, with an egg for installation purposes that
>> depends on everything.  PyObjC can be broken up into about 30 eggs,
>> one for each package, one for the Xcode support (which depends on
>> py2app and altgraph), one for all of the tests (or maybe separate
>> eggs for each test suite).  Obviously I'm not looking to create 30+
>> setup.py files, so what do I do?
>
> Um, create a script to generate the setup.py files for you?  ;)
>
> If you don't mind building everything together, you could just  
> create a data structure and then call setup() in a "for" loop that  
> loops through all the projects...  no, scratch that, it won't work  
> unless you clean out the build directory every single time.
>
> Seriously, distutils isn't made for this.  I originally intended to  
> make setuptools do something like it for PEAK, and then gave up  
> because it really just doesn't work.  Too many of the "install"  
> commands depend on just copying everything that's in the build  
> directories, which means you'll get inter-project crosstalk.
>
> The One Obvious Way (30+ setup.py files) is in fact the only  
> practical way without some pretty major work on the distutils or  
> scaffolding to work around them.  I'd suggest, however, that maybe  
> one egg per package is too fine-grained, and you break it into just  
> a handful of eggs instead.

Ok, so I'll start off with a handful of setup.py files to see how it  
works out..

For install_requires and setup_requires, how I can I let ez_setup  
know that the subprojects are in the same tarball relative to the  
main setup.py?

-bob


From etrepum at mac.com  Mon Dec 12 04:02:43 2005
From: etrepum at mac.com (Bob Ippolito)
Date: Sun, 11 Dec 2005 19:02:43 -0800
Subject: [Distutils] Splitting large packages into multiple eggs
In-Reply-To: <28E84919-5544-46EA-A0D9-E50413053543@redivi.com>
References: <5.1.1.6.0.20051211170340.03340a30@mail.telecommunity.com>
	<28E84919-5544-46EA-A0D9-E50413053543@redivi.com>
Message-ID: <A000A1AF-A063-45A6-9123-9129BFFCC086@mac.com>


On Dec 11, 2005, at 2:49 PM, Bob Ippolito wrote:

>
> On Dec 11, 2005, at 2:11 PM, Phillip J. Eby wrote:
>
>> At 01:37 PM 12/11/2005 -0800, Bob Ippolito wrote:
>>> I'm currently refactoring PyObjC and py2app to be setuptools-
>>> friendly, but one issue I'm coming across is that PyObjC is a large
>>> package, and normally people wouldn't want to include all of it.   
>>> The
>>> way to do that would be to split it up into lots of eggs, so py2app
>>> can include the subset of eggs that are necessary.
>>>
>>> The issue at hand is how to structure the setup.py to support
>>> creation of multiple eggs, with an egg for installation purposes  
>>> that
>>> depends on everything.  PyObjC can be broken up into about 30 eggs,
>>> one for each package, one for the Xcode support (which depends on
>>> py2app and altgraph), one for all of the tests (or maybe separate
>>> eggs for each test suite).  Obviously I'm not looking to create 30+
>>> setup.py files, so what do I do?
>>
>> Um, create a script to generate the setup.py files for you?  ;)
>>
>> If you don't mind building everything together, you could just  
>> create a data structure and then call setup() in a "for" loop that  
>> loops through all the projects...  no, scratch that, it won't work  
>> unless you clean out the build directory every single time.
>>
>> Seriously, distutils isn't made for this.  I originally intended  
>> to make setuptools do something like it for PEAK, and then gave up  
>> because it really just doesn't work.  Too many of the "install"  
>> commands depend on just copying everything that's in the build  
>> directories, which means you'll get inter-project crosstalk.
>>
>> The One Obvious Way (30+ setup.py files) is in fact the only  
>> practical way without some pretty major work on the distutils or  
>> scaffolding to work around them.  I'd suggest, however, that maybe  
>> one egg per package is too fine-grained, and you break it into  
>> just a handful of eggs instead.
>
> Ok, so I'll start off with a handful of setup.py files to see how  
> it works out..
>
> For install_requires and setup_requires, how I can I let ez_setup  
> know that the subprojects are in the same tarball relative to the  
> main setup.py?

It looks like easy_install can't do this yet.  Here's the layout of  
what I have right now:
http://svn.red-bean.com/pyobjc/branches/pyobjc-setuptools/

The main setup.py is at:
http://svn.red-bean.com/pyobjc/branches/pyobjc-setuptools/setup.py

and each subproject lives in here:
http://svn.red-bean.com/pyobjc/branches/pyobjc-setuptools/subprojects/

Each subproject contains only a setup.py... The setup.py performs its  
job by changing back to the main source directory and then building  
some stuff as if it were just the main setup.py.  It's done this way  
because there are a lot of include files that these things share, and  
it makes the refactoring less painful.

It seems that the PackageIndex only wants to find eggs and source  
packages, but not source dirs... I can't make source packages out of  
them because they refer to source that exists elsewhere (in the  
parent tree).  I'm not very familiar with the sources yet, so it  
would take a while for me to write such a patch.

Also, I'm going to want a way to have "setup.py develop" ensure that  
all of the subprojects are up to date.

-bob


From robert.kern at gmail.com  Mon Dec 12 04:55:27 2005
From: robert.kern at gmail.com (Robert Kern)
Date: Sun, 11 Dec 2005 19:55:27 -0800
Subject: [Distutils] Splitting large packages into multiple eggs
In-Reply-To: <8417B72C-0F3F-4404-91AB-8E1BE71CA4F0@redivi.com>
References: <8417B72C-0F3F-4404-91AB-8E1BE71CA4F0@redivi.com>
Message-ID: <dnisbg$hee$1@sea.gmane.org>

Bob Ippolito wrote:

> The issue at hand is how to structure the setup.py to support  
> creation of multiple eggs, with an egg for installation purposes that  
> depends on everything.  PyObjC can be broken up into about 30 eggs,  
> one for each package, one for the Xcode support (which depends on  
> py2app and altgraph), one for all of the tests (or maybe separate  
> eggs for each test suite).  Obviously I'm not looking to create 30+  
> setup.py files, so what do I do?

You may want to look at how the new scipy.distutils scheme works. We have a
utility class Configuration which encapsulates everything about the setup.
Eventually, it creates the **kwds for setup() from that information. The
important bits of our main setup.py looks something like this:

  from scipy.distutils.misc_util import Configuration
  config = Configuration(
                         maintainer = "SciPy Developers",
                         # ...
                         )
  # Force scipy to be a package (its __init__.py file comes from scipy_core)
  config.packages.append('scipy')
  config.package_dir['scipy'] = os.path.join(config.local_path,'Lib')

  config.add_subpackage('Lib')

  setup(**config.todict())

In this case, the Lib/ subdirectory in the source tree will become the main
scipy package. It has its own setup.py:

  def configuration(parent_package='',top_path=None):
    from scipy.distutils.misc_util import Configuration
    config = Configuration('scipy',parent_package,top_path)
    config.add_subpackage('sandbox')
    config.add_subpackage('utils')
    config.add_subpackage('io')
    config.add_subpackage('fftpack')
    config.add_subpackage('signal')
    config.add_subpackage('integrate')
    config.add_subpackage('linalg')
    config.add_subpackage('special')
    config.add_subpackage('optimize')
    config.add_subpackage('stats')
    config.add_subpackage('interpolate')
    config.add_subpackage('sparse')
    config.add_subpackage('cluster')
    config.add_subpackage('lib')
    config.make_svn_version_py()  # installs __svn_version__.py
    config.make_config_py('__scipy_config__')
    return config

Each of these subdirectories has their own setup.py files, too. E.g.:

  def configuration(parent_package='',top_path=None):
    from scipy.distutils.misc_util import Configuration
    config = Configuration('io', parent_package, top_path)
    config.add_extension('numpyio',
                         sources = ['numpyiomodule.c'])
    config.add_data_dir('tests')
    config.add_data_dir('examples')
    config.add_data_dir('docs')
    return config

All of these file names and directories are local. I could move subpackages from
scipy.sandbox.*, say, into scipy.*, and all I'd have to change are the
config.add_subpackage() calls. The Configuration class does all of the bookkeeping.

One could selectively build certain subpackages. One of these days I am going to
write a script that will read data from a configuration file to determine which
subpackages to build.

The Configuration class seems to be fairly decoupled from the rest of
scipy.distutils. I think you only have to change a few

  from scipy.distutils.core import Extension

to

  from setuptools import Extension

http://svn.scipy.org/svn/scipy_core/trunk/scipy/distutils/misc_util.py

-- 
Robert Kern
robert.kern at gmail.com

"In the fields of hell where the grass grows high
 Are the graves of dreams allowed to die."
  -- Richard Harter


From bob at redivi.com  Mon Dec 12 05:23:30 2005
From: bob at redivi.com (Bob Ippolito)
Date: Sun, 11 Dec 2005 20:23:30 -0800
Subject: [Distutils] Splitting large packages into multiple eggs
In-Reply-To: <dnisbg$hee$1@sea.gmane.org>
References: <8417B72C-0F3F-4404-91AB-8E1BE71CA4F0@redivi.com>
	<dnisbg$hee$1@sea.gmane.org>
Message-ID: <5C282174-93C0-4C2B-9966-9E80CEF4081F@redivi.com>


On Dec 11, 2005, at 7:55 PM, Robert Kern wrote:

> Bob Ippolito wrote:
>
>> The issue at hand is how to structure the setup.py to support
>> creation of multiple eggs, with an egg for installation purposes that
>> depends on everything.  PyObjC can be broken up into about 30 eggs,
>> one for each package, one for the Xcode support (which depends on
>> py2app and altgraph), one for all of the tests (or maybe separate
>> eggs for each test suite).  Obviously I'm not looking to create 30+
>> setup.py files, so what do I do?
>
> You may want to look at how the new scipy.distutils scheme works.  
> We have a
> utility class Configuration which encapsulates everything about the  
> setup.
> Eventually, it creates the **kwds for setup() from that  
> information. The
> important bits of our main setup.py looks something like this:

That's interesting, but it's not relevant...  The point is that there  
needs to be separate setup.py files because we're making separate  
eggs.  I'm making separate eggs for packaging reasons.  Basically:

- there's 3.4M of tests for the core
- a handful of the packages aren't compatible with some supported  
platforms
- the Xcode support depends on a whole slew of other packages you  
don't otherwise want.

This was traditionally not a problem because there was no dependency  
management, plus py2app simply included whatever it detected was used  
on a per-module level.  For the next release of py2app I'm going to  
make it wholly include any egg that is used, which would be very bad  
for PyObjC if it were a single egg.

-bob


From robert.kern at gmail.com  Mon Dec 12 05:28:22 2005
From: robert.kern at gmail.com (Robert Kern)
Date: Sun, 11 Dec 2005 20:28:22 -0800
Subject: [Distutils] Splitting large packages into multiple eggs
In-Reply-To: <5C282174-93C0-4C2B-9966-9E80CEF4081F@redivi.com>
References: <8417B72C-0F3F-4404-91AB-8E1BE71CA4F0@redivi.com>	<dnisbg$hee$1@sea.gmane.org>
	<5C282174-93C0-4C2B-9966-9E80CEF4081F@redivi.com>
Message-ID: <dniu97$l83$1@sea.gmane.org>

Bob Ippolito wrote:

> That's interesting, but it's not relevant...  The point is that there  
> needs to be separate setup.py files because we're making separate  
> eggs.  I'm making separate eggs for packaging reasons.

Ah, that's right. PyObjC isn't organized into a namespace package. Never mind.

-- 
Robert Kern
robert.kern at gmail.com

"In the fields of hell where the grass grows high
 Are the graves of dreams allowed to die."
  -- Richard Harter


From bob at redivi.com  Mon Dec 12 22:13:40 2005
From: bob at redivi.com (Bob Ippolito)
Date: Mon, 12 Dec 2005 13:13:40 -0800
Subject: [Distutils] [Pyobjc-checkins] [PyObjC-svn] r1763 - in
	branches/pyobjc-setuptools: . Lib/pyobjc.egg-info
In-Reply-To: <549ACA27-584E-424B-898A-36FF16682886@mac.com>
References: <E1ElYiM-00082X-5U@sanpietro.red-bean.com>
	<549ACA27-584E-424B-898A-36FF16682886@mac.com>
Message-ID: <51DF1258-632D-4E96-9DCC-5F4BA36D00A1@redivi.com>

On Dec 11, 2005, at 1:45 PM, Ronald Oussoren wrote:

> On 11-dec-2005, at 22:26, bob at red-bean.com wrote:
>
>> Author: bob
>> Date: Sun Dec 11 15:26:36 2005
>> New Revision: 1763
>>
>> Modified:
>>    branches/pyobjc-setuptools/Lib/pyobjc.egg-info/PKG-INFO
>>    branches/pyobjc-setuptools/Lib/pyobjc.egg-info/requires.txt
>
> Should these be in the repository in the first place? AFAIK  
> the .egg-info directory is created and maintained by setuptools.

I don't know, I've seen them in other repositories and there's one in  
setuptools itself.

-bob


From pje at telecommunity.com  Mon Dec 12 22:32:24 2005
From: pje at telecommunity.com (Phillip J. Eby)
Date: Mon, 12 Dec 2005 16:32:24 -0500
Subject: [Distutils] [Pyobjc-checkins] [PyObjC-svn] r1763 - in
 branches/pyobjc-setuptools: . Lib/pyobjc.egg-info
In-Reply-To: <51DF1258-632D-4E96-9DCC-5F4BA36D00A1@redivi.com>
References: <549ACA27-584E-424B-898A-36FF16682886@mac.com>
	<E1ElYiM-00082X-5U@sanpietro.red-bean.com>
	<549ACA27-584E-424B-898A-36FF16682886@mac.com>
Message-ID: <5.1.1.6.0.20051212162950.0353c9f0@mail.telecommunity.com>

At 01:13 PM 12/12/2005 -0800, Bob Ippolito wrote:
>On Dec 11, 2005, at 1:45 PM, Ronald Oussoren wrote:
>
> > On 11-dec-2005, at 22:26, bob at red-bean.com wrote:
> >
> >> Author: bob
> >> Date: Sun Dec 11 15:26:36 2005
> >> New Revision: 1763
> >>
> >> Modified:
> >>    branches/pyobjc-setuptools/Lib/pyobjc.egg-info/PKG-INFO
> >>    branches/pyobjc-setuptools/Lib/pyobjc.egg-info/requires.txt
> >
> > Should these be in the repository in the first place? AFAIK
> > the .egg-info directory is created and maintained by setuptools.
>
>I don't know, I've seen them in other repositories and there's one in
>setuptools itself.

It is only needed if you have manually maintained files in 
there.  Setuptools needs one because its own entry points can't be 
bootstrapped any other way.  Other projects have manually-created files 
like for SQLObject schema stuff, and an older version of Trac had a 
configuration file in there.

If you aren't manually maintaining files in there, then no, you don't need 
it in Subversion and can add it to your svn:ignore list for the parent 
directory.

(Btw, I haven't forgotten about the split-up discussion and have more 
comments, but I've got a deadline to meet today; will write more this evening.)


From mal at egenix.com  Tue Dec 13 12:08:29 2005
From: mal at egenix.com (M.-A. Lemburg)
Date: Tue, 13 Dec 2005 12:08:29 +0100
Subject: [Distutils] eggs on distutils-sig
In-Reply-To: <C2A49395-EF6A-4344-B086-1B63063AD76D@nist.gov>
References: <4396AD9F.1050009@egenix.com>	<3f085ecd0512070516h34290103gaaa37a44e952dfb7@mail.gmail.com>	<43971782.2070801@colorstudy.com>
	<C2A49395-EF6A-4344-B086-1B63063AD76D@nist.gov>
Message-ID: <439EABAD.5030008@egenix.com>

Stephen Langer wrote:
> On Dec 7, 2005, at 12:10 PM, Ian Bicking wrote:
> 
>> Kevin Dangoor wrote:
>>> -1
>>>
>>> In the months that I've been subscribed to this list, I haven't  
>>> really
>>> seen any *other* discussion that seems geared towards improving
>>> distribution of Python modules and add-ons.
>> Ditto.  Setuptools is the most active distutils-related development
>> right now, no surprise there's lots of messages on it.  The volume  
>> is a
>> *good* thing ;)
> 
> I guess the concern is that the dominance of setuptools message is  
> scaring away people with old-fashioned distutils questions.  There  
> haven't been many of those lately, and some of them seem to have  
> received no response.  I don't think that splitting the list in two  
> is a solution, though.  There's not enough traffic to warrant that  
> (except for a brief spurt a week or so ago...)

Point taken.

I would like to get the development and integration of important
distribution formats going again.

For me, bdist_deb, bdist_msi, bdist_nsis, bdist_inno as well as
some other formats used by HP-UX and Solaris are more important
than inventing yet another package manager.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Dec 13 2005)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! ::::

From mal at egenix.com  Tue Dec 13 12:26:17 2005
From: mal at egenix.com (M.-A. Lemburg)
Date: Tue, 13 Dec 2005 12:26:17 +0100
Subject: [Distutils] bdist_nsis, bdist_deb, bdist_msi, bdist_inno,
 bdist_pkgtool, bdist_sdux, etc.
Message-ID: <439EAFD9.8060005@egenix.com>

Hi,

I'd like to focus attention on these distribution formats
that distutils is missing:

bdist_nsis:
  Nullsoft installer for Windows (NSIS); by Jean-Christophe G.
  http://mail.python.org/pipermail/distutils-sig/2005-July/004724.html
  http://bdist-nsi.sourceforge.net/

bdist_deb:
  Debian package; by Geoffrey T. Dairiki
  http://lists.debian.org/debian-python/2004/10/msg00017.html

bdist_msi:
  MS Installer (MSI); by Martin von L?wis
  http://mail.python.org/pipermail/distutils-sig/2005-August/005008.html

bdist_inno:
  InnoSetup installer for Windows
  Included in py2exe by Thomas Heller:
  http://starship.python.net/crew/theller/py2exe/

bdist_pkgtool:
  Package format used on Solaris
  There was a code contribution by Mark Alexander for this, but it had
to be removed due to licensing issues:

http://cvs.sourceforge.net/viewcvs.py/python/python/dist/src/Lib/distutils/command/Attic/

bdist_sdux:
  Package format used on HP-UX
  Same issues as with bdist_pkgtool.

Ideally, I'd like to see all of these added to distutils. The
standard procedure for this is to:

1. Find a maintainer for the code; ideally the author him/herself

2. Get the author to sign the PSF contribution form:

   http://www.python.org/psf/contrib.html

3. Add the code to distutils

Since these are plain additions to distutils, I don't think
we need a PEP for the integration.

Would any of the authors (I've tried to CC them) be willing to
contribute their distutils command to the PSF for integration ?

Thanks,
-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Dec 13 2005)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! ::::

From mal at egenix.com  Tue Dec 13 12:34:07 2005
From: mal at egenix.com (M.-A. Lemburg)
Date: Tue, 13 Dec 2005 12:34:07 +0100
Subject: [Distutils] bdist_nsis, bdist_deb, bdist_msi, bdist_inno,
 bdist_pkgtool, bdist_sdux, bdist_mpkg, etc.
In-Reply-To: <439EAFD9.8060005@egenix.com>
References: <439EAFD9.8060005@egenix.com>
Message-ID: <439EB1AF.4040809@egenix.com>

M.-A. Lemburg wrote:
> Hi,
> 
> I'd like to focus attention on these distribution formats
> that distutils is missing:
> 

I'd like to another important candidate to the list:

bdist_mpkg:
   Mac OS X installer format
   Part of py2app by Bob Ippolito
   http://undefined.org/python/py2app.html#bdist-mpkg-documentation

> Ideally, I'd like to see all of these added to distutils. The
> standard procedure for this is to:
> 
> 1. Find a maintainer for the code; ideally the author him/herself
> 
> 2. Get the author to sign the PSF contribution form:
> 
>    http://www.python.org/psf/contrib.html
> 
> 3. Add the code to distutils
> 
> Since these are plain additions to distutils, I don't think
> we need a PEP for the integration.
> 
> Would any of the authors (I've tried to CC them) be willing to
> contribute their distutils command to the PSF for integration ?

Thanks,
-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Dec 13 2005)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! ::::

From theller at python.net  Tue Dec 13 17:06:25 2005
From: theller at python.net (Thomas Heller)
Date: Tue, 13 Dec 2005 17:06:25 +0100
Subject: [Distutils] bdist_nsis, bdist_deb, bdist_msi, bdist_inno,
 bdist_pkgtool, bdist_sdux, etc.
References: <439EAFD9.8060005@egenix.com>
Message-ID: <u0ddkmm6.fsf@python.net>

"M.-A. Lemburg" <mal at egenix.com> writes:

> Hi,
>
> I'd like to focus attention on these distribution formats
> that distutils is missing:
>
> bdist_nsis:
>   Nullsoft installer for Windows (NSIS); by Jean-Christophe G.
>   http://mail.python.org/pipermail/distutils-sig/2005-July/004724.html
>   http://bdist-nsi.sourceforge.net/
>
> bdist_deb:
>   Debian package; by Geoffrey T. Dairiki
>   http://lists.debian.org/debian-python/2004/10/msg00017.html
>
> bdist_msi:
>   MS Installer (MSI); by Martin von L?wis
>   http://mail.python.org/pipermail/distutils-sig/2005-August/005008.html
>
> bdist_inno:
>   InnoSetup installer for Windows
>   Included in py2exe by Thomas Heller:
>   http://starship.python.net/crew/theller/py2exe/

That's not really a distutils distribution format, it is a sample setup
script which builds an exe file plus the needed support files with
py2exe, and then builds a Windows installer to install all this as an
application.  What do you have in mind - a bdist_wininst lookalike for
Python packages, built with inno?

Thomas


From mal at egenix.com  Tue Dec 13 18:01:12 2005
From: mal at egenix.com (M.-A. Lemburg)
Date: Tue, 13 Dec 2005 18:01:12 +0100
Subject: [Distutils] bdist_nsis, bdist_deb, bdist_msi, bdist_inno,
 bdist_pkgtool, bdist_sdux, etc.
In-Reply-To: <u0ddkmm6.fsf@python.net>
References: <439EAFD9.8060005@egenix.com> <u0ddkmm6.fsf@python.net>
Message-ID: <439EFE58.7050504@egenix.com>

Thomas Heller wrote:
> "M.-A. Lemburg" <mal at egenix.com> writes:
> 
>> Hi,
>>
>> I'd like to focus attention on these distribution formats
>> that distutils is missing:
>>
>> bdist_nsis:
>>   Nullsoft installer for Windows (NSIS); by Jean-Christophe G.
>>   http://mail.python.org/pipermail/distutils-sig/2005-July/004724.html
>>   http://bdist-nsi.sourceforge.net/
>>
>> bdist_deb:
>>   Debian package; by Geoffrey T. Dairiki
>>   http://lists.debian.org/debian-python/2004/10/msg00017.html
>>
>> bdist_msi:
>>   MS Installer (MSI); by Martin von L?wis
>>   http://mail.python.org/pipermail/distutils-sig/2005-August/005008.html
>>
>> bdist_inno:
>>   InnoSetup installer for Windows
>>   Included in py2exe by Thomas Heller:
>>   http://starship.python.net/crew/theller/py2exe/
> 
> That's not really a distutils distribution format, it is a sample setup
> script which builds an exe file plus the needed support files with
> py2exe, and then builds a Windows installer to install all this as an
> application. 

Ah, I see.

> What do you have in mind - a bdist_wininst lookalike for
> Python packages, built with inno?

Yes. One that creates an Inno setup file and then compiles
everything into an installer using the Inno, but installing
things much like bdist_wininst (and with the option of
building upon the many extra features you get from the
InnoSetup system).

Bernhard Herzog started one some time ago:
http://mail.python.org/pipermail/distutils-sig/2001-September/002597.html

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Dec 13 2005)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! ::::

From bob at redivi.com  Tue Dec 13 18:41:33 2005
From: bob at redivi.com (Bob Ippolito)
Date: Tue, 13 Dec 2005 09:41:33 -0800
Subject: [Distutils] bdist_nsis, bdist_deb, bdist_msi, bdist_inno,
	bdist_pkgtool, bdist_sdux, bdist_mpkg, etc.
In-Reply-To: <439EB1AF.4040809@egenix.com>
References: <439EAFD9.8060005@egenix.com> <439EB1AF.4040809@egenix.com>
Message-ID: <4F0360B5-9C0B-487E-BA8C-1210328F6C8C@redivi.com>

On Dec 13, 2005, at 3:34 AM, M.-A. Lemburg wrote:

> M.-A. Lemburg wrote:
>> Hi,
>>
>> I'd like to focus attention on these distribution formats
>> that distutils is missing:
>>
>
> I'd like to another important candidate to the list:
>
> bdist_mpkg:
>    Mac OS X installer format
>    Part of py2app by Bob Ippolito
>    http://undefined.org/python/py2app.html#bdist-mpkg-documentation

bdist_mpkg is actually available on its own now, but I'm not sure  
adding it to distutils is a great idea.  The current version is  
already moved over to setuptools, and I'm pretty close to adding egg  
support to it.  I may even require that the Python bits of the  
installation package are eggs, as that would allow me to remove a  
bunch of code.  Eggs mostly obsolete what bdist_mpkg is for, but it's  
also used to install things like documentation and example code, so  
I'm not going to drop it altogether.

-bob


From bh at intevation.de  Tue Dec 13 18:20:47 2005
From: bh at intevation.de (Bernhard Herzog)
Date: Tue, 13 Dec 2005 18:20:47 +0100
Subject: [Distutils] bdist_nsis, bdist_deb, bdist_msi, bdist_inno,
 bdist_pkgtool, bdist_sdux, etc.
In-Reply-To: <439EFE58.7050504@egenix.com> (M.'s message of "Tue, 13 Dec
	2005 18:01:12 +0100")
References: <439EAFD9.8060005@egenix.com> <u0ddkmm6.fsf@python.net>
	<439EFE58.7050504@egenix.com>
Message-ID: <s9z4q5cq5g0.fsf@thetis.intevation.de>

"M.-A. Lemburg" <mal at egenix.com> writes:

>> What do you have in mind - a bdist_wininst lookalike for
>> Python packages, built with inno?
>
> Yes. One that creates an Inno setup file and then compiles
> everything into an installer using the Inno, but installing
> things much like bdist_wininst (and with the option of
> building upon the many extra features you get from the
> InnoSetup system).
>
> Bernhard Herzog started one some time ago:
> http://mail.python.org/pipermail/distutils-sig/2001-September/002597.html

FWIW, the code that's linked from that posting is still unchanged in
Thuban and worked fine last time I made a release about a year ago with
python 2.3.

   Bernhard

-- 
Intevation GmbH                                 http://intevation.de/
Skencil                                           http://skencil.org/
Thuban                                  http://thuban.intevation.org/

From mal at egenix.com  Tue Dec 13 20:13:09 2005
From: mal at egenix.com (M.-A. Lemburg)
Date: Tue, 13 Dec 2005 20:13:09 +0100
Subject: [Distutils] bdist_nsis, bdist_deb, bdist_msi, bdist_inno,
 bdist_pkgtool, bdist_sdux, bdist_mpkg, etc.
In-Reply-To: <4F0360B5-9C0B-487E-BA8C-1210328F6C8C@redivi.com>
References: <439EAFD9.8060005@egenix.com> <439EB1AF.4040809@egenix.com>
	<4F0360B5-9C0B-487E-BA8C-1210328F6C8C@redivi.com>
Message-ID: <439F1D45.9090509@egenix.com>

Bob Ippolito wrote:
> On Dec 13, 2005, at 3:34 AM, M.-A. Lemburg wrote:
> 
>> M.-A. Lemburg wrote:
>>> Hi,
>>>
>>> I'd like to focus attention on these distribution formats
>>> that distutils is missing:
>>>
>>
>> I'd like to another important candidate to the list:
>>
>> bdist_mpkg:
>>    Mac OS X installer format
>>    Part of py2app by Bob Ippolito
>>    http://undefined.org/python/py2app.html#bdist-mpkg-documentation
> 
> bdist_mpkg is actually available on its own now, but I'm not sure adding
> it to distutils is a great idea.  The current version is already moved
> over to setuptools, and I'm pretty close to adding egg support to it.  I
> may even require that the Python bits of the installation package are
> eggs, as that would allow me to remove a bunch of code.  Eggs mostly
> obsolete what bdist_mpkg is for, but it's also used to install things
> like documentation and example code, so I'm not going to drop it
> altogether.

Looks like now is a good time to add it to distutils then :-)

Eggs are nice for plugins and application add-ons, but I don't
think they should be used for system wide installation of
Python extensions - these need to be under the control of
the system's installer which also deals with the Python
installation itself.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Dec 13 2005)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! ::::

From bob at redivi.com  Tue Dec 13 20:24:03 2005
From: bob at redivi.com (Bob Ippolito)
Date: Tue, 13 Dec 2005 11:24:03 -0800
Subject: [Distutils] bdist_nsis, bdist_deb, bdist_msi, bdist_inno,
	bdist_pkgtool, bdist_sdux, bdist_mpkg, etc.
In-Reply-To: <439F1D45.9090509@egenix.com>
References: <439EAFD9.8060005@egenix.com> <439EB1AF.4040809@egenix.com>
	<4F0360B5-9C0B-487E-BA8C-1210328F6C8C@redivi.com>
	<439F1D45.9090509@egenix.com>
Message-ID: <BB8FCDBE-8F26-47E7-8D4F-42C7CE490FED@redivi.com>

On Dec 13, 2005, at 11:13 AM, M.-A. Lemburg wrote:

> Bob Ippolito wrote:
>> On Dec 13, 2005, at 3:34 AM, M.-A. Lemburg wrote:
>>
>>> M.-A. Lemburg wrote:
>>>> Hi,
>>>>
>>>> I'd like to focus attention on these distribution formats
>>>> that distutils is missing:
>>>>
>>>
>>> I'd like to another important candidate to the list:
>>>
>>> bdist_mpkg:
>>>    Mac OS X installer format
>>>    Part of py2app by Bob Ippolito
>>>    http://undefined.org/python/py2app.html#bdist-mpkg-documentation
>>
>> bdist_mpkg is actually available on its own now, but I'm not sure  
>> adding
>> it to distutils is a great idea.  The current version is already  
>> moved
>> over to setuptools, and I'm pretty close to adding egg support to  
>> it.  I
>> may even require that the Python bits of the installation package are
>> eggs, as that would allow me to remove a bunch of code.  Eggs mostly
>> obsolete what bdist_mpkg is for, but it's also used to install things
>> like documentation and example code, so I'm not going to drop it
>> altogether.
>
> Looks like now is a good time to add it to distutils then :-)

No.  Definitely not.

> Eggs are nice for plugins and application add-ons, but I don't
> think they should be used for system wide installation of
> Python extensions - these need to be under the control of
> the system's installer which also deals with the Python
> installation itself.

Well you're wrong in the context of Mac OS X (and everywhere else,  
for the most part).  They solve real problems that the system package  
manager doesn't (dependencies, conflict detection, ...), and they  
provide metadata for py2app to use to build standalone applications  
reliably.  Even for platforms that have a better underlying system,  
the metadata provided by eggs is a huge step forward for Python and  
the sanity of users and application developers.

"eggs are nice for plugins" is certainly true -- bdist_mpkg provides  
itself as a setuptools plugin (via a distutils.commands entry point),  
which removes all the import hackery (hacking the Distribution object  
and the distutils packages to add the command)... of course it'll  
probably still be there but deprecated for a version due to backwards  
compatibility, but it'll go away soon.

In other words, I don't consent to adding bdist_mpkg to Python until  
it supports eggs.

-bob


From mal at egenix.com  Tue Dec 13 22:42:13 2005
From: mal at egenix.com (M.-A. Lemburg)
Date: Tue, 13 Dec 2005 22:42:13 +0100
Subject: [Distutils] bdist_nsis, bdist_deb, bdist_msi, bdist_inno,
 bdist_pkgtool, bdist_sdux, bdist_mpkg, etc.
In-Reply-To: <BB8FCDBE-8F26-47E7-8D4F-42C7CE490FED@redivi.com>
References: <439EAFD9.8060005@egenix.com>
	<439EB1AF.4040809@egenix.com>	<4F0360B5-9C0B-487E-BA8C-1210328F6C8C@redivi.com>	<439F1D45.9090509@egenix.com>
	<BB8FCDBE-8F26-47E7-8D4F-42C7CE490FED@redivi.com>
Message-ID: <439F4035.5070305@egenix.com>

Bob Ippolito wrote:
> On Dec 13, 2005, at 11:13 AM, M.-A. Lemburg wrote:
>>>>> I'd like to focus attention on these distribution formats
>>>>> that distutils is missing:
>>>>>
>>>>
>>>> I'd like to another important candidate to the list:
>>>>
>>>> bdist_mpkg:
>>>>    Mac OS X installer format
>>>>    Part of py2app by Bob Ippolito
>>>>    http://undefined.org/python/py2app.html#bdist-mpkg-documentation
>>>
>>> bdist_mpkg is actually available on its own now, but I'm not sure adding
>>> it to distutils is a great idea.  The current version is already moved
>>> over to setuptools, and I'm pretty close to adding egg support to it.  I
>>> may even require that the Python bits of the installation package are
>>> eggs, as that would allow me to remove a bunch of code.  Eggs mostly
>>> obsolete what bdist_mpkg is for, but it's also used to install things
>>> like documentation and example code, so I'm not going to drop it
>>> altogether.
>>
>> Looks like now is a good time to add it to distutils then :-)
> 
> No.  Definitely not.
> ... 
> In other words, I don't consent to adding bdist_mpkg to Python until it
> supports eggs.

Ok, then I'll remove the above candidate from the list.

Anyone else interested in writing a bdist_mpkg command
that doesn't require setuptools ?

Given the good documentation for the format, this shouldn't
be hard:

http://s.sudre.free.fr/Stuff/PackageMaker_Howto.html

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Dec 13 2005)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! ::::

From bob at redivi.com  Tue Dec 13 23:26:14 2005
From: bob at redivi.com (Bob Ippolito)
Date: Tue, 13 Dec 2005 14:26:14 -0800
Subject: [Distutils] bdist_nsis, bdist_deb, bdist_msi, bdist_inno,
	bdist_pkgtool, bdist_sdux, bdist_mpkg, etc.
In-Reply-To: <439F4035.5070305@egenix.com>
References: <439EAFD9.8060005@egenix.com>
	<439EB1AF.4040809@egenix.com>	<4F0360B5-9C0B-487E-BA8C-1210328F6C8C@redivi.com>	<439F1D45.9090509@egenix.com>
	<BB8FCDBE-8F26-47E7-8D4F-42C7CE490FED@redivi.com>
	<439F4035.5070305@egenix.com>
Message-ID: <E134B0C8-F027-4C8B-9B1A-41BA3DF0390E@redivi.com>

On Dec 13, 2005, at 1:42 PM, M.-A. Lemburg wrote:

> Bob Ippolito wrote:
>> On Dec 13, 2005, at 11:13 AM, M.-A. Lemburg wrote:
>>>>>> I'd like to focus attention on these distribution formats
>>>>>> that distutils is missing:
>>>>>>
>>>>>
>>>>> I'd like to another important candidate to the list:
>>>>>
>>>>> bdist_mpkg:
>>>>>    Mac OS X installer format
>>>>>    Part of py2app by Bob Ippolito
>>>>>    http://undefined.org/python/py2app.html#bdist-mpkg- 
>>>>> documentation
>>>>
>>>> bdist_mpkg is actually available on its own now, but I'm not  
>>>> sure adding
>>>> it to distutils is a great idea.  The current version is already  
>>>> moved
>>>> over to setuptools, and I'm pretty close to adding egg support  
>>>> to it.  I
>>>> may even require that the Python bits of the installation  
>>>> package are
>>>> eggs, as that would allow me to remove a bunch of code.  Eggs  
>>>> mostly
>>>> obsolete what bdist_mpkg is for, but it's also used to install  
>>>> things
>>>> like documentation and example code, so I'm not going to drop it
>>>> altogether.
>>>
>>> Looks like now is a good time to add it to distutils then :-)
>>
>> No.  Definitely not.
>> ...
>> In other words, I don't consent to adding bdist_mpkg to Python  
>> until it
>> supports eggs.
>
> Ok, then I'll remove the above candidate from the list.
>
> Anyone else interested in writing a bdist_mpkg command
> that doesn't require setuptools ?

That would be a disservice to the Python community.  Since Apple's  
installation packages don't provide for dependency metadata, conflict  
resolution, etc. it's really just a serialized version of "python  
setup.py install".  It doesn't even support relocation in any  
meaningful way to Python users (e.g. you'd have to make separate  
packages for home vs. root installation).

I have definitely had real-world experience where bdist_mpkg fails  
miserably.  The worst cases are when a module turns into a package,  
or is moved from inside a package to top-level, etc.  The old code is  
still around and gets imported, requiring the user to manually  
obliterate something inside their site-packages.

Moving to eggs is the right solution, taking what bdist_mpkg  
currently does and making it available to everyone is a bad idea.

The workaround for this is to include scripts in the installer  
package (pre-install, upgrade, etc.).  The functionality that needs  
to go into those scripts is already part of setuptools and I'd really  
rather not reinvent any wheels.

The other alternative is to use eggs exclusively and simply put  
together an egg management GUI alternative to the easy_install shell  
script.  The egg management GUI would include an egg with some  
version of setuptools for bootstrapping itself, but could use a newer  
system version if available.

In reality, this is an probably a much better option because Apple's  
installer application does not provide for downloading anything,  
checking for updates, or uninstallation.  The only particularly good  
reason I'm keeping bdist_mpkg around is that it is convenient for  
packages that install non-Python things (documentation, Xcode  
templates, examples).

> Given the good documentation for the format, this shouldn't
> be hard:

Apple has more current and official documentation for the format at  
developer.apple.com.  There is one component that still isn't  
documented though: the format used for the Archive.bom file.  This  
file is a binary bill of materials for the package contents, and must  
be generated with /usr/bin/mkbom (or it needs to be reverse  
engineered).  That means that bdist_mpkg can only be used from the  
Mac OS X platform, even for pure Python packages.  bdist_egg, on the  
other hand, makes much more sense.  The .egg file can be built on any  
platform, and can be installed on any other platform (with working  
scripts!), and can even be relocated.

The license for bdist_mpkg is amenable to copying/inclusion, so you  
could probably go ahead and do it if you really really wanted to and  
I'm not going to sue anyone over it... but I still think it would be  
a bad idea to include it in its current or former state, so you're  
not going to see any maintenance or consent from me if you do include  
it.

A version in distutils proper would also be a pain in my ass, because  
that would trump the third party command name and compel me to rename  
it.

-bob


From ianb at colorstudy.com  Wed Dec 14 00:17:26 2005
From: ianb at colorstudy.com (Ian Bicking)
Date: Tue, 13 Dec 2005 17:17:26 -0600
Subject: [Distutils] bdist_nsis, bdist_deb, bdist_msi, bdist_inno,
 bdist_pkgtool, bdist_sdux, bdist_mpkg, etc.
In-Reply-To: <439F1D45.9090509@egenix.com>
References: <439EAFD9.8060005@egenix.com>
	<439EB1AF.4040809@egenix.com>	<4F0360B5-9C0B-487E-BA8C-1210328F6C8C@redivi.com>
	<439F1D45.9090509@egenix.com>
Message-ID: <439F5686.9060108@colorstudy.com>

M.-A. Lemburg wrote:
> Eggs are nice for plugins and application add-ons, but I don't
> think they should be used for system wide installation of
> Python extensions - these need to be under the control of
> the system's installer which also deals with the Python
> installation itself.

I don't see where you come to that conclusion.  After a long and rather 
difficult discussion regarding Debian packages, while I don't think we 
all came away entirely happy, there seemed to be some reasonable peace 
made about eggs and packages.  There's still some outstanding issues, 
but there's also a plan to resolve those issues.

Honestly I'm still baffled by the hostility towards eggs, as though the 
fact that eggs duplicate some system package functionality means that 
they must be in conflict.  They don't need to be.  Eggs and setuptools 
goes to great length to be compatible with current Python functionality 
and conventions, while adding functionality using the current systems. 
It also goes to great length to be compatible with distutils.  And it 
also has served useful as a backport of distutils, so for instance I can 
make use of the upload command (which I believe is only in distutils in 
2.5).  Phillip has put a lot of effort into making setuptools play 
nicely with current Python conventions.

Anyway, getting back to something concrete, what specifically are the 
issues you see with Eggs as a general distribution format?

-- 
Ian Bicking  /  ianb at colorstudy.com  /  http://blog.ianbicking.org

From mal at egenix.com  Wed Dec 14 12:28:11 2005
From: mal at egenix.com (M.-A. Lemburg)
Date: Wed, 14 Dec 2005 12:28:11 +0100
Subject: [Distutils] bdist_nsis, bdist_deb, bdist_msi, bdist_inno,
 bdist_pkgtool, bdist_sdux, etc.
In-Reply-To: <439F005E.7080004@v.loewis.de>
References: <439EAFD9.8060005@egenix.com> <439F005E.7080004@v.loewis.de>
Message-ID: <43A001CB.3000905@egenix.com>

Martin v. L?wis wrote:
> M.-A. Lemburg wrote:
>> Would any of the authors (I've tried to CC them) be willing to
>> contribute their distutils command to the PSF for integration ?
> 
> I was hoping to contribute bdist_msi before 2.5. Before that, I need
> atleast two more releases; I have some feedback on which I haven't
> reacted on, yet.

Would be nice for have bdist_msi in 2.5, but it's good to know
that this is on your schedule.

Thanks,
-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Dec 14 2005)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! ::::

From mal at egenix.com  Wed Dec 14 12:54:16 2005
From: mal at egenix.com (M.-A. Lemburg)
Date: Wed, 14 Dec 2005 12:54:16 +0100
Subject: [Distutils] bdist_nsis, bdist_deb, bdist_msi, bdist_inno,
 bdist_pkgtool, bdist_sdux, etc.
In-Reply-To: <s9z4q5cq5g0.fsf@thetis.intevation.de>
References: <439EAFD9.8060005@egenix.com>
	<u0ddkmm6.fsf@python.net>	<439EFE58.7050504@egenix.com>
	<s9z4q5cq5g0.fsf@thetis.intevation.de>
Message-ID: <43A007E8.8040703@egenix.com>

Bernhard Herzog wrote:
> "M.-A. Lemburg" <mal at egenix.com> writes:
> 
>>> What do you have in mind - a bdist_wininst lookalike for
>>> Python packages, built with inno?
>> Yes. One that creates an Inno setup file and then compiles
>> everything into an installer using the Inno, but installing
>> things much like bdist_wininst (and with the option of
>> building upon the many extra features you get from the
>> InnoSetup system).
>>
>> Bernhard Herzog started one some time ago:
>> http://mail.python.org/pipermail/distutils-sig/2001-September/002597.html
> 
> FWIW, the code that's linked from that posting is still unchanged in
> Thuban and worked fine last time I made a release about a year ago with
> python 2.3.

Would you be willing to contribute the code to the PSF ?

We could then use is as starting point for a more elaborate
implementation.

Thanks,
-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Dec 14 2005)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! ::::

From mal at egenix.com  Wed Dec 14 14:49:14 2005
From: mal at egenix.com (M.-A. Lemburg)
Date: Wed, 14 Dec 2005 14:49:14 +0100
Subject: [Distutils] bdist_nsis, bdist_deb, bdist_msi, bdist_inno,
 bdist_pkgtool, bdist_sdux, bdist_mpkg, etc.
In-Reply-To: <E134B0C8-F027-4C8B-9B1A-41BA3DF0390E@redivi.com>
References: <439EAFD9.8060005@egenix.com>
	<439EB1AF.4040809@egenix.com>	<4F0360B5-9C0B-487E-BA8C-1210328F6C8C@redivi.com>	<439F1D45.9090509@egenix.com>	<BB8FCDBE-8F26-47E7-8D4F-42C7CE490FED@redivi.com>	<439F4035.5070305@egenix.com>
	<E134B0C8-F027-4C8B-9B1A-41BA3DF0390E@redivi.com>
Message-ID: <43A022DA.6090905@egenix.com>

Bob Ippolito wrote:
> On Dec 13, 2005, at 1:42 PM, M.-A. Lemburg wrote:
> 
>> Bob Ippolito wrote:
>>> On Dec 13, 2005, at 11:13 AM, M.-A. Lemburg wrote:
>>>>>> I'd like to another important candidate to the list:
>>>>>>
>>>>>> bdist_mpkg:
>>>>>>    Mac OS X installer format
>>>>>>    Part of py2app by Bob Ippolito
>>>>>>    http://undefined.org/python/py2app.html#bdist-mpkg-documentation
>>>>>
>>>>> bdist_mpkg is actually available on its own now, but I'm not sure
>>>>> adding
>>>>> it to distutils is a great idea.  The current version is already moved
>>>>> over to setuptools, and I'm pretty close to adding egg support to
>>>>> it.  I
>>>>> may even require that the Python bits of the installation package are
>>>>> eggs, as that would allow me to remove a bunch of code.  Eggs mostly
>>>>> obsolete what bdist_mpkg is for, but it's also used to install things
>>>>> like documentation and example code, so I'm not going to drop it
>>>>> altogether.
>>>>
>>>> Looks like now is a good time to add it to distutils then :-)
>>>
>>> No.  Definitely not.
>>> ...
>>> In other words, I don't consent to adding bdist_mpkg to Python until it
>>> supports eggs.
>>
>> Ok, then I'll remove the above candidate from the list.
>>
>> Anyone else interested in writing a bdist_mpkg command
>> that doesn't require setuptools ?
> 
> That would be a disservice to the Python community. 

I don't follow you there. It's the standard Mac OS X installation
format and Mac users are used to installing things that way.

> Since Apple's
> installation packages don't provide for dependency metadata, conflict
> resolution, etc. it's really just a serialized version of "python
> setup.py install".  It doesn't even support relocation in any meaningful
> way to Python users (e.g. you'd have to make separate packages for home
> vs. root installation).

The target audience for these installers are users which
use the standard Python installation that comes with
Mac OS X. Later on, we might want to add support for
specifying the target path for installing e.g.
into Zope/Plone or other applications that use custom
extensions.

> I have definitely had real-world experience where bdist_mpkg fails
> miserably.  The worst cases are when a module turns into a package, or
> is moved from inside a package to top-level, etc.  The old code is still
> around and gets imported, requiring the user to manually obliterate
> something inside their site-packages.

The installer preinstall/upgrade scripts can take care
of this.

> Moving to eggs is the right solution, taking what bdist_mpkg currently
> does and making it available to everyone is a bad idea.

I think differently, but you know that, so I won't elaborate.

> The workaround for this is to include scripts in the installer package
> (pre-install, upgrade, etc.).  The functionality that needs to go into
> those scripts is already part of setuptools and I'd really rather not
> reinvent any wheels.

You don't change package hierarchies that often and if you
do, you have to be careful not to break backwards compatibility
anyway. This will always require writing extra code and being
extra careful regarding import paths stored in Python pickles
and similar formats.

> The other alternative is to use eggs exclusively and simply put together
> an egg management GUI alternative to the easy_install shell script.  The
> egg management GUI would include an egg with some version of setuptools
> for bootstrapping itself, but could use a newer system version if
> available.
>
> In reality, this is an probably a much better option because Apple's
> installer application does not provide for downloading anything,
> checking for updates, or uninstallation.  The only particularly good
> reason I'm keeping bdist_mpkg around is that it is convenient for
> packages that install non-Python things (documentation, Xcode templates,
> examples).

Now who's reinventing wheels here ? ;-)

I'm sure that Apple's installer is going to grow at least
the uninstall option sooner or later. Everything else
can easily be done in the installer hook scripts. Please
remember that you already have Python installed on the
machine, so there are plenty things you can do in these
scripts.

>> Given the good documentation for the format, this shouldn't
>> be hard:
> 
> Apple has more current and official documentation for the format at
> developer.apple.com.  There is one component that still isn't documented
> though: the format used for the Archive.bom file.  This file is a binary
> bill of materials for the package contents, and must be generated with
> /usr/bin/mkbom (or it needs to be reverse engineered).  That means that
> bdist_mpkg can only be used from the Mac OS X platform, even for pure
> Python packages. 

No problem with that: you need to test things on the platform
itself anyway (including whether the installer works or not).

> bdist_egg, on the other hand, makes much more sense. 
> The .egg file can be built on any platform, and can be installed on any
> other platform (with working scripts!), and can even be relocated.

Whether it makes sense or not should be left for the user
to decide. I don't want to argue over this. Instead, I would
like to give users the choice of using a native and well-known
interface for installing software or some other method.

> The license for bdist_mpkg is amenable to copying/inclusion, so you
> could probably go ahead and do it if you really really wanted to and I'm
> not going to sue anyone over it... but I still think it would be a bad
> idea to include it in its current or former state, so you're not going
> to see any maintenance or consent from me if you do include it.

Fair enough.

> A version in distutils proper would also be a pain in my ass, because
> that would trump the third party command name and compel me to rename it.

We could use a different command name, e.g. bdist_macosx.

Regards,
-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Dec 14 2005)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! ::::

From tgreenwoodgeer at yahoo.com  Wed Dec 14 19:18:22 2005
From: tgreenwoodgeer at yahoo.com (Todd Greenwood-Geer)
Date: Wed, 14 Dec 2005 10:18:22 -0800
Subject: [Distutils] Naming conventions for module unit tests?
In-Reply-To: <Pine.LNX.4.58.0512111302340.6246@alice>
References: <5.1.1.6.0.20051208114741.03eb3680@mail.telecommunity.com>
	<439907CE.7070503@yahoo.com>
	<Pine.WNT.4.64.0512091421360.3480@shaolin>
	<439B8F29.3080404@yahoo.com>
	<Pine.LNX.4.58.0512111302340.6246@alice>
Message-ID: <43A061EE.1050001@yahoo.com>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

To the list :

Anyone have any good naming conventions for module tests?

[module name]_unittest.py
test_[module_name].py
unittest_[module_name].py

Then there's the issue of differentiating unit tests from functional
tests, scenario tests...

Here's my off the cuff definition for these things:

unit test : fine grained tests, usually testing individual methods
functional test : coarser grained tests, usually testing an aggregation
of methods, usually a single module
scenario tests : even coarser grained tests, not focused on code testing
per se, focused on testing actual usage scenarios from beginning to end.

Then there are the aggregations of tests, drts and bvts.

Anyone out there come across a useful naming structure for all this?

Perhaps:
[aggregation]_[level]_[module].py
	drt_unittest_foo.py
	bvt_functionaltest_foo.py

Or am I overly complicating things again?

- -Todd

John J Lee wrote:
| On Sat, 10 Dec 2005, Todd Greenwood-Geer wrote:
|
|> -----BEGIN PGP SIGNED MESSAGE-----
|> Hash: SHA1
|>
|> Thanks John. That was biting me in several places. Is there a standard
|> practice/naming convention for module unit tests? I was trying something
|> like [module name]_unittest.py.
|
| That's a list question, surely -- post to the list!
|
|
| John
|
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.1 (GNU/Linux)

iD8DBQFDoGHuz6uXX4lQc/URAntjAJ0UAs/e4CP7LO/y0w/ukO57TSRoVQCaApbC
1My/W9koS6HaJ1aeh4pGdWU=
=eWeG
-----END PGP SIGNATURE-----

From tgreenwoodgeer at yahoo.com  Wed Dec 14 19:32:20 2005
From: tgreenwoodgeer at yahoo.com (Todd Greenwood-Geer)
Date: Wed, 14 Dec 2005 10:32:20 -0800
Subject: [Distutils] offline coding, svn:externals, and ez_setup
In-Reply-To: <5.1.1.6.0.20051210212653.01f2e2d0@mail.telecommunity.com>
References: <5.1.1.6.0.20051208234423.02d580e8@mail.telecommunity.com>
	<5.1.1.6.0.20051208114741.03eb3680@mail.telecommunity.com>
	<5.1.1.6.0.20051208114741.03eb3680@mail.telecommunity.com>
	<5.1.1.6.0.20051208234423.02d580e8@mail.telecommunity.com>
	<5.1.1.6.0.20051210212653.01f2e2d0@mail.telecommunity.com>
Message-ID: <43A06534.9080306@yahoo.com>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

I'm finding that I'm coding online and offline. When I'm offline, I
would really like to use the svn:externals definition for ez_setup. It
seems that the best way to do this would be to create a 'peak' project
just for the ez_setup files. Then, when I'm online, I can sync 'peak'
and get the latest ez_setup files. When I'm offline, I could reference
the 'peak' project in my other python projects that use setuptools. I've
tried this, but so far, I keep getting just a blank directory when my
projects reference 'peak'.

REPRO:

$ cd temp
$ mkdir peak
$ svn import peak file:///home/me/svn/peak -m 'local peak'
$ cd ~/active
$ svn co file:///home/me/svn/peak
$ cd peak
$ svn propedit svn:externals .
|> ez_setup svn://svn.eby-sarna.com/svnroot/ez_setup
$ svn update
$ ls
|> ez_setup
$ ls ez_setup
|> __init__.py README.txt
$ cd ~/active
$ cd SomeExistingProjectInSVN
$ svn propedit svn:externals .
|> # ez_setup svn://svn.eby-sarna.com/svnroot/ez_setup
|> ez_setup file:///home/me/svn/peak
$ svn update
|> Fetching external item into 'ez_setup'
|> External at revision 21.
|>
|> At revision 21.
$ ls
|> ez_setup
$ ls ez_setup
|> total 0

ERROR:
I expect to see these files:
|> __init__.py README.txt

- -Todd


-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.1 (GNU/Linux)

iD8DBQFDoGU0z6uXX4lQc/URAg2QAJ9EQsR/WHYhPyebWT2Dx7ikJWhUxQCfXwAm
XHyCKVMpQNApSJOf1Qihbaw=
=VBoR
-----END PGP SIGNATURE-----

From tgreenwoodgeer at yahoo.com  Wed Dec 14 21:59:21 2005
From: tgreenwoodgeer at yahoo.com (Todd Greenwood-Geer)
Date: Wed, 14 Dec 2005 12:59:21 -0800
Subject: [Distutils] python setup.py bdist_egg results depend on dir location
In-Reply-To: <5.1.1.6.0.20051208234423.02d580e8@mail.telecommunity.com>
References: <5.1.1.6.0.20051208114741.03eb3680@mail.telecommunity.com>
	<5.1.1.6.0.20051208114741.03eb3680@mail.telecommunity.com>
	<5.1.1.6.0.20051208234423.02d580e8@mail.telecommunity.com>
Message-ID: <43A087A9.6050000@yahoo.com>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

PROBLEM:
I would like to be able to execute the egg build from several
directories, like so:

$ cd active/SimpleExampleEgg
$ python setup.py bdist_egg

then, at other times, i'd like to change to a package directory and
execute the builds from there:

$ cd active/SimpleExampleEgg/fruit/docs
$ python ../../setup.py bdist_egg

this places the output in active/SimpleExampleEgg/fruit/docs (build,
dist), which is fine by me.

REASON:
I'm executing builds as part of my documentation process using vim and a
restructured text pluggin to vim. Everything seems to hang together a
bit better if I execute the vim/rest code from the directory it is in,
[project]/fruit/docs, in this case.

ALTERNATIVE:
I suppose I could stay in the project root directory. This means I'd
have to include my documentation via resource_filename(...), and is not
the end of the world.

BUG:
Perhaps this is exposing an interesting bug. I would expect the egg
creation to either be identical for these two cases, or fail altogether
for the second case.

REPRO:
greenwo at luxor~/active/SimpleExampleEgg$ python setup.py bdist_egg
running bdist_egg
running egg_info
writing ./SimpleExampleEgg.egg-info/PKG-INFO
writing top-level names to ./SimpleExampleEgg.egg-info/top_level.txt
installing library code to build/bdist.linux-i686/egg
running install_lib
running build_py
creating build/bdist.linux-i686/egg
creating build/bdist.linux-i686/egg/fruit
copying build/lib/fruit/orange.py -> build/bdist.linux-i686/egg/fruit
copying build/lib/fruit/apple.py -> build/bdist.linux-i686/egg/fruit
copying build/lib/fruit/__init__.py -> build/bdist.linux-i686/egg/fruit
copying build/lib/fruit/simpletests.py -> build/bdist.linux-i686/egg/fruit
creating build/bdist.linux-i686/egg/fruit/docs
copying build/lib/fruit/docs/traditional.css ->
build/bdist.linux-i686/egg/fruit/docs
copying build/lib/fruit/docs/readme.rest ->
build/bdist.linux-i686/egg/fruit/docs
copying build/lib/fruit/docs/readme.html ->
build/bdist.linux-i686/egg/fruit/docs
copying build/lib/fruit/docs/default.css ->
build/bdist.linux-i686/egg/fruit/docs
copying build/lib/fruit/docs/oldstyle.css ->
build/bdist.linux-i686/egg/fruit/docs
copying build/lib/fruit/docs/vstrc.vim ->
build/bdist.linux-i686/egg/fruit/docs
byte-compiling build/bdist.linux-i686/egg/fruit/orange.py to orange.pyc
byte-compiling build/bdist.linux-i686/egg/fruit/apple.py to apple.pyc
byte-compiling build/bdist.linux-i686/egg/fruit/__init__.py to __init__.pyc
byte-compiling build/bdist.linux-i686/egg/fruit/simpletests.py to
simpletests.pyc
creating build/bdist.linux-i686/egg/EGG-INFO
copying ./SimpleExampleEgg.egg-info/PKG-INFO ->
build/bdist.linux-i686/egg/EGG-INFO
copying ./SimpleExampleEgg.egg-info/top_level.txt ->
build/bdist.linux-i686/egg/EGG-INFO
creating 'dist/SimpleExampleEgg-0.1-py2.4.egg' and adding
'build/bdist.linux-i686/egg' to it
removing 'build/bdist.linux-i686/egg' (and everything under it)
tgreenwo at luxor~/active/SimpleExampleEgg$ unzip -l
dist/SimpleExampleEgg-0.1-py2.4.egg
Archive:  dist/SimpleExampleEgg-0.1-py2.4.egg
~  Length     Date   Time    Name
~ --------    ----   ----    ----
~      290  12-14-05 12:11   fruit/orange.py
~      286  12-14-05 12:11   fruit/apple.py
~        0  12-14-05 12:13   fruit/__init__.py
~      240  12-14-05 12:26   fruit/simpletests.py
~      556  12-14-05 12:47   fruit/orange.pyc
~      625  12-14-05 12:47   fruit/apple.pyc
~      124  12-14-05 12:47   fruit/__init__.pyc
~      537  12-14-05 12:47   fruit/simpletests.pyc
~    14863  12-14-05 12:36   fruit/docs/traditional.css
~     5079  12-14-05 12:43   fruit/docs/readme.rest
~    12958  12-14-05 12:43   fruit/docs/readme.html
~     6464  12-14-05 12:43   fruit/docs/default.css
~    14140  12-14-05 12:36   fruit/docs/oldstyle.css
~      141  12-14-05 12:36   fruit/docs/vstrc.vim
~      532  12-14-05 12:47   EGG-INFO/PKG-INFO
~        6  12-14-05 12:47   EGG-INFO/top_level.txt
~        0  12-14-05 12:47   EGG-INFO/zip-safe
~ --------                   -------
~    56841                   17 files
tgreenwo at luxor~/active/SimpleExampleEgg$ cd fruit/docs/
tgreenwo at luxor~/active/SimpleExampleEgg/fruit/docs$ python
../../setup.py bdist_egg
running bdist_egg
running egg_info
writing ./SimpleExampleEgg.egg-info/PKG-INFO
writing top-level names to ./SimpleExampleEgg.egg-info/top_level.txt
installing library code to build/bdist.linux-i686/egg
running install_lib
warning: install_lib: 'build/lib' does not exist -- no Python modules to
install
creating build/bdist.linux-i686/egg
creating build/bdist.linux-i686/egg/EGG-INFO
copying ./SimpleExampleEgg.egg-info/PKG-INFO ->
build/bdist.linux-i686/egg/EGG-INFO
copying ./SimpleExampleEgg.egg-info/top_level.txt ->
build/bdist.linux-i686/egg/EGG-INFO
creating 'dist/SimpleExampleEgg-0.1-py2.4.egg' and adding
'build/bdist.linux-i686/egg' to it
removing 'build/bdist.linux-i686/egg' (and everything under it)
tgreenwo at luxor~/active/SimpleExampleEgg/fruit/docs$ unzip -l
dist/SimpleExampleEgg-0.1-py2.4.egg
Archive:  dist/SimpleExampleEgg-0.1-py2.4.egg
~  Length     Date   Time    Name
~ --------    ----   ----    ----
~      532  12-14-05 12:48   EGG-INFO/PKG-INFO
~        1  12-14-05 12:48   EGG-INFO/top_level.txt
~        0  12-14-05 12:48   EGG-INFO/zip-safe
~ --------                   -------
~      533                   3 files
tgreenwo at luxor~/active/SimpleExampleEgg/fruit/docs$


- -Todd
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.1 (GNU/Linux)

iD8DBQFDoIepz6uXX4lQc/URAmBeAJ9MNkw8rvTpZedpH7seUY8kjA6QKgCfe+sK
D/zaLhfsLcju0dOUMDOVZyo=
=jtO/
-----END PGP SIGNATURE-----

From pje at telecommunity.com  Wed Dec 14 22:32:34 2005
From: pje at telecommunity.com (Phillip J. Eby)
Date: Wed, 14 Dec 2005 16:32:34 -0500
Subject: [Distutils] python setup.py bdist_egg results depend on dir
	location
In-Reply-To: <43A087A9.6050000@yahoo.com>
References: <5.1.1.6.0.20051208234423.02d580e8@mail.telecommunity.com>
	<5.1.1.6.0.20051208114741.03eb3680@mail.telecommunity.com>
	<5.1.1.6.0.20051208114741.03eb3680@mail.telecommunity.com>
	<5.1.1.6.0.20051208234423.02d580e8@mail.telecommunity.com>
Message-ID: <5.1.1.6.0.20051214162713.01de1460@mail.telecommunity.com>

At 12:59 PM 12/14/2005 -0800, Todd Greenwood-Geer wrote:
>-----BEGIN PGP SIGNED MESSAGE-----
>Hash: SHA1
>
>PROBLEM:
>I would like to be able to execute the egg build from several
>directories, like so:
>
>$ cd active/SimpleExampleEgg
>$ python setup.py bdist_egg
>
>then, at other times, i'd like to change to a package directory and
>execute the builds from there:
>
>$ cd active/SimpleExampleEgg/fruit/docs
>$ python ../../setup.py bdist_egg

Stop right there.  The distutils do *not* support this, and neither does 
setuptools.  'setup.py' scripts *must* be run from the directory where they 
are located, so I didn't bother reading the rest of your message.

If you want to run setup.py this way, you should 'easy_install buildutils' 
and then use its "pbu" command, e.g.:

    pbu bdist_egg

The "pbu" command automatically searches parent directories for a setup.py, 
then changes to the directory where the setup.py was found, and runs it for 
you.  This is the only safe way to do it.

(When "pbu" exits, you will however still be in the directory you were in 
before.)


From pje at telecommunity.com  Wed Dec 14 22:39:04 2005
From: pje at telecommunity.com (Phillip J. Eby)
Date: Wed, 14 Dec 2005 16:39:04 -0500
Subject: [Distutils] offline coding, svn:externals, and ez_setup
In-Reply-To: <43A06534.9080306@yahoo.com>
References: <5.1.1.6.0.20051210212653.01f2e2d0@mail.telecommunity.com>
	<5.1.1.6.0.20051208234423.02d580e8@mail.telecommunity.com>
	<5.1.1.6.0.20051208114741.03eb3680@mail.telecommunity.com>
	<5.1.1.6.0.20051208114741.03eb3680@mail.telecommunity.com>
	<5.1.1.6.0.20051208234423.02d580e8@mail.telecommunity.com>
	<5.1.1.6.0.20051210212653.01f2e2d0@mail.telecommunity.com>
Message-ID: <5.1.1.6.0.20051214163705.03a9e258@mail.telecommunity.com>

At 10:32 AM 12/14/2005 -0800, Todd Greenwood-Geer wrote:
>-----BEGIN PGP SIGNED MESSAGE-----
>Hash: SHA1
>
>I'm finding that I'm coding online and offline. When I'm offline, I
>would really like to use the svn:externals definition for ez_setup. It
>seems that the best way to do this would be to create a 'peak' project
>just for the ez_setup files. Then, when I'm online, I can sync 'peak'
>and get the latest ez_setup files. When I'm offline, I could reference
>the 'peak' project in my other python projects that use setuptools. I've
>tried this, but so far, I keep getting just a blank directory when my
>projects reference 'peak'.

Subversion doesn't chain externals, so there isn't a good solution for what 
you're trying to do.  You should probably just use ez_setup.py instead of 
ez_setup/__init__.py.

The main advantage to using svn:externals is that it gets you (and anyone 
who checks out your project) an updated ez_setup/__init__.py just by 
running svn update.


From pje at telecommunity.com  Wed Dec 14 22:44:47 2005
From: pje at telecommunity.com (Phillip J. Eby)
Date: Wed, 14 Dec 2005 16:44:47 -0500
Subject: [Distutils] Naming conventions for module unit tests?
In-Reply-To: <43A061EE.1050001@yahoo.com>
References: <Pine.LNX.4.58.0512111302340.6246@alice>
	<5.1.1.6.0.20051208114741.03eb3680@mail.telecommunity.com>
	<439907CE.7070503@yahoo.com>
	<Pine.WNT.4.64.0512091421360.3480@shaolin>
	<439B8F29.3080404@yahoo.com>
	<Pine.LNX.4.58.0512111302340.6246@alice>
Message-ID: <5.1.1.6.0.20051214163917.03aa6cf0@mail.telecommunity.com>

At 10:18 AM 12/14/2005 -0800, Todd Greenwood-Geer wrote:
>test_[module_name].py

This is a common convention used by large Python projects.  Other 
conventions include test/foo.py and tests/bar.py.


>Then there's the issue of differentiating unit tests from functional
>tests, scenario tests...

I haven't seen any two projects use the same approach for these, I'm afraid.


>Or am I overly complicating things again?

If there's only one or two modules involved, definitely; small modules 
often include all their tests in the same file with the code!

Python is very good for incremental, iterative development; try starting 
with the simplest thing that could possibly work, and only add complexity 
when you discover an actual problem with your current layout.  Modern 
revision control tools like Subversion allow you to reorganize your source 
without losing history, so there's no need for you to worry in advance 
about what your layout will be when your project gets big.  Start small and 
figure it out as you grow, and then you will have empirical data about how 
many tests you have, how big they are, etc., that will help you define an 
organization that works for your project.

Currently, I would say that there is no community consensus as to a One 
Obvious Way to lay out tests for Python applications or libraries.


From pje at telecommunity.com  Wed Dec 14 22:59:25 2005
From: pje at telecommunity.com (Phillip J. Eby)
Date: Wed, 14 Dec 2005 16:59:25 -0500
Subject: [Distutils] bdist_nsis, bdist_deb, bdist_msi, bdist_inno,
 bdist_pkgtool, bdist_sdux, bdist_mpkg, etc.
In-Reply-To: <43A022DA.6090905@egenix.com>
References: <E134B0C8-F027-4C8B-9B1A-41BA3DF0390E@redivi.com>
	<439EAFD9.8060005@egenix.com> <439EB1AF.4040809@egenix.com>
	<4F0360B5-9C0B-487E-BA8C-1210328F6C8C@redivi.com>
	<439F1D45.9090509@egenix.com>
	<BB8FCDBE-8F26-47E7-8D4F-42C7CE490FED@redivi.com>
	<439F4035.5070305@egenix.com>
	<E134B0C8-F027-4C8B-9B1A-41BA3DF0390E@redivi.com>
Message-ID: <5.1.1.6.0.20051214164533.0359be20@mail.telecommunity.com>

(Disclaimer: this email does not advocate using eggs; it's a comment on 
needed functionality in bdist_mpkg, however it gets implemented.)


At 02:49 PM 12/14/2005 +0100, M.-A. Lemburg wrote:
> > I have definitely had real-world experience where bdist_mpkg fails
> > miserably.  The worst cases are when a module turns into a package, or
> > is moved from inside a package to top-level, etc.  The old code is still
> > around and gets imported, requiring the user to manually obliterate
> > something inside their site-packages.
>
>The installer preinstall/upgrade scripts can take care
>of this.

Only if the upgrade script knows what files are installed; this is much 
easier to handle with an .egg, since PEP 262 was never added to the distutils.


> > Moving to eggs is the right solution, taking what bdist_mpkg currently
> > does and making it available to everyone is a bad idea.
>
>I think differently, but you know that, so I won't elaborate.

Until/unless Apple adds uninstallation support, you'll need to duplicate 
egg functionality in some other way to support this.  If you implement a 
bdist_mpkg that *doesn't* duplicate at least some egg functionality, then 
Bob is correct: it's a *very* bad idea.


>You don't change package hierarchies that often and if you
>do, you have to be careful not to break backwards compatibility
>anyway.

Your argument here appears to be, "things are hard the way we do them now, 
but that's okay because we experts know they're hard and if you're not an 
expert you don't deserve to have things work.  There's no need to do things 
more easily because we already know how to do them the hard way."  I'm not 
sure how that approach helps the distutils "move forward" in any way.


> > A version in distutils proper would also be a pain in my ass, because
> > that would trump the third party command name and compel me to rename it.
>
>We could use a different command name, e.g. bdist_macosx.

Actually, setuptools-based commands trump non-setuptools commands, as long 
as the project uses setuptools.  If it doesn't use setuptools, you can 
force it to by using:

     python -c "import setuptools;execfile('setup.py')" ...

in place of:

     python setup.py ...

So I don't think adding a bdist_mpkg to distutils would actually result in 
any chapping of Bob's posterior.  :)

But I do think that anybody building a bdist_mpkg needs to have it support 
uninstallation (e.g. by creating an uninstall script) and upgrading (by 
finding any previous version and running its uninstall script).  Otherwise, 
there is no reason (that I understand, anyway) to *have* a bdist_mpkg 
format; it won't be doing anything you can't do with say, bdist_dumb or a 
plain "setup.py install".


From tgreenwoodgeer at yahoo.com  Wed Dec 14 23:13:02 2005
From: tgreenwoodgeer at yahoo.com (Todd Greenwood-Geer)
Date: Wed, 14 Dec 2005 14:13:02 -0800
Subject: [Distutils] offline coding, svn:externals, and ez_setup
In-Reply-To: <5.1.1.6.0.20051214163705.03a9e258@mail.telecommunity.com>
References: <5.1.1.6.0.20051210212653.01f2e2d0@mail.telecommunity.com>
	<5.1.1.6.0.20051208234423.02d580e8@mail.telecommunity.com>
	<5.1.1.6.0.20051208114741.03eb3680@mail.telecommunity.com>
	<5.1.1.6.0.20051208114741.03eb3680@mail.telecommunity.com>
	<5.1.1.6.0.20051208234423.02d580e8@mail.telecommunity.com>
	<5.1.1.6.0.20051210212653.01f2e2d0@mail.telecommunity.com>
	<5.1.1.6.0.20051214163705.03a9e258@mail.telecommunity.com>
Message-ID: <43A098EE.7080806@yahoo.com>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Ok. I'll fall back to copying the file in the offline scenario. Thanks.

Phillip J. Eby wrote:
| At 10:32 AM 12/14/2005 -0800, Todd Greenwood-Geer wrote:
|> -----BEGIN PGP SIGNED MESSAGE-----
|> Hash: SHA1
|>
|> I'm finding that I'm coding online and offline. When I'm offline, I
|> would really like to use the svn:externals definition for ez_setup. It
|> seems that the best way to do this would be to create a 'peak' project
|> just for the ez_setup files. Then, when I'm online, I can sync 'peak'
|> and get the latest ez_setup files. When I'm offline, I could reference
|> the 'peak' project in my other python projects that use setuptools. I've
|> tried this, but so far, I keep getting just a blank directory when my
|> projects reference 'peak'.
|
| Subversion doesn't chain externals, so there isn't a good solution for
what
| you're trying to do.  You should probably just use ez_setup.py instead of
| ez_setup/__init__.py.
|
| The main advantage to using svn:externals is that it gets you (and anyone
| who checks out your project) an updated ez_setup/__init__.py just by
| running svn update.
|
|
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.1 (GNU/Linux)

iD8DBQFDoJjuz6uXX4lQc/URAl8+AKC8BX0GjYNHzOQH8RkWNqBw7xDIQwCfU3YD
LjgdJfneCmzOumzc4HuhNDU=
=1wd7
-----END PGP SIGNATURE-----

From pje at telecommunity.com  Wed Dec 14 23:16:50 2005
From: pje at telecommunity.com (Phillip J. Eby)
Date: Wed, 14 Dec 2005 17:16:50 -0500
Subject: [Distutils] Splitting large packages into multiple eggs
In-Reply-To: <A000A1AF-A063-45A6-9123-9129BFFCC086@mac.com>
References: <28E84919-5544-46EA-A0D9-E50413053543@redivi.com>
	<5.1.1.6.0.20051211170340.03340a30@mail.telecommunity.com>
	<28E84919-5544-46EA-A0D9-E50413053543@redivi.com>
Message-ID: <5.1.1.6.0.20051214170051.03b35e90@mail.telecommunity.com>

At 07:02 PM 12/11/2005 -0800, Bob Ippolito wrote:
>It looks like easy_install can't do this yet.

And it's not going to be able to.  The closest thing I can envision is that 
if you have a master project with a setup.py that runs each child project's 
setup.py (or just a setup() call), such that each gets passed a "bdist_egg" 
command by easy_install.  EasyInstall already detects when multiple eggs 
are built by a single setup.py, and processes all of them.

That way, you could have one PyObjC project that contains all the others, 
and builds multiple eggs, including one that just specifies dependencies on 
the others.

The only possible issue that might arise is if there are inter-egg 
dependencies and the eggs are built out-of-order.  In that case, 
easy_install might incorrectly conclude that it needs one of the built 
eggs, before it has processed it.  I could probably add some code to make 
this more robust.

Anyway, the only limitation of this approach is that you won't be able to 
build any individual packages from source, only the overall package as a 
whole.  However, it will be possible to build and even upload all the eggs 
using "setup.py bdist_egg upload", so easy_install will be able to find the 
binary packages.  Or you can just have your PyPI download URL point to a 
directory where you dump the latest eggs; if you're running your bdist 
commands on the web server (or rsync it), you can use the 'rotate' command 
to delete outdated snapshots.

But anybody who wants to build from source will have to use the master 
package, rather than any individual subpackages.


>Also, I'm going to want a way to have "setup.py develop" ensure that
>all of the subprojects are up to date.

This should work normally if your setup.py just calls setup() for each 
subproject and the master; again this should be in dependencies-first order.


From tgreenwoodgeer at yahoo.com  Wed Dec 14 23:17:05 2005
From: tgreenwoodgeer at yahoo.com (Todd Greenwood-Geer)
Date: Wed, 14 Dec 2005 14:17:05 -0800
Subject: [Distutils] python setup.py bdist_egg results depend on dir
	location
In-Reply-To: <5.1.1.6.0.20051214162713.01de1460@mail.telecommunity.com>
References: <5.1.1.6.0.20051208234423.02d580e8@mail.telecommunity.com>
	<5.1.1.6.0.20051208114741.03eb3680@mail.telecommunity.com>
	<5.1.1.6.0.20051208114741.03eb3680@mail.telecommunity.com>
	<5.1.1.6.0.20051208234423.02d580e8@mail.telecommunity.com>
	<5.1.1.6.0.20051214162713.01de1460@mail.telecommunity.com>
Message-ID: <43A099E1.8000802@yahoo.com>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Great. The pbu utility is perfect. Thanks, yet again.

Phillip J. Eby wrote:
| At 12:59 PM 12/14/2005 -0800, Todd Greenwood-Geer wrote:
|> -----BEGIN PGP SIGNED MESSAGE-----
|> Hash: SHA1
|>
|> PROBLEM:
|> I would like to be able to execute the egg build from several
|> directories, like so:
|>
|> $ cd active/SimpleExampleEgg
|> $ python setup.py bdist_egg
|>
|> then, at other times, i'd like to change to a package directory and
|> execute the builds from there:
|>
|> $ cd active/SimpleExampleEgg/fruit/docs
|> $ python ../../setup.py bdist_egg
|
| Stop right there.  The distutils do *not* support this, and neither does
| setuptools.  'setup.py' scripts *must* be run from the directory where
they
| are located, so I didn't bother reading the rest of your message.
|
| If you want to run setup.py this way, you should 'easy_install
buildutils'
| and then use its "pbu" command, e.g.:
|
|     pbu bdist_egg
|
| The "pbu" command automatically searches parent directories for a
setup.py,
| then changes to the directory where the setup.py was found, and runs
it for
| you.  This is the only safe way to do it.
|
| (When "pbu" exits, you will however still be in the directory you were in
| before.)
|
|
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.1 (GNU/Linux)

iD8DBQFDoJnhz6uXX4lQc/URAj1NAKDG6yXdR0w6Y3dqjIaKmZ9InjRN+QCeLe5s
L2/6Bs3qn/rTjAPs6RrHn4k=
=Z29e
-----END PGP SIGNATURE-----

From pje at telecommunity.com  Thu Dec 15 01:06:35 2005
From: pje at telecommunity.com (Phillip J. Eby)
Date: Wed, 14 Dec 2005 19:06:35 -0500
Subject: [Distutils] Egg support for system packages (including
	bdist_wininst)
Message-ID: <5.1.1.6.0.20051214184924.03033a28@mail.telecommunity.com>

Setuptools in SVN now provides preliminary support for installing eggs in 
.egg-info directory style, so that setuptools-based projects can be wrapped 
by system packagers who wish to avoid using .egg files or directories.  In 
addition, you can now use setuptools to install non-setuptools packages in 
such a way that an .egg-info directory is included, so that 
setuptools-based packages can detect the presence of the 
non-setuptools-based package.

If you are building packages using a distutils command such as 
'bdist_wininst' that internally invokes the distutils "install" command, 
you do not need to do anything special.  Setuptools detects when the 
"install" command is invoked programmatically (via run_command()) and 
automatically enters its backward-compatibility mode.

However, if you are using a command such as 'bdist_rpm' that generates 
external scripts which then run "setup.py install", you will need to modify 
the generated scripts to run this instead:

     setup.py install --single-version-externally-managed

You will also need to include the --root or --record options.

For the 'bdist_rpm' command, setuptools automatically adjusts the 'spec' 
files before they're written, so bdist_rpm *should* work correctly, 
although I haven't tested it yet.  Please let me know if you find any 
problems, as I have not yet tested this installation mode with anything but 
bdist_wininst or direct manual installation.

For persons like Paul Moore who want to wrap egg-based packages in Windows 
installers, note that you can now take a setuptools-based package's source 
and run "setup.py bdist_wininst" and it will build a usable .exe.

In addition, you can build Windows installer-wrapped eggs for 
non-setuptools packages by changing your command line from:

     python setup.py bdist_wininst

to:
     python -c "import setuptools;execfile('setup.py')" bdist_wininst

This will build an .exe that includes .egg-info, so that other packages 
will be able to detect it at runtime.

Similarly, if you are building any other kind of system package using 
"python setup.py install", you can force .egg-info to be included by using 
this instead:

     python -c "import setuptools;execfile('setup.py')" install \
         --single-version-externally-managed ...

This support for system packages of eggs is still early and experimental; 
please let me know about any problems you may encounter or any questions 
you may have.  Thanks.


From jjl at pobox.com  Thu Dec 15 02:04:03 2005
From: jjl at pobox.com (John J Lee)
Date: Thu, 15 Dec 2005 01:04:03 +0000 (UTC)
Subject: [Distutils] Naming conventions for module unit tests?
In-Reply-To: <43A061EE.1050001@yahoo.com>
References: <5.1.1.6.0.20051208114741.03eb3680@mail.telecommunity.com>
	<439907CE.7070503@yahoo.com> <Pine.WNT.4.64.0512091421360.3480@shaolin>
	<439B8F29.3080404@yahoo.com> <Pine.LNX.4.58.0512111302340.6246@alice>
	<43A061EE.1050001@yahoo.com>
Message-ID: <Pine.LNX.4.58.0512150034310.6251@alice>

On Wed, 14 Dec 2005, Todd Greenwood-Geer wrote:
[...]
> Anyone have any good naming conventions for module tests?
> 
> [module name]_unittest.py
> test_[module_name].py
> unittest_[module_name].py

Hmm, I actually misread your question in your original email...

Python itself uses test_[module_name].py, so I guess that's a reasonable
standard.


> Then there's the issue of differentiating unit tests from functional
> tests, scenario tests...

personally I put my functional tests in a functional_tests.py, but then I
have too few functional tests in my open source stuff...


[...]
> Perhaps:
> [aggregation]_[level]_[module].py
> 	drt_unittest_foo.py
> 	bvt_functionaltest_foo.py
> 
> Or am I overly complicating things again?
[...]

That surely depends entirely on whether you're writing myfirstbabypackage
or some massive framework.

On the general subject of conventions, note that Barry Warsaw just
released a new PEP 8 draft (see last day or two's posts in python-dev
archives, particularly from Barry Warsaw).  python-dev is currently
discussing the momentous subject of whether function_names_like_this
should be blessed over functionNamesLikeThis, or just to give the standard
"pick one and stick to it" advice.


John

From tgreenwoodgeer at yahoo.com  Thu Dec 15 07:00:49 2005
From: tgreenwoodgeer at yahoo.com (Todd Greenwood-Geer)
Date: Wed, 14 Dec 2005 22:00:49 -0800
Subject: [Distutils] Easy Install / Setup Tools Example
In-Reply-To: <5.1.1.6.0.20051208234423.02d580e8@mail.telecommunity.com>
References: <5.1.1.6.0.20051208114741.03eb3680@mail.telecommunity.com>
	<5.1.1.6.0.20051208114741.03eb3680@mail.telecommunity.com>
	<5.1.1.6.0.20051208234423.02d580e8@mail.telecommunity.com>
Message-ID: <43A10691.6020900@yahoo.com>

I've taken to heart your suggestions. The result is what I hope is a 
very simple example / tutorial that demonstrates:

  * the file structure of a project destined to be an egg
  * how to use doctests and unit tests with setuptools / eggs
  * how to build and deploy an egg
  * how to register with pypi
  * how to (and not to) generate wrapper scripts
  * how to write documentation using restructured text

Instructions:
$ easy_install SimpleExampleEgg

That's it. You have now installed v 0.2. The tutorial docs
are the egg:

/usr/lib/python2.4/site-packages/SimpleExampleEgg-0.2-py2.4.egg:fruit/docs/readme.html

Could I get a quick review of this? I'm attaching the readme.html.

-Todd

Phillip J. Eby wrote:
> At 08:27 PM 12/8/2005 -0800, Todd Greenwood-Geer wrote:
> small_unittest.py::
> 
>>         import small
>>         import unittest
>>         import doctest
>>
>>         def getTestSuite():
>>                 suite = unittest.TestSuite()
>>                 for mod in small,:
>>                         suite.addTest(doctest.DocTestSuite(mod))
>>                 return suite
>>
>>         runner = unittest.TextTestRunner()
>>         runner.run(getTestSuite())
> 
> If 'small.small_unittest' is the actual name of the module, then doing this:
> 
>    python -c "from unittest import main; main(None)" 
> small.small_unittest.getTestSuite
> 
> will run the test, even if it's inside an egg.  No unzipping is 
> necessary.  If the egg wasn't installed with a .pth file (i.e. 
> --multi-version or to a non-standard location), you'll need to make sure 
> it's on PYTHONPATH, along with its dependencies.
> 
>> Here is what the src tree for the simple example looks like:
>>
>> dir listing::
>>
>>         small/src/small/test/test.txt
>>         small/src/small/small2.py
>>         small/src/small/small.py
>>         small/src/small/small_unittest.pyc
>>         small/src/small/__init__.py
>>         small/src/small/small_unittest.py
>>         small/src/small_unittest.py
> 
> This looks like a duplicate.  I'm also not sure why you have a small2, or 
> why you bother having a separate test subdirectory.  Indeed, for this 
> "small" of an example, a src/ directory is probably overkill.  For clarity, 
> I'd also suggest calling the topmost directory something like SmallExample 
> to help distinguish your project name (SmallExample) from your package name 
> (small).
> 
>>         small/setup_mult.py
> 
> What's this file for?
> 
> 
>>         small/ez_setup.py
> 
> If you're using Subversion, btw, you can use the svn:externals trick so 
> that you don't have to manually maintain this file; see the setuptools 
> manual under:
> 
> http://peak.telecommunity.com/DevCenter/setuptools#using-setuptools-without-bundling-it
> 
> 
> 
>> The thing to note here is how small.small and small.small2 are referenced::
>>
>>         from small import small as s1
>>         from small import small2 as s2
>>
>> For some reason, this gave me plenty of problems.
> 
> It's generally not a good idea to name a module the same as a package, and 
> definitely not the same as a class within the module, or else it gets 
> unclear in code which one you're talking about.  In a language like Java, 
> there's no such ambiguity because syntactically you can't refer to a class 
> where a package should be or vice versa, but in Python there are only 
> objects, so you should not name them all the same thing or you (and/or the 
> interpreter) will be confused.  :)
> 
> Similarly, I suspect that your example has way too many files perhaps 
> because you have a Java background?  It really isn't necessary to split 
> Python projects up into so many small pieces; it just makes more work for 
> you and doesn't get you any benefit until the files get too large to 
> conveniently work with.
> 
> 
>> Complex Case
>> ------------
>>
>> The complex case splits out the test modules into a nested package 
>> hierarchy like this:
> ...
>> Package Hierarchy
>> +++++++++++++++++
>>
>> package hierarchy::
>>
>>         small/src/
>>         small/src/size/
>>         small/src/size/small
>>         small/src/size/large
> 
> Ouch.  I thought the previous example was the complex one.  ;)  Seriously, 
> your simple example is way more complex than it needs to be.  This bigger 
> one makes my head hurt, so I'm not going to try to comment on it further, 
> except to suggest that its top-level directory should be named 
> "small-mult-test" since that's your project name.  Also, I'm guessing you 
> mean your package hiearchy is size, size.small, and size.large.  The 
> small/src stuff isn't part of the package hierarchy, just the directory 
> hierarchy.
> 
> By the way, the only reason to have a 'src' directory under your main 
> project directory is if you have a bunch of stuff that's *not* source code 
> there.  Such as documentation, documentation directories, test directories, 
> examples, etc.  Your projects here have none of those things, only a setup 
> script, so there's no reason to have a 'src' subdirectory; you could just 
> put the 'small' or 'size' package directly under the project directory, and 
> in fact this is the most common layout for published Python packages.  The 
> 'src' or 'lib' subdirectory approach is mainly used by large projects with 
> a lot of other things in their project directory, and by people who got 
> used to that format by working on large projects.  :)
> 
> 
>>         from size.small import small
>>         from size.large import large
> 
> I said I wasn't going to comment further, but this is technically a 
> repetition of my earlier comment: please don't name packages and modules 
> the same thing, you will confuse yourself and everyone else who will never 
> be sure if 'small' means 'size.small' or 'size.small.small' or worse, 
> 'size.small.small.small'.  Eeek!
> 
> 
>>  * url : need to see if that works for downloading dependencies (next 
>> tutorial, not this one)
> 
> Yes, it does, *if* you register your package with PyPI.  The URL given will 
> be scanned for download links when and if easy_install looks for your 
> package on PyPI.
> 
> 
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/distutils-sig/attachments/20051214/12eb0df8/readme-0001.html
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 189 bytes
Desc: OpenPGP digital signature
Url : http://mail.python.org/pipermail/distutils-sig/attachments/20051214/12eb0df8/signature-0001.pgp

From ronaldoussoren at mac.com  Thu Dec 15 08:17:32 2005
From: ronaldoussoren at mac.com (Ronald Oussoren)
Date: Thu, 15 Dec 2005 08:17:32 +0100
Subject: [Distutils] bdist_nsis, bdist_deb, bdist_msi, bdist_inno,
	bdist_pkgtool, bdist_sdux, bdist_mpkg, etc.
In-Reply-To: <439F4035.5070305@egenix.com>
References: <439EAFD9.8060005@egenix.com> <439EB1AF.4040809@egenix.com>
	<4F0360B5-9C0B-487E-BA8C-1210328F6C8C@redivi.com>
	<439F1D45.9090509@egenix.com>
	<BB8FCDBE-8F26-47E7-8D4F-42C7CE490FED@redivi.com>
	<439F4035.5070305@egenix.com>
Message-ID: <67071656-74BA-4CFF-BA9B-A5DA89A172FF@mac.com>


On 13-dec-2005, at 22:42, M.-A. Lemburg wrote:

> Bob Ippolito wrote:
>> On Dec 13, 2005, at 11:13 AM, M.-A. Lemburg wrote:
>>>>>> I'd like to focus attention on these distribution formats
>>>>>> that distutils is missing:
>>>>>>
>>>>>
>>>>> I'd like to another important candidate to the list:
>>>>>
>>>>> bdist_mpkg:
>>>>>    Mac OS X installer format
>>>>>    Part of py2app by Bob Ippolito
>>>>>    http://undefined.org/python/py2app.html#bdist-mpkg- 
>>>>> documentation
>>>>
>>>> bdist_mpkg is actually available on its own now, but I'm not  
>>>> sure adding
>>>> it to distutils is a great idea.  The current version is already  
>>>> moved
>>>> over to setuptools, and I'm pretty close to adding egg support  
>>>> to it.  I
>>>> may even require that the Python bits of the installation  
>>>> package are
>>>> eggs, as that would allow me to remove a bunch of code.  Eggs  
>>>> mostly
>>>> obsolete what bdist_mpkg is for, but it's also used to install  
>>>> things
>>>> like documentation and example code, so I'm not going to drop it
>>>> altogether.
>>>
>>> Looks like now is a good time to add it to distutils then :-)
>>
>> No.  Definitely not.
>> ...
>> In other words, I don't consent to adding bdist_mpkg to Python  
>> until it
>> supports eggs.
>
> Ok, then I'll remove the above candidate from the list.
>
> Anyone else interested in writing a bdist_mpkg command
> that doesn't require setuptools ?
>
> Given the good documentation for the format, this shouldn't
> be hard:
>
> http://s.sudre.free.fr/Stuff/PackageMaker_Howto.html

Should the volunteer also write the package management tools? One  
reason why
I'm interested in setuptools is that OSX's 'Installer.app' is just  
that, a tool
to install .pkg/.mpkg archives. You cannot uninstall packages and  
upgradeing
seems to leave files around unless you manually remove them in a pre/ 
post script.

OSX is great, but its installer application sucks. Setuptools/ 
easy_install solves
several problems for me:
1) uninstall
2) dependency management
3) automaticly downloading required software

And BTW the most common way to install software on OSX is by dragging  
an application
icon from a CD or disk image to the folder containing  
applications ;-) ;-)

Ronald

>
> -- 
> Marc-Andre Lemburg
> eGenix.com
>
> Professional Python Services directly from the Source  (#1, Dec 13  
> 2005)
>>>> Python/Zope Consulting and Support ...        http:// 
>>>> www.egenix.com/
>>>> mxODBC.Zope.Database.Adapter ...             http:// 
>>>> zope.egenix.com/
>>>> mxODBC, mxDateTime, mxTextTools ...        http:// 
>>>> python.egenix.com/
> ______________________________________________________________________ 
> __
>
> ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for  
> free ! ::::
> _______________________________________________
> Distutils-SIG maillist  -  Distutils-SIG at python.org
> http://mail.python.org/mailman/listinfo/distutils-sig


From p.f.moore at gmail.com  Thu Dec 15 10:19:00 2005
From: p.f.moore at gmail.com (Paul Moore)
Date: Thu, 15 Dec 2005 09:19:00 +0000
Subject: [Distutils] Egg support for system packages (including
	bdist_wininst)
In-Reply-To: <5.1.1.6.0.20051214184924.03033a28@mail.telecommunity.com>
References: <5.1.1.6.0.20051214184924.03033a28@mail.telecommunity.com>
Message-ID: <79990c6b0512150119t67be66eesb626d2c02e854265@mail.gmail.com>

On 12/15/05, Phillip J. Eby <pje at telecommunity.com> wrote:
> For persons like Paul Moore who want to wrap egg-based packages in Windows
> installers, note that you can now take a setuptools-based package's source
> and run "setup.py bdist_wininst" and it will build a usable .exe.
>
> In addition, you can build Windows installer-wrapped eggs for
> non-setuptools packages by changing your command line from:
>
>      python setup.py bdist_wininst
>
> to:
>      python -c "import setuptools;execfile('setup.py')" bdist_wininst
>
> This will build an .exe that includes .egg-info, so that other packages
> will be able to detect it at runtime.

Massively cool! Thank you for putting this in place.
Paul.

From bh at intevation.de  Thu Dec 15 15:53:33 2005
From: bh at intevation.de (Bernhard Herzog)
Date: Thu, 15 Dec 2005 15:53:33 +0100
Subject: [Distutils] bdist_nsis, bdist_deb, bdist_msi, bdist_inno,
 bdist_pkgtool, bdist_sdux, etc.
In-Reply-To: <43A007E8.8040703@egenix.com> (M.'s message of "Wed, 14 Dec
	2005 12:54:16 +0100")
References: <439EAFD9.8060005@egenix.com> <u0ddkmm6.fsf@python.net>
	<439EFE58.7050504@egenix.com> <s9z4q5cq5g0.fsf@thetis.intevation.de>
	<43A007E8.8040703@egenix.com>
Message-ID: <s9zek4e76oi.fsf@thetis.intevation.de>

"M.-A. Lemburg" <mal at egenix.com> writes:

> Bernhard Herzog wrote:
>> "M.-A. Lemburg" <mal at egenix.com> writes:
>>> Bernhard Herzog started one some time ago:
>>> http://mail.python.org/pipermail/distutils-sig/2001-September/002597.html
>> 
>> FWIW, the code that's linked from that posting is still unchanged in
>> Thuban and worked fine last time I made a release about a year ago with
>> python 2.3.
>
> Would you be willing to contribute the code to the PSF ?

Yes.  I take it we need to fill out and sign the PSF contributor
agreement and send or fax it to the PSF?  "We" being Intevation GmbH, my
employer and actual copyright holder for the code in question.

> We could then use is as starting point for a more elaborate
> implementation.

Great!

   Bernhard

-- 
Intevation GmbH                                 http://intevation.de/
Skencil                                           http://skencil.org/
Thuban                                  http://thuban.intevation.org/

From pje at telecommunity.com  Fri Dec 16 21:14:27 2005
From: pje at telecommunity.com (Phillip J. Eby)
Date: Fri, 16 Dec 2005 15:14:27 -0500
Subject: [Distutils] Updated installation instructions for
	EasyInstall/setuptools
Message-ID: <5.1.1.6.0.20051216134912.0372f900@mail.telecommunity.com>

I just checked in (and updated on the Wiki) the installation instructions 
for setuptools and EasyInstall to cover a lot of frequently-asked questions 
and problems (e.g. how do I install without net access?), as well as to 
explain additional approaches to custom installation locations.

The new docs on custom installation are at:

http://peak.telecommunity.com/DevCenter/EasyInstall#custom-installation-locations

And the main installation instructions, now with better troubleshooting and 
alternatives information, is at:

http://peak.telecommunity.com/DevCenter/EasyInstall#installation-instructions

Please let me know whether these are more helpful than the old versions.

I've also added a new section to the setuptools manual called "What Your 
Users Should Know":

http://peak.telecommunity.com/DevCenter/setuptools#what-your-users-should-know

This section provides developers using setuptools with notes on what they 
should tell users about installing their project -- such as that they 
should read the "custom installation" docs if they plan to install anywhere 
but the primary site-packages, and tips on issues ranging from system 
packaging (0.6a9dev+ only) to lack of network access.

Feedback on this new manual section would also be appreciated.

There is only one more feature I'm planning to add before the official 
release of setuptools 0.6a9: support for building and installing shared 
libraries linked to Python extensions.  If you are not using the 0.6a9 
development version yet, I encourage you to upgrade as soon as practical so 
that I can address any problems you find before making the official 
release.  To install the in-development version, just run:

     ez_setup.py setuptools==dev

after first making sure you have done any pre-installation steps described 
by the updated installation instructions above.  Thanks, and please let me 
know of any questions or problems you may have with any of the above.


From mcantor at stanford.edu  Fri Dec 16 21:44:54 2005
From: mcantor at stanford.edu (mike cantor)
Date: Fri, 16 Dec 2005 12:44:54 -0800
Subject: [Distutils] problem finding dll with C extension
Message-ID: <6.0.1.1.2.20051216123008.01ec9d70@mcantor.pobox.stanford.edu>

Hi,

I have just built a C extension.  "python setup.py install" successfully 
builds the extension and puts the library , PGModule.pyd, in the 
Python23\Lib\site-packages folder.  However, from within Python, "import 
PGModule" yields "ImportError: DLL load failed: The specified module could 
not be found"

The setup.py file I am using looks like this:

from distutils.core import setup, Extension
setup(name='PGModule', ext_modules=[ Extension('PGModule',
	sources = ['PGfit_DataTypes.c', 'PGModule.c'],
	include_dirs=["E:\Program Files\GnuWin32\include","E:\sundials\includes"],
	libraries = ["libgsl"],
	library_dirs = ["E:\Program Files\GnuWin32\lib"]) ])

A few more clues/oddities:  The extension uses the gsl (gnu scientific 
library) library.  Without the "libraries" and "library_dir" flags, my code 
compiles fine but does not link (I get error messages like "in PGModule.o 
unidentified reference gsl_set_vector" etc).  Other extensions that I 
compile from the same directory and using the same setup file, but minus 
the library flags, build fine and are importable and usable from within 
Python.  When I look in the site-packages folder, PGModule.pyd is right 
there along with other modules (e.g. PGFit.pyd), yet "import PGModule" from 
a python shell yields the aforementioned error while "import PGFit" 
executes successfully.

Any ideas?

Thanks a ton,
-mike



From mcantor at stanford.edu  Sat Dec 17 01:31:41 2005
From: mcantor at stanford.edu (mike cantor)
Date: Fri, 16 Dec 2005 16:31:41 -0800
Subject: [Distutils] problem finding dll with C extension
In-Reply-To: <6.0.1.1.2.20051216123008.01ec9d70@mcantor.pobox.stanford.e
 du>
References: <6.0.1.1.2.20051216123008.01ec9d70@mcantor.pobox.stanford.edu>
Message-ID: <6.0.1.1.2.20051216162920.01ef6418@mcantor.pobox.stanford.edu>

Another piece of information.  The error message I get ("ImportError: DLL 
load failed: The specified module could not be found") is different that 
the error message I get if I try to import a non-existent module (namely, 
"ImportError: No module named Foo").  So somewhere within the module a DLL 
is failing to load.  Any ideas????




At 12:44 PM 12/16/2005, mike cantor wrote:
>Hi,
>
>I have just built a C extension.  "python setup.py install" successfully 
>builds the extension and puts the library , PGModule.pyd, in the 
>Python23\Lib\site-packages folder.  However, from within Python, "import 
>PGModule" yields "ImportError: DLL load failed: The specified module could 
>not be found"
>
>The setup.py file I am using looks like this:
>
>from distutils.core import setup, Extension
>setup(name='PGModule', ext_modules=[ Extension('PGModule',
>         sources = ['PGfit_DataTypes.c', 'PGModule.c'],
>         include_dirs=["E:\Program 
> Files\GnuWin32\include","E:\sundials\includes"],
>         libraries = ["libgsl"],
>         library_dirs = ["E:\Program Files\GnuWin32\lib"]) ])
>
>A few more clues/oddities:  The extension uses the gsl (gnu scientific 
>library) library.  Without the "libraries" and "library_dir" flags, my 
>code compiles fine but does not link (I get error messages like "in 
>PGModule.o unidentified reference gsl_set_vector" etc).  Other extensions 
>that I compile from the same directory and using the same setup file, but 
>minus the library flags, build fine and are importable and usable from 
>within Python.  When I look in the site-packages folder, PGModule.pyd is 
>right there along with other modules (e.g. PGFit.pyd), yet "import 
>PGModule" from a python shell yields the aforementioned error while 
>"import PGFit" executes successfully.
>
>Any ideas?
>
>Thanks a ton,
>-mike



From pje at telecommunity.com  Sat Dec 17 01:42:56 2005
From: pje at telecommunity.com (Phillip J. Eby)
Date: Fri, 16 Dec 2005 19:42:56 -0500
Subject: [Distutils] problem finding dll with C extension
In-Reply-To: <6.0.1.1.2.20051216162920.01ef6418@mcantor.pobox.stanford.e
 du>
References: <6.0.1.1.2.20051216123008.01ec9d70@mcantor.pobox.stanford.e du>
	<6.0.1.1.2.20051216123008.01ec9d70@mcantor.pobox.stanford.edu>
Message-ID: <5.1.1.6.0.20051216194205.01e06e70@mail.telecommunity.com>

At 04:31 PM 12/16/2005 -0800, mike cantor wrote:
>Another piece of information.  The error message I get ("ImportError: DLL
>load failed: The specified module could not be found") is different that
>the error message I get if I try to import a non-existent module (namely,
>"ImportError: No module named Foo").  So somewhere within the module a DLL
>is failing to load.  Any ideas????

Either libgsl or one of its dependencies isn't loading.  My guess would be 
that E:\Program Files\GnuWin32\lib isn't on your PATH.


From mcantor at stanford.edu  Sat Dec 17 01:50:48 2005
From: mcantor at stanford.edu (mike cantor)
Date: Fri, 16 Dec 2005 16:50:48 -0800
Subject: [Distutils] problem finding dll with C extension
In-Reply-To: <5.1.1.6.0.20051216194205.01e06e70@mail.telecommunity.com>
References: <6.0.1.1.2.20051216123008.01ec9d70@mcantor.pobox.stanford.e du>
	<6.0.1.1.2.20051216123008.01ec9d70@mcantor.pobox.stanford.edu>
	<5.1.1.6.0.20051216194205.01e06e70@mail.telecommunity.com>
Message-ID: <6.0.1.1.2.20051216164959.01e8acb8@mcantor.pobox.stanford.edu>

Thanks so much - that was it.  Actually it was "E:\Program 
Files\GnuWin32\bin" that was missing.

-mike


At 04:42 PM 12/16/2005, Phillip J. Eby wrote:
>At 04:31 PM 12/16/2005 -0800, mike cantor wrote:
>>Another piece of information.  The error message I get ("ImportError: DLL
>>load failed: The specified module could not be found") is different that
>>the error message I get if I try to import a non-existent module (namely,
>>"ImportError: No module named Foo").  So somewhere within the module a DLL
>>is failing to load.  Any ideas????
>
>Either libgsl or one of its dependencies isn't loading.  My guess would be 
>that E:\Program Files\GnuWin32\lib isn't on your PATH.



From rasky at develer.com  Sat Dec 17 02:18:11 2005
From: rasky at develer.com (Giovanni Bajo)
Date: Sat, 17 Dec 2005 02:18:11 +0100
Subject: [Distutils] Bug with bdist_rpm and version containing '-'
Message-ID: <0d3301c602a7$bf42ae10$f74a2597@bagio>

Hello,

while using distutils shipped with Python 2.4.2, I found out a problem when
running bdist_rpm if the version of the package contains an hyphen ('-'). The
problem is that RPMs do not support hyphens in versions, and bdist_rpm tries to
work around this by replacing hyphens with underscores ('_'). While this is the
correct solution, it appears to be incomplete, because bdist_rpm invokes
'sdist' to build the .tar.gz (or .tar.bz2) without notifying it of the change
of the version string (and thus of the change of the filename that sdist will
have to generate).

I have succesfully worked around this bug with this hack in my setup.py:

class _bdist_rpm(bdist_rpm):
    def run_command(self, cmd, *args, **kwargs):
        if cmd == "sdist":
            old_version = self.distribution.metadata.version
            if self.distribution.metadata.version is not None:
                self.distribution.metadata.version = old_version.replace("-",
"_")
            bdist_rpm.run_command(self, cmd, *args, **kwargs)
            self.distribution.metadata.version = old_version
            return
        bdist_rpm.run_command(self, cmd, *args, **kwargs)

This works by patching the version at the moment of the call to 'sdist' from
within bdist_rpm. Of course it's a gross hack but I couldn't find a better way
:)

Thanks,
Giovanni Bajo


From pje at telecommunity.com  Sat Dec 17 03:02:37 2005
From: pje at telecommunity.com (Phillip J. Eby)
Date: Fri, 16 Dec 2005 21:02:37 -0500
Subject: [Distutils] Bug with bdist_rpm and version containing '-'
In-Reply-To: <0d3301c602a7$bf42ae10$f74a2597@bagio>
Message-ID: <5.1.1.6.0.20051216205558.01de19c0@mail.telecommunity.com>

At 02:18 AM 12/17/2005 +0100, Giovanni Bajo wrote:
>Hello,
>
>while using distutils shipped with Python 2.4.2, I found out a problem when
>running bdist_rpm if the version of the package contains an hyphen ('-'). The
>problem is that RPMs do not support hyphens in versions, and bdist_rpm 
>tries to
>work around this by replacing hyphens with underscores ('_'). While this 
>is the
>correct solution, it appears to be incomplete, because bdist_rpm invokes
>'sdist' to build the .tar.gz (or .tar.bz2) without notifying it of the change
>of the version string (and thus of the change of the filename that sdist will
>have to generate).

For a more direct fix (patching the bdist_rpm command to handle this 
correctly), see:

http://mail.python.org/pipermail/patches/2005-November/018408.html

Setuptools includes a bdist_rpm subclass that implements the same fix for 
both Python 2.3 and 2.4, so using setuptools is also a workaround for this 
problem.


From rasky at develer.com  Sat Dec 17 10:26:24 2005
From: rasky at develer.com (Giovanni Bajo)
Date: Sat, 17 Dec 2005 10:26:24 +0100
Subject: [Distutils] Bug with bdist_rpm and version containing '-'
References: <5.1.1.6.0.20051216205558.01de19c0@mail.telecommunity.com>
Message-ID: <0f6201c602eb$f3587ab0$f74a2597@bagio>

Phillip J. Eby <pje at telecommunity.com> wrote:

> For a more direct fix (patching the bdist_rpm command to handle this
> correctly), see:
>
> http://mail.python.org/pipermail/patches/2005-November/018408.html
>
> Setuptools includes a bdist_rpm subclass that implements the same fix
> for both Python 2.3 and 2.4, so using setuptools is also a workaround
> for this problem.


Thanks for the reference. I will try the patch in the SF tracker but I assume
it's going to work. Can we have that patch applied to the Python tree?

Giovanni Bajo


From sharkypt at gmail.com  Sat Dec 17 14:24:40 2005
From: sharkypt at gmail.com (Sharky On PTNet)
Date: Sat, 17 Dec 2005 13:24:40 +0000
Subject: [Distutils] Upgrade a package
Message-ID: <40b686290512170524x370a558t@mail.gmail.com>

Hi,

Every time I do an 'easy_install -U ...' for a package, it installs ok but
remain with the old package in site-packages.

Is there a way do remove old packages?

ex.:
$ ls -ltrd /usr/lib/python2.4/site-packages/Tur*
drwxr-xr-x  4 root root 4096 2005-12-07 22:14
/usr/lib/python2.4/site-packages/TurboGears-0.8a5-py2.4.egg
drwxr-xr-x  4 root root 4096 2005-12-17 13:13
/usr/lib/python2.4/site-packages/TurboGears-0.9a0dev-py2.4.egg

$ ls -ltrd /usr/lib/python2.4/site-packages/setu*
drwxr-xr-x  4 root root 4096 2005-11-24 19:08
/usr/lib/python2.4/site-packages/setuptools-0.6a7-py2.4.egg
drwxr-xr-x  4 root root 4096 2005-12-07 21:24
/usr/lib/python2.4/site-packages/setuptools-0.6a9dev_r41630-py2.4.egg
-rw-r--r--  1 root root   70 2005-12-07 21:24
/usr/lib/python2.4/site-packages/setuptools.pth
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/distutils-sig/attachments/20051217/eb47da7d/attachment.html

From pje at telecommunity.com  Sat Dec 17 16:52:38 2005
From: pje at telecommunity.com (Phillip J. Eby)
Date: Sat, 17 Dec 2005 10:52:38 -0500
Subject: [Distutils] Upgrade a package
In-Reply-To: <40b686290512170524x370a558t@mail.gmail.com>
Message-ID: <5.1.1.6.0.20051217104949.01de3390@mail.telecommunity.com>

At 01:24 PM 12/17/2005 +0000, Sharky On PTNet wrote:
>Hi,
>
>Every time I do an 'easy_install -U ...' for a package, it installs ok but 
>remain with the old package in site-packages.
>
>Is there a way do remove old packages?

See:

http://peak.telecommunity.com/DevCenter/EasyInstall#uninstalling-packages

This will get better in the 0.7 series, when the "nest" command gets 
implemented.  "nest" will have a variety of subcommands including ones to 
remove a particular egg and to get rid of unused eggs ("nest rm" and "nest 
purge").  It will also have commands for commonly-used combinations of 
easy_install options, as easy_install has grown way too many options at 
this point.


From jjl at pobox.com  Sun Dec 18 02:03:08 2005
From: jjl at pobox.com (John J Lee)
Date: Sun, 18 Dec 2005 01:03:08 +0000 (UTC)
Subject: [Distutils] [setuptools] setup.py develop inaccurate error message?
Message-ID: <Pine.LNX.4.58.0512180016580.6216@alice>

I got an error message while running setup.py develop that I don't
understand: seems the error message claims an automatic install of a
dependency failed, even though the dependency's egg actually did get
installed and appears correct.


(A bit tangential, but explanatory) background: Noting PJE's doc update
repeating your recomendation to run setup.py develop.  I wondered how
setuptools would handle fetching a newer SVN revision of a dependency D
installed with setup.py develop from an SVN checkout when running setup.py
develop on a project P that depends on D.  Would it use the .egg-link file
to find and update the original working copy from which D was installed,
or instead create a fresh SVN checkout?

D == ClientForm

P == mechanize



So, I tried the following:

1. Installed ClientForm in site-packages from an SVN checkout using 
setup.py develop

2. Installed mechanize, which depends on Clientform, from an SVN checkout
using setup.py develop

3. Modified my local copy of mechanize's setup.py to require a newer
revision of ClientForm than I had installed in step 1.  The modified
dependency reads "ClientForm>=0.2.2-r21160, ==dev"

4. Ran python setup.py develop again for mechanize.


That last step installed an egg with the right name (using a fresh
checkout, which I guess answers the question above that motivated me), but
also ended with this error message:

error: Could not find required distribution ClientForm==dev,>=0.2.2-r21160


However, the ClientForm egg did in fact get installed and does indeed
contain revision 21160.  So, why the error message?


Full output of step 4:


mechanize[0]# python setup.py develop
running develop
running egg_info
writing requirements to ./mechanize.egg-info/requires.txt
writing ./mechanize.egg-info/PKG-INFO
writing top-level names to ./mechanize.egg-info/top_level.txt
running build_ext
Creating /usr/lib/python2.3/site-packages/mechanize.egg-link (link to .)
mechanize 0.0.12a.dev-r20866 is already the active version in 
easy-install.pth

Installed /hda/usr/local/buf/comp/dev/wwwsearch/wwwsearch/mechanize
Processing dependencies for mechanize==0.0.12a.dev-r20866
Searching for ClientForm==dev,>=0.2.2-r21160
Reading http://www.python.org/pypi/ClientForm/
Reading http://wwwsearch.sourceforge.net/ClientForm/
Best match: ClientForm dev
Downloading 
http://codespeak.net/svn/wwwsearch/ClientForm/trunk#egg=ClientForm-dev
Doing subversion checkout from 
http://codespeak.net/svn/wwwsearch/ClientForm/trunk to 
/tmp/easy_install-31mf3S/trunk
Processing trunk
Running setup.py -q bdist_egg --dist-dir 
/tmp/easy_install-31mf3S/trunk/egg-dist-tmp-qFDl23
Removing ClientForm 0.2.2.dev-r20620 from easy-install.pth file
Adding ClientForm 0.2.2.dev-r21160 to easy-install.pth file

Installed 
/usr/lib/python2.3/site-packages/ClientForm-0.2.2.dev_r21160-py2.3.egg
error: Could not find required distribution ClientForm==dev,>=0.2.2-r21160
mechanize[0]#
mechanize[0]# cd ../ClientForm
ClientForm[0]# svn st -u
?                   README.html
       *    20620   README.html.in
       *    20620   test.py
       *    20620   setup.py
M      *    20620   ClientForm.py
?                   ClientForm.egg-info
?                   ClientForm.pyc
X                   ez_setup
Status against revision:  21261

Performing status on external item at 'ez_setup'
?                   ez_setup/__init__.pyc
Status against revision:   2110
ClientForm[0]# cat /usr/lib/python2.3/site-packages/mechanize.egg-link
/hda/usr/local/buf/comp/dev/wwwsearch/wwwsearch/mechanizeClientForm[0]# pwd
/home/john/comp/dev/wwwsearch/wwwsearch/ClientForm
ClientForm[0]#


John

From pje at telecommunity.com  Sun Dec 18 02:02:38 2005
From: pje at telecommunity.com (Phillip J. Eby)
Date: Sat, 17 Dec 2005 20:02:38 -0500
Subject: [Distutils] [setuptools] setup.py develop inaccurate error
 message?
In-Reply-To: <Pine.LNX.4.58.0512180016580.6216@alice>
Message-ID: <5.1.1.6.0.20051217195818.01f45ec0@mail.telecommunity.com>

At 01:03 AM 12/18/2005 +0000, John J Lee wrote:
>...
>However, the ClientForm egg did in fact get installed and does indeed
>contain revision 21160.  So, why the error message?
>...
>Adding ClientForm 0.2.2.dev-r21160 to easy-install.pth file
>...
>error: Could not find required distribution ClientForm==dev,>=0.2.2-r21160

0.2.2.dev-r21160 was installed, but your requirement was for 0.2.2-r21160.

The installed version is r21160 of an in-development pre-release of 0.2.2, 
while your requirement is for an r21160 *post*-release patch of 0.2.2.

In other words, the error is due to a typo in your requirement.

With respect to the idea of doing SVN updates of dependencies (as opposed 
to doing fresh checkouts and egg builds), it's an interesting idea but it 
seems like the implementation would be complex, to say the least.  A key 
issue is that .egg-link files don't have any way to point to their 
project's setup script.  But I'll give it some thought for when I start 
work on 0.7, which will be refactoring some internals of the 
requirement/resolution system anyway.  Maybe there will be a way to work 
something in then.


From jjl at pobox.com  Sun Dec 18 15:47:55 2005
From: jjl at pobox.com (John J Lee)
Date: Sun, 18 Dec 2005 14:47:55 +0000 (UTC)
Subject: [Distutils] [setuptools] How to maintain version strings?
Message-ID: <Pine.LNX.4.58.0512180313540.6216@alice>

Annoying little issue: I have version strings here and there in my
projects, and would prefer to keep the version in a single place.  In
particular, I have __version__ strings in module code, and also in
setup.py.

I also want to avoid importing code from setup.py, because people
sometimes want to install using a Python executable other than the one
with which they intend to import the package (I assume that's sensible &
possible with setuptools packages), and imports in setup.py have broken
that in the past.

Seems I can't just open('mymodule.py') and find the version string with a
regexp, I assume because of setuptools' "sandbox".  So, I just have to
have my release script check that I remembered to update both.  No big
deal, but it annoys me :-)

Anybody have a way of doing this (or a slap to my forehead to teach me the
simple way to do it ;-)?


John

From jjl at pobox.com  Sun Dec 18 15:52:47 2005
From: jjl at pobox.com (John J Lee)
Date: Sun, 18 Dec 2005 14:52:47 +0000 (UTC)
Subject: [Distutils] [setuptools] Keeping up with SVN (was: Re: setup.py
 develop inaccurate...)
In-Reply-To: <5.1.1.6.0.20051217195818.01f45ec0@mail.telecommunity.com>
References: <5.1.1.6.0.20051217195818.01f45ec0@mail.telecommunity.com>
Message-ID: <Pine.LNX.4.58.0512180303530.6216@alice>

On Sat, 17 Dec 2005, Phillip J. Eby wrote:
[...]
> 0.2.2.dev-r21160 was installed, but your requirement was for 0.2.2-r21160.
> 
> The installed version is r21160 of an in-development pre-release of 0.2.2, 
> while your requirement is for an r21160 *post*-release patch of 0.2.2.
> 
> In other words, the error is due to a typo in your requirement.

Think-o, in fact.  Sorry, I posted too quickly.


> With respect to the idea of doing SVN updates of dependencies (as opposed 
> to doing fresh checkouts and egg builds), it's an interesting idea but it 
> seems like the implementation would be complex, to say the least.  A key 
[...]

I wasn't surprised to find it works the way it does, but mildly
disappointed.

Do other people work with multi-egg projects (that don't necessarily share
a single version control root), and want to make it possible for people to
keep up with version control without updating multiple SVN checkouts by
hand (and without doing multiple checkouts)?  SVN in particular.  I know
one can always just write a simple script to do this kind of thing, but
multiple platforms, changing dependencies, mucking around with PYTHONPATH
or .pth files etc. make it less simple, which makes setuptools attractive.  

Probably I should instead work to get rid of my scrappy little release
scripts, use branches a bit more, and just release much more often...


John

From pje at telecommunity.com  Sun Dec 18 17:33:34 2005
From: pje at telecommunity.com (Phillip J. Eby)
Date: Sun, 18 Dec 2005 11:33:34 -0500
Subject: [Distutils] [setuptools] How to maintain version strings?
In-Reply-To: <Pine.LNX.4.58.0512180313540.6216@alice>
Message-ID: <5.1.1.6.0.20051218112508.01f63410@mail.telecommunity.com>

At 02:47 PM 12/18/2005 +0000, John J Lee wrote:
>I also want to avoid importing code from setup.py, because people
>sometimes want to install using a Python executable other than the one
>with which they intend to import the package (I assume that's sensible &
>possible with setuptools packages), and imports in setup.py have broken
>that in the past.

It's also a big problem if you have dependencies that you import; 
setuptools doesn't install your dependencies until after the setup script 
has effectively run to completion, so you might be trying to import stuff 
that's not yet installed.


>Seems I can't just open('mymodule.py') and find the version string with a
>regexp, I assume because of setuptools' "sandbox".

I don't see why not; the sandbox should only restrict *writing* to files, 
not reading them, and in any case it only restricts writing *outside* the 
subdirectory tree where the setup script is located.


>Anybody have a way of doing this (or a slap to my forehead to teach me the
>simple way to do it ;-)?

The 'networkx' project on PyPI does this by having a 'release.py' inside 
the main package that contains all the variables that need to be shared 
between the package and the setup script.  The setup script uses 
execfile('packagename/release.py')  to obtain the variables, while the 
package does a 'from release import *'.  Thus, the variables are in a 
single file.

I use a slightly different technique myself; PEAK includes a tool called 
'version' that can automatically edit a variety of files and update the 
included version numbers in a variety of user-defined formats.  When I bump 
setuptools alpha versions, I just run "version incr build" to bump the 
micro number.  When I move from say, 0.6 to 0.7, I run "version incr 
minor".  Anyway, it basically works by having a configuration file 
describing what to search for and edit in each file that contains the 
version number, as well as what parts there are in my versioning scheme and 
the different formats for "spelling" that versioning scheme.  The 
configuration file itself has a "#!peak version-config" line that makes it 
executable, so I just run it to update my release scripts, setup.py, 
__version__ variables, etc.

I believe there is at least one other tool out there that does something 
similar on SourceForge, but I don't remember its name.


From pje at telecommunity.com  Sun Dec 18 17:42:57 2005
From: pje at telecommunity.com (Phillip J. Eby)
Date: Sun, 18 Dec 2005 11:42:57 -0500
Subject: [Distutils] [setuptools] Keeping up with SVN (was: Re: setup.py
 develop inaccurate...)
In-Reply-To: <Pine.LNX.4.58.0512180303530.6216@alice>
References: <5.1.1.6.0.20051217195818.01f45ec0@mail.telecommunity.com>
	<5.1.1.6.0.20051217195818.01f45ec0@mail.telecommunity.com>
Message-ID: <5.1.1.6.0.20051218113747.038bd7b0@mail.telecommunity.com>

At 02:52 PM 12/18/2005 +0000, John J Lee wrote:
>Do other people work with multi-egg projects (that don't necessarily share
>a single version control root), and want to make it possible for people to
>keep up with version control without updating multiple SVN checkouts by
>hand (and without doing multiple checkouts)?  SVN in particular.  I know
>one can always just write a simple script to do this kind of thing, but
>multiple platforms, changing dependencies, mucking around with PYTHONPATH
>or .pth files etc. make it less simple, which makes setuptools attractive.

After giving it some more thought, I think having a 'nest update --develop' 
command might be the way to do this, assuming that 'nest update' ends up 
being a tool to check whether there are any updated versions of stuff you 
have installed.  The --develop would tell it to only update all the SVN or 
CVS checkouts referenced by egg-links, and rerun "develop" on all of them 
afterwards.  (As opposed to doing that and then updating any eggs too.)

Anyway, "nest" won't appear until the 0.7 series starts, but this would 
probably be a cleaner way to implement the feature than hacking the normal 
easy_install process to incorporate it.


From Scott.Daniels at Acm.Org  Mon Dec 19 07:51:04 2005
From: Scott.Daniels at Acm.Org (Scott David Daniels)
Date: Sun, 18 Dec 2005 22:51:04 -0800
Subject: [Distutils] specify zip method (ZIP_STORED, ZIP_DEFLATED)
In-Reply-To: <dmeeti$rmu$1@sea.gmane.org>
References: <dmeeti$rmu$1@sea.gmane.org>
Message-ID: <do5l8b$dop$1@sea.gmane.org>

Wolfgang Langner wrote:
> I think it is possible to speed up something if ZIP_STORED is
> used to "compress" egg packages.
It might be, but it might be faster on a modern machine to do less
I/O and more calculation in reading a file.

--Scott David Daniels
Scott.Daniels at Acm.Org


From jjl at pobox.com  Mon Dec 19 11:51:52 2005
From: jjl at pobox.com (John J Lee)
Date: Mon, 19 Dec 2005 10:51:52 +0000 (GMT Standard Time)
Subject: [Distutils] [setuptools] Interactive help (was Re: documentation
	for mechanize)
In-Reply-To: <20051219050823.GA26599@e-smith.com>
References: <20051219050823.GA26599@e-smith.com>
Message-ID: <Pine.WNT.4.64.0512191049270.3204@shaolin>

On Mon, 19 Dec 2005, Michael P. Soulier wrote:

>>>> import mechanize
>>>> help(mechanize)
> Traceback (most recent call last):
[...]
> OSError: [Errno 20] Not a directory:
> '/usr/lib/python2.3/site-packages/mechanize-0.0.11a-py2.3.egg/mechanize'
>
> Does this not work with egg files? Has anyone told the PEAK people about
> this problem? Is there another way to get at the docstrings? Could we
> just dump the html output from pydoc to the website?

Is this expected?


John

From pje at telecommunity.com  Mon Dec 19 16:50:32 2005
From: pje at telecommunity.com (Phillip J. Eby)
Date: Mon, 19 Dec 2005 10:50:32 -0500
Subject: [Distutils] [setuptools] Interactive help (was Re:
 documentation for mechanize)
In-Reply-To: <Pine.WNT.4.64.0512191049270.3204@shaolin>
References: <20051219050823.GA26599@e-smith.com>
	<20051219050823.GA26599@e-smith.com>
Message-ID: <5.1.1.6.0.20051219104859.020a7c50@mail.telecommunity.com>

At 10:51 AM 12/19/2005 +0000, John J Lee wrote:
>On Mon, 19 Dec 2005, Michael P. Soulier wrote:
>
> >>>> import mechanize
> >>>> help(mechanize)
> > Traceback (most recent call last):
>[...]
> > OSError: [Errno 20] Not a directory:
> > '/usr/lib/python2.3/site-packages/mechanize-0.0.11a-py2.3.egg/mechanize'
> >
> > Does this not work with egg files? Has anyone told the PEAK people about
> > this problem? Is there another way to get at the docstrings? Could we
> > just dump the html output from pydoc to the website?
>
>Is this expected?

pydoc doesn't work on packages in zip files.  It works on *modules* in 
zipfiles, and anything in the modules; it's only packages that it's broken 
for.  I may try to include a monkeypatch for that, as well as a backported 
fix for the Python source, but at the moment the only way to get around it 
is to install with --always-unzip/-Z or set the "not zip-safe" flag in your 
project's setup.py


From daniel.gloeckner at ipsi.fraunhofer.de  Mon Dec 19 16:59:26 2005
From: daniel.gloeckner at ipsi.fraunhofer.de (=?ISO-8859-1?Q?Daniel_Gl=F6ckner?=)
Date: Mon, 19 Dec 2005 16:59:26 +0100
Subject: [Distutils] using distutils / setuptools on Windows (MinGW)
Message-ID: <43A6D8DE.2060501@ipsi.fraunhofer.de>

Hi all!

In our working group, we're starting to use python for some machine 
learning tasks.
Performance needs force us to deliver some of our code as optimized C code.

To make a long story short: I was the one to discover howto deploy 
native python extensions on windows.
The online documentation on various sites about that topic is quite old 
and sometimes faulty.

I think the setuptools are a big step forward for professional 
deployment of python applicatoions. Keep up the good work guys!

Maybe I can contribute to the project by giving you a more recent 
documentation on compiling and deploying python code and python C 
extensions on windows using
-MinGW
-setuptools / distutils
-SWIG.

Cheers,
Daniel


From pje at telecommunity.com  Mon Dec 19 19:20:42 2005
From: pje at telecommunity.com (Phillip J. Eby)
Date: Mon, 19 Dec 2005 13:20:42 -0500
Subject: [Distutils] using distutils / setuptools on Windows (MinGW)
In-Reply-To: <43A6D8DE.2060501@ipsi.fraunhofer.de>
Message-ID: <5.1.1.6.0.20051219131311.020a8fd8@mail.telecommunity.com>

At 04:59 PM 12/19/2005 +0100, Daniel Gl?ckner wrote:
>Maybe I can contribute to the project by giving you a more recent
>documentation on compiling and deploying python code and python C
>extensions on windows using
>-MinGW

FYI, the shortest path to using MinGW with distutils or setuptools is to 
run this script and follow its directions:

http://mail.python.org/pipermail/python-dev/2004-January/041676.html

Once you've done that, the version of Python you ran it with will use the 
MinGW compiler automatically for all building, with no '-cmingw32' required.

The script should probably be added to Python's "tools" directory, 
particularly for Windows.  I've just never gotten around to going through 
the necessary ceremony to get it in.

Martin v Loewis has also offered to change Python's Windows distribution a 
bit so that the script would be unnecessary, except for setting the [build] 
options in distutils.cfg, but I'm not sure if that's happened yet either, 
even for Python 2.5.  But the script will work for Python 2.3 and up, maybe 
even 2.2.


From p.f.moore at gmail.com  Mon Dec 19 21:01:25 2005
From: p.f.moore at gmail.com (Paul Moore)
Date: Mon, 19 Dec 2005 20:01:25 +0000
Subject: [Distutils] using distutils / setuptools on Windows (MinGW)
In-Reply-To: <5.1.1.6.0.20051219131311.020a8fd8@mail.telecommunity.com>
References: <43A6D8DE.2060501@ipsi.fraunhofer.de>
	<5.1.1.6.0.20051219131311.020a8fd8@mail.telecommunity.com>
Message-ID: <79990c6b0512191201r33e4a3f4r240460e6ba71ea75@mail.gmail.com>

On 12/19/05, Phillip J. Eby <pje at telecommunity.com> wrote:
> At 04:59 PM 12/19/2005 +0100, Daniel Gl?ckner wrote:
> FYI, the shortest path to using MinGW with distutils or setuptools is to
> run this script and follow its directions:
>
> http://mail.python.org/pipermail/python-dev/2004-January/041676.html
>
> Once you've done that, the version of Python you ran it with will use the
> MinGW compiler automatically for all building, with no '-cmingw32' required.

Actually, the referenced script just prints instructions on modifying
distutils.cfg, it doesn't do it automatically.

> The script should probably be added to Python's "tools" directory,
> particularly for Windows.  I've just never gotten around to going through
> the necessary ceremony to get it in.

Probably not any more - see below.

> Martin v Loewis has also offered to change Python's Windows distribution a
> bit so that the script would be unnecessary, except for setting the [build]
> options in distutils.cfg, but I'm not sure if that's happened yet either,
> even for Python 2.5.  But the script will work for Python 2.3 and up, maybe
> even 2.2.

As of Python 2.4, this is no longer needed - libpython24.a is shipped
with the standard Python Windows installer (and yes, I mean all
versions of Python 2.4 - memo to self, must upgrade home PC...)

The distutils.cfg change is still needed if you want to make mingw the
default - personally, I just use the command line flag
--compiler=mingw32.

Paul.

From pje at telecommunity.com  Mon Dec 19 21:11:26 2005
From: pje at telecommunity.com (Phillip J. Eby)
Date: Mon, 19 Dec 2005 15:11:26 -0500
Subject: [Distutils] using distutils / setuptools on Windows (MinGW)
In-Reply-To: <79990c6b0512191201r33e4a3f4r240460e6ba71ea75@mail.gmail.co
 m>
References: <5.1.1.6.0.20051219131311.020a8fd8@mail.telecommunity.com>
	<43A6D8DE.2060501@ipsi.fraunhofer.de>
	<5.1.1.6.0.20051219131311.020a8fd8@mail.telecommunity.com>
Message-ID: <5.1.1.6.0.20051219150546.0209f3a0@mail.telecommunity.com>

At 08:01 PM 12/19/2005 +0000, Paul Moore wrote:
>On 12/19/05, Phillip J. Eby <pje at telecommunity.com> wrote:
> > At 04:59 PM 12/19/2005 +0100, Daniel Gl?ckner wrote:
> > FYI, the shortest path to using MinGW with distutils or setuptools is to
> > run this script and follow its directions:
> >
> > http://mail.python.org/pipermail/python-dev/2004-January/041676.html
> >
> > Once you've done that, the version of Python you ran it with will use the
> > MinGW compiler automatically for all building, with no '-cmingw32' 
> required.
>
>Actually, the referenced script just prints instructions on modifying
>distutils.cfg, it doesn't do it automatically.

That's why I said "run this script and follow its directions".  :)  The 
script builds libpython2X.a, which is the hard part for most earlier Python 
versions.


>As of Python 2.4, this is no longer needed - libpython24.a is shipped
>with the standard Python Windows installer (and yes, I mean all
>versions of Python 2.4 - memo to self, must upgrade home PC...)
>
>The distutils.cfg change is still needed if you want to make mingw the
>default - personally, I just use the command line flag
>--compiler=mingw32.

This can be abrreviated -cmingw32, of course, but if you use easy_install 
it's best to update the distutils.cfg, so that it will use that compiler by 
default.

Of course, if you have a setuptools-based project you can always do this once:

    setup.py build -cmingw32 saveopts -g

This will update distutils.cfg automatically after running the build, to 
save any options you used on the command line in the (-g) global 
configuration.  See the "saveopts" command in the setuptools manual for 
details.  (This is one of several setuptools commands that could probably 
be non-controversial additions to the official distutils: alias, rotate, 
saveopts, and setopt.)


From mal at egenix.com  Tue Dec 20 15:40:51 2005
From: mal at egenix.com (M.-A. Lemburg)
Date: Tue, 20 Dec 2005 15:40:51 +0100
Subject: [Distutils] bdist_nsis, bdist_deb, bdist_msi, bdist_inno,
 bdist_pkgtool, bdist_sdux, bdist_mpkg, etc.
In-Reply-To: <67071656-74BA-4CFF-BA9B-A5DA89A172FF@mac.com>
References: <439EAFD9.8060005@egenix.com>
	<439EB1AF.4040809@egenix.com>	<4F0360B5-9C0B-487E-BA8C-1210328F6C8C@redivi.com>	<439F1D45.9090509@egenix.com>	<BB8FCDBE-8F26-47E7-8D4F-42C7CE490FED@redivi.com>	<439F4035.5070305@egenix.com>
	<67071656-74BA-4CFF-BA9B-A5DA89A172FF@mac.com>
Message-ID: <43A817F3.4000807@egenix.com>

Ronald Oussoren wrote:
> 
> On 13-dec-2005, at 22:42, M.-A. Lemburg wrote:
> 
>> Bob Ippolito wrote:
>>> On Dec 13, 2005, at 11:13 AM, M.-A. Lemburg wrote:
>>>>>>> I'd like to focus attention on these distribution formats
>>>>>>> that distutils is missing:
>>>>>>>
>>>>>>
>>>>>> I'd like to another important candidate to the list:
>>>>>>
>>>>>> bdist_mpkg:
>>>>>>    Mac OS X installer format
>>>>>>    Part of py2app by Bob Ippolito
>>>>>>    http://undefined.org/python/py2app.html#bdist-mpkg-documentation
>>>>>
>>>>> bdist_mpkg is actually available on its own now, but I'm not sure
>>>>> adding
>>>>> it to distutils is a great idea.  The current version is already moved
>>>>> over to setuptools, and I'm pretty close to adding egg support to
>>>>> it.  I
>>>>> may even require that the Python bits of the installation package are
>>>>> eggs, as that would allow me to remove a bunch of code.  Eggs mostly
>>>>> obsolete what bdist_mpkg is for, but it's also used to install things
>>>>> like documentation and example code, so I'm not going to drop it
>>>>> altogether.
>>>>
>>>> Looks like now is a good time to add it to distutils then :-)
>>>
>>> No.  Definitely not.
>>> ...
>>> In other words, I don't consent to adding bdist_mpkg to Python until it
>>> supports eggs.
>>
>> Ok, then I'll remove the above candidate from the list.
>>
>> Anyone else interested in writing a bdist_mpkg command
>> that doesn't require setuptools ?
>>
>> Given the good documentation for the format, this shouldn't
>> be hard:
>>
>> http://s.sudre.free.fr/Stuff/PackageMaker_Howto.html
> 
> Should the volunteer also write the package management tools? One reason
> why
> I'm interested in setuptools is that OSX's 'Installer.app' is just that,
> a tool
> to install .pkg/.mpkg archives. You cannot uninstall packages and
> upgradeing
> seems to leave files around unless you manually remove them in a
> pre/post script.
> 
> OSX is great, but its installer application sucks.
> Setuptools/easy_install solves
> several problems for me:
> 1) uninstall
> 2) dependency management
> 3) automaticly downloading required software

I'm sure Apple will add support for uninstall soon. Even if not,
we can easily add an uninstall script to the created package which
then removes all traces of the installed package - much like
bdist_wininst does in order to support uninstall of the Windows
package.

Dependency management is a nice to have feature, but I
wouldn't consider it essential.

Automatic downloads are nice to have as well, but sys admins
rarely trust tools that automatically download and install
software, unless they have complete control of where things
are downloaded, when and whether there's proper authenticity
control built into the tool.

AFAIK, easy_install doesn't
provide any kind of authenticity control - probably due to
the fact that PyPI doesn't have this feature. At least the
MD5 sum should be checked, but for this, the PyPI registry
would have to provide this information in some way.

> And BTW the most common way to install software on OSX is by dragging an
> application
> icon from a CD or disk image to the folder containing applications ;-) ;-)

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Dec 20 2005)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! ::::

From mal at egenix.com  Tue Dec 20 15:48:28 2005
From: mal at egenix.com (M.-A. Lemburg)
Date: Tue, 20 Dec 2005 15:48:28 +0100
Subject: [Distutils] bdist_nsis, bdist_deb, bdist_msi, bdist_inno,
 bdist_pkgtool, bdist_sdux, bdist_mpkg, etc.
In-Reply-To: <43A817F3.4000807@egenix.com>
References: <439EAFD9.8060005@egenix.com>
	<439EB1AF.4040809@egenix.com>	<4F0360B5-9C0B-487E-BA8C-1210328F6C8C@redivi.com>	<439F1D45.9090509@egenix.com>	<BB8FCDBE-8F26-47E7-8D4F-42C7CE490FED@redivi.com>	<439F4035.5070305@egenix.com>
	<67071656-74BA-4CFF-BA9B-A5DA89A172FF@mac.com>
	<43A817F3.4000807@egenix.com>
Message-ID: <43A819BC.1050402@egenix.com>

M.-A. Lemburg wrote:
> Automatic downloads are nice to have as well, but sys admins
> rarely trust tools that automatically download and install
> software, unless they have complete control of where things
> are downloaded, when and whether there's proper authenticity
> control built into the tool.
> 
> AFAIK, easy_install doesn't
> provide any kind of authenticity control - probably due to
> the fact that PyPI doesn't have this feature. At least the
> MD5 sum should be checked, but for this, the PyPI registry
> would have to provide this information in some way.

I have to correct myself: Looks like PyPI has grown this feature
recently. Does easy_install check the MD5 sum ?

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Dec 20 2005)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! ::::

From ronaldoussoren at mac.com  Tue Dec 20 16:06:27 2005
From: ronaldoussoren at mac.com (Ronald Oussoren)
Date: Tue, 20 Dec 2005 16:06:27 +0100
Subject: [Distutils] bdist_nsis, bdist_deb, bdist_msi, bdist_inno,
	bdist_pkgtool, bdist_sdux, bdist_mpkg, etc.
In-Reply-To: <43A817F3.4000807@egenix.com>
References: <439EAFD9.8060005@egenix.com> <439EB1AF.4040809@egenix.com>
	<4F0360B5-9C0B-487E-BA8C-1210328F6C8C@redivi.com>
	<439F1D45.9090509@egenix.com>
	<BB8FCDBE-8F26-47E7-8D4F-42C7CE490FED@redivi.com>
	<439F4035.5070305@egenix.com>
	<67071656-74BA-4CFF-BA9B-A5DA89A172FF@mac.com>
	<43A817F3.4000807@egenix.com>
Message-ID: <F3D97137-7567-4A7D-887B-D5A6B8BAB950@mac.com>


On 20-dec-2005, at 15:40, M.-A. Lemburg wrote:
>
> I'm sure Apple will add support for uninstall soon. Even if not,
> we can easily add an uninstall script to the created package which
> then removes all traces of the installed package - much like
> bdist_wininst does in order to support uninstall of the Windows
> package.

You're much more optimistic than me w.r.t. Apple provided  
uninstallers. Apple's installer does offer no protection whatsoever  
against replacing files that belong to another package, I suppose at  
least partially because Apple itself uses installer packages to ship  
patches. This makes uninstalling a lot harder.

It should not be too hard to write a limited uninstaller for OSX,  
information about installed packages is stored in /Library/Receipts  
and that includes information about all installed files.
>
> Dependency management is a nice to have feature, but I
> wouldn't consider it essential.

I've lived without dependency management for ages, but am disliking  
it because I end up playing the dependency engine myself. It would be  
very nice if the standard distutils package could at least provide a  
way to store dependency information in setup.py, a PKG-INFO file and/ 
or on PyPI. That way users at least wouldn't have to search for  
dependencies in package documentation.

>
> Automatic downloads are nice to have as well, but sys admins
> rarely trust tools that automatically download and install
> software, unless they have complete control of where things
> are downloaded, when and whether there's proper authenticity
> control built into the tool.

I'm not too happy about the downloading feature of setuptools because  
of the lack of verification. I haven't spent enough time with  
setuptools yet to decide how to reach a save-enough situation. There  
will always be some trust involved, even if packagse included digital  
signatures I'd have to trust the signer enough to assume that he  
won't insert malware into his package.

>
> AFAIK, easy_install doesn't
> provide any kind of authenticity control - probably due to
> the fact that PyPI doesn't have this feature. At least the
> MD5 sum should be checked, but for this, the PyPI registry
> would have to provide this information in some way.

Checking checksums is very nice of cource, but doesn't help against  
someone that registers hostile code with PyPI. Luckily PyPI protects  
against users that try to register new versions of projects they  
don't own.

>
>> And BTW the most common way to install software on OSX is by  
>> dragging an
>> application
>> icon from a CD or disk image to the folder containing  
>> applications ;-) ;-)
>
> -- 
> Marc-Andre Lemburg
> eGenix.com
>
> Professional Python Services directly from the Source  (#1, Dec 20  
> 2005)
>>>> Python/Zope Consulting and Support ...        http:// 
>>>> www.egenix.com/
>>>> mxODBC.Zope.Database.Adapter ...             http:// 
>>>> zope.egenix.com/
>>>> mxODBC, mxDateTime, mxTextTools ...        http:// 
>>>> python.egenix.com/
> ______________________________________________________________________ 
> __
>
> ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for  
> free ! ::::


From pje at telecommunity.com  Tue Dec 20 16:41:30 2005
From: pje at telecommunity.com (Phillip J. Eby)
Date: Tue, 20 Dec 2005 10:41:30 -0500
Subject: [Distutils] bdist_nsis, bdist_deb, bdist_msi, bdist_inno,
 bdist_pkgtool, bdist_sdux, bdist_mpkg, etc.
In-Reply-To: <43A817F3.4000807@egenix.com>
References: <67071656-74BA-4CFF-BA9B-A5DA89A172FF@mac.com>
	<439EAFD9.8060005@egenix.com> <439EB1AF.4040809@egenix.com>
	<4F0360B5-9C0B-487E-BA8C-1210328F6C8C@redivi.com>
	<439F1D45.9090509@egenix.com>
	<BB8FCDBE-8F26-47E7-8D4F-42C7CE490FED@redivi.com>
	<439F4035.5070305@egenix.com>
	<67071656-74BA-4CFF-BA9B-A5DA89A172FF@mac.com>
Message-ID: <5.1.1.6.0.20051220102945.04423e40@mail.telecommunity.com>

At 03:40 PM 12/20/2005 +0100, M.-A. Lemburg wrote:
>AFAIK, easy_install doesn't
>provide any kind of authenticity control - probably due to
>the fact that PyPI doesn't have this feature. At least the
>MD5 sum should be checked, but for this, the PyPI registry
>would have to provide this information in some way.

EasyInstall verifies the MD5 sums for files downloaded from PyPI, and the 
--allow-hosts option allows you to restrict what hosts will be 
contacted.  It also allows you to create your own package lists with MD5 
sums (using "#md5=...." on the end of download URLs).  You can then 
restrict to hosts in your intranet, or only download from python.org, 
etc.  You can also just block downloading altogether (--allow-hosts=none) 
and just see what URLs it spits out as rejected, then go manually download 
things into a directory, and use --find-links=downloadsdir or just pass the 
filenames on the command line.  And since easy_install is a distutils 
command, you can configure all of those options in the standard distutils 
config files.

--allow-hosts is in the EasyInstall manual, but the MD5 bit isn't currently 
documented.  The manual should probably have a section on setting up your 
own package index(es) and access control in general, as right now you have 
to just dig through the options reference to find out about all of this 
stuff.  In 0.7 there will be a "nest" command that will offer the same 
functionality through a different interface, as right now the EasyInstall 
options are a combinatorial explosion of different things you can do.  Nest 
will offer separate commands for separate functions.  For example, to fetch 
a package's source into a subdirectory of the current directory, you would 
now do:

     easy_install -eb. somepackage

but in 0.7 there will be a "nest source" command:

     nest source somepackage

In other words, different use cases for the tools will have their own 
commands and their own documentation, rather than using a slew of options 
that interact with each other in non-obvious ways.


From theller at python.net  Tue Dec 20 19:14:01 2005
From: theller at python.net (Thomas Heller)
Date: Tue, 20 Dec 2005 19:14:01 +0100
Subject: [Distutils] bdist_nsis, bdist_deb, bdist_msi, bdist_inno,
 bdist_pkgtool, bdist_sdux, etc.
References: <439EAFD9.8060005@egenix.com> <u0ddkmm6.fsf@python.net>
	<439EFE58.7050504@egenix.com> <s9z4q5cq5g0.fsf@thetis.intevation.de>
Message-ID: <3bkntz4m.fsf@python.net>

Bernhard Herzog <bh at intevation.de> writes:

> "M.-A. Lemburg" <mal at egenix.com> writes:
>
>>> What do you have in mind - a bdist_wininst lookalike for
>>> Python packages, built with inno?
>>
>> Yes. One that creates an Inno setup file and then compiles
>> everything into an installer using the Inno, but installing
>> things much like bdist_wininst (and with the option of
>> building upon the many extra features you get from the
>> InnoSetup system).
>>
>> Bernhard Herzog started one some time ago:
>> http://mail.python.org/pipermail/distutils-sig/2001-September/002597.html
>
> FWIW, the code that's linked from that posting is still unchanged in
> Thuban and worked fine last time I made a release about a year ago with
> python 2.3.

>From the code linked in the posting above:

        # and invoke 
        if self.run_inno:
            self.spawn(["iscc", iss_file])

This code compiles the generated iss file, but it requires that ISCC is
on the PATH.  I guess that it would be better to use

  os.startfile(iss_file, "compile")

but this requires that os.startfile, which is implemented with
ShellExecute() on Windows, grows an optional second option.
Another example use case would be

  os.startfile("mydocs.pdf", "print")

If this would be accepted, I could upload/commit a patch for that.

Thomas


From mal at egenix.com  Tue Dec 20 19:52:08 2005
From: mal at egenix.com (M.-A. Lemburg)
Date: Tue, 20 Dec 2005 19:52:08 +0100
Subject: [Distutils] bdist_nsis, bdist_deb, bdist_msi, bdist_inno,
 bdist_pkgtool, bdist_sdux, bdist_mpkg, etc.
In-Reply-To: <F3D97137-7567-4A7D-887B-D5A6B8BAB950@mac.com>
References: <439EAFD9.8060005@egenix.com>
	<439EB1AF.4040809@egenix.com>	<4F0360B5-9C0B-487E-BA8C-1210328F6C8C@redivi.com>	<439F1D45.9090509@egenix.com>	<BB8FCDBE-8F26-47E7-8D4F-42C7CE490FED@redivi.com>	<439F4035.5070305@egenix.com>
	<67071656-74BA-4CFF-BA9B-A5DA89A172FF@mac.com>	<43A817F3.4000807@egenix.com>
	<F3D97137-7567-4A7D-887B-D5A6B8BAB950@mac.com>
Message-ID: <43A852D8.7080306@egenix.com>

Ronald Oussoren wrote:
> 
> On 20-dec-2005, at 15:40, M.-A. Lemburg wrote:
>>
>> I'm sure Apple will add support for uninstall soon. Even if not,
>> we can easily add an uninstall script to the created package which
>> then removes all traces of the installed package - much like
>> bdist_wininst does in order to support uninstall of the Windows
>> package.
> 
> You're much more optimistic than me w.r.t. Apple provided uninstallers.
> Apple's installer does offer no protection whatsoever against replacing
> files that belong to another package, I suppose at least partially
> because Apple itself uses installer packages to ship patches. This makes
> uninstalling a lot harder.

There's a tool called "Uninstaller" which seems to go a long
way to fill the gap:

	http://macmagna.free.fr/?l=en&s=uninstaller

> It should not be too hard to write a limited uninstaller for OSX,
> information about installed packages is stored in /Library/Receipts and
> that includes information about all installed files.

Indeed.

>> Dependency management is a nice to have feature, but I
>> wouldn't consider it essential.
> 
> I've lived without dependency management for ages, but am disliking it
> because I end up playing the dependency engine myself. It would be very
> nice if the standard distutils package could at least provide a way to
> store dependency information in setup.py, a PKG-INFO file and/or on
> PyPI. That way users at least wouldn't have to search for dependencies
> in package documentation.

SVN distutils already supports PEP 314 style
provides/requires/obsoletes. It also uses this information
for building RPMs and for registering with PyPI.

Looking at PyPI, these are not yet in wide-spread use, though.

>> Automatic downloads are nice to have as well, but sys admins
>> rarely trust tools that automatically download and install
>> software, unless they have complete control of where things
>> are downloaded, when and whether there's proper authenticity
>> control built into the tool.
> 
> I'm not too happy about the downloading feature of setuptools because of
> the lack of verification. I haven't spent enough time with setuptools
> yet to decide how to reach a save-enough situation. There will always be
> some trust involved, even if packagse included digital signatures I'd
> have to trust the signer enough to assume that he won't insert malware
> into his package.

I think an author provided MD5 sum will already go
a long way for making sure that the registered upload
has not been tampered with. The signature will
additionally confirm authorship, provided you trust
the signer.

The PyPI upload command can automatically sign packages
that you upload. However, the same problem as for PEP 314
applies to this I guess.

>> AFAIK, easy_install doesn't
>> provide any kind of authenticity control - probably due to
>> the fact that PyPI doesn't have this feature. At least the
>> MD5 sum should be checked, but for this, the PyPI registry
>> would have to provide this information in some way.
> 
> Checking checksums is very nice of cource, but doesn't help against
> someone that registers hostile code with PyPI. Luckily PyPI protects
> against users that try to register new versions of projects they don't own.

True. MD5 sums are the minimum requirement, I suppose.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Dec 20 2005)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! ::::

From bob at redivi.com  Tue Dec 20 22:09:57 2005
From: bob at redivi.com (Bob Ippolito)
Date: Tue, 20 Dec 2005 13:09:57 -0800 (PST)
Subject: [Distutils] bdist_nsis, bdist_deb, bdist_msi, bdist_inno,
 bdist_pkgtool, bdist_sdux, bdist_mpkg, etc.
In-Reply-To: <43A817F3.4000807@egenix.com>
References: <439EAFD9.8060005@egenix.com>
	<439EB1AF.4040809@egenix.com>	<4F0360B5-9C0B-487E-BA8C-1210328F6C8C@redivi.com>	<439F1D45.9090509@egenix.com>	<BB8FCDBE-8F26-47E7-8D4F-42C7CE490FED@redivi.com>	<439F4035.5070305@egenix.com>
	<67071656-74BA-4CFF-BA9B-A5DA89A172FF@mac.com>
	<43A817F3.4000807@egenix.com>
Message-ID: <33555.216.27.181.31.1135112997.squirrel@webmail.redivi.com>

> Ronald Oussoren wrote:
>>
>> On 13-dec-2005, at 22:42, M.-A. Lemburg wrote:
>>
>>> Bob Ippolito wrote:
>>>> On Dec 13, 2005, at 11:13 AM, M.-A. Lemburg wrote:
>>>>>>>> I'd like to focus attention on these distribution formats
>>>>>>>> that distutils is missing:
>>>>>>>>
>>>>>>>
>>>>>>> I'd like to another important candidate to the list:
>>>>>>>
>>>>>>> bdist_mpkg:
>>>>>>>    Mac OS X installer format
>>>>>>>    Part of py2app by Bob Ippolito
>>>>>>>    http://undefined.org/python/py2app.html#bdist-mpkg-documentation
>>>>>>
>>>>>> bdist_mpkg is actually available on its own now, but I'm not sure
>>>>>> adding
>>>>>> it to distutils is a great idea.  The current version is already
>>>>>> moved
>>>>>> over to setuptools, and I'm pretty close to adding egg support to
>>>>>> it.  I
>>>>>> may even require that the Python bits of the installation package
>>>>>> are
>>>>>> eggs, as that would allow me to remove a bunch of code.  Eggs mostly
>>>>>> obsolete what bdist_mpkg is for, but it's also used to install
>>>>>> things
>>>>>> like documentation and example code, so I'm not going to drop it
>>>>>> altogether.
>>>>>
>>>>> Looks like now is a good time to add it to distutils then :-)
>>>>
>>>> No.  Definitely not.
>>>> ...
>>>> In other words, I don't consent to adding bdist_mpkg to Python until
>>>> it
>>>> supports eggs.
>>>
>>> Ok, then I'll remove the above candidate from the list.
>>>
>>> Anyone else interested in writing a bdist_mpkg command
>>> that doesn't require setuptools ?
>>>
>>> Given the good documentation for the format, this shouldn't
>>> be hard:
>>>
>>> http://s.sudre.free.fr/Stuff/PackageMaker_Howto.html
>>
>> Should the volunteer also write the package management tools? One reason
>> why
>> I'm interested in setuptools is that OSX's 'Installer.app' is just that,
>> a tool
>> to install .pkg/.mpkg archives. You cannot uninstall packages and
>> upgradeing
>> seems to leave files around unless you manually remove them in a
>> pre/post script.
>>
>> OSX is great, but its installer application sucks.
>> Setuptools/easy_install solves
>> several problems for me:
>> 1) uninstall
>> 2) dependency management
>> 3) automaticly downloading required software
>
> I'm sure Apple will add support for uninstall soon. Even if not,
> we can easily add an uninstall script to the created package which
> then removes all traces of the installed package - much like
> bdist_wininst does in order to support uninstall of the Windows
> package.

I thought they'd add support for uninstall soon too... 5 years ago.  I
gave up.  The installer portion of OS X is not important to Apple because
the majority of users will only see it when bootstrapping the OS and
installing a select few applications.  The majority of other things are
either self-contained applications or built with InstallerVISE.

Uninstall scripts just aren't commonplace for Mac OS X, even for things
that have installer packages.  If we're going to come up with a new idiom
we might as well pick one that works, and that's easy_install.

> Dependency management is a nice to have feature, but I
> wouldn't consider it essential.

I disagree, and so do plenty of other users.  I've dealt with this
platform and Python for several years, and have been providing packages
for users and writing the software to make it possible.  Without
dependency management, even double-click installation is daunting since
you have to choose several different packages in the right order and find
the right package for your Python and Mac OS X version.

setuptools not only knows which order to install things (for supporting
pacakges) but it'll go ahead and find it all for you and install it as
well.

Mac OS X users are used to easy.  Python is not currently easy on the Mac
unless you're using DarwinPorts or Fink, but those package management
systems have their own problems.  easy_install fixes almost everything for
the Python on Mac OS X situation.

> Automatic downloads are nice to have as well, but sys admins
> rarely trust tools that automatically download and install
> software, unless they have complete control of where things
> are downloaded, when and whether there's proper authenticity
> control built into the tool.

Mac OS X users are very rarely sysadmins, and easy_install supports that
use case too.  You can tell it not to download anything, or you can tell
it where to download things from (even just a local path).

> AFAIK, easy_install doesn't
> provide any kind of authenticity control - probably due to
> the fact that PyPI doesn't have this feature. At least the
> MD5 sum should be checked, but for this, the PyPI registry
> would have to provide this information in some way.

The PyPI registry provides for GPG signatures and MD5, and I'm relatively
certain that easy_install does check MD5.  The ez_setup bootstrap
definitely checks the md5 for known versions of setuptools, and I'm
assuming that a similar feature is in easy_install for normal packages.

-bob



From pje at telecommunity.com  Tue Dec 20 23:08:08 2005
From: pje at telecommunity.com (Phillip J. Eby)
Date: Tue, 20 Dec 2005 17:08:08 -0500
Subject: [Distutils] bdist_nsis, bdist_deb, bdist_msi, bdist_inno,
 bdist_pkgtool, bdist_sdux, bdist_mpkg, etc.
In-Reply-To: <33555.216.27.181.31.1135112997.squirrel@webmail.redivi.com
 >
References: <43A817F3.4000807@egenix.com> <439EAFD9.8060005@egenix.com>
	<439EB1AF.4040809@egenix.com>
	<4F0360B5-9C0B-487E-BA8C-1210328F6C8C@redivi.com>
	<439F1D45.9090509@egenix.com>
	<BB8FCDBE-8F26-47E7-8D4F-42C7CE490FED@redivi.com>
	<439F4035.5070305@egenix.com>
	<67071656-74BA-4CFF-BA9B-A5DA89A172FF@mac.com>
	<43A817F3.4000807@egenix.com>
Message-ID: <5.1.1.6.0.20051220170516.02094bb8@mail.telecommunity.com>

At 01:09 PM 12/20/2005 -0800, Bob Ippolito wrote:
>The ez_setup bootstrap
>definitely checks the md5 for known versions of setuptools, and I'm
>assuming that a similar feature is in easy_install for normal packages.

No need to assume; see the 'check_md5' method of the PackageIndex class in 
setuptools.package_index.  ;)

*Any* URL processed by PackageIndex that ends with a "#md5=" anchor will 
have its md5 verified upon download.  If the match fails, the downloaded 
file is immediately deleted and an error is raised.

(In addition, PyPI pages' MD5 sums for downloads are preprocessed into the 
"#md5=" format using a regular expression search.)


From rasky at develer.com  Wed Dec 21 15:33:14 2005
From: rasky at develer.com (Giovanni Bajo)
Date: Wed, 21 Dec 2005 15:33:14 +0100
Subject: [Distutils] Generation of script during installation
Message-ID: <02a301c6063b$79ee1100$bf03030a@trilan>

Hello,

I have a problem with distutils. During installation, I need to generate a
Python module containing information about the target installation paths.
For instance, if an user does:

setup.py install --install-headers=/foo

my module will have to contain something along the lines of:

headers= "/foo"

I managed to do this by subclassing install_lib (overriding method run) and
generating the module there. I access the target paths with things like
self.get_finalized_command("install_headers").install_dir. The module is
created into the build_dir, so that install_lib.run() then copy it together
with the other modules at the correct location.

This works like a charm *but* breaks when doing a bdist_wininst (and
probably a bdist_rpm, I haven't checked). I'm not sure how to deal with
that, as the final paths will be decided at installation time (depending on
the user's system), and that's outside of the scope of my setup.py script.

Suggestions? For instance, is there a way so that the Windows installer
generate an installation log, which my Python script can parse to extract
the directory information?
-- 
Giovanni Bajo


From pje at telecommunity.com  Wed Dec 21 18:20:23 2005
From: pje at telecommunity.com (Phillip J. Eby)
Date: Wed, 21 Dec 2005 12:20:23 -0500
Subject: [Distutils] Generation of script during installation
In-Reply-To: <02a301c6063b$79ee1100$bf03030a@trilan>
Message-ID: <5.1.1.6.0.20051221121927.020acce8@mail.telecommunity.com>

At 03:33 PM 12/21/2005 +0100, Giovanni Bajo wrote:
>Suggestions?

You might want to just include all your data and/or header files inside 
your package directory; this is the one directory you can always find at 
runtime, no matter how your package gets installed.


From ianb at colorstudy.com  Wed Dec 21 18:30:40 2005
From: ianb at colorstudy.com (Ian Bicking)
Date: Wed, 21 Dec 2005 11:30:40 -0600
Subject: [Distutils] default installs and plugins
Message-ID: <43A99140.7080900@colorstudy.com>

I'm still finding it impossible to use multiple versions of a package 
unless none of them show up in a .pth file (i.e., none are available 
without requiring), I get a VersionConflict.  I think this may be 
because I iterate over entry points early on, then do a require later.

Conversely, if I don't have a default version installed, I don't see any 
entry points while iterating until the package is required.  I can kind 
of resolve this by adding a --require option to any command that uses 
plugins, but this feels like I'm going down the path of CLASSPATH-like 
command invocation.

I'm not even sure what the resolution would or can be, except that it 
means that I don't see any way to reasonably use versioning and plugins 
together.  This worries me, since I have lots of code that is both 
package and plugin, and if this is the situation then I'll have to move 
to non-root installs everywhere I want version isolation.  (Which I 
actually suspect I'll have to do anyway, but I was hoping that things 
would at least be usable without it).

-- 
Ian Bicking  /  ianb at colorstudy.com  /  http://blog.ianbicking.org

From pje at telecommunity.com  Wed Dec 21 19:09:34 2005
From: pje at telecommunity.com (Phillip J. Eby)
Date: Wed, 21 Dec 2005 13:09:34 -0500
Subject: [Distutils] default installs and plugins
In-Reply-To: <43A99140.7080900@colorstudy.com>
Message-ID: <5.1.1.6.0.20051221130104.02786800@mail.telecommunity.com>

At 11:30 AM 12/21/2005 -0600, Ian Bicking wrote:
>I'm still finding it impossible to use multiple versions of a package
>unless none of them show up in a .pth file (i.e., none are available
>without requiring), I get a VersionConflict.  I think this may be
>because I iterate over entry points early on, then do a require later.

No, it's because the initial working set is fixed once pkg_resources is 
imported.  I'm contemplating ways to make the working set more fluid in 
0.7, so that which version of something you're using can change until 
you've imported it, searched for it, or require()d it.  Right now, though, 
anything that's the default version gets locked in as the available version 
as soon as pkg_resources is imported.  The only way around it is to have a 
__requires__ specified in __main__, which is what easy_install-generated 
wrapper scripts do.  So, if a particular package version is explicitly 
required by a script, then it can override the default version.  But 
currently that is the *only* way to override a default version and it has 
to be done from __main__ *before* pkg_resources is imported.

I definitely want to change this in 0.7, but am waiting until then because 
it's a potentially destabilizing change and I want a stable (ha!) 0.6a9 or 
a10 out there for people to use while 0.7a1 is in development.


>Conversely, if I don't have a default version installed, I don't see any
>entry points while iterating until the package is required.  I can kind
>of resolve this by adding a --require option to any command that uses
>plugins, but this feels like I'm going down the path of CLASSPATH-like
>command invocation.
>
>I'm not even sure what the resolution would or can be, except that it
>means that I don't see any way to reasonably use versioning and plugins
>together.

That's only possible/practical in the case where you have a specific 
directory for plugins; i.e. a pluggable application with possibly multiple 
instances.  For example, Zope and Trac do this, although I don't believe 
either has specific support for multiple versions.

In situations using more global tools, it's necessary to have plugins be 
global, which in turn implies a particular version of the plugin.


>   This worries me, since I have lots of code that is both
>package and plugin, and if this is the situation then I'll have to move
>to non-root installs everywhere I want version isolation.  (Which I
>actually suspect I'll have to do anyway, but I was hoping that things
>would at least be usable without it).

There are a number other custom installation strategies available besides 
the "virtual python" one, now.  You might check those out on the 
EasyInstall page.


From rasky at develer.com  Wed Dec 21 19:29:55 2005
From: rasky at develer.com (Giovanni Bajo)
Date: Wed, 21 Dec 2005 19:29:55 +0100
Subject: [Distutils] Generation of script during installation
References: <5.1.1.6.0.20051221121927.020acce8@mail.telecommunity.com>
Message-ID: <00d401c6065c$8a984270$bf03030a@trilan>

Phillip J. Eby <pje at telecommunity.com> wrote:

>> Suggestions?
>
> You might want to just include all your data and/or header files inside
> your package directory; this is the one directory you can always find at
> runtime, no matter how your package gets installed.

I'm not sure I understand. Are you suggesting to ignore/override what the
user specifies with --install-headers or similar options?
-- 
Giovanni Bajo


From pje at telecommunity.com  Wed Dec 21 19:47:05 2005
From: pje at telecommunity.com (Phillip J. Eby)
Date: Wed, 21 Dec 2005 13:47:05 -0500
Subject: [Distutils] Generation of script during installation
In-Reply-To: <00d401c6065c$8a984270$bf03030a@trilan>
References: <5.1.1.6.0.20051221121927.020acce8@mail.telecommunity.com>
Message-ID: <5.1.1.6.0.20051221134545.0279c610@mail.telecommunity.com>

At 07:29 PM 12/21/2005 +0100, Giovanni Bajo wrote:
>Phillip J. Eby <pje at telecommunity.com> wrote:
>
> >> Suggestions?
> >
> > You might want to just include all your data and/or header files inside
> > your package directory; this is the one directory you can always find at
> > runtime, no matter how your package gets installed.
>
>I'm not sure I understand. Are you suggesting to ignore/override what the
>user specifies with --install-headers or similar options?

I'm suggesting that if your package needs to have a reliable location for 
the data, then it should include it with the package.  This is orthogonal 
to whether you *also* install the headers in a user-specified 
location.  I'm just saying that if you have the data in your package, you 
no longer have to care where the user may have *also* put it.


From ianb at colorstudy.com  Wed Dec 21 22:05:33 2005
From: ianb at colorstudy.com (Ian Bicking)
Date: Wed, 21 Dec 2005 15:05:33 -0600
Subject: [Distutils] default installs and plugins
In-Reply-To: <5.1.1.6.0.20051221130104.02786800@mail.telecommunity.com>
References: <5.1.1.6.0.20051221130104.02786800@mail.telecommunity.com>
Message-ID: <43A9C39D.8050405@colorstudy.com>

Relatedly -- I'm realizing that it would be *really* useful when dealing 
with some of these conflicts to have logging in setuptools.  Then I'd 
have a much better idea of what is happening when, and exactly what 
versions of code I'm working with.  Without that, I have a hard time 
actually debugging stuff without putting prints into pkg_resources.

-- 
Ian Bicking  /  ianb at colorstudy.com  /  http://blog.ianbicking.org

From slash at dotnetslash.net  Wed Dec 21 23:54:48 2005
From: slash at dotnetslash.net (Mark W. Alexander)
Date: Wed, 21 Dec 2005 17:54:48 -0500
Subject: [Distutils] bdist_nsis, bdist_deb, bdist_msi, bdist_inno,
	bdist_pkgtool, bdist_sdux, etc.
In-Reply-To: <439EAFD9.8060005@egenix.com>
References: <439EAFD9.8060005@egenix.com>
Message-ID: <20051221225448.GA18559@dotnetslash.net>

On Tue, Dec 13, 2005 at 12:26:17PM +0100, M.-A. Lemburg wrote:
> Hi,
> 
> I'd like to focus attention on these distribution formats
> that distutils is missing:

[snip]

> bdist_pkgtool:
>   Package format used on Solaris
>   There was a code contribution by Mark Alexander for this, but it had
> to be removed due to licensing issues:
> 
> http://cvs.sourceforge.net/viewcvs.py/python/python/dist/src/Lib/distutils/command/Attic/
> 
> bdist_sdux:
>   Package format used on HP-UX
>   Same issues as with bdist_pkgtool.

While I'm now employed where I could probably re-implement those without any
licensing issues, I no longer need them nor do I have any HP or Sun boxes to
beat about with them. I'd be willing to help if someone else's interested, but
the previous implementation was left with my previous employer.

I agree that Distutils support for these is key to broader acceptance of
Python. As a former-proprietary-enterprise-sysadmin, having something that
integrates with the native package manager beats the CPAN approach hands down
for sw inventory and configuration management.

mwa
-- 
Mark W. Alexander
slash at dotnetslash.net

The contents of this message authored by Mark W. Alexander are released under
the Creative Commons Attribution-NonCommercial license.  Copyright of quoted
materials, if any, are retained by the original author(s).

http://creativecommons.org/licenses/by-nc/2.0/

From rasky at develer.com  Thu Dec 22 00:35:29 2005
From: rasky at develer.com (Giovanni Bajo)
Date: Thu, 22 Dec 2005 00:35:29 +0100
Subject: [Distutils] Generation of script during installation
References: <5.1.1.6.0.20051221121927.020acce8@mail.telecommunity.com>
	<5.1.1.6.0.20051221134545.0279c610@mail.telecommunity.com>
Message-ID: <067e01c60687$3a3f0cc0$e14f2a97@bagio>

Phillip J. Eby <pje at telecommunity.com> wrote:

>>> You might want to just include all your data and/or header files
>>> inside your package directory; this is the one directory you can
>>> always find at runtime, no matter how your package gets installed.
>>
>> I'm not sure I understand. Are you suggesting to ignore/override
>> what the user specifies with --install-headers or similar options?
>
> I'm suggesting that if your package needs to have a reliable location
> for the data, then it should include it with the package.  This is
> orthogonal to whether you *also* install the headers in a
> user-specified
> location.  I'm just saying that if you have the data in your package,
> you no longer have to care where the user may have *also* put it.


OK, so you're suggesting to install two copies of the same files: one in a
fixed location (within site-packages) so that it's always accessible, and a
copy for the user wherever he wants. This might work for my case, but how do I
implement it? Do I need to override most of the install_* cmd to implement this
logic, or there is some trick I can exploit?

Thanks!

Giovanni Bajo


From bob at redivi.com  Thu Dec 22 01:06:00 2005
From: bob at redivi.com (Bob Ippolito)
Date: Wed, 21 Dec 2005 16:06:00 -0800 (PST)
Subject: [Distutils] Generation of script during installation
In-Reply-To: <067e01c60687$3a3f0cc0$e14f2a97@bagio>
References: <5.1.1.6.0.20051221121927.020acce8@mail.telecommunity.com>
	<5.1.1.6.0.20051221134545.0279c610@mail.telecommunity.com>
	<067e01c60687$3a3f0cc0$e14f2a97@bagio>
Message-ID: <33753.216.27.181.31.1135209960.squirrel@webmail.redivi.com>

> Phillip J. Eby <pje at telecommunity.com> wrote:
>
>>>> You might want to just include all your data and/or header files
>>>> inside your package directory; this is the one directory you can
>>>> always find at runtime, no matter how your package gets installed.
>>>
>>> I'm not sure I understand. Are you suggesting to ignore/override
>>> what the user specifies with --install-headers or similar options?
>>
>> I'm suggesting that if your package needs to have a reliable location
>> for the data, then it should include it with the package.  This is
>> orthogonal to whether you *also* install the headers in a
>> user-specified
>> location.  I'm just saying that if you have the data in your package,
>> you no longer have to care where the user may have *also* put it.
>
>
> OK, so you're suggesting to install two copies of the same files: one in a
> fixed location (within site-packages) so that it's always accessible, and
> a
> copy for the user wherever he wants. This might work for my case, but how
> do I
> implement it? Do I need to override most of the install_* cmd to implement
> this
> logic, or there is some trick I can exploit?

Why not just include the header files in both install_headers and
install_data (or equivalent)?

-bob



From pje at telecommunity.com  Thu Dec 22 01:56:46 2005
From: pje at telecommunity.com (Phillip J. Eby)
Date: Wed, 21 Dec 2005 19:56:46 -0500
Subject: [Distutils] Generation of script during installation
In-Reply-To: <33753.216.27.181.31.1135209960.squirrel@webmail.redivi.com
 >
References: <067e01c60687$3a3f0cc0$e14f2a97@bagio>
	<5.1.1.6.0.20051221121927.020acce8@mail.telecommunity.com>
	<5.1.1.6.0.20051221134545.0279c610@mail.telecommunity.com>
	<067e01c60687$3a3f0cc0$e14f2a97@bagio>
Message-ID: <5.1.1.6.0.20051221195358.020b3c58@mail.telecommunity.com>

At 04:06 PM 12/21/2005 -0800, Bob Ippolito wrote:
> > OK, so you're suggesting to install two copies of the same files: one in a
> > fixed location (within site-packages) so that it's always accessible, and
> > a
> > copy for the user wherever he wants. This might work for my case, but how
> > do I
> > implement it? Do I need to override most of the install_* cmd to implement
> > this
> > logic, or there is some trick I can exploit?
>
>Why not just include the header files in both install_headers and
>install_data (or equivalent)?

Well, if you're using Python 2.4 (or Python 2.3 with setuptools), just put 
the header files inside your package directories and use:

     package_data = {'': ['*.h']},
     headers = ['srcdir/mypkg/foo.h', 'srcdir/mypkg/bar.h', ...],

to include all .h files that are found in your package directories, and to 
also install the same files as headers.


From mal at egenix.com  Thu Dec 22 11:57:47 2005
From: mal at egenix.com (M.-A. Lemburg)
Date: Thu, 22 Dec 2005 11:57:47 +0100
Subject: [Distutils] bdist_nsis, bdist_deb, bdist_msi, bdist_inno,
 bdist_pkgtool, bdist_sdux, etc.
In-Reply-To: <3bkntz4m.fsf@python.net>
References: <439EAFD9.8060005@egenix.com>
	<u0ddkmm6.fsf@python.net>	<439EFE58.7050504@egenix.com>
	<s9z4q5cq5g0.fsf@thetis.intevation.de> <3bkntz4m.fsf@python.net>
Message-ID: <43AA86AB.8050701@egenix.com>

Thomas Heller wrote:
> Bernhard Herzog <bh at intevation.de> writes:
> 
>> "M.-A. Lemburg" <mal at egenix.com> writes:
>>
>>>> What do you have in mind - a bdist_wininst lookalike for
>>>> Python packages, built with inno?
>>> Yes. One that creates an Inno setup file and then compiles
>>> everything into an installer using the Inno, but installing
>>> things much like bdist_wininst (and with the option of
>>> building upon the many extra features you get from the
>>> InnoSetup system).
>>>
>>> Bernhard Herzog started one some time ago:
>>> http://mail.python.org/pipermail/distutils-sig/2001-September/002597.html
>> FWIW, the code that's linked from that posting is still unchanged in
>> Thuban and worked fine last time I made a release about a year ago with
>> python 2.3.
> 
>>From the code linked in the posting above:
> 
>         # and invoke 
>         if self.run_inno:
>             self.spawn(["iscc", iss_file])
> 
> This code compiles the generated iss file, but it requires that ISCC is
> on the PATH.  I guess that it would be better to use
> 
>   os.startfile(iss_file, "compile")
> 
> but this requires that os.startfile, which is implemented with
> ShellExecute() on Windows, grows an optional second option.
> Another example use case would be
> 
>   os.startfile("mydocs.pdf", "print")
> 
> If this would be accepted, I could upload/commit a patch for that.

Sounds like a good idea.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Dec 22 2005)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! ::::

From mal at egenix.com  Thu Dec 22 12:24:28 2005
From: mal at egenix.com (M.-A. Lemburg)
Date: Thu, 22 Dec 2005 12:24:28 +0100
Subject: [Distutils] Generation of script during installation
In-Reply-To: <02a301c6063b$79ee1100$bf03030a@trilan>
References: <02a301c6063b$79ee1100$bf03030a@trilan>
Message-ID: <43AA8CEC.9030806@egenix.com>

Giovanni Bajo wrote:
> Hello,
> 
> I have a problem with distutils. During installation, I need to generate a
> Python module containing information about the target installation paths.
> For instance, if an user does:
> 
> setup.py install --install-headers=/foo
> 
> my module will have to contain something along the lines of:
> 
> headers= "/foo"
> 
> I managed to do this by subclassing install_lib (overriding method run) and
> generating the module there. I access the target paths with things like
> self.get_finalized_command("install_headers").install_dir. The module is
> created into the build_dir, so that install_lib.run() then copy it together
> with the other modules at the correct location.
> 
> This works like a charm *but* breaks when doing a bdist_wininst (and
> probably a bdist_rpm, I haven't checked). I'm not sure how to deal with
> that, as the final paths will be decided at installation time (depending on
> the user's system), and that's outside of the scope of my setup.py script.
> 
> Suggestions? For instance, is there a way so that the Windows installer
> generate an installation log, which my Python script can parse to extract
> the directory information?

wininst does create an installation log (called
<packagename>-wininst.log) which is placed into the
Python directory along with an uninstall .exe which
takes care of removing the package if uninstall is
selected in the Windows software administration tool.

However, I'm not sure whether you really need this,
since only the Python installation directory is
selectable during installation with wininst.

Note that it's easy to find the installation directory
of an installed package by importing it and looking
at the package.__path__ or .__file__.

For ZIP package imports, these two attributes include
the location of the ZIP file, e.g.
/home/lemburg/tmp/mxMisc.zip/mx/Misc

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Dec 22 2005)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! ::::

From mal at egenix.com  Thu Dec 22 12:36:12 2005
From: mal at egenix.com (M.-A. Lemburg)
Date: Thu, 22 Dec 2005 12:36:12 +0100
Subject: [Distutils] Multiple version support (default installs and
	plugins)
In-Reply-To: <43A99140.7080900@colorstudy.com>
References: <43A99140.7080900@colorstudy.com>
Message-ID: <43AA8FAC.1060808@egenix.com>

Ian Bicking wrote:
> I'm still finding it impossible to use multiple versions of a package 
> unless none of them show up in a .pth file (i.e., none are available 
> without requiring), I get a VersionConflict. 

This comes up every now and then when discussing the benefits
of easy_install.

I'd really like to understand what the use case is for having
multiple versions of a package around.

Note that sys.modules (the registry of loaded modules) does
not support having multiple versions of a module loaded.

Even if you do trick Python into loading a module twice,
either by mistake or on purpose, you end up with only
one being referenced in sys.modules.

Having a module loaded twice (the same or different versions)
will cause lots of very subtle problems with various mechanisms
in Python, e.g. Python pickles may get loaded using the wrong
module, classes may reference the wrong module (causing
isinstance() or issubclass() tests to fail without apparent
reason), initialization code may have been applied twice
which could lead to duplication (e.g. if you append data
to a list during initialization), etc.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Dec 22 2005)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! ::::

From mal at egenix.com  Thu Dec 22 12:48:18 2005
From: mal at egenix.com (M.-A. Lemburg)
Date: Thu, 22 Dec 2005 12:48:18 +0100
Subject: [Distutils] Egg support for system packages
	(including	bdist_wininst)
In-Reply-To: <5.1.1.6.0.20051214184924.03033a28@mail.telecommunity.com>
References: <5.1.1.6.0.20051214184924.03033a28@mail.telecommunity.com>
Message-ID: <43AA9282.5040806@egenix.com>

Phillip J. Eby wrote:
> Setuptools in SVN now provides preliminary support for installing eggs in 
> .egg-info directory style, so that setuptools-based projects can be wrapped 
> by system packagers who wish to avoid using .egg files or directories.  In 
> addition, you can now use setuptools to install non-setuptools packages in 
> such a way that an .egg-info directory is included, so that 
> setuptools-based packages can detect the presence of the 
> non-setuptools-based package.

Thanks for adding this.

I'd just wish that this would be the default and the .egg ZIP
file installation approach be made an option.

setuptools would then finally be compatible to the rest of the
distutils world again and avoid all the added overhead and
problems of ZIP file imports.

Perhaps you could have two commands, e.g. the default install
would create the normal package directory (with added .egg-info
dir) and a new install_eggfile to install the .egg ZIP file
instead.

Thanks,
-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Dec 22 2005)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! ::::

From mal at egenix.com  Thu Dec 22 13:52:15 2005
From: mal at egenix.com (M.-A. Lemburg)
Date: Thu, 22 Dec 2005 13:52:15 +0100
Subject: [Distutils] Egg support for system
	packages	(including	bdist_wininst)
In-Reply-To: <43AA9282.5040806@egenix.com>
References: <5.1.1.6.0.20051214184924.03033a28@mail.telecommunity.com>
	<43AA9282.5040806@egenix.com>
Message-ID: <43AAA17F.5040203@egenix.com>

M.-A. Lemburg wrote:
> Phillip J. Eby wrote:
>> Setuptools in SVN now provides preliminary support for installing eggs in 
>> .egg-info directory style, so that setuptools-based projects can be wrapped 
>> by system packagers who wish to avoid using .egg files or directories.  In 
>> addition, you can now use setuptools to install non-setuptools packages in 
>> such a way that an .egg-info directory is included, so that 
>> setuptools-based packages can detect the presence of the 
>> non-setuptools-based package.
> 
> Thanks for adding this.
> 
> I'd just wish that this would be the default and the .egg ZIP
> file installation approach be made an option.
> 
> setuptools would then finally be compatible to the rest of the
> distutils world again and avoid all the added overhead and
> problems of ZIP file imports.
> 
> Perhaps you could have two commands, e.g. the default install
> would create the normal package directory (with added .egg-info
> dir) and a new install_eggfile to install the .egg ZIP file
> instead.

Somewhat related to this: wouldn't it be possible to build
.egg ZIP files using the same layout, ie. with the .egg-info
subdir inside the package instead of having a top-level
EGG-INFO directory in the ZIP file ?!

That way, you could simply unzip the .egg file somewhere
on your PYTHONPATH in order to install it and still have
the meta information at hand.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Dec 22 2005)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! ::::

From p.f.moore at gmail.com  Thu Dec 22 15:13:24 2005
From: p.f.moore at gmail.com (Paul Moore)
Date: Thu, 22 Dec 2005 14:13:24 +0000
Subject: [Distutils] Multiple version support (default installs and
	plugins)
In-Reply-To: <43AA8FAC.1060808@egenix.com>
References: <43A99140.7080900@colorstudy.com> <43AA8FAC.1060808@egenix.com>
Message-ID: <79990c6b0512220613n5e494b5fp27e66c44d3ae6d48@mail.gmail.com>

On 12/22/05, M.-A. Lemburg <mal at egenix.com> wrote:
> Ian Bicking wrote:
> > I'm still finding it impossible to use multiple versions of a package
> > unless none of them show up in a .pth file (i.e., none are available
> > without requiring), I get a VersionConflict.
>
> This comes up every now and then when discussing the benefits
> of easy_install.
>
> I'd really like to understand what the use case is for having
> multiple versions of a package around.
>
> Note that sys.modules (the registry of loaded modules) does
> not support having multiple versions of a module loaded.

I think the point is to allow multiple versions to be installed on the
system, but allow an individual program, at run time, to specify which
one is to be loaded. There will never be more than one version loaded
in any specific instance, but if program A requires version 1 of
package X, and program B requires version 2 of package X, both can run
without needing a reinstall of package X in between.

Of course, this is also possible via PYTHONPATH manipulations, or
sys.path modifications in the program before doing the import. All
setuptools is doing is to provide a common infrastructure for handling
this.

Paul.

From pje at telecommunity.com  Thu Dec 22 16:26:52 2005
From: pje at telecommunity.com (Phillip J. Eby)
Date: Thu, 22 Dec 2005 10:26:52 -0500
Subject: [Distutils] Multiple version support (default installs and
 plugins)
In-Reply-To: <79990c6b0512220613n5e494b5fp27e66c44d3ae6d48@mail.gmail.co
 m>
References: <43AA8FAC.1060808@egenix.com> <43A99140.7080900@colorstudy.com>
	<43AA8FAC.1060808@egenix.com>
Message-ID: <5.1.1.6.0.20051222102351.01b07cd8@mail.telecommunity.com>

At 02:13 PM 12/22/2005 +0000, Paul Moore wrote:
>On 12/22/05, M.-A. Lemburg <mal at egenix.com> wrote:
> > Ian Bicking wrote:
> > > I'm still finding it impossible to use multiple versions of a package
> > > unless none of them show up in a .pth file (i.e., none are available
> > > without requiring), I get a VersionConflict.
> >
> > This comes up every now and then when discussing the benefits
> > of easy_install.
> >
> > I'd really like to understand what the use case is for having
> > multiple versions of a package around.
> >
> > Note that sys.modules (the registry of loaded modules) does
> > not support having multiple versions of a module loaded.
>
>I think the point is to allow multiple versions to be installed on the
>system, but allow an individual program, at run time, to specify which
>one is to be loaded. There will never be more than one version loaded
>in any specific instance, but if program A requires version 1 of
>package X, and program B requires version 2 of package X, both can run
>without needing a reinstall of package X in between.
>
>Of course, this is also possible via PYTHONPATH manipulations, or
>sys.path modifications in the program before doing the import. All
>setuptools is doing is to provide a common infrastructure for handling
>this.

Yes, Paul is correct; supporting multiple programs that use different 
versions of a module is indeed the primary use case.  Also, during 
development and debugging it can be quite convenient to switch versions of 
a dependency back and forth to identify the source of a problem.


From pje at telecommunity.com  Thu Dec 22 16:42:41 2005
From: pje at telecommunity.com (Phillip J. Eby)
Date: Thu, 22 Dec 2005 10:42:41 -0500
Subject: [Distutils] Egg support for system packages
 (including	bdist_wininst)
In-Reply-To: <43AA9282.5040806@egenix.com>
References: <5.1.1.6.0.20051214184924.03033a28@mail.telecommunity.com>
	<5.1.1.6.0.20051214184924.03033a28@mail.telecommunity.com>
Message-ID: <5.1.1.6.0.20051222102709.04406a20@mail.telecommunity.com>

At 12:48 PM 12/22/2005 +0100, M.-A. Lemburg wrote:
>I'd just wish that this would be the default and the .egg ZIP
>file installation approach be made an option.
>
>setuptools would then finally be compatible to the rest of the
>distutils world again and avoid all the added overhead and
>problems of ZIP file imports.
>
>Perhaps you could have two commands, e.g. the default install
>would create the normal package directory (with added .egg-info
>dir) and a new install_eggfile to install the .egg ZIP file
>instead.

The only way this could happen is to add a manifest of installed files, in 
order to allow uninstallation and upgrades to be made safe in the absence 
of a packaging system.  The .egg file/directory technique currently makes 
uninstallation as simple as "rm -rf package-version.egg", and multi-version 
installs work with no special action.

Note, too, that using .egg files or directories means that installation 
*never breaks existing programs* as long as they have specified their 
dependencies accurately.  Traditional distutils installation is destructive 
in the absence of a system packager.

However, if you *have* a system packager, then you should not be using 
"setup.py install" anyway, so I'm not sure I see how this is an issue.  You 
should be simply building system packages.  And it's quite straightforward 
to make most bdist_* commands work: bdist_wininst and bdist_dumb work with 
setuptools "out of the box" because they call the "install" command 
internally; only bdist_rpm required some monkeypatching to add 
--single-version-externally-managed, because it writes script files that 
run "install" later.

Anyway, I guess what I'm saying is, if you want traditional distutils-style 
installation, you should use system packages, since they're the only way to 
allow uninstallation and avoid corrupted upgrades.  Otherwise, you should 
use .egg files or directories, in which case easy_install (and in the 
future, nest) will be your package manager.


From pje at telecommunity.com  Thu Dec 22 16:44:03 2005
From: pje at telecommunity.com (Phillip J. Eby)
Date: Thu, 22 Dec 2005 10:44:03 -0500
Subject: [Distutils] Egg support for system
 packages	(including	bdist_wininst)
In-Reply-To: <43AAA17F.5040203@egenix.com>
References: <43AA9282.5040806@egenix.com>
	<5.1.1.6.0.20051214184924.03033a28@mail.telecommunity.com>
	<43AA9282.5040806@egenix.com>
Message-ID: <5.1.1.6.0.20051222104251.04319f10@mail.telecommunity.com>

At 01:52 PM 12/22/2005 +0100, M.-A. Lemburg wrote:
>Somewhat related to this: wouldn't it be possible to build
>.egg ZIP files using the same layout, ie. with the .egg-info
>subdir inside the package instead of having a top-level
>EGG-INFO directory in the ZIP file ?!
>
>That way, you could simply unzip the .egg file somewhere
>on your PYTHONPATH in order to install it and still have
>the meta information at hand.

And the advantage of manually doing this over having easy_install do it 
would be...?


From mal at egenix.com  Thu Dec 22 22:39:26 2005
From: mal at egenix.com (M.-A. Lemburg)
Date: Thu, 22 Dec 2005 22:39:26 +0100
Subject: [Distutils] Egg support for system
	packages	(including	bdist_wininst)
In-Reply-To: <5.1.1.6.0.20051222104251.04319f10@mail.telecommunity.com>
References: <43AA9282.5040806@egenix.com>
	<5.1.1.6.0.20051214184924.03033a28@mail.telecommunity.com>
	<43AA9282.5040806@egenix.com>
	<5.1.1.6.0.20051222104251.04319f10@mail.telecommunity.com>
Message-ID: <43AB1D0E.2000204@egenix.com>

Phillip J. Eby wrote:
> At 01:52 PM 12/22/2005 +0100, M.-A. Lemburg wrote:
>> Somewhat related to this: wouldn't it be possible to build
>> .egg ZIP files using the same layout, ie. with the .egg-info
>> subdir inside the package instead of having a top-level
>> EGG-INFO directory in the ZIP file ?!
>>
>> That way, you could simply unzip the .egg file somewhere
>> on your PYTHONPATH in order to install it and still have
>> the meta information at hand.
> 
> And the advantage of manually doing this over having easy_install do it
> would be...?

* No magic, whatsoever.
* No dependencies on a specific package manager.
* No commands to learn.
* No cluttering up PYTHONPATH.
* No mangling of sys.path at runtime.
* No Python installation slowdown due to frequent scans of ZIP files.

I guess what I want to say is: eggs would become a no-strings-attached,
easy to use drop-in install format.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Dec 22 2005)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! ::::

From mal at egenix.com  Thu Dec 22 23:22:34 2005
From: mal at egenix.com (M.-A. Lemburg)
Date: Thu, 22 Dec 2005 23:22:34 +0100
Subject: [Distutils] Egg support for system packages
	(including	bdist_wininst)
In-Reply-To: <5.1.1.6.0.20051222102709.04406a20@mail.telecommunity.com>
References: <5.1.1.6.0.20051214184924.03033a28@mail.telecommunity.com>
	<5.1.1.6.0.20051214184924.03033a28@mail.telecommunity.com>
	<5.1.1.6.0.20051222102709.04406a20@mail.telecommunity.com>
Message-ID: <43AB272A.3010508@egenix.com>

Phillip J. Eby wrote:
> At 12:48 PM 12/22/2005 +0100, M.-A. Lemburg wrote:
>> I'd just wish that this would be the default and the .egg ZIP
>> file installation approach be made an option.
>>
>> setuptools would then finally be compatible to the rest of the
>> distutils world again and avoid all the added overhead and
>> problems of ZIP file imports.
>>
>> Perhaps you could have two commands, e.g. the default install
>> would create the normal package directory (with added .egg-info
>> dir) and a new install_eggfile to install the .egg ZIP file
>> instead.
> 
> The only way this could happen is to add a manifest of installed files,
> in order to allow uninstallation and upgrades to be made safe in the
> absence of a packaging system. 

Sure, why not put this manifest into the .egg-info dir ?!

The advantage of the .egg-info dir is that it's easy
to find using package introspection.

> The .egg file/directory technique
> currently makes uninstallation as simple as "rm -rf
> package-version.egg", and multi-version installs work with no special
> action.
>
> Note, too, that using .egg files or directories means that installation
> *never breaks existing programs* as long as they have specified their
> dependencies accurately.  Traditional distutils installation is
> destructive in the absence of a system packager.
>
> However, if you *have* a system packager, then you should not be using
> "setup.py install" anyway, so I'm not sure I see how this is an issue. 

The issue is that "setup.py install" should continue to
install things the (one-and-only) Python way. Any different
way should use a different command name.

You are currently overloading the install command with a
completely different approach, which is not compatible to
the standard distutils system or the Python import mechanism.

> You should be simply building system packages.  And it's quite
> straightforward to make most bdist_* commands work: bdist_wininst and
> bdist_dumb work with setuptools "out of the box" because they call the
> "install" command internally; only bdist_rpm required some
> monkeypatching to add --single-version-externally-managed, because it
> writes script files that run "install" later.
> 
> Anyway, I guess what I'm saying is, if you want traditional
> distutils-style installation, you should use system packages, since
> they're the only way to allow uninstallation and avoid corrupted
> upgrades.  Otherwise, you should use .egg files or directories, in which
> case easy_install (and in the future, nest) will be your package manager.

I guess what I want to say is that I only want to be forced to
use the easy_install package manager (or any other
non-system package manager) if I have a need or requirement
for the features it offers.

The egg format is not too far away from being a nice drop-in
format for Python binary extensions.

All it takes is:

* making sure that an unzip will create a
  proper Python package (with the meta information embedded
  into it)

With this changes, I think that eggs could actually
become a prime distribution format - even for system
extensions since it then no longer interferes with
the system installer mechanisms.

BTW, in case you still don't believe that placing ZIP
files on PYTHONPATH causes a slowdown, just install
10-20 eggs on PYTHONPATH and then do a

python -v

Python will tell you that it scanned all the ZIP files
on startup - even before having imported any of the
system lib modules such as warnings. Note that disabling
the import of site.py doesn't change this (python -S -v).

Here's an example with 20 eggs (using C shell):

tmp/eggs/> setenv PYTHONPATH
egg1.egg:egg10.egg:egg11.egg:egg12.egg:egg13.egg:egg14.egg:egg15.egg:egg16.egg:egg17.egg:egg18.egg:egg19.egg:egg2.egg:egg20.egg:egg3.egg:egg4.egg:egg5.egg:egg6.egg:egg7.egg:egg8.egg:egg9.egg
tmp/eggs> time python -S -c '0'
0.014u 0.006s 0:00.02 50.0%     0+0k 0+0io 0pf+0w
tmp/eggs> unsetenv PYTHONPATH
tmp/eggs> time python -S -c '0'
0.006u 0.003s 0:00.01 0.0%      0+0k 0+0io 0pf+0w

System time for startup doubles.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Dec 22 2005)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! ::::

From mal at egenix.com  Thu Dec 22 23:38:45 2005
From: mal at egenix.com (M.-A. Lemburg)
Date: Thu, 22 Dec 2005 23:38:45 +0100
Subject: [Distutils] Multiple version support (default installs and
	plugins)
In-Reply-To: <5.1.1.6.0.20051222102351.01b07cd8@mail.telecommunity.com>
References: <43AA8FAC.1060808@egenix.com> <43A99140.7080900@colorstudy.com>
	<43AA8FAC.1060808@egenix.com>
	<5.1.1.6.0.20051222102351.01b07cd8@mail.telecommunity.com>
Message-ID: <43AB2AF5.7050800@egenix.com>

Phillip J. Eby wrote:
> At 02:13 PM 12/22/2005 +0000, Paul Moore wrote:
>> On 12/22/05, M.-A. Lemburg <mal at egenix.com> wrote:
>> > Ian Bicking wrote:
>> > > I'm still finding it impossible to use multiple versions of a package
>> > > unless none of them show up in a .pth file (i.e., none are available
>> > > without requiring), I get a VersionConflict.
>> >
>> > This comes up every now and then when discussing the benefits
>> > of easy_install.
>> >
>> > I'd really like to understand what the use case is for having
>> > multiple versions of a package around.
>> >
>> > Note that sys.modules (the registry of loaded modules) does
>> > not support having multiple versions of a module loaded.
>>
>> I think the point is to allow multiple versions to be installed on the
>> system, but allow an individual program, at run time, to specify which
>> one is to be loaded. There will never be more than one version loaded
>> in any specific instance, but if program A requires version 1 of
>> package X, and program B requires version 2 of package X, both can run
>> without needing a reinstall of package X in between.
>>
>> Of course, this is also possible via PYTHONPATH manipulations, or
>> sys.path modifications in the program before doing the import. All
>> setuptools is doing is to provide a common infrastructure for handling
>> this.
> 
> Yes, Paul is correct; supporting multiple programs that use different
> versions of a module is indeed the primary use case.  Also, during
> development and debugging it can be quite convenient to switch versions
> of a dependency back and forth to identify the source of a problem.

Ok, I can see your point.

How do you assure that only one version of the package
gets loaded into the Python process ?

If you use a module A that needs version 1 and at the
same time a module B that requires version 2, how will
this conflict get resolved ?

If you find this feature important enough, shouldn't we
try to get a solution into the standard Python import
mechanism rather than playing tricks with sys.path ?

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Dec 22 2005)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! ::::

From pje at telecommunity.com  Fri Dec 23 00:10:42 2005
From: pje at telecommunity.com (Phillip J. Eby)
Date: Thu, 22 Dec 2005 18:10:42 -0500
Subject: [Distutils] Egg support for system
 packages	(including	bdist_wininst)
In-Reply-To: <43AB1D0E.2000204@egenix.com>
References: <5.1.1.6.0.20051222104251.04319f10@mail.telecommunity.com>
	<43AA9282.5040806@egenix.com>
	<5.1.1.6.0.20051214184924.03033a28@mail.telecommunity.com>
	<43AA9282.5040806@egenix.com>
	<5.1.1.6.0.20051222104251.04319f10@mail.telecommunity.com>
Message-ID: <5.1.1.6.0.20051222180744.020e8e58@mail.telecommunity.com>

At 10:39 PM 12/22/2005 +0100, M.-A. Lemburg wrote:
>Phillip J. Eby wrote:
> > At 01:52 PM 12/22/2005 +0100, M.-A. Lemburg wrote:
> >> Somewhat related to this: wouldn't it be possible to build
> >> .egg ZIP files using the same layout, ie. with the .egg-info
> >> subdir inside the package instead of having a top-level
> >> EGG-INFO directory in the ZIP file ?!
> >>
> >> That way, you could simply unzip the .egg file somewhere
> >> on your PYTHONPATH in order to install it and still have
> >> the meta information at hand.
> >
> > And the advantage of manually doing this over having easy_install do it
> > would be...?
>
>* No magic, whatsoever.
>* No dependencies on a specific package manager.
>* No commands to learn.
>* No cluttering up PYTHONPATH.
>* No mangling of sys.path at runtime.
>* No Python installation slowdown due to frequent scans of ZIP files.
>
>I guess what I want to say is: eggs would become a no-strings-attached,
>easy to use drop-in install format.

Perhaps I was unclear.  My question is what benefit there was, compared to 
having easy_install (or any other tool) do the unzipping and rename 
EGG-INFO to Package-Version.egg-info?

Note also that unzipping an egg to a directory with the same name as its 
.egg file produces a similar result, but has the additional advantage of 
supporting a trivial uninstallation - with no package manager required for 
either installation or uninstallation.  So, the use case of "just unzip and 
go" actually works already, if you either manually add it to a .pth file or 
just use require() at runtime.


From pje at telecommunity.com  Fri Dec 23 01:11:04 2005
From: pje at telecommunity.com (Phillip J. Eby)
Date: Thu, 22 Dec 2005 19:11:04 -0500
Subject: [Distutils] Egg support for system packages
 (including	bdist_wininst)
In-Reply-To: <43AB272A.3010508@egenix.com>
References: <5.1.1.6.0.20051222102709.04406a20@mail.telecommunity.com>
	<5.1.1.6.0.20051214184924.03033a28@mail.telecommunity.com>
	<5.1.1.6.0.20051214184924.03033a28@mail.telecommunity.com>
	<5.1.1.6.0.20051222102709.04406a20@mail.telecommunity.com>
Message-ID: <5.1.1.6.0.20051222181054.04faeaa8@mail.telecommunity.com>

At 11:22 PM 12/22/2005 +0100, M.-A. Lemburg wrote:
>Phillip J. Eby wrote:
> > At 12:48 PM 12/22/2005 +0100, M.-A. Lemburg wrote:
> >> I'd just wish that this would be the default and the .egg ZIP
> >> file installation approach be made an option.
> >>
> >> setuptools would then finally be compatible to the rest of the
> >> distutils world again and avoid all the added overhead and
> >> problems of ZIP file imports.
> >>
> >> Perhaps you could have two commands, e.g. the default install
> >> would create the normal package directory (with added .egg-info
> >> dir) and a new install_eggfile to install the .egg ZIP file
> >> instead.
> >
> > The only way this could happen is to add a manifest of installed files,
> > in order to allow uninstallation and upgrades to be made safe in the
> > absence of a packaging system.
>
>Sure, why not put this manifest into the .egg-info dir ?!

Because then people will need a special tool to uninstall it.  I'm kind of 
confused because in one email you argue against having easy_install as a 
package manager, and then you seem to be arguing for making more package 
management tools.  Which would you prefer?  :)

Here are the use cases as I see them:

Just unzip an egg and use it:  Sure, just unzip it as a directory with the 
.egg name and add it to sys.path or use require().

Flat layout, short sys.path:  Use a system package manager like bdist_rpm 
or bdist_wininst, or some other bdist wrapper.

No package manager, or other use cases: Use easy_install as your package 
manager.

Looks to me like everything is covered here.


>The advantage of the .egg-info dir is that it's easy
>to find using package introspection.

I don't understand you.  Package != distribution.  .egg-info directories go 
with a project's distribution, and have nothing to do with packages in the 
import sense.  A distribution might contain only a module, for example.  So 
there is no connection between a package and its .egg-info in the sense you 
seem to be implying.


> > The .egg file/directory technique
> > currently makes uninstallation as simple as "rm -rf
> > package-version.egg", and multi-version installs work with no special
> > action.
> >
> > Note, too, that using .egg files or directories means that installation
> > *never breaks existing programs* as long as they have specified their
> > dependencies accurately.  Traditional distutils installation is
> > destructive in the absence of a system packager.
> >
> > However, if you *have* a system packager, then you should not be using
> > "setup.py install" anyway, so I'm not sure I see how this is an issue.
>
>The issue is that "setup.py install" should continue to
>install things the (one-and-only) Python way.

Which is currently broken without a package manager for uninstallation or 
upgrade.  Your argument implies that:

1. Broken things should not be fixed or improved, ever
2. One must never have more than one implementation of an interface

The first one needs no further comment; the second negates the entire 
purpose of having an interface.  (In case it's unclear, I mean that 
"setup.py install" is an interface that simply promises to make the project 
available for use, not how or in what format.)


>  Any different
>way should use a different command name.
>
>You are currently overloading the install command with a
>completely different approach, which is not compatible to
>the standard distutils system or the Python import mechanism.

Please define "compatible" and explain why you believe the above to be 
true, providing adequate technical detail to demonstrate the supposed 
incompatibility.  When you have enough information to do that, you'll see 
why it's not true.

You'll save some time, however, if you first investigate the 'extra_path' 
option to 'setup()' that is provided by the distutils.  It is well within 
the power of a normal "setup.py install" to produce *precisely* the same 
results as a setuptools "setup.py install", and I know this for a fact 
because early versions of setuptools actually *used* it to bootstrap the 
installation of setuptools itself as an egg!

In other words, there is nothing "completely different" going on 
here.  Nothing.  This "non-standard" and "not compatible" nonsense is pure 
FUD, unless of course the distutils are non-standard and not compatible 
with themselves.  :)


> > Anyway, I guess what I'm saying is, if you want traditional
> > distutils-style installation, you should use system packages, since
> > they're the only way to allow uninstallation and avoid corrupted
> > upgrades.  Otherwise, you should use .egg files or directories, in which
> > case easy_install (and in the future, nest) will be your package manager.
>
>I guess what I want to say is that I only want to be forced to
>use the easy_install package manager (or any other
>non-system package manager) if I have a need or requirement
>for the features it offers.

Which is fine: use your system packager.  But people who don't have system 
packagers need a way to handle upgrades and uninstallation safely - and 
that *should* be the default behavior.  I don't give a flying hoot what the 
distutils default behavior is; only what it *should* be, which is to say 
*safe*.  The current default behavior is not safe.  Resorting to "it's the 
standard" as an argument therefore carries zero weight for me.

Nobody should be using unadorned distutils installation, except for the 
Python build process and tools that build system packages or other "bdist" 
formats.  Therefore, setuptools does not provide it as a default behavior.

Yes, setuptools is valuing safety over performance.  If you prefer to be 
unsafe, you'll have to read the docs and use a command line option to get 
it the other way.  If you want to be fast *and* safe, you need a system 
package manager.


>The egg format is not too far away from being a nice drop-in
>format for Python binary extensions.
>
>All it takes is:
>
>* making sure that an unzip will create a
>   proper Python package (with the meta information embedded
>   into it)

It does this now; unzip to a directory with a .egg name and add that 
directory to sys.path.


>With this changes, I think that eggs could actually
>become a prime distribution format - even for system
>extensions since it then no longer interferes with
>the system installer mechanisms.

If this is the *only* thing stopping you from liking eggs, I'll certainly 
consider adding it in 0.7.  There may be some backward compatibility 
issues, in that eggs built with 0.7 would then *not* be usable by older 
versions of the pkg_resources module, which would not know to look for 
.egg-info *inside* a zipped egg.  It seems to me that it would be much 
cleaner to have a script that take an .egg and do the equivalent of 
"install --single-version-externally-managed" with it, than to change the 
format to support using an arbitrary "unzip" utility.  Such an arbitrary 
utility also won't have any way to uninstall or upgrade the files, either.


>Here's an example with 20 eggs (using C shell):
>
>tmp/eggs/> setenv PYTHONPATH
>egg1.egg:egg10.egg:egg11.egg:egg12.egg:egg13.egg:egg14.egg:egg15.egg:egg16.egg:egg17.egg:egg18.egg:egg19.egg:egg2.egg:egg20.egg:egg3.egg:egg4.egg:egg5.egg:egg6.egg:egg7.egg:egg8.egg:egg9.egg
>tmp/eggs> time python -S -c '0'
>0.014u 0.006s 0:00.02 50.0%     0+0k 0+0io 0pf+0w
>tmp/eggs> unsetenv PYTHONPATH
>tmp/eggs> time python -S -c '0'
>0.006u 0.003s 0:00.01 0.0%      0+0k 0+0io 0pf+0w
>
>System time for startup doubles.

If I'm reading this correctly, that's 1/100th of a second to scan 20 
eggs.  If that's a linear progression, one could have *every single project 
listed on PyPI* as an egg on PYTHONPATH simultaneously, with only 54/100ths 
of a second being added to startup.  That actually sounds *really* good to 
me for anything but short scripts!

Note too that setuptools encourages the use of "virtual" Python 
installations for doing software development; this makes it really easy to 
install Python modules for a single project through having its own virtual 
Python.  It's likely that the system Python will have system packages (not 
affecting sys.path length), while user-installed packages will be more 
likely to go in a user-private virtual Python or extended PYTHONPATH 
anyway.  Meanwhile, the short scripts will likely use the system Python, or 
else you'll set up a stripped-down virtual Python just for those scripts if 
you need one.

Since setuptools now supports system packaging, there's no longer any 
reason to treat sys.path length as an argument against using 
setuptools.  The only reason to use easy_install in the first place is for 
things your packaging system can't supply, e.g. because a package is too 
new, or because you're a non-root user who can't install system packages 
anyway, or because you have application instances with different package 
needs, you need isolated Python installations for testing, etc. etc. 
etc.  As a general rule, system packagers don't support *any* of these 
things, so easy_install is really your only practical choice.  That's why 
it exists.


From pje at telecommunity.com  Fri Dec 23 01:25:59 2005
From: pje at telecommunity.com (Phillip J. Eby)
Date: Thu, 22 Dec 2005 19:25:59 -0500
Subject: [Distutils] Multiple version support (default installs and
 plugins)
In-Reply-To: <43AB2AF5.7050800@egenix.com>
References: <5.1.1.6.0.20051222102351.01b07cd8@mail.telecommunity.com>
	<43AA8FAC.1060808@egenix.com> <43A99140.7080900@colorstudy.com>
	<43AA8FAC.1060808@egenix.com>
	<5.1.1.6.0.20051222102351.01b07cd8@mail.telecommunity.com>
Message-ID: <5.1.1.6.0.20051222191118.04fcee20@mail.telecommunity.com>

At 11:38 PM 12/22/2005 +0100, M.-A. Lemburg wrote:
>How do you assure that only one version of the package
>gets loaded into the Python process ?

I went back in the time machine and convinced Guido to make the import 
system only load the first module it finds with a given name.  ;-)

Sorry, couldn't resist.  :)


>If you use a module A that needs version 1 and at the
>same time a module B that requires version 2, how will
>this conflict get resolved ?

pkg_resources.VersionConflict is raised at the point of require()-ing 
something that conflicts with something already in the working set.

Note that you might not get a conflict if module A accepts versions >=1, 
and B accepts versions >=2, and you have version 2, and it's the default 
version or was requested by the __main__ program, effectively making it the 
default.

For example, TurboGears specifies requirements for certain modules that are 
also depended on by its dependencies.  I don't remember what they are, but 
for the sake of example I'll say that TurboGears depends on a particular 
version of FormEncode, which is also depended on by SQLObject - which 
TurboGears also requires.  So it's something like this:

TurboGears:
     FormEncode>=0.5dev
     SQLObject>=0.8dev

SQLObject:
     FormEncode>=0.4

If you run a TurboGears-requiring script, it's going to request a newer 
FormEncode, which will then become the default.  SQLObject also gets 
processed for dependencies, since TurboGears requires it, but since the 
default (FormEncode>=0.5dev) is compatible with the SQLObject requirement 
(FormEncode>=0.4), life just goes on with no conflict.

There are currently a few issues with this algorithm, mainly that it 
decides on a "default version" too early; as soon as pkg_resources is 
finished importing, all "default versions" for eggs that are already 
importable on sys.path are established and can't be changed.  In 0.7, I'll 
be doing some work to loosen this so that the default version of a project 
won't be set in stone until an explicit choice is made or until the default 
default version has been imported.


>If you find this feature important enough, shouldn't we
>try to get a solution into the standard Python import
>mechanism rather than playing tricks with sys.path ?

"Playing tricks with sys.path" *is* the standard way to do it; just look at 
what Guido answers every time anybody asks about needing specific versions 
of things.  "The application should just set up its sys.path however it 
needs to" is more or less his standard response.  The pkg_resources module 
just provides you with convenient routines to do what Guido has already 
declared to be the One Obvious Way to do it, i.e. sys.path manipulation.


From b88zhou at gmail.com  Mon Dec 26 19:41:03 2005
From: b88zhou at gmail.com (Brian Zhou)
Date: Mon, 26 Dec 2005 10:41:03 -0800
Subject: [Distutils] [build_scripts] executable
Message-ID: <859e91e70512261041l476ddd16q549521c0ac725e47@mail.gmail.com>

Hi all,

I'm working on some system packages for http://nslu2-linux.org optware with
setuptools (svn dev so I can use the --single-version-externally-managed).
Thanks to your hard work, so far it looks very promising.

However, since the platform I'm working on requires cross build, the python
on debian host /usr/bin/python2.4 should not be used for scripts in the
package. In particular we need all scripts in the package to have

    #!/opt/bin/python

As the first line. With traditional distutils, we can append the following
to setup.cfg

    [build_scripts]
    executable=/opt/bin/python

Can we add that back please?

Thanks,

-Brian Zhou
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/distutils-sig/attachments/20051226/2ccb1c64/attachment.html

From b88zhou at gmail.com  Mon Dec 26 19:52:51 2005
From: b88zhou at gmail.com (Brian Zhou)
Date: Mon, 26 Dec 2005 10:52:51 -0800
Subject: [Distutils] [build_scripts] executable
In-Reply-To: <859e91e70512261041l476ddd16q549521c0ac725e47@mail.gmail.com>
References: <859e91e70512261041l476ddd16q549521c0ac725e47@mail.gmail.com>
Message-ID: <859e91e70512261052r667d40a9s6448a138502cd33b@mail.gmail.com>

Never mind. I think it's problem of the way turbogears uses setuptools, in
the case of sqlobject for example, it works ok.

-Brian Zhou

On 12/26/05, Brian Zhou <b88zhou at gmail.com> wrote:
>
> Hi all,
>
> I'm working on some system packages for http://nslu2-linux.org optware
> with setuptools (svn dev so I can use the
> --single-version-externally-managed). Thanks to your hard work, so far it
> looks very promising.
>
> However, since the platform I'm working on requires cross build, the
> python on debian host /usr/bin/python2.4 should not be used for scripts in
> the package. In particular we need all scripts in the package to have
>
>     #!/opt/bin/python
>
> As the first line. With traditional distutils, we can append the following
> to setup.cfg
>
>     [build_scripts]
>     executable=/opt/bin/python
>
> Can we add that back please?
>
> Thanks,
>
> -Brian Zhou
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/distutils-sig/attachments/20051226/700150d3/attachment.htm

From pje at telecommunity.com  Mon Dec 26 20:26:15 2005
From: pje at telecommunity.com (Phillip J. Eby)
Date: Mon, 26 Dec 2005 14:26:15 -0500
Subject: [Distutils] [build_scripts] executable
In-Reply-To: <859e91e70512261041l476ddd16q549521c0ac725e47@mail.gmail.co
 m>
Message-ID: <5.1.1.6.0.20051226140926.039b46c8@mail.telecommunity.com>

At 10:41 AM 12/26/2005 -0800, Brian Zhou wrote:
>Hi all,
>
>I'm working on some system packages for 
><http://nslu2-linux.org>http://nslu2-linux.org optware with setuptools 
>(svn dev so I can use the --single-version-externally-managed). Thanks to 
>your hard work, so far it looks very promising.
>
>However, since the platform I'm working on requires cross build, the 
>python on debian host /usr/bin/python2.4 should not be used for scripts in 
>the package. In particular we need all scripts in the package to have
>
>     #!/opt/bin/python
>
>As the first line. With traditional distutils, we can append the following 
>to setup.cfg
>
>     [build_scripts]
>     executable=/opt/bin/python
>
>Can we add that back please?

I've checked a fix in now.  I didn't know that option existed.  Note that 
it needs to be specified in the config file or on the command line at the 
time install_scripts is run, since setuptools-generated scripts don't have 
a separate "build" step.

Also, easy_install does not respect this option; I haven't evaluated yet 
whether it should, and the change is more complex.  So at the moment, I've 
only changed it for "install --single-version-externally-managed".


From pje at telecommunity.com  Mon Dec 26 20:27:53 2005
From: pje at telecommunity.com (Phillip J. Eby)
Date: Mon, 26 Dec 2005 14:27:53 -0500
Subject: [Distutils] [build_scripts] executable
In-Reply-To: <859e91e70512261052r667d40a9s6448a138502cd33b@mail.gmail.co
 m>
References: <859e91e70512261041l476ddd16q549521c0ac725e47@mail.gmail.com>
	<859e91e70512261041l476ddd16q549521c0ac725e47@mail.gmail.com>
Message-ID: <5.1.1.6.0.20051226142634.042c6bb0@mail.telecommunity.com>

At 10:52 AM 12/26/2005 -0800, Brian Zhou wrote:
>Never mind. I think it's problem of the way turbogears uses setuptools, in 
>the case of sqlobject for example, it works ok.

The issue is that setuptools-generated scripts were not respecting 
build_scripts --executable.  I've fixed that now.

Projects that don't use entry-point scripts wouldn't have been affected, 
and projects with both old and new-style scripts would have had some 
scripts work and some not.


From sharkypt at gmail.com  Wed Dec 28 12:44:09 2005
From: sharkypt at gmail.com (Sharky On PTNet)
Date: Wed, 28 Dec 2005 11:44:09 +0000
Subject: [Distutils] Problems with svn in windows
Message-ID: <40b686290512280344v3f668e24n@mail.gmail.com>

Hi,


  I?m trying to do "easy_install setuptools==dev" in a windows box but have
an error with svn command.

"""... 'svn' is not recognized as an internal or external command, ..."""

  I have installed TortoiseSVN but it seems that don't have command line, is
all with the Explorer.

  Can you tell me what client I may use in windows?

Thanks for all,
Helio Pereira

Sharky @ PTNet
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/distutils-sig/attachments/20051228/b9ee2220/attachment.htm

From tim.peters at gmail.com  Wed Dec 28 16:39:20 2005
From: tim.peters at gmail.com (Tim Peters)
Date: Wed, 28 Dec 2005 10:39:20 -0500
Subject: [Distutils] Problems with svn in windows
In-Reply-To: <40b686290512280344v3f668e24n@mail.gmail.com>
References: <40b686290512280344v3f668e24n@mail.gmail.com>
Message-ID: <1f7befae0512280739h67a1b080w26857f65752eb488@mail.gmail.com>

[Sharky On PTNet]
>    I?m trying to do "easy_install setuptools==dev" in a windows box but have
> an error with svn command.
>
>  """... 'svn' is not recognized as an internal or external command, ..."""
>
>    I have installed TortoiseSVN but it seems that don't have command line,
> is all with the Explorer.
>
>    Can you tell me what client I may use in windows?

Easiest is to run Jostein Andersen's SVN Windows installer, by
following the instructions on the SVN download page:

    http://subversion.tigris.org/project_packages.html

From jjl at pobox.com  Wed Dec 28 17:32:12 2005
From: jjl at pobox.com (John J Lee)
Date: Wed, 28 Dec 2005 16:32:12 +0000 (GMT Standard Time)
Subject: [Distutils] Problems with svn in windows
In-Reply-To: <40b686290512280344v3f668e24n@mail.gmail.com>
References: <40b686290512280344v3f668e24n@mail.gmail.com>
Message-ID: <Pine.WNT.4.64.0512281630090.2916@shaolin>

On Wed, 28 Dec 2005, Sharky On PTNet wrote:

>  I?m trying to do "easy_install setuptools==dev" in a windows box but have
> an error with svn command.
>
> """... 'svn' is not recognized as an internal or external command, ..."""
[...]
>  Can you tell me what client I may use in windows?

http://subversion.tigris.org/files/documents/15/25364/svn-1.2.3-setup.exe


(you may need to put svn on your PATH after running the installer above -- 
not sure, but I don't think it's necessary).


John

From jeff.pitman at gmail.com  Thu Dec 29 16:05:24 2005
From: jeff.pitman at gmail.com (Jeff Pitman)
Date: Thu, 29 Dec 2005 23:05:24 +0800
Subject: [Distutils] Setuptools' goal in rpm/deb-based systems
Message-ID: <6b9c17630512290705m22a4a9f1i4bfb8439a250e227@mail.gmail.com>

Hi,

This may have been discussed ad nauseum already, but as I tried to
package turbogears for Fedora/CentOS, I thought I'd throw in 2 cents.

From a .deb and .rpm perspective, it is highly discouraged to install
non-system managed files into non-$HOME areas (esp. /usr). Even for
many, /usr/local is highly frowned upon (eg. ./configure; make; make
install mantra).

I see that --single-version-externally-managed has been added as an
option so that .deb/.rpm packaging can take advantage of that relying
upon apt/yum/smart to take care of business. Hopefully, this will be
pushed out soon. (Because turbogears script generation is not trivial
to package. I think I'll just include a generated script as one of the
Sources, for now.)

I think the goal for setuptools on rpm/deb-based systems should focus
on getting $HOME setup and managed correctly for packages not provided
by the system. For example, if I had an ISP which provided python2.4,
but not TurboGears, then I could use .egg to manage a setup in $HOME
to use the packages correctly. Currently, -d/-s and other options can
be used. But, if the user isn't root, maybe it should be the default
behavior: install and manage in $HOME.

Also, are there efforts to detect if a package is already installed?
For example, if "python-sqlobject" is installed, can setuptools detect
the lib if the install_requires has "SQLObject"? I think peering
between .egg and .deb/.rpm to be a difficult problem. So, having it
all or nothing would be a cleaner policy. .deb/.rpm to manage /usr
installed packages/modules and .egg to manage $HOME installed
packages.

Thoughts?

--
-jeff

From dangoor at gmail.com  Thu Dec 29 16:31:49 2005
From: dangoor at gmail.com (Kevin Dangoor)
Date: Thu, 29 Dec 2005 10:31:49 -0500
Subject: [Distutils] Setuptools' goal in rpm/deb-based systems
In-Reply-To: <6b9c17630512290705m22a4a9f1i4bfb8439a250e227@mail.gmail.com>
References: <6b9c17630512290705m22a4a9f1i4bfb8439a250e227@mail.gmail.com>
Message-ID: <3f085ecd0512290731ib409502vaacd46675bc3e1d5@mail.gmail.com>

On 12/29/05, Jeff Pitman <jeff.pitman at gmail.com> wrote:
> This may have been discussed ad nauseum already, but as I tried to
> package turbogears for Fedora/CentOS, I thought I'd throw in 2 cents.

Hey, great! I'm sure many will appreciate the effort!

> I see that --single-version-externally-managed has been added as an
> option so that .deb/.rpm packaging can take advantage of that relying
> upon apt/yum/smart to take care of business. Hopefully, this will be
> pushed out soon. (Because turbogears script generation is not trivial
> to package. I think I'll just include a generated script as one of the
> Sources, for now.)

Note that TurboGears 0.9 absolutely needs to run as a proper egg.
(When you say you're including the generated script as part of the
sources, I'm not sure if that means that you're bypassing the egg
packaging entirely...)

> I think the goal for setuptools on rpm/deb-based systems should focus
> on getting $HOME setup and managed correctly for packages not provided
> by the system. For example, if I had an ISP which provided python2.4,
> but not TurboGears, then I could use .egg to manage a setup in $HOME
> to use the packages correctly. Currently, -d/-s and other options can
> be used. But, if the user isn't root, maybe it should be the default
> behavior: install and manage in $HOME.

Setting things up in $HOME is definitely a good thing, and doesn't
take very much effort. I hope more ISPs will do so (and it's something
I'm working on with TurboGears hosting providers).

> Also, are there efforts to detect if a package is already installed?
> For example, if "python-sqlobject" is installed, can setuptools detect
> the lib if the install_requires has "SQLObject"? I think peering
> between .egg and .deb/.rpm to be a difficult problem. So, having it
> all or nothing would be a cleaner policy. .deb/.rpm to manage /usr
> installed packages/modules and .egg to manage $HOME installed
> packages.

The problem is with the missing metadata. setuptools doesn't know what
version of SQLObject that is, and TurboGears most definitely requires
0.7 and up. I think the goal is ultimately to have system-installed
.debs/.rpms that are also eggs (with the proper metadata). TurboGears
0.9 makes use of setuptools' extension mechanism which is why the egg
metadata needs to be available.

Kevin

From dangoor at gmail.com  Thu Dec 29 16:51:24 2005
From: dangoor at gmail.com (Kevin Dangoor)
Date: Thu, 29 Dec 2005 10:51:24 -0500
Subject: [Distutils] Setuptools' goal in rpm/deb-based systems
In-Reply-To: <6b9c17630512290745o5da75422p81c17b481bc2f5f2@mail.gmail.com>
References: <6b9c17630512290705m22a4a9f1i4bfb8439a250e227@mail.gmail.com>
	<3f085ecd0512290731ib409502vaacd46675bc3e1d5@mail.gmail.com>
	<6b9c17630512290745o5da75422p81c17b481bc2f5f2@mail.gmail.com>
Message-ID: <3f085ecd0512290751m60745d34o6530f6e855129fd4@mail.gmail.com>

On 12/29/05, Jeff Pitman <jeff.pitman at gmail.com> wrote:
> On 12/29/05, Kevin Dangoor <dangoor at gmail.com> wrote:
> > On 12/29/05, Jeff Pitman <jeff.pitman at gmail.com> wrote:
> > > (Because turbogears script generation is not trivial
> > > to package. I think I'll just include a generated script as one of the
> > > Sources, for now.)
> >
> > Note that TurboGears 0.9 absolutely needs to run as a proper egg.
> > (When you say you're including the generated script as part of the
> > sources, I'm not sure if that means that you're bypassing the egg
> > packaging entirely...)
>
> It's just /usr/bin/tg-admin.

Oh, OK. If that's all you did, that shouldn't pose a problem.

> > The problem is with the missing metadata. setuptools doesn't know what
> > version of SQLObject that is, and TurboGears most definitely requires
> > 0.7 and up. I think the goal is ultimately to have system-installed
> > .debs/.rpms that are also eggs (with the proper metadata).
>
> Do you mean package the egg-info in with the rpm/deb?

I believe that's what Phillip's recent changes have been about. They
provide a way to put the egg metadata inside the package directory so
that it can install like a traditional package but still have the
appropriate metadata. I'll stop commenting at this point, though,
because Phillip's the authoritative source and I've never had to
personally walk down the path of packaging an egg up into something
else.

> BTW, TurboGears
> needs to be installed as rpm to ever be included in Fedora Extras (and
> as .deb in Debian/Ubuntu land). This is a matter of policy. And people
> are very wary about pollution of system directories like /usr.
> Technically, files updated during runtime should go in /var (like
> logs, state, and crap).

Absolutely.

> > TurboGears
> > 0.9 makes use of setuptools' extension mechanism which is why the egg
> > metadata needs to be available.
>
> During runtime or during installation?

During runtime. Certain parts of TurboGears can be extended just by
installing an egg with the additional functionality. TurboGears itself
uses this mechanism to define the base pieces. (tg-admin commands are
an example of this: anyone can create an egg that adds tg-admin
commands. the existing commands are all defined in the same way that a
user would create commands.)

Kevin

From jeff.pitman at gmail.com  Thu Dec 29 16:45:12 2005
From: jeff.pitman at gmail.com (Jeff Pitman)
Date: Thu, 29 Dec 2005 23:45:12 +0800
Subject: [Distutils] Setuptools' goal in rpm/deb-based systems
In-Reply-To: <3f085ecd0512290731ib409502vaacd46675bc3e1d5@mail.gmail.com>
References: <6b9c17630512290705m22a4a9f1i4bfb8439a250e227@mail.gmail.com>
	<3f085ecd0512290731ib409502vaacd46675bc3e1d5@mail.gmail.com>
Message-ID: <6b9c17630512290745o5da75422p81c17b481bc2f5f2@mail.gmail.com>

On 12/29/05, Kevin Dangoor <dangoor at gmail.com> wrote:
> On 12/29/05, Jeff Pitman <jeff.pitman at gmail.com> wrote:
> > (Because turbogears script generation is not trivial
> > to package. I think I'll just include a generated script as one of the
> > Sources, for now.)
>
> Note that TurboGears 0.9 absolutely needs to run as a proper egg.
> (When you say you're including the generated script as part of the
> sources, I'm not sure if that means that you're bypassing the egg
> packaging entirely...)

It's just /usr/bin/tg-admin.

> > Also, are there efforts to detect if a package is already installed?
> > For example, if "python-sqlobject" is installed, can setuptools detect
> > the lib if the install_requires has "SQLObject"? I think peering
> > between .egg and .deb/.rpm to be a difficult problem. So, having it
> > all or nothing would be a cleaner policy. .deb/.rpm to manage /usr
> > installed packages/modules and .egg to manage $HOME installed
> > packages.
>
> The problem is with the missing metadata. setuptools doesn't know what
> version of SQLObject that is, and TurboGears most definitely requires
> 0.7 and up. I think the goal is ultimately to have system-installed
> .debs/.rpms that are also eggs (with the proper metadata).

Do you mean package the egg-info in with the rpm/deb? BTW, TurboGears
needs to be installed as rpm to ever be included in Fedora Extras (and
as .deb in Debian/Ubuntu land). This is a matter of policy. And people
are very wary about pollution of system directories like /usr.
Technically, files updated during runtime should go in /var (like
logs, state, and crap).

> TurboGears
> 0.9 makes use of setuptools' extension mechanism which is why the egg
> metadata needs to be available.

During runtime or during installation?

--
-jeff

From jeff.pitman at gmail.com  Thu Dec 29 16:56:36 2005
From: jeff.pitman at gmail.com (Jeff Pitman)
Date: Thu, 29 Dec 2005 23:56:36 +0800
Subject: [Distutils] Setuptools' goal in rpm/deb-based systems
In-Reply-To: <3f085ecd0512290751m60745d34o6530f6e855129fd4@mail.gmail.com>
References: <6b9c17630512290705m22a4a9f1i4bfb8439a250e227@mail.gmail.com>
	<3f085ecd0512290731ib409502vaacd46675bc3e1d5@mail.gmail.com>
	<6b9c17630512290745o5da75422p81c17b481bc2f5f2@mail.gmail.com>
	<3f085ecd0512290751m60745d34o6530f6e855129fd4@mail.gmail.com>
Message-ID: <6b9c17630512290756u71805c93i339a4fc28eeefa4@mail.gmail.com>

On 12/29/05, Kevin Dangoor <dangoor at gmail.com> wrote:
> On 12/29/05, Jeff Pitman <jeff.pitman at gmail.com> wrote:
> > On 12/29/05, Kevin Dangoor <dangoor at gmail.com> wrote:
> > > TurboGears
> > > 0.9 makes use of setuptools' extension mechanism which is why the egg
> > > metadata needs to be available.
> >
> > During runtime or during installation?
>
> During runtime. Certain parts of TurboGears can be extended just by
> installing an egg with the additional functionality. TurboGears itself
> uses this mechanism to define the base pieces. (tg-admin commands are
> an example of this: anyone can create an egg that adds tg-admin
> commands. the existing commands are all defined in the same way that a
> user would create commands.)

So, they're plugins that extend tg-admin commands. In the .rpm/.deb
space what would happen here is that these plugins would also be
installed as .rpm/.deb. This is kind of like Twisted 2.0+ after the
split. What would be good here is to have .eggs install into $HOME and
tg-admin still pickup on them.

--
-jeff

From dangoor at gmail.com  Thu Dec 29 17:11:26 2005
From: dangoor at gmail.com (Kevin Dangoor)
Date: Thu, 29 Dec 2005 11:11:26 -0500
Subject: [Distutils] Setuptools' goal in rpm/deb-based systems
In-Reply-To: <6b9c17630512290756u71805c93i339a4fc28eeefa4@mail.gmail.com>
References: <6b9c17630512290705m22a4a9f1i4bfb8439a250e227@mail.gmail.com>
	<3f085ecd0512290731ib409502vaacd46675bc3e1d5@mail.gmail.com>
	<6b9c17630512290745o5da75422p81c17b481bc2f5f2@mail.gmail.com>
	<3f085ecd0512290751m60745d34o6530f6e855129fd4@mail.gmail.com>
	<6b9c17630512290756u71805c93i339a4fc28eeefa4@mail.gmail.com>
Message-ID: <3f085ecd0512290811n20c3da3ap75f270eb29183f14@mail.gmail.com>

On 12/29/05, Jeff Pitman <jeff.pitman at gmail.com> wrote:
> So, they're plugins that extend tg-admin commands. In the .rpm/.deb
> space what would happen here is that these plugins would also be
> installed as .rpm/.deb. This is kind of like Twisted 2.0+ after the
> split. What would be good here is to have .eggs install into $HOME and
> tg-admin still pickup on them.

If the system is set up correctly, the users could install the plugins
as .rpms/.debs *or* install the eggs to their $HOME. Either way should
work.

Kevin

--
Kevin Dangoor
Author of the Zesty News RSS newsreader

email: kid at blazingthings.com
company: http://www.BlazingThings.com
blog: http://www.BlueSkyOnMars.com

From pje at telecommunity.com  Thu Dec 29 19:12:13 2005
From: pje at telecommunity.com (Phillip J. Eby)
Date: Thu, 29 Dec 2005 13:12:13 -0500
Subject: [Distutils] Setuptools' goal in rpm/deb-based systems
In-Reply-To: <3f085ecd0512290811n20c3da3ap75f270eb29183f14@mail.gmail.co
 m>
References: <6b9c17630512290756u71805c93i339a4fc28eeefa4@mail.gmail.com>
	<6b9c17630512290705m22a4a9f1i4bfb8439a250e227@mail.gmail.com>
	<3f085ecd0512290731ib409502vaacd46675bc3e1d5@mail.gmail.com>
	<6b9c17630512290745o5da75422p81c17b481bc2f5f2@mail.gmail.com>
	<3f085ecd0512290751m60745d34o6530f6e855129fd4@mail.gmail.com>
	<6b9c17630512290756u71805c93i339a4fc28eeefa4@mail.gmail.com>
Message-ID: <5.1.1.6.0.20051229130933.01f79008@mail.telecommunity.com>

At 11:11 AM 12/29/2005 -0500, Kevin Dangoor wrote:
>On 12/29/05, Jeff Pitman <jeff.pitman at gmail.com> wrote:
> > So, they're plugins that extend tg-admin commands. In the .rpm/.deb
> > space what would happen here is that these plugins would also be
> > installed as .rpm/.deb. This is kind of like Twisted 2.0+ after the
> > split. What would be good here is to have .eggs install into $HOME and
> > tg-admin still pickup on them.
>
>If the system is set up correctly, the users could install the plugins
>as .rpms/.debs *or* install the eggs to their $HOME. Either way should
>work.

By the way, what Jeff is asking for regarding $HOME is now a documented 
procedure in the EasyInstall documentation, known as an "Administrator 
Installation".  See:

http://peak.telecommunity.com/DevCenter/EasyInstall#custom-installation-locations

for more details.


From pje at telecommunity.com  Thu Dec 29 19:28:58 2005
From: pje at telecommunity.com (Phillip J. Eby)
Date: Thu, 29 Dec 2005 13:28:58 -0500
Subject: [Distutils] Setuptools' goal in rpm/deb-based systems
In-Reply-To: <6b9c17630512290705m22a4a9f1i4bfb8439a250e227@mail.gmail.co
 m>
Message-ID: <5.1.1.6.0.20051229132412.04254c58@mail.telecommunity.com>

At 11:05 PM 12/29/2005 +0800, Jeff Pitman wrote:
>I see that --single-version-externally-managed has been added as an
>option so that .deb/.rpm packaging can take advantage of that relying
>upon apt/yum/smart to take care of business. Hopefully, this will be
>pushed out soon.

FYI, unless there's some policy preventing it, you can use the current SVN 
snapshot as a basis for a Debian package of setuptools.  If built from a 
source checkout, it will correctly include its Subversion revision number 
as part of its name, e.g. "0.6a9dev-r41780".  In setuptools' version 
numbering, this is an *older* release than "0.6a9", so when the final 0.6a9 
comes out it will replace the "dev" versions.  I realize that this isn't 
necessarily the same thing as how Debian package versions work, but I 
assume it must have some facility for pre-release tags.  If all else fails, 
you could treat it as "0.6a8-r41780" instead, which might be a more 
compatible numbering approach.

At this point, the only thing left on my to-do list for 0.6a9 is more 
documentation for EasyInstall, so don't let that stop you from packaging 
it.  The only reason to wait would be to see if anybody reports bugs with 
the current version, which probably won't happen until *after* 0.6a9 is 
released.  :)


From ianb at colorstudy.com  Thu Dec 29 23:39:30 2005
From: ianb at colorstudy.com (Ian Bicking)
Date: Thu, 29 Dec 2005 16:39:30 -0600
Subject: [Distutils] default installs and plugins
In-Reply-To: <5.1.1.6.0.20051221130104.02786800@mail.telecommunity.com>
References: <5.1.1.6.0.20051221130104.02786800@mail.telecommunity.com>
Message-ID: <43B465A2.8030009@colorstudy.com>

Phillip J. Eby wrote:
>>   This worries me, since I have lots of code that is both
>> package and plugin, and if this is the situation then I'll have to move
>> to non-root installs everywhere I want version isolation.  (Which I
>> actually suspect I'll have to do anyway, but I was hoping that things
>> would at least be usable without it).
> 
> 
> There are a number other custom installation strategies available 
> besides the "virtual python" one, now.  You might check those out on the 
> EasyInstall page.

Incidentally, I went with the administrator installation, but based off 
an environmental variable instead of the home directory.  Except that 
distutils.dist does not substitute environmental variables in 
configuration files, so I had to patch that.  It's a very small change, 
but I don't know if I can really justify submitting it upstream. 
Anyway, now I select my environment (a particular website) by setting an 
environmental variable, but sticking with the same user account.  It's 
actually reasonable that I could login as the user account for the site 
I'm working on, but I kind of prefer always being logged in as myself.

Since I've found it far too difficult to maintain version specifications 
that really keep sites isolated in the way I want, the separate paths 
work much better.

I notice that Ruby Gems have an option to "freeze" the Gems you are 
using, which copies them all to some directory.  In the context of Rails 
that's a subdirectory of the project directory.  I find Rails a bit 
funny, as it jams configuration, runtime stuff like libraries, scripts, 
and the actual code all in one big directory hierarchy.  But anyway, 
simply installing stuff in the environment that I want works well, so 
long as things aren't also globally installed.  We'll probably install 
some stuff globally, but only if it is really stable and we aren't 
involved in development at all (e.g., psycopg).

-- 
Ian Bicking  /  ianb at colorstudy.com  /  http://blog.ianbicking.org

From pje at telecommunity.com  Fri Dec 30 17:51:44 2005
From: pje at telecommunity.com (Phillip J. Eby)
Date: Fri, 30 Dec 2005 11:51:44 -0500
Subject: [Distutils] Setuptools projects with '-' in their names (was Re:
 some kind of problem with setuptools 0.6a9)
In-Reply-To: <3f085ecd0512300704w61be76a0y27a233b96ba3b38e@mail.gmail.co
 m>
Message-ID: <5.1.1.6.0.20051230110343.03b9b010@mail.telecommunity.com>

At 10:04 AM 12/30/2005 -0500, Kevin Dangoor wrote:
>The "tg-admin sql" command uses sqlobject which, in turn uses egg
>metadata. Something I just noticed is that these commands are no
>longer working on a project i have called "Wiki-20". They *used* to
>work. Now they get a "DistributionNotFound" error for Wiki-20.
>...
>I had also tried to just do "pkg_resources.require("WIki-20")" from
>the Python shell, but that didn't work either. This is new behavior
>between a8 and a9. Let me know if there's anything else I can tell you
>about the problem...

Hi all.  Due to the recent addition of versioned '.egg-info' support for 
system packages, you may have to rename any current '.egg-info' if your 
project name has a '-' in it.  Otherwise, recent versions of setuptools 
will think the '-' is the separator between the project name and version 
number, leading to problems like the ones described by Kevin above.

I've added code to the latest SVN setuptools to detect this problem and 
warn you about it, as well as tell you what you should rename the .egg-info 
directory to.

If you have a '-' in your project's name, and you are using an 0.6a9dev 
version of setuptools, you should upgrade and rename the .egg-info 
directory.  If you distribute any source releases of your project or have 
users tracking SVN or CVS releases of it, you *must* make your project 
require setuptools>=0.6a9dev_r41857, by passing it to ez_setup, e.g.::

     from ez_setup import use_setuptools
     use_setuptools('0.6a9dev_r41857', 
'http://some/dir/to/download/the/egg/from/')

If you do not do this, and a user has an older version of setuptools, they 
will encounter problems.

If you wait for the official release of 0.6a9 to upgrade, and you use its 
ez_setup.py or ez_setup/__init__.py, you can avoid having to change the 
setup script as shown.

Again, all this applies only to projects with a '-' in their names, and 
only if *you* are upgrading from 0.6a8 to an 0.6a9 development or final 
release.  Your project's users can upgrade to 0.6a9 without breakage, as 
there is a backward compatibility mode for everything but "setup.py 
develop" (which will refuse to run until the .egg-info directory has been 
renamed).



From gnuguy at thelink.net  Fri Dec 30 18:11:55 2005
From: gnuguy at thelink.net (cmire)
Date: Fri, 30 Dec 2005 11:11:55 -0600
Subject: [Distutils] SuSE 10.0 & distutils
Message-ID: <200512301111.55947.gnuguy@thelink.net>

I have SuSE 10.0 installed on my machine, and it comes with Python 2.4.1.  
I've been trying to build Zope on this box, and I'm getting an error that it 
can't find distutils.cmd.  I did some digging, and it appears this is part of 
distutils.core, and according to the distutils download page, distutils 
should be already included with Python versions above 1.6.

I checked where my system has Python installed, and it's 
at /usr/lib/python2.4.  I did a find for that directory and all the 
subdirectories for anything distutils, and nothing came back.

Is it just a straightforward thing to download the distutils tar file and 
install it, or since it wasn't included with my Python install (and should 
have been), is there anything special I should check or be aware of?

Thanks,
Charles Mire

From pje at telecommunity.com  Fri Dec 30 18:25:13 2005
From: pje at telecommunity.com (Phillip J. Eby)
Date: Fri, 30 Dec 2005 12:25:13 -0500
Subject: [Distutils] SuSE 10.0 & distutils
In-Reply-To: <200512301111.55947.gnuguy@thelink.net>
Message-ID: <5.1.1.6.0.20051230122340.020e3908@mail.telecommunity.com>

At 11:11 AM 12/30/2005 -0600, cmire wrote:
>I have SuSE 10.0 installed on my machine, and it comes with Python 2.4.1.
>I've been trying to build Zope on this box, and I'm getting an error that it
>can't find distutils.cmd.  I did some digging, and it appears this is part of
>distutils.core, and according to the distutils download page, distutils
>should be already included with Python versions above 1.6.
>
>I checked where my system has Python installed, and it's
>at /usr/lib/python2.4.  I did a find for that directory and all the
>subdirectories for anything distutils, and nothing came back.
>
>Is it just a straightforward thing to download the distutils tar file and
>install it, or since it wasn't included with my Python install (and should
>have been), is there anything special I should check or be aware of?

Some Linux distributions split Python into multiple system packages; the 
missing package is probably an RPM (or whatever SuSE uses for packaging) 
called "python-dev" or "python-devel".  Install that and you should be good 
to go.


From jeff.pitman at gmail.com  Sat Dec 31 04:45:10 2005
From: jeff.pitman at gmail.com (Jeff Pitman)
Date: Sat, 31 Dec 2005 11:45:10 +0800
Subject: [Distutils] Setuptools projects with '-' in their names (was
	Re: some kind of problem with setuptools 0.6a9)
In-Reply-To: <5.1.1.6.0.20051230110343.03b9b010@mail.telecommunity.com>
References: <5.1.1.6.0.20051230110343.03b9b010@mail.telecommunity.com>
Message-ID: <6b9c17630512301945l246468d6sf7977966524070de@mail.gmail.com>

On 12/31/05, Phillip J. Eby <pje at telecommunity.com> wrote:
> At 10:04 AM 12/30/2005 -0500, Kevin Dangoor wrote:
> >The "tg-admin sql" command uses sqlobject which, in turn uses egg
> >metadata. Something I just noticed is that these commands are no
> >longer working on a project i have called "Wiki-20". They *used* to
> >work. Now they get a "DistributionNotFound" error for Wiki-20.
> >...
> >I had also tried to just do "pkg_resources.require("WIki-20")" from
> >the Python shell, but that didn't work either. This is new behavior
> >between a8 and a9. Let me know if there's anything else I can tell you
> >about the problem...

> Otherwise, recent versions of setuptools
> will think the '-' is the separator between the project name and version
> number, leading to problems like the ones described by Kevin above.

There is a standard regular expression that can be used to split out
N-V which is a pretty common thing in deb/rpm. (ie. python-foo). I use
the following in some py scripts to manage these pairings for my rpm
repo:

'(?i)(.*)-([^-]*)-(([^-]*)\.([^-]*)\.%s)(.*)'

Which is N-V-R.dist.repo.ext

The form that you might want would be:

'(.*)-([^-]*)'

Name-Version

This would allow Name to have -. Obviously, - cannot be used in Version.

Also, two side notes, 1) it's kind of annoying that package names
interchange between `-' and `_', one in the file name of the archive
and one used in the extracted directory name. 2) I've also see the use
of `_' in versioning, such as json.py. I also highly discourage that
practice.

--
-jeff

From pje at telecommunity.com  Sat Dec 31 06:30:17 2005
From: pje at telecommunity.com (Phillip J. Eby)
Date: Sat, 31 Dec 2005 00:30:17 -0500
Subject: [Distutils] Setuptools projects with '-' in their names (was
 Re: some kind of problem with setuptools 0.6a9)
In-Reply-To: <6b9c17630512301945l246468d6sf7977966524070de@mail.gmail.co
 m>
References: <5.1.1.6.0.20051230110343.03b9b010@mail.telecommunity.com>
	<5.1.1.6.0.20051230110343.03b9b010@mail.telecommunity.com>
Message-ID: <5.1.1.6.0.20051231002538.020ebd40@mail.telecommunity.com>

At 11:45 AM 12/31/2005 +0800, Jeff Pitman wrote:
>There is a standard regular expression that can be used to split out N-V 
>which is a pretty common thing in deb/rpm. (ie. python-foo). I use the 
>following in some py scripts to manage these pairings for my rpm repo: 
>'(?i)(.*)-([^-]*)-(([^-]*)\.([^-]*)\.%s)(.*)' Which is N-V-R.dist.repo.ext 
>The form that you might want would be: '(.*)-([^-]*)' Name-Version This 
>would allow Name to have -. Obviously, - cannot be used in Version. Also, 
>two side notes, 1) it's kind of annoying that package names interchange 
>between `-' and `_', one in the file name of the archive and one used in 
>the extracted directory name. 2) I've also see the use of `_' in 
>versioning, such as json.py. I also highly discourage that practice. -- -jeff

The format used by setuptools condenses non-alphanumeric runs to a single 
'-' in the project's actual name and version; these are then transposed to 
a '_' in filenames, with '-' to separate name parts.  The canonical form is 
'-', but the .egg-info directory format previously didn't transpose it to 
'_'.  This caused a problem when support for versioned .egg-info was added 
to support system packages.  I didn't realize that the setup-directory 
.egg-info wasn't being translated.  Anyway, this is fixed now in the SVN 
version; all of the other places in setuptools that worked with '-'/'_' 
filenames were using the same (correct) code to do the processing.  The 
relevant .egg-info code, however, was different, because it needed to *not* 
include the version number, so it couldn't use the standard dist.egg_name() 
method to generate a properly escaped filename including the version, 
Python version, etc.


From jeff.pitman at gmail.com  Sat Dec 31 09:35:19 2005
From: jeff.pitman at gmail.com (Jeff Pitman)
Date: Sat, 31 Dec 2005 16:35:19 +0800
Subject: [Distutils] Setuptools projects with '-' in their names (was
	Re: some kind of problem with setuptools 0.6a9)
In-Reply-To: <5.1.1.6.0.20051231002538.020ebd40@mail.telecommunity.com>
References: <5.1.1.6.0.20051230110343.03b9b010@mail.telecommunity.com>
	<5.1.1.6.0.20051231002538.020ebd40@mail.telecommunity.com>
Message-ID: <6b9c17630512310035y44701a8eva69a5d3a39e59be0@mail.gmail.com>

On 12/31/05, Phillip J. Eby <pje at telecommunity.com> wrote:
> At 11:45 AM 12/31/2005 +0800, Jeff Pitman wrote:
> >1) it's kind of annoying that package names interchange
> >between `-' and `_', one in the file name of the archive and one used in
> >the extracted directory name. 2) I've also see the use of `_' in
> >versioning, such as json.py. I also highly discourage that practice. -- -jeff
>
> The format used by setuptools condenses non-alphanumeric runs to a single
> '-' in the project's actual name and version; these are then transposed to
> a '_' in filenames, with '-' to separate name parts.

What's the purpose of transposition between `-' and `_'?

--
-jeff

From postmaster at renesas.com  Sat Dec 31 09:28:05 2005
From: postmaster at renesas.com (postmaster@renesas.com)
Date: Sat, 31 Dec 2005 17:28:05 +0900
Subject: [Distutils] =?utf-8?q?Undelivered_Message?=
Message-ID: <20051231083757.5966E1E4002@bag.python.org>

The email was deleted by system policy.
Attached file might be containing virus.

Connection From: 127.0.0.1
From: distutils-sig at python.org
To: stephen.clarke at superh.com, joern.rennecke at superh.com
Date: Sat, 31 Dec 2005 17:28:04 +0900
Subject: hi, ive a new mail address

--- Scan information follows ---

Virus Name: W32.Sober.X at mm!zip
File Attachment: mailtext.zip
Attachment Status: infected

Virus Name: W32.Sober.X at mm
File Attachment: mailtext.zip/File-packed_dataInfo.exe
Attachment Status: infected
--- File name Block information follows ---

File Attachment: mailtext.zip
Matching file name: Message is considered to be a mass-mailer.

The following text is Japanese.
??????????????????????????
?????????????????????????
????????



From pje at telecommunity.com  Sat Dec 31 16:19:54 2005
From: pje at telecommunity.com (Phillip J. Eby)
Date: Sat, 31 Dec 2005 10:19:54 -0500
Subject: [Distutils] Setuptools projects with '-' in their names (was
 Re: some kind of problem with setuptools 0.6a9)
In-Reply-To: <6b9c17630512310035y44701a8eva69a5d3a39e59be0@mail.gmail.co
 m>
References: <5.1.1.6.0.20051231002538.020ebd40@mail.telecommunity.com>
	<5.1.1.6.0.20051230110343.03b9b010@mail.telecommunity.com>
	<5.1.1.6.0.20051231002538.020ebd40@mail.telecommunity.com>
Message-ID: <5.1.1.6.0.20051231101659.034cf2d8@mail.telecommunity.com>

At 04:35 PM 12/31/2005 +0800, Jeff Pitman wrote:
>What's the purpose of transposition between `-' and `_'? -- -jeff

The distutils put '-' between different parts of the filenames it 
generates.  If dashes aren't escaped, it prevents the filename from being 
unambiguously parseable by a computer.  As it is, when looking for source 
distributions or win32.exe files, EasyInstall has to treat filenames as 
having every possible ambiguous interpretation, since the distutils 
currently generate such ambiguous filenames for non-setuptools projects.


From jeff.pitman at gmail.com  Sat Dec 31 16:26:42 2005
From: jeff.pitman at gmail.com (Jeff Pitman)
Date: Sat, 31 Dec 2005 23:26:42 +0800
Subject: [Distutils] Setuptools projects with '-' in their names (was
	Re: some kind of problem with setuptools 0.6a9)
In-Reply-To: <5.1.1.6.0.20051231101659.034cf2d8@mail.telecommunity.com>
References: <5.1.1.6.0.20051230110343.03b9b010@mail.telecommunity.com>
	<5.1.1.6.0.20051231002538.020ebd40@mail.telecommunity.com>
	<5.1.1.6.0.20051231101659.034cf2d8@mail.telecommunity.com>
Message-ID: <6b9c17630512310726r7d4fed51kca5d1d6d31cd509c@mail.gmail.com>

On 12/31/05, Phillip J. Eby <pje at telecommunity.com> wrote:
> At 04:35 PM 12/31/2005 +0800, Jeff Pitman wrote:
> >What's the purpose of transposition between `-' and `_'? -- -jeff
>
> The distutils put '-' between different parts of the filenames it
> generates.  If dashes aren't escaped, it prevents the filename from being
> unambiguously parseable by a computer.

What's ambiguous? Unless distutils allows `-' in Version, I believe
Name and Version can easily be extractable without transposition. Do
you have a specific example that's not obvious?

--
-jeff

From pje at telecommunity.com  Sat Dec 31 19:39:39 2005
From: pje at telecommunity.com (Phillip J. Eby)
Date: Sat, 31 Dec 2005 13:39:39 -0500
Subject: [Distutils] Setuptools projects with '-' in their names (was
 Re: some kind of problem with setuptools 0.6a9)
In-Reply-To: <6b9c17630512310726r7d4fed51kca5d1d6d31cd509c@mail.gmail.co
 m>
References: <5.1.1.6.0.20051231101659.034cf2d8@mail.telecommunity.com>
	<5.1.1.6.0.20051230110343.03b9b010@mail.telecommunity.com>
	<5.1.1.6.0.20051231002538.020ebd40@mail.telecommunity.com>
	<5.1.1.6.0.20051231101659.034cf2d8@mail.telecommunity.com>
Message-ID: <5.1.1.6.0.20051231133546.03b91210@mail.telecommunity.com>

At 11:26 PM 12/31/2005 +0800, Jeff Pitman wrote:
>What's ambiguous? Unless distutils allows `-' in Version, I believe Name 
>and Version can easily be extractable without transposition. Do you have a 
>specific example that's not obvious?

The distutils allow *anything* in both name and version; the arguments 
aren't checked in any way.  Take a quick skim through PyPI and see how many 
packages have a '-' in either the name or the version number.  I don't know 
of any that do both.

An obvious example: CherryPy 2.1.0-rc2 had a '-' in the version number.  I 
know of at least one other project that uses '-rc' suffixes on its 
versions, though its name escapes my recall at the moment.  There are also 
distutils projects that have numbers in their names - distinct from the 
actual version number.