From dholth at gmail.com  Sat Sep  1 19:53:19 2012
From: dholth at gmail.com (Daniel Holth)
Date: Sat, 1 Sep 2012 13:53:19 -0400
Subject: [Distutils] [tahoe-dev] wheel file signatures scheme
In-Reply-To: <504248FF.9000406@jacaranda.org>
References: <CAG8k2+64-5ojP6S86kWD1w9+w5iOVmH4HwG41gKHLb5DmWRp5g@mail.gmail.com>
	<504248FF.9000406@jacaranda.org>
Message-ID: <CAG8k2+5Ur7V_x13-VwJvxj5hhK3vLbfJ61Lg7Wga4bg0X=Sipg@mail.gmail.com>

The file "record" inside the archive is signed.
On Sep 1, 2012 1:42 PM, "David-Sarah Hopwood" <david-sarah at jacaranda.org>
wrote:

> On 22/08/12 03:52, Daniel Holth wrote:
> > I've made what I think is exciting progress on the digital signatures
> > design for wheel (updated built/binary packages for Python; intended
> > to replace egg). The insight is that we can overload the "extras"
> > syntax as a convenient way to mention the public key we expect:
> >
> > package[extra, ed25519=ouBJlTJJ4SJXoy8Bi1KRlewWLU6JW7HUXTgvU1YRuiA]
>
> I missed this when you first posted it. I like it a lot.
>
> The signature would be on the wheel binary package file?
>
> --
> David-Sarah Hopwood ?
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20120901/216597e4/attachment.html>

From david-sarah at jacaranda.org  Sat Sep  1 19:42:23 2012
From: david-sarah at jacaranda.org (David-Sarah Hopwood)
Date: Sat, 01 Sep 2012 18:42:23 +0100
Subject: [Distutils] [tahoe-dev] wheel file signatures scheme
In-Reply-To: <CAG8k2+64-5ojP6S86kWD1w9+w5iOVmH4HwG41gKHLb5DmWRp5g@mail.gmail.com>
References: <CAG8k2+64-5ojP6S86kWD1w9+w5iOVmH4HwG41gKHLb5DmWRp5g@mail.gmail.com>
Message-ID: <504248FF.9000406@jacaranda.org>

On 22/08/12 03:52, Daniel Holth wrote:
> I've made what I think is exciting progress on the digital signatures
> design for wheel (updated built/binary packages for Python; intended
> to replace egg). The insight is that we can overload the "extras"
> syntax as a convenient way to mention the public key we expect:
> 
> package[extra, ed25519=ouBJlTJJ4SJXoy8Bi1KRlewWLU6JW7HUXTgvU1YRuiA]

I missed this when you first posted it. I like it a lot.

The signature would be on the wheel binary package file?

-- 
David-Sarah Hopwood ?

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 554 bytes
Desc: OpenPGP digital signature
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20120901/7c6ef524/attachment.pgp>

From ron.duplain at gmail.com  Sun Sep  2 02:50:15 2012
From: ron.duplain at gmail.com (Ron DuPlain)
Date: Sat, 1 Sep 2012 20:50:15 -0400
Subject: [Distutils] Reliable means in site.py to reinitialize sys.path?
Message-ID: <CAJyZNZL7qZTY165NwyBZ-66_uNLgtpw4b6bEDCvPDekn7eVVkA@mail.gmail.com>

Hello,

Does site.py provide a reliable means to reinitialize sys.path of a
running process?

I see that Python 2.7 added site.getsitepackages() which gives the
paths needed to reinitialize explicitly, but it appears that
virtualenv's site.py does not include this change.
https://github.com/pypa/virtualenv/blob/5b904f88583d40954701c5471527fc07c7ae9056/virtualenv_embedded/site.py

To give some context, I'm interested in code reloading inside
virtualenvs on production systems where I would like to reduce
downtime to 0. Packages installed as non-zip eggs are added to a .pth
file, and attempts to reload(module) or even __import__ in a child
process do not get the new code since sys.path is still pointing at
the old paths given at the time the .pth file was read.

Thanks,

Ron

From vs at it.uu.se  Sun Sep  2 18:23:50 2012
From: vs at it.uu.se (Virgil Stokes)
Date: Sun, 02 Sep 2012 18:23:50 +0200
Subject: [Distutils] Two questions on setup.py/distutils.py
Message-ID: <50438816.10806@it.uu.se>

I have only recently joined this list --- hopefully my first mail to this list 
is acceptable.

*First*, is it possible to place comments in the distutils.cfg file? If yes, 
then how?

*Second*, I have been unable to install yappi (included in the PyPI list) on my 
Windows 7 (64-bit) platform with Python 2.7 (32-bit) and would greatly 
appreciate help on this installation.

Note, I have installed many other PyPI packages on this same system with no 
problems where I have cygwin, minGW (32-bit) and minGW (64-bit) gcc and g++ 
compilers installed and working properly. I have installed yappi on my Windows 
Vista (32-bit) platform with Python 2.7 and there were no problems. I have also 
studied the information posted by Greg Ward at 
http://docs.python.org/install/index.html.

Here are some of the results of my attempts to install yappi on the Windows 7 
system.

I looked at what compilers are available for building from my setup.py
file:

c:\Software\Python\yappi\yappi-0.62>python setup.py build --help-compiler
      List of available compilers:
        --compiler=bcpp     Borland C++ Compiler
        --compiler=cygwin   Cygwin port of GNU C Compiler for Win32
        --compiler=emx      EMX port of GNU C Compiler for OS/2
        --compiler=mingw32  Mingw32 port of GNU C Compiler for Win32
        --compiler=msvc     Microsoft Visual C++
        --compiler=unix     standard UNIX-style compiler

I then tried the following:

c:\Software\Python\yappi\yappi-0.62>python setup.py build --compiler=cygwin
       running build
       running build_py
       running build_ext
       building '_yappi' extension
       gcc -mcygwin -mdll -O -Wall -IC:\Python27\include -IC:\Python27\PC -c  
_yappi.c -o build\temp.win32-2.7\Release\_yappi.o
       error: command 'gcc' failed: No such file or directory

I then tried to be more specific,

c:\Software\Python\yappi\yappi-0.62>python setup.py build
       --compiler=c:\cygwin\bin\gcc.exe
       running build
       running build_py
       running build_ext
       building '_yappi' extension
       error: don't don't know how to compile C/C++ code on platform 'nt' with 
'c:\cygwin\bin\gcc.exe' compiler

I also tried many other possible (at least IMHO) solutions --- these also
failed. For example,

c:\Software\Python\yappi\yappi-0.62>python setup.py build --compiler=msvc
       running build
       running build_py
       running build_ext
       building '_yappi' extension
       error: Unable to find vcvarsall.bat

I then modified the c:\python27\Lib\distutils\distutils.cfg file and set
different compilers in it --- these also failed.

The bottom line --- all my trials (attempts) failed for one reason or another.

Any suggestions as to how to proceed? I will be glad to provide any additional
information on my trials.

--V

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20120902/fd091df8/attachment.html>

From ronaldoussoren at mac.com  Mon Sep  3 08:15:47 2012
From: ronaldoussoren at mac.com (Ronald Oussoren)
Date: Mon, 03 Sep 2012 08:15:47 +0200
Subject: [Distutils] wheel file signatures scheme
In-Reply-To: <CAG8k2+64-5ojP6S86kWD1w9+w5iOVmH4HwG41gKHLb5DmWRp5g@mail.gmail.com>
References: <CAG8k2+64-5ojP6S86kWD1w9+w5iOVmH4HwG41gKHLb5DmWRp5g@mail.gmail.com>
Message-ID: <8061CC66-8D92-4969-9BB0-1F91A6DB3EA7@mac.com>


On 22 Aug, 2012, at 4:52, Daniel Holth <dholth at gmail.com> wrote:

> I've made what I think is exciting progress on the digital signatures
> design for wheel (updated built/binary packages for Python; intended
> to replace egg). The insight is that we can overload the "extras"
> syntax as a convenient way to mention the public key we expect:
> 
> package[extra, ed25519=ouBJlTJJ4SJXoy8Bi1KRlewWLU6JW7HUXTgvU1YRuiA]

Why this hack instead of providing explict syntax for this?

Also the format doesn't seem to have any way to verify the validity of the signing key,
the documentation even says that "key distribution is out of scope for this spec". That's
odd for feature that's intended to add security. 

Why did you decide to use JSON Web Signatures instead of PGP signatures, or even
X.509 signatures? With the latter two the key distribution problem is already solved, and
PGP signatures are used a lot in the opensource world.

Ronald

> 
> http://wheel.readthedocs.org/en/latest/index.html#signed-wheel-files
> _______________________________________________
> Distutils-SIG maillist  -  Distutils-SIG at python.org
> http://mail.python.org/mailman/listinfo/distutils-sig

-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 4788 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20120903/71fb8a03/attachment.bin>

From dholth at gmail.com  Mon Sep  3 13:24:32 2012
From: dholth at gmail.com (Daniel Holth)
Date: Mon, 3 Sep 2012 07:24:32 -0400
Subject: [Distutils] wheel file signatures scheme
In-Reply-To: <8061CC66-8D92-4969-9BB0-1F91A6DB3EA7@mac.com>
References: <CAG8k2+64-5ojP6S86kWD1w9+w5iOVmH4HwG41gKHLb5DmWRp5g@mail.gmail.com>
	<8061CC66-8D92-4969-9BB0-1F91A6DB3EA7@mac.com>
Message-ID: <CAG8k2+6S4nMMasr6e4XjXKEjvan8JiXGCkcSDfUS_pL0USjkVw@mail.gmail.com>

On Mon, Sep 3, 2012 at 2:15 AM, Ronald Oussoren <ronaldoussoren at mac.com> wrote:
>
> On 22 Aug, 2012, at 4:52, Daniel Holth <dholth at gmail.com> wrote:
>
>> I've made what I think is exciting progress on the digital signatures
>> design for wheel (updated built/binary packages for Python; intended
>> to replace egg). The insight is that we can overload the "extras"
>> syntax as a convenient way to mention the public key we expect:
>>
>> package[extra, ed25519=ouBJlTJJ4SJXoy8Bi1KRlewWLU6JW7HUXTgvU1YRuiA]
>
> Why this hack instead of providing explict syntax for this?

It's because it can be made to be backwards-compatible. Declare that
your package provides the extra "ed25519=key" and
non-signature-verifying setuptools won't complain. I like the idea of
using array indexing [] for "which package? the one signed with a
particular key."

> Also the format doesn't seem to have any way to verify the validity of the signing key,
> the documentation even says that "key distribution is out of scope for this spec". That's
> odd for feature that's intended to add security.
>
> Why did you decide to use JSON Web Signatures instead of PGP signatures, or even
> X.509 signatures? With the latter two the key distribution problem is already solved, and
> PGP signatures are used a lot in the opensource world.

I wanted to use Ed25519, an elliptic-curve signatures system that has
32-byte public keys and a very fast (~100k cycle) implementation. I
wanted to be able to generate throwaway keys in 2.5 milliseconds and
use them to verify transfers between a build server and an
installation, and have keys represent the build server or the package,
not necessarily a person. I wanted to include the entire signing and
verifying system in Python instead of relying on an external package.

Currently so few Python packages are signed that popular opinion is
that signing isn't supported at all (it is). I think part of the
problem is that PGP is totally inconvenient. A new system gives us the
opportunity to try to make signing so effortless that publishers are
more likely to provide us with signed packages.

Key distribution is outside the scope of the wheel spec which just
says how signatures are included in the file. I think we are figuring
out some elegant solutions though, the first being "a PGP-signed
e-mail containing package[ed25519=key] lines" (distributed by the
publisher of a Python application that has dependencies). Signed wheel
archives include the verifying key as part of the signature. Maybe
PyPI will eventually provide a place for it too.

Thanks,

Daniel

From oscar.benjamin at bristol.ac.uk  Mon Sep  3 16:51:03 2012
From: oscar.benjamin at bristol.ac.uk (Oscar Benjamin)
Date: Mon, 3 Sep 2012 15:51:03 +0100
Subject: [Distutils] Two questions on setup.py/distutils.py
In-Reply-To: <50438816.10806@it.uu.se>
References: <50438816.10806@it.uu.se>
Message-ID: <CAHVvXxS_GH8mWnQhqwbTjLuUTFDSXD=mOHHzW4qACp1BKK3bdw@mail.gmail.com>

On 2 September 2012 17:23, Virgil Stokes <vs at it.uu.se> wrote:

>  I have only recently joined this list --- hopefully my first mail to
> this list is acceptable.
>
> *First*, is it possible to place comments in the distutils.cfg file? If
> yes, then how?
>

I don't know. There isn't much documentation for these things. Have you
tried just using a '#' symbol like in python code?


>
> *Second*, I have been unable to install yappi (included in the PyPI list)
> on my Windows 7 (64-bit) platform with Python 2.7 (32-bit) and would
> greatly appreciate help on this installation.
>
> Note, I have installed many other PyPI packages on this same system with
> no problems where I have cygwin, minGW (32-bit) and minGW (64-bit) gcc and
> g++ compilers installed and working properly. I have installed yappi on
> my Windows Vista (32-bit) platform with Python 2.7 and there were no
> problems. I have also studied the information posted by Greg Ward at
> http://docs.python.org/install/index.html.
>
> Here are some of the results of my attempts to install yappi on the
> Windows 7 system.
>
> I looked at what compilers are available for building from my setup.py
> file:
>
>    c:\Software\Python\yappi\yappi-0.62>python setup.py build
> --help-compiler
>      List of available compilers:
>        --compiler=bcpp     Borland C++ Compiler
>        --compiler=cygwin   Cygwin port of GNU C Compiler for Win32
>        --compiler=emx      EMX port of GNU C Compiler for OS/2
>        --compiler=mingw32  Mingw32 port of GNU C Compiler for Win32
>        --compiler=msvc     Microsoft Visual C++
>        --compiler=unix     standard UNIX-style compiler
>
> I then tried the following:
>
>    c:\Software\Python\yappi\yappi-0.62>python setup.py build
> --compiler=cygwin
>       running build
>       running build_py
>       running build_ext
>       building '_yappi' extension
>       gcc -mcygwin -mdll -O -Wall -IC:\Python27\include -IC:\Python27\PC
> -c  _yappi.c -o build\temp.win32-2.7\Release\_yappi.o
>       error: command 'gcc' failed: No such file or directory
>
>
Maybe I'm missing something but are you running this from within cygwin or
just from a normal cmd.exe window? It looks as if you're trying to use the
cygwin compiler from outside of cygwin.


> I then tried to be more specific,
>
>    c:\Software\Python\yappi\yappi-0.62>python setup.py build
>       --compiler=c:\cygwin\bin\gcc.exe
>       running build
>       running build_py
>       running build_ext
>       building '_yappi' extension
>       error: don't don't know how to compile C/C++ code on platform 'nt'
> with 'c:\cygwin\bin\gcc.exe' compiler
>
>
It says platform 'nt' maening that you are not in cygwin. You should be
using the mingw compiler to compile outside of cygwin or the cygwin
compiler compiler to compile inside cygwin. When you installed cygwin it
should have given you an icon that you can click to open the cygwin shell.
This cygwin shell looks like the ordinary cmd.exe window but it runs a bash
shell and the programs that you run will be run inside cygwin.


> I also tried many other possible (at least IMHO) solutions --- these also
> failed. For example,
>
>    c:\Software\Python\yappi\yappi-0.62>python setup.py build
> --compiler=msvc
>       running build
>       running build_py
>       running build_ext
>       building '_yappi' extension
>       error: Unable to find vcvarsall.bat
>
>
I think this error comes from using a version of msvc that is too old for
the python version that you are trying to install with. You need to have
the version that matches your python version. See here:
http://stackoverflow.com/questions/2676763/what-version-of-visual-studio-is-this-python-compiled-with


> I then modified the c:\python27\Lib\distutils\distutils.cfg file and set
> different compilers in it --- these also failed.
>
> The bottom line --- all my trials (attempts) failed for one reason or
> another.
>

> Any suggestions as to how to proceed? I will be glad to provide any
> additional
> information on my trials.
>

There are three possible ways to fix this:
1) Get the right version of msvc (see link above)
2) Use --compiler=mingw in the normal cmd.exe window
3) Use --compiler=cygwin from inside the cygwin environment

Oscar
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20120903/89603ef2/attachment.html>

From lele at metapensiero.it  Wed Sep  5 11:55:52 2012
From: lele at metapensiero.it (Lele Gaifax)
Date: Wed, 05 Sep 2012 11:55:52 +0200
Subject: [Distutils] Struggling with a modular buildout configuration
References: <87lihgp6gn.fsf@metapensiero.it>
	<loom.20120818T204418-92@post.gmane.org>
	<87mx1rghmv.fsf@metapensiero.it>
Message-ID: <87sjawbxp3.fsf@metapensiero.it>

Lele Gaifax <lele at metapensiero.it> writes:

>> can you provide an example that would illustrate the problem? The file
>> you submitted fails because projects do not include setup.py. I would
>> advise to use zc.buildout 1.6.0 and see if issue persists.
>
> Sure, I added minimalistic setup.py to both subprojects:
>
>   http://artiemestieri.tn.it/~lele/test-buildout-2.tar.gz

Hi Domen,

just to shake a doubt, was that enough for you to exercise the problem,
or can/should I provide more details?

Thank you,
ciao, lele.
-- 
nickname: Lele Gaifax | Quando vivr? di quello che ho pensato ieri
real: Emanuele Gaifas | comincer? ad aver paura di chi mi copia.
lele at metapensiero.it  |                 -- Fortunato Depero, 1929.


From dholth at gmail.com  Thu Sep  6 18:34:54 2012
From: dholth at gmail.com (Daniel Holth)
Date: Thu, 6 Sep 2012 12:34:54 -0400
Subject: [Distutils] add hash algorithm agility to RECORD
Message-ID: <CAG8k2+4XBF3XSnsEROrgq=OOkrQ7fKO231+3jXhg6B1TEgpnig@mail.gmail.com>

Proposed edits to https://bitbucket.org/dholth/python-peps/changeset/9c26fa50

In wheel I use urlsafe_b64encode_nopad() which omits the trailing =
characters, but although very easy to implement isn't included in the
stdlib. In this spec I use the stdlib urlsafe_b64encode().

diff -r 23f9640c2020 -r 9c26fa508424 pep-0376.txt
--- a/pep-0376.txt      Thu Sep 06 12:09:58 2012 -0400
+++ b/pep-0376.txt      Thu Sep 06 12:24:40 2012 -0400
@@ -218,11 +218,16 @@

   - an absolute path, using the local platform separator

-- the **MD5** hash of the file, encoded in hex. Notice that `pyc` and `pyo`
-  generated files don't have any hash because they are automatically produced
-  from `py` files. So checking the hash of the corresponding `py` file is
-  enough to decide if the file and its associated `pyc` or `pyo` files have
-  changed.
+- a hash of the file's contents.
+  Notice that `pyc` and `pyo` generated files don't have any hash because
+  they are automatically produced from `py` files. So checking the hash
+  of the corresponding `py` file is enough to decide if the file and
+  its associated `pyc` or `pyo` files have changed.
+
+  The hash is either the empty string, the **MD5** hash of
+  the file, encoded in hex, or the hash algorithm as named in
+  ``hashlib.algorithms``, followed by the equals character ``=``,
+  followed by the hash digest as encoded with ``urlsafe_b64encode``.

 - the file's size in bytes

@@ -391,9 +396,9 @@

 And following methods:

-- ``get_installed_files(local=False)`` -> iterator of (path, md5, size)
+- ``get_installed_files(local=False)`` -> iterator of (path, hash, size)

-  Iterates over the `RECORD` entries and return a tuple ``(path, md5, size)``
+  Iterates over the `RECORD` entries and return a tuple ``(path, hash, size)``
   for each line. If ``local`` is ``True``, the path is transformed into a
   local absolute path. Otherwise the raw value from `RECORD` is returned.

From p.f.moore at gmail.com  Thu Sep  6 22:54:31 2012
From: p.f.moore at gmail.com (Paul Moore)
Date: Thu, 6 Sep 2012 21:54:31 +0100
Subject: [Distutils] Some clarifications and/or corrections to PEP 376
Message-ID: <CACac1F9mAYzJGdEBPS55Z56m732sSh8C6ma9M0bUEKN5s126xg@mail.gmail.com>

I've been looking at the details of PEP 376 (Database of Installed
Python Distributions) and there are a couple of aspects that I don't
think work properly alongside PEP 370 (Per-user site-packages
directory)..

1. The dist-info directory for a distribution is stated as being
"located in the site-packages directory".

It's not clear how this is intended to work in a PEP 370 world with
multiple site-packages. What I propose is that the description be
changed to be worded in terms of sysconfig-style locations: the
dist-info directory is located in whichever of purelib or platlib is
used by the distribution. When the distribution uses both, purelib is
preferred, when it uses neither (!) purelib is used. In nearly all
cases, this is the same as currently. The exceptions are posix_home
(where the directory name isn't "site-packages" but its function is
the same), posix_prefix (where purelib and platlib differ, and PEP 376
is currently ambiguous as to which is implied), and any custom schemes
that might be created (where PEP 376 is silent, and this proposal has
the benefit of at least being specific). I do not believe this changes
any actual practice - as far as I have been able to determine any code
using dist-info at the moment follows this proposal in the corner
cases where it differs from PEP 376.

2. File paths in RECORD are stated as being relative to the "base
location" as long as the files sit under that location, or under the
"install prefix" where that is above the base location. That's a messy
but precise way of saying you can go up a bit as long as you stay
within sys.prefix. (Other paths are platform format absolute paths,
which is an ugly discrepancy but the only really viable option).
Again, this doesn't work as I'd expect it should in a PEP 370 user
site-packages.

In this case, the answer isn't quite as clear. I would suggest,
looking at the schemes defined in sysconfig, allowing any files
located in one of the defined sysconfig paths (for the current scheme)
to be recorded as relative paths. The paths should be relative to
whichever of purelib and platlib was chosen in (1) above. The
exception is that where pureXXX and platXXX do not use the same base
(posix_prefix and possibly custom schemas) it should not be allowed to
have a relative path from a pureXXX to a platXXX or vice versa. That
equates to something slightly stricter than the current scheme
(although not in a way that's likely to be used in practice), but
using the appropriate one of {base} or {userbase} rather than
sys.prefix. As noted, the two exceptions are custom schemes (where PEP
376 is silent and would probably end up effectively mandating platform
absolute paths throughout) and posix_prefix, where
platlib/platinclude/platstdlib are located under {platbase} rather
than {base} and the current proposal would probably allow relative
paths to be used a little more often that PEP 376 allows. Current
practice that I've seen here is not as clear cut, but mostly in the
sense that code doesn't consider the corner cases at all, and probably
only follows PEP 376 by luck, if at all...

Comments would be appreciated. I'm pretty happy that my proposal for
(1) is an improvement. With (2) I'm concerned about the posix_prefix
case, particularly as I'm a Windows developer and my understanding of
Python's layout on POSIX is limited. But I honestly don't think it's
possible to do much better than my proposal, and I do think that PEP
376 as it stands now is distinctly worse because it doesn't consider
PEP 370 or the posix_prefix case.

Also, while these proposals are not 100% backward compatible, I don't
believe they change behaviour in the most common scenarios, and given
that PEP 376 currently has limited adoption (mainly in distutils2, I
believe) I'd suggest that the minor compatibility break now is better
than keeping the current system until it's more widely adopted and
there are more users to impact. Obviously I'd be looking for views
from Tarek and ?ric in particular here, as it's their code in
distutils2 that would be directly impacted.

Paul.

From dholth at gmail.com  Thu Sep  6 23:18:25 2012
From: dholth at gmail.com (Daniel Holth)
Date: Thu, 6 Sep 2012 17:18:25 -0400
Subject: [Distutils] Some clarifications and/or corrections to PEP 376
In-Reply-To: <CACac1F9mAYzJGdEBPS55Z56m732sSh8C6ma9M0bUEKN5s126xg@mail.gmail.com>
References: <CACac1F9mAYzJGdEBPS55Z56m732sSh8C6ma9M0bUEKN5s126xg@mail.gmail.com>
Message-ID: <CAG8k2+5ZLR-CvDYOGnhx1nxb6b8n1H0aAQuZALsT4kM45uCHjQ@mail.gmail.com>

I prefer that paths in record are always relative to the parent directory
of record (usually site-packages) unless a relative path would not work
(drive letter boundaries). Consider installing and then chroot.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20120906/d7e31772/attachment.html>

From p.f.moore at gmail.com  Thu Sep  6 23:33:53 2012
From: p.f.moore at gmail.com (Paul Moore)
Date: Thu, 6 Sep 2012 22:33:53 +0100
Subject: [Distutils] Some clarifications and/or corrections to PEP 376
In-Reply-To: <CAG8k2+5ZLR-CvDYOGnhx1nxb6b8n1H0aAQuZALsT4kM45uCHjQ@mail.gmail.com>
References: <CACac1F9mAYzJGdEBPS55Z56m732sSh8C6ma9M0bUEKN5s126xg@mail.gmail.com>
	<CAG8k2+5ZLR-CvDYOGnhx1nxb6b8n1H0aAQuZALsT4kM45uCHjQ@mail.gmail.com>
Message-ID: <CACac1F_Xnqv5Y-CvWirawZSkN5mH8n+F8jT=HD2vi6scV6qVVQ@mail.gmail.com>

On 6 September 2012 22:18, Daniel Holth <dholth at gmail.com> wrote:
> I prefer that paths in record are always relative to the parent directory of
> record (usually site-packages) unless a relative path would not work (drive
> letter boundaries). Consider installing and then chroot.

That's a much simpler rule and I agree in principle that it's
preferable. The problems are:

1. It's much further away from what PEP 376 specifies. I will accept
this happily if there is general agreement that it's OK, but I
preferred to start with a more conservative suggestion :-)
2. Do you really want long strings of ../../.. if a distribution
specifies a file to be installed in an absolute location (possible,
although probably not well supported by current tools). Consider a
package that installs something to /var/python (I'm not a Unix user,
so this may be an unconvincing example, but I understand that similar
things *are* possible). If Python is in /usr, you'd have RECORD with
something like ../../../../var/python. I don't have enough Unix
experience to know if anyone would care about this. Of course what I
know about chroot implies this would break in that scenario anyway...

As I say, if the Unix people are OK with it, I'm happy to go this way.

Of course, I'd be happy to mandate that files in a distribution should
never be installed anywhere that isn't defined as one of the sysconfig
defined paths - but suggesting that would be sure to start a much
bigger debate that I don't want to get into as I don't have the
expertise to referee it.

Paul.

From dholth at gmail.com  Thu Sep  6 23:38:58 2012
From: dholth at gmail.com (Daniel Holth)
Date: Thu, 6 Sep 2012 17:38:58 -0400
Subject: [Distutils] Some clarifications and/or corrections to PEP 376
In-Reply-To: <CACac1F_Xnqv5Y-CvWirawZSkN5mH8n+F8jT=HD2vi6scV6qVVQ@mail.gmail.com>
References: <CACac1F9mAYzJGdEBPS55Z56m732sSh8C6ma9M0bUEKN5s126xg@mail.gmail.com>
	<CAG8k2+5ZLR-CvDYOGnhx1nxb6b8n1H0aAQuZALsT4kM45uCHjQ@mail.gmail.com>
	<CACac1F_Xnqv5Y-CvWirawZSkN5mH8n+F8jT=HD2vi6scV6qVVQ@mail.gmail.com>
Message-ID: <CAG8k2+4FJmrjYDYKHzVN3J=rizznsS=F=i2xU9kb5upA0Ks=Og@mail.gmail.com>

On Sep 6, 2012 5:33 PM, "Paul Moore" <p.f.moore at gmail.com> wrote:
>
> On 6 September 2012 22:18, Daniel Holth <dholth at gmail.com> wrote:
> > I prefer that paths in record are always relative to the parent
directory of
> > record (usually site-packages) unless a relative path would not work
(drive
> > letter boundaries). Consider installing and then chroot.
>
> That's a much simpler rule and I agree in principle that it's
> preferable. The problems are:
>
> 1. It's much further away from what PEP 376 specifies. I will accept
> this happily if there is general agreement that it's OK, but I
> preferred to start with a more conservative suggestion :-)
> 2. Do you really want long strings of ../../.. if a distribution
> specifies a file to be installed in an absolute location (possible,
> although probably not well supported by current tools). Consider a
> package that installs something to /var/python (I'm not a Unix user,
> so this may be an unconvincing example, but I understand that similar
> things *are* possible). If Python is in /usr, you'd have RECORD with
> something like ../../../../var/python. I don't have enough Unix
> experience to know if anyone would care about this. Of course what I
> know about chroot implies this would break in that scenario anyway...

Fyi in pip all the installed - files paths are relative to
installed-files.txt itself - even more dots.

> As I say, if the Unix people are OK with it, I'm happy to go this way.
>
> Of course, I'd be happy to mandate that files in a distribution should
> never be installed anywhere that isn't defined as one of the sysconfig
> defined paths - but suggesting that would be sure to start a much
> bigger debate that I don't want to get into as I don't have the
> expertise to referee it.
>
> Paul.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20120906/2a89589d/attachment-0001.html>

From barry at python.org  Fri Sep  7 04:53:38 2012
From: barry at python.org (Barry Warsaw)
Date: Thu, 6 Sep 2012 22:53:38 -0400
Subject: [Distutils] Latest z3c.recipe.tag upload to PyPI is broken
Message-ID: <20120906225338.261f51ff@resist.wooz.org>

The latest upload of z3c.recipe.tag appears to be broken.  I'm sending this
message to zope-dev at zope.org because that's the maintainer_email from the
setup.py, and I can't find any indication of where to submit bug reports.  I'm
also CC'ing distutils-sig, just 'cause. :)

Doing a buildout of the Mailman 3 trunk crashes today:

% bin/buildout
Develop: '/home/barry/projects/mailman/3.0/.'
[...]
Got z3c.recipe.scripts 1.0.1.
Getting distribution for 'zc.recipe.egg>=1.3.0'.
Got zc.recipe.egg 1.3.2.
Getting distribution for 'z3c.recipe.tag'.
error: /tmp/easy_install-ganUUb/z3c.recipe.tag-0.5/CHANGES.txt: No such file or directory
An error occurred when trying to install z3c.recipe.tag 0.5. Look above this message for any errors that were output by easy_install.
While:
  Installing.
  Getting section tags.
  Initializing section tags.
  Installing recipe z3c.recipe.tag.
  Getting distribution for 'z3c.recipe.tag'.
Error: Couldn't install: z3c.recipe.tag 0.5

PyPI says a new version of z3c.recipe.tags was uploaded today (2012-09-06).
It looks like a file was missing from the zip.

Cheers,
-Barry
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 836 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20120906/9714e376/attachment.pgp>

From p.f.moore at gmail.com  Fri Sep  7 12:58:02 2012
From: p.f.moore at gmail.com (Paul Moore)
Date: Fri, 7 Sep 2012 11:58:02 +0100
Subject: [Distutils] add hash algorithm agility to RECORD
In-Reply-To: <CAG8k2+4XBF3XSnsEROrgq=OOkrQ7fKO231+3jXhg6B1TEgpnig@mail.gmail.com>
References: <CAG8k2+4XBF3XSnsEROrgq=OOkrQ7fKO231+3jXhg6B1TEgpnig@mail.gmail.com>
Message-ID: <CACac1F_K1Bt0+rPxt7f7sqGWcHHJQTJboq8V6Gu6DtvD6frXLg@mail.gmail.com>

On 6 September 2012 17:34, Daniel Holth <dholth at gmail.com> wrote:
> Proposed edits to https://bitbucket.org/dholth/python-peps/changeset/9c26fa50
>
> In wheel I use urlsafe_b64encode_nopad() which omits the trailing =
> characters, but although very easy to implement isn't included in the
> stdlib. In this spec I use the stdlib urlsafe_b64encode().

Why urlsafe_b64encode, rather than just hexdigest which is what is used for md5?

Paul.

From dholth at gmail.com  Fri Sep  7 12:59:58 2012
From: dholth at gmail.com (Daniel Holth)
Date: Fri, 7 Sep 2012 06:59:58 -0400
Subject: [Distutils] add hash algorithm agility to RECORD
In-Reply-To: <CACac1F_K1Bt0+rPxt7f7sqGWcHHJQTJboq8V6Gu6DtvD6frXLg@mail.gmail.com>
References: <CAG8k2+4XBF3XSnsEROrgq=OOkrQ7fKO231+3jXhg6B1TEgpnig@mail.gmail.com>
	<CACac1F_K1Bt0+rPxt7f7sqGWcHHJQTJboq8V6Gu6DtvD6frXLg@mail.gmail.com>
Message-ID: <CAG8k2+6sL0ytKEAHT=0ktEXXv1K+OLYkMFgikxGJB-3c94wz0A@mail.gmail.com>

It's shorter, and it's used extensively in the digital signature format I'm
using.
On Sep 7, 2012 6:58 AM, "Paul Moore" <p.f.moore at gmail.com> wrote:

> On 6 September 2012 17:34, Daniel Holth <dholth at gmail.com> wrote:
> > Proposed edits to
> https://bitbucket.org/dholth/python-peps/changeset/9c26fa50
> >
> > In wheel I use urlsafe_b64encode_nopad() which omits the trailing =
> > characters, but although very easy to implement isn't included in the
> > stdlib. In this spec I use the stdlib urlsafe_b64encode().
>
> Why urlsafe_b64encode, rather than just hexdigest which is what is used
> for md5?
>
> Paul.
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20120907/4f7c4cf6/attachment.html>

From dholth at gmail.com  Fri Sep  7 19:58:39 2012
From: dholth at gmail.com (Daniel Holth)
Date: Fri, 7 Sep 2012 13:58:39 -0400
Subject: [Distutils] add hash algorithm agility to RECORD
In-Reply-To: <CAG8k2+6sL0ytKEAHT=0ktEXXv1K+OLYkMFgikxGJB-3c94wz0A@mail.gmail.com>
References: <CAG8k2+4XBF3XSnsEROrgq=OOkrQ7fKO231+3jXhg6B1TEgpnig@mail.gmail.com>
	<CACac1F_K1Bt0+rPxt7f7sqGWcHHJQTJboq8V6Gu6DtvD6frXLg@mail.gmail.com>
	<CAG8k2+6sL0ytKEAHT=0ktEXXv1K+OLYkMFgikxGJB-3c94wz0A@mail.gmail.com>
Message-ID: <CAG8k2+5bp8cVG6pAwiyFtbF60Pc_EXJFxu9pj9FSp1ZEj08V9w@mail.gmail.com>

On Fri, Sep 7, 2012 at 6:59 AM, Daniel Holth <dholth at gmail.com> wrote:
> It's shorter, and it's used extensively in the digital signature format I'm
> using.
>
> On Sep 7, 2012 6:58 AM, "Paul Moore" <p.f.moore at gmail.com> wrote:
>>
>> On 6 September 2012 17:34, Daniel Holth <dholth at gmail.com> wrote:
>> > Proposed edits to
>> > https://bitbucket.org/dholth/python-peps/changeset/9c26fa50
>> >
>> > In wheel I use urlsafe_b64encode_nopad() which omits the trailing =
>> > characters, but although very easy to implement isn't included in the
>> > stdlib. In this spec I use the stdlib urlsafe_b64encode().
>>
>> Why urlsafe_b64encode, rather than just hexdigest which is what is used
>> for md5?
>>
>> Paul.

On the other hand, why encode it at all? CSV supports shoving the raw
bytes in there, no problem. ;-)

From p.f.moore at gmail.com  Fri Sep  7 21:36:39 2012
From: p.f.moore at gmail.com (Paul Moore)
Date: Fri, 7 Sep 2012 20:36:39 +0100
Subject: [Distutils] add hash algorithm agility to RECORD
In-Reply-To: <CAG8k2+5bp8cVG6pAwiyFtbF60Pc_EXJFxu9pj9FSp1ZEj08V9w@mail.gmail.com>
References: <CAG8k2+4XBF3XSnsEROrgq=OOkrQ7fKO231+3jXhg6B1TEgpnig@mail.gmail.com>
	<CACac1F_K1Bt0+rPxt7f7sqGWcHHJQTJboq8V6Gu6DtvD6frXLg@mail.gmail.com>
	<CAG8k2+6sL0ytKEAHT=0ktEXXv1K+OLYkMFgikxGJB-3c94wz0A@mail.gmail.com>
	<CAG8k2+5bp8cVG6pAwiyFtbF60Pc_EXJFxu9pj9FSp1ZEj08V9w@mail.gmail.com>
Message-ID: <CACac1F8qYLm5KfXBi7rMsOMQxFdjZ68eCb7HkWxN=U+KmUTJnQ@mail.gmail.com>

On 7 September 2012 18:58, Daniel Holth <dholth at gmail.com> wrote:
> On Fri, Sep 7, 2012 at 6:59 AM, Daniel Holth <dholth at gmail.com> wrote:
>> It's shorter, and it's used extensively in the digital signature format I'm
>> using.
>
> On the other hand, why encode it at all? CSV supports shoving the raw
> bytes in there, no problem. ;-)

CSV is text format. Feed it binary at your own peril :-)

I've just spent most of the day fighting a (non-Python) system that is
going barmy because of line ending funnies in data within CSV fields.
That was bad enough - I have no intention of ever having to deal with
a CSV file with binary data in it...

Paul.

From m.van.rees at zestsoftware.nl  Fri Sep  7 23:52:07 2012
From: m.van.rees at zestsoftware.nl (Maurits van Rees)
Date: Fri, 07 Sep 2012 23:52:07 +0200
Subject: [Distutils] Latest z3c.recipe.tag upload to PyPI is broken
In-Reply-To: <20120906225338.261f51ff@resist.wooz.org>
References: <20120906225338.261f51ff@resist.wooz.org>
Message-ID: <k2dqa7$vo9$1@ger.gmane.org>

Op 07-09-12 04:53, Barry Warsaw schreef:
> The latest upload of z3c.recipe.tag appears to be broken.  I'm sending this
> message to zope-dev at zope.org because that's the maintainer_email from the
> setup.py, and I can't find any indication of where to submit bug reports.  I'm
> also CC'ing distutils-sig, just 'cause. :)
>
> Doing a buildout of the Mailman 3 trunk crashes today:
>
> % bin/buildout
> Develop: '/home/barry/projects/mailman/3.0/.'
> [...]
> Got z3c.recipe.scripts 1.0.1.
> Getting distribution for 'zc.recipe.egg>=1.3.0'.
> Got zc.recipe.egg 1.3.2.
> Getting distribution for 'z3c.recipe.tag'.
> error: /tmp/easy_install-ganUUb/z3c.recipe.tag-0.5/CHANGES.txt: No such file or directory
> An error occurred when trying to install z3c.recipe.tag 0.5. Look above this message for any errors that were output by easy_install.
> While:
>    Installing.
>    Getting section tags.
>    Initializing section tags.
>    Installing recipe z3c.recipe.tag.
>    Getting distribution for 'z3c.recipe.tag'.
> Error: Couldn't install: z3c.recipe.tag 0.5
>
> PyPI says a new version of z3c.recipe.tags was uploaded today (2012-09-06).
> It looks like a file was missing from the zip.

Fixed and 0.6 released by Christian Theune.  Thanks!


-- 
Maurits van Rees: http://maurits.vanrees.org/
Zest Software: http://zestsoftware.nl


From aclark at aclark.net  Sat Sep  8 05:19:36 2012
From: aclark at aclark.net (Alex Clark)
Date: Fri, 7 Sep 2012 23:19:36 -0400
Subject: [Distutils] wheel file signatures scheme
References: <CAG8k2+64-5ojP6S86kWD1w9+w5iOVmH4HwG41gKHLb5DmWRp5g@mail.gmail.com>
	<8061CC66-8D92-4969-9BB0-1F91A6DB3EA7@mac.com>
	<CAG8k2+6S4nMMasr6e4XjXKEjvan8JiXGCkcSDfUS_pL0USjkVw@mail.gmail.com>
Message-ID: <k2edg8$vnp$1@ger.gmane.org>

On 2012-09-03 11:24:32 +0000, Daniel Holth said:

> On Mon, Sep 3, 2012 at 2:15 AM, Ronald Oussoren 
> <ronaldoussoren at mac.com> wrote:
>> 
>> On 22 Aug, 2012, at 4:52, Daniel Holth 
>> <dholth at gmail.com> wrote:
>> 
>>> I've made what I think is exciting progress on the digital signatures
>>> design for wheel (updated built/binary packages for Python; intended
>>> to replace egg). The insight is that we can overload the "extras"
>>> syntax as a convenient way to mention the public key we expect:
>>> 
>>> package[extra, ed25519=ouBJlTJJ4SJXoy8Bi1KRlewWLU6JW7HUXTgvU1YRuiA]
>> 
>> Why this hack instead of providing explict syntax for this?
> 
> It's because it can be made to be backwards-compatible. Declare that
> your package provides the extra "ed25519=key" and
> non-signature-verifying setuptools won't complain. I like the idea of
> using array indexing [] for "which package? the one signed with a
> particular key."
> 
>> Also the format doesn't seem to have any way to verify the validity of 
>> the signing key,
>> the documentation even says that "key distribution is out of scope for 
>> this spec". That's
>> odd for feature that's intended to add security.
>> 
>> Why did you decide to use JSON Web Signatures instead of PGP 
>> signatures, or even
>> X.509 signatures? With the latter two the key distribution problem is 
>> already solved, and
>> PGP signatures are used a lot in the opensource world.
> 
> I wanted to use Ed25519, an elliptic-curve signatures system that has
> 32-byte public keys and a very fast (~100k cycle) implementation. I
> wanted to be able to generate throwaway keys in 2.5 milliseconds and
> use them to verify transfers between a build server and an
> installation, and have keys represent the build server or the package,
> not necessarily a person. I wanted to include the entire signing and
> verifying system in Python instead of relying on an external package.
> 
> Currently so few Python packages are signed that popular opinion is
> that signing isn't supported at all (it is). I think part of the
> problem is that PGP is totally inconvenient. A new system gives us the
> opportunity to try to make signing so effortless that publishers are
> more likely to provide us with signed packages.
> 
> Key distribution is outside the scope of the wheel spec which just
> says how signatures are included in the file. I think we are figuring
> out some elegant solutions though, the first being "a PGP-signed
> e-mail containing package[ed25519=key] lines" (distributed by the
> publisher of a Python application that has dependencies). Signed wheel
> archives include the verifying key as part of the signature. Maybe
> PyPI will eventually provide a place for it too.


Good answer :-)

> 
> Thanks,
> 
> Daniel


-- 
Alex Clark ? http://pythonpackages.com



From aclark at aclark.net  Sat Sep  8 05:21:18 2012
From: aclark at aclark.net (Alex Clark)
Date: Fri, 7 Sep 2012 23:21:18 -0400
Subject: [Distutils] Struggling with a modular buildout configuration
References: <87lihgp6gn.fsf@metapensiero.it>
	<loom.20120818T204418-92@post.gmane.org>
	<87mx1rghmv.fsf@metapensiero.it> <87sjawbxp3.fsf@metapensiero.it>
Message-ID: <k2edjd$1a9$1@ger.gmane.org>

On 2012-09-05 09:55:52 +0000, Lele Gaifax said:

> Lele Gaifax <lele at metapensiero.it> writes:
> 
>>> can you provide an example that would illustrate the problem? The file
>>> you submitted fails because projects do not include setup.py. I would
>>> advise to use zc.buildout 1.6.0 and see if issue persists.
>> 
>> Sure, I added minimalistic setup.py to both subprojects:
>> 
>> http://artiemestieri.tn.it/~lele/test-buildout-2.tar.gz
> 
> Hi Domen,
> 
> just to shake a doubt, was that enough for you to exercise the problem,
> or can/should I provide more details?


Did zc.buildout 1.6.3 help at all?


> 
> Thank you,
> ciao, lele.


-- 
Alex Clark ? http://pythonpackages.com



From barry at python.org  Sat Sep  8 18:58:54 2012
From: barry at python.org (Barry Warsaw)
Date: Sat, 8 Sep 2012 12:58:54 -0400
Subject: [Distutils] Latest z3c.recipe.tag upload to PyPI is broken
In-Reply-To: <k2dqa7$vo9$1@ger.gmane.org>
References: <20120906225338.261f51ff@resist.wooz.org>
	<k2dqa7$vo9$1@ger.gmane.org>
Message-ID: <20120908125854.4d24e736@limelight.wooz.org>

On Sep 07, 2012, at 11:52 PM, Maurits van Rees wrote:

>> PyPI says a new version of z3c.recipe.tags was uploaded today (2012-09-06).
>> It looks like a file was missing from the zip.
>
>Fixed and 0.6 released by Christian Theune.  Thanks!

Looks great, thanks!

-Barry
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 836 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20120908/e28ed2b1/attachment.pgp>

From lele at metapensiero.it  Mon Sep 10 15:04:30 2012
From: lele at metapensiero.it (Lele Gaifax)
Date: Mon, 10 Sep 2012 15:04:30 +0200
Subject: [Distutils] Struggling with a modular buildout configuration
References: <87lihgp6gn.fsf@metapensiero.it>
	<loom.20120818T204418-92@post.gmane.org>
	<87mx1rghmv.fsf@metapensiero.it> <87sjawbxp3.fsf@metapensiero.it>
	<k2edjd$1a9$1@ger.gmane.org>
Message-ID: <877gs2jag1.fsf@metapensiero.it>

Alex Clark <aclark at aclark.net> writes:

>
> Did zc.buildout 1.6.3 help at all?
>

No, it exhibits the very same behaviour as described in my previous
post, that is, the configuration bits from proj2 clobber those coming
from proj1. While it's enterely possible I'm misusing the extends
functionality, it seems something odd is happening in the
_update_section() function...

Thanks and bye,
lele.
-- 
nickname: Lele Gaifax | Quando vivr? di quello che ho pensato ieri
real: Emanuele Gaifas | comincer? ad aver paura di chi mi copia.
lele at metapensiero.it  |                 -- Fortunato Depero, 1929.


From donald.stufft at gmail.com  Tue Sep 11 20:35:31 2012
From: donald.stufft at gmail.com (Donald Stufft)
Date: Tue, 11 Sep 2012 14:35:31 -0400
Subject: [Distutils] PEP345 - Distutils2 - Cannot pin to exact version
Message-ID: <86B0D55791FB4654A1A219CE1DA82F0E@gmail.com>

I was digging through PEP386 & PEP345 today, and I noticed something odd about the wording of PEP345. 

It states:

    When a version is provided, it always includes all versions that starts with the same value. For
    example the "2.5" version of Python will include versions like "2.5.2" or "2.5.3". Pre and post
    releases in that case are excluded. So in our example, versions like "2.5a1" are not included
    when "2.5" is used. If the first version of the range is required, it has to be explicitly given. In
    our example, it will be "2.5.0".

It also states:

    In that case, "2.5.0" will have to be explicitly used to avoid any confusion between the "2.5"
    notation that represents the full range. It is a recommended practice to use schemes of the
    same length for a series to completely avoid this problem.

This effectively translates to an inability to pin to an exact version. Even in the case of specifying
== it checks that the version "starts with" the value you selected. So if you pin to "2.5", and the
author then releases "2.5.1", that will count as ==2.5. If you try to then pin to "2.5.0", and the
author releases "2.5.0.1", then that will count as ==2.5.0.

Essentially this translates to:

    ==2.5       -> >=2.5<2.6
    ==2.5.0    -> >=2.5.0<2.5.1
    ==2.5.0.0 -> >=2.5.0.0<2.5.0.1

Which means that version specifiers are _always_ ranges and are never exact versions. The PEP
as written relies on authors to decide beforehand how many digits they are going to use in their
versions, and for them to never increase or decrease that number.

I also checked to see if Distutils2/packaging implemented VersionPredicates that way or if they
allowed specifying an exact version. It turned out that it implements the PEP as written:

>>> from distutils2 import version
>>> predicate = version.VersionPredicate("foo (==2.5)")
>>> print predicate
foo (==2.5)
>>> predicate.match("2.5")
True
>>> predicate.match("2.5.0")
True
>>> predicate.match("2.5.0.0")
True
>>> predicate.mach("2.5.0.5")
True

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20120911/93cf691d/attachment.html>

From erik.m.bray at gmail.com  Tue Sep 11 21:11:27 2012
From: erik.m.bray at gmail.com (Erik Bray)
Date: Tue, 11 Sep 2012 15:11:27 -0400
Subject: [Distutils] PEP345 - Distutils2 - Cannot pin to exact version
In-Reply-To: <86B0D55791FB4654A1A219CE1DA82F0E@gmail.com>
References: <86B0D55791FB4654A1A219CE1DA82F0E@gmail.com>
Message-ID: <CAOTD34b8_V81ecjsBxA143_96t6631kLtNTOFE4wOSg6c1FZsw@mail.gmail.com>

On Tue, Sep 11, 2012 at 2:35 PM, Donald Stufft <donald.stufft at gmail.com> wrote:
> I was digging through PEP386 & PEP345 today, and I noticed something odd
> about the wording of PEP345.
>
> It states:
>
>     When a version is provided, it always includes all versions that starts
> with the same value. For
>     example the "2.5" version of Python will include versions like "2.5.2"
> or "2.5.3". Pre and post
>     releases in that case are excluded. So in our example, versions like
> "2.5a1" are not included
>     when "2.5" is used. If the first version of the range is required, it
> has to be explicitly given. In
>     our example, it will be "2.5.0".
>
> It also states:
>
>     In that case, "2.5.0" will have to be explicitly used to avoid any
> confusion between the "2.5"
>     notation that represents the full range. It is a recommended practice to
> use schemes of the
>     same length for a series to completely avoid this problem.
>
> This effectively translates to an inability to pin to an exact version. Even
> in the case of specifying
> == it checks that the version "starts with" the value you selected. So if
> you pin to "2.5", and the
> author then releases "2.5.1", that will count as ==2.5. If you try to then
> pin to "2.5.0", and the
> author releases "2.5.0.1", then that will count as ==2.5.0.
>
> Essentially this translates to:
>
>     ==2.5       -> >=2.5<2.6
>     ==2.5.0    -> >=2.5.0<2.5.1
>     ==2.5.0.0 -> >=2.5.0.0<2.5.0.1
>
> Which means that version specifiers are _always_ ranges and are never exact
> versions. The PEP
> as written relies on authors to decide beforehand how many digits they are
> going to use in their
> versions, and for them to never increase or decrease that number.
>
> I also checked to see if Distutils2/packaging implemented VersionPredicates
> that way or if they
> allowed specifying an exact version. It turned out that it implements the
> PEP as written:
>
>>>> from distutils2 import version
>>>> predicate = version.VersionPredicate("foo (==2.5)")
>>>> print predicate
> foo (==2.5)
>>>> predicate.match("2.5")
> True
>>>> predicate.match("2.5.0")
> True
>>>> predicate.match("2.5.0.0")
> True
>>>> predicate.mach("2.5.0.5")
> True

That's kind of annoying.  Does anyone know if this is by design?

FWIW there is a workaround. For example if you want to pin to exactly 2.5.1:

>>> predicate = version.VersionPredicate("foo (==2.5.1,<2.5.1.1)")
>>> predicate.match('2.5.1')
True
>>> predicate.match('2.5.2')
False
>>> predicate.match('2.5.1.0')
True
>>> predicate.match('2.5.1.1')


Erik

From dholth at gmail.com  Tue Sep 11 21:19:11 2012
From: dholth at gmail.com (Daniel Holth)
Date: Tue, 11 Sep 2012 15:19:11 -0400
Subject: [Distutils] PEP345 - Distutils2 - Cannot pin to exact version
In-Reply-To: <CAOTD34b8_V81ecjsBxA143_96t6631kLtNTOFE4wOSg6c1FZsw@mail.gmail.com>
References: <86B0D55791FB4654A1A219CE1DA82F0E@gmail.com>
	<CAOTD34b8_V81ecjsBxA143_96t6631kLtNTOFE4wOSg6c1FZsw@mail.gmail.com>
Message-ID: <CAG8k2+6ak7vs=+tvGEfGe+iY99uwGc5t7X9R99H4-n2FZVuP0A@mail.gmail.com>

On Tue, Sep 11, 2012 at 3:11 PM, Erik Bray <erik.m.bray at gmail.com> wrote:
> On Tue, Sep 11, 2012 at 2:35 PM, Donald Stufft <donald.stufft at gmail.com> wrote:
>> I was digging through PEP386 & PEP345 today, and I noticed something odd
>> about the wording of PEP345.
>>
>> It states:
>>
>>     When a version is provided, it always includes all versions that starts
>> with the same value. For
>>     example the "2.5" version of Python will include versions like "2.5.2"
>> or "2.5.3". Pre and post
>>     releases in that case are excluded. So in our example, versions like
>> "2.5a1" are not included
>>     when "2.5" is used. If the first version of the range is required, it
>> has to be explicitly given. In
>>     our example, it will be "2.5.0".
>>
>> It also states:
>>
>>     In that case, "2.5.0" will have to be explicitly used to avoid any
>> confusion between the "2.5"
>>     notation that represents the full range. It is a recommended practice to
>> use schemes of the
>>     same length for a series to completely avoid this problem.
>>
>> This effectively translates to an inability to pin to an exact version. Even
>> in the case of specifying
>> == it checks that the version "starts with" the value you selected. So if
>> you pin to "2.5", and the
>> author then releases "2.5.1", that will count as ==2.5. If you try to then
>> pin to "2.5.0", and the
>> author releases "2.5.0.1", then that will count as ==2.5.0.
>>
>> Essentially this translates to:
>>
>>     ==2.5       -> >=2.5<2.6
>>     ==2.5.0    -> >=2.5.0<2.5.1
>>     ==2.5.0.0 -> >=2.5.0.0<2.5.0.1
>>
>> Which means that version specifiers are _always_ ranges and are never exact
>> versions. The PEP
>> as written relies on authors to decide beforehand how many digits they are
>> going to use in their
>> versions, and for them to never increase or decrease that number.
>>
>> I also checked to see if Distutils2/packaging implemented VersionPredicates
>> that way or if they
>> allowed specifying an exact version. It turned out that it implements the
>> PEP as written:
>>
>>>>> from distutils2 import version
>>>>> predicate = version.VersionPredicate("foo (==2.5)")
>>>>> print predicate
>> foo (==2.5)
>>>>> predicate.match("2.5")
>> True
>>>>> predicate.match("2.5.0")
>> True
>>>>> predicate.match("2.5.0.0")
>> True
>>>>> predicate.mach("2.5.0.5")
>> True
>
> That's kind of annoying.  Does anyone know if this is by design?
>
> FWIW there is a workaround. For example if you want to pin to exactly 2.5.1:
>
>>>> predicate = version.VersionPredicate("foo (==2.5.1,<2.5.1.1)")
>>>> predicate.match('2.5.1')
> True
>>>> predicate.match('2.5.2')
> False
>>>> predicate.match('2.5.1.0')
> True
>>>> predicate.match('2.5.1.1')

But you could still release 2.5.1.0.0? I suppose we limit the number
of version parts these days.

Why don't we update the spec so that (2.0) means (2.0) the range, and
(==2.0) means 2.0 (exactly).

Daniel

From erik.m.bray at gmail.com  Tue Sep 11 21:25:02 2012
From: erik.m.bray at gmail.com (Erik Bray)
Date: Tue, 11 Sep 2012 15:25:02 -0400
Subject: [Distutils] PEP345 - Distutils2 - Cannot pin to exact version
In-Reply-To: <CAG8k2+6ak7vs=+tvGEfGe+iY99uwGc5t7X9R99H4-n2FZVuP0A@mail.gmail.com>
References: <86B0D55791FB4654A1A219CE1DA82F0E@gmail.com>
	<CAOTD34b8_V81ecjsBxA143_96t6631kLtNTOFE4wOSg6c1FZsw@mail.gmail.com>
	<CAG8k2+6ak7vs=+tvGEfGe+iY99uwGc5t7X9R99H4-n2FZVuP0A@mail.gmail.com>
Message-ID: <CAOTD34bRSqp7Ex2TP7AOu8A2oWAB+LMk9omJ6u83Ofi4YFBjtg@mail.gmail.com>

On Tue, Sep 11, 2012 at 3:19 PM, Daniel Holth <dholth at gmail.com> wrote:
> On Tue, Sep 11, 2012 at 3:11 PM, Erik Bray <erik.m.bray at gmail.com> wrote:
>> On Tue, Sep 11, 2012 at 2:35 PM, Donald Stufft <donald.stufft at gmail.com> wrote:
>>> I was digging through PEP386 & PEP345 today, and I noticed something odd
>>> about the wording of PEP345.
>>>
>>> It states:
>>>
>>>     When a version is provided, it always includes all versions that starts
>>> with the same value. For
>>>     example the "2.5" version of Python will include versions like "2.5.2"
>>> or "2.5.3". Pre and post
>>>     releases in that case are excluded. So in our example, versions like
>>> "2.5a1" are not included
>>>     when "2.5" is used. If the first version of the range is required, it
>>> has to be explicitly given. In
>>>     our example, it will be "2.5.0".
>>>
>>> It also states:
>>>
>>>     In that case, "2.5.0" will have to be explicitly used to avoid any
>>> confusion between the "2.5"
>>>     notation that represents the full range. It is a recommended practice to
>>> use schemes of the
>>>     same length for a series to completely avoid this problem.
>>>
>>> This effectively translates to an inability to pin to an exact version. Even
>>> in the case of specifying
>>> == it checks that the version "starts with" the value you selected. So if
>>> you pin to "2.5", and the
>>> author then releases "2.5.1", that will count as ==2.5. If you try to then
>>> pin to "2.5.0", and the
>>> author releases "2.5.0.1", then that will count as ==2.5.0.
>>>
>>> Essentially this translates to:
>>>
>>>     ==2.5       -> >=2.5<2.6
>>>     ==2.5.0    -> >=2.5.0<2.5.1
>>>     ==2.5.0.0 -> >=2.5.0.0<2.5.0.1
>>>
>>> Which means that version specifiers are _always_ ranges and are never exact
>>> versions. The PEP
>>> as written relies on authors to decide beforehand how many digits they are
>>> going to use in their
>>> versions, and for them to never increase or decrease that number.
>>>
>>> I also checked to see if Distutils2/packaging implemented VersionPredicates
>>> that way or if they
>>> allowed specifying an exact version. It turned out that it implements the
>>> PEP as written:
>>>
>>>>>> from distutils2 import version
>>>>>> predicate = version.VersionPredicate("foo (==2.5)")
>>>>>> print predicate
>>> foo (==2.5)
>>>>>> predicate.match("2.5")
>>> True
>>>>>> predicate.match("2.5.0")
>>> True
>>>>>> predicate.match("2.5.0.0")
>>> True
>>>>>> predicate.mach("2.5.0.5")
>>> True
>>
>> That's kind of annoying.  Does anyone know if this is by design?
>>
>> FWIW there is a workaround. For example if you want to pin to exactly 2.5.1:
>>
>>>>> predicate = version.VersionPredicate("foo (==2.5.1,<2.5.1.1)")
>>>>> predicate.match('2.5.1')
>> True
>>>>> predicate.match('2.5.2')
>> False
>>>>> predicate.match('2.5.1.0')
>> True
>>>>> predicate.match('2.5.1.1')
>
> But you could still release 2.5.1.0.0? I suppose we limit the number
> of version parts these days.
>
> Why don't we update the spec so that (2.0) means (2.0) the range, and
> (==2.0) means 2.0 (exactly).
>
> Daniel

The PEP is ambiguous on this, but you could get away with reading it
as "When a version is provided (without a conditional operator) it
always includes all versions that start with the same value".
Although it's unwritten in the PEP exactly how the operators are meant
to be interpreted, I would say they should be interpreted strictly.

Erik

From tarek at ziade.org  Tue Sep 11 21:31:16 2012
From: tarek at ziade.org (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Tue, 11 Sep 2012 21:31:16 +0200
Subject: [Distutils] PEP345 - Distutils2 - Cannot pin to exact version
In-Reply-To: <CAOTD34bRSqp7Ex2TP7AOu8A2oWAB+LMk9omJ6u83Ofi4YFBjtg@mail.gmail.com>
References: <86B0D55791FB4654A1A219CE1DA82F0E@gmail.com>
	<CAOTD34b8_V81ecjsBxA143_96t6631kLtNTOFE4wOSg6c1FZsw@mail.gmail.com>
	<CAG8k2+6ak7vs=+tvGEfGe+iY99uwGc5t7X9R99H4-n2FZVuP0A@mail.gmail.com>
	<CAOTD34bRSqp7Ex2TP7AOu8A2oWAB+LMk9omJ6u83Ofi4YFBjtg@mail.gmail.com>
Message-ID: <504F9184.5010806@ziade.org>

On 9/11/12 9:25 PM, Erik Bray wrote:
> ...
>> But you could still release 2.5.1.0.0? I suppose we limit the number
>> of version parts these days.
>>
>> Why don't we update the spec so that (2.0) means (2.0) the range, and
>> (==2.0) means 2.0 (exactly).
>>
>> Daniel
> The PEP is ambiguous on this, but you could get away with reading it
> as "When a version is provided (without a conditional operator) it
> always includes all versions that start with the same value".
> Although it's unwritten in the PEP exactly how the operators are meant
> to be interpreted, I would say they should be interpreted strictly.

  Yeah that's what we wanted to do back then, I recall now.

+1 for the clarification - since it's just rephrasing thing we don't 
need a new PEP I guess

and we can fix the implementations

Cheers
Tarek


>
> Erik
> _______________________________________________
> Distutils-SIG maillist  -  Distutils-SIG at python.org
> http://mail.python.org/mailman/listinfo/distutils-sig


From dholth at gmail.com  Tue Sep 11 21:34:23 2012
From: dholth at gmail.com (Daniel Holth)
Date: Tue, 11 Sep 2012 15:34:23 -0400
Subject: [Distutils] PEP345 - Distutils2 - Cannot pin to exact version
In-Reply-To: <504F9184.5010806@ziade.org>
References: <86B0D55791FB4654A1A219CE1DA82F0E@gmail.com>
	<CAOTD34b8_V81ecjsBxA143_96t6631kLtNTOFE4wOSg6c1FZsw@mail.gmail.com>
	<CAG8k2+6ak7vs=+tvGEfGe+iY99uwGc5t7X9R99H4-n2FZVuP0A@mail.gmail.com>
	<CAOTD34bRSqp7Ex2TP7AOu8A2oWAB+LMk9omJ6u83Ofi4YFBjtg@mail.gmail.com>
	<504F9184.5010806@ziade.org>
Message-ID: <CAG8k2+5bd=cSB-+keaQw5w9K9s9tx61DfOTSCZf9izo33aWUpg@mail.gmail.com>

On Tue, Sep 11, 2012 at 3:31 PM, Tarek Ziad? <tarek at ziade.org> wrote:
> On 9/11/12 9:25 PM, Erik Bray wrote:
>>
>> ...
>>
>>> But you could still release 2.5.1.0.0? I suppose we limit the number
>>> of version parts these days.
>>>
>>> Why don't we update the spec so that (2.0) means (2.0) the range, and
>>> (==2.0) means 2.0 (exactly).
>>>
>>> Daniel
>>
>> The PEP is ambiguous on this, but you could get away with reading it
>> as "When a version is provided (without a conditional operator) it
>> always includes all versions that start with the same value".
>> Although it's unwritten in the PEP exactly how the operators are meant
>> to be interpreted, I would say they should be interpreted strictly.
>
>
>  Yeah that's what we wanted to do back then, I recall now.
>
> +1 for the clarification - since it's just rephrasing thing we don't need a
> new PEP I guess
>
> and we can fix the implementations
>
> Cheers
> Tarek

We do have a new PEP anyway.

http://www.python.org/dev/peps/pep-0426/ Metadata 1.3

From tarek at ziade.org  Tue Sep 11 21:36:11 2012
From: tarek at ziade.org (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Tue, 11 Sep 2012 21:36:11 +0200
Subject: [Distutils] PEP345 - Distutils2 - Cannot pin to exact version
In-Reply-To: <CAG8k2+5bd=cSB-+keaQw5w9K9s9tx61DfOTSCZf9izo33aWUpg@mail.gmail.com>
References: <86B0D55791FB4654A1A219CE1DA82F0E@gmail.com>
	<CAOTD34b8_V81ecjsBxA143_96t6631kLtNTOFE4wOSg6c1FZsw@mail.gmail.com>
	<CAG8k2+6ak7vs=+tvGEfGe+iY99uwGc5t7X9R99H4-n2FZVuP0A@mail.gmail.com>
	<CAOTD34bRSqp7Ex2TP7AOu8A2oWAB+LMk9omJ6u83Ofi4YFBjtg@mail.gmail.com>
	<504F9184.5010806@ziade.org>
	<CAG8k2+5bd=cSB-+keaQw5w9K9s9tx61DfOTSCZf9izo33aWUpg@mail.gmail.com>
Message-ID: <504F92AB.2080803@ziade.org>

On 9/11/12 9:34 PM, Daniel Holth wrote:
> On Tue, Sep 11, 2012 at 3:31 PM, Tarek Ziad? <tarek at ziade.org> wrote:
>> On 9/11/12 9:25 PM, Erik Bray wrote:
>>> ...
>>>
>>>> But you could still release 2.5.1.0.0? I suppose we limit the number
>>>> of version parts these days.
>>>>
>>>> Why don't we update the spec so that (2.0) means (2.0) the range, and
>>>> (==2.0) means 2.0 (exactly).
>>>>
>>>> Daniel
>>> The PEP is ambiguous on this, but you could get away with reading it
>>> as "When a version is provided (without a conditional operator) it
>>> always includes all versions that start with the same value".
>>> Although it's unwritten in the PEP exactly how the operators are meant
>>> to be interpreted, I would say they should be interpreted strictly.
>>
>>   Yeah that's what we wanted to do back then, I recall now.
>>
>> +1 for the clarification - since it's just rephrasing thing we don't need a
>> new PEP I guess
>>
>> and we can fix the implementations
>>
>> Cheers
>> Tarek
> We do have a new PEP anyway.
>
> http://www.python.org/dev/peps/pep-0426/ Metadata 1.3
We need to fix PEP 386 though

From donald.stufft at gmail.com  Tue Sep 11 21:56:05 2012
From: donald.stufft at gmail.com (Donald Stufft)
Date: Tue, 11 Sep 2012 15:56:05 -0400
Subject: [Distutils] PEP345 - Distutils2 - Cannot pin to exact version
In-Reply-To: <504F92AB.2080803@ziade.org>
References: <86B0D55791FB4654A1A219CE1DA82F0E@gmail.com>
	<CAOTD34b8_V81ecjsBxA143_96t6631kLtNTOFE4wOSg6c1FZsw@mail.gmail.com>
	<CAG8k2+6ak7vs=+tvGEfGe+iY99uwGc5t7X9R99H4-n2FZVuP0A@mail.gmail.com>
	<CAOTD34bRSqp7Ex2TP7AOu8A2oWAB+LMk9omJ6u83Ofi4YFBjtg@mail.gmail.com>
	<504F9184.5010806@ziade.org>
	<CAG8k2+5bd=cSB-+keaQw5w9K9s9tx61DfOTSCZf9izo33aWUpg@mail.gmail.com>
	<504F92AB.2080803@ziade.org>
Message-ID: <C9F7E47A96D04158BB871682F7AD6B98@gmail.com>

On Tuesday, September 11, 2012 at 3:36 PM, Tarek Ziad? wrote:
> We need to fix PEP 386 though  
On the topic of PEP386, It also doesn't specify how versions like 2.5 and 2.5.0  
should be treated. That's what originally got me on the thread of how PEP345
handles the == case. I believe that PEP386 should specify that additional .0's
on the end of a version are semantically noops. So 2.5 == 2.5.0 == 2.5.0.0 etc.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20120911/a7c97fd0/attachment.html>

From donald.stufft at gmail.com  Tue Sep 11 23:04:04 2012
From: donald.stufft at gmail.com (Donald Stufft)
Date: Tue, 11 Sep 2012 17:04:04 -0400
Subject: [Distutils] PEP386 Pre releases
Message-ID: <8906B719F6704854A5F3B01E8E719963@gmail.com>

Why does 1.0a1 sort before 1.0.dev1? It appears to me that common
usage in the wild of dev1 releases is that they are used for the
development version before any sort of alpha, beta, rc, or final has
been cut? 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20120911/3df1a137/attachment.html>

From erik.m.bray at gmail.com  Wed Sep 12 19:43:54 2012
From: erik.m.bray at gmail.com (Erik Bray)
Date: Wed, 12 Sep 2012 13:43:54 -0400
Subject: [Distutils] PEP386 Pre releases
In-Reply-To: <8906B719F6704854A5F3B01E8E719963@gmail.com>
References: <8906B719F6704854A5F3B01E8E719963@gmail.com>
Message-ID: <CAOTD34bK-E2aNLrzoaAy0aZKb1+gGcPuivFQLEOOLx+B=COyJA@mail.gmail.com>

On Tue, Sep 11, 2012 at 5:04 PM, Donald Stufft <donald.stufft at gmail.com> wrote:
> Why does 1.0a1 sort before 1.0.dev1? It appears to me that common
> usage in the wild of dev1 releases is that they are used for the
> development version before any sort of alpha, beta, rc, or final has
> been cut?

If I had to guess, the rationale might be that when developing a new
version you're developing toward the first "alpha" release, so
something like .dev1 would be a development preview of a final release
after all a/b/rc pre-releases have been released.

That said, this doesn't match my workflow at all.  After releasing
"1.0" the next version is going to be "1.1", and any development
pre-release will be "1.1.devX".  "1.1a" might not ever even exist.  I
think others brought up this critique at the time PEP 386 was being
discussed, but then nothing was ever done about it >_>

Erik

From dholth at gmail.com  Wed Sep 12 19:46:47 2012
From: dholth at gmail.com (Daniel Holth)
Date: Wed, 12 Sep 2012 13:46:47 -0400
Subject: [Distutils] PEP386 Pre releases
In-Reply-To: <CAOTD34bK-E2aNLrzoaAy0aZKb1+gGcPuivFQLEOOLx+B=COyJA@mail.gmail.com>
References: <8906B719F6704854A5F3B01E8E719963@gmail.com>
	<CAOTD34bK-E2aNLrzoaAy0aZKb1+gGcPuivFQLEOOLx+B=COyJA@mail.gmail.com>
Message-ID: <CAG8k2+511nwtGA9agumw_ksqUizcDVvagwFt6Uq55XRSWi0kbw@mail.gmail.com>

Isn't the pep carefully constructed to respect lexicographical sorting?
On Sep 12, 2012 1:44 PM, "Erik Bray" <erik.m.bray at gmail.com> wrote:

> On Tue, Sep 11, 2012 at 5:04 PM, Donald Stufft <donald.stufft at gmail.com>
> wrote:
> > Why does 1.0a1 sort before 1.0.dev1? It appears to me that common
> > usage in the wild of dev1 releases is that they are used for the
> > development version before any sort of alpha, beta, rc, or final has
> > been cut?
>
> If I had to guess, the rationale might be that when developing a new
> version you're developing toward the first "alpha" release, so
> something like .dev1 would be a development preview of a final release
> after all a/b/rc pre-releases have been released.
>
> That said, this doesn't match my workflow at all.  After releasing
> "1.0" the next version is going to be "1.1", and any development
> pre-release will be "1.1.devX".  "1.1a" might not ever even exist.  I
> think others brought up this critique at the time PEP 386 was being
> discussed, but then nothing was ever done about it >_>
>
> Erik
> _______________________________________________
> Distutils-SIG maillist  -  Distutils-SIG at python.org
> http://mail.python.org/mailman/listinfo/distutils-sig
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20120912/f998da77/attachment.html>

From donald.stufft at gmail.com  Wed Sep 12 19:47:39 2012
From: donald.stufft at gmail.com (Donald Stufft)
Date: Wed, 12 Sep 2012 13:47:39 -0400
Subject: [Distutils] PEP386 Pre releases
In-Reply-To: <CAOTD34bK-E2aNLrzoaAy0aZKb1+gGcPuivFQLEOOLx+B=COyJA@mail.gmail.com>
References: <8906B719F6704854A5F3B01E8E719963@gmail.com>
	<CAOTD34bK-E2aNLrzoaAy0aZKb1+gGcPuivFQLEOOLx+B=COyJA@mail.gmail.com>
Message-ID: <C7B7C99A1E0C49E2BE38AB845EDD26EF@gmail.com>

On Wednesday, September 12, 2012 at 1:43 PM, Erik Bray wrote:
> That said, this doesn't match my workflow at all. After releasing
> "1.0" the next version is going to be "1.1", and any development
> pre-release will be "1.1.devX". "1.1a" might not ever even exist. I
> think others brought up this critique at the time PEP 386 was being
> discussed, but then nothing was ever done about it >_>
> 
Yea, this concerned me because 1.1.devX < 1.1a1 < 1.1b1 < 1.1c1 < 1.1
is how i've seen it used in the wild. Looks like most everyone i've seen
using it so far has been doing it wrong. Don't think ive seen a single
person do it right.

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20120912/3736f2f0/attachment.html>

From erik.m.bray at gmail.com  Wed Sep 12 19:48:10 2012
From: erik.m.bray at gmail.com (Erik Bray)
Date: Wed, 12 Sep 2012 13:48:10 -0400
Subject: [Distutils] PEP386 Pre releases
In-Reply-To: <CAG8k2+511nwtGA9agumw_ksqUizcDVvagwFt6Uq55XRSWi0kbw@mail.gmail.com>
References: <8906B719F6704854A5F3B01E8E719963@gmail.com>
	<CAOTD34bK-E2aNLrzoaAy0aZKb1+gGcPuivFQLEOOLx+B=COyJA@mail.gmail.com>
	<CAG8k2+511nwtGA9agumw_ksqUizcDVvagwFt6Uq55XRSWi0kbw@mail.gmail.com>
Message-ID: <CAOTD34ZOZbb7cuA6ywRR6MiVaTgPV8OVc=E11Q=8nbu9D=FGcg@mail.gmail.com>

On Wed, Sep 12, 2012 at 1:46 PM, Daniel Holth <dholth at gmail.com> wrote:
> Isn't the pep carefully constructed to respect lexicographical sorting?

Yes.  That's why it's ".dev" and not just "dev".  Lexicographically,
"1.0.dev" < "1.0a".

> On Sep 12, 2012 1:44 PM, "Erik Bray" <erik.m.bray at gmail.com> wrote:
>>
>> On Tue, Sep 11, 2012 at 5:04 PM, Donald Stufft <donald.stufft at gmail.com>
>> wrote:
>> > Why does 1.0a1 sort before 1.0.dev1? It appears to me that common
>> > usage in the wild of dev1 releases is that they are used for the
>> > development version before any sort of alpha, beta, rc, or final has
>> > been cut?
>>
>> If I had to guess, the rationale might be that when developing a new
>> version you're developing toward the first "alpha" release, so
>> something like .dev1 would be a development preview of a final release
>> after all a/b/rc pre-releases have been released.
>>
>> That said, this doesn't match my workflow at all.  After releasing
>> "1.0" the next version is going to be "1.1", and any development
>> pre-release will be "1.1.devX".  "1.1a" might not ever even exist.  I
>> think others brought up this critique at the time PEP 386 was being
>> discussed, but then nothing was ever done about it >_>
>>
>> Erik
>> _______________________________________________
>> Distutils-SIG maillist  -  Distutils-SIG at python.org
>> http://mail.python.org/mailman/listinfo/distutils-sig

From spiccinini at codigosur.org  Thu Sep 13 03:21:24 2012
From: spiccinini at codigosur.org (SAn)
Date: Wed, 12 Sep 2012 22:21:24 -0300
Subject: [Distutils] PEP386 Pre releases
Message-ID: <CAESYoks+erPUU34s7-=mf4iGmwT2HweeAk-trN=3c6dWzfi14w@mail.gmail.com>

>On Wed Sep 12 19:47:39 CEST 2012, Donald Stufft wrote:
>
>On Wednesday, September 12, 2012 at 1:43 PM, Erik Bray wrote:
>> That said, this doesn't match my workflow at all. After releasing
>> "1.0" the next version is going to be "1.1", and any development
>> pre-release will be "1.1.devX". "1.1a" might not ever even exist. I
>> think others brought up this critique at the time PEP 386 was being
>> discussed, but then nothing was ever done about it >_>
>
> Yea, this concerned me because 1.1.devX < 1.1a1 < 1.1b1 < 1.1c1 < 1.1
> is how i've seen it used in the wild. Looks like most everyone i've seen
> using it so far has been doing it wrong. Don't think ive seen a single
> person do it right.

Hi, just yesterday i got bitten by this issue. FYI:

# verlib "pep386" (from https://bitbucket.org/tarek/distutilsversion)
>>> from verlib import NormalizedVersion as V
>>> V("0.2a1") <  V("0.2.dev0") < V("0.2")
True

Also there is a bug in that version of verlib because it contradicts
pep386 here:

>>> V("0.2rc1") < V("0.2")
False


# setuptools ??
>>> from pkg_resources import parse_version as V
>>> V("0.2.dev0") < V("0.2a1") < V("0.2")
True

# distutils
>>> distutils.version.LooseVersion("0.2.dev0") < distutils.version.LooseVersion("0.2")
False
# O_O

cheers,
SAn
PD: i wasn't subcribed to the list, sorry if this mail breaks the "thread".

From dholth at gmail.com  Thu Sep 13 07:50:45 2012
From: dholth at gmail.com (Daniel Holth)
Date: Thu, 13 Sep 2012 01:50:45 -0400
Subject: [Distutils] Extension: Entry-Point
Message-ID: <CAG8k2+5NtD8mXasT3Nob4bC+VjbcTh0pM0WrdH4ZAU3aNp=bTA@mail.gmail.com>

I propose mapping a typical entry_points.txt to Metadata 1.3 like so:

        [pyramid.scaffold]
        starter=pyramid.scaffolds:StarterProjectTemplate
        zodb=pyramid.scaffolds:ZODBProjectTemplate
        alchemy=pyramid.scaffolds:AlchemyProjectTemplate
        [console_scripts]
        bfg2pyramid = pyramid.fixers.fix_bfg_imports:main
        pcreate = pyramid.scripts.pcreate:main
        pserve = pyramid.scripts.pserve:main
        pshell = pyramid.scripts.pshell:main
        proutes = pyramid.scripts.proutes:main
        pviews = pyramid.scripts.pviews:main
        ptweens = pyramid.scripts.ptweens:main
        prequest = pyramid.scripts.prequest:main
        [paste.server_runner]
        wsgiref = pyramid.scripts.pserve:wsgiref_server_runner
        cherrypy = pyramid.scripts.pserve:cherrypy_server_runner


Extension: Entry-Point
Entry-Point/pyramid.scaffold: starter=pyramid.scaffolds:StarterProjectTemplate
Entry-Point/paste.server_runner: wsgiref =
pyramid.scripts.pserve:wsgiref_server_runner

From chrism at plope.com  Thu Sep 13 09:26:44 2012
From: chrism at plope.com (Chris McDonough)
Date: Thu, 13 Sep 2012 03:26:44 -0400
Subject: [Distutils] extension to pep 376: "develop" installation
Message-ID: <1347521204.3464.55.camel@thinko>

I propose an extension to PEP 376 to support the development of projects
such that:

- Installation tools should expect to need to cope with files
  present in purelib named "foo.dist-link".  Each of these files
  will be of the form "/path/containing/foo.dist-info\nreldir"
  For example, the contents of such a file might be:

      /home/chrism/projects/zope.interface
      src

  Presence of such files would indicate a link to a directory
  which contains a .dist-info directory for the project as well
  as the subdirectory in the project which actually contains
  packages and modules.

- Installation tools should need to cope with a "develop.pth"
  file in purelib.  Its presence will indicate that there are
  projects in development mode in this environment.
  Its absence indicates that there are no projects in development
  mode in this environment.

- Installer tools should place dist-link files into purelib as the
  result of a notional "develop" command e.g.

      pysetup develop .

  If "pysetup develop ." were run in zope.interface, 
  zope.interface.dist-link would be installed into  purelib

- Installer tools should mutate "develop.pth" as the result of 
  "develop", e.g. "pysetup develop ." would create this
  file if it didn't already exist.

  After a "pysetup develop" of zope.interface it might look like:

  import pkgutil.develop; pkgutil.develop.save_path_order()
  /home/chrism/projects/zope.interface/src
  import pkgutil.develop; pkgutil.develop.reorder_sys_path()

  The "import" lines would ensure that the paths mentioned in the
  .pth file would precede stdlib sys.paths but would come after
  paths mentioned on PYTHONPATH envvar.  Some sort of hack like
  this anyway; maybe Python could offer better support for
  this without hackiness of relying on executing code via
  a .pth file at startup time.

- "pysetup undevelop ." of zope.interface would undo the changes
  made via "pysetup develop"; it would also remove develop.pth
  if no other developed projects existed anymore after 
  zope.interface was undeveloped.

- It will be permissible to have both the "develop" version and
  the "install" version of a library installed in the same purelib.
  Installer tools should be able to cope with this and display
  both versions when they are asked to list installed distributions.

That's the basics.


From chrism at plope.com  Thu Sep 13 10:04:14 2012
From: chrism at plope.com (Chris McDonough)
Date: Thu, 13 Sep 2012 04:04:14 -0400
Subject: [Distutils] Extension: Entry-Point
In-Reply-To: <CAG8k2+5NtD8mXasT3Nob4bC+VjbcTh0pM0WrdH4ZAU3aNp=bTA@mail.gmail.com>
References: <CAG8k2+5NtD8mXasT3Nob4bC+VjbcTh0pM0WrdH4ZAU3aNp=bTA@mail.gmail.com>
Message-ID: <1347523454.3464.58.camel@thinko>

On Thu, 2012-09-13 at 01:50 -0400, Daniel Holth wrote:
> I propose mapping a typical entry_points.txt to Metadata 1.3 like so:
> 
>         [pyramid.scaffold]
>         starter=pyramid.scaffolds:StarterProjectTemplate
>         zodb=pyramid.scaffolds:ZODBProjectTemplate
>         alchemy=pyramid.scaffolds:AlchemyProjectTemplate
>         [console_scripts]
>         bfg2pyramid = pyramid.fixers.fix_bfg_imports:main
>         pcreate = pyramid.scripts.pcreate:main
>         pserve = pyramid.scripts.pserve:main
>         pshell = pyramid.scripts.pshell:main
>         proutes = pyramid.scripts.proutes:main
>         pviews = pyramid.scripts.pviews:main
>         ptweens = pyramid.scripts.ptweens:main
>         prequest = pyramid.scripts.prequest:main
>         [paste.server_runner]
>         wsgiref = pyramid.scripts.pserve:wsgiref_server_runner
>         cherrypy = pyramid.scripts.pserve:cherrypy_server_runner
> 
> 
> Extension: Entry-Point
> Entry-Point/pyramid.scaffold: starter=pyramid.scaffolds:StarterProjectTemplate
> Entry-Point/paste.server_runner: wsgiref =
> pyramid.scripts.pserve:wsgiref_server_runner

I see nothing wrong with this as long as it's understood that individual
"Entry-Point/foo" lines can be multiple use.  The spec only indicates
that the "Entry-Point" moniker itself is multiple use.

- C




From chrism at plope.com  Thu Sep 13 10:15:52 2012
From: chrism at plope.com (Chris McDonough)
Date: Thu, 13 Sep 2012 04:15:52 -0400
Subject: [Distutils] optional binary compilation under future installation
	tools
Message-ID: <1347524152.3464.68.camel@thinko>

Some packages we maintain currently provide largely identical
side-by-side implementations of features: one implementation is written
in C, the other implementation is written in Python.  The C module is
just an optimized version of the Python code.

There is logic at module scope within modules in the same package which
attempts to import the C version.  If the C version cannot be imported
(an ImportError is raised), the logic falls back to importing the Python
version:

try:
    from . import cmodule as module
except ImportError
    from . import pymodule as module

This means that the package can be used if the C implementation is
compiled or not.  It will run more slowly, but that's OK in lots of
cases.

In our setup.py for these kinds of distributions, we have code which
under Py2 uses a setuptools "feature" and under Py3 registers a
different kind of build_ext which, at install time, will:

- Attempt to compile the C if suitable build tools seem to exist on the
  target system.

- If suitable build tools don't seem to exist on the target system, will
  print a message and continue.

How can we support such a feature in the brave new declarative packaging
world?

- C



From chrism at plope.com  Thu Sep 13 10:23:34 2012
From: chrism at plope.com (Chris McDonough)
Date: Thu, 13 Sep 2012 04:23:34 -0400
Subject: [Distutils] extension to pep 376: "develop" installation
In-Reply-To: <1347521204.3464.55.camel@thinko>
References: <1347521204.3464.55.camel@thinko>
Message-ID: <1347524614.3464.70.camel@thinko>

On Thu, 2012-09-13 at 03:26 -0400, Chris McDonough wrote:

> That's the basics.

One additional thing implementations in installer tools of "develop"
should probably do: cause test-time-only dependencies to be installed.

- C



From tarek at ziade.org  Thu Sep 13 10:44:24 2012
From: tarek at ziade.org (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Thu, 13 Sep 2012 10:44:24 +0200
Subject: [Distutils] optional binary compilation under future
 installation tools
In-Reply-To: <1347524152.3464.68.camel@thinko>
References: <1347524152.3464.68.camel@thinko>
Message-ID: <50519CE8.4050102@ziade.org>

On 9/13/12 10:15 AM, Chris McDonough wrote:
> Some packages we maintain currently provide largely identical
> side-by-side implementations of features: one implementation is written
> in C, the other implementation is written in Python.  The C module is
> just an optimized version of the Python code.
>
> There is logic at module scope within modules in the same package which
> attempts to import the C version.  If the C version cannot be imported
> (an ImportError is raised), the logic falls back to importing the Python
> version:
>
> try:
>      from . import cmodule as module
> except ImportError
>      from . import pymodule as module
>
> This means that the package can be used if the C implementation is
> compiled or not.  It will run more slowly, but that's OK in lots of
> cases.
>
> In our setup.py for these kinds of distributions, we have code which
> under Py2 uses a setuptools "feature" and under Py3 registers a
> different kind of build_ext which, at install time, will:
>
> - Attempt to compile the C if suitable build tools seem to exist on the
>    target system.
>
> - If suitable build tools don't seem to exist on the target system, will
>    print a message and continue.
>
> How can we support such a feature in the brave new declarative packaging
> world?
>
> - C
>
FWIW there's an 'optional' option for the Extension class in Python 2.7 
in distutils.

see 
http://hg.python.org/cpython/file/9b40d018e4cf/Lib/distutils/command/build_ext.py#l458

If the compilation fails it just warns and continues.

That option is also present in distutils2, so you can use it in the 
setup.cfg file.

Does that work ?

Cheers
Tarek




> _______________________________________________
> Distutils-SIG maillist  -  Distutils-SIG at python.org
> http://mail.python.org/mailman/listinfo/distutils-sig


From tarek at ziade.org  Thu Sep 13 10:47:47 2012
From: tarek at ziade.org (=?UTF-8?B?VGFyZWsgWmlhZMOp?=)
Date: Thu, 13 Sep 2012 10:47:47 +0200
Subject: [Distutils] PEP386 Pre releases
In-Reply-To: <CAESYoks+erPUU34s7-=mf4iGmwT2HweeAk-trN=3c6dWzfi14w@mail.gmail.com>
References: <CAESYoks+erPUU34s7-=mf4iGmwT2HweeAk-trN=3c6dWzfi14w@mail.gmail.com>
Message-ID: <50519DB3.3060008@ziade.org>

On 9/13/12 3:21 AM, SAn wrote:
>> On Wed Sep 12 19:47:39 CEST 2012, Donald Stufft wrote:
>>
>> On Wednesday, September 12, 2012 at 1:43 PM, Erik Bray wrote:
>>> That said, this doesn't match my workflow at all. After releasing
>>> "1.0" the next version is going to be "1.1", and any development
>>> pre-release will be "1.1.devX". "1.1a" might not ever even exist. I
>>> think others brought up this critique at the time PEP 386 was being
>>> discussed, but then nothing was ever done about it >_>
>> Yea, this concerned me because 1.1.devX < 1.1a1 < 1.1b1 < 1.1c1 < 1.1
>> is how i've seen it used in the wild. Looks like most everyone i've seen
>> using it so far has been doing it wrong. Don't think ive seen a single
>> person do it right.
> Hi, just yesterday i got bitten by this issue. FYI:
>
> # verlib "pep386" (from https://bitbucket.org/tarek/distutilsversion)
you should use the one at 
http://hg.python.org/distutils2/file/0291648eb2b2/distutils2/version.py

for all your tests, and file bugs at bugs.python.org

verlib is an old version and is properly a bit different

Cheers
Tarek




From tarek at ziade.org  Thu Sep 13 10:48:09 2012
From: tarek at ziade.org (=?UTF-8?B?VGFyZWsgWmlhZMOp?=)
Date: Thu, 13 Sep 2012 10:48:09 +0200
Subject: [Distutils] PEP345 - Distutils2 - Cannot pin to exact version
In-Reply-To: <C9F7E47A96D04158BB871682F7AD6B98@gmail.com>
References: <86B0D55791FB4654A1A219CE1DA82F0E@gmail.com>
	<CAOTD34b8_V81ecjsBxA143_96t6631kLtNTOFE4wOSg6c1FZsw@mail.gmail.com>
	<CAG8k2+6ak7vs=+tvGEfGe+iY99uwGc5t7X9R99H4-n2FZVuP0A@mail.gmail.com>
	<CAOTD34bRSqp7Ex2TP7AOu8A2oWAB+LMk9omJ6u83Ofi4YFBjtg@mail.gmail.com>
	<504F9184.5010806@ziade.org>
	<CAG8k2+5bd=cSB-+keaQw5w9K9s9tx61DfOTSCZf9izo33aWUpg@mail.gmail.com>
	<504F92AB.2080803@ziade.org>
	<C9F7E47A96D04158BB871682F7AD6B98@gmail.com>
Message-ID: <50519DC9.9080504@ziade.org>

On 9/11/12 9:56 PM, Donald Stufft wrote:
> On Tuesday, September 11, 2012 at 3:36 PM, Tarek Ziad? wrote:
>> We need to fix PEP 386 though 
> On the topic of PEP386, It also doesn't specify how versions like 2.5 
> and 2.5.0
> should be treated. That's what originally got me on the thread of how 
> PEP345
> handles the == case. I believe that PEP386 should specify that 
> additional .0's
> on the end of a version are semantically noops. So 2.5 == 2.5.0 == 
> 2.5.0.0 etc.
+1
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20120913/8e1f04eb/attachment-0001.html>

From tarek at ziade.org  Thu Sep 13 10:55:25 2012
From: tarek at ziade.org (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Thu, 13 Sep 2012 10:55:25 +0200
Subject: [Distutils] extension to pep 376: "develop" installation
In-Reply-To: <1347521204.3464.55.camel@thinko>
References: <1347521204.3464.55.camel@thinko>
Message-ID: <50519F7D.1080301@ziade.org>

On 9/13/12 9:26 AM, Chris McDonough wrote:
> I propose an extension to PEP 376 to support the development of projects
> such that:
Since PEP 376 is accepted, we'll need a new PEP for this kind of addition.

>
> - Installation tools should expect to need to cope with files
>    present in purelib named "foo.dist-link".  Each of these files
>    will be of the form "/path/containing/foo.dist-info\nreldir"
>    For example, the contents of such a file might be:
>
>        /home/chrism/projects/zope.interface
>        src

why do we need to place this file in purelib ?

In my understanding, if we add a file in the root of the developed 
project, that lists all
directories that should be included in the python path, we'd only need to
scan that file and add its content in develop.pth

IOW - the only impact in the purelib directory could be a single 
develop.pth file.

Now what if I am not sudoer ? how can I have the develop feature without 
being able to
write develop.pth in purelib ?

Could we find a way to have a per-user *and/or* per local project 
directory, that get
scan by Python at startup ?

Cheers
Tarek



From tarek at ziade.org  Thu Sep 13 10:59:02 2012
From: tarek at ziade.org (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Thu, 13 Sep 2012 10:59:02 +0200
Subject: [Distutils] extension to pep 376: "develop" installation
In-Reply-To: <50519F7D.1080301@ziade.org>
References: <1347521204.3464.55.camel@thinko> <50519F7D.1080301@ziade.org>
Message-ID: <5051A056.2010902@ziade.org>

On 9/13/12 10:55 AM, Tarek Ziad? wrote:
...
>
> Now what if I am not sudoer ? how can I have the develop feature 
> without being able to
> write develop.pth in purelib ?
>
> Could we find a way to have a per-user *and/or* per local project 
> directory, that get
> scan by Python at startup ?
On a second thought, I guess virtualenv fixes this issue..

From chrism at plope.com  Thu Sep 13 11:04:39 2012
From: chrism at plope.com (Chris McDonough)
Date: Thu, 13 Sep 2012 05:04:39 -0400
Subject: [Distutils] optional binary compilation under future
 installation tools
In-Reply-To: <50519CE8.4050102@ziade.org>
References: <1347524152.3464.68.camel@thinko> <50519CE8.4050102@ziade.org>
Message-ID: <1347527079.3464.72.camel@thinko>

On Thu, 2012-09-13 at 10:44 +0200, Tarek Ziad? wrote:
> On 9/13/12 10:15 AM, Chris McDonough wrote:
> > Some packages we maintain currently provide largely identical
> > side-by-side implementations of features: one implementation is written
> > in C, the other implementation is written in Python.  The C module is
> > just an optimized version of the Python code.
> >
> > There is logic at module scope within modules in the same package which
> > attempts to import the C version.  If the C version cannot be imported
> > (an ImportError is raised), the logic falls back to importing the Python
> > version:
> >
> > try:
> >      from . import cmodule as module
> > except ImportError
> >      from . import pymodule as module
> >
> > This means that the package can be used if the C implementation is
> > compiled or not.  It will run more slowly, but that's OK in lots of
> > cases.
> >
> > In our setup.py for these kinds of distributions, we have code which
> > under Py2 uses a setuptools "feature" and under Py3 registers a
> > different kind of build_ext which, at install time, will:
> >
> > - Attempt to compile the C if suitable build tools seem to exist on the
> >    target system.
> >
> > - If suitable build tools don't seem to exist on the target system, will
> >    print a message and continue.
> >
> > How can we support such a feature in the brave new declarative packaging
> > world?
> >
> > - C
> >
> FWIW there's an 'optional' option for the Extension class in Python 2.7 
> in distutils.
> 
> see 
> http://hg.python.org/cpython/file/9b40d018e4cf/Lib/distutils/command/build_ext.py#l458
> 
> If the compilation fails it just warns and continues.
> 
> That option is also present in distutils2, so you can use it in the 
> setup.cfg file.
> 
> Does that work ?

Yep.

- C



From dholth at gmail.com  Thu Sep 13 14:23:01 2012
From: dholth at gmail.com (Daniel Holth)
Date: Thu, 13 Sep 2012 08:23:01 -0400
Subject: [Distutils] Extension: Entry-Point
In-Reply-To: <1347523454.3464.58.camel@thinko>
References: <CAG8k2+5NtD8mXasT3Nob4bC+VjbcTh0pM0WrdH4ZAU3aNp=bTA@mail.gmail.com>
	<1347523454.3464.58.camel@thinko>
Message-ID: <CAG8k2+7_j_Nw8yOSSx0jNDey-nH4VXt4aa+vOZwQCAdL6gBd5g@mail.gmail.com>

On Thu, Sep 13, 2012 at 4:04 AM, Chris McDonough <chrism at plope.com> wrote:
> On Thu, 2012-09-13 at 01:50 -0400, Daniel Holth wrote:
>> I propose mapping a typical entry_points.txt to Metadata 1.3 like so:
>>
>>         [pyramid.scaffold]
>>         starter=pyramid.scaffolds:StarterProjectTemplate
>>         zodb=pyramid.scaffolds:ZODBProjectTemplate
>>         alchemy=pyramid.scaffolds:AlchemyProjectTemplate
>>         [console_scripts]
>>         bfg2pyramid = pyramid.fixers.fix_bfg_imports:main
>>         pcreate = pyramid.scripts.pcreate:main
>>         pserve = pyramid.scripts.pserve:main
>>         pshell = pyramid.scripts.pshell:main
>>         proutes = pyramid.scripts.proutes:main
>>         pviews = pyramid.scripts.pviews:main
>>         ptweens = pyramid.scripts.ptweens:main
>>         prequest = pyramid.scripts.prequest:main
>>         [paste.server_runner]
>>         wsgiref = pyramid.scripts.pserve:wsgiref_server_runner
>>         cherrypy = pyramid.scripts.pserve:cherrypy_server_runner
>>
>>
>> Extension: Entry-Point
>> Entry-Point/pyramid.scaffold: starter=pyramid.scaffolds:StarterProjectTemplate
>> Entry-Point/paste.server_runner: wsgiref =
>> pyramid.scripts.pserve:wsgiref_server_runner
>
> I see nothing wrong with this as long as it's understood that individual
> "Entry-Point/foo" lines can be multiple use.  The spec only indicates
> that the "Entry-Point" moniker itself is multiple use.

Of course. It would be a dumb translation of the old file, with the
same semantics.

Alternatively it could just mention the [sections] and keep entry-points.txt

I don't suppose there is any meaningful performance difference either
way. I think we will have to parse every METADATA most of the time any
packaging queries are made with this system.

From spiccinini at codigosur.org  Thu Sep 13 15:56:04 2012
From: spiccinini at codigosur.org (SAn)
Date: Thu, 13 Sep 2012 10:56:04 -0300
Subject: [Distutils] PEP386 Pre releases
In-Reply-To: <50519DB3.3060008@ziade.org>
References: <CAESYoks+erPUU34s7-=mf4iGmwT2HweeAk-trN=3c6dWzfi14w@mail.gmail.com>
	<50519DB3.3060008@ziade.org>
Message-ID: <CAESYokvELcaX1mDgbG_F0Z5b6WkTb5yNd6f9-NRbgw1_YG+VNA@mail.gmail.com>

On Thu, Sep 13, 2012 at 5:47 AM, Tarek Ziad? <tarek at ziade.org> wrote:
> On 9/13/12 3:21 AM, SAn wrote:
>>>
>>> On Wed Sep 12 19:47:39 CEST 2012, Donald Stufft wrote:
>>>
>>> On Wednesday, September 12, 2012 at 1:43 PM, Erik Bray wrote:
>>>>
>>>> That said, this doesn't match my workflow at all. After releasing
>>>> "1.0" the next version is going to be "1.1", and any development
>>>> pre-release will be "1.1.devX". "1.1a" might not ever even exist. I
>>>> think others brought up this critique at the time PEP 386 was being
>>>> discussed, but then nothing was ever done about it >_>
>>>
>>> Yea, this concerned me because 1.1.devX < 1.1a1 < 1.1b1 < 1.1c1 < 1.1
>>> is how i've seen it used in the wild. Looks like most everyone i've seen
>>> using it so far has been doing it wrong. Don't think ive seen a single
>>> person do it right.
>>
>> Hi, just yesterday i got bitten by this issue. FYI:
>>
>> # verlib "pep386" (from https://bitbucket.org/tarek/distutilsversion)
>
> you should use the one at
> http://hg.python.org/distutils2/file/0291648eb2b2/distutils2/version.py
>
> for all your tests, and file bugs at bugs.python.org
>
> verlib is an old version and is properly a bit different
>

Thanks Tarek. The code at distutils2 doesn't have that bug :)

Cheers,
SAn

From dholth at gmail.com  Thu Sep 13 17:26:21 2012
From: dholth at gmail.com (Daniel Holth)
Date: Thu, 13 Sep 2012 11:26:21 -0400
Subject: [Distutils] [Python-Dev] packaging location ?
In-Reply-To: <CAOTD34ZONfYrSBni6Z5=uMKK0--0mu4p-0DtAV80kux_48KBPg@mail.gmail.com>
References: <50503E19.1030604@ziade.org> <5050DC52.508@netwok.org>
	<CAL0kPAUzY5zyPob==fnyf5NDwhSvwRqrRUhJeBMYNPnkR52UGw@mail.gmail.com>
	<CAP1=2W7r-Zj_kQknok+FS-6DpxoznBnc0gzjowh_xg3zZ+g_=w@mail.gmail.com>
	<20120912224308.56B48250180@webabinitio.net>
	<CADiSq7d67906UWn4LaLqKZv_SyF0XsaJWTBQk0j-a=mSSUwqrw@mail.gmail.com>
	<20120913113857.69dafcb1@pitrou.net>
	<CAOTD34ZONfYrSBni6Z5=uMKK0--0mu4p-0DtAV80kux_48KBPg@mail.gmail.com>
Message-ID: <CAG8k2+5k0-eWDOtjbga9RCox4w6t4qkHNfhiXGCtrExxKK-oFQ@mail.gmail.com>

Bento's documentation and source code should be required reading for
anyone doing Python packaging work. It is actually well designed,
which comes as something of a shock if you are accustomed to hacking
on nested-subcommands-that-initialize-and-call-each-other. Bento
cleanly separates the build and install phases, it includes an
intermediate JSON metadata "ipkg.info" with the PEP metadata, install
paths, and file lists, it is easy to hack on. It needs a better lexer.

https://gist.github.com/3715068 - the intermediate (not human
editable) package info.

From pje at telecommunity.com  Thu Sep 13 22:24:59 2012
From: pje at telecommunity.com (PJ Eby)
Date: Thu, 13 Sep 2012 16:24:59 -0400
Subject: [Distutils] Extension: Entry-Point
In-Reply-To: <CAG8k2+7_j_Nw8yOSSx0jNDey-nH4VXt4aa+vOZwQCAdL6gBd5g@mail.gmail.com>
References: <CAG8k2+5NtD8mXasT3Nob4bC+VjbcTh0pM0WrdH4ZAU3aNp=bTA@mail.gmail.com>
	<1347523454.3464.58.camel@thinko>
	<CAG8k2+7_j_Nw8yOSSx0jNDey-nH4VXt4aa+vOZwQCAdL6gBd5g@mail.gmail.com>
Message-ID: <CALeMXf6nhkwkjZCouRZVtUdpewqfgKwP3mTCtL=F+hL6UEeWUw@mail.gmail.com>

On Thu, Sep 13, 2012 at 8:23 AM, Daniel Holth <dholth at gmail.com> wrote:
> On Thu, Sep 13, 2012 at 4:04 AM, Chris McDonough <chrism at plope.com> wrote:
>> On Thu, 2012-09-13 at 01:50 -0400, Daniel Holth wrote:
>>> I propose mapping a typical entry_points.txt to Metadata 1.3 like so:
>>>
>>>         [pyramid.scaffold]
>>>         starter=pyramid.scaffolds:StarterProjectTemplate
>>>         zodb=pyramid.scaffolds:ZODBProjectTemplate
>>>         alchemy=pyramid.scaffolds:AlchemyProjectTemplate
>>>         [console_scripts]
>>>         bfg2pyramid = pyramid.fixers.fix_bfg_imports:main
>>>         pcreate = pyramid.scripts.pcreate:main
>>>         pserve = pyramid.scripts.pserve:main
>>>         pshell = pyramid.scripts.pshell:main
>>>         proutes = pyramid.scripts.proutes:main
>>>         pviews = pyramid.scripts.pviews:main
>>>         ptweens = pyramid.scripts.ptweens:main
>>>         prequest = pyramid.scripts.prequest:main
>>>         [paste.server_runner]
>>>         wsgiref = pyramid.scripts.pserve:wsgiref_server_runner
>>>         cherrypy = pyramid.scripts.pserve:cherrypy_server_runner
>>>
>>>
>>> Extension: Entry-Point
>>> Entry-Point/pyramid.scaffold: starter=pyramid.scaffolds:StarterProjectTemplate
>>> Entry-Point/paste.server_runner: wsgiref =
>>> pyramid.scripts.pserve:wsgiref_server_runner
>>
>> I see nothing wrong with this as long as it's understood that individual
>> "Entry-Point/foo" lines can be multiple use.  The spec only indicates
>> that the "Entry-Point" moniker itself is multiple use.
>
> Of course. It would be a dumb translation of the old file, with the
> same semantics.
>
> Alternatively it could just mention the [sections] and keep entry-points.txt
>
> I don't suppose there is any meaningful performance difference either
> way. I think we will have to parse every METADATA most of the time any
> packaging queries are made with this system.
> _______________________________________________
> Distutils-SIG maillist  -  Distutils-SIG at python.org
> http://mail.python.org/mailman/listinfo/distutils-sig


I don't see the point of jamming all metadata into a single file.
Well, I mean, I do see it if you want to pull a single file off of
PyPI and know everything about a distribution.

But for runtime performance, separate files are better because then
you don't have to actually read them for a bunch of projects with no
entry points.  Putting the entry points in a file that *every*
installed distribution has is going to positively murder the
performance.

From vs at it.uu.se  Fri Sep 14 11:14:04 2012
From: vs at it.uu.se (Virgil Stokes)
Date: Fri, 14 Sep 2012 11:14:04 +0200
Subject: [Distutils] [Python-Dev] packaging location ?
In-Reply-To: <CAG8k2+5k0-eWDOtjbga9RCox4w6t4qkHNfhiXGCtrExxKK-oFQ@mail.gmail.com>
References: <50503E19.1030604@ziade.org> <5050DC52.508@netwok.org>
	<CAL0kPAUzY5zyPob==fnyf5NDwhSvwRqrRUhJeBMYNPnkR52UGw@mail.gmail.com>
	<CAP1=2W7r-Zj_kQknok+FS-6DpxoznBnc0gzjowh_xg3zZ+g_=w@mail.gmail.com>
	<20120912224308.56B48250180@webabinitio.net>
	<CADiSq7d67906UWn4LaLqKZv_SyF0XsaJWTBQk0j-a=mSSUwqrw@mail.gmail.com>
	<20120913113857.69dafcb1@pitrou.net>
	<CAOTD34ZONfYrSBni6Z5=uMKK0--0mu4p-0DtAV80kux_48KBPg@mail.gmail.com>
	<CAG8k2+5k0-eWDOtjbga9RCox4w6t4qkHNfhiXGCtrExxKK-oFQ@mail.gmail.com>
Message-ID: <5052F55C.1050009@it.uu.se>

On 13-Sep-2012 17:26, Daniel Holth wrote:
> Bento's documentation and source code should be required reading for
> anyone doing Python packaging work. It is actually well designed,
> which comes as something of a shock if you are accustomed to hacking
> on nested-subcommands-that-initialize-and-call-each-other. Bento
> cleanly separates the build and install phases, it includes an
> intermediate JSON metadata "ipkg.info" with the PEP metadata, install
> paths, and file lists, it is easy to hack on. It needs a better lexer.
>
> https://gist.github.com/3715068 - the intermediate (not human
> editable) package info.
> _______________________________________________
> Distutils-SIG maillist  -  Distutils-SIG at python.org
> http://mail.python.org/mailman/listinfo/distutils-sig
I downloaded the compressed file; but, found no documentation on the use of this 
file --- where can I get information on how to use this file?

Best,
--V

From dholth at gmail.com  Fri Sep 14 18:21:05 2012
From: dholth at gmail.com (Daniel Holth)
Date: Fri, 14 Sep 2012 12:21:05 -0400
Subject: [Distutils] Extension: Entry-Point
In-Reply-To: <CALeMXf6nhkwkjZCouRZVtUdpewqfgKwP3mTCtL=F+hL6UEeWUw@mail.gmail.com>
References: <CAG8k2+5NtD8mXasT3Nob4bC+VjbcTh0pM0WrdH4ZAU3aNp=bTA@mail.gmail.com>
	<1347523454.3464.58.camel@thinko>
	<CAG8k2+7_j_Nw8yOSSx0jNDey-nH4VXt4aa+vOZwQCAdL6gBd5g@mail.gmail.com>
	<CALeMXf6nhkwkjZCouRZVtUdpewqfgKwP3mTCtL=F+hL6UEeWUw@mail.gmail.com>
Message-ID: <CAG8k2+5fBuh6UUB5zvDKz-je+goTFqxrFxXpnNjzOLTTvGb3yw@mail.gmail.com>

> I don't see the point of jamming all metadata into a single file.
> Well, I mean, I do see it if you want to pull a single file off of
> PyPI and know everything about a distribution.
>
> But for runtime performance, separate files are better because then
> you don't have to actually read them for a bunch of projects with no
> entry points.  Putting the entry points in a file that *every*
> installed distribution has is going to positively murder the
> performance.

Agreed. I will let entry_points.txt stay as is.

From dholth at gmail.com  Sat Sep 15 14:22:57 2012
From: dholth at gmail.com (Daniel Holth)
Date: Sat, 15 Sep 2012 08:22:57 -0400
Subject: [Distutils] add hash algorithm agility to RECORD
In-Reply-To: <CACac1F8qYLm5KfXBi7rMsOMQxFdjZ68eCb7HkWxN=U+KmUTJnQ@mail.gmail.com>
References: <CAG8k2+4XBF3XSnsEROrgq=OOkrQ7fKO231+3jXhg6B1TEgpnig@mail.gmail.com>
	<CACac1F_K1Bt0+rPxt7f7sqGWcHHJQTJboq8V6Gu6DtvD6frXLg@mail.gmail.com>
	<CAG8k2+6sL0ytKEAHT=0ktEXXv1K+OLYkMFgikxGJB-3c94wz0A@mail.gmail.com>
	<CAG8k2+5bp8cVG6pAwiyFtbF60Pc_EXJFxu9pj9FSp1ZEj08V9w@mail.gmail.com>
	<CACac1F8qYLm5KfXBi7rMsOMQxFdjZ68eCb7HkWxN=U+KmUTJnQ@mail.gmail.com>
Message-ID: <CAG8k2+62h8-dUAKXa-892+LUomFCsdenJFP5MBgAqKPcW0e6DA@mail.gmail.com>

OK. Making MD5 optional in RECORD doesn't seem to be very
controversial any more, and it will make life easier for an entire
class of systems that compile md5() to produce a crash instead of a
message digest. It will not lull anyone into a false sense of security
or degrade performance.

The edit now suggests the installer pick from one of
hashlib.algorithms_guaranteed, currently:

{'sha1', 'sha224', 'sha384', 'sha256', 'sha512', 'md5'}

All valid arguments to hashlib.new(). The hash value is now either
empty, the md5 hexdigest, or the name of the hash, followed by =,
followed by the urlsafe-b64encode-nopad (base64 with trailing =
removed) of the digest.

Does an uninstaller exist that checks the hashes during uninstall?

Daniel Holth

From dholth at gmail.com  Sun Sep 16 05:17:22 2012
From: dholth at gmail.com (Daniel Holth)
Date: Sat, 15 Sep 2012 23:17:22 -0400
Subject: [Distutils] No >, >=, <, <= in environment markers spec
Message-ID: <CAG8k2+6j0U38qGE=Zs5O9i_zD7BG=7vB+uExdhsx6Q15_PYrKg@mail.gmail.com>

Noticed that there is no >= etc. in the environment markers spec, but
both environment markers implementations allow it. Shall I fix the
spec?

As of wheel 0.9.5, you can add to your setup.cfg lines to be included
as Requires-Dist: in the output METADATA. They supplement setup.py's
values. Only problem is they are only resolved during install from
wheel, and not during an install directly from the sdist.

[metadata]
requires_dist = beaglevote; python_version == 2.7
   reportlab
   quux; sys.platform == 'win32'

From dholth at gmail.com  Sun Sep 16 05:19:49 2012
From: dholth at gmail.com (Daniel Holth)
Date: Sat, 15 Sep 2012 23:19:49 -0400
Subject: [Distutils] zip imports ftw
Message-ID: <CAG8k2+48MMoMWeXDFJpSQq63p8ebMew-VM5WOOe6qx_YoiieKA@mail.gmail.com>

I'd like to take this moment to note how awesome importing from
zipfile is. Did you know you can execute a zip file with Python, and
if it has a __main__.py in the root it will be executed, with the
zipfile's contents on the PYTHONPATH?

file.zip:
__main__.py
somedep.py
other/code.py

python file.zip # runs!

From chris.jerdonek at gmail.com  Sun Sep 16 05:44:21 2012
From: chris.jerdonek at gmail.com (Chris Jerdonek)
Date: Sat, 15 Sep 2012 20:44:21 -0700
Subject: [Distutils] zip imports ftw
In-Reply-To: <CAG8k2+48MMoMWeXDFJpSQq63p8ebMew-VM5WOOe6qx_YoiieKA@mail.gmail.com>
References: <CAG8k2+48MMoMWeXDFJpSQq63p8ebMew-VM5WOOe6qx_YoiieKA@mail.gmail.com>
Message-ID: <CAOTb1wcLiYUrV2=tsKUietmmM3zQQtrQuyqGbk1b42hyQN4GHg@mail.gmail.com>

On Sat, Sep 15, 2012 at 8:19 PM, Daniel Holth <dholth at gmail.com> wrote:
> I'd like to take this moment to note how awesome importing from
> zipfile is. Did you know you can execute a zip file with Python, and
> if it has a __main__.py in the root it will be executed, with the
> zipfile's contents on the PYTHONPATH?

I've known this, but can you say a little about the caveats (e.g. why
you wouldn't always want to store your packages as zip files)?  For
example, IIRC you can't really use __file__ from within such modules.

--Chris

From dholth at gmail.com  Sun Sep 16 14:07:55 2012
From: dholth at gmail.com (Daniel Holth)
Date: Sun, 16 Sep 2012 08:07:55 -0400
Subject: [Distutils] zip imports ftw
In-Reply-To: <CAOTb1wcLiYUrV2=tsKUietmmM3zQQtrQuyqGbk1b42hyQN4GHg@mail.gmail.com>
References: <CAG8k2+48MMoMWeXDFJpSQq63p8ebMew-VM5WOOe6qx_YoiieKA@mail.gmail.com>
	<CAOTb1wcLiYUrV2=tsKUietmmM3zQQtrQuyqGbk1b42hyQN4GHg@mail.gmail.com>
Message-ID: <1B50CBF2-804D-4BFE-B5CE-3D26940507B3@gmail.com>

Of course the major potentially non obvious caveats are that a lot of python code doesn't use the API to load data from the importer path, instead using __file__ and assuming everything is one, and that sometimes (shared libraries) the system can't load it unless its a real file.

Daniel Holth

On Sep 15, 2012, at 11:44 PM, Chris Jerdonek <chris.jerdonek at gmail.com> wrote:

> On Sat, Sep 15, 2012 at 8:19 PM, Daniel Holth <dholth at gmail.com> wrote:
>> I'd like to take this moment to note how awesome importing from
>> zipfile is. Did you know you can execute a zip file with Python, and
>> if it has a __main__.py in the root it will be executed, with the
>> zipfile's contents on the PYTHONPATH?
> 
> I've known this, but can you say a little about the caveats (e.g. why
> you wouldn't always want to store your packages as zip files)?  For
> example, IIRC you can't really use __file__ from within such modules.
> 
> --Chris

From chrism at plope.com  Sun Sep 16 19:23:43 2012
From: chrism at plope.com (Chris McDonough)
Date: Sun, 16 Sep 2012 13:23:43 -0400
Subject: [Distutils] zip imports ftw
In-Reply-To: <1B50CBF2-804D-4BFE-B5CE-3D26940507B3@gmail.com>
References: <CAG8k2+48MMoMWeXDFJpSQq63p8ebMew-VM5WOOe6qx_YoiieKA@mail.gmail.com>
	<CAOTb1wcLiYUrV2=tsKUietmmM3zQQtrQuyqGbk1b42hyQN4GHg@mail.gmail.com>
	<1B50CBF2-804D-4BFE-B5CE-3D26940507B3@gmail.com>
Message-ID: <1347816223.3426.7.camel@thinko>

On Sun, 2012-09-16 at 08:07 -0400, Daniel Holth wrote:
> Of course the major potentially non obvious caveats are that a lot of python code doesn't use the API to load data from the importer path, instead using __file__ and assuming everything is one, and that sometimes (shared libraries) the system can't load it unless its a real file.

This is easy to work around in brand new code using pkg_resources, so
it's generally only a problem with older code, or code which can depend
only on the stdlib.  Unfortunately, the stdlib doesn't have much to help
here.  There's pkgutil.get_data but it doesn't really work well if you
need to get a filelike handle to a resource, or you need to list the
contents of a directory, and other things which pkg_resources does very
nicely, so it seems like Python isn't really serious about allowing code
in zipfiles to be a viable alternative to file-and-directory-based code.

- C



From dholth at gmail.com  Sun Sep 16 19:41:07 2012
From: dholth at gmail.com (Daniel Holth)
Date: Sun, 16 Sep 2012 13:41:07 -0400
Subject: [Distutils] zip imports ftw
In-Reply-To: <1347816223.3426.7.camel@thinko>
References: <CAG8k2+48MMoMWeXDFJpSQq63p8ebMew-VM5WOOe6qx_YoiieKA@mail.gmail.com>
	<CAOTb1wcLiYUrV2=tsKUietmmM3zQQtrQuyqGbk1b42hyQN4GHg@mail.gmail.com>
	<1B50CBF2-804D-4BFE-B5CE-3D26940507B3@gmail.com>
	<1347816223.3426.7.camel@thinko>
Message-ID: <CAG8k2+6paxOumii6U+iTrza2KN7d=nCuQHwBeN-SbH1NKYLB=g@mail.gmail.com>

> This is easy to work around in brand new code using pkg_resources, so
> it's generally only a problem with older code, or code which can depend
> only on the stdlib.  Unfortunately, the stdlib doesn't have much to help
> here.  There's pkgutil.get_data but it doesn't really work well if you
> need to get a filelike handle to a resource, or you need to list the
> contents of a directory, and other things which pkg_resources does very
> nicely, so it seems like Python isn't really serious about allowing code
> in zipfiles to be a viable alternative to file-and-directory-based code.

I got the impression that killing zip imports is an explicit goal of
packaging. This is sure to be a contentious issue.

To all the haters, let me say that no one is going to make you import
code from a zip if you don't want to. That easy_install mistake will
not be repeated. However, those of us who like importing from places
other than the filesystem should be able to enjoy continually
improving support for doing so. Also, remember that thing about the
debugger not showing the source code for zipped code? Fixed.

From p.f.moore at gmail.com  Sun Sep 16 20:54:21 2012
From: p.f.moore at gmail.com (Paul Moore)
Date: Sun, 16 Sep 2012 19:54:21 +0100
Subject: [Distutils] zip imports ftw
In-Reply-To: <CAG8k2+6paxOumii6U+iTrza2KN7d=nCuQHwBeN-SbH1NKYLB=g@mail.gmail.com>
References: <CAG8k2+48MMoMWeXDFJpSQq63p8ebMew-VM5WOOe6qx_YoiieKA@mail.gmail.com>
	<CAOTb1wcLiYUrV2=tsKUietmmM3zQQtrQuyqGbk1b42hyQN4GHg@mail.gmail.com>
	<1B50CBF2-804D-4BFE-B5CE-3D26940507B3@gmail.com>
	<1347816223.3426.7.camel@thinko>
	<CAG8k2+6paxOumii6U+iTrza2KN7d=nCuQHwBeN-SbH1NKYLB=g@mail.gmail.com>
Message-ID: <CACac1F_SwcfB+0v5__qTURvnNG0UnpDtzOhtR9NNLbFNE=fV5A@mail.gmail.com>

On 16 September 2012 18:41, Daniel Holth <dholth at gmail.com> wrote:
>> This is easy to work around in brand new code using pkg_resources, so
>> it's generally only a problem with older code, or code which can depend
>> only on the stdlib.  Unfortunately, the stdlib doesn't have much to help
>> here.  There's pkgutil.get_data but it doesn't really work well if you
>> need to get a filelike handle to a resource, or you need to list the
>> contents of a directory, and other things which pkg_resources does very
>> nicely, so it seems like Python isn't really serious about allowing code
>> in zipfiles to be a viable alternative to file-and-directory-based code.
>
> I got the impression that killing zip imports is an explicit goal of
> packaging. This is sure to be a contentious issue.

I think packaging possibly only considered installing *to* filesystem
locations. But that's no different from distutils - at worst,
packaging explicitly ignored the setuptools features around creating
eggs (importable zip formats). It's not that you can't do it, just
that packaging didn't want to get sucked into that area and so you're
on your own.

Python *itself* very definitely supports zip imports. And with the new
implementation of the core import machinery using importlib in Python
3.3, it goes a lot further, supporting imports from any locations you
care to write the code for. There's no core support for putting your
package into (say) a sqlite database, but once it's in there, the core
supports you adding hooks in so you can put that sqlite database on
sys.path and have it "just work". That's a huge amount of flexibility,
and no-one has even scratched the surface of it yet.

Paul.

From barry at python.org  Sun Sep 16 21:12:26 2012
From: barry at python.org (Barry Warsaw)
Date: Sun, 16 Sep 2012 15:12:26 -0400
Subject: [Distutils] zip imports ftw
In-Reply-To: <1347816223.3426.7.camel@thinko>
References: <CAG8k2+48MMoMWeXDFJpSQq63p8ebMew-VM5WOOe6qx_YoiieKA@mail.gmail.com>
	<CAOTb1wcLiYUrV2=tsKUietmmM3zQQtrQuyqGbk1b42hyQN4GHg@mail.gmail.com>
	<1B50CBF2-804D-4BFE-B5CE-3D26940507B3@gmail.com>
	<1347816223.3426.7.camel@thinko>
Message-ID: <8F8983CF-FD94-4831-8753-44A1EE61BB7E@python.org>

On Sep 16, 2012, at 1:23 PM, Chris McDonough wrote:

> On Sun, 2012-09-16 at 08:07 -0400, Daniel Holth wrote:
>> Of course the major potentially non obvious caveats are that a lot of python code doesn't use the API to load data from the importer path, instead using __file__ and assuming everything is one, and that sometimes (shared libraries) the system can't load it unless its a real file.
> 
> This is easy to work around in brand new code using pkg_resources, so
> it's generally only a problem with older code, or code which can depend
> only on the stdlib.  Unfortunately, the stdlib doesn't have much to help
> here.  There's pkgutil.get_data but it doesn't really work well if you
> need to get a filelike handle to a resource, or you need to list the
> contents of a directory, and other things which pkg_resources does very
> nicely, so it seems like Python isn't really serious about allowing code
> in zipfiles to be a viable alternative to file-and-directory-based code.

Remind me again why pkg_resources (or an API exactly like it) isn't in the stdlib already?

-Barry


From dholth at gmail.com  Sun Sep 16 21:33:08 2012
From: dholth at gmail.com (Daniel Holth)
Date: Sun, 16 Sep 2012 15:33:08 -0400
Subject: [Distutils] zip imports ftw
In-Reply-To: <8F8983CF-FD94-4831-8753-44A1EE61BB7E@python.org>
References: <CAG8k2+48MMoMWeXDFJpSQq63p8ebMew-VM5WOOe6qx_YoiieKA@mail.gmail.com>
	<CAOTb1wcLiYUrV2=tsKUietmmM3zQQtrQuyqGbk1b42hyQN4GHg@mail.gmail.com>
	<1B50CBF2-804D-4BFE-B5CE-3D26940507B3@gmail.com>
	<1347816223.3426.7.camel@thinko>
	<8F8983CF-FD94-4831-8753-44A1EE61BB7E@python.org>
Message-ID: <CAG8k2+6Ne7VrN-WZQNio2zM+49BV2ec9ohn5mJy_rB8ov+hXoA@mail.gmail.com>

It is because the file has a controversial coding style, with up to 25
lines of whitespace between classes, and includes functionality for
eggs and some other stuff that was unwanted on top of the resources
stuff that partly made its way into pkgutil.

I am really impressed with the file after seeing how easy it was to
add PEP-345 and PEP-376 support to pkg_resources.py without having to
present a different API to the user, and seeing how it provides a lot
of what an installer needs especially being able to iterate over all
the distributions in a particular folder, but I am glad it is not in
the stdlib because I wouldn't have been able to do that if it was.

From chrism at plope.com  Sun Sep 16 21:40:07 2012
From: chrism at plope.com (Chris McDonough)
Date: Sun, 16 Sep 2012 15:40:07 -0400
Subject: [Distutils] zip imports ftw
In-Reply-To: <CAG8k2+6Ne7VrN-WZQNio2zM+49BV2ec9ohn5mJy_rB8ov+hXoA@mail.gmail.com>
References: <CAG8k2+48MMoMWeXDFJpSQq63p8ebMew-VM5WOOe6qx_YoiieKA@mail.gmail.com>
	<CAOTb1wcLiYUrV2=tsKUietmmM3zQQtrQuyqGbk1b42hyQN4GHg@mail.gmail.com>
	<1B50CBF2-804D-4BFE-B5CE-3D26940507B3@gmail.com>
	<1347816223.3426.7.camel@thinko>
	<8F8983CF-FD94-4831-8753-44A1EE61BB7E@python.org>
	<CAG8k2+6Ne7VrN-WZQNio2zM+49BV2ec9ohn5mJy_rB8ov+hXoA@mail.gmail.com>
Message-ID: <1347824407.3426.14.camel@thinko>

On Sun, 2012-09-16 at 15:33 -0400, Daniel Holth wrote:
> It is because the file has a controversial coding style, with up to 25
> lines of whitespace between classes, and includes functionality for
> eggs and some other stuff that was unwanted on top of the resources
> stuff that partly made its way into pkgutil.

I think maybe there's also a latent notion floating around that "a
resource (aka non-Python files) shouldn't be addressable within Python
packages" aka. "Python should only make it easy to address Python files
in Python packages, but no other kind of file".  I'm not sure I've ever
seen any of the core dev team express this opinion, but I've seen it
floating around elsewhere.

- C



From dholth at gmail.com  Sun Sep 16 21:42:31 2012
From: dholth at gmail.com (Daniel Holth)
Date: Sun, 16 Sep 2012 15:42:31 -0400
Subject: [Distutils] zip imports ftw
In-Reply-To: <1347824407.3426.14.camel@thinko>
References: <CAG8k2+48MMoMWeXDFJpSQq63p8ebMew-VM5WOOe6qx_YoiieKA@mail.gmail.com>
	<CAOTb1wcLiYUrV2=tsKUietmmM3zQQtrQuyqGbk1b42hyQN4GHg@mail.gmail.com>
	<1B50CBF2-804D-4BFE-B5CE-3D26940507B3@gmail.com>
	<1347816223.3426.7.camel@thinko>
	<8F8983CF-FD94-4831-8753-44A1EE61BB7E@python.org>
	<CAG8k2+6Ne7VrN-WZQNio2zM+49BV2ec9ohn5mJy_rB8ov+hXoA@mail.gmail.com>
	<1347824407.3426.14.camel@thinko>
Message-ID: <CAG8k2+5vf6--zkVpz9kBG9XAKdLL_USTuk8EqxsYPOt9orX4bA@mail.gmail.com>

On Sun, Sep 16, 2012 at 3:40 PM, Chris McDonough <chrism at plope.com> wrote:
> On Sun, 2012-09-16 at 15:33 -0400, Daniel Holth wrote:
>> It is because the file has a controversial coding style, with up to 25
>> lines of whitespace between classes, and includes functionality for
>> eggs and some other stuff that was unwanted on top of the resources
>> stuff that partly made its way into pkgutil.
>
> I think maybe there's also a latent notion floating around that "a
> resource (aka non-Python files) shouldn't be addressable within Python
> packages" aka. "Python should only make it easy to address Python files
> in Python packages, but no other kind of file".  I'm not sure I've ever
> seen any of the core dev team express this opinion, but I've seen it
> floating around elsewhere.

I think we should talk about something else.

Let's talk about how to replace the stdlib's implementation of the
packaging API with another implementation, like import hooks. How
would that work? Should we define a strict API/implementation
separation by using the amazing apipkg, and then let Python choose the
actual implementation based on an environment variable/site.py/.pth
file with 'import; do something'; or perhaps a configure.zcml in
site-packages with zope.interface?

From vinay_sajip at yahoo.co.uk  Mon Sep 17 02:24:45 2012
From: vinay_sajip at yahoo.co.uk (Vinay Sajip)
Date: Mon, 17 Sep 2012 01:24:45 +0100 (BST)
Subject: [Distutils] [Python-Dev] packaging location ?
In-Reply-To: <CACac1F_LBUpPm=DVmjh9KjKFtQsxNyS+=Xi3ab1PYWnRTiWkBg@mail.gmail.com>
References: <50503E19.1030604@ziade.org> <5050DC52.508@netwok.org>
	<CAL0kPAUzY5zyPob==fnyf5NDwhSvwRqrRUhJeBMYNPnkR52UGw@mail.gmail.com>
	<CAP1=2W7r-Zj_kQknok+FS-6DpxoznBnc0gzjowh_xg3zZ+g_=w@mail.gmail.com>
	<20120912224308.56B48250180@webabinitio.net>
	<CADiSq7d67906UWn4LaLqKZv_SyF0XsaJWTBQk0j-a=mSSUwqrw@mail.gmail.com>
	<20120913113857.69dafcb1@pitrou.net> <5051C453.3080003@ziade.org>
	<CAOTb1wdiumKfTZKRw-0n6Z8D6y3gvV+MKE4AkY0h2pQgty6Hag@mail.gmail.com>
	<CADiSq7cQVYSz50msJh9raeFH1L4xZjgdLA3vkaPU4wyvYkTy-A@mail.gmail.com>
	<5051F00D.3040807@netwok.org>
	<CADiSq7cDMKmiSW+nE1oLhqL3zgGa1zs2Sh14xmK9JbedQgY=bw@mail.gmail.com>
	<loom.20120914T165543-69@post.gmane.org>
	<CACac1F9vQn17=x07-3e0uL_7LkqRAthO2CYoVRHcRxdeg---gw@mail.gmail.com>
	<loom.20120917T001634-295@post.gmane.org>
	<CACac1F_LBUpPm=DVmjh9KjKFtQsxNyS+=Xi3ab1PYWnRTiWkBg@mail.gmail.com>
Message-ID: <1347841485.5303.YahooMailNeo@web171404.mail.ir2.yahoo.com>

[moved from python-dev]

> Possibly also useful (if only to standardise the approach) would be a

> method to check hashes against installed files.

Added as Distribution.check_installed_files() in distlib repo[1]. Docstring:

"""
Checks that the hashes and sizes of the files in ``RECORD`` are
matched by the files themselves. Returns a (possibly empty) list of
mismatches. Each entry in the mismatch list will be a tuple consisting
of the path, 'exists', 'size' or 'hash' according to what didn't match
(existence is checked first, then size, then hash), the expected
value and the actual value.
"""

Regards,

Vinay Sajip

[1] https://bitbucket.org/vinay.sajip/distlib/

From dholth at gmail.com  Mon Sep 17 03:40:01 2012
From: dholth at gmail.com (Daniel Holth)
Date: Sun, 16 Sep 2012 21:40:01 -0400
Subject: [Distutils] .egg-info metadata
Message-ID: <CAG8k2+5oj9zZRdjQ42_6SDD6zeLzZ7cXcmaA0Vq2DVEnDoZQ9w@mail.gmail.com>

> I've been chatting to Chris McDonough a bit as well, and one
> potentially useful thing would be to clearly document the
> setuptools/distribute metadata precisely as it is generated today.
> Currently these formats are entirely implicit in the implementation of
> the code that reads and writes them, as far as I can tell anyway. The
> distribute docs seem to do a decent job of explaining setup.py and the
> various setuptools specific arguments, but *not* what the file formats
> will look like inside the metadata directory when installed.
>
> The main advantages of this would be to make it clear:
> 1. What can setuptools metadata describe, that v1.2 of the official
> metadata standard cannot?
> 2. Does v1.3 allow everything that setuptools can currently describe
> (either directly, or as an extension)?
> 3. Does v1.3 allow some things to be expressed more clearly than they
> can be with setuptools?

The big new feature with Metadata 1.2 and up that has no
representation in setuptools is the environment markers specification,
which for common cases like 'require x when Python version is y'
allows the generated requirements file to be the same no matter which
version of Python was used to produce the dist:

Requires-Dist: argparse; python_version < '2.7'

would be a common example. The only thing I don't like about it is
that I cannot remember whether to use _ or . to separate the words.
Metadata 1.3 restores extras from setuptools by adding 'extra' as a
variable in environment markers.

In setuptools:

requires.txt:
normal
requirements
not-in-a-section

[extra_name]
extra_requirement

Nothing prevents an installer from generating requires.txt from
Metadata 1.3, but you can't go the other way due to the environment
markers.


entry_points.txt is the plugin mechanism. I am leaving this unchanged
in my implementation; I'm not motivated to put them into PKG-INFO and
there's no benefit. Each section is a kind of entry point, and each
entry is

key = module.name:callable

Split on the colon :, import the first part, return the second part.
console_scripts is the most widely used by far, but thousands of
packages provide other kinds of entry points. I wish pypi was smart
enough to index this file.

entry_points.txt:
[console_scripts]
wininst2wheel = wheel.wininst2wheel:main
egg2wheel = wheel.egg2wheel:main
wheel = wheel.__main__:main

[distutils.commands]
bdist_wheel = wheel.bdist_wheel:bdist_wheel


top_level.txt is just a list of top-level packages defined by the
dist, one per line. Is this still used?

namespace_packages.txt - same format as top_level.txt. Is used.

SOURCES.txt - list of source files, one per line. Not used.

not-zip-safe: present if package should not be zipped up. empty.

dependency_links.txt: url's of the package's dependencies. Speak up if
you use this; it is very surprising, and has a much better replacement
with pip --index options and requirements files.


Provides works the same way in setuptools, it is in PKG-INFO.

From a.klein at science-applied.nl  Mon Sep 17 09:44:44 2012
From: a.klein at science-applied.nl (Almar Klein)
Date: Mon, 17 Sep 2012 09:44:44 +0200
Subject: [Distutils] Confused about distutils2/packaging
Message-ID: <CAB_T6=mtSCHN95czWv26z_uqQjLmFcoAbF3sAojTTPeCQfB+Ug@mail.gmail.com>

Hi,

I'm eagerly looking forward to the improvements that distutils2/packaging
will (hopefully) bring, and I very much appreciate the efforts of those
involved.

However, I'm a bit confused about the status. On
http://pypi.python.org/pypi/Distutils2 it says packaging is included in
Python 3.3, and on http://docs.python.org/dev/packaging/ there is guidance
for how to write a .cfg file. On the other hand, on
http://python.org/dev/peps/pep-0398/ (the 3.3 release schedule) it says
that the inclusion of packaging is deferred.

Which is it? Maybe it's a good idea to make the different docs consistent.

Thanks,
  Almar

-- 
Almar Klein, PhD
Science Applied
phone: +31 6 19268652
e-mail: a.klein at science-applied.nl
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20120917/0dd44ed4/attachment.html>

From donald.stufft at gmail.com  Mon Sep 17 09:54:16 2012
From: donald.stufft at gmail.com (Donald Stufft)
Date: Mon, 17 Sep 2012 03:54:16 -0400
Subject: [Distutils] Confused about distutils2/packaging
In-Reply-To: <CAB_T6=mtSCHN95czWv26z_uqQjLmFcoAbF3sAojTTPeCQfB+Ug@mail.gmail.com>
References: <CAB_T6=mtSCHN95czWv26z_uqQjLmFcoAbF3sAojTTPeCQfB+Ug@mail.gmail.com>
Message-ID: <9B6D65CA77B94EE489C7260E6915E22E@gmail.com>

It part of 3.3 for awhile, but it was ultimately removed because it was determined that it was not ready for inclusion in the stdlib. 


On Monday, September 17, 2012 at 3:44 AM, Almar Klein wrote:

> Hi,
> 
> I'm eagerly looking forward to the improvements that distutils2/packaging will (hopefully) bring, and I very much appreciate the efforts of those involved.
> 
> However, I'm a bit confused about the status. On http://pypi.python.org/pypi/Distutils2 it says packaging is included in Python 3.3, and on http://docs.python.org/dev/packaging/ there is guidance for how to write a .cfg file. On the other hand, on http://python.org/dev/peps/pep-0398/ (the 3.3 release schedule) it says that the inclusion of packaging is deferred. 
> 
> Which is it? Maybe it's a good idea to make the different docs consistent.
> 
> Thanks,
>   Almar
> 
> -- 
> Almar Klein, PhD
> Science Applied
> phone: +31 6 19268652 (tel:%2B31%206%2019268652)
> e-mail: a.klein at science-applied.nl (mailto:a.klein at science-applied.nl)
> 
> _______________________________________________
> Distutils-SIG maillist - Distutils-SIG at python.org (mailto:Distutils-SIG at python.org)
> http://mail.python.org/mailman/listinfo/distutils-sig
> 
> 


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20120917/0f9d7f2e/attachment-0001.html>

From barry at python.org  Mon Sep 17 16:34:45 2012
From: barry at python.org (Barry Warsaw)
Date: Mon, 17 Sep 2012 10:34:45 -0400
Subject: [Distutils] zip imports ftw
In-Reply-To: <CAG8k2+6Ne7VrN-WZQNio2zM+49BV2ec9ohn5mJy_rB8ov+hXoA@mail.gmail.com>
References: <CAG8k2+48MMoMWeXDFJpSQq63p8ebMew-VM5WOOe6qx_YoiieKA@mail.gmail.com>
	<CAOTb1wcLiYUrV2=tsKUietmmM3zQQtrQuyqGbk1b42hyQN4GHg@mail.gmail.com>
	<1B50CBF2-804D-4BFE-B5CE-3D26940507B3@gmail.com>
	<1347816223.3426.7.camel@thinko>
	<8F8983CF-FD94-4831-8753-44A1EE61BB7E@python.org>
	<CAG8k2+6Ne7VrN-WZQNio2zM+49BV2ec9ohn5mJy_rB8ov+hXoA@mail.gmail.com>
Message-ID: <20120917103445.526466de@resist.wooz.org>

On Sep 16, 2012, at 03:33 PM, Daniel Holth wrote:

>It is because the file has a controversial coding style, with up to 25
>lines of whitespace between classes, and includes functionality for
>eggs and some other stuff that was unwanted on top of the resources
>stuff that partly made its way into pkgutil.

Right, which is why I passively-aggressively suggested an API like it. :)

>I am really impressed with the file after seeing how easy it was to
>add PEP-345 and PEP-376 support to pkg_resources.py without having to
>present a different API to the user, and seeing how it provides a lot
>of what an installer needs especially being able to iterate over all
>the distributions in a particular folder, but I am glad it is not in
>the stdlib because I wouldn't have been able to do that if it was.

Interesting, but of course, this is the same with any module in the stdlib.
That's why it's (somewhat humorously) said the stdlib is where good modules go
to die.  ;)

-Barry
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 836 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20120917/92b750b2/attachment.pgp>

From barry at python.org  Mon Sep 17 16:38:58 2012
From: barry at python.org (Barry Warsaw)
Date: Mon, 17 Sep 2012 10:38:58 -0400
Subject: [Distutils] zip imports ftw
In-Reply-To: <1347824407.3426.14.camel@thinko>
References: <CAG8k2+48MMoMWeXDFJpSQq63p8ebMew-VM5WOOe6qx_YoiieKA@mail.gmail.com>
	<CAOTb1wcLiYUrV2=tsKUietmmM3zQQtrQuyqGbk1b42hyQN4GHg@mail.gmail.com>
	<1B50CBF2-804D-4BFE-B5CE-3D26940507B3@gmail.com>
	<1347816223.3426.7.camel@thinko>
	<8F8983CF-FD94-4831-8753-44A1EE61BB7E@python.org>
	<CAG8k2+6Ne7VrN-WZQNio2zM+49BV2ec9ohn5mJy_rB8ov+hXoA@mail.gmail.com>
	<1347824407.3426.14.camel@thinko>
Message-ID: <20120917103858.070cbee3@resist.wooz.org>

On Sep 16, 2012, at 03:40 PM, Chris McDonough wrote:

>On Sun, 2012-09-16 at 15:33 -0400, Daniel Holth wrote:
>> It is because the file has a controversial coding style, with up to 25
>> lines of whitespace between classes, and includes functionality for
>> eggs and some other stuff that was unwanted on top of the resources
>> stuff that partly made its way into pkgutil.
>
>I think maybe there's also a latent notion floating around that "a
>resource (aka non-Python files) shouldn't be addressable within Python
>packages" aka. "Python should only make it easy to address Python files
>in Python packages, but no other kind of file".  I'm not sure I've ever
>seen any of the core dev team express this opinion, but I've seen it
>floating around elsewhere.

Yeah.  Personally, I think that's just silly.  It's way, way, way too
convenient to put some data files in your packages, and with pkg_resources,
there's actually a principled way to do that consistently regardless of
whether you're running from source or installed.  I'm not even sure *not*
putting your data files in your packages has any kind of common systematic
support in Python.

OTOH, it's definitely true that data files that people have to edit
(e.g. configuration files) shouldn't live in your packages.  That's just an
invitation to violate all kinds of standards (he says, painfully aware of
Mailman 2's mm_cfg.py file ;).

-Barry
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 836 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20120917/38e8e99b/attachment.pgp>

From dholth at gmail.com  Mon Sep 17 16:47:58 2012
From: dholth at gmail.com (Daniel Holth)
Date: Mon, 17 Sep 2012 10:47:58 -0400
Subject: [Distutils] zip imports ftw
In-Reply-To: <20120917103858.070cbee3@resist.wooz.org>
References: <CAG8k2+48MMoMWeXDFJpSQq63p8ebMew-VM5WOOe6qx_YoiieKA@mail.gmail.com>
	<CAOTb1wcLiYUrV2=tsKUietmmM3zQQtrQuyqGbk1b42hyQN4GHg@mail.gmail.com>
	<1B50CBF2-804D-4BFE-B5CE-3D26940507B3@gmail.com>
	<1347816223.3426.7.camel@thinko>
	<8F8983CF-FD94-4831-8753-44A1EE61BB7E@python.org>
	<CAG8k2+6Ne7VrN-WZQNio2zM+49BV2ec9ohn5mJy_rB8ov+hXoA@mail.gmail.com>
	<1347824407.3426.14.camel@thinko>
	<20120917103858.070cbee3@resist.wooz.org>
Message-ID: <CAG8k2+7piB+NvpMZNAPHZuBUQSARssC4YDVEg4jd6TMmbjGjkw@mail.gmail.com>

On Mon, Sep 17, 2012 at 10:38 AM, Barry Warsaw <barry at python.org> wrote:
> On Sep 16, 2012, at 03:40 PM, Chris McDonough wrote:
>
>>On Sun, 2012-09-16 at 15:33 -0400, Daniel Holth wrote:
>>> It is because the file has a controversial coding style, with up to 25
>>> lines of whitespace between classes, and includes functionality for
>>> eggs and some other stuff that was unwanted on top of the resources
>>> stuff that partly made its way into pkgutil.
>>
>>I think maybe there's also a latent notion floating around that "a
>>resource (aka non-Python files) shouldn't be addressable within Python
>>packages" aka. "Python should only make it easy to address Python files
>>in Python packages, but no other kind of file".  I'm not sure I've ever
>>seen any of the core dev team express this opinion, but I've seen it
>>floating around elsewhere.
>
> Yeah.  Personally, I think that's just silly.  It's way, way, way too
> convenient to put some data files in your packages, and with pkg_resources,
> there's actually a principled way to do that consistently regardless of
> whether you're running from source or installed.  I'm not even sure *not*
> putting your data files in your packages has any kind of common systematic
> support in Python.
>
> OTOH, it's definitely true that data files that people have to edit
> (e.g. configuration files) shouldn't live in your packages.  That's just an
> invitation to violate all kinds of standards (he says, painfully aware of
> Mailman 2's mm_cfg.py file ;).
>
> -Barry

I don't think pkg_resources's much better version of pkgutil.get_data
is seriously under dispute.

What's really missing in action is a better api than
zipimport._zip_directory_cache to list everything on the importer
path, not just the modules.

distutils2 does try to add some support for 'data files not on sys.path'

Bento will write, on request only, a Python file with the root paths
used during installation. You get to choose the name. Something like:

yourpackage/__config.py:

SCRIPTS='/path/to/scripts'

I think this is a better idea than .dist-info/RESOURCES being a file
with original-path -> installed-path  for every file in the
distribution.

From erik.m.bray at gmail.com  Mon Sep 17 18:39:49 2012
From: erik.m.bray at gmail.com (Erik Bray)
Date: Mon, 17 Sep 2012 12:39:49 -0400
Subject: [Distutils] zip imports ftw
In-Reply-To: <CAG8k2+6Ne7VrN-WZQNio2zM+49BV2ec9ohn5mJy_rB8ov+hXoA@mail.gmail.com>
References: <CAG8k2+48MMoMWeXDFJpSQq63p8ebMew-VM5WOOe6qx_YoiieKA@mail.gmail.com>
	<CAOTb1wcLiYUrV2=tsKUietmmM3zQQtrQuyqGbk1b42hyQN4GHg@mail.gmail.com>
	<1B50CBF2-804D-4BFE-B5CE-3D26940507B3@gmail.com>
	<1347816223.3426.7.camel@thinko>
	<8F8983CF-FD94-4831-8753-44A1EE61BB7E@python.org>
	<CAG8k2+6Ne7VrN-WZQNio2zM+49BV2ec9ohn5mJy_rB8ov+hXoA@mail.gmail.com>
Message-ID: <CAOTD34ZwrxHgz0BEeizrsQ6JGOG=xjeiABmxBOT1pC9=m7skvg@mail.gmail.com>

On Sun, Sep 16, 2012 at 3:33 PM, Daniel Holth <dholth at gmail.com> wrote:
> It is because the file has a controversial coding style, with up to 25
> lines of whitespace between classes, and includes functionality for
> eggs and some other stuff that was unwanted on top of the resources
> stuff that partly made its way into pkgutil.

I don't see why that couldn't be fixed though.  As it is, almost every
time I make a patch to distribute I clean things up a bit, in part
because I have vim configured by default to run autopep8 and some
other scripts on every Python file I open so I don't have to look at
*quite* as much ugly code (this has proven very useful at work).

> I am really impressed with the file after seeing how easy it was to
> add PEP-345 and PEP-376 support to pkg_resources.py without having to
> present a different API to the user, and seeing how it provides a lot
> of what an installer needs especially being able to iterate over all
> the distributions in a particular folder, but I am glad it is not in
> the stdlib because I wouldn't have been able to do that if it was.

Yes--I frequently have found myself wishing pkg_resources'
availability could be assumed. Especially for getting handles to
non-code resources, and for things like get_distribution().

Erik

From setuptools at bugs.python.org  Wed Sep 19 18:55:35 2012
From: setuptools at bugs.python.org (Chris Nehren)
Date: Wed, 19 Sep 2012 16:55:35 +0000
Subject: [Distutils] [issue145] .egg files not regenerated when .so file
	changes
Message-ID: <1348073735.5.0.782284757559.issue145@psf.upfronthosting.co.za>


New submission from Chris Nehren:

I'm trying to package Clearsilver for OmniOS (illumos derivative). I built it incorrectly the first time round (causing a symbol to not be found in the python neo_cgi.so) and then fixed it. I fixed and reinstalled the package on the target machine but the .egg with the broken neo_cgi.so was still around, and wasn't regenerated when I ran the code again. This caused trac to reference the old, broken .so in the old, broken .egg, causing it to fail despite it having a perfectly serviceable neo_cgi.so right where it knows where to find it. 

I would expect make(1)-like logic for regeneration of .egg files so that this sort of thing does not happen.

Alternately, if I've misunderstood something, I would be very grateful to pointed at tfm so I can read. Thank you!

----------
messages: 694
nosy: cnehren
priority: bug
status: unread
title: .egg files not regenerated when .so file changes

_______________________________________________
Setuptools tracker <setuptools at bugs.python.org>
<http://bugs.python.org/setuptools/issue145>
_______________________________________________

From dholth at gmail.com  Thu Sep 20 03:26:09 2012
From: dholth at gmail.com (Daniel Holth)
Date: Wed, 19 Sep 2012 21:26:09 -0400
Subject: [Distutils] environment markers in setup.cfg -- vs ;
Message-ID: <CAG8k2+6M5O1r2wG-2PK-gAU+AAMntE5sb1qDsmTLpz80qDB4pQ@mail.gmail.com>

I am implementing requirements-in-setup.cfg for wheel because there is
no place to put environment markers in setup.py

The Requires-Dist: lines in PKG-INFO use ; to separate environment
markers from the dep name. Is setup.cfg supposed to use -- ? Why?

requires-dist = keyring; python_version == '2.6'

versus

requires-dist = keyring -- python_version == '2.6'


Do I correctly recall an earlier syntax that used longer section names
as a place to store environment markers?

[foo; marker]


Thanks,

Daniel Holth

From dholth at gmail.com  Thu Sep 20 05:30:37 2012
From: dholth at gmail.com (Daniel Holth)
Date: Wed, 19 Sep 2012 23:30:37 -0400
Subject: [Distutils] weekend project for someone...
Message-ID: <CAG8k2+7d=hs-Y2Yhqg_rEK-P0o7OwN_jOb4riUc+Ejz=QTeaTA@mail.gmail.com>

Got a project for someone...

Take holger krekel's wonderful apipkg and setuptools' versatile
pkg_resources.py, put them together, and define a "nice" API that
exposes only the parts that are used externally. Including both the
"installed/requirements" database and the actual resources "files on
__path__" bits.

Bonus points if you download every sdist from pypi and grep for
"import pkg_resources" to see how it is really used.

Daniel H

https://crate.io/packages/apipkg/

http://nullege.com/codes/search?cq=pkg_resources

From erik.m.bray at gmail.com  Thu Sep 20 19:32:33 2012
From: erik.m.bray at gmail.com (Erik Bray)
Date: Thu, 20 Sep 2012 13:32:33 -0400
Subject: [Distutils] environment markers in setup.cfg -- vs ;
In-Reply-To: <CAG8k2+6M5O1r2wG-2PK-gAU+AAMntE5sb1qDsmTLpz80qDB4pQ@mail.gmail.com>
References: <CAG8k2+6M5O1r2wG-2PK-gAU+AAMntE5sb1qDsmTLpz80qDB4pQ@mail.gmail.com>
Message-ID: <CAOTD34bbHMbM0tSjOwEFjJ52N_gsQEM2R1Yt4ho+ONMvqN63Rw@mail.gmail.com>

On Wed, Sep 19, 2012 at 9:26 PM, Daniel Holth <dholth at gmail.com> wrote:
> I am implementing requirements-in-setup.cfg for wheel because there is
> no place to put environment markers in setup.py
>
> The Requires-Dist: lines in PKG-INFO use ; to separate environment
> markers from the dep name. Is setup.cfg supposed to use -- ? Why?
>
> requires-dist = keyring; python_version == '2.6'
>
> versus
>
> requires-dist = keyring -- python_version == '2.6'

(Sorry, meant to reply-all; hate how that's not a default on
python.org mailing lists.)

Where does it say that?  I see to recall it always using ; both in the
PKG-INFO and in setup.cfg.  I'm probably mistaken, but I'm just
wondering where you got this from?  And if so I wonder if it could be
changed. The -- syntax is just confusing IMO.

Erik

From dholth at gmail.com  Thu Sep 20 19:39:37 2012
From: dholth at gmail.com (Daniel Holth)
Date: Thu, 20 Sep 2012 13:39:37 -0400
Subject: [Distutils] environment markers in setup.cfg -- vs ;
In-Reply-To: <CAOTD34bbHMbM0tSjOwEFjJ52N_gsQEM2R1Yt4ho+ONMvqN63Rw@mail.gmail.com>
References: <CAG8k2+6M5O1r2wG-2PK-gAU+AAMntE5sb1qDsmTLpz80qDB4pQ@mail.gmail.com>
	<CAOTD34bbHMbM0tSjOwEFjJ52N_gsQEM2R1Yt4ho+ONMvqN63Rw@mail.gmail.com>
Message-ID: <CAG8k2+6wcVMSAWBx-qkiM4dhA7n8s_hacYtPrkokRfR2ryDyHQ@mail.gmail.com>

On Thu, Sep 20, 2012 at 1:32 PM, Erik Bray <erik.m.bray at gmail.com> wrote:
> On Wed, Sep 19, 2012 at 9:26 PM, Daniel Holth <dholth at gmail.com> wrote:
>> I am implementing requirements-in-setup.cfg for wheel because there is
>> no place to put environment markers in setup.py
>>
>> The Requires-Dist: lines in PKG-INFO use ; to separate environment
>> markers from the dep name. Is setup.cfg supposed to use -- ? Why?
>>
>> requires-dist = keyring; python_version == '2.6'
>>
>> versus
>>
>> requires-dist = keyring -- python_version == '2.6'
>
> (Sorry, meant to reply-all; hate how that's not a default on
> python.org mailing lists.)
>
> Where does it say that?  I see to recall it always using ; both in the
> PKG-INFO and in setup.cfg.  I'm probably mistaken, but I'm just
> wondering where you got this from?  And if so I wonder if it could be
> changed. The -- syntax is just confusing IMO.
>
> Erik

Reading from http://www.python.org/~gbrandl/build/html2/packaging/setupcfg.html#extension-modules-sections

Of course if you are always installing from wheel you would probably
continue to specify anything build-related in Python, and only the
install-time metadata from setup.cfg's [metadata] section would need
environment markers. Wheel doesn't include setup.cfg or setup.py, just
the .dist-info/METADATA generated from those files.

From pje at telecommunity.com  Thu Sep 20 21:30:54 2012
From: pje at telecommunity.com (PJ Eby)
Date: Thu, 20 Sep 2012 15:30:54 -0400
Subject: [Distutils] .egg-info metadata
In-Reply-To: <CAG8k2+5oj9zZRdjQ42_6SDD6zeLzZ7cXcmaA0Vq2DVEnDoZQ9w@mail.gmail.com>
References: <CAG8k2+5oj9zZRdjQ42_6SDD6zeLzZ7cXcmaA0Vq2DVEnDoZQ9w@mail.gmail.com>
Message-ID: <CALeMXf7K=w7uN05qPZPWk_qP1Et2+bv2DZadjpa07Oovz2ZP2g@mail.gmail.com>

On Sun, Sep 16, 2012 at 9:40 PM, Daniel Holth <dholth at gmail.com> wrote:
>> I've been chatting to Chris McDonough a bit as well, and one
>> potentially useful thing would be to clearly document the
>> setuptools/distribute metadata precisely as it is generated today.

The official docs for all egg formats can be found at:

  http://peak.telecommunity.com/DevCenter/EggFormats

And they answer most of your questions.  A few call-outs below:

> dependency_links.txt: url's of the package's dependencies. Speak up if
> you use this; it is very surprising, and has a much better replacement
> with pip --index options and requirements files.

No, actually, those are not replacements at all, let alone better
ones.  They're for a package's supplier to offer third-party builds to
fulfill requirements not available via PyPI.  E.g. the author of
package Foo offers unofficial Windows builds of package Bar, and
includes dependency links so that those dependencies can be fulfilled
even if not present on PyPI.

> Provides works the same way in setuptools, it is in PKG-INFO.

Setuptools doesn't actually use Provides.

Lexical formats of the files are described in terms of pkg_resources
API calls -- these calls (and more precise syntax documentation) are
documented here:

  http://peak.telecommunity.com/DevCenter/PkgResources

Mainly in the "Parsing Utilities" section:

  http://peak.telecommunity.com/DevCenter/PkgResources#parsing-utilities

From dholth at gmail.com  Thu Sep 20 22:24:49 2012
From: dholth at gmail.com (Daniel Holth)
Date: Thu, 20 Sep 2012 16:24:49 -0400
Subject: [Distutils] .egg-info metadata
In-Reply-To: <CALeMXf7K=w7uN05qPZPWk_qP1Et2+bv2DZadjpa07Oovz2ZP2g@mail.gmail.com>
References: <CAG8k2+5oj9zZRdjQ42_6SDD6zeLzZ7cXcmaA0Vq2DVEnDoZQ9w@mail.gmail.com>
	<CALeMXf7K=w7uN05qPZPWk_qP1Et2+bv2DZadjpa07Oovz2ZP2g@mail.gmail.com>
Message-ID: <CAG8k2+5g2JD9X5b1HCKQD-A+aOZUV474-YZQ2-MCkAtR1JiGEQ@mail.gmail.com>

> And they answer most of your questions.  A few call-outs below:
>
>> dependency_links.txt: url's of the package's dependencies. Speak up if
>> you use this; it is very surprising, and has a much better replacement
>> with pip --index options and requirements files.

I will check to see how often this is used.

> No, actually, those are not replacements at all, let alone better
> ones.  They're for a package's supplier to offer third-party builds to
> fulfill requirements not available via PyPI.  E.g. the author of
> package Foo offers unofficial Windows builds of package Bar, and
> includes dependency links so that those dependencies can be fulfilled
> even if not present on PyPI.

It's probably a style thing, but I like to distribute this information
out-of-band. So as the author of foopackage (an application) I list
all the requirements and their requirements and where they can be
found, including git+http://github urls and so forth. I'm sure this
has all been discussed to death over the last 6+ years.

>> Provides works the same way in setuptools, it is in PKG-INFO.
>
> Setuptools doesn't actually use Provides.

I noticed that. An installer could generate a file
provided-name.egg-info based on this information. Ironically
distribute would probably be the main client of this feature.

> Lexical formats of the files are described in terms of pkg_resources
> API calls -- these calls (and more precise syntax documentation) are
> documented here:
>
>   http://peak.telecommunity.com/DevCenter/PkgResources
>
> Mainly in the "Parsing Utilities" section:
>
>   http://peak.telecommunity.com/DevCenter/PkgResources#parsing-utilities

I am impressed by pkg_resources.

From donald.stufft at gmail.com  Thu Sep 20 22:53:55 2012
From: donald.stufft at gmail.com (Donald Stufft)
Date: Thu, 20 Sep 2012 16:53:55 -0400
Subject: [Distutils] .egg-info metadata
In-Reply-To: <CAG8k2+5g2JD9X5b1HCKQD-A+aOZUV474-YZQ2-MCkAtR1JiGEQ@mail.gmail.com>
References: <CAG8k2+5oj9zZRdjQ42_6SDD6zeLzZ7cXcmaA0Vq2DVEnDoZQ9w@mail.gmail.com>
	<CALeMXf7K=w7uN05qPZPWk_qP1Et2+bv2DZadjpa07Oovz2ZP2g@mail.gmail.com>
	<CAG8k2+5g2JD9X5b1HCKQD-A+aOZUV474-YZQ2-MCkAtR1JiGEQ@mail.gmail.com>
Message-ID: <3F2F5CBEBDBD4E1F961B1AF1971AF15F@gmail.com>

On Thursday, September 20, 2012 at 4:24 PM, Daniel Holth wrote:
> It's probably a style thing, but I like to distribute this information
> out-of-band. So as the author of foopackage (an application) I list
> all the requirements and their requirements and where they can be
> found, including git+http://github urls and so forth. I'm sure this
> has all been discussed to death over the last 6+ years.

For what it's worth I agree with you here. I think that setup.(py|cfg) should 
point towards abstract requirements. (I need a distribution named foo) not
concrete requirements (I need the distribution named foo from example.com).
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20120920/ef9a4086/attachment.html>

From pje at telecommunity.com  Fri Sep 21 03:24:31 2012
From: pje at telecommunity.com (PJ Eby)
Date: Thu, 20 Sep 2012 21:24:31 -0400
Subject: [Distutils] .egg-info metadata
In-Reply-To: <3F2F5CBEBDBD4E1F961B1AF1971AF15F@gmail.com>
References: <CAG8k2+5oj9zZRdjQ42_6SDD6zeLzZ7cXcmaA0Vq2DVEnDoZQ9w@mail.gmail.com>
	<CALeMXf7K=w7uN05qPZPWk_qP1Et2+bv2DZadjpa07Oovz2ZP2g@mail.gmail.com>
	<CAG8k2+5g2JD9X5b1HCKQD-A+aOZUV474-YZQ2-MCkAtR1JiGEQ@mail.gmail.com>
	<3F2F5CBEBDBD4E1F961B1AF1971AF15F@gmail.com>
Message-ID: <CALeMXf55zLx1HRn6xz8rC4nfz9nPgLHeTa9WB=jXsLeF+o1FNQ@mail.gmail.com>

On Thu, Sep 20, 2012 at 4:53 PM, Donald Stufft <donald.stufft at gmail.com> wrote:
> On Thursday, September 20, 2012 at 4:24 PM, Daniel Holth wrote:
>
> It's probably a style thing, but I like to distribute this information
> out-of-band. So as the author of foopackage (an application) I list
> all the requirements and their requirements and where they can be
> found, including git+http://github urls and so forth. I'm sure this
> has all been discussed to death over the last 6+ years.
>
> For what it's worth I agree with you here.

As a practical matter (and practicality beats purity), it's sometimes
necessary to provide this information in order to depend on packages
that aren't cataloged in PyPI, or when third-party build support is
required, and your goal is to offer a library or application which
itself is used as a dependency.

That is, if project A depends on project B, which depends on C, which
is not on PyPI, then A cannot simply depend on B without *copying*
this "out of band" information, propagating a maintenance burden.
Dependency links work around this problem (among others).

So, IMO, this is actually a crucial part of making a composable
dependency system in *practice*, even if it is not required in
*theory*.

From donald.stufft at gmail.com  Fri Sep 21 04:08:05 2012
From: donald.stufft at gmail.com (Donald Stufft)
Date: Thu, 20 Sep 2012 22:08:05 -0400
Subject: [Distutils] .egg-info metadata
In-Reply-To: <CALeMXf55zLx1HRn6xz8rC4nfz9nPgLHeTa9WB=jXsLeF+o1FNQ@mail.gmail.com>
References: <CAG8k2+5oj9zZRdjQ42_6SDD6zeLzZ7cXcmaA0Vq2DVEnDoZQ9w@mail.gmail.com>
	<CALeMXf7K=w7uN05qPZPWk_qP1Et2+bv2DZadjpa07Oovz2ZP2g@mail.gmail.com>
	<CAG8k2+5g2JD9X5b1HCKQD-A+aOZUV474-YZQ2-MCkAtR1JiGEQ@mail.gmail.com>
	<3F2F5CBEBDBD4E1F961B1AF1971AF15F@gmail.com>
	<CALeMXf55zLx1HRn6xz8rC4nfz9nPgLHeTa9WB=jXsLeF+o1FNQ@mail.gmail.com>
Message-ID: <21A3D1C1F4D24590AB762016391DE77D@gmail.com>

On Thursday, September 20, 2012 at 9:24 PM, PJ Eby wrote:
> As a practical matter (and practicality beats purity), it's sometimes
> necessary to provide this information in order to depend on packages
> that aren't cataloged in PyPI, or when third-party build support is
> required, and your goal is to offer a library or application which
> itself is used as a dependency.

This is likely the cause of the difference in opinion. I don't think that 
packaging metadata should contain information other than informative
for dependencies external to the ecosystem.

In this case If something isn't on PyPI then you don't get to have it
automatically installed. If I depend on lxml (or somewhere in my
dependency chain something does) then I have to convey that
requirements for libxml2 and libxslt. I don't see things external to PyPI
as any different. In metadata 1.2+ this could be expressed via
requires-external. It's a dependency external to that of the Python
packaging ecosystem.

Maybe that's just me though.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20120920/24320d32/attachment.html>

From sklein at cpcug.org  Fri Sep 21 04:19:54 2012
From: sklein at cpcug.org (Stanley A. Klein)
Date: Thu, 20 Sep 2012 22:19:54 -0400 (EDT)
Subject: [Distutils] Upstream packaging feedback
In-Reply-To: <mailman.87.1347537619.23756.python-devel@lists.fedoraproject.org>
References: <mailman.87.1347537619.23756.python-devel@lists.fedoraproject.org>
Message-ID: <60935.71.163.73.57.1348193994.squirrel@www.cpcug.org>

This is a timely discussion for me because I need to get back to packaging
some tools for RHEL/Centos that we have as dependencies.  The tools are
available on pypi or from a git repository.  They seem to be available in
older versions for Fedora 14 but not in EPEL or other sites for
RHEL/Centos.

One issue I encountered when previously working on the packaging for
Fedora was that pypi packages do not necessarily follow the Linux
Filesystem Hierarchy Standard.  In fact, the packages may be implicitly
organized for something like Windows where documentation goes in the same
directory as code.  I found I needed to use pypi2rpm bdist_rpm2 and modify
the spec file to get things into the right places.

BTW, the tools right now use python 2.7.

So, please add the FHS issue to your list.

I've also found that everything seems to be in a state of flux right now,
with setuptools, distribute, distutils, distutils2, and
pypi2rpm/bdist_rpm2 all out there and bumping into each other.  When I
tried to do a yum update to my Fedora 14 system, I ran into trouble
because of conflicts among these packages.  I hope this all settles down
soon.


Stan Klein



On Thu, September 13, 2012 8:00 am, Toshio Kuratomi <a.badger at gmail.com>
wrote:

>
> Message: 1
> Date: Wed, 12 Sep 2012 15:14:07 -0700
> From: Toshio Kuratomi <a.badger at gmail.com>
> To: Fedora Python SIG <python-devel at lists.fedoraproject.org>
> Subject: Re: Upstream packaging feedback

>
> On Wed, Sep 12, 2012 at 01:41:22PM +1000, Nick Coghlan wrote:
>> I'll likely be helping to guide updates to the Python packaging format
>> standards over the coming months. While they won't hit the standard
>> library until 3.4, there will likely be third party tool support in
>> earlier versions (since the whole point of the exercise is to eliminate
>> the current implementation coupling to distutils and setuptools in
>> favour of better defined metadata standards for communication between
>> multiple tools).
>>
>> The first step will be reviewing the status quo and then creating a
>> plausible road map (as well as describing current efforts for various
>> aspects). I've started on that here:
>> http://python-notes.boredomandlaziness.org/en/latest/pep_ideas/core_packaging_api.html
>>
>> One thing I would *love* to be able to enable is adding support for
>> automatic mapping of PyPI distribution names (similar to what already
>> exists for Perl and CPAN) where (for example), a developer could just
>> write "Requires: python(south)" instead of having to figure out manually
>> the name of the appropriate RPM package in Fedora.
>>
>> I believe that the new metadata fields defined in PEP 345 and PEP 426
>> should be enough to support that when generating a SPEC file from the
>> PyPI metadata.
>
> Hey Nick!  Thanks for working on this whole thing.  I haven't been looking
> at things in the past year but was active on it a few years ago if you
> want
> to bounce any ideas around, get some idea of what's been discussed i nthe
> past by whom, or talk about what Fedora-specifically is currently
> doing/has
> found troublesome in the past.
>
> Mapping of pypi distribution names was something that I looked into a
> small
> bit with some Canonical people at PyCon two years ago.  IIRC, it was part
> of
> trying to map distribution packages with each other and with pypi in order
> to figure out the state of python3 porting.  Barry Warsaw still works
> there
> and might know more about what happened to it -- Allison Randal is who I
> was
> working with but I got the impression that she isn't working on that
> anymore
> so I don't know if she'll still have code around or not.
>
> There are numerous caveats to trying to do this, none of them
> insurmountable.  I believe we were trying to compare versions as well as
> package names which was even tougher.  Some things I can remember off the
> top of my head:
>
> * Multiple names for a package
>   * pypi usually has the same name as the setup.py name field which is
> also
>     encoded in the egg file name and metadata
>   * The name of the module that is imported
> * One upstream package having multiple downstream names -- for instance,
>   Debian has setuptools and pkg_resources as two separate binary packages.
>   However, they're provided by the setuptools pypi entry.
> * A module that is provided by multiple upstreams:  For instance,
> setuptools
>   is provided by setuptools and distribute.  pexpect is provided by
> pexpect
>   and pexpect-u.
> * Some packages aren't present on pypi.  Many library bindings
>   provided by the libraries are this way, for instance libselinux-python.
> * Some libraries have conflicting names upstream (mock was one example in
>   the past.  ming still is one: http://www.libming.org/ and
>   http://merciless.sourceforge.net/
> * The naming for packages in the distribution isn't always simple.  In
>   Fedora, for instance, we have several styles and not all of them are
>   mutually exclusive:
>   * python-foo is the common case for libraries (python-docutils)
>   * foo-python are the majority (but not all) of libraries which are
>     bindings to C libraries (selinux-python)
>   * Fedora allows modules that have py as a prefix to not add python- to
>     their name. (pygtk2)
>   * Applications may provide libraries that other packages need but since
>     they're "primarily applications" they may simply bear the name of the
>     application. (bzr)
>   * Case sensitivity can get you.  There are some maintainers that prefer
> to
>     lowercase the package names and others who prefer to do exactly what
>     upstream did.
>   * dashes, periods, and other punctuation can also get you.  Sometimes
>     those are translated into dashes, other times they're left alone, and
>     other times they're omitted.
> * Even today, not all packages provide egg-info.  For instance, a useful
>   python module might consist of a single .py file so upstream might not
>   provide a setup.py for it.  We install it by copying the .py file to
>   site-packages.
> * We'll need to differentiate between things provided for python2 and
> python3
>   (and python26 in EPEL)
>
> All that said.... you can probably sidestep some of these issues by having
> python packages contain explicit virtual Provides.  These might be
> manually
> added or automatically generated by a tool like pypi2rpm with the
> maintainer editting them afterwards to make sure they didn't hit any of
> the
> above cornercases.
>
> There's some prior work done by other people:
> * http://www.rpm.org/ticket/154
> * http://lists.fedoraproject.org/pipermail/packaging/2008-June/004715.html
>   (Despite my having written that email, the hard parts were all dmalcolm
> :-)
>
> -Toshio
> -------------- next part --------------
> A non-text attachment was scrubbed...
> Name: not available
> Type: application/pgp-signature
> Size: 198 bytes
> Desc: not available
> URL:
> <http://lists.fedoraproject.org/pipermail/python-devel/attachments/20120912/a4d3ecfe/attachment-0001.sig>



From ncoghlan at gmail.com  Fri Sep 21 04:51:18 2012
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Fri, 21 Sep 2012 12:51:18 +1000
Subject: [Distutils] .egg-info metadata
In-Reply-To: <21A3D1C1F4D24590AB762016391DE77D@gmail.com>
References: <CAG8k2+5oj9zZRdjQ42_6SDD6zeLzZ7cXcmaA0Vq2DVEnDoZQ9w@mail.gmail.com>
	<CALeMXf7K=w7uN05qPZPWk_qP1Et2+bv2DZadjpa07Oovz2ZP2g@mail.gmail.com>
	<CAG8k2+5g2JD9X5b1HCKQD-A+aOZUV474-YZQ2-MCkAtR1JiGEQ@mail.gmail.com>
	<3F2F5CBEBDBD4E1F961B1AF1971AF15F@gmail.com>
	<CALeMXf55zLx1HRn6xz8rC4nfz9nPgLHeTa9WB=jXsLeF+o1FNQ@mail.gmail.com>
	<21A3D1C1F4D24590AB762016391DE77D@gmail.com>
Message-ID: <CADiSq7c69krd0Z-HPxW6ZUZ6dgwG5XiD65j5VnSGGfy+PDNVdg@mail.gmail.com>

On Fri, Sep 21, 2012 at 12:08 PM, Donald Stufft <donald.stufft at gmail.com> wrote:
> Maybe that's just me though.

Nope, it's "In the face of ambiguity, refuse the temptation to guess"
applied to dependency metadata. It's one of the core philosophical
differences between the stdlib and systems like setuptools.

If *others* want to guess in their automated systems, that's their
prerogative. We prefer to put up a barrier and say "be explicit about
what you want". You see the same difference in perspective between
datetime and dateutil - even though we agree there are contexts where
guessing is useful, we will generally consider such tools a poor fit
for the standard library, instead leaving them to the third party
ecosystem.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia

From pje at telecommunity.com  Fri Sep 21 05:58:31 2012
From: pje at telecommunity.com (PJ Eby)
Date: Thu, 20 Sep 2012 23:58:31 -0400
Subject: [Distutils] .egg-info metadata
In-Reply-To: <CADiSq7c69krd0Z-HPxW6ZUZ6dgwG5XiD65j5VnSGGfy+PDNVdg@mail.gmail.com>
References: <CAG8k2+5oj9zZRdjQ42_6SDD6zeLzZ7cXcmaA0Vq2DVEnDoZQ9w@mail.gmail.com>
	<CALeMXf7K=w7uN05qPZPWk_qP1Et2+bv2DZadjpa07Oovz2ZP2g@mail.gmail.com>
	<CAG8k2+5g2JD9X5b1HCKQD-A+aOZUV474-YZQ2-MCkAtR1JiGEQ@mail.gmail.com>
	<3F2F5CBEBDBD4E1F961B1AF1971AF15F@gmail.com>
	<CALeMXf55zLx1HRn6xz8rC4nfz9nPgLHeTa9WB=jXsLeF+o1FNQ@mail.gmail.com>
	<21A3D1C1F4D24590AB762016391DE77D@gmail.com>
	<CADiSq7c69krd0Z-HPxW6ZUZ6dgwG5XiD65j5VnSGGfy+PDNVdg@mail.gmail.com>
Message-ID: <CALeMXf5QZiSFPesBL7hkZ+urmKhTAgWGxBj53Wge5C339Mz=xA@mail.gmail.com>

On Thu, Sep 20, 2012 at 10:51 PM, Nick Coghlan <ncoghlan at gmail.com> wrote:
> On Fri, Sep 21, 2012 at 12:08 PM, Donald Stufft <donald.stufft at gmail.com> wrote:
>> Maybe that's just me though.
>
> Nope, it's "In the face of ambiguity, refuse the temptation to guess"
> applied to dependency metadata.

How are dependency links guessing anything?

From ncoghlan at gmail.com  Fri Sep 21 06:24:55 2012
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Fri, 21 Sep 2012 14:24:55 +1000
Subject: [Distutils] .egg-info metadata
In-Reply-To: <CALeMXf5QZiSFPesBL7hkZ+urmKhTAgWGxBj53Wge5C339Mz=xA@mail.gmail.com>
References: <CAG8k2+5oj9zZRdjQ42_6SDD6zeLzZ7cXcmaA0Vq2DVEnDoZQ9w@mail.gmail.com>
	<CALeMXf7K=w7uN05qPZPWk_qP1Et2+bv2DZadjpa07Oovz2ZP2g@mail.gmail.com>
	<CAG8k2+5g2JD9X5b1HCKQD-A+aOZUV474-YZQ2-MCkAtR1JiGEQ@mail.gmail.com>
	<3F2F5CBEBDBD4E1F961B1AF1971AF15F@gmail.com>
	<CALeMXf55zLx1HRn6xz8rC4nfz9nPgLHeTa9WB=jXsLeF+o1FNQ@mail.gmail.com>
	<21A3D1C1F4D24590AB762016391DE77D@gmail.com>
	<CADiSq7c69krd0Z-HPxW6ZUZ6dgwG5XiD65j5VnSGGfy+PDNVdg@mail.gmail.com>
	<CALeMXf5QZiSFPesBL7hkZ+urmKhTAgWGxBj53Wge5C339Mz=xA@mail.gmail.com>
Message-ID: <CADiSq7fOR+U9zaDpQi1yiTFq78vJW--5r9h0F9q4x3TouDt3fg@mail.gmail.com>

On Fri, Sep 21, 2012 at 1:58 PM, PJ Eby <pje at telecommunity.com> wrote:
> On Thu, Sep 20, 2012 at 10:51 PM, Nick Coghlan <ncoghlan at gmail.com> wrote:
>> On Fri, Sep 21, 2012 at 12:08 PM, Donald Stufft <donald.stufft at gmail.com> wrote:
>>> Maybe that's just me though.
>>
>> Nope, it's "In the face of ambiguity, refuse the temptation to guess"
>> applied to dependency metadata.
>
> How are dependency links guessing anything?

What are they a link to? What OSes will it run on? How do you install
them? As you soon as you step outside the defined dependency
framework, it's all guesswork.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia

From pje at telecommunity.com  Fri Sep 21 06:57:28 2012
From: pje at telecommunity.com (PJ Eby)
Date: Fri, 21 Sep 2012 00:57:28 -0400
Subject: [Distutils] .egg-info metadata
In-Reply-To: <CADiSq7fOR+U9zaDpQi1yiTFq78vJW--5r9h0F9q4x3TouDt3fg@mail.gmail.com>
References: <CAG8k2+5oj9zZRdjQ42_6SDD6zeLzZ7cXcmaA0Vq2DVEnDoZQ9w@mail.gmail.com>
	<CALeMXf7K=w7uN05qPZPWk_qP1Et2+bv2DZadjpa07Oovz2ZP2g@mail.gmail.com>
	<CAG8k2+5g2JD9X5b1HCKQD-A+aOZUV474-YZQ2-MCkAtR1JiGEQ@mail.gmail.com>
	<3F2F5CBEBDBD4E1F961B1AF1971AF15F@gmail.com>
	<CALeMXf55zLx1HRn6xz8rC4nfz9nPgLHeTa9WB=jXsLeF+o1FNQ@mail.gmail.com>
	<21A3D1C1F4D24590AB762016391DE77D@gmail.com>
	<CADiSq7c69krd0Z-HPxW6ZUZ6dgwG5XiD65j5VnSGGfy+PDNVdg@mail.gmail.com>
	<CALeMXf5QZiSFPesBL7hkZ+urmKhTAgWGxBj53Wge5C339Mz=xA@mail.gmail.com>
	<CADiSq7fOR+U9zaDpQi1yiTFq78vJW--5r9h0F9q4x3TouDt3fg@mail.gmail.com>
Message-ID: <CALeMXf4tSNUe1s7eKQt+ZX0fFSyLAha=uRd-XEfO-571iJbzYg@mail.gmail.com>

On Fri, Sep 21, 2012 at 12:24 AM, Nick Coghlan <ncoghlan at gmail.com> wrote:
> On Fri, Sep 21, 2012 at 1:58 PM, PJ Eby <pje at telecommunity.com> wrote:
>> On Thu, Sep 20, 2012 at 10:51 PM, Nick Coghlan <ncoghlan at gmail.com> wrote:
>>> On Fri, Sep 21, 2012 at 12:08 PM, Donald Stufft <donald.stufft at gmail.com> wrote:
>>>> Maybe that's just me though.
>>>
>>> Nope, it's "In the face of ambiguity, refuse the temptation to guess"
>>> applied to dependency metadata.
>>
>> How are dependency links guessing anything?
>
> What are they a link to? What OSes will it run on? How do you install
> them? As you soon as you step outside the defined dependency
> framework, it's all guesswork.

I'm not seeing how a documented file discovery protocol is
"guesswork".  Perhaps you've not read the documentation?  e.g.:

http://peak.telecommunity.com/DevCenter/setuptools#dependencies-that-aren-t-in-pypi
http://peak.telecommunity.com/DevCenter/setuptools#making-your-package-available-for-easyinstall

This is no more guesswork than the PyPI /simple index discovery protocol is.

The only part I can think of in the whole process that is actually
guesswork is interpreting distutils-generated (as opposed to
setuptools-generated) distribution filenames -- a necessary nod to
backward-compatibility.

As far as the practicality vs. purity question, Python has already had
Provides/Requires in the metadata format for several years, and it
contained all the features that were needed for a "pure" dependency
resolution system.  In *theory* you could meet everyone's use cases
with that, if you're going to assume that all packages live in PyPI
and everyone has a compiler.  And then there wouldn't be any need for
any of the new metadata specs.

In practice, however, not everything's in PyPI and not everybody has
compilers, and not every package is instantly updated or maintained
indefinitely.  If you don't like how dependency links solve the
problem, perhaps you could propose some other way of handling those
use cases, such that developer B's dependency maintenance burdens are
not propagated to developer A and his peer/parent dependers?

From donald.stufft at gmail.com  Fri Sep 21 07:28:20 2012
From: donald.stufft at gmail.com (Donald Stufft)
Date: Fri, 21 Sep 2012 01:28:20 -0400
Subject: [Distutils] .egg-info metadata
In-Reply-To: <CALeMXf4tSNUe1s7eKQt+ZX0fFSyLAha=uRd-XEfO-571iJbzYg@mail.gmail.com>
References: <CAG8k2+5oj9zZRdjQ42_6SDD6zeLzZ7cXcmaA0Vq2DVEnDoZQ9w@mail.gmail.com>
	<CALeMXf7K=w7uN05qPZPWk_qP1Et2+bv2DZadjpa07Oovz2ZP2g@mail.gmail.com>
	<CAG8k2+5g2JD9X5b1HCKQD-A+aOZUV474-YZQ2-MCkAtR1JiGEQ@mail.gmail.com>
	<3F2F5CBEBDBD4E1F961B1AF1971AF15F@gmail.com>
	<CALeMXf55zLx1HRn6xz8rC4nfz9nPgLHeTa9WB=jXsLeF+o1FNQ@mail.gmail.com>
	<21A3D1C1F4D24590AB762016391DE77D@gmail.com>
	<CADiSq7c69krd0Z-HPxW6ZUZ6dgwG5XiD65j5VnSGGfy+PDNVdg@mail.gmail.com>
	<CALeMXf5QZiSFPesBL7hkZ+urmKhTAgWGxBj53Wge5C339Mz=xA@mail.gmail.com>
	<CADiSq7fOR+U9zaDpQi1yiTFq78vJW--5r9h0F9q4x3TouDt3fg@mail.gmail.com>
	<CALeMXf4tSNUe1s7eKQt+ZX0fFSyLAha=uRd-XEfO-571iJbzYg@mail.gmail.com>
Message-ID: <DC2B739C51404F778AFDA37E861F3A09@gmail.com>

On Friday, September 21, 2012 at 12:57 AM, PJ Eby wrote:
> As far as the practicality vs. purity question, Python has already had
> Provides/Requires in the metadata format for several years, and it
> contained all the features that were needed for a "pure" dependency
> resolution system. In *theory* you could meet everyone's use cases
> with that, if you're going to assume that all packages live in PyPI
> and everyone has a compiler. And then there wouldn't be any need for
> any of the new metadata specs.

That field only allows importable names which many distributions do not
have (e.g. you can't put in `django-tastypie` into requires).

The exact wording is:
     The format of a requirement string is identical to that of a
      module or package name usable with the 'import' statement,
      optionally followed by a version declaration within parentheses.


These fields were _not_ for saying that it required a particular distribution/project
and _were_ for saying it requires a particular module or package (in the import sense). 
> 
> In practice, however, not everything's in PyPI and not everybody has
> compilers, and not every package is instantly updated or maintained
> indefinitely. If you don't like how dependency links solve the
> problem, perhaps you could propose some other way of handling those
> use cases, such that developer B's dependency maintenance burdens are
> not propagated to developer A and his peer/parent dependers?

I don't see why pushing the maintenance burden down the stack isn't an ok thing
to do. We already do it for External requirements that either aren't python packages
or are packaged in a way that standard tools cannot handle them. Obviously you
have to draw the line somewhere between packages that you can depend on
automatically and ones you can't.

My problems with dependency_links is less to do with guessing and more about
the fact it ties dependency installation to a particular host. The fact that dependencies
are abstract requirements (a distribution named foo at version 2.0) and not a concrete
location (distribution foo version 2.0 found at http://example.com/dists/) means that
you get to choose at install time where your requirements come from. So a user could
easily host their own intranet PyPI style server that they installed their dependencies
from. A dependency that comes from dependency_links can't be installed from your
own internal PyPI (or another public one) without rewriting the setup.py scripts (Unless
my understanding of dependency links is wrong).

Additionally by tying a dependency to an external system you decrease the overall
installablity of a set of requirements* and you increase the number of locations that
someone can use to exploit developers installing Python packages. These are important
considerations when deciding to install or depend on a package and the dependency_links
system "hides" it and makes it non obvious to end users what all is entailed in
installing their set of requirements. (Note I have the same basic problem with external
links on PyPI).

I don't think it's entirely unreasonable for a packaging tool to have the ability to install
from other locations that PyPI or a PyPI like server, I think that a packaging tool
probably should have a dependency_links like system, however I think that the choice
to use them should be in the hands of the person _installing_ the package, not the
person creating the package.

* If You depend on 5 different hosts for installing any particular set of requirements,
  and each of those 5 servers have a 99% uptime then the total combined uptime
  would be (0.99 * 0.99 * 0.99 * 0.99 * 0.99) = ~95%. In an average month you'd 
  go from ~7 hours of uninstallable time per month (for one server at 99% uptime)
  to over a day of uninstallable time per month (for 5 servers at 99% uptime each).
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20120921/b3ee7785/attachment.html>

From ncoghlan at gmail.com  Fri Sep 21 09:06:28 2012
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Fri, 21 Sep 2012 17:06:28 +1000
Subject: [Distutils] .egg-info metadata
In-Reply-To: <CALeMXf4tSNUe1s7eKQt+ZX0fFSyLAha=uRd-XEfO-571iJbzYg@mail.gmail.com>
References: <CAG8k2+5oj9zZRdjQ42_6SDD6zeLzZ7cXcmaA0Vq2DVEnDoZQ9w@mail.gmail.com>
	<CALeMXf7K=w7uN05qPZPWk_qP1Et2+bv2DZadjpa07Oovz2ZP2g@mail.gmail.com>
	<CAG8k2+5g2JD9X5b1HCKQD-A+aOZUV474-YZQ2-MCkAtR1JiGEQ@mail.gmail.com>
	<3F2F5CBEBDBD4E1F961B1AF1971AF15F@gmail.com>
	<CALeMXf55zLx1HRn6xz8rC4nfz9nPgLHeTa9WB=jXsLeF+o1FNQ@mail.gmail.com>
	<21A3D1C1F4D24590AB762016391DE77D@gmail.com>
	<CADiSq7c69krd0Z-HPxW6ZUZ6dgwG5XiD65j5VnSGGfy+PDNVdg@mail.gmail.com>
	<CALeMXf5QZiSFPesBL7hkZ+urmKhTAgWGxBj53Wge5C339Mz=xA@mail.gmail.com>
	<CADiSq7fOR+U9zaDpQi1yiTFq78vJW--5r9h0F9q4x3TouDt3fg@mail.gmail.com>
	<CALeMXf4tSNUe1s7eKQt+ZX0fFSyLAha=uRd-XEfO-571iJbzYg@mail.gmail.com>
Message-ID: <CADiSq7fkpybBqFoBXG-gh-juB=0Us8nBTRNa-0ZQTjjUAxJQAw@mail.gmail.com>

On Fri, Sep 21, 2012 at 2:57 PM, PJ Eby <pje at telecommunity.com> wrote:
> This is no more guesswork than the PyPI /simple index discovery protocol is.

You have zero idea what's at the end of a URL link. You're just hoping
it's the file you expect.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia

From pje at telecommunity.com  Fri Sep 21 17:00:53 2012
From: pje at telecommunity.com (PJ Eby)
Date: Fri, 21 Sep 2012 11:00:53 -0400
Subject: [Distutils] .egg-info metadata
In-Reply-To: <CADiSq7fkpybBqFoBXG-gh-juB=0Us8nBTRNa-0ZQTjjUAxJQAw@mail.gmail.com>
References: <CAG8k2+5oj9zZRdjQ42_6SDD6zeLzZ7cXcmaA0Vq2DVEnDoZQ9w@mail.gmail.com>
	<CALeMXf7K=w7uN05qPZPWk_qP1Et2+bv2DZadjpa07Oovz2ZP2g@mail.gmail.com>
	<CAG8k2+5g2JD9X5b1HCKQD-A+aOZUV474-YZQ2-MCkAtR1JiGEQ@mail.gmail.com>
	<3F2F5CBEBDBD4E1F961B1AF1971AF15F@gmail.com>
	<CALeMXf55zLx1HRn6xz8rC4nfz9nPgLHeTa9WB=jXsLeF+o1FNQ@mail.gmail.com>
	<21A3D1C1F4D24590AB762016391DE77D@gmail.com>
	<CADiSq7c69krd0Z-HPxW6ZUZ6dgwG5XiD65j5VnSGGfy+PDNVdg@mail.gmail.com>
	<CALeMXf5QZiSFPesBL7hkZ+urmKhTAgWGxBj53Wge5C339Mz=xA@mail.gmail.com>
	<CADiSq7fOR+U9zaDpQi1yiTFq78vJW--5r9h0F9q4x3TouDt3fg@mail.gmail.com>
	<CALeMXf4tSNUe1s7eKQt+ZX0fFSyLAha=uRd-XEfO-571iJbzYg@mail.gmail.com>
	<CADiSq7fkpybBqFoBXG-gh-juB=0Us8nBTRNa-0ZQTjjUAxJQAw@mail.gmail.com>
Message-ID: <CALeMXf7z2WT7TygEHako0EahdCepmtPx8UKFv5oCcYFf9ejM2g@mail.gmail.com>

On Fri, Sep 21, 2012 at 3:06 AM, Nick Coghlan <ncoghlan at gmail.com> wrote:
> On Fri, Sep 21, 2012 at 2:57 PM, PJ Eby <pje at telecommunity.com> wrote:
>> This is no more guesswork than the PyPI /simple index discovery protocol is.
>
> You have zero idea what's at the end of a URL link. You're just hoping
> it's the file you expect.

If that's really a problem, you can include an #md5 tag in the link,
and easy_install will refuse to install it if it doesn't match.

However, in a major common use case for dependency links, the person
who put the link in knows *exactly* what's at the end of that link,
because it's a URL they control with contents they produced.

I think, though, that maybe you're confused about how these links
work.  A dependency link has to be one of two types:

1. A "direct link" (a link that's recognizable as a distribution,
either because its final path component is a distribution filename or
because it's tagged with an #egg identifier), or

2. An "indirect link", which will be treated as an HTML page and
scanned for direct links.

This is the *same* protocol used with the PyPI /simple index, or more
precisely, it's a *subset* of that protocol.  (Index pages are scanned
for indirect links tagged as a home page or download URL; this
extension is not supported for dependency links.)

From brett at python.org  Fri Sep 21 17:04:49 2012
From: brett at python.org (Brett Cannon)
Date: Fri, 21 Sep 2012 11:04:49 -0400
Subject: [Distutils] .egg-info metadata
In-Reply-To: <DC2B739C51404F778AFDA37E861F3A09@gmail.com>
References: <CAG8k2+5oj9zZRdjQ42_6SDD6zeLzZ7cXcmaA0Vq2DVEnDoZQ9w@mail.gmail.com>
	<CALeMXf7K=w7uN05qPZPWk_qP1Et2+bv2DZadjpa07Oovz2ZP2g@mail.gmail.com>
	<CAG8k2+5g2JD9X5b1HCKQD-A+aOZUV474-YZQ2-MCkAtR1JiGEQ@mail.gmail.com>
	<3F2F5CBEBDBD4E1F961B1AF1971AF15F@gmail.com>
	<CALeMXf55zLx1HRn6xz8rC4nfz9nPgLHeTa9WB=jXsLeF+o1FNQ@mail.gmail.com>
	<21A3D1C1F4D24590AB762016391DE77D@gmail.com>
	<CADiSq7c69krd0Z-HPxW6ZUZ6dgwG5XiD65j5VnSGGfy+PDNVdg@mail.gmail.com>
	<CALeMXf5QZiSFPesBL7hkZ+urmKhTAgWGxBj53Wge5C339Mz=xA@mail.gmail.com>
	<CADiSq7fOR+U9zaDpQi1yiTFq78vJW--5r9h0F9q4x3TouDt3fg@mail.gmail.com>
	<CALeMXf4tSNUe1s7eKQt+ZX0fFSyLAha=uRd-XEfO-571iJbzYg@mail.gmail.com>
	<DC2B739C51404F778AFDA37E861F3A09@gmail.com>
Message-ID: <CAP1=2W7R6SP8kTtRMft6xXV7kMvkMUWrSHEt1_icVuTos=XJ7w@mail.gmail.com>

On Fri, Sep 21, 2012 at 1:28 AM, Donald Stufft <donald.stufft at gmail.com>wrote:

>  On Friday, September 21, 2012 at 12:57 AM, PJ Eby wrote:
>
> As far as the practicality vs. purity question, Python has already had
> Provides/Requires in the metadata format for several years, and it
> contained all the features that were needed for a "pure" dependency
> resolution system. In *theory* you could meet everyone's use cases
> with that, if you're going to assume that all packages live in PyPI
> and everyone has a compiler. And then there wouldn't be any need for
> any of the new metadata specs.
>
> That field only allows importable names which many distributions do not
> have (e.g. you can't put in `django-tastypie` into requires).
>
> The exact wording is:
>      The format of a requirement string is identical to that of a
>       module or package name usable with the 'import' statement,
>       optionally followed by a version declaration within parentheses.
>
> These fields were _not_ for saying that it required a particular
> distribution/project
> and _were_ for saying it requires a particular module or package (in the
> import sense).
>
>
> In practice, however, not everything's in PyPI and not everybody has
> compilers, and not every package is instantly updated or maintained
> indefinitely. If you don't like how dependency links solve the
> problem, perhaps you could propose some other way of handling those
> use cases, such that developer B's dependency maintenance burdens are
> not propagated to developer A and his peer/parent dependers?
>
> I don't see why pushing the maintenance burden down the stack isn't an ok
> thing
> to do. We already do it for External requirements that either aren't
> python packages
> or are packaged in a way that standard tools cannot handle them. Obviously
> you
> have to draw the line somewhere between packages that you can depend on
> automatically and ones you can't.
>
> My problems with dependency_links is less to do with guessing and more
> about
> the fact it ties dependency installation to a particular host. The fact
> that dependencies
> are abstract requirements (a distribution named foo at version 2.0) and
> not a concrete
> location (distribution foo version 2.0 found at http://example.com/dists/)
> means that
> you get to choose at install time where your requirements come from. So a
> user could
> easily host their own intranet PyPI style server that they installed their
> dependencies
> from. A dependency that comes from dependency_links can't be installed
> from your
> own internal PyPI (or another public one) without rewriting the setup.py
> scripts (Unless
> my understanding of dependency links is wrong).
>

I have tangentially following this discussion, but Donald's point about
keeping dependencies as abstract against a project and version and not
concrete as to where to grab it from seems very reasonable for metadata.
While PyPI might be considered a special-case as it's essentially the
canonical place to find packages and it is under our control (and thus a
place that we might provide a basic tool to pull from), past that I think
it's the purview of pip to decide how it wants to find places to resolve
things. This very much ties into the library vs. tool dividing line.


>
> Additionally by tying a dependency to an external system you decrease the
> overall
> installablity of a set of requirements* and you increase the number of
> locations that
> someone can use to exploit developers installing Python packages. These
> are important
> considerations when deciding to install or depend on a package and the
> dependency_links
> system "hides" it and makes it non obvious to end users what all is
> entailed in
> installing their set of requirements. (Note I have the same basic problem
> with external
> links on PyPI).
>
> I don't think it's entirely unreasonable for a packaging tool to have the
> ability to install
> from other locations that PyPI or a PyPI like server, I think that a
> packaging tool
> probably should have a dependency_links like system, however I think that
> the choice
> to use them should be in the hands of the person _installing_ the package,
> not the
> person creating the package.
>

Agreed. While I might have a suggestion to grab some project from some
place on e.g. bitbucket, that doesn't mean it is the best for someone else
(e.g. the NSA probably wants to pull certain projects form their own
super-secret, vetted project repo rather than from joe-schmoe.com). It
really shouldn't be a packager's job to figure out where to get something,
just that something is needed (e.g. I don't tell you where to go buy your
milk to make waffles, just that you need milk); it is a recipe and thus
should be kept abstract as such.

-Brett


>
> * If You depend on 5 different hosts for installing any particular set of
> requirements,
>   and each of those 5 servers have a 99% uptime then the total combined
> uptime
>   would be (0.99 * 0.99 * 0.99 * 0.99 * 0.99) = ~95%. In an average month
> you'd
>   go from ~7 hours of uninstallable time per month (for one server at 99%
> uptime)
>   to over a day of uninstallable time per month (for 5 servers at 99%
> uptime each).
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20120921/94b60661/attachment.html>

From dholth at gmail.com  Fri Sep 21 17:14:11 2012
From: dholth at gmail.com (Daniel Holth)
Date: Fri, 21 Sep 2012 11:14:11 -0400
Subject: [Distutils] .egg-info metadata
In-Reply-To: <CAP1=2W7R6SP8kTtRMft6xXV7kMvkMUWrSHEt1_icVuTos=XJ7w@mail.gmail.com>
References: <CAG8k2+5oj9zZRdjQ42_6SDD6zeLzZ7cXcmaA0Vq2DVEnDoZQ9w@mail.gmail.com>
	<CALeMXf7K=w7uN05qPZPWk_qP1Et2+bv2DZadjpa07Oovz2ZP2g@mail.gmail.com>
	<CAG8k2+5g2JD9X5b1HCKQD-A+aOZUV474-YZQ2-MCkAtR1JiGEQ@mail.gmail.com>
	<3F2F5CBEBDBD4E1F961B1AF1971AF15F@gmail.com>
	<CALeMXf55zLx1HRn6xz8rC4nfz9nPgLHeTa9WB=jXsLeF+o1FNQ@mail.gmail.com>
	<21A3D1C1F4D24590AB762016391DE77D@gmail.com>
	<CADiSq7c69krd0Z-HPxW6ZUZ6dgwG5XiD65j5VnSGGfy+PDNVdg@mail.gmail.com>
	<CALeMXf5QZiSFPesBL7hkZ+urmKhTAgWGxBj53Wge5C339Mz=xA@mail.gmail.com>
	<CADiSq7fOR+U9zaDpQi1yiTFq78vJW--5r9h0F9q4x3TouDt3fg@mail.gmail.com>
	<CALeMXf4tSNUe1s7eKQt+ZX0fFSyLAha=uRd-XEfO-571iJbzYg@mail.gmail.com>
	<DC2B739C51404F778AFDA37E861F3A09@gmail.com>
	<CAP1=2W7R6SP8kTtRMft6xXV7kMvkMUWrSHEt1_icVuTos=XJ7w@mail.gmail.com>
Message-ID: <CAG8k2+7GKqQ_Ua75qg2d6XkxeVCzzfjs=BXQGpuAuhQ=JQ9vFg@mail.gmail.com>

Let's agree that there is/should be an --ignore flag for
dependency_links.txt and argue about whether it is on by default.

From pje at telecommunity.com  Fri Sep 21 17:14:50 2012
From: pje at telecommunity.com (PJ Eby)
Date: Fri, 21 Sep 2012 11:14:50 -0400
Subject: [Distutils] .egg-info metadata
In-Reply-To: <DC2B739C51404F778AFDA37E861F3A09@gmail.com>
References: <CAG8k2+5oj9zZRdjQ42_6SDD6zeLzZ7cXcmaA0Vq2DVEnDoZQ9w@mail.gmail.com>
	<CALeMXf7K=w7uN05qPZPWk_qP1Et2+bv2DZadjpa07Oovz2ZP2g@mail.gmail.com>
	<CAG8k2+5g2JD9X5b1HCKQD-A+aOZUV474-YZQ2-MCkAtR1JiGEQ@mail.gmail.com>
	<3F2F5CBEBDBD4E1F961B1AF1971AF15F@gmail.com>
	<CALeMXf55zLx1HRn6xz8rC4nfz9nPgLHeTa9WB=jXsLeF+o1FNQ@mail.gmail.com>
	<21A3D1C1F4D24590AB762016391DE77D@gmail.com>
	<CADiSq7c69krd0Z-HPxW6ZUZ6dgwG5XiD65j5VnSGGfy+PDNVdg@mail.gmail.com>
	<CALeMXf5QZiSFPesBL7hkZ+urmKhTAgWGxBj53Wge5C339Mz=xA@mail.gmail.com>
	<CADiSq7fOR+U9zaDpQi1yiTFq78vJW--5r9h0F9q4x3TouDt3fg@mail.gmail.com>
	<CALeMXf4tSNUe1s7eKQt+ZX0fFSyLAha=uRd-XEfO-571iJbzYg@mail.gmail.com>
	<DC2B739C51404F778AFDA37E861F3A09@gmail.com>
Message-ID: <CALeMXf7OA=Y4_ARcPbQ7cg7wGj8ZOkB2i6FN2o1Ti=8YsZhh3w@mail.gmail.com>

On Fri, Sep 21, 2012 at 1:28 AM, Donald Stufft <donald.stufft at gmail.com> wrote:
> These fields were _not_ for saying that it required a particular
> distribution/project
> and _were_ for saying it requires a particular module or package (in the
> import sense).

Yes, but that was still sufficient information to implement a
dependency system, in theory.  Specifically, it would have worked for
the case where all projects are on PyPI and have correct metadata.

If you assume that condition, you can trivially solve all
dependencies; but if you don't assume that condition, you need
something like dependency links.


> I don't see why pushing the maintenance burden down the stack isn't an ok thing to do.

For the same reason that requiring someone to read the source code of
every function called by every function called by every function
called by code they use isn't an ok thing to do.


> A dependency that comes from dependency_links can't be installed from
> your  own internal PyPI (or another public one) without rewriting the setup.py
> scripts (Unless my understanding of dependency links is wrong).

This is indeed wrong.  Dependency links are a last resort, used after
all other provided dependency resolution sources have failed.

I think I see where Nick is coming from now - when he said "you don't
know what's at the end of the URL", my first reaction was "WTF?", only
much more elaborate than "WTF", because I couldn't conceive of how
that's even remotely a meaningful comment.

However, if he, like you, thought that a dependency link means
"download and install this along with the package", then that's a
perfectly understandable further misconception.

That isn't how they work at all: they are simply links that
easy_install is *allowed* to use to resolve dependencies.  It doesn't
just download those URLs and install them.

If you depend on Foo and Bar, your standard requirements list *still
has to include Foo and Bar*!

The dependency links only say, "here are some places to *look* for Foo
and Bar, *if* you can't find them through the normal channels" (e.g.
PyPI, your local index server, already-installed files, links
specified on the command line, etc.).

In other words, they *supplement* the requirements metadata and index
system, they don't *subvert* it.


> I think that the choice to use them should be in the hands of the person
> _installing_ the package, not the person creating the package.

That's what --allow-hosts is for: easy_install can be locked down to
not install except from your local intranet or PyPI or whatever, and
it can be done on a sitewide basis by default (by settings in the
site's distutils.cfg).

From pje at telecommunity.com  Fri Sep 21 18:52:17 2012
From: pje at telecommunity.com (PJ Eby)
Date: Fri, 21 Sep 2012 12:52:17 -0400
Subject: [Distutils] .egg-info metadata
In-Reply-To: <CAP1=2W7R6SP8kTtRMft6xXV7kMvkMUWrSHEt1_icVuTos=XJ7w@mail.gmail.com>
References: <CAG8k2+5oj9zZRdjQ42_6SDD6zeLzZ7cXcmaA0Vq2DVEnDoZQ9w@mail.gmail.com>
	<CALeMXf7K=w7uN05qPZPWk_qP1Et2+bv2DZadjpa07Oovz2ZP2g@mail.gmail.com>
	<CAG8k2+5g2JD9X5b1HCKQD-A+aOZUV474-YZQ2-MCkAtR1JiGEQ@mail.gmail.com>
	<3F2F5CBEBDBD4E1F961B1AF1971AF15F@gmail.com>
	<CALeMXf55zLx1HRn6xz8rC4nfz9nPgLHeTa9WB=jXsLeF+o1FNQ@mail.gmail.com>
	<21A3D1C1F4D24590AB762016391DE77D@gmail.com>
	<CADiSq7c69krd0Z-HPxW6ZUZ6dgwG5XiD65j5VnSGGfy+PDNVdg@mail.gmail.com>
	<CALeMXf5QZiSFPesBL7hkZ+urmKhTAgWGxBj53Wge5C339Mz=xA@mail.gmail.com>
	<CADiSq7fOR+U9zaDpQi1yiTFq78vJW--5r9h0F9q4x3TouDt3fg@mail.gmail.com>
	<CALeMXf4tSNUe1s7eKQt+ZX0fFSyLAha=uRd-XEfO-571iJbzYg@mail.gmail.com>
	<DC2B739C51404F778AFDA37E861F3A09@gmail.com>
	<CAP1=2W7R6SP8kTtRMft6xXV7kMvkMUWrSHEt1_icVuTos=XJ7w@mail.gmail.com>
Message-ID: <CALeMXf7gnGTRn1EmFP+6P6zz5+Xn5xuwmPM+0zSAniCozTnQpg@mail.gmail.com>

On Fri, Sep 21, 2012 at 11:04 AM, Brett Cannon <brett at python.org> wrote:
> Agreed. While I might have a suggestion to grab some project from some place
> on e.g. bitbucket,

This might've been buried in the thread, but that's *precisely* what
dependency links are: a *suggestion* as to where a *normally declared
dependency* can be filled.

Tools have the option to consume these suggestions, or not.  And
easy_install *only* uses these suggestions after all other available
options for finding the dependency have been exhausted.  (E.g. PyPI,
already-installed packages, links or indexes supplied on the command
line, etc.)

So there are two ways to block the use of dependency links:

1. Host restrictions (which can come from the command-line, personal
config, or site config)
2. Alternate indexes or command-line overrides for dependency links
(also configurable in the above-mentioned places) that satisfy all the
needed dependencies (so that easy_install doesn't bother with the
suggested links)

And you can easily combine the two, even if you're not the NSA.  ;-)

(You can also ask easy_install not to install dependencies at all, and
handle it all yourself.)

My point here, btw, isn't to advocate for the exact implementation of
dependency links; I just want to make sure everybody knows what they
*are*, because AFAICT there has been some huge misunderstanding and
FUD going on in this thread, and people making judgments based on
amplifying other people's misinterpretations of other people's
misunderstandings.

From dholth at gmail.com  Fri Sep 21 18:59:19 2012
From: dholth at gmail.com (Daniel Holth)
Date: Fri, 21 Sep 2012 12:59:19 -0400
Subject: [Distutils] wheel as a PEP
Message-ID: <CAG8k2+6neJpvVB9RkNQMdRnsbmcByYkHZX0oHT8bbEzKpx-YLA@mail.gmail.com>

I intend to write up the wheel binary package format version 1.0 as a
Python PEP. The draft is at

https://docs.google.com/document/d/13B9mYo2WVDP42BYQFhWq71b9ZuMyAWhtilfwmasW5Z8/edit

It is quite similar to the spec as it has existed since July. I think
the best explanation is how it is installed:

Installing a wheel ?distribution-1.0.py32.none.any.whl?

    Unpack.
        Parse distribution-1.0.dist-info/WHEEL.
        Check that installer is compatible with Wheel-Version. Warn if
minor version is greater, abort if major version is greater.
        If Root-Is-Purelib == ?true?, unpack archive into purelib
(site-packages).
        Else unpack archive into platlib (site-packages).
    Spread.
        Unpacked archive includes distribution-1.0.dist-info/ and (if
there is data) distribution-1.0.data/
        Move each subtree of distribution-1.0.data/ onto its
destination path. Each subdirectory of distribution-1.0.data/ is a key
into sysconfig.get_paths(), such as
distribution-1.0.data/(config|purelib|platlib|scripts). See
http://docs.python.org/dev/packaging/setupcfg.html#resources
        If applicable, update scripts starting with ?#!python? to
point to the correct interpreter.
        Update distribution-1.0.dist.info/RECORD with the installed paths.
        Remove empty distribution-1.0.data directory.

Thanks,

Daniel Holth

From donald.stufft at gmail.com  Fri Sep 21 18:59:54 2012
From: donald.stufft at gmail.com (Donald Stufft)
Date: Fri, 21 Sep 2012 12:59:54 -0400
Subject: [Distutils] .egg-info metadata
In-Reply-To: <CALeMXf7OA=Y4_ARcPbQ7cg7wGj8ZOkB2i6FN2o1Ti=8YsZhh3w@mail.gmail.com>
References: <CAG8k2+5oj9zZRdjQ42_6SDD6zeLzZ7cXcmaA0Vq2DVEnDoZQ9w@mail.gmail.com>
	<CALeMXf7K=w7uN05qPZPWk_qP1Et2+bv2DZadjpa07Oovz2ZP2g@mail.gmail.com>
	<CAG8k2+5g2JD9X5b1HCKQD-A+aOZUV474-YZQ2-MCkAtR1JiGEQ@mail.gmail.com>
	<3F2F5CBEBDBD4E1F961B1AF1971AF15F@gmail.com>
	<CALeMXf55zLx1HRn6xz8rC4nfz9nPgLHeTa9WB=jXsLeF+o1FNQ@mail.gmail.com>
	<21A3D1C1F4D24590AB762016391DE77D@gmail.com>
	<CADiSq7c69krd0Z-HPxW6ZUZ6dgwG5XiD65j5VnSGGfy+PDNVdg@mail.gmail.com>
	<CALeMXf5QZiSFPesBL7hkZ+urmKhTAgWGxBj53Wge5C339Mz=xA@mail.gmail.com>
	<CADiSq7fOR+U9zaDpQi1yiTFq78vJW--5r9h0F9q4x3TouDt3fg@mail.gmail.com>
	<CALeMXf4tSNUe1s7eKQt+ZX0fFSyLAha=uRd-XEfO-571iJbzYg@mail.gmail.com>
	<DC2B739C51404F778AFDA37E861F3A09@gmail.com>
	<CALeMXf7OA=Y4_ARcPbQ7cg7wGj8ZOkB2i6FN2o1Ti=8YsZhh3w@mail.gmail.com>
Message-ID: <339B2BB2574447B39C8EC1B88913F721@gmail.com>

On Friday, September 21, 2012 at 11:14 AM, PJ Eby wrote:
> Yes, but that was still sufficient information to implement a
> dependency system, in theory. Specifically, it would have worked for
> the case where all projects are on PyPI and have correct metadata.
> 
> If you assume that condition, you can trivially solve all
> dependencies; but if you don't assume that condition, you need
> something like dependency links.
> 
> 

How would you take ``requires: tastypie`` and turn it into `django-tastypie`? Remebering
that PyPI enforces a unique distribution name, not a unique python package (in the import
sense) name. In tastypie's case there are 2 libraries that both provide a top level ``tastypie``
import, one is on PyPI as just ``tastypie`` and the other is ``django-tastypie``.

That also sidesteps the issue that those fields were already in use and if they were
hijacked for this new purpose all that old data would (could?) have created issues. The
new meta-data sidestepped all these issues.
> For the same reason that requiring someone to read the source code of
> every function called by every function called by every function
> called by code they use isn't an ok thing to do.

As I said before, obviously you must draw the line somewhere. setuptools
choose to draw it at a different place than I believe it should have. I don't
believe saying "you must be using some combination of indexes that
include all the dependencies or else you'll need to manually fetch them" is
that large of a burden or requirement to make, obviously you disagree.
> This is indeed wrong. Dependency links are a last resort, used after
> all other provided dependency resolution sources have failed.

This makes them better than I thought, but still have the "installability" and
security. The question this raises in my head, can they be used for any
dependency, or only dependencies that the distribution defining them has?
e.g. if I install Foo, and it depends on Bar, and Wat, and Bar has a
dependency_links, will it only work for Bar and it's dependencies
or will it work for Wat's dependencies as well? 
> That's what --allow-hosts is for: easy_install can be locked down to
> not install except from your local intranet or PyPI or whatever, and
> it can be done on a sitewide basis by default (by settings in the
> site's distutils.cfg).

I'd have been less uneasy about them if they were opt in instead of 
opt out, but that'd require a time machine. I'm mostly trying to stress
why I'm against them to make sure we don't repeat (in my opinion)
the mistakes of the past.

I should probably mention here that I don't particularly blame setuptools
for their invention, I think it's a side effect of one tool owning the entire
packaging chain from top to bottom and not any fault with the implementer(s)
and because easy_install (to my knowledge) does not have anything like
a requirements file from pip so without that the obvious place for these
"extra locations" is a setup.py. I just think it's better to split them out and
keep the packaging metadata abstract, and them have the concrete meta-data
be inferred by taking that abstract data and combining it with a requirements.txt
like file.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20120921/b758fc58/attachment.html>

From pje at telecommunity.com  Fri Sep 21 21:09:21 2012
From: pje at telecommunity.com (PJ Eby)
Date: Fri, 21 Sep 2012 15:09:21 -0400
Subject: [Distutils] .egg-info metadata
In-Reply-To: <339B2BB2574447B39C8EC1B88913F721@gmail.com>
References: <CAG8k2+5oj9zZRdjQ42_6SDD6zeLzZ7cXcmaA0Vq2DVEnDoZQ9w@mail.gmail.com>
	<CALeMXf7K=w7uN05qPZPWk_qP1Et2+bv2DZadjpa07Oovz2ZP2g@mail.gmail.com>
	<CAG8k2+5g2JD9X5b1HCKQD-A+aOZUV474-YZQ2-MCkAtR1JiGEQ@mail.gmail.com>
	<3F2F5CBEBDBD4E1F961B1AF1971AF15F@gmail.com>
	<CALeMXf55zLx1HRn6xz8rC4nfz9nPgLHeTa9WB=jXsLeF+o1FNQ@mail.gmail.com>
	<21A3D1C1F4D24590AB762016391DE77D@gmail.com>
	<CADiSq7c69krd0Z-HPxW6ZUZ6dgwG5XiD65j5VnSGGfy+PDNVdg@mail.gmail.com>
	<CALeMXf5QZiSFPesBL7hkZ+urmKhTAgWGxBj53Wge5C339Mz=xA@mail.gmail.com>
	<CADiSq7fOR+U9zaDpQi1yiTFq78vJW--5r9h0F9q4x3TouDt3fg@mail.gmail.com>
	<CALeMXf4tSNUe1s7eKQt+ZX0fFSyLAha=uRd-XEfO-571iJbzYg@mail.gmail.com>
	<DC2B739C51404F778AFDA37E861F3A09@gmail.com>
	<CALeMXf7OA=Y4_ARcPbQ7cg7wGj8ZOkB2i6FN2o1Ti=8YsZhh3w@mail.gmail.com>
	<339B2BB2574447B39C8EC1B88913F721@gmail.com>
Message-ID: <CALeMXf6YEsa_7+hzv5Ngmc29AU-r0zGXuE7k=+dxUq2dqR0ovA@mail.gmail.com>

On Fri, Sep 21, 2012 at 12:59 PM, Donald Stufft <donald.stufft at gmail.com> wrote:
> How would you take ``requires: tastypie`` and turn it into
> `django-tastypie`?

By matching Requires with Provides at the index level.  That is, by
having PyPI index packages' Provides so that you can search for
packages that match a given Requires.

> As I said before, obviously you must draw the line somewhere. setuptools
> choose to draw it at a different place than I believe it should have. I
> don't
> believe saying "you must be using some combination of indexes that
> include all the dependencies or else you'll need to manually fetch them" is
> that large of a burden or requirement to make, obviously you disagree.

This wasn't added because I personally disagreed, it's the managers of
projects that use large dependency sets while also being relied upon
as part of other projects' large dependency sets.


> This makes them better than I thought, but still have the "installability"
> and
> security.

How is this any different than any other part of PyPI?

> The question this raises in my head, can they be used for any
> dependency, or only dependencies that the distribution defining them has?
> e.g. if I install Foo, and it depends on Bar, and Wat, and Bar has a
> dependency_links, will it only work for Bar and it's dependencies
> or will it work for Wat's dependencies as well?

I would have to dig into the code to be sure, actually.  But I don't
see any problem with restricting them to the dependency subtree being
resolved.  It's possible that the current easy_install implementation
leaks dependencies to sibling installs, but if it does, it's probably
not hard to fix.

> and because easy_install (to my knowledge) does not have anything like
> a requirements file from pip

I just googled requirements.txt, so now I understand why people are so
confused about dependency_links.  Dependency links are *not*
requirements.txt.  A pip requirements.txt is basically equivalent to a
set of easy_install command line options.

So, technically, you can do something  "easy_install `cat
requirements.txt`" and use a requirements.txt with it.  ;-)

Dependency links, OTOH, are only parameters to the easy_install
--find-links option: a *much* more limited functionality.  Not only is
it not the same functionality, the *use case* is quite different as
well.  Requirements.txt is for people installing packages, dependency
links are for people *providing* packages.  Shipping it in-band is the
whole point of the operation.


> so without that the obvious place for these
> "extra locations" is a setup.py.

Again: the entire point of dependency_links is to have an in-band,
automated way to ship third-party indexing information that would
otherwise require manual, out-of-band, *recursive* distribution.
Dependency links are download locations, and are independent of
version specifications.


> I just think it's better to split them out

They *are* split out - that's why they're not in requires.txt or PKG-INFO

> and keep the packaging metadata abstract, and them have the concrete meta-data

I agree that dependency links should not be part of the PKG-INFO
format.  They are indeed a concrete, practical matter for installation
tools.


> be inferred by taking that abstract data and combining it with a requirements.txt like file.

That is basically how dependency links work now, *except* that they
are not as powerful as a requirements.txt file, and they are shipped
with the distribution.

As with entry points, though, there's no need for it to be in
PKG-INFO, since it is for the convenience of installation tools and
end users; it isn't really about the project itself.

From donald.stufft at gmail.com  Fri Sep 21 21:25:46 2012
From: donald.stufft at gmail.com (Donald Stufft)
Date: Fri, 21 Sep 2012 15:25:46 -0400
Subject: [Distutils] .egg-info metadata
In-Reply-To: <CALeMXf6YEsa_7+hzv5Ngmc29AU-r0zGXuE7k=+dxUq2dqR0ovA@mail.gmail.com>
References: <CAG8k2+5oj9zZRdjQ42_6SDD6zeLzZ7cXcmaA0Vq2DVEnDoZQ9w@mail.gmail.com>
	<CALeMXf7K=w7uN05qPZPWk_qP1Et2+bv2DZadjpa07Oovz2ZP2g@mail.gmail.com>
	<CAG8k2+5g2JD9X5b1HCKQD-A+aOZUV474-YZQ2-MCkAtR1JiGEQ@mail.gmail.com>
	<3F2F5CBEBDBD4E1F961B1AF1971AF15F@gmail.com>
	<CALeMXf55zLx1HRn6xz8rC4nfz9nPgLHeTa9WB=jXsLeF+o1FNQ@mail.gmail.com>
	<21A3D1C1F4D24590AB762016391DE77D@gmail.com>
	<CADiSq7c69krd0Z-HPxW6ZUZ6dgwG5XiD65j5VnSGGfy+PDNVdg@mail.gmail.com>
	<CALeMXf5QZiSFPesBL7hkZ+urmKhTAgWGxBj53Wge5C339Mz=xA@mail.gmail.com>
	<CADiSq7fOR+U9zaDpQi1yiTFq78vJW--5r9h0F9q4x3TouDt3fg@mail.gmail.com>
	<CALeMXf4tSNUe1s7eKQt+ZX0fFSyLAha=uRd-XEfO-571iJbzYg@mail.gmail.com>
	<DC2B739C51404F778AFDA37E861F3A09@gmail.com>
	<CALeMXf7OA=Y4_ARcPbQ7cg7wGj8ZOkB2i6FN2o1Ti=8YsZhh3w@mail.gmail.com>
	<339B2BB2574447B39C8EC1B88913F721@gmail.com>
	<CALeMXf6YEsa_7+hzv5Ngmc29AU-r0zGXuE7k=+dxUq2dqR0ovA@mail.gmail.com>
Message-ID: <D144B3052DA6494FBC6F67445BBC8B32@gmail.com>

On Friday, September 21, 2012 at 3:09 PM, PJ Eby wrote:
> On Fri, Sep 21, 2012 at 12:59 PM, Donald Stufft <donald.stufft at gmail.com (mailto:donald.stufft at gmail.com)> wrote:
> > How would you take ``requires: tastypie`` and turn it into
> > `django-tastypie`?
> > 
> 
> 
> By matching Requires with Provides at the index level. That is, by
> having PyPI index packages' Provides so that you can search for
> packages that match a given Requires.
> 
> 

And when you have 2 packages that both provides: tastypie? Which is a real
world occurence. I actually got my projects switched up, it's haystack and
django-haystack which have different names on PyPI (and entirely different functions)
but are both ``import haystack`` and thus would both be Provides: haystack.

https://crate.io/packages/haystack/
https://crate.io/packages/django-haystack/
> > This makes them better than I thought, but still have the "installability"
> > and
> > security.
> > 
> 
> 
> How is this any different than any other part of PyPI?
I hate it there too and I've done a little bit to try and make it more obvious
to people when people rely on dependencies that aren't hosted on PyPI and
I plan to do more in the future. (I realize the historical significance of it, I just
don't think it should bind us to be stuck with it forever). But that's a separate
argument for a different day.
> I would have to dig into the code to be sure, actually. But I don't
> see any problem with restricting them to the dependency subtree being
> resolved. It's possible that the current easy_install implementation
> leaks dependencies to sibling installs, but if it does, it's probably
> not hard to fix.

I only asked this because if it works for siblings I could imagine a case
where someones install set worked, but then they remove a dependency
and suddenly unrelated installs stopped working. Not really pertinent to the
discussion at hand, just a possible bug (or surprising behavior) that popped
into my head. 
> Again: the entire point of dependency_links is to have an in-band,
> automated way to ship third-party indexing information that would
> otherwise require manual, out-of-band, *recursive* distribution.
> Dependency links are download locations, and are independent of
> version specifications.

Yea I get that, I think at this point it's just a differing of opinion about
wether that is a desirable trait for a system to have or not. Obviously
I think not, and you I believe think it is. Setuptools isn't (and shouldn't)
be removing them and as you said they don't belong in PKG-INFO so I
I think we are in agreement about that :)

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20120921/cdee377d/attachment.html>

From pje at telecommunity.com  Fri Sep 21 22:16:20 2012
From: pje at telecommunity.com (PJ Eby)
Date: Fri, 21 Sep 2012 16:16:20 -0400
Subject: [Distutils] .egg-info metadata
In-Reply-To: <D144B3052DA6494FBC6F67445BBC8B32@gmail.com>
References: <CAG8k2+5oj9zZRdjQ42_6SDD6zeLzZ7cXcmaA0Vq2DVEnDoZQ9w@mail.gmail.com>
	<CALeMXf7K=w7uN05qPZPWk_qP1Et2+bv2DZadjpa07Oovz2ZP2g@mail.gmail.com>
	<CAG8k2+5g2JD9X5b1HCKQD-A+aOZUV474-YZQ2-MCkAtR1JiGEQ@mail.gmail.com>
	<3F2F5CBEBDBD4E1F961B1AF1971AF15F@gmail.com>
	<CALeMXf55zLx1HRn6xz8rC4nfz9nPgLHeTa9WB=jXsLeF+o1FNQ@mail.gmail.com>
	<21A3D1C1F4D24590AB762016391DE77D@gmail.com>
	<CADiSq7c69krd0Z-HPxW6ZUZ6dgwG5XiD65j5VnSGGfy+PDNVdg@mail.gmail.com>
	<CALeMXf5QZiSFPesBL7hkZ+urmKhTAgWGxBj53Wge5C339Mz=xA@mail.gmail.com>
	<CADiSq7fOR+U9zaDpQi1yiTFq78vJW--5r9h0F9q4x3TouDt3fg@mail.gmail.com>
	<CALeMXf4tSNUe1s7eKQt+ZX0fFSyLAha=uRd-XEfO-571iJbzYg@mail.gmail.com>
	<DC2B739C51404F778AFDA37E861F3A09@gmail.com>
	<CALeMXf7OA=Y4_ARcPbQ7cg7wGj8ZOkB2i6FN2o1Ti=8YsZhh3w@mail.gmail.com>
	<339B2BB2574447B39C8EC1B88913F721@gmail.com>
	<CALeMXf6YEsa_7+hzv5Ngmc29AU-r0zGXuE7k=+dxUq2dqR0ovA@mail.gmail.com>
	<D144B3052DA6494FBC6F67445BBC8B32@gmail.com>
Message-ID: <CALeMXf5b3Pi-J=aQKE1o=kDDPCT9hW+ZBC1HDMOJPjhOZSXK-g@mail.gmail.com>

On Fri, Sep 21, 2012 at 3:25 PM, Donald Stufft <donald.stufft at gmail.com> wrote:
> And when you have 2 packages that both provides: tastypie? Which is a real
> world occurence.

Hey, I didn't say it would work in practice, I said it would work in
*theory*.  ;-)  The point was, if you only have to have something work
in theory, you can make much simpler design choices.


> I think at this point it's just a differing of opinion about
> wether that is a desirable trait for a system to have or not. Obviously
> I think not, and you I believe think it is.

It's not a matter of opinion whether the trait is desirable: I added
it because people actually desired it, not because I personally have
an opinion about it.  (I'm certainly not a frequent user of the
feature, myself.)  And the use case exists, even if you personally
don't have that use case.

IOW, people have in fact had the use case and desired the feature, so
framing it as a matter of opinion is wrong as a matter of fact.  ;-)

Now, I am not arguing that any particular metadata scheme *must*
support that use case.  I am simply pointing out that the use case
exists and hand-waving it away as a non-existent requirement is
pointless.  If you want to investigate who's using it and decide
whether you are okay with telling those people using it to find
another way to meet the use case, that's fine by me.

However, bear in mind that potentially every download of a package
using dependency links represents a user who would have to manually
handle some dependency downloads, and that the burden propagates to
packages *depending* on that package, not just the direct package.

(I suspect you will also find that dependency links are more likely to
be used in projects that are frequently downloaded; the whole point is
to save a lot of people time hunting down dependencies that would
otherwise be difficult to find.)

Anyway, the point is that arguing whether the use case is desirable
from a purity perspective, or necessary in an ideal world, is
irrelevant to the practical question of whether there should be an
official (or semi-official) way to support the use case in practice.

IOW, in theory, the use case is unnecessary and undesirable.  In
practice, it's not a theoretical question.  ;-)

You can say "We understand the use case, but no, we won't support it."
 I am only trying to point out that "we think this use case doesn't or
shouldn't exist" is not the same thing.

(And at the beginning, I was only trying to answer Daniel's questions
about the specs of existing .egg-info metadata.)

From dholth at gmail.com  Fri Sep 21 22:46:48 2012
From: dholth at gmail.com (Daniel Holth)
Date: Fri, 21 Sep 2012 16:46:48 -0400
Subject: [Distutils] .egg-info metadata
In-Reply-To: <CALeMXf5b3Pi-J=aQKE1o=kDDPCT9hW+ZBC1HDMOJPjhOZSXK-g@mail.gmail.com>
References: <CAG8k2+5oj9zZRdjQ42_6SDD6zeLzZ7cXcmaA0Vq2DVEnDoZQ9w@mail.gmail.com>
	<CALeMXf7K=w7uN05qPZPWk_qP1Et2+bv2DZadjpa07Oovz2ZP2g@mail.gmail.com>
	<CAG8k2+5g2JD9X5b1HCKQD-A+aOZUV474-YZQ2-MCkAtR1JiGEQ@mail.gmail.com>
	<3F2F5CBEBDBD4E1F961B1AF1971AF15F@gmail.com>
	<CALeMXf55zLx1HRn6xz8rC4nfz9nPgLHeTa9WB=jXsLeF+o1FNQ@mail.gmail.com>
	<21A3D1C1F4D24590AB762016391DE77D@gmail.com>
	<CADiSq7c69krd0Z-HPxW6ZUZ6dgwG5XiD65j5VnSGGfy+PDNVdg@mail.gmail.com>
	<CALeMXf5QZiSFPesBL7hkZ+urmKhTAgWGxBj53Wge5C339Mz=xA@mail.gmail.com>
	<CADiSq7fOR+U9zaDpQi1yiTFq78vJW--5r9h0F9q4x3TouDt3fg@mail.gmail.com>
	<CALeMXf4tSNUe1s7eKQt+ZX0fFSyLAha=uRd-XEfO-571iJbzYg@mail.gmail.com>
	<DC2B739C51404F778AFDA37E861F3A09@gmail.com>
	<CALeMXf7OA=Y4_ARcPbQ7cg7wGj8ZOkB2i6FN2o1Ti=8YsZhh3w@mail.gmail.com>
	<339B2BB2574447B39C8EC1B88913F721@gmail.com>
	<CALeMXf6YEsa_7+hzv5Ngmc29AU-r0zGXuE7k=+dxUq2dqR0ovA@mail.gmail.com>
	<D144B3052DA6494FBC6F67445BBC8B32@gmail.com>
	<CALeMXf5b3Pi-J=aQKE1o=kDDPCT9hW+ZBC1HDMOJPjhOZSXK-g@mail.gmail.com>
Message-ID: <CAG8k2+69tMyPh0y9uYMDp7p6HUS7qG95dMvE0S3fN_ATGxORqA@mail.gmail.com>

FWIW bdist_wheel knows about dependency_links.txt. It deletes it if it
is only whitespace, otherwise it keeps it.

As of last May when I downloaded pypi,

16984 of the latest sdists

13028 have dependency_links.txt

The 211 in the postscript have a non-empty dependency_links.txt

These point at peak.telecomunity.com (pardon my dict):

{'HTConsole-0.2.tar.gz':
'http://peak.telecommunity.com/snapshots/\nhttp://prdownloads.sourceforge.net/cheetahtemplate/Cheetah-2.0rc6.tar.gz?download\n',
 'plone.checksum-0.1.tar.gz': 'http://peak.telecommunity.com/snapshots/\n',
 'tagger-0.3.1.tar.gz':
'http://www.openplans.org/projects/yucca/install-tagger\nhttp://svn.rdflib.net/branches/michel-events#egg=rdflib-2.3.1events\nhttp://peak.telecommunity.com/snapshots/\n',
 'tw.starrating-1.0.2.tar.gz': 'http://peak.telecommunity.com/snapshots/\n'}

-- Daniel

p.s.

['DataPipeline-1.0.tar.gz',
 'DjangoSkel-0.1.3.tar.gz',
 'Dozer-0.1.tar.gz',
 'Flask-Security-1.2.2.tar.gz',
 'Flask-Social-1.0.0.tar.gz',
 'HTConsole-0.2.tar.gz',
 'Ibid-0.1.1.tar.gz',
 'MathBench-1.1.0.tar.gz',
 'Mtrax-2.2.07.zip',
 'OpenGLContext-full-2.1.0a9.tar.gz',
 'OpencoreRedirect-0.5.1.tar.gz',
 'Orange-Bioinformatics-2.5a4.tar.gz',
 'Pylons-1.0.1rc1.tar.gz',
 'SatchmoBeanstream-0.2.tar.gz',
 'Schevo-3.0.zip',
 'SchevoGears-1.0a1.zip',
 'SchevoPolicy-1.0a1.zip',
 'SchevoSql-1.0a1.zip',
 'SchevoWeb-1.0a2.zip',
 'Shabti-0.4.4.tar.gz',
 'Sunflower-1.1.0.tar.gz',
 'TaskTracker-0.4.tar.gz',
 'TurboGears-1.5.1.tar.gz',
 'TurboGears2-2.1.5.tar.gz',
 'Twirlip-0.6.tar.gz',
 'Twitabit-1.0a1.zip',
 'ZSI-2.0-rc3.tar.gz',
 'aipy-1.1.1.tar.gz',
 'airi-1.0.c11.tar.gz',
 'android2po-1.2.0.tar.gz',
 'asilib-1.0.2.tar.gz',
 'bda.bfg.tile-1.0.tar.gz',
 'bitsyauth-0.1.2.tar.gz',
 'bitsyblog-2.4.1.tar.gz',
 'boprox_client-0.2.1.tar.gz',
 'boprox_server-0.3.0.tar.gz',
 'buildout.python_libevent-0.9.1.tar.gz',
 'causal-1.0.21.tar.gz',
 'cc.licenserdf-0.2.29.tar.gz',
 'cdat_lite-5.2rc1.tar.gz',
 'chimera-python-0.1.zip',
 'cocograph-0.1.0.tar.gz',
 'cocos2d-0.5.0.zip',
 'collective.recipe.seleniumrc-0.6.1.1.tar.gz',
 'compoze-0.4.1.1.tar.gz',
 'configglue-1.0.3.tar.gz',
 'daversy-0.1.0.dev-r330.tar.gz',
 'dbsh-0.1.6.tar.gz',
 'decoupage-0.10.3.tar.gz',
 'dexy-0.5.7.tar.gz',
 'django-apps-0.1.0.tar.gz',
 'django-chronograph-0.3.1.tar.gz',
 'django-ckeditor-3.6.2.tar.gz',
 'django-dynamic-rules-0.3.2.tar.gz',
 'django-dynamic-validation-0.2.5.tar.gz',
 'django-dzenlog-0.3.1.tar.gz',
 'django-faces-0.2.1.tar.gz',
 'django-fiber-0.9.6.6.tar.gz',
 'django-lfs-0.7.4.tar.gz',
 'django-markdown2-0.1.4.tar.gz',
 'django-modeldict-1.3.1.tar.gz',
 'django-mutant-0.0.2.tar.gz',
 'django-profile-0.0.3.tar.gz',
 'django-pushserver-0.1.8.tar.gz',
 'django-star-0.999.tar.gz',
 'django-stories-0.6.6.tar.gz',
 'django-twitter-tag-0.4.0.tar.gz',
 'django-userprofile-0.0.5.tar.gz',
 'dzenlog-link-0.1.2.tar.gz',
 'ebpub-1.2.tar.gz',
 'elaphe-0.5.3.tar.gz',
 'evogrid-0.1.0.tar.gz',
 'externalator-0.7.0.tar.gz',
 'eyefi-0.2.tar.gz',
 'fassembler-0.6.tar.gz',
 'fetcher-0.9.2.6.zip',
 'finch-cms-0.8.3.tar.gz',
 'fiveruns.dash.django-0.2.1.tar.gz',
 'flunc-0.8.tar.gz',
 'getpaid.report-0.1.1.tar.gz',
 'getpaid.wizard-0.4.zip',
 'gocept.zope3instance-2.0a2.tar.gz',
 'googl-0.1dev.tar.gz',
 'graphite-0.71.tar.gz',
 'graphtool-0.5.2.tar.gz',
 'gtwpy-1.3.9.tar.gz',
 'hapicake-0.1.1.tar.gz',
 'hiringpond-0.1dev.tar.gz',
 'huImages-2.2.tar.gz',
 'huSoftM-0.65p2.tar.gz',
 'hyde-0.8.4.tar.gz',
 'ice.adverlet-0.2.3.tar.gz',
 'ice.template-0.1.0.tar.gz',
 'isk-daemon-0.9.3.zip',
 'isotoma.zope.testpythonscript-1.0.0.tar.gz',
 'iw.subscriber-0.1.1.tar.gz',
 'jmbo-0.2.5.tar.gz',
 'jmbo-social-0.0.6.tar.gz',
 'jquery.jsonform-0.1.0c1.tar.gz',
 'jquery.widget-0.1.0c1.tar.gz',
 'kegbot-0.8.5.tar.gz',
 'korovic-1.0.5.tar.gz',
 'kpwrapper-1.0.4.tar.gz',
 'ks.smartimage-1.2.2.tar.gz',
 'kytten-6.0.0.tar.gz',
 'labjacku12-0.4.tar.gz',
 'lfs-contact-1.0.tar.gz',
 'lfs-order-numbers-1.0b1.tar.gz',
 'll-orasql-1.27.1.zip',
 'll-toxic-0.10.zip',
 'll-xist-3.25.zip',
 'localwiki-0.3.1.tar.gz',
 'logfollow-server-0.0.9.tar.gz',
 'lojbansuggest-0.2.2.tar.gz',
 'massmedia-0.6.4.tar.gz',
 'matter-0.8-r960-20110308.tar.gz',
 'mdemos.server-0.1.3dev.tar.gz',
 'mobi.devices-1.2.tar.gz',
 'moxie-8.12.1.tar.gz',
 'nagare-0.4.1.tar.gz',
 'nagare.examples-0.3.0.tar.gz',
 'ndg_xacml-0.5.0.tar.gz',
 'neo4django-0.1.5.tar.gz',
 'numericube-twistranet-1.1.4.zip',
 'nydus-0.9.0.tar.gz',
 'opencore-0.9a4.tar.gz',
 'opencore-listen-0.8.0.tar.gz',
 'ore.viewlet-0.2.1.tar.gz',
 'panya-0.1.6.tar.gz',
 'panya-calendar-0.0.2.tar.gz',
 'panya-contact-0.0.6.tar.gz',
 'panya-social-0.0.4.tar.gz',
 'panya-tube-0.0.3.tar.gz',
 'pbp.recipe.trac-0.4.3.tar.gz',
 'pdfserver-0.6.tar.gz',
 'peer-0.10.0.tar.gz',
 'pg_boss-1.0dev2.tar.gz',
 'pg_foundation-1.1.1.tar.gz',
 'pg_greentrunk-1.0.1.tar.gz',
 'pg_pqueue-1.0.2.tar.gz',
 'pg_proboscis-1.0.5.tar.gz',
 'pg_tin-1.0dev.tar.gz',
 'pg_typical-1.2.tar.gz',
 'philo-0.9.2.tar.gz',
 'plone.checksum-0.1.tar.gz',
 'plone.recipe.deliverance-1.0rc1.tar.gz',
 'plone.recipe.dzhandle-1.0-beta4.tar.gz',
 'plone.recipe.plone-3.1.7.tar.gz',
 'plone.recipe.plone21install-1.0.zip',
 'plone.recipe.plone25install-0.2.tar.gz',
 'plone.recipe.precompiler-0.4.tar.gz',
 'plone.recipe.unifiedinstaller-4.1.zip',
 'plotlib-0.3.tar.gz',
 'pmxbot-1101.7.3.zip',
 'puke-1.5.5.tar.gz',
 'py-soocial-0.2.1.tar.gz',
 'pyCaCORE-0.3.0.zip',
 'pyemir-0.6.4.tar.gz',
 'pyunpack-0.0.2.tar.gz',
 'rapidsms-xforms-0.4.1.tar.gz',
 'rdfadict-0.7.4.tar.gz',
 'repopencore-0.3.tar.gz',
 'rhizome-0.2.4.tar.gz',
 'ritremixerator-0.3.tar.gz',
 'seishub.plugins.exupery-1.2.1.zip',
 'seishub.plugins.seismology-1.2.2.zip',
 'sentry-4.4.6.tar.gz',
 'seriesoftubes-0.9.2.tar.gz',
 'stsci.distutils-0.3.tar.gz',
 'superpy-1.2.6.zip',
 'swab-0.1.2.tar.gz',
 'swg-1.2.8.1.tar.gz',
 'tagger-0.3.1.tar.gz',
 'tastytweets-0.2.2.tar.gz',
 'tempodb-0.0.9.tar.gz',
 'tesla-0.2.6.tar.gz',
 'tg.devtools-2.1.5.tar.gz',
 'tg.ext.repoze.who-0.1-r26.tar.gz',
 'tgapp-photos-0.2.2.tar.gz',
 'tgapp-registration-0.1.tar.gz',
 'tgapp-smallpress-0.2.1.tar.gz',
 'tgext.menu-1.0rc3.tar.gz',
 'tgidproviders-0.2.tar.gz',
 'topp.build.lib-0.5.1.tar.gz',
 'topp.build.opencore-0.5.1.tar.gz',
 'transifex-1.2.1.tar.gz',
 'tw.starrating-1.0.2.tar.gz',
 'tw.tinymce-0.8.tar.gz',
 'viewpoint-0.9.6.tar.gz',
 'whatsup-0.1.1.tar.gz',
 'z3c.breadcrumb-1.1.1.tar.gz',
 'z3c.csvvocabulary-1.1.0.tar.gz',
 'z3c.currency-0.1.0.tar.gz',
 'z3c.formjsdemo-0.3.1.tar.gz',
 'z3c.language.session-1.0.2.zip',
 'z3c.macroviewlet-1.1.0.tar.gz',
 'z3c.recipe.runscript-0.1.3.tar.gz',
 'z3c.referercredentials-0.1.0.tar.gz',
 'zc.creditcard-1.0.tar.gz',
 'zc.resumelb-0.5.2.tar.gz',
 'zc.shortcut-1.0.tar.gz',
 'zc.ssl-1.2.tar.gz',
 'zcontact-0.1.0a11.tar.gz',
 'zerobin-0.1.tar.gz',
 'zicbee-0.9-rc9.zip',
 'zicbee-gst-0.6.1.tar.gz',
 'zicbee-lib-0.7.3.zip',
 'zicbee-mplayer-0.9.1.tar.gz',
 'zicbee-vlc-0.9.2.tar.gz',
 'zif.jsonserver-0.6.tar.gz',
 'zif.xtemplate-0.2.2.tar.gz']

From erik.m.bray at gmail.com  Fri Sep 21 23:07:55 2012
From: erik.m.bray at gmail.com (Erik Bray)
Date: Fri, 21 Sep 2012 17:07:55 -0400
Subject: [Distutils] .egg-info metadata
In-Reply-To: <CAG8k2+5g2JD9X5b1HCKQD-A+aOZUV474-YZQ2-MCkAtR1JiGEQ@mail.gmail.com>
References: <CAG8k2+5oj9zZRdjQ42_6SDD6zeLzZ7cXcmaA0Vq2DVEnDoZQ9w@mail.gmail.com>
	<CALeMXf7K=w7uN05qPZPWk_qP1Et2+bv2DZadjpa07Oovz2ZP2g@mail.gmail.com>
	<CAG8k2+5g2JD9X5b1HCKQD-A+aOZUV474-YZQ2-MCkAtR1JiGEQ@mail.gmail.com>
Message-ID: <CAOTD34b79_sRV6sN=bAUkW6XWn_6Sb7-DWxYBB-HuXTHwwZiiQ@mail.gmail.com>

On Thu, Sep 20, 2012 at 4:24 PM, Daniel Holth <dholth at gmail.com> wrote:
>> And they answer most of your questions.  A few call-outs below:
>>
>>> dependency_links.txt: url's of the package's dependencies. Speak up if
>>> you use this; it is very surprising, and has a much better replacement
>>> with pip --index options and requirements files.
>
> I will check to see how often this is used.

I definitely use this. This way all my packages can get their
dependencies from my own package index instead of PyPI. This gives a
greater deal of control in dealing with users who are angry/frustrated
when they can't reach PyPI for some reason. Though I think this will
be less of an issue with PyPI mirror support.  Though it's also nice
to be able to release development packages that have dev versions of
other packages as dependencies without having to put all my dev
releases on PyPI.

From erik.m.bray at gmail.com  Fri Sep 21 23:18:35 2012
From: erik.m.bray at gmail.com (Erik Bray)
Date: Fri, 21 Sep 2012 17:18:35 -0400
Subject: [Distutils] .egg-info metadata
In-Reply-To: <CALeMXf4tSNUe1s7eKQt+ZX0fFSyLAha=uRd-XEfO-571iJbzYg@mail.gmail.com>
References: <CAG8k2+5oj9zZRdjQ42_6SDD6zeLzZ7cXcmaA0Vq2DVEnDoZQ9w@mail.gmail.com>
	<CALeMXf7K=w7uN05qPZPWk_qP1Et2+bv2DZadjpa07Oovz2ZP2g@mail.gmail.com>
	<CAG8k2+5g2JD9X5b1HCKQD-A+aOZUV474-YZQ2-MCkAtR1JiGEQ@mail.gmail.com>
	<3F2F5CBEBDBD4E1F961B1AF1971AF15F@gmail.com>
	<CALeMXf55zLx1HRn6xz8rC4nfz9nPgLHeTa9WB=jXsLeF+o1FNQ@mail.gmail.com>
	<21A3D1C1F4D24590AB762016391DE77D@gmail.com>
	<CADiSq7c69krd0Z-HPxW6ZUZ6dgwG5XiD65j5VnSGGfy+PDNVdg@mail.gmail.com>
	<CALeMXf5QZiSFPesBL7hkZ+urmKhTAgWGxBj53Wge5C339Mz=xA@mail.gmail.com>
	<CADiSq7fOR+U9zaDpQi1yiTFq78vJW--5r9h0F9q4x3TouDt3fg@mail.gmail.com>
	<CALeMXf4tSNUe1s7eKQt+ZX0fFSyLAha=uRd-XEfO-571iJbzYg@mail.gmail.com>
Message-ID: <CAOTD34aFjkOf0_35nGVvvXnauMxniaQPijDiKZsdejBdJ5nUzg@mail.gmail.com>

On Fri, Sep 21, 2012 at 12:57 AM, PJ Eby <pje at telecommunity.com> wrote:
> I'm not seeing how a documented file discovery protocol is
> "guesswork".  Perhaps you've not read the documentation?  e.g.:
>
> http://peak.telecommunity.com/DevCenter/setuptools#dependencies-that-aren-t-in-pypi
> http://peak.telecommunity.com/DevCenter/setuptools#making-your-package-available-for-easyinstall

Exactly.  This is the opposite of guesswork, in fact.  I have use
cases where I want to know/specify exactly where my dependencies are
coming from, and not just put out an abstract name.  I agree this
information shouldn't be a part of the package's Metadata.  I think
the existing solution in setuptools of including it in a
dependency_links.txt file is fine.  This wouldn't be an option though
with Nick's "one JSON file to rule them all" proposal (which I'm not
against--it just precludes keeping dependency-links-like info out of
band).

> As far as the practicality vs. purity question, Python has already had
> Provides/Requires in the metadata format for several years, and it
> contained all the features that were needed for a "pure" dependency
> resolution system.  In *theory* you could meet everyone's use cases
> with that, if you're going to assume that all packages live in PyPI
> and everyone has a compiler.  And then there wouldn't be any need for
> any of the new metadata specs.
>
> In practice, however, not everything's in PyPI and not everybody has
> compilers, and not every package is instantly updated or maintained
> indefinitely.  If you don't like how dependency links solve the
> problem, perhaps you could propose some other way of handling those
> use cases, such that developer B's dependency maintenance burdens are
> not propagated to developer A and his peer/parent dependers?

And not everybody wants to put everything on PyPI. In fact, I once had
a use case to build a system on top of pkg_resources that could locate
dependencies in SVN repository checkouts by adding their paths to
dependency_links/find_links.  There are all kinds of uses for this
kind of thing that I don't think should be ignored, and pkg_resources
has already provided the means to implement most of them (the
aforementioned example also required subclassing easy_install to work
properly, but that wasn't too big a deal).

I think a broader point is that package distribution and dependency
resolution has some ugly and messy corners and there's no way around
that.  Just saying, "Well all the dependencies should be clearly
documented and it's up to the user to figure how to assemble them all"
isn't going to fly either.

Erik

From erik.m.bray at gmail.com  Fri Sep 21 23:25:54 2012
From: erik.m.bray at gmail.com (Erik Bray)
Date: Fri, 21 Sep 2012 17:25:54 -0400
Subject: [Distutils] .egg-info metadata
In-Reply-To: <CADiSq7fkpybBqFoBXG-gh-juB=0Us8nBTRNa-0ZQTjjUAxJQAw@mail.gmail.com>
References: <CAG8k2+5oj9zZRdjQ42_6SDD6zeLzZ7cXcmaA0Vq2DVEnDoZQ9w@mail.gmail.com>
	<CALeMXf7K=w7uN05qPZPWk_qP1Et2+bv2DZadjpa07Oovz2ZP2g@mail.gmail.com>
	<CAG8k2+5g2JD9X5b1HCKQD-A+aOZUV474-YZQ2-MCkAtR1JiGEQ@mail.gmail.com>
	<3F2F5CBEBDBD4E1F961B1AF1971AF15F@gmail.com>
	<CALeMXf55zLx1HRn6xz8rC4nfz9nPgLHeTa9WB=jXsLeF+o1FNQ@mail.gmail.com>
	<21A3D1C1F4D24590AB762016391DE77D@gmail.com>
	<CADiSq7c69krd0Z-HPxW6ZUZ6dgwG5XiD65j5VnSGGfy+PDNVdg@mail.gmail.com>
	<CALeMXf5QZiSFPesBL7hkZ+urmKhTAgWGxBj53Wge5C339Mz=xA@mail.gmail.com>
	<CADiSq7fOR+U9zaDpQi1yiTFq78vJW--5r9h0F9q4x3TouDt3fg@mail.gmail.com>
	<CALeMXf4tSNUe1s7eKQt+ZX0fFSyLAha=uRd-XEfO-571iJbzYg@mail.gmail.com>
	<CADiSq7fkpybBqFoBXG-gh-juB=0Us8nBTRNa-0ZQTjjUAxJQAw@mail.gmail.com>
Message-ID: <CAOTD34bmVHfmz7CntzBi9eJkh4XTsn5+TS+bJo8BG_6mBvU9CQ@mail.gmail.com>

On Fri, Sep 21, 2012 at 3:06 AM, Nick Coghlan <ncoghlan at gmail.com> wrote:
> On Fri, Sep 21, 2012 at 2:57 PM, PJ Eby <pje at telecommunity.com> wrote:
>> This is no more guesswork than the PyPI /simple index discovery protocol is.
>
> You have zero idea what's at the end of a URL link. You're just hoping
> it's the file you expect.

That's not really something the average science user is going to care
about.  All they care is that when they run `setup.py install` like I
told them to it installs the package and gets all the right versions
of the dependencies, some of which may not be on PyPI, or have been
repackaged for some reason because the official package that is on
PyPI was created by someone who doesn't know what they're doing and
isn't inclined to fix it.

Erik

From dholth at gmail.com  Sun Sep 23 05:31:21 2012
From: dholth at gmail.com (Daniel Holth)
Date: Sat, 22 Sep 2012 23:31:21 -0400
Subject: [Distutils] Wheel PEP
Message-ID: <CAG8k2+4rvHSw9xnLrySEr_Fr3WsnrUdtFcETfp70azRRtZh+YQ@mail.gmail.com>

Initial wheel PEP text up in approximate PEP format at
https://bitbucket.org/dholth/python-peps/changeset/5ffb06c1de6

Is it OK to push this to python.org/peps?

Thanks,

Daniel Holth

From ncoghlan at gmail.com  Sun Sep 23 08:26:28 2012
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sun, 23 Sep 2012 16:26:28 +1000
Subject: [Distutils] Wheel PEP
In-Reply-To: <CAG8k2+4rvHSw9xnLrySEr_Fr3WsnrUdtFcETfp70azRRtZh+YQ@mail.gmail.com>
References: <CAG8k2+4rvHSw9xnLrySEr_Fr3WsnrUdtFcETfp70azRRtZh+YQ@mail.gmail.com>
Message-ID: <CADiSq7cONVuT6gdp75LzW_D30RAtGfXZ90y8ascWfeujc=Co1g@mail.gmail.com>

On Sun, Sep 23, 2012 at 1:31 PM, Daniel Holth <dholth at gmail.com> wrote:
> Initial wheel PEP text up in approximate PEP format at
> https://bitbucket.org/dholth/python-peps/changeset/5ffb06c1de6
>
> Is it OK to push this to python.org/peps?

Yes, much better to have that as an initial draft rather than the
current placeholder.

I do have a comment prompted by the existing FAQ entry attempting to
explain the ".data" suffix. I'd favour something more like
".dist-scheme" or ".dist-paths", to provide a better link to the
relevant part of the sysconfig API
(http://docs.python.org/dev/library/sysconfig.html#installation-paths).
The PEP should also link to that section of the sysconfig docs when
first discussing the "root-is-purelib" setting.

It's also not quite true that the contents of a wheel aren't
importable - since they're still just a zipfile, they can still be
added to an import path in the same manner as egg files. It's just not
*recommended* to use them that way, as the format is designed
primarily for use in distribution rather than runtime imports.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia

From dholth at gmail.com  Sun Sep 23 14:57:33 2012
From: dholth at gmail.com (Daniel Holth)
Date: Sun, 23 Sep 2012 08:57:33 -0400
Subject: [Distutils] Wheel PEP
In-Reply-To: <CADiSq7cONVuT6gdp75LzW_D30RAtGfXZ90y8ascWfeujc=Co1g@mail.gmail.com>
References: <CAG8k2+4rvHSw9xnLrySEr_Fr3WsnrUdtFcETfp70azRRtZh+YQ@mail.gmail.com>
	<CADiSq7cONVuT6gdp75LzW_D30RAtGfXZ90y8ascWfeujc=Co1g@mail.gmail.com>
Message-ID: <CAG8k2+7Z2i9+VWpOw0wtSi7bDBMAHz4c79fOR3qd8D6QHP=3tg@mail.gmail.com>

> It's also not quite true that the contents of a wheel aren't
> importable - since they're still just a zipfile, they can still be
> added to an import path in the same manner as egg files. It's just not
> *recommended* to use them that way, as the format is designed
> primarily for use in distribution rather than runtime imports.

That is intentional, rather than the bdist_wininst scheme that puts
nothing but subdirectories at the root. I wouldn't be surprised if
someone writes a tool to append the appropriate .pyc to wheels.

PURELIB/
SCRIPTS/
HEADERS/

And fun fact, you can add a subpath of a zip file to PYTHONPATH. You
could probably import one of the wininst .exe files if you added
instaler.exe/PURELIB to your path. But in zip all those files in
PURELIB/ also represent an extra 8 uncompressed bytes len('PURELIB/')
for every entry in the most common path.

I may explicitly mention in the spec that wheel is allowed to compress
with xz or bz2. Those particular wheels would not be importable and
would only work on Python 3.3+.

From vinay_sajip at yahoo.co.uk  Sun Sep 23 17:57:39 2012
From: vinay_sajip at yahoo.co.uk (Vinay Sajip)
Date: Sun, 23 Sep 2012 15:57:39 +0000 (UTC)
Subject: [Distutils] distlib updated with resources API
Message-ID: <loom.20120923T174554-748@post.gmane.org>

I've updated distlib[1] with a resources API - functionality for accessing data
files co-located with code in Python packages. This is missing from the stdlib
and currently people use pkg_resources to achieve this.

The design and implementation allows for accessing resources from packages
imported from the file system or from .zip files, and is intended to be
extensible to other PEP 302-compliant import systems.

The update includes:

A short tutorial showing how to use the API. [2]
A discussion of the design of the API. [3]
Reference documentation for the API. [4]

I'd be interested in any and all feedback on this - be it on design, code, docs.
Please regard it as very much a first draft - it would appear to cover pretty
much the same functionality as the corresponding part of pkg_resources [5].

Regards,

Vinay Sajip

[1] https://bitbucket.org/vinay.sajip/distlib/
[2] http://distlib.readthedocs.org/en/latest/tutorial.html#using-the-resource-api
[3] http://distlib.readthedocs.org/en/latest/internals.html#the-resources-api
[4]
http://distlib.readthedocs.org/en/latest/reference.html#the-distlib-resources-package
[5] http://packages.python.org/distribute/pkg_resources.html#basic-resource-access


From conrado at buhrer.net  Wed Sep 26 16:05:13 2012
From: conrado at buhrer.net (Conrado Buhrer)
Date: Wed, 26 Sep 2012 11:05:13 -0300
Subject: [Distutils] buildout.org A record
Message-ID: <CANoAZMCh4uvnvOXKo2ZEmUkNjTCfpupfsp3_A3vhky9=sQCLPQ@mail.gmail.com>

Hi there,

I've contacted gocept.com who defered me to zoneedit.com, who I also
sent a mail to. Since there is also an issue with the site and I don't
know who administers it I've come to spam you all with this
information.

Currently the A record for the naked buildout.org domain is missing.

http://buildout.org/bootstrap.py (tested through www.buildout.org) is
also missing. This causes buildout's bootstrap.py to fail checking for
a new version.

Several pages from the buildout.org site are also missing, possibly
due to a bad server migration?

Best,
Conrado.

From jim at zope.com  Wed Sep 26 16:56:11 2012
From: jim at zope.com (Jim Fulton)
Date: Wed, 26 Sep 2012 10:56:11 -0400
Subject: [Distutils] buildout.org A record
In-Reply-To: <CANoAZMCh4uvnvOXKo2ZEmUkNjTCfpupfsp3_A3vhky9=sQCLPQ@mail.gmail.com>
References: <CANoAZMCh4uvnvOXKo2ZEmUkNjTCfpupfsp3_A3vhky9=sQCLPQ@mail.gmail.com>
Message-ID: <CAPDm-Fhd4WRNugMuZY7mDuYqdymHPg3uxTgh20t+vWgem2SGmw@mail.gmail.com>

On Wed, Sep 26, 2012 at 10:05 AM, Conrado Buhrer <conrado at buhrer.net> wrote:
> http://buildout.org/bootstrap.py (tested through www.buildout.org) is
> also missing. This causes buildout's bootstrap.py to fail checking for
> a new version.

What?  I'm not aware of (nor can I find) anything in buildout that
depends on buildout.org.

Jim

-- 
Jim Fulton
http://www.linkedin.com/in/jimfulton
Jerky is better than bacon! http://zo.pe/Kqm

From conrado at buhrer.net  Wed Sep 26 17:03:35 2012
From: conrado at buhrer.net (Conrado Buhrer)
Date: Wed, 26 Sep 2012 12:03:35 -0300
Subject: [Distutils] buildout.org A record
In-Reply-To: <CAPDm-Fhd4WRNugMuZY7mDuYqdymHPg3uxTgh20t+vWgem2SGmw@mail.gmail.com>
References: <CANoAZMCh4uvnvOXKo2ZEmUkNjTCfpupfsp3_A3vhky9=sQCLPQ@mail.gmail.com>
	<CAPDm-Fhd4WRNugMuZY7mDuYqdymHPg3uxTgh20t+vWgem2SGmw@mail.gmail.com>
Message-ID: <CANoAZMB0W0b0AFtVzDO85a5eqy6aQGWNrUKaUk7ma6et7S--JQ@mail.gmail.com>

Hi Jim,

A few of us on #buildout at freenode have noticed the issue, we believe
that some links on pypi are the cause.

See lines 84-89 of the source of http://pypi.python.org/simple/zc.buildout/

Best,
Conrado

> What?  I'm not aware of (nor can I find) anything in buildout that
> depends on buildout.org.
>
> Jim
>
> --
> Jim Fulton
> http://www.linkedin.com/in/jimfulton
> Jerky is better than bacon! http://zo.pe/Kqm

From jim at zope.com  Wed Sep 26 17:35:00 2012
From: jim at zope.com (Jim Fulton)
Date: Wed, 26 Sep 2012 11:35:00 -0400
Subject: [Distutils] buildout.org A record
In-Reply-To: <CANoAZMB0W0b0AFtVzDO85a5eqy6aQGWNrUKaUk7ma6et7S--JQ@mail.gmail.com>
References: <CANoAZMCh4uvnvOXKo2ZEmUkNjTCfpupfsp3_A3vhky9=sQCLPQ@mail.gmail.com>
	<CAPDm-Fhd4WRNugMuZY7mDuYqdymHPg3uxTgh20t+vWgem2SGmw@mail.gmail.com>
	<CANoAZMB0W0b0AFtVzDO85a5eqy6aQGWNrUKaUk7ma6et7S--JQ@mail.gmail.com>
Message-ID: <CAPDm-FgdfTPKbkOC-EB=+KWyX9KvPprSYdQdTYzjOSAnKfGtag@mail.gmail.com>

On Wed, Sep 26, 2012 at 11:03 AM, Conrado Buhrer <conrado at buhrer.net> wrote:
> Hi Jim,
>
> A few of us on #buildout at freenode have noticed the issue, we believe
> that some links on pypi are the cause.
>
> See lines 84-89 of the source of http://pypi.python.org/simple/zc.buildout/

Fixed. I removed the home-page meta data from those releases.
I don't know what's going on with buildout.org. :(

Jim

-- 
Jim Fulton
http://www.linkedin.com/in/jimfulton
Jerky is better than bacon! http://zo.pe/Kqm

From donald.stufft at gmail.com  Wed Sep 26 21:09:19 2012
From: donald.stufft at gmail.com (Donald Stufft)
Date: Wed, 26 Sep 2012 15:09:19 -0400
Subject: [Distutils] Differences in PEP386 and setuptools
Message-ID: <AE15D652194A4C70BADC05166BAEF0DF@gmail.com>

I've been comparing how PEP386/distutils2.version treats versions
and how pkg_resources from setuptools treats versions and it
confirmed a worry of me with the way that dev is treated in PEP386.

In PEP386 you have several kinds of "top level" releases, these 
are alpha, beta, candidate, and final and they are sorted as you
would expect. On top of that it has pre and post releases for
any of those top level releases. The pre releases are tagged
with a ".dev" and post releases are tagged with a ".post". They
are sorted immediately before/after the "main" release that they
are pre/post for.

In pkg_resources dev is treated as it's own "main" release and
sorts it in front of an alpha.

This means that given the versions:

    ["0.1.dev1", "0.1a1", "0.1b1", "0.1c1", "0.1"]

PEP386 will sort them as:
    
    ["0.1a1", "0.1b1", "0.1c1", "0.1.dev1", "0.1"]

and pkg_resources will sort them as:

    ["0.1.dev1", "0.1a1", "0.1b1", "0.1c1", "0.1"]


To further complicate things the most common usage I've personally seen
in the wild is that of "0.1dev" or "0.1dev1" which the author expects to sort
before an alpha (In this case distutils2.version throws an error, but the suggest
function is able to turn it into 0.1.dev1).

I think this difference is going to cause confusion, especially during the the
transition period when you're going to have people using both pkg_resources
and the new PEP386 functions.

Since PEP386 is only in the "Accepted" stage, and isn't part of the official
implementation yet, is it at all possible to revise it? Ideally I think to follow
with the prior art, and people's expectations "dev" should be moved to a "main"
release type sorted before an alpha, and to take it's place as a pre release modifier
perhaps something like "pre" can be used instead (e.g. "0.1.pre1").

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20120926/698356df/attachment.html>

From a.badger at gmail.com  Wed Sep 26 22:00:28 2012
From: a.badger at gmail.com (Toshio Kuratomi)
Date: Wed, 26 Sep 2012 13:00:28 -0700
Subject: [Distutils] Differences in PEP386 and setuptools
In-Reply-To: <AE15D652194A4C70BADC05166BAEF0DF@gmail.com>
References: <AE15D652194A4C70BADC05166BAEF0DF@gmail.com>
Message-ID: <20120926200028.GH2974@unaka.lan>

On Wed, Sep 26, 2012 at 03:09:19PM -0400, Donald Stufft wrote:
> I've been comparing how PEP386/distutils2.version treats versions
> and how pkg_resources from setuptools treats versions and it
> confirmed a worry of me with the way that dev is treated in PEP386.
> 
> In PEP386 you have several kinds of "top level" releases, these 
> are alpha, beta, candidate, and final and they are sorted as you
> would expect. On top of that it has pre and post releases for
> any of those top level releases. The pre releases are tagged
> with a ".dev" and post releases are tagged with a ".post". They
> are sorted immediately before/after the "main" release that they
> are pre/post for.
> 
> In pkg_resources dev is treated as it's own "main" release and
> sorts it in front of an alpha.
> 
> This means that given the versions:
> 
>     ["0.1.dev1", "0.1a1", "0.1b1", "0.1c1", "0.1"]
> 
> PEP386 will sort them as:
>     
>     ["0.1a1", "0.1b1", "0.1c1", "0.1.dev1", "0.1"]
> 
> and pkg_resources will sort them as:
> 
>     ["0.1.dev1", "0.1a1", "0.1b1", "0.1c1", "0.1"]
> 
> 
> To further complicate things the most common usage I've personally seen
> in the wild is that of "0.1dev" or "0.1dev1" which the author expects to sort
> before an alpha (In this case distutils2.version throws an error, but the
> suggest
> function is able to turn it into 0.1.dev1).
> 
> I think this difference is going to cause confusion, especially during the the
> transition period when you're going to have people using both pkg_resources
> and the new PEP386 functions.
> 
> Since PEP386 is only in the "Accepted" stage, and isn't part of the official
> implementation yet, is it at all possible to revise it? Ideally I think to
> follow
> with the prior art, and people's expectations "dev" should be moved to a "main"
> release type sorted before an alpha, and to take it's place as a pre release
> modifier
> perhaps something like "pre" can be used instead (e.g. "0.1.pre1").

Note that this was an intentional difference with setuptools.

-Toshio
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 198 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20120926/7b33b71d/attachment-0001.pgp>

From donald.stufft at gmail.com  Wed Sep 26 22:24:42 2012
From: donald.stufft at gmail.com (Donald Stufft)
Date: Wed, 26 Sep 2012 16:24:42 -0400
Subject: [Distutils] Differences in PEP386 and setuptools
In-Reply-To: <20120926200028.GH2974@unaka.lan>
References: <AE15D652194A4C70BADC05166BAEF0DF@gmail.com>
	<20120926200028.GH2974@unaka.lan>
Message-ID: <215BFDFF990F49E5A920022A9C8E1F47@gmail.com>

On Wednesday, September 26, 2012 at 4:00 PM, Toshio Kuratomi wrote:
> Note that this was an intentional difference with setuptools. 
Do you have any idea _why_? It seems from my perspective (of someone 
who wasn't really paying attention back then) to be a fairly arbitrary difference
in both prior art and what I've personally seen as the common usage that has
a good chance of causing confusion (as it already has).
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20120926/35e0d249/attachment.html>

From tarek at ziade.org  Wed Sep 26 22:40:37 2012
From: tarek at ziade.org (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Wed, 26 Sep 2012 22:40:37 +0200
Subject: [Distutils] distlib updated with resources API
In-Reply-To: <loom.20120923T174554-748@post.gmane.org>
References: <loom.20120923T174554-748@post.gmane.org>
Message-ID: <50636845.8080204@ziade.org>

On 9/23/12 5:57 PM, Vinay Sajip wrote:
> I've updated distlib[1] with a resources API - functionality for accessing data
> files co-located with code in Python packages. This is missing from the stdlib
> and currently people use pkg_resources to achieve this.
>
> The design and implementation allows for accessing resources from packages
> imported from the file system or from .zip files, and is intended to be
> extensible to other PEP 302-compliant import systems.
>
> The update includes:
>
> A short tutorial showing how to use the API. [2]
> A discussion of the design of the API. [3]
> Reference documentation for the API. [4]
>
> I'd be interested in any and all feedback on this - be it on design, code, docs.
> Please regard it as very much a first draft - it would appear to cover pretty
> much the same functionality as the corresponding part of pkg_resources [5].
Nice work !

On a side note, since these are the original modules that were taking 
out of Python's packaging implementation,
I don't think you can copyright them under your name like what I have 
seen in setup.py

Unless your plan is not to have distlib incorporated into Python but 
roll a separate project, I'd recommend having it in hg.python.org
under the PSF umbrella.

If it's a separate project I think the original licencing must remain

Cheers
Tarek

From vinay_sajip at yahoo.co.uk  Wed Sep 26 23:58:31 2012
From: vinay_sajip at yahoo.co.uk (Vinay Sajip)
Date: Wed, 26 Sep 2012 21:58:31 +0000 (UTC)
Subject: [Distutils] distlib updated with resources API
References: <loom.20120923T174554-748@post.gmane.org>
	<50636845.8080204@ziade.org>
Message-ID: <loom.20120926T233609-992@post.gmane.org>

Tarek Ziad? <tarek <at> ziade.org> writes:

> On a side note, since these are the original modules that were taking 
> out of Python's packaging implementation,
> I don't think you can copyright them under your name like what I have 
> seen in setup.py

AFAIK I've only added my copyright to the individual files I've created, but not
to any of the files I've copied over from packaging. The plan is to move the
project over to hg.python.org at some point, but Antoine suggested (on
python-dev) leaving it on BitBucket until it gets a little more mature. I'm fine
with that too; there's still a lot to do on it.

> Unless your plan is not to have distlib incorporated into Python but 
> roll a separate project, I'd recommend having it in hg.python.org
> under the PSF umbrella.

I think the suggestion was to keep it out of the stdlib for now, and add all or
part of it later, as and when there's a general consensus on python-dev about
its fitness for purpose. I can certainly update the files which I copied from
packaging with a PSF copyright (or other copyright, like the Fellowship of the
Packaging), if that would make you more comfortable, and / or state "Portions
Copyright XXX" in setup.py. I certainly don't intend for copyright issues to be
contentious :-)

If distlib doesn't make it into the stdlib, it'll be dead in the water anyway,
as the purpose of it is to find a common underpinning in the stdlib that higher
level tools can use, to achieve interoperability and consistency across
different packaging tools. I will be adding more features as time permits - e.g.
I have started adding scripts functionality including binary launchers for
Windows, and I will soon be adding a "plugins" package to handle the "entry
point" functionality. At the moment the distlib stuff is really in limbo
awaiting feedback from distutils-sig and python-dev members; I'll be adding bits
and pieces, but without some kind of endorsement/suggestions/contributions from
others, no real progress can be made.

Regards,

Vinay Sajip


From aclark at aclark.net  Thu Sep 27 04:38:45 2012
From: aclark at aclark.net (Alex Clark)
Date: Wed, 26 Sep 2012 22:38:45 -0400
Subject: [Distutils] buildout.org A record
References: <CANoAZMCh4uvnvOXKo2ZEmUkNjTCfpupfsp3_A3vhky9=sQCLPQ@mail.gmail.com>
	<CAPDm-Fhd4WRNugMuZY7mDuYqdymHPg3uxTgh20t+vWgem2SGmw@mail.gmail.com>
	<CANoAZMB0W0b0AFtVzDO85a5eqy6aQGWNrUKaUk7ma6et7S--JQ@mail.gmail.com>
	<CAPDm-FgdfTPKbkOC-EB=+KWyX9KvPprSYdQdTYzjOSAnKfGtag@mail.gmail.com>
Message-ID: <k40e7j$fhj$1@ger.gmane.org>

On 2012-09-26 15:35:00 +0000, Jim Fulton said:

> On Wed, Sep 26, 2012 at 11:03 AM, Conrado Buhrer <conrado at buhrer.net> wrote:
>> Hi Jim,
>> 
>> A few of us on #buildout at freenode have noticed the issue, we believe
>> that some links on pypi are the cause.
>> 
>> See lines 84-89 of the source of http://pypi.python.org/simple/zc.buildout/
> 
> Fixed. I removed the home-page meta data from those releases.
> I don't know what's going on with buildout.org. :(


If anyone does know, please change the DNS A record to 207.97.227.245 
so we can host the site on github pages.


> 
> Jim


-- 
Alex Clark ? http://pythonpackages.com



From dholth at gmail.com  Thu Sep 27 05:34:52 2012
From: dholth at gmail.com (Daniel Holth)
Date: Wed, 26 Sep 2012 23:34:52 -0400
Subject: [Distutils] distlib updated with resources API
In-Reply-To: <loom.20120926T233609-992@post.gmane.org>
References: <loom.20120923T174554-748@post.gmane.org>
	<50636845.8080204@ziade.org>
	<loom.20120926T233609-992@post.gmane.org>
Message-ID: <CAG8k2+6kJevxaXmivECB8+Pm1pPOE1ANCqyH1FgA_CXt09+JeQ@mail.gmail.com>

> If distlib doesn't make it into the stdlib, it'll be dead in the water anyway,
> as the purpose of it is to find a common underpinning in the stdlib that higher
> level tools can use, to achieve interoperability and consistency across
> different packaging tools. I will be adding more features as time permits - e.g.
> I have started adding scripts functionality including binary launchers for
> Windows, and I will soon be adding a "plugins" package to handle the "entry
> point" functionality. At the moment the distlib stuff is really in limbo
> awaiting feedback from distutils-sig and python-dev members; I'll be adding bits
> and pieces, but without some kind of endorsement/suggestions/contributions from
> others, no real progress can be made.

I feel it is necessary to implement pkg_resources.py on top of any new
API. For example just monkey-patch pkg_resources with the parts you've
implemented, or implement the new API by calling pkg_resources. It
would be a very healthy exercise to see whether you've forgotten
something, and avoids having to port 2/3rds of the software on pypi to
use the new API.

Does pkgutil's resource API (get_data) fit in in any way? I notice it
includes the "list zip contents" implementation but it doesn't expose
it in any way. It would be a chore to have to implement 3 importer
adapters to make all 3 APIs work.

From vinay_sajip at yahoo.co.uk  Thu Sep 27 08:41:09 2012
From: vinay_sajip at yahoo.co.uk (Vinay Sajip)
Date: Thu, 27 Sep 2012 06:41:09 +0000 (UTC)
Subject: [Distutils] distlib updated with resources API
References: <loom.20120923T174554-748@post.gmane.org>
	<50636845.8080204@ziade.org>
	<loom.20120926T233609-992@post.gmane.org>
	<CAG8k2+6kJevxaXmivECB8+Pm1pPOE1ANCqyH1FgA_CXt09+JeQ@mail.gmail.com>
Message-ID: <loom.20120927T072155-429@post.gmane.org>

Daniel Holth <dholth <at> gmail.com> writes:

> I feel it is necessary to implement pkg_resources.py on top of any new
> API. For example just monkey-patch pkg_resources with the parts you've
> implemented, or implement the new API by calling pkg_resources. It
> would be a very healthy exercise to see whether you've forgotten
> something, and avoids having to port 2/3rds of the software on pypi to
> use the new API.

That can be looked at in due course. However, weaning projects off distutils/
setuptools APIs and onto anything else will not be pain-free. ISTM one can't
promise to fulfil the contract of the pkg_resources resource functions, without
essentially duplicating large chunks of pkg_resources; for example, the
resource functions take a Requirement object as well as a package name, so any
usage of the APIs with Requirements would not work with a simple wrapper.

There are specific functions in pkg_resources which I have specifically left
out: for example, there's no "get a real filename for this resource, extracting
it from zip to a cache folder if you need to". This type of functionality
doesn't seem to be a core requirement and could be provided with a specialised
finder if really needed, but I see no case for it in the base classes which
might go in the stdlib.

The resources API is IMO small and simple enough to evaluate on its own (in
terms of whether it meets the stated requirements, and whether the stated
requirements are complete).

PEP 365, proposing inclusion of pkg_resources in the stdlib, was rejected. It
doesn't seem right to have something which might potentially be in the stdlib
monkey-patching external projects. There might be other approaches to ease
porting to new APIs, e.g. 2to3 style fixers.

> Does pkgutil's resource API (get_data) fit in in any way? I notice it
> includes the "list zip contents" implementation but it doesn't expose
> it in any way. It would be a chore to have to implement 3 importer
> adapters to make all 3 APIs work.

I'm not sure I quite understand what you mean, as I couldn't see any reference
to listing zip contents in the pkgutil docs, but pkgutil's get_data wraps the
PEP 302 loader's get_data API, as does distlib.resources. However,
pkgutil.get_data doesn't allow you to customise its behaviour, whereas
distlib.resources does (by allowing you to implement and register your own
finder).

One of the reasons why Python packaging is in its current state is that
distutils is hard to extend. Any replacement APIs we introduce should not
repeat that mistake. While pkg_resources is not quite so bad in that
department, since it allows for providers for different loaders, its provider
hierarchy seems to be such that eggs are an integral part of the mix, rather
than an option you can do without:

IResourceProvider
NullProvider
  EmptyProvider
  EggProvider
    DefaultProvider
    ZipProvider

In distlib.resources, there are only three equivalents:

Resource -> IResourceProvider
ResourceFinder -> DefaultProvider
ZipResourceFinder -> ZipProvider

Part of the feedback I'm looking for is whether these provide a suitable basis
for extending by packaging tool providers where needed.

Regards,

Vinay Sajip


From vinay_sajip at yahoo.co.uk  Thu Sep 27 12:43:31 2012
From: vinay_sajip at yahoo.co.uk (Vinay Sajip)
Date: Thu, 27 Sep 2012 10:43:31 +0000 (UTC)
Subject: [Distutils] distlib updated with scripts API
Message-ID: <loom.20120927T122121-586@post.gmane.org>

I've updated distlib[1] with a scripts API - functionality for installing
scripts.

The design and implementation covers shebang adjustment and optional native
Windows executable installation (like setuptools).

The update includes:

A short tutorial showing how to use the API. [2]
A discussion of the design of the API. [3]
Reference documentation for the API. [4]

I'd be interested in any and all feedback on this.

Regards,

Vinay Sajip

[1] https://bitbucket.org/vinay.sajip/distlib/
[2] http://distlib.readthedocs.org/en/latest/tutorial.html#using-the-scripts-api
[3] http://distlib.readthedocs.org/en/latest/internals.html#the-scripts-api
[4]
http://distlib.readthedocs.org/en/latest/reference.html#the-distlib-scripts-package



From dholth at gmail.com  Thu Sep 27 14:15:28 2012
From: dholth at gmail.com (Daniel Holth)
Date: Thu, 27 Sep 2012 08:15:28 -0400
Subject: [Distutils] distlib updated with scripts API
In-Reply-To: <loom.20120927T122121-586@post.gmane.org>
References: <loom.20120927T122121-586@post.gmane.org>
Message-ID: <CAG8k2+4OB+BXhes8Q8vdcTEP6mGqba9HpG9UPrJ2d8f8cQpr5A@mail.gmail.com>

Not bad. The wheel installer could probably use the scripts wrapper.

On Thu, Sep 27, 2012 at 6:43 AM, Vinay Sajip <vinay_sajip at yahoo.co.uk> wrote:
> I've updated distlib[1] with a scripts API - functionality for installing
> scripts.
>
> The design and implementation covers shebang adjustment and optional native
> Windows executable installation (like setuptools).
>
> The update includes:
>
> A short tutorial showing how to use the API. [2]
> A discussion of the design of the API. [3]
> Reference documentation for the API. [4]
>
> I'd be interested in any and all feedback on this.
>
> Regards,
>
> Vinay Sajip
>
> [1] https://bitbucket.org/vinay.sajip/distlib/
> [2] http://distlib.readthedocs.org/en/latest/tutorial.html#using-the-scripts-api
> [3] http://distlib.readthedocs.org/en/latest/internals.html#the-scripts-api
> [4]
> http://distlib.readthedocs.org/en/latest/reference.html#the-distlib-scripts-package
>
>
> _______________________________________________
> Distutils-SIG maillist  -  Distutils-SIG at python.org
> http://mail.python.org/mailman/listinfo/distutils-sig

From dholth at gmail.com  Thu Sep 27 14:39:01 2012
From: dholth at gmail.com (Daniel Holth)
Date: Thu, 27 Sep 2012 08:39:01 -0400
Subject: [Distutils] distlib updated with resources API
In-Reply-To: <loom.20120927T072155-429@post.gmane.org>
References: <loom.20120923T174554-748@post.gmane.org>
	<50636845.8080204@ziade.org>
	<loom.20120926T233609-992@post.gmane.org>
	<CAG8k2+6kJevxaXmivECB8+Pm1pPOE1ANCqyH1FgA_CXt09+JeQ@mail.gmail.com>
	<loom.20120927T072155-429@post.gmane.org>
Message-ID: <CAG8k2+5ynKdXmP1HPbmK4769ZnT_VJxj_dRNdZZQPFJsX4pMvg@mail.gmail.com>

What you can do with pkgutil is call
pkgutil.iter_importer_modules.register(importer, function) so that
your new importer works with iter_modules. Unfortunately even though
it implements most of what you would need to listdir() a zip file,
using zipimport._zip_directory_cache[], it does not expose the
functionality.

https://bitbucket.org/dholth/cpython/src/b534ce119c3c/Lib/pkgutil.py#cl-357

For this particular resources feature you could probably rename Egg to
Dist as far as the pkg_resources code goes, there is barely anything
in the EggProvider class.

Make sure you have a test where the package is not in the root of the .zip file

foo.zip/site-packages/bar.py

Speaking of old code, does anyone feel like replacing

# @decorator
def fn(): ...
fn = decorator(fn)

with the probably-ok-to-use-by-now decorator syntax?

Daniel

From vinay_sajip at yahoo.co.uk  Thu Sep 27 14:56:34 2012
From: vinay_sajip at yahoo.co.uk (Vinay Sajip)
Date: Thu, 27 Sep 2012 12:56:34 +0000 (UTC)
Subject: [Distutils] distlib updated with scripts API
References: <loom.20120927T122121-586@post.gmane.org>
	<CAG8k2+4OB+BXhes8Q8vdcTEP6mGqba9HpG9UPrJ2d8f8cQpr5A@mail.gmail.com>
Message-ID: <loom.20120927T145536-405@post.gmane.org>

Daniel Holth <dholth <at> gmail.com> writes:

> 
> Not bad. The wheel installer could probably use the scripts wrapper.
> 

Good to know, thanks. I noticed that the PEP referred to script installation but
didn't notice it in the wheel implementation.

Regards,

Vinay Sajip


From erik.m.bray at gmail.com  Thu Sep 27 15:08:00 2012
From: erik.m.bray at gmail.com (Erik Bray)
Date: Thu, 27 Sep 2012 09:08:00 -0400
Subject: [Distutils] Differences in PEP386 and setuptools
In-Reply-To: <AE15D652194A4C70BADC05166BAEF0DF@gmail.com>
References: <AE15D652194A4C70BADC05166BAEF0DF@gmail.com>
Message-ID: <CAOTD34Yb7WoQAosnEW3bsKXRxWFEtueKwEYMh=rMomnA79NduQ@mail.gmail.com>

On Wed, Sep 26, 2012 at 3:09 PM, Donald Stufft <donald.stufft at gmail.com> wrote:
> I've been comparing how PEP386/distutils2.version treats versions
> and how pkg_resources from setuptools treats versions and it
> confirmed a worry of me with the way that dev is treated in PEP386.
>
> In PEP386 you have several kinds of "top level" releases, these
> are alpha, beta, candidate, and final and they are sorted as you
> would expect. On top of that it has pre and post releases for
> any of those top level releases. The pre releases are tagged
> with a ".dev" and post releases are tagged with a ".post". They
> are sorted immediately before/after the "main" release that they
> are pre/post for.
>
> In pkg_resources dev is treated as it's own "main" release and
> sorts it in front of an alpha.
>
> This means that given the versions:
>
>     ["0.1.dev1", "0.1a1", "0.1b1", "0.1c1", "0.1"]
>
> PEP386 will sort them as:
>
>     ["0.1a1", "0.1b1", "0.1c1", "0.1.dev1", "0.1"]
>
> and pkg_resources will sort them as:
>
>     ["0.1.dev1", "0.1a1", "0.1b1", "0.1c1", "0.1"]
>
>
> To further complicate things the most common usage I've personally seen
> in the wild is that of "0.1dev" or "0.1dev1" which the author expects to
> sort
> before an alpha (In this case distutils2.version throws an error, but the
> suggest
> function is able to turn it into 0.1.dev1).
>
> I think this difference is going to cause confusion, especially during the
> the
> transition period when you're going to have people using both pkg_resources
> and the new PEP386 functions.
>
> Since PEP386 is only in the "Accepted" stage, and isn't part of the official
> implementation yet, is it at all possible to revise it? Ideally I think to
> follow
> with the prior art, and people's expectations "dev" should be moved to a
> "main"
> release type sorted before an alpha, and to take it's place as a pre release
> modifier
> perhaps something like "pre" can be used instead (e.g. "0.1.pre1").

+1  Thanks for re-raising this issue.

The PEP 386 ordering completely breaks the expectations inherent in my
dev/release process.  That may just be me, but anecdotal evidence
seems to imply that this is generally the case.

Erik

From erik.m.bray at gmail.com  Thu Sep 27 15:12:44 2012
From: erik.m.bray at gmail.com (Erik Bray)
Date: Thu, 27 Sep 2012 09:12:44 -0400
Subject: [Distutils] Differences in PEP386 and setuptools
In-Reply-To: <215BFDFF990F49E5A920022A9C8E1F47@gmail.com>
References: <AE15D652194A4C70BADC05166BAEF0DF@gmail.com>
	<20120926200028.GH2974@unaka.lan>
	<215BFDFF990F49E5A920022A9C8E1F47@gmail.com>
Message-ID: <CAOTD34Yr7A4gCtTOuvK0WUcCruBEox7zFzdXEoCFsC+C7gn4Og@mail.gmail.com>

On Wed, Sep 26, 2012 at 4:24 PM, Donald Stufft <donald.stufft at gmail.com> wrote:
> On Wednesday, September 26, 2012 at 4:00 PM, Toshio Kuratomi wrote:
>
> Note that this was an intentional difference with setuptools.
>
> Do you have any idea _why_? It seems from my perspective (of someone
> who wasn't really paying attention back then) to be a fairly arbitrary
> difference
> in both prior art and what I've personally seen as the common usage that has
> a good chance of causing confusion (as it already has).
>

My guess--and this is pure speculation as I wasn't following the
original discussions either--is that it fell out of the desire to
require versions to be lexicographically ordered.  So "dev" would
*have* to come after "a", "b", and "c" whether we like it or not.  But
at some point the convention to use ".dev" was proposed, which solves
this issue--this puts it before "a" lexicographically and thus makes
more sense in general.  Although ".dev" made it into PEP 386 it seems
the rest of the PEP wasn't changed to reflect the fact that ".dev" now
comes first.

Again--just speculation.  But I don't see any way that the PEP as
written makes sense without assuming some kind of editorial oversight.

Erik

From vinay_sajip at yahoo.co.uk  Thu Sep 27 15:14:21 2012
From: vinay_sajip at yahoo.co.uk (Vinay Sajip)
Date: Thu, 27 Sep 2012 13:14:21 +0000 (UTC)
Subject: [Distutils] distlib updated with resources API
References: <loom.20120923T174554-748@post.gmane.org>
	<50636845.8080204@ziade.org>
	<loom.20120926T233609-992@post.gmane.org>
	<CAG8k2+6kJevxaXmivECB8+Pm1pPOE1ANCqyH1FgA_CXt09+JeQ@mail.gmail.com>
	<loom.20120927T072155-429@post.gmane.org>
	<CAG8k2+5ynKdXmP1HPbmK4769ZnT_VJxj_dRNdZZQPFJsX4pMvg@mail.gmail.com>
Message-ID: <loom.20120927T145654-634@post.gmane.org>

Daniel Holth <dholth <at> gmail.com> writes:

> 
> What you can do with pkgutil is call
> pkgutil.iter_importer_modules.register(importer, function) so that
> your new importer works with iter_modules. Unfortunately even though
> it implements most of what you would need to listdir() a zip file,
> using zipimport._zip_directory_cache[], it does not expose the
> functionality.

I haven't defined any new importers. In terms of listing a whole zip's contents,
that's not part of the idea of a resource: you need to know what resource you
want, before you can get it, at least at the top level. Of
course, you can iterate over a resource tree once you have a top-level
resource. Possibly an iterator could be provided for convenience, but I'm not
sure what the use case is. For example, should particular files like __pycache__
and .pyc be excluded when operating on file system resources?

I don't use _zip_directory_cache directly - it's exposed as the zipimport
loader's "_files" attribute, and I use that.

> Make sure you have a test where the package is not in the root of the .zip file
> 
> foo.zip/site-packages/bar.py
>

Thanks for pointing that out, I've raised an issue to remind myself.

> Speaking of old code, does anyone feel like replacing
> 
> # @decorator
> def fn(): ...
> fn = decorator(fn)
> 
> with the probably-ok-to-use-by-now decorator syntax?

Where's the anachronistic code you're referring to?

Regards,

Vinay Sajip


From erik.m.bray at gmail.com  Thu Sep 27 15:17:40 2012
From: erik.m.bray at gmail.com (Erik Bray)
Date: Thu, 27 Sep 2012 09:17:40 -0400
Subject: [Distutils] Differences in PEP386 and setuptools
In-Reply-To: <CAOTD34Yr7A4gCtTOuvK0WUcCruBEox7zFzdXEoCFsC+C7gn4Og@mail.gmail.com>
References: <AE15D652194A4C70BADC05166BAEF0DF@gmail.com>
	<20120926200028.GH2974@unaka.lan>
	<215BFDFF990F49E5A920022A9C8E1F47@gmail.com>
	<CAOTD34Yr7A4gCtTOuvK0WUcCruBEox7zFzdXEoCFsC+C7gn4Og@mail.gmail.com>
Message-ID: <CAOTD34aFFdtsvvEwHfLjfT119Xr_6cAyuZbRoySuNFzZr7eiQg@mail.gmail.com>

On Thu, Sep 27, 2012 at 9:12 AM, Erik Bray <erik.m.bray at gmail.com> wrote:
> On Wed, Sep 26, 2012 at 4:24 PM, Donald Stufft <donald.stufft at gmail.com> wrote:
>> On Wednesday, September 26, 2012 at 4:00 PM, Toshio Kuratomi wrote:
>>
>> Note that this was an intentional difference with setuptools.
>>
>> Do you have any idea _why_? It seems from my perspective (of someone
>> who wasn't really paying attention back then) to be a fairly arbitrary
>> difference
>> in both prior art and what I've personally seen as the common usage that has
>> a good chance of causing confusion (as it already has).
>>
>
> My guess--and this is pure speculation as I wasn't following the
> original discussions either--is that it fell out of the desire to
> require versions to be lexicographically ordered.  So "dev" would
> *have* to come after "a", "b", and "c" whether we like it or not.  But
> at some point the convention to use ".dev" was proposed, which solves
> this issue--this puts it before "a" lexicographically and thus makes
> more sense in general.  Although ".dev" made it into PEP 386 it seems
> the rest of the PEP wasn't changed to reflect the fact that ".dev" now
> comes first.
>
> Again--just speculation.  But I don't see any way that the PEP as
> written makes sense without assuming some kind of editorial oversight.

I should add: I'm not saying that this scheme actually *does* allow
versions to be lexicographically ordered.  For example, "0.1.dev" >
"0.1.1" which is certainly not desired.  I'm just guessing that
somebody was trying to tweak the scheme so that it would work this
way.

Erik

From dholth at gmail.com  Thu Sep 27 15:25:26 2012
From: dholth at gmail.com (Daniel Holth)
Date: Thu, 27 Sep 2012 09:25:26 -0400
Subject: [Distutils] distlib updated with resources API
In-Reply-To: <loom.20120927T145654-634@post.gmane.org>
References: <loom.20120923T174554-748@post.gmane.org>
	<50636845.8080204@ziade.org>
	<loom.20120926T233609-992@post.gmane.org>
	<CAG8k2+6kJevxaXmivECB8+Pm1pPOE1ANCqyH1FgA_CXt09+JeQ@mail.gmail.com>
	<loom.20120927T072155-429@post.gmane.org>
	<CAG8k2+5ynKdXmP1HPbmK4769ZnT_VJxj_dRNdZZQPFJsX4pMvg@mail.gmail.com>
	<loom.20120927T145654-634@post.gmane.org>
Message-ID: <CAG8k2+4CLTbft0PjnWv4FJYLi_cvDKEJDPNmvOsyTiQO1+SxDw@mail.gmail.com>

On Thu, Sep 27, 2012 at 9:14 AM, Vinay Sajip <vinay_sajip at yahoo.co.uk> wrote:
> I haven't defined any new importers. In terms of listing a whole zip's contents,
> that's not part of the idea of a resource: you need to know what resource you
> want, before you can get it, at least at the top level. Of
> course, you can iterate over a resource tree once you have a top-level
> resource. Possibly an iterator could be provided for convenience, but I'm not
> sure what the use case is. For example, should particular files like __pycache__
> and .pyc be excluded when operating on file system resources?

The pkgutil "walk modules/packages" API uses listdir / equivalent to
find the packages. So does the code that finds .egg-info / .dist-info.

> I don't use _zip_directory_cache directly - it's exposed as the zipimport
> loader's "_files" attribute, and I use that.

Really the importer API is deficient, but that is a different problem.

>> Speaking of old code, does anyone feel like replacing
>>
>> # @decorator
>> def fn(): ...
>> fn = decorator(fn)
>>
>> with the probably-ok-to-use-by-now decorator syntax?
>
> Where's the anachronistic code you're referring to?

pkgutil and pkg_resources

From donald.stufft at gmail.com  Thu Sep 27 15:25:51 2012
From: donald.stufft at gmail.com (Donald Stufft)
Date: Thu, 27 Sep 2012 09:25:51 -0400
Subject: [Distutils] Differences in PEP386 and setuptools
In-Reply-To: <CAOTD34Yr7A4gCtTOuvK0WUcCruBEox7zFzdXEoCFsC+C7gn4Og@mail.gmail.com>
References: <AE15D652194A4C70BADC05166BAEF0DF@gmail.com>
	<20120926200028.GH2974@unaka.lan>
	<215BFDFF990F49E5A920022A9C8E1F47@gmail.com>
	<CAOTD34Yr7A4gCtTOuvK0WUcCruBEox7zFzdXEoCFsC+C7gn4Og@mail.gmail.com>
Message-ID: <93ECE7DE9BE749399E79589762F936CE@gmail.com>

On Thursday, September 27, 2012 at 9:12 AM, Erik Bray wrote:
> My guess--and this is pure speculation as I wasn't following the
> original discussions either--is that it fell out of the desire to
> require versions to be lexicographically ordered. So "dev" would
> *have* to come after "a", "b", and "c" whether we like it or not. But
> at some point the convention to use ".dev" was proposed, which solves
> this issue--this puts it before "a" lexicographically and thus makes
> more sense in general. Although ".dev" made it into PEP 386 it seems
> the rest of the PEP wasn't changed to reflect the fact that ".dev" now
> comes first.
> 
> Again--just speculation. But I don't see any way that the PEP as
> written makes sense without assuming some kind of editorial oversight.
> 
If that's the reason (and I wasn't around during the original discussions) then
that's already been broken by rc/final (final is given an internal representation
of `z` to allow ordering to work. 

The way the PEP is written .devN is a pre-release of a specific version (a version
being a numbered release of either alpha, beta, candidate, or final). While I think
pre-releases of a specific version is likely useful in some context, I think that it
should be using an identifier more like `pre`. Using the `dev` tag to me violates
what a user expects from history.

I'm not discounting that there is a reason that makes sense though, hopefully if
there is one someone will be able to come forth with it. I personally feel though
that the current behavior is unexpected, will cause confusion as long as releases
are being ordered by both pkg_resources and PEP386, and goes against common
usage of the release tags. My proposal would be to replace the .devN pre-release
tag with .preN, and add a `devN` tag (perhaps optional N, with an omitted N being
equal to `dev0`. I feel like this will fit into people's expectations better, and make
migration easier.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20120927/cf5fe507/attachment.html>

From dholth at gmail.com  Thu Sep 27 15:56:10 2012
From: dholth at gmail.com (Daniel Holth)
Date: Thu, 27 Sep 2012 09:56:10 -0400
Subject: [Distutils] Upstream packaging feedback
In-Reply-To: <60935.71.163.73.57.1348193994.squirrel@www.cpcug.org>
References: <mailman.87.1347537619.23756.python-devel@lists.fedoraproject.org>
	<60935.71.163.73.57.1348193994.squirrel@www.cpcug.org>
Message-ID: <CAG8k2+7UG0L1PeyXK2EcXcb+dq3tDDG4fxjFhYJj9rUHaXm2=w@mail.gmail.com>

On Thu, Sep 20, 2012 at 10:19 PM, Stanley A. Klein <sklein at cpcug.org> wrote:
> So, please add the FHS issue to your list.
>
> I've also found that everything seems to be in a state of flux right now,
> with setuptools, distribute, distutils, distutils2, and
> pypi2rpm/bdist_rpm2 all out there and bumping into each other.  When I
> tried to do a yum update to my Fedora 14 system, I ran into trouble
> because of conflicts among these packages.  I hope this all settles down
> soon.

The FHS issues are on the list, which is why there are "resource
categories" that can be installed wherever in new-packaging land.

Unfortunately my experience is that system Python packaging doesn't
work for application developers. That is why developers remove
themselves from the system Python as quickly as possible by using
virtualenv or by compiling an up-to-date version of Python, and use
only the packages installed in that virtualenv.

It's unfortunate because there are a few impossible-to-compile things
like qt or cairo bindings that would be really nice to be able to get
at, but for example if I install the wrong packages on Ubuntu,

"python -c 'import keyring'" segfaults.

The GTK/gobject bindings are also fraught with peril.

Basically Python's site-packages directory is the equivalent of
automatically re-linking every C program with every library in /lib
every time a new library is installed.

We could use a solution that is like egg except politically
acceptable. Maybe an import hook that knows which top-level packages
you actually want, and hides the complexity of excluding the other
ones instead of producing a gut-wrechingly long sys.path...

From tarek at ziade.org  Thu Sep 27 16:01:24 2012
From: tarek at ziade.org (=?UTF-8?B?VGFyZWsgWmlhZMOp?=)
Date: Thu, 27 Sep 2012 16:01:24 +0200
Subject: [Distutils] distlib updated with resources API
In-Reply-To: <loom.20120926T233609-992@post.gmane.org>
References: <loom.20120923T174554-748@post.gmane.org>
	<50636845.8080204@ziade.org>
	<loom.20120926T233609-992@post.gmane.org>
Message-ID: <50645C34.6070606@ziade.org>

On 9/26/12 11:58 PM, Vinay Sajip wrote:
> Tarek Ziad? <tarek <at> ziade.org> writes:
>
>> On a side note, since these are the original modules that were taking
>> out of Python's packaging implementation,
>> I don't think you can copyright them under your name like what I have
>> seen in setup.py
> AFAIK I've only added my copyright to the individual files I've created, but not
> to any of the files I've copied over from packaging. The plan is to move the
> project over to hg.python.org at some point, but Antoine suggested (on
> python-dev) leaving it on BitBucket until it gets a little more mature. I'm fine
> with that too; there's still a lot to do on it.

I think it's perfectly fine to have hg.python.org/distlib and do what 
you described.

It can mature there - it does not bother cpython or other repositories.

Plus, it makes it easier to avoid any licensing headache in a few 
months/years,
since any contributor just have to sign the Python contributor agreement.


>
>> Unless your plan is not to have distlib incorporated into Python but
>> roll a separate project, I'd recommend having it in hg.python.org
>> under the PSF umbrella.
> I think the suggestion was to keep it out of the stdlib for now, and add all or
> part of it later, as and when there's a general consensus on python-dev about
> its fitness for purpose. I can certainly update the files which I copied from
> packaging with a PSF copyright (or other copyright, like the Fellowship of the
> Packaging), if that would make you more comfortable, and / or state "Portions
> Copyright XXX" in setup.py. I certainly don't intend for copyright issues to be
> contentious :-)

I would just leave it in hg.python.org and drop any single author 
header, and
have the project driven by the community under the PSF governance.

If we want to thank contributors like you or me or anyone that helped in 
this code
base we can always maintain a CONTRIBUTORS.txt file.

What I'd like to avoid is distlib becoming a project that's owned/driven 
by a single
person -- even if in the current effort you are the person that is 
contributing
and driving things.

I hope this does not sound like harsh or anything - I am thankful you 
are doing this.

I just think it's important that this project stays under the python-dev 
umbrella
as the "official" subproject of packaging/distutils2 to have a smoother
transition later - and have you as its de-facto maintainer.

Cheers
Tarek



From p.f.moore at gmail.com  Thu Sep 27 16:22:39 2012
From: p.f.moore at gmail.com (Paul Moore)
Date: Thu, 27 Sep 2012 15:22:39 +0100
Subject: [Distutils] distlib updated with resources API
In-Reply-To: <50645C34.6070606@ziade.org>
References: <loom.20120923T174554-748@post.gmane.org>
	<50636845.8080204@ziade.org>
	<loom.20120926T233609-992@post.gmane.org>
	<50645C34.6070606@ziade.org>
Message-ID: <CACac1F-CgXw2t3rwF2B4e1efgYR6jJmt0yY1cdKmwa-AMX7fbA@mail.gmail.com>

On 27 September 2012 15:01, Tarek Ziad? <tarek at ziade.org> wrote:
> I think it's perfectly fine to have hg.python.org/distlib and do what you
> described.
>
> It can mature there - it does not bother cpython or other repositories.

One advantage of distlib being on bitbucket is that anyone can fork it
and create pull requests. I've been doing that recently on a number of
projects, and it's a very efficient workflow. I don't know how the
workflow would go if hg.python.org was used, but I'd be much less
likely to contribute code if I had to create a patch, submit it via
the tracker, and manually maintain it in a local copy of the code
until it gets accepted.

Maybe it's possible to integrate hg.python.org with bitbucket - I see
bitbucket allows you to create a repo by forking an externally hosted
Mercurial repository, but I've never tried it, and I don't know if
pull requests would work.

Paul.

From tarek at ziade.org  Thu Sep 27 16:26:33 2012
From: tarek at ziade.org (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Thu, 27 Sep 2012 16:26:33 +0200
Subject: [Distutils] distlib updated with resources API
In-Reply-To: <CACac1F-CgXw2t3rwF2B4e1efgYR6jJmt0yY1cdKmwa-AMX7fbA@mail.gmail.com>
References: <loom.20120923T174554-748@post.gmane.org>
	<50636845.8080204@ziade.org>
	<loom.20120926T233609-992@post.gmane.org>
	<50645C34.6070606@ziade.org>
	<CACac1F-CgXw2t3rwF2B4e1efgYR6jJmt0yY1cdKmwa-AMX7fbA@mail.gmail.com>
Message-ID: <50646219.9080205@ziade.org>

On 9/27/12 4:22 PM, Paul Moore wrote:
> On 27 September 2012 15:01, Tarek Ziad? <tarek at ziade.org> wrote:
>> I think it's perfectly fine to have hg.python.org/distlib and do what you
>> described.
>>
>> It can mature there - it does not bother cpython or other repositories.
> One advantage of distlib being on bitbucket is that anyone can fork it
> and create pull requests. I've been doing that recently on a number of
> projects, and it's a very efficient workflow. I don't know how the
> workflow would go if hg.python.org was used, but I'd be much less
> likely to contribute code if I had to create a patch, submit it via
> the tracker, and manually maintain it in a local copy of the code
> until it gets accepted.

But then, unless I am mistaken, you have to ask each contributor to sign 
the agreement to avoid any issue
the day it goes into Python.

I think you can mirror hg.python.org on bitbucket, see below

>
> Maybe it's possible to integrate hg.python.org with bitbucket - I see
> bitbucket allows you to create a repo by forking an externally hosted
> Mercurial repository, but I've never tried it, and I don't know if
> pull requests would work.

bitbucket has hg.python.org mirrors IIRC - there's one for cpython and I 
guess we could ask one for another repo ?

>
> Paul.


From p.f.moore at gmail.com  Thu Sep 27 16:40:00 2012
From: p.f.moore at gmail.com (Paul Moore)
Date: Thu, 27 Sep 2012 15:40:00 +0100
Subject: [Distutils] distlib updated with resources API
In-Reply-To: <50646219.9080205@ziade.org>
References: <loom.20120923T174554-748@post.gmane.org>
	<50636845.8080204@ziade.org>
	<loom.20120926T233609-992@post.gmane.org>
	<50645C34.6070606@ziade.org>
	<CACac1F-CgXw2t3rwF2B4e1efgYR6jJmt0yY1cdKmwa-AMX7fbA@mail.gmail.com>
	<50646219.9080205@ziade.org>
Message-ID: <CACac1F-WDQirObA3tsw=XEtK04V1PFQJ=N51mQMn5sxF1Ni1nQ@mail.gmail.com>

(Sorry, meant to post to the list)

On 27 September 2012 15:26, Tarek Ziad? <tarek at ziade.org> wrote:
> But then, unless I am mistaken, you have to ask each contributor to sign the
> agreement to avoid any issue the day it goes into Python.

TBH, I'm not sure. I've contributed some patches to Python via the
tracker, and I honestly don't recall if I ever signed an agreement (I
don't mind doing so, just don't know if I did). How would a pull
request on bitbucket be any different?

Paul

From tarek at ziade.org  Thu Sep 27 16:45:10 2012
From: tarek at ziade.org (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Thu, 27 Sep 2012 16:45:10 +0200
Subject: [Distutils] distlib updated with resources API
In-Reply-To: <CACac1F-WDQirObA3tsw=XEtK04V1PFQJ=N51mQMn5sxF1Ni1nQ@mail.gmail.com>
References: <loom.20120923T174554-748@post.gmane.org>
	<50636845.8080204@ziade.org>
	<loom.20120926T233609-992@post.gmane.org>
	<50645C34.6070606@ziade.org>
	<CACac1F-CgXw2t3rwF2B4e1efgYR6jJmt0yY1cdKmwa-AMX7fbA@mail.gmail.com>
	<50646219.9080205@ziade.org>
	<CACac1F-WDQirObA3tsw=XEtK04V1PFQJ=N51mQMn5sxF1Ni1nQ@mail.gmail.com>
Message-ID: <50646676.3050705@ziade.org>

On 9/27/12 4:40 PM, Paul Moore wrote:
> (Sorry, meant to post to the list)
>
> On 27 September 2012 15:26, Tarek Ziad? <tarek at ziade.org> wrote:
>> But then, unless I am mistaken, you have to ask each contributor to sign the
>> agreement to avoid any issue the day it goes into Python.
> TBH, I'm not sure. I've contributed some patches to Python via the
> tracker, and I honestly don't recall if I ever signed an agreement (I
> don't mind doing so, just don't know if I did). How would a pull
> request on bitbucket be any different?

I think it's a border case where you are under the 'authority' of the 
person that do the merge.

I do recall I had to make 10+ people sign the agreement at the last 
packaging sprint, even if some of them just did pull requests...

But I am not a lawyer - ccing Van!



>
> Paul


From solipsis at pitrou.net  Thu Sep 27 16:28:06 2012
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Thu, 27 Sep 2012 16:28:06 +0200
Subject: [Distutils] distlib updated with resources API
In-Reply-To: <50645C34.6070606@ziade.org>
References: <loom.20120923T174554-748@post.gmane.org>
	<50636845.8080204@ziade.org> <loom.20120926T233609-992@post.gmane.org>
	<50645C34.6070606@ziade.org>
Message-ID: <1348756086.3495.1.camel@localhost.localdomain>

Le jeudi 27 septembre 2012 ? 16:01 +0200, Tarek Ziad? a ?crit :
> On 9/26/12 11:58 PM, Vinay Sajip wrote:
> > Tarek Ziad? <tarek <at> ziade.org> writes:
> >
> >> On a side note, since these are the original modules that were taking
> >> out of Python's packaging implementation,
> >> I don't think you can copyright them under your name like what I have
> >> seen in setup.py
> > AFAIK I've only added my copyright to the individual files I've created, but not
> > to any of the files I've copied over from packaging. The plan is to move the
> > project over to hg.python.org at some point, but Antoine suggested (on
> > python-dev) leaving it on BitBucket until it gets a little more mature. I'm fine
> > with that too; there's still a lot to do on it.
> 
> I think it's perfectly fine to have hg.python.org/distlib and do what 
> you described.
> 
> It can mature there - it does not bother cpython or other repositories.
> 
> Plus, it makes it easier to avoid any licensing headache in a few 
> months/years,
> since any contributor just have to sign the Python contributor agreement.

There are two of us maintaining hg.python.org: Georg and I. So, I don't
know about Georg, but I don't want to maintain repositories for every
third-party library that might one day become part of Python. OTOH, if
Georg wants to handle it, then fine :-)

Regards

Antoine.


-- 
Software development and contracting: http://pro.pitrou.net



From tarek at ziade.org  Thu Sep 27 17:13:03 2012
From: tarek at ziade.org (=?UTF-8?B?VGFyZWsgWmlhZMOp?=)
Date: Thu, 27 Sep 2012 17:13:03 +0200
Subject: [Distutils] distlib updated with resources API
In-Reply-To: <1348756086.3495.1.camel@localhost.localdomain>
References: <loom.20120923T174554-748@post.gmane.org>
	<50636845.8080204@ziade.org>
	<loom.20120926T233609-992@post.gmane.org>
	<50645C34.6070606@ziade.org>
	<1348756086.3495.1.camel@localhost.localdomain>
Message-ID: <50646CFF.3010601@ziade.org>

On 9/27/12 4:28 PM, Antoine Pitrou wrote:
> Le jeudi 27 septembre 2012 ? 16:01 +0200, Tarek Ziad? a ?crit :
>> On 9/26/12 11:58 PM, Vinay Sajip wrote:
>>> Tarek Ziad? <tarek <at> ziade.org> writes:
>>>
>>>> On a side note, since these are the original modules that were taking
>>>> out of Python's packaging implementation,
>>>> I don't think you can copyright them under your name like what I have
>>>> seen in setup.py
>>> AFAIK I've only added my copyright to the individual files I've created, but not
>>> to any of the files I've copied over from packaging. The plan is to move the
>>> project over to hg.python.org at some point, but Antoine suggested (on
>>> python-dev) leaving it on BitBucket until it gets a little more mature. I'm fine
>>> with that too; there's still a lot to do on it.
>> I think it's perfectly fine to have hg.python.org/distlib and do what
>> you described.
>>
>> It can mature there - it does not bother cpython or other repositories.
>>
>> Plus, it makes it easier to avoid any licensing headache in a few
>> months/years,
>> since any contributor just have to sign the Python contributor agreement.
> There are two of us maintaining hg.python.org: Georg and I. So, I don't
> know about Georg, but I don't want to maintain repositories for every
> third-party library that might one day become part of Python. OTOH, if
> Georg wants to handle it, then fine :-)

Since I see distutils2, unittest2, stackless and many users repo in there,
Please define the exact rules here - rather than you willingness to do 
the benevolent work.

and what you mean by maintaining an extra repo exactly.




From solipsis at pitrou.net  Thu Sep 27 17:40:12 2012
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Thu, 27 Sep 2012 15:40:12 +0000 (UTC)
Subject: [Distutils] distlib updated with resources API
References: <loom.20120923T174554-748@post.gmane.org>
	<50636845.8080204@ziade.org>
	<loom.20120926T233609-992@post.gmane.org>
	<50645C34.6070606@ziade.org>
	<1348756086.3495.1.camel@localhost.localdomain>
	<50646CFF.3010601@ziade.org>
Message-ID: <loom.20120927T173348-617@post.gmane.org>

Tarek Ziad? <tarek <at> ziade.org> writes:
> > There are two of us maintaining hg.python.org: Georg and I. So, I don't
> > know about Georg, but I don't want to maintain repositories for every
> > third-party library that might one day become part of Python. OTOH, if
> > Georg wants to handle it, then fine 
> 
> Since I see distutils2, unittest2, stackless and many users repo in there,
> Please define the exact rules here - rather than you willingness to do 
> the benevolent work.
> 
> and what you mean by maintaining an extra repo exactly.

Maintaining a repo means setting it up and possibly changing the configuration 
(e.g. commit hooks) when required. This is all done by hand by one of us.

My main point is not that hg.p.o is closed to new repositories, but that it is 
not meant to become a "forge" or an incubator where anyone can create new 
Python repos. Is distlib important enough yet?

Regards

Antoine.



From tarek at ziade.org  Thu Sep 27 17:49:45 2012
From: tarek at ziade.org (=?UTF-8?B?VGFyZWsgWmlhZMOp?=)
Date: Thu, 27 Sep 2012 17:49:45 +0200
Subject: [Distutils] distlib updated with resources API
In-Reply-To: <loom.20120927T173348-617@post.gmane.org>
References: <loom.20120923T174554-748@post.gmane.org>
	<50636845.8080204@ziade.org>
	<loom.20120926T233609-992@post.gmane.org>
	<50645C34.6070606@ziade.org>
	<1348756086.3495.1.camel@localhost.localdomain>
	<50646CFF.3010601@ziade.org>
	<loom.20120927T173348-617@post.gmane.org>
Message-ID: <50647599.3030507@ziade.org>

On 9/27/12 5:40 PM, Antoine Pitrou wrote:
> Tarek Ziad? <tarek <at> ziade.org> writes:
>>> There are two of us maintaining hg.python.org: Georg and I. So, I don't
>>> know about Georg, but I don't want to maintain repositories for every
>>> third-party library that might one day become part of Python. OTOH, if
>>> Georg wants to handle it, then fine
>> Since I see distutils2, unittest2, stackless and many users repo in there,
>> Please define the exact rules here - rather than you willingness to do
>> the benevolent work.
>>
>> and what you mean by maintaining an extra repo exactly.
> Maintaining a repo means setting it up and possibly changing the configuration
> (e.g. commit hooks) when required. This is all done by hand by one of us.
>
> My main point is not that hg.p.o is closed to new repositories, but that it is
> not meant to become a "forge" or an incubator where anyone can create new
> Python repos. Is distlib important enough yet?

I don't know what you mean by 'important'

We removed packaging from Python and said we would work on a smaller set 
for inclusion.

I've also said that I believed that it's simpler to include back wrt 
licensing if it's a code base
that's under the contributors agreement.

Last but not least, distlib is the plan forward endorsed by python-dev, 
so having it
in hg.python.org makes that plan more legitimate, no ?

I frankly don't follow your reluctance here, and I find your definition 
of what can
be in hg.python.org to be very vague

When I asked for a distutils2 repository it was done in 2 minutes and I 
did not
have to argue for hours and potentially push the discussion to yet 
another packaging drama

This is really annoying  :/

>
> Regards
>
> Antoine.
>
>
> _______________________________________________
> Distutils-SIG maillist  -  Distutils-SIG at python.org
> http://mail.python.org/mailman/listinfo/distutils-sig


From a.badger at gmail.com  Thu Sep 27 17:59:00 2012
From: a.badger at gmail.com (Toshio Kuratomi)
Date: Thu, 27 Sep 2012 08:59:00 -0700
Subject: [Distutils] Differences in PEP386 and setuptools
In-Reply-To: <93ECE7DE9BE749399E79589762F936CE@gmail.com>
References: <AE15D652194A4C70BADC05166BAEF0DF@gmail.com>
	<20120926200028.GH2974@unaka.lan>
	<215BFDFF990F49E5A920022A9C8E1F47@gmail.com>
	<CAOTD34Yr7A4gCtTOuvK0WUcCruBEox7zFzdXEoCFsC+C7gn4Og@mail.gmail.com>
	<93ECE7DE9BE749399E79589762F936CE@gmail.com>
Message-ID: <20120927155900.GI2974@unaka.lan>

On Thu, Sep 27, 2012 at 09:25:51AM -0400, Donald Stufft wrote:
> On Thursday, September 27, 2012 at 9:12 AM, Erik Bray wrote:
> 
>     My guess--and this is pure speculation as I wasn't following the
>     original discussions either--is that it fell out of the desire to
>     require versions to be lexicographically ordered.
> 
This was not it.  I was for a lexicographic ordering but it was rejected.
> 
> 
> If that's the reason (and I wasn't around during the original discussions) then
> that's already been broken by rc/final (final is given an internal
> representation
> of `z` to allow ordering to work. 

Seems a little fragile :-)  But it is internal...
> 
> The way the PEP is written .devN is a pre-release of a specific version (a
> version
> being a numbered release of either alpha, beta, candidate, or final). While I
> think
> pre-releases of a specific version is likely useful in some context, I think
> that it
> should be using an identifier more like `pre`. Using the `dev` tag to me
> violates
> what a user expects from history.
> 
> I'm not discounting that there is a reason that makes sense though, hopefully
> if
> there is one someone will be able to come forth with it.
>
I was hoping that Tarek would chime in since the discussion about this was
years ago (2009, I think) and my memory is slightly fuzzy.  I'll try to
remember the rationale and Tarek can correct me.

setuptools allows any random string into the versions.  Certain strings have
special meaning.  Unknown strings are lexicographically sorted.  Rereading
this section of the PEP
http://www.python.org/dev/peps/pep-0386/#caveats-of-existing-systems

will give you the background on problems with the setuptools concept.
For part of the rationale about .dev, the first paragraph of the rationale
is important.  setuptools allowed too many things to be allowed into
versions.  Remembering how the specialcases mix with the lexicographic
sorting lead to confusion when some project starts working with non-standard
version strings.  So one of the goals of PEP 386 was to make a small,
discrete number of legal version strings.

Keeping that in mind, these are the alpha tags that PEP386 allows:
a, b, c, rc, .post, .dev

* a, b, c, and rc are the typical alpha beta release candidate tags.  We
  debated whether to allow the single letter versions or the long versions
  "alpha", "beta", "rc".  In the end, which set of tags was used was
  a bikeshed argument so it was chosen by PEP-author decree.  I personally
  felt that having both "c" and "rc" was counter to the goals of the PEP but
  we at least agreed that "rc" would intuitively sort after "c" so it wasn't
  as bad as having both "alpha" and "a" where there would be no intuitively
  correct sorting.
* For .post there was a desire to have a tag that sorted after release.
  Choice of .post for the bikeshed color was not controversial.
* For .dev there was a desire to have a tag that sorted before release.
  Choice of .dev for the bikeshed was debated but in the end it was
  a bikeshed and we were all very tired of it.  I'm not sure at this point
  who was championing .dev over .pre or similar or what their rationale was.


> I personally feel
> though
> that the current behavior is unexpected, will cause confusion as long as
> releases
> are being ordered by both pkg_resources and PEP386, and goes against common
> usage of the release tags. My proposal would be to replace the .devN
> pre-release
> tag with .preN, and add a `devN` tag (perhaps optional N, with an omitted N
> being
> equal to `dev0`. I feel like this will fit into people's expectations better,
> and make
> migration easier.
>
I would be for renaming .dev to .pre[1]_ but I would be against the rest of this
proposal.  Having one and only one way to spell things was one of the goals
of the pep.  Having two post release tags that don't already have a defined
meaning leads to confusion about what the ordering should be:

   foo-1.0.dev486 > foo-1.0.post486
   foo-1.0.dev486 < foo-1.0.post486

   foo-1.0.dev486 > foo-1.0.post386
   foo-1.0.dev486 < foo-1.0.post386

To someone who doesn't know the sorting order specified by PEP386, there's
no obvious way to tell which of those is true and which is false.

.. [1]_: rationale for .pre since someone probably has a rationale for .dev
         to counter this:
           * .pre makes it obvious that it has the opposite meaning and use
             case as .post.
           * .pre is more descriptive of how it is supposed to sort than .dev

-Toshio
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 198 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20120927/b731a68a/attachment.pgp>

From g.brandl at gmx.net  Thu Sep 27 18:37:37 2012
From: g.brandl at gmx.net (Georg Brandl)
Date: Thu, 27 Sep 2012 18:37:37 +0200
Subject: [Distutils] distlib updated with resources API
In-Reply-To: <50647599.3030507@ziade.org>
References: <loom.20120923T174554-748@post.gmane.org>
	<50636845.8080204@ziade.org>
	<loom.20120926T233609-992@post.gmane.org>
	<50645C34.6070606@ziade.org>
	<1348756086.3495.1.camel@localhost.localdomain>
	<50646CFF.3010601@ziade.org>
	<loom.20120927T173348-617@post.gmane.org>
	<50647599.3030507@ziade.org>
Message-ID: <k41vce$89g$1@ger.gmane.org>

On 09/27/2012 05:49 PM, Tarek Ziad? wrote:
> On 9/27/12 5:40 PM, Antoine Pitrou wrote:
>> Tarek Ziad? <tarek <at> ziade.org> writes:
>>>> There are two of us maintaining hg.python.org: Georg and I. So, I don't
>>>> know about Georg, but I don't want to maintain repositories for every
>>>> third-party library that might one day become part of Python. OTOH, if
>>>> Georg wants to handle it, then fine
>>> Since I see distutils2, unittest2, stackless and many users repo in there,
>>> Please define the exact rules here - rather than you willingness to do
>>> the benevolent work.
>>>
>>> and what you mean by maintaining an extra repo exactly.
>> Maintaining a repo means setting it up and possibly changing the configuration
>> (e.g. commit hooks) when required. This is all done by hand by one of us.
>>
>> My main point is not that hg.p.o is closed to new repositories, but that it is
>> not meant to become a "forge" or an incubator where anyone can create new
>> Python repos. Is distlib important enough yet?
>
> I don't know what you mean by 'important'
>
> We removed packaging from Python and said we would work on a smaller set
> for inclusion.
>
> I've also said that I believed that it's simpler to include back wrt
> licensing if it's a code base
> that's under the contributors agreement.
>
> Last but not least, distlib is the plan forward endorsed by python-dev,
> so having it
> in hg.python.org makes that plan more legitimate, no ?

No, but I can see why you want it there, and I for one have no problems
creating and maintaining that repository.

> I frankly don't follow your reluctance here, and I find your definition
> of what can
> be in hg.python.org to be very vague

And we'd like to keep it that way.

> When I asked for a distutils2 repository it was done in 2 minutes and I
> did not
> have to argue for hours and potentially push the discussion to yet
> another packaging drama
>
> This is really annoying  :/

Are you sure you're not overreacting?

All that Antoine is saying is that hg.p.o is not a professional hosting
service, and the more repositories we have, the more (potential) requests
we may get that have to be processed.  If you can wait a few days for a
new hook to be installed -- then everything is fine.

Georg



From vinay_sajip at yahoo.co.uk  Thu Sep 27 18:43:40 2012
From: vinay_sajip at yahoo.co.uk (Vinay Sajip)
Date: Thu, 27 Sep 2012 16:43:40 +0000 (UTC)
Subject: [Distutils] distlib updated with resources API
References: <loom.20120923T174554-748@post.gmane.org>
	<50636845.8080204@ziade.org>
	<loom.20120926T233609-992@post.gmane.org>
	<50645C34.6070606@ziade.org>
Message-ID: <loom.20120927T181536-234@post.gmane.org>

Tarek Ziad? <tarek <at> ziade.org> writes:

> I would just leave it in hg.python.org and drop any single author 
> header, and
> have the project driven by the community under the PSF governance.
> 
> If we want to thank contributors like you or me or anyone that helped in 
> this code
> base we can always maintain a CONTRIBUTORS.txt file.
> 
> What I'd like to avoid is distlib becoming a project that's owned/driven 
> by a single 
> person -- even if in the current effort you are the person that is 
> contributing
> and driving things.

Well, my focus here is certainly not on ownership, but I generally add
copyright notices to all the software I produce as a matter of course. If all
it takes is an additional "Licensed to the PSF through a Contributor
Agreement" to regularise things, I'll certainly do that. But the only way it can
be a multi-person project is for multiple people to get practically involved on
a day-to-day level.

I might seem to driving the project right now, but that's only an illusion
due to the fact that I've got spare bandwidth at the moment to think about
how things could work, and write some code, docs and tests. Other priorities
might intervene at any point, so I certainly agree that the project shouldn't
have a low bus factor. Contributors welcome! Just form an orderly line :-)

> I just think it's important that this project stays under the python-dev 
> umbrella
> as the "official" subproject of packaging/distutils2 to have a smoother
> transition later - and have you as its de-facto maintainer.

Hmmm ... holy grail, or poisoned chalice? ;-)

Regards,

Vinay Sajip


From vinay_sajip at yahoo.co.uk  Thu Sep 27 18:48:49 2012
From: vinay_sajip at yahoo.co.uk (Vinay Sajip)
Date: Thu, 27 Sep 2012 16:48:49 +0000 (UTC)
Subject: [Distutils] distlib updated with resources API
References: <loom.20120923T174554-748@post.gmane.org>
	<50636845.8080204@ziade.org>
	<loom.20120926T233609-992@post.gmane.org>
	<50645C34.6070606@ziade.org>
	<1348756086.3495.1.camel@localhost.localdomain>
	<50646CFF.3010601@ziade.org>
	<loom.20120927T173348-617@post.gmane.org>
	<50647599.3030507@ziade.org> <k41vce$89g$1@ger.gmane.org>
Message-ID: <loom.20120927T184442-444@post.gmane.org>

Georg Brandl <g.brandl <at> gmx.net> writes:

>
> No, but I can see why you want it there, and I for one have no problems
> creating and maintaining that repository.
>

Great. Thanks!
 
> 
> And we'd like to keep it that way.
>

:-)

> 
> Are you sure you're not overreacting?
> 
> All that Antoine is saying is that hg.p.o is not a professional hosting
> service, and the more repositories we have, the more (potential) requests
> we may get that have to be processed.  If you can wait a few days for a
> new hook to be installed -- then everything is fine.
> 

I assume you'll just clone the distlib repo on BitBucket. I'm happy to wait.

Thanks, again!

Regards,

Vinay Sajip



From tarek at ziade.org  Thu Sep 27 18:52:10 2012
From: tarek at ziade.org (=?UTF-8?B?VGFyZWsgWmlhZMOp?=)
Date: Thu, 27 Sep 2012 18:52:10 +0200
Subject: [Distutils] distlib updated with resources API
In-Reply-To: <k41vce$89g$1@ger.gmane.org>
References: <loom.20120923T174554-748@post.gmane.org>
	<50636845.8080204@ziade.org>
	<loom.20120926T233609-992@post.gmane.org>
	<50645C34.6070606@ziade.org>
	<1348756086.3495.1.camel@localhost.localdomain>
	<50646CFF.3010601@ziade.org>
	<loom.20120927T173348-617@post.gmane.org>
	<50647599.3030507@ziade.org> <k41vce$89g$1@ger.gmane.org>
Message-ID: <5064843A.6090304@ziade.org>

On 9/27/12 6:37 PM, Georg Brandl wrote:
> On 09/27/2012 05:49 PM, Tarek Ziad? wrote:
>> On 9/27/12 5:40 PM, Antoine Pitrou wrote:
>>> Tarek Ziad? <tarek <at> ziade.org> writes:
>>>>> There are two of us maintaining hg.python.org: Georg and I. So, I 
>>>>> don't
>>>>> know about Georg, but I don't want to maintain repositories for every
>>>>> third-party library that might one day become part of Python. 
>>>>> OTOH, if
>>>>> Georg wants to handle it, then fine
>>>> Since I see distutils2, unittest2, stackless and many users repo in 
>>>> there,
>>>> Please define the exact rules here - rather than you willingness to do
>>>> the benevolent work.
>>>>
>>>> and what you mean by maintaining an extra repo exactly.
>>> Maintaining a repo means setting it up and possibly changing the 
>>> configuration
>>> (e.g. commit hooks) when required. This is all done by hand by one 
>>> of us.
>>>
>>> My main point is not that hg.p.o is closed to new repositories, but 
>>> that it is
>>> not meant to become a "forge" or an incubator where anyone can 
>>> create new
>>> Python repos. Is distlib important enough yet?
>>
>> I don't know what you mean by 'important'
>>
>> We removed packaging from Python and said we would work on a smaller set
>> for inclusion.
>>
>> I've also said that I believed that it's simpler to include back wrt
>> licensing if it's a code base
>> that's under the contributors agreement.
>>
>> Last but not least, distlib is the plan forward endorsed by python-dev,
>> so having it
>> in hg.python.org makes that plan more legitimate, no ?
>
> No, but I can see why you want it there, and I for one have no problems
> creating and maintaining that repository.

Ok thanks
>
>> I frankly don't follow your reluctance here, and I find your definition
>> of what can
>> be in hg.python.org to be very vague
>
> And we'd like to keep it that way.
>
>> When I asked for a distutils2 repository it was done in 2 minutes and I
>> did not
>> have to argue for hours and potentially push the discussion to yet
>> another packaging drama
>>
>> This is really annoying  :/
>
> Are you sure you're not overreacting?
>

Of course I am




From donald.stufft at gmail.com  Thu Sep 27 19:00:10 2012
From: donald.stufft at gmail.com (Donald Stufft)
Date: Thu, 27 Sep 2012 13:00:10 -0400
Subject: [Distutils] Differences in PEP386 and setuptools
In-Reply-To: <20120927155900.GI2974@unaka.lan>
References: <AE15D652194A4C70BADC05166BAEF0DF@gmail.com>
	<20120926200028.GH2974@unaka.lan>
	<215BFDFF990F49E5A920022A9C8E1F47@gmail.com>
	<CAOTD34Yr7A4gCtTOuvK0WUcCruBEox7zFzdXEoCFsC+C7gn4Og@mail.gmail.com>
	<93ECE7DE9BE749399E79589762F936CE@gmail.com>
	<20120927155900.GI2974@unaka.lan>
Message-ID: <FAEA0E647131499088126D0269DD9A9E@gmail.com>

On Thursday, September 27, 2012 at 11:59 AM, Toshio Kuratomi wrote:
> setuptools allows any random string into the versions. Certain strings have
> special meaning. Unknown strings are lexicographically sorted. Rereading
> this section of the PEP
> http://www.python.org/dev/peps/pep-0386/#caveats-of-existing-systems
> 
> will give you the background on problems with the setuptools concept.
> For part of the rationale about .dev, the first paragraph of the rationale
> is important. setuptools allowed too many things to be allowed into
> versions. Remembering how the specialcases mix with the lexicographic
> sorting lead to confusion when some project starts working with non-standard
> version strings. So one of the goals of PEP 386 was to make a small,
> discrete number of legal version strings.

No arguments here on the need of this goal. 
> 
> Keeping that in mind, these are the alpha tags that PEP386 allows:
> a, b, c, rc, .post, .dev
> 
> * a, b, c, and rc are the typical alpha beta release candidate tags. We
> debated whether to allow the single letter versions or the long versions
> "alpha", "beta", "rc". In the end, which set of tags was used was
> a bikeshed argument so it was chosen by PEP-author decree. I personally
> felt that having both "c" and "rc" was counter to the goals of the PEP but
> we at least agreed that "rc" would intuitively sort after "c" so it wasn't
> as bad as having both "alpha" and "a" where there would be no intuitively
> correct sorting.
> * For .post there was a desire to have a tag that sorted after release.
> Choice of .post for the bikeshed color was not controversial.
> * For .dev there was a desire to have a tag that sorted before release.
> Choice of .dev for the bikeshed was debated but in the end it was
> a bikeshed and we were all very tired of it. I'm not sure at this point
> who was championing .dev over .pre or similar or what their rationale was.

Yes I've read this, but sadly all it says about the issue really is "I don't remember". 
> I would be for renaming .dev to .pre[1]_ but I would be against the rest of this
> proposal. Having one and only one way to spell things was one of the goals
> of the pep. Having two post release tags that don't already have a defined
> meaning leads to confusion about what the ordering should be:

dev (in my proposal, and in the way I've seen it used) isn't a post release tag,
it is a separate release at the same level as alpha, beta, rc, final, and it's
meaning tends to be "this is the in development version of what will become
release X.Y", so foo-1.0dev1 means the first development release of what
will become foo-1.0, (just like foo-1.0a1 is the first alpha release of what
will become foo-1.0).

With the current layout (and with your changes to the proposal) there is no
good way to have a development version. With the current layout the best
approximation you can get is 1.0a1.dev1 (similarly with your changes it would
be 1.0a1.pre1).

On the surface 1.0a1.dev1 looks like it might be alright, but my issues with it:

    1. It's not intuitive (at least not to me, development snapshots have
        always come before alphas in my mind)
    2. Goes against what is currently being used for either no good reason or
        a yet to be remembered good reason.
    3. Requires you to decide ahead of time if you're going to go from dev
        to alpha, or straight to a beta/candidate/final. (This is something that
        most small projects don't have set in stone).
     4. It's semantics are different, this is a development release of
         1.0a1 as opposed to a development release of the 1.0 version.
     5. It's just plain ugly.


So to be clear my proposal would be:

    1.0dev1 < 1.0a1 < 1.0b1 < 1.0c1 < 1.0.pre1 < 1.0 < 1.0.post1

as opposed to the current:

    1.0a1 < 1.0b1 < 1.0c1 < 1.0.dev1 < 1.0 < 1.0.post1

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20120927/9c08e2dc/attachment.html>

From merwok at netwok.org  Thu Sep 27 19:46:25 2012
From: merwok at netwok.org (=?UTF-8?B?w4lyaWMgQXJhdWpv?=)
Date: Thu, 27 Sep 2012 13:46:25 -0400
Subject: [Distutils] Differences in PEP386 and setuptools
In-Reply-To: <20120927155900.GI2974@unaka.lan>
References: <AE15D652194A4C70BADC05166BAEF0DF@gmail.com>
	<20120926200028.GH2974@unaka.lan>
	<215BFDFF990F49E5A920022A9C8E1F47@gmail.com>
	<CAOTD34Yr7A4gCtTOuvK0WUcCruBEox7zFzdXEoCFsC+C7gn4Og@mail.gmail.com>
	<93ECE7DE9BE749399E79589762F936CE@gmail.com>
	<20120927155900.GI2974@unaka.lan>
Message-ID: <506490F1.7080909@netwok.org>

Le 27/09/2012 11:59, Toshio Kuratomi a ?crit :
> * a, b, c, and rc are the typical alpha beta release candidate tags.  We
>   debated whether to allow the single letter versions or the long versions
>   "alpha", "beta", "rc".  In the end, which set of tags was used was
>   a bikeshed argument so it was chosen by PEP-author decree.  I personally
>   felt that having both "c" and "rc" was counter to the goals of the PEP but
>   we at least agreed that "rc" would intuitively sort after "c" so it wasn't
>   as bad as having both "alpha" and "a" where there would be no intuitively
>   correct sorting.

Well, c and rc should really compare equal in my opinion.

From dholth at gmail.com  Thu Sep 27 19:50:07 2012
From: dholth at gmail.com (Daniel Holth)
Date: Thu, 27 Sep 2012 13:50:07 -0400
Subject: [Distutils] Differences in PEP386 and setuptools
In-Reply-To: <506490F1.7080909@netwok.org>
References: <AE15D652194A4C70BADC05166BAEF0DF@gmail.com>
	<20120926200028.GH2974@unaka.lan>
	<215BFDFF990F49E5A920022A9C8E1F47@gmail.com>
	<CAOTD34Yr7A4gCtTOuvK0WUcCruBEox7zFzdXEoCFsC+C7gn4Og@mail.gmail.com>
	<93ECE7DE9BE749399E79589762F936CE@gmail.com>
	<20120927155900.GI2974@unaka.lan> <506490F1.7080909@netwok.org>
Message-ID: <CAG8k2+7xwEjyjiExEDHYaEsEh+C-3sQgMA-UEBQ1nEGb9oTvDg@mail.gmail.com>

On Thu, Sep 27, 2012 at 1:46 PM, ?ric Araujo <merwok at netwok.org> wrote:
> Le 27/09/2012 11:59, Toshio Kuratomi a ?crit :
>> * a, b, c, and rc are the typical alpha beta release candidate tags.  We
>>   debated whether to allow the single letter versions or the long versions
>>   "alpha", "beta", "rc".  In the end, which set of tags was used was
>>   a bikeshed argument so it was chosen by PEP-author decree.  I personally
>>   felt that having both "c" and "rc" was counter to the goals of the PEP but
>>   we at least agreed that "rc" would intuitively sort after "c" so it wasn't
>>   as bad as having both "alpha" and "a" where there would be no intuitively
>>   correct sorting.
>
> Well, c and rc should really compare equal in my opinion.
> _______________________________________________
> Distutils-SIG maillist  -  Distutils-SIG at python.org
> http://mail.python.org/mailman/listinfo/distutils-sig

Does that mean p == np?

Ruby gems use http://semver.org/

From fdrake at acm.org  Thu Sep 27 19:58:00 2012
From: fdrake at acm.org (Fred Drake)
Date: Thu, 27 Sep 2012 13:58:00 -0400
Subject: [Distutils] Differences in PEP386 and setuptools
In-Reply-To: <CAG8k2+7xwEjyjiExEDHYaEsEh+C-3sQgMA-UEBQ1nEGb9oTvDg@mail.gmail.com>
References: <AE15D652194A4C70BADC05166BAEF0DF@gmail.com>
	<20120926200028.GH2974@unaka.lan>
	<215BFDFF990F49E5A920022A9C8E1F47@gmail.com>
	<CAOTD34Yr7A4gCtTOuvK0WUcCruBEox7zFzdXEoCFsC+C7gn4Og@mail.gmail.com>
	<93ECE7DE9BE749399E79589762F936CE@gmail.com>
	<20120927155900.GI2974@unaka.lan> <506490F1.7080909@netwok.org>
	<CAG8k2+7xwEjyjiExEDHYaEsEh+C-3sQgMA-UEBQ1nEGb9oTvDg@mail.gmail.com>
Message-ID: <CAFT4OTFnQuHYT-ujJSs7UkPi+E4CY2d6KZd+XJRObqtUK1wuFw@mail.gmail.com>

On Thu, Sep 27, 2012 at 1:50 PM, Daniel Holth <dholth at gmail.com> wrote:
> Ruby gems use http://semver.org/

I don't remember if I was involved in the past discussions (it's possible),
but I'm in favor of using (and requiring) semantic versioning, if only
because we don't have to invent notation or interpretation.


  -Fred

-- 
Fred L. Drake, Jr.    <fdrake at acm.org>
"A person who won't read has no advantage over one who can't read."
   --Samuel Langhorne Clemens

From a.badger at gmail.com  Thu Sep 27 20:03:35 2012
From: a.badger at gmail.com (Toshio Kuratomi)
Date: Thu, 27 Sep 2012 11:03:35 -0700
Subject: [Distutils] Differences in PEP386 and setuptools
In-Reply-To: <FAEA0E647131499088126D0269DD9A9E@gmail.com>
References: <AE15D652194A4C70BADC05166BAEF0DF@gmail.com>
	<20120926200028.GH2974@unaka.lan>
	<215BFDFF990F49E5A920022A9C8E1F47@gmail.com>
	<CAOTD34Yr7A4gCtTOuvK0WUcCruBEox7zFzdXEoCFsC+C7gn4Og@mail.gmail.com>
	<93ECE7DE9BE749399E79589762F936CE@gmail.com>
	<20120927155900.GI2974@unaka.lan>
	<FAEA0E647131499088126D0269DD9A9E@gmail.com>
Message-ID: <20120927180335.GL2974@unaka.lan>

On Thu, Sep 27, 2012 at 01:00:10PM -0400, Donald Stufft wrote:
> On Thursday, September 27, 2012 at 11:59 AM, Toshio Kuratomi wrote:
> 
>     I would be for renaming .dev to .pre[1]_ but I would be against the rest of
>     this
>     proposal. Having one and only one way to spell things was one of the goals
>     of the pep. Having two post release tags that don't already have a defined
>     meaning leads to confusion about what the ordering should be:
> 
> dev (in my proposal, and in the way I've seen it used) isn't a post release
> tag,
> it is a separate release at the same level as alpha, beta, rc, final, and it's
> meaning tends to be "this is the in development version of what will become
> release X.Y", so foo-1.0dev1 means the first development release of what
> will become foo-1.0, (just like foo-1.0a1 is the first alpha release of what
> will become foo-1.0).
> 
> With the current layout (and with your changes to the proposal) there is no
> good way to have a development version. With the current layout the best
> approximation you can get is 1.0a1.dev1 (similarly with your changes it would
> be 1.0a1.pre1).
> 
That's your preference, not capability.  I'd also say you should use
1.0.post1 rather than 1.0a1pre1.

> On the surface 1.0a1.dev1 looks like it might be alright, but my issues with
> it:
> 
>     1. It's not intuitive (at least not to me, development snapshots have
>         always come before alphas in my mind)
>
There's no such consensus.  This is both from dealing with upstream
versioning as a distro packager and from discussion with other people who
worked on the versioning spec.  People were very clear that they wanted to
be able to stick dev versions in the middle of their attempt to release
alphas, betas, rcs, and finals.

You can see this in current, setuptools-driven practice in the snapshots
that pje hosts, for example:
  http://peak.telecommunity.com/snapshots/


>     2. Goes against what is currently being used for either no good reason or
>         a yet to be remembered good reason.

I've given you the reasons above.  I'd also point out that with your
clarification that you want .dev to be a toplevel alongside 'a', 'b', and
'c' you are also changing how dev is being used for no good reason -- simply
to fit your personal use case.

>     3. Requires you to decide ahead of time if you're going to go from dev
>         to alpha, or straight to a beta/candidate/final. (This is something
> that
>         most small projects don't have set in stone).

Really?

Today I release:
   1.0a1.dev2012

Tomorrow I release:
   1.0b1

The versions sort.  There was no forethought needed.  Now if I was to do
this, I'd use:
   1.0.post2012

as my first release instead.


>      4. It's semantics are different, this is a development release of
>          1.0a1 as opposed to a development release of the 1.0 version.

Correct.  And "this is a development release of 1.0a1" is what people that
designed the version specification wnated.  The "development release of the
1.0 version" case is taken care of by .post.

>      5. It's just plain ugly.
> 
I'd say the same thing about 'a', 'b', 'c' vs 'alpha', 'beta', 'rc'; .post
and .pre, etc.  Ugly seems to be the path to painting bikesheds.

> 
> So to be clear my proposal would be:
> 
>     1.0dev1 < 1.0a1 < 1.0b1 < 1.0c1 < 1.0.pre1 < 1.0 < 1.0.post1

So is your proposal to get rid of all modifiers?  ie: there will be
toplevels that sort at the same level as the alpha/beta/rc tags?  If so, I'd
get rid of the pre tag in there and rename dev to pre as dev means "post" to
some people and "pre" to others.  I'd also remove the "." since they're all
at the same level:

1.0pre1 < 1.0a1 < 1.0b1 < 1.0c1 < 1.0 < 1.0post1

While I'd be for this, this was definitely counter to the desires of a good
number of the other people who participated.  They definitely wanted
modifiers to denote that a snapshot dev instance applied before or after an
alpha/beta/rc.

If you do intend to keep .pre and .post as modifiers, I would be against
adding dev as a toplevel.  it's just adding another name with no clear
definition in the collective minds of the people consuming the versions
where the functionality is already provided for by the .post and .pre
(current .dev)

> as opposed to the current:
> 
>     1.0a1 < 1.0b1 < 1.0c1 < 1.0.dev1 < 1.0 < 1.0.post1


(Note: I assume rc1 is also in all of these examples and sorts between c and
"final")

-Toshio
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 198 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20120927/aeb5abe9/attachment.pgp>

From benji at benjiyork.com  Thu Sep 27 20:07:44 2012
From: benji at benjiyork.com (Benji York)
Date: Thu, 27 Sep 2012 18:07:44 +0000
Subject: [Distutils] Differences in PEP386 and setuptools
In-Reply-To: <CAFT4OTFnQuHYT-ujJSs7UkPi+E4CY2d6KZd+XJRObqtUK1wuFw@mail.gmail.com>
References: <AE15D652194A4C70BADC05166BAEF0DF@gmail.com>
	<20120926200028.GH2974@unaka.lan>
	<215BFDFF990F49E5A920022A9C8E1F47@gmail.com>
	<CAOTD34Yr7A4gCtTOuvK0WUcCruBEox7zFzdXEoCFsC+C7gn4Og@mail.gmail.com>
	<93ECE7DE9BE749399E79589762F936CE@gmail.com>
	<20120927155900.GI2974@unaka.lan> <506490F1.7080909@netwok.org>
	<CAG8k2+7xwEjyjiExEDHYaEsEh+C-3sQgMA-UEBQ1nEGb9oTvDg@mail.gmail.com>
	<CAFT4OTFnQuHYT-ujJSs7UkPi+E4CY2d6KZd+XJRObqtUK1wuFw@mail.gmail.com>
Message-ID: <CAC8tyQ7myAgMMRQnXvNr=t_VTeu0Mi2UAP8LJ2RRu5zxQqar3Q@mail.gmail.com>

On Thu, Sep 27, 2012 at 5:58 PM, Fred Drake <fdrake at acm.org> wrote:
> On Thu, Sep 27, 2012 at 1:50 PM, Daniel Holth <dholth at gmail.com> wrote:
>> Ruby gems use http://semver.org/
>
> I don't remember if I was involved in the past discussions (it's possible),
> but I'm in favor of using (and requiring) semantic versioning, if only
> because we don't have to invent notation or interpretation.

+1
-- 
Benji York

From dholth at gmail.com  Thu Sep 27 20:30:41 2012
From: dholth at gmail.com (Daniel Holth)
Date: Thu, 27 Sep 2012 14:30:41 -0400
Subject: [Distutils] Differences in PEP386 and setuptools
In-Reply-To: <CAC8tyQ7myAgMMRQnXvNr=t_VTeu0Mi2UAP8LJ2RRu5zxQqar3Q@mail.gmail.com>
References: <AE15D652194A4C70BADC05166BAEF0DF@gmail.com>
	<20120926200028.GH2974@unaka.lan>
	<215BFDFF990F49E5A920022A9C8E1F47@gmail.com>
	<CAOTD34Yr7A4gCtTOuvK0WUcCruBEox7zFzdXEoCFsC+C7gn4Og@mail.gmail.com>
	<93ECE7DE9BE749399E79589762F936CE@gmail.com>
	<20120927155900.GI2974@unaka.lan> <506490F1.7080909@netwok.org>
	<CAG8k2+7xwEjyjiExEDHYaEsEh+C-3sQgMA-UEBQ1nEGb9oTvDg@mail.gmail.com>
	<CAFT4OTFnQuHYT-ujJSs7UkPi+E4CY2d6KZd+XJRObqtUK1wuFw@mail.gmail.com>
	<CAC8tyQ7myAgMMRQnXvNr=t_VTeu0Mi2UAP8LJ2RRu5zxQqar3Q@mail.gmail.com>
Message-ID: <CAG8k2+46niaSRwGY889U+ry_wv1vNNXJ+w1Yxv0GcaXf4cGX+w@mail.gmail.com>

This is the worst bikeshed topic. I would like to keep PEP-386 as much
as possible, only because the time-consuming agreement has already
happened.

For wheel I had to add an alphanumeric build number apart from the
main version because you need to be able to distinguish e.g. a RHEL
build from a Debian build just by including a string like RedHat's 1fc
or 1el build numbers.

From a.badger at gmail.com  Thu Sep 27 20:33:26 2012
From: a.badger at gmail.com (Toshio Kuratomi)
Date: Thu, 27 Sep 2012 11:33:26 -0700
Subject: [Distutils] Differences in PEP386 and setuptools
In-Reply-To: <506490F1.7080909@netwok.org>
References: <AE15D652194A4C70BADC05166BAEF0DF@gmail.com>
	<20120926200028.GH2974@unaka.lan>
	<215BFDFF990F49E5A920022A9C8E1F47@gmail.com>
	<CAOTD34Yr7A4gCtTOuvK0WUcCruBEox7zFzdXEoCFsC+C7gn4Og@mail.gmail.com>
	<93ECE7DE9BE749399E79589762F936CE@gmail.com>
	<20120927155900.GI2974@unaka.lan> <506490F1.7080909@netwok.org>
Message-ID: <20120927183326.GM2974@unaka.lan>

On Thu, Sep 27, 2012 at 01:46:25PM -0400, ?ric Araujo wrote:
> Le 27/09/2012 11:59, Toshio Kuratomi a ?crit :
> > * a, b, c, and rc are the typical alpha beta release candidate tags.  We
> >   debated whether to allow the single letter versions or the long versions
> >   "alpha", "beta", "rc".  In the end, which set of tags was used was
> >   a bikeshed argument so it was chosen by PEP-author decree.  I personally
> >   felt that having both "c" and "rc" was counter to the goals of the PEP but
> >   we at least agreed that "rc" would intuitively sort after "c" so it wasn't
> >   as bad as having both "alpha" and "a" where there would be no intuitively
> >   correct sorting.
> 
> Well, c and rc should really compare equal in my opinion.
>
That would be a bad thing.  What do you do in the face of a project
releasing:

foo-1.0c1
foo-1.0rc1

Does your tool get to download either one of those depending on who coded
it, the timestamp for the upload, or input from the RNG?

And saying that the foo project maintainers shouldn't do that because
they're semantically the same is well ad good but in practice people do
things that are wrong all the time.  With a single version string inside of
a package we can provide functions that can validate whether the version is
correct or not as part of using the versions in the library to mitigate
that.  We cannot do the same thing with version strings from two separate
releases of the package because where those releases are stored is
site/project specific.

Documenting that even though c and rc are menat to be semantically the same
they should always sort "c" followed by "rc" protects you from these
problems.

-Toshio
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 198 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20120927/96f86145/attachment-0001.pgp>

From donald.stufft at gmail.com  Thu Sep 27 20:42:50 2012
From: donald.stufft at gmail.com (Donald Stufft)
Date: Thu, 27 Sep 2012 14:42:50 -0400
Subject: [Distutils] Differences in PEP386 and setuptools
In-Reply-To: <20120927180335.GL2974@unaka.lan>
References: <AE15D652194A4C70BADC05166BAEF0DF@gmail.com>
	<20120926200028.GH2974@unaka.lan>
	<215BFDFF990F49E5A920022A9C8E1F47@gmail.com>
	<CAOTD34Yr7A4gCtTOuvK0WUcCruBEox7zFzdXEoCFsC+C7gn4Og@mail.gmail.com>
	<93ECE7DE9BE749399E79589762F936CE@gmail.com>
	<20120927155900.GI2974@unaka.lan>
	<FAEA0E647131499088126D0269DD9A9E@gmail.com>
	<20120927180335.GL2974@unaka.lan>
Message-ID: <0371FC78A53B43FEA000E06F3F5C2BF1@gmail.com>



On Thursday, September 27, 2012 at 2:03 PM, Toshio Kuratomi wrote:

> That's your preference, not capability. I'd also say you should use
> 1.0.post1 rather than 1.0a1pre1.

So how do you encode the very first development release then if you're
using post? Can you point me towards somewhere where their conceptual
release cycle is

Final -> Dev -> Alpha -> Beta -> RC

or

Alpha -> Beta -> RC -> Final -> Dev

How can you release any sort of Alpha/Beta/RC/Final prior to having done
development? If Dev is a post release what do you call it before you've had
your first Alpha?
> 
> > On the surface 1.0a1.dev1 looks like it might be alright, but my issues with
> > it:
> > 
> > 1. It's not intuitive (at least not to me, development snapshots have
> > always come before alphas in my mind)
> > 
> 
> There's no such consensus. This is both from dealing with upstream
> versioning as a distro packager and from discussion with other people who
> worked on the versioning spec. People were very clear that they wanted to
> be able to stick dev versions in the middle of their attempt to release
> alphas, betas, rcs, and finals.
> 
> You can see this in current, setuptools-driven practice in the snapshots
> that pje hosts, for example:
> http://peak.telecommunity.com/snapshots/
> 
> 
> > 2. Goes against what is currently being used for either no good reason or
> > a yet to be remembered good reason.
> > 
> 
> 
> I've given you the reasons above. I'd also point out that with your
> clarification that you want .dev to be a toplevel alongside 'a', 'b', and
> 'c' you are also changing how dev is being used for no good reason -- simply
> to fit your personal use case.

I'm actually reverting it back to the current defacto standard of what setuptools does. 

However upon further examination I've figured out I believe what's caused the
mismatch between expectations here. Essentially pkg_resources does it both ways.
given the tags dev, a, b, c and an implicit f, it allows you to "nest" them.

So:

    0.1dev1 < 0.1a1 < 0.1b1 < 0.1c1 < 0.1

However it also allows you to add modifiers to these "top level" releases such as:

    0.1.dev < 0.1a1.dev1 < 0.1.a1 < 0.1.b1 < 0.1c1 < 0.1

This also makes sense to me in my particular use case, and it sounds like it
makes sense in yours? My main concern is that shifting 0.1dev1 from
sorting before alpha/beta/candidate to sorting after them is going to cause
headaches. So perhaps a better proposal would be to allow .dev at any
level (as it does currently), but dictate that .dev sorts first at whatever level
it occurs at (which is what setuptools does). This fixes my major problem of
getting bitten because setuptools and PEP386 treat "1.0dev1" differently while
still giving people the ability to do "1.0a1.dev1". The major drawback being
that you can't do a "dev/pre" release between an rc and a final, but as pkg_resources
doesn't currently allow for that I would think that isn't a huge concern? That might
solve the use case for both sides?
> 
> (Note: I assume rc1 is also in all of these examples and sorts between c and
> "final")

Yes essentially, I mean conceptually c and rc should sort the same but practicality
beats purity and what not.
> 
> -Toshio 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20120927/66efda41/attachment.html>

From a.badger at gmail.com  Thu Sep 27 21:45:29 2012
From: a.badger at gmail.com (Toshio Kuratomi)
Date: Thu, 27 Sep 2012 12:45:29 -0700
Subject: [Distutils] Differences in PEP386 and setuptools
In-Reply-To: <0371FC78A53B43FEA000E06F3F5C2BF1@gmail.com>
References: <AE15D652194A4C70BADC05166BAEF0DF@gmail.com>
	<20120926200028.GH2974@unaka.lan>
	<215BFDFF990F49E5A920022A9C8E1F47@gmail.com>
	<CAOTD34Yr7A4gCtTOuvK0WUcCruBEox7zFzdXEoCFsC+C7gn4Og@mail.gmail.com>
	<93ECE7DE9BE749399E79589762F936CE@gmail.com>
	<20120927155900.GI2974@unaka.lan>
	<FAEA0E647131499088126D0269DD9A9E@gmail.com>
	<20120927180335.GL2974@unaka.lan>
	<0371FC78A53B43FEA000E06F3F5C2BF1@gmail.com>
Message-ID: <20120927194529.GO2974@unaka.lan>

On Thu, Sep 27, 2012 at 02:42:50PM -0400, Donald Stufft wrote:
> 
> On Thursday, September 27, 2012 at 2:03 PM, Toshio Kuratomi wrote:
> 
>     That's your preference, not capability. I'd also say you should use
>     1.0.post1 rather than 1.0a1pre1.
> 
> So how do you encode the very first development release then if you're
> using post?

Took me a couple readings but I think you mean:

I start a project this morning.
I make a snapshot this afternoon.

What should the version be?

Using the current versioning scheme, I'd personally do:
0.dev1

By the time I get to an alpha release, we're going to have a version number
(for instance, 0.1).  So an initial version on the first day of 0 is fine:

0.dev1 < 0.1a1 < [...] < 0.1

/me has seen this in the wild as well... not just in my theoretical mind :-)

> Can you point me towards somewhere where their conceptual
> release cycle is
> 
> Final -> Dev -> Alpha -> Beta -> RC
> 
> or
> 
> Alpha -> Beta -> RC -> Final -> Dev
> 
I'm not sure what you're asking here.  All projects change code and release
it.  The changes may get released as something "official"-y like an alpha,
a beta, an rc, or a final.  They may also be released like a snapshot.
Those snapshots can be taken from any point on the timeline against any
release.  Trying to say that "snapshots only happen before alpha beta rc" or
"snapshots only happen after alpha beta rc" doesn't map to what people in
the real world are doing.

I think we both get to this same place down below, though, so I'll skip down
to there.


> This also makes sense to me in my particular use case, and it sounds like it
> makes sense in yours? My main concern is that shifting 0.1dev1 from
> sorting before alpha/beta/candidate to sorting after them is going to cause
> headaches. So perhaps a better proposal would be to allow .dev at any
> level (as it does currently), but dictate that .dev sorts first at whatever
> level
> it occurs at (which is what setuptools does). This fixes my major problem of
> getting bitten because setuptools and PEP386 treat "1.0dev1" differently while
> still giving people the ability to do "1.0a1.dev1". The major drawback being
> that you can't do a "dev/pre" release between an rc and a final, but as
> pkg_resources
> doesn't currently allow for that I would think that isn't a huge concern? That
> might
> solve the use case for both sides?
> 
If we're just changing the position of X.Y.preN from this:

0.1 < 0.1.post1 < 0.2a1.pre1 < 0.2a1 < 0.2a1.post1 < 0.2b1 < 0.2pre1 < 0.2

to this:

0.1 < 0.1.post1 < 0.2pre1 < 0.2a1.pre1 < 0.2a1 < 0.2a1.post1 < 0.2b1 < 0.2

then that works perfectly fine for me.  To put out a snapshot between rc and
final, I'd do 0.2rc1.post1 so not having a pre/dev in there doesn't bother
me.

However, now that you mention it, ISTR that this is the way that it was
originally and the lack of pre/dev in that position did bother another (set
of?) the PEP authors enough that it was changed.  For me, switching the
search order of that particular piece is a bikeshed so I seem to have
reallocated the memory for storing the arguments made in favor of shifting
it to the current position :-)

-Toshio
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 198 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20120927/e42811d2/attachment.pgp>

From donald.stufft at gmail.com  Thu Sep 27 22:02:33 2012
From: donald.stufft at gmail.com (Donald Stufft)
Date: Thu, 27 Sep 2012 16:02:33 -0400
Subject: [Distutils] Differences in PEP386 and setuptools
In-Reply-To: <20120927194529.GO2974@unaka.lan>
References: <AE15D652194A4C70BADC05166BAEF0DF@gmail.com>
	<20120926200028.GH2974@unaka.lan>
	<215BFDFF990F49E5A920022A9C8E1F47@gmail.com>
	<CAOTD34Yr7A4gCtTOuvK0WUcCruBEox7zFzdXEoCFsC+C7gn4Og@mail.gmail.com>
	<93ECE7DE9BE749399E79589762F936CE@gmail.com>
	<20120927155900.GI2974@unaka.lan>
	<FAEA0E647131499088126D0269DD9A9E@gmail.com>
	<20120927180335.GL2974@unaka.lan>
	<0371FC78A53B43FEA000E06F3F5C2BF1@gmail.com>
	<20120927194529.GO2974@unaka.lan>
Message-ID: <D626A5AFC34B42708B92FCDBB0AB8842@gmail.com>

On Thursday, September 27, 2012 at 3:45 PM, Toshio Kuratomi wrote:
> then that works perfectly fine for me. To put out a snapshot between rc and
> final, I'd do 0.2rc1.post1 so not having a pre/dev in there doesn't bother
> me.
> 
> However, now that you mention it, ISTR that this is the way that it was
> originally and the lack of pre/dev in that position did bother another (set
> of?) the PEP authors enough that it was changed. For me, switching the
> search order of that particular piece is a bikeshed so I seem to have
> reallocated the memory for storing the arguments made in favor of shifting
> it to the current position :-)

So I guess the question is does losing the ability to not have a dev that
is sorted between the rc and the final outweigh the legacy concerns of
dealing with the number of packages in the wild (which is hard to count
since due to other issues very few non final releases make it to PyPI) that
expect 0.1dev1 to sort before an alpha.

Personally I would want to fall on the side of not breaking the current assumptions
since given a particular version string we do not have a good way of determining
if it's expecting dev to fall before alpha or expecting it to fall after the rc.

I should note that I originally stumbled upon this issue working on trying to convert
existing tools to use PEP386 and warn when a version wasn't PEP386 compat and
the difference between the way PEP386 handles the ordering and the way pkg_resources
handled the differences caused a problem with a different version getting installed before
and after my changes that caused me to dig into why.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20120927/b013fe91/attachment-0001.html>

From jim at zope.com  Thu Sep 27 23:12:05 2012
From: jim at zope.com (Jim Fulton)
Date: Thu, 27 Sep 2012 17:12:05 -0400
Subject: [Distutils] buildout.org A record
In-Reply-To: <k40e7j$fhj$1@ger.gmane.org>
References: <CANoAZMCh4uvnvOXKo2ZEmUkNjTCfpupfsp3_A3vhky9=sQCLPQ@mail.gmail.com>
	<CAPDm-Fhd4WRNugMuZY7mDuYqdymHPg3uxTgh20t+vWgem2SGmw@mail.gmail.com>
	<CANoAZMB0W0b0AFtVzDO85a5eqy6aQGWNrUKaUk7ma6et7S--JQ@mail.gmail.com>
	<CAPDm-FgdfTPKbkOC-EB=+KWyX9KvPprSYdQdTYzjOSAnKfGtag@mail.gmail.com>
	<k40e7j$fhj$1@ger.gmane.org>
Message-ID: <CAPDm-FjKj5ZvwurCVpeAfMXXyp6tv63Qu1e1+_Wvu+Urjhuymw@mail.gmail.com>

On Wed, Sep 26, 2012 at 10:38 PM, Alex Clark <aclark at aclark.net> wrote:
> On 2012-09-26 15:35:00 +0000, Jim Fulton said:
>
>> On Wed, Sep 26, 2012 at 11:03 AM, Conrado Buhrer <conrado at buhrer.net>
>> wrote:
>>>
>>> Hi Jim,
>>>
>>> A few of us on #buildout at freenode have noticed the issue, we believe
>>> that some links on pypi are the cause.
>>>
>>> See lines 84-89 of the source of
>>> http://pypi.python.org/simple/zc.buildout/
>>
>>
>> Fixed. I removed the home-page meta data from those releases.
>> I don't know what's going on with buildout.org. :(
>
>
>
> If anyone does know, please change the DNS A record to 207.97.227.245 so we
> can host the site on github pages.

I'm all for that (where A record to 207... is a euphemism for CNAME record to
pages.github.com) once something's set up there. :)

In the mean time, I'd just like to get DNS under control. :(

Jim

-- 
Jim Fulton
http://www.linkedin.com/in/jimfulton
Jerky is better than bacon! http://zo.pe/Kqm

From bradallen137 at gmail.com  Fri Sep 28 00:00:56 2012
From: bradallen137 at gmail.com (Brad Allen)
Date: Thu, 27 Sep 2012 17:00:56 -0500
Subject: [Distutils] Differences in PEP386 and setuptools
In-Reply-To: <20120927194529.GO2974@unaka.lan>
References: <AE15D652194A4C70BADC05166BAEF0DF@gmail.com>
	<20120926200028.GH2974@unaka.lan>
	<215BFDFF990F49E5A920022A9C8E1F47@gmail.com>
	<CAOTD34Yr7A4gCtTOuvK0WUcCruBEox7zFzdXEoCFsC+C7gn4Og@mail.gmail.com>
	<93ECE7DE9BE749399E79589762F936CE@gmail.com>
	<20120927155900.GI2974@unaka.lan>
	<FAEA0E647131499088126D0269DD9A9E@gmail.com>
	<20120927180335.GL2974@unaka.lan>
	<0371FC78A53B43FEA000E06F3F5C2BF1@gmail.com>
	<20120927194529.GO2974@unaka.lan>
Message-ID: <CAA_9swrAdirrR5YYo5dFn5ch83WTzf-UJJ3_y5MA6kSeDMxOnQ@mail.gmail.com>

On Thu, Sep 27, 2012 at 2:45 PM, Toshio Kuratomi <a.badger at gmail.com> wrote:

> However, now that you mention it, ISTR that this is the way that it was
> originally and the lack of pre/dev in that position did bother another (set
> of?) the PEP authors enough that it was changed.  For me, switching the
> search order of that particular piece is a bikeshed so I seem to have
> reallocated the memory for storing the arguments made in favor of shifting
> it to the current position :-)

Why is this perceived as a bikeshed issue? It seems like a discussion
about substantial functionality which is tied to the semantics of these
special tags. For many projects and organizations, terms like 'alpha'
and 'beta' and 'dev' have meaning in a release context, and rearranging
the order of these concepts has an impact.

> So to be clear my proposal would be:
>
>     1.0dev1 < 1.0a1 < 1.0b1 < 1.0c1 < 1.0.pre1 < 1.0 < 1.0.post1
>
> as opposed to the current:
>
>     1.0a1 < 1.0b1 < 1.0c1 < 1.0.dev1 < 1.0 < 1.0.post1
>

+1 for the proposal to change the PEP so that 'dev' versions are
earlier than 'a' versions.
I'm not a contributor to any packaging projects so this is really just a vote
from an appreciative user of setuptools and distribute.

From solipsis at pitrou.net  Fri Sep 28 00:55:48 2012
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Thu, 27 Sep 2012 22:55:48 +0000 (UTC)
Subject: [Distutils] distlib updated with resources API
References: <loom.20120923T174554-748@post.gmane.org>
	<50636845.8080204@ziade.org>
	<loom.20120926T233609-992@post.gmane.org>
	<50645C34.6070606@ziade.org>
	<1348756086.3495.1.camel@localhost.localdomain>
	<50646CFF.3010601@ziade.org>
	<loom.20120927T173348-617@post.gmane.org>
	<50647599.3030507@ziade.org>
Message-ID: <loom.20120928T004648-800@post.gmane.org>

Tarek Ziad? <tarek <at> ziade.org> writes:
> 
> I've also said that I believed that it's simpler to include back wrt 
> licensing if it's a code base
> that's under the contributors agreement.

What does that have to do with hg.p.o? You can ask contributor agreements if
your code is hosted on bitbucket. There's no magical relationship between hg.p.o
and contributor agreements.

> Last but not least, distlib is the plan forward endorsed by python-dev, 

Is it? I haven't seen a PEP or an official decision about that. Just because
someone proposed it on a mailing-list doesn't mean it is "endorsed by
python-dev".

By the way, you can already create repository clones from the Web interface.
They just won't have any fancy configuration (hooks).

> This is really annoying  :/

Why is it so? Why does being on hg.p.o have anything to do with you being able
to work on distlib? As Georg said, this is not a professional service that
happens to be offered by a company. This is run by volunteers, and I doubt any
of us enjoys sysadmin-related tasks.

Regards

Antoine.



From a.badger at gmail.com  Fri Sep 28 01:15:17 2012
From: a.badger at gmail.com (Toshio Kuratomi)
Date: Thu, 27 Sep 2012 16:15:17 -0700
Subject: [Distutils] Differences in PEP386 and setuptools
In-Reply-To: <CAA_9swrAdirrR5YYo5dFn5ch83WTzf-UJJ3_y5MA6kSeDMxOnQ@mail.gmail.com>
References: <20120926200028.GH2974@unaka.lan>
	<215BFDFF990F49E5A920022A9C8E1F47@gmail.com>
	<CAOTD34Yr7A4gCtTOuvK0WUcCruBEox7zFzdXEoCFsC+C7gn4Og@mail.gmail.com>
	<93ECE7DE9BE749399E79589762F936CE@gmail.com>
	<20120927155900.GI2974@unaka.lan>
	<FAEA0E647131499088126D0269DD9A9E@gmail.com>
	<20120927180335.GL2974@unaka.lan>
	<0371FC78A53B43FEA000E06F3F5C2BF1@gmail.com>
	<20120927194529.GO2974@unaka.lan>
	<CAA_9swrAdirrR5YYo5dFn5ch83WTzf-UJJ3_y5MA6kSeDMxOnQ@mail.gmail.com>
Message-ID: <20120927231516.GP2974@unaka.lan>

On Thu, Sep 27, 2012 at 05:00:56PM -0500, Brad Allen wrote:
> On Thu, Sep 27, 2012 at 2:45 PM, Toshio Kuratomi <a.badger at gmail.com> wrote:
> 
> > However, now that you mention it, ISTR that this is the way that it was
> > originally and the lack of pre/dev in that position did bother another (set
> > of?) the PEP authors enough that it was changed.  For me, switching the
> > search order of that particular piece is a bikeshed so I seem to have
> > reallocated the memory for storing the arguments made in favor of shifting
> > it to the current position :-)
> 
> Why is this perceived as a bikeshed issue? It seems like a discussion
> about substantial functionality which is tied to the semantics of these
> special tags. For many projects and organizations, terms like 'alpha'
> and 'beta' and 'dev' have meaning in a release context, and rearranging
> the order of these concepts has an impact.
> 
dev has no universal defined relation to alpha, beta, etc in a release
context.

We all seem to be amenable to:

1.0alpha1.dev1  < 1.0alpha1

But people have not agreed whether:

1.0.dev1 < 1.0alpha1
or
1.0alpha1 < 1.0.dev1

Either of these cases can also be functionally replaced with the appropriate
.post tag:

1.0.dev1 < 1.0alpha1 ===>  0.9.post1 < 1.0alpha1

1.0alpha1 < 1.0.dev1 ===>  1.0alpha1 < 1.0rc1.post1

(Substitute the last "final" release for 0.9 in the above and substitute
whatever version you're snapshoting after for 1.0rc1)

Some organizations and projects use it one way and others the other way.
And unlike alpha, beta, rc ordering which is both long established in the
computing industry and based on the sorting of the greek alphabet, the
choice with .dev is arbitrary.  That's what makes this a bikeshed.  Either
way of doing this is going to confuse and upset some of the consumers and
there's no solid reason to say that the upset people are "wrong".  At best
you can say that they're late to the party.

> > So to be clear my proposal would be:
> >
> >     1.0dev1 < 1.0a1 < 1.0b1 < 1.0c1 < 1.0.pre1 < 1.0 < 1.0.post1
> >
> > as opposed to the current:
> >
> >     1.0a1 < 1.0b1 < 1.0c1 < 1.0.dev1 < 1.0 < 1.0.post1
> >
> 
> +1 for the proposal to change the PEP so that 'dev' versions are
> earlier than 'a' versions.
>
Note, just in case my chart is misleading, the proposal doesn't change all
'dev' versions to be earlier than the 'a' versions.  '.dev' is a modifier.
So it can be applied to any of the toplevel portions of the version.

So to be more complete, the proposal is to do this::

0.1
  < 0.1.post1
  < 1.0.dev1        # The .dev that's changing position
  < 1.0a1.dev1
  < 1.0a1
  < 1.0a1.post1
  < 1.0b1.dev1
  < 1.0b1
  < 1.0b1.post1
  < 1.0c1.dev1
  < 1.0c1
  < 1.0c1.post1
                    # The current PEP386 position of 1.0.dev1
  < 1.0
  < 1.0.post1

-Toshio
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 198 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20120927/4a950297/attachment.pgp>

From donald.stufft at gmail.com  Fri Sep 28 01:26:26 2012
From: donald.stufft at gmail.com (Donald Stufft)
Date: Thu, 27 Sep 2012 19:26:26 -0400
Subject: [Distutils] Differences in PEP386 and setuptools
In-Reply-To: <20120927231516.GP2974@unaka.lan>
References: <20120926200028.GH2974@unaka.lan>
	<215BFDFF990F49E5A920022A9C8E1F47@gmail.com>
	<CAOTD34Yr7A4gCtTOuvK0WUcCruBEox7zFzdXEoCFsC+C7gn4Og@mail.gmail.com>
	<93ECE7DE9BE749399E79589762F936CE@gmail.com>
	<20120927155900.GI2974@unaka.lan>
	<FAEA0E647131499088126D0269DD9A9E@gmail.com>
	<20120927180335.GL2974@unaka.lan>
	<0371FC78A53B43FEA000E06F3F5C2BF1@gmail.com>
	<20120927194529.GO2974@unaka.lan>
	<CAA_9swrAdirrR5YYo5dFn5ch83WTzf-UJJ3_y5MA6kSeDMxOnQ@mail.gmail.com>
	<20120927231516.GP2974@unaka.lan>
Message-ID: <2AE51853A9C74B6BB70224FB40BAD3CD@gmail.com>

On Thursday, September 27, 2012 at 7:15 PM, Toshio Kuratomi wrote:
> Some organizations and projects use it one way and others the other way.
> And unlike alpha, beta, rc ordering which is both long established in the
> computing industry and based on the sorting of the greek alphabet, the
> choice with .dev is arbitrary. That's what makes this a bikeshed. Either
> way of doing this is going to confuse and upset some of the consumers and
> there's no solid reason to say that the upset people are "wrong". At best
> you can say that they're late to the party.

My biggest reason is that it changes what 0.1dev1 means based on if you
use setuptools or pep386 and setuptools is the current defacto standard and by
breaking that you break expectations that already exist in the wild. The way pep386
does it is different then the way things are now. If we fast forward into a world
with both a pep386 compatible packaging toolbelt and setuptools we are going
to end up with two different definitions of where a top level devN sorts.

>From PEP386 we have distutils's StrictVersion, LooseVersion and pkg_resources's
parse_version.

Of the 3 parse_version is the only one that handles this sanely at all, (StrictVersion
it errors out on, LooseVersion sorts alphabetically, so a < b < c < dev < rc).

Moving forward with PEP386 as is will break existing assumptions (I've already encountered
this with my local testing) and leaves us no way to tell if the author meant
to order things ala PEP386, or ala pkg_resources. Moving back to the behavior
by pkg_resources in this area leaves us with the "bikeshed" color that people
already expect and will not leave us in a state where a valid set of versions in both
systems parse to two different orders which keeps the status quo.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20120927/470da8d6/attachment-0001.html>

From tarek at ziade.org  Fri Sep 28 01:37:01 2012
From: tarek at ziade.org (=?UTF-8?B?VGFyZWsgWmlhZMOp?=)
Date: Fri, 28 Sep 2012 01:37:01 +0200
Subject: [Distutils] distlib updated with resources API
In-Reply-To: <loom.20120928T004648-800@post.gmane.org>
References: <loom.20120923T174554-748@post.gmane.org>
	<50636845.8080204@ziade.org>
	<loom.20120926T233609-992@post.gmane.org>
	<50645C34.6070606@ziade.org>
	<1348756086.3495.1.camel@localhost.localdomain>
	<50646CFF.3010601@ziade.org>
	<loom.20120927T173348-617@post.gmane.org>
	<50647599.3030507@ziade.org>
	<loom.20120928T004648-800@post.gmane.org>
Message-ID: <5064E31D.80601@ziade.org>

On 9/28/12 12:55 AM, Antoine Pitrou wrote:
>> Last but not least, distlib is the plan forward endorsed by python-dev,
> Is it? I haven't seen a PEP or an official decision about that. Just because
> someone proposed it on a mailing-list doesn't mean it is "endorsed by
> python-dev".

We discussed about this with Vinay, Nick and al on python-dev, based on 
Nick's document
that describes what 'distlib' is.

The document has changed since then, 
http://python-notes.boredomandlaziness.org/en/latest/pep_ideas/core_packaging_api.html

But the idea was to create a subset of 4 or 5 modules that implement the 
various PEPs.

Vinay started to work on this and made progress.

When I said "endorsed", I mean that most of the people in python-dev 
that care about packaging agreed or did not disagree.

Now, if you disagree please say it. Or if you need an official decision, 
we need to first declare who is the current packaging BDFL maybe ?

And since you seem interested in the topic maybe you could take that role ?

Cheers
Tarek


From solipsis at pitrou.net  Fri Sep 28 01:44:20 2012
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Thu, 27 Sep 2012 23:44:20 +0000 (UTC)
Subject: [Distutils] distlib updated with resources API
References: <loom.20120923T174554-748@post.gmane.org>
	<50636845.8080204@ziade.org>
	<loom.20120926T233609-992@post.gmane.org>
	<50645C34.6070606@ziade.org>
	<1348756086.3495.1.camel@localhost.localdomain>
	<50646CFF.3010601@ziade.org>
	<loom.20120927T173348-617@post.gmane.org>
	<50647599.3030507@ziade.org>
	<loom.20120928T004648-800@post.gmane.org>
	<5064E31D.80601@ziade.org>
Message-ID: <loom.20120928T013957-911@post.gmane.org>

Tarek Ziad? <tarek <at> ziade.org> writes:
> 
> On 9/28/12 12:55 AM, Antoine Pitrou wrote:
> >> Last but not least, distlib is the plan forward endorsed by python-dev,
> > Is it? I haven't seen a PEP or an official decision about that. Just because
> > someone proposed it on a mailing-list doesn't mean it is "endorsed by
> > python-dev".
> 
> We discussed about this with Vinay, Nick and al on python-dev, based on 
> Nick's document
> that describes what 'distlib' is.
> 
> The document has changed since then, 
> http://python-notes.boredomandlaziness.org/en/latest/pep_ideas
> /core_packaging_api.html

Yep, so it's still a draft, even though it may be promising.

> Now, if you disagree please say it. Or if you need an official decision, 
> we need to first declare who is the current packaging BDFL maybe ?
> 
> And since you seem interested in the topic maybe you could take that role ?

I have no problem with distlib on the principle. However, if I were the 
packaging BDFL, my decision would be "integrate it all in distutils" :-)

By the way, if you want to help with hg.python.org and manage the distlib repo 
there, you can send an email on
http://mail.python.org/mailman/listinfo/infrastructure and ask for ssh access 
to the virtual machine.

Regards

Antoine.



From chrism at plope.com  Fri Sep 28 02:03:24 2012
From: chrism at plope.com (Chris McDonough)
Date: Thu, 27 Sep 2012 20:03:24 -0400
Subject: [Distutils] Differences in PEP386 and setuptools
In-Reply-To: <20120927231516.GP2974@unaka.lan>
References: <20120926200028.GH2974@unaka.lan>
	<215BFDFF990F49E5A920022A9C8E1F47@gmail.com>
	<CAOTD34Yr7A4gCtTOuvK0WUcCruBEox7zFzdXEoCFsC+C7gn4Og@mail.gmail.com>
	<93ECE7DE9BE749399E79589762F936CE@gmail.com>
	<20120927155900.GI2974@unaka.lan>
	<FAEA0E647131499088126D0269DD9A9E@gmail.com>
	<20120927180335.GL2974@unaka.lan>
	<0371FC78A53B43FEA000E06F3F5C2BF1@gmail.com>
	<20120927194529.GO2974@unaka.lan>
	<CAA_9swrAdirrR5YYo5dFn5ch83WTzf-UJJ3_y5MA6kSeDMxOnQ@mail.gmail.com>
	<20120927231516.GP2974@unaka.lan>
Message-ID: <1348790604.2967.18.camel@thinko>

On Thu, 2012-09-27 at 16:15 -0700, Toshio Kuratomi wrote:
> On Thu, Sep 27, 2012 at 05:00:56PM -0500, Brad Allen wrote:
> > On Thu, Sep 27, 2012 at 2:45 PM, Toshio Kuratomi <a.badger at gmail.com> wrote:
> > 
> > > However, now that you mention it, ISTR that this is the way that it was
> > > originally and the lack of pre/dev in that position did bother another (set
> > > of?) the PEP authors enough that it was changed.  For me, switching the
> > > search order of that particular piece is a bikeshed so I seem to have
> > > reallocated the memory for storing the arguments made in favor of shifting
> > > it to the current position :-)
> > 
> > Why is this perceived as a bikeshed issue? It seems like a discussion
> > about substantial functionality which is tied to the semantics of these
> > special tags. For many projects and organizations, terms like 'alpha'
> > and 'beta' and 'dev' have meaning in a release context, and rearranging
> > the order of these concepts has an impact.
> > 
> dev has no universal defined relation to alpha, beta, etc in a release
> context.

The tools that people use to install a Python package sort it before
alpha and have for many years.  That seems to be a pretty good argument
in favor, all other things considered.

- C



From ncoghlan at gmail.com  Fri Sep 28 04:25:37 2012
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Fri, 28 Sep 2012 12:25:37 +1000
Subject: [Distutils] [Python-Dev]  distlib updated with resources API
In-Reply-To: <5064E31D.80601@ziade.org>
References: <loom.20120923T174554-748@post.gmane.org>
	<50636845.8080204@ziade.org>
	<loom.20120926T233609-992@post.gmane.org>
	<50645C34.6070606@ziade.org>
	<1348756086.3495.1.camel@localhost.localdomain>
	<50646CFF.3010601@ziade.org>
	<loom.20120927T173348-617@post.gmane.org>
	<50647599.3030507@ziade.org>
	<loom.20120928T004648-800@post.gmane.org>
	<5064E31D.80601@ziade.org>
Message-ID: <CADiSq7et7kT7NKSjNJ-D3oWAWcbNNF43=nqaj=VrCkThJoyYXA@mail.gmail.com>

On Fri, Sep 28, 2012 at 9:37 AM, Tarek Ziad? <tarek at ziade.org> wrote:
> On 9/28/12 12:55 AM, Antoine Pitrou wrote:
>>>
>>> Last but not least, distlib is the plan forward endorsed by python-dev,
>>
>> Is it? I haven't seen a PEP or an official decision about that. Just
>> because
>> someone proposed it on a mailing-list doesn't mean it is "endorsed by
>> python-dev".
>
>
> We discussed about this with Vinay, Nick and al on python-dev, based on
> Nick's document
> that describes what 'distlib' is.
>
> The document has changed since then,
> http://python-notes.boredomandlaziness.org/en/latest/pep_ideas/core_packaging_api.html

Yeah, don't read too much into the current state of that - it will
eventually become a proposal for a standardised *in-memory* data
structure to better support metadata interoperability between
packaging tools, but it isn't there yet (although scrubbing every
reference to "JSON file" and replacing it with "API data structure"
would get you close - think of the overall idea as "like dictConfig,
but for distribution metadata rather than logging configurations". We
need something like that in order to allow import hooks to correctly
supply distribution metadata).

The original email thread from the removal of packaging from 3.3 is
probably a better point of reference, with a concrete "distlib" PEP
still on the todo list.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia

From donald.stufft at gmail.com  Fri Sep 28 04:46:26 2012
From: donald.stufft at gmail.com (Donald Stufft)
Date: Thu, 27 Sep 2012 22:46:26 -0400
Subject: [Distutils] Differences in PEP386 and setuptools
In-Reply-To: <1348790604.2967.18.camel@thinko>
References: <20120926200028.GH2974@unaka.lan>
	<215BFDFF990F49E5A920022A9C8E1F47@gmail.com>
	<CAOTD34Yr7A4gCtTOuvK0WUcCruBEox7zFzdXEoCFsC+C7gn4Og@mail.gmail.com>
	<93ECE7DE9BE749399E79589762F936CE@gmail.com>
	<20120927155900.GI2974@unaka.lan>
	<FAEA0E647131499088126D0269DD9A9E@gmail.com>
	<20120927180335.GL2974@unaka.lan>
	<0371FC78A53B43FEA000E06F3F5C2BF1@gmail.com>
	<20120927194529.GO2974@unaka.lan>
	<CAA_9swrAdirrR5YYo5dFn5ch83WTzf-UJJ3_y5MA6kSeDMxOnQ@mail.gmail.com>
	<20120927231516.GP2974@unaka.lan> <1348790604.2967.18.camel@thinko>
Message-ID: <AFD8FF3FF8B54AB493B63A91FD7F34AE@gmail.com>

On Thursday, September 27, 2012 at 8:03 PM, Chris McDonough wrote:
> The tools that people use to install a Python package sort it before
> alpha and have for many years. That seems to be a pretty good argument
> in favor, all other things considered.
> 
Yes this is what I've been trying to say, I agree that in a clean room implementation
the difference here would be mostly bikeshedding, however since it is a bikeshed
then we should endeavor to keep the existing bikeshed instead of switching. 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20120927/7294a72a/attachment.html>

From dholth at gmail.com  Fri Sep 28 06:37:43 2012
From: dholth at gmail.com (Daniel Holth)
Date: Fri, 28 Sep 2012 00:37:43 -0400
Subject: [Distutils] [Python-Dev] distlib updated with resources API
In-Reply-To: <CADiSq7et7kT7NKSjNJ-D3oWAWcbNNF43=nqaj=VrCkThJoyYXA@mail.gmail.com>
References: <loom.20120923T174554-748@post.gmane.org>
	<50636845.8080204@ziade.org>
	<loom.20120926T233609-992@post.gmane.org>
	<50645C34.6070606@ziade.org>
	<1348756086.3495.1.camel@localhost.localdomain>
	<50646CFF.3010601@ziade.org>
	<loom.20120927T173348-617@post.gmane.org>
	<50647599.3030507@ziade.org>
	<loom.20120928T004648-800@post.gmane.org>
	<5064E31D.80601@ziade.org>
	<CADiSq7et7kT7NKSjNJ-D3oWAWcbNNF43=nqaj=VrCkThJoyYXA@mail.gmail.com>
Message-ID: <CAG8k2+54-cyWCg81FF1epjH+4Gb9c9ZQhfV4WgSdPdRYuiM1Kw@mail.gmail.com>

The wheel strategy has just been to implement the peps in setuptools
and in pkg_resources. It is nearly effortless to do so since the PEPs
are so similar to the existing system of eggs. Plus you get to play
with it using 900 * 1.6 ** (year-2005) packages.

Afterwards, with binary packages, you can deprecate the distutils
build system. You still have to build package with distutils forever,
but it doesn't matter as much because you don't have to do it on your
production machine.

When a particular package has trouble with distutils, you tell them to
choose among a healthy ecosystem of superior build systems rather than
trying to add features to distutils.

Are we trying to kill setuptools? I'm not entirely sure, but we should
stop trying to do that. The migration should take essentially forever
as soon as it makes sense for each pypi publisher.

From g.brandl at gmx.net  Fri Sep 28 08:37:45 2012
From: g.brandl at gmx.net (Georg Brandl)
Date: Fri, 28 Sep 2012 08:37:45 +0200
Subject: [Distutils] distlib updated with resources API
In-Reply-To: <loom.20120927T184442-444@post.gmane.org>
References: <loom.20120923T174554-748@post.gmane.org>
	<50636845.8080204@ziade.org>
	<loom.20120926T233609-992@post.gmane.org>
	<50645C34.6070606@ziade.org>
	<1348756086.3495.1.camel@localhost.localdomain>
	<50646CFF.3010601@ziade.org>
	<loom.20120927T173348-617@post.gmane.org>
	<50647599.3030507@ziade.org> <k41vce$89g$1@ger.gmane.org>
	<loom.20120927T184442-444@post.gmane.org>
Message-ID: <k43gjn$37l$1@ger.gmane.org>

On 09/27/2012 06:48 PM, Vinay Sajip wrote:
> Georg Brandl <g.brandl <at> gmx.net> writes:
>
>>
>> No, but I can see why you want it there, and I for one have no problems
>> creating and maintaining that repository.
>>
>
> Great. Thanks!
>
>>
>> And we'd like to keep it that way.
>>
>
> :-)
>
>>
>> Are you sure you're not overreacting?
>>
>> All that Antoine is saying is that hg.p.o is not a professional hosting
>> service, and the more repositories we have, the more (potential) requests
>> we may get that have to be processed.  If you can wait a few days for a
>> new hook to be installed -- then everything is fine.
>>
>
> I assume you'll just clone the distlib repo on BitBucket. I'm happy to wait.

I've cloned the bitbucket.org/vinay.sajip/distlib repo to hg.python.org/distlib.
At the moment no hooks are set up; let me know if you want e.g. email
notification, CIA or roundup integration.

cheers,
Georg


From ct at gocept.com  Fri Sep 28 08:47:08 2012
From: ct at gocept.com (Christian Theune)
Date: Fri, 28 Sep 2012 08:47:08 +0200
Subject: [Distutils] buildout.org A record
References: <CANoAZMCh4uvnvOXKo2ZEmUkNjTCfpupfsp3_A3vhky9=sQCLPQ@mail.gmail.com>
	<CAPDm-Fhd4WRNugMuZY7mDuYqdymHPg3uxTgh20t+vWgem2SGmw@mail.gmail.com>
	<CANoAZMB0W0b0AFtVzDO85a5eqy6aQGWNrUKaUk7ma6et7S--JQ@mail.gmail.com>
	<CAPDm-FgdfTPKbkOC-EB=+KWyX9KvPprSYdQdTYzjOSAnKfGtag@mail.gmail.com>
	<k40e7j$fhj$1@ger.gmane.org>
	<CAPDm-FjKj5ZvwurCVpeAfMXXyp6tv63Qu1e1+_Wvu+Urjhuymw@mail.gmail.com>
Message-ID: <k43h56$78c$1@ger.gmane.org>

Hi,

On 2012-09-27 21:12:05 +0000, Jim Fulton said:

> On Wed, Sep 26, 2012 at 10:38 PM, Alex Clark <aclark at aclark.net> wrote:
>> On 2012-09-26 15:35:00 +0000, Jim Fulton said:
>> 
>>> On Wed, Sep 26, 2012 at 11:03 AM, Conrado Buhrer <conrado at buhrer.net>
>>> wrote:
>>>> 
>>>> Hi Jim,
>>>> 
>>>> A few of us on #buildout at freenode have noticed the issue, we believe
>>>> that some links on pypi are the cause.
>>>> 
>>>> See lines 84-89 of the source of
>>>> http://pypi.python.org/simple/zc.buildout/
>>> 
>>> 
>>> Fixed. I removed the home-page meta data from those releases.
>>> I don't know what's going on with buildout.org. :(
>> 
>> 
>> 
>> If anyone does know, please change the DNS A record to 207.97.227.245 so we
>> can host the site on github pages.
> 
> I'm all for that (where A record to 207... is a euphemism for CNAME record to
> pages.github.com) once something's set up there. :)
> 
> In the mean time, I'd just like to get DNS under control. :(

OK - DNS is under control now. We took it back to the DNS servers at 
gocept to get the status quo back to a working state. We're happy to 
make any changes that are deemed necessary. Looking at the fact that we 
get contacted because the WHOIS points to us I'd prefer to also have 
authority over DNS - otherwise we just get flooded with requests that 
we can't do anything about.

So: what's next?

Christian



From vinay_sajip at yahoo.co.uk  Fri Sep 28 10:17:57 2012
From: vinay_sajip at yahoo.co.uk (Vinay Sajip)
Date: Fri, 28 Sep 2012 08:17:57 +0000 (UTC)
Subject: [Distutils] distlib updated with resources API
References: <loom.20120923T174554-748@post.gmane.org>
	<50636845.8080204@ziade.org>
	<loom.20120926T233609-992@post.gmane.org>
	<50645C34.6070606@ziade.org>
	<1348756086.3495.1.camel@localhost.localdomain>
	<50646CFF.3010601@ziade.org>
	<loom.20120927T173348-617@post.gmane.org>
	<50647599.3030507@ziade.org> <k41vce$89g$1@ger.gmane.org>
	<loom.20120927T184442-444@post.gmane.org>
	<k43gjn$37l$1@ger.gmane.org>
Message-ID: <loom.20120928T100621-122@post.gmane.org>

Georg Brandl <g.brandl <at> gmx.net> writes:

> I've cloned the bitbucket.org/vinay.sajip/distlib repo to
> hg.python.org/distlib.

Thank you very much.

> At the moment no hooks are set up; let me know if you want e.g. email
> notification, CIA or roundup integration.

Are the coding style (whitespace) hooks set up? They would be useful to have.

Re. the integration with CI, roundup and email, I'm not sure exactly which
features are easy to provide - how much work would they mean for you? I don't
see any urgent need to implement these right away - perhaps we can add these
when some more work has been done, and there are more contributors who can
benefit.

Regards,

Vinay Sajip


From g.brandl at gmx.net  Fri Sep 28 10:29:13 2012
From: g.brandl at gmx.net (Georg Brandl)
Date: Fri, 28 Sep 2012 10:29:13 +0200
Subject: [Distutils] distlib updated with resources API
In-Reply-To: <loom.20120928T100621-122@post.gmane.org>
References: <loom.20120923T174554-748@post.gmane.org>
	<50636845.8080204@ziade.org>
	<loom.20120926T233609-992@post.gmane.org>
	<50645C34.6070606@ziade.org>
	<1348756086.3495.1.camel@localhost.localdomain>
	<50646CFF.3010601@ziade.org>
	<loom.20120927T173348-617@post.gmane.org>
	<50647599.3030507@ziade.org> <k41vce$89g$1@ger.gmane.org>
	<loom.20120927T184442-444@post.gmane.org>
	<k43gjn$37l$1@ger.gmane.org>
	<loom.20120928T100621-122@post.gmane.org>
Message-ID: <k43n4m$mgl$1@ger.gmane.org>

On 09/28/2012 10:17 AM, Vinay Sajip wrote:
> Georg Brandl <g.brandl <at> gmx.net> writes:
>
>> I've cloned the bitbucket.org/vinay.sajip/distlib repo to
>> hg.python.org/distlib.
>
> Thank you very much.
>
>> At the moment no hooks are set up; let me know if you want e.g. email
>> notification, CIA or roundup integration.
>
> Are the coding style (whitespace) hooks set up? They would be useful to have.

I've now set up the checkwhitespace hook.  Let me know if it works as intended.

> Re. the integration with CI, roundup and email, I'm not sure exactly which
> features are easy to provide - how much work would they mean for you? I don't
> see any urgent need to implement these right away - perhaps we can add these
> when some more work has been done, and there are more contributors who can
> benefit.

If you can tell me an address to send the notifications to, email is easy to
set up.  For CIA, I need the user and project names to submit for.

The Roundup hook works by sending email to the Roundup email gateway.

Georg


From jim at zope.com  Fri Sep 28 15:16:53 2012
From: jim at zope.com (Jim Fulton)
Date: Fri, 28 Sep 2012 09:16:53 -0400
Subject: [Distutils] buildout.org A record
In-Reply-To: <k43h56$78c$1@ger.gmane.org>
References: <CANoAZMCh4uvnvOXKo2ZEmUkNjTCfpupfsp3_A3vhky9=sQCLPQ@mail.gmail.com>
	<CAPDm-Fhd4WRNugMuZY7mDuYqdymHPg3uxTgh20t+vWgem2SGmw@mail.gmail.com>
	<CANoAZMB0W0b0AFtVzDO85a5eqy6aQGWNrUKaUk7ma6et7S--JQ@mail.gmail.com>
	<CAPDm-FgdfTPKbkOC-EB=+KWyX9KvPprSYdQdTYzjOSAnKfGtag@mail.gmail.com>
	<k40e7j$fhj$1@ger.gmane.org>
	<CAPDm-FjKj5ZvwurCVpeAfMXXyp6tv63Qu1e1+_Wvu+Urjhuymw@mail.gmail.com>
	<k43h56$78c$1@ger.gmane.org>
Message-ID: <CAPDm-Fi1SB4+wE5suh625v+q3Wsjpn0OAR19BNWLcmxpqZwZoQ@mail.gmail.com>

On Fri, Sep 28, 2012 at 2:47 AM, Christian Theune <ct at gocept.com> wrote:
> Hi,
>
>
> On 2012-09-27 21:12:05 +0000, Jim Fulton said:
>
>> On Wed, Sep 26, 2012 at 10:38 PM, Alex Clark <aclark at aclark.net> wrote:

...

>>> If anyone does know, please change the DNS A record to 207.97.227.245 so
>>> we
>>> can host the site on github pages.
>>
>>
>> I'm all for that (where A record to 207... is a euphemism for CNAME record
>> to
>> pages.github.com) once something's set up there. :)
>>
>> In the mean time, I'd just like to get DNS under control. :(
>
>
> OK - DNS is under control now. We took it back to the DNS servers at gocept
> to get the status quo back to a working state. We're happy to make any
> changes that are deemed necessary. Looking at the fact that we get contacted
> because the WHOIS points to us I'd prefer to also have authority over DNS -
> otherwise we just get flooded with requests that we can't do anything about.

+1

> So: what's next?

Status quo for now.  I like the idea of using github pages to host buildout.org.
When someone sets that up, we'll switch over.

Jim


-- 
Jim Fulton
http://www.linkedin.com/in/jimfulton
Jerky is better than bacon! http://zo.pe/Kqm

From ct at gocept.com  Fri Sep 28 15:18:06 2012
From: ct at gocept.com (Christian Theune)
Date: Fri, 28 Sep 2012 15:18:06 +0200
Subject: [Distutils] buildout.org A record
In-Reply-To: <CAPDm-Fi1SB4+wE5suh625v+q3Wsjpn0OAR19BNWLcmxpqZwZoQ@mail.gmail.com>
References: <CANoAZMCh4uvnvOXKo2ZEmUkNjTCfpupfsp3_A3vhky9=sQCLPQ@mail.gmail.com>
	<CAPDm-Fhd4WRNugMuZY7mDuYqdymHPg3uxTgh20t+vWgem2SGmw@mail.gmail.com>
	<CANoAZMB0W0b0AFtVzDO85a5eqy6aQGWNrUKaUk7ma6et7S--JQ@mail.gmail.com>
	<CAPDm-FgdfTPKbkOC-EB=+KWyX9KvPprSYdQdTYzjOSAnKfGtag@mail.gmail.com>
	<k40e7j$fhj$1@ger.gmane.org>
	<CAPDm-FjKj5ZvwurCVpeAfMXXyp6tv63Qu1e1+_Wvu+Urjhuymw@mail.gmail.com>
	<k43h56$78c$1@ger.gmane.org>
	<CAPDm-Fi1SB4+wE5suh625v+q3Wsjpn0OAR19BNWLcmxpqZwZoQ@mail.gmail.com>
Message-ID: <D88253DF-045D-46E0-822B-B6FAF63D5AA8@gocept.com>


On Sep 28, 2012, at 3:16 PM, Jim Fulton <jim at zope.com> wrote:
> 
> Status quo for now.  I like the idea of using github pages to host buildout.org.
> When someone sets that up, we'll switch over.

Ack.

-- 
Christian Theune ? ct at gocept.com
gocept gmbh & co. kg ? Forsterstra?e 29 ? 06112 Halle (Saale) ? Germany
http://gocept.com ? Tel +49 345 1229889-7
Python, Pyramid, Plone, Zope ? consulting, development, hosting, operations


From marius at pov.lt  Fri Sep 28 16:32:09 2012
From: marius at pov.lt (Marius Gedminas)
Date: Fri, 28 Sep 2012 17:32:09 +0300
Subject: [Distutils] buildout.org A record
In-Reply-To: <D88253DF-045D-46E0-822B-B6FAF63D5AA8@gocept.com>
References: <CANoAZMCh4uvnvOXKo2ZEmUkNjTCfpupfsp3_A3vhky9=sQCLPQ@mail.gmail.com>
	<CAPDm-Fhd4WRNugMuZY7mDuYqdymHPg3uxTgh20t+vWgem2SGmw@mail.gmail.com>
	<CANoAZMB0W0b0AFtVzDO85a5eqy6aQGWNrUKaUk7ma6et7S--JQ@mail.gmail.com>
	<CAPDm-FgdfTPKbkOC-EB=+KWyX9KvPprSYdQdTYzjOSAnKfGtag@mail.gmail.com>
	<k40e7j$fhj$1@ger.gmane.org>
	<CAPDm-FjKj5ZvwurCVpeAfMXXyp6tv63Qu1e1+_Wvu+Urjhuymw@mail.gmail.com>
	<k43h56$78c$1@ger.gmane.org>
	<CAPDm-Fi1SB4+wE5suh625v+q3Wsjpn0OAR19BNWLcmxpqZwZoQ@mail.gmail.com>
	<D88253DF-045D-46E0-822B-B6FAF63D5AA8@gocept.com>
Message-ID: <20120928143209.GA17543@fridge.pov.lt>

On Fri, Sep 28, 2012 at 03:18:06PM +0200, Christian Theune wrote:
> 
> On Sep 28, 2012, at 3:16 PM, Jim Fulton <jim at zope.com> wrote:
> > 
> > Status quo for now.  I like the idea of using github pages to host buildout.org.
> > When someone sets that up, we'll switch over.
> 
> Ack.

Um, status quo is "www.buildout.org is broken", or am I missing
something?

    $ links buildout.org
    Error loading http://www.buildout.org/: Host not found

    $ host -t ns buildout.org
    buildout.org name server ns1.gocept.com.
    buildout.org name server ns2.gocept.com.

    $ host buildout.org ns1.gocept.com.
    buildout.org has address 176.9.22.59

    $ host www.buildout.org ns1.gocept.com.
    Host www.buildout.org.lan not found: 5(REFUSED)

No idea where that '.lan' comes from

    $ host www.buildout.org. ns1.gocept.com.
    Host www.buildout.org. not found: 3(NXDOMAIN)

Marius Gedminas
-- 
BYTE editors are people who separate the wheat from the chaff, and then
carefully print the chaff.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 190 bytes
Desc: Digital signature
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20120928/563f5a51/attachment.pgp>

From merwok at netwok.org  Fri Sep 28 18:01:57 2012
From: merwok at netwok.org (=?UTF-8?B?w4lyaWMgQXJhdWpv?=)
Date: Fri, 28 Sep 2012 12:01:57 -0400
Subject: [Distutils] Panel on packaging at PyCon 2013
Message-ID: <5065C9F5.5080202@netwok.org>

Hi everybody,

I?m putting up a last-minute proposal for a panel about directions for
the packaging ecosystem at the next PyCon.  For that I would need a list
of panelists.  I think it would be interesting to have developers (say
from distribute, buildout, pip, wheel) as well as users from
subcommunities (packaging people tend to be web developers, but their
experience doesn?t match the needs of the scipy community for example).

The threads of the panel would be something like:
- working on a better distutils vs. rethinking the whole thing;
- what to put in the stdlib (distlib vs. distutils2);
- decide if we revise the accepted PEPs to address the known problems or
start new ones
- devise a roadmap for new PEPs.

Who wants to volunteer?

I would be a panelist too, so I need a moderator.  Nick, would you like
to take that role?

Regards

From barry at python.org  Fri Sep 28 18:08:48 2012
From: barry at python.org (Barry Warsaw)
Date: Fri, 28 Sep 2012 12:08:48 -0400
Subject: [Distutils] Panel on packaging at PyCon 2013
In-Reply-To: <5065C9F5.5080202@netwok.org>
References: <5065C9F5.5080202@netwok.org>
Message-ID: <20120928120848.11f11880@resist.wooz.org>

On Sep 28, 2012, at 12:01 PM, ?ric Araujo wrote:

>I?m putting up a last-minute proposal for a panel about directions for
>the packaging ecosystem at the next PyCon.  For that I would need a list
>of panelists.  I think it would be interesting to have developers (say
>from distribute, buildout, pip, wheel) as well as users from
>subcommunities (packaging people tend to be web developers, but their
>experience doesn?t match the needs of the scipy community for example).

Great idea.  Please include folks from distros.  Toshio would be a great
representative of the RPM faction, and although I'm sure there's someone
better, if no one else volunteers, I'll represent the .deb cabal.  It would be
nice to have other *nix representatives if available, as well as Mac(Ports?)
and Windows experts.

Cheers,
-Barry

From aclark at aclark.net  Fri Sep 28 18:14:52 2012
From: aclark at aclark.net (Alex Clark)
Date: Fri, 28 Sep 2012 12:14:52 -0400
Subject: [Distutils] pythonpackages.com "hail mary"
Message-ID: <k44idr$9q4$1@ger.gmane.org>

Hi,

The pythonpackages.com "hail mary" call has gone out:

- http://blog.aclark.net/2012/09/28/pythonpackages-com-one-year-later/

If you are interested in this project and can help financially (even in 
a small way), please do so now (the blog entry has the details.) 
Otherwise, I'll mostly likely open source the application by the end of 
the year, and chop it up for parts :-).


Thanks for considering,


Alex



-- 
Alex Clark ? http://pythonpackages.com



From aclark at aclark.net  Fri Sep 28 18:27:46 2012
From: aclark at aclark.net (Alex Clark)
Date: Fri, 28 Sep 2012 12:27:46 -0400
Subject: [Distutils] pythonpackages.com "hail mary"
References: <k44idr$9q4$1@ger.gmane.org>
Message-ID: <k44j62$gj2$1@ger.gmane.org>

On 2012-09-28 16:14:52 +0000, Alex Clark said:

> Hi,
> 
> The pythonpackages.com "hail mary" call has gone out:
> 
> - http://blog.aclark.net/2012/09/28/pythonpackages-com-one-year-later/
> 
> If you are interested in this project and can help financially (even 
> ina small way), please do so now (the blog entry has the 
> details.)Otherwise, I'll mostly likely open source the application by 
> the end ofthe year, and chop it up for parts :-).
> 
> 
> Thanks for considering,


Sorry, I think I meant to send this to catalog-sig.


> 
> 
> Alex


-- 
Alex Clark ? http://pythonpackages.com



From aclark at aclark.net  Fri Sep 28 18:58:38 2012
From: aclark at aclark.net (Alex Clark)
Date: Fri, 28 Sep 2012 12:58:38 -0400
Subject: [Distutils] buildout.org A record
References: <CANoAZMCh4uvnvOXKo2ZEmUkNjTCfpupfsp3_A3vhky9=sQCLPQ@mail.gmail.com>
	<CAPDm-Fhd4WRNugMuZY7mDuYqdymHPg3uxTgh20t+vWgem2SGmw@mail.gmail.com>
	<CANoAZMB0W0b0AFtVzDO85a5eqy6aQGWNrUKaUk7ma6et7S--JQ@mail.gmail.com>
	<CAPDm-FgdfTPKbkOC-EB=+KWyX9KvPprSYdQdTYzjOSAnKfGtag@mail.gmail.com>
	<k40e7j$fhj$1@ger.gmane.org>
	<CAPDm-FjKj5ZvwurCVpeAfMXXyp6tv63Qu1e1+_Wvu+Urjhuymw@mail.gmail.com>
	<k43h56$78c$1@ger.gmane.org>
Message-ID: <k44kvs$16j$1@ger.gmane.org>

On 2012-09-28 06:47:08 +0000, Christian Theune said:

> Hi,
> 
> On 2012-09-27 21:12:05 +0000, Jim Fulton said:
> 
>> On Wed, Sep 26, 2012 at 10:38 PM, Alex Clark <aclark at aclark.net> wrote:
>>> On 2012-09-26 15:35:00 +0000, Jim Fulton said:
>>> 
>>>> On Wed, Sep 26, 2012 at 11:03 AM, Conrado Buhrer <conrado at buhrer.net>
>>>> wrote:
>>>>> 
>>>>> Hi Jim,
>>>>> 
>>>>> A few of us on #buildout at freenode have noticed the issue, we believe
>>>>> that some links on pypi are the cause.
>>>>> 
>>>>> See lines 84-89 of the source of
>>>>> http://pypi.python.org/simple/zc.buildout/
>>>> 
>>>> 
>>>> Fixed. I removed the home-page meta data from those releases.
>>>> I don't know what's going on with buildout.org. :(
>>> 
>>> 
>>> 
>>> If anyone does know, please change the DNS A record to 207.97.227.245 so we
>>> can host the site on github pages.
>> 
>> I'm all for that (where A record to 207... is a euphemism for CNAME record to
>> pages.github.com) once something's set up there. :)
>> 
>> In the mean time, I'd just like to get DNS under control. :(
> 
> OK - DNS is under control now. We took it back to the DNS servers at
> gocept to get the status quo back to a working state. We're happy to
> make any changes that are deemed necessary. Looking at the fact that we
> get contacted because the WHOIS points to us I'd prefer to also have
> authority over DNS - otherwise we just get flooded with requests that
> we can't do anything about.
> 
> So: what's next?

Please change the A record. Once that change propagates, these docs 
will be displayed at buildout.org:

- https://github.com/buildout/buildout.github.com


Alex


> 
> Christian
> 
> 
> _______________________________________________
> Distutils-SIG maillist  -  Distutils-SIG at python.org
> http://mail.python.org/mailman/listinfo/distutils-sig


-- 
Alex Clark ? http://pythonpackages.com



From ncoghlan at gmail.com  Fri Sep 28 19:05:01 2012
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Fri, 28 Sep 2012 22:35:01 +0530
Subject: [Distutils] [Python-Dev] distlib updated with resources API
In-Reply-To: <CAG8k2+54-cyWCg81FF1epjH+4Gb9c9ZQhfV4WgSdPdRYuiM1Kw@mail.gmail.com>
References: <loom.20120923T174554-748@post.gmane.org>
	<50636845.8080204@ziade.org>
	<loom.20120926T233609-992@post.gmane.org>
	<50645C34.6070606@ziade.org>
	<1348756086.3495.1.camel@localhost.localdomain>
	<50646CFF.3010601@ziade.org>
	<loom.20120927T173348-617@post.gmane.org>
	<50647599.3030507@ziade.org>
	<loom.20120928T004648-800@post.gmane.org>
	<5064E31D.80601@ziade.org>
	<CADiSq7et7kT7NKSjNJ-D3oWAWcbNNF43=nqaj=VrCkThJoyYXA@mail.gmail.com>
	<CAG8k2+54-cyWCg81FF1epjH+4Gb9c9ZQhfV4WgSdPdRYuiM1Kw@mail.gmail.com>
Message-ID: <CADiSq7eYnfDTkKSBjen+Q=gZaTN8QbQ5kjzWrfn+zrbCW6kzBA@mail.gmail.com>

On Fri, Sep 28, 2012 at 10:07 AM, Daniel Holth <dholth at gmail.com> wrote:
> Are we trying to kill setuptools? I'm not entirely sure, but we should
> stop trying to do that. The migration should take essentially forever
> as soon as it makes sense for each pypi publisher.

I'd certainly like to kill easy_install, and see any popular elements
of setuptools metadata become officially defined *independently* of
any given implementation.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia

From jim at zope.com  Fri Sep 28 19:50:56 2012
From: jim at zope.com (Jim Fulton)
Date: Fri, 28 Sep 2012 13:50:56 -0400
Subject: [Distutils] buildout.org A record
In-Reply-To: <k44kvs$16j$1@ger.gmane.org>
References: <CANoAZMCh4uvnvOXKo2ZEmUkNjTCfpupfsp3_A3vhky9=sQCLPQ@mail.gmail.com>
	<CAPDm-Fhd4WRNugMuZY7mDuYqdymHPg3uxTgh20t+vWgem2SGmw@mail.gmail.com>
	<CANoAZMB0W0b0AFtVzDO85a5eqy6aQGWNrUKaUk7ma6et7S--JQ@mail.gmail.com>
	<CAPDm-FgdfTPKbkOC-EB=+KWyX9KvPprSYdQdTYzjOSAnKfGtag@mail.gmail.com>
	<k40e7j$fhj$1@ger.gmane.org>
	<CAPDm-FjKj5ZvwurCVpeAfMXXyp6tv63Qu1e1+_Wvu+Urjhuymw@mail.gmail.com>
	<k43h56$78c$1@ger.gmane.org> <k44kvs$16j$1@ger.gmane.org>
Message-ID: <CAPDm-FjxtQqCO0uBxkCGtv9h4xiKuEf4j_tAtnZ5tXXgykVbhw@mail.gmail.com>

On Fri, Sep 28, 2012 at 12:58 PM, Alex Clark <aclark at aclark.net> wrote:
> Please change the A record. Once that change propagates, these docs will be
> displayed at buildout.org:
>
> - https://github.com/buildout/buildout.github.com

Not so fast.

1) To test this, I added this to my host file:

    207.97.227.245 www.buildout.org	buildout.org

    What I get when I go there has no content.
    I don't want us to switch to this until it's on par
    (or at least vaguely close) to the existing site.

2) When we do this, if I understand this correctly, we'll
     want to use a CNAME to pages.github.com.

Jim


-- 
Jim Fulton
http://www.linkedin.com/in/jimfulton
Jerky is better than bacon! http://zo.pe/Kqm

From dholth at gmail.com  Fri Sep 28 19:59:34 2012
From: dholth at gmail.com (Daniel Holth)
Date: Fri, 28 Sep 2012 13:59:34 -0400
Subject: [Distutils] [Python-Dev] distlib updated with resources API
In-Reply-To: <CADiSq7eYnfDTkKSBjen+Q=gZaTN8QbQ5kjzWrfn+zrbCW6kzBA@mail.gmail.com>
References: <loom.20120923T174554-748@post.gmane.org>
	<50636845.8080204@ziade.org>
	<loom.20120926T233609-992@post.gmane.org>
	<50645C34.6070606@ziade.org>
	<1348756086.3495.1.camel@localhost.localdomain>
	<50646CFF.3010601@ziade.org>
	<loom.20120927T173348-617@post.gmane.org>
	<50647599.3030507@ziade.org>
	<loom.20120928T004648-800@post.gmane.org>
	<5064E31D.80601@ziade.org>
	<CADiSq7et7kT7NKSjNJ-D3oWAWcbNNF43=nqaj=VrCkThJoyYXA@mail.gmail.com>
	<CAG8k2+54-cyWCg81FF1epjH+4Gb9c9ZQhfV4WgSdPdRYuiM1Kw@mail.gmail.com>
	<CADiSq7eYnfDTkKSBjen+Q=gZaTN8QbQ5kjzWrfn+zrbCW6kzBA@mail.gmail.com>
Message-ID: <CAG8k2+4et=av7RS6NtpNa=V8_tAnReBSq5ogNkmXQEcEqrjgYg@mail.gmail.com>

> I'd certainly like to kill easy_install, and see any popular elements
> of setuptools metadata become officially defined *independently* of
> any given implementation.

I would like to kill distutils without killing setuptools, if that
makes any sense.

I think the most important thing to do is to clear the confusion and
doubt about packaging so you can just relax and make a package rather
than wondering if new-packaging-stuff is working yet or if what you
are doing is going to suddenly stop working. So my message is relax,
make your package how you please, it will continue to be useful,
packaging is under control.

I think we have the documenting the egg metadata bit pretty well
covered at least in e-mail and in the existing PEAK documentation.

The next step for me is probably to give setup.cfg's [metadata]
section another shot. It would be easy to implement in setuptools and
pip, and we would solve the "what do I have to install to run
setup.py?" problem and the "where do I put the environment markers"
(because install_requires=[] can't parse them) problem.

setup.cfg:

[metadata]
setup_requires_dist = one
   two ; python_version < '4.0'

requires_dist = ...

From a.badger at gmail.com  Fri Sep 28 20:08:21 2012
From: a.badger at gmail.com (Toshio Kuratomi)
Date: Fri, 28 Sep 2012 11:08:21 -0700
Subject: [Distutils] Panel on packaging at PyCon 2013
In-Reply-To: <20120928120848.11f11880@resist.wooz.org>
References: <5065C9F5.5080202@netwok.org>
	<20120928120848.11f11880@resist.wooz.org>
Message-ID: <20120928180821.GQ2974@unaka.lan>

On Fri, Sep 28, 2012 at 12:08:48PM -0400, Barry Warsaw wrote:
> On Sep 28, 2012, at 12:01 PM, ?ric Araujo wrote:
> 
> >I?m putting up a last-minute proposal for a panel about directions for
> >the packaging ecosystem at the next PyCon.  For that I would need a list
> >of panelists.  I think it would be interesting to have developers (say
> >from distribute, buildout, pip, wheel) as well as users from
> >subcommunities (packaging people tend to be web developers, but their
> >experience doesn?t match the needs of the scipy community for example).
> 
> Great idea.  Please include folks from distros.  Toshio would be a great
> representative of the RPM faction, and although I'm sure there's someone
> better, if no one else volunteers, I'll represent the .deb cabal.  It would be
> nice to have other *nix representatives if available, as well as Mac(Ports?)
> and Windows experts.
> 
<nod> I could be on a panel although I haven't been keeping up with the
changes in distutils2/packaging/distlib/wheel/etc recently.  Nick Coghlin
had mentioned on one of the Fedora Python lists that he was working on
packaging and distros.  If he's planning on attending pycon he'd be an
alternate good choice for an rpm-based distro.

-Toshio
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 198 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20120928/29e5222e/attachment.pgp>

From aclark at aclark.net  Fri Sep 28 20:49:28 2012
From: aclark at aclark.net (Alex Clark)
Date: Fri, 28 Sep 2012 14:49:28 -0400
Subject: [Distutils] buildout.org A record
References: <CANoAZMCh4uvnvOXKo2ZEmUkNjTCfpupfsp3_A3vhky9=sQCLPQ@mail.gmail.com>
	<CAPDm-Fhd4WRNugMuZY7mDuYqdymHPg3uxTgh20t+vWgem2SGmw@mail.gmail.com>
	<CANoAZMB0W0b0AFtVzDO85a5eqy6aQGWNrUKaUk7ma6et7S--JQ@mail.gmail.com>
	<CAPDm-FgdfTPKbkOC-EB=+KWyX9KvPprSYdQdTYzjOSAnKfGtag@mail.gmail.com>
	<k40e7j$fhj$1@ger.gmane.org>
	<CAPDm-FjKj5ZvwurCVpeAfMXXyp6tv63Qu1e1+_Wvu+Urjhuymw@mail.gmail.com>
	<k43h56$78c$1@ger.gmane.org> <k44kvs$16j$1@ger.gmane.org>
	<CAPDm-FjxtQqCO0uBxkCGtv9h4xiKuEf4j_tAtnZ5tXXgykVbhw@mail.gmail.com>
Message-ID: <k44rfn$r1c$1@ger.gmane.org>

On 2012-09-28 17:50:56 +0000, Jim Fulton said:

> On Fri, Sep 28, 2012 at 12:58 PM, Alex Clark <aclark at aclark.net> wrote:
>> Please change the A record. Once that change propagates, these docs will be
>> displayed at buildout.org:
>> 
>> - https://github.com/buildout/buildout.github.com
> 
> Not so fast.
> 
> 1) To test this, I added this to my host file:
> 
>     207.97.227.245 www.buildout.org	buildout.org
> 
>     What I get when I go there has no content.
>     I don't want us to switch to this until it's on par
>     (or at least vaguely close) to the existing site.

OK I added files from the wayback machine, so it should look exactly 
like it used to. e.g.

- https://raw.github.com/buildout/buildout.github.com/master/screenshot.png


> 
> 2) When we do this, if I understand this correctly, we'll
>      want to use a CNAME to pages.github.com.


Not exactly. Check:

- https://help.github.com/articles/setting-up-a-custom-domain-with-pages

So if you want to use the naked domain (and redir www to buildout.org) then do:

buildout.org A 204.232.175.78

Otherwise you'd create a CNAME for www.buildout.org:

www CNAME buildout.github.com

(and redir buildout.org to www)


Alex





> 
> Jim


-- 
Alex Clark ? http://pythonpackages.com



From dholth at gmail.com  Fri Sep 28 22:14:43 2012
From: dholth at gmail.com (Daniel Holth)
Date: Fri, 28 Sep 2012 16:14:43 -0400
Subject: [Distutils] indexing sys.meta_path hook?
Message-ID: <CAG8k2+63BUdRv4n7SFepPPqPnvkwAX0iSf8e03tPjZGvEJZ9EA@mail.gmail.com>

Has anyone bothered to write a meta path hook that checks all the
top_level.txt on a sys.path full of eggs, and uses
imp.load_module(name, short list of paths) to find the module instead
of the default load_module(name, full sys.path)?

From pje at telecommunity.com  Sat Sep 29 00:23:18 2012
From: pje at telecommunity.com (PJ Eby)
Date: Fri, 28 Sep 2012 18:23:18 -0400
Subject: [Distutils] indexing sys.meta_path hook?
In-Reply-To: <CAG8k2+63BUdRv4n7SFepPPqPnvkwAX0iSf8e03tPjZGvEJZ9EA@mail.gmail.com>
References: <CAG8k2+63BUdRv4n7SFepPPqPnvkwAX0iSf8e03tPjZGvEJZ9EA@mail.gmail.com>
Message-ID: <CALeMXf4-82tEodBwU1P0ufhFfR=N6SaDsWwm=KgvhQFVBtcyuA@mail.gmail.com>

On Fri, Sep 28, 2012 at 4:14 PM, Daniel Holth <dholth at gmail.com> wrote:
> Has anyone bothered to write a meta path hook that checks all the
> top_level.txt on a sys.path full of eggs, and uses
> imp.load_module(name, short list of paths) to find the module instead
> of the default load_module(name, full sys.path)?

If you're talking about runtime caching, Python 3.3 does something
similar by default.  For older Pythons, using zipped eggs does
something similar by default (which is why eggs default to installing
zipped under easy_install, actually).

From dholth at gmail.com  Sat Sep 29 00:27:39 2012
From: dholth at gmail.com (Daniel Holth)
Date: Fri, 28 Sep 2012 18:27:39 -0400
Subject: [Distutils] indexing sys.meta_path hook?
In-Reply-To: <CALeMXf4-82tEodBwU1P0ufhFfR=N6SaDsWwm=KgvhQFVBtcyuA@mail.gmail.com>
References: <CAG8k2+63BUdRv4n7SFepPPqPnvkwAX0iSf8e03tPjZGvEJZ9EA@mail.gmail.com>
	<CALeMXf4-82tEodBwU1P0ufhFfR=N6SaDsWwm=KgvhQFVBtcyuA@mail.gmail.com>
Message-ID: <CAG8k2+6CnbZHa30xu_EjMCfRjm=oXYjYzK2hud0LzCoq_EMxfg@mail.gmail.com>

Is the new python 3.3 optimization documented anywhere? I am having trouble
finding it.
On Sep 28, 2012 6:23 PM, "PJ Eby" <pje at telecommunity.com> wrote:

> On Fri, Sep 28, 2012 at 4:14 PM, Daniel Holth <dholth at gmail.com> wrote:
> > Has anyone bothered to write a meta path hook that checks all the
> > top_level.txt on a sys.path full of eggs, and uses
> > imp.load_module(name, short list of paths) to find the module instead
> > of the default load_module(name, full sys.path)?
>
> If you're talking about runtime caching, Python 3.3 does something
> similar by default.  For older Pythons, using zipped eggs does
> something similar by default (which is why eggs default to installing
> zipped under easy_install, actually).
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20120928/ab33f848/attachment.html>

From pje at telecommunity.com  Sat Sep 29 00:42:29 2012
From: pje at telecommunity.com (PJ Eby)
Date: Fri, 28 Sep 2012 18:42:29 -0400
Subject: [Distutils] indexing sys.meta_path hook?
In-Reply-To: <CAG8k2+6CnbZHa30xu_EjMCfRjm=oXYjYzK2hud0LzCoq_EMxfg@mail.gmail.com>
References: <CAG8k2+63BUdRv4n7SFepPPqPnvkwAX0iSf8e03tPjZGvEJZ9EA@mail.gmail.com>
	<CALeMXf4-82tEodBwU1P0ufhFfR=N6SaDsWwm=KgvhQFVBtcyuA@mail.gmail.com>
	<CAG8k2+6CnbZHa30xu_EjMCfRjm=oXYjYzK2hud0LzCoq_EMxfg@mail.gmail.com>
Message-ID: <CALeMXf7wT7g1gfkH90efugDH83BM_yBCF=Cj6ZxKErSzSPxRpw@mail.gmail.com>

On Fri, Sep 28, 2012 at 6:27 PM, Daniel Holth <dholth at gmail.com> wrote:
> Is the new python 3.3 optimization documented anywhere? I am having trouble
> finding it.

Well, it's mentioned several places in
http://docs.python.org/dev/whatsnew/3.3.html, albeit rather obscurely.
 Most important mention is probably this:

"""The default finders used by import now utilize a cache of what is
contained within a specific directory. If you create a Python source
file or sourceless bytecode file, make sure to call
importlib.invalidate_caches() to clear out the cache for the finders
to notice the new file."""

Basically, the new system does for directories what older Pythons did
for zipfiles: read in the a directory listing on first use, and keep
it cached in memory.  Unlike zipfiles, though, there's a stat() call
done on each import to verify the directory listing isn't out of date.

Technically, you don't need to manually flush the caches every time
you put new modules in a directory, though.  The only way for the
cache to become invalid is if you try to import a module between two
changes to the same directory, there is a potential race condition on
filesystems with a highly granular filestamp type.  For example, on a
FAT filesystem, timestamps only change every two seconds, so if you
make two changes to a directory within two seconds, and an import is
attempted between those two changes, then the cache for that directory
could become stale.  A bit of info on this can be found at
http://docs.python.org/dev/library/importlib.html#importlib.machinery.FileFinder

From vinay_sajip at yahoo.co.uk  Sat Sep 29 11:54:46 2012
From: vinay_sajip at yahoo.co.uk (Vinay Sajip)
Date: Sat, 29 Sep 2012 09:54:46 +0000 (UTC)
Subject: [Distutils] [Python-Dev]  distlib updated with resources API
References: <loom.20120923T174554-748@post.gmane.org>
	<50636845.8080204@ziade.org>
	<loom.20120926T233609-992@post.gmane.org>
	<50645C34.6070606@ziade.org>
	<1348756086.3495.1.camel@localhost.localdomain>
	<50646CFF.3010601@ziade.org>
	<loom.20120927T173348-617@post.gmane.org>
	<50647599.3030507@ziade.org>
	<loom.20120928T004648-800@post.gmane.org>
	<5064E31D.80601@ziade.org>
	<CADiSq7et7kT7NKSjNJ-D3oWAWcbNNF43=nqaj=VrCkThJoyYXA@mail.gmail.com>
Message-ID: <loom.20120929T112825-998@post.gmane.org>

Nick Coghlan <ncoghlan <at> gmail.com> writes:


> > The document has changed since then,
> >
http://python-notes.boredomandlaziness.org/en/latest/pep_ideas/core_packaging_api.html

I read from your page there that Donald Stufft is working on a JSON-based
metadata format. I've been looking that the same thing - a more flexible
metadata format which directly maps to dicts - but I used YAML, as looking at
too much JSON gives me eye-strain from all the extraneous quotes and braces.

I believe that JSON is the right format to use at the moment, because
PyYAML still has some bugs which I've run into while doing this work (also,
of course, it's not in the stdlib). As the formats are readily
interchangeable, there might be interest here in looking at the package.yaml
that I've come up with.

Since the metadata needs to support both the existing metadata and the
additional things that e.g. setuptools supports via additional kwargs to
setup(), I put together an ugly hack where I essentially mocked parts of
distutils and setuptools, including the setup() call. This allows me to
generate the YAML format automatically from most distributions on PyPI, using
their setup.py.

Here's a GIST with sample package.yaml files automatically generated from PyPI
downloads of SQLAlchemy 0.7.8, Jinja2 2.6, Flask 0.9 and wheel 0.9.4.

https://gist.github.com/3803556

The JSON format of the metadata is actually appended as a comment on the last
line of the YAML metadata (I use that to report YAML bugs).

I've not yet documented the schema for the metadata, as I'm still thinking
about the details.

I ran my hack on around 18,000 PyPI releases (basically, all the latest
releases which are hosted on PyPI). For all but around 1300, I was
able to generate package.yaml files. The ones which failed are those where
no setup.py is present, or it's present, but can't be imported because it
assumes that some third-party package is available.

Comments welcome.

Regards,

Vinay Sajip


From dholth at gmail.com  Sun Sep 30 05:55:58 2012
From: dholth at gmail.com (Daniel Holth)
Date: Sat, 29 Sep 2012 23:55:58 -0400
Subject: [Distutils] [Python-Dev] distlib updated with resources API
In-Reply-To: <loom.20120929T112825-998@post.gmane.org>
References: <loom.20120923T174554-748@post.gmane.org>
	<50636845.8080204@ziade.org>
	<loom.20120926T233609-992@post.gmane.org>
	<50645C34.6070606@ziade.org>
	<1348756086.3495.1.camel@localhost.localdomain>
	<50646CFF.3010601@ziade.org>
	<loom.20120927T173348-617@post.gmane.org>
	<50647599.3030507@ziade.org>
	<loom.20120928T004648-800@post.gmane.org>
	<5064E31D.80601@ziade.org>
	<CADiSq7et7kT7NKSjNJ-D3oWAWcbNNF43=nqaj=VrCkThJoyYXA@mail.gmail.com>
	<loom.20120929T112825-998@post.gmane.org>
Message-ID: <CAG8k2+4anCPWpR4JNvLqvb=CF5-tMLqvmA1BJFRfSA2CLQ2_Ow@mail.gmail.com>

I like this kind of study. Fixing 1300 packages sounds a lot more
manageable than fixing 18,000. (I took a similar look at setup.py but
with the ast module instead of actually running the things. Your
method is probably more accurate.) It would be very cool to know how
many packages use if: statements to affect install_requires...

I have tried to include the vital setuptools metadata in Metadata 1.3
without the json. It maps to an ordered dict. A few files like
entry_points.txt stay in their own files, and are better off (more
performant) that way, since you may be able to avoid parsing METADATA
at all if you just want to know if a package has entry_points
(os.path.exists(entry_points.txt)).

Did you look at the bento ipkg (internal package metadata) format? A
barebones one is at https://gist.github.com/3715068

Is there a good "download the latest versions of everything hosted on
pypi" script? Mine was pretty terrible as it could not resume after a
crash or after the data got stale.

From donald.stufft at gmail.com  Sun Sep 30 07:03:58 2012
From: donald.stufft at gmail.com (Donald Stufft)
Date: Sun, 30 Sep 2012 01:03:58 -0400
Subject: [Distutils] [Python-Dev] distlib updated with resources API
In-Reply-To: <loom.20120929T112825-998@post.gmane.org>
References: <loom.20120923T174554-748@post.gmane.org>
	<50636845.8080204@ziade.org> <loom.20120926T233609-992@post.gmane.org>
	<50645C34.6070606@ziade.org>
	<1348756086.3495.1.camel@localhost.localdomain>
	<50646CFF.3010601@ziade.org> <loom.20120927T173348-617@post.gmane.org>
	<50647599.3030507@ziade.org> <loom.20120928T004648-800@post.gmane.org>
	<5064E31D.80601@ziade.org>
	<CADiSq7et7kT7NKSjNJ-D3oWAWcbNNF43=nqaj=VrCkThJoyYXA@mail.gmail.com>
	<loom.20120929T112825-998@post.gmane.org>
Message-ID: <472D350E502F4B2FA46F3B6475281BE6@gmail.com>

I have some unpublished work I should publish. 

Part of the point with what i'm trying to do is to define a standard for what is inside
a package, but not really for how you take a particular set of files and turn it into
that. So if you want to edit yaml you can have a yaml file and have a package
creation tool that turns that yaml into the standard json that gets added to the
package. The same can be said for a python file or a setup.cfg or whatever. Ideally
the roles of package creation, building, and installation should be able to be completely
separate. So my goal is to facilitate that by creating a standard way to describe
all the data about a distribution, including extensible data in a way that any tool
can serialize or deserialize it losslessly.



On Saturday, September 29, 2012 at 5:54 AM, Vinay Sajip wrote:

> Nick Coghlan <ncoghlan <at> gmail.com (http://gmail.com)> writes:
> 
> 
> > > The document has changed since then,
> http://python-notes.boredomandlaziness.org/en/latest/pep_ideas/core_packaging_api.html
> 
> I read from your page there that Donald Stufft is working on a JSON-based
> metadata format. I've been looking that the same thing - a more flexible
> metadata format which directly maps to dicts - but I used YAML, as looking at
> too much JSON gives me eye-strain from all the extraneous quotes and braces.
> 
> I believe that JSON is the right format to use at the moment, because
> PyYAML still has some bugs which I've run into while doing this work (also,
> of course, it's not in the stdlib). As the formats are readily
> interchangeable, there might be interest here in looking at the package.yaml
> that I've come up with.
> 
> Since the metadata needs to support both the existing metadata and the
> additional things that e.g. setuptools supports via additional kwargs to
> setup(), I put together an ugly hack where I essentially mocked parts of
> distutils and setuptools, including the setup() call. This allows me to
> generate the YAML format automatically from most distributions on PyPI, using
> their setup.py.
> 
> Here's a GIST with sample package.yaml files automatically generated from PyPI
> downloads of SQLAlchemy 0.7.8, Jinja2 2.6, Flask 0.9 and wheel 0.9.4.
> 
> https://gist.github.com/3803556
> 
> The JSON format of the metadata is actually appended as a comment on the last
> line of the YAML metadata (I use that to report YAML bugs).
> 
> I've not yet documented the schema for the metadata, as I'm still thinking
> about the details.
> 
> I ran my hack on around 18,000 PyPI releases (basically, all the latest
> releases which are hosted on PyPI). For all but around 1300, I was
> able to generate package.yaml files. The ones which failed are those where
> no setup.py is present, or it's present, but can't be imported because it
> assumes that some third-party package is available.
> 
> Comments welcome.
> 
> Regards,
> 
> Vinay Sajip
> 
> _______________________________________________
> Distutils-SIG maillist - Distutils-SIG at python.org (mailto:Distutils-SIG at python.org)
> http://mail.python.org/mailman/listinfo/distutils-sig
> 
> 


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20120930/89198bc9/attachment.html>

From dholth at gmail.com  Sun Sep 30 07:33:25 2012
From: dholth at gmail.com (Daniel Holth)
Date: Sun, 30 Sep 2012 01:33:25 -0400
Subject: [Distutils] [Python-Dev] distlib updated with resources API
In-Reply-To: <472D350E502F4B2FA46F3B6475281BE6@gmail.com>
References: <loom.20120923T174554-748@post.gmane.org>
	<50636845.8080204@ziade.org>
	<loom.20120926T233609-992@post.gmane.org>
	<50645C34.6070606@ziade.org>
	<1348756086.3495.1.camel@localhost.localdomain>
	<50646CFF.3010601@ziade.org>
	<loom.20120927T173348-617@post.gmane.org>
	<50647599.3030507@ziade.org>
	<loom.20120928T004648-800@post.gmane.org>
	<5064E31D.80601@ziade.org>
	<CADiSq7et7kT7NKSjNJ-D3oWAWcbNNF43=nqaj=VrCkThJoyYXA@mail.gmail.com>
	<loom.20120929T112825-998@post.gmane.org>
	<472D350E502F4B2FA46F3B6475281BE6@gmail.com>
Message-ID: <B6EF91EF-B385-4980-AE83-A6AEA68F7692@gmail.com>

Why not just let any line in pkg-info that starts with json/tag name: decode as json.

On Sep 30, 2012, at 1:03 AM, Donald Stufft <donald.stufft at gmail.com> wrote:

> I have some unpublished work I should publish.
> 
> Part of the point with what i'm trying to do is to define a standard for what is inside
> a package, but not really for how you take a particular set of files and turn it into
> that. So if you want to edit yaml you can have a yaml file and have a package
> creation tool that turns that yaml into the standard json that gets added to the
> package. The same can be said for a python file or a setup.cfg or whatever. Ideally
> the roles of package creation, building, and installation should be able to be completely
> separate. So my goal is to facilitate that by creating a standard way to describe
> all the data about a distribution, including extensible data in a way that any tool
> can serialize or deserialize it losslessly.
> 
> 
> On Saturday, September 29, 2012 at 5:54 AM, Vinay Sajip wrote:
> 
>> Nick Coghlan <ncoghlan <at> gmail.com> writes:
>> 
>> 
>>>> The document has changed since then,
>> http://python-notes.boredomandlaziness.org/en/latest/pep_ideas/core_packaging_api.html
>> 
>> I read from your page there that Donald Stufft is working on a JSON-based
>> metadata format. I've been looking that the same thing - a more flexible
>> metadata format which directly maps to dicts - but I used YAML, as looking at
>> too much JSON gives me eye-strain from all the extraneous quotes and braces.
>> 
>> I believe that JSON is the right format to use at the moment, because
>> PyYAML still has some bugs which I've run into while doing this work (also,
>> of course, it's not in the stdlib). As the formats are readily
>> interchangeable, there might be interest here in looking at the package.yaml
>> that I've come up with.
>> 
>> Since the metadata needs to support both the existing metadata and the
>> additional things that e.g. setuptools supports via additional kwargs to
>> setup(), I put together an ugly hack where I essentially mocked parts of
>> distutils and setuptools, including the setup() call. This allows me to
>> generate the YAML format automatically from most distributions on PyPI, using
>> their setup.py.
>> 
>> Here's a GIST with sample package.yaml files automatically generated from PyPI
>> downloads of SQLAlchemy 0.7.8, Jinja2 2.6, Flask 0.9 and wheel 0.9.4.
>> 
>> https://gist.github.com/3803556
>> 
>> The JSON format of the metadata is actually appended as a comment on the last
>> line of the YAML metadata (I use that to report YAML bugs).
>> 
>> I've not yet documented the schema for the metadata, as I'm still thinking
>> about the details.
>> 
>> I ran my hack on around 18,000 PyPI releases (basically, all the latest
>> releases which are hosted on PyPI). For all but around 1300, I was
>> able to generate package.yaml files. The ones which failed are those where
>> no setup.py is present, or it's present, but can't be imported because it
>> assumes that some third-party package is available.
>> 
>> Comments welcome.
>> 
>> Regards,
>> 
>> Vinay Sajip
>> 
>> _______________________________________________
>> Distutils-SIG maillist - Distutils-SIG at python.org
>> http://mail.python.org/mailman/listinfo/distutils-sig
> 
> _______________________________________________
> Distutils-SIG maillist  -  Distutils-SIG at python.org
> http://mail.python.org/mailman/listinfo/distutils-sig
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20120930/4d2a6a64/attachment-0001.html>

From donald.stufft at gmail.com  Sun Sep 30 08:07:07 2012
From: donald.stufft at gmail.com (Donald Stufft)
Date: Sun, 30 Sep 2012 02:07:07 -0400
Subject: [Distutils] [Python-Dev] distlib updated with resources API
In-Reply-To: <B6EF91EF-B385-4980-AE83-A6AEA68F7692@gmail.com>
References: <loom.20120923T174554-748@post.gmane.org>
	<50636845.8080204@ziade.org> <loom.20120926T233609-992@post.gmane.org>
	<50645C34.6070606@ziade.org>
	<1348756086.3495.1.camel@localhost.localdomain>
	<50646CFF.3010601@ziade.org> <loom.20120927T173348-617@post.gmane.org>
	<50647599.3030507@ziade.org> <loom.20120928T004648-800@post.gmane.org>
	<5064E31D.80601@ziade.org>
	<CADiSq7et7kT7NKSjNJ-D3oWAWcbNNF43=nqaj=VrCkThJoyYXA@mail.gmail.com>
	<loom.20120929T112825-998@post.gmane.org>
	<472D350E502F4B2FA46F3B6475281BE6@gmail.com>
	<B6EF91EF-B385-4980-AE83-A6AEA68F7692@gmail.com>
Message-ID: <675E0C6BE3F046E092A88103CD504D9A@gmail.com>

On Sunday, September 30, 2012 at 1:33 AM, Daniel Holth wrote:
> Why not just let any line in pkg-info that starts with json/tag name: decode as json.

Why use a format that requires custom parsing knowledge to create an internal Python representation. It cannot even accurately represent all of the Metadata 1.2 data. The Project Url feature is a good example. The parser has to know that in order to take "Docs, http://docs.com/" and turn it into {"Docs": "https://docs.com/"} that it has to split on the , and strip the whitespace. Now in the future we add another item that needs a namespace, but this one needs to have "," in it's value, so now for this tag we decide to use : instead of ,. Leads to inconsistent syntax that needs to be reimplemented by anyone who wants to parse the file.

The lack of power of the current encoding already causes issue needing to layer on custom bits, why not just use a standard serialization format that can properly serialize all of the values. 
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20120930/cd875b6e/attachment.html>

From dholth at gmail.com  Sun Sep 30 13:30:49 2012
From: dholth at gmail.com (Daniel Holth)
Date: Sun, 30 Sep 2012 07:30:49 -0400
Subject: [Distutils] [Python-Dev] distlib updated with resources API
In-Reply-To: <675E0C6BE3F046E092A88103CD504D9A@gmail.com>
References: <loom.20120923T174554-748@post.gmane.org>
	<50636845.8080204@ziade.org>
	<loom.20120926T233609-992@post.gmane.org>
	<50645C34.6070606@ziade.org>
	<1348756086.3495.1.camel@localhost.localdomain>
	<50646CFF.3010601@ziade.org>
	<loom.20120927T173348-617@post.gmane.org>
	<50647599.3030507@ziade.org>
	<loom.20120928T004648-800@post.gmane.org>
	<5064E31D.80601@ziade.org>
	<CADiSq7et7kT7NKSjNJ-D3oWAWcbNNF43=nqaj=VrCkThJoyYXA@mail.gmail.com>
	<loom.20120929T112825-998@post.gmane.org>
	<472D350E502F4B2FA46F3B6475281BE6@gmail.com>
	<B6EF91EF-B385-4980-AE83-A6AEA68F7692@gmail.com>
	<675E0C6BE3F046E092A88103CD504D9A@gmail.com>
Message-ID: <8C4B359E-75BD-46BB-B508-55C0F5BAACDB@gmail.com>

On Sep 30, 2012, at 2:07 AM, Donald Stufft <donald.stufft at gmail.com> wrote:

> On Sunday, September 30, 2012 at 1:33 AM, Daniel Holth wrote:
>> Why not just let any line in pkg-info that starts with json/tag name: decode as json.
> Why use a format that requires custom parsing knowledge to create an internal Python representation. It cannot even accurately represent all of the Metadata 1.2 data. The Project Url feature is a good example. The parser has to know that in order to take "Docs, http://docs.com/" and turn it into {"Docs": "https://docs.com/"} that it has to split on the , and strip the whitespace. Now in the future we add another item that needs a namespace, but this one needs to have "," in it's value, so now for this tag we decide to use : instead of ,. Leads to inconsistent syntax that needs to be reimplemented by anyone who wants to parse the file.
> 
> The lack of power of the current encoding already causes issue needing to layer on custom bits, why not just use a standard serialization format that can properly serialize all of the values. 

Ok
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20120930/3d8f0ff7/attachment.html>

From dholth at gmail.com  Sun Sep 30 14:56:49 2012
From: dholth at gmail.com (Daniel Holth)
Date: Sun, 30 Sep 2012 08:56:49 -0400
Subject: [Distutils] the 'wheel' binary package format
In-Reply-To: <502CF29D.7010802@egenix.com>
References: <CAG8k2+4YMgSDbjVsOY8J1qwkjoOdKLD28Gnp5RYDNMH8iesK5Q@mail.gmail.com>
	<502BC678.2040503@egenix.com>
	<CAG8k2+6FQ7DYOygt_YruC_UTjTLJ7881fkYRodbeRGeGUwyOkQ@mail.gmail.com>
	<502CF29D.7010802@egenix.com>
Message-ID: <CAG8k2+4JC__=Xs+8aXy3E2V8TTAGDp+XvRxCeyqgztVgaM60bw@mail.gmail.com>

Here is the reference implementation (so far) for PEP 425
"Compatibility tags...". It produces a list like so, intended to
express which similarly tagged binary distributions the current Python
can run.

import pep425tags, pprint
pprint.pprint(pep425tags.get_supported())

[('cp33', 'cp33m', 'linux_x86_64'),
 ('cp33', 'abi3', 'linux_x86_64'),
 ('cp33', 'none', 'linux_x86_64'),
 ('cp33', 'none', 'any'),
 ('cp3', 'none', 'any'),
 ('cp32', 'none', 'any'),
 ('cp31', 'none', 'any'),
 ('cp30', 'none', 'any'),
 ('py33', 'none', 'any'),
 ('py3', 'none', 'any'),
 ('py32', 'none', 'any'),
 ('py31', 'none', 'any'),
 ('py30', 'none', 'any')]

https://bitbucket.org/dholth/pep425tags/src/542429eaa92d/pep425tags.py

I would like to figure out what to do with abi3, have better freebsd
support, make sure OSX works, figure out whether the rules can make
sense in English as well as in Python.

Thanks,

Daniel

From cournape at gmail.com  Sun Sep 30 14:59:00 2012
From: cournape at gmail.com (David Cournapeau)
Date: Sun, 30 Sep 2012 13:59:00 +0100
Subject: [Distutils] [Python-Dev] distlib updated with resources API
In-Reply-To: <472D350E502F4B2FA46F3B6475281BE6@gmail.com>
References: <loom.20120923T174554-748@post.gmane.org>
	<50636845.8080204@ziade.org>
	<loom.20120926T233609-992@post.gmane.org>
	<50645C34.6070606@ziade.org>
	<1348756086.3495.1.camel@localhost.localdomain>
	<50646CFF.3010601@ziade.org>
	<loom.20120927T173348-617@post.gmane.org>
	<50647599.3030507@ziade.org>
	<loom.20120928T004648-800@post.gmane.org>
	<5064E31D.80601@ziade.org>
	<CADiSq7et7kT7NKSjNJ-D3oWAWcbNNF43=nqaj=VrCkThJoyYXA@mail.gmail.com>
	<loom.20120929T112825-998@post.gmane.org>
	<472D350E502F4B2FA46F3B6475281BE6@gmail.com>
Message-ID: <CAGY4rcUYA2t=JjauLbC5gkt49Lps+96wRWVejgs5COE0OQjytw@mail.gmail.com>

On Sun, Sep 30, 2012 at 6:03 AM, Donald Stufft <donald.stufft at gmail.com> wrote:
> I have some unpublished work I should publish.
>
> Part of the point with what i'm trying to do is to define a standard for
> what is inside
> a package, but not really for how you take a particular set of files and
> turn it into
> that. So if you want to edit yaml you can have a yaml file and have a
> package
> creation tool that turns that yaml into the standard json that gets added to
> the
> package. The same can be said for a python file or a setup.cfg or whatever.
> Ideally
> the roles of package creation, building, and installation should be able to
> be completely
> separate. So my goal is to facilitate that by creating a standard way to
> describe
> all the data about a distribution, including extensible data in a way that
> any tool
> can serialize or deserialize it losslessly.

Note that all this work has already been done in Bento.

I understand the appeal of using an existing format like yaml, but it
is not clear to me how one can handle conditional with it, and I think
you want to handle conditionals in there (for platform-dependent
dependencies).

Bento also has a command to convert setup.py-based projects to bento
internal format, adapting it to use another format should not be too
difficult.

David

From cournape at gmail.com  Sun Sep 30 15:04:21 2012
From: cournape at gmail.com (David Cournapeau)
Date: Sun, 30 Sep 2012 14:04:21 +0100
Subject: [Distutils] [Python-Dev] distlib updated with resources API
In-Reply-To: <CAG8k2+4anCPWpR4JNvLqvb=CF5-tMLqvmA1BJFRfSA2CLQ2_Ow@mail.gmail.com>
References: <loom.20120923T174554-748@post.gmane.org>
	<50636845.8080204@ziade.org>
	<loom.20120926T233609-992@post.gmane.org>
	<50645C34.6070606@ziade.org>
	<1348756086.3495.1.camel@localhost.localdomain>
	<50646CFF.3010601@ziade.org>
	<loom.20120927T173348-617@post.gmane.org>
	<50647599.3030507@ziade.org>
	<loom.20120928T004648-800@post.gmane.org>
	<5064E31D.80601@ziade.org>
	<CADiSq7et7kT7NKSjNJ-D3oWAWcbNNF43=nqaj=VrCkThJoyYXA@mail.gmail.com>
	<loom.20120929T112825-998@post.gmane.org>
	<CAG8k2+4anCPWpR4JNvLqvb=CF5-tMLqvmA1BJFRfSA2CLQ2_Ow@mail.gmail.com>
Message-ID: <CAGY4rcVfgt8shoHVTR-+bmVTYJFqHMw10TFZQjO8Sc4EACuZYA@mail.gmail.com>

On Sun, Sep 30, 2012 at 4:55 AM, Daniel Holth <dholth at gmail.com> wrote:
> I like this kind of study. Fixing 1300 packages sounds a lot more
> manageable than fixing 18,000. (I took a similar look at setup.py but
> with the ast module instead of actually running the things. Your
> method is probably more accurate.) It would be very cool to know how
> many packages use if: statements to affect install_requires...

Note that being able to convert a package does not mean the conversion
is working. You need to make sure that installing something from this
new format gives the same thing as installing from the setup.py.
That's harder to test, obviously.

>
> Is there a good "download the latest versions of everything hosted on
> pypi" script? Mine was pretty terrible as it could not resume after a
> crash or after the data got stale.

I would be interested in that as well, I wanted to do the same kind of
analysis for Bento's convert command.

David

From dholth at gmail.com  Sun Sep 30 16:19:14 2012
From: dholth at gmail.com (Daniel Holth)
Date: Sun, 30 Sep 2012 10:19:14 -0400
Subject: [Distutils] [Python-Dev] distlib updated with resources API
In-Reply-To: <CAGY4rcUYA2t=JjauLbC5gkt49Lps+96wRWVejgs5COE0OQjytw@mail.gmail.com>
References: <loom.20120923T174554-748@post.gmane.org>
	<50636845.8080204@ziade.org>
	<loom.20120926T233609-992@post.gmane.org>
	<50645C34.6070606@ziade.org>
	<1348756086.3495.1.camel@localhost.localdomain>
	<50646CFF.3010601@ziade.org>
	<loom.20120927T173348-617@post.gmane.org>
	<50647599.3030507@ziade.org>
	<loom.20120928T004648-800@post.gmane.org>
	<5064E31D.80601@ziade.org>
	<CADiSq7et7kT7NKSjNJ-D3oWAWcbNNF43=nqaj=VrCkThJoyYXA@mail.gmail.com>
	<loom.20120929T112825-998@post.gmane.org>
	<472D350E502F4B2FA46F3B6475281BE6@gmail.com>
	<CAGY4rcUYA2t=JjauLbC5gkt49Lps+96wRWVejgs5COE0OQjytw@mail.gmail.com>
Message-ID: <CAG8k2+7uOOF5WQc13eM0SuivMxV2TT19uZF=QC3fSoNKyeKOeQ@mail.gmail.com>

On Sun, Sep 30, 2012 at 8:59 AM, David Cournapeau <cournape at gmail.com> wrote:
> On Sun, Sep 30, 2012 at 6:03 AM, Donald Stufft <donald.stufft at gmail.com> wrote:
>> I have some unpublished work I should publish.
>>
>> Part of the point with what i'm trying to do is to define a standard for
>> what is inside
>> a package, but not really for how you take a particular set of files and
>> turn it into
>> that. So if you want to edit yaml you can have a yaml file and have a
>> package
>> creation tool that turns that yaml into the standard json that gets added to
>> the
>> package. The same can be said for a python file or a setup.cfg or whatever.
>> Ideally
>> the roles of package creation, building, and installation should be able to
>> be completely
>> separate. So my goal is to facilitate that by creating a standard way to
>> describe
>> all the data about a distribution, including extensible data in a way that
>> any tool
>> can serialize or deserialize it losslessly.
>
> Note that all this work has already been done in Bento.

I am a huge fan of Bento. I will be converting my packages to use it
as soon as is practical.

I would like to add the "where are things installed.py" generation to
the wheel standard somehow.

From vinay_sajip at yahoo.co.uk  Sun Sep 30 17:25:55 2012
From: vinay_sajip at yahoo.co.uk (Vinay Sajip)
Date: Sun, 30 Sep 2012 15:25:55 +0000 (UTC)
Subject: [Distutils] [Python-Dev] distlib updated with resources API
References: <loom.20120923T174554-748@post.gmane.org>
	<50636845.8080204@ziade.org>
	<loom.20120926T233609-992@post.gmane.org>
	<50645C34.6070606@ziade.org>
	<1348756086.3495.1.camel@localhost.localdomain>
	<50646CFF.3010601@ziade.org>
	<loom.20120927T173348-617@post.gmane.org>
	<50647599.3030507@ziade.org>
	<loom.20120928T004648-800@post.gmane.org>
	<5064E31D.80601@ziade.org>
	<CADiSq7et7kT7NKSjNJ-D3oWAWcbNNF43=nqaj=VrCkThJoyYXA@mail.gmail.com>
	<loom.20120929T112825-998@post.gmane.org>
	<CAG8k2+4anCPWpR4JNvLqvb=CF5-tMLqvmA1BJFRfSA2CLQ2_Ow@mail.gmail.com>
	<CAGY4rcVfgt8shoHVTR-+bmVTYJFqHMw10TFZQjO8Sc4EACuZYA@mail.gmail.com>
Message-ID: <loom.20120930T165808-259@post.gmane.org>

David Cournapeau <cournape <at> gmail.com> writes:

> Note that being able to convert a package does not mean the conversion
> is working. You need to make sure that installing something from this
> new format gives the same thing as installing from the setup.py.
> That's harder to test, obviously.

Right, and I've considered this. The most additional basic test I've done is to
do the equivalent of "python setup.py sdist" using *only* the package.yaml,
actually running "python setup.py sdist" and then comparing the two archives.

While I don't have the stats to hand, most of the 18,000 PyPI packages are such
that the comparison of archives shows no meaningful differences. Many of the
failures are due to custom code in setup.py, such as command classes.

My work so far also doesn't faithfully mock e.g. Cython or numpy, so that
calls to their customisations to distutils from setup.py aren't captured,
and so in those cases the resulting package.yaml is incomplete. However,
that's perhaps just a matter on spending some more time on the mocking
approach.

Obviously checking sdist is just a first step, but it's disappointing to
see how many packages on PyPI just fail the "download from PyPI, then
run python setup.py sdist" test because of e.g. importing packages which
don't exist.

To my mind, any source package on PyPI should be downloadable and be able to
have an sdist run on it to regenerate the archive, without needing any other
packages to be present: it's a source package already, right? But perhaps
that's what you get by allowing arbitrary code in setup.py, and eliminating
setup.py is definitely a step in the right direction.

> >
> > Is there a good "download the latest versions of everything hosted on
> > pypi" script? Mine was pretty terrible as it could not resume after a
> > crash or after the data got stale.
> 
> I would be interested in that as well, I wanted to do the same kind of
> analysis for Bento's convert command.
> 

Not as a standalone script that anyone can use, unfortunately. I don't have the
space to store all those downloads, and stuff on PyPI keeps getting updated,
anyway: so what I did was to run a first pass using the XML-RPC API to get
a list of package, version and archive URLs into a text file; my scripts then
pick up individual packages, download them and do the mocking/capture to
package.yaml, followed by the sdist comparison. I just use grep to filter the
archive list to determine which packages to process.

Every now and again I update that text file of archives and versions, see what
changed and arrange to run my code over updated and new packages. I just keep
the package.yaml files and the listings of the archives produced by the
distutils/setuptools dist operation and my package.yaml-using version.

Regards,

Vinay Sajip


From donald.stufft at gmail.com  Sun Sep 30 18:07:29 2012
From: donald.stufft at gmail.com (Donald Stufft)
Date: Sun, 30 Sep 2012 12:07:29 -0400
Subject: [Distutils] [Python-Dev] distlib updated with resources API
In-Reply-To: <CAGY4rcUYA2t=JjauLbC5gkt49Lps+96wRWVejgs5COE0OQjytw@mail.gmail.com>
References: <loom.20120923T174554-748@post.gmane.org>
	<50636845.8080204@ziade.org> <loom.20120926T233609-992@post.gmane.org>
	<50645C34.6070606@ziade.org>
	<1348756086.3495.1.camel@localhost.localdomain>
	<50646CFF.3010601@ziade.org> <loom.20120927T173348-617@post.gmane.org>
	<50647599.3030507@ziade.org> <loom.20120928T004648-800@post.gmane.org>
	<5064E31D.80601@ziade.org>
	<CADiSq7et7kT7NKSjNJ-D3oWAWcbNNF43=nqaj=VrCkThJoyYXA@mail.gmail.com>
	<loom.20120929T112825-998@post.gmane.org>
	<472D350E502F4B2FA46F3B6475281BE6@gmail.com>
	<CAGY4rcUYA2t=JjauLbC5gkt49Lps+96wRWVejgs5COE0OQjytw@mail.gmail.com>
Message-ID: <14DC3D7670F4434E970EDD724F1D7660@gmail.com>

On Sunday, September 30, 2012 at 8:59 AM, David Cournapeau wrote:
> Note that all this work has already been done in Bento.
> 
> I understand the appeal of using an existing format like yaml, but it
> is not clear to me how one can handle conditional with it, and I think
> you want to handle conditionals in there (for platform-dependent
> dependencies).
> 
> Bento also has a command to convert setup.py-based projects to bento
> internal format, adapting it to use another format should not be too
> difficult.
> 
> David 
Instead of conditionals the existing ideas use an environment marker, 
so instead of (pseudo format, I just woke up):

     if python_version < 2.6:
        require: simplejson

You would do:
    require: simplejson; python_version < 2.6

This gives you the same sort of ability however instead of using if statements
it encodes it into the requirement string. I'm not completely in love with either
system but I prefer the ; solution over conditionals because it makes the metadata
static no matter what system you run it on and it makes it easy to use an existing
format.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20120930/f5c46cfd/attachment.html>

From dholth at gmail.com  Sun Sep 30 18:38:54 2012
From: dholth at gmail.com (Daniel Holth)
Date: Sun, 30 Sep 2012 12:38:54 -0400
Subject: [Distutils] [Python-Dev] distlib updated with resources API
In-Reply-To: <14DC3D7670F4434E970EDD724F1D7660@gmail.com>
References: <loom.20120923T174554-748@post.gmane.org>
	<50636845.8080204@ziade.org>
	<loom.20120926T233609-992@post.gmane.org>
	<50645C34.6070606@ziade.org>
	<1348756086.3495.1.camel@localhost.localdomain>
	<50646CFF.3010601@ziade.org>
	<loom.20120927T173348-617@post.gmane.org>
	<50647599.3030507@ziade.org>
	<loom.20120928T004648-800@post.gmane.org>
	<5064E31D.80601@ziade.org>
	<CADiSq7et7kT7NKSjNJ-D3oWAWcbNNF43=nqaj=VrCkThJoyYXA@mail.gmail.com>
	<loom.20120929T112825-998@post.gmane.org>
	<472D350E502F4B2FA46F3B6475281BE6@gmail.com>
	<CAGY4rcUYA2t=JjauLbC5gkt49Lps+96wRWVejgs5COE0OQjytw@mail.gmail.com>
	<14DC3D7670F4434E970EDD724F1D7660@gmail.com>
Message-ID: <CAG8k2+7ZGxr2NgULNpssXykaC5-49m3jVQLU2jcQSz-dRAONMw@mail.gmail.com>

It's the same. Someone will write a bento conditionals to pep markers
compiler and it will be trivial.
On Sep 30, 2012 12:07 PM, "Donald Stufft" <donald.stufft at gmail.com> wrote:

> On Sunday, September 30, 2012 at 8:59 AM, David Cournapeau wrote:
>
> Note that all this work has already been done in Bento.
>
> I understand the appeal of using an existing format like yaml, but it
> is not clear to me how one can handle conditional with it, and I think
> you want to handle conditionals in there (for platform-dependent
> dependencies).
>
> Bento also has a command to convert setup.py-based projects to bento
> internal format, adapting it to use another format should not be too
> difficult.
>
> David
>
>  Instead of conditionals the existing ideas use an environment marker,
> so instead of (pseudo format, I just woke up):
>
>      if python_version < 2.6:
>         require: simplejson
>
> You would do:
>     require: simplejson; python_version < 2.6
>
> This gives you the same sort of ability however instead of using if
> statements
> it encodes it into the requirement string. I'm not completely in love with
> either
> system but I prefer the ; solution over conditionals because it makes the
> metadata
> static no matter what system you run it on and it makes it easy to use an
> existing
> format.
>
> _______________________________________________
> Distutils-SIG maillist  -  Distutils-SIG at python.org
> http://mail.python.org/mailman/listinfo/distutils-sig
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20120930/8b9440eb/attachment.html>

From donald.stufft at gmail.com  Sun Sep 30 18:48:28 2012
From: donald.stufft at gmail.com (Donald Stufft)
Date: Sun, 30 Sep 2012 12:48:28 -0400
Subject: [Distutils] [Python-Dev] distlib updated with resources API
In-Reply-To: <CAG8k2+7ZGxr2NgULNpssXykaC5-49m3jVQLU2jcQSz-dRAONMw@mail.gmail.com>
References: <loom.20120923T174554-748@post.gmane.org>
	<50636845.8080204@ziade.org> <loom.20120926T233609-992@post.gmane.org>
	<50645C34.6070606@ziade.org>
	<1348756086.3495.1.camel@localhost.localdomain>
	<50646CFF.3010601@ziade.org> <loom.20120927T173348-617@post.gmane.org>
	<50647599.3030507@ziade.org> <loom.20120928T004648-800@post.gmane.org>
	<5064E31D.80601@ziade.org>
	<CADiSq7et7kT7NKSjNJ-D3oWAWcbNNF43=nqaj=VrCkThJoyYXA@mail.gmail.com>
	<loom.20120929T112825-998@post.gmane.org>
	<472D350E502F4B2FA46F3B6475281BE6@gmail.com>
	<CAGY4rcUYA2t=JjauLbC5gkt49Lps+96wRWVejgs5COE0OQjytw@mail.gmail.com>
	<14DC3D7670F4434E970EDD724F1D7660@gmail.com>
	<CAG8k2+7ZGxr2NgULNpssXykaC5-49m3jVQLU2jcQSz-dRAONMw@mail.gmail.com>
Message-ID: <E2994CABE9484D579E907C9C18CDF2AF@gmail.com>

On Sunday, September 30, 2012 at 12:38 PM, Daniel Holth wrote:
> It's the same. Someone will write a bento conditionals to pep markers compiler and it will be trivial. 
> 
> 

Right same concept, the difference being one requires a specialized parser and the other uses a standard parser available in every language. 
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20120930/b8d81ff1/attachment.html>