From steve at pearwood.info  Sun Nov  1 12:21:15 2009
From: steve at pearwood.info (Steven D'Aprano)
Date: Sun, 1 Nov 2009 22:21:15 +1100
Subject: [Python-Dev] Retrieve an arbitrary element from a
	setwithoutremoving it
In-Reply-To: <874opgi8rd.fsf@uwakimon.sk.tsukuba.ac.jp>
References: <200910231132.45504.w.richert@gmx.net>
	<200910310211.33590.steve@pearwood.info>
	<874opgi8rd.fsf@uwakimon.sk.tsukuba.ac.jp>
Message-ID: <200911012221.16289.steve@pearwood.info>

It seems that even those originally asking for set retrieval have gone 
silent, so I guess this isn't going anywhere.

However, for the benefit of future discussions (because I'm sure this 
question will be raised again), I'd like to answer a couple of points 
raised by Stephen.

On Sat, 31 Oct 2009 01:42:46 pm Stephen J. Turnbull wrote:
> ?> If folks prefer a different name, by all means suggest it. I like
> ?> the name suggested by Wikipedia, "pick".
>
> "Pick" has a connotation of removal in many contexts: "Pick a card,
> any card". ?

And in just as many, there is no connotation of removal. "Pick a 
colour".

For what it's worth, Forth and similar stack-based languages usually 
have a function "pick" equivalent to pop-without-removal. pick seems to 
be a standard term for this behaviour.


> Like get, to me it has a flavor of 
> "according to some criterion": "a band of picked men". ?I would 
> expect a pick or get operation to take an optional predicate. 

I think you are underestimating the power of context here. In practice, 
method names are interpreted in the context of the data being operated 
on. We don't get confused that ConfigParser.get() has a different 
signature to dict.get(), which is different again from Queue.get().

list.pop() takes an index; dict.pop() takes a key and an optional second 
argument; set.pop() takes no argument at all. Is anyone actually 
confused by this?

If not, why would set.get() be more confusing? I think far too many 
people say "this is confusing" when what they really mean is "I don't 
like this".


> The use case I have so far
> observed people to have in mind is a cast from an equivalence class
> to a representative member.

The difficulty is that we actually have two related, but different, 
behaviours, and sadly we've conflated the two of them by using the same 
name "get" for both.

One behaviour is what Wikipedia calls pick:

set.pick() -> return an arbitrary element from set without removing it

This is not useful for the cast you describe. For that, you need a 
method that takes an argument. I'm going to call it "retrieve", to 
avoid the baggage of "get":

set.retrieve(x) -> return the element from set equal to x


In the first case, it seems self-evident to me that having "arbitrary" 
mean "the same element each time it is called" really would condemn 
pick() to be unused, but I don't care enough to argue.

In the second case, the retrieval isn't arbitrary, so this is not a 
problem that needs solving.


> ?> No. Since sets are unordered, there's no way to seek to a specific
> ?> element.
>
> I think people will realize that in fact *these* sets are ordered and
> they will demand a seek function for various practical purposes.

We can iterate over dicts, and the order is arbitrary but consistent. 
Where are the people clamouring for dict.seek()?




-- 
Steven D'Aprano

From solipsis at pitrou.net  Sun Nov  1 12:33:19 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Sun, 1 Nov 2009 11:33:19 +0000 (UTC)
Subject: [Python-Dev] Reworking the GIL
References: <1256502140.5621.163.camel@localhost>
	<bbaeab100910251457t64616f62x3216ebb4863223f5@mail.gmail.com>
Message-ID: <loom.20091101T123110-331@post.gmane.org>


Hello again,

Brett Cannon <brett <at> python.org> writes:
> 
> I think it's worth it. Removal of the GIL is a totally open-ended problem 
> with no solution in sight. This, on the other hand, is a performance benefit 
> now. I say move forward with this. If it happens to be short-lived because 
> some actually figures out how to remove the GIL then great, but is that 
> really going to happen between now and Python 3.2? I doubt it.

Based on this whole discussion, I think I am going to merge the new GIL work
into the py3k branch, with priority requests disabled.

If you think this is premature or uncalled for, or if you just want to review
the changes before making a judgement, please voice up :)

Regards

Antoine.



From w.richert at gmx.net  Sun Nov  1 14:55:30 2009
From: w.richert at gmx.net (Willi Richert)
Date: Sun, 1 Nov 2009 14:55:30 +0100
Subject: [Python-Dev] Retrieve an arbitrary element from a
	setwithoutremoving it
In-Reply-To: <200911012221.16289.steve@pearwood.info>
References: <200910231132.45504.w.richert@gmx.net>
	<874opgi8rd.fsf@uwakimon.sk.tsukuba.ac.jp>
	<200911012221.16289.steve@pearwood.info>
Message-ID: <200911011455.30615.w.richert@gmx.net>

Am Sonntag, 1. November 2009 12:21:15 schrieben Sie:
> It seems that even those originally asking for set retrieval have gone 
> silent

Nope. Stilll following and waiting for the verdict of the community after 
having filed the corpus delicti [1] 

wr

[1]: http://bugs.python.org/issue7212


From solipsis at pitrou.net  Sun Nov  1 16:23:19 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Sun, 1 Nov 2009 15:23:19 +0000 (UTC)
Subject: [Python-Dev] Bizarre mtime behaviour
Message-ID: <loom.20091101T161805-983@post.gmane.org>

Hello,

I wondered if someone had a clue about the following behaviour.
While debugging an erratic test_mailbox failure on RDM's buildbot (and other
machines), it turned out that the system sometimes set the wrong mtime on a
directory:

$ date && python -c 'import os; os.link("setup.py", "t/c")' && stat t && date

Sun Nov  1 09:49:04 EST 2009
  File: `t'
  Size: 144             Blocks: 0          IO Block: 4096   directory
Device: 811h/2065d      Inode: 223152      Links: 2
Access: (0755/drwxr-xr-x)  Uid: ( 1001/  pitrou)   Gid: ( 1005/  pitrou)
Access: 2009-11-01 09:10:11.000000000 -0500
Modify: 2009-11-01 09:49:03.000000000 -0500
Change: 2009-11-01 09:49:03.000000000 -0500
Sun Nov  1 09:49:04 EST 2009

As you see above, the mtime for directory 't' is set to a full second before the
actual modification has happened.

Sprinkling traces of time.time() and os.path.getmtime() on Lib/mailbox.py shows
this is exactly what trips up test_mailbox. I've got posted a patch to fix it
(see issue #6896), but I would like to know if such OS behaviour is normal.

Regards

Antoine.



From rhamph at gmail.com  Sun Nov  1 17:25:39 2009
From: rhamph at gmail.com (Adam Olsen)
Date: Sun, 1 Nov 2009 09:25:39 -0700
Subject: [Python-Dev] Bizarre mtime behaviour
In-Reply-To: <loom.20091101T161805-983@post.gmane.org>
References: <loom.20091101T161805-983@post.gmane.org>
Message-ID: <aac2c7cb0911010825y10651428v4fd192b63b7020e8@mail.gmail.com>

On Sun, Nov 1, 2009 at 08:23, Antoine Pitrou <solipsis at pitrou.net> wrote:
> Hello,
>
> I wondered if someone had a clue about the following behaviour.
> While debugging an erratic test_mailbox failure on RDM's buildbot (and other
> machines), it turned out that the system sometimes set the wrong mtime on a
> directory:
>
> $ date && python -c 'import os; os.link("setup.py", "t/c")' && stat t && date
>
> Sun Nov ?1 09:49:04 EST 2009
> ?File: `t'
> ?Size: 144 ? ? ? ? ? ? Blocks: 0 ? ? ? ? ?IO Block: 4096 ? directory
> Device: 811h/2065d ? ? ?Inode: 223152 ? ? ?Links: 2
> Access: (0755/drwxr-xr-x) ?Uid: ( 1001/ ?pitrou) ? Gid: ( 1005/ ?pitrou)
> Access: 2009-11-01 09:10:11.000000000 -0500
> Modify: 2009-11-01 09:49:03.000000000 -0500
> Change: 2009-11-01 09:49:03.000000000 -0500
> Sun Nov ?1 09:49:04 EST 2009
>
> As you see above, the mtime for directory 't' is set to a full second before the
> actual modification has happened.
>
> Sprinkling traces of time.time() and os.path.getmtime() on Lib/mailbox.py shows
> this is exactly what trips up test_mailbox. I've got posted a patch to fix it
> (see issue #6896), but I would like to know if such OS behaviour is normal.

Looks like an OS bug to me.  Linux I'm guessing?


-- 
Adam Olsen, aka Rhamphoryncus

From solipsis at pitrou.net  Sun Nov  1 17:29:11 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Sun, 1 Nov 2009 16:29:11 +0000 (UTC)
Subject: [Python-Dev] Bizarre mtime behaviour
References: <loom.20091101T161805-983@post.gmane.org>
	<aac2c7cb0911010825y10651428v4fd192b63b7020e8@mail.gmail.com>
Message-ID: <loom.20091101T172820-503@post.gmane.org>

Adam Olsen <rhamph <at> gmail.com> writes:
> 
> Looks like an OS bug to me.  Linux I'm guessing?

Yes, but only on certain boxes. I could never reproduce on my home box.
RDM (David)'s buildbot is a Gentoo vserver with a reiserfs filesystem.



From eric at trueblade.com  Sun Nov  1 18:03:36 2009
From: eric at trueblade.com (Eric Smith)
Date: Sun, 01 Nov 2009 12:03:36 -0500
Subject: [Python-Dev] Bizarre mtime behaviour
In-Reply-To: <loom.20091101T172820-503@post.gmane.org>
References: <loom.20091101T161805-983@post.gmane.org>	<aac2c7cb0911010825y10651428v4fd192b63b7020e8@mail.gmail.com>
	<loom.20091101T172820-503@post.gmane.org>
Message-ID: <4AEDBF68.2060902@trueblade.com>

Antoine Pitrou wrote:
> Adam Olsen <rhamph <at> gmail.com> writes:
>> Looks like an OS bug to me.  Linux I'm guessing?
> 
> Yes, but only on certain boxes. I could never reproduce on my home box.
> RDM (David)'s buildbot is a Gentoo vserver with a reiserfs filesystem.

You'll occasionally see something similar on Windows boxes running FAT, 
because the timestamps stored in the filesystem have a 2 second 
granularity there. Not sure if this might be similar, or just a reiserfs 
issue.

Eric.

From exarkun at twistedmatrix.com  Sun Nov  1 18:54:32 2009
From: exarkun at twistedmatrix.com (exarkun at twistedmatrix.com)
Date: Sun, 01 Nov 2009 17:54:32 -0000
Subject: [Python-Dev] EC2 buildslaves
In-Reply-To: <loom.20091031T210845-748@post.gmane.org>
References: <5c6f2a5d0910250154nfcdabafyc051389c956ef258@mail.gmail.com>
	<4AE41E9E.4050706@v.loewis.de>
	<20091025140512.11571.1513702964.divmod.xquotient.1543@localhost.localdomain>
	<hcf3pi$vgs$1@ger.gmane.org> <4AEB6C06.3070907@v.loewis.de>
	<loom.20091031T210845-748@post.gmane.org>
Message-ID: <20091101175432.12668.1205553571.divmod.xquotient.130@localhost.localdomain>

On 31 Oct, 08:13 pm, solipsis at pitrou.net wrote:
>Martin v. L?wis <martin <at> v.loewis.de> writes:
>>
>>Not sure whether it's still relevant after the offers of individually
>>donated hardware.
>
>We'll see, indeed.
>>However, if you want to look into this, feel free to
>>set up EC2 slaves.
>
>I only know to setup mainstream Linux distros though (Debian- or
>Redhat-lookalikes :-)). I've just played a bit and, after the hassle of 
>juggling
>with a bunch of different keys and credentials, setting up an instance 
>and
>saving an image for future use is quite easy.

Starting with a mainstream distro doesn't seem like a bad idea.  For 
example, there isn't currently a 32bit Ubuntu (any version) slave.  That 
would be a nice gap to fill in, right?

Jean-Paul

From robertc at robertcollins.net  Sun Nov  1 22:02:32 2009
From: robertc at robertcollins.net (Robert Collins)
Date: Mon, 02 Nov 2009 08:02:32 +1100
Subject: [Python-Dev] Bizarre mtime behaviour
In-Reply-To: <4AEDBF68.2060902@trueblade.com>
References: <loom.20091101T161805-983@post.gmane.org>
	<aac2c7cb0911010825y10651428v4fd192b63b7020e8@mail.gmail.com>
	<loom.20091101T172820-503@post.gmane.org>
	<4AEDBF68.2060902@trueblade.com>
Message-ID: <1257109353.1235.23.camel@lifeless-64>

On Sun, 2009-11-01 at 12:03 -0500, Eric Smith wrote:
> Antoine Pitrou wrote:
> > Adam Olsen <rhamph <at> gmail.com> writes:
> >> Looks like an OS bug to me.  Linux I'm guessing?
> > 
> > Yes, but only on certain boxes. I could never reproduce on my home box.
> > RDM (David)'s buildbot is a Gentoo vserver with a reiserfs filesystem.
> 
> You'll occasionally see something similar on Windows boxes running FAT, 
> because the timestamps stored in the filesystem have a 2 second 
> granularity there. Not sure if this might be similar, or just a reiserfs 
> issue.

Some years ago there was a bug where the dentry cache held timestamps
more granular than those that the filesystem could represent. We saw
this cause build failures which were troubling to reproduce :) I haven't
seen Linux arbitrarily invent time travel so far:).

The FAT rounding issue is a possibility, but I didn't think reiserfs was
short that much precision.

I'd check that the work area you had really was reiser, not a mounted AT
partition, and if its not look up the ReiserFS precision for mtime.

-Rob
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 197 bytes
Desc: This is a digitally signed message part
URL: <http://mail.python.org/pipermail/python-dev/attachments/20091102/c7b81020/attachment.pgp>

From solipsis at pitrou.net  Sun Nov  1 22:08:14 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Sun, 01 Nov 2009 22:08:14 +0100
Subject: [Python-Dev] Bizarre mtime behaviour
In-Reply-To: <1257109353.1235.23.camel@lifeless-64>
References: <loom.20091101T161805-983@post.gmane.org>
	<aac2c7cb0911010825y10651428v4fd192b63b7020e8@mail.gmail.com>
	<loom.20091101T172820-503@post.gmane.org>
	<4AEDBF68.2060902@trueblade.com>
	<1257109353.1235.23.camel@lifeless-64>
Message-ID: <1257109694.30946.7.camel@localhost>

Le lundi 02 novembre 2009 ? 08:02 +1100, Robert Collins a ?crit :
> 
> The FAT rounding issue is a possibility, but I didn't think reiserfs was
> short that much precision.
> 
> I'd check that the work area you had really was reiser, not a mounted AT
> partition, and if its not look up the ReiserFS precision for mtime.

Yes it was reiserfs. As for the mtime granularity, since some displayed
mtimes were even and some were odd, I assume it's one second or less.

Another possibility would be that the filesystem would use a slightly
different clock than time.time(), but why?



From brett at python.org  Sun Nov  1 22:17:24 2009
From: brett at python.org (Brett Cannon)
Date: Sun, 1 Nov 2009 13:17:24 -0800
Subject: [Python-Dev] Reworking the GIL
In-Reply-To: <loom.20091101T123110-331@post.gmane.org>
References: <1256502140.5621.163.camel@localhost>
	<bbaeab100910251457t64616f62x3216ebb4863223f5@mail.gmail.com> 
	<loom.20091101T123110-331@post.gmane.org>
Message-ID: <bbaeab100911011317o6a849bf9o23f264203ea4d7e7@mail.gmail.com>

On Sun, Nov 1, 2009 at 03:33, Antoine Pitrou <solipsis at pitrou.net> wrote:
>
> Hello again,
>
> Brett Cannon <brett <at> python.org> writes:
>>
>> I think it's worth it. Removal of the GIL is a totally open-ended problem
>> with no solution in sight. This, on the other hand, is a performance benefit
>> now. I say move forward with this. If it happens to be short-lived because
>> some actually figures out how to remove the GIL then great, but is that
>> really going to happen between now and Python 3.2? I doubt it.
>
> Based on this whole discussion, I think I am going to merge the new GIL work
> into the py3k branch, with priority requests disabled.

This will be a nice Py3K carrot!

>
> If you think this is premature or uncalled for, or if you just want to review
> the changes before making a judgement, please voice up :)

I know I personally trust you to not mess it up, Antoine, but that
might also come from mental exhaustion and laziness. =)

-Brett

From regebro at gmail.com  Sun Nov  1 22:26:58 2009
From: regebro at gmail.com (Lennart Regebro)
Date: Sun, 1 Nov 2009 22:26:58 +0100
Subject: [Python-Dev] nonlocal keyword in 2.x?
In-Reply-To: <4AE95F18.4040807@gmail.com>
References: <da7032ce0910211856g45e45632n550aa5b4a10f9a62@mail.gmail.com>
	<4AE0BF83.4080903@v.loewis.de>
	<20091022204621.11571.1616253771.divmod.xquotient.1454@localhost.localdomain>
	<4AE0C73C.9070805@v.loewis.de>
	<319e029f0910272330me6847d4i5f9a4edb8f9359fc@mail.gmail.com>
	<loom.20091028T101119-295@post.gmane.org>
	<19176.18618.662587.613771@montanaro.dyndns.org>
	<loom.20091028T154811-994@post.gmane.org>
	<319e029f0910290115y49be844co78da6636452445f@mail.gmail.com>
	<4AE95F18.4040807@gmail.com>
Message-ID: <319e029f0911011326i45bf52ew35b9d7e438df6570@mail.gmail.com>

2009/10/29 Nick Coghlan <ncoghlan at gmail.com>:
> Lennart Regebro wrote:
>> 2009/10/28 Antoine Pitrou <solipsis at pitrou.net>:
>>> <skip <at> pobox.com> writes:
>>>> ? ? >> So 2.7 support will for the most part be a case not of supporting
>>>> ? ? >> Python versions, but Python *users*.
>>>>
>>>> ? ? Antoine> That's still not a good reason to backport nonlocal. The same
>>>> ? ? Antoine> reasoning could be used to backport new features to the 2.6
>>>> ? ? Antoine> branch after all.
>>>>
>>>> No, because 2.6 is in feature freeze (bug fixes only). ?2.7 is the current
>>>> version of 2.x where new features are allowed to be added.
>>> That was precisely my point.
>>
>> Then I don't understand what you are saying. Obviously we shouldn't
>> backport to the 2.6 branch, it's in bugfix mode. This is about 2.7. I
>> don't see what 2.6 has to do with it.
>>
>>> There are development practices which mitigate the
>>> idea that backporting is always helpful to the user.
>>
>> And those are?
>
> You said it above yourself: "bugfix mode"

Again: Then I don't understand what you are saying. Obviously we
shouldn't backport to the 2.6 branch, it's in bugfix mode. This is
about 2.7. I don't see what 2.6 has to do with it.

> That's all Antoine's point was - backporting of new features to previous
> branches is not automatically a good idea.

Obviously we shouldn't backport to the 2.6 branch, it's in bugfix
mode. This is about 2.7. I don't see what 2.6 has to do with it.

> In the case of 3.2 -> 2.7 backports, there are issues with the initial
> development time investment to do the backport, future double-keying of
> additional maintenance issues, consideration of possible poor
> interaction with legacy features in the 2.x series. It's a bunch of
> additional work that isn't going to happen without someone volunteering
> to do it.

Yes?

I feel that you are saying things that are obvious, and that you then
expect me to draw a conclusion from this, that you probably find
obvious too, but I don't. So I still don't understand what you are
saying, or how this in any way contradicts what I said, or clarified
or expanded on the matter.


As I said: Python 2 support is not only about supporting old versions of Python,
but also supporting users of Python2-only modules. So 2.7 support will
for the most part be a case not of supporting Python versions, but
Python *users*. And contrary to what Antoine said, that *is* a good
reason to backport it.

-- 
Lennart Regebro: Python, Zope, Plone, Grok
http://regebro.wordpress.com/
+33 661 58 14 64

From python at rcn.com  Sun Nov  1 22:39:36 2009
From: python at rcn.com (Raymond Hettinger)
Date: Sun, 1 Nov 2009 13:39:36 -0800
Subject: [Python-Dev] nonlocal keyword in 2.x?
References: <da7032ce0910211856g45e45632n550aa5b4a10f9a62@mail.gmail.com><4AE0BF83.4080903@v.loewis.de><20091022204621.11571.1616253771.divmod.xquotient.1454@localhost.localdomain><4AE0C73C.9070805@v.loewis.de><319e029f0910272330me6847d4i5f9a4edb8f9359fc@mail.gmail.com><loom.20091028T101119-295@post.gmane.org><19176.18618.662587.613771@montanaro.dyndns.org><loom.20091028T154811-994@post.gmane.org><319e029f0910290115y49be844co78da6636452445f@mail.gmail.com><4AE95F18.4040807@gmail.com>
	<319e029f0911011326i45bf52ew35b9d7e438df6570@mail.gmail.com>
Message-ID: <02A812F3B69E46D180B45133D52D1F2B@RaymondLaptop1>

> As I said: Python 2 support is not only about supporting old versions of Python,
> but also supporting users of Python2-only modules. So 2.7 support will
> for the most part be a case not of supporting Python versions, but
> Python *users*. And contrary to what Antoine said, that *is* a good
> reason to backport it.

FWIW, I support backporting the nonlocal-keyword in 2.7.
All of the reasons for introducting nonlocal to 3.x also apply to 2.x.
Using the nonlocal keyword in clear and explicit, especially
when compared to the existing workarounds which are not pretty.


Raymond

From lists at cheimes.de  Sun Nov  1 23:12:16 2009
From: lists at cheimes.de (Christian Heimes)
Date: Sun, 01 Nov 2009 23:12:16 +0100
Subject: [Python-Dev] Reworking the GIL
In-Reply-To: <loom.20091101T123110-331@post.gmane.org>
References: <1256502140.5621.163.camel@localhost>	<bbaeab100910251457t64616f62x3216ebb4863223f5@mail.gmail.com>
	<loom.20091101T123110-331@post.gmane.org>
Message-ID: <4AEE07C0.5090006@cheimes.de>

Antoine Pitrou wrote:
> Based on this whole discussion, I think I am going to merge the new GIL work
> into the py3k branch, with priority requests disabled.
> 
> If you think this is premature or uncalled for, or if you just want to review
> the changes before making a judgement, please voice up :)

+1 from me. I trust you like Brett does.

How much work would it cost to make your patch optional at compile time?
For what it's worth we could compare your work on different machines and
on different platforms before it gets enabled by default. Can you
imagine scenarios where your implementation might be slower than the
current GIL implementation?


From solipsis at pitrou.net  Sun Nov  1 23:27:35 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Sun, 1 Nov 2009 22:27:35 +0000 (UTC)
Subject: [Python-Dev] Reworking the GIL
References: <1256502140.5621.163.camel@localhost>	<bbaeab100910251457t64616f62x3216ebb4863223f5@mail.gmail.com>
	<loom.20091101T123110-331@post.gmane.org>
	<4AEE07C0.5090006@cheimes.de>
Message-ID: <loom.20091101T232101-655@post.gmane.org>

Christian Heimes <lists <at> cheimes.de> writes:
> 
> +1 from me. I trust you like Brett does.
> 
> How much work would it cost to make your patch optional at compile time?

Quite a bit, because it changes the logic for processing asynchronous pending
calls (signals) and asynchronous exceptions in the eval loop. The #defines would
get quite convoluted, I think; I'd prefer not to do that.

> For what it's worth we could compare your work on different machines and
> on different platforms before it gets enabled by default. Can you
> imagine scenarios where your implementation might be slower than the
> current GIL implementation?

I don't really think so. The GIL is taken and released much more predictably
than it was before. The thing that might be worth checking is a workload with
many threads (say 50 or 100). Does anyone have that?

Regards

Antoine.



From lists at cheimes.de  Sun Nov  1 23:43:50 2009
From: lists at cheimes.de (Christian Heimes)
Date: Sun, 01 Nov 2009 23:43:50 +0100
Subject: [Python-Dev] Reworking the GIL
In-Reply-To: <loom.20091101T232101-655@post.gmane.org>
References: <1256502140.5621.163.camel@localhost>	<bbaeab100910251457t64616f62x3216ebb4863223f5@mail.gmail.com>	<loom.20091101T123110-331@post.gmane.org>	<4AEE07C0.5090006@cheimes.de>
	<loom.20091101T232101-655@post.gmane.org>
Message-ID: <4AEE0F26.2000209@cheimes.de>

Antoine Pitrou wrote:
> Christian Heimes <lists <at> cheimes.de> writes:
>> +1 from me. I trust you like Brett does.
>>
>> How much work would it cost to make your patch optional at compile time?
> 
> Quite a bit, because it changes the logic for processing asynchronous pending
> calls (signals) and asynchronous exceptions in the eval loop. The #defines would
> get quite convoluted, I think; I'd prefer not to do that.

Based on the new piece of information I totally agree.

> I don't really think so. The GIL is taken and released much more predictably
> than it was before. The thing that might be worth checking is a workload with
> many threads (say 50 or 100). Does anyone have that?

I don't have an application that works on Python 3 and uses that many
threads, sorry.

Christian

From brett at python.org  Mon Nov  2 00:57:21 2009
From: brett at python.org (Brett Cannon)
Date: Sun, 1 Nov 2009 15:57:21 -0800
Subject: [Python-Dev] nonlocal keyword in 2.x?
In-Reply-To: <02A812F3B69E46D180B45133D52D1F2B@RaymondLaptop1>
References: <da7032ce0910211856g45e45632n550aa5b4a10f9a62@mail.gmail.com> 
	<4AE0C73C.9070805@v.loewis.de>
	<319e029f0910272330me6847d4i5f9a4edb8f9359fc@mail.gmail.com> 
	<loom.20091028T101119-295@post.gmane.org>
	<19176.18618.662587.613771@montanaro.dyndns.org> 
	<loom.20091028T154811-994@post.gmane.org>
	<319e029f0910290115y49be844co78da6636452445f@mail.gmail.com> 
	<4AE95F18.4040807@gmail.com>
	<319e029f0911011326i45bf52ew35b9d7e438df6570@mail.gmail.com> 
	<02A812F3B69E46D180B45133D52D1F2B@RaymondLaptop1>
Message-ID: <bbaeab100911011557t714e3b4cxcdc91b3fb9a49b84@mail.gmail.com>

On Sun, Nov 1, 2009 at 13:39, Raymond Hettinger <python at rcn.com> wrote:
>> As I said: Python 2 support is not only about supporting old versions of
>> Python,
>> but also supporting users of Python2-only modules. So 2.7 support will
>> for the most part be a case not of supporting Python versions, but
>> Python *users*. And contrary to what Antoine said, that *is* a good
>> reason to backport it.
>
> FWIW, I support backporting the nonlocal-keyword in 2.7.
> All of the reasons for introducting nonlocal to 3.x also apply to 2.x.
> Using the nonlocal keyword in clear and explicit, especially
> when compared to the existing workarounds which are not pretty.

We are voting, I'm -0.

-Brett

From sturla at molden.no  Mon Nov  2 04:22:52 2009
From: sturla at molden.no (Sturla Molden)
Date: Mon, 02 Nov 2009 04:22:52 +0100
Subject: [Python-Dev] Integer behaviour in Python 2.6.4
Message-ID: <4AEE508C.1070807@molden.no>


Why does this happen?

 >>> type(2**31-1)
<type 'long'>

It seems to have broken NumPy's RNG on Win32.







From curt at hagenlocher.org  Mon Nov  2 04:49:18 2009
From: curt at hagenlocher.org (Curt Hagenlocher)
Date: Sun, 1 Nov 2009 20:49:18 -0700
Subject: [Python-Dev] Integer behaviour in Python 2.6.4
In-Reply-To: <4AEE508C.1070807@molden.no>
References: <4AEE508C.1070807@molden.no>
Message-ID: <d2155e360911011949w58f27308lfa6329535d280485@mail.gmail.com>

On Sun, Nov 1, 2009 at 8:22 PM, Sturla Molden <sturla at molden.no> wrote:
>
> Why does this happen?
>
> >>> type(2**31-1)
> <type 'long'>

Does that not happen on non-Windows platforms? 2**31 can't be
represented as a 32-bit signed integer, so it's automatically promoted
to a long.

--
Curt Hagenlocher
curt at hagenlocher.org

From sturla at molden.no  Mon Nov  2 05:21:06 2009
From: sturla at molden.no (Sturla Molden)
Date: Mon, 02 Nov 2009 05:21:06 +0100
Subject: [Python-Dev] Integer behaviour in Python 2.6.4
In-Reply-To: <d2155e360911011949w58f27308lfa6329535d280485@mail.gmail.com>
References: <4AEE508C.1070807@molden.no>
	<d2155e360911011949w58f27308lfa6329535d280485@mail.gmail.com>
Message-ID: <4AEE5E32.1090506@molden.no>

Curt Hagenlocher skrev:
> Does that not happen on non-Windows platforms? 2**31 can't be
> represented as a 32-bit signed integer, so it's automatically promoted
> to a long.
>
>   
Yes you are right.

I've now traced down the problem to an integer overflow in NumPy.

It seems to have this Pyrex code:

cdef long lo, hi, diff
[...]
diff = hi - lo - 1

:-D


Sturla



From cs at zip.com.au  Mon Nov  2 06:29:00 2009
From: cs at zip.com.au (Cameron Simpson)
Date: Mon, 2 Nov 2009 16:29:00 +1100
Subject: [Python-Dev] Retrieve an arbitrary element from a set
	withoutremoving it
In-Reply-To: <7b0045ed0910301843y7059d316s8a851142b78b7aeb@mail.gmail.com>
References: <7b0045ed0910301843y7059d316s8a851142b78b7aeb@mail.gmail.com>
Message-ID: <20091102052900.GA25160@cskk.homeip.net>

On 30Oct2009 20:43, Chris Bergstresser <chris at subtlety.com> wrote:
| On Fri, Oct 30, 2009 at 8:29 PM, Steven D'Aprano <steve at pearwood.info> wrote:
| >> > Iterating over an iterable is
| >> > what iterators are for.
| >
| > set.get(), or set.pick() as Wikipedia calls it, isn't for iterating over
| > sets. It is for getting an arbitrary element from the set.
[...]
| > The purpose is to
| > return an arbitrary item each time it is called. If two threads get the
| > same element, that's not a problem; if one thread misses an element
| > because another thread grabbed it first, that's not a problem either.
| > If people prefer a random element instead, I have no problem with
| > that -- personally I think that's overkill, but maybe that's just me.
| 
|    I also think returning a random one is overkill, in the base set.
| And I'd have the base implementation do the simplest thing possible:
| return a fixed element (either the first returned when iterated over,
| or the last stored, or whatever's easiest based on the internals).
| For me, leaving the choice of *which* element to return on each call
| is a feature.  It allows subclasses to change the behavior to support
| other use cases, like a random or round-robin behavior.

Personally, I'm for the iteration spec in a lot of ways.

Firstly, a .get()/.pick() that always returns the same element feels
horrible. Is there anyone here who _likes_ it?

Secondly, I think the thread-unsafe arguments are weak. Question: is the
existing set iteration scheme thread safe? i.e. does it return a fresh
iterator that a thread can happily use concurrently while another thread
uses its own iterator?  (Absent set modifications). If the answer is yes,
then I don't buy the thread-unsafe complaints - there are already plenty
of thread unsafe things much as lots of iterators will break in the face
of changes to the data structures over which they're iterating. If
iter(set) gets you a safe iterator (absent set modification and multiple
users of that iterator) then thread safe methods exist and are easy to
use. No presently thread-safe program is going to be broken by adding an
iterating .get() method.

Thirdly, an iteration-like .get() is dead easy to provide: you just keep a
_single_, cycling, internal iterator made on demand the same way __iter__
already works. So why not do just do it? There's no need to construct it
before anyone calls .get() the first time. Somewhat like:

  def get(self):
    for x in self:
      return x
    raise something here

but not making a new iterator for every caller. Indeed, making a new
iterater per caller, in addition to being expensive, might easily return
the same element to every caller.

Do anyone feel an iteration capable .get() unduely burdens subclasses
that want to impement different .get()s? Both the suggested potential
subclass choices (round robin and random) suggest iteration capable
.get()s (presuming "random" means "shuffle order" rather than potentially
returning the same element twice).

Cheers,
-- 
Cameron Simpson <cs at zip.com.au> DoD#743
http://www.cskk.ezoshosting.com/cs/

Why doesn't DOS ever say EXCELLENT command or filename?

From greg at krypto.org  Mon Nov  2 07:39:21 2009
From: greg at krypto.org (Gregory P. Smith)
Date: Sun, 1 Nov 2009 22:39:21 -0800
Subject: [Python-Dev] Reworking the GIL
In-Reply-To: <loom.20091101T123110-331@post.gmane.org>
References: <1256502140.5621.163.camel@localhost>
	<bbaeab100910251457t64616f62x3216ebb4863223f5@mail.gmail.com> 
	<loom.20091101T123110-331@post.gmane.org>
Message-ID: <52dc1c820911012239m627036ccx2b5eca83c0b4b94f@mail.gmail.com>

On Sun, Nov 1, 2009 at 3:33 AM, Antoine Pitrou <solipsis at pitrou.net> wrote:
>
> Hello again,
>
> Brett Cannon <brett <at> python.org> writes:
>>
>> I think it's worth it. Removal of the GIL is a totally open-ended problem
>> with no solution in sight. This, on the other hand, is a performance benefit
>> now. I say move forward with this. If it happens to be short-lived because
>> some actually figures out how to remove the GIL then great, but is that
>> really going to happen between now and Python 3.2? I doubt it.
>
> Based on this whole discussion, I think I am going to merge the new GIL work
> into the py3k branch, with priority requests disabled.
>
> If you think this is premature or uncalled for, or if you just want to review
> the changes before making a judgement, please voice up :)

+1 Good idea.  Thats the best way to make sure this work gets
anywhere.  It can be iterated on from there if anyone has objections.

From fuzzyman at voidspace.org.uk  Mon Nov  2 09:29:22 2009
From: fuzzyman at voidspace.org.uk (Michael Foord)
Date: Mon, 02 Nov 2009 09:29:22 +0100
Subject: [Python-Dev] nonlocal keyword in 2.x?
In-Reply-To: <bbaeab100911011557t714e3b4cxcdc91b3fb9a49b84@mail.gmail.com>
References: <da7032ce0910211856g45e45632n550aa5b4a10f9a62@mail.gmail.com>
	<4AE0C73C.9070805@v.loewis.de>	<319e029f0910272330me6847d4i5f9a4edb8f9359fc@mail.gmail.com>
	<loom.20091028T101119-295@post.gmane.org>	<19176.18618.662587.613771@montanaro.dyndns.org>
	<loom.20091028T154811-994@post.gmane.org>	<319e029f0910290115y49be844co78da6636452445f@mail.gmail.com>
	<4AE95F18.4040807@gmail.com>	<319e029f0911011326i45bf52ew35b9d7e438df6570@mail.gmail.com>
	<02A812F3B69E46D180B45133D52D1F2B@RaymondLaptop1>
	<bbaeab100911011557t714e3b4cxcdc91b3fb9a49b84@mail.gmail.com>
Message-ID: <4AEE9862.8010201@voidspace.org.uk>

Brett Cannon wrote:
> On Sun, Nov 1, 2009 at 13:39, Raymond Hettinger <python at rcn.com> wrote:
>   
>>> As I said: Python 2 support is not only about supporting old versions of
>>> Python,
>>> but also supporting users of Python2-only modules. So 2.7 support will
>>> for the most part be a case not of supporting Python versions, but
>>> Python *users*. And contrary to what Antoine said, that *is* a good
>>> reason to backport it.
>>>       
>> FWIW, I support backporting the nonlocal-keyword in 2.7.
>> All of the reasons for introducting nonlocal to 3.x also apply to 2.x.
>> Using the nonlocal keyword in clear and explicit, especially
>> when compared to the existing workarounds which are not pretty.
>>     
>
> We are voting, I'm -0.
>   

nonlocal is really useful for tests so I'd love to have this in 2.7, but 
obviously someone has to do the work so I'm only +0.

Michael

> -Brett
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk
>   


-- 
http://www.ironpythoninaction.com/


From martin at v.loewis.de  Mon Nov  2 10:08:48 2009
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Mon, 02 Nov 2009 10:08:48 +0100
Subject: [Python-Dev] Reworking the GIL
In-Reply-To: <1256502140.5621.163.camel@localhost>
References: <1256502140.5621.163.camel@localhost>
Message-ID: <4AEEA1A0.8050908@v.loewis.de>

> The new GIL does away with this by ditching _Py_Ticker entirely and
> instead using a fixed interval (by default 5 milliseconds, but settable)
> after which we ask the main thread to release the GIL and let another
> thread be scheduled.

I've looked at this part of the implementation, and have a few comments.
a) why is gil_interval a double? Make it an int, counting in
   microseconds.
b) notice that, on Windows, minimum wait resolution may be as large as
   15ms (e.g. on XP, depending on the hardware). Not sure what this
   means for WaitForMultipleObjects; most likely, if you ask for a 5ms
   wait, it waits until the next clock tick. It would be bad if, on
   some systems, a wait of 5ms would mean that it immediately returns.
c) is the gil_drop_request handling thread-safe? If your answer is
   "yes", you should add a comment as to what the assumptions are of
   this code (ISTM that multiple threads may simultaneously attempt
   to set the drop request, overlapping with the holding thread actually
   dropping the GIL). There is also the question whether it is
   thread-safe to write into a "volatile int"; I keep forgetting the
   answer.

Regards,
Martin



From dickinsm at gmail.com  Mon Nov  2 10:18:49 2009
From: dickinsm at gmail.com (Mark Dickinson)
Date: Mon, 2 Nov 2009 09:18:49 +0000
Subject: [Python-Dev] thanks to everyone cleaning up the tests
In-Reply-To: <bbaeab100910311447l656c4f17tee3d88225e1cdcfc@mail.gmail.com>
References: <bbaeab100910311447l656c4f17tee3d88225e1cdcfc@mail.gmail.com>
Message-ID: <5c6f2a5d0911020118h4cfee72cl430ef45a1cbf9bc7@mail.gmail.com>

On Sat, Oct 31, 2009 at 9:47 PM, Brett Cannon <brett at python.org> wrote:
> Just wanted to publicly thank everyone who has been causing all the
> checkins to fix and stabilize the test suite (I think it's mostly
> Antoine and Mark, but I could be missing somebody; I'm under a
> deadline so only have marginal higher brain functionality).

Not guilty, your honour.  Blame Antoine and David.  :)

David's new buildslave seems to be making a difference, too.

Mark

From solipsis at pitrou.net  Mon Nov  2 11:21:05 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Mon, 2 Nov 2009 10:21:05 +0000 (UTC)
Subject: [Python-Dev]
	=?utf-8?q?Retrieve_an_arbitrary_element_from_a_set?=
	=?utf-8?q?=09withoutremoving_it?=
References: <7b0045ed0910301843y7059d316s8a851142b78b7aeb@mail.gmail.com>
	<20091102052900.GA25160@cskk.homeip.net>
Message-ID: <loom.20091102T111744-750@post.gmane.org>

Cameron Simpson <cs <at> zip.com.au> writes:
> 
> Personally, I'm for the iteration spec in a lot of ways.
> 
> Firstly, a .get()/.pick() that always returns the same element feels
> horrible. Is there anyone here who _likes_ it?

I do. Since the caller is asking for an arbitrary element, returning the same
element is legitimate. It's funny how people seem to have a problem with the
word "arbitrary" :)

And I'm -1 on any implicit iteration attaching some state to the object. If you
want to iterate, there's already an obvious way to it.

Regards

Antoine.



From sturla at molden.no  Mon Nov  2 11:28:39 2009
From: sturla at molden.no (Sturla Molden)
Date: Mon, 02 Nov 2009 11:28:39 +0100
Subject: [Python-Dev] Reworking the GIL
In-Reply-To: <4AEEA1A0.8050908@v.loewis.de>
References: <1256502140.5621.163.camel@localhost>
	<4AEEA1A0.8050908@v.loewis.de>
Message-ID: <4AEEB457.6010808@molden.no>

Martin v. L?wis skrev:
> b) notice that, on Windows, minimum wait resolution may be as large as
>    15ms (e.g. on XP, depending on the hardware). Not sure what this
>    means for WaitForMultipleObjects; most likely, if you ask for a 5ms
>    wait, it waits until the next clock tick. It would be bad if, on
>    some systems, a wait of 5ms would mean that it immediately returns.
>   
Which is why one should use multimedia timers with QPC on Windows.

To get a wait function with much better resolution than Windows' 
default, do this:

1. Set a high resolution with timeBeginPeriod.

2. Loop using a time-out of 0 for WaitForMultipleObjects and put a 
Sleep(0) in the loop not to burn the CPU. Call QPF to get a precise 
timing, and break the loop when the requested time-out has been reached.

3. When you are done, call timeBeginPeriod to turn the multimedia timer off.

This is how you create usleep() in Windows as well: Just loop on QPF and 
Sleep(0) after setting timeBeginPeriod(1).







From martin at v.loewis.de  Mon Nov  2 11:42:47 2009
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Mon, 02 Nov 2009 11:42:47 +0100
Subject: [Python-Dev] Reworking the GIL
In-Reply-To: <4AEEB457.6010808@molden.no>
References: <1256502140.5621.163.camel@localhost>	<4AEEA1A0.8050908@v.loewis.de>
	<4AEEB457.6010808@molden.no>
Message-ID: <4AEEB7A7.3020102@v.loewis.de>

Sturla Molden wrote:
> Martin v. L?wis skrev:
>> b) notice that, on Windows, minimum wait resolution may be as large as
>>    15ms (e.g. on XP, depending on the hardware). Not sure what this
>>    means for WaitForMultipleObjects; most likely, if you ask for a 5ms
>>    wait, it waits until the next clock tick. It would be bad if, on
>>    some systems, a wait of 5ms would mean that it immediately returns.
>>   
> Which is why one should use multimedia timers with QPC on Windows.

Maybe you should study the code under discussion before making such
a proposal.

Regards,
Martin

From solipsis at pitrou.net  Mon Nov  2 11:53:37 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Mon, 2 Nov 2009 10:53:37 +0000 (UTC)
Subject: [Python-Dev] Reworking the GIL
References: <1256502140.5621.163.camel@localhost>
	<4AEEA1A0.8050908@v.loewis.de>
Message-ID: <loom.20091102T113830-495@post.gmane.org>

Martin v. L?wis <martin <at> v.loewis.de> writes:
> 
> I've looked at this part of the implementation, and have a few comments.
> a) why is gil_interval a double? Make it an int, counting in
>    microseconds.

Ok, I'll do that.

> b) notice that, on Windows, minimum wait resolution may be as large as
>    15ms (e.g. on XP, depending on the hardware). Not sure what this
>    means for WaitForMultipleObjects; most likely, if you ask for a 5ms
>    wait, it waits until the next clock tick. It would be bad if, on
>    some systems, a wait of 5ms would mean that it immediately returns.

I'll let someone else give an authoritative answer. I did the Windows version
mainly by reading online MSDN docs, and testing it worked fine in an XP VM.

Anyway, here is what the online docs have to say :

? The system clock "ticks" at a constant rate. If the time-out interval is less
than the resolution of the system clock, the wait may time out in less than the
specified length of time. If the time-out interval is greater than one tick but
less than two, the wait can be anywhere between one and two ticks, and so on. ?

So it seems that the timeout always happens on a Windows tick boundary, which
means that it can immediately return, but only very rarely so and never more
than once in a row...

> c) is the gil_drop_request handling thread-safe? If your answer is
>    "yes", you should add a comment as to what the assumptions are of
>    this code (ISTM that multiple threads may simultaneously attempt
>    to set the drop request, overlapping with the holding thread actually
>    dropping the GIL).

The gil_drop_request is volatile mainly out of precaution, but it's probably not
necessary. It is only written (set/cleared) when holding the gil_mutex; however,
it can be read while not holding that mutex. Exceptionally reading the "wrong"
value would not have any adverse consequences, it would just decrease the
switching quality at the said instant.

Regards

Antoine.



From w.richert at gmx.net  Mon Nov  2 12:00:24 2009
From: w.richert at gmx.net (Willi Richert)
Date: Mon, 2 Nov 2009 12:00:24 +0100
Subject: [Python-Dev] Retrieve an arbitrary element from a set
	withoutremoving it
In-Reply-To: <20091102052900.GA25160@cskk.homeip.net>
References: <7b0045ed0910301843y7059d316s8a851142b78b7aeb@mail.gmail.com>
	<20091102052900.GA25160@cskk.homeip.net>
Message-ID: <200911021200.24249.w.richert@gmx.net>

Hi,

all your points are valid -- for the experienced Python programmer who has 
stumbled across this issue already and solved it in one of several ways. All 
your points, however, do not support the "one obvious way to do it" philosophy 
of Python. It's all about making Python even more clean and beautiful.

wr

Am Montag, 2. November 2009 06:29:00 schrieb Cameron Simpson:
> On 30Oct2009 20:43, Chris Bergstresser <chris at subtlety.com> wrote:
> | On Fri, Oct 30, 2009 at 8:29 PM, Steven D'Aprano <steve at pearwood.info> 
wrote:
> | >> > Iterating over an iterable is
> | >> > what iterators are for.
> | >
> | > set.get(), or set.pick() as Wikipedia calls it, isn't for iterating
> | > over sets. It is for getting an arbitrary element from the set.
> 
> [...]
> 
> | > The purpose is to
> | > return an arbitrary item each time it is called. If two threads get the
> | > same element, that's not a problem; if one thread misses an element
> | > because another thread grabbed it first, that's not a problem either.
> | > If people prefer a random element instead, I have no problem with
> | > that -- personally I think that's overkill, but maybe that's just me.
> |
> |    I also think returning a random one is overkill, in the base set.
> | And I'd have the base implementation do the simplest thing possible:
> | return a fixed element (either the first returned when iterated over,
> | or the last stored, or whatever's easiest based on the internals).
> | For me, leaving the choice of *which* element to return on each call
> | is a feature.  It allows subclasses to change the behavior to support
> | other use cases, like a random or round-robin behavior.
> 
> Personally, I'm for the iteration spec in a lot of ways.
> 
> Firstly, a .get()/.pick() that always returns the same element feels
> horrible. Is there anyone here who _likes_ it?
> 
> Secondly, I think the thread-unsafe arguments are weak. Question: is the
> existing set iteration scheme thread safe? i.e. does it return a fresh
> iterator that a thread can happily use concurrently while another thread
> uses its own iterator?  (Absent set modifications). If the answer is yes,
> then I don't buy the thread-unsafe complaints - there are already plenty
> of thread unsafe things much as lots of iterators will break in the face
> of changes to the data structures over which they're iterating. If
> iter(set) gets you a safe iterator (absent set modification and multiple
> users of that iterator) then thread safe methods exist and are easy to
> use. No presently thread-safe program is going to be broken by adding an
> iterating .get() method.
> 
> Thirdly, an iteration-like .get() is dead easy to provide: you just keep a
> _single_, cycling, internal iterator made on demand the same way __iter__
> already works. So why not do just do it? There's no need to construct it
> before anyone calls .get() the first time. Somewhat like:
> 
>   def get(self):
>     for x in self:
>       return x
>     raise something here
> 
> but not making a new iterator for every caller. Indeed, making a new
> iterater per caller, in addition to being expensive, might easily return
> the same element to every caller.
> 
> Do anyone feel an iteration capable .get() unduely burdens subclasses
> that want to impement different .get()s? Both the suggested potential
> subclass choices (round robin and random) suggest iteration capable
> .get()s (presuming "random" means "shuffle order" rather than potentially
> returning the same element twice).
> 
> Cheers,
> 


From solipsis at pitrou.net  Mon Nov  2 12:30:56 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Mon, 2 Nov 2009 11:30:56 +0000 (UTC)
Subject: [Python-Dev] EC2 buildslaves
References: <5c6f2a5d0910250154nfcdabafyc051389c956ef258@mail.gmail.com>
	<4AE41E9E.4050706@v.loewis.de>
	<20091025140512.11571.1513702964.divmod.xquotient.1543@localhost.localdomain>
	<hcf3pi$vgs$1@ger.gmane.org> <4AEB6C06.3070907@v.loewis.de>
	<loom.20091031T210845-748@post.gmane.org>
	<20091101175432.12668.1205553571.divmod.xquotient.130@localhost.localdomain>
Message-ID: <loom.20091102T122730-452@post.gmane.org>

<exarkun <at> twistedmatrix.com> writes:
> 
> Starting with a mainstream distro doesn't seem like a bad idea.  For 
> example, there isn't currently a 32bit Ubuntu (any version) slave.  That 
> would be a nice gap to fill in, right?

I've setup a buildslave on an EC2 Ubuntu Karmic instance here:
http://www.python.org/dev/buildbot/all/buildslaves/pitrou-ubuntu

However, since it's right now billed on my credit card, I'll shut it down in a
couple of days. I wonder how we can get the PSF to be billed instead of me, if
the PSF accepts to fund such an instance (which, given EC2 prices, is perhaps
not the best use of money?).

Regards

Antoine.



From martin at v.loewis.de  Mon Nov  2 12:42:19 2009
From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=)
Date: Mon, 02 Nov 2009 12:42:19 +0100
Subject: [Python-Dev] Reworking the GIL
In-Reply-To: <loom.20091102T113830-495@post.gmane.org>
References: <1256502140.5621.163.camel@localhost>	<4AEEA1A0.8050908@v.loewis.de>
	<loom.20091102T113830-495@post.gmane.org>
Message-ID: <4AEEC59B.1040902@v.loewis.de>

>> c) is the gil_drop_request handling thread-safe? If your answer is
>>    "yes", you should add a comment as to what the assumptions are of
>>    this code (ISTM that multiple threads may simultaneously attempt
>>    to set the drop request, overlapping with the holding thread actually
>>    dropping the GIL).
> 
> The gil_drop_request is volatile mainly out of precaution, but it's probably not
> necessary. It is only written (set/cleared) when holding the gil_mutex; however,
> it can be read while not holding that mutex. Exceptionally reading the "wrong"
> value would not have any adverse consequences, it would just decrease the
> switching quality at the said instant.

I think it then needs definitely to be volatile - otherwise, the
compiler may chose to cache it in a register (assuming enough registers
are available), instead of re-reading it from memory each time (which is
exactly what volatile then forces it to).

Even if it is read from memory, I still wonder what might happen on
systems that require explicit memory barriers to synchronize across
CPUs. What if CPU 1 keeps reading a 0 value out of its cache, even
though CPU 1 has written an 1 value a long time ago?

IIUC, any (most?) pthread calls would cause synchronization in that
case, which is why applications that also use locks for reading:

http://www.opengroup.org/onlinepubs/009695399/basedefs/xbd_chap04.html#tag_04_10

Of course, on x86, you won't see any issues, because it's cache-coherent
anyway.

Regards,
Martin

From solipsis at pitrou.net  Mon Nov  2 13:15:28 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Mon, 2 Nov 2009 12:15:28 +0000 (UTC)
Subject: [Python-Dev] Reworking the GIL
References: <1256502140.5621.163.camel@localhost>	<4AEEA1A0.8050908@v.loewis.de>
	<loom.20091102T113830-495@post.gmane.org>
	<4AEEC59B.1040902@v.loewis.de>
Message-ID: <loom.20091102T124905-364@post.gmane.org>

Martin v. L?wis <martin <at> v.loewis.de> writes:
> 
[gil_drop_request]
> Even if it is read from memory, I still wonder what might happen on
> systems that require explicit memory barriers to synchronize across
> CPUs. What if CPU 1 keeps reading a 0 value out of its cache, even
> though CPU 1 has written an 1 value a long time ago?
> 
> IIUC, any (most?) pthread calls would cause synchronization in that
> case, which is why applications that also use locks for reading:
> 
> http://www.opengroup.org/onlinepubs/009695399/basedefs/xbd_chap04.html#tag_04_10
> 
> Of course, on x86, you won't see any issues, because it's cache-coherent
> anyway.

I think there are two things here:
- all machines Python runs on should AFAIK be cache-coherent: CPUs synchronize
their views of memory in a rather timely fashion.
- memory ordering: writes made by a CPU can be seen in a different order by
another CPU (i.e. CPU 1 writes A before B, but CPU 2 sees B written before A). I
don't see how this can apply to gil_drop_request, since it's a single variable
(and, moreover, only a single bit of it is significant).

(there's an explanation of memory ordering issues here:
http://www.linuxjournal.com/article/8211)

As a side note, I remember Jeffrey Yasskin trying to specify an ordering model
for Python code
(see http://code.google.com/p/unladen-swallow/wiki/MemoryModel).

Regards

Antoine.



From ncoghlan at gmail.com  Mon Nov  2 13:27:48 2009
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Mon, 02 Nov 2009 22:27:48 +1000
Subject: [Python-Dev] Retrieve an arbitrary element from a set
 withoutremoving it
In-Reply-To: <loom.20091102T111744-750@post.gmane.org>
References: <7b0045ed0910301843y7059d316s8a851142b78b7aeb@mail.gmail.com>	<20091102052900.GA25160@cskk.homeip.net>
	<loom.20091102T111744-750@post.gmane.org>
Message-ID: <4AEED044.4020405@gmail.com>

Antoine Pitrou wrote:
> Cameron Simpson <cs <at> zip.com.au> writes:
>> Personally, I'm for the iteration spec in a lot of ways.
>>
>> Firstly, a .get()/.pick() that always returns the same element feels
>> horrible. Is there anyone here who _likes_ it?
> 
> I do. Since the caller is asking for an arbitrary element, returning the same
> element is legitimate. It's funny how people seem to have a problem with the
> word "arbitrary" :)

Agreed. Arbitrary is arbitrary - for a stateless method that returns an
arbitrary result, return the same value every time is fine.

> And I'm -1 on any implicit iteration attaching some state to the object. If you
> want to iterate, there's already an obvious way to it.

Also agreed.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------

From martin at v.loewis.de  Mon Nov  2 13:31:25 2009
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Mon, 02 Nov 2009 13:31:25 +0100
Subject: [Python-Dev] EC2 buildslaves
In-Reply-To: <loom.20091102T122730-452@post.gmane.org>
References: <5c6f2a5d0910250154nfcdabafyc051389c956ef258@mail.gmail.com>	<4AE41E9E.4050706@v.loewis.de>	<20091025140512.11571.1513702964.divmod.xquotient.1543@localhost.localdomain>	<hcf3pi$vgs$1@ger.gmane.org>
	<4AEB6C06.3070907@v.loewis.de>	<loom.20091031T210845-748@post.gmane.org>	<20091101175432.12668.1205553571.divmod.xquotient.130@localhost.localdomain>
	<loom.20091102T122730-452@post.gmane.org>
Message-ID: <4AEED11D.3000602@v.loewis.de>

> I've setup a buildslave on an EC2 Ubuntu Karmic instance here:
> http://www.python.org/dev/buildbot/all/buildslaves/pitrou-ubuntu
> 
> However, since it's right now billed on my credit card, I'll shut it down in a
> couple of days. I wonder how we can get the PSF to be billed instead of me, if
> the PSF accepts to fund such an instance (which, given EC2 prices, is perhaps
> not the best use of money?).

Send a request to psf-board at python.org. Such request should include:
- contact information (who'll be running this project)
- project purpose/description
- estimated expenses (in case of doubt, round up rather than
  rounding down)
- a proposal of how payment should proceed. I'm not quite sure whether
  it could be billed on the PSF credit card (you may ask Kurt Kaiser
  in advance); traditionally, it worked best when we received invoices.
There will be a board meeting next Monday, so it might be useful to
have a proposal by then.

As for whether that's good use of the money, I'm skeptical as well.
I don't actually know what EC2 prices *are*, or what the current pricing
for a root server is (plus there had been various offers from people
donating hardware - from people who would be unable to also donate
time).

There was discussion that an EC2 instance can be turned on only when
needed, so we could try to set up something like that (the build master
could then trigger activation of the machine, IIUC). However, it might
be that the machine would have to be up most of the day, as there would
be one build going on always, anyway.

Regards,
Martin

From solipsis at pitrou.net  Mon Nov  2 13:41:20 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Mon, 02 Nov 2009 13:41:20 +0100
Subject: [Python-Dev] EC2 buildslaves
In-Reply-To: <4AEED11D.3000602@v.loewis.de>
References: <5c6f2a5d0910250154nfcdabafyc051389c956ef258@mail.gmail.com>
	<4AE41E9E.4050706@v.loewis.de>
	<20091025140512.11571.1513702964.divmod.xquotient.1543@localhost.localdomain>
	<hcf3pi$vgs$1@ger.gmane.org> <4AEB6C06.3070907@v.loewis.de>
	<loom.20091031T210845-748@post.gmane.org>
	<20091101175432.12668.1205553571.divmod.xquotient.130@localhost.localdomain>
	<loom.20091102T122730-452@post.gmane.org>
	<4AEED11D.3000602@v.loewis.de>
Message-ID: <1257165680.5521.11.camel@localhost>

Le lundi 02 novembre 2009 ? 13:31 +0100, "Martin v. L?wis" a ?crit :
> 
> There was discussion that an EC2 instance can be turned on only when
> needed, so we could try to set up something like that (the build master
> could then trigger activation of the machine, IIUC). However, it might
> be that the machine would have to be up most of the day, as there would
> be one build going on always, anyway.

Yes, I think that would be the case. We have frequent commits on each of
the 4 branches under test, with a test suite that takes quite a bit of
time to run in debug mode with -uall.

Moreover, a standard ("small", which also means cheapest) EC2 instance
apparently provides (based on a quick test) between 25% and 50% of the
power of a full CPU core, which makes builds longer. I thought a full
CPU core was provided, but it is not.

An always-on "small" EC2 instance is at least 500$ a year, with a small
storage cost added to that.

Therefore, I think EC2 buildslaves would be interesting under the
condition that they are donated rather than paid for. I don't know
whether anyone has contacts at Amazon.
(but then any donated piece of hardware would be good, not necessary an
EC2 instance)

I assume Jean-Paul made his original suggestion under the premise that
the EC2 instances would only be run when necessary, which is probably
very economical with Twisted's development model (few commits on trunk)
but not with ours.

Regards

Antoine.



From ssteinerx at gmail.com  Mon Nov  2 13:42:35 2009
From: ssteinerx at gmail.com (ssteinerX@gmail.com)
Date: Mon, 2 Nov 2009 07:42:35 -0500
Subject: [Python-Dev] EC2 buildslaves
In-Reply-To: <loom.20091102T122730-452@post.gmane.org>
References: <5c6f2a5d0910250154nfcdabafyc051389c956ef258@mail.gmail.com>
	<4AE41E9E.4050706@v.loewis.de>
	<20091025140512.11571.1513702964.divmod.xquotient.1543@localhost.localdomain>
	<hcf3pi$vgs$1@ger.gmane.org> <4AEB6C06.3070907@v.loewis.de>
	<loom.20091031T210845-748@post.gmane.org>
	<20091101175432.12668.1205553571.divmod.xquotient.130@localhost.localdomain>
	<loom.20091102T122730-452@post.gmane.org>
Message-ID: <008C8A20-798A-45F0-81D4-C90785EEC8CA@gmail.com>


On Nov 2, 2009, at 6:30 AM, Antoine Pitrou wrote:

> <exarkun <at> twistedmatrix.com> writes:
>>
>> Starting with a mainstream distro doesn't seem like a bad idea.  For
>> example, there isn't currently a 32bit Ubuntu (any version) slave.   
>> That
>> would be a nice gap to fill in, right?
>
> I've setup a buildslave on an EC2 Ubuntu Karmic instance here:
> http://www.python.org/dev/buildbot/all/buildslaves/pitrou-ubuntu
>

If you could send me the script that you used to set it up, I could  
give it a shot on RackSpace where it's cheaper (and I have a temporary  
developer account).

S


From solipsis at pitrou.net  Mon Nov  2 13:49:10 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Mon, 02 Nov 2009 13:49:10 +0100
Subject: [Python-Dev] EC2 buildslaves
In-Reply-To: <008C8A20-798A-45F0-81D4-C90785EEC8CA@gmail.com>
References: <5c6f2a5d0910250154nfcdabafyc051389c956ef258@mail.gmail.com>
	<4AE41E9E.4050706@v.loewis.de>
	<20091025140512.11571.1513702964.divmod.xquotient.1543@localhost.localdomain>
	<hcf3pi$vgs$1@ger.gmane.org> <4AEB6C06.3070907@v.loewis.de>
	<loom.20091031T210845-748@post.gmane.org>
	<20091101175432.12668.1205553571.divmod.xquotient.130@localhost.localdomain>
	<loom.20091102T122730-452@post.gmane.org>
	<008C8A20-798A-45F0-81D4-C90785EEC8CA@gmail.com>
Message-ID: <1257166150.5521.19.camel@localhost>

Le lundi 02 novembre 2009 ? 07:42 -0500, ssteinerX at gmail.com a ?crit :
> 
> If you could send me the script that you used to set it up, I could  
> give it a shot on RackSpace where it's cheaper (and I have a temporary  
> developer account).

There's no need for a special script, really.
Install Python, buildbot, subversion (all packaged by your Linux
provider), then a couple of development libraries so that enough
extension modules get built: dev headers for zlib, bz2, openssl and
sqlite3 would be enough IMO.

When you have done that, go to http://wiki.python.org/moin/BuildBot and
follow the instructions at the end. Skip "install buildbot from source"
if you've installed it from your distro's package repo.
Also, you may have to replace "buildbot slave" with "buildbot
create-slave".




From martin at v.loewis.de  Mon Nov  2 13:51:43 2009
From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=)
Date: Mon, 02 Nov 2009 13:51:43 +0100
Subject: [Python-Dev] Reworking the GIL
In-Reply-To: <loom.20091102T124905-364@post.gmane.org>
References: <1256502140.5621.163.camel@localhost>	<4AEEA1A0.8050908@v.loewis.de>	<loom.20091102T113830-495@post.gmane.org>	<4AEEC59B.1040902@v.loewis.de>
	<loom.20091102T124905-364@post.gmane.org>
Message-ID: <4AEED5DF.8040105@v.loewis.de>

> - all machines Python runs on should AFAIK be cache-coherent: CPUs synchronize
> their views of memory in a rather timely fashion.

Ok. I thought that Itanium was an example where this assumption is
actually violated (as many web pages claim such a restriction), however,
it seems that on Itanium, caches are indeed synchronized using MESI.

So claims wrt. lack of cache consistency on Itanium, and the need for
barrier instruction, seem to be caused by the Itanium feature that
allows the processor to fetch memory out-of-order, i.e. an earlier read
may see a later memory state. This is apparently used to satisfy reads
as soon as the cache line is read (so that the cache line can be
discarded earlier). Wrt. to your requirement ("rather timely fashion",
this still seems to be fine).

Still, this all needs to be documented in the code :-)

Regards,
Martin


From martin at v.loewis.de  Mon Nov  2 13:59:37 2009
From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=)
Date: Mon, 02 Nov 2009 13:59:37 +0100
Subject: [Python-Dev] EC2 buildslaves
In-Reply-To: <1257165680.5521.11.camel@localhost>
References: <5c6f2a5d0910250154nfcdabafyc051389c956ef258@mail.gmail.com>	<4AE41E9E.4050706@v.loewis.de>	<20091025140512.11571.1513702964.divmod.xquotient.1543@localhost.localdomain>	<hcf3pi$vgs$1@ger.gmane.org>
	<4AEB6C06.3070907@v.loewis.de>	<loom.20091031T210845-748@post.gmane.org>	<20091101175432.12668.1205553571.divmod.xquotient.130@localhost.localdomain>	<loom.20091102T122730-452@post.gmane.org>	<4AEED11D.3000602@v.loewis.de>
	<1257165680.5521.11.camel@localhost>
Message-ID: <4AEED7B9.2000209@v.loewis.de>

> An always-on "small" EC2 instance is at least 500$ a year, with a small
> storage cost added to that.

OTOH, that isn't that expensive (compared to the other PSF expenses),
plus people keep donating money, so when we say what we use it for,
there may be a larger return than just the test results.

OTTH, the same could be achieved by buying a hosted server elsewhere.

Regards,
Martin

P.S. Perhaps this *is* the time for Steve to ask for "always-on" machines.

From solipsis at pitrou.net  Mon Nov  2 14:06:14 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Mon, 02 Nov 2009 14:06:14 +0100
Subject: [Python-Dev] EC2 buildslaves
In-Reply-To: <4AEED7B9.2000209@v.loewis.de>
References: <5c6f2a5d0910250154nfcdabafyc051389c956ef258@mail.gmail.com>
	<4AE41E9E.4050706@v.loewis.de>
	<20091025140512.11571.1513702964.divmod.xquotient.1543@localhost.localdomain>
	<hcf3pi$vgs$1@ger.gmane.org> <4AEB6C06.3070907@v.loewis.de>
	<loom.20091031T210845-748@post.gmane.org>
	<20091101175432.12668.1205553571.divmod.xquotient.130@localhost.localdomain>
	<loom.20091102T122730-452@post.gmane.org>	<4AEED11D.3000602@v.loewis.de>
	<1257165680.5521.11.camel@localhost>  <4AEED7B9.2000209@v.loewis.de>
Message-ID: <1257167174.5521.23.camel@localhost>


> OTOH, that isn't that expensive (compared to the other PSF expenses),
> plus people keep donating money, so when we say what we use it for,
> there may be a larger return than just the test results.
> 
> OTTH, the same could be achieved by buying a hosted server elsewhere.

One advantage of a real hosted server is that we could have a bunch of
our own VMs on it, which is probably not possible (and perhaps not
allowed) on an EC2 instance (not to mention it could really be slow). 

(I'm not volunteering to install and manage VMs, however; I don't think
I'm competent to do that)



From jnoller at gmail.com  Mon Nov  2 15:04:03 2009
From: jnoller at gmail.com (Jesse Noller)
Date: Mon, 2 Nov 2009 09:04:03 -0500
Subject: [Python-Dev] EC2 buildslaves
In-Reply-To: <1257167174.5521.23.camel@localhost>
References: <5c6f2a5d0910250154nfcdabafyc051389c956ef258@mail.gmail.com>
	<hcf3pi$vgs$1@ger.gmane.org> <4AEB6C06.3070907@v.loewis.de>
	<loom.20091031T210845-748@post.gmane.org>
	<20091101175432.12668.1205553571.divmod.xquotient.130@localhost.localdomain>
	<loom.20091102T122730-452@post.gmane.org>
	<4AEED11D.3000602@v.loewis.de> <1257165680.5521.11.camel@localhost>
	<4AEED7B9.2000209@v.loewis.de> <1257167174.5521.23.camel@localhost>
Message-ID: <4222a8490911020604s73de78bj44c197b43b30e38b@mail.gmail.com>

On Mon, Nov 2, 2009 at 8:06 AM, Antoine Pitrou <solipsis at pitrou.net> wrote:
>
>> OTOH, that isn't that expensive (compared to the other PSF expenses),
>> plus people keep donating money, so when we say what we use it for,
>> there may be a larger return than just the test results.
>>
>> OTTH, the same could be achieved by buying a hosted server elsewhere.
>
> One advantage of a real hosted server is that we could have a bunch of
> our own VMs on it, which is probably not possible (and perhaps not
> allowed) on an EC2 instance (not to mention it could really be slow).
>
> (I'm not volunteering to install and manage VMs, however; I don't think
> I'm competent to do that)

Now that the bulk of my pycon work is "completed" - I'll thread swap
to my VM host proposal and the moratorium PEP. So don't worry about
the VM host.

jesse

From sturla at molden.no  Mon Nov  2 15:07:14 2009
From: sturla at molden.no (Sturla Molden)
Date: Mon, 02 Nov 2009 15:07:14 +0100
Subject: [Python-Dev] Reworking the GIL
In-Reply-To: <4AEEB7A7.3020102@v.loewis.de>
References: <1256502140.5621.163.camel@localhost>	<4AEEA1A0.8050908@v.loewis.de>
	<4AEEB457.6010808@molden.no> <4AEEB7A7.3020102@v.loewis.de>
Message-ID: <4AEEE792.9080108@molden.no>

Martin v. L?wis skrev:
> Maybe you should study the code under discussion before making such
> a proposal.
>
I did, and it does nothing of what I suggested. I am sure I can make the 
Windows GIL
in ceval_gil.h and the mutex in thread_nt.h at lot more precise and 
efficient.

This is the kind of code I was talking about, from ceval_gil.h:

r = WaitForMultipleObjects(2, objects, TRUE, milliseconds);

I would turn on multimedia timer (it is not on by default), and replace 
this
call with a loop, approximately like this:

for (;;) {
   r = WaitForMultipleObjects(2, objects, TRUE, 0);
   /* blah blah blah */   
   QueryPerformanceCounter(&cnt);  
   if (cnt > timeout) break;
   Sleep(0);
}

And the timeout "milliseconds" would now be computed from querying the 
performance
counter, instead of unreliably by the Windows NT kernel.



Sturla











From solipsis at pitrou.net  Mon Nov  2 15:16:58 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Mon, 2 Nov 2009 14:16:58 +0000 (UTC)
Subject: [Python-Dev] Reworking the GIL
References: <1256502140.5621.163.camel@localhost>	<4AEEA1A0.8050908@v.loewis.de>
	<4AEEB457.6010808@molden.no> <4AEEB7A7.3020102@v.loewis.de>
	<4AEEE792.9080108@molden.no>
Message-ID: <loom.20091102T151603-346@post.gmane.org>

Sturla Molden <sturla <at> molden.no> writes:
> 
> And the timeout "milliseconds" would now be computed from querying the 
> performance
> counter, instead of unreliably by the Windows NT kernel.

Could you actually test your proposal under Windows and report what kind of
concrete benefits it brings?

Thank you

Antoine.



From sturla at molden.no  Mon Nov  2 15:22:02 2009
From: sturla at molden.no (Sturla Molden)
Date: Mon, 02 Nov 2009 15:22:02 +0100
Subject: [Python-Dev] Reworking the GIL
In-Reply-To: <4AEEE792.9080108@molden.no>
References: <1256502140.5621.163.camel@localhost>	<4AEEA1A0.8050908@v.loewis.de>	<4AEEB457.6010808@molden.no>
	<4AEEB7A7.3020102@v.loewis.de> <4AEEE792.9080108@molden.no>
Message-ID: <4AEEEB0A.9070604@molden.no>

Sturla Molden skrev:
>
> I would turn on multimedia timer (it is not on by default), and 
> replace this
> call with a loop, approximately like this:
>
> for (;;) {
>   r = WaitForMultipleObjects(2, objects, TRUE, 0);
>   /* blah blah blah */     QueryPerformanceCounter(&cnt);    if (cnt > 
> timeout) break;
>   Sleep(0);
> }

And just so you don't ask: There should not just be a Sleep(0) in the 
loop, but a sleep that gets shorter and shorter until a lower threshold 
is reached, where it skips to Sleep(0). That way we avoid hammering om 
WaitForMultipleObjects and QueryPerformanceCounter more than we need. 
And for all that to work better than just giving a timeout to 
WaitForMultipleObjects, we need the multimedia timer turned on.

Sturla




From sturla at molden.no  Mon Nov  2 15:40:43 2009
From: sturla at molden.no (Sturla Molden)
Date: Mon, 02 Nov 2009 15:40:43 +0100
Subject: [Python-Dev] Reworking the GIL
In-Reply-To: <4AEEEB0A.9070604@molden.no>
References: <1256502140.5621.163.camel@localhost>	<4AEEA1A0.8050908@v.loewis.de>	<4AEEB457.6010808@molden.no>	<4AEEB7A7.3020102@v.loewis.de>
	<4AEEE792.9080108@molden.no> <4AEEEB0A.9070604@molden.no>
Message-ID: <4AEEEF6B.80100@molden.no>

Sturla Molden skrev:
>
> And just so you don't ask: There should not just be a Sleep(0) in the 
> loop, but a sleep that gets shorter and shorter until a lower 
> threshold is reached, where it skips to Sleep(0). That way we avoid 
> hammering om WaitForMultipleObjects and QueryPerformanceCounter more 
> than we need. And for all that to work better than just giving a 
> timeout to WaitForMultipleObjects, we need the multimedia timer turned 
> on.

The important thing about multimedia timer is that the granularity of 
Sleep() and WaitForMultipleObjects() by default is "10 ms or at most 20 
ms". But if we call

timeBeginPeriod(1);

the MM timer is on and granularity becomes 1 ms or at most 2 ms. But we 
can get even more precise than that by hammering on Sleep(0) for 
timeouts less than 2 ms. We can get typical granularity in the order of 
10 ?s, with the occational 100 ?s now and then. I know this because I 
was using Windows 2000 to generate TTL signals on the LPT port some 
years ago, and watched them on the oscilloscope.

~ 15 ms granularity is Windows default. But that is brain dead.

By the way Antoine, if you think granularity of 1-2 ms is sufficient, 
i.e. no need for ?s precision, then just calling timeBeginPeriod(1) will 
be sufficient.

Sturla



 





From solipsis at pitrou.net  Mon Nov  2 15:53:30 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Mon, 2 Nov 2009 14:53:30 +0000 (UTC)
Subject: [Python-Dev] Reworking the GIL
References: <1256502140.5621.163.camel@localhost>	<4AEEA1A0.8050908@v.loewis.de>	<4AEEB457.6010808@molden.no>	<4AEEB7A7.3020102@v.loewis.de>
	<4AEEE792.9080108@molden.no> <4AEEEB0A.9070604@molden.no>
	<4AEEEF6B.80100@molden.no>
Message-ID: <loom.20091102T155123-793@post.gmane.org>

Sturla Molden <sturla <at> molden.no> writes:
> 
> By the way Antoine, if you think granularity of 1-2 ms is sufficient, 

It certainly is.
But once again, I'm no Windows developer and I don't have a native Windost host
to test on; therefore someone else (you?) has to try.

Also, the MSDN doc (*) says timeBeginPeriod() can have a detrimental effect on
system performance; I don't know how much of it is true.

(*) http://msdn.microsoft.com/en-us/library/dd757624(VS.85).aspx



From sturla at molden.no  Mon Nov  2 16:07:06 2009
From: sturla at molden.no (Sturla Molden)
Date: Mon, 02 Nov 2009 16:07:06 +0100
Subject: [Python-Dev] Reworking the GIL
In-Reply-To: <loom.20091102T155123-793@post.gmane.org>
References: <1256502140.5621.163.camel@localhost>	<4AEEA1A0.8050908@v.loewis.de>	<4AEEB457.6010808@molden.no>	<4AEEB7A7.3020102@v.loewis.de>	<4AEEE792.9080108@molden.no>
	<4AEEEB0A.9070604@molden.no>	<4AEEEF6B.80100@molden.no>
	<loom.20091102T155123-793@post.gmane.org>
Message-ID: <4AEEF59A.10504@molden.no>

Antoine Pitrou skrev:
> It certainly is.
> But once again, I'm no Windows developer and I don't have a native Windost host
> to test on; therefore someone else (you?) has to try.
>   
I'd love to try, but I don't have VC++ to build Python, I use GCC on 
Windows.

Anyway, the first thing to try then is to call
  
   timeBeginPeriod(1);

once on startup, and leave the rest of the code as it is. If 2-4 ms is 
sufficient we can use timeBeginPeriod(2), etc. Microsoft is claiming 
Windows performs better with high granularity, which is why it is 10 ms 
by default.


Sturla







From john.arbash.meinel at gmail.com  Mon Nov  2 16:22:52 2009
From: john.arbash.meinel at gmail.com (John Arbash Meinel)
Date: Mon, 02 Nov 2009 09:22:52 -0600
Subject: [Python-Dev] Reworking the GIL
In-Reply-To: <4AEEF59A.10504@molden.no>
References: <1256502140.5621.163.camel@localhost>	<4AEEA1A0.8050908@v.loewis.de>	<4AEEB457.6010808@molden.no>	<4AEEB7A7.3020102@v.loewis.de>	<4AEEE792.9080108@molden.no>	<4AEEEB0A.9070604@molden.no>	<4AEEEF6B.80100@molden.no>	<loom.20091102T155123-793@post.gmane.org>
	<4AEEF59A.10504@molden.no>
Message-ID: <4AEEF94C.4090701@gmail.com>

Sturla Molden wrote:
> Antoine Pitrou skrev:
>> It certainly is.
>> But once again, I'm no Windows developer and I don't have a native
>> Windost host
>> to test on; therefore someone else (you?) has to try.
>>   
> I'd love to try, but I don't have VC++ to build Python, I use GCC on
> Windows.
> 
> Anyway, the first thing to try then is to call
>  
>   timeBeginPeriod(1);
> 
> once on startup, and leave the rest of the code as it is. If 2-4 ms is
> sufficient we can use timeBeginPeriod(2), etc. Microsoft is claiming
> Windows performs better with high granularity, which is why it is 10 ms
> by default.
> 
> 
> Sturla

That page claims:
  Windows uses the lowest value (that is, highest resolution) requested
  by any process.

I would posit that the chance of having some random process on your
machine request a high-speed timer is high enough that the overhead for
Python doing the same is probably low.

John
=:->

From solipsis at pitrou.net  Mon Nov  2 16:28:12 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Mon, 2 Nov 2009 15:28:12 +0000 (UTC)
Subject: [Python-Dev] Reworking the GIL
References: <1256502140.5621.163.camel@localhost>	<4AEEA1A0.8050908@v.loewis.de>	<4AEEB457.6010808@molden.no>	<4AEEB7A7.3020102@v.loewis.de>	<4AEEE792.9080108@molden.no>
	<4AEEEB0A.9070604@molden.no>	<4AEEEF6B.80100@molden.no>
	<loom.20091102T155123-793@post.gmane.org>
	<4AEEF59A.10504@molden.no>
Message-ID: <loom.20091102T162744-24@post.gmane.org>

Sturla Molden <sturla <at> molden.no> writes:
> 
> I'd love to try, but I don't have VC++ to build Python, I use GCC on 
> Windows.

You can use Visual Studio Express, which is free (gratis).




From martin at v.loewis.de  Mon Nov  2 18:27:40 2009
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Mon, 02 Nov 2009 18:27:40 +0100
Subject: [Python-Dev] Reworking the GIL
In-Reply-To: <4AEEE792.9080108@molden.no>
References: <1256502140.5621.163.camel@localhost>	<4AEEA1A0.8050908@v.loewis.de>	<4AEEB457.6010808@molden.no>
	<4AEEB7A7.3020102@v.loewis.de> <4AEEE792.9080108@molden.no>
Message-ID: <4AEF168C.1020102@v.loewis.de>

> I did, and it does nothing of what I suggested. I am sure I can make the
> Windows GIL in ceval_gil.h and the mutex in thread_nt.h at lot more precise and
> efficient.

Hmm. I'm skeptical that your code makes it more accurate, and I
completely fail to see that it makes it more efficient (by what
measurement of efficiency?)

Also, why would making it more accurate make it better? IIUC,
accuracy is completely irrelevant here, though efficiency
(low overhead) does matter.

> This is the kind of code I was talking about, from ceval_gil.h:
> 
> r = WaitForMultipleObjects(2, objects, TRUE, milliseconds);
> 
> I would turn on multimedia timer (it is not on by default), and replace
> this
> call with a loop, approximately like this:
> 
> for (;;) {
>   r = WaitForMultipleObjects(2, objects, TRUE, 0);
>   /* blah blah blah */     QueryPerformanceCounter(&cnt);    if (cnt >
> timeout) break;
>   Sleep(0);
> }
> 
> And the timeout "milliseconds" would now be computed from querying the
> performance counter, instead of unreliably by the Windows NT kernel.

Hmm. This creates a busy wait loop; if you add larger sleep values,
then it loses accuracy.

Why not just call timeBeginPeriod, and then rely on the higher clock
rate for WaitForMultipleObjects?

Regards,
Martin

From daniel at stutzbachenterprises.com  Mon Nov  2 18:40:29 2009
From: daniel at stutzbachenterprises.com (Daniel Stutzbach)
Date: Mon, 2 Nov 2009 11:40:29 -0600
Subject: [Python-Dev] Reworking the GIL
In-Reply-To: <4AEF168C.1020102@v.loewis.de>
References: <1256502140.5621.163.camel@localhost>
	<4AEEA1A0.8050908@v.loewis.de> <4AEEB457.6010808@molden.no>
	<4AEEB7A7.3020102@v.loewis.de> <4AEEE792.9080108@molden.no>
	<4AEF168C.1020102@v.loewis.de>
Message-ID: <eae285400911020940h3de2a260lb62abddeaf722f7e@mail.gmail.com>

On Mon, Nov 2, 2009 at 11:27 AM, "Martin v. L?wis" <martin at v.loewis.de>wrote:

> Hmm. This creates a busy wait loop; if you add larger sleep values,
> then it loses accuracy.
>

I thought that at first, too, but then I checked the documentation for
Sleep(0):

"A value of zero causes the thread to relinquish the remainder of its time
slice to any other thread of equal priority that is ready to run."

(this is not to say that I think the solution with Sleep is worthwhile,
though...)

--
Daniel Stutzbach, Ph.D.
President, Stutzbach Enterprises, LLC <http://stutzbachenterprises.com>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20091102/9c0c8ec7/attachment.htm>

From jyasskin at gmail.com  Mon Nov  2 18:56:11 2009
From: jyasskin at gmail.com (Jeffrey Yasskin)
Date: Mon, 2 Nov 2009 09:56:11 -0800
Subject: [Python-Dev] Reworking the GIL
In-Reply-To: <loom.20091102T124905-364@post.gmane.org>
References: <1256502140.5621.163.camel@localhost>
	<4AEEA1A0.8050908@v.loewis.de>
	<loom.20091102T113830-495@post.gmane.org>
	<4AEEC59B.1040902@v.loewis.de>
	<loom.20091102T124905-364@post.gmane.org>
Message-ID: <5d44f72f0911020956t4e21bf7bp8fa3f096cbb894fd@mail.gmail.com>

On Mon, Nov 2, 2009 at 4:15 AM, Antoine Pitrou <solipsis at pitrou.net> wrote:
> Martin v. L?wis <martin <at> v.loewis.de> writes:
>>
> [gil_drop_request]
>> Even if it is read from memory, I still wonder what might happen on
>> systems that require explicit memory barriers to synchronize across
>> CPUs. What if CPU 1 keeps reading a 0 value out of its cache, even
>> though CPU 1 has written an 1 value a long time ago?
>>
>> IIUC, any (most?) pthread calls would cause synchronization in that
>> case, which is why applications that also use locks for reading:
>>
>> http://www.opengroup.org/onlinepubs/009695399/basedefs/xbd_chap04.html#tag_04_10
>>
>> Of course, on x86, you won't see any issues, because it's cache-coherent
>> anyway.
>
> I think there are two things here:
> - all machines Python runs on should AFAIK be cache-coherent: CPUs synchronize
> their views of memory in a rather timely fashion.
> - memory ordering: writes made by a CPU can be seen in a different order by
> another CPU (i.e. CPU 1 writes A before B, but CPU 2 sees B written before A). I
> don't see how this can apply to gil_drop_request, since it's a single variable
> (and, moreover, only a single bit of it is significant).
>
> (there's an explanation of memory ordering issues here:
> http://www.linuxjournal.com/article/8211)
>
> As a side note, I remember Jeffrey Yasskin trying to specify an ordering model
> for Python code
> (see http://code.google.com/p/unladen-swallow/wiki/MemoryModel).

Note that that memory model was only for Python code; the C code
implementing it is subject to the C memory model, which is weaker (and
not even fully defined until the next C standard comes out).

To be really safe, we ought to have a couple primitives implementing
"atomic" rather than just "volatile" instructions, but until then a
signal that's just saying "do something" rather than "here's some data
you should look at" should be ok as a volatile int.

I'd like to look at the patch in detail, but I can't guarantee that
I'll get to it in a timely manner. I'd say check it in and let more
threading experts look at it in the tree. We've got some time before a
release for people to fix problems and make further improvements. +1
to Martin's request for detailed documentation though. :)

From guido at python.org  Mon Nov  2 19:01:54 2009
From: guido at python.org (Guido van Rossum)
Date: Mon, 2 Nov 2009 10:01:54 -0800
Subject: [Python-Dev] nonlocal keyword in 2.x?
In-Reply-To: <4AEE9862.8010201@voidspace.org.uk>
References: <da7032ce0910211856g45e45632n550aa5b4a10f9a62@mail.gmail.com> 
	<loom.20091028T101119-295@post.gmane.org>
	<19176.18618.662587.613771@montanaro.dyndns.org> 
	<loom.20091028T154811-994@post.gmane.org>
	<319e029f0910290115y49be844co78da6636452445f@mail.gmail.com> 
	<4AE95F18.4040807@gmail.com>
	<319e029f0911011326i45bf52ew35b9d7e438df6570@mail.gmail.com> 
	<02A812F3B69E46D180B45133D52D1F2B@RaymondLaptop1>
	<bbaeab100911011557t714e3b4cxcdc91b3fb9a49b84@mail.gmail.com> 
	<4AEE9862.8010201@voidspace.org.uk>
Message-ID: <ca471dc20911021001u102da717wcf4da0eada0867e@mail.gmail.com>

I'm -0 on backporting nonlocal to 2.7. I could be +0 if we added "from
__future__ import nonlocal_keyword" (or some such phrasing) to enable
it.

-- 
--Guido van Rossum (python.org/~guido)

From python at rcn.com  Mon Nov  2 19:06:27 2009
From: python at rcn.com (Raymond Hettinger)
Date: Mon, 2 Nov 2009 10:06:27 -0800
Subject: [Python-Dev] nonlocal keyword in 2.x?
References: <da7032ce0910211856g45e45632n550aa5b4a10f9a62@mail.gmail.com>
	<loom.20091028T101119-295@post.gmane.org><19176.18618.662587.613771@montanaro.dyndns.org>
	<loom.20091028T154811-994@post.gmane.org><319e029f0910290115y49be844co78da6636452445f@mail.gmail.com>
	<4AE95F18.4040807@gmail.com><319e029f0911011326i45bf52ew35b9d7e438df6570@mail.gmail.com>
	<02A812F3B69E46D180B45133D52D1F2B@RaymondLaptop1><bbaeab100911011557t714e3b4cxcdc91b3fb9a49b84@mail.gmail.com>
	<4AEE9862.8010201@voidspace.org.uk>
	<ca471dc20911021001u102da717wcf4da0eada0867e@mail.gmail.com>
Message-ID: <5C4388816CE0428FB27EA69CE77CFCC0@RaymondLaptop1>


[Guido van Rossum]
> I'm -0 on backporting nonlocal to 2.7. I could be +0 if we added "from
> __future__ import nonlocal_keyword" (or some such phrasing) to enable
> it.

With the "from __future__" option, what keeps you from being
a full +1 on nonlocal?   Is there something that makes it a better
solution for 3.x than 2.x?  Just curious about the pros and cons
from your point of view.


Raymond

From guido at python.org  Mon Nov  2 19:09:40 2009
From: guido at python.org (Guido van Rossum)
Date: Mon, 2 Nov 2009 10:09:40 -0800
Subject: [Python-Dev] nonlocal keyword in 2.x?
In-Reply-To: <5C4388816CE0428FB27EA69CE77CFCC0@RaymondLaptop1>
References: <da7032ce0910211856g45e45632n550aa5b4a10f9a62@mail.gmail.com> 
	<loom.20091028T154811-994@post.gmane.org>
	<319e029f0910290115y49be844co78da6636452445f@mail.gmail.com> 
	<4AE95F18.4040807@gmail.com>
	<319e029f0911011326i45bf52ew35b9d7e438df6570@mail.gmail.com> 
	<02A812F3B69E46D180B45133D52D1F2B@RaymondLaptop1>
	<bbaeab100911011557t714e3b4cxcdc91b3fb9a49b84@mail.gmail.com> 
	<4AEE9862.8010201@voidspace.org.uk>
	<ca471dc20911021001u102da717wcf4da0eada0867e@mail.gmail.com> 
	<5C4388816CE0428FB27EA69CE77CFCC0@RaymondLaptop1>
Message-ID: <ca471dc20911021009u7485adcdgde8fae04a5061059@mail.gmail.com>

On Mon, Nov 2, 2009 at 10:06 AM, Raymond Hettinger <python at rcn.com> wrote:
>
> [Guido van Rossum]
>>
>> I'm -0 on backporting nonlocal to 2.7. I could be +0 if we added "from
>> __future__ import nonlocal_keyword" (or some such phrasing) to enable
>> it.
>
> With the "from __future__" option, what keeps you from being
> a full +1 on nonlocal? ? Is there something that makes it a better
> solution for 3.x than 2.x? ?Just curious about the pros and cons
> from your point of view.

I think the number of projects that can afford to drop support for 2.6
is pretty limited, so I think the utility of the feature is thereby
also limited.

-- 
--Guido van Rossum (python.org/~guido)

From sturla at molden.no  Mon Nov  2 19:10:12 2009
From: sturla at molden.no (Sturla Molden)
Date: Mon, 02 Nov 2009 19:10:12 +0100
Subject: [Python-Dev] Reworking the GIL
In-Reply-To: <4AEF168C.1020102@v.loewis.de>
References: <1256502140.5621.163.camel@localhost>	<4AEEA1A0.8050908@v.loewis.de>	<4AEEB457.6010808@molden.no>
	<4AEEB7A7.3020102@v.loewis.de> <4AEEE792.9080108@molden.no>
	<4AEF168C.1020102@v.loewis.de>
Message-ID: <4AEF2084.2040905@molden.no>

Martin v. L?wis skrev:
>> I did, and it does nothing of what I suggested. I am sure I can make the
>> Windows GIL in ceval_gil.h and the mutex in thread_nt.h at lot more precise and
>> efficient.
>>     
>
> Hmm. I'm skeptical that your code makes it more accurate, and I
> completely fail to see that it makes it more efficient (by what
> measurement of efficiency?)
>
> Also, why would making it more accurate make it better? IIUC,
> accuracy is completely irrelevant here, though efficiency
> (low overhead) does matter.
>
>   
>> This is the kind of code I was talking about, from ceval_gil.h:
>>
>> r = WaitForMultipleObjects(2, objects, TRUE, milliseconds);
>>
>> I would turn on multimedia timer (it is not on by default), and replace
>> this
>> call with a loop, approximately like this:
>>
>> for (;;) {
>>   r = WaitForMultipleObjects(2, objects, TRUE, 0);
>>   /* blah blah blah */     QueryPerformanceCounter(&cnt);    if (cnt >
>> timeout) break;
>>   Sleep(0);
>> }
>>
>> And the timeout "milliseconds" would now be computed from querying the
>> performance counter, instead of unreliably by the Windows NT kernel.
>>     
>
> Hmm. This creates a busy wait loop; if you add larger sleep values,
> then it loses accuracy.
>
>   
Actually an usleep lookes like this, and the call to the wait function 
must go into the for loop. But no, it's not a busy sleep.

static int inited = 0;
static __int64 hz;
static double dhz;
const double sleep_granularity = 2.0E10-3;

void usleep( long us )
{
    __int64 cnt, end;
    double diff;
    if (!inited) {
        timeBeginPeriod(1);
        QueryPerformanceFrequency((LARGE_INTEGER*)&hz);
        dhz = (double)hz;
        inited = 1;
    }      
    QueryPerformanceCounter((LARGE_INTEGER*)&cnt);
    end = cnt + (__int64)(1.0E10-6 * (double)(us) * dhz);
    for (;;) {
        QueryPerformanceCounter((LARGE_INTEGER*)&cnt);
        if (cnt >= end) break;
        diff = (double)(end - cnt)/dhz;
        if (diff > sleep_granularity)
            Sleep((DWORD)(diff - sleep_granularity));
        else
            Sleep(0);  
    }
}


> Why not just call timeBeginPeriod, and then rely on the higher clock
> rate for WaitForMultipleObjects?
>
>   
That is what I suggested when Antoine said 1-2 ms was enough.



Sturla





From martin at v.loewis.de  Mon Nov  2 19:39:53 2009
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Mon, 02 Nov 2009 19:39:53 +0100
Subject: [Python-Dev] language summit topic: issue tracker
In-Reply-To: <4AE17C12.1080007@gmail.com>
References: <bbaeab100910222147u41749107r1032545147fdc2e6@mail.gmail.com>
	<4AE17C12.1080007@gmail.com>
Message-ID: <4AEF2779.5040402@v.loewis.de>

> +lots on adding a module field (independent of automatically adding
> maintainers to the nosy list, it would assist in "I just did a major
> cleanup of module X, are there any old bugs I can kill off").

Link (1:1) or Multilink (1:n)? What is the impact on the Component field?

Would you be willing to manage the field (in the sense of managing the
set of values)? If so, please send me a list of values.

Regards,
Martin

From martin at v.loewis.de  Mon Nov  2 19:42:22 2009
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Mon, 02 Nov 2009 19:42:22 +0100
Subject: [Python-Dev] language summit topic: issue tracker
In-Reply-To: <19169.43241.70138.170239@montanaro.dyndns.org>
References: <bbaeab100910222147u41749107r1032545147fdc2e6@mail.gmail.com>
	<19169.43241.70138.170239@montanaro.dyndns.org>
Message-ID: <4AEF280E.6040503@v.loewis.de>

skip at pobox.com wrote:
>     Brett> Another summit, another potential time to see if people want to
>     Brett> change anything about the issue tracker.
> 
> I have no idea how hard this would be to implement and won't be at the
> language summit to formally present the idea, but it seems to me that some
> integration between the issue tracker and Rietveld would be beneficial. 

See

http://psf.upfronthosting.co.za/roundup/meta/issue247

Regards,
Martin

From rdmurray at bitdance.com  Mon Nov  2 20:18:48 2009
From: rdmurray at bitdance.com (R. David Murray)
Date: Mon, 2 Nov 2009 14:18:48 -0500 (EST)
Subject: [Python-Dev] nonlocal keyword in 2.x?
In-Reply-To: <ca471dc20911021009u7485adcdgde8fae04a5061059@mail.gmail.com>
References: <da7032ce0910211856g45e45632n550aa5b4a10f9a62@mail.gmail.com> 
	<loom.20091028T154811-994@post.gmane.org>
	<319e029f0910290115y49be844co78da6636452445f@mail.gmail.com>
	<4AE95F18.4040807@gmail.com>
	<319e029f0911011326i45bf52ew35b9d7e438df6570@mail.gmail.com>
	<02A812F3B69E46D180B45133D52D1F2B@RaymondLaptop1>
	<bbaeab100911011557t714e3b4cxcdc91b3fb9a49b84@mail.gmail.com> 
	<4AEE9862.8010201@voidspace.org.uk>
	<ca471dc20911021001u102da717wcf4da0eada0867e@mail.gmail.com>
	<5C4388816CE0428FB27EA69CE77CFCC0@RaymondLaptop1>
	<ca471dc20911021009u7485adcdgde8fae04a5061059@mail.gmail.com>
Message-ID: <Pine.LNX.4.64.0911021405420.21420@kimball.webabinitio.net>

On Mon, 2 Nov 2009 at 10:09, Guido van Rossum wrote:
> On Mon, Nov 2, 2009 at 10:06 AM, Raymond Hettinger <python at rcn.com> wrote:
>>
>> [Guido van Rossum]
>>>
>>> I'm -0 on backporting nonlocal to 2.7. I could be +0 if we added "from
>>> __future__ import nonlocal_keyword" (or some such phrasing) to enable
>>> it.
>>
>> With the "from __future__" option, what keeps you from being
>> a full +1 on nonlocal? ? Is there something that makes it a better
>> solution for 3.x than 2.x? ?Just curious about the pros and cons
>> from your point of view.
>
> I think the number of projects that can afford to drop support for 2.6
> is pretty limited, so I think the utility of the feature is thereby
> also limited.

I don't currently have an opinion on this backport proposal, but in
regard to this argument:  if we do not do any 2.x releases after 2.7,
then over time the number of packages that can afford to drop 2.6 support
will grow, yet many will need to retain 2.7 support for much longer.

--David

From benjamin at python.org  Mon Nov  2 21:00:21 2009
From: benjamin at python.org (Benjamin Peterson)
Date: Mon, 2 Nov 2009 14:00:21 -0600
Subject: [Python-Dev] 2.7/3.2 release schedule
Message-ID: <1afaf6160911021200m1bcf27cck4a78db471db038a7@mail.gmail.com>

I've updated PEP 373 with my proposed release schedule:

- 2.7/3.2 alpha 1 2009-12-05
- 2.7/3.2 alpha 2 2010-01-09
- 2.7/3.2 alpha 3 2010-02-06
- 2.7/3.2 alpha 4 2010-03-06
- 2.7/3.2 beta 1 2010-04-03
- 2.7/3.2 beta 2 2010-05-01
- 2.7/3.2 rc1 2010-05-29
- 2.7/3.2 rc2 2010-06-12
- 2.7/3.2 final 2010-06-26

-- 
Regards,
Benjamin

From martin at v.loewis.de  Mon Nov  2 22:17:07 2009
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Mon, 02 Nov 2009 22:17:07 +0100
Subject: [Python-Dev] nonlocal keyword in 2.x?
In-Reply-To: <Pine.LNX.4.64.0911021405420.21420@kimball.webabinitio.net>
References: <da7032ce0910211856g45e45632n550aa5b4a10f9a62@mail.gmail.com>
	<loom.20091028T154811-994@post.gmane.org>	<319e029f0910290115y49be844co78da6636452445f@mail.gmail.com>	<4AE95F18.4040807@gmail.com>	<319e029f0911011326i45bf52ew35b9d7e438df6570@mail.gmail.com>	<02A812F3B69E46D180B45133D52D1F2B@RaymondLaptop1>	<bbaeab100911011557t714e3b4cxcdc91b3fb9a49b84@mail.gmail.com>
	<4AEE9862.8010201@voidspace.org.uk>	<ca471dc20911021001u102da717wcf4da0eada0867e@mail.gmail.com>	<5C4388816CE0428FB27EA69CE77CFCC0@RaymondLaptop1>	<ca471dc20911021009u7485adcdgde8fae04a5061059@mail.gmail.com>
	<Pine.LNX.4.64.0911021405420.21420@kimball.webabinitio.net>
Message-ID: <4AEF4C53.7090902@v.loewis.de>

> I don't currently have an opinion on this backport proposal, but in
> regard to this argument:  if we do not do any 2.x releases after 2.7,
> then over time the number of packages that can afford to drop 2.6 support
> will grow, yet many will need to retain 2.7 support for much longer.

I don't think the argument applies to 2.7 as much as it applied to
earlier releases: 2.7 will have a life time of 18 months perhaps (I
think we still need to decide formally against 2.8, and also decide
when to make the last 2.7 bug fix release). There is some likelihood
that by the time people can agree to drop 2.6 support, 2.7 will be
out of bug fix maintenance already (*). I'm fairly skeptical that people
will be interested in adding new code to specifically clean up their
2.x codebase.

Regards,
Martin

(*) just as it happened with any other release: people are now dropping
support for 2.3, when this got out of security fixes a year ago. By that
measure, people will drop 2.6 support in 2015. I think there are
*really* good chances that many packages will drop 2.6 support along
with dropping 2.x altogether.

From rdmurray at bitdance.com  Mon Nov  2 22:27:10 2009
From: rdmurray at bitdance.com (R. David Murray)
Date: Mon, 2 Nov 2009 16:27:10 -0500 (EST)
Subject: [Python-Dev] nonlocal keyword in 2.x?
In-Reply-To: <4AEF4C53.7090902@v.loewis.de>
References: <da7032ce0910211856g45e45632n550aa5b4a10f9a62@mail.gmail.com> 
	<loom.20091028T154811-994@post.gmane.org>
	<319e029f0910290115y49be844co78da6636452445f@mail.gmail.com>
	<4AE95F18.4040807@gmail.com>
	<319e029f0911011326i45bf52ew35b9d7e438df6570@mail.gmail.com>
	<02A812F3B69E46D180B45133D52D1F2B@RaymondLaptop1>
	<bbaeab100911011557t714e3b4cxcdc91b3fb9a49b84@mail.gmail.com> 
	<4AEE9862.8010201@voidspace.org.uk>
	<ca471dc20911021001u102da717wcf4da0eada0867e@mail.gmail.com>
	<5C4388816CE0428FB27EA69CE77CFCC0@RaymondLaptop1>
	<ca471dc20911021009u7485adcdgde8fae04a5061059@mail.gmail.com>
	<Pine.LNX.4.64.0911021405420.21420@kimball.webabinitio.net>
	<4AEF4C53.7090902@v.loewis.de>
Message-ID: <Pine.LNX.4.64.0911021624110.21420@kimball.webabinitio.net>

On Mon, 2 Nov 2009 at 22:17, "Martin v. L?wis" wrote:
>> I don't currently have an opinion on this backport proposal, but in
>> regard to this argument:  if we do not do any 2.x releases after 2.7,
>> then over time the number of packages that can afford to drop 2.6 support
>> will grow, yet many will need to retain 2.7 support for much longer.
>
> I don't think the argument applies to 2.7 as much as it applied to
> earlier releases: 2.7 will have a life time of 18 months perhaps (I
> think we still need to decide formally against 2.8, and also decide
> when to make the last 2.7 bug fix release). There is some likelihood

I was under the impression that if 2.7 was the last release that it
would be maintained (ie: bugfixed) until we decided 3.x uptake was
"sufficient", and that that might be considerably longer than 18 months.
If that is not the case, then what you say is true.

--David

From guido at python.org  Mon Nov  2 22:33:10 2009
From: guido at python.org (Guido van Rossum)
Date: Mon, 2 Nov 2009 13:33:10 -0800
Subject: [Python-Dev] nonlocal keyword in 2.x?
In-Reply-To: <Pine.LNX.4.64.0911021624110.21420@kimball.webabinitio.net>
References: <da7032ce0910211856g45e45632n550aa5b4a10f9a62@mail.gmail.com> 
	<02A812F3B69E46D180B45133D52D1F2B@RaymondLaptop1>
	<bbaeab100911011557t714e3b4cxcdc91b3fb9a49b84@mail.gmail.com> 
	<4AEE9862.8010201@voidspace.org.uk>
	<ca471dc20911021001u102da717wcf4da0eada0867e@mail.gmail.com> 
	<5C4388816CE0428FB27EA69CE77CFCC0@RaymondLaptop1>
	<ca471dc20911021009u7485adcdgde8fae04a5061059@mail.gmail.com> 
	<Pine.LNX.4.64.0911021405420.21420@kimball.webabinitio.net> 
	<4AEF4C53.7090902@v.loewis.de>
	<Pine.LNX.4.64.0911021624110.21420@kimball.webabinitio.net>
Message-ID: <ca471dc20911021333r168e257eja1e5f79dbe4ae234@mail.gmail.com>

On Mon, Nov 2, 2009 at 1:27 PM, R. David Murray <rdmurray at bitdance.com> wrote:
> On Mon, 2 Nov 2009 at 22:17, "Martin v. L?wis" wrote:
>>>
>>> I don't currently have an opinion on this backport proposal, but in
>>> regard to this argument: ?if we do not do any 2.x releases after 2.7,
>>> then over time the number of packages that can afford to drop 2.6 support
>>> will grow, yet many will need to retain 2.7 support for much longer.
>>
>> I don't think the argument applies to 2.7 as much as it applied to
>> earlier releases: 2.7 will have a life time of 18 months perhaps (I
>> think we still need to decide formally against 2.8, and also decide
>> when to make the last 2.7 bug fix release). There is some likelihood
>
> I was under the impression that if 2.7 was the last release that it
> would be maintained (ie: bugfixed) until we decided 3.x uptake was
> "sufficient", and that that might be considerably longer than 18 months.
> If that is not the case, then what you say is true.

Is it even wort doing a 2.7 release? Isn't the effort better spent on
3.2 alone? (Note, these aren't rhetorical questions. It's well
possible that there are good reasons for pushing along with 2.7. Maybe
considering those reasons will also help answering questions about
whether to backport things like nonlocal.)

-- 
--Guido van Rossum (python.org/~guido)

From martin at v.loewis.de  Mon Nov  2 22:33:38 2009
From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=)
Date: Mon, 02 Nov 2009 22:33:38 +0100
Subject: [Python-Dev] nonlocal keyword in 2.x?
In-Reply-To: <Pine.LNX.4.64.0911021624110.21420@kimball.webabinitio.net>
References: <da7032ce0910211856g45e45632n550aa5b4a10f9a62@mail.gmail.com>
	<loom.20091028T154811-994@post.gmane.org>
	<319e029f0910290115y49be844co78da6636452445f@mail.gmail.com>
	<4AE95F18.4040807@gmail.com>
	<319e029f0911011326i45bf52ew35b9d7e438df6570@mail.gmail.com>
	<02A812F3B69E46D180B45133D52D1F2B@RaymondLaptop1>
	<bbaeab100911011557t714e3b4cxcdc91b3fb9a49b84@mail.gmail.com>
	<4AEE9862.8010201@voidspace.org.uk>
	<ca471dc20911021001u102da717wcf4da0eada0867e@mail.gmail.com>
	<5C4388816CE0428FB27EA69CE77CFCC0@RaymondLaptop1>
	<ca471dc20911021009u7485adcdgde8fae04a5061059@mail.gmail.com>
	<Pine.LNX.4.64.0911021405420.21420@kimball.webabinitio.net>
	<4AEF4C53.7090902@v.loewis.de>
	<Pine.LNX.4.64.0911021624110.21420@kimball.webabinitio.net>
Message-ID: <4AEF5032.6020403@v.loewis.de>

R. David Murray wrote:
> On Mon, 2 Nov 2009 at 22:17, "Martin v. L?wis" wrote:
>>> I don't currently have an opinion on this backport proposal, but in
>>> regard to this argument:  if we do not do any 2.x releases after 2.7,
>>> then over time the number of packages that can afford to drop 2.6
>>> support
>>> will grow, yet many will need to retain 2.7 support for much longer.
>>
>> I don't think the argument applies to 2.7 as much as it applied to
>> earlier releases: 2.7 will have a life time of 18 months perhaps (I
>> think we still need to decide formally against 2.8, and also decide
>> when to make the last 2.7 bug fix release). There is some likelihood
> 
> I was under the impression that if 2.7 was the last release that it
> would be maintained (ie: bugfixed) until we decided 3.x uptake was
> "sufficient", and that that might be considerably longer than 18 months.
> If that is not the case, then what you say is true.

I think that's as-yet undecided. My understanding was that we would
decide, at some point, whether to create a 2.8 release or not, and if
not, that 2.7 would be the final release. To me, this always implied
that there wouldn't be any bug fix releases after 18 months, and no
security releases after five years.

If we would decide to continue doing 2.x releases, the we definitely
wouldn't go beyond 2.9 (because there can't be a 2.10 release,
numerically). At a rate of a release every 18 month, this would mean
we would make bug fix releases for 2.x for no more than 5 years
(from now). But I would really hope that we stop before 2.9.

Regards,
Martin

From solipsis at pitrou.net  Mon Nov  2 22:41:45 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Mon, 2 Nov 2009 21:41:45 +0000 (UTC)
Subject: [Python-Dev] nonlocal keyword in 2.x?
References: <da7032ce0910211856g45e45632n550aa5b4a10f9a62@mail.gmail.com>
	<02A812F3B69E46D180B45133D52D1F2B@RaymondLaptop1>
	<bbaeab100911011557t714e3b4cxcdc91b3fb9a49b84@mail.gmail.com>
	<4AEE9862.8010201@voidspace.org.uk>
	<ca471dc20911021001u102da717wcf4da0eada0867e@mail.gmail.com>
	<5C4388816CE0428FB27EA69CE77CFCC0@RaymondLaptop1>
	<ca471dc20911021009u7485adcdgde8fae04a5061059@mail.gmail.com>
	<Pine.LNX.4.64.0911021405420.21420@kimball.webabinitio.net>
	<4AEF4C53.7090902@v.loewis.de>
	<Pine.LNX.4.64.0911021624110.21420@kimball.webabinitio.net>
	<ca471dc20911021333r168e257eja1e5f79dbe4ae234@mail.gmail.com>
Message-ID: <loom.20091102T223839-797@post.gmane.org>

Guido van Rossum <guido <at> python.org> writes:
> 
> Is it even wort doing a 2.7 release? Isn't the effort better spent on
> 3.2 alone? (Note, these aren't rhetorical questions. It's well
> possible that there are good reasons for pushing along with 2.7. Maybe
> considering those reasons will also help answering questions about
> whether to backport things like nonlocal.)

2.7 has an up-to-date backport of the C IO lib (as well as the memoryview
object), which means it is better for people wanting to ease transition to 3.x.

But of course, as Martin said, few people will want to support 2.7 only and not 
2.6.




From martin at v.loewis.de  Mon Nov  2 22:42:40 2009
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Mon, 02 Nov 2009 22:42:40 +0100
Subject: [Python-Dev] nonlocal keyword in 2.x?
In-Reply-To: <ca471dc20911021333r168e257eja1e5f79dbe4ae234@mail.gmail.com>
References: <da7032ce0910211856g45e45632n550aa5b4a10f9a62@mail.gmail.com>
	<02A812F3B69E46D180B45133D52D1F2B@RaymondLaptop1>
	<bbaeab100911011557t714e3b4cxcdc91b3fb9a49b84@mail.gmail.com>
	<4AEE9862.8010201@voidspace.org.uk>
	<ca471dc20911021001u102da717wcf4da0eada0867e@mail.gmail.com>
	<5C4388816CE0428FB27EA69CE77CFCC0@RaymondLaptop1>
	<ca471dc20911021009u7485adcdgde8fae04a5061059@mail.gmail.com>
	<Pine.LNX.4.64.0911021405420.21420@kimball.webabinitio.net>
	<4AEF4C53.7090902@v.loewis.de>
	<Pine.LNX.4.64.0911021624110.21420@kimball.webabinitio.net>
	<ca471dc20911021333r168e257eja1e5f79dbe4ae234@mail.gmail.com>
Message-ID: <4AEF5250.9090905@v.loewis.de>

> Is it even wort doing a 2.7 release? Isn't the effort better spent on
> 3.2 alone? (Note, these aren't rhetorical questions. It's well
> possible that there are good reasons for pushing along with 2.7. Maybe
> considering those reasons will also help answering questions about
> whether to backport things like nonlocal.)

If no 2.7 is produced, people will ask for the life time of 2.6. I think
users will appreciate not being forced to 3.x as long as possible, which
essentially means as long as we create 2.x releases. It may be that
users could also "live" with 2.6 bug fix releases only - in this case,
we would have to decide for how long these will have to be created.

There is also the issue of committers who already added stuff to the
trunk after the 2.6 release - some may be unhappy to learn that their
stuff is not going to be released (at least not on that branch).

In the end, the EOL for 2.x will need to be decided by BDFL
pronouncement. Users would continue to ask for 2.x releases until
2020 and beyond if all they have to do is ask. Some committers would
likely also want to continue creating 2.x releases for a few more
years (not sure whether that's because they see themselves also as
users, or because they sympathize with the users, or for other reasons).

In any case, a decision not to release 2.7 should be made before
Benjamin produces the first alpha release.

Regards,
Martin


From olemis at gmail.com  Mon Nov  2 22:43:07 2009
From: olemis at gmail.com (Olemis Lang)
Date: Mon, 2 Nov 2009 16:43:07 -0500
Subject: [Python-Dev] A new way to configure logging
In-Reply-To: <24ea26600910070926g6b742492xd6be4fa07a932a0c@mail.gmail.com>
References: <loom.20091007T164835-226@post.gmane.org>
	<24ea26600910070819r364f6715l9164a754bc20b8f4@mail.gmail.com>
	<loom.20091007T174509-394@post.gmane.org>
	<24ea26600910070926g6b742492xd6be4fa07a932a0c@mail.gmail.com>
Message-ID: <24ea26600911021343t4f26a319hbcc8093aeb016937@mail.gmail.com>

On Wed, Oct 7, 2009 at 11:26 AM, Olemis Lang <olemis at gmail.com> wrote:
> On Wed, Oct 7, 2009 at 10:49 AM, Vinay Sajip <vinay_sajip at yahoo.co.uk> wrote:
>> Olemis Lang <olemis <at> gmail.com> writes:
>>
>>> This kind of problems is similar to the one mentioned in another
>>> thread about modifying config options after executing commands. In
>>> that case I mentioned that the same dict-like interface also holds for
>>> WinReg and so on ...
>>>
>>> So thinking big (yes ! I have a big head ! ) I think that the best
>>> approach in this case is to build an adaptation (simple) layer on top
>>> of ConfigParser, JSON, WinReg, PyCode, YAML, ... and build specific
>>> extensions for these formats . Perhaps the proper interfaces are
>>> already there (e.g. `dict`, `shelve` ) and I'm just blind and looking
>>> somewhere else ;o)
>>
>> Sorry, you've lost me :-)
>>
>
> Never mind . I was just trying to say that using `dict`, an adapter
> could be implemented (if not already there ;o) for multiple formats
> like the ones I mentioned above and the solution would cover many
> config formats ... and also I was saying that I was not sure about
> whether this is correct , I mean for *ANY* config formats,

Intention = answer myself + sharing with you => for a better Pyland

Maybe we could learn something from Augeas [1]_ and use their
experience to include some of those features to `stdlib` . It seems to
be an ambitious attempt to put all the pieces in config-land together;
and AFAICS they handle (a lof of) config formats and have an API for
that, and ... CMIIW anyway

.. [1] Augeas
        (http://augeas.net/)

-- 
Regards,

Olemis.

Blog ES: http://simelo-es.blogspot.com/
Blog EN: http://simelo-en.blogspot.com/

Featured article:

From greg.ewing at canterbury.ac.nz  Mon Nov  2 23:19:38 2009
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Tue, 03 Nov 2009 11:19:38 +1300
Subject: [Python-Dev] Retrieve an arbitrary element from a set
 withoutremoving it
In-Reply-To: <20091102052900.GA25160@cskk.homeip.net>
References: <7b0045ed0910301843y7059d316s8a851142b78b7aeb@mail.gmail.com>
	<20091102052900.GA25160@cskk.homeip.net>
Message-ID: <4AEF5AFA.5060806@canterbury.ac.nz>

Cameron Simpson wrote:

> Personally, I'm for the iteration spec in a lot of ways.
> 
> Firstly, a .get()/.pick() that always returns the same element feels
> horrible. Is there anyone here who _likes_ it?

It doesn't sound very useful to me. However, an iterating
version of it doesn't sound any more useful. What would it
gain you? Why not just iterate over the set? Or make a
copy and repeatedly pop() it?

I completely fail to see a use case for this.

-- 
Greg

From ssteinerx at gmail.com  Tue Nov  3 00:24:54 2009
From: ssteinerx at gmail.com (ssteinerX@gmail.com)
Date: Mon, 2 Nov 2009 18:24:54 -0500
Subject: [Python-Dev] 2.7 Release?  2.7 == last of the 2.x line?
Message-ID: <E077843B-919A-4AC7-95D7-0313E4D0328A@gmail.com>

Not that anyone has asked yet, but here's my opinion on two issues  
that have been raised on the python-dev mailing list lately:

	+1 on 2.7 release with as much 3.0 "easy-port goo" as is practicable  
without delaying the product beyond the tentative schedule.  Sooner  
would, of course, be better but I'm sure PEP 373 was produced after  
due consideration of all pertinent factors.

	+1 on 2.7 being the last of the 2.x series.  Enough already!

I, personally, haven't even written my first line of 3.x code, nor  
have I had any good reason to.

If I saw the actual end of the line at 2.7, I would actually start  
looking for 3.x versions of my favorite tools and would be much more  
inclined to help push them along ASAP.

Right now, so much that I use on a daily basis doesn't even have a 3.x  
roadmap, much less any sort of working implementation, that I don't  
see switching to 3.x ever unless the 2.x line ends, and soon!

Just my one vote.

S
  

From foom at fuhm.net  Tue Nov  3 01:26:29 2009
From: foom at fuhm.net (James Y Knight)
Date: Mon, 2 Nov 2009 19:26:29 -0500
Subject: [Python-Dev] 2.7 Release?  2.7 == last of the 2.x line?
In-Reply-To: <E077843B-919A-4AC7-95D7-0313E4D0328A@gmail.com>
References: <E077843B-919A-4AC7-95D7-0313E4D0328A@gmail.com>
Message-ID: <F245110A-2FDF-460B-8E0D-87785F0E6DB0@fuhm.net>


On Nov 2, 2009, at 6:24 PM, ssteinerX at gmail.com wrote:

> 	+1 on 2.7 being the last of the 2.x series.  Enough already!

-1. (not that it matters)

> I, personally, haven't even written my first line of 3.x code, nor  
> have I had any good reason to.

Me neither.

> If I saw the actual end of the line at 2.7, I would actually start  
> looking for 3.x versions of my favorite tools and would be much more  
> inclined to help push them along ASAP.

I'd probably keep using 2.7 to be able to keep using those tools,  
instead.

> Right now, so much that I use on a daily basis doesn't even have a  
> 3.x roadmap, much less any sort of working implementation, that I  
> don't see switching to 3.x ever unless the 2.x line ends, and soon!


I don't see switching to 3.x anytime soon either. But what's the rush?

2.x seems to be a fine edition of Python, why not let it keep going to  
2.8 and beyond? Then you wouldn't have to switch to 3.x at all, and  
that'd save you a ton of work. (and save all the people you will have  
to convince to make a 3.x roadmap and do the port a ton of work too!)

It really sounds like you're saying that switching to 3.x isn't worth  
the cost to you, but you want to force people (including yourself) to  
do so anyways, because ...?

James

From skip at pobox.com  Tue Nov  3 02:56:18 2009
From: skip at pobox.com (skip at pobox.com)
Date: Mon, 2 Nov 2009 19:56:18 -0600
Subject: [Python-Dev] 2.7 Release?  2.7 == last of the 2.x line?
In-Reply-To: <F245110A-2FDF-460B-8E0D-87785F0E6DB0@fuhm.net>
References: <E077843B-919A-4AC7-95D7-0313E4D0328A@gmail.com>
	<F245110A-2FDF-460B-8E0D-87785F0E6DB0@fuhm.net>
Message-ID: <19183.36290.623351.810217@montanaro.dyndns.org>


    James> 2.x seems to be a fine edition of Python, why not let it keep
    James> going to 2.8 and beyond?

Resources would be my guess.  In much the same way that people move on to
newer releases of Windows, Mac OSX or Linux leaving an ever-dwindling group
available to support old versions, the same will be true of Python.  Over
time more and more of the core developers (and end users) will switch to 3.x
leaving fewer and fewer people with the time or inclination to support 2.x.

I think there are probably enough functional differences between 2.6 and 2.7
that releasing 2.7 makes sense.  The discussion at this point should
probably be what to do when 2.7 is out the door.  It makes sense to me to
merge the py3k branch to trunk coincident with the 2.7/3.2 releases and
creation of the release27-maint and release32-maint branches.  3.3 and
future versions would then be released from trunk and there would be no
further 2.x releases.

Skip

From skip at pobox.com  Tue Nov  3 03:26:02 2009
From: skip at pobox.com (skip at pobox.com)
Date: Mon, 2 Nov 2009 20:26:02 -0600
Subject: [Python-Dev] language summit topic: issue tracker
In-Reply-To: <4AEF280E.6040503@v.loewis.de>
References: <bbaeab100910222147u41749107r1032545147fdc2e6@mail.gmail.com>
	<19169.43241.70138.170239@montanaro.dyndns.org>
	<4AEF280E.6040503@v.loewis.de>
Message-ID: <19183.38074.390745.534425@montanaro.dyndns.org>

    >> ... it seems to me that some integration between the issue tracker
    >> and Rietveld would be beneficial.

    Martin> See

    Martin> http://psf.upfronthosting.co.za/roundup/meta/issue247

Cool.  I still haven't used Rietveld for anything, though I am getting
comfortable with Review Board and like the tool support for code reviews.

Skip

From ssteinerx at gmail.com  Tue Nov  3 04:48:46 2009
From: ssteinerx at gmail.com (ssteinerX@gmail.com)
Date: Mon, 2 Nov 2009 22:48:46 -0500
Subject: [Python-Dev] 2.7 Release?  2.7 == last of the 2.x line?
In-Reply-To: <F245110A-2FDF-460B-8E0D-87785F0E6DB0@fuhm.net>
References: <E077843B-919A-4AC7-95D7-0313E4D0328A@gmail.com>
	<F245110A-2FDF-460B-8E0D-87785F0E6DB0@fuhm.net>
Message-ID: <BBB79ADA-7E31-4DD8-AAA5-07537FA74C5A@gmail.com>


On Nov 2, 2009, at 7:26 PM, James Y Knight wrote:
>
> It really sounds like you're saying that switching to 3.x isn't  
> worth the cost to you, but you want to force people (including  
> yourself) to do so anyways, because ...?

Because that's the future of Python, where the developers who make  
real base language improvements are focusing their attention, and  
because the language would advance faster without this split attention.

A better language, i.e. Python 3.x, will become better faster without  
dragging the 2.x series out any longer.

S




From brett at python.org  Tue Nov  3 05:22:56 2009
From: brett at python.org (Brett Cannon)
Date: Mon, 2 Nov 2009 20:22:56 -0800
Subject: [Python-Dev] 2.7 Release? 2.7 == last of the 2.x line?
In-Reply-To: <F245110A-2FDF-460B-8E0D-87785F0E6DB0@fuhm.net>
References: <E077843B-919A-4AC7-95D7-0313E4D0328A@gmail.com> 
	<F245110A-2FDF-460B-8E0D-87785F0E6DB0@fuhm.net>
Message-ID: <bbaeab100911022022p7c9b2f91u27105097be6abd71@mail.gmail.com>

On Mon, Nov 2, 2009 at 16:26, James Y Knight <foom at fuhm.net> wrote:
>
> On Nov 2, 2009, at 6:24 PM, ssteinerX at gmail.com wrote:
>
>> ? ? ? ?+1 on 2.7 being the last of the 2.x series. ?Enough already!
>
> -1. (not that it matters)
>
>> I, personally, haven't even written my first line of 3.x code, nor have I
>> had any good reason to.
>
> Me neither.
>
>> If I saw the actual end of the line at 2.7, I would actually start looking
>> for 3.x versions of my favorite tools and would be much more inclined to
>> help push them along ASAP.
>
> I'd probably keep using 2.7 to be able to keep using those tools, instead.
>
>> Right now, so much that I use on a daily basis doesn't even have a 3.x
>> roadmap, much less any sort of working implementation, that I don't see
>> switching to 3.x ever unless the 2.x line ends, and soon!
>
>
> I don't see switching to 3.x anytime soon either. But what's the rush?
>
> 2.x seems to be a fine edition of Python, why not let it keep going to 2.8
> and beyond? Then you wouldn't have to switch to 3.x at all, and that'd save
> you a ton of work. (and save all the people you will have to convince to
> make a 3.x roadmap and do the port a ton of work too!)
>
> It really sounds like you're saying that switching to 3.x isn't worth the
> cost to you, but you want to force people (including yourself) to do so
> anyways, because ...?

... I think a decent number of us no longer want to maintain the 2.x
series. Honestly, if we go past 2.7 I am simply going to stop
backporting features and bug fixes. It's just too much work keeping so
many branches fixed.

-Brett

From barry at python.org  Tue Nov  3 05:28:27 2009
From: barry at python.org (Barry Warsaw)
Date: Mon, 2 Nov 2009 23:28:27 -0500
Subject: [Python-Dev] 2.7 Release?  2.7 == last of the 2.x line?
In-Reply-To: <BBB79ADA-7E31-4DD8-AAA5-07537FA74C5A@gmail.com>
References: <E077843B-919A-4AC7-95D7-0313E4D0328A@gmail.com>
	<F245110A-2FDF-460B-8E0D-87785F0E6DB0@fuhm.net>
	<BBB79ADA-7E31-4DD8-AAA5-07537FA74C5A@gmail.com>
Message-ID: <D99BA4CF-E16C-47EB-9C60-F1EC5C27CDDC@python.org>

On Nov 2, 2009, at 10:48 PM, ssteinerX at gmail.com wrote:

> A better language, i.e. Python 3.x, will become better faster  
> without dragging the 2.x series out any longer.

If Python 2.7 becomes the last of the 2.x series, then I personally  
favor back porting as many features from Python 3 as possible.  I  
still think doing so will help people migrate to Python 3 by getting  
their Python 2 code base as close to Python 3 as possible without  
biting the ultimate bullet.  E.g. for me "from __future__ import  
absolute_import, unicode_literals" in Python 2.6 has helped quite a bit.

I also think Guido's call for feature freeze makes a lot more sense  
when 2.7 is the EOL.  Let's give people migrating to Python 3 a nice  
big stable target to hit.  Improving the stdlib also gives people a  
big carrot to move.

I think it's also necessary to give third party library and  
application authors as much help as possible to provide Python 3  
compatible software.  Putting together Python tools involves so many  
dependencies in a fairly deep stack that even one unconverted library  
can cause everything above it to stall on Python 2.

-Barry

-------------- next part --------------
A non-text attachment was scrubbed...
Name: PGP.sig
Type: application/pgp-signature
Size: 832 bytes
Desc: This is a digitally signed message part
URL: <http://mail.python.org/pipermail/python-dev/attachments/20091102/c820079e/attachment.pgp>

From ssteinerx at gmail.com  Tue Nov  3 05:51:06 2009
From: ssteinerx at gmail.com (ssteinerX@gmail.com)
Date: Mon, 2 Nov 2009 23:51:06 -0500
Subject: [Python-Dev] 2.7 Release?  2.7 == last of the 2.x line?
In-Reply-To: <D99BA4CF-E16C-47EB-9C60-F1EC5C27CDDC@python.org>
References: <E077843B-919A-4AC7-95D7-0313E4D0328A@gmail.com>
	<F245110A-2FDF-460B-8E0D-87785F0E6DB0@fuhm.net>
	<BBB79ADA-7E31-4DD8-AAA5-07537FA74C5A@gmail.com>
	<D99BA4CF-E16C-47EB-9C60-F1EC5C27CDDC@python.org>
Message-ID: <C5BC05A6-27EF-48A2-9416-64C1AA715321@gmail.com>

On Nov 2, 2009, at 11:28 PM, Barry Warsaw wrote:

> On Nov 2, 2009, at 10:48 PM, ssteinerX at gmail.com wrote:
>
>> A better language, i.e. Python 3.x, will become better faster  
>> without dragging the 2.x series out any longer.
>
> If Python 2.7 becomes the last of the 2.x series, then I personally  
> favor back porting as many features from Python 3 as possible.  I  
> still think doing so will help people migrate to Python 3 by getting  
> their Python 2 code base as close to Python 3 as possible without  
> biting the ultimate bullet.  E.g. for me "from __future__ import  
> absolute_import, unicode_literals" in Python 2.6 has helped quite a  
> bit.

I agree as long as:
	A> 2.7 comes out as soon as possible, even if it's missing helpful  
porting features.
	B> 2.7 will get ONLY new features that make it easier to port to 3.x,  
not every feature added to 3.x or all you've done is make "Python 2.7,  
the Python 3 Version." and core developer time will continue to be  
wasted on Python 2.7 instead of moving forward.

> I also think Guido's call for feature freeze makes a lot more sense  
> when 2.7 is the EOL.  Let's give people migrating to Python 3 a nice  
> big stable target to hit.  Improving the stdlib also gives people a  
> big carrot to move.

Agreed. And specifically NOT porting every shiny new toy from Python 3  
back to 2.7 makes sure the carrots are only in the 3.x series.

> I think it's also necessary to give third party library and  
> application authors as much help as possible to provide Python 3  
> compatible software.  Putting together Python tools involves so many  
> dependencies in a fairly deep stack that even one unconverted  
> library can cause everything above it to stall on Python 2.

And that's one of the reasons my explorations into Python 3 have been  
limited to pretty much nothing.

I don't have time to do a bunch of work only to find out that the tool  
I absolutely have to have to finish a project doesn't have a Python 3  
version or has been crippled to make a Python 3 version.

BeautifulSoup, which I use every day, is one such product.  Since the  
crappy old SMGL parser's gone, BeautifulSoup uses the one that's left  
in Python 3 and it makes BeautifulSoup completely useless for my daily  
work.

That's not to say I can't fix that one particular project, but  
customers get cranky when their project is taking longer than expected  
and "Oh, I'm having to convert a lot of things to use Python 3"  
doesn't seem to improve their mood much.

Thanks,

S


From guido at python.org  Tue Nov  3 06:06:18 2009
From: guido at python.org (Guido van Rossum)
Date: Mon, 2 Nov 2009 22:06:18 -0700
Subject: [Python-Dev] 2.7 Release? 2.7 == last of the 2.x line?
In-Reply-To: <C5BC05A6-27EF-48A2-9416-64C1AA715321@gmail.com>
References: <E077843B-919A-4AC7-95D7-0313E4D0328A@gmail.com> 
	<F245110A-2FDF-460B-8E0D-87785F0E6DB0@fuhm.net>
	<BBB79ADA-7E31-4DD8-AAA5-07537FA74C5A@gmail.com> 
	<D99BA4CF-E16C-47EB-9C60-F1EC5C27CDDC@python.org>
	<C5BC05A6-27EF-48A2-9416-64C1AA715321@gmail.com>
Message-ID: <ca471dc20911022106h45856c22q9ba1cb359997ef1e@mail.gmail.com>

On Mon, Nov 2, 2009 at 9:51 PM, ssteinerX at gmail.com <ssteinerx at gmail.com> wrote:
> BeautifulSoup, which I use every day, is one such product. ?Since the crappy
> old SMGL parser's gone, BeautifulSoup uses the one that's left in Python 3
> and it makes BeautifulSoup completely useless for my daily work.

This sounds an area where some help might be useful. Perhaps the
quickest solution would simply be to copy the old crappy "sgml" based
html parser into a new version of BeautifulSoup. Though I imagine what
it really needs is a "quirks mode" parser that is compatible with the
HTML dialect accepted by, say, IE6. Maybe a summer of code project?

-- 
--Guido van Rossum (python.org/~guido)

From foom at fuhm.net  Tue Nov  3 06:29:15 2009
From: foom at fuhm.net (James Y Knight)
Date: Tue, 3 Nov 2009 00:29:15 -0500
Subject: [Python-Dev] 2.7 Release? 2.7 == last of the 2.x line?
In-Reply-To: <ca471dc20911022106h45856c22q9ba1cb359997ef1e@mail.gmail.com>
References: <E077843B-919A-4AC7-95D7-0313E4D0328A@gmail.com>
	<F245110A-2FDF-460B-8E0D-87785F0E6DB0@fuhm.net>
	<BBB79ADA-7E31-4DD8-AAA5-07537FA74C5A@gmail.com>
	<D99BA4CF-E16C-47EB-9C60-F1EC5C27CDDC@python.org>
	<C5BC05A6-27EF-48A2-9416-64C1AA715321@gmail.com>
	<ca471dc20911022106h45856c22q9ba1cb359997ef1e@mail.gmail.com>
Message-ID: <24CE11F0-415E-4D0B-B241-19E1763A695D@fuhm.net>


On Nov 3, 2009, at 12:06 AM, Guido van Rossum wrote:
> Though I imagine what
> it really needs is a "quirks mode" parser that is compatible with the
> HTML dialect accepted by, say, IE6. Maybe a summer of code project?

Already exists: html5lib.
http://code.google.com/p/html5lib/

Or if you want a faster (yet I think less exact) HTML parser,  
libxml2's HTML parser, via lxml:
http://codespeak.net/lxml/parsing.html#parsing-html

James

From martin at v.loewis.de  Tue Nov  3 07:07:03 2009
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Tue, 03 Nov 2009 07:07:03 +0100
Subject: [Python-Dev] 2.7 Release?  2.7 == last of the 2.x line?
In-Reply-To: <D99BA4CF-E16C-47EB-9C60-F1EC5C27CDDC@python.org>
References: <E077843B-919A-4AC7-95D7-0313E4D0328A@gmail.com>	<F245110A-2FDF-460B-8E0D-87785F0E6DB0@fuhm.net>	<BBB79ADA-7E31-4DD8-AAA5-07537FA74C5A@gmail.com>
	<D99BA4CF-E16C-47EB-9C60-F1EC5C27CDDC@python.org>
Message-ID: <4AEFC887.6020801@v.loewis.de>

Barry Warsaw wrote:
> On Nov 2, 2009, at 10:48 PM, ssteinerX at gmail.com wrote:
> 
>> A better language, i.e. Python 3.x, will become better faster without
>> dragging the 2.x series out any longer.
> 
> If Python 2.7 becomes the last of the 2.x series, then I personally
> favor back porting as many features from Python 3 as possible.

And if *2.6* becomes the last of the 2.x series?

Regards,
Martin

From sturla at molden.no  Tue Nov  3 08:20:14 2009
From: sturla at molden.no (Sturla Molden)
Date: Tue, 03 Nov 2009 08:20:14 +0100
Subject: [Python-Dev] 2.7 Release?  2.7 == last of the 2.x line?
In-Reply-To: <E077843B-919A-4AC7-95D7-0313E4D0328A@gmail.com>
References: <E077843B-919A-4AC7-95D7-0313E4D0328A@gmail.com>
Message-ID: <4AEFD9AE.7090104@molden.no>

I'd just like to mention that the scientific community is highly 
dependent on NumPy. As long as NumPy is not ported to Py3k, migration is 
out of the question. Porting NumPy is not a trivial issue. It might take 
a complete rewrite of the whole C base using Cython. NumPy's ABI is not 
even PEP 3118 compliant. Changing the ABI for Py3k might break extension 
code written for NumPy using C. And scientists tend to write CPU-bound 
routines in languages like C and Fortran, not Python, so that is a major 
issue as well. If we port NumPy to Py3k, everyone using NumPy will have 
to port their C code to the new ABI. There are lot of people stuck with 
Python 2.x for this reason. It does not just affect individual 
scientists, but also large projects like IBM and CERN's blue brain and 
NASA's space telecope. So please, do not cancel 2.x support before we 
have ported NumPy, Matplotlib and most of their dependant extensions to 
Py3k. The community of scientists and engineers using Python is growing, 
but shutting down 2.x support might bring an end to that.

Sturla Molden



From cs at zip.com.au  Tue Nov  3 08:30:46 2009
From: cs at zip.com.au (Cameron Simpson)
Date: Tue, 3 Nov 2009 18:30:46 +1100
Subject: [Python-Dev] Python-Dev Retrieve an arbitrary element from
	a	set?withoutremoving it
In-Reply-To: <loom.20091102T111744-750@post.gmane.org>
References: <loom.20091102T111744-750@post.gmane.org>
Message-ID: <20091103073046.GA31310@cskk.homeip.net>

On 02Nov2009 10:21, Antoine Pitrou <solipsis at pitrou.net> wrote:
| Cameron Simpson <cs <at> zip.com.au> writes:
| > 
| > Personally, I'm for the iteration spec in a lot of ways.
| > 
| > Firstly, a .get()/.pick() that always returns the same element feels
| > horrible. Is there anyone here who _likes_ it?
| 
| I do. Since the caller is asking for an arbitrary element, returning the same
| element is legitimate. It's funny how people seem to have a problem with the
| word "arbitrary" :)
| 
| And I'm -1 on any implicit iteration attaching some state to the object. If you
| want to iterate, there's already an obvious way to it.

Good point. Colour me convinced by this.

I suggest the doco is really clear about the word arbitrary:-)
-- 
Cameron Simpson <cs at zip.com.au> DoD#743
http://www.cskk.ezoshosting.com/cs/

I must really get a thinner pencil. I can't manage this one a bit. It writes
all manner of things I don't intend.
        - rheney at csugrad.cs.vt.edu (Bolo Mk XXXIX)

From nnorwitz at gmail.com  Tue Nov  3 08:41:09 2009
From: nnorwitz at gmail.com (Neal Norwitz)
Date: Tue, 3 Nov 2009 00:41:09 -0700
Subject: [Python-Dev] thanks to everyone cleaning up the tests
In-Reply-To: <5c6f2a5d0911020118h4cfee72cl430ef45a1cbf9bc7@mail.gmail.com>
References: <bbaeab100910311447l656c4f17tee3d88225e1cdcfc@mail.gmail.com>
	<5c6f2a5d0911020118h4cfee72cl430ef45a1cbf9bc7@mail.gmail.com>
Message-ID: <ee2a432c0911022341s14738c83obb3bc0c25cac290c@mail.gmail.com>

On Mon, Nov 2, 2009 at 2:18 AM, Mark Dickinson <dickinsm at gmail.com> wrote:
> On Sat, Oct 31, 2009 at 9:47 PM, Brett Cannon <brett at python.org> wrote:
>> Just wanted to publicly thank everyone who has been causing all the
>> checkins to fix and stabilize the test suite (I think it's mostly
>> Antoine and Mark, but I could be missing somebody; I'm under a
>> deadline so only have marginal higher brain functionality).
>
> Not guilty, your honour. ?Blame Antoine and David. ?:)
>
> David's new buildslave seems to be making a difference, too.

It's really been awesome to see all the work improving the test suite.
 This will make it easier to implement change without fear of breaking
everything.  These changes will reap benefits for a long time to come.

Kudos for picking up this work and to Brett for calling you out. :-)

Thanks,
n

From ubershmekel at gmail.com  Tue Nov  3 09:20:52 2009
From: ubershmekel at gmail.com (Yuvgoog Greenle)
Date: Tue, 3 Nov 2009 10:20:52 +0200
Subject: [Python-Dev] Retrieve an arbitrary element from a set
	withoutremoving it
In-Reply-To: <4AEF5AFA.5060806@canterbury.ac.nz>
References: <7b0045ed0910301843y7059d316s8a851142b78b7aeb@mail.gmail.com>
	<20091102052900.GA25160@cskk.homeip.net>
	<4AEF5AFA.5060806@canterbury.ac.nz>
Message-ID: <9d153b7c0911030020n336fd15bobeea4ea713cd8b23@mail.gmail.com>

On Tue, Nov 3, 2009 at 12:19 AM, Greg Ewing <greg.ewing at canterbury.ac.nz>wrote:

> Cameron Simpson wrote:
>
>  Personally, I'm for the iteration spec in a lot of ways.
>>
>> Firstly, a .get()/.pick() that always returns the same element feels
>> horrible. Is there anyone here who _likes_ it?
>>
>
>
State might cause people to use this to iterate which would be just plain
wrong. The 2 things I have a bad feeling about is:
1. random.choice could be a pythonic alternative to what some have mentioned
here but it doesn't work on sets. The following code raises TypeError: 'set'
object does not support indexing:
    import random
    random.choice(set([1,2,3,4,5,6]))
this is kinda ironic: http://en.wikipedia.org/wiki/Axiom_of_choice
2. If I store objects in a set and modify their attributes I have no O(1)
way of getting the objects back if I stumble upon an equivalent object. In
cases like that I'd have to either iterate over the set or use a dict with
key == value. I have a feeling the "get" or "peek" api could cater to this
need. A use case for this could be an implementation of a cookie jar with a
set of cookies where equivalence is defined by the name and domain.

--yuv
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20091103/3163271b/attachment.htm>

From greg at krypto.org  Tue Nov  3 09:29:15 2009
From: greg at krypto.org (Gregory P. Smith)
Date: Tue, 3 Nov 2009 00:29:15 -0800
Subject: [Python-Dev] language summit topic: issue tracker
In-Reply-To: <4AE17C12.1080007@gmail.com>
References: <bbaeab100910222147u41749107r1032545147fdc2e6@mail.gmail.com> 
	<4AE17C12.1080007@gmail.com>
Message-ID: <52dc1c820911030029m72b10895xae947b60068ad0c4@mail.gmail.com>

On Fri, Oct 23, 2009 at 1:49 AM, Nick Coghlan <ncoghlan at gmail.com> wrote:
> Brett Cannon wrote:
>> Another summit, another potential time to see if people want to change
>> anything about the issue tracker. I would bring up:
>>
>> - Dropping Stage in favor of some keywords (e.g. 'needs unit test',
>> 'needs docs')
>> - Adding a freestyle text box to delineate which, if any, stdlib module
>> is the cause of a bug and tie that into Misc/maintainers.rst; would
>> potentially scale back the Component box
>
> +lots on adding a module field (independent of automatically adding
> maintainers to the nosy list, it would assist in "I just did a major
> cleanup of module X, are there any old bugs I can kill off").

yet another feature request or two to be lost to a mailing list thread
along those lines:

Maintainer or not i'd like to be able to setup triggers so that i'm
automatically cc'ed on any bugs matching a simple search query i
specify.

The email sent out to people cc'ed when a new bug is opened and
unassigned should have a simple links in it when cc'ed to someone who
can be assigned bugs:  'Assign to me' that if followed will assign
that bug to them without requiring a login.

>
> Of course, it will take a while for the field to be filled in on
> existing issues, but even having it be possible would be very nice.
>
> Cheers,
> Nick.
>
> --
> Nick Coghlan ? | ? ncoghlan at gmail.com ? | ? Brisbane, Australia
> ---------------------------------------------------------------
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/greg%40krypto.org
>

From mal at egenix.com  Tue Nov  3 10:10:59 2009
From: mal at egenix.com (M.-A. Lemburg)
Date: Tue, 03 Nov 2009 10:10:59 +0100
Subject: [Python-Dev] nonlocal keyword in 2.x?
In-Reply-To: <loom.20091102T223839-797@post.gmane.org>
References: <da7032ce0910211856g45e45632n550aa5b4a10f9a62@mail.gmail.com>	<02A812F3B69E46D180B45133D52D1F2B@RaymondLaptop1>	<bbaeab100911011557t714e3b4cxcdc91b3fb9a49b84@mail.gmail.com>	<4AEE9862.8010201@voidspace.org.uk>	<ca471dc20911021001u102da717wcf4da0eada0867e@mail.gmail.com>	<5C4388816CE0428FB27EA69CE77CFCC0@RaymondLaptop1>	<ca471dc20911021009u7485adcdgde8fae04a5061059@mail.gmail.com>	<Pine.LNX.4.64.0911021405420.21420@kimball.webabinitio.net>	<4AEF4C53.7090902@v.loewis.de>	<Pine.LNX.4.64.0911021624110.21420@kimball.webabinitio.net>	<ca471dc20911021333r168e257eja1e5f79dbe4ae234@mail.gmail.com>
	<loom.20091102T223839-797@post.gmane.org>
Message-ID: <4AEFF3A3.3020103@egenix.com>

Antoine Pitrou wrote:
> Guido van Rossum <guido <at> python.org> writes:
>>
>> Is it even wort doing a 2.7 release? Isn't the effort better spent on
>> 3.2 alone? (Note, these aren't rhetorical questions. It's well
>> possible that there are good reasons for pushing along with 2.7. Maybe
>> considering those reasons will also help answering questions about
>> whether to backport things like nonlocal.)
> 
> 2.7 has an up-to-date backport of the C IO lib (as well as the memoryview
> object), which means it is better for people wanting to ease transition to 3.x.
> 
> But of course, as Martin said, few people will want to support 2.7 only and not 
> 2.6.

Since 2.7 will be closer to 3.2 than 2.6, the result will more likely
be: many people will want to support 2.7 and 3.x - which is really
what we should be after.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Nov 03 2009)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try our new mxODBC.Connect Python Database Interface for free ! ::::


   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
           Registered at Amtsgericht Duesseldorf: HRB 46611
               http://www.egenix.com/company/contact/

From fuzzyman at voidspace.org.uk  Tue Nov  3 10:13:52 2009
From: fuzzyman at voidspace.org.uk (Michael Foord)
Date: Tue, 03 Nov 2009 10:13:52 +0100
Subject: [Python-Dev] 2.7 Release?  2.7 == last of the 2.x line?
In-Reply-To: <4AEFD9AE.7090104@molden.no>
References: <E077843B-919A-4AC7-95D7-0313E4D0328A@gmail.com>
	<4AEFD9AE.7090104@molden.no>
Message-ID: <4AEFF450.9060007@voidspace.org.uk>

Sturla Molden wrote:
> I'd just like to mention that the scientific community is highly 
> dependent on NumPy. As long as NumPy is not ported to Py3k, migration 
> is out of the question. Porting NumPy is not a trivial issue. It might 
> take a complete rewrite of the whole C base using Cython. NumPy's ABI 
> is not even PEP 3118 compliant. Changing the ABI for Py3k might break 
> extension code written for NumPy using C. And scientists tend to write 
> CPU-bound routines in languages like C and Fortran, not Python, so 
> that is a major issue as well. If we port NumPy to Py3k, everyone 
> using NumPy will have to port their C code to the new ABI. There are 
> lot of people stuck with Python 2.x for this reason. It does not just 
> affect individual scientists, but also large projects like IBM and 
> CERN's blue brain and NASA's space telecope. So please, do not cancel 
> 2.x support before we have ported NumPy, Matplotlib and most of their 
> dependant extensions to Py3k.

What will it take to *start* the port? (Or is it already underway?) For 
many projects I fear that it is only the impending obsolescence (real 
rather than theoretical) of Python 2 that will convince projects to port.

Python 2.X is not about to 'stop working', but there will come a point 
where it will 'stop being worked on'.

All the best,

Michael
> The community of scientists and engineers using Python is growing, but 
> shutting down 2.x support might bring an end to that.
>
> Sturla Molden
>
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: 
> http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk 
>


-- 
http://www.ironpythoninaction.com/


From ncoghlan at gmail.com  Tue Nov  3 10:21:43 2009
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Tue, 03 Nov 2009 19:21:43 +1000
Subject: [Python-Dev] language summit topic: issue tracker
In-Reply-To: <4AEF2779.5040402@v.loewis.de>
References: <bbaeab100910222147u41749107r1032545147fdc2e6@mail.gmail.com>
	<4AE17C12.1080007@gmail.com> <4AEF2779.5040402@v.loewis.de>
Message-ID: <4AEFF627.4060601@gmail.com>

Martin v. L?wis wrote:
>> +lots on adding a module field (independent of automatically adding
>> maintainers to the nosy list, it would assist in "I just did a major
>> cleanup of module X, are there any old bugs I can kill off").
> 
> Link (1:1) or Multilink (1:n)? What is the impact on the Component field?

I was thinking multilink, and leaving component alone - the module field
would largely come into play when the component was just the "Lib"
catch-all.

> Would you be willing to manage the field (in the sense of managing the
> set of values)? If so, please send me a list of values.

I would suggest just using the module index from the documentation to
seed any such list of modules in the tracker:
http://docs.python.org/modindex.html

Packages could generally be left as a single entry in the list. The only
exception I think is that there should be an "xml.etree" entry separate
from the main "xml" entry, and perhaps a separate entry for "os.path".

Deprecated modules could either be left out of the list, or else moved
to appear at the end.

Regards,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------

From cournape at gmail.com  Tue Nov  3 10:55:39 2009
From: cournape at gmail.com (David Cournapeau)
Date: Tue, 3 Nov 2009 18:55:39 +0900
Subject: [Python-Dev] 2.7 Release? 2.7 == last of the 2.x line?
In-Reply-To: <4AEFF450.9060007@voidspace.org.uk>
References: <E077843B-919A-4AC7-95D7-0313E4D0328A@gmail.com>
	<4AEFD9AE.7090104@molden.no> <4AEFF450.9060007@voidspace.org.uk>
Message-ID: <5b8d13220911030155x57615540x1f59e4b95b6c2f56@mail.gmail.com>

On Tue, Nov 3, 2009 at 6:13 PM, Michael Foord <fuzzyman at voidspace.org.uk> wrote:
> Sturla Molden wrote:
>>
>> I'd just like to mention that the scientific community is highly dependent
>> on NumPy. As long as NumPy is not ported to Py3k, migration is out of the
>> question. Porting NumPy is not a trivial issue. It might take a complete
>> rewrite of the whole C base using Cython. NumPy's ABI is not even PEP 3118
>> compliant. Changing the ABI for Py3k might break extension code written for
>> NumPy using C. And scientists tend to write CPU-bound routines in languages
>> like C and Fortran, not Python, so that is a major issue as well. If we port
>> NumPy to Py3k, everyone using NumPy will have to port their C code to the
>> new ABI. There are lot of people stuck with Python 2.x for this reason. It
>> does not just affect individual scientists, but also large projects like IBM
>> and CERN's blue brain and NASA's space telecope. So please, do not cancel
>> 2.x support before we have ported NumPy, Matplotlib and most of their
>> dependant extensions to Py3k.
>
> What will it take to *start* the port? (Or is it already underway?) For many
> projects I fear that it is only the impending obsolescence (real rather than
> theoretical) of Python 2 that will convince projects to port.

I feel the same way. Given how much resources it will take to port to
py3k, I doubt the port will happen soon. I don't know what other numpy
developers think, but I consider py3k to simply not worth the hassle -
I know we will have to port eventually, though.

To answer your question, the main issues are:
 - are two branches are necessary or not ? If two branches are
necessary, I think we simply do not have the resources at the moment.
 - how to maintain a compatible C API across 2.x and 3.x
 - is it practically possible to support and maintain numpy from 2.4
to 3.x ? For example, I don't think the python 2.6 py3k warnings are
very useful when you need to maintain compatibility with 2.4 and 2.5.

There is also little documentation on how to port a significant C
codebase to py3k.

David

From python at rcn.com  Tue Nov  3 10:58:27 2009
From: python at rcn.com (Raymond Hettinger)
Date: Tue, 3 Nov 2009 01:58:27 -0800
Subject: [Python-Dev] 2.7 Release?  2.7 == last of the 2.x line?
References: <E077843B-919A-4AC7-95D7-0313E4D0328A@gmail.com><4AEFD9AE.7090104@molden.no>
	<4AEFF450.9060007@voidspace.org.uk>
Message-ID: <79EADA413EFB44F6864EF71D5B830088@RaymondLaptop1>


[Michael Foord]
> What will it take to *start* the port? (Or is it already underway?) For many projects I fear that it is only the impending 
> obsolescence (real rather than theoretical) of Python 2 that will convince projects to port.

FWIW, I do not buy into the several premises that have arisen in this thread:

* For 3.x to succeed, something bad has to happen to 2.x.  (which in my book translates to intentionally harming 2.x users, either 
through neglect or force, in order to bait them into switching to 3.x).

* Core developers will are losing time supporting 2.x. (backports take some time but it
is small in comparison to getting a patch to work in the first place -- if anyone can comment on this assertion,  it is the people 
who have been doing it already (such as AP, MD, BP, GB, and myself)).

* That 3.x has proven its readiness to supplant 2.x.  (It hasn't been exercised that heavily and there are a lot of things that may 
or may not prove to be successful in the end -- bytes/text issues, tuple comparison challenges, new io, mapping views with set 
operations, etc).

In all these matters, I think the users should get a vote.  And that vote should be cast with their decision to stay with 2.x, or 
switch to 3.x, or try to support both.  We should not muck with their rational decision making by putting "carrots" in one pile and 
abandoning the other.


Raymond


P.S.  I found it curious that one of the strongest proponents of killing 2.x also mentioned that he has never written a line of 3.x 
code.   Since this discussion is a matter of great consequence, I would hope that advocates will only take informed positions --  
this isn't really time for shooting from the hip and killing 2.x.


From p.f.moore at gmail.com  Tue Nov  3 12:23:33 2009
From: p.f.moore at gmail.com (Paul Moore)
Date: Tue, 3 Nov 2009 11:23:33 +0000
Subject: [Python-Dev] 2.7 Release? 2.7 == last of the 2.x line?
In-Reply-To: <79EADA413EFB44F6864EF71D5B830088@RaymondLaptop1>
References: <E077843B-919A-4AC7-95D7-0313E4D0328A@gmail.com>
	<4AEFD9AE.7090104@molden.no> <4AEFF450.9060007@voidspace.org.uk>
	<79EADA413EFB44F6864EF71D5B830088@RaymondLaptop1>
Message-ID: <79990c6b0911030323x47a9f008ue7256c56e88c0ee8@mail.gmail.com>

2009/11/3 Raymond Hettinger <python at rcn.com>:
> In all these matters, I think the users should get a vote. ?And that vote
> should be cast with their decision to stay with 2.x, or switch to 3.x, or
> try to support both. ?We should not muck with their rational decision making
> by putting "carrots" in one pile and abandoning the other.

Agreed (up to a point).

The biggest issue to my mind is that adoption by the ultimate end
users is significantly hampered by the fact that big projects like
Twisted, numpy and the like, have no current plans to move to Python
3. Even end users with a reasonable level of coding expertise don't
have the time or resources to offer much in the way of help with a
port, when the project as a whole isn't interested in starting the
process.

At the moment, it seems to me that this is the biggest blocker to
Python 3 adoption. And it's a chicken and egg situation - I don't use
Python 3, so I don't test the new features, so the projects I need see
little take-up, so I can't use them in Python 3, so I don't use Python
3...

And while I know I can run Python 2.x and Python 3.x side by side, at
the end of the day, I want to just be able to type "python" to get my
interpreter.

I don't know how to solve this (assuming that "just wait" isn't going
to do it). Maybe the core devs will have to offer resource to some of
the key projects to get things moving (but as this is a volunteer
effort, that isn't something that "just happens" either...)

Paul.

From solipsis at pitrou.net  Tue Nov  3 12:40:38 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Tue, 3 Nov 2009 11:40:38 +0000 (UTC)
Subject: [Python-Dev] 2.7 Release?  2.7 == last of the 2.x line?
References: <E077843B-919A-4AC7-95D7-0313E4D0328A@gmail.com>
	<4AEFD9AE.7090104@molden.no>
Message-ID: <loom.20091103T123514-202@post.gmane.org>

Sturla Molden <sturla <at> molden.no> writes:
> 
> Porting NumPy is not a trivial issue. It might take 
> a complete rewrite of the whole C base using Cython.

I don't see why they would need a rewrite. Little of the C API has changed
between 2.x and 3.x. Cython itself is supposed to support both 2.x and 3.x,
isn't it?

> NumPy's ABI is not even PEP 3118 compliant.

That's interesting, because PEP 3118 was pushed mainly by a prominent member of
the NumPy community and some of its features are almost dedicated to NumPy.

Regards

Antoine.



From solipsis at pitrou.net  Tue Nov  3 12:43:30 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Tue, 3 Nov 2009 11:43:30 +0000 (UTC)
Subject: [Python-Dev] 2.7 Release? 2.7 == last of the 2.x line?
References: <E077843B-919A-4AC7-95D7-0313E4D0328A@gmail.com>
	<4AEFD9AE.7090104@molden.no> <4AEFF450.9060007@voidspace.org.uk>
	<5b8d13220911030155x57615540x1f59e4b95b6c2f56@mail.gmail.com>
Message-ID: <loom.20091103T124246-962@post.gmane.org>

David Cournapeau <cournape <at> gmail.com> writes:
> 
> To answer your question, the main issues are:
>  - are two branches are necessary or not ? If two branches are
> necessary, I think we simply do not have the resources at the moment.
>  - how to maintain a compatible C API across 2.x and 3.x
>  - is it practically possible to support and maintain numpy from 2.4
> to 3.x ? For example, I don't think the python 2.6 py3k warnings are
> very useful when you need to maintain compatibility with 2.4 and 2.5.

You should ask all those questions on the dedicated mailing-list:
http://mail.python.org/mailman/listinfo/python-porting

Regards

Antoine.



From rdmurray at bitdance.com  Tue Nov  3 13:41:04 2009
From: rdmurray at bitdance.com (R. David Murray)
Date: Tue, 3 Nov 2009 07:41:04 -0500 (EST)
Subject: [Python-Dev] 2.7 Release? 2.7 == last of the 2.x line?
In-Reply-To: <ca471dc20911022106h45856c22q9ba1cb359997ef1e@mail.gmail.com>
References: <E077843B-919A-4AC7-95D7-0313E4D0328A@gmail.com> 
	<F245110A-2FDF-460B-8E0D-87785F0E6DB0@fuhm.net>
	<BBB79ADA-7E31-4DD8-AAA5-07537FA74C5A@gmail.com>
	<D99BA4CF-E16C-47EB-9C60-F1EC5C27CDDC@python.org>
	<C5BC05A6-27EF-48A2-9416-64C1AA715321@gmail.com>
	<ca471dc20911022106h45856c22q9ba1cb359997ef1e@mail.gmail.com>
Message-ID: <Pine.LNX.4.64.0911030737260.5039@kimball.webabinitio.net>

On Mon, 2 Nov 2009 at 22:06, Guido van Rossum wrote:
> On Mon, Nov 2, 2009 at 9:51 PM, ssteinerX at gmail.com <ssteinerx at gmail.com> wrote:
>> BeautifulSoup, which I use every day, is one such product. ?Since the crappy
>> old SMGL parser's gone, BeautifulSoup uses the one that's left in Python 3
>> and it makes BeautifulSoup completely useless for my daily work.
>
> This sounds an area where some help might be useful. Perhaps the
> quickest solution would simply be to copy the old crappy "sgml" based
> html parser into a new version of BeautifulSoup. Though I imagine what
> it really needs is a "quirks mode" parser that is compatible with the
> HTML dialect accepted by, say, IE6. Maybe a summer of code project?

It's not a matter of quirks.  It's a matter of being able to parse
truly broken html/xml, which browsers unfortunately do too well
for everyone else's sanity.

So, call it a "sloppy mode" parser, and then yes, that would solve the
problem.

--David (RDM)

From barry at python.org  Tue Nov  3 13:55:34 2009
From: barry at python.org (Barry Warsaw)
Date: Tue, 3 Nov 2009 07:55:34 -0500
Subject: [Python-Dev] 2.7 Release?  2.7 == last of the 2.x line?
In-Reply-To: <4AEFC887.6020801@v.loewis.de>
References: <E077843B-919A-4AC7-95D7-0313E4D0328A@gmail.com>	<F245110A-2FDF-460B-8E0D-87785F0E6DB0@fuhm.net>	<BBB79ADA-7E31-4DD8-AAA5-07537FA74C5A@gmail.com>
	<D99BA4CF-E16C-47EB-9C60-F1EC5C27CDDC@python.org>
	<4AEFC887.6020801@v.loewis.de>
Message-ID: <3DE1D8FB-896E-441D-937D-AB63EB3A8E22@python.org>

On Nov 3, 2009, at 1:07 AM, Martin v. L?wis wrote:

> Barry Warsaw wrote:
>> On Nov 2, 2009, at 10:48 PM, ssteinerX at gmail.com wrote:
>>
>>> A better language, i.e. Python 3.x, will become better faster  
>>> without
>>> dragging the 2.x series out any longer.
>>
>> If Python 2.7 becomes the last of the 2.x series, then I personally
>> favor back porting as many features from Python 3 as possible.
>
> And if *2.6* becomes the last of the 2.x series?

Then clearly we can't back port features.

I'd like to read some case studies of people who have migrated  
applications from 2.6 to 3.0.  Having just gone through a 2 week  
sprint to migrate Launchpad from 2.4 to 2.6, and only making it to  
2.5, I can say that I was unpleasantly surprised at the amount of work  
it took.  A lot of that was working out the dependency upgrades, with  
some amount of fixing our code (mostly tests) for things that have  
changed (e.g. exception print/str format).  We didn't make it to  
Python 2.6 because dealing with deprecation warnings for sha, md5, and  
sets (a little in our code but tons in our dependencies) consumed most  
of our remaining time.

Given another week or so I think we would have made it to Python 2.6,  
but I'm not at all confident that that would have been a good enough  
platform to attempt an upgrade to Python 3, even if all of our very  
numerous large dependencies were available for Python 3.  Maybe it  
wouldn't be so bad, but my recent experience informs me that I'm  
probably being too optimistic rather than too pessimistic.

-Barry

-------------- next part --------------
A non-text attachment was scrubbed...
Name: PGP.sig
Type: application/pgp-signature
Size: 832 bytes
Desc: This is a digitally signed message part
URL: <http://mail.python.org/pipermail/python-dev/attachments/20091103/c11881f4/attachment.pgp>

From barry at python.org  Tue Nov  3 14:09:06 2009
From: barry at python.org (Barry Warsaw)
Date: Tue, 3 Nov 2009 08:09:06 -0500
Subject: [Python-Dev] 2.7 Release?  2.7 == last of the 2.x line?
In-Reply-To: <C5BC05A6-27EF-48A2-9416-64C1AA715321@gmail.com>
References: <E077843B-919A-4AC7-95D7-0313E4D0328A@gmail.com>
	<F245110A-2FDF-460B-8E0D-87785F0E6DB0@fuhm.net>
	<BBB79ADA-7E31-4DD8-AAA5-07537FA74C5A@gmail.com>
	<D99BA4CF-E16C-47EB-9C60-F1EC5C27CDDC@python.org>
	<C5BC05A6-27EF-48A2-9416-64C1AA715321@gmail.com>
Message-ID: <F50DEBD8-309E-4D4F-AAB4-D8996FA8E7D6@python.org>

On Nov 2, 2009, at 11:51 PM, ssteinerX at gmail.com wrote:

> I agree as long as:
> 	A> 2.7 comes out as soon as possible, even if it's missing helpful  
> porting features.
> 	B> 2.7 will get ONLY new features that make it easier to port to  
> 3.x, not every feature added to 3.x or all you've done is make  
> "Python 2.7, the Python 3 Version." and core developer time will  
> continue to be wasted on Python 2.7 instead of moving forward.

I'm not sure I agree with that core developer time will be "wasted on  
Python 2.7 instead of moving forward".  However, I completely  
understand core developers seeing Python 2.x as a dead end and not  
wanting to work on it.  If that's a real issue, we should acknowledge  
that as a factor in the decision.  This is a volunteer organization  
and if the majority of developers are sick and tired of Python 2, then  
it absolutely makes no sense to slog through a Python 2.7 release.   
I'd much rather take all the enthusiastic energy that decision will  
reclaim and focus it on, oh, Python 3's email package instead <wink>.

>> I also think Guido's call for feature freeze makes a lot more sense  
>> when 2.7 is the EOL.  Let's give people migrating to Python 3 a  
>> nice big stable target to hit.  Improving the stdlib also gives  
>> people a big carrot to move.
>
> Agreed. And specifically NOT porting every shiny new toy from Python  
> 3 back to 2.7 makes sure the carrots are only in the 3.x series.

Python 3's got enough carrots and sticks already.  One huge carrot  
that will never make it back into Python 2 is bytes/strings of  
course.  The huge stick is that Python 2 is end-of-lifed, if not now,  
eventually.  It isn't going to get a reprieve.  Everyone knows that  
everyone will have to get to Python 3.  The question is, what can we  
as a community  do to make that inevitability as easy to swallow as  
possible?

>> I think it's also necessary to give third party library and  
>> application authors as much help as possible to provide Python 3  
>> compatible software.  Putting together Python tools involves so  
>> many dependencies in a fairly deep stack that even one unconverted  
>> library can cause everything above it to stall on Python 2.
>
> And that's one of the reasons my explorations into Python 3 have  
> been limited to pretty much nothing.
>
> I don't have time to do a bunch of work only to find out that the  
> tool I absolutely have to have to finish a project doesn't have a  
> Python 3 version or has been crippled to make a Python 3 version.

Unfortunately, I think we have to do those explorations, fail, hit  
roadblocks, complain, etc. but most importantly identify the packages  
that need to be ported.  Then work with those package authors to make  
the upgrades happen.  And improve Python and Pythonic tools so that  
migrations can go smoothly.

Speaking as a package author, I know how much work it is just to get a  
bug fix release out.  The three lines of code fix means 50 lines of  
test writing, a half a day of documenting, packaging, uploading, and  
announcing.  Porting even one of my packages to Python 3 is a  
significant undertaking which frankly I don't have the cycles for.   
Anything large and complicated is hopeless.  Witness how long and  
difficult it's been just to get a standard library module updated  
(email) and you get a sense of how much work it will be to get an  
entire stack of code onto Python 3.

> BeautifulSoup, which I use every day, is one such product.  Since  
> the crappy old SMGL parser's gone, BeautifulSoup uses the one that's  
> left in Python 3 and it makes BeautifulSoup completely useless for  
> my daily work.
>
> That's not to say I can't fix that one particular project, but  
> customers get cranky when their project is taking longer than  
> expected and "Oh, I'm having to convert a lot of things to use  
> Python 3" doesn't seem to improve their mood much.

I completely agree.  What happens when your application depends on a  
half dozen Zope packages, Twisted, and 15 or 20 other established,  
mature packages?  It's a daunting prospect.

-Barry

-------------- next part --------------
A non-text attachment was scrubbed...
Name: PGP.sig
Type: application/pgp-signature
Size: 832 bytes
Desc: This is a digitally signed message part
URL: <http://mail.python.org/pipermail/python-dev/attachments/20091103/278fa416/attachment-0001.pgp>

From mal at egenix.com  Tue Nov  3 14:21:34 2009
From: mal at egenix.com (M.-A. Lemburg)
Date: Tue, 03 Nov 2009 14:21:34 +0100
Subject: [Python-Dev] 2.7 Release?  2.7 == last of the 2.x line?
In-Reply-To: <79EADA413EFB44F6864EF71D5B830088@RaymondLaptop1>
References: <E077843B-919A-4AC7-95D7-0313E4D0328A@gmail.com><4AEFD9AE.7090104@molden.no>	<4AEFF450.9060007@voidspace.org.uk>
	<79EADA413EFB44F6864EF71D5B830088@RaymondLaptop1>
Message-ID: <4AF02E5E.4020501@egenix.com>

Raymond Hettinger wrote:
> In all these matters, I think the users should get a vote.  And that
> vote should be cast with their decision to stay with 2.x, or switch to
> 3.x, or try to support both.  We should not muck with their rational
> decision making by putting "carrots" in one pile and abandoning the other.

I don't think users will really go for carrots. Python 2.x is mature
enough already and at least the users I know are really happy with it
(including myself).

IMHO, the main benefit of backporting features from 3.x to 2.x is
to make the transition from 2.x to 3.x a gradual one based on evolution
rather than revolution.

The other aspect is maintenance. Users do care about bug fixes and
security patches. They also care about fixes needed to make Python
work on new platforms (such as Windows 7) - mainly to keep their
existing code running on these new platforms.

The question is: Who is going to continue working on such 2.x
releases, review patches, etc. ?

If there are no core developers willing to do this, it's likely
not going to happen.

OTOH, I'm sure that companies who have invested in Python 2.x
applications will gladly pay a yearly fee to the PSF to have
this done for them.

It's simply a whole lot cheaper than to port a few 100K lines of
Python/C code, not to mention having to wait for all the used
3rd party libs to get ported as well.

The PSF could then pay a core developer to take care of the
extra Python 2.x releases.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Nov 03 2009)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try our new mxODBC.Connect Python Database Interface for free ! ::::


   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
           Registered at Amtsgericht Duesseldorf: HRB 46611
               http://www.egenix.com/company/contact/

From cournape at gmail.com  Tue Nov  3 14:26:00 2009
From: cournape at gmail.com (David Cournapeau)
Date: Tue, 3 Nov 2009 22:26:00 +0900
Subject: [Python-Dev] 2.7 Release? 2.7 == last of the 2.x line?
In-Reply-To: <loom.20091103T123514-202@post.gmane.org>
References: <E077843B-919A-4AC7-95D7-0313E4D0328A@gmail.com>
	<4AEFD9AE.7090104@molden.no> <loom.20091103T123514-202@post.gmane.org>
Message-ID: <5b8d13220911030526y7dce8ce0gecb427ea009eaf1a@mail.gmail.com>

On Tue, Nov 3, 2009 at 8:40 PM, Antoine Pitrou <solipsis at pitrou.net> wrote:
> Sturla Molden <sturla <at> molden.no> writes:
>>
>> Porting NumPy is not a trivial issue. It might take
>> a complete rewrite of the whole C base using Cython.
>
> I don't see why they would need a rewrite.

(let me know if those numpy-specific discussions are considered OT0

There is certainly no need for a full rewrite, no. I am still unclear
on the range of things to change for 3.x, but the C changes are not
small, especially since numpy uses "dark" areas of python C extension.
The long vs int, strings vs bytes will take some time.

AFAIK, the only thing which has been attempted so far is porting our
own distutils extension to python 3.x, but I have not integrated those
changes yet.

> between 2.x and 3.x. Cython itself is supposed to support both 2.x and 3.x,
> isn't it?

Yes - but no numpy code use cython ATM, except for the random
generators, which would almost certainly be trivial to convert.

The idea which has been discussed so far is that for *some* code which
need significant changes or rewrite, using cython instead of C may be
beneficial, as it would give the 3.x code "for free". Having more
cython and less C could also bring more contributors - that would
actually be the biggest incentive, as the number of people who know
the core C code of numpy is too small.

> That's interesting, because PEP 3118 was pushed mainly by a prominent member of
> the NumPy community and some of its features are almost dedicated to NumPy.

I have not been involved with PEP 3118 discussion, so cannot comment
on the reason why it is not fully supported yet by numpy.

But I think that's a different issue altogether - PEP 3118 goal is for
interoperation with other packages. We can port to PEP 3118 without
porting to 3.x, and we can port to 3.x without taking care of PEP
3118.

David

From cournape at gmail.com  Tue Nov  3 14:40:04 2009
From: cournape at gmail.com (David Cournapeau)
Date: Tue, 3 Nov 2009 22:40:04 +0900
Subject: [Python-Dev] 2.7 Release? 2.7 == last of the 2.x line?
In-Reply-To: <3DE1D8FB-896E-441D-937D-AB63EB3A8E22@python.org>
References: <E077843B-919A-4AC7-95D7-0313E4D0328A@gmail.com>
	<F245110A-2FDF-460B-8E0D-87785F0E6DB0@fuhm.net>
	<BBB79ADA-7E31-4DD8-AAA5-07537FA74C5A@gmail.com>
	<D99BA4CF-E16C-47EB-9C60-F1EC5C27CDDC@python.org>
	<4AEFC887.6020801@v.loewis.de>
	<3DE1D8FB-896E-441D-937D-AB63EB3A8E22@python.org>
Message-ID: <5b8d13220911030540s3d700265ib00f4277b7ce4a2b@mail.gmail.com>

On Tue, Nov 3, 2009 at 9:55 PM, Barry Warsaw <barry at python.org> wrote:

>
> Then clearly we can't back port features.
>
> I'd like to read some case studies of people who have migrated applications
> from 2.6 to 3.0.

+1, especially for packages which have a lot of C code: the current
documentation is sparse :) The only helpful reference I have found so
far is an email by MvL concerning psycopg2 port.

David

From ssteinerx at gmail.com  Tue Nov  3 14:40:48 2009
From: ssteinerx at gmail.com (ssteinerX@gmail.com)
Date: Tue, 3 Nov 2009 08:40:48 -0500
Subject: [Python-Dev] 2.7 Release? 2.7 == last of the 2.x line?
In-Reply-To: <ca471dc20911022106h45856c22q9ba1cb359997ef1e@mail.gmail.com>
References: <E077843B-919A-4AC7-95D7-0313E4D0328A@gmail.com>
	<F245110A-2FDF-460B-8E0D-87785F0E6DB0@fuhm.net>
	<BBB79ADA-7E31-4DD8-AAA5-07537FA74C5A@gmail.com>
	<D99BA4CF-E16C-47EB-9C60-F1EC5C27CDDC@python.org>
	<C5BC05A6-27EF-48A2-9416-64C1AA715321@gmail.com>
	<ca471dc20911022106h45856c22q9ba1cb359997ef1e@mail.gmail.com>
Message-ID: <A210A69F-FBA5-4C82-9D7E-AC278E237F4A@gmail.com>


On Nov 3, 2009, at 12:06 AM, Guido van Rossum wrote:

> On Mon, Nov 2, 2009 at 9:51 PM, ssteinerX at gmail.com <ssteinerx at gmail.com 
> > wrote:
>> BeautifulSoup, which I use every day, is one such product.  Since  
>> the crappy
>> old SMGL parser's gone, BeautifulSoup uses the one that's left in  
>> Python 3
>> and it makes BeautifulSoup completely useless for my daily work.
>
> This sounds an area where some help might be useful. Perhaps the
> quickest solution would simply be to copy the old crappy "sgml" based
> html parser into a new version of BeautifulSoup.

That is what we're discussing doing on the old-soup branch at http://github.com/adevore/old-beautiful-soup 
  .  I'm not exactly sure why the old SGML parser was dropped but it  
seems that porting it to Python 3 would be enough of an effort that it  
caused the Python library to drop it, and the current developer of the  
mainline of Beautiful Soup to decide to just use what was available in  
Python 3 natively.

> Though I imagine what it really needs is a "quirks mode" parser that  
> is compatible with the
> HTML dialect accepted by, say, IE6. Maybe a summer of code project?

I think it just relies on the old SGML parser's not blowing up on  
completely bogus HTML (like most of the web) and does the best it can  
with the 'chunks' that come back; nothing to do with quirks mode per se.

As for a Summer of Code project, I have no idea what would be  
involved.  I know there are lots of users for Beautiful soup; as far  
as I know it is the best scraper of HTML code, valid or not, that's  
out there and it's been around a long time and I see it in projects in  
the "html scraping" realm all the time.

At any rate, it's just one example of where the developer has taken  
the easy route out with a 3.0 port and has produced a product that's  
"Python 3" but, instead of getting better with Python's new features,  
has actually become useless for the majority of use-cases where  
formerly it shined.

S


From ssteinerx at gmail.com  Tue Nov  3 14:43:06 2009
From: ssteinerx at gmail.com (ssteinerX@gmail.com)
Date: Tue, 3 Nov 2009 08:43:06 -0500
Subject: [Python-Dev] 2.7 Release?  2.7 == last of the 2.x line?
In-Reply-To: <4AEFC887.6020801@v.loewis.de>
References: <E077843B-919A-4AC7-95D7-0313E4D0328A@gmail.com>	<F245110A-2FDF-460B-8E0D-87785F0E6DB0@fuhm.net>	<BBB79ADA-7E31-4DD8-AAA5-07537FA74C5A@gmail.com>
	<D99BA4CF-E16C-47EB-9C60-F1EC5C27CDDC@python.org>
	<4AEFC887.6020801@v.loewis.de>
Message-ID: <5B548FA6-995A-4519-B614-982794FA85DE@gmail.com>


On Nov 3, 2009, at 1:07 AM, Martin v. L?wis wrote:

> Barry Warsaw wrote:
>> On Nov 2, 2009, at 10:48 PM, ssteinerX at gmail.com wrote:
>>
>>> A better language, i.e. Python 3.x, will become better faster  
>>> without
>>> dragging the 2.x series out any longer.
>>
>> If Python 2.7 becomes the last of the 2.x series, then I personally
>> favor back porting as many features from Python 3 as possible.
>
> And if *2.6* becomes the last of the 2.x series?

Then I'd guess that that would annoy the crap out of everyone who's  
put so much effort into 2.7 and all of us who have been waiting for  
what I hope will finally be the "now ports way more easily to 3.0"  
version.

S


From ssteinerx at gmail.com  Tue Nov  3 14:45:59 2009
From: ssteinerx at gmail.com (ssteinerX@gmail.com)
Date: Tue, 3 Nov 2009 08:45:59 -0500
Subject: [Python-Dev] 2.7 Release?  2.7 == last of the 2.x line?
In-Reply-To: <4AEFD9AE.7090104@molden.no>
References: <E077843B-919A-4AC7-95D7-0313E4D0328A@gmail.com>
	<4AEFD9AE.7090104@molden.no>
Message-ID: <DF3EB3B9-EB18-42C3-BE34-AADC80006DC4@gmail.com>


On Nov 3, 2009, at 2:20 AM, Sturla Molden wrote:

> I'd just like to mention that the scientific community is highly  
> dependent on NumPy. As long as NumPy is not ported to Py3k,  
> migration is out of the question. Porting NumPy is not a trivial  
> issue. It might take a complete rewrite of the whole C base using  
> Cython. NumPy's ABI is not even PEP 3118 compliant. Changing the ABI  
> for Py3k might break extension code written for NumPy using C. And  
> scientists tend to write CPU-bound routines in languages like C and  
> Fortran, not Python, so that is a major issue as well. If we port  
> NumPy to Py3k, everyone using NumPy will have to port their C code  
> to the new ABI. There are lot of people stuck with Python 2.x for  
> this reason.

> It does not just affect individual scientists, but also large  
> projects like IBM and CERN's blue brain and NASA's space telecope.

Then, perhaps, if 2.7 is known to be the last release of the 2.x line,  
some of those deep pockets can cough up some $$$ or developers to  
actually _do_ the port.

A Python 3 version of NumPy might be enough of an improvement to bring  
*more* scientists and engineers onboard if the Python 3.x version  
shows what great productivity gains are to be had with Python 3.x over  
2.x.

S


From ssteinerx at gmail.com  Tue Nov  3 14:47:58 2009
From: ssteinerx at gmail.com (ssteinerX@gmail.com)
Date: Tue, 3 Nov 2009 08:47:58 -0500
Subject: [Python-Dev] 2.7 Release? 2.7 == last of the 2.x line?
In-Reply-To: <5b8d13220911030155x57615540x1f59e4b95b6c2f56@mail.gmail.com>
References: <E077843B-919A-4AC7-95D7-0313E4D0328A@gmail.com>
	<4AEFD9AE.7090104@molden.no> <4AEFF450.9060007@voidspace.org.uk>
	<5b8d13220911030155x57615540x1f59e4b95b6c2f56@mail.gmail.com>
Message-ID: <30C6A234-DC87-45CD-A82D-62733B2BD9E3@gmail.com>


On Nov 3, 2009, at 4:55 AM, David Cournapeau wrote:

> On Tue, Nov 3, 2009 at 6:13 PM, Michael Foord <fuzzyman at voidspace.org.uk 
> > wrote:
>
> There is also little documentation on how to port a significant C
> codebase to py3k.

Now there's a good Summer of Code project:  to produce a pre-processor  
that will flag all C constructs that need to be modified in some way,  
with direct pointers to the relevant documentation, and "code  
suggestions" wherever practicable.

S


From ssteinerx at gmail.com  Tue Nov  3 14:55:13 2009
From: ssteinerx at gmail.com (ssteinerX@gmail.com)
Date: Tue, 3 Nov 2009 08:55:13 -0500
Subject: [Python-Dev] 2.7 Release?  2.7 == last of the 2.x line?
In-Reply-To: <79EADA413EFB44F6864EF71D5B830088@RaymondLaptop1>
References: <E077843B-919A-4AC7-95D7-0313E4D0328A@gmail.com><4AEFD9AE.7090104@molden.no>
	<4AEFF450.9060007@voidspace.org.uk>
	<79EADA413EFB44F6864EF71D5B830088@RaymondLaptop1>
Message-ID: <16F0B6B0-677F-464C-9C89-CF92D3713A65@gmail.com>


On Nov 3, 2009, at 4:58 AM, Raymond Hettinger wrote:
>
> P.S.  I found it curious that one of the strongest proponents of  
> killing 2.x also mentioned that he has never written a line of 3.x  
> code.   Since this discussion is a matter of great consequence, I  
> would hope that advocates will only take informed positions --  this  
> isn't really time for shooting from the hip and killing 2.x.

Uh, that would be me.

I'm only a proponent of making a decision.

I *want* to have a better development language, library, and add-on  
tools.

If 3.x is where future core development time is going to be focused,  
then I have faith that they will be able to make it the compelling  
path that it will become with Guido having put as much effort into it  
as he has.  Or, maybe he's completely lost his mind as sometimes  
happens with "dictators for life" benevolent or other-wise. ;-)

In any case, splitting time between 2.x and 3.x, with limited  
developer resources is going to lead to slower progress on both fronts.

And, as you point out, if 3.x doesn't start getting the crap beat out  
of it in the real world sooner rather than later, we may find  
ourselves, collectively with a stale 2.x, an under battle-tested 3.x,  
and nowhere to go.

S


From solipsis at pitrou.net  Tue Nov  3 15:04:06 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Tue, 3 Nov 2009 14:04:06 +0000 (UTC)
Subject: [Python-Dev] 2.7 Release? 2.7 == last of the 2.x line?
References: <E077843B-919A-4AC7-95D7-0313E4D0328A@gmail.com>
	<4AEFD9AE.7090104@molden.no>
	<loom.20091103T123514-202@post.gmane.org>
	<5b8d13220911030526y7dce8ce0gecb427ea009eaf1a@mail.gmail.com>
Message-ID: <loom.20091103T150228-755@post.gmane.org>

David Cournapeau <cournape <at> gmail.com> writes:
> 
> We can port to PEP 3118 without
> porting to 3.x, and we can port to 3.x without taking care of PEP
> 3118.

I'm not sure you can do the latter. The old buffer API (the one PEP 3118
replaces) doesn't exist in py3k.


Antoine.



From daniel at stutzbachenterprises.com  Tue Nov  3 15:14:46 2009
From: daniel at stutzbachenterprises.com (Daniel Stutzbach)
Date: Tue, 3 Nov 2009 08:14:46 -0600
Subject: [Python-Dev] 2.7 Release? 2.7 == last of the 2.x line?
In-Reply-To: <5b8d13220911030155x57615540x1f59e4b95b6c2f56@mail.gmail.com>
References: <E077843B-919A-4AC7-95D7-0313E4D0328A@gmail.com>
	<4AEFD9AE.7090104@molden.no> <4AEFF450.9060007@voidspace.org.uk>
	<5b8d13220911030155x57615540x1f59e4b95b6c2f56@mail.gmail.com>
Message-ID: <eae285400911030614tdac889fqb5e655027565a962@mail.gmail.com>

On Tue, Nov 3, 2009 at 3:55 AM, David Cournapeau <cournape at gmail.com> wrote:

>  - are two branches are necessary or not ? If two branches are
> necessary, I think we simply do not have the resources at the moment.
>  - how to maintain a compatible C API across 2.x and 3.x
>  - is it practically possible to support and maintain numpy from 2.4
> to 3.x ? For example, I don't think the python 2.6 py3k warnings are
> very useful when you need to maintain compatibility with 2.4 and 2.5.
>

I've already ported some of my Python extension modules to Python 3.  Here's
how I would answer your questions based on my experience.

Writing C code that compiles with Python 2.4 through 3.1 is pretty easy.
Python's C API hasn't changed much and you can use #ifdef and #define for
any bits that must be version-specific.

It's pretty easy to make Python source that works under 2.6 and 3.x.  It's
basically impossible to make Python source that works under 2.4/2.5 and
3.x.  You may be able to write code that works under 2.4/2.5 and works
cleanly with 2to3 to produce 3.x code.  I haven't tried that route, though
in theory it should work.  All you really need is syntax compatibility.  For
the rest, you can check sys.version_info.

In a nutshell, I don't think you need two branches to support an extension
module on Python 2 and Python 3.

YMMV.

--
Daniel Stutzbach, Ph.D.
President, Stutzbach Enterprises, LLC <http://stutzbachenterprises.com>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20091103/d4485062/attachment.htm>

From skip at pobox.com  Tue Nov  3 16:23:39 2009
From: skip at pobox.com (skip at pobox.com)
Date: Tue, 3 Nov 2009 09:23:39 -0600
Subject: [Python-Dev] 2.7 Release?  2.7 == last of the 2.x line?
In-Reply-To: <4AEFC887.6020801@v.loewis.de>
References: <E077843B-919A-4AC7-95D7-0313E4D0328A@gmail.com>
	<F245110A-2FDF-460B-8E0D-87785F0E6DB0@fuhm.net>
	<BBB79ADA-7E31-4DD8-AAA5-07537FA74C5A@gmail.com>
	<D99BA4CF-E16C-47EB-9C60-F1EC5C27CDDC@python.org>
	<4AEFC887.6020801@v.loewis.de>
Message-ID: <19184.19195.484852.432070@montanaro.dyndns.org>


    Martin> And if *2.6* becomes the last of the 2.x series?

With all due respect, I don't think you can make that decision now.  The
time to have stated 2.6 would be the end of the 2.x line was when 2.6 was
released.  At that point instead of opening up the trunk for changes you
would have closed it off or merged the py3k branch to trunk.  2.6.0 was
released over a year ago and there has been no effort to suppress bug fix or
feature additions to trunk since then.  If you call 2.6 "the end of 2.x"
you'll have wasted a year of work on 2.7 with about a month to go before the
first 2.7 alpha release.

If you want to accelerate release of 2.7 (fewer alphas, compressed schedule,
etc) that's fine, but I don't think you can turn back the clock at this
point and decree that 2.7 is dead.

Skip

From skip at pobox.com  Tue Nov  3 16:32:04 2009
From: skip at pobox.com (skip at pobox.com)
Date: Tue, 3 Nov 2009 09:32:04 -0600
Subject: [Python-Dev] 2.7 Release?  2.7 == last of the 2.x line?
In-Reply-To: <4AF02E5E.4020501@egenix.com>
References: <E077843B-919A-4AC7-95D7-0313E4D0328A@gmail.com>
	<4AEFD9AE.7090104@molden.no> <4AEFF450.9060007@voidspace.org.uk>
	<79EADA413EFB44F6864EF71D5B830088@RaymondLaptop1>
	<4AF02E5E.4020501@egenix.com>
Message-ID: <19184.19700.781398.906305@montanaro.dyndns.org>


    mal> I don't think users will really go for carrots. Python 2.x is
    mal> mature enough already and at least the users I know are really
    mal> happy with it (including myself).

Taking that thought further back one step (or three), from

    http://effbot.org/tkinterbook/listbox.htm

I pull this quote:

    In versions before Python 1.5, use string.atoi instead of int.

:-)

Skip

From guido at python.org  Tue Nov  3 16:54:32 2009
From: guido at python.org (Guido van Rossum)
Date: Tue, 3 Nov 2009 07:54:32 -0800
Subject: [Python-Dev] 2.7 Release? 2.7 == last of the 2.x line?
In-Reply-To: <3DE1D8FB-896E-441D-937D-AB63EB3A8E22@python.org>
References: <E077843B-919A-4AC7-95D7-0313E4D0328A@gmail.com> 
	<F245110A-2FDF-460B-8E0D-87785F0E6DB0@fuhm.net>
	<BBB79ADA-7E31-4DD8-AAA5-07537FA74C5A@gmail.com> 
	<D99BA4CF-E16C-47EB-9C60-F1EC5C27CDDC@python.org>
	<4AEFC887.6020801@v.loewis.de> 
	<3DE1D8FB-896E-441D-937D-AB63EB3A8E22@python.org>
Message-ID: <ca471dc20911030754scf07219h7dcf6e44825cd516@mail.gmail.com>

On Tue, Nov 3, 2009 at 4:55 AM, Barry Warsaw <barry at python.org> wrote:
> I'd like to read some case studies of people who have migrated applications
> from 2.6 to 3.0. ?Having just gone through a 2 week sprint to migrate
> Launchpad from 2.4 to 2.6, and only making it to 2.5, I can say that I was
> unpleasantly surprised at the amount of work it took. ?A lot of that was
> working out the dependency upgrades, with some amount of fixing our code
> (mostly tests) for things that have changed (e.g. exception print/str
> format). ?We didn't make it to Python 2.6 because dealing with deprecation
> warnings for sha, md5, and sets (a little in our code but tons in our
> dependencies) consumed most of our remaining time.

Ouch. sets. I'm guessing you are referring to code that was still
using the pre-2.4 sets.py module? Yes, that must have been painful.

> Given another week or so I think we would have made it to Python 2.6, but
> I'm not at all confident that that would have been a good enough platform to
> attempt an upgrade to Python 3, even if all of our very numerous large
> dependencies were available for Python 3. ?Maybe it wouldn't be so bad, but
> my recent experience informs me that I'm probably being too optimistic
> rather than too pessimistic.

There are two stages of porting to 2.6 you have to go through. The
first one, which you would probably have reached in that week, is
running on 2.6 period. You can then release your code for the benefit
of others wanting to use it on 2.6.

But the second stage, which can take much more time (depending on the
state of your code base) is to run on 2.6 *free of warnings with the
-3 flag on*. You pretty much have to consider this a separate port,
and it is here where you do much of the prep work for 3.x (at least
for Python code -- for C extensions it's not so clear). The good news
is that you can claim 2.6 support before you've even started this
stage; the other good news is that doing this right will really help
you prepare for 3.x. And in most cases you can even (with some effort)
maintain compatibility with 2.5 or even 2.4 -- though you may have to
work around some things like the md5 and sha warnings.

The bad news is that this stage may well take more time than porting
from 2.4 to 2.6.

-- 
--Guido van Rossum (python.org/~guido)

From eric at trueblade.com  Tue Nov  3 16:56:24 2009
From: eric at trueblade.com (Eric Smith)
Date: Tue, 03 Nov 2009 10:56:24 -0500
Subject: [Python-Dev] 2.7 Release?  2.7 == last of the 2.x line?
In-Reply-To: <19184.19700.781398.906305@montanaro.dyndns.org>
References: <E077843B-919A-4AC7-95D7-0313E4D0328A@gmail.com>	<4AEFD9AE.7090104@molden.no>
	<4AEFF450.9060007@voidspace.org.uk>	<79EADA413EFB44F6864EF71D5B830088@RaymondLaptop1>	<4AF02E5E.4020501@egenix.com>
	<19184.19700.781398.906305@montanaro.dyndns.org>
Message-ID: <4AF052A8.6060007@trueblade.com>

skip at pobox.com wrote:
>     mal> I don't think users will really go for carrots. Python 2.x is
>     mal> mature enough already and at least the users I know are really
>     mal> happy with it (including myself).
> 
> Taking that thought further back one step (or three), from
> 
>     http://effbot.org/tkinterbook/listbox.htm
> 
> I pull this quote:
> 
>     In versions before Python 1.5, use string.atoi instead of int.

Which reminds me: I've been meaning to add -3 warnings for these string 
module functions!

Eric.

From foom at fuhm.net  Tue Nov  3 18:04:00 2009
From: foom at fuhm.net (James Y Knight)
Date: Tue, 3 Nov 2009 12:04:00 -0500
Subject: [Python-Dev] 2.7 Release?  2.7 == last of the 2.x line?
In-Reply-To: <16F0B6B0-677F-464C-9C89-CF92D3713A65@gmail.com>
References: <E077843B-919A-4AC7-95D7-0313E4D0328A@gmail.com><4AEFD9AE.7090104@molden.no>
	<4AEFF450.9060007@voidspace.org.uk>
	<79EADA413EFB44F6864EF71D5B830088@RaymondLaptop1>
	<16F0B6B0-677F-464C-9C89-CF92D3713A65@gmail.com>
Message-ID: <116885F3-819E-4B7C-86F7-8E5B246590EB@fuhm.net>

On Nov 3, 2009, at 8:55 AM, ssteinerX at gmail.com wrote:
> And, as you point out, if 3.x doesn't start getting the crap beat  
> out of it in the real world sooner rather than later, we may find  
> ourselves, collectively with a stale 2.x, an under battle-tested  
> 3.x, and nowhere to go.

If that happens, it's not true that there's *nowhere* to go. A  
solution would be to discard 3.x as a failed experiment, take  
everything that is useful from it and port it to 2.x, and simply  
continue development from the last 2.x release. And from there,  
features can be deprecated and then removed a few releases later, as  
is the usual policy.

Been there, done that, on a couple other projects. It's unfortunate  
when you have to throw out work you've done because it failed to gain  
traction over the thing you tried to replace, but sometimes that's life.

James

From debatem1 at gmail.com  Tue Nov  3 18:10:17 2009
From: debatem1 at gmail.com (geremy condra)
Date: Tue, 3 Nov 2009 12:10:17 -0500
Subject: [Python-Dev] 2.7 Release? 2.7 == last of the 2.x line?
In-Reply-To: <30C6A234-DC87-45CD-A82D-62733B2BD9E3@gmail.com>
References: <E077843B-919A-4AC7-95D7-0313E4D0328A@gmail.com>
	<4AEFD9AE.7090104@molden.no> <4AEFF450.9060007@voidspace.org.uk>
	<5b8d13220911030155x57615540x1f59e4b95b6c2f56@mail.gmail.com>
	<30C6A234-DC87-45CD-A82D-62733B2BD9E3@gmail.com>
Message-ID: <f3cc57c60911030910n209fae74te0de5426c1867276@mail.gmail.com>

On Tue, Nov 3, 2009 at 8:47 AM, ssteinerX at gmail.com <ssteinerx at gmail.com> wrote:
>
> On Nov 3, 2009, at 4:55 AM, David Cournapeau wrote:
>
>> On Tue, Nov 3, 2009 at 6:13 PM, Michael Foord <fuzzyman at voidspace.org.uk>
>> wrote:
>>
>> There is also little documentation on how to port a significant C
>> codebase to py3k.
>
> Now there's a good Summer of Code project: ?to produce a pre-processor that
> will flag all C constructs that need to be modified in some way, with direct
> pointers to the relevant documentation, and "code suggestions" wherever
> practicable.
>
> S

How much interest is there in this?

Geremy Condra

From guido at python.org  Tue Nov  3 18:23:16 2009
From: guido at python.org (Guido van Rossum)
Date: Tue, 3 Nov 2009 09:23:16 -0800
Subject: [Python-Dev] 2.7 Release? 2.7 == last of the 2.x line?
In-Reply-To: <116885F3-819E-4B7C-86F7-8E5B246590EB@fuhm.net>
References: <E077843B-919A-4AC7-95D7-0313E4D0328A@gmail.com> 
	<4AEFD9AE.7090104@molden.no> <4AEFF450.9060007@voidspace.org.uk> 
	<79EADA413EFB44F6864EF71D5B830088@RaymondLaptop1>
	<16F0B6B0-677F-464C-9C89-CF92D3713A65@gmail.com> 
	<116885F3-819E-4B7C-86F7-8E5B246590EB@fuhm.net>
Message-ID: <ca471dc20911030923p7e5fe1bega0699084bfda98af@mail.gmail.com>

On Tue, Nov 3, 2009 at 9:04 AM, James Y Knight <foom at fuhm.net> wrote:
> If that happens, it's not true that there's *nowhere* to go. A solution
> would be to discard 3.x as a failed experiment, take everything that is
> useful from it and port it to 2.x, and simply continue development from the
> last 2.x release. And from there, features can be deprecated and then
> removed a few releases later, as is the usual policy.
>
> Been there, done that, on a couple other projects. It's unfortunate when you
> have to throw out work you've done because it failed to gain traction over
> the thing you tried to replace, but sometimes that's life.

I'm not ready for that yet. I think there's plenty of time before we
have to agree to such a bleak view. In the mean time let's do
something practical like help NumPy port to Py3k.

-- 
--Guido van Rossum (python.org/~guido)

From martin at v.loewis.de  Tue Nov  3 18:27:37 2009
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Tue, 03 Nov 2009 18:27:37 +0100
Subject: [Python-Dev] 2.7 Release? 2.7 == last of the 2.x line?
In-Reply-To: <5b8d13220911030155x57615540x1f59e4b95b6c2f56@mail.gmail.com>
References: <E077843B-919A-4AC7-95D7-0313E4D0328A@gmail.com>	<4AEFD9AE.7090104@molden.no>
	<4AEFF450.9060007@voidspace.org.uk>
	<5b8d13220911030155x57615540x1f59e4b95b6c2f56@mail.gmail.com>
Message-ID: <4AF06809.3040009@v.loewis.de>

> To answer your question, the main issues are:
>  - are two branches are necessary or not ? If two branches are
> necessary, I think we simply do not have the resources at the moment.

No, it should be well possible to have the same source being used in
both 2.x and 3.x. I've ported ZODB to Python 3 (which includes both
C and Python code), and it works quite well.

>  - how to maintain a compatible C API across 2.x and 3.x

Not sure what the "C API" is: if it is the actual implementation of
the C modules, then I recommend to use preprocessor macros a lot.
If you need specific solutions, you'll have to ask specific questions.

>  - is it practically possible to support and maintain numpy from 2.4
> to 3.x ? 

Absolutely, yes.

> For example, I don't think the python 2.6 py3k warnings are
> very useful when you need to maintain compatibility with 2.4 and 2.5.

These I don't know. I found that I had little use for the 3k warnings
in porting to 3k; I usually rely on tests to find out whether the code
still works correctly.

> There is also little documentation on how to port a significant C
> codebase to py3k.

Please do ask specific questions.

Regards,
Martin

From arcriley at gmail.com  Tue Nov  3 18:28:47 2009
From: arcriley at gmail.com (Arc Riley)
Date: Tue, 3 Nov 2009 12:28:47 -0500
Subject: [Python-Dev] 2.7 Release? 2.7 == last of the 2.x line?
In-Reply-To: <116885F3-819E-4B7C-86F7-8E5B246590EB@fuhm.net>
References: <E077843B-919A-4AC7-95D7-0313E4D0328A@gmail.com> 
	<4AEFD9AE.7090104@molden.no> <4AEFF450.9060007@voidspace.org.uk> 
	<79EADA413EFB44F6864EF71D5B830088@RaymondLaptop1>
	<16F0B6B0-677F-464C-9C89-CF92D3713A65@gmail.com> 
	<116885F3-819E-4B7C-86F7-8E5B246590EB@fuhm.net>
Message-ID: <bad82a80911030928x1f594a92yc315f2ae8b8246d9@mail.gmail.com>

+1 on ending with 2.6.

I'm the maintainer of 3rd party Python 3-only packages and have ported a few
modules that we needed with some help from the 2to3 tool.  It's really not a
big deal - and Py3 really is a massive improvement.

The main thing holding back the community are lazy and/or obstinate package
maintainers.  If they spent half the time they've put into complaining about
Py3 into actually working to upgrade their code they'd be done now.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20091103/f1ee989d/attachment-0001.htm>

From steven.bethard at gmail.com  Tue Nov  3 18:33:20 2009
From: steven.bethard at gmail.com (Steven Bethard)
Date: Tue, 3 Nov 2009 09:33:20 -0800
Subject: [Python-Dev] 2.7 Release? 2.7 == last of the 2.x line?
In-Reply-To: <3DE1D8FB-896E-441D-937D-AB63EB3A8E22@python.org>
References: <E077843B-919A-4AC7-95D7-0313E4D0328A@gmail.com>
	<F245110A-2FDF-460B-8E0D-87785F0E6DB0@fuhm.net>
	<BBB79ADA-7E31-4DD8-AAA5-07537FA74C5A@gmail.com>
	<D99BA4CF-E16C-47EB-9C60-F1EC5C27CDDC@python.org>
	<4AEFC887.6020801@v.loewis.de>
	<3DE1D8FB-896E-441D-937D-AB63EB3A8E22@python.org>
Message-ID: <d11dcfba0911030933o74405be9q1851699fda0937f8@mail.gmail.com>

On Tue, Nov 3, 2009 at 4:55 AM, Barry Warsaw <barry at python.org> wrote:
> I'd like to read some case studies of people who have migrated applications
> from 2.6 to 3.0.

For what it's worth, it was pretty easy to migrate argparse:

  http://code.google.com/p/argparse/source/detail?r=12

It was mostly just adding a few aliases, and doing a little
sys.exc_info() dance in a couple places because argparse supports
Python 2.3 - 3.1 all in the same code base.

Steve
-- 
Where did you get that preposterous hypothesis?
Did Steve tell you that?
        --- The Hiphopopotamus

From martin at v.loewis.de  Tue Nov  3 18:34:04 2009
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Tue, 03 Nov 2009 18:34:04 +0100
Subject: [Python-Dev] 2.7 Release? 2.7 == last of the 2.x line?
In-Reply-To: <5b8d13220911030540s3d700265ib00f4277b7ce4a2b@mail.gmail.com>
References: <E077843B-919A-4AC7-95D7-0313E4D0328A@gmail.com>	<F245110A-2FDF-460B-8E0D-87785F0E6DB0@fuhm.net>	<BBB79ADA-7E31-4DD8-AAA5-07537FA74C5A@gmail.com>	<D99BA4CF-E16C-47EB-9C60-F1EC5C27CDDC@python.org>	<4AEFC887.6020801@v.loewis.de>	<3DE1D8FB-896E-441D-937D-AB63EB3A8E22@python.org>
	<5b8d13220911030540s3d700265ib00f4277b7ce4a2b@mail.gmail.com>
Message-ID: <4AF0698C.2030706@v.loewis.de>

>> I'd like to read some case studies of people who have migrated applications
>> from 2.6 to 3.0.
> 
> +1, especially for packages which have a lot of C code: the current
> documentation is sparse :) The only helpful reference I have found so
> far is an email by MvL concerning psycopg2 port.

You may also want to take a look at my ZODB port:

https://www.dcl.hpi.uni-potsdam.de/home/loewis/zodb/

Regards,
Martin

From ssteinerx at gmail.com  Tue Nov  3 18:35:16 2009
From: ssteinerx at gmail.com (ssteinerX@gmail.com)
Date: Tue, 3 Nov 2009 12:35:16 -0500
Subject: [Python-Dev] 2.7 Release? 2.7 == last of the 2.x line?
In-Reply-To: <ca471dc20911030923p7e5fe1bega0699084bfda98af@mail.gmail.com>
References: <E077843B-919A-4AC7-95D7-0313E4D0328A@gmail.com>
	<4AEFD9AE.7090104@molden.no> <4AEFF450.9060007@voidspace.org.uk>
	<79EADA413EFB44F6864EF71D5B830088@RaymondLaptop1>
	<16F0B6B0-677F-464C-9C89-CF92D3713A65@gmail.com>
	<116885F3-819E-4B7C-86F7-8E5B246590EB@fuhm.net>
	<ca471dc20911030923p7e5fe1bega0699084bfda98af@mail.gmail.com>
Message-ID: <3B364FE0-7616-41F8-82C7-528EA1282E40@gmail.com>


On Nov 3, 2009, at 12:23 PM, Guido van Rossum wrote:

> On Tue, Nov 3, 2009 at 9:04 AM, James Y Knight <foom at fuhm.net> wrote:
>> If that happens, it's not true that there's *nowhere* to go. A  
>> solution
>> would be to discard 3.x as a failed experiment, take everything  
>> that is
>> useful from it and port it to 2.x, and simply continue development  
>> from the
>> last 2.x release. And from there, features can be deprecated and then
>> removed a few releases later, as is the usual policy.
>>
>> Been there, done that, on a couple other projects. It's unfortunate  
>> when you
>> have to throw out work you've done because it failed to gain  
>> traction over
>> the thing you tried to replace, but sometimes that's life.
>
> I'm not ready for that yet. I think there's plenty of time before we
> have to agree to such a bleak view. In the mean time let's do
> something practical like help NumPy port to Py3k.

Or, for example, Django...

S


From guido at python.org  Tue Nov  3 18:35:10 2009
From: guido at python.org (Guido van Rossum)
Date: Tue, 3 Nov 2009 09:35:10 -0800
Subject: [Python-Dev] PEP 3003 - Python Language Moratorium
Message-ID: <ca471dc20911030935l25251e0diec64427f5d7bcbf2@mail.gmail.com>

I've checked draft (!) PEP 3003, "Python Language Moratorium", into
SVN. As authors I've listed Jesse, Brett and myself.

On python-ideas the moratorium idea got fairly positive responses
(more positive than I'd expected, in fact) but I'm bracing myself for
fierce discussion here on python-dev. It's important to me that if if
this is accepted it is a "rough consensus" decision (working code we
already have plenty of :-), not something enforced by a vocal minority
or an influential individual such as myself. If there's too much
opposition I'll withdraw the PEP so as not to waste everybody's time
with a fruitless discussion.

The PEP tries to spell out some gray areas but I'm sure there will be
others; that's life. Do note that the PEP proposes to be *retroactive*
back to the 3.1 release, i.e. the "frozen" version of the language is
the state in which it was released as 3.1.

-- 
--Guido van Rossum (python.org/~guido)
-------------- next part --------------
PEP: 3003
Title: Python Language Moratorium
Version: $Revision$
Last-Modified: $Date$
Author: Brett Cannon, Jesse Noller, Guido van Rossum
Status: Draft
Type: Process
Content-Type: text/x-rst
Created: 21-Oct-2009
Post-History:


Abstract
========

This PEP proposes a temporary moratorium (suspension) of all changes
to the Python language syntax, semantics, and built-ins for a period
of at least two years from the release of Python 3.1.

This suspension of features is designed to allow non-CPython implementations
to "catch up" to the core implementation of the language, help ease adoption
of Python 3.x, and provide a more stable base for the community.


Rationale
=========

This idea was proposed by Guido van Rossum on the python-ideas [1]_ mailing
list. The premise of his email was to slow the alteration of the  Python core
syntax, builtins and semantics to allow non-CPython implementations to catch
up to the current state of Python, both 2.x and 3.x.

Python, as a language is more than the core implementation --
CPython -- with a rich, mature and vibrant community of implementations, such
as Jython [2]_, IronPython [3]_ and PyPy [4]_ that are a benefit not only to
the community, but to the language itself.

Still others, such as Unladen Swallow [5]_ (a branch of CPython) seek not to
create an alternative implementation, but rather they seek to enhance the
performance and implementation of CPython itself.

Python 3.x was a large part of the last several years of Python's
development. Its release, as well as a bevy of changes to the language
introduce by it and the previous 2.6.x releases, puts alternative
implementations at a severe disadvantage in "keeping pace" with core python
development.

Additionally, many of the changes put into the recent releases of the language
as implemented by CPython have not yet seen widespread usage amongst the
general user population. For example, most users are beholden to the version
of the interpreter (typically CPython) which comes pre-installed with their
operating system. Most OS vendors are just barely beginning to ship Python 2.6
-- even fewer are shipping Python 3.x.

As it is expected that Python 2.7 be the effective "end of life" of the Python
2.x code line, with Python 3.x being the future, it is in the best interest of
Python core development to temporarily suspend the alteration of the language
itself to allow all of these external entities to catch up and to assist in
the adoption of, and migration to, Python 3.x

Finally, the moratorium is intended to free up cycles within core development
to focus on other issues, such as the CPython interpreter and improvements
therein, the standard library, etc.

This moratorium does not allow for exceptions -- once accepted, any pending
changes to the syntax or semantics of the language will be postponed until the
moratorium is lifted.

This moratorium does not attempt to apply to any other Python implementation
meaning that if desired other implementations may add features which deviate
from the standard implementation.


Details
=======

Cannot Change
-------------

* New built-ins
* Language syntax
    The grammar file essentially becomes immutable apart from ambiguity
    fixes.
* General language semantics
    The language operates as-is with only specific exemptions (see
    below).


Case-by-Case Exemptions
-----------------------

* New methods on built-ins
    The case for adding a method to a built-in object can be made.
* Incorrect language semantics 
    If the language semantics turn out to be ambiguous or improperly
    implemented based on the intention of the original design then the
    semantics may change.
* Language semantics that are difficult to implement
    Because other VMs have not begun implementing Python 3.x semantics
    there is a possibility that certain semantics are too difficult to
    replicate. In those cases they can be changed to ease adoption of
    Python 3.x by the other VMs.


Allowed to Change
-----------------

* C API
    It is entirely acceptable to change the underlying C code of
    CPython as long as other restrictions of this moratorium are not
    broken. E.g. removing the GIL would be fine assuming certain
    operations that are currently atomic remain atomic.
* The standard library
    As the standard library is not directly tied to the language
    definition it is not covered by this moratorium.
* Backports of 3.x features to to 2.x
    The moratorium only affects features that would be new in 3.x.


Retroactive
===========

It is important to note that the moratorium covers all changes since the
release of Python 3.1. This rule is intended to avoid features being rushed or
smuggled into the CPython source tree while the moratorium is being discussed.


Copyright
=========

This document has been placed in the public domain.

References
==========

.. [1] http://mail.python.org/pipermail/python-ideas/2009-October/006305.html
.. [2] http://www.jython.org/
.. [3] http://www.codeplex.com/IronPython
.. [4] http://codespeak.net/pypy/
.. [5] http://code.google.com/p/unladen-swallow/ 


..
   Local Variables:
   mode: indented-text
   indent-tabs-mode: nil
   sentence-end-double-space: t
   fill-column: 80
   coding: utf-8
   End:

From solipsis at pitrou.net  Tue Nov  3 18:35:57 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Tue, 3 Nov 2009 17:35:57 +0000 (UTC)
Subject: [Python-Dev] 2.7 Release? 2.7 == last of the 2.x line?
References: <E077843B-919A-4AC7-95D7-0313E4D0328A@gmail.com>
	<4AEFD9AE.7090104@molden.no> <4AEFF450.9060007@voidspace.org.uk>
	<79EADA413EFB44F6864EF71D5B830088@RaymondLaptop1>
	<16F0B6B0-677F-464C-9C89-CF92D3713A65@gmail.com>
	<116885F3-819E-4B7C-86F7-8E5B246590EB@fuhm.net>
	<bad82a80911030928x1f594a92yc315f2ae8b8246d9@mail.gmail.com>
Message-ID: <loom.20091103T183408-637@post.gmane.org>

Arc Riley <arcriley <at> gmail.com> writes:
> 
> +1 on ending with 2.6.I'm the maintainer of 3rd party Python 3-only packages
> and have ported a few modules that we needed with some help from the 2to3 
> tool.? It's really not a big deal - and Py3 really is a massive improvement.
> The main thing holding back the community are lazy and/or obstinate package 
> maintainers.? If they spent half the time they've put into complaining about 
> Py3 into actually working to upgrade their code they'd be done now.

One thing you could do is explain (do you have a blog?) how Py3 is a massive
improvement for you as a developer and package maintainer.
We core developers obviously agree that py3k is better than 2.x, but the same
opinion coming from a third-party developer would carry a different weight.

Regards

Antoine.



From martin at v.loewis.de  Tue Nov  3 18:37:35 2009
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Tue, 03 Nov 2009 18:37:35 +0100
Subject: [Python-Dev] 2.7 Release?  2.7 == last of the 2.x line?
In-Reply-To: <DF3EB3B9-EB18-42C3-BE34-AADC80006DC4@gmail.com>
References: <E077843B-919A-4AC7-95D7-0313E4D0328A@gmail.com>	<4AEFD9AE.7090104@molden.no>
	<DF3EB3B9-EB18-42C3-BE34-AADC80006DC4@gmail.com>
Message-ID: <4AF06A5F.6090501@v.loewis.de>

> A Python 3 version of NumPy might be enough of an improvement to bring
> *more* scientists and engineers onboard if the Python 3.x version shows
> what great productivity gains are to be had with Python 3.x over 2.x.

I would be really surprised if 2.7 would simplify porting to 3.x. How
could that possibly work?

(and no, adding things like nonlocal to 2.7 doesn't making porting of
a real application or library any easier, since the existing application
or library simply doesn't use that keyword. In fact, no change to 2.x
can reasonably simplify porting - only changes to 3.x might - except
for changes to 2to3, which can simplify porting a lot. But 2to3 should
be run under 3.x, IMO.)

Regards,
Martin

From guido at python.org  Tue Nov  3 18:38:09 2009
From: guido at python.org (Guido van Rossum)
Date: Tue, 3 Nov 2009 09:38:09 -0800
Subject: [Python-Dev] 2.7 Release? 2.7 == last of the 2.x line?
In-Reply-To: <3B364FE0-7616-41F8-82C7-528EA1282E40@gmail.com>
References: <E077843B-919A-4AC7-95D7-0313E4D0328A@gmail.com> 
	<4AEFD9AE.7090104@molden.no> <4AEFF450.9060007@voidspace.org.uk> 
	<79EADA413EFB44F6864EF71D5B830088@RaymondLaptop1>
	<16F0B6B0-677F-464C-9C89-CF92D3713A65@gmail.com> 
	<116885F3-819E-4B7C-86F7-8E5B246590EB@fuhm.net>
	<ca471dc20911030923p7e5fe1bega0699084bfda98af@mail.gmail.com> 
	<3B364FE0-7616-41F8-82C7-528EA1282E40@gmail.com>
Message-ID: <ca471dc20911030938l26be2d0fw4c184c21bc251b1d@mail.gmail.com>

On Tue, Nov 3, 2009 at 9:35 AM, ssteinerX at gmail.com <ssteinerx at gmail.com> wrote:
>
> On Nov 3, 2009, at 12:23 PM, Guido van Rossum wrote:
>
>> On Tue, Nov 3, 2009 at 9:04 AM, James Y Knight <foom at fuhm.net> wrote:
>>>
>>> If that happens, it's not true that there's *nowhere* to go. A solution
>>> would be to discard 3.x as a failed experiment, take everything that is
>>> useful from it and port it to 2.x, and simply continue development from
>>> the
>>> last 2.x release. And from there, features can be deprecated and then
>>> removed a few releases later, as is the usual policy.
>>>
>>> Been there, done that, on a couple other projects. It's unfortunate when
>>> you
>>> have to throw out work you've done because it failed to gain traction
>>> over
>>> the thing you tried to replace, but sometimes that's life.
>>
>> I'm not ready for that yet. I think there's plenty of time before we
>> have to agree to such a bleak view. In the mean time let's do
>> something practical like help NumPy port to Py3k.
>
> Or, for example, Django...

Wasn't Django ported to Py3k by MvL as a demo? The problem seems to be
more to port the Django *community* to Py3k...

-- 
--Guido van Rossum (python.org/~guido)

From g.brandl at gmx.net  Tue Nov  3 18:39:21 2009
From: g.brandl at gmx.net (Georg Brandl)
Date: Tue, 03 Nov 2009 18:39:21 +0100
Subject: [Python-Dev] 2.7 Release?  2.7 == last of the 2.x line?
In-Reply-To: <19184.19195.484852.432070@montanaro.dyndns.org>
References: <E077843B-919A-4AC7-95D7-0313E4D0328A@gmail.com>	<F245110A-2FDF-460B-8E0D-87785F0E6DB0@fuhm.net>	<BBB79ADA-7E31-4DD8-AAA5-07537FA74C5A@gmail.com>	<D99BA4CF-E16C-47EB-9C60-F1EC5C27CDDC@python.org>	<4AEFC887.6020801@v.loewis.de>
	<19184.19195.484852.432070@montanaro.dyndns.org>
Message-ID: <hcpps3$r8u$1@ger.gmane.org>

skip at pobox.com schrieb:
>     Martin> And if *2.6* becomes the last of the 2.x series?
> 
> With all due respect, I don't think you can make that decision now.  The
> time to have stated 2.6 would be the end of the 2.x line was when 2.6 was
> released.  At that point instead of opening up the trunk for changes you
> would have closed it off or merged the py3k branch to trunk.  2.6.0 was
> released over a year ago and there has been no effort to suppress bug fix or
> feature additions to trunk since then.  If you call 2.6 "the end of 2.x"
> you'll have wasted a year of work on 2.7 with about a month to go before the
> first 2.7 alpha release.

+1.

Georg


From martin at v.loewis.de  Tue Nov  3 18:39:53 2009
From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=)
Date: Tue, 03 Nov 2009 18:39:53 +0100
Subject: [Python-Dev] 2.7 Release? 2.7 == last of the 2.x line?
In-Reply-To: <eae285400911030614tdac889fqb5e655027565a962@mail.gmail.com>
References: <E077843B-919A-4AC7-95D7-0313E4D0328A@gmail.com>	<4AEFD9AE.7090104@molden.no>
	<4AEFF450.9060007@voidspace.org.uk>	<5b8d13220911030155x57615540x1f59e4b95b6c2f56@mail.gmail.com>
	<eae285400911030614tdac889fqb5e655027565a962@mail.gmail.com>
Message-ID: <4AF06AE9.2010706@v.loewis.de>

> It's pretty easy to make Python source that works under 2.6 and 3.x. 
> It's basically impossible to make Python source that works under 2.4/2.5
> and 3.x.  You may be able to write code that works under 2.4/2.5 and
> works cleanly with 2to3 to produce 3.x code.  I haven't tried that
> route, though in theory it should work.  All you really need is syntax
> compatibility.

I have tried that route for a number of projects, and I think it works
really well. It is also supported by distribute.

> In a nutshell, I don't think you need two branches to support an
> extension module on Python 2 and Python 3.
> 
> YMMV.

Exactly my experience as well.

Regards,
Martin

From martin at v.loewis.de  Tue Nov  3 18:43:29 2009
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Tue, 03 Nov 2009 18:43:29 +0100
Subject: [Python-Dev] 2.7 Release? 2.7 == last of the 2.x line?
In-Reply-To: <3B364FE0-7616-41F8-82C7-528EA1282E40@gmail.com>
References: <E077843B-919A-4AC7-95D7-0313E4D0328A@gmail.com>	<4AEFD9AE.7090104@molden.no>
	<4AEFF450.9060007@voidspace.org.uk>	<79EADA413EFB44F6864EF71D5B830088@RaymondLaptop1>	<16F0B6B0-677F-464C-9C89-CF92D3713A65@gmail.com>	<116885F3-819E-4B7C-86F7-8E5B246590EB@fuhm.net>	<ca471dc20911030923p7e5fe1bega0699084bfda98af@mail.gmail.com>
	<3B364FE0-7616-41F8-82C7-528EA1282E40@gmail.com>
Message-ID: <4AF06BC1.6060004@v.loewis.de>

>> I'm not ready for that yet. I think there's plenty of time before we
>> have to agree to such a bleak view. In the mean time let's do
>> something practical like help NumPy port to Py3k.
> 
> Or, for example, Django...

See

http://wiki.python.org/moin/PortingDjangoTo3k

Regards,
Martin

From ssteinerx at gmail.com  Tue Nov  3 18:44:08 2009
From: ssteinerx at gmail.com (ssteinerX@gmail.com)
Date: Tue, 3 Nov 2009 12:44:08 -0500
Subject: [Python-Dev] 2.7 Release? 2.7 == last of the 2.x line?
In-Reply-To: <ca471dc20911030938l26be2d0fw4c184c21bc251b1d@mail.gmail.com>
References: <E077843B-919A-4AC7-95D7-0313E4D0328A@gmail.com>
	<4AEFD9AE.7090104@molden.no> <4AEFF450.9060007@voidspace.org.uk>
	<79EADA413EFB44F6864EF71D5B830088@RaymondLaptop1>
	<16F0B6B0-677F-464C-9C89-CF92D3713A65@gmail.com>
	<116885F3-819E-4B7C-86F7-8E5B246590EB@fuhm.net>
	<ca471dc20911030923p7e5fe1bega0699084bfda98af@mail.gmail.com>
	<3B364FE0-7616-41F8-82C7-528EA1282E40@gmail.com>
	<ca471dc20911030938l26be2d0fw4c184c21bc251b1d@mail.gmail.com>
Message-ID: <35DEB5D3-41C5-482A-8D84-577606EE2C70@gmail.com>


On Nov 3, 2009, at 12:38 PM, Guido van Rossum wrote:

> On Tue, Nov 3, 2009 at 9:35 AM, ssteinerX at gmail.com <ssteinerx at gmail.com 
> > wrote:
>>
>> On Nov 3, 2009, at 12:23 PM, Guido van Rossum wrote:
>>
>>> On Tue, Nov 3, 2009 at 9:04 AM, James Y Knight <foom at fuhm.net>  
>>> wrote:
>>>>
>>>> If that happens, it's not true that there's *nowhere* to go. A  
>>>> solution
>>>> would be to discard 3.x as a failed experiment, take everything  
>>>> that is
>>>> useful from it and port it to 2.x, and simply continue  
>>>> development from
>>>> the
>>>> last 2.x release. And from there, features can be deprecated and  
>>>> then
>>>> removed a few releases later, as is the usual policy.
>>>>
>>>> Been there, done that, on a couple other projects. It's  
>>>> unfortunate when
>>>> you
>>>> have to throw out work you've done because it failed to gain  
>>>> traction
>>>> over
>>>> the thing you tried to replace, but sometimes that's life.
>>>
>>> I'm not ready for that yet. I think there's plenty of time before we
>>> have to agree to such a bleak view. In the mean time let's do
>>> something practical like help NumPy port to Py3k.
>>
>> Or, for example, Django...
>
> Wasn't Django ported to Py3k by MvL as a demo? The problem seems to be
> more to port the Django *community* to Py3k...

I do remember seeing something about that somewhere but it sure isn't  
jumping into my workflow at the moment.

If I can get a Python 3 version of Django, that's keeping up with  
trunk, I hereby declare that I will start using it on my current  
project as soon as the client takes the blowtorch off my toes for the  
current deliverable.

And...I'll help keep it up to date with trunk as best I can and also  
help pull along all the modules I need (incidentally, including  
Beautiful Soup).

I really _want_ Python 3 to be better, I hope actual use convinces me  
that it is...

There, now I'm committed (or, maybe I should _be_ committed).

Thanks,

S


From p.f.moore at gmail.com  Tue Nov  3 18:44:24 2009
From: p.f.moore at gmail.com (Paul Moore)
Date: Tue, 3 Nov 2009 17:44:24 +0000
Subject: [Python-Dev] 2.7 Release? 2.7 == last of the 2.x line?
In-Reply-To: <4AF0698C.2030706@v.loewis.de>
References: <E077843B-919A-4AC7-95D7-0313E4D0328A@gmail.com>
	<F245110A-2FDF-460B-8E0D-87785F0E6DB0@fuhm.net>
	<BBB79ADA-7E31-4DD8-AAA5-07537FA74C5A@gmail.com>
	<D99BA4CF-E16C-47EB-9C60-F1EC5C27CDDC@python.org>
	<4AEFC887.6020801@v.loewis.de>
	<3DE1D8FB-896E-441D-937D-AB63EB3A8E22@python.org>
	<5b8d13220911030540s3d700265ib00f4277b7ce4a2b@mail.gmail.com>
	<4AF0698C.2030706@v.loewis.de>
Message-ID: <79990c6b0911030944r26fd668ev37e56a4a7d79559b@mail.gmail.com>

2009/11/3 "Martin v. L?wis" <martin at v.loewis.de>:
>>> I'd like to read some case studies of people who have migrated applications
>>> from 2.6 to 3.0.
>>
>> +1, especially for packages which have a lot of C code: the current
>> documentation is sparse :) The only helpful reference I have found so
>> far is an email by MvL concerning psycopg2 port.
>
> You may also want to take a look at my ZODB port:
>
> https://www.dcl.hpi.uni-potsdam.de/home/loewis/zodb/

Has that port been integrated back into the zodb project? If not, it
would be interesting to know the reasons (zodb project maintainers not
interested, barriers to contribution are too high, port is not 100%
complete and no-one willing to take it over and complete it...?)

Paul.

From g.brandl at gmx.net  Tue Nov  3 18:44:58 2009
From: g.brandl at gmx.net (Georg Brandl)
Date: Tue, 03 Nov 2009 18:44:58 +0100
Subject: [Python-Dev] 2.7 Release?  2.7 == last of the 2.x line?
In-Reply-To: <79EADA413EFB44F6864EF71D5B830088@RaymondLaptop1>
References: <E077843B-919A-4AC7-95D7-0313E4D0328A@gmail.com><4AEFD9AE.7090104@molden.no>	<4AEFF450.9060007@voidspace.org.uk>
	<79EADA413EFB44F6864EF71D5B830088@RaymondLaptop1>
Message-ID: <hcpq6k$r8u$2@ger.gmane.org>

Raymond Hettinger schrieb:
> [Michael Foord]
>> What will it take to *start* the port? (Or is it already underway?) For
>> many projects I fear that it is only the impending obsolescence (real
>> rather than theoretical) of Python 2 that will convince projects to port.
> 
> FWIW, I do not buy into the several premises that have arisen in this thread:
> 
> 
> * For 3.x to succeed, something bad has to happen to 2.x.  (which in my book
> translates to intentionally harming 2.x users, either through neglect or
> force, in order to bait them into switching to 3.x).
> 
> * Core developers will are losing time supporting 2.x. (backports take some
> time but it is small in comparison to getting a patch to work in the first
> place -- if anyone can comment on this assertion,  it is the people who have
> been doing it already (such as AP, MD, BP, GB, and myself)).

I agree.  However I wouldn't want to lose the amount of work I've put into 2.7.

While reviewing the 2.6 "svnmerge avail" output, I also got the impression that
a *significant* number of fixes were not backported to 2.6.  I don't have the
time to go through a 300+ kb change log and find out what to backport, just
based on commit messages that are not always clear on whether a fix or feature
was added.

So if we kill 2.7, we at least need to make sure no real improvements that
should have been in 2.x are lost.

Georg


From solipsis at pitrou.net  Tue Nov  3 18:47:01 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Tue, 3 Nov 2009 17:47:01 +0000 (UTC)
Subject: [Python-Dev] PEP 3003 - Python Language Moratorium
References: <ca471dc20911030935l25251e0diec64427f5d7bcbf2@mail.gmail.com>
Message-ID: <loom.20091103T184405-383@post.gmane.org>

Guido van Rossum <guido <at> python.org> writes:
> 
> The PEP tries to spell out some gray areas but I'm sure there will be
> others; that's life. Do note that the PEP proposes to be *retroactive*
> back to the 3.1 release, i.e. the "frozen" version of the language is
> the state in which it was released as 3.1.

I've done a quick check through the NEWS file and it seems no post-3.1 change
would be affected by this PEP.

I'm for the PEP myself.

Antoine.



From charlesc-lists-python-dev at pyropus.ca  Tue Nov  3 18:41:32 2009
From: charlesc-lists-python-dev at pyropus.ca (Charles Cazabon)
Date: Tue, 3 Nov 2009 11:41:32 -0600
Subject: [Python-Dev] 2.7 Release? 2.7 == last of the 2.x line?
In-Reply-To: <bad82a80911030928x1f594a92yc315f2ae8b8246d9@mail.gmail.com>
References: <E077843B-919A-4AC7-95D7-0313E4D0328A@gmail.com>
	<4AEFD9AE.7090104@molden.no> <4AEFF450.9060007@voidspace.org.uk>
	<79EADA413EFB44F6864EF71D5B830088@RaymondLaptop1>
	<16F0B6B0-677F-464C-9C89-CF92D3713A65@gmail.com>
	<116885F3-819E-4B7C-86F7-8E5B246590EB@fuhm.net>
	<bad82a80911030928x1f594a92yc315f2ae8b8246d9@mail.gmail.com>
Message-ID: <20091103174131.GA10645@pyropus.ca>

Arc Riley <arcriley at gmail.com> wrote:
> +1 on ending with 2.6.

That seems precipitous.

> I'm the maintainer of 3rd party Python 3-only packages and have ported a few
> modules that we needed with some help from the 2to3 tool.  It's really not a
> big deal - and Py3 really is a massive improvement.
> 
> The main thing holding back the community are lazy and/or obstinate package
> maintainers.

I wouldn't say that.  For instance, I'm just starting a refactoring that will
result in getmail v.5, but I need to target Python 2.5 and up, so there's
essentially no way the code will run in Python 3.x (as another list member
posted).

Why do I need to target Python 2.5?  Because that's the most current default
version of Python shipped in Debian stable and various other distributions
that don't stay on the bleeding edge.  getmail v.4 targeted Python 2.3 and up,
getmail v.3 targeted Python 1.5.2 and up.  I may be able to target Python 2.6
in a year or two, at which point Python 3 compatibility becomes a reasonable
goal.

Saying "2.6 is the last Python 2.x" seems to me to be a death sentence for
Python 3.  People will stay with 2.x much longer than you seem to want them
to, and making it harder for them to upgrade will only hurt Python 3.

Charles
-- 
-----------------------------------------------------------------------
Charles Cazabon
GPL'ed software available at:               http://pyropus.ca/software/
-----------------------------------------------------------------------

From martin at v.loewis.de  Tue Nov  3 18:51:33 2009
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Tue, 03 Nov 2009 18:51:33 +0100
Subject: [Python-Dev] language summit topic: issue tracker
In-Reply-To: <52dc1c820911030029m72b10895xae947b60068ad0c4@mail.gmail.com>
References: <bbaeab100910222147u41749107r1032545147fdc2e6@mail.gmail.com>
	<4AE17C12.1080007@gmail.com>
	<52dc1c820911030029m72b10895xae947b60068ad0c4@mail.gmail.com>
Message-ID: <4AF06DA5.6080508@v.loewis.de>

> yet another feature request or two to be lost to a mailing list thread
> along those lines:
> 
> Maintainer or not i'd like to be able to setup triggers so that i'm
> automatically cc'ed on any bugs matching a simple search query i
> specify.

Please add that to the meta tracker (if you really want it, rather
than just losing it on the mailing list :-)

Implementing it would be quite involved, I think. It should probably
go into the saved query feature. Evaluating them on every change would
might be expensive, so doing so only regularly (e.g. hourly) would
be ok?

> The email sent out to people cc'ed when a new bug is opened and
> unassigned should have a simple links in it when cc'ed to someone who
> can be assigned bugs:  'Assign to me' that if followed will assign
> that bug to them without requiring a login.

Unfortunately, this is now defeated by the fear of XSS attacks. If
such a link was possible, it would be also possible to embed it
into a XSS attack, right?

It would be possible to protect the link with a token, but then,
efficient token validation might be tricky. So this would also need
to go into the meta tracker.

If you are really interested in these, it would be best to add them
as feature requests to roundup itself (since that really is the place
where they should be provided):

http://issues.roundup-tracker.org/

[but then, roundup could also use more contributors]

Regards,
Martin


From martin at v.loewis.de  Tue Nov  3 18:53:36 2009
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Tue, 03 Nov 2009 18:53:36 +0100
Subject: [Python-Dev] nonlocal keyword in 2.x?
In-Reply-To: <4AEFF3A3.3020103@egenix.com>
References: <da7032ce0910211856g45e45632n550aa5b4a10f9a62@mail.gmail.com>	<02A812F3B69E46D180B45133D52D1F2B@RaymondLaptop1>	<bbaeab100911011557t714e3b4cxcdc91b3fb9a49b84@mail.gmail.com>	<4AEE9862.8010201@voidspace.org.uk>	<ca471dc20911021001u102da717wcf4da0eada0867e@mail.gmail.com>	<5C4388816CE0428FB27EA69CE77CFCC0@RaymondLaptop1>	<ca471dc20911021009u7485adcdgde8fae04a5061059@mail.gmail.com>	<Pine.LNX.4.64.0911021405420.21420@kimball.webabinitio.net>	<4AEF4C53.7090902@v.loewis.de>	<Pine.LNX.4.64.0911021624110.21420@kimball.webabinitio.net>	<ca471dc20911021333r168e257eja1e5f79dbe4ae234@mail.gmail.com>	<loom.20091102T223839-797@post.gmane.org>
	<4AEFF3A3.3020103@egenix.com>
Message-ID: <4AF06E20.7050307@v.loewis.de>

>> 2.7 has an up-to-date backport of the C IO lib (as well as the memoryview
>> object), which means it is better for people wanting to ease transition to 3.x.
>>
>> But of course, as Martin said, few people will want to support 2.7 only and not 
>> 2.6.
> 
> Since 2.7 will be closer to 3.2 than 2.6, the result will more likely
> be: many people will want to support 2.7 and 3.x - which is really
> what we should be after.

Supporting 2.7 doesn't mean that you have to use the nonlocal keyword
(nor does support for 3.2 require that keyword). So I'm not sure how
the backport of a feature to 2.x could encourage people to forward-port
to 3.x.

Regards,
Martin


From martin at v.loewis.de  Tue Nov  3 18:56:11 2009
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Tue, 03 Nov 2009 18:56:11 +0100
Subject: [Python-Dev] 2.7 Release? 2.7 == last of the 2.x line?
In-Reply-To: <ca471dc20911030938l26be2d0fw4c184c21bc251b1d@mail.gmail.com>
References: <E077843B-919A-4AC7-95D7-0313E4D0328A@gmail.com>
	<4AEFD9AE.7090104@molden.no> <4AEFF450.9060007@voidspace.org.uk>
	<79EADA413EFB44F6864EF71D5B830088@RaymondLaptop1>	<16F0B6B0-677F-464C-9C89-CF92D3713A65@gmail.com>
	<116885F3-819E-4B7C-86F7-8E5B246590EB@fuhm.net>	<ca471dc20911030923p7e5fe1bega0699084bfda98af@mail.gmail.com>
	<3B364FE0-7616-41F8-82C7-528EA1282E40@gmail.com>
	<ca471dc20911030938l26be2d0fw4c184c21bc251b1d@mail.gmail.com>
Message-ID: <4AF06EBB.8060100@v.loewis.de>

> Wasn't Django ported to Py3k by MvL as a demo? The problem seems to be
> more to port the Django *community* to Py3k...

Exactly so. At the last Pycon, we then agreed that I would get a branch
in the Django code repository, but then progress stalled due to
inactivity on boths sides.

Regards,
Martin


From zookog at gmail.com  Tue Nov  3 18:59:42 2009
From: zookog at gmail.com (Zooko O'Whielacronx)
Date: Tue, 3 Nov 2009 10:59:42 -0700
Subject: [Python-Dev] 2.7 Release? 2.7 == last of the 2.x line?
In-Reply-To: <ca471dc20911030923p7e5fe1bega0699084bfda98af@mail.gmail.com>
References: <E077843B-919A-4AC7-95D7-0313E4D0328A@gmail.com>
	<4AEFD9AE.7090104@molden.no> <4AEFF450.9060007@voidspace.org.uk>
	<79EADA413EFB44F6864EF71D5B830088@RaymondLaptop1>
	<16F0B6B0-677F-464C-9C89-CF92D3713A65@gmail.com>
	<116885F3-819E-4B7C-86F7-8E5B246590EB@fuhm.net>
	<ca471dc20911030923p7e5fe1bega0699084bfda98af@mail.gmail.com>
Message-ID: <cd6401a0911030959t12595529vb8d13f262ccae954@mail.gmail.com>

Folks:

I really don't want to make anyone feel bad or to criticize, but I
should mention that I have no plans to use Python 3 or to support
Python 3.  My best guess at this time is that the current projects
that I'm involved in will still require Python 2 for the forseeable
future (let's say 5 years.  I can see 5 years into the future.), and
that as I start new projects I will probably try out interesting
alternative programming languages like Haskell, Newspeak [1],
Jacaranda [2], and other new things that appear in the coming years.

Of course, I reserve the right to change my mind and start using and
supporting Python 3.  That might happen if there is some combination
of: 1. my users start asking for it (no-one has yet), 2. my
dependencies start providing it (I use Python because it has Twisted.
Twisted requires Python 2.), 3. it becomes more possible for me to
write code which is still Python-2-compatible and also is more and
more close to being Python-3-compatible.

By the way, one significant detail which makes Python 3 less
interesting to me is [3].  Those two languages that I mentioned --
Newspeak and Jacaranda -- both have object-capability nature.  If that
issue in [3] were fixed then Python 3 would join Python 2 as a
language that can (with the CapPython extension) have
object-capability nature.

Regards,

Zooko

[1] http://bracha.org/Site/Newspeak.html
[2] http://jacaranda.org
[3] http://lackingrhoticity.blogspot.com/2008/09/cappython-unbound-methods-and-python-30.html
---
Your cloud storage provider does not need access to your data.
Tahoe-LAFS -- http://allmydata.org

From flashk at gmail.com  Tue Nov  3 19:01:22 2009
From: flashk at gmail.com (Farshid Lashkari)
Date: Tue, 3 Nov 2009 10:01:22 -0800
Subject: [Python-Dev] 2.7 Release? 2.7 == last of the 2.x line?
In-Reply-To: <4AF06A5F.6090501@v.loewis.de>
References: <E077843B-919A-4AC7-95D7-0313E4D0328A@gmail.com> 
	<4AEFD9AE.7090104@molden.no>
	<DF3EB3B9-EB18-42C3-BE34-AADC80006DC4@gmail.com> 
	<4AF06A5F.6090501@v.loewis.de>
Message-ID: <978d1eac0911031001o40d69d9evfae42e7921fbf00a@mail.gmail.com>

On Tue, Nov 3, 2009 at 9:37 AM, "Martin v. L?wis" <martin at v.loewis.de>wrote:
>
> (and no, adding things like nonlocal to 2.7 doesn't making porting of
> a real application or library any easier, since the existing application
> or library simply doesn't use that keyword. In fact, no change to 2.x
> can reasonably simplify porting - only changes to 3.x might - except
> for changes to 2to3, which can simplify porting a lot. But 2to3 should
> be run under 3.x, IMO.)
>

I agree that adding these features doesn't really help much for people that
are porting from 2.x to 3.x. However, I can see it being useful for package
developers who want to support both 3.x and 2.x. The more these two versions
have in common the easier it will be to support them both. Obviously, this
means the package might not work with 2.6 and earlier. But some users out
there might be in some situation where they cannot upgrade to 3.x but can
jump from 2.4/5/6 to 2.7.

-Farshid
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20091103/4cb6da94/attachment.htm>

From solipsis at pitrou.net  Tue Nov  3 19:03:25 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Tue, 3 Nov 2009 18:03:25 +0000 (UTC)
Subject: [Python-Dev] 2.7 Release?  2.7 == last of the 2.x line?
References: <E077843B-919A-4AC7-95D7-0313E4D0328A@gmail.com>	<F245110A-2FDF-460B-8E0D-87785F0E6DB0@fuhm.net>	<BBB79ADA-7E31-4DD8-AAA5-07537FA74C5A@gmail.com>	<D99BA4CF-E16C-47EB-9C60-F1EC5C27CDDC@python.org>	<4AEFC887.6020801@v.loewis.de>
	<19184.19195.484852.432070@montanaro.dyndns.org>
	<hcpps3$r8u$1@ger.gmane.org>
Message-ID: <loom.20091103T190051-631@post.gmane.org>

Georg Brandl <g.brandl <at> gmx.net> writes:
> 
> skip <at> pobox.com schrieb:
> >     Martin> And if *2.6* becomes the last of the 2.x series?
> > 
> > With all due respect, I don't think you can make that decision now.  The
> > time to have stated 2.6 would be the end of the 2.x line was when 2.6 was
> > released.  At that point instead of opening up the trunk for changes you
> > would have closed it off or merged the py3k branch to trunk.  2.6.0 was
> > released over a year ago and there has been no effort to suppress bug fix or
> > feature additions to trunk since then.  If you call 2.6 "the end of 2.x"
> > you'll have wasted a year of work on 2.7 with about a month to go before the
> > first 2.7 alpha release.
> 
> +1.

+1 as well. Besides, it is much better communication to release 2.7 and say up
front that it will be the last major release in the 2.x line. Announcing that
there won't be a 2.7, however, would give the impression of poor planning.

Regards

Antoine.



From martin at v.loewis.de  Tue Nov  3 19:09:29 2009
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Tue, 03 Nov 2009 19:09:29 +0100
Subject: [Python-Dev] 2.7 Release? 2.7 == last of the 2.x line?
In-Reply-To: <79990c6b0911030944r26fd668ev37e56a4a7d79559b@mail.gmail.com>
References: <E077843B-919A-4AC7-95D7-0313E4D0328A@gmail.com>	
	<F245110A-2FDF-460B-8E0D-87785F0E6DB0@fuhm.net>	
	<BBB79ADA-7E31-4DD8-AAA5-07537FA74C5A@gmail.com>	
	<D99BA4CF-E16C-47EB-9C60-F1EC5C27CDDC@python.org>	
	<4AEFC887.6020801@v.loewis.de>	
	<3DE1D8FB-896E-441D-937D-AB63EB3A8E22@python.org>	
	<5b8d13220911030540s3d700265ib00f4277b7ce4a2b@mail.gmail.com>	
	<4AF0698C.2030706@v.loewis.de>
	<79990c6b0911030944r26fd668ev37e56a4a7d79559b@mail.gmail.com>
Message-ID: <4AF071D9.5000900@v.loewis.de>

>> You may also want to take a look at my ZODB port:
>>
>> https://www.dcl.hpi.uni-potsdam.de/home/loewis/zodb/
> 
> Has that port been integrated back into the zodb project?

Only partially so.

> If not, it
> would be interesting to know the reasons (zodb project maintainers not
> interested, barriers to contribution are too high, port is not 100%
> complete and no-one willing to take it over and complete it...?)

ZODB is a lot of layers, really (zope.interfaces, zope.lockfile,
zope.proxy, ZConfig, etc..., finally ZODB).

I failed to port buildout and zope.testing. With zope.testing not
ported, it is not easy to actually run the test suites for all of
these; this is where the integration stalled.

In any case, I only did the port in September; at the DZUG sprint,
people from gocept then started looking into integrating it.
We didn't get through at the sprint, and I think there has been
little progress since.

One specific issue was about specifying the root of the zope.interfaces
hierarchy. In current zope.interfaces, None could be used (implicitly)
instead of Interface, to indicate the root interface type. In 3.x, this
would break tests, because you can't sort a list of interfaces anymore
if that list also contains None. So people decided that zope.interfaces
needs to be changed to disallow None, and that was a significant change
that has not yet been fully understood.

Regards,
Martin

From eric at trueblade.com  Tue Nov  3 19:13:18 2009
From: eric at trueblade.com (Eric Smith)
Date: Tue, 03 Nov 2009 13:13:18 -0500
Subject: [Python-Dev] 2.7 Release?  2.7 == last of the 2.x line?
In-Reply-To: <4AF06A5F.6090501@v.loewis.de>
References: <E077843B-919A-4AC7-95D7-0313E4D0328A@gmail.com>	<4AEFD9AE.7090104@molden.no>	<DF3EB3B9-EB18-42C3-BE34-AADC80006DC4@gmail.com>
	<4AF06A5F.6090501@v.loewis.de>
Message-ID: <4AF072BE.2010603@trueblade.com>

Martin v. L?wis wrote:
>> A Python 3 version of NumPy might be enough of an improvement to bring
>> *more* scientists and engineers onboard if the Python 3.x version shows
>> what great productivity gains are to be had with Python 3.x over 2.x.
> 
> I would be really surprised if 2.7 would simplify porting to 3.x. How
> could that possibly work?

The only things I can think of that would go into this category are 
features like:
- PEP 3118, revised buffer protocol. If the buffer API that numpy
   uses is not present in py3k (I'm no expert on the subject, but
   it seems this way from a recent thread on python-dev), then if
   they could move to PEP 3118 in 2.7 their migration to 3.x would
   be easier
- short float repr. This would remove a class of hard-to-find
   problems from a migration from 2.7 to 3.x.
- Maybe io, but I don't know enough about it to say.

But I definitely agree that backporting language features new to 3.x 
don't make it easier. Examples are nonlocal and required keyword args.

Eric.

From ssteinerx at gmail.com  Tue Nov  3 19:15:18 2009
From: ssteinerx at gmail.com (ssteinerX@gmail.com)
Date: Tue, 3 Nov 2009 13:15:18 -0500
Subject: [Python-Dev] 2.7 Release? 2.7 == last of the 2.x line?
In-Reply-To: <bad82a80911030928x1f594a92yc315f2ae8b8246d9@mail.gmail.com>
References: <E077843B-919A-4AC7-95D7-0313E4D0328A@gmail.com>
	<4AEFD9AE.7090104@molden.no> <4AEFF450.9060007@voidspace.org.uk>
	<79EADA413EFB44F6864EF71D5B830088@RaymondLaptop1>
	<16F0B6B0-677F-464C-9C89-CF92D3713A65@gmail.com>
	<116885F3-819E-4B7C-86F7-8E5B246590EB@fuhm.net>
	<bad82a80911030928x1f594a92yc315f2ae8b8246d9@mail.gmail.com>
Message-ID: <3FE75D51-199C-4221-9B38-94FD2FE9BC4E@gmail.com>


On Nov 3, 2009, at 12:28 PM, Arc Riley wrote:
>
> The main thing holding back the community are lazy and/or obstinate  
> package maintainers.  If they spent half the time they've put into  
> complaining about Py3 into actually working to upgrade their code  
> they'd be done now.

That's an inflammatory, defamatory, unsubstantiated, hyperbolic,  
sweeping overgeneralization.

S

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20091103/85176163/attachment.htm>

From martin at v.loewis.de  Tue Nov  3 19:15:38 2009
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Tue, 03 Nov 2009 19:15:38 +0100
Subject: [Python-Dev] 2.7 Release? 2.7 == last of the 2.x line?
In-Reply-To: <20091103174131.GA10645@pyropus.ca>
References: <E077843B-919A-4AC7-95D7-0313E4D0328A@gmail.com>	<4AEFD9AE.7090104@molden.no>
	<4AEFF450.9060007@voidspace.org.uk>	<79EADA413EFB44F6864EF71D5B830088@RaymondLaptop1>	<16F0B6B0-677F-464C-9C89-CF92D3713A65@gmail.com>	<116885F3-819E-4B7C-86F7-8E5B246590EB@fuhm.net>	<bad82a80911030928x1f594a92yc315f2ae8b8246d9@mail.gmail.com>
	<20091103174131.GA10645@pyropus.ca>
Message-ID: <4AF0734A.3080705@v.loewis.de>

> I wouldn't say that.  For instance, I'm just starting a refactoring that will
> result in getmail v.5, but I need to target Python 2.5 and up, so there's
> essentially no way the code will run in Python 3.x (as another list member
> posted).

That's a common myth. It is very well possible, using 2to3. You don't
have to wait until you can drop 2.5 to start supporting 3.x, out of a
single code base.

> Why do I need to target Python 2.5?  Because that's the most current default
> version of Python shipped in Debian stable and various other distributions
> that don't stay on the bleeding edge.  getmail v.4 targeted Python 2.3 and up,
> getmail v.3 targeted Python 1.5.2 and up.  I may be able to target Python 2.6
> in a year or two, at which point Python 3 compatibility becomes a reasonable
> goal.

Are you saying that it doesn't *run* on 2.6? Why?
(not sure what you mean by "targetting")

Regards,
Martin

From g.brandl at gmx.net  Tue Nov  3 19:16:14 2009
From: g.brandl at gmx.net (Georg Brandl)
Date: Tue, 03 Nov 2009 18:16:14 +0000
Subject: [Python-Dev] 2.7 Release? 2.7 == last of the 2.x line?
In-Reply-To: <4AF06AE9.2010706@v.loewis.de>
References: <E077843B-919A-4AC7-95D7-0313E4D0328A@gmail.com>	<4AEFD9AE.7090104@molden.no>	<4AEFF450.9060007@voidspace.org.uk>	<5b8d13220911030155x57615540x1f59e4b95b6c2f56@mail.gmail.com>	<eae285400911030614tdac889fqb5e655027565a962@mail.gmail.com>
	<4AF06AE9.2010706@v.loewis.de>
Message-ID: <hcps18$54e$1@ger.gmane.org>

Martin v. L?wis schrieb:
>> It's pretty easy to make Python source that works under 2.6 and 3.x. 
>> It's basically impossible to make Python source that works under 2.4/2.5
>> and 3.x.  You may be able to write code that works under 2.4/2.5 and
>> works cleanly with 2to3 to produce 3.x code.  I haven't tried that
>> route, though in theory it should work.  All you really need is syntax
>> compatibility.
> 
> I have tried that route for a number of projects, and I think it works
> really well. It is also supported by distribute.

I've ported both Docutils and Pygments using that strategy, and I'll gladly
agree to that.

Georg


From ssteinerx at gmail.com  Tue Nov  3 19:18:46 2009
From: ssteinerx at gmail.com (ssteinerX@gmail.com)
Date: Tue, 3 Nov 2009 13:18:46 -0500
Subject: [Python-Dev] 2.7 Release? 2.7 == last of the 2.x line?
In-Reply-To: <4AF06BC1.6060004@v.loewis.de>
References: <E077843B-919A-4AC7-95D7-0313E4D0328A@gmail.com>	<4AEFD9AE.7090104@molden.no>
	<4AEFF450.9060007@voidspace.org.uk>	<79EADA413EFB44F6864EF71D5B830088@RaymondLaptop1>	<16F0B6B0-677F-464C-9C89-CF92D3713A65@gmail.com>	<116885F3-819E-4B7C-86F7-8E5B246590EB@fuhm.net>	<ca471dc20911030923p7e5fe1bega0699084bfda98af@mail.gmail.com>
	<3B364FE0-7616-41F8-82C7-528EA1282E40@gmail.com>
	<4AF06BC1.6060004@v.loewis.de>
Message-ID: <4A0DF493-B46D-436A-AEF3-BF1EC8DFE0A8@gmail.com>


On Nov 3, 2009, at 12:43 PM, Martin v. L?wis wrote:

>>> I'm not ready for that yet. I think there's plenty of time before we
>>> have to agree to such a bleak view. In the mean time let's do
>>> something practical like help NumPy port to Py3k.
>>
>> Or, for example, Django...
>
> See
>
> http://wiki.python.org/moin/PortingDjangoTo3k

Well, that's certainly a  start.

I guess the logical question is: Now what?

S


From ssteinerx at gmail.com  Tue Nov  3 19:19:58 2009
From: ssteinerx at gmail.com (ssteinerX@gmail.com)
Date: Tue, 3 Nov 2009 13:19:58 -0500
Subject: [Python-Dev] 2.7 Release? 2.7 == last of the 2.x line?
In-Reply-To: <loom.20091103T183408-637@post.gmane.org>
References: <E077843B-919A-4AC7-95D7-0313E4D0328A@gmail.com>
	<4AEFD9AE.7090104@molden.no> <4AEFF450.9060007@voidspace.org.uk>
	<79EADA413EFB44F6864EF71D5B830088@RaymondLaptop1>
	<16F0B6B0-677F-464C-9C89-CF92D3713A65@gmail.com>
	<116885F3-819E-4B7C-86F7-8E5B246590EB@fuhm.net>
	<bad82a80911030928x1f594a92yc315f2ae8b8246d9@mail.gmail.com>
	<loom.20091103T183408-637@post.gmane.org>
Message-ID: <5877F924-3E2B-4B18-BE55-4543C08BA149@gmail.com>


On Nov 3, 2009, at 12:35 PM, Antoine Pitrou wrote:

> Arc Riley <arcriley <at> gmail.com> writes:
>>
>> +1 on ending with 2.6.I'm the maintainer of 3rd party Python 3-only  
>> packages
>> and have ported a few modules that we needed with some help from  
>> the 2to3
>> tool.  It's really not a big deal - and Py3 really is a massive  
>> improvement.
>> The main thing holding back the community are lazy and/or obstinate  
>> package
>> maintainers.  If they spent half the time they've put into  
>> complaining about
>> Py3 into actually working to upgrade their code they'd be done now.
>
> One thing you could do is explain (do you have a blog?) how Py3 is a  
> massive
> improvement for you as a developer and package maintainer.
> We core developers obviously agree that py3k is better than 2.x, but  
> the same
> opinion coming from a third-party developer would carry a different  
> weight.

Maybe I haven't been looking, but has anyone collected the "Here's why  
3.x is better and here's how it saved my bacon on project XYZ" stories?

S



From barry at python.org  Tue Nov  3 19:20:18 2009
From: barry at python.org (Barry Warsaw)
Date: Tue, 3 Nov 2009 13:20:18 -0500
Subject: [Python-Dev] 2.7 Release? 2.7 == last of the 2.x line?
In-Reply-To: <ca471dc20911030754scf07219h7dcf6e44825cd516@mail.gmail.com>
References: <E077843B-919A-4AC7-95D7-0313E4D0328A@gmail.com>
	<F245110A-2FDF-460B-8E0D-87785F0E6DB0@fuhm.net>
	<BBB79ADA-7E31-4DD8-AAA5-07537FA74C5A@gmail.com>
	<D99BA4CF-E16C-47EB-9C60-F1EC5C27CDDC@python.org>
	<4AEFC887.6020801@v.loewis.de>
	<3DE1D8FB-896E-441D-937D-AB63EB3A8E22@python.org>
	<ca471dc20911030754scf07219h7dcf6e44825cd516@mail.gmail.com>
Message-ID: <22B9056E-E74A-40C2-B7B6-BC75FD6D8F4E@python.org>

On Nov 3, 2009, at 10:54 AM, Guido van Rossum wrote:

> Ouch. sets. I'm guessing you are referring to code that was still
> using the pre-2.4 sets.py module? Yes, that must have been painful.

Indeed. :)  What's nice though is that these changes could be made for  
the Python 2.5 branch and didn't have to wait until 2.6 (e.g. sha/md5 - 
 > hashlib, sets -> builtin-set).

What was actually most painful about this were all the tests that were  
checking stdout/stderr for clean subprocess runs.  The  
DeprecationWarnings that were produced killed us for a long while  
because so many of those tests failed.

It's actually quite difficult to suppress DeprecationWarnings across  
the board, especially when you have lots of subprocess that call third  
party code (where most of the deprecated code lived in, and which we  
couldn't change).  We added filterwarnings where we could, and called  
subprocesses with -W every place we thought we needed to but we never  
did kill them all off.  I think a $PYTHONWARNING environment variable  
might have helped here.

>> Given another week or so I think we would have made it to Python  
>> 2.6, but
>> I'm not at all confident that that would have been a good enough  
>> platform to
>> attempt an upgrade to Python 3, even if all of our very numerous  
>> large
>> dependencies were available for Python 3.  Maybe it wouldn't be so  
>> bad, but
>> my recent experience informs me that I'm probably being too  
>> optimistic
>> rather than too pessimistic.
>
> There are two stages of porting to 2.6 you have to go through. The
> first one, which you would probably have reached in that week, is
> running on 2.6 period. You can then release your code for the benefit
> of others wanting to use it on 2.6.

Yep.

> But the second stage, which can take much more time (depending on the
> state of your code base) is to run on 2.6 *free of warnings with the
> -3 flag on*. You pretty much have to consider this a separate port,
> and it is here where you do much of the prep work for 3.x (at least
> for Python code -- for C extensions it's not so clear). The good news
> is that you can claim 2.6 support before you've even started this
> stage; the other good news is that doing this right will really help
> you prepare for 3.x. And in most cases you can even (with some effort)
> maintain compatibility with 2.5 or even 2.4 -- though you may have to
> work around some things like the md5 and sha warnings.
>
> The bad news is that this stage may well take more time than porting
> from 2.4 to 2.6.

We have another window for this (since it's all open source I don't  
mind talking about it) when the next version of Ubuntu comes out.   
Running with the -3 is a great idea, and something I will definitely  
try after finishing the straight 2.6 port.

-Barry

-------------- next part --------------
A non-text attachment was scrubbed...
Name: PGP.sig
Type: application/pgp-signature
Size: 832 bytes
Desc: This is a digitally signed message part
URL: <http://mail.python.org/pipermail/python-dev/attachments/20091103/04ffe5d3/attachment.pgp>

From martin at v.loewis.de  Tue Nov  3 19:20:49 2009
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Tue, 03 Nov 2009 19:20:49 +0100
Subject: [Python-Dev] PEP 3003 - Python Language Moratorium
In-Reply-To: <ca471dc20911030935l25251e0diec64427f5d7bcbf2@mail.gmail.com>
References: <ca471dc20911030935l25251e0diec64427f5d7bcbf2@mail.gmail.com>
Message-ID: <4AF07481.3080106@v.loewis.de>

> * General language semantics
>     The language operates as-is with only specific exemptions (see
>     below).

Would PEP 382 (namespace packages) fall under the moratorium?

Regards,
Martin

From martin at v.loewis.de  Tue Nov  3 19:25:14 2009
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Tue, 03 Nov 2009 19:25:14 +0100
Subject: [Python-Dev] 2.7 Release?  2.7 == last of the 2.x line?
In-Reply-To: <4AF072BE.2010603@trueblade.com>
References: <E077843B-919A-4AC7-95D7-0313E4D0328A@gmail.com>	<4AEFD9AE.7090104@molden.no>	<DF3EB3B9-EB18-42C3-BE34-AADC80006DC4@gmail.com>
	<4AF06A5F.6090501@v.loewis.de> <4AF072BE.2010603@trueblade.com>
Message-ID: <4AF0758A.6070108@v.loewis.de>

>> I would be really surprised if 2.7 would simplify porting to 3.x. How
>> could that possibly work?
> 
> The only things I can think of that would go into this category are
> features like:
> - PEP 3118, revised buffer protocol. If the buffer API that numpy
>   uses is not present in py3k (I'm no expert on the subject, but
>   it seems this way from a recent thread on python-dev), then if
>   they could move to PEP 3118 in 2.7 their migration to 3.x would
>   be easier

But only if NumPy would drop support for 2.x, for x < 7, right?
That would probably be many years in the future.

Regards,
Martin

From martin at v.loewis.de  Tue Nov  3 19:25:55 2009
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Tue, 03 Nov 2009 19:25:55 +0100
Subject: [Python-Dev] 2.7 Release? 2.7 == last of the 2.x line?
In-Reply-To: <4A0DF493-B46D-436A-AEF3-BF1EC8DFE0A8@gmail.com>
References: <E077843B-919A-4AC7-95D7-0313E4D0328A@gmail.com>	<4AEFD9AE.7090104@molden.no>
	<4AEFF450.9060007@voidspace.org.uk>	<79EADA413EFB44F6864EF71D5B830088@RaymondLaptop1>	<16F0B6B0-677F-464C-9C89-CF92D3713A65@gmail.com>	<116885F3-819E-4B7C-86F7-8E5B246590EB@fuhm.net>	<ca471dc20911030923p7e5fe1bega0699084bfda98af@mail.gmail.com>
	<3B364FE0-7616-41F8-82C7-528EA1282E40@gmail.com>
	<4AF06BC1.6060004@v.loewis.de>
	<4A0DF493-B46D-436A-AEF3-BF1EC8DFE0A8@gmail.com>
Message-ID: <4AF075B3.2060301@v.loewis.de>

>>>> I'm not ready for that yet. I think there's plenty of time before we
>>>> have to agree to such a bleak view. In the mean time let's do
>>>> something practical like help NumPy port to Py3k.
>>>
>>> Or, for example, Django...
>>
>> See
>>
>> http://wiki.python.org/moin/PortingDjangoTo3k
> 
> Well, that's certainly a  start.
> 
> I guess the logical question is: Now what?

Use it, and report bugs.

Regards,
Martin

From eric at trueblade.com  Tue Nov  3 19:26:34 2009
From: eric at trueblade.com (Eric Smith)
Date: Tue, 03 Nov 2009 13:26:34 -0500
Subject: [Python-Dev] 2.7 Release?  2.7 == last of the 2.x line?
In-Reply-To: <4AF0758A.6070108@v.loewis.de>
References: <E077843B-919A-4AC7-95D7-0313E4D0328A@gmail.com>	<4AEFD9AE.7090104@molden.no>	<DF3EB3B9-EB18-42C3-BE34-AADC80006DC4@gmail.com>
	<4AF06A5F.6090501@v.loewis.de> <4AF072BE.2010603@trueblade.com>
	<4AF0758A.6070108@v.loewis.de>
Message-ID: <4AF075DA.5080607@trueblade.com>

Martin v. L?wis wrote:
>>> I would be really surprised if 2.7 would simplify porting to 3.x. How
>>> could that possibly work?
>> The only things I can think of that would go into this category are
>> features like:
>> - PEP 3118, revised buffer protocol. If the buffer API that numpy
>>   uses is not present in py3k (I'm no expert on the subject, but
>>   it seems this way from a recent thread on python-dev), then if
>>   they could move to PEP 3118 in 2.7 their migration to 3.x would
>>   be easier
> 
> But only if NumPy would drop support for 2.x, for x < 7, right?
> That would probably be many years in the future.

Right. But that might be their best migration strategy: wait for 2.7 to 
be available everywhere, port to 2.7, then port to 3.4 (or whatever the 
current version of 3.x would be, then).

From guido at python.org  Tue Nov  3 19:39:56 2009
From: guido at python.org (Guido van Rossum)
Date: Tue, 3 Nov 2009 10:39:56 -0800
Subject: [Python-Dev] 2.7 Release? 2.7 == last of the 2.x line?
In-Reply-To: <4AF06A5F.6090501@v.loewis.de>
References: <E077843B-919A-4AC7-95D7-0313E4D0328A@gmail.com> 
	<4AEFD9AE.7090104@molden.no>
	<DF3EB3B9-EB18-42C3-BE34-AADC80006DC4@gmail.com> 
	<4AF06A5F.6090501@v.loewis.de>
Message-ID: <ca471dc20911031039o4931d7c7xc5be220c100c33e5@mail.gmail.com>

On Tue, Nov 3, 2009 at 9:37 AM, "Martin v. L?wis" <martin at v.loewis.de> wrote:
> (and no, adding things like nonlocal to 2.7 doesn't making porting of
> a real application or library any easier, since the existing application
> or library simply doesn't use that keyword.

Agreed.

> In fact, no change to 2.x
> can reasonably simplify porting - only changes to 3.x might - except
> for changes to 2to3, which can simplify porting a lot. But 2to3 should
> be run under 3.x, IMO.)

Disagreed. Better "-3" warnings could make porting easier. (Not just
more warnings -- "better" might mean fewer false positives for
warnings already issued.)

FWIW, it doesn't sound like killing 2.7 is a productive thing to do.
However making 2.7 the end of the line (though with indefinite bugfix
releases) might be. (Indefinite != infinite.)

-- 
--Guido van Rossum (python.org/~guido)

From g.brandl at gmx.net  Tue Nov  3 19:40:56 2009
From: g.brandl at gmx.net (Georg Brandl)
Date: Tue, 03 Nov 2009 18:40:56 +0000
Subject: [Python-Dev] 2.7 Release?  2.7 == last of the 2.x line?
In-Reply-To: <loom.20091103T190051-631@post.gmane.org>
References: <E077843B-919A-4AC7-95D7-0313E4D0328A@gmail.com>	<F245110A-2FDF-460B-8E0D-87785F0E6DB0@fuhm.net>	<BBB79ADA-7E31-4DD8-AAA5-07537FA74C5A@gmail.com>	<D99BA4CF-E16C-47EB-9C60-F1EC5C27CDDC@python.org>	<4AEFC887.6020801@v.loewis.de>	<19184.19195.484852.432070@montanaro.dyndns.org>	<hcpps3$r8u$1@ger.gmane.org>
	<loom.20091103T190051-631@post.gmane.org>
Message-ID: <hcptfi$9ql$1@ger.gmane.org>

Antoine Pitrou schrieb:
> Georg Brandl <g.brandl <at> gmx.net> writes:
>> 
>> skip <at> pobox.com schrieb:
>> >     Martin> And if *2.6* becomes the last of the 2.x series?
>> > 
>> > With all due respect, I don't think you can make that decision now.  The
>> > time to have stated 2.6 would be the end of the 2.x line was when 2.6 was
>> > released.  At that point instead of opening up the trunk for changes you
>> > would have closed it off or merged the py3k branch to trunk.  2.6.0 was
>> > released over a year ago and there has been no effort to suppress bug fix or
>> > feature additions to trunk since then.  If you call 2.6 "the end of 2.x"
>> > you'll have wasted a year of work on 2.7 with about a month to go before the
>> > first 2.7 alpha release.
>> 
>> +1.
> 
> +1 as well. Besides, it is much better communication to release 2.7 and say up
> front that it will be the last major release in the 2.x line. Announcing that
> there won't be a 2.7, however, would give the impression of poor planning.

And of a rush to get rid of 2.x.

Georg


From g.brandl at gmx.net  Tue Nov  3 19:42:12 2009
From: g.brandl at gmx.net (Georg Brandl)
Date: Tue, 03 Nov 2009 18:42:12 +0000
Subject: [Python-Dev] 2.7 Release? 2.7 == last of the 2.x line?
In-Reply-To: <3FE75D51-199C-4221-9B38-94FD2FE9BC4E@gmail.com>
References: <E077843B-919A-4AC7-95D7-0313E4D0328A@gmail.com>	<4AEFD9AE.7090104@molden.no>
	<4AEFF450.9060007@voidspace.org.uk>	<79EADA413EFB44F6864EF71D5B830088@RaymondLaptop1>	<16F0B6B0-677F-464C-9C89-CF92D3713A65@gmail.com>	<116885F3-819E-4B7C-86F7-8E5B246590EB@fuhm.net>	<bad82a80911030928x1f594a92yc315f2ae8b8246d9@mail.gmail.com>
	<3FE75D51-199C-4221-9B38-94FD2FE9BC4E@gmail.com>
Message-ID: <hcpthu$9ql$2@ger.gmane.org>

ssteinerX at gmail.com schrieb:
> 
> On Nov 3, 2009, at 12:28 PM, Arc Riley wrote:
>>
>> The main thing holding back the community are lazy and/or obstinate
>> package maintainers.  If they spent half the time they've put into
>> complaining about Py3 into actually working to upgrade their code
>> they'd be done now.
> 
> That's an inflammatory, defamatory, unsubstantiated, hyperbolic,
> sweeping overgeneralization.

I know a few maintainers, and I have no problem seeing how Arc came
to that conclusion.

Georg


From guido at python.org  Tue Nov  3 19:44:58 2009
From: guido at python.org (Guido van Rossum)
Date: Tue, 3 Nov 2009 10:44:58 -0800
Subject: [Python-Dev] PEP 3003 - Python Language Moratorium
In-Reply-To: <4AF07481.3080106@v.loewis.de>
References: <ca471dc20911030935l25251e0diec64427f5d7bcbf2@mail.gmail.com> 
	<4AF07481.3080106@v.loewis.de>
Message-ID: <ca471dc20911031044t22900521qc4b4295209ad04a8@mail.gmail.com>

On Tue, Nov 3, 2009 at 10:20 AM, "Martin v. L?wis" <martin at v.loewis.de> wrote:
>> * General language semantics
>> ? ? The language operates as-is with only specific exemptions (see
>> ? ? below).
>
> Would PEP 382 (namespace packages) fall under the moratorium?

Import semantics are a bit of a gray area. However I think the
moratorium shouldn't be interpreted to prohibit this. I think there
was a question on python-ideas about import semantics, and my response
was something like "import semantics vary between Python
implementations anyway." I'll add this to the PEP.

-- 
--Guido van Rossum (python.org/~guido)

From charlesc-lists-python-dev at pyropus.ca  Tue Nov  3 19:56:41 2009
From: charlesc-lists-python-dev at pyropus.ca (Charles Cazabon)
Date: Tue, 3 Nov 2009 12:56:41 -0600
Subject: [Python-Dev] 2.7 Release? 2.7 == last of the 2.x line?
In-Reply-To: <4AF0734A.3080705@v.loewis.de>
References: <E077843B-919A-4AC7-95D7-0313E4D0328A@gmail.com>
	<4AEFD9AE.7090104@molden.no> <4AEFF450.9060007@voidspace.org.uk>
	<79EADA413EFB44F6864EF71D5B830088@RaymondLaptop1>
	<16F0B6B0-677F-464C-9C89-CF92D3713A65@gmail.com>
	<116885F3-819E-4B7C-86F7-8E5B246590EB@fuhm.net>
	<bad82a80911030928x1f594a92yc315f2ae8b8246d9@mail.gmail.com>
	<20091103174131.GA10645@pyropus.ca> <4AF0734A.3080705@v.loewis.de>
Message-ID: <20091103185641.GA13969@pyropus.ca>

"Martin v. L?wis" <martin at v.loewis.de> wrote:
> > I wouldn't say that.  For instance, I'm just starting a refactoring that will
> > result in getmail v.5, but I need to target Python 2.5 and up, so there's
> > essentially no way the code will run in Python 3.x (as another list member
> > posted).
> 
> That's a common myth. It is very well possible, using 2to3. You don't
> have to wait until you can drop 2.5 to start supporting 3.x, out of a
> single code base.

I haven't tried this, but I was relying on Daniel Stutzbach's opinion posted
here earlier:
http://mail.python.org/pipermail/python-dev/2009-November/093608.html

  It's pretty easy to make Python source that works under 2.6 and 3.x.  It's
  basically impossible to make Python source that works under 2.4/2.5 and
  3.x. 

> > Why do I need to target Python 2.5?  Because that's the most current
> > default version of Python shipped in Debian stable and various other
> > distributions that don't stay on the bleeding edge.
> 
> Are you saying that it doesn't *run* on 2.6?

No.  getmail v.4 runs fine on Python 2.3.4 through 2.6.x; getmail's code has
always been pretty forward-compatible.

> Why?  (not sure what you mean by "targetting")

By target, I mean backwards compatibility -- the minimum version of Python
which is required to run getmail.  getmail v.4 came out of beta about five
years ago targetting Python 2.3 and higher, and 2.3 was too bleeding-edge for
many users -- it wasn't shipped by many Linux distributions for a long time
after getmail v.4 was released.  Debian *still* shipps getmail v3 (which
supports back to Python 1.5.2) today, although they also ship v4.

getmail v.5 will be released in a month or three.  And many of its users will
still have Python 2.5, so that's what getmail has to run on.  Perhaps Daniel's
comment is incorrect (I have no evidence either way), but if it is true that
having a single getmail codebase run on Python 2.5 and Python 3.x is
"basically impossible", then I won't be too concerned about 3.x for a while
yet.

I've been an avid Python user and promoter since 1.2, but saying "drop Python
2.x and switch to 3 now" is simply not realistic in any of the environments in
which I use Python daily.  My $0.02.

Charles
-- 
-----------------------------------------------------------------------
Charles Cazabon
GPL'ed software available at:               http://pyropus.ca/software/
-----------------------------------------------------------------------

From guido at python.org  Tue Nov  3 19:56:44 2009
From: guido at python.org (Guido van Rossum)
Date: Tue, 3 Nov 2009 10:56:44 -0800
Subject: [Python-Dev] Retrieve an arbitrary element from a set
	withoutremoving it
In-Reply-To: <9d153b7c0911030020n336fd15bobeea4ea713cd8b23@mail.gmail.com>
References: <7b0045ed0910301843y7059d316s8a851142b78b7aeb@mail.gmail.com> 
	<20091102052900.GA25160@cskk.homeip.net>
	<4AEF5AFA.5060806@canterbury.ac.nz> 
	<9d153b7c0911030020n336fd15bobeea4ea713cd8b23@mail.gmail.com>
Message-ID: <ca471dc20911031056j4c9849bep1a83e1582db9bf4b@mail.gmail.com>

On Tue, Nov 3, 2009 at 12:20 AM, Yuvgoog Greenle <ubershmekel at gmail.com> wrote:
> On Tue, Nov 3, 2009 at 12:19 AM, Greg Ewing <greg.ewing at canterbury.ac.nz>
> wrote:
>>
>> Cameron Simpson wrote:
>>
>>> Personally, I'm for the iteration spec in a lot of ways.
>>>
>>> Firstly, a .get()/.pick() that always returns the same element feels
>>> horrible. Is there anyone here who _likes_ it?
>>
>
> State might cause people to use this to iterate which would be just plain
> wrong. The 2 things I have a bad feeling about is:
> 1. random.choice could be a pythonic alternative to what some have mentioned
> here but it doesn't work on sets. The following code raises TypeError: 'set'
> object does not support indexing:
> ?? ?import random
> ?? ?random.choice(set([1,2,3,4,5,6]))

There is a huge difference between picking a random element of a set
and picking an arbitrary one. Algorithms that need randomness *must*
use the random generator (and because the hash table implementation
doesn't provide O(1) access they will have to use a list first).
Algorithms that don't need randomness should not be forced to pay for
the (considerable!) cost of calling the random number generator, and
must accept that the element selected may be predictable.

> this is kinda ironic:?http://en.wikipedia.org/wiki/Axiom_of_choice

Also irrelevant. That Axiom is only interesting for infinite sets,
which Python does not support (at least not using the set builtin --
you can of course write your own symbolic algebra package in Python
that can be used to represent certain infinite sets, though you still
won't be able to iterate over all of their elements :-).

> 2. If I store objects in a set and modify their attributes I have no O(1)
> way of getting the objects back if I stumble upon an equivalent object. In
> cases like that I'd have to either iterate over the set or use a dict with
> key == value. I have a feeling the "get" or "peek" api could cater to this
> need. A use case for this could be an implementation of a cookie jar with a
> set of cookies where equivalence is defined by the name and domain.

Sets are what they are. If they don't fit your requirements, don't use
them. Don't get fooled by the name -- a dict also isn't a very good
data structure to implement an actual Chinese-English dictionary, for
example. :-)

-- 
--Guido van Rossum (python.org/~guido)

From mal at egenix.com  Tue Nov  3 20:15:54 2009
From: mal at egenix.com (M.-A. Lemburg)
Date: Tue, 03 Nov 2009 20:15:54 +0100
Subject: [Python-Dev] nonlocal keyword in 2.x?
In-Reply-To: <4AF06E20.7050307@v.loewis.de>
References: <da7032ce0910211856g45e45632n550aa5b4a10f9a62@mail.gmail.com>	<02A812F3B69E46D180B45133D52D1F2B@RaymondLaptop1>	<bbaeab100911011557t714e3b4cxcdc91b3fb9a49b84@mail.gmail.com>	<4AEE9862.8010201@voidspace.org.uk>	<ca471dc20911021001u102da717wcf4da0eada0867e@mail.gmail.com>	<5C4388816CE0428FB27EA69CE77CFCC0@RaymondLaptop1>	<ca471dc20911021009u7485adcdgde8fae04a5061059@mail.gmail.com>	<Pine.LNX.4.64.0911021405420.21420@kimball.webabinitio.net>	<4AEF4C53.7090902@v.loewis.de>	<Pine.LNX.4.64.0911021624110.21420@kimball.webabinitio.net>	<ca471dc20911021333r168e257eja1e5f79dbe4ae234@mail.gmail.com>	<loom.20091102T223839-797@post.gmane.org>	<4AEFF3A3.3020103@egenix.com>
	<4AF06E20.7050307@v.loewis.de>
Message-ID: <4AF0816A.706@egenix.com>

"Martin v. L?wis" wrote:
>>> 2.7 has an up-to-date backport of the C IO lib (as well as the memoryview
>>> object), which means it is better for people wanting to ease transition to 3.x.
>>>
>>> But of course, as Martin said, few people will want to support 2.7 only and not 
>>> 2.6.
>>
>> Since 2.7 will be closer to 3.2 than 2.6, the result will more likely
>> be: many people will want to support 2.7 and 3.x - which is really
>> what we should be after.
> 
> Supporting 2.7 doesn't mean that you have to use the nonlocal keyword
> (nor does support for 3.2 require that keyword). So I'm not sure how
> the backport of a feature to 2.x could encourage people to forward-port
> to 3.x.

True. This particular backport is not all that useful - it is well
possible to write code that doesn't need the feature, both in 2.7 and
3.2.

However, if a 3.2 feature is required for code to work out of the
box without changes, then I'd regard such a feature as potential
backport candidate.

memoryviews are a good example. In 3.x the buffer object is gone
and memoryviews are the new replacement for it.

Code written to support both 2.7 and 3.2 would need to have
access to these memoryviews in order to avoid having to
write special support for buffer objects (for 2.7) and
memoryviews (for 3.x).

By backporting memoryviews to 2.7, users no longer have to
write support code for buffer objects and that makes things
easier for them.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Nov 03 2009)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try our new mxODBC.Connect Python Database Interface for free ! ::::


   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
           Registered at Amtsgericht Duesseldorf: HRB 46611
               http://www.egenix.com/company/contact/

From brett at python.org  Tue Nov  3 20:21:06 2009
From: brett at python.org (Brett Cannon)
Date: Tue, 3 Nov 2009 11:21:06 -0800
Subject: [Python-Dev] PEP 3003 - Python Language Moratorium
In-Reply-To: <ca471dc20911030935l25251e0diec64427f5d7bcbf2@mail.gmail.com>
References: <ca471dc20911030935l25251e0diec64427f5d7bcbf2@mail.gmail.com>
Message-ID: <bbaeab100911031121y4ba5342eva7a2240cee55f8c0@mail.gmail.com>

On Tue, Nov 3, 2009 at 09:35, Guido van Rossum <guido at python.org> wrote:
> I've checked draft (!) PEP 3003, "Python Language Moratorium", into
> SVN. As authors I've listed Jesse, Brett and myself.
>
> On python-ideas the moratorium idea got fairly positive responses
> (more positive than I'd expected, in fact) but I'm bracing myself for
> fierce discussion here on python-dev. It's important to me that if if
> this is accepted it is a "rough consensus" decision (working code we
> already have plenty of :-), not something enforced by a vocal minority
> or an influential individual such as myself. If there's too much
> opposition I'll withdraw the PEP so as not to waste everybody's time
> with a fruitless discussion.
>

Are you going to gauge it roughly from python-dev feedback, or should
we take a more formal vote on python-committers once the PEP has
settled?

> The PEP tries to spell out some gray areas but I'm sure there will be
> others; that's life. Do note that the PEP proposes to be *retroactive*
> back to the 3.1 release, i.e. the "frozen" version of the language is
> the state in which it was released as 3.1.

I'm obviously +1 for this.

I think one thing to decide is how long the moratorium is in effect.
As of right now the PEP's abstract says for "at least two years",
which taking into account the release schedule Benjamin is proposing
and assuming acceptance at the end of the year puts us at roughly the
release of Python 3.3. Should we just say this will be in affect until
development is open for Python 3.4?

-Brett

From martin at v.loewis.de  Tue Nov  3 20:22:41 2009
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Tue, 03 Nov 2009 20:22:41 +0100
Subject: [Python-Dev] 2.7 Release? 2.7 == last of the 2.x line?
In-Reply-To: <20091103185641.GA13969@pyropus.ca>
References: <E077843B-919A-4AC7-95D7-0313E4D0328A@gmail.com>	<4AEFD9AE.7090104@molden.no>
	<4AEFF450.9060007@voidspace.org.uk>	<79EADA413EFB44F6864EF71D5B830088@RaymondLaptop1>	<16F0B6B0-677F-464C-9C89-CF92D3713A65@gmail.com>	<116885F3-819E-4B7C-86F7-8E5B246590EB@fuhm.net>	<bad82a80911030928x1f594a92yc315f2ae8b8246d9@mail.gmail.com>	<20091103174131.GA10645@pyropus.ca>
	<4AF0734A.3080705@v.loewis.de> <20091103185641.GA13969@pyropus.ca>
Message-ID: <4AF08301.6070000@v.loewis.de>

>>> I wouldn't say that.  For instance, I'm just starting a refactoring that will
>>> result in getmail v.5, but I need to target Python 2.5 and up, so there's
>>> essentially no way the code will run in Python 3.x (as another list member
>>> posted).
>> That's a common myth. It is very well possible, using 2to3. You don't
>> have to wait until you can drop 2.5 to start supporting 3.x, out of a
>> single code base.
> 
> I haven't tried this, but I was relying on Daniel Stutzbach's opinion posted
> here earlier:
> http://mail.python.org/pipermail/python-dev/2009-November/093608.html
> 
>   It's pretty easy to make Python source that works under 2.6 and 3.x.  It's
>   basically impossible to make Python source that works under 2.4/2.5 and
>   3.x. 

But that's a completely different statement. "make code that works on
both 2.x and 3.x" is *entirely* different from "make code written for
2.x also run on 3.x". The latter is less constrained: to run it on 3.x,
you can convert it to 3.x first, using 2to3. So it would not be the
*same* code that runs, but you can have single source code for both
2.x and 3.x - only that what you run is not literally the source code.

> getmail v.5 will be released in a month or three.  And many of its users will
> still have Python 2.5, so that's what getmail has to run on.  Perhaps Daniel's
> comment is incorrect (I have no evidence either way), but if it is true that
> having a single getmail codebase run on Python 2.5 and Python 3.x is
> "basically impossible", then I won't be too concerned about 3.x for a while
> yet.

No no no. Daniels comment is correct, in that you can't run the code
*unmodified*. However, it is well possible to have a single code base
for both 2.x and 3.x - you just don't *run* the code on 3.x (but convert
it first, e.g. at installation time).

> I've been an avid Python user and promoter since 1.2, but saying "drop Python
> 2.x and switch to 3 now" is simply not realistic in any of the environments in
> which I use Python daily. 

And I wouldn't say that. Instead, I say "support both 2.x and 3.x from a
single code base". That approach can work for supporting everything from
2.3 to 3.2 (perhaps even from 2.0 to 3.x).

Regards,
Martin

From mal at egenix.com  Tue Nov  3 20:23:49 2009
From: mal at egenix.com (M.-A. Lemburg)
Date: Tue, 03 Nov 2009 20:23:49 +0100
Subject: [Python-Dev] PEP 3003 - Python Language Moratorium
In-Reply-To: <ca471dc20911030935l25251e0diec64427f5d7bcbf2@mail.gmail.com>
References: <ca471dc20911030935l25251e0diec64427f5d7bcbf2@mail.gmail.com>
Message-ID: <4AF08345.6040603@egenix.com>

Guido van Rossum wrote:
> I've checked draft (!) PEP 3003, "Python Language Moratorium", into
> SVN. As authors I've listed Jesse, Brett and myself.
> 
> On python-ideas the moratorium idea got fairly positive responses
> (more positive than I'd expected, in fact) but I'm bracing myself for
> fierce discussion here on python-dev. It's important to me that if if
> this is accepted it is a "rough consensus" decision (working code we
> already have plenty of :-), not something enforced by a vocal minority
> or an influential individual such as myself. If there's too much
> opposition I'll withdraw the PEP so as not to waste everybody's time
> with a fruitless discussion.
> 
> The PEP tries to spell out some gray areas but I'm sure there will be
> others; that's life. Do note that the PEP proposes to be *retroactive*
> back to the 3.1 release, i.e. the "frozen" version of the language is
> the state in which it was released as 3.1.

+1.

> This suspension of features is designed to allow non-CPython implementations
> to "catch up" to the core implementation of the language, help ease adoption
> of Python 3.x, and provide a more stable base for the community.

I'd also add "to allow users to catch up"... ;-)

One question:

There are currently number of patch waiting on the tracker for
additional Unicode feature support and it's also likely that we'll
want to upgrade to a more recent Unicode version within the
next few years.

How would such indirect changes be seen under the moratorium ?

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Nov 03 2009)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try our new mxODBC.Connect Python Database Interface for free ! ::::


   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
           Registered at Amtsgericht Duesseldorf: HRB 46611
               http://www.egenix.com/company/contact/

From skip at pobox.com  Tue Nov  3 20:41:06 2009
From: skip at pobox.com (skip at pobox.com)
Date: Tue, 3 Nov 2009 13:41:06 -0600
Subject: [Python-Dev] PEP 3003 - Python Language Moratorium
In-Reply-To: <ca471dc20911030935l25251e0diec64427f5d7bcbf2@mail.gmail.com>
References: <ca471dc20911030935l25251e0diec64427f5d7bcbf2@mail.gmail.com>
Message-ID: <19184.34642.648427.211546@montanaro.dyndns.org>


    Guido> I've checked draft (!) PEP 3003, "Python Language Moratorium",
    Guido> into SVN.

LGTM.

Skip

From brett at python.org  Tue Nov  3 20:53:02 2009
From: brett at python.org (Brett Cannon)
Date: Tue, 3 Nov 2009 11:53:02 -0800
Subject: [Python-Dev] 2.7 Release? 2.7 == last of the 2.x line?
In-Reply-To: <5b8d13220911030540s3d700265ib00f4277b7ce4a2b@mail.gmail.com>
References: <E077843B-919A-4AC7-95D7-0313E4D0328A@gmail.com> 
	<F245110A-2FDF-460B-8E0D-87785F0E6DB0@fuhm.net>
	<BBB79ADA-7E31-4DD8-AAA5-07537FA74C5A@gmail.com> 
	<D99BA4CF-E16C-47EB-9C60-F1EC5C27CDDC@python.org>
	<4AEFC887.6020801@v.loewis.de> 
	<3DE1D8FB-896E-441D-937D-AB63EB3A8E22@python.org>
	<5b8d13220911030540s3d700265ib00f4277b7ce4a2b@mail.gmail.com>
Message-ID: <bbaeab100911031153h61c356dfw850c53d4662aca7@mail.gmail.com>

On Tue, Nov 3, 2009 at 05:40, David Cournapeau <cournape at gmail.com> wrote:
> On Tue, Nov 3, 2009 at 9:55 PM, Barry Warsaw <barry at python.org> wrote:
>
>>
>> Then clearly we can't back port features.
>>
>> I'd like to read some case studies of people who have migrated applications
>> from 2.6 to 3.0.
>
> +1, especially for packages which have a lot of C code: the current
> documentation is sparse :) The only helpful reference I have found so
> far is an email by MvL concerning psycopg2 port.

There used to be a page on the wiki of case studies, but I couldn't
find it. But one C-specific case study is the port of Wing's extension
modules to support from Python 2.0 to 3.0:
http://pythonology.blogspot.com/2009/02/making-code-run-on-python-20-through-30.html
. There is also a wiki pages explicitly about porting C code:
http://wiki.python.org/moin/cporting and
http://wiki.python.org/moin/Py3kExtensionModules . Those latter pages
include links to the official porting docs:
http://docs.python.org/3.1/howto/cporting.html .

And as Georg pointed out there is always the python-porting mailing
list: http://mail.python.org/mailman/listinfo/python-porting .

I'm afraid there is some FUD going around here, which is
understandable since no one wants to burn a ton of time on something
that will be difficult or take a lot of time. But I have not heard
anyone in this email thread (or anywhere for that matter) say that
they tried a port in earnest and it turned out to be difficult.

-Brett

From barry at python.org  Tue Nov  3 20:56:28 2009
From: barry at python.org (Barry Warsaw)
Date: Tue, 3 Nov 2009 14:56:28 -0500
Subject: [Python-Dev] PEP 3003 - Python Language Moratorium
In-Reply-To: <ca471dc20911030935l25251e0diec64427f5d7bcbf2@mail.gmail.com>
References: <ca471dc20911030935l25251e0diec64427f5d7bcbf2@mail.gmail.com>
Message-ID: <B9E66EB8-54B5-4578-AA2A-7C8614FE785F@python.org>

On Nov 3, 2009, at 12:35 PM, Guido van Rossum wrote:

> I've checked draft (!) PEP 3003, "Python Language Moratorium", into
> SVN. As authors I've listed Jesse, Brett and myself.
>
> On python-ideas the moratorium idea got fairly positive responses
> (more positive than I'd expected, in fact) but I'm bracing myself for
> fierce discussion here on python-dev. It's important to me that if if
> this is accepted it is a "rough consensus" decision (working code we
> already have plenty of :-), not something enforced by a vocal minority
> or an influential individual such as myself. If there's too much
> opposition I'll withdraw the PEP so as not to waste everybody's time
> with a fruitless discussion.
>
> The PEP tries to spell out some gray areas but I'm sure there will be
> others; that's life. Do note that the PEP proposes to be *retroactive*
> back to the 3.1 release, i.e. the "frozen" version of the language is
> the state in which it was released as 3.1.

I think this is a great idea.  I'd love to see the energy normally put  
into evolving the language into making the stdlib really kick ass.

-Barry

-------------- next part --------------
A non-text attachment was scrubbed...
Name: PGP.sig
Type: application/pgp-signature
Size: 832 bytes
Desc: This is a digitally signed message part
URL: <http://mail.python.org/pipermail/python-dev/attachments/20091103/15780ad9/attachment.pgp>

From jnoller at gmail.com  Tue Nov  3 21:01:22 2009
From: jnoller at gmail.com (Jesse Noller)
Date: Tue, 3 Nov 2009 15:01:22 -0500
Subject: [Python-Dev] PEP 3003 - Python Language Moratorium
In-Reply-To: <ca471dc20911030935l25251e0diec64427f5d7bcbf2@mail.gmail.com>
References: <ca471dc20911030935l25251e0diec64427f5d7bcbf2@mail.gmail.com>
Message-ID: <4222a8490911031201y3a70087dv531255a73241dab3@mail.gmail.com>

On Tue, Nov 3, 2009 at 12:35 PM, Guido van Rossum <guido at python.org> wrote:
> I've checked draft (!) PEP 3003, "Python Language Moratorium", into
> SVN. As authors I've listed Jesse, Brett and myself.
>
> On python-ideas the moratorium idea got fairly positive responses
> (more positive than I'd expected, in fact) but I'm bracing myself for
> fierce discussion here on python-dev. It's important to me that if if
> this is accepted it is a "rough consensus" decision (working code we
> already have plenty of :-), not something enforced by a vocal minority
> or an influential individual such as myself. If there's too much
> opposition I'll withdraw the PEP so as not to waste everybody's time
> with a fruitless discussion.
>
> The PEP tries to spell out some gray areas but I'm sure there will be
> others; that's life. Do note that the PEP proposes to be *retroactive*
> back to the 3.1 release, i.e. the "frozen" version of the language is
> the state in which it was released as 3.1.
>
> --
> --Guido van Rossum (python.org/~guido)
>

+1 - I'll work on some grammar kinks in my prose asap

From guido at python.org  Tue Nov  3 21:01:07 2009
From: guido at python.org (Guido van Rossum)
Date: Tue, 3 Nov 2009 12:01:07 -0800
Subject: [Python-Dev] PEP 3003 - Python Language Moratorium
In-Reply-To: <bbaeab100911031121y4ba5342eva7a2240cee55f8c0@mail.gmail.com>
References: <ca471dc20911030935l25251e0diec64427f5d7bcbf2@mail.gmail.com> 
	<bbaeab100911031121y4ba5342eva7a2240cee55f8c0@mail.gmail.com>
Message-ID: <ca471dc20911031201k81e5fc2qe49fa44ab7537223@mail.gmail.com>

On Tue, Nov 3, 2009 at 11:21 AM, Brett Cannon <brett at python.org> wrote:
> Are you going to gauge it roughly from python-dev feedback, or should
> we take a more formal vote on python-committers once the PEP has
> settled?

I'll not take a formal vote unless the discussion suggests there's a
lot of pushback. So far I've seen very little.

>> The PEP tries to spell out some gray areas but I'm sure there will be
>> others; that's life. Do note that the PEP proposes to be *retroactive*
>> back to the 3.1 release, i.e. the "frozen" version of the language is
>> the state in which it was released as 3.1.
>
> I'm obviously +1 for this.
>
> I think one thing to decide is how long the moratorium is in effect.
> As of right now the PEP's abstract says for "at least two years",

Yeah, I changed that from Jesse's and your "not to exceed two years".

> which taking into account the release schedule Benjamin is proposing
> and assuming acceptance at the end of the year puts us at roughly the
> release of Python 3.3. Should we just say this will be in affect until
> development is open for Python 3.4?

I like the actual time limit, because it doesn't allow the backdoor of
releasing 3.4 early. I'm fine with saying the moratorium effectively
ends after 3.3 is released.

-- 
--Guido van Rossum (python.org/~guido)

From guido at python.org  Tue Nov  3 21:03:13 2009
From: guido at python.org (Guido van Rossum)
Date: Tue, 3 Nov 2009 12:03:13 -0800
Subject: [Python-Dev] PEP 3003 - Python Language Moratorium
In-Reply-To: <4AF08345.6040603@egenix.com>
References: <ca471dc20911030935l25251e0diec64427f5d7bcbf2@mail.gmail.com> 
	<4AF08345.6040603@egenix.com>
Message-ID: <ca471dc20911031203i2dd83f7ft32f4487cd3edacb4@mail.gmail.com>

On Tue, Nov 3, 2009 at 11:23 AM, M.-A. Lemburg <mal at egenix.com> wrote:
>> This suspension of features is designed to allow non-CPython implementations
>> to "catch up" to the core implementation of the language, help ease adoption
>> of Python 3.x, and provide a more stable base for the community.
>
> I'd also add "to allow users to catch up"... ;-)

IMO that's implied by "help ease adoption".

> One question:
>
> There are currently number of patch waiting on the tracker for
> additional Unicode feature support and it's also likely that we'll
> want to upgrade to a more recent Unicode version within the
> next few years.
>
> How would such indirect changes be seen under the moratorium ?

That would fall under the Case-by-Case Exemptions section. "Within the
next few years" sounds like it might well wait until the moratorium is
ended though. :-)

-- 
--Guido van Rossum (python.org/~guido)

From tjreedy at udel.edu  Tue Nov  3 21:23:38 2009
From: tjreedy at udel.edu (Terry Reedy)
Date: Tue, 03 Nov 2009 15:23:38 -0500
Subject: [Python-Dev] 2.7 Release?  2.7 == last of the 2.x line?
In-Reply-To: <116885F3-819E-4B7C-86F7-8E5B246590EB@fuhm.net>
References: <E077843B-919A-4AC7-95D7-0313E4D0328A@gmail.com><4AEFD9AE.7090104@molden.no>	<4AEFF450.9060007@voidspace.org.uk>	<79EADA413EFB44F6864EF71D5B830088@RaymondLaptop1>	<16F0B6B0-677F-464C-9C89-CF92D3713A65@gmail.com>
	<116885F3-819E-4B7C-86F7-8E5B246590EB@fuhm.net>
Message-ID: <hcq3g8$lb$1@ger.gmane.org>

James Y Knight wrote:

> If that happens, it's not true that there's *nowhere* to go. A solution 
> would be to discard 3.x as a failed experiment, take everything that is 
> useful from it and port it to 2.x, and simply continue development from 
> the last 2.x release. And from there, features can be deprecated and 
> then removed a few releases later, as is the usual policy.

The once 'usual policy' of removal was changed several years ago to 
'defer removals until 3.0' because people wanted a more stable language 
and claimed that they would prefer to deal with several removals all at 
once. So old-style classes were kept around long past when they would 
have been removed under the old 'usual policy'. Ditto for old-style int 
/ int and some others.  Or one can simply recognize that 3.0 was the 
'few releases later' release of that policy.

The other big change was switching to unicode strings from ascii strings 
with optional unicode string add-on. That was/is/will-be a hassle 
regardless of when and what name, but necessary for Python to be a 
modern world language.

tjr


From facundobatista at gmail.com  Tue Nov  3 21:27:24 2009
From: facundobatista at gmail.com (Facundo Batista)
Date: Tue, 3 Nov 2009 17:27:24 -0300
Subject: [Python-Dev] PEP 3003 - Python Language Moratorium
In-Reply-To: <ca471dc20911030935l25251e0diec64427f5d7bcbf2@mail.gmail.com>
References: <ca471dc20911030935l25251e0diec64427f5d7bcbf2@mail.gmail.com>
Message-ID: <e04bdf310911031227m47306ea8yeeabdebcb2624c50@mail.gmail.com>

On Tue, Nov 3, 2009 at 2:35 PM, Guido van Rossum <guido at python.org> wrote:

> fierce discussion here on python-dev. It's important to me that if if
> this is accepted it is a "rough consensus" decision (working code we

+1 to the PEP.

-- 
.    Facundo

Blog: http://www.taniquetil.com.ar/plog/
PyAr: http://www.python.org/ar/

From fuzzyman at voidspace.org.uk  Tue Nov  3 21:42:09 2009
From: fuzzyman at voidspace.org.uk (Michael Foord)
Date: Tue, 03 Nov 2009 21:42:09 +0100
Subject: [Python-Dev] PEP 3003 - Python Language Moratorium
In-Reply-To: <B9E66EB8-54B5-4578-AA2A-7C8614FE785F@python.org>
References: <ca471dc20911030935l25251e0diec64427f5d7bcbf2@mail.gmail.com>
	<B9E66EB8-54B5-4578-AA2A-7C8614FE785F@python.org>
Message-ID: <4AF095A1.6040406@voidspace.org.uk>

Barry Warsaw wrote:
> On Nov 3, 2009, at 12:35 PM, Guido van Rossum wrote:
>
>> I've checked draft (!) PEP 3003, "Python Language Moratorium", into
>> SVN. As authors I've listed Jesse, Brett and myself.
>>
>> On python-ideas the moratorium idea got fairly positive responses
>> (more positive than I'd expected, in fact) but I'm bracing myself for
>> fierce discussion here on python-dev. It's important to me that if if
>> this is accepted it is a "rough consensus" decision (working code we
>> already have plenty of :-), not something enforced by a vocal minority
>> or an influential individual such as myself. If there's too much
>> opposition I'll withdraw the PEP so as not to waste everybody's time
>> with a fruitless discussion.
>>
>> The PEP tries to spell out some gray areas but I'm sure there will be
>> others; that's life. Do note that the PEP proposes to be *retroactive*
>> back to the 3.1 release, i.e. the "frozen" version of the language is
>> the state in which it was released as 3.1.
>
> I think this is a great idea.  I'd love to see the energy normally put 
> into evolving the language into making the stdlib really kick ass.
>

+lots

Michael

> -Barry
>
> ------------------------------------------------------------------------
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk
>   


-- 
http://www.ironpythoninaction.com/


From g.brandl at gmx.net  Tue Nov  3 22:00:21 2009
From: g.brandl at gmx.net (Georg Brandl)
Date: Tue, 03 Nov 2009 21:00:21 +0000
Subject: [Python-Dev] 2.7 Release?  2.7 == last of the 2.x line?
In-Reply-To: <hcq3g8$lb$1@ger.gmane.org>
References: <E077843B-919A-4AC7-95D7-0313E4D0328A@gmail.com><4AEFD9AE.7090104@molden.no>	<4AEFF450.9060007@voidspace.org.uk>	<79EADA413EFB44F6864EF71D5B830088@RaymondLaptop1>	<16F0B6B0-677F-464C-9C89-CF92D3713A65@gmail.com>	<116885F3-819E-4B7C-86F7-8E5B246590EB@fuhm.net>
	<hcq3g8$lb$1@ger.gmane.org>
Message-ID: <hcq5ku$75n$1@ger.gmane.org>

Terry Reedy schrieb:
> James Y Knight wrote:
> 
>> If that happens, it's not true that there's *nowhere* to go. A solution 
>> would be to discard 3.x as a failed experiment, take everything that is 
>> useful from it and port it to 2.x, and simply continue development from 
>> the last 2.x release. And from there, features can be deprecated and 
>> then removed a few releases later, as is the usual policy.
> 
> The once 'usual policy' of removal was changed several years ago to 
> 'defer removals until 3.0' because people wanted a more stable language 
> and claimed that they would prefer to deal with several removals all at 
> once. So old-style classes were kept around long past when they would 
> have been removed under the old 'usual policy'. Ditto for old-style int 
> / int and some others.  Or one can simply recognize that 3.0 was the 
> 'few releases later' release of that policy.
> 
> The other big change was switching to unicode strings from ascii strings 
> with optional unicode string add-on. That was/is/will-be a hassle 
> regardless of when and what name, but necessary for Python to be a 
> modern world language.

>From my experience, string to unicode migration really is the biggest
pain when porting anything that handles real-world data.

An interesting experiment would have been to split the big changes in two
parts, e.g., a 2.95 that only has the string to unicode changes, and
a 3.0 that has all the rest.  Of course, people would have complained
about having to port twice :)

Georg


From mike.klaas at gmail.com  Tue Nov  3 22:05:19 2009
From: mike.klaas at gmail.com (Mike Klaas)
Date: Tue, 3 Nov 2009 13:05:19 -0800
Subject: [Python-Dev] 2.7 Release? 2.7 == last of the 2.x line?
In-Reply-To: <hcpthu$9ql$2@ger.gmane.org>
References: <E077843B-919A-4AC7-95D7-0313E4D0328A@gmail.com>
	<4AEFD9AE.7090104@molden.no> <4AEFF450.9060007@voidspace.org.uk>
	<79EADA413EFB44F6864EF71D5B830088@RaymondLaptop1>
	<16F0B6B0-677F-464C-9C89-CF92D3713A65@gmail.com>
	<116885F3-819E-4B7C-86F7-8E5B246590EB@fuhm.net>
	<bad82a80911030928x1f594a92yc315f2ae8b8246d9@mail.gmail.com>
	<3FE75D51-199C-4221-9B38-94FD2FE9BC4E@gmail.com>
	<hcpthu$9ql$2@ger.gmane.org>
Message-ID: <3d2ce8cb0911031305y330b832fo58e6907011e69efb@mail.gmail.com>

On Tue, Nov 3, 2009 at 10:42 AM, Georg Brandl <g.brandl at gmx.net> wrote:

> ssteinerX at gmail.com schrieb:
> >
> > On Nov 3, 2009, at 12:28 PM, Arc Riley wrote:
> >>
> >> The main thing holding back the community are lazy and/or obstinate
> >> package maintainers.  If they spent half the time they've put into
> >> complaining about Py3 into actually working to upgrade their code
> >> they'd be done now.
> >
> > That's an inflammatory, defamatory, unsubstantiated, hyperbolic,
> > sweeping overgeneralization.
>
> I know a few maintainers, and I have no problem seeing how Arc came
> to that conclusion.


Be that as it may, the only way python 3 will be widely adopted if people
have motivation to (need to be compatible with other libs, pressure from
users, their own interest in fostering python 3.0, etc.).  Deriding them as
"lazy" accomplishes nothing and obscures the fact that it is the python
maintainers responsibility to bring about this motivation if they want
python 3.0 to be adopted.  No-one is going to convert to python 3.0 because
you called them lazy.

-Mike

>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20091103/47d6f86a/attachment.htm>

From guido at python.org  Tue Nov  3 22:10:14 2009
From: guido at python.org (Guido van Rossum)
Date: Tue, 3 Nov 2009 13:10:14 -0800
Subject: [Python-Dev] 2.7 Release? 2.7 == last of the 2.x line?
In-Reply-To: <hcq5ku$75n$1@ger.gmane.org>
References: <E077843B-919A-4AC7-95D7-0313E4D0328A@gmail.com> 
	<4AEFD9AE.7090104@molden.no> <4AEFF450.9060007@voidspace.org.uk> 
	<79EADA413EFB44F6864EF71D5B830088@RaymondLaptop1>
	<16F0B6B0-677F-464C-9C89-CF92D3713A65@gmail.com> 
	<116885F3-819E-4B7C-86F7-8E5B246590EB@fuhm.net>
	<hcq3g8$lb$1@ger.gmane.org> <hcq5ku$75n$1@ger.gmane.org>
Message-ID: <ca471dc20911031310k5a4232eah310fb4031bbf8800@mail.gmail.com>

On Tue, Nov 3, 2009 at 1:00 PM, Georg Brandl <g.brandl at gmx.net> wrote:
> From my experience, string to unicode migration really is the biggest
> pain when porting anything that handles real-world data.

Of course, handling Unicode right is also the biggest pain when
writing code for 2.x in the first place -- writing greenfield code
targeted at 3.x that does Unicode right is a lot easier. Alas, this
does nothing for those folks who already have working Unicode handling
code for 2.x...

-- 
--Guido van Rossum (python.org/~guido)

From solipsis at pitrou.net  Tue Nov  3 22:56:19 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Tue, 3 Nov 2009 21:56:19 +0000 (UTC)
Subject: [Python-Dev] Porting C extensions
References: <E077843B-919A-4AC7-95D7-0313E4D0328A@gmail.com>
	<F245110A-2FDF-460B-8E0D-87785F0E6DB0@fuhm.net>
	<BBB79ADA-7E31-4DD8-AAA5-07537FA74C5A@gmail.com>
	<D99BA4CF-E16C-47EB-9C60-F1EC5C27CDDC@python.org>
	<4AEFC887.6020801@v.loewis.de>
	<3DE1D8FB-896E-441D-937D-AB63EB3A8E22@python.org>
	<5b8d13220911030540s3d700265ib00f4277b7ce4a2b@mail.gmail.com>
	<bbaeab100911031153h61c356dfw850c53d4662aca7@mail.gmail.com>
Message-ID: <loom.20091103T225333-378@post.gmane.org>

Brett Cannon <brett <at> python.org> writes:
> 
> I'm afraid there is some FUD going around here, which is
> understandable since no one wants to burn a ton of time on something
> that will be difficult or take a lot of time. But I have not heard
> anyone in this email thread (or anywhere for that matter) say that
> they tried a port in earnest and it turned out to be difficult.

One obvious example is the C extensions for the interpreter itself.
The code for those C extensions is actually almost the same from 2.x to 3.x. The
two main changes are the module initialization sequence (see PEP 3121) and the
bytes->unicode change for special methods such as tp_repr.

Regards

Antoine.



From p.f.moore at gmail.com  Tue Nov  3 23:16:51 2009
From: p.f.moore at gmail.com (Paul Moore)
Date: Tue, 3 Nov 2009 22:16:51 +0000
Subject: [Python-Dev] 2.7 Release? 2.7 == last of the 2.x line?
In-Reply-To: <bbaeab100911031153h61c356dfw850c53d4662aca7@mail.gmail.com>
References: <E077843B-919A-4AC7-95D7-0313E4D0328A@gmail.com>
	<F245110A-2FDF-460B-8E0D-87785F0E6DB0@fuhm.net>
	<BBB79ADA-7E31-4DD8-AAA5-07537FA74C5A@gmail.com>
	<D99BA4CF-E16C-47EB-9C60-F1EC5C27CDDC@python.org>
	<4AEFC887.6020801@v.loewis.de>
	<3DE1D8FB-896E-441D-937D-AB63EB3A8E22@python.org>
	<5b8d13220911030540s3d700265ib00f4277b7ce4a2b@mail.gmail.com>
	<bbaeab100911031153h61c356dfw850c53d4662aca7@mail.gmail.com>
Message-ID: <79990c6b0911031416r36c6947ldd19984108d46f7e@mail.gmail.com>

2009/11/3 Brett Cannon <brett at python.org>:
> I'm afraid there is some FUD going around here, which is
> understandable since no one wants to burn a ton of time on something
> that will be difficult or take a lot of time. But I have not heard
> anyone in this email thread (or anywhere for that matter) say that
> they tried a port in earnest and it turned out to be difficult.

Sadly, what I've heard a lot of is people (projects) saying that they
*won't* try a port (yet, for the forseeable future, take your pick)
because they *expect* it to be difficult. The worst thing is that even
Martin's (and probably others') efforts in actually producing ports,
and demonstrating that it's not difficult, don't seem to be changing
minds.

FWIW, I did a quick survey of some packages (a sampling of packages
I've used or considered using in the past):

Twisted - no plans yet for Python 3
wxPython - no mention of Python 3
numpy - no plans yet for Python 3
pyQt - supports Python 3
cx_Oracle - supports Python 3
pywin32 - supports Python 3
pygame - python 3 support "mostly completed"
Django - not yet, FAQ suggests it will be "a year or two"
TurboGears - Python 3 "currently unsupported", no timescale given
PIL - 1.1.7 (due very soon) supports Python 3
lxml - supports Python 3
pyCrypto - doesn't appear to support Python 3 yet
gmpy - 1.10 beta supports Python 3
pyYaml - supports Python 3
mod_wsgi - 3.0 RC 5 supports Python 3 (but see below)

Here, "supports Python 3" either means that explicit support is
mentioned on the website, or Windows binaries exist. I guess there's
also some pure python code that "might work", possibly only requiring
testing to confirm this.

That's a lot better picture than I expected. How come this message
isn't getting across? I suspect the big answer is that there's no web
framework (that I'm aware of) which works with Python 3 - mod_wsgi
supports Python 3, but from what I've seen on web-sig, the WSGI
picture for Python 3 is unclear, to say the least.

It seems to me that Python 3 adoption is pretty healthy, at least in
these terms. If a credible Python 3 web development framework appeared
(ie, one of the "big ones" got a port done) things would start to be
in pretty good shape.

Paul.

From solipsis at pitrou.net  Tue Nov  3 23:23:55 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Tue, 3 Nov 2009 22:23:55 +0000 (UTC)
Subject: [Python-Dev] 2.7 Release? 2.7 == last of the 2.x line?
References: <E077843B-919A-4AC7-95D7-0313E4D0328A@gmail.com>
	<F245110A-2FDF-460B-8E0D-87785F0E6DB0@fuhm.net>
	<BBB79ADA-7E31-4DD8-AAA5-07537FA74C5A@gmail.com>
	<D99BA4CF-E16C-47EB-9C60-F1EC5C27CDDC@python.org>
	<4AEFC887.6020801@v.loewis.de>
	<3DE1D8FB-896E-441D-937D-AB63EB3A8E22@python.org>
	<5b8d13220911030540s3d700265ib00f4277b7ce4a2b@mail.gmail.com>
	<bbaeab100911031153h61c356dfw850c53d4662aca7@mail.gmail.com>
	<79990c6b0911031416r36c6947ldd19984108d46f7e@mail.gmail.com>
Message-ID: <loom.20091103T232112-660@post.gmane.org>

Paul Moore <p.f.moore <at> gmail.com> writes:
> 
> FWIW, I did a quick survey of some packages (a sampling of packages
> I've used or considered using in the past):
> 
> Twisted - no plans yet for Python 3

Well Twisted depends on zope.interface which is not ported yet.
Twisted apparently has plans to reduce or remove the warnings generated with the
"-3" option, we'll see if the patches get committed:
http://twistedmatrix.com/trac/ticket/2484
http://twistedmatrix.com/trac/ticket/4053
http://twistedmatrix.com/trac/ticket/4065

> TurboGears - Python 3 "currently unsupported", no timescale given

TurboGears is Pylons-based, so I suppose the actual question is when Pylons gets
ported.

Regards

Antoine.



From skippy.hammond at gmail.com  Tue Nov  3 23:39:38 2009
From: skippy.hammond at gmail.com (Mark Hammond)
Date: Wed, 04 Nov 2009 09:39:38 +1100
Subject: [Python-Dev] PEP 3003 - Python Language Moratorium
In-Reply-To: <ca471dc20911030935l25251e0diec64427f5d7bcbf2@mail.gmail.com>
References: <ca471dc20911030935l25251e0diec64427f5d7bcbf2@mail.gmail.com>
Message-ID: <4AF0B12A.9000406@gmail.com>

On 4/11/2009 4:35 AM, Guido van Rossum wrote:
> I've checked draft (!) PEP 3003, "Python Language Moratorium", into
> SVN. As authors I've listed Jesse, Brett and myself.

Good move, +1.

Cheers,

Mark

From steve at pearwood.info  Tue Nov  3 23:46:15 2009
From: steve at pearwood.info (Steven D'Aprano)
Date: Wed, 4 Nov 2009 09:46:15 +1100
Subject: [Python-Dev] Retrieve an arbitrary element from a set
	withoutremoving it
In-Reply-To: <4AEF5AFA.5060806@canterbury.ac.nz>
References: <7b0045ed0910301843y7059d316s8a851142b78b7aeb@mail.gmail.com>
	<20091102052900.GA25160@cskk.homeip.net>
	<4AEF5AFA.5060806@canterbury.ac.nz>
Message-ID: <200911040946.15805.steve@pearwood.info>

On Tue, 3 Nov 2009 09:19:38 am Greg Ewing wrote:
> Cameron Simpson wrote:
> > Personally, I'm for the iteration spec in a lot of ways.
> >
> > Firstly, a .get()/.pick() that always returns the same element
> > feels horrible. Is there anyone here who _likes_ it?
>
> It doesn't sound very useful to me. However, an iterating
> version of it doesn't sound any more useful. What would it
> gain you? Why not just iterate over the set? Or make a
> copy and repeatedly pop() it?

Since I was the person who decided that "arbitrary" meant "give a 
different result each time", I should answer that.

My intention was not that people should use repeated calls to pick() for 
iteration. I described that as an abuse of the method. But I can 
imagine scenarios where the caller calls set.pick() sequentially:

def pick_two_cards(hand):
    assert isinstance(hand, (set, frozenset))
    assert len(hand) == 5
    return (hand.pick(), hand.pick())


As I said in a reply to Raymond, optimization was not my primary 
concern, but as a fundamental set operation[1] pick() should be 
efficient. It may be called on a set with millions of items, not just 
small sets. Copying to another set, or to a list, will be O(N) and 
potentially slow -- at the very least, it is wasteful to copy millions 
of elements in order to access one.

I don't know how expensive it is to create a set iterator, but (my 
reasoning went) surely we can do better? The set already has access to 
its own elements, it just needs to return one of them. pop() also knows 
how to retrieve an arbitrary element. pick() is just like pop(), except 
without removal.


> I completely fail to see a use case for this.

Nevertheless, people keep requesting it, so obviously they have a use 
for it. None of the standard solutions are obvious or easily 
discoverable, and in my experience the usual solution people come up 
with is to pop() an element, then add() it back in, but of course 
that's not just inelegant but it fails on frozensets.



[1] Obviously there is disagreement on whether or not pick() is a 
fundamental operation or not. As Raymond points out, it is uncommon in 
other languages. But Wikipedia lists it as fundamental, and it is not 
just Python users who ask for it:

http://stackoverflow.com/questions/124671/picking-a-random-element-from-a-set


-- 
Steven D'Aprano

From jackdied at gmail.com  Wed Nov  4 00:07:39 2009
From: jackdied at gmail.com (Jack Diederich)
Date: Tue, 3 Nov 2009 18:07:39 -0500
Subject: [Python-Dev] PEP 3003 - Python Language Moratorium
In-Reply-To: <ca471dc20911030935l25251e0diec64427f5d7bcbf2@mail.gmail.com>
References: <ca471dc20911030935l25251e0diec64427f5d7bcbf2@mail.gmail.com>
Message-ID: <b8e622740911031507s73daad8bw88628a16bc660b81@mail.gmail.com>

+1.  There are no compelling language changes on the horizon ("yield
from" is nice but not necessary).  I see the main benefit of a
moratorium as social rather than technical by encouraging people to
work on the lib instead of the language.  Plus, I'd gladly proxy my
vote to any one of the three PEP authors so 3/3 is a no-brainer.

-Jack

From ssteinerx at gmail.com  Wed Nov  4 00:10:00 2009
From: ssteinerx at gmail.com (ssteinerX@gmail.com)
Date: Tue, 3 Nov 2009 18:10:00 -0500
Subject: [Python-Dev] 2.7 Release? 2.7 == last of the 2.x line?
In-Reply-To: <hcpthu$9ql$2@ger.gmane.org>
References: <E077843B-919A-4AC7-95D7-0313E4D0328A@gmail.com>	<4AEFD9AE.7090104@molden.no>
	<4AEFF450.9060007@voidspace.org.uk>	<79EADA413EFB44F6864EF71D5B830088@RaymondLaptop1>	<16F0B6B0-677F-464C-9C89-CF92D3713A65@gmail.com>	<116885F3-819E-4B7C-86F7-8E5B246590EB@fuhm.net>	<bad82a80911030928x1f594a92yc315f2ae8b8246d9@mail.gmail.com>
	<3FE75D51-199C-4221-9B38-94FD2FE9BC4E@gmail.com>
	<hcpthu$9ql$2@ger.gmane.org>
Message-ID: <BD8FECB6-3C68-413A-968E-0018CCE0F3CF@gmail.com>


On Nov 3, 2009, at 1:42 PM, Georg Brandl wrote:

> ssteinerX at gmail.com schrieb:
>>
>> On Nov 3, 2009, at 12:28 PM, Arc Riley wrote:
>>>
>>> The main thing holding back the community are lazy and/or obstinate
>>> package maintainers.  If they spent half the time they've put into
>>> complaining about Py3 into actually working to upgrade their code
>>> they'd be done now.
>>
>> That's an inflammatory, defamatory, unsubstantiated, hyperbolic,
>> sweeping overgeneralization.
>
> I know a few maintainers, and I have no problem seeing how Arc came
> to that conclusion.

Yah, me neither, but the other two are still cool in my book ;-)

S


From guido at python.org  Wed Nov  4 00:10:30 2009
From: guido at python.org (Guido van Rossum)
Date: Tue, 3 Nov 2009 15:10:30 -0800
Subject: [Python-Dev] Retrieve an arbitrary element from a set
	withoutremoving it
In-Reply-To: <200911040946.15805.steve@pearwood.info>
References: <7b0045ed0910301843y7059d316s8a851142b78b7aeb@mail.gmail.com> 
	<20091102052900.GA25160@cskk.homeip.net>
	<4AEF5AFA.5060806@canterbury.ac.nz> 
	<200911040946.15805.steve@pearwood.info>
Message-ID: <ca471dc20911031510j3cd6f7fek76894942518f8726@mail.gmail.com>

On Tue, Nov 3, 2009 at 2:46 PM, Steven D'Aprano <steve at pearwood.info> wrote:
> On Tue, 3 Nov 2009 09:19:38 am Greg Ewing wrote:
>> Cameron Simpson wrote:
>> > Personally, I'm for the iteration spec in a lot of ways.
>> >
>> > Firstly, a .get()/.pick() that always returns the same element
>> > feels horrible. Is there anyone here who _likes_ it?
>>
>> It doesn't sound very useful to me. However, an iterating
>> version of it doesn't sound any more useful. What would it
>> gain you? Why not just iterate over the set? Or make a
>> copy and repeatedly pop() it?
>
> Since I was the person who decided that "arbitrary" meant "give a
> different result each time", I should answer that.
>
> My intention was not that people should use repeated calls to pick() for
> iteration. I described that as an abuse of the method. But I can
> imagine scenarios where the caller calls set.pick() sequentially:
>
> def pick_two_cards(hand):
> ? ?assert isinstance(hand, (set, frozenset))
> ? ?assert len(hand) == 5
> ? ?return (hand.pick(), hand.pick())
>
>
> As I said in a reply to Raymond, optimization was not my primary
> concern, but as a fundamental set operation[1] pick() should be
> efficient. It may be called on a set with millions of items, not just
> small sets. Copying to another set, or to a list, will be O(N) and
> potentially slow -- at the very least, it is wasteful to copy millions
> of elements in order to access one.
>
> I don't know how expensive it is to create a set iterator, but (my
> reasoning went) surely we can do better? The set already has access to
> its own elements, it just needs to return one of them. pop() also knows
> how to retrieve an arbitrary element. pick() is just like pop(), except
> without removal.
>
>
>> I completely fail to see a use case for this.
>
> Nevertheless, people keep requesting it, so obviously they have a use
> for it. None of the standard solutions are obvious or easily
> discoverable, and in my experience the usual solution people come up
> with is to pop() an element, then add() it back in, but of course
> that's not just inelegant but it fails on frozensets.
>
>
>
> [1] Obviously there is disagreement on whether or not pick() is a
> fundamental operation or not. As Raymond points out, it is uncommon in
> other languages. But Wikipedia lists it as fundamental, and it is not
> just Python users who ask for it:
>
> http://stackoverflow.com/questions/124671/picking-a-random-element-from-a-set

You're obviously talking about a *random* element. This is a separate
use case (though I agree many people don't know the difference).

Picking a random element can be done in O(1) only if the data
structure supports access by index, which Python's hash tables don't.

-- 
--Guido van Rossum (python.org/~guido)

From daniel at stutzbachenterprises.com  Wed Nov  4 00:11:47 2009
From: daniel at stutzbachenterprises.com (Daniel Stutzbach)
Date: Tue, 3 Nov 2009 17:11:47 -0600
Subject: [Python-Dev] Retrieve an arbitrary element from a set
	withoutremoving it
In-Reply-To: <200911040946.15805.steve@pearwood.info>
References: <7b0045ed0910301843y7059d316s8a851142b78b7aeb@mail.gmail.com>
	<20091102052900.GA25160@cskk.homeip.net>
	<4AEF5AFA.5060806@canterbury.ac.nz>
	<200911040946.15805.steve@pearwood.info>
Message-ID: <eae285400911031511q140383d5vb5fdf6327254a279@mail.gmail.com>

On Tue, Nov 3, 2009 at 4:46 PM, Steven D'Aprano <steve at pearwood.info> wrote:

> def pick_two_cards(hand):
>    assert isinstance(hand, (set, frozenset))
>    assert len(hand) == 5
>    return (hand.pick(), hand.pick())
>

Even if pick() chose random, you still might end up picking the same card
twice.  Is that really what you intended?

FWIW, I've been working on an extension module that supplies a "sortedset"
type [1].  In most ways, it's similar to a set except it's indexable like a
list.  The items are kept in sorted order, so index 0 is always the lowest
item, index 1 is the next-to-lowest, etc.  Because they're indexable, it's
easy and efficient to retrieve random elements using the standard library's
"random" module.

With the sortedset type, that function would become:

def pick_two_cards(hand):
    assert isinstance(hand, (set, frozenset))
    assert len(hand) == 5
    return random.choice(hand), random.choice(hand)

or if you want to avoid duplicates:

    return random.sample(hand, 2)


Would something like that fit your needs?


[1] It's already implemented, along with sortedlist, weaksortedlist, and
weaksortedset types.  I'm just putting them through the paces in my own
products before releasing them.

--
Daniel Stutzbach, Ph.D.
President, Stutzbach Enterprises, LLC <http://stutzbachenterprises.com>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20091103/693532ba/attachment.htm>

From solipsis at pitrou.net  Wed Nov  4 00:21:56 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Tue, 3 Nov 2009 23:21:56 +0000 (UTC)
Subject: [Python-Dev]
	=?utf-8?q?Retrieve_an_arbitrary_element_from_a_set?=
	=?utf-8?q?=09withoutremoving_it?=
References: <7b0045ed0910301843y7059d316s8a851142b78b7aeb@mail.gmail.com>
	<20091102052900.GA25160@cskk.homeip.net>
	<4AEF5AFA.5060806@canterbury.ac.nz>
	<200911040946.15805.steve@pearwood.info>
	<ca471dc20911031510j3cd6f7fek76894942518f8726@mail.gmail.com>
Message-ID: <loom.20091104T002022-633@post.gmane.org>

Guido van Rossum <guido <at> python.org> writes:
> 
> You're obviously talking about a *random* element. This is a separate
> use case (though I agree many people don't know the difference).
> 
> Picking a random element can be done in O(1) only if the data
> structure supports access by index, which Python's hash tables don't.

Well, at the implementation level, they can. You'd just have to pick a new
random index until it points to a non-empty slot.

Regards

Antoine.



From amauryfa at gmail.com  Wed Nov  4 00:57:32 2009
From: amauryfa at gmail.com (Amaury Forgeot d'Arc)
Date: Wed, 4 Nov 2009 00:57:32 +0100
Subject: [Python-Dev] No buildbot to test wide unicode?
Message-ID: <e27efe130911031557n7839ab25g28fc6dafe3174ab7@mail.gmail.com>

Hello,

It seems that there is no buildbot to test a wide unicode build.

On http://www.python.org/dev/buildbot/3.x/ all outputs of the "configure"
steps show this message::
  checking what type to use for str... unsigned short
which looks like a ucs2 build to me.

Since wide unicode is the standard chosen by some Linux distributions,
it would make sense to have at least one buildbot running with
--with-wide-unicode (3.x) or --enable-unicode=ucs4 (2.x).

-- 
Amaury Forgeot d'Arc

From cournape at gmail.com  Wed Nov  4 01:06:07 2009
From: cournape at gmail.com (David Cournapeau)
Date: Wed, 4 Nov 2009 09:06:07 +0900
Subject: [Python-Dev] 2.7 Release? 2.7 == last of the 2.x line?
In-Reply-To: <4AF0758A.6070108@v.loewis.de>
References: <E077843B-919A-4AC7-95D7-0313E4D0328A@gmail.com>
	<4AEFD9AE.7090104@molden.no>
	<DF3EB3B9-EB18-42C3-BE34-AADC80006DC4@gmail.com>
	<4AF06A5F.6090501@v.loewis.de> <4AF072BE.2010603@trueblade.com>
	<4AF0758A.6070108@v.loewis.de>
Message-ID: <5b8d13220911031606g7a1c40d9jbfa51137661a6d4@mail.gmail.com>

On Wed, Nov 4, 2009 at 3:25 AM, "Martin v. L?wis" <martin at v.loewis.de> wrote:

> But only if NumPy would drop support for 2.x, for x < 7, right?
> That would probably be many years in the future.

Yes. Today, given the choice of supporting py 3.x and dropping python
< 2.7 and continue support for 2.4, the latter is by far my preferred
choice today (RHEL still require 2.4, for example).

cheers,

David

From amcnabb at mcnabbs.org  Tue Nov  3 22:28:43 2009
From: amcnabb at mcnabbs.org (Andrew McNabb)
Date: Tue, 3 Nov 2009 14:28:43 -0700
Subject: [Python-Dev] 2.7 Release? 2.7 == last of the 2.x line?
In-Reply-To: <bad82a80911030928x1f594a92yc315f2ae8b8246d9@mail.gmail.com>
References: <E077843B-919A-4AC7-95D7-0313E4D0328A@gmail.com>
	<4AEFD9AE.7090104@molden.no> <4AEFF450.9060007@voidspace.org.uk>
	<79EADA413EFB44F6864EF71D5B830088@RaymondLaptop1>
	<16F0B6B0-677F-464C-9C89-CF92D3713A65@gmail.com>
	<116885F3-819E-4B7C-86F7-8E5B246590EB@fuhm.net>
	<bad82a80911030928x1f594a92yc315f2ae8b8246d9@mail.gmail.com>
Message-ID: <20091103212843.GA2444@mcnabbs.org>

On Tue, Nov 03, 2009 at 12:28:47PM -0500, Arc Riley wrote:
> 
> The main thing holding back the community are lazy and/or obstinate package
> maintainers.  If they spent half the time they've put into complaining about
> Py3 into actually working to upgrade their code they'd be done now.

The main reason the package maintainers are so "lazy and/or obstinate"
is that Python 2 is much more available than Python 3.  For example,
work is being completed on a python3 RPM, but it will first appear in
Fedora 13, not Fedora 12.  RHEL is still using Python 2.4 and won't even
get Python 2.6 until RHEL 6 comes out.  I don't think it's worth
worrying about packages being upgraded to Python 3 when many or most
users still don't have access to Python 3.

In my opinion, the best place for current efforts is on infrastructure:

1) Python 3 packages for all Linux distributions (I've recently been
involved with bringing Python 3 to Fedora)

2) WSGI/mod_python, etc.: according to
http://wsgi.org/wsgi/Amendments_1.0 and various blog posts, the WSGI
standard isn't defined for Python 3 yet, and the modwsgi page at
http://code.google.com/p/modwsgi/ does not yet list a Python 3 release.

3) Numpy/scipy: as others have mentioned, numpy is essential for
scientific work, and apparently the project needs help to add support
for Python 3

Before basic infrastructure is available for Python 3, it's absurd to
expect package maintainers to flock to it.

-- 
Andrew McNabb
http://www.mcnabbs.org/andrew/
PGP Fingerprint: 8A17 B57C 6879 1863 DE55  8012 AB4D 6098 8826 6868

From bugbee at seanet.com  Wed Nov  4 00:27:43 2009
From: bugbee at seanet.com (Larry Bugbee)
Date: Tue, 3 Nov 2009 15:27:43 -0800
Subject: [Python-Dev] 2.7 Release? 2.7 == last of the 2.x line?
In-Reply-To: <261064E43172924BBADB00DA14BB4B672A14855D01@XCH-NW-14V.nw.nos.boeing.com>
References: <261064E43172924BBADB00DA14BB4B672A14855D01@XCH-NW-14V.nw.nos.boeing.com>
Message-ID: <93A0C3CE-3388-49B9-B994-8AB84E6F48B8@seanet.com>

This is my problem to solve...  I work with a lot of [non-production]  
crypto where byte strings are a normal way of manipulating and storing  
keys and other artifacts.  Under Python2 I have few/no Unicode  
issues.  With Python3's default string type being Unicode, there are a  
lot of subtle ways where my codes go south.  Like I say, it is my  
problem.

I will convert to Py3, but not likely before summer.  Besides 2to3,  
any other tools/suggestions would be welcome.  (Perhaps there is a  
Master switch to disable Unicode? ;-)

Changing the subject...  I say -1 on backporting more 3.x features to  
2.6/2.7.  As previously mentioned, leave the carrot.  And best  
intentions aside, backports can introduce problems of their own.




From ben+python at benfinney.id.au  Wed Nov  4 01:42:29 2009
From: ben+python at benfinney.id.au (Ben Finney)
Date: Wed, 04 Nov 2009 11:42:29 +1100
Subject: [Python-Dev] 2.7 Release? 2.7 == last of the 2.x line?
References: <E077843B-919A-4AC7-95D7-0313E4D0328A@gmail.com>
	<F245110A-2FDF-460B-8E0D-87785F0E6DB0@fuhm.net>
	<BBB79ADA-7E31-4DD8-AAA5-07537FA74C5A@gmail.com>
	<D99BA4CF-E16C-47EB-9C60-F1EC5C27CDDC@python.org>
	<4AEFC887.6020801@v.loewis.de>
	<3DE1D8FB-896E-441D-937D-AB63EB3A8E22@python.org>
	<5b8d13220911030540s3d700265ib00f4277b7ce4a2b@mail.gmail.com>
	<bbaeab100911031153h61c356dfw850c53d4662aca7@mail.gmail.com>
	<79990c6b0911031416r36c6947ldd19984108d46f7e@mail.gmail.com>
	<loom.20091103T232112-660@post.gmane.org>
Message-ID: <87y6mnf7d6.fsf@benfinney.id.au>

Antoine Pitrou <solipsis at pitrou.net> writes:

> Paul Moore <p.f.moore <at> gmail.com> writes:
> > TurboGears - Python 3 "currently unsupported", no timescale given
>
> TurboGears is Pylons-based, so I suppose the actual question is when
> Pylons gets ported.

And there's the rub. I expect this general problem of ?project FOO
depends on library BAR, so there's no point thinking about migration of
FOO until BAR is ready for Python 3? will prove to be quite common in
widespread projects.

-- 
 \           ?If you do not trust the source do not use this program.? |
  `\                                ?Microsoft Vista security dialogue |
_o__)                                                                  |
Ben Finney


From ssteinerx at gmail.com  Wed Nov  4 01:50:36 2009
From: ssteinerx at gmail.com (ssteinerX@gmail.com)
Date: Tue, 3 Nov 2009 19:50:36 -0500
Subject: [Python-Dev] 2.7 Release? 2.7 == last of the 2.x line?
In-Reply-To: <87y6mnf7d6.fsf@benfinney.id.au>
References: <E077843B-919A-4AC7-95D7-0313E4D0328A@gmail.com>
	<F245110A-2FDF-460B-8E0D-87785F0E6DB0@fuhm.net>
	<BBB79ADA-7E31-4DD8-AAA5-07537FA74C5A@gmail.com>
	<D99BA4CF-E16C-47EB-9C60-F1EC5C27CDDC@python.org>
	<4AEFC887.6020801@v.loewis.de>
	<3DE1D8FB-896E-441D-937D-AB63EB3A8E22@python.org>
	<5b8d13220911030540s3d700265ib00f4277b7ce4a2b@mail.gmail.com>
	<bbaeab100911031153h61c356dfw850c53d4662aca7@mail.gmail.com>
	<79990c6b0911031416r36c6947ldd19984108d46f7e@mail.gmail.com>
	<loom.20091103T232112-660@post.gmane.org>
	<87y6mnf7d6.fsf@benfinney.id.au>
Message-ID: <A6B292E4-14DB-4F76-9E95-65F16F73E418@gmail.com>


On Nov 3, 2009, at 7:42 PM, Ben Finney wrote:

> Antoine Pitrou <solipsis at pitrou.net> writes:
>
>> Paul Moore <p.f.moore <at> gmail.com> writes:
>>> TurboGears - Python 3 "currently unsupported", no timescale given
>>
>> TurboGears is Pylons-based, so I suppose the actual question is when
>> Pylons gets ported.
>
> And there's the rub. I expect this general problem of ?project FOO
> depends on library BAR, so there's no point thinking about migration  
> of
> FOO until BAR is ready for Python 3? will prove to be quite common in
> widespread projects.

There's no "will prove", it is.

S


From greg.ewing at canterbury.ac.nz  Wed Nov  4 01:54:47 2009
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Wed, 04 Nov 2009 13:54:47 +1300
Subject: [Python-Dev] Retrieve an arbitrary element from a set
 withoutremoving it
In-Reply-To: <200911040946.15805.steve@pearwood.info>
References: <7b0045ed0910301843y7059d316s8a851142b78b7aeb@mail.gmail.com>
	<20091102052900.GA25160@cskk.homeip.net>
	<4AEF5AFA.5060806@canterbury.ac.nz>
	<200911040946.15805.steve@pearwood.info>
Message-ID: <4AF0D0D7.3090403@canterbury.ac.nz>

Steven D'Aprano wrote:

> I don't know how expensive it is to create a set iterator,

Not expensive enough to justify burdening the set type with
extra functionality that will be extremely rarely used.

-- 
Greg

From greg.ewing at canterbury.ac.nz  Wed Nov  4 02:00:12 2009
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Wed, 04 Nov 2009 14:00:12 +1300
Subject: [Python-Dev] Retrieve an arbitrary element from a set
 withoutremoving it
In-Reply-To: <loom.20091104T002022-633@post.gmane.org>
References: <7b0045ed0910301843y7059d316s8a851142b78b7aeb@mail.gmail.com>
	<20091102052900.GA25160@cskk.homeip.net>
	<4AEF5AFA.5060806@canterbury.ac.nz>
	<200911040946.15805.steve@pearwood.info>
	<ca471dc20911031510j3cd6f7fek76894942518f8726@mail.gmail.com>
	<loom.20091104T002022-633@post.gmane.org>
Message-ID: <4AF0D21C.8040909@canterbury.ac.nz>

Antoine Pitrou wrote:
> Guido van Rossum <guido <at> python.org> writes:
> 
>>Picking a random element can be done in O(1) only if the data
>>structure supports access by index, which Python's hash tables don't.
> 
> Well, at the implementation level, they can. You'd just have to pick a new
> random index until it points to a non-empty slot.

But that's hardly O(1).

-- 
Greg

From solipsis at pitrou.net  Wed Nov  4 02:10:58 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Wed, 4 Nov 2009 01:10:58 +0000 (UTC)
Subject: [Python-Dev] Retrieve an arbitrary element from a set
	withoutremoving it
References: <7b0045ed0910301843y7059d316s8a851142b78b7aeb@mail.gmail.com>
	<20091102052900.GA25160@cskk.homeip.net>
	<4AEF5AFA.5060806@canterbury.ac.nz>
	<200911040946.15805.steve@pearwood.info>
	<ca471dc20911031510j3cd6f7fek76894942518f8726@mail.gmail.com>
	<loom.20091104T002022-633@post.gmane.org>
	<4AF0D21C.8040909@canterbury.ac.nz>
Message-ID: <loom.20091104T020819-105@post.gmane.org>

Greg Ewing <greg.ewing <at> canterbury.ac.nz> writes:
> > 
> >>Picking a random element can be done in O(1) only if the data
> >>structure supports access by index, which Python's hash tables don't.
> > 
> > Well, at the implementation level, they can. You'd just have to pick a new
> > random index until it points to a non-empty slot.
> 
> But that's hardly O(1).

It is, assuming that the set has a built-in minimum fill level (e.g. it always
keeps at least x% of its entries filled), and the random generator is good.

(of course, it is "statistically O(1)", like lookups in a hash table actually)

Regards

Antoine.



From stephen at xemacs.org  Wed Nov  4 02:39:23 2009
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Wed, 04 Nov 2009 10:39:23 +0900
Subject: [Python-Dev] PEP 3003 - Python Language Moratorium
In-Reply-To: <ca471dc20911031203i2dd83f7ft32f4487cd3edacb4@mail.gmail.com>
References: <ca471dc20911030935l25251e0diec64427f5d7bcbf2@mail.gmail.com>
	<4AF08345.6040603@egenix.com>
	<ca471dc20911031203i2dd83f7ft32f4487cd3edacb4@mail.gmail.com>
Message-ID: <87tyxb131w.fsf@uwakimon.sk.tsukuba.ac.jp>

Guido van Rossum writes:
 > On Tue, Nov 3, 2009 at 11:23 AM, M.-A. Lemburg <mal at egenix.com> wrote:

 > > One question:
 > >
 > > There are currently number of patch waiting on the tracker for
 > > additional Unicode feature support and it's also likely that we'll
 > > want to upgrade to a more recent Unicode version within the
 > > next few years.
 > >
 > > How would such indirect changes be seen under the moratorium ?
 > 
 > That would fall under the Case-by-Case Exemptions section. "Within the
 > next few years" sounds like it might well wait until the moratorium is
 > ended though. :-)

(1) Unicode "feature support" (eg, implementing access to various
    databases, implementation of new algorithms) strikes me as worth a
    hearing but very dubious under case-by-case (the case I have in
    mind is when necessary to support (2) or (3)).

(2) Adjusting existing implementations of Unicode features (eg,
    universal newline, which currently does not conform to the
    relevant TR AFAIK, thus dubious IMO) to conform to (new versions
    of) Unicode seems worthy of consideration on a cost/benefit basis.
    I would expect that in most cases the costs that lead to a
    moratorium would outweigh prospective benefits, though.  I don't
    know of an example where I'd support an exception to the
    moratorium for this, though.  It's hard to imagine that things
    like the algorithm for UTF-16 are going to change, and support for
    things like bidi are currently outside of the scope of the
    moratorium as I understand it -- they'd be stdlib facilities.

(3) Upgrading the character repertoire and property databases (eg,
    case and composition) seems like a shoo-in to me.  That's why the
    databases are versioned.  Of course there are costs to upgrading
    (backward compatibility in some corner cases, eg) that might
    outweigh the benefits at any given time, but I don't think this
    should fall under the moratorium if someone finds a benefit to
    upgrading the databases within the moratorium period.

From guido at python.org  Wed Nov  4 03:11:02 2009
From: guido at python.org (Guido van Rossum)
Date: Tue, 3 Nov 2009 18:11:02 -0800
Subject: [Python-Dev] Retrieve an arbitrary element from a set
	withoutremoving it
In-Reply-To: <loom.20091104T020819-105@post.gmane.org>
References: <7b0045ed0910301843y7059d316s8a851142b78b7aeb@mail.gmail.com> 
	<20091102052900.GA25160@cskk.homeip.net>
	<4AEF5AFA.5060806@canterbury.ac.nz> 
	<200911040946.15805.steve@pearwood.info>
	<ca471dc20911031510j3cd6f7fek76894942518f8726@mail.gmail.com> 
	<loom.20091104T002022-633@post.gmane.org>
	<4AF0D21C.8040909@canterbury.ac.nz> 
	<loom.20091104T020819-105@post.gmane.org>
Message-ID: <ca471dc20911031811m6abc75c8t5aa9b44fcc417ae9@mail.gmail.com>

On Tue, Nov 3, 2009 at 5:10 PM, Antoine Pitrou <solipsis at pitrou.net> wrote:
> Greg Ewing <greg.ewing <at> canterbury.ac.nz> writes:
>> >
>> >>Picking a random element can be done in O(1) only if the data
>> >>structure supports access by index, which Python's hash tables don't.
>> >
>> > Well, at the implementation level, they can. You'd just have to pick a new
>> > random index until it points to a non-empty slot.
>>
>> But that's hardly O(1).
>
> It is, assuming that the set has a built-in minimum fill level (e.g. it always
> keeps at least x% of its entries filled), and the random generator is good.
>
> (of course, it is "statistically O(1)", like lookups in a hash table actually)

Clever. But I don't think the set implementation should have a
dependency on random(), so it would have to expose an "indexing"
operation, which smells like it would expose too much of the
implementation for comfort.

-- 
--Guido van Rossum (python.org/~guido)

From tjreedy at udel.edu  Wed Nov  4 03:26:25 2009
From: tjreedy at udel.edu (Terry Reedy)
Date: Tue, 03 Nov 2009 21:26:25 -0500
Subject: [Python-Dev] 2.7 Release? 2.7 == last of the 2.x line?
In-Reply-To: <ca471dc20911031039o4931d7c7xc5be220c100c33e5@mail.gmail.com>
References: <E077843B-919A-4AC7-95D7-0313E4D0328A@gmail.com>
	<4AEFD9AE.7090104@molden.no>	<DF3EB3B9-EB18-42C3-BE34-AADC80006DC4@gmail.com>
	<4AF06A5F.6090501@v.loewis.de>
	<ca471dc20911031039o4931d7c7xc5be220c100c33e5@mail.gmail.com>
Message-ID: <hcqoof$2uj$1@ger.gmane.org>

Guido van Rossum wrote:
> On Tue, Nov 3, 2009 at 9:37 AM, "Martin v. L?wis" <martin at v.loewis.de> wrote:
>> (and no, adding things like nonlocal to 2.7 doesn't making porting of
>> a real application or library any easier, since the existing application
>> or library simply doesn't use that keyword.
> 
> Agreed.
> 
>> In fact, no change to 2.x
>> can reasonably simplify porting - only changes to 3.x might - except
>> for changes to 2to3, which can simplify porting a lot. But 2to3 should
>> be run under 3.x, IMO.)
> 
> Disagreed. Better "-3" warnings could make porting easier. (Not just
> more warnings -- "better" might mean fewer false positives for
> warnings already issued.)

There is also Eric Smith's list to consider: PEP3118 new buffer 
protocol, short float repr, and maybe io.

> FWIW, it doesn't sound like killing 2.7 is a productive thing to do.
> However making 2.7 the end of the line (though with indefinite bugfix
> releases) might be. (Indefinite != infinite.)

I think you should decide and announce something like the following:
'''
Python 2.7 will be the final, stable release in the Python 2 line.
It will be released in mid-2010 with the first alpha scheduled for 
December 2009.
It will not intentionally break valid older 2.x code; this means no 
removals. (Valid == not exploiting a bug.)
Being the last of its line, there will be no deprecation warnings unless 
explicitly requested with the -3 flag to warn about incompatibilities 
with 3.x.
There will be lots of bug fixes since 2.6.
There will be only a few new features, with those aimed at easing 
eventual transition of libraries to 3.x.
The period of 2.7.z bugfix releases should be longer than for previous 
x.y releases (as long as there are volunteers to write and review 
patches and prepare distributions).

The developers urge people with 2.x code, especially library 
maintainers, to test it with preliminary alpha or beta releases so 2.7 
can be as good as possible.

The developers hope OS distributions can move to including 2.7 as soon 
as feasible, even if it means 'skipping' 2.6 as a default version.
'''
[My intention with the last is to promote 2.7 as the definitive version 
of 2.x, though there might be better wording.]

Terry Jan Reedy



From guido at python.org  Wed Nov  4 04:11:47 2009
From: guido at python.org (Guido van Rossum)
Date: Tue, 3 Nov 2009 19:11:47 -0800
Subject: [Python-Dev] 2.7 Release? 2.7 == last of the 2.x line?
In-Reply-To: <93A0C3CE-3388-49B9-B994-8AB84E6F48B8@seanet.com>
References: <261064E43172924BBADB00DA14BB4B672A14855D01@XCH-NW-14V.nw.nos.boeing.com>
	<93A0C3CE-3388-49B9-B994-8AB84E6F48B8@seanet.com>
Message-ID: <ca471dc20911031911l1757ba8cq58a7ee733cb9164e@mail.gmail.com>

On Tue, Nov 3, 2009 at 3:27 PM, Larry Bugbee <bugbee at seanet.com> wrote:
> This is my problem to solve... ?I work with a lot of [non-production] crypto
> where byte strings are a normal way of manipulating and storing keys and
> other artifacts. ?Under Python2 I have few/no Unicode issues. ?With
> Python3's default string type being Unicode, there are a lot of subtle ways
> where my codes go south. ?Like I say, it is my problem.

It may be interesting for others if you posted some examples of what
you are used to doing in 2.x. Maybe somebody will suggest a better way
of doing it in 3.x than you have thought of. :-)

> I will convert to Py3, but not likely before summer. ?Besides 2to3, any
> other tools/suggestions would be welcome. ?(Perhaps there is a Master switch
> to disable Unicode? ;-)

Yes, go back to 1.5.2. :-)

-- 
--Guido van Rossum (python.org/~guido)

From debatem1 at gmail.com  Wed Nov  4 04:46:41 2009
From: debatem1 at gmail.com (geremy condra)
Date: Tue, 3 Nov 2009 22:46:41 -0500
Subject: [Python-Dev] 2.7 Release? 2.7 == last of the 2.x line?
In-Reply-To: <93A0C3CE-3388-49B9-B994-8AB84E6F48B8@seanet.com>
References: <261064E43172924BBADB00DA14BB4B672A14855D01@XCH-NW-14V.nw.nos.boeing.com>
	<93A0C3CE-3388-49B9-B994-8AB84E6F48B8@seanet.com>
Message-ID: <f3cc57c60911031946k5c5e2594g81c8a1dac68298a4@mail.gmail.com>

On Tue, Nov 3, 2009 at 6:27 PM, Larry Bugbee <bugbee at seanet.com> wrote:
> This is my problem to solve... ?I work with a lot of [non-production] crypto
> where byte strings are a normal way of manipulating and storing keys and
> other artifacts. ?Under Python2 I have few/no Unicode issues. ?With
> Python3's default string type being Unicode, there are a lot of subtle ways
> where my codes go south. ?Like I say, it is my problem.
>

Interesting- I've been doing crypto in python 3 for quite some time and
haven't had any issues except with getting things to play nicely with
C. Any specific examples?

Geremy Condra

From steven.bethard at gmail.com  Wed Nov  4 05:57:27 2009
From: steven.bethard at gmail.com (Steven Bethard)
Date: Tue, 3 Nov 2009 20:57:27 -0800
Subject: [Python-Dev] 2.7 Release? 2.7 == last of the 2.x line?
In-Reply-To: <eae285400911030614tdac889fqb5e655027565a962@mail.gmail.com>
References: <E077843B-919A-4AC7-95D7-0313E4D0328A@gmail.com>
	<4AEFD9AE.7090104@molden.no> <4AEFF450.9060007@voidspace.org.uk>
	<5b8d13220911030155x57615540x1f59e4b95b6c2f56@mail.gmail.com>
	<eae285400911030614tdac889fqb5e655027565a962@mail.gmail.com>
Message-ID: <d11dcfba0911032057r7d76c899i10afcc5a3d8dc6ff@mail.gmail.com>

On Tue, Nov 3, 2009 at 6:14 AM, Daniel Stutzbach
<daniel at stutzbachenterprises.com> wrote:
> It's pretty easy to make Python source that works under 2.6 and 3.x.? It's
> basically impossible to make Python source that works under 2.4/2.5 and
> 3.x.

This keeps getting quoted later in the thread so I just wanted to say
again that this is not true. The argparse module supports Python
2.3-3.1 all from the same codebase, and it really wasn't all that
hard:

  http://code.google.com/p/argparse/source/detail?r=12

Steve
-- 
Where did you get that preposterous hypothesis?
Did Steve tell you that?
        --- The Hiphopopotamus

From glyph at twistedmatrix.com  Wed Nov  4 06:20:18 2009
From: glyph at twistedmatrix.com (Glyph Lefkowitz)
Date: Wed, 4 Nov 2009 00:20:18 -0500
Subject: [Python-Dev] 2.7 Release? 2.7 == last of the 2.x line?
In-Reply-To: <79990c6b0911031416r36c6947ldd19984108d46f7e@mail.gmail.com>
References: <E077843B-919A-4AC7-95D7-0313E4D0328A@gmail.com>
	<F245110A-2FDF-460B-8E0D-87785F0E6DB0@fuhm.net>
	<BBB79ADA-7E31-4DD8-AAA5-07537FA74C5A@gmail.com>
	<D99BA4CF-E16C-47EB-9C60-F1EC5C27CDDC@python.org>
	<4AEFC887.6020801@v.loewis.de>
	<3DE1D8FB-896E-441D-937D-AB63EB3A8E22@python.org>
	<5b8d13220911030540s3d700265ib00f4277b7ce4a2b@mail.gmail.com>
	<bbaeab100911031153h61c356dfw850c53d4662aca7@mail.gmail.com>
	<79990c6b0911031416r36c6947ldd19984108d46f7e@mail.gmail.com>
Message-ID: <CCFA9232-8222-4714-8356-F6C501744077@twistedmatrix.com>

On Nov 3, 2009, at 5:16 PM, Paul Moore wrote:
> 2009/11/3 Brett Cannon <brett at python.org>:
>> I'm afraid there is some FUD going around here, which is
>> understandable since no one wants to burn a ton of time on something
>> that will be difficult or take a lot of time. But I have not heard
>> anyone in this email thread (or anywhere for that matter) say that
>> they tried a port in earnest and it turned out to be difficult.

> FWIW, I did a quick survey of some packages (a sampling of packages
> I've used or considered using in the past):
>
> Twisted - no plans yet for Python 3


Speaking of FUD, we've had a plan for Python 3 support for some time:

     http://twistedmatrix.com/trac/ticket/2484
     http://stackoverflow.com/questions/172306/how-are-you-planning-on-handling-the-migration-to-python-3/214601#214601

Not only that, but progress is actually being made on that plan, as it  
is being slowly executed by contributors from the community, a  
sampling of which you can see on these tickets, linked from the bottom  
of the "master plan" ticket I mentioned above (#2484):

     http://twistedmatrix.com/trac/ticket/4053
     http://twistedmatrix.com/trac/ticket/4065
     http://twistedmatrix.com/trac/ticket/4066

If you're interested in helping, our core team has all not had much  
time for Twisted lately and we need volunteers who are interested in  
doing code reviews and becoming a committer to help shepherd these  
tickets through the process.

From cs at zip.com.au  Wed Nov  4 06:26:40 2009
From: cs at zip.com.au (Cameron Simpson)
Date: Wed, 4 Nov 2009 16:26:40 +1100
Subject: [Python-Dev] Retrieve an arbitrary element from a set
	withoutremoving it
In-Reply-To: <200911040946.15805.steve@pearwood.info>
References: <200911040946.15805.steve@pearwood.info>
Message-ID: <20091104052640.GA15378@cskk.homeip.net>

On 04Nov2009 09:46, Steven D'Aprano <steve at pearwood.info> wrote:
| On Tue, 3 Nov 2009 09:19:38 am Greg Ewing wrote:
| > Cameron Simpson wrote:
| > > Personally, I'm for the iteration spec in a lot of ways.

For the record, I've since, in mere hours or day, been convinced my
preference was misguided.

I _do_ still feel that a hypothetical .pick() (a name I think better than
.get()) would be "nicer" if it didn't always return the same item. Not
that I've imagined any use case not equally well served by the existing
iterator facility :-(

Greg Ewing:
| > I completely fail to see a use case for this.

Steven D'Aprano:
| Nevertheless, people keep requesting it, so obviously they have a use 
| for it. None of the standard solutions are obvious or easily 
| discoverable, and in my experience the usual solution people come up 
| with is to pop() an element, then add() it back in, but of course 
| that's not just inelegant but it fails on frozensets.

Question: has anyone asked for .pick()/.get() with a use case _not_ well
served by the existing facilities?

Personally, I have found it useful in doco I write to have a section on
"Common Tasks", with recommended/suggested examples of how to do them and
short rationale for the chosen method. It seems to me that if .pick()
is frequently desired and "None of the standard solutions are obvious
or easily discoverable" then they should be _made_ so with documentation.

Comments?
-- 
Cameron Simpson <cs at zip.com.au> DoD#743
http://www.cskk.ezoshosting.com/cs/

Politics, n. Strife of interests masquerading as a contest of principles.
- Ambrose Bierce, _The_Devil's_Dictionary_

From glyph at twistedmatrix.com  Wed Nov  4 06:51:09 2009
From: glyph at twistedmatrix.com (Glyph Lefkowitz)
Date: Wed, 4 Nov 2009 00:51:09 -0500
Subject: [Python-Dev] 2.7 Release? 2.7 == last of the 2.x line?
In-Reply-To: <79990c6b0911030323x47a9f008ue7256c56e88c0ee8@mail.gmail.com>
References: <E077843B-919A-4AC7-95D7-0313E4D0328A@gmail.com>
	<4AEFD9AE.7090104@molden.no> <4AEFF450.9060007@voidspace.org.uk>
	<79EADA413EFB44F6864EF71D5B830088@RaymondLaptop1>
	<79990c6b0911030323x47a9f008ue7256c56e88c0ee8@mail.gmail.com>
Message-ID: <5CB59A21-D8C9-4BC1-A1F6-4438B86A8BC5@twistedmatrix.com>

On Nov 3, 2009, at 6:23 AM, Paul Moore wrote:

> 2009/11/3 Raymond Hettinger <python at rcn.com>:
>> In all these matters, I think the users should get a vote.  And  
>> that vote
>> should be cast with their decision to stay with 2.x, or switch to  
>> 3.x, or
>> try to support both.  We should not muck with their rational  
>> decision making
>> by putting "carrots" in one pile and abandoning the other.

> The biggest issue to my mind is that adoption by the ultimate end
> users is significantly hampered by the fact that big projects like
> Twisted, numpy and the like, have no current plans to move to Python
> 3. Even end users with a reasonable level of coding expertise don't
> have the time or resources to offer much in the way of help with a
> port, when the project as a whole isn't interested in starting the
> process.

For what it's worth, the official position of the Twisted project is  
not that we have "no plan" to move to Python 3.  It's that our plan is  
to do exactly as Raymond suggests, and give the users a vote - in this  
case, you vote with your patches :).

We are actively and frequently encouraging our users to contribute  
patches that clean up warnings, which is the biggest impediment to a  
py3 port of Twisted.  Some of you would probably expecting to whinge  
about how people never contribute anything, but actually, users *have*  
shown up and started doing this.  Our biggest problem at the moment is  
that we don't have enough people doing code reviews so the  
contributions are starting to pile up.  As I said in my other message,  
if someone would like to help, signing up to do code reviews would be  
a good way.

Despite this progress, my hope is that there will be a robust 2.x  
series up through 2.9.

For one thing, we have a very long row to hoe here.  The migration to  
3.0 is a long, tedious process with little tangible benefit.  I hope  
that sometime in the next decade Twisted can accelerate the process of  
dropping old 2.x versions, but I seriously doubt we could do a feature- 
complete 3.1/2.6 version.  I get the general impression that a 3.2/2.7  
port would be more feasible; hopefully a 3.3/2.8 would be even moreso.

Also, the benefits of migrating to python 3.x are still negligible, as  
far as I can tell.  On the one hand, you've got a Python with no old- 
style classes and a clear unicode/bytes situation, and that's great.   
On the other hand, you've got NumPy, PyGTK, Unladen Swallow, PyPy,  
Jython, IronPython, and so on and so forth.  Since I started using it,  
the strength of Python has been in its ecosystem, and the 3.x  
ecosystem is not yet viable.

As long as we're tossing out modest proposals here, I still think that  
(as I believe James Knight already proposed) abandoning the current  
3.x branch, backporting everything to 2.7, and continuing forward with  
a migration strategy that introduces individual deprecations every  
major version until 2.x == 3.x is the way to go.  For example, 2.8  
could emit a deprecation warning for every old-style class that was  
defined, 2.9 could emit a deprecation warning for every string  
constant declared without a 'b' or 'u' prefix unless the module in  
question were in "3.x mode" (i.e. no-prefix == 'u').  (I leave the  
determination of whether the parser should be in 3.x mode for a  
particular module as an exercise for the reader, but a 'from  
__future__' import would suffice.)

I realize that there are other issues here, like the C ABI changes  
some NumPy folks have raised.  Also, I'm not planning to actually do  
any *work* on this suggestion, so you can take it for what it is,  
which is to say, armchair quarterbacking.

There have been some other comments in this thread indicating that  
this was not the case because some users indicated that they'd rather  
deal with lots of changes "all at once".  My understanding is that it  
was done this way so that the *developers* of Python could make a  
clean break, and design and implement a new version of Python without  
being constrained by compatibility concerns.  If you can show me an  
actual application or library developer in Python who wanted this one- 
big-jump migration, I will show you a crazy person.

The main reason I want a long 2.x series is that I believe it would  
more easily allow us infrastructure folks to drop support for *older*  
versions.  With this big 2.x->3.x chasm, I can't really see an end in  
sight for Twisted using Python 2.x as its _source_ language,  
translating with 2to3.  Some projects which depend on Twisted and want  
new versions (and security fixes, etc) are going to want Python 2.x  
for a really long time.  Maybe they're just really conservative, maybe  
they don't have a lot of maintenance energy, or maybe they have other  
dependencies which haven't got a port; it doesn't really matter,  
empirically speaking people want older versions of Python.

Keep in mind also that the 2.x translation process is extremely slow  
and results in a clunky development process.  There's no '2to3 -- 
interactive' commandline that lets me type python 2 at a >>> prompt  
and get python 3 results out so that I can try experiments on the 3.x  
interpreter; I have to actually put my experiments into my unit tests  
and wait 10 minutes to see if it works.  It's like writing C++.

With the 2.x series, users and operating systems seem to move on  
fairly rapidly, because dependencies generally continue to work if you  
upgrade just one version.  This isn't quite as formal a requirement as  
I would like (warnings get generated, unit tests fail, things do  
break) but in practice, users can rely on it for most functionality.   
If 3.x could be broken into a series of transitions like that, where  
you can upgrade one version, fix some stuff, then upgrade another  
version, even if you couldn't actually support more than 2 versions at  
once, I think that we could pick up the migration pace to the point  
where we might actually be using 3.x syntax in a few years.  Having a  
2.x series which goes to 2.9 and then stops isn't *quite* the same  
thing as having one that moves over continuously to some 3.x version,  
but it does seem to me that by that point the chasm between versions  
will have narrowed to a crack, and the migration will be a little hop  
over it rather than the currently-required great flying leap.

(To be fair, that leap is not the surface-to-orbit rocket-propelled  
jump that I originally expected it to be.  Still, it's still not a  
small effort, especially if you're interfacing deeply with a lot of I/ 
O APIs.)


From regebro at gmail.com  Wed Nov  4 07:05:02 2009
From: regebro at gmail.com (Lennart Regebro)
Date: Wed, 4 Nov 2009 07:05:02 +0100
Subject: [Python-Dev] 2.7 Release? 2.7 == last of the 2.x line?
In-Reply-To: <E077843B-919A-4AC7-95D7-0313E4D0328A@gmail.com>
References: <E077843B-919A-4AC7-95D7-0313E4D0328A@gmail.com>
Message-ID: <319e029f0911032205y56f52984q433e8e8d4221a4c5@mail.gmail.com>

2009/11/3 ssteinerX at gmail.com <ssteinerx at gmail.com>:
> Not that anyone has asked yet, but here's my opinion on two issues that have
> been raised on the python-dev mailing list lately:
>
> ? ? ? ?+1 on 2.7 release with as much 3.0 "easy-port goo" as is practicable
> without delaying the product beyond the tentative schedule. ?Sooner would,
> of course, be better but I'm sure PEP 373 was produced after due
> consideration of all pertinent factors.

+1

> ? ? ? ?+1 on 2.7 being the last of the 2.x series. ?Enough already!

-1

The last of the series should be the last one needed, and we don't
know that until after it's been made. No official decision should be
taken on this.

> I, personally, haven't even written my first line of 3.x code, nor have I
> had any good reason to.
>
> If I saw the actual end of the line at 2.7, I would actually start looking
> for 3.x versions of my favorite tools and would be much more inclined to
> help push them along ASAP.

That's a terrible reason. It's a stick, and sticks suck. If Python 3
is not enough of a carrot, then people should not migrate at all.

> Right now, so much that I use on a daily basis doesn't even have a 3.x
> roadmap, much less any sort of working implementation, that I don't see
> switching to 3.x ever unless the 2.x line ends, and soon!

Wrong. Most things do not have a 3.x roadmap or a working
implementation, because most things depend on another thing, that also
doesn't have an implementation, and therefore can't be ported or
planned. The major hangup for this has been setuptools. As you know,
distribute came with a Python 3 release fairly recently, which means
that most projects out there could not be ported or even planned (as
setuptools had no Python 3 plans) until something like a month ago.

-- 
Lennart Regebro: Python, Zope, Plone, Grok
http://regebro.wordpress.com/
+33 661 58 14 64

From regebro at gmail.com  Wed Nov  4 07:06:09 2009
From: regebro at gmail.com (Lennart Regebro)
Date: Wed, 4 Nov 2009 07:06:09 +0100
Subject: [Python-Dev] 2.7 Release? 2.7 == last of the 2.x line?
In-Reply-To: <BBB79ADA-7E31-4DD8-AAA5-07537FA74C5A@gmail.com>
References: <E077843B-919A-4AC7-95D7-0313E4D0328A@gmail.com>
	<F245110A-2FDF-460B-8E0D-87785F0E6DB0@fuhm.net>
	<BBB79ADA-7E31-4DD8-AAA5-07537FA74C5A@gmail.com>
Message-ID: <319e029f0911032206y64386499hf4564839ec9acb1a@mail.gmail.com>

2009/11/3 ssteinerX at gmail.com <ssteinerx at gmail.com>:
>
> On Nov 2, 2009, at 7:26 PM, James Y Knight wrote:
>>
>> It really sounds like you're saying that switching to 3.x isn't worth the
>> cost to you, but you want to force people (including yourself) to do so
>> anyways, because ...?
>
> Because that's the future of Python

Or not. Maybe it's a dead branch of Python?

-- 
Lennart Regebro: Python, Zope, Plone, Grok
http://regebro.wordpress.com/
+33 661 58 14 64

From martin at v.loewis.de  Wed Nov  4 07:07:24 2009
From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=)
Date: Wed, 04 Nov 2009 07:07:24 +0100
Subject: [Python-Dev] 2.7 Release? 2.7 == last of the 2.x line?
In-Reply-To: <5b8d13220911031606g7a1c40d9jbfa51137661a6d4@mail.gmail.com>
References: <E077843B-919A-4AC7-95D7-0313E4D0328A@gmail.com>	
	<4AEFD9AE.7090104@molden.no>	
	<DF3EB3B9-EB18-42C3-BE34-AADC80006DC4@gmail.com>	
	<4AF06A5F.6090501@v.loewis.de> <4AF072BE.2010603@trueblade.com>	
	<4AF0758A.6070108@v.loewis.de>
	<5b8d13220911031606g7a1c40d9jbfa51137661a6d4@mail.gmail.com>
Message-ID: <4AF11A1C.9080107@v.loewis.de>

>> But only if NumPy would drop support for 2.x, for x < 7, right?
>> That would probably be many years in the future.
> 
> Yes. Today, given the choice of supporting py 3.x and dropping python
> < 2.7 and continue support for 2.4, the latter is by far my preferred
> choice today (RHEL still require 2.4, for example).

However, you don't have to make that choice. You can support 2.4 and 3.x
simultaneously; IMO, any simplification resulting from possible
backports is really minor compared to the rest of the work that you
would have to do, anyway.

For example, porting to the new buffer interface is may be difficult
and/or a lot of work, and you'll have to do it, anyway. Making it then
conditional (with preprocessor statements), and maintaining both APIs
in parallel for some time, is really not that hard, IMO.

Regards,
Martin

From regebro at gmail.com  Wed Nov  4 07:31:27 2009
From: regebro at gmail.com (Lennart Regebro)
Date: Wed, 4 Nov 2009 07:31:27 +0100
Subject: [Python-Dev] 2.7 Release? 2.7 == last of the 2.x line?
In-Reply-To: <79990c6b0911030944r26fd668ev37e56a4a7d79559b@mail.gmail.com>
References: <E077843B-919A-4AC7-95D7-0313E4D0328A@gmail.com>
	<F245110A-2FDF-460B-8E0D-87785F0E6DB0@fuhm.net>
	<BBB79ADA-7E31-4DD8-AAA5-07537FA74C5A@gmail.com>
	<D99BA4CF-E16C-47EB-9C60-F1EC5C27CDDC@python.org>
	<4AEFC887.6020801@v.loewis.de>
	<3DE1D8FB-896E-441D-937D-AB63EB3A8E22@python.org>
	<5b8d13220911030540s3d700265ib00f4277b7ce4a2b@mail.gmail.com>
	<4AF0698C.2030706@v.loewis.de>
	<79990c6b0911030944r26fd668ev37e56a4a7d79559b@mail.gmail.com>
Message-ID: <319e029f0911032231l5fb294dbic4fe0ae996dd7c10@mail.gmail.com>

2009/11/3 Paul Moore <p.f.moore at gmail.com>:
> Has that port been integrated back into the zodb project?

That is on the way. Setuptools has been ported via the distribute
project, I'm hoping to make a test release of the zope.interface
branch (to test Python 2 compatibility mainly) soon, and then
hopefully merge it, etc.

It's slow, but moving.

-- 
Lennart Regebro: Python, Zope, Plone, Grok
http://regebro.wordpress.com/
+33 661 58 14 64

From python at rcn.com  Wed Nov  4 07:35:57 2009
From: python at rcn.com (Raymond Hettinger)
Date: Tue, 3 Nov 2009 22:35:57 -0800
Subject: [Python-Dev] Retrieve an arbitrary element from a
	setwithoutremoving it
References: <200911040946.15805.steve@pearwood.info>
	<20091104052640.GA15378@cskk.homeip.net>
Message-ID: <31742B2077EC4E57836315709D494E34@RaymondLaptop1>

> Personally, I have found it useful in doco I write to have a section on
> "Common Tasks", with recommended/suggested examples of how to do them and
> short rationale for the chosen method. It seems to me that if .pick()
> is frequently desired and "None of the standard solutions are obvious
> or easily discoverable" then they should be _made_ so with documentation.
>
> Comments?

That's reasonable.  It's in the same category as people who can't figure-out how to clear a list because they forgot about slice 
notation.

Raymond



From regebro at gmail.com  Wed Nov  4 07:43:06 2009
From: regebro at gmail.com (Lennart Regebro)
Date: Wed, 4 Nov 2009 07:43:06 +0100
Subject: [Python-Dev] 2.7 Release? 2.7 == last of the 2.x line?
In-Reply-To: <20091103185641.GA13969@pyropus.ca>
References: <E077843B-919A-4AC7-95D7-0313E4D0328A@gmail.com>
	<4AEFD9AE.7090104@molden.no> <4AEFF450.9060007@voidspace.org.uk>
	<79EADA413EFB44F6864EF71D5B830088@RaymondLaptop1>
	<16F0B6B0-677F-464C-9C89-CF92D3713A65@gmail.com>
	<116885F3-819E-4B7C-86F7-8E5B246590EB@fuhm.net>
	<bad82a80911030928x1f594a92yc315f2ae8b8246d9@mail.gmail.com>
	<20091103174131.GA10645@pyropus.ca> <4AF0734A.3080705@v.loewis.de>
	<20091103185641.GA13969@pyropus.ca>
Message-ID: <319e029f0911032243y48dfb71fpf399d9fd7881b434@mail.gmail.com>

2009/11/3 Charles Cazabon <charlesc-lists-python-dev at pyropus.ca>:
> "Martin v. L?wis" <martin at v.loewis.de> wrote:
> ?It's pretty easy to make Python source that works under 2.6 and 3.x. ?It's
> ?basically impossible to make Python source that works under 2.4/2.5 and
> ?3.x.

Without using 2to3, yes.

See http://code.google.com/p/python-incompatibility/ where I've
attempted to document most 2.5/3.0 incompatibilities and explain how
to get around them to run the same unmodified code under 2.6 and 3.0.

(It's not complete, help appreciated, especially with the C-API part)

> No. ?getmail v.4 runs fine on Python 2.3.4 through 2.6.x; getmail's code has
> always been pretty forward-compatible.

Then it's going to be dead easy to support both 2.x and 3.x by using 2to3.

If you don't want to have separate packages youcan use distribute:
http://packages.python.org/distribute/python3.html

Also see my talk at PyCon: http://pycon.blip.tv/file/1949281/

-- 
Lennart Regebro: Python, Zope, Plone, Grok
http://regebro.wordpress.com/
+33 661 58 14 64

From arcriley at gmail.com  Wed Nov  4 08:05:42 2009
From: arcriley at gmail.com (Arc Riley)
Date: Wed, 4 Nov 2009 02:05:42 -0500
Subject: [Python-Dev] 2.7 Release? 2.7 == last of the 2.x line?
In-Reply-To: <3d2ce8cb0911031305y330b832fo58e6907011e69efb@mail.gmail.com>
References: <E077843B-919A-4AC7-95D7-0313E4D0328A@gmail.com> 
	<4AEFD9AE.7090104@molden.no> <4AEFF450.9060007@voidspace.org.uk> 
	<79EADA413EFB44F6864EF71D5B830088@RaymondLaptop1>
	<16F0B6B0-677F-464C-9C89-CF92D3713A65@gmail.com> 
	<116885F3-819E-4B7C-86F7-8E5B246590EB@fuhm.net>
	<bad82a80911030928x1f594a92yc315f2ae8b8246d9@mail.gmail.com> 
	<3FE75D51-199C-4221-9B38-94FD2FE9BC4E@gmail.com>
	<hcpthu$9ql$2@ger.gmane.org> 
	<3d2ce8cb0911031305y330b832fo58e6907011e69efb@mail.gmail.com>
Message-ID: <bad82a80911032305o3e92f90bpc04fd6b585e89c21@mail.gmail.com>

I'm not aware of any currently active project that isn't in the process of
adding Py3 support (or who has already done so).  By "most maintainers" I'm
referring to the long tail; the hundreds of 3rd party modules used in niche
cases and can be easily replaced by those who need the functionality they
provide.

Free software is an evolutionary process.  Evolution dictates that those who
cannot adapt will die when faced with a changing environment.  It's better
for us all that they do.

Thus, I am not attempting to persuade anyone to upgrade their package,
rather, I'm explaining why it's best we ignore them and get on with doing
productive work.

On Tue, Nov 3, 2009 at 4:05 PM, Mike Klaas <mike.klaas at gmail.com> wrote:

>
> Be that as it may, the only way python 3 will be widely adopted if people
> have motivation to (need to be compatible with other libs, pressure from
> users, their own interest in fostering python 3.0, etc.).  Deriding them as
> "lazy" accomplishes nothing and obscures the fact that it is the python
> maintainers responsibility to bring about this motivation if they want
> python 3.0 to be adopted.  No-one is going to convert to python 3.0 because
> you called them lazy.
>
> -Mike
>
>>
>
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> http://mail.python.org/mailman/options/python-dev/arcriley%40gmail.com
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20091104/8a22eb00/attachment.htm>

From regebro at gmail.com  Wed Nov  4 08:06:50 2009
From: regebro at gmail.com (Lennart Regebro)
Date: Wed, 4 Nov 2009 08:06:50 +0100
Subject: [Python-Dev] nonlocal keyword in 2.x?
In-Reply-To: <4AF0816A.706@egenix.com>
References: <da7032ce0910211856g45e45632n550aa5b4a10f9a62@mail.gmail.com>
	<ca471dc20911021009u7485adcdgde8fae04a5061059@mail.gmail.com>
	<Pine.LNX.4.64.0911021405420.21420@kimball.webabinitio.net>
	<4AEF4C53.7090902@v.loewis.de>
	<Pine.LNX.4.64.0911021624110.21420@kimball.webabinitio.net>
	<ca471dc20911021333r168e257eja1e5f79dbe4ae234@mail.gmail.com>
	<loom.20091102T223839-797@post.gmane.org>
	<4AEFF3A3.3020103@egenix.com> <4AF06E20.7050307@v.loewis.de>
	<4AF0816A.706@egenix.com>
Message-ID: <319e029f0911032306j746bc263jcf6d0dfeaba9f24d@mail.gmail.com>

I should maybe point out that although I'm generally +1 on
backporting, I'm not specifically anything on backporting the nonlocal
keyword. There are probably things that would help more from a
compatibility standpoint than that.

For example, from __future__ import unicode_literals doesn't switch
the types. So this code will rais an assertion error in 2.6.

>>> from __future__ import unicode_literals
>>> assert isinstance("fghjkl", str)

While it of course will run fine in 3.0. Now this has a fairly trivial
workaround or two, but still.

I also would really like to see a real port of the bytes class to 2.6,
but I have a vague memory that there was some reason that wouldn't
work.

-- 
Lennart Regebro: Python, Zope, Plone, Grok
http://regebro.wordpress.com/
+33 661 58 14 64

From p.f.moore at gmail.com  Wed Nov  4 09:40:35 2009
From: p.f.moore at gmail.com (Paul Moore)
Date: Wed, 4 Nov 2009 08:40:35 +0000
Subject: [Python-Dev] 2.7 Release? 2.7 == last of the 2.x line?
In-Reply-To: <CCFA9232-8222-4714-8356-F6C501744077@twistedmatrix.com>
References: <E077843B-919A-4AC7-95D7-0313E4D0328A@gmail.com>
	<F245110A-2FDF-460B-8E0D-87785F0E6DB0@fuhm.net>
	<BBB79ADA-7E31-4DD8-AAA5-07537FA74C5A@gmail.com>
	<D99BA4CF-E16C-47EB-9C60-F1EC5C27CDDC@python.org>
	<4AEFC887.6020801@v.loewis.de>
	<3DE1D8FB-896E-441D-937D-AB63EB3A8E22@python.org>
	<5b8d13220911030540s3d700265ib00f4277b7ce4a2b@mail.gmail.com>
	<bbaeab100911031153h61c356dfw850c53d4662aca7@mail.gmail.com>
	<79990c6b0911031416r36c6947ldd19984108d46f7e@mail.gmail.com>
	<CCFA9232-8222-4714-8356-F6C501744077@twistedmatrix.com>
Message-ID: <79990c6b0911040040h4b7932fbla8b731c0c057d460@mail.gmail.com>

2009/11/4 Glyph Lefkowitz <glyph at twistedmatrix.com>:
> On Nov 3, 2009, at 5:16 PM, Paul Moore wrote:
>>
>> 2009/11/3 Brett Cannon <brett at python.org>:
>>>
>>> I'm afraid there is some FUD going around here, which is
>>> understandable since no one wants to burn a ton of time on something
>>> that will be difficult or take a lot of time. But I have not heard
>>> anyone in this email thread (or anywhere for that matter) say that
>>> they tried a port in earnest and it turned out to be difficult.
>
>> FWIW, I did a quick survey of some packages (a sampling of packages
>> I've used or considered using in the past):
>>
>> Twisted - no plans yet for Python 3
>
>
> Speaking of FUD, we've had a plan for Python 3 support for some time:
[...]

Thanks, and my sincere apologies for spreading FUD - I did try to find
details, and in fact I did spot a couple of the specific python 3
compatibility tickets you mentioned, but missed the link back to the
master plan.

Having said that,

> ?http://stackoverflow.com/questions/172306/how-are-you-planning-on-handling-the-migration-to-python-3/214601#214601

seems pretty negative in terms of when Twisted will support Python 3.
Based on my reading, the focus is more on when 2.x support will be
dropped than on when there will be a version of Twisted which works
with Python 3 (which I'd expect to be much sooner!)

> If you're interested in helping, our core team has all not had much time for
> Twisted lately and we need volunteers who are interested in doing code
> reviews and becoming a committer to help shepherd these tickets through the
> process.

Personally, I don't *use* twisted. I occasionally play with it, and I
sometimes end up using applications which rely on it, but I don't use
it myself. So I couldn't be much direct help. (And yes, I know that
means I'm not one of your users, so me asking for Python 3 support
isn't an issue. No problem there).

Paul.

From doug.hellmann at gmail.com  Wed Nov  4 11:05:34 2009
From: doug.hellmann at gmail.com (Doug Hellmann)
Date: Wed, 4 Nov 2009 05:05:34 -0500
Subject: [Python-Dev] PEP 3003 - Python Language Moratorium
In-Reply-To: <4AF095A1.6040406@voidspace.org.uk>
References: <ca471dc20911030935l25251e0diec64427f5d7bcbf2@mail.gmail.com>
	<B9E66EB8-54B5-4578-AA2A-7C8614FE785F@python.org>
	<4AF095A1.6040406@voidspace.org.uk>
Message-ID: <1B2F0471-AE7C-4AAC-9AD1-5C4A68014DFB@gmail.com>


On Nov 3, 2009, at 3:42 PM, Michael Foord wrote:

> Barry Warsaw wrote:
>> On Nov 3, 2009, at 12:35 PM, Guido van Rossum wrote:
>>
>>> I've checked draft (!) PEP 3003, "Python Language Moratorium", into
>>> SVN. As authors I've listed Jesse, Brett and myself.
>>>
>>> On python-ideas the moratorium idea got fairly positive responses
>>> (more positive than I'd expected, in fact) but I'm bracing myself  
>>> for
>>> fierce discussion here on python-dev. It's important to me that if  
>>> if
>>> this is accepted it is a "rough consensus" decision (working code we
>>> already have plenty of :-), not something enforced by a vocal  
>>> minority
>>> or an influential individual such as myself. If there's too much
>>> opposition I'll withdraw the PEP so as not to waste everybody's time
>>> with a fruitless discussion.
>>>
>>> The PEP tries to spell out some gray areas but I'm sure there will  
>>> be
>>> others; that's life. Do note that the PEP proposes to be  
>>> *retroactive*
>>> back to the 3.1 release, i.e. the "frozen" version of the language  
>>> is
>>> the state in which it was released as 3.1.
>>
>> I think this is a great idea.  I'd love to see the energy normally  
>> put into evolving the language into making the stdlib really kick  
>> ass.
>>
>
> +lots

Ditto.

Doug


From steve at pearwood.info  Wed Nov  4 12:00:49 2009
From: steve at pearwood.info (Steven D'Aprano)
Date: Wed, 4 Nov 2009 22:00:49 +1100
Subject: [Python-Dev] Retrieve an arbitrary element from a set
	withoutremoving it
In-Reply-To: <ca471dc20911031510j3cd6f7fek76894942518f8726@mail.gmail.com>
References: <7b0045ed0910301843y7059d316s8a851142b78b7aeb@mail.gmail.com>
	<200911040946.15805.steve@pearwood.info>
	<ca471dc20911031510j3cd6f7fek76894942518f8726@mail.gmail.com>
Message-ID: <200911042200.50850.steve@pearwood.info>

On Wed, 4 Nov 2009 10:10:30 am Guido van Rossum wrote:
> On Tue, Nov 3, 2009 at 2:46 PM, Steven D'Aprano <steve at pearwood.info> 
wrote:
> > Since I was the person who decided that "arbitrary" meant "give a
> > different result each time", I should answer that.

> You're obviously talking about a *random* element. This is a separate
> use case (though I agree many people don't know the difference).

I'm aware of the difference between random and arbitrary, and in an 
earlier post I said that the One Obvious Way of getting a random 
element from a list would be to convert to a list and call 
random.choice(). Sorry for muddying the waters by linking to a page 
discussing such random selections.


-- 
Steven D'Aprano

From steve at pearwood.info  Wed Nov  4 12:34:36 2009
From: steve at pearwood.info (Steven D'Aprano)
Date: Wed, 4 Nov 2009 22:34:36 +1100
Subject: [Python-Dev] Retrieve an arbitrary element from a set
	withoutremoving it
In-Reply-To: <4AF0D0D7.3090403@canterbury.ac.nz>
References: <7b0045ed0910301843y7059d316s8a851142b78b7aeb@mail.gmail.com>
	<200911040946.15805.steve@pearwood.info>
	<4AF0D0D7.3090403@canterbury.ac.nz>
Message-ID: <200911042234.36796.steve@pearwood.info>

On Wed, 4 Nov 2009 11:54:47 am Greg Ewing wrote:
> Steven D'Aprano wrote:
> > I don't know how expensive it is to create a set iterator,
>
> Not expensive enough to justify burdening the set type with
> extra functionality that will be extremely rarely used.

As my previous posts on this topic tried to convey, this isn't primarily 
about efficiency, but about discoverability and obviousness.

Anyway, given the level of opposition to the suggestion, I'm no longer 
willing to carry the flag for it. If anyone else -- perhaps the OP -- 
feels they want to take it any further, be my guest.



-- 
Steven D'Aprano

From ncoghlan at gmail.com  Wed Nov  4 12:53:15 2009
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Wed, 04 Nov 2009 21:53:15 +1000
Subject: [Python-Dev] PEP 3003 - Python Language Moratorium
In-Reply-To: <b8e622740911031507s73daad8bw88628a16bc660b81@mail.gmail.com>
References: <ca471dc20911030935l25251e0diec64427f5d7bcbf2@mail.gmail.com>
	<b8e622740911031507s73daad8bw88628a16bc660b81@mail.gmail.com>
Message-ID: <4AF16B2B.6010807@gmail.com>

Jack Diederich wrote:
> +1.  There are no compelling language changes on the horizon ("yield
> from" is nice but not necessary).

Another +1 here.

A note in the PEP that there are no changes to SVN that would need to be
rolled back due to the moratorium would be a good addition (as per
Antoine's review of the NEWS file).

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------

From ncoghlan at gmail.com  Wed Nov  4 13:00:07 2009
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Wed, 04 Nov 2009 22:00:07 +1000
Subject: [Python-Dev] Py3k bytes type in 2.x (Re: nonlocal keyword in 2.x?)
In-Reply-To: <319e029f0911032306j746bc263jcf6d0dfeaba9f24d@mail.gmail.com>
References: <da7032ce0910211856g45e45632n550aa5b4a10f9a62@mail.gmail.com>	<ca471dc20911021009u7485adcdgde8fae04a5061059@mail.gmail.com>	<Pine.LNX.4.64.0911021405420.21420@kimball.webabinitio.net>	<4AEF4C53.7090902@v.loewis.de>	<Pine.LNX.4.64.0911021624110.21420@kimball.webabinitio.net>	<ca471dc20911021333r168e257eja1e5f79dbe4ae234@mail.gmail.com>	<loom.20091102T223839-797@post.gmane.org>	<4AEFF3A3.3020103@egenix.com>
	<4AF06E20.7050307@v.loewis.de>	<4AF0816A.706@egenix.com>
	<319e029f0911032306j746bc263jcf6d0dfeaba9f24d@mail.gmail.com>
Message-ID: <4AF16CC7.6000705@gmail.com>

Lennart Regebro wrote:
> I also would really like to see a real port of the bytes class to 2.6,
> but I have a vague memory that there was some reason that wouldn't
> work.

Not so much that it wouldn't work, but that the interfaces to support
using it effectively really aren't there - lots of areas in the standard
library needed to be tweaked to cope with bytes objects in 3.x.

Generally speaking, the "bytes = str" trick represents a reasonable
compromise as the APIs that you would pass a bytes object to in 3.x
expect an 8-bit str instance in 2.x.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------

From mrts.pydev at gmail.com  Wed Nov  4 13:20:27 2009
From: mrts.pydev at gmail.com (=?ISO-8859-1?Q?Mart_S=F5mermaa?=)
Date: Wed, 4 Nov 2009 14:20:27 +0200
Subject: [Python-Dev] A wordcode-based Python
In-Reply-To: <4213.88.149.182.147.1242111241.squirrel@webmail5.pair.com>
References: <49D4DA72.60401@v.loewis.de> <4A0590C4.1020904@v.loewis.de>
	<20090509143829.17F293A4080@sparrow.telecommunity.com>
	<4A059639.7040505@v.loewis.de>
	<20090509153716.D44633A4080@sparrow.telecommunity.com>
	<7FF9D9A9-211E-4E5D-BDD0-9C0315123975@zooko.com>
	<4c8bd6707712f01ccf3841c2c26169ef@preisshare.net>
	<1024.88.149.182.147.1242064816.squirrel@webmail5.pair.com>
	<43aa6ff70905111414n62d20099r9bb2b3ebd52a26ec@mail.gmail.com>
	<4213.88.149.182.147.1242111241.squirrel@webmail5.pair.com>
Message-ID: <ad1f81530911040420q3f3f8976p1fd1295cb84c2637@mail.gmail.com>

On Tue, May 12, 2009 at 8:54 AM, Cesare Di Mauro
<cesare.dimauro at a-tono.com> wrote:
>> Also, I checked out wpython at head to run Unladen Swallow's
>> benchmarks against it, but it refuses to compile with either gcc 4.0.1
>> or 4.3.1 on Linux (fails in Python/ast.c). I can send you the build
>> failures off-list, if you're interested.
>>
>> Thanks,
>> Collin Winter
>
> I'm very interested, thanks. That's because I worked only on Windows
> machines, so I definitely need to test and fix it to let it run on any other
> platform.
>
> Cesare

Re-animating an old discussion -- Cesare, any news on the wpython front?

I did a checkout from http://wpython.googlecode.com/svn/trunk and
was able to ./configure and make successfully on my 64-bit Linux box
as well as to run the Unladen benchmarks.

Given svn co http://svn.python.org/projects/python/tags/r261 in py261
and svn co http://wpython.googlecode.com/svn/trunk in wpy,

$ python unladen-tests/perf.py -rm --benchmarks=-2to3,all py261/python
wpy/python

gives the following results:

Report on Linux foo 2.6.31-14-generic #48-Ubuntu SMP Fri Oct 16
14:05:01 UTC 2009 x86_64
Total CPU cores: 2

ai:
Min: 0.640516 -> 0.586532: 9.20% faster
Avg: 0.677346 -> 0.632785: 7.04% faster
Significant (t=4.336740, a=0.95)
Stddev: 0.05839 -> 0.08455: 30.94% larger

Mem max: 7412.000 -> 6768.000: 9.52% smaller
Usage over time: http://tinyurl.com/ykwhmcc


call_simple:
Min: 1.880816 -> 1.701622: 10.53% faster
Avg: 1.944320 -> 1.778701: 9.31% faster
Significant (t=14.323045, a=0.95)
Stddev: 0.09885 -> 0.06000: 64.74% smaller

Mem max: 8100.000 -> 6636.000: 22.06% smaller
Usage over time: http://tinyurl.com/yzsswgp


django:
Min: 1.287158 -> 1.315700: 2.17% slower
Avg: 1.330423 -> 1.366978: 2.67% slower
Significant (t=-4.475769, a=0.95)
Stddev: 0.05663 -> 0.05885: 3.78% larger

Mem max: 15508.000 -> 16228.000: 4.44% larger
Usage over time: http://tinyurl.com/yfpbmjn


iterative_count:
Min: 0.211620 -> 0.124646: 69.78% faster
Avg: 0.222778 -> 0.159868: 39.35% faster
Significant (t=9.291635, a=0.95)
Stddev: 0.04239 -> 0.05279: 19.69% larger

Mem max: 7388.000 -> 6680.000: 10.60% smaller
Usage over time: http://tinyurl.com/yj7s8h4


normal_startup:
Min: 1.060017 -> 0.991366: 6.92% faster
Avg: 1.189612 -> 1.170067: 1.67% faster
Significant (t=2.002086, a=0.95)
Stddev: 0.06942 -> 0.06864: 1.13% smaller

Mem max: 3252.000 -> 4648.000: 30.03% larger
Usage over time: http://tinyurl.com/ygo3bwt


pickle:
Min: 2.027566 -> 1.948784: 4.04% faster
Avg: 2.051633 -> 2.043656: 0.39% faster
Not significant
Stddev: 0.03095 -> 0.07348: 57.88% larger

Mem max: 8544.000 -> 7340.000: 16.40% smaller
Usage over time: http://tinyurl.com/ykg9dn2


pickle_dict:
Min: 1.658693 -> 1.656844: 0.11% faster
Avg: 1.689483 -> 1.698176: 0.51% slower
Not significant
Stddev: 0.16945 -> 0.09403: 80.20% smaller

Mem max: 6716.000 -> 7636.000: 12.05% larger
Usage over time: http://tinyurl.com/yjhyame


pickle_list:
Min: 0.919083 -> 0.894758: 2.72% faster
Avg: 0.956513 -> 0.921314: 3.82% faster
Significant (t=2.131237, a=0.95)
Stddev: 0.12744 -> 0.10506: 21.31% smaller

Mem max: 6804.000 -> 8792.000: 22.61% larger
Usage over time: http://tinyurl.com/ylc3ezf


pybench:
Min: 58781 -> 50836: 15.63% faster
Avg: 60009 -> 51788: 15.87% faster

regex_compile:
Min: 0.934131 -> 0.862323: 8.33% faster
Avg: 0.962159 -> 0.884848: 8.74% faster
Significant (t=13.587168, a=0.95)
Stddev: 0.04685 -> 0.03229: 45.11% smaller

Mem max: 12584.000 -> 12740.000: 1.22% larger
Usage over time: http://tinyurl.com/yjngu8j


regex_effbot:
Min: 0.130686 -> 0.122483: 6.70% faster
Avg: 0.143453 -> 0.138078: 3.89% faster
Not significant
Stddev: 0.01864 -> 0.03177: 41.32% larger

Mem max: 7652.000 -> 6660.000: 14.89% smaller
Usage over time: http://tinyurl.com/ykcgntf


regex_v8:
Min: 0.135130 -> 0.150092: 9.97% slower
Avg: 0.138027 -> 0.177309: 22.15% slower
Significant (t=-8.197595, a=0.95)
Stddev: 0.00258 -> 0.04785: 94.60% larger

Mem max: 11124.000 -> 12236.000: 9.09% larger
Usage over time: http://tinyurl.com/ykb5vzu


rietveld:
Min: 0.848245 -> 0.816473: 3.89% faster
Avg: 1.033925 -> 1.019889: 1.38% faster
Not significant
Stddev: 0.11242 -> 0.13006: 13.56% larger

Mem max: 23792.000 -> 24548.000: 3.08% larger
Usage over time: http://tinyurl.com/yhdvz5v


slowpickle:
Min: 0.876736 -> 0.800203: 9.56% faster
Avg: 0.932808 -> 0.870577: 7.15% faster
Significant (t=5.020426, a=0.95)
Stddev: 0.05600 -> 0.11059: 49.36% larger

Mem max: 7200.000 -> 7276.000: 1.04% larger
Usage over time: http://tinyurl.com/ykt2brq


slowspitfire:
Min: 1.029100 -> 0.948458: 8.50% faster
Avg: 1.062486 -> 1.020777: 4.09% faster
Significant (t=4.581669, a=0.95)
Stddev: 0.05441 -> 0.07298: 25.44% larger

Mem max: 139792.000 -> 129264.000: 8.14% smaller
Usage over time: http://tinyurl.com/yh7vmlh


slowunpickle:
Min: 0.411744 -> 0.356784: 15.40% faster
Avg: 0.444638 -> 0.393261: 13.06% faster
Significant (t=7.009269, a=0.95)
Stddev: 0.04147 -> 0.06044: 31.38% larger

Mem max: 7132.000 -> 7848.000: 9.12% larger
Usage over time: http://tinyurl.com/yfwvz3g


startup_nosite:
Min: 0.664456 -> 0.598770: 10.97% faster
Avg: 0.933034 -> 0.761228: 22.57% faster
Significant (t=20.660776, a=0.95)
Stddev: 0.09645 -> 0.06728: 43.37% smaller

Mem max: 1940.000 -> 1940.000: -0.00% smaller
Usage over time: http://tinyurl.com/yzzxcmd


threaded_count:
Min: 0.220059 -> 0.138708: 58.65% faster
Avg: 0.232347 -> 0.156120: 48.83% faster
Significant (t=23.804797, a=0.95)
Stddev: 0.01889 -> 0.02586: 26.96% larger

Mem max: 6460.000 -> 7664.000: 15.71% larger
Usage over time: http://tinyurl.com/yzm3awu


unpack_sequence:
Min: 0.000129 -> 0.000120: 7.57% faster
Avg: 0.000218 -> 0.000194: 12.14% faster
Significant (t=3.946194, a=0.95)
Stddev: 0.00139 -> 0.00128: 8.13% smaller

Mem max: 18948.000 -> 19056.000: 0.57% larger
Usage over time: http://tinyurl.com/yf8es3f


unpickle:
Min: 1.191468 -> 1.206198: 1.22% slower
Avg: 1.248471 -> 1.281957: 2.61% slower
Significant (t=-2.658526, a=0.95)
Stddev: 0.05513 -> 0.11325: 51.32% larger

Mem max: 7776.000 -> 8676.000: 10.37% larger
Usage over time: http://tinyurl.com/yz96gw2


unpickle_list:
Min: 0.922200 -> 0.861167: 7.09% faster
Avg: 0.955964 -> 0.976829: 2.14% slower
Not significant
Stddev: 0.04374 -> 0.21061: 79.23% larger

Mem max: 6820.000 -> 8324.000: 18.07% larger
Usage over time: http://tinyurl.com/yjbraxg

---

The diff between the two trees is at
http://dpaste.org/RpIv/

Best,
Mart S?mermaa

From mal at egenix.com  Wed Nov  4 13:21:01 2009
From: mal at egenix.com (M.-A. Lemburg)
Date: Wed, 04 Nov 2009 13:21:01 +0100
Subject: [Python-Dev] Py3k bytes type in 2.x (Re: nonlocal keyword in
 2.x?)
In-Reply-To: <4AF16CC7.6000705@gmail.com>
References: <da7032ce0910211856g45e45632n550aa5b4a10f9a62@mail.gmail.com>	<ca471dc20911021009u7485adcdgde8fae04a5061059@mail.gmail.com>	<Pine.LNX.4.64.0911021405420.21420@kimball.webabinitio.net>	<4AEF4C53.7090902@v.loewis.de>	<Pine.LNX.4.64.0911021624110.21420@kimball.webabinitio.net>	<ca471dc20911021333r168e257eja1e5f79dbe4ae234@mail.gmail.com>	<loom.20091102T223839-797@post.gmane.org>	<4AEFF3A3.3020103@egenix.com>	<4AF06E20.7050307@v.loewis.de>	<4AF0816A.706@egenix.com>	<319e029f0911032306j746bc263jcf6d0dfeaba9f24d@mail.gmail.com>
	<4AF16CC7.6000705@gmail.com>
Message-ID: <4AF171AD.1000703@egenix.com>

Nick Coghlan wrote:
> Lennart Regebro wrote:
>> I also would really like to see a real port of the bytes class to 2.6,
>> but I have a vague memory that there was some reason that wouldn't
>> work.
> 
> Not so much that it wouldn't work, but that the interfaces to support
> using it effectively really aren't there - lots of areas in the standard
> library needed to be tweaked to cope with bytes objects in 3.x.
> 
> Generally speaking, the "bytes = str" trick represents a reasonable
> compromise as the APIs that you would pass a bytes object to in 3.x
> expect an 8-bit str instance in 2.x.

Could you please add such hints, tricks and tips to the wiki
page:

	http://wiki.python.org/moin/PortingToPy3k

Thanks,
-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Nov 04 2009)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try our new mxODBC.Connect Python Database Interface for free ! ::::


   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
           Registered at Amtsgericht Duesseldorf: HRB 46611
               http://www.egenix.com/company/contact/

From ziade.tarek at gmail.com  Wed Nov  4 13:29:35 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Wed, 4 Nov 2009 13:29:35 +0100
Subject: [Python-Dev] Refactoring installation schemes
In-Reply-To: <94bdd2610910281105q328bc3a5qaed020ef3afe28df@mail.gmail.com>
References: <94bdd2610910271455s7854e6a9u94e3aeeeacb69c02@mail.gmail.com>
	<4AE8206A.9020108@egenix.com> <4AE824DA.4090701@voidspace.org.uk>
	<ED004FA5-9F77-4215-B06D-6CBECEF6D023@gmail.com>
	<94bdd2610910281105q328bc3a5qaed020ef3afe28df@mail.gmail.com>
Message-ID: <94bdd2610911040429j50a6df87m51aac2464dd13bd7@mail.gmail.com>

On Wed, Oct 28, 2009 at 7:05 PM, Tarek Ziad? <ziade.tarek at gmail.com> wrote:
> Ok then I'll work on a patch for that change and start an issue about it soon.

As I expected, being able to provide all those paths pulls a lot of
other stuffs out of distutils.

In fact, almost all the APIs that are located in distutils/sysconfig
can be taken out of distutils, and cleaned up for stdlib's benefit.
I've started to refactor the code in a module I have called
"sysconfig", reusing distutils/sysconfig, distutils/command/install
and site code.

This "sysconfig" module should provide at the end very useful APIs to
query the current Python environment.

That's a work in progress but: I've started a branch at
/python/branches/tarek_sysconfig so it's easier to get some feedbacks
if anyone want to help on this.

Regards
Tarek

From solipsis at pitrou.net  Wed Nov  4 13:43:19 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Wed, 4 Nov 2009 12:43:19 +0000 (UTC)
Subject: [Python-Dev] 2to3 interactive mode
References: <E077843B-919A-4AC7-95D7-0313E4D0328A@gmail.com>
	<4AEFD9AE.7090104@molden.no> <4AEFF450.9060007@voidspace.org.uk>
	<79EADA413EFB44F6864EF71D5B830088@RaymondLaptop1>
	<79990c6b0911030323x47a9f008ue7256c56e88c0ee8@mail.gmail.com>
	<5CB59A21-D8C9-4BC1-A1F6-4438B86A8BC5@twistedmatrix.com>
Message-ID: <loom.20091104T133942-829@post.gmane.org>

Glyph Lefkowitz <glyph <at> twistedmatrix.com> writes:
> 
> Keep in mind also that the 2.x translation process is extremely slow  
> and results in a clunky development process.  There's no '2to3 -- 
> interactive' commandline that lets me type python 2 at a >>> prompt  
> and get python 3 results out so that I can try experiments on the 3.x  
> interpreter; I have to actually put my experiments into my unit tests  
> and wait 10 minutes to see if it works.  It's like writing C++.

Please enter a feature request into the bug tracker.

Regards

Antoine.



From ncoghlan at gmail.com  Wed Nov  4 13:45:45 2009
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Wed, 04 Nov 2009 22:45:45 +1000
Subject: [Python-Dev] 2.7 Release? 2.7 == last of the 2.x line?
In-Reply-To: <hcqoof$2uj$1@ger.gmane.org>
References: <E077843B-919A-4AC7-95D7-0313E4D0328A@gmail.com>	<4AEFD9AE.7090104@molden.no>	<DF3EB3B9-EB18-42C3-BE34-AADC80006DC4@gmail.com>	<4AF06A5F.6090501@v.loewis.de>	<ca471dc20911031039o4931d7c7xc5be220c100c33e5@mail.gmail.com>
	<hcqoof$2uj$1@ger.gmane.org>
Message-ID: <4AF17779.40206@gmail.com>

Terry Reedy wrote:
> Guido van Rossum wrote:
>> On Tue, Nov 3, 2009 at 9:37 AM, "Martin v. L?wis" <martin at v.loewis.de>
>> wrote:
>>> (and no, adding things like nonlocal to 2.7 doesn't making porting of
>>> a real application or library any easier, since the existing application
>>> or library simply doesn't use that keyword.
>>
>> Agreed.
>>
>>> In fact, no change to 2.x
>>> can reasonably simplify porting - only changes to 3.x might - except
>>> for changes to 2to3, which can simplify porting a lot. But 2to3 should
>>> be run under 3.x, IMO.)
>>
>> Disagreed. Better "-3" warnings could make porting easier. (Not just
>> more warnings -- "better" might mean fewer false positives for
>> warnings already issued.)
> 
> There is also Eric Smith's list to consider: PEP3118 new buffer
> protocol, short float repr, and maybe io.

The pure Python io module was already backported for the 2.6 release
[1], as was the C API aspect of PEP 3118 [2].

Short float repr has since been backported for 2.7, as has the C
accelerated io module implementation and the Python API (memoryview)
aspect of PEP 3118.

I believe those 3 features alone are more than enough justification to
proceed with at least a 2.7 release (that is probably the point Eric was
making in posting that list of features in the first place).

As to how those backports can help with forward ports to Py3k, someone
made the point elsewhere in the thread that testing/experimenting via
2to3 is a very C++ like development cycle - there is a long build time
before you get to see the results of running a test. With features
backported to 2.x, you can instead use more traditional version checks
(or the interactive prompt) and get the usual quick feedback cycle via
the 2.7 version, before submitting your code to the tender mercies of
the 2to3 converter (or possibly avoid 2to3 altogether if the version
checks turn out to suffice for a given use case).

Cheers,
Nick.

[1] http://docs.python.org/whatsnew/2.6.html#pep-3116-new-i-o-library
[2]
http://docs.python.org/whatsnew/2.6.html#pep-3118-revised-buffer-protocol

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------

From list at qtrac.plus.com  Wed Nov  4 14:18:24 2009
From: list at qtrac.plus.com (Mark Summerfield)
Date: Wed, 4 Nov 2009 13:18:24 +0000
Subject: [Python-Dev] 2.7 Release? 2.7 == last of the 2.x line?
In-Reply-To: <4AF17779.40206@gmail.com>
References: <E077843B-919A-4AC7-95D7-0313E4D0328A@gmail.com>
	<hcqoof$2uj$1@ger.gmane.org> <4AF17779.40206@gmail.com>
Message-ID: <200911041318.24648.list@qtrac.plus.com>

Hi,

I wanted to make some brief comments on this thread:

- 2to3 encourages people to see Python 3 as exotic and other---and not
  to actually write in it.

- 3to2 encourages people to use Python 3 and also provides a route to
  Python 2 compatibility.

I hope that a point will be reached where people are encouraged to do a
one off 2to3, hand fix, and once it passes their tests to keep a single
Python 3 source and use 3to2 to support their users of older Pythons.

- Unicode strings is the solution, not the problem, and is one of Python
  3's most important advances.

- Have any big ports been done? Yes, PyQt4.
  PyQt4 supports both Python 2 and Python 3---and the port was done by
  one person in his "spare" time over a period of months. PyQt4 wraps at
  least 700,000 lines of C++ code---and it isn't just GUI stuff, it has
  networking, threading, etc., and works on Linux, Mac, Windows, etc.

- I do hope NumPy gets ported, since both on and off the lists it seems
  like a show-stopper for many people.

- I hope the "ditch 3" calls are ignored. Python 3 is significantly
  better than (an already excellent) Python 2: eventually people will
  port---or those who start out with Python 3 will build their own
  libraries for what's missing, just as people did when Python 2 came
  out.

- I think the developers have done a fantastic job with Python 3.
  I just wish more people realised how good it is!

Regarding the Moratorium:

+inf

since I'd really love to see more time devoted to improving the standard
library.

My 2c:-)

-- 
Mark Summerfield, Qtrac Ltd, www.qtrac.eu
    C++, Python, Qt, PyQt - training and consultancy
        "Advanced Qt Programming" - ISBN 0321635906

From ncoghlan at gmail.com  Wed Nov  4 14:20:55 2009
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Wed, 04 Nov 2009 23:20:55 +1000
Subject: [Python-Dev] Py3k bytes type in 2.x (Re: nonlocal keyword in
 2.x?)
In-Reply-To: <4AF171AD.1000703@egenix.com>
References: <da7032ce0910211856g45e45632n550aa5b4a10f9a62@mail.gmail.com>	<ca471dc20911021009u7485adcdgde8fae04a5061059@mail.gmail.com>	<Pine.LNX.4.64.0911021405420.21420@kimball.webabinitio.net>	<4AEF4C53.7090902@v.loewis.de>	<Pine.LNX.4.64.0911021624110.21420@kimball.webabinitio.net>	<ca471dc20911021333r168e257eja1e5f79dbe4ae234@mail.gmail.com>	<loom.20091102T223839-797@post.gmane.org>	<4AEFF3A3.3020103@egenix.com>	<4AF06E20.7050307@v.loewis.de>	<4AF0816A.706@egenix.com>	<319e029f0911032306j746bc263jcf6d0dfeaba9f24d@mail.gmail.com>	<4AF16CC7.6000705@gmail.com>
	<4AF171AD.1000703@egenix.com>
Message-ID: <4AF17FB7.5090708@gmail.com>

M.-A. Lemburg wrote:
> Nick Coghlan wrote:
>> Lennart Regebro wrote:
>>> I also would really like to see a real port of the bytes class to 2.6,
>>> but I have a vague memory that there was some reason that wouldn't
>>> work.
>> Not so much that it wouldn't work, but that the interfaces to support
>> using it effectively really aren't there - lots of areas in the standard
>> library needed to be tweaked to cope with bytes objects in 3.x.
>>
>> Generally speaking, the "bytes = str" trick represents a reasonable
>> compromise as the APIs that you would pass a bytes object to in 3.x
>> expect an 8-bit str instance in 2.x.
> 
> Could you please add such hints, tricks and tips to the wiki
> page:
> 
> 	http://wiki.python.org/moin/PortingToPy3k

Done (although the task of figuring out how to get the wiki to display
code properly defeated me... the normal Python documentation syntax for
it seemed to give the wiki's ReST interpreter fits).

I also mentioned the trick someone mentioned about "from __future__
import unicode_literals" not changing the meaning of 'str' since it only
alters the parser but not the builtins.

In writing it up, it occurred to me that having that kind of thing in a
"py3_compat" compatibility module (to be used as, e.g., "from py3_compat
import str, bytes") would not only make it easier to use in multiple
modules, but also easier for 2to3 to remove when forward porting.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------

From cesare.di.mauro at gmail.com  Wed Nov  4 14:30:01 2009
From: cesare.di.mauro at gmail.com (Cesare Di Mauro)
Date: Wed, 4 Nov 2009 14:30:01 +0100
Subject: [Python-Dev] A wordcode-based Python
In-Reply-To: <ad1f81530911040420q3f3f8976p1fd1295cb84c2637@mail.gmail.com>
References: <49D4DA72.60401@v.loewis.de>
	<20090509143829.17F293A4080@sparrow.telecommunity.com>
	<4A059639.7040505@v.loewis.de>
	<20090509153716.D44633A4080@sparrow.telecommunity.com>
	<7FF9D9A9-211E-4E5D-BDD0-9C0315123975@zooko.com>
	<4c8bd6707712f01ccf3841c2c26169ef@preisshare.net>
	<1024.88.149.182.147.1242064816.squirrel@webmail5.pair.com>
	<43aa6ff70905111414n62d20099r9bb2b3ebd52a26ec@mail.gmail.com>
	<4213.88.149.182.147.1242111241.squirrel@webmail5.pair.com>
	<ad1f81530911040420q3f3f8976p1fd1295cb84c2637@mail.gmail.com>
Message-ID: <c184dbd10911040530g1eed71e4t5a603cf9fdfd3b48@mail.gmail.com>

Hi Mart

I had some problems and little time to dedicate to wpython in the last
period, but I restarted again with it in the last month.

Currently I'm working on changing and documenting the code so that almost
every optimization can be selected. So you'll compile it enabling only the
ones you are interested in.

I've also investigated about some ideas which Antoine told me on grouping
together FASTs and CONSTs in order to reduce bytecodes, but I've found that
the suggested solution brings some problems with the current function call
implementation that can hurt performance on some situations (mostly with
recursive ones, because usually they need to create new frames, and
constants references must be copied and INCREFed).
Since it will require huge changes to the current code base, I don't know if
it's worth the effort just to verify the idea. I'll think about it when the
project will be "finalized".

My plan is to finish the current work in a few days, and then remove the
(may be ugly) hacks that I made to the Python object model that were needed
to let tuples, lists and dictionaries be loaded as CONSTs.
May be a the end of the month it'll be fixed (and the diffs against CPython
will be reduced a lot, since a few files results changed).

Next, I need to changed the trace code (in frameobject.c) to let the
test_trace.py pass (at this time two tests are disabled because the VM
crashes).

Finally, I think to update the code base to 2.6.4.

I think to release everything at the end of the year, but if someone is
interested I can do a partial release at the end of November.

Regarding your tests, they are very interesting, particularly for regex_v8
that showed an unexpected result for me. I'll investigate about it after
I'll release wpython.

I you have any questions, I'm at your disposal (thanks for your tests!)

Cesare

2009/11/4 Mart S?mermaa <mrts.pydev at gmail.com>

> On Tue, May 12, 2009 at 8:54 AM, Cesare Di Mauro
> <cesare.dimauro at a-tono.com> wrote:
> >> Also, I checked out wpython at head to run Unladen Swallow's
> >> benchmarks against it, but it refuses to compile with either gcc 4.0.1
> >> or 4.3.1 on Linux (fails in Python/ast.c). I can send you the build
> >> failures off-list, if you're interested.
> >>
> >> Thanks,
> >> Collin Winter
> >
> > I'm very interested, thanks. That's because I worked only on Windows
> > machines, so I definitely need to test and fix it to let it run on any
> other
> > platform.
> >
> > Cesare
>
> Re-animating an old discussion -- Cesare, any news on the wpython front?
>
> I did a checkout from http://wpython.googlecode.com/svn/trunk and
> was able to ./configure and make successfully on my 64-bit Linux box
> as well as to run the Unladen benchmarks.
>
> Given svn co http://svn.python.org/projects/python/tags/r261 in py261
> and svn co http://wpython.googlecode.com/svn/trunk in wpy,
>
> $ python unladen-tests/perf.py -rm --benchmarks=-2to3,all py261/python
> wpy/python
>
> gives the following results:
>
> Report on Linux foo 2.6.31-14-generic #48-Ubuntu SMP Fri Oct 16
> 14:05:01 UTC 2009 x86_64
> Total CPU cores: 2
>
> ai:
> Min: 0.640516 -> 0.586532: 9.20% faster
> Avg: 0.677346 -> 0.632785: 7.04% faster
> Significant (t=4.336740, a=0.95)
> Stddev: 0.05839 -> 0.08455: 30.94% larger
>
> Mem max: 7412.000 -> 6768.000: 9.52% smaller
> Usage over time: http://tinyurl.com/ykwhmcc
>
>
> call_simple:
> Min: 1.880816 -> 1.701622: 10.53% faster
> Avg: 1.944320 -> 1.778701: 9.31% faster
> Significant (t=14.323045, a=0.95)
> Stddev: 0.09885 -> 0.06000: 64.74% smaller
>
> Mem max: 8100.000 -> 6636.000: 22.06% smaller
> Usage over time: http://tinyurl.com/yzsswgp
>
>
> django:
> Min: 1.287158 -> 1.315700: 2.17% slower
> Avg: 1.330423 -> 1.366978: 2.67% slower
> Significant (t=-4.475769, a=0.95)
> Stddev: 0.05663 -> 0.05885: 3.78% larger
>
> Mem max: 15508.000 -> 16228.000: 4.44% larger
> Usage over time: http://tinyurl.com/yfpbmjn
>
>
> iterative_count:
> Min: 0.211620 -> 0.124646: 69.78% faster
> Avg: 0.222778 -> 0.159868: 39.35% faster
> Significant (t=9.291635, a=0.95)
> Stddev: 0.04239 -> 0.05279: 19.69% larger
>
> Mem max: 7388.000 -> 6680.000: 10.60% smaller
> Usage over time: http://tinyurl.com/yj7s8h4
>
>
> normal_startup:
> Min: 1.060017 -> 0.991366: 6.92% faster
> Avg: 1.189612 -> 1.170067: 1.67% faster
> Significant (t=2.002086, a=0.95)
> Stddev: 0.06942 -> 0.06864: 1.13% smaller
>
> Mem max: 3252.000 -> 4648.000: 30.03% larger
> Usage over time: http://tinyurl.com/ygo3bwt
>
>
> pickle:
> Min: 2.027566 -> 1.948784: 4.04% faster
> Avg: 2.051633 -> 2.043656: 0.39% faster
> Not significant
> Stddev: 0.03095 -> 0.07348: 57.88% larger
>
> Mem max: 8544.000 -> 7340.000: 16.40% smaller
> Usage over time: http://tinyurl.com/ykg9dn2
>
>
> pickle_dict:
> Min: 1.658693 -> 1.656844: 0.11% faster
> Avg: 1.689483 -> 1.698176: 0.51% slower
> Not significant
> Stddev: 0.16945 -> 0.09403: 80.20% smaller
>
> Mem max: 6716.000 -> 7636.000: 12.05% larger
> Usage over time: http://tinyurl.com/yjhyame
>
>
> pickle_list:
> Min: 0.919083 -> 0.894758: 2.72% faster
> Avg: 0.956513 -> 0.921314: 3.82% faster
> Significant (t=2.131237, a=0.95)
> Stddev: 0.12744 -> 0.10506: 21.31% smaller
>
> Mem max: 6804.000 -> 8792.000: 22.61% larger
> Usage over time: http://tinyurl.com/ylc3ezf
>
>
> pybench:
> Min: 58781 -> 50836: 15.63% faster
> Avg: 60009 -> 51788: 15.87% faster
>
> regex_compile:
> Min: 0.934131 -> 0.862323: 8.33% faster
> Avg: 0.962159 -> 0.884848: 8.74% faster
> Significant (t=13.587168, a=0.95)
> Stddev: 0.04685 -> 0.03229: 45.11% smaller
>
> Mem max: 12584.000 -> 12740.000: 1.22% larger
> Usage over time: http://tinyurl.com/yjngu8j
>
>
> regex_effbot:
> Min: 0.130686 -> 0.122483: 6.70% faster
> Avg: 0.143453 -> 0.138078: 3.89% faster
> Not significant
> Stddev: 0.01864 -> 0.03177: 41.32% larger
>
> Mem max: 7652.000 -> 6660.000: 14.89% smaller
> Usage over time: http://tinyurl.com/ykcgntf
>
>
> regex_v8:
> Min: 0.135130 -> 0.150092: 9.97% slower
> Avg: 0.138027 -> 0.177309: 22.15% slower
> Significant (t=-8.197595, a=0.95)
> Stddev: 0.00258 -> 0.04785: 94.60% larger
>
> Mem max: 11124.000 -> 12236.000: 9.09% larger
> Usage over time: http://tinyurl.com/ykb5vzu
>
>
> rietveld:
> Min: 0.848245 -> 0.816473: 3.89% faster
> Avg: 1.033925 -> 1.019889: 1.38% faster
> Not significant
> Stddev: 0.11242 -> 0.13006: 13.56% larger
>
> Mem max: 23792.000 -> 24548.000: 3.08% larger
> Usage over time: http://tinyurl.com/yhdvz5v
>
>
> slowpickle:
> Min: 0.876736 -> 0.800203: 9.56% faster
> Avg: 0.932808 -> 0.870577: 7.15% faster
> Significant (t=5.020426, a=0.95)
> Stddev: 0.05600 -> 0.11059: 49.36% larger
>
> Mem max: 7200.000 -> 7276.000: 1.04% larger
> Usage over time: http://tinyurl.com/ykt2brq
>
>
> slowspitfire:
> Min: 1.029100 -> 0.948458: 8.50% faster
> Avg: 1.062486 -> 1.020777: 4.09% faster
> Significant (t=4.581669, a=0.95)
> Stddev: 0.05441 -> 0.07298: 25.44% larger
>
> Mem max: 139792.000 -> 129264.000: 8.14% smaller
> Usage over time: http://tinyurl.com/yh7vmlh
>
>
> slowunpickle:
> Min: 0.411744 -> 0.356784: 15.40% faster
> Avg: 0.444638 -> 0.393261: 13.06% faster
> Significant (t=7.009269, a=0.95)
> Stddev: 0.04147 -> 0.06044: 31.38% larger
>
> Mem max: 7132.000 -> 7848.000: 9.12% larger
> Usage over time: http://tinyurl.com/yfwvz3g
>
>
> startup_nosite:
> Min: 0.664456 -> 0.598770: 10.97% faster
> Avg: 0.933034 -> 0.761228: 22.57% faster
> Significant (t=20.660776, a=0.95)
> Stddev: 0.09645 -> 0.06728: 43.37% smaller
>
> Mem max: 1940.000 -> 1940.000: -0.00% smaller
> Usage over time: http://tinyurl.com/yzzxcmd
>
>
> threaded_count:
> Min: 0.220059 -> 0.138708: 58.65% faster
> Avg: 0.232347 -> 0.156120: 48.83% faster
> Significant (t=23.804797, a=0.95)
> Stddev: 0.01889 -> 0.02586: 26.96% larger
>
> Mem max: 6460.000 -> 7664.000: 15.71% larger
> Usage over time: http://tinyurl.com/yzm3awu
>
>
> unpack_sequence:
> Min: 0.000129 -> 0.000120: 7.57% faster
> Avg: 0.000218 -> 0.000194: 12.14% faster
> Significant (t=3.946194, a=0.95)
> Stddev: 0.00139 -> 0.00128: 8.13% smaller
>
> Mem max: 18948.000 -> 19056.000: 0.57% larger
> Usage over time: http://tinyurl.com/yf8es3f
>
>
> unpickle:
> Min: 1.191468 -> 1.206198: 1.22% slower
> Avg: 1.248471 -> 1.281957: 2.61% slower
> Significant (t=-2.658526, a=0.95)
> Stddev: 0.05513 -> 0.11325: 51.32% larger
>
> Mem max: 7776.000 -> 8676.000: 10.37% larger
> Usage over time: http://tinyurl.com/yz96gw2
>
>
> unpickle_list:
> Min: 0.922200 -> 0.861167: 7.09% faster
> Avg: 0.955964 -> 0.976829: 2.14% slower
> Not significant
> Stddev: 0.04374 -> 0.21061: 79.23% larger
>
> Mem max: 6820.000 -> 8324.000: 18.07% larger
> Usage over time: http://tinyurl.com/yjbraxg
>
> ---
>
> The diff between the two trees is at
> http://dpaste.org/RpIv/
>
> Best,
> Mart S?mermaa
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> http://mail.python.org/mailman/options/python-dev/cesare.di.mauro%40gmail.com
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20091104/314237d9/attachment-0001.htm>

From guido at python.org  Wed Nov  4 16:06:55 2009
From: guido at python.org (Guido van Rossum)
Date: Wed, 4 Nov 2009 07:06:55 -0800
Subject: [Python-Dev] 2.7 Release? 2.7 == last of the 2.x line?
In-Reply-To: <5CB59A21-D8C9-4BC1-A1F6-4438B86A8BC5@twistedmatrix.com>
References: <E077843B-919A-4AC7-95D7-0313E4D0328A@gmail.com> 
	<4AEFD9AE.7090104@molden.no> <4AEFF450.9060007@voidspace.org.uk> 
	<79EADA413EFB44F6864EF71D5B830088@RaymondLaptop1>
	<79990c6b0911030323x47a9f008ue7256c56e88c0ee8@mail.gmail.com> 
	<5CB59A21-D8C9-4BC1-A1F6-4438B86A8BC5@twistedmatrix.com>
Message-ID: <ca471dc20911040706t22d2cff7j796ade3a8388200@mail.gmail.com>

On Tue, Nov 3, 2009 at 9:51 PM, Glyph Lefkowitz
<glyph at twistedmatrix.com> wrote (amongst way too many words):
> [...]?For example, 2.8 could emit a deprecation
> warning for every old-style class that was defined, 2.9 could emit a
> deprecation warning for every string constant declared without a 'b' or 'u'
> prefix unless the module in question were in "3.x mode" (i.e. no-prefix ==
> 'u').

This proposal is hopelessly naive. It has been considered seriously
from all possible sides before, and there just isn't a way to make
this work. Not even with several releases as stepping points.

-- 
--Guido van Rossum (python.org/~guido)

From pje at telecommunity.com  Wed Nov  4 16:37:26 2009
From: pje at telecommunity.com (P.J. Eby)
Date: Wed, 04 Nov 2009 10:37:26 -0500
Subject: [Python-Dev] 2.7 Release? 2.7 == last of the 2.x line?
In-Reply-To: <5CB59A21-D8C9-4BC1-A1F6-4438B86A8BC5@twistedmatrix.com>
References: <E077843B-919A-4AC7-95D7-0313E4D0328A@gmail.com>
	<4AEFD9AE.7090104@molden.no> <4AEFF450.9060007@voidspace.org.uk>
	<79EADA413EFB44F6864EF71D5B830088@RaymondLaptop1>
	<79990c6b0911030323x47a9f008ue7256c56e88c0ee8@mail.gmail.com>
	<5CB59A21-D8C9-4BC1-A1F6-4438B86A8BC5@twistedmatrix.com>
Message-ID: <20091104153728.C2D043A408C@sparrow.telecommunity.com>

At 12:51 AM 11/4/2009 -0500, Glyph Lefkowitz wrote:
>With the 2.x series, users and operating systems seem to move on
>fairly rapidly, because dependencies generally continue to work if you
>upgrade just one version.  This isn't quite as formal a requirement as
>I would like (warnings get generated, unit tests fail, things do
>break) but in practice, users can rely on it for most functionality.
>If 3.x could be broken into a series of transitions like that, where
>you can upgrade one version, fix some stuff, then upgrade another
>version, even if you couldn't actually support more than 2 versions at
>once, I think that we could pick up the migration pace to the point
>where we might actually be using 3.x syntax in a few years.  Having a
>2.x series which goes to 2.9 and then stops isn't *quite* the same
>thing as having one that moves over continuously to some 3.x version,
>but it does seem to me that by that point the chasm between versions
>will have narrowed to a crack, and the migration will be a little hop
>over it rather than the currently-required great flying leap.

+1 (I actually thought this was the original plan.)


From collinw at gmail.com  Wed Nov  4 16:54:13 2009
From: collinw at gmail.com (Collin Winter)
Date: Wed, 4 Nov 2009 07:54:13 -0800
Subject: [Python-Dev] A wordcode-based Python
In-Reply-To: <ad1f81530911040420q3f3f8976p1fd1295cb84c2637@mail.gmail.com>
References: <49D4DA72.60401@v.loewis.de>
	<20090509143829.17F293A4080@sparrow.telecommunity.com>
	<4A059639.7040505@v.loewis.de>
	<20090509153716.D44633A4080@sparrow.telecommunity.com>
	<7FF9D9A9-211E-4E5D-BDD0-9C0315123975@zooko.com>
	<4c8bd6707712f01ccf3841c2c26169ef@preisshare.net>
	<1024.88.149.182.147.1242064816.squirrel@webmail5.pair.com>
	<43aa6ff70905111414n62d20099r9bb2b3ebd52a26ec@mail.gmail.com>
	<4213.88.149.182.147.1242111241.squirrel@webmail5.pair.com>
	<ad1f81530911040420q3f3f8976p1fd1295cb84c2637@mail.gmail.com>
Message-ID: <43aa6ff70911040754k745015faocadb167e67664432@mail.gmail.com>

On Wed, Nov 4, 2009 at 4:20 AM, Mart S?mermaa <mrts.pydev at gmail.com> wrote:
> On Tue, May 12, 2009 at 8:54 AM, Cesare Di Mauro
> <cesare.dimauro at a-tono.com> wrote:
>>> Also, I checked out wpython at head to run Unladen Swallow's
>>> benchmarks against it, but it refuses to compile with either gcc 4.0.1
>>> or 4.3.1 on Linux (fails in Python/ast.c). I can send you the build
>>> failures off-list, if you're interested.
>>>
>>> Thanks,
>>> Collin Winter
>>
>> I'm very interested, thanks. That's because I worked only on Windows
>> machines, so I definitely need to test and fix it to let it run on any other
>> platform.
>>
>> Cesare
>
> Re-animating an old discussion -- Cesare, any news on the wpython front?
>
> I did a checkout from http://wpython.googlecode.com/svn/trunk and
> was able to ./configure and make successfully on my 64-bit Linux box
> as well as to run the Unladen benchmarks.
>
> Given svn co http://svn.python.org/projects/python/tags/r261 in py261
> and svn co http://wpython.googlecode.com/svn/trunk in wpy,
>
> $ python unladen-tests/perf.py -rm --benchmarks=-2to3,all py261/python
> wpy/python

Do note that the --track_memory option to perf.py imposes some
overhead that interferes with the performance figures. I'd recommend
running the benchmarks again without --track_memory. That extra
overhead is almost certainly what's causing some of the variability in
the results.

Collin Winter

From guido at python.org  Wed Nov  4 17:53:12 2009
From: guido at python.org (Guido van Rossum)
Date: Wed, 4 Nov 2009 08:53:12 -0800
Subject: [Python-Dev] nonlocal keyword in 2.x?
In-Reply-To: <319e029f0911032306j746bc263jcf6d0dfeaba9f24d@mail.gmail.com>
References: <da7032ce0910211856g45e45632n550aa5b4a10f9a62@mail.gmail.com> 
	<Pine.LNX.4.64.0911021405420.21420@kimball.webabinitio.net> 
	<4AEF4C53.7090902@v.loewis.de>
	<Pine.LNX.4.64.0911021624110.21420@kimball.webabinitio.net> 
	<ca471dc20911021333r168e257eja1e5f79dbe4ae234@mail.gmail.com> 
	<loom.20091102T223839-797@post.gmane.org> <4AEFF3A3.3020103@egenix.com>
	<4AF06E20.7050307@v.loewis.de> <4AF0816A.706@egenix.com>
	<319e029f0911032306j746bc263jcf6d0dfeaba9f24d@mail.gmail.com>
Message-ID: <ca471dc20911040853l2c10cf85m5c6a1b17969940d5@mail.gmail.com>

On Tue, Nov 3, 2009 at 11:06 PM, Lennart Regebro <regebro at gmail.com> wrote:
> I should maybe point out that although I'm generally +1 on
> backporting, I'm not specifically anything on backporting the nonlocal
> keyword. There are probably things that would help more from a
> compatibility standpoint than that.
>
> For example, from __future__ import unicode_literals doesn't switch
> the types. So this code will rais an assertion error in 2.6.
>
>>>> from __future__ import unicode_literals
>>>> assert isinstance("fghjkl", str)
>
> While it of course will run fine in 3.0. Now this has a fairly trivial
> workaround or two, but still.
>
> I also would really like to see a real port of the bytes class to 2.6,
> but I have a vague memory that there was some reason that wouldn't
> work.

There are many interfaces that return a str instance which should
become a bytes instance for code that is bytes-aware, but should stay
a str instance (which after all has the same representation) for code
that is not bytes-aware. In 3.x it's easy, such interfaces always
return bytes instances. But in 2.x, it would have to be context-aware,
and there are many reasons why that can't work. (E.g. an object could
be shared between two modules, one of which is bytes-aware, while the
other is not. Or a call could be made by a module that isn't
bytes-aware and the return value passed to a module that is
bytes-aware. Or vice versa. It's a mess unless *everyone* is
bytes-aware, which is the 3.x world.)

-- 
--Guido van Rossum (python.org/~guido)

From mrts.pydev at gmail.com  Wed Nov  4 18:32:07 2009
From: mrts.pydev at gmail.com (=?ISO-8859-1?Q?Mart_S=F5mermaa?=)
Date: Wed, 4 Nov 2009 19:32:07 +0200
Subject: [Python-Dev] A wordcode-based Python
In-Reply-To: <43aa6ff70911040754k745015faocadb167e67664432@mail.gmail.com>
References: <49D4DA72.60401@v.loewis.de> <4A059639.7040505@v.loewis.de>
	<20090509153716.D44633A4080@sparrow.telecommunity.com>
	<7FF9D9A9-211E-4E5D-BDD0-9C0315123975@zooko.com>
	<4c8bd6707712f01ccf3841c2c26169ef@preisshare.net>
	<1024.88.149.182.147.1242064816.squirrel@webmail5.pair.com>
	<43aa6ff70905111414n62d20099r9bb2b3ebd52a26ec@mail.gmail.com>
	<4213.88.149.182.147.1242111241.squirrel@webmail5.pair.com>
	<ad1f81530911040420q3f3f8976p1fd1295cb84c2637@mail.gmail.com>
	<43aa6ff70911040754k745015faocadb167e67664432@mail.gmail.com>
Message-ID: <ad1f81530911040932v23bff594w113eb7d5f80f5011@mail.gmail.com>

On Wed, Nov 4, 2009 at 5:54 PM, Collin Winter <collinw at gmail.com> wrote:
> Do note that the --track_memory option to perf.py imposes some
> overhead that interferes with the performance figures.

Thanks for the notice, without -m/--track_memory the deviation in
results is indeed much smaller.

> I'd recommend
> running the benchmarks again without --track_memory.

Done:

$ python unladen-tests/perf.py -r --benchmarks=-2to3,all py261/python wpy/python

Report on Linux zeus 2.6.31-14-generic #48-Ubuntu SMP Fri Oct 16
14:05:01 UTC 2009 x86_64
Total CPU cores: 2

ai:
Min: 0.629343 -> 0.576259: 9.21% faster
Avg: 0.634689 -> 0.581551: 9.14% faster
Significant (t=39.404870, a=0.95)
Stddev: 0.01259 -> 0.00484: 160.04% smaller


call_simple:
Min: 1.796710 -> 1.700046: 5.69% faster
Avg: 1.801533 -> 1.716367: 4.96% faster
Significant (t=137.452069, a=0.95)
Stddev: 0.00522 -> 0.00333: 56.64% smaller


django:
Min: 1.280840 -> 1.275350: 0.43% faster
Avg: 1.287179 -> 1.287233: 0.00% slower
Not significant
Stddev: 0.01055 -> 0.00581: 81.60% smaller


iterative_count:
Min: 0.211744 -> 0.123271: 71.77% faster
Avg: 0.213148 -> 0.128596: 65.75% faster
Significant (t=88.510311, a=0.95)
Stddev: 0.00233 -> 0.00926: 74.80% larger


normal_startup:
Min: 0.520829 -> 0.516412: 0.86% faster
Avg: 0.559170 -> 0.554678: 0.81% faster
Not significant
Stddev: 0.02031 -> 0.02093: 2.98% larger


pickle:
Min: 1.988127 -> 1.926643: 3.19% faster
Avg: 2.000676 -> 1.936185: 3.33% faster
Significant (t=36.712505, a=0.95)
Stddev: 0.01650 -> 0.00603: 173.67% smaller


pickle_dict:
Min: 1.681116 -> 1.619192: 3.82% faster
Avg: 1.701952 -> 1.629548: 4.44% faster
Significant (t=34.513963, a=0.95)
Stddev: 0.01721 -> 0.01200: 43.46% smaller


pickle_list:
Min: 0.918128 -> 0.884967: 3.75% faster
Avg: 0.925534 -> 0.891200: 3.85% faster
Significant (t=60.451407, a=0.95)
Stddev: 0.00496 -> 0.00276: 80.00% smaller


pybench:
Min: 58692 -> 51128: 14.79% faster
Avg: 59914 -> 52316: 14.52% faster

regex_compile:
Min: 0.894190 -> 0.816447: 9.52% faster
Avg: 0.900353 -> 0.826003: 9.00% faster
Significant (t=24.974080, a=0.95)
Stddev: 0.00448 -> 0.02943: 84.78% larger


regex_effbot:
Min: 0.124442 -> 0.123750: 0.56% faster
Avg: 0.134908 -> 0.126137: 6.95% faster
Significant (t=5.496357, a=0.95)
Stddev: 0.01581 -> 0.00218: 625.68% smaller


regex_v8:
Min: 0.132730 -> 0.143494: 7.50% slower
Avg: 0.134287 -> 0.147387: 8.89% slower
Significant (t=-40.654627, a=0.95)
Stddev: 0.00108 -> 0.00304: 64.34% larger


rietveld:
Min: 0.754050 -> 0.737335: 2.27% faster
Avg: 0.770227 -> 0.754642: 2.07% faster
Significant (t=7.547765, a=0.95)
Stddev: 0.01434 -> 0.01486: 3.49% larger


slowpickle:
Min: 0.858494 -> 0.795162: 7.96% faster
Avg: 0.862350 -> 0.799479: 7.86% faster
Significant (t=133.690989, a=0.95)
Stddev: 0.00394 -> 0.00257: 52.92% smaller


slowspitfire:
Min: 0.955587 -> 0.909843: 5.03% faster
Avg: 0.965960 -> 0.925845: 4.33% faster
Significant (t=16.351067, a=0.95)
Stddev: 0.01237 -> 0.02119: 41.63% larger


slowunpickle:
Min: 0.409312 -> 0.346982: 17.96% faster
Avg: 0.412381 -> 0.349148: 18.11% faster
Significant (t=242.889869, a=0.95)
Stddev: 0.00198 -> 0.00169: 17.61% smaller


startup_nosite:
Min: 0.195620 -> 0.194328: 0.66% faster
Avg: 0.230811 -> 0.238523: 3.23% slower
Significant (t=-3.869944, a=0.95)
Stddev: 0.01932 -> 0.02052: 5.87% larger


threaded_count:
Min: 0.222133 -> 0.133764: 66.06% faster
Avg: 0.236670 -> 0.147750: 60.18% faster
Significant (t=57.472693, a=0.95)
Stddev: 0.01317 -> 0.00813: 61.98% smaller


unpack_sequence:
Min: 0.000129 -> 0.000119: 8.43% faster
Avg: 0.000132 -> 0.000123: 7.22% faster
Significant (t=24.614061, a=0.95)
Stddev: 0.00003 -> 0.00011: 77.02% larger


unpickle:
Min: 1.191255 -> 1.149132: 3.67% faster
Avg: 1.218023 -> 1.162351: 4.79% faster
Significant (t=21.222711, a=0.95)
Stddev: 0.02242 -> 0.01362: 64.54% smaller


unpickle_list:
Min: 0.880991 -> 0.965611: 8.76% slower
Avg: 0.898949 -> 0.985231: 8.76% slower
Significant (t=-17.387537, a=0.95)
Stddev: 0.04838 -> 0.01103: 338.79% smaller

From ssteinerx at gmail.com  Wed Nov  4 19:20:42 2009
From: ssteinerx at gmail.com (ssteinerX@gmail.com)
Date: Wed, 4 Nov 2009 13:20:42 -0500
Subject: [Python-Dev] 2.7 Release? 2.7 == last of the 2.x line?
In-Reply-To: <319e029f0911032206y64386499hf4564839ec9acb1a@mail.gmail.com>
References: <E077843B-919A-4AC7-95D7-0313E4D0328A@gmail.com>
	<F245110A-2FDF-460B-8E0D-87785F0E6DB0@fuhm.net>
	<BBB79ADA-7E31-4DD8-AAA5-07537FA74C5A@gmail.com>
	<319e029f0911032206y64386499hf4564839ec9acb1a@mail.gmail.com>
Message-ID: <B4DEA4E2-E808-4A07-8A9A-AFDEB5A0CD68@gmail.com>


On Nov 4, 2009, at 1:06 AM, Lennart Regebro wrote:

> 2009/11/3 ssteinerX at gmail.com <ssteinerx at gmail.com>:
>>
>> On Nov 2, 2009, at 7:26 PM, James Y Knight wrote:
>>>
>>> It really sounds like you're saying that switching to 3.x isn't  
>>> worth the
>>> cost to you, but you want to force people (including yourself) to  
>>> do so
>>> anyways, because ...?
>>
>> Because that's the future of Python
>
> Or not. Maybe it's a dead branch of Python?

Maybe the 3.x line should just be put out of our misery, merged back  
to 2.7, 2.8, 2.9, and proceed as Glyph suggested in passing with  
increasing levels of deprecation until it just turns into 3.x on its  
own by running out of numbers.

S


From martin at v.loewis.de  Wed Nov  4 19:37:51 2009
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Wed, 04 Nov 2009 19:37:51 +0100
Subject: [Python-Dev] Py3k bytes type in 2.x (Re: nonlocal keyword in
 2.x?)
In-Reply-To: <4AF16CC7.6000705@gmail.com>
References: <da7032ce0910211856g45e45632n550aa5b4a10f9a62@mail.gmail.com>	<ca471dc20911021009u7485adcdgde8fae04a5061059@mail.gmail.com>	<Pine.LNX.4.64.0911021405420.21420@kimball.webabinitio.net>	<4AEF4C53.7090902@v.loewis.de>	<Pine.LNX.4.64.0911021624110.21420@kimball.webabinitio.net>	<ca471dc20911021333r168e257eja1e5f79dbe4ae234@mail.gmail.com>	<loom.20091102T223839-797@post.gmane.org>	<4AEFF3A3.3020103@egenix.com>
	<4AF06E20.7050307@v.loewis.de>	<4AF0816A.706@egenix.com>
	<319e029f0911032306j746bc263jcf6d0dfeaba9f24d@mail.gmail.com>
	<4AF16CC7.6000705@gmail.com>
Message-ID: <4AF1C9FF.2060302@v.loewis.de>

Nick Coghlan wrote:
> Lennart Regebro wrote:
>> I also would really like to see a real port of the bytes class to 2.6,
>> but I have a vague memory that there was some reason that wouldn't
>> work.
> 
> Not so much that it wouldn't work, but that the interfaces to support
> using it effectively really aren't there - lots of areas in the standard
> library needed to be tweaked to cope with bytes objects in 3.x.

I see the problem differently: if a bytes type was added, nothing would
use it. In particular, IO wouldn't start returning bytes (although it
could accept them); IO would continue to return str. Therefore, I'm
skeptical that adding a *third* string type to 3.x would do any good.

Regards,
Martin

From ctrachte at gmail.com  Wed Nov  4 19:39:43 2009
From: ctrachte at gmail.com (Carl Trachte)
Date: Wed, 4 Nov 2009 11:39:43 -0700
Subject: [Python-Dev] 2.7 Release? 2.7 == last of the 2.x line?
In-Reply-To: <B4DEA4E2-E808-4A07-8A9A-AFDEB5A0CD68@gmail.com>
References: <E077843B-919A-4AC7-95D7-0313E4D0328A@gmail.com>
	<F245110A-2FDF-460B-8E0D-87785F0E6DB0@fuhm.net>
	<BBB79ADA-7E31-4DD8-AAA5-07537FA74C5A@gmail.com>
	<319e029f0911032206y64386499hf4564839ec9acb1a@mail.gmail.com>
	<B4DEA4E2-E808-4A07-8A9A-AFDEB5A0CD68@gmail.com>
Message-ID: <426ada670911041039m6df85994v321342231fbd3805@mail.gmail.com>

On 11/4/09, ssteinerX at gmail.com <ssteinerx at gmail.com> wrote:

> Maybe the 3.x line should just be put out of our misery, merged back
> to 2.7, 2.8, 2.9, and proceed as Glyph suggested in passing with
> increasing levels of deprecation until it just turns into 3.x on its
> own by running out of numbers.

<delurk>
As a user, I'm horrified.  Granted, I'm not the most high powered
user, but . . .
my employer is already providing me with a 3.0 Python version on one
of my work computers with the expectation that I'll be using it more
and more.

Sorry to butt in, but is this a joke?  I thought all this was hashed
out prior to inventing python 3.0.
</delurk>

From arcriley at gmail.com  Wed Nov  4 19:43:02 2009
From: arcriley at gmail.com (Arc Riley)
Date: Wed, 4 Nov 2009 13:43:02 -0500
Subject: [Python-Dev] 2.7 Release? 2.7 == last of the 2.x line?
In-Reply-To: <B4DEA4E2-E808-4A07-8A9A-AFDEB5A0CD68@gmail.com>
References: <E077843B-919A-4AC7-95D7-0313E4D0328A@gmail.com> 
	<F245110A-2FDF-460B-8E0D-87785F0E6DB0@fuhm.net>
	<BBB79ADA-7E31-4DD8-AAA5-07537FA74C5A@gmail.com> 
	<319e029f0911032206y64386499hf4564839ec9acb1a@mail.gmail.com> 
	<B4DEA4E2-E808-4A07-8A9A-AFDEB5A0CD68@gmail.com>
Message-ID: <bad82a80911041043x2e57fa56g96a8737a0d3c1fce@mail.gmail.com>

That's not going to happen.  Stop trolling the python-dev list.

On Wed, Nov 4, 2009 at 1:20 PM, ssteinerX at gmail.com <ssteinerx at gmail.com>wrote:

> Maybe the 3.x line should just be put out of our misery, merged back to
> 2.7, 2.8, 2.9, and proceed as Glyph suggested in passing with increasing
> levels of deprecation until it just turns into 3.x on its own by running out
> of numbers.
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20091104/3986dd82/attachment.htm>

From guido at python.org  Wed Nov  4 19:45:59 2009
From: guido at python.org (Guido van Rossum)
Date: Wed, 4 Nov 2009 10:45:59 -0800
Subject: [Python-Dev] 2.7 Release? 2.7 == last of the 2.x line?
In-Reply-To: <426ada670911041039m6df85994v321342231fbd3805@mail.gmail.com>
References: <E077843B-919A-4AC7-95D7-0313E4D0328A@gmail.com> 
	<F245110A-2FDF-460B-8E0D-87785F0E6DB0@fuhm.net>
	<BBB79ADA-7E31-4DD8-AAA5-07537FA74C5A@gmail.com> 
	<319e029f0911032206y64386499hf4564839ec9acb1a@mail.gmail.com> 
	<B4DEA4E2-E808-4A07-8A9A-AFDEB5A0CD68@gmail.com>
	<426ada670911041039m6df85994v321342231fbd3805@mail.gmail.com>
Message-ID: <ca471dc20911041045nf22baa2nb6f6d0d4d4e2960f@mail.gmail.com>

On Wed, Nov 4, 2009 at 10:39 AM, Carl Trachte <ctrachte at gmail.com> wrote:
> On 11/4/09, ssteinerX at gmail.com <ssteinerx at gmail.com> wrote:
>
>> Maybe the 3.x line should just be put out of our misery, merged back
>> to 2.7, 2.8, 2.9, and proceed as Glyph suggested in passing with
>> increasing levels of deprecation until it just turns into 3.x on its
>> own by running out of numbers.
>
> <delurk>
> As a user, I'm horrified. ?Granted, I'm not the most high powered
> user, but . . .
> my employer is already providing me with a 3.0 Python version on one
> of my work computers with the expectation that I'll be using it more
> and more.
>
> Sorry to butt in, but is this a joke? ?I thought all this was hashed
> out prior to inventing python 3.0.
> </delurk>

I have no idea who "ssteinerX" is. He certainly doesn't speak for the
core developers.

-- 
--Guido van Rossum (python.org/~guido)

From martin at v.loewis.de  Wed Nov  4 19:46:39 2009
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Wed, 04 Nov 2009 19:46:39 +0100
Subject: [Python-Dev] 2.7 Release? 2.7 == last of the 2.x line?
In-Reply-To: <loom.20091103T232112-660@post.gmane.org>
References: <E077843B-919A-4AC7-95D7-0313E4D0328A@gmail.com>	<F245110A-2FDF-460B-8E0D-87785F0E6DB0@fuhm.net>	<BBB79ADA-7E31-4DD8-AAA5-07537FA74C5A@gmail.com>	<D99BA4CF-E16C-47EB-9C60-F1EC5C27CDDC@python.org>	<4AEFC887.6020801@v.loewis.de>	<3DE1D8FB-896E-441D-937D-AB63EB3A8E22@python.org>	<5b8d13220911030540s3d700265ib00f4277b7ce4a2b@mail.gmail.com>	<bbaeab100911031153h61c356dfw850c53d4662aca7@mail.gmail.com>	<79990c6b0911031416r36c6947ldd19984108d46f7e@mail.gmail.com>
	<loom.20091103T232112-660@post.gmane.org>
Message-ID: <4AF1CC0F.8090709@v.loewis.de>

Antoine Pitrou wrote:
> Paul Moore <p.f.moore <at> gmail.com> writes:
>> FWIW, I did a quick survey of some packages (a sampling of packages
>> I've used or considered using in the past):
>>
>> Twisted - no plans yet for Python 3
> 
> Well Twisted depends on zope.interface which is not ported yet.

That's not strictly true, see

http://svn.zope.org/zope.interface/branches/regebro-python3/

While this isn't released yet, Lennart and myself have been working
to make it work on 2.x and 3.x. So porting activities *could* start
now (requiring 3.x user to use that branch).

Regards,
Martin

From martin at v.loewis.de  Wed Nov  4 20:02:05 2009
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Wed, 04 Nov 2009 20:02:05 +0100
Subject: [Python-Dev] 2.7 Release? 2.7 == last of the 2.x line?
In-Reply-To: <5CB59A21-D8C9-4BC1-A1F6-4438B86A8BC5@twistedmatrix.com>
References: <E077843B-919A-4AC7-95D7-0313E4D0328A@gmail.com>	<4AEFD9AE.7090104@molden.no>
	<4AEFF450.9060007@voidspace.org.uk>	<79EADA413EFB44F6864EF71D5B830088@RaymondLaptop1>	<79990c6b0911030323x47a9f008ue7256c56e88c0ee8@mail.gmail.com>
	<5CB59A21-D8C9-4BC1-A1F6-4438B86A8BC5@twistedmatrix.com>
Message-ID: <4AF1CFAD.6000305@v.loewis.de>

> For one thing, we have a very long row to hoe here.  The migration to
> 3.0 is a long, tedious process with little tangible benefit.  I hope
> that sometime in the next decade Twisted can accelerate the process of
> dropping old 2.x versions, but I seriously doubt we could do a
> feature-complete 3.1/2.6 version.  I get the general impression that a
> 3.2/2.7 port would be more feasible; hopefully a 3.3/2.8 would be even
> moreso.

Please understand that you will not need to drop 2.x versions in order
to support 3.x. Just add 3.x support now and make sure it won't break
2.x support.

> Also, the benefits of migrating to python 3.x are still negligible, as
> far as I can tell.

For Twisted, most definitely - you will need to support 2.x and 3.x
simultaneously, so you can't really benefit from 3.x-only changes
for a long time to come - perhaps until a 3to2 tool has a good quality,
and probably not even then (since it will restrict you what you can
do in 3.x code).

> On the other hand, you've got NumPy, PyGTK, Unladen Swallow,
> PyPy, Jython, IronPython, and so on and so forth.  Since I started using
> it, the strength of Python has been in its ecosystem, and the 3.x
> ecosystem is not yet viable.

Right - the advantages wouldn't be for Twisted itself, but for users
of Twisted, which would see a larger ecosystem if Twisted was available.

> The main reason I want a long 2.x series is that I believe it would more
> easily allow us infrastructure folks to drop support for *older*
> versions.  With this big 2.x->3.x chasm, I can't really see an end in
> sight for Twisted using Python 2.x as its _source_ language, translating
> with 2to3.

Well, 3to2 would then be an option for you: use Python 3 as the source
language.

> Some projects which depend on Twisted and want new versions
> (and security fixes, etc) are going to want Python 2.x for a really long
> time.  Maybe they're just really conservative, maybe they don't have a
> lot of maintenance energy, or maybe they have other dependencies which
> haven't got a port; it doesn't really matter, empirically speaking
> people want older versions of Python.

But wouldn't these applications also break as Twisted drops support
for old 2.x versions, and the applications fail to work on the newer
2.x version (say, 2.34)?

> Keep in mind also that the 2.x translation process is extremely slow and
> results in a clunky development process.  There's no '2to3
> --interactive' commandline that lets me type python 2 at a >>> prompt
> and get python 3 results out so that I can try experiments on the 3.x
> interpreter; I have to actually put my experiments into my unit tests
> and wait 10 minutes to see if it works.  It's like writing C++.

That's not my experience. I see a change in source (say, on Django)
available for 3.x within 5 seconds.

Regards,
Martin

From martin at v.loewis.de  Wed Nov  4 20:08:58 2009
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Wed, 04 Nov 2009 20:08:58 +0100
Subject: [Python-Dev] No buildbot to test wide unicode?
In-Reply-To: <e27efe130911031557n7839ab25g28fc6dafe3174ab7@mail.gmail.com>
References: <e27efe130911031557n7839ab25g28fc6dafe3174ab7@mail.gmail.com>
Message-ID: <4AF1D14A.6040007@v.loewis.de>

> It seems that there is no buildbot to test a wide unicode build.
> 
> On http://www.python.org/dev/buildbot/3.x/ all outputs of the "configure"
> steps show this message::
>   checking what type to use for str... unsigned short
> which looks like a ucs2 build to me.
> 
> Since wide unicode is the standard chosen by some Linux distributions,
> it would make sense to have at least one buildbot running with
> --with-wide-unicode (3.x) or --enable-unicode=ucs4 (2.x).

Can you propose some (one? two? more?) systems that might be best as
candidates? I'd then setup two (sets of) builders; they would share
the slave lock, so builds would run sequentially (unless the slave
operator agrees to setup two slaves on one machine).

Regards,
Martin

From ssteinerx at gmail.com  Wed Nov  4 20:13:48 2009
From: ssteinerx at gmail.com (ssteinerX@gmail.com)
Date: Wed, 4 Nov 2009 14:13:48 -0500
Subject: [Python-Dev] 2.7 Release? 2.7 == last of the 2.x line?
In-Reply-To: <426ada670911041039m6df85994v321342231fbd3805@mail.gmail.com>
References: <E077843B-919A-4AC7-95D7-0313E4D0328A@gmail.com>
	<F245110A-2FDF-460B-8E0D-87785F0E6DB0@fuhm.net>
	<BBB79ADA-7E31-4DD8-AAA5-07537FA74C5A@gmail.com>
	<319e029f0911032206y64386499hf4564839ec9acb1a@mail.gmail.com>
	<B4DEA4E2-E808-4A07-8A9A-AFDEB5A0CD68@gmail.com>
	<426ada670911041039m6df85994v321342231fbd3805@mail.gmail.com>
Message-ID: <404F0418-E18D-4F44-9F8A-31D8EF38B601@gmail.com>


On Nov 4, 2009, at 1:39 PM, Carl Trachte wrote:

> On 11/4/09, ssteinerX at gmail.com <ssteinerx at gmail.com> wrote:
>
>> Maybe the 3.x line should just be put out of our misery, merged back
>> to 2.7, 2.8, 2.9, and proceed as Glyph suggested in passing with
>> increasing levels of deprecation until it just turns into 3.x on its
>> own by running out of numbers.
>
> <delurk>
> As a user, I'm horrified.  Granted, I'm not the most high powered
> user, but . . .
> my employer is already providing me with a 3.0 Python version on one
> of my work computers with the expectation that I'll be using it more
> and more.
>
> Sorry to butt in, but is this a joke?  I thought all this was hashed
> out prior to inventing python 3.0.

Yes, of course it was a joke.

2.7 won't "turn into" Python 3.x any more that Perl will turn into Ruby.

Oh, wait, maybe that was a bad example.

The point was, that Python 3.x does not seem to be something that can  
be "evolved" into and, all along, I have been suggesting that, if  
Python 3.x is the future, let's let 2.7 be the last of the 2.x series,  
backport whatever will make it easiest to make 2to3 do as much of the  
work as possible, and just decide that 2.7 is the end of the line.

I shudder to think how much time has been spent hacking things around  
to make them compatible with the 2.x series while trying to move to 3.x.

If 2.x is over, let it be over and let's all focus on moving into  
Python 3.x with no more time doing other than bug-fixes on 2.x  
versions of things.

S


From cournape at gmail.com  Wed Nov  4 20:15:59 2009
From: cournape at gmail.com (David Cournapeau)
Date: Thu, 5 Nov 2009 04:15:59 +0900
Subject: [Python-Dev] 2.7 Release? 2.7 == last of the 2.x line?
In-Reply-To: <4AF1CFAD.6000305@v.loewis.de>
References: <E077843B-919A-4AC7-95D7-0313E4D0328A@gmail.com>
	<4AEFD9AE.7090104@molden.no> <4AEFF450.9060007@voidspace.org.uk>
	<79EADA413EFB44F6864EF71D5B830088@RaymondLaptop1>
	<79990c6b0911030323x47a9f008ue7256c56e88c0ee8@mail.gmail.com>
	<5CB59A21-D8C9-4BC1-A1F6-4438B86A8BC5@twistedmatrix.com>
	<4AF1CFAD.6000305@v.loewis.de>
Message-ID: <5b8d13220911041115y207a7625j65007a6e6fe7a9a7@mail.gmail.com>

On Thu, Nov 5, 2009 at 4:02 AM, "Martin v. L?wis" <martin at v.loewis.de> wrote:

>
> That's not my experience. I see a change in source (say, on Django)
> available for 3.x within 5 seconds.

This is for which version of 2to3 ? I got similar experience (several
minutes), but maybe I am using 2to3 the wrong way. On my machine, with
2to3 from 3.1.1, it takes ~ 1s to convert one single file of 200
lines, and converting a tiny subset of numpy takes > one minute.

David

From mbk.lists at gmail.com  Wed Nov  4 20:30:47 2009
From: mbk.lists at gmail.com (Mike Krell)
Date: Wed, 4 Nov 2009 12:30:47 -0700
Subject: [Python-Dev] 2.7 Release? 2.7 == last of the 2.x line?
In-Reply-To: <4AF1CFAD.6000305@v.loewis.de>
References: <E077843B-919A-4AC7-95D7-0313E4D0328A@gmail.com>
	<4AEFD9AE.7090104@molden.no> <4AEFF450.9060007@voidspace.org.uk>
	<79EADA413EFB44F6864EF71D5B830088@RaymondLaptop1>
	<79990c6b0911030323x47a9f008ue7256c56e88c0ee8@mail.gmail.com>
	<5CB59A21-D8C9-4BC1-A1F6-4438B86A8BC5@twistedmatrix.com>
	<4AF1CFAD.6000305@v.loewis.de>
Message-ID: <da7032ce0911041130h40ebd3adu49cd9b08734ce74f@mail.gmail.com>

On Wed, Nov 4, 2009 at 12:02 PM, "Martin v. L?wis" <martin at v.loewis.de>wrote:


> > The main reason I want a long 2.x series is that I believe it would more
> > easily allow us infrastructure folks to drop support for *older*
> > versions.  With this big 2.x->3.x chasm, I can't really see an end in
> > sight for Twisted using Python 2.x as its _source_ language, translating
> > with 2to3.
>
> Well, 3to2 would then be an option for you: use Python 3 as the source
> language.
>

A migration path which would be made all the more compelling with the
addition of the nonlocal keyword to 2.7 ;-)

    Mike
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20091104/1ca4b659/attachment.htm>

From martin at v.loewis.de  Wed Nov  4 20:56:27 2009
From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=)
Date: Wed, 04 Nov 2009 20:56:27 +0100
Subject: [Python-Dev] 2.7 Release? 2.7 == last of the 2.x line?
In-Reply-To: <5b8d13220911041115y207a7625j65007a6e6fe7a9a7@mail.gmail.com>
References: <E077843B-919A-4AC7-95D7-0313E4D0328A@gmail.com>	
	<4AEFD9AE.7090104@molden.no> <4AEFF450.9060007@voidspace.org.uk>	
	<79EADA413EFB44F6864EF71D5B830088@RaymondLaptop1>	
	<79990c6b0911030323x47a9f008ue7256c56e88c0ee8@mail.gmail.com>	
	<5CB59A21-D8C9-4BC1-A1F6-4438B86A8BC5@twistedmatrix.com>	
	<4AF1CFAD.6000305@v.loewis.de>
	<5b8d13220911041115y207a7625j65007a6e6fe7a9a7@mail.gmail.com>
Message-ID: <4AF1DC6B.1000305@v.loewis.de>

>> That's not my experience. I see a change in source (say, on Django)
>> available for 3.x within 5 seconds.
> 
> This is for which version of 2to3 ? I got similar experience (several
> minutes), but maybe I am using 2to3 the wrong way. On my machine, with
> 2to3 from 3.1.1, it takes ~ 1s to convert one single file of 200
> lines, and converting a tiny subset of numpy takes > one minute.

The version released with 3.1. The trick is not to recompile all 2to3
code every time you make a source change. Instead, cache the 2to3 output
in the build area, and have setup.py only invoke 2to3 for the files
that got modified.

So whenever I make a change, I do "python3 setup.py install". This
checks all timestamps, finds the modified files (which will only be
one), runs 2to3 on it, and then copies it into my 3.1 installation,
where I can test the change. Recompiling a single file typically takes a
few seconds, or less. It would be possible to also run out of the build
area; you still would need to run "setup.py build" after every change.

There is already support for this in both distutils (as released in
3.1), and distribute.

Regards,
Martin

From alexandre at peadrop.com  Wed Nov  4 21:19:30 2009
From: alexandre at peadrop.com (Alexandre Vassalotti)
Date: Wed, 4 Nov 2009 15:19:30 -0500
Subject: [Python-Dev] PEP 3003 - Python Language Moratorium
In-Reply-To: <ca471dc20911030935l25251e0diec64427f5d7bcbf2@mail.gmail.com>
References: <ca471dc20911030935l25251e0diec64427f5d7bcbf2@mail.gmail.com>
Message-ID: <acd65fa20911041219g4d532601ha153c6ce3f0a364c@mail.gmail.com>

On Tue, Nov 3, 2009 at 12:35 PM, Guido van Rossum <guido at python.org> wrote:
> I've checked draft (!) PEP 3003, "Python Language Moratorium", into
> SVN. As authors I've listed Jesse, Brett and myself.
>

+1 from me.

-- Alexandre

From debatem1 at gmail.com  Wed Nov  4 22:32:39 2009
From: debatem1 at gmail.com (geremy condra)
Date: Wed, 4 Nov 2009 16:32:39 -0500
Subject: [Python-Dev] Retrieve an arbitrary element from a set
	withoutremoving it
In-Reply-To: <200911042234.36796.steve@pearwood.info>
References: <7b0045ed0910301843y7059d316s8a851142b78b7aeb@mail.gmail.com>
	<200911040946.15805.steve@pearwood.info>
	<4AF0D0D7.3090403@canterbury.ac.nz>
	<200911042234.36796.steve@pearwood.info>
Message-ID: <f3cc57c60911041332v479cc3b0t19d9601e52002128@mail.gmail.com>

On Wed, Nov 4, 2009 at 6:34 AM, Steven D'Aprano <steve at pearwood.info> wrote:
> On Wed, 4 Nov 2009 11:54:47 am Greg Ewing wrote:
>> Steven D'Aprano wrote:
>> > I don't know how expensive it is to create a set iterator,
>>
>> Not expensive enough to justify burdening the set type with
>> extra functionality that will be extremely rarely used.
>
> As my previous posts on this topic tried to convey, this isn't primarily
> about efficiency, but about discoverability and obviousness.
>
> Anyway, given the level of opposition to the suggestion, I'm no longer
> willing to carry the flag for it. If anyone else -- perhaps the OP --
> feels they want to take it any further, be my guest.
>
>
>
> --
> Steven D'Aprano

I've said before that I'd like there to be one, standard way of
doing this. A function call- set.pick() seems reasonably named
to me- is probably the cleanest way to do that. Absent that,
an example in the docs that illustrates the preferred idiom
would be great. Is there any kind of consensus on either point?

Geremy Condra

From brett at python.org  Wed Nov  4 23:13:41 2009
From: brett at python.org (Brett Cannon)
Date: Wed, 4 Nov 2009 14:13:41 -0800
Subject: [Python-Dev] 2.7 Release? 2.7 == last of the 2.x line?
In-Reply-To: <B4DEA4E2-E808-4A07-8A9A-AFDEB5A0CD68@gmail.com>
References: <E077843B-919A-4AC7-95D7-0313E4D0328A@gmail.com> 
	<F245110A-2FDF-460B-8E0D-87785F0E6DB0@fuhm.net>
	<BBB79ADA-7E31-4DD8-AAA5-07537FA74C5A@gmail.com> 
	<319e029f0911032206y64386499hf4564839ec9acb1a@mail.gmail.com> 
	<B4DEA4E2-E808-4A07-8A9A-AFDEB5A0CD68@gmail.com>
Message-ID: <bbaeab100911041413m2f3073f2r13f1df8c9c06a5b@mail.gmail.com>

On Wed, Nov 4, 2009 at 10:20, ssteinerX at gmail.com <ssteinerx at gmail.com> wrote:
>
> On Nov 4, 2009, at 1:06 AM, Lennart Regebro wrote:
>
>> 2009/11/3 ssteinerX at gmail.com <ssteinerx at gmail.com>:
>>>
>>> On Nov 2, 2009, at 7:26 PM, James Y Knight wrote:
>>>>
>>>> It really sounds like you're saying that switching to 3.x isn't worth
>>>> the
>>>> cost to you, but you want to force people (including yourself) to do so
>>>> anyways, because ...?
>>>
>>> Because that's the future of Python
>>
>> Or not. Maybe it's a dead branch of Python?
>
> Maybe the 3.x line should just be put out of our misery, merged back to 2.7,
> 2.8, 2.9, and proceed as Glyph suggested in passing with increasing levels
> of deprecation until it just turns into 3.x on its own by running out of
> numbers.
>

I am going to say this once: we are not killing off Python 3.

First off, Python 3 is not even a year old! Considering people have
not fully migrated to 2.6, should we kill it off as well? There is a
certain lack of perspective on time scale. This is especially true
when Guido himself has said on multiple occasions that moving the
community to 3.x would be a mult-year process, as in 3-5 years
process, not 11 months.

Second, the people calling for us to potentially kill 3.x and just
keep 2.x floating along have yet to say that they have tried porting
their code and that it was difficult. Every person who has stepped
forward stating they have done a port has said it was actually
relatively straight-forward. Not only that, we have anecdotal evidence
from multiple people that you can support code way back to whatever
old version of Python RHEL is running.

Third, the same people calling for the death of 3.x have not suggested
they have used it extensively (if at all). I have yet to hear anyone
say that 3.x is not at least a nice improvement, if not a huge one. I
for one find 3.x more enjoyable to work in than 2.x, and that's saying
a lot since I obviously loved Python 2.x enough to get involved in its
development. I have also never heard anyone ever say, "I gave 3.x a
fair shake and honestly, I wish I had not wasted the time." Wait until
3to2 gets to a good state (which will happen; it's my next project --
after I either get us moved to Hg or I simply give up on it -- and I
know I am not the only core developer planning on making it happen).

I realize that there is some fear that it will be time wasted if
people port their code to 3.x if it somehow burns out. But do you
honestly think that python-dev would leave you hanging like that?
Let's take a worst-case scenario here and say that direct pick-up of
3.x after a couple years never happens. Fine, we then begin to
backport features. But if you already ported your code then chances
are you already support the new features. And you know what one of the
first things we would back port would be? Unicode strings and bytes.
And since that is the hardest thing to port to, you will have not
wasted any time.

In other words the calling for the death of 3.x is rather premature
and honestly unfair until people have actually tried to port their
code in earnest and it has been a couple of years for the community to
catch up to what python-dev is pushing out the door (which always
takes a while no matter what minor version has been released).

-Brett

From david.lyon at preisshare.net  Wed Nov  4 23:16:22 2009
From: david.lyon at preisshare.net (David Lyon)
Date: Wed, 04 Nov 2009 17:16:22 -0500
Subject: [Python-Dev] Refactoring installation schemes
In-Reply-To: <94bdd2610911040429j50a6df87m51aac2464dd13bd7@mail.gmail.com>
References: <94bdd2610910271455s7854e6a9u94e3aeeeacb69c02@mail.gmail.com>
	<4AE8206A.9020108@egenix.com> <4AE824DA.4090701@voidspace.org.uk>
	<ED004FA5-9F77-4215-B06D-6CBECEF6D023@gmail.com>
	<94bdd2610910281105q328bc3a5qaed020ef3afe28df@mail.gmail.com>
	<94bdd2610911040429j50a6df87m51aac2464dd13bd7@mail.gmail.com>
Message-ID: <c77acedd9a48dd7036cca55fde42de0a@preisshare.net>

On Wed, 4 Nov 2009 13:29:35 +0100, Tarek Ziad? <ziade.tarek at gmail.com>
wrote:
> I've started to refactor the code in a module I have called
> "sysconfig", reusing distutils/sysconfig, distutils/command/install
> and site code.
> 
> This "sysconfig" module should provide at the end very useful APIs to
> query the current Python environment.
> 
> That's a work in progress but: I've started a branch at
> /python/branches/tarek_sysconfig so it's easier to get some feedbacks
> if anyone want to help on this.

Good job so far. Keep going.

imho a refactoring of the installation and build schemes of distutils
is due. It seems like you easily have the skills to do it.

My advice would be to do it gradually, as you are. And focus on
simplification, and filling in the functional gaps. Ask people what
the functional gaps are on their platforms and try to mould it
together in an unplatform specific way.

Myself and others can assist with this, but best to do it on 
distutils-sig.

I would even go so far as to use the python 3 as a carrot for
the new work.

imho The biggest and best thing that you could do for python
packaging is to do a refactor of the .EGG format.

What I mean here, is *take* the egg, and run with it. Modernise
it and revamp it into a platform independent thing. People
'get' the egg idea, they just hate the current implementation.

What we all *need* is for the egg to be what the source
distribution now is. Not for it to be some proprietory out
out of python object.
 
Call it the new Tarek egg...

Then refactor distutils build to focus on good egg creation..

Many people will help you.. including myself..

David









From ben+python at benfinney.id.au  Wed Nov  4 23:23:33 2009
From: ben+python at benfinney.id.au (Ben Finney)
Date: Thu, 05 Nov 2009 09:23:33 +1100
Subject: [Python-Dev] 2to3,
	3to2: official status (was: 2.7 Release? 2.7 == last of the 2.x
	line?)
References: <E077843B-919A-4AC7-95D7-0313E4D0328A@gmail.com>
	<4AEFD9AE.7090104@molden.no> <4AEFF450.9060007@voidspace.org.uk>
	<79EADA413EFB44F6864EF71D5B830088@RaymondLaptop1>
	<79990c6b0911030323x47a9f008ue7256c56e88c0ee8@mail.gmail.com>
	<5CB59A21-D8C9-4BC1-A1F6-4438B86A8BC5@twistedmatrix.com>
	<4AF1CFAD.6000305@v.loewis.de>
Message-ID: <87skcuc4ka.fsf_-_@benfinney.id.au>

"Martin v. L?wis" <martin at v.loewis.de> writes:

> Well, 3to2 would then be an option for you: use Python 3 as the source
> language.

I was under the impression that 2to3 was officially supported as part of
Python, but 3to2 was a third-party tool. What's the status of 3to2 now?
Is it an official part of Python?

-- 
 \           ?A free society is one where it is safe to be unpopular.? |
  `\                                            ?Adlai Ewing Stevenson |
_o__)                                                                  |
Ben Finney


From brett at python.org  Wed Nov  4 23:28:57 2009
From: brett at python.org (Brett Cannon)
Date: Wed, 4 Nov 2009 14:28:57 -0800
Subject: [Python-Dev] 2to3,
	3to2: official status (was: 2.7 Release? 2.7 == last of the 2.x
	line?)
In-Reply-To: <87skcuc4ka.fsf_-_@benfinney.id.au>
References: <E077843B-919A-4AC7-95D7-0313E4D0328A@gmail.com> 
	<4AEFD9AE.7090104@molden.no> <4AEFF450.9060007@voidspace.org.uk> 
	<79EADA413EFB44F6864EF71D5B830088@RaymondLaptop1>
	<79990c6b0911030323x47a9f008ue7256c56e88c0ee8@mail.gmail.com> 
	<5CB59A21-D8C9-4BC1-A1F6-4438B86A8BC5@twistedmatrix.com>
	<4AF1CFAD.6000305@v.loewis.de> <87skcuc4ka.fsf_-_@benfinney.id.au>
Message-ID: <bbaeab100911041428j3d3ef55fv8465375643c5df20@mail.gmail.com>

On Wed, Nov 4, 2009 at 14:23, Ben Finney <ben+python at benfinney.id.au> wrote:
> "Martin v. L?wis" <martin at v.loewis.de> writes:
>
>> Well, 3to2 would then be an option for you: use Python 3 as the source
>> language.
>
> I was under the impression that 2to3 was officially supported as part of
> Python, but 3to2 was a third-party tool. What's the status of 3to2 now?
> Is it an official part of Python?

Nope, third-party while it continues to mature.

-Brett

From fdrake at acm.org  Wed Nov  4 23:40:35 2009
From: fdrake at acm.org (Fred Drake)
Date: Wed, 4 Nov 2009 17:40:35 -0500
Subject: [Python-Dev] Refactoring installation schemes
In-Reply-To: <c77acedd9a48dd7036cca55fde42de0a@preisshare.net>
References: <94bdd2610910271455s7854e6a9u94e3aeeeacb69c02@mail.gmail.com> 
	<4AE8206A.9020108@egenix.com> <4AE824DA.4090701@voidspace.org.uk> 
	<ED004FA5-9F77-4215-B06D-6CBECEF6D023@gmail.com>
	<94bdd2610910281105q328bc3a5qaed020ef3afe28df@mail.gmail.com> 
	<94bdd2610911040429j50a6df87m51aac2464dd13bd7@mail.gmail.com> 
	<c77acedd9a48dd7036cca55fde42de0a@preisshare.net>
Message-ID: <9cee7ab80911041440j43e51e39qc77a426547d02bcb@mail.gmail.com>

On Wed, Nov 4, 2009 at 5:16 PM, David Lyon <david.lyon at preisshare.net> wrote:
> I would even go so far as to use the python 3 as a carrot for
> the new work.

The packaging story is in such bad shape that it needs the work
regardless, and keeping it to Python 3 would significantly reduce the
set of potential volunteers.

> imho The biggest and best thing that you could do for python
> packaging is to do a refactor of the .EGG format.
>
> What I mean here, is *take* the egg, and run with it. Modernise
> it and revamp it into a platform independent thing. People
> 'get' the egg idea, they just hate the current implementation.

There's certainly work on that, but... is it that people hate the
format?  Or working with setuptools?

The fact that there's more than one egg format doesn't help, so you
may be right.

> Call it the new Tarek egg...

The tegg?  ;-)


  -Fred

-- 
Fred L. Drake, Jr.    <fdrake at gmail.com>
"Chaos is the score upon which reality is written." --Henry Miller

From martin at v.loewis.de  Wed Nov  4 23:41:49 2009
From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=)
Date: Wed, 04 Nov 2009 23:41:49 +0100
Subject: [Python-Dev] 2to3,	3to2: official status
In-Reply-To: <87skcuc4ka.fsf_-_@benfinney.id.au>
References: <E077843B-919A-4AC7-95D7-0313E4D0328A@gmail.com>	<4AEFD9AE.7090104@molden.no>
	<4AEFF450.9060007@voidspace.org.uk>	<79EADA413EFB44F6864EF71D5B830088@RaymondLaptop1>	<79990c6b0911030323x47a9f008ue7256c56e88c0ee8@mail.gmail.com>	<5CB59A21-D8C9-4BC1-A1F6-4438B86A8BC5@twistedmatrix.com>	<4AF1CFAD.6000305@v.loewis.de>
	<87skcuc4ka.fsf_-_@benfinney.id.au>
Message-ID: <4AF2032D.5050406@v.loewis.de>

Ben Finney wrote:
> "Martin v. L?wis" <martin at v.loewis.de> writes:
> 
>> Well, 3to2 would then be an option for you: use Python 3 as the source
>> language.
> 
> I was under the impression that 2to3 was officially supported as part of
> Python, but 3to2 was a third-party tool. What's the status of 3to2 now?
> Is it an official part of Python?

No, the status is exactly as you describe it.

Regards,
Martin


From zookog at gmail.com  Wed Nov  4 23:54:44 2009
From: zookog at gmail.com (Zooko O'Whielacronx)
Date: Wed, 4 Nov 2009 15:54:44 -0700
Subject: [Python-Dev] 2.7 Release? 2.7 == last of the 2.x line?
In-Reply-To: <93A0C3CE-3388-49B9-B994-8AB84E6F48B8@seanet.com>
References: <261064E43172924BBADB00DA14BB4B672A14855D01@XCH-NW-14V.nw.nos.boeing.com>
	<93A0C3CE-3388-49B9-B994-8AB84E6F48B8@seanet.com>
Message-ID: <cd6401a0911041454i3dbf6f65w946e33d88b2ed9d1@mail.gmail.com>

Folks:

It occurred to me to wonder why I haven't investigated how hard it
would be to make my Python packages Python-3-compatible.  That's right
-- I haven't even looked closely.  I couldn't even tell you off the
top of my head what is in Python 3 that I would have to think about
except for the new unicode regime.  I think the answer is that the
payoff is just *so* low to me at this point that it doesn't even
justify me taking 15 minutes to read "What's New In Python 3" or to
execute 2to3 on my smallest package and see what it does.

On the other hand, I'm totally committed to supporting Python 2.7,
because my customers will demand it and because I expect that it will
be easy.

So, if you guys slip in your favorite new Python 3 feature into 2.7
and add a deprecation warning for your least favorite Python 2
misfeature, then probably within about 24 months I'll have fixed all
code that uses the deprecated feature, and probably within about five
years I'll consider dropping backwards-compatibility with Python 2.6
and starting to use that new feature that you added to Python 2.7.

(I'm currently considering dropping Python 2.4 compatibility for the
next releases of most of my code.)

Regards,

Zooko

From p.f.moore at gmail.com  Thu Nov  5 00:23:31 2009
From: p.f.moore at gmail.com (Paul Moore)
Date: Wed, 4 Nov 2009 23:23:31 +0000
Subject: [Python-Dev] 2.7 Release? 2.7 == last of the 2.x line?
In-Reply-To: <cd6401a0911041454i3dbf6f65w946e33d88b2ed9d1@mail.gmail.com>
References: <261064E43172924BBADB00DA14BB4B672A14855D01@XCH-NW-14V.nw.nos.boeing.com>
	<93A0C3CE-3388-49B9-B994-8AB84E6F48B8@seanet.com>
	<cd6401a0911041454i3dbf6f65w946e33d88b2ed9d1@mail.gmail.com>
Message-ID: <79990c6b0911041523q514dd227jc71eb7809ebbf06e@mail.gmail.com>

2009/11/4 Zooko O'Whielacronx <zookog at gmail.com>:
> On the other hand, I'm totally committed to supporting Python 2.7,
> because my customers will demand it and because I expect that it will
> be easy.

Why do you think your customers will demand 2.7 support but not 3.1
support? If I were one of your customers, I'd want 3.1 support...

Paul.

From floris.bruynooghe at gmail.com  Thu Nov  5 00:27:43 2009
From: floris.bruynooghe at gmail.com (Floris Bruynooghe)
Date: Wed, 4 Nov 2009 23:27:43 +0000
Subject: [Python-Dev] 2.7 Release? 2.7 == last of the 2.x line?
In-Reply-To: <cd6401a0911041454i3dbf6f65w946e33d88b2ed9d1@mail.gmail.com>
References: <261064E43172924BBADB00DA14BB4B672A14855D01@XCH-NW-14V.nw.nos.boeing.com>
	<93A0C3CE-3388-49B9-B994-8AB84E6F48B8@seanet.com>
	<cd6401a0911041454i3dbf6f65w946e33d88b2ed9d1@mail.gmail.com>
Message-ID: <20091104232743.GD19537@laurie.devork>

On Wed, Nov 04, 2009 at 03:54:44PM -0700, Zooko O'Whielacronx wrote:
> It occurred to me to wonder why I haven't investigated how hard it
> would be to make my Python packages Python-3-compatible.  That's right
> -- I haven't even looked closely.  I couldn't even tell you off the
> top of my head what is in Python 3 that I would have to think about
> except for the new unicode regime.  I think the answer is that the
> payoff is just *so* low to me at this point that it doesn't even
> justify me taking 15 minutes to read "What's New In Python 3" or to
> execute 2to3 on my smallest package and see what it does.

<cynical mode>

You have time to read this thread but no time to read "What's New In
Python 3"?

</cynical mode>

Personally I found porting to Python 3 a rather pleasant experience
(include C extension module).  I can't wait until I can drop support
for Python 2.2-2.X.

Regards
Floris

PS: I have to admit that the commerial code base I work on is still at
Python 2.5, but that doesn't make me worry in any way.  It'll get to
Python 3 in time (it's running on 2.6 already in development).

-- 
Debian GNU/Linux -- The Power of Freedom
www.debian.org | www.gnu.org | www.kernel.org

From aahz at pythoncraft.com  Thu Nov  5 00:53:25 2009
From: aahz at pythoncraft.com (Aahz)
Date: Wed, 4 Nov 2009 15:53:25 -0800
Subject: [Python-Dev] PyCon 2010: Poster sessions
Message-ID: <20091104235325.GA936@panix.com>

PyCon 2010: Poster sessions
===============================================================

Due date: November 30, 2009

PyCon 2010 introduces a new type of presentation, the poster session.
Poster sessions consist of two pieces:

* A display space where you can put up information about a topic

* Live Q&A during a plenary timeslot where people can get more
information from you while you stand next to your display

For more information and to submit a poster proposal, visit
http://us.pycon.org/2010/conference/posters/
-- 
Aahz (aahz at pythoncraft.com)           <*>         http://www.pythoncraft.com/

[on old computer technologies and programmers]  "Fancy tail fins on a
brand new '59 Cadillac didn't mean throwing out a whole generation of
mechanics who started with model As."  --Andrew Dalke

From david.lyon at preisshare.net  Thu Nov  5 01:13:17 2009
From: david.lyon at preisshare.net (David Lyon)
Date: Wed, 04 Nov 2009 19:13:17 -0500
Subject: [Python-Dev] Refactoring installation schemes
In-Reply-To: <9cee7ab80911041440j43e51e39qc77a426547d02bcb@mail.gmail.com>
References: <94bdd2610910271455s7854e6a9u94e3aeeeacb69c02@mail.gmail.com>
	<4AE8206A.9020108@egenix.com> <4AE824DA.4090701@voidspace.org.uk>
	<ED004FA5-9F77-4215-B06D-6CBECEF6D023@gmail.com>
	<94bdd2610910281105q328bc3a5qaed020ef3afe28df@mail.gmail.com>
	<94bdd2610911040429j50a6df87m51aac2464dd13bd7@mail.gmail.com>
	<c77acedd9a48dd7036cca55fde42de0a@preisshare.net>
	<9cee7ab80911041440j43e51e39qc77a426547d02bcb@mail.gmail.com>
Message-ID: <39deb0d1eef08419dad65d7354a8ffb1@preisshare.net>

On Wed, 4 Nov 2009 17:40:35 -0500, Fred Drake <fdrake at acm.org> wrote:

> The packaging story is in such bad shape that it needs the work
> regardless, and keeping it to Python 3 would significantly reduce the
> set of potential volunteers.

Well I guess that is a 'marketing decision'. Not a coding issue.

Actually, I don't honestly think that there is a lack of volounteers
or any lack of resources.

The main challenge imho is to get peoples bikesheds (including my own
bikeshed) to line up in such a way that people can ride together
sharing some sort of parts. 

Tarek has been doing just fine at facilitating this.

> There's certainly work on that, but... is it that people hate the
> format?  Or working with setuptools?

In my experience, working with the setuptools implementation is
the difficult part. There's a barrage of impossible to remember
command line options and the documentation is imho more convoluted
than it needs to be.

Let me put the distutils documentation forward as some sort of
reference as to where the setuptools documentation should be.

Then, there are some relatively minor issues that just annoy
users to no end. The simple one is that they live in your
site-packages directory unpacked. People wouldn't have so
many issues if they lived like every other 'normal' package.

As for the format itself, there's nothing to hate that I
can see.

> The fact that there's more than one egg format doesn't help, so you
> may be right.

Yes. The implementation can be confusing. That's the only
problem imo.

>> Call it the new Tarek egg...
> 
> The tegg?  ;-)

l'oeuf ?

In summmary, it doesn't matter so much what an 'egg' is.

All that is important is that it works on lots of python
platforms, distutils can easily pop them out, they are well
documented and relatively trouble free.

Most importantly, credit for the original idea goes back
to PJE and everything gets refactored to make it 'nice'.

I don't think it's a massive refactoring operation myself,
and I'm very sure well within Tareks skillset to take on.

David




From tjreedy at udel.edu  Thu Nov  5 01:42:31 2009
From: tjreedy at udel.edu (Terry Reedy)
Date: Wed, 04 Nov 2009 19:42:31 -0500
Subject: [Python-Dev] 2.7 Release? 2.7 == last of the 2.x line?
In-Reply-To: <5CB59A21-D8C9-4BC1-A1F6-4438B86A8BC5@twistedmatrix.com>
References: <E077843B-919A-4AC7-95D7-0313E4D0328A@gmail.com>	<4AEFD9AE.7090104@molden.no>
	<4AEFF450.9060007@voidspace.org.uk>	<79EADA413EFB44F6864EF71D5B830088@RaymondLaptop1>	<79990c6b0911030323x47a9f008ue7256c56e88c0ee8@mail.gmail.com>
	<5CB59A21-D8C9-4BC1-A1F6-4438B86A8BC5@twistedmatrix.com>
Message-ID: <hct71l$1tl$1@ger.gmane.org>

Glyph Lefkowitz wrote:

> For what it's worth, the official position of the Twisted project is not 
> that we have "no plan" to move to Python 3.  It's that our plan is to do 
> exactly as Raymond suggests, and give the users a vote - in this case, 
> you vote with your patches :).

You probably will not hear from potential users who skip Twisted because 
it is not available for 3.x. I suspect you do not hear much either from 
new users who only installed 2.x to use Twisted, but would prefer 3.x. 
There are regular questions on python-list about 'web programming with 
3.0' or some such.

> For one thing, we have a very long row to hoe here.  The migration to 
> 3.0 is a long, tedious process with little tangible benefit.

One group that benefits is new Python programmers. Python 3 is easier to 
learn, and is being used to teach Python in at least some schools and 
universities, and will be used more as more libraries are available. 
Hardly a week goes by on Python list without someone posting a problem 
using 2.x that has been solved in 3.x.

Another group is existing programmers who were/are sufficiently annoyed 
by some of the things that got cleaned up.

A third group is people who want to use non-ascii in identifiers, and 
who are delighted now that they can.

Since you do not fall in these groups, I understand your impatience and 
reluctance with the change. I can also imagine that Twisted may be more 
affected by some of the changes than most other projects.

[snip more] ...

> There have been some other comments in this thread indicating that this 
> was not the case because some users indicated that they'd rather deal 
> with lots of changes "all at once".

I wrote that based on both my reading of clp/pylist posts during the 
discussion of the int/int semantic change and Guido's report of private 
conversations he had had.

 > My understanding is that it was
> done this way so that the *developers* of Python could make a clean 
> break, and design and implement a new version of Python without being 
> constrained by compatibility concerns.

I do not believe that was ever intended. It certainly is not what 
happened; many changes were not made *because* of compatibility concerns 
and all went through the filter of 'is the benefit of this change worth 
the pain of breakage'. There is a big difference between not being 
straightjacketed and being unconstrained.

 > If you can show me an actual
> application or library developer in Python who wanted this one-big-jump 
> migration, I will show you a crazy person.

Be careful of labels.

Once the prolonged and intense int/int debate shifted from the ontology 
of ints to the pragmatics of the proposed change, most people agreed 
that int/int 'should' have meant float(int)/float(int) from the 
beginning. But some were still strongly opposed to making the change 
because they (understandably) did not want to have to scan (by eye) 
possibly 10000s or even 100000s of lines for every a/b to determine 
whether any fix was needed. Some said that that would be such a major 
change that it should not be done until there was a new major release, a 
Python 3 off in the then distant future.  Well, that future is now.

I half-jokingly suggested that the change be made on Guido's original
timetable, but that the '2.5' that completed the change simply be 
relabeled '3.0'. I personally would have preferred that it had been 
completed in 2.5. But that did not happen and more changes were made 
once they were made, and here we are.

Terry Jan Reedy


From tjreedy at udel.edu  Thu Nov  5 01:42:56 2009
From: tjreedy at udel.edu (Terry Reedy)
Date: Wed, 04 Nov 2009 19:42:56 -0500
Subject: [Python-Dev] 2.7 Release? 2.7 == last of the 2.x line?
In-Reply-To: <cd6401a0911041454i3dbf6f65w946e33d88b2ed9d1@mail.gmail.com>
References: <261064E43172924BBADB00DA14BB4B672A14855D01@XCH-NW-14V.nw.nos.boeing.com>	<93A0C3CE-3388-49B9-B994-8AB84E6F48B8@seanet.com>
	<cd6401a0911041454i3dbf6f65w946e33d88b2ed9d1@mail.gmail.com>
Message-ID: <hct72d$1tl$2@ger.gmane.org>

Zooko O'Whielacronx wrote:
> Folks:

> So, if you guys slip in your favorite new Python 3 feature into 2.7
> and add a deprecation warning for your least favorite Python 2
> misfeature, 

Just run with the -3 flag for warnings.

Also see my response to Glyph.

Terry Jan Reedy


From python at rcn.com  Thu Nov  5 02:07:23 2009
From: python at rcn.com (Raymond Hettinger)
Date: Wed, 4 Nov 2009 17:07:23 -0800
Subject: [Python-Dev] Retrieve an arbitrary element from a
	setwithoutremoving it
References: <7b0045ed0910301843y7059d316s8a851142b78b7aeb@mail.gmail.com><200911040946.15805.steve@pearwood.info><4AF0D0D7.3090403@canterbury.ac.nz><200911042234.36796.steve@pearwood.info>
	<f3cc57c60911041332v479cc3b0t19d9601e52002128@mail.gmail.com>
Message-ID: <99BB308202224D31AA5300223DB59912@RaymondLaptop1>



[Steven D'Aprano]
>> Anyway, given the level of opposition to the suggestion, I'm no longer
>> willing to carry the flag for it. If anyone else -- perhaps the OP --
>> feels they want to take it any further, be my guest.

[geremy condra]
> I've said before that I'd like there to be one, standard way of
> doing this. A function call- set.pick() seems reasonably named
> to me- is probably the cleanest way to do that. Absent that,
> an example in the docs that illustrates the preferred idiom
> would be great. 

Summarizing my opposition to a new set method:
1) there already are at least two succinct ways to get the same effect
2) those ways work with any container, not just sets
3) set implementations in other languages show that this isn't needed.
4) there is value to keeping the API compact
5) isn't needed for optimization (selecting the same value in a loop makes no sense)
6) absence of real-world code examples that would be meaningfully improved

I would be happy to add an example to the docs so that this thread
can finally end.


Raymond

From eric at trueblade.com  Thu Nov  5 02:12:13 2009
From: eric at trueblade.com (Eric Smith)
Date: Wed, 04 Nov 2009 20:12:13 -0500
Subject: [Python-Dev] Retrieve an arbitrary element from
 a	setwithoutremoving it
In-Reply-To: <99BB308202224D31AA5300223DB59912@RaymondLaptop1>
References: <7b0045ed0910301843y7059d316s8a851142b78b7aeb@mail.gmail.com><200911040946.15805.steve@pearwood.info><4AF0D0D7.3090403@canterbury.ac.nz><200911042234.36796.steve@pearwood.info>	<f3cc57c60911041332v479cc3b0t19d9601e52002128@mail.gmail.com>
	<99BB308202224D31AA5300223DB59912@RaymondLaptop1>
Message-ID: <4AF2266D.2020706@trueblade.com>

Raymond Hettinger wrote:
> Summarizing my opposition to a new set method:
> 1) there already are at least two succinct ways to get the same effect
> 2) those ways work with any container, not just sets
> 3) set implementations in other languages show that this isn't needed.
> 4) there is value to keeping the API compact
> 5) isn't needed for optimization (selecting the same value in a loop 
> makes no sense)
> 6) absence of real-world code examples that would be meaningfully improved
> 
> I would be happy to add an example to the docs so that this thread
> can finally end.

Please do!

Eric.


From tjreedy at udel.edu  Thu Nov  5 02:48:25 2009
From: tjreedy at udel.edu (Terry Reedy)
Date: Wed, 04 Nov 2009 20:48:25 -0500
Subject: [Python-Dev] Retrieve an arbitrary element from a
	setwithoutremoving it
In-Reply-To: <4AF2266D.2020706@trueblade.com>
References: <7b0045ed0910301843y7059d316s8a851142b78b7aeb@mail.gmail.com><200911040946.15805.steve@pearwood.info><4AF0D0D7.3090403@canterbury.ac.nz><200911042234.36796.steve@pearwood.info>	<f3cc57c60911041332v479cc3b0t19d9601e52002128@mail.gmail.com>	<99BB308202224D31AA5300223DB59912@RaymondLaptop1>
	<4AF2266D.2020706@trueblade.com>
Message-ID: <hctat6$a5a$1@ger.gmane.org>

Eric Smith wrote:
> Raymond Hettinger wrote:
>> Summarizing my opposition to a new set method:
>> 1) there already are at least two succinct ways to get the same effect
>> 2) those ways work with any container, not just sets
>> 3) set implementations in other languages show that this isn't needed.
>> 4) there is value to keeping the API compact
>> 5) isn't needed for optimization (selecting the same value in a loop 
>> makes no sense)
>> 6) absence of real-world code examples that would be meaningfully 
>> improved

Agreed

>> I would be happy to add an example to the docs so that this thread
>> can finally end.
> 
> Please do!

Yes!


From ben+python at benfinney.id.au  Thu Nov  5 03:49:22 2009
From: ben+python at benfinney.id.au (Ben Finney)
Date: Thu, 05 Nov 2009 13:49:22 +1100
Subject: [Python-Dev] 2to3,	3to2: official status
References: <E077843B-919A-4AC7-95D7-0313E4D0328A@gmail.com>
	<4AEFD9AE.7090104@molden.no> <4AEFF450.9060007@voidspace.org.uk>
	<79EADA413EFB44F6864EF71D5B830088@RaymondLaptop1>
	<79990c6b0911030323x47a9f008ue7256c56e88c0ee8@mail.gmail.com>
	<5CB59A21-D8C9-4BC1-A1F6-4438B86A8BC5@twistedmatrix.com>
	<4AF1CFAD.6000305@v.loewis.de> <87skcuc4ka.fsf_-_@benfinney.id.au>
	<4AF2032D.5050406@v.loewis.de>
Message-ID: <87bpjhd6tp.fsf@benfinney.id.au>

"Martin v. L?wis" <martin at v.loewis.de> writes:

> Ben Finney wrote:
> > "Martin v. L?wis" <martin at v.loewis.de> writes:
> > 
> >> Well, 3to2 would then be an option for you: use Python 3 as the
> >> source language.
> > 
> > I was under the impression that 2to3 was officially supported as
> > part of Python, but 3to2 was a third-party tool. [?] Is it an
> > official part of Python?
>
> No, the status is exactly as you describe it.

Okay. It's probably best for anyone with their Python developer hat on
(which, in this forum, is all the time for any Python developer) to make
the status of 3to2 clear when recommending it to people concerned about
future plans.

-- 
 \        ?Odious ideas are not entitled to hide from criticism behind |
  `\          the human shield of their believers' feelings.? ?Richard |
_o__)                                                         Stallman |
Ben Finney


From collinw at gmail.com  Thu Nov  5 04:09:09 2009
From: collinw at gmail.com (Collin Winter)
Date: Wed, 4 Nov 2009 19:09:09 -0800
Subject: [Python-Dev] 2to3, 3to2: official status
In-Reply-To: <87bpjhd6tp.fsf@benfinney.id.au>
References: <E077843B-919A-4AC7-95D7-0313E4D0328A@gmail.com>
	<4AEFD9AE.7090104@molden.no> <4AEFF450.9060007@voidspace.org.uk>
	<79EADA413EFB44F6864EF71D5B830088@RaymondLaptop1>
	<79990c6b0911030323x47a9f008ue7256c56e88c0ee8@mail.gmail.com>
	<5CB59A21-D8C9-4BC1-A1F6-4438B86A8BC5@twistedmatrix.com>
	<4AF1CFAD.6000305@v.loewis.de> <87skcuc4ka.fsf_-_@benfinney.id.au>
	<4AF2032D.5050406@v.loewis.de> <87bpjhd6tp.fsf@benfinney.id.au>
Message-ID: <43aa6ff70911041909w2a8051e6r3201ed77617b6743@mail.gmail.com>

Hi Ben,

On Wed, Nov 4, 2009 at 6:49 PM, Ben Finney <ben+python at benfinney.id.au> wrote:
> "Martin v. L?wis" <martin at v.loewis.de> writes:
>
>> Ben Finney wrote:
>> > "Martin v. L?wis" <martin at v.loewis.de> writes:
>> >
>> >> Well, 3to2 would then be an option for you: use Python 3 as the
>> >> source language.
>> >
>> > I was under the impression that 2to3 was officially supported as
>> > part of Python, but 3to2 was a third-party tool. [?] Is it an
>> > official part of Python?
>>
>> No, the status is exactly as you describe it.
>
> Okay. It's probably best for anyone with their Python developer hat on
> (which, in this forum, is all the time for any Python developer) to make
> the status of 3to2 clear when recommending it to people concerned about
> future plans.

Are you implying that we shouldn't recommend 3to2 to people wanting to
develop in Py3k and back-translate to 2.x?

Thanks,
Collin Winter

From ben+python at benfinney.id.au  Thu Nov  5 04:20:41 2009
From: ben+python at benfinney.id.au (Ben Finney)
Date: Thu, 05 Nov 2009 14:20:41 +1100
Subject: [Python-Dev] 2to3, 3to2: official status
References: <E077843B-919A-4AC7-95D7-0313E4D0328A@gmail.com>
	<4AEFD9AE.7090104@molden.no> <4AEFF450.9060007@voidspace.org.uk>
	<79EADA413EFB44F6864EF71D5B830088@RaymondLaptop1>
	<79990c6b0911030323x47a9f008ue7256c56e88c0ee8@mail.gmail.com>
	<5CB59A21-D8C9-4BC1-A1F6-4438B86A8BC5@twistedmatrix.com>
	<4AF1CFAD.6000305@v.loewis.de> <87skcuc4ka.fsf_-_@benfinney.id.au>
	<4AF2032D.5050406@v.loewis.de> <87bpjhd6tp.fsf@benfinney.id.au>
	<43aa6ff70911041909w2a8051e6r3201ed77617b6743@mail.gmail.com>
Message-ID: <87pr7xbqt2.fsf@benfinney.id.au>

Collin Winter <collinw at gmail.com> writes:

> On Wed, Nov 4, 2009 at 6:49 PM, Ben Finney <ben+python at benfinney.id.au> wrote:
> > Okay. It's probably best for anyone with their Python developer hat
> > on (which, in this forum, is all the time for any Python developer)
> > to make the status of 3to2 clear when recommending it to people
> > concerned about future plans.
>
> Are you implying that we shouldn't recommend 3to2 to people wanting to
> develop in Py3k and back-translate to 2.x?

No, I'm implying that mentioning Python 3, 3to2, and 2to3 all together
in a discussion about how to migrate can easily give the false
impression that they're all equally supported by the Python developers.

That 3to2 is *not* supported officially is certainly something I'd want
to know if a Python core developer was recommending it to me to reassure
me about the migration path to Python 3, and I didn't get that
impression from the way it's been casually referenced in this
discussion.

-- 
 \           ?Never express yourself more clearly than you are able to |
  `\                                               think.? ?Niels Bohr |
_o__)                                                                  |
Ben Finney


From brett at python.org  Thu Nov  5 04:52:28 2009
From: brett at python.org (Brett Cannon)
Date: Wed, 4 Nov 2009 19:52:28 -0800
Subject: [Python-Dev] 2.7 Release? 2.7 == last of the 2.x line?
In-Reply-To: <cd6401a0911041454i3dbf6f65w946e33d88b2ed9d1@mail.gmail.com>
References: <261064E43172924BBADB00DA14BB4B672A14855D01@XCH-NW-14V.nw.nos.boeing.com>
	<93A0C3CE-3388-49B9-B994-8AB84E6F48B8@seanet.com>
	<cd6401a0911041454i3dbf6f65w946e33d88b2ed9d1@mail.gmail.com>
Message-ID: <bbaeab100911041952o596d8f7cr4b9134e50cf82c5c@mail.gmail.com>

On Wed, Nov 4, 2009 at 14:54, Zooko O'Whielacronx <zookog at gmail.com> wrote:
> Folks:
>
> It occurred to me to wonder why I haven't investigated how hard it
> would be to make my Python packages Python-3-compatible. ?That's right
> -- I haven't even looked closely. ?I couldn't even tell you off the
> top of my head what is in Python 3 that I would have to think about
> except for the new unicode regime. ?I think the answer is that the
> payoff is just *so* low to me at this point that it doesn't even
> justify me taking 15 minutes to read "What's New In Python 3" or to
> execute 2to3 on my smallest package and see what it does.
>

But the payoff is low for you because you want an object-capabilities
system and Python 3 doesn't support that kind of use (nor was Python
designed to support objcap in the first place, so it's already a
strained use in 2.x).

-Brett

From stefan_ml at behnel.de  Thu Nov  5 08:28:26 2009
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Thu, 05 Nov 2009 08:28:26 +0100
Subject: [Python-Dev] PEP 3003 - Python Language Moratorium
In-Reply-To: <ca471dc20911030935l25251e0diec64427f5d7bcbf2@mail.gmail.com>
References: <ca471dc20911030935l25251e0diec64427f5d7bcbf2@mail.gmail.com>
Message-ID: <hctuqq$in0$1@ger.gmane.org>

Guido van Rossum, 03.11.2009 18:35:
> Python, as a language is more than the core implementation --
> CPython -- with a rich, mature and vibrant community of implementations,
> such as Jython [2]_, IronPython [3]_ and PyPy [4]_ that are a benefit
> not only to the community, but to the language itself.

I noticed that this says "such as", but if it's intended to name the major
implementations, I wonder why Cython is missing from that list.

Stefan


From fuzzyman at voidspace.org.uk  Thu Nov  5 10:13:55 2009
From: fuzzyman at voidspace.org.uk (Michael Foord)
Date: Thu, 05 Nov 2009 10:13:55 +0100
Subject: [Python-Dev] PEP 3003 - Python Language Moratorium
In-Reply-To: <hctuqq$in0$1@ger.gmane.org>
References: <ca471dc20911030935l25251e0diec64427f5d7bcbf2@mail.gmail.com>
	<hctuqq$in0$1@ger.gmane.org>
Message-ID: <4AF29753.3040801@voidspace.org.uk>

Stefan Behnel wrote:
> Guido van Rossum, 03.11.2009 18:35:
>   
>> Python, as a language is more than the core implementation --
>> CPython -- with a rich, mature and vibrant community of implementations,
>> such as Jython [2]_, IronPython [3]_ and PyPy [4]_ that are a benefit
>> not only to the community, but to the language itself.
>>     
>
> I noticed that this says "such as", but if it's intended to name the major
> implementations, I wonder why Cython is missing from that list.
>   

Is Cython an implementation of Python?

It's a grey area, but it seems like Cython is a "Python inspired" 
language (similar to Shedskin) rather than an implementation of Python:

    The Cython language is very close to the Python language, but Cython 
additionally supports calling C functions and declaring C types on 
variables and class attributes. This allows the compiler to generate 
very efficient C code from Cython code.

(From http://www.cython.org/ )

There are several partial implementations, including Python inspired 
languages, but if we are looking at 'major complete implementations' 
then the current list seems to be: CPython, Jython, IronPython and PyPy. 
Even Unladen Swallow is a fork (sorry - I mean branch) of CPython rather 
than a separate implementation.

All the best,

Michael Foord

> Stefan
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk
>   


-- 
http://www.ironpythoninaction.com/


From stefan_ml at behnel.de  Thu Nov  5 11:11:41 2009
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Thu, 05 Nov 2009 11:11:41 +0100
Subject: [Python-Dev] PEP 3003 - Python Language Moratorium
In-Reply-To: <4AF29753.3040801@voidspace.org.uk>
References: <ca471dc20911030935l25251e0diec64427f5d7bcbf2@mail.gmail.com>	<hctuqq$in0$1@ger.gmane.org>
	<4AF29753.3040801@voidspace.org.uk>
Message-ID: <hcu8ct$dog$1@ger.gmane.org>

Michael Foord, 05.11.2009 10:13:
> Stefan Behnel wrote:
>> if it's intended to name the major
>> implementations, I wonder why Cython is missing from that list.
> 
> Is Cython an implementation of Python?

We certainly aim for Python compatibility. That's a proclaimed 1.0 goal.
Not sure if that means Py3 or Py2 syntax, but that's just a parser detail,
after all.


> It's a grey area, but it seems like Cython is a "Python inspired"
> language (similar to Shedskin) rather than an implementation of Python:
> 
>    The Cython language is very close to the Python language, but Cython
> additionally supports calling C functions and declaring C types on
> variables and class attributes. This allows the compiler to generate
> very efficient C code from Cython code.
>
> (From http://www.cython.org/ )

Cython is different from Shedskin because Shedskin /requires/ the 'Python'
code to be static. Cython does not. It /optimises/ static code, but it will
not complain about dynamic code.

It /does/ make some static assumptions in that it considers builtins true
builtins. However, it does not prevent you from replacing them in your
code, as long as you do it inside the module. Certainly a restriction
compared to Python, where you can import a module into a changed dict
environment that redefines 'object', but not a major restriction IMO, and
certainly not one that impacts much code.


> There are several partial implementations, including Python inspired
> languages, but if we are looking at 'major complete implementations'
> then the current list seems to be: CPython, Jython, IronPython and PyPy.

Cython certainly is not a complete implementation, sure. It still lacks
support for local classes, for example, metaclasses aren't handled, and
generators are still in design stage. But if one of the goals of the
moratorium is to let implementations catch up with the syntax, then Cython
is certainly in line with that. I doubt that there will be much left on the
TODO list two years from now.


> Even Unladen Swallow is a fork (sorry - I mean branch) of CPython rather
> than a separate implementation.

Cython is somewhere in the middle. It's not a fork or branch. Rather, it
depends on the normal CPython runtime for certain features, such as the
standard library, the type/object implementation, garbage collection, and a
couple of dynamic features. But that doesn't prevent it from compiling your
module into an executable application.

Stefan


From fuzzyman at voidspace.org.uk  Thu Nov  5 11:18:17 2009
From: fuzzyman at voidspace.org.uk (Michael Foord)
Date: Thu, 05 Nov 2009 11:18:17 +0100
Subject: [Python-Dev] PEP 3003 - Python Language Moratorium
In-Reply-To: <hcu8ct$dog$1@ger.gmane.org>
References: <ca471dc20911030935l25251e0diec64427f5d7bcbf2@mail.gmail.com>	<hctuqq$in0$1@ger.gmane.org>	<4AF29753.3040801@voidspace.org.uk>
	<hcu8ct$dog$1@ger.gmane.org>
Message-ID: <4AF2A669.4010708@voidspace.org.uk>

Stefan Behnel wrote:
> Michael Foord, 05.11.2009 10:13:
>   
>> Stefan Behnel wrote:
>>     
>>> if it's intended to name the major
>>> implementations, I wonder why Cython is missing from that list.
>>>       
>> Is Cython an implementation of Python?
>>     
>
> We certainly aim for Python compatibility. That's a proclaimed 1.0 goal.
> Not sure if that means Py3 or Py2 syntax, but that's just a parser detail,
> after all.
>
>   

Right, but I think the main point is that Cython is not an 
implementation of Python. It uses Python syntax, calling into the Python 
runtime for many of its features, and *extending* Python syntax for 
compilation to C. (Right?)

Obviously a grey area - but if you take Cython code and try to run it on 
ClassicPython then you are likely to have syntax errors. The same is not 
true of the other full implementations (although where they use native 
platform features you may not have the same libraries available - but 
the language is unchanged).

All the best,

Michael

>   
>> It's a grey area, but it seems like Cython is a "Python inspired"
>> language (similar to Shedskin) rather than an implementation of Python:
>>
>>    The Cython language is very close to the Python language, but Cython
>> additionally supports calling C functions and declaring C types on
>> variables and class attributes. This allows the compiler to generate
>> very efficient C code from Cython code.
>>
>> (From http://www.cython.org/ )
>>     
>
> Cython is different from Shedskin because Shedskin /requires/ the 'Python'
> code to be static. Cython does not. It /optimises/ static code, but it will
> not complain about dynamic code.
>
> It /does/ make some static assumptions in that it considers builtins true
> builtins. However, it does not prevent you from replacing them in your
> code, as long as you do it inside the module. Certainly a restriction
> compared to Python, where you can import a module into a changed dict
> environment that redefines 'object', but not a major restriction IMO, and
> certainly not one that impacts much code.
>
>
>   
>> There are several partial implementations, including Python inspired
>> languages, but if we are looking at 'major complete implementations'
>> then the current list seems to be: CPython, Jython, IronPython and PyPy.
>>     
>
> Cython certainly is not a complete implementation, sure. It still lacks
> support for local classes, for example, metaclasses aren't handled, and
> generators are still in design stage. But if one of the goals of the
> moratorium is to let implementations catch up with the syntax, then Cython
> is certainly in line with that. I doubt that there will be much left on the
> TODO list two years from now.
>
>
>   
>> Even Unladen Swallow is a fork (sorry - I mean branch) of CPython rather
>> than a separate implementation.
>>     
>
> Cython is somewhere in the middle. It's not a fork or branch. Rather, it
> depends on the normal CPython runtime for certain features, such as the
> standard library, the type/object implementation, garbage collection, and a
> couple of dynamic features. But that doesn't prevent it from compiling your
> module into an executable application.
>
> Stefan
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk
>   


-- 
http://www.ironpythoninaction.com/


From ziade.tarek at gmail.com  Thu Nov  5 11:35:41 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Thu, 5 Nov 2009 11:35:41 +0100
Subject: [Python-Dev] Refactoring installation schemes
In-Reply-To: <39deb0d1eef08419dad65d7354a8ffb1@preisshare.net>
References: <94bdd2610910271455s7854e6a9u94e3aeeeacb69c02@mail.gmail.com>
	<4AE8206A.9020108@egenix.com> <4AE824DA.4090701@voidspace.org.uk>
	<ED004FA5-9F77-4215-B06D-6CBECEF6D023@gmail.com>
	<94bdd2610910281105q328bc3a5qaed020ef3afe28df@mail.gmail.com>
	<94bdd2610911040429j50a6df87m51aac2464dd13bd7@mail.gmail.com>
	<c77acedd9a48dd7036cca55fde42de0a@preisshare.net>
	<9cee7ab80911041440j43e51e39qc77a426547d02bcb@mail.gmail.com>
	<39deb0d1eef08419dad65d7354a8ffb1@preisshare.net>
Message-ID: <94bdd2610911050235l4e851612i46e0392d352eaa20@mail.gmail.com>

2009/11/5 David Lyon <david.lyon at preisshare.net>:
>>> Call it the new Tarek egg...
>>
>> The tegg? ?;-)
>
> l'oeuf ?

While I like 'tegg' a lot ;) , I would like to make it clear that
"eggs" are not part of Distutils. That's the Setuptools format.

Although there's a distinction to make between the *installation*
format, and the *distribution* format (sdist, bdist, etc)

Right now we have several installation formats which leads to problems
I won't mention here.

PEP 376 is working on a default, unified, *installation* format, that
tries to gather the good ideas of Pip, Setuptools etc.. and propose a
unified format for our site-packages. This new standard will come with
APIs in pkgutil to be able to query installed distribution etc. This
work is also linked to PEP 345 work where we are modifying the
Metadata, and to PEP 386 that proposes a standard version comparison
scheme.

But there's no plan to include a new *distribution* format in Distutils.

In any case those PEPs are not finished yet, so everyone can help at
distutils-SIG

Regards
Tarek

From stefan_ml at behnel.de  Thu Nov  5 11:47:46 2009
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Thu, 05 Nov 2009 11:47:46 +0100
Subject: [Python-Dev] PEP 3003 - Python Language Moratorium
In-Reply-To: <4AF2A669.4010708@voidspace.org.uk>
References: <ca471dc20911030935l25251e0diec64427f5d7bcbf2@mail.gmail.com>	<hctuqq$in0$1@ger.gmane.org>	<4AF29753.3040801@voidspace.org.uk>	<hcu8ct$dog$1@ger.gmane.org>
	<4AF2A669.4010708@voidspace.org.uk>
Message-ID: <hcuagi$kqu$1@ger.gmane.org>

Michael Foord, 05.11.2009 11:18:
> Stefan Behnel wrote:
>> We certainly aim for Python compatibility. That's a proclaimed 1.0 goal.
> 
> Right, but I think the main point is that Cython is not an
> implementation of Python. It uses Python syntax, calling into the Python
> runtime for many of its features

It is a Python implementation in the sense that it implements syntax and
semantics of the Python language, be it through generated C code or by
calling into the CPython runtime. You could theoretically drop the entire
interpreter from CPython, and Cython would still work with the remaining
parts - not that it would be worth the effort on either side...


>  and *extending* Python syntax for compilation to C.

Programmers can decide themselves if they want to statically type
variables, and they can do so without breaking with Python syntax. It is
certainly not required to use non-Python syntax to compile Python code, not
even if you want to get a fast binary module as a result.


> Obviously a grey area - but if you take Cython code and try to run it on
> ClassicPython then you are likely to have syntax errors. The same is not
> true of the other full implementations (although where they use native
> platform features you may not have the same libraries available - but
> the language is unchanged).

To me, the term "Python implementation" just means something that
implements the Python language. That shouldn't prevent it from providing
additional features. Jython can interface with Java code, IronPython can
interface with .NET code, Cython can interface with C code. Not sure what
PyPy allows in addition to being a Python implementation, but there surely
are language features also in PyPy that no other Python implementation
provides.

Stefan


From mal at egenix.com  Thu Nov  5 11:56:28 2009
From: mal at egenix.com (M.-A. Lemburg)
Date: Thu, 05 Nov 2009 11:56:28 +0100
Subject: [Python-Dev] Py3k bytes type in 2.x (Re: nonlocal keyword in
 2.x?)
In-Reply-To: <4AF17FB7.5090708@gmail.com>
References: <da7032ce0910211856g45e45632n550aa5b4a10f9a62@mail.gmail.com>	<ca471dc20911021009u7485adcdgde8fae04a5061059@mail.gmail.com>	<Pine.LNX.4.64.0911021405420.21420@kimball.webabinitio.net>	<4AEF4C53.7090902@v.loewis.de>	<Pine.LNX.4.64.0911021624110.21420@kimball.webabinitio.net>	<ca471dc20911021333r168e257eja1e5f79dbe4ae234@mail.gmail.com>	<loom.20091102T223839-797@post.gmane.org>	<4AEFF3A3.3020103@egenix.com>	<4AF06E20.7050307@v.loewis.de>	<4AF0816A.706@egenix.com>	<319e029f0911032306j746bc263jcf6d0dfeaba9f24d@mail.gmail.com>	<4AF16CC7.6000705@gmail.com>	<4AF171AD.1000703@egenix.com>
	<4AF17FB7.5090708@gmail.com>
Message-ID: <4AF2AF5C.4050401@egenix.com>

Nick Coghlan wrote:
> M.-A. Lemburg wrote:
>> Nick Coghlan wrote:
>>> Lennart Regebro wrote:
>>>> I also would really like to see a real port of the bytes class to 2.6,
>>>> but I have a vague memory that there was some reason that wouldn't
>>>> work.
>>> Not so much that it wouldn't work, but that the interfaces to support
>>> using it effectively really aren't there - lots of areas in the standard
>>> library needed to be tweaked to cope with bytes objects in 3.x.
>>>
>>> Generally speaking, the "bytes = str" trick represents a reasonable
>>> compromise as the APIs that you would pass a bytes object to in 3.x
>>> expect an 8-bit str instance in 2.x.
>>
>> Could you please add such hints, tricks and tips to the wiki
>> page:
>>
>> 	http://wiki.python.org/moin/PortingToPy3k
> 
> Done (although the task of figuring out how to get the wiki to display
> code properly defeated me... the normal Python documentation syntax for
> it seemed to give the wiki's ReST interpreter fits).

Thanks.

> I also mentioned the trick someone mentioned about "from __future__
> import unicode_literals" not changing the meaning of 'str' since it only
> alters the parser but not the builtins.
> 
> In writing it up, it occurred to me that having that kind of thing in a
> "py3_compat" compatibility module (to be used as, e.g., "from py3_compat
> import str, bytes") would not only make it easier to use in multiple
> modules, but also easier for 2to3 to remove when forward porting.

That would certainly be useful to have, yes, since all
developers wanting to support both 2.7 and 3.2 using the same
code base will have a need for these things.

It would also help to have a package that contains proxy modules
using the Python 3.x module/package names which map the functionality
back to the Python 2.x modules.

This avoids problems with freezing tools that scan the source
code for imports as well as pickles which store the full
dotted path name to classes.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Nov 05 2009)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try our new mxODBC.Connect Python Database Interface for free ! ::::


   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
           Registered at Amtsgericht Duesseldorf: HRB 46611
               http://www.egenix.com/company/contact/

From fijall at gmail.com  Thu Nov  5 11:58:40 2009
From: fijall at gmail.com (Maciej Fijalkowski)
Date: Thu, 5 Nov 2009 11:58:40 +0100
Subject: [Python-Dev] PEP 3003 - Python Language Moratorium
In-Reply-To: <hcuagi$kqu$1@ger.gmane.org>
References: <ca471dc20911030935l25251e0diec64427f5d7bcbf2@mail.gmail.com>
	<hctuqq$in0$1@ger.gmane.org> <4AF29753.3040801@voidspace.org.uk>
	<hcu8ct$dog$1@ger.gmane.org> <4AF2A669.4010708@voidspace.org.uk>
	<hcuagi$kqu$1@ger.gmane.org>
Message-ID: <693bc9ab0911050258p56cd81dcvca46e4fc087effa3@mail.gmail.com>

>
> To me, the term "Python implementation" just means something that
> implements the Python language. That shouldn't prevent it from providing
> additional features. Jython can interface with Java code, IronPython can
> interface with .NET code, Cython can interface with C code. Not sure what
> PyPy allows in addition to being a Python implementation, but there surely
> are language features also in PyPy that no other Python implementation
> provides.
>
> Stefan
>

Hi.

Features available in PyPy (or Java or IronPython) are not language features.
These are features are available through normal python syntax, just
some libraries
are available extra. Python is something that passes most of CPython's test
suite, modulo stuff that is considered implementation detail (like
reference counting).
In fact, some of extra PyPy features can be achieved via extension modules like
greenlets.

Cython might as well at some point become one, but so far it does not even
import the whole test suite (correct me if I'm wrong), hence it's not
yet python.

Even if the ultimate goal is to support whole Python, it's not
achieved yet, so it
should not be considered implementation of Python (as of today, it might as well
be one tomorrow).

Cheers,
fijal

From fuzzyman at voidspace.org.uk  Thu Nov  5 12:38:49 2009
From: fuzzyman at voidspace.org.uk (Michael Foord)
Date: Thu, 05 Nov 2009 12:38:49 +0100
Subject: [Python-Dev] PEP 3003 - Python Language Moratorium
In-Reply-To: <hcuagi$kqu$1@ger.gmane.org>
References: <ca471dc20911030935l25251e0diec64427f5d7bcbf2@mail.gmail.com>	<hctuqq$in0$1@ger.gmane.org>	<4AF29753.3040801@voidspace.org.uk>	<hcu8ct$dog$1@ger.gmane.org>	<4AF2A669.4010708@voidspace.org.uk>
	<hcuagi$kqu$1@ger.gmane.org>
Message-ID: <4AF2B949.9030706@voidspace.org.uk>

Stefan Behnel wrote:
> Michael Foord, 05.11.2009 11:18:
>   
>> Stefan Behnel wrote:
>>     
>>> We certainly aim for Python compatibility. That's a proclaimed 1.0 goal.
>>>       
>> Right, but I think the main point is that Cython is not an
>> implementation of Python. It uses Python syntax, calling into the Python
>> runtime for many of its features
>>     
>
> It is a Python implementation in the sense that it implements syntax and
> semantics of the Python language, be it through generated C code or by
> calling into the CPython runtime.

Implementing Python by calling into Python does not seem to really be a 
'new implementation'... In that sense Cython is a wrapper not an 
implementation.
>  You could theoretically drop the entire
> interpreter from CPython, and Cython would still work with the remaining
> parts - not that it would be worth the effort on either side...
>
>
>   
>>  and *extending* Python syntax for compilation to C.
>>     
>
> Programmers can decide themselves if they want to statically type
> variables, and they can do so without breaking with Python syntax. It is
> certainly not required to use non-Python syntax to compile Python code, not
> even if you want to get a fast binary module as a result.
>
>   

But you extend the language. Coders can write Python or 'not-Python' (or 
perhaps 'Python plus'). Where you extend the language you are no longer 
implementing Python but a language oriented tool with Python inspired 
syntax intended for use with Python.
>   
>> Obviously a grey area - but if you take Cython code and try to run it on
>> ClassicPython then you are likely to have syntax errors. The same is not
>> true of the other full implementations (although where they use native
>> platform features you may not have the same libraries available - but
>> the language is unchanged).
>>     
>
> To me, the term "Python implementation" just means something that
> implements the Python language. 

Which Cython doesn't - it can parse Python and it calls into Python. It 
doesn't implement runtime features (from the sounds of it).
> That shouldn't prevent it from providing
> additional features. 

Heh. Imagine if Microsoft had taken IronPython and added new syntax, 
that wasn't valid Python, and still tried to claim that IronPython was 
an implementation of Python. Few people would agree...

> Jython can interface with Java code, IronPython can
> interface with .NET code, Cython can interface with C code. Not sure what
> PyPy allows in addition to being a Python implementation, but there surely
> are language features also in PyPy that no other Python implementation
> provides.
>   
No. All of these implementation provide their integration through 
existing language features of Python, not by adding new *language* level 
features.

All the best,

Michael

> Stefan
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk
>   


-- 
http://www.ironpythoninaction.com/


From ncoghlan at gmail.com  Thu Nov  5 13:08:18 2009
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Thu, 05 Nov 2009 22:08:18 +1000
Subject: [Python-Dev] 2.7 Release? 2.7 == last of the 2.x line?
In-Reply-To: <da7032ce0911041130h40ebd3adu49cd9b08734ce74f@mail.gmail.com>
References: <E077843B-919A-4AC7-95D7-0313E4D0328A@gmail.com>	<4AEFD9AE.7090104@molden.no>
	<4AEFF450.9060007@voidspace.org.uk>	<79EADA413EFB44F6864EF71D5B830088@RaymondLaptop1>	<79990c6b0911030323x47a9f008ue7256c56e88c0ee8@mail.gmail.com>	<5CB59A21-D8C9-4BC1-A1F6-4438B86A8BC5@twistedmatrix.com>	<4AF1CFAD.6000305@v.loewis.de>
	<da7032ce0911041130h40ebd3adu49cd9b08734ce74f@mail.gmail.com>
Message-ID: <4AF2C032.4040605@gmail.com>

Mike Krell wrote:
>     Well, 3to2 would then be an option for you: use Python 3 as the source
>     language.

Making life easier for 3to2 is an *excellent* rationale for backports.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------

From ncoghlan at gmail.com  Thu Nov  5 13:10:27 2009
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Thu, 05 Nov 2009 22:10:27 +1000
Subject: [Python-Dev] 2to3, 3to2: official status
In-Reply-To: <87pr7xbqt2.fsf@benfinney.id.au>
References: <E077843B-919A-4AC7-95D7-0313E4D0328A@gmail.com>	<4AEFD9AE.7090104@molden.no>
	<4AEFF450.9060007@voidspace.org.uk>	<79EADA413EFB44F6864EF71D5B830088@RaymondLaptop1>	<79990c6b0911030323x47a9f008ue7256c56e88c0ee8@mail.gmail.com>	<5CB59A21-D8C9-4BC1-A1F6-4438B86A8BC5@twistedmatrix.com>	<4AF1CFAD.6000305@v.loewis.de>
	<87skcuc4ka.fsf_-_@benfinney.id.au>	<4AF2032D.5050406@v.loewis.de>
	<87bpjhd6tp.fsf@benfinney.id.au>	<43aa6ff70911041909w2a8051e6r3201ed77617b6743@mail.gmail.com>
	<87pr7xbqt2.fsf@benfinney.id.au>
Message-ID: <4AF2C0B3.3010601@gmail.com>

Ben Finney wrote:
> That 3to2 is *not* supported officially is certainly something I'd want
> to know if a Python core developer was recommending it to me to reassure
> me about the migration path to Python 3, and I didn't get that
> impression from the way it's been casually referenced in this
> discussion.

I would stick a "yet" in that sentence (i.e. "not supported officially
yet").

It's certainly an addition we should be considering for 3.2 if it is
sufficiently mature by then (and if not 3.2, then we can take another
look for 3.3).

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------

From stefan_ml at behnel.de  Thu Nov  5 13:13:13 2009
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Thu, 05 Nov 2009 13:13:13 +0100
Subject: [Python-Dev] Cython as a Python implementation (was: PEP 3003 -
 Python Language Moratorium)
In-Reply-To: <4AF2B949.9030706@voidspace.org.uk>
References: <ca471dc20911030935l25251e0diec64427f5d7bcbf2@mail.gmail.com>	<hctuqq$in0$1@ger.gmane.org>	<4AF29753.3040801@voidspace.org.uk>	<hcu8ct$dog$1@ger.gmane.org>	<4AF2A669.4010708@voidspace.org.uk>	<hcuagi$kqu$1@ger.gmane.org>
	<4AF2B949.9030706@voidspace.org.uk>
Message-ID: <hcufgp$4ui$1@ger.gmane.org>

Michael Foord, 05.11.2009 12:38:
> Stefan Behnel wrote:
>> Michael Foord, 05.11.2009 11:18:
>>> I think the main point is that Cython is not an
>>> implementation of Python. It uses Python syntax, calling into the Python
>>> runtime for many of its features
>>
>> It is a Python implementation in the sense that it implements syntax and
>> semantics of the Python language, be it through generated C code or by
>> calling into the CPython runtime.
> 
> Implementing Python by calling into Python does not seem to really be a
> 'new implementation'... In that sense Cython is a wrapper not an
> implementation.

Would it help anyone if we started forking CPython, just so we can say
"look, we implement Python"? (note that this will never happen, I'm just
trying to find out where we can draw the line here)

To clarify this statement:

>> You could theoretically drop the entire interpreter from CPython, and
>> Cython would still work with the remaining parts

I think the main parts of CPython that we (re-)use are:

- all built-in types
- ref-counting and garbage collection
- frames for traceback reporting
- stdlib

There may be other parts, but that's what comes to my mind right now.

You can additionally use the CPython interpreter through the 'exec'
statement or by calling into uncompiled code.

For a Python implementation, I don't see any use in reimplementing the
built-in types, for example. They are well done and pretty fast, especially
when used from C code, i.e. outside of the interpreter.

The frame/traceback handling code is only for interaction with uncompiled
Python code, so that you get nice looking tracebacks (which I personally
wouldn't consider a required part of the language itself).

Most Python implementations do not reimplement the stdlib, or at most a
minor part of it, so that's right out of the discussion.

I'm not sure how to value the ref-counting/GC reuse, but I doubt that it is
worth being considered an argument for Cython being a CPython wrapper.


>> Programmers can decide themselves if they want to statically type
>> variables, and they can do so without breaking with Python syntax. It is
>> certainly not required to use non-Python syntax to compile Python
>> code, not even if you want to get a fast binary module as a result.
> 
> But you extend the language. Coders can write Python or 'not-Python' (or
> perhaps 'Python plus'). Where you extend the language you are no longer
> implementing Python but a language oriented tool with Python inspired
> syntax intended for use with Python.

But that's just a detail of the parser. It wouldn't be hard at all to
switch off the support for extended features when compiling .py files (as
opposed to .pyx files). It's just not currently done.

Would you consider Cython a Python implementation if we implemented this?
(which I guess we will do anyway at some point, simply because the
extensions are actually invalid code in the given context)

Stefan


From dirkjan at ochtman.nl  Thu Nov  5 13:18:23 2009
From: dirkjan at ochtman.nl (Dirkjan Ochtman)
Date: Thu, 5 Nov 2009 13:18:23 +0100
Subject: [Python-Dev] Cython as a Python implementation (was: PEP 3003 -
	Python Language Moratorium)
In-Reply-To: <hcufgp$4ui$1@ger.gmane.org>
References: <ca471dc20911030935l25251e0diec64427f5d7bcbf2@mail.gmail.com>
	<hctuqq$in0$1@ger.gmane.org> <4AF29753.3040801@voidspace.org.uk>
	<hcu8ct$dog$1@ger.gmane.org> <4AF2A669.4010708@voidspace.org.uk>
	<hcuagi$kqu$1@ger.gmane.org> <4AF2B949.9030706@voidspace.org.uk>
	<hcufgp$4ui$1@ger.gmane.org>
Message-ID: <ea2499da0911050418i498f8f09g5be3d3dd4a5c3f56@mail.gmail.com>

On Thu, Nov 5, 2009 at 13:13, Stefan Behnel <stefan_ml at behnel.de> wrote:
> Would you consider Cython a Python implementation if we implemented this?
> (which I guess we will do anyway at some point, simply because the
> extensions are actually invalid code in the given context)

Why do you want to be recognized as a Python implementation, anyway? I
don't really understand why this seems so important to you. It's not
like python-dev doesn't want to listen to you, right?

Cheers,

Dirkjan

From ncoghlan at gmail.com  Thu Nov  5 13:19:45 2009
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Thu, 05 Nov 2009 22:19:45 +1000
Subject: [Python-Dev] Py3k bytes type in 2.x (Re: nonlocal keyword in
 2.x?)
In-Reply-To: <4AF1C9FF.2060302@v.loewis.de>
References: <da7032ce0910211856g45e45632n550aa5b4a10f9a62@mail.gmail.com>	<ca471dc20911021009u7485adcdgde8fae04a5061059@mail.gmail.com>	<Pine.LNX.4.64.0911021405420.21420@kimball.webabinitio.net>	<4AEF4C53.7090902@v.loewis.de>	<Pine.LNX.4.64.0911021624110.21420@kimball.webabinitio.net>	<ca471dc20911021333r168e257eja1e5f79dbe4ae234@mail.gmail.com>	<loom.20091102T223839-797@post.gmane.org>	<4AEFF3A3.3020103@egenix.com>	<4AF06E20.7050307@v.loewis.de>	<4AF0816A.706@egenix.com>	<319e029f0911032306j746bc263jcf6d0dfeaba9f24d@mail.gmail.com>	<4AF16CC7.6000705@gmail.com>
	<4AF1C9FF.2060302@v.loewis.de>
Message-ID: <4AF2C2E1.3050601@gmail.com>

Martin v. L?wis wrote:
> Nick Coghlan wrote:
>> Lennart Regebro wrote:
>>> I also would really like to see a real port of the bytes class to 2.6,
>>> but I have a vague memory that there was some reason that wouldn't
>>> work.
>> Not so much that it wouldn't work, but that the interfaces to support
>> using it effectively really aren't there - lots of areas in the standard
>> library needed to be tweaked to cope with bytes objects in 3.x.
> 
> I see the problem differently: if a bytes type was added, nothing would
> use it. In particular, IO wouldn't start returning bytes (although it
> could accept them); IO would continue to return str. Therefore, I'm
> skeptical that adding a *third* string type to 3.x would do any good.

I'm guessing you meant '2.x' in that last sentence, in which case we
agree (just emphasising different parts of the "binary data" ecosystem
that would be necessary to make a backported bytes type effective).

Although I think Guido did a better job than either of us in explaining
why backporting the full bytes type to 2.x really wouldn't help all that
much.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------

From ncoghlan at gmail.com  Thu Nov  5 13:31:52 2009
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Thu, 05 Nov 2009 22:31:52 +1000
Subject: [Python-Dev] 2.7 Release? 2.7 == last of the 2.x line?
In-Reply-To: <426ada670911041039m6df85994v321342231fbd3805@mail.gmail.com>
References: <E077843B-919A-4AC7-95D7-0313E4D0328A@gmail.com>	<F245110A-2FDF-460B-8E0D-87785F0E6DB0@fuhm.net>	<BBB79ADA-7E31-4DD8-AAA5-07537FA74C5A@gmail.com>	<319e029f0911032206y64386499hf4564839ec9acb1a@mail.gmail.com>	<B4DEA4E2-E808-4A07-8A9A-AFDEB5A0CD68@gmail.com>
	<426ada670911041039m6df85994v321342231fbd3805@mail.gmail.com>
Message-ID: <4AF2C5B8.8070204@gmail.com>

Carl Trachte wrote:
> On 11/4/09, ssteinerX at gmail.com <ssteinerx at gmail.com> wrote:
> 
>> Maybe the 3.x line should just be put out of our misery, merged back
>> to 2.7, 2.8, 2.9, and proceed as Glyph suggested in passing with
>> increasing levels of deprecation until it just turns into 3.x on its
>> own by running out of numbers.
> 
> <delurk>
> As a user, I'm horrified.  Granted, I'm not the most high powered
> user, but . . .
> my employer is already providing me with a 3.0 Python version on one
> of my work computers with the expectation that I'll be using it more
> and more.
> 
> Sorry to butt in, but is this a joke?  I thought all this was hashed
> out prior to inventing python 3.0.
> </delurk>

Don't worry, 3.x is still the future of the Python language. Some of the
interested onlookers are just rehashing discussions that happened years
ago *before* the 3.x branch was created. It boils down to the fact that
the real beneficiaries of the 2.x to 3.x transition are the people that
aren't using Python yet, so existing users (especially maintainers of
large libraries and frameworks) bear a disproportionate amount of the
cost of the transition while gaining little of the benefit. They're
understandably irritated by that and the situation is likely to take a
couple more years to sort itself out.

While it may be hard to tell without knowing who is and isn't a core
developer, the only point seriously under discussion is whether there is
going to be a 2.8 after 2.7, and the current answer to that is looking
to be "probably not".

Planning on that basis probably isn't a bad idea. Even if we do decide
to create a 2.8 after 3k has already been merged back to the trunk, a
new 2.8 development branch could easily be created based on the 2.7
maintenance branch instead of the trunk.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------

From theller at ctypes.org  Thu Nov  5 13:53:46 2009
From: theller at ctypes.org (Thomas Heller)
Date: Thu, 05 Nov 2009 13:53:46 +0100
Subject: [Python-Dev] Cython as a Python implementation
In-Reply-To: <ea2499da0911050418i498f8f09g5be3d3dd4a5c3f56@mail.gmail.com>
References: <ca471dc20911030935l25251e0diec64427f5d7bcbf2@mail.gmail.com>	<hctuqq$in0$1@ger.gmane.org>
	<4AF29753.3040801@voidspace.org.uk>	<hcu8ct$dog$1@ger.gmane.org>
	<4AF2A669.4010708@voidspace.org.uk>	<hcuagi$kqu$1@ger.gmane.org>
	<4AF2B949.9030706@voidspace.org.uk>	<hcufgp$4ui$1@ger.gmane.org>
	<ea2499da0911050418i498f8f09g5be3d3dd4a5c3f56@mail.gmail.com>
Message-ID: <hcuhsq$b0t$1@ger.gmane.org>

Dirkjan Ochtman schrieb:
> On Thu, Nov 5, 2009 at 13:13, Stefan Behnel <stefan_ml at behnel.de> wrote:
>> Would you consider Cython a Python implementation if we implemented this?
>> (which I guess we will do anyway at some point, simply because the
>> extensions are actually invalid code in the given context)
> 
> Why do you want to be recognized as a Python implementation, anyway? I
> don't really understand why this seems so important to you.

Advertising, I assume.  That's what Stefan often does (at least it feels
this way, for me).


-- 
Thanks,
Thomas


From solipsis at pitrou.net  Thu Nov  5 14:02:35 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Thu, 5 Nov 2009 13:02:35 +0000 (UTC)
Subject: [Python-Dev] Cython as a Python implementation
References: <ca471dc20911030935l25251e0diec64427f5d7bcbf2@mail.gmail.com>	<hctuqq$in0$1@ger.gmane.org>
	<4AF29753.3040801@voidspace.org.uk>	<hcu8ct$dog$1@ger.gmane.org>
	<4AF2A669.4010708@voidspace.org.uk>	<hcuagi$kqu$1@ger.gmane.org>
	<4AF2B949.9030706@voidspace.org.uk>	<hcufgp$4ui$1@ger.gmane.org>
	<ea2499da0911050418i498f8f09g5be3d3dd4a5c3f56@mail.gmail.com>
	<hcuhsq$b0t$1@ger.gmane.org>
Message-ID: <loom.20091105T140142-435@post.gmane.org>

Thomas Heller <theller <at> ctypes.org> writes:
> 
> > Why do you want to be recognized as a Python implementation, anyway? I
> > don't really understand why this seems so important to you.
> 
> Advertising, I assume.  That's what Stefan often does (at least it feels
> this way, for me).

Well, it's true that their work often goes unnoticed in favour of more
pie-in-the-sky things such as PyPy and unladen-swallow.



From abpillai at gmail.com  Thu Nov  5 15:05:49 2009
From: abpillai at gmail.com (Anand Balachandran Pillai)
Date: Thu, 5 Nov 2009 19:35:49 +0530
Subject: [Python-Dev] Cython as a Python implementation
In-Reply-To: <loom.20091105T140142-435@post.gmane.org>
References: <ca471dc20911030935l25251e0diec64427f5d7bcbf2@mail.gmail.com>
	<4AF29753.3040801@voidspace.org.uk> <hcu8ct$dog$1@ger.gmane.org>
	<4AF2A669.4010708@voidspace.org.uk> <hcuagi$kqu$1@ger.gmane.org>
	<4AF2B949.9030706@voidspace.org.uk> <hcufgp$4ui$1@ger.gmane.org>
	<ea2499da0911050418i498f8f09g5be3d3dd4a5c3f56@mail.gmail.com>
	<hcuhsq$b0t$1@ger.gmane.org> <loom.20091105T140142-435@post.gmane.org>
Message-ID: <8548c5f30911050605o14982697i6a7cc4be68243959@mail.gmail.com>

On Thu, Nov 5, 2009 at 6:32 PM, Antoine Pitrou <solipsis at pitrou.net> wrote:

> Thomas Heller <theller <at> ctypes.org> writes:
> >
> > > Why do you want to be recognized as a Python implementation, anyway? I
> > > don't really understand why this seems so important to you.
> >
> > Advertising, I assume.  That's what Stefan often does (at least it feels
> > this way, for me).
>
> Well, it's true that their work often goes unnoticed in favour of more
> pie-in-the-sky things such as PyPy and unladen-swallow.
>

 Perhaps something to do with naming. "PyPy" and "unladen-swallow"
 are obviously more eye-catching than "Cython".

 Just my $0.02.

>
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> http://mail.python.org/mailman/options/python-dev/abpillai%40gmail.com
>

Regards,

-- 
--Anand
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20091105/c77d3916/attachment.htm>

From metawilm at gmail.com  Thu Nov  5 16:21:27 2009
From: metawilm at gmail.com (Willem Broekema)
Date: Thu, 5 Nov 2009 16:21:27 +0100
Subject: [Python-Dev] Cython as a Python implementation
In-Reply-To: <8548c5f30911050605o14982697i6a7cc4be68243959@mail.gmail.com>
References: <ca471dc20911030935l25251e0diec64427f5d7bcbf2@mail.gmail.com>
	<hcu8ct$dog$1@ger.gmane.org> <4AF2A669.4010708@voidspace.org.uk>
	<hcuagi$kqu$1@ger.gmane.org> <4AF2B949.9030706@voidspace.org.uk>
	<hcufgp$4ui$1@ger.gmane.org>
	<ea2499da0911050418i498f8f09g5be3d3dd4a5c3f56@mail.gmail.com>
	<hcuhsq$b0t$1@ger.gmane.org> <loom.20091105T140142-435@post.gmane.org>
	<8548c5f30911050605o14982697i6a7cc4be68243959@mail.gmail.com>
Message-ID: <f6bc9b490911050721u769e3bcdh2e5658c2e46d4358@mail.gmail.com>

On Thu, Nov 5, 2009 at 3:05 PM, Anand Balachandran Pillai
<abpillai at gmail.com> wrote:
> ?Perhaps something to do with naming. "PyPy" and "unladen-swallow"
> ?are obviously more eye-catching than "Cython".

Heh, CLPython is not very catchy either. Maybe that's why it's still
under the radar here.

- Willem

From stefan_ml at behnel.de  Thu Nov  5 16:45:39 2009
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Thu, 05 Nov 2009 16:45:39 +0100
Subject: [Python-Dev] Cython as a Python implementation
In-Reply-To: <hcuhsq$b0t$1@ger.gmane.org>
References: <ca471dc20911030935l25251e0diec64427f5d7bcbf2@mail.gmail.com>	<hctuqq$in0$1@ger.gmane.org>	<4AF29753.3040801@voidspace.org.uk>	<hcu8ct$dog$1@ger.gmane.org>	<4AF2A669.4010708@voidspace.org.uk>	<hcuagi$kqu$1@ger.gmane.org>	<4AF2B949.9030706@voidspace.org.uk>	<hcufgp$4ui$1@ger.gmane.org>	<ea2499da0911050418i498f8f09g5be3d3dd4a5c3f56@mail.gmail.com>
	<hcuhsq$b0t$1@ger.gmane.org>
Message-ID: <hcurv3$ir5$1@ger.gmane.org>

Thomas Heller, 05.11.2009 13:53:
> Advertising, I assume.  That's what Stefan often does (at least it feels
> this way, for me).

Well, there's nothing inherently wrong with telling people about the right
tool for the job, especially when it's free. And don't forget that postings
are self-selecting by their topic, so you may simply not have come across
the non-advertising posts of mine, and I may have skipped worthy
non-advertising-affine topics that flew below my minimum level of conscious
reading.

Stefan


From guido at python.org  Thu Nov  5 16:57:19 2009
From: guido at python.org (Guido van Rossum)
Date: Thu, 5 Nov 2009 07:57:19 -0800
Subject: [Python-Dev] Cython as a Python implementation
In-Reply-To: <hcuhsq$b0t$1@ger.gmane.org>
References: <ca471dc20911030935l25251e0diec64427f5d7bcbf2@mail.gmail.com> 
	<hctuqq$in0$1@ger.gmane.org> <4AF29753.3040801@voidspace.org.uk> 
	<hcu8ct$dog$1@ger.gmane.org> <4AF2A669.4010708@voidspace.org.uk> 
	<hcuagi$kqu$1@ger.gmane.org> <4AF2B949.9030706@voidspace.org.uk> 
	<hcufgp$4ui$1@ger.gmane.org>
	<ea2499da0911050418i498f8f09g5be3d3dd4a5c3f56@mail.gmail.com> 
	<hcuhsq$b0t$1@ger.gmane.org>
Message-ID: <ca471dc20911050757s79e671cfydf120a8ee6425aad@mail.gmail.com>

On Thu, Nov 5, 2009 at 4:53 AM, Thomas Heller <theller at ctypes.org> wrote:
> Dirkjan Ochtman schrieb:
>> On Thu, Nov 5, 2009 at 13:13, Stefan Behnel <stefan_ml at behnel.de> wrote:
>>> Would you consider Cython a Python implementation if we implemented this?
>>> (which I guess we will do anyway at some point, simply because the
>>> extensions are actually invalid code in the given context)
>>
>> Why do you want to be recognized as a Python implementation, anyway? I
>> don't really understand why this seems so important to you.
>
> Advertising, I assume. ?That's what Stefan often does (at least it feels
> this way, for me).

I'm skeptical about Cython as a Python implementation. AFAIK it
requires CPython to work (for the runtime functionality) and while
Cython's syntax is very Python-like, it is probably best characterized
as an extended subset (:-). I guess when it runs the entire standard
test suite without errors it could claim to be a Python
implementation; until then, I would rather consider it a useful tool
to go with CPython. (The name is actually a bit confusing, but I don't
have a desire to change it.)

-- 
--Guido van Rossum (python.org/~guido)

From solipsis at pitrou.net  Thu Nov  5 17:06:03 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Thu, 5 Nov 2009 16:06:03 +0000 (UTC)
Subject: [Python-Dev] No buildbot to test wide unicode?
References: <e27efe130911031557n7839ab25g28fc6dafe3174ab7@mail.gmail.com>
	<4AF1D14A.6040007@v.loewis.de>
Message-ID: <hcut5b$n6b$1@ger.gmane.org>

Le Wed, 04 Nov 2009 20:08:58 +0100, Martin v. L?wis a ?crit?:
>> 
>> Since wide unicode is the standard chosen by some Linux distributions,
>> it would make sense to have at least one buildbot running with
>> --with-wide-unicode (3.x) or --enable-unicode=ucs4 (2.x).
> 
> Can you propose some (one? two? more?) systems that might be best as
> candidates? I'd then setup two (sets of) builders; they would share the
> slave lock, so builds would run sequentially (unless the slave operator
> agrees to setup two slaves on one machine).

I would certainly agree to setup two slaves on mine. There are ample 
resources available.

Regards

Antoine.


From stefan_ml at behnel.de  Thu Nov  5 17:08:52 2009
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Thu, 05 Nov 2009 17:08:52 +0100
Subject: [Python-Dev] Cython as a Python implementation
In-Reply-To: <loom.20091105T140142-435@post.gmane.org>
References: <ca471dc20911030935l25251e0diec64427f5d7bcbf2@mail.gmail.com>	<hctuqq$in0$1@ger.gmane.org>	<4AF29753.3040801@voidspace.org.uk>	<hcu8ct$dog$1@ger.gmane.org>	<4AF2A669.4010708@voidspace.org.uk>	<hcuagi$kqu$1@ger.gmane.org>	<4AF2B949.9030706@voidspace.org.uk>	<hcufgp$4ui$1@ger.gmane.org>	<ea2499da0911050418i498f8f09g5be3d3dd4a5c3f56@mail.gmail.com>	<hcuhsq$b0t$1@ger.gmane.org>
	<loom.20091105T140142-435@post.gmane.org>
Message-ID: <hcutal$o1s$1@ger.gmane.org>

Antoine Pitrou, 05.11.2009 14:02:
> Thomas Heller writes:
>>> Why do you want to be recognized as a Python implementation, anyway? I
>>> don't really understand why this seems so important to you.
>>
>> Advertising, I assume.  That's what Stefan often does (at least it feels
>> this way, for me).
>
> Well, it's true that their work often goes unnoticed in favour of more
> pie-in-the-sky things such as PyPy and unladen-swallow.

That's likely got to do with it. It's not the first time I read about the
"four" Python implementations. It may be related to the lack of Cython
developers at the language summits, but, well, it's not like those were
cheap to join either, if you count travel time/costs etc.

I'm well aware, and I openly admit that Cython is laging behind in terms of
Python language features when compared with other Python implementations. A
reason for this is that the goal of the Cython project to design a
/superset/ of the Python language is simply more ambitious (and certainly
more time consuming) than just writing "yet another Python implementation".
We put a lot of thought work into language design between Python and C, and
also into code performance optimisations, where others can just go straight
for CPython compatibility. But this doesn't mean we are not writing a
Python implementation. It just means that we do more than that, and that
this 'more' takes time.

It has often been argued that it's easy to implement Python faster if you
actually don't implement Python, but that doesn't apply to the current
Cython anymore. What it lacks in terms of features is nothing that would
impact the performance of existing features in any way.

Like most of the other Python implementations, Cython is ready to use now.
Unlike most others, it has all batteries included. And unlike some others,
it's fast and even scales extremely well with various performance
requirements. Personally, I consider Cython one of the killer arguments
against performance prejudices about the CPython platform, if not about the
Python language itself.

Stefan


From rdmurray at bitdance.com  Thu Nov  5 17:29:26 2009
From: rdmurray at bitdance.com (R. David Murray)
Date: Thu, 5 Nov 2009 11:29:26 -0500 (EST)
Subject: [Python-Dev] No buildbot to test wide unicode?
In-Reply-To: <hcut5b$n6b$1@ger.gmane.org>
References: <e27efe130911031557n7839ab25g28fc6dafe3174ab7@mail.gmail.com>
	<4AF1D14A.6040007@v.loewis.de> <hcut5b$n6b$1@ger.gmane.org>
Message-ID: <Pine.LNX.4.64.0911051125270.11243@kimball.webabinitio.net>

On Thu, 5 Nov 2009 at 16:06, Antoine Pitrou wrote:
> Le Wed, 04 Nov 2009 20:08:58 +0100, Martin v. L??wis a ??crit??:
>>> Since wide unicode is the standard chosen by some Linux distributions,
>>> it would make sense to have at least one buildbot running with
>>> --with-wide-unicode (3.x) or --enable-unicode=ucs4 (2.x).
>>
>> Can you propose some (one? two? more?) systems that might be best as
>> candidates? I'd then setup two (sets of) builders; they would share the
>> slave lock, so builds would run sequentially (unless the slave operator
>> agrees to setup two slaves on one machine).
>
> I would certainly agree to setup two slaves on mine. There are ample
> resources available.

I could do so as well.  Gentoo is one of the distributions that uses
the wide build by default, so that would make it a good test candidate
as well.

--David

From stefan_ml at behnel.de  Thu Nov  5 17:40:11 2009
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Thu, 05 Nov 2009 17:40:11 +0100
Subject: [Python-Dev] Cython as a Python implementation
In-Reply-To: <693bc9ab0911050258p56cd81dcvca46e4fc087effa3@mail.gmail.com>
References: <ca471dc20911030935l25251e0diec64427f5d7bcbf2@mail.gmail.com>	<hctuqq$in0$1@ger.gmane.org>
	<4AF29753.3040801@voidspace.org.uk>	<hcu8ct$dog$1@ger.gmane.org>
	<4AF2A669.4010708@voidspace.org.uk>	<hcuagi$kqu$1@ger.gmane.org>
	<693bc9ab0911050258p56cd81dcvca46e4fc087effa3@mail.gmail.com>
Message-ID: <hcuv5c$uup$1@ger.gmane.org>

Maciej Fijalkowski, 05.11.2009 11:58:
> Python is something that passes most of CPython's test
> suite, modulo stuff that is considered implementation detail (like
> reference counting).

Certainly not an implementation detail to Cython. ;)


> Cython might as well at some point become one, but so far it does not even
> import the whole test suite (correct me if I'm wrong), hence it's not
> yet python.

The test suite is a problem anyway, given that Cython doesn't implement a
specific level of the Python language. It aims for 2.6 compatibility, but
it already implements many of the Py3k PEPs, for example.

But, yes, it doesn't currently import the test suite (without cheating).
Many of the tests fail to compile due to the use of lambda statements
(which won't take long to support on top of the existing closures) and
conditionally defined classes (which will likely take longer, unless we
cheat a bit by special casing the simple cases).

BTW, other implementations lack some standard library modules. I find it a
lot easier to work around missing syntax than to work around missing
dependencies. But maybe that's just me.

Stefan


From ubershmekel at gmail.com  Thu Nov  5 18:34:29 2009
From: ubershmekel at gmail.com (Yuvgoog Greenle)
Date: Thu, 5 Nov 2009 19:34:29 +0200
Subject: [Python-Dev] 2.7 Release? 2.7 == last of the 2.x line?
In-Reply-To: <4AF2C5B8.8070204@gmail.com>
References: <E077843B-919A-4AC7-95D7-0313E4D0328A@gmail.com>
	<F245110A-2FDF-460B-8E0D-87785F0E6DB0@fuhm.net>
	<BBB79ADA-7E31-4DD8-AAA5-07537FA74C5A@gmail.com>
	<319e029f0911032206y64386499hf4564839ec9acb1a@mail.gmail.com>
	<B4DEA4E2-E808-4A07-8A9A-AFDEB5A0CD68@gmail.com>
	<426ada670911041039m6df85994v321342231fbd3805@mail.gmail.com>
	<4AF2C5B8.8070204@gmail.com>
Message-ID: <9d153b7c0911050934k714aca73q62dd78cf9033584e@mail.gmail.com>

On Thu, Nov 5, 2009 at 2:31 PM, Nick Coghlan <ncoghlan at gmail.com> wrote:

> While it may be hard to tell without knowing who is and isn't a core
> developer
>

Maybe there should be badges or something, hmmm, sounds like making an
svn-commits-hall-of-fame for python could be a nice project.

--yuv
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20091105/3671acda/attachment-0001.htm>

From fijall at gmail.com  Thu Nov  5 19:01:56 2009
From: fijall at gmail.com (Maciej Fijalkowski)
Date: Thu, 5 Nov 2009 19:01:56 +0100
Subject: [Python-Dev] Cython as a Python implementation (was: PEP 3003 -
	Python Language Moratorium)
In-Reply-To: <hcufgp$4ui$1@ger.gmane.org>
References: <ca471dc20911030935l25251e0diec64427f5d7bcbf2@mail.gmail.com>
	<hctuqq$in0$1@ger.gmane.org> <4AF29753.3040801@voidspace.org.uk>
	<hcu8ct$dog$1@ger.gmane.org> <4AF2A669.4010708@voidspace.org.uk>
	<hcuagi$kqu$1@ger.gmane.org> <4AF2B949.9030706@voidspace.org.uk>
	<hcufgp$4ui$1@ger.gmane.org>
Message-ID: <693bc9ab0911051001x495f3305v92e8312fcc143d04@mail.gmail.com>

> Most Python implementations do not reimplement the stdlib, or at most a
> minor part of it, so that's right out of the discussion.

Did you actually check?

From dinov at microsoft.com  Thu Nov  5 19:35:08 2009
From: dinov at microsoft.com (Dino Viehland)
Date: Thu, 5 Nov 2009 18:35:08 +0000
Subject: [Python-Dev] PEP 3003 - Python Language Moratorium
In-Reply-To: <hcu8ct$dog$1@ger.gmane.org>
References: <ca471dc20911030935l25251e0diec64427f5d7bcbf2@mail.gmail.com>
	<hctuqq$in0$1@ger.gmane.org>	<4AF29753.3040801@voidspace.org.uk>
	<hcu8ct$dog$1@ger.gmane.org>
Message-ID: <1A472770E042064698CB5ADC83A12ACD04C4A6C6@TK5EX14MBXC118.redmond.corp.microsoft.com>

Stefan wrote:
> It /does/ make some static assumptions in that it considers builtins
> true
> builtins. However, it does not prevent you from replacing them in your
> code, as long as you do it inside the module. Certainly a restriction
> compared to Python, where you can import a module into a changed dict
> environment that redefines 'object', but not a major restriction IMO,
> and certainly not one that impacts much code.

To me this is a deal breaker which prevents Cython from being a Python
implementation.  From a talk given by Colin Winter at the LLVM dev meeting
(http://llvm.org/devmtg/2009-10/) it seems like Unladen Swallow wanted to
do something like this as well and Guido said no.  In this case the breaking
change is so subtle that I'd personally hate to run into something like 
this porting code to Cython and having to figure out why it's not working.

More generally if all the implementations pick and choose what features of
Python we want to ignore for performance reasons then we'll just end up 
w/ a bunch of incompatible things that look very similar.  



From collinw at gmail.com  Thu Nov  5 19:40:48 2009
From: collinw at gmail.com (Collin Winter)
Date: Thu, 5 Nov 2009 10:40:48 -0800
Subject: [Python-Dev] PEP 3003 - Python Language Moratorium
In-Reply-To: <1A472770E042064698CB5ADC83A12ACD04C4A6C6@TK5EX14MBXC118.redmond.corp.microsoft.com>
References: <ca471dc20911030935l25251e0diec64427f5d7bcbf2@mail.gmail.com>
	<hctuqq$in0$1@ger.gmane.org> <4AF29753.3040801@voidspace.org.uk>
	<hcu8ct$dog$1@ger.gmane.org>
	<1A472770E042064698CB5ADC83A12ACD04C4A6C6@TK5EX14MBXC118.redmond.corp.microsoft.com>
Message-ID: <43aa6ff70911051040t3a915217o7ffec1f432e9def7@mail.gmail.com>

On Thu, Nov 5, 2009 at 10:35 AM, Dino Viehland <dinov at microsoft.com> wrote:
> Stefan wrote:
>> It /does/ make some static assumptions in that it considers builtins
>> true
>> builtins. However, it does not prevent you from replacing them in your
>> code, as long as you do it inside the module. Certainly a restriction
>> compared to Python, where you can import a module into a changed dict
>> environment that redefines 'object', but not a major restriction IMO,
>> and certainly not one that impacts much code.
>
> To me this is a deal breaker which prevents Cython from being a Python
> implementation. ?From a talk given by Colin Winter at the LLVM dev meeting
> (http://llvm.org/devmtg/2009-10/) it seems like Unladen Swallow wanted to
> do something like this as well and Guido said no. ?In this case the breaking
> change is so subtle that I'd personally hate to run into something like
> this porting code to Cython and having to figure out why it's not working.

To clarify, I was joking when I told that story (or at least I was
joking with Guido when I asked him if we could break that). It clearly
*would* be easier if we could just ignore this point of Python
compatibility, but that's not an option, so we've had to optimize
around it. It's not that hard to do, but it's still extra work.

Collin Winter

From stefan_ml at behnel.de  Thu Nov  5 19:55:17 2009
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Thu, 05 Nov 2009 19:55:17 +0100
Subject: [Python-Dev] Cython as a Python implementation
In-Reply-To: <693bc9ab0911051001x495f3305v92e8312fcc143d04@mail.gmail.com>
References: <ca471dc20911030935l25251e0diec64427f5d7bcbf2@mail.gmail.com>	<hctuqq$in0$1@ger.gmane.org>
	<4AF29753.3040801@voidspace.org.uk>	<hcu8ct$dog$1@ger.gmane.org>
	<4AF2A669.4010708@voidspace.org.uk>	<hcuagi$kqu$1@ger.gmane.org>
	<4AF2B949.9030706@voidspace.org.uk>	<hcufgp$4ui$1@ger.gmane.org>
	<693bc9ab0911051001x495f3305v92e8312fcc143d04@mail.gmail.com>
Message-ID: <hcv72l$rvg$1@ger.gmane.org>

Maciej Fijalkowski, 05.11.2009 19:01:
>> Most Python implementations do not reimplement the stdlib, or at most a
>> minor part of it, so that's right out of the discussion.
> 
> Did you actually check?

Well, I know that Jython uses the original stdlib modules through
svn:externals in the build, last thing I heard about IronPython was that
they are allowed to use other people's code now, so I imagine they do the
same thing, and I wouldn't expect PyPy to rewrite the existing Python code
that exists in the stdlib. So the only remaining problem are the C
extensions in CPython, and that's the minor part I mentioned above.

Stefan


From pje at telecommunity.com  Thu Nov  5 20:18:22 2009
From: pje at telecommunity.com (P.J. Eby)
Date: Thu, 05 Nov 2009 14:18:22 -0500
Subject: [Python-Dev] 2.7 Release? 2.7 == last of the 2.x line?
In-Reply-To: <20091104232743.GD19537@laurie.devork>
References: <261064E43172924BBADB00DA14BB4B672A14855D01@XCH-NW-14V.nw.nos.boeing.com>
	<93A0C3CE-3388-49B9-B994-8AB84E6F48B8@seanet.com>
	<cd6401a0911041454i3dbf6f65w946e33d88b2ed9d1@mail.gmail.com>
	<20091104232743.GD19537@laurie.devork>
Message-ID: <20091105191827.896D13A40A8@sparrow.telecommunity.com>

At 11:27 PM 11/4/2009 +0000, Floris Bruynooghe wrote:
><cynical mode>
>
>You have time to read this thread but no time to read "What's New In
>Python 3"?
>
></cynical mode>

Given that the average developer using tons of existing libraries on 
2.x is unlikely to see any killer benefits in moving to 3.x, it's a 
natural attitude to have.

I fought this same battle with setuptools for a long time before it 
sank in that people who don't perceive a need aren't going to RTFM, 
no matter how much time said RTFMing would save them in the long run, 
vs. sitting around complaining.

IOW, once someone has become annoyed by the mere appearance of a 
necessity to deal with something that appears to be being foisted on 
them (whether it's setuptools or Python 3), the natural tendency is 
to minimize any actual work that would move in the direction of the 
thing they feel forced to deal with.

For me, the closest thing to a killer feature in 3.x is argument type 
declarations, and it'd be a mild convenience at best.  From a 
distance, many of the other changes appear like anti-features, if 
only because they're changing what I've been used to for twelve-plus 
years.  (A few, like the removal of __metaclass__-in-locals support, 
are an active hindrance to porting.)

So no, I haven't actually tried to port anything, nor have I done 
more than lightly skim the porting docs...  looking for some reason 
why I'd *want* to move to Python 3.  Heck, I have yet to use 2.6 to 
run any production code, and find some of *its* changes a bit 
annoying from a porting perspective.  (E.g. dropping the "sets" module.)

To make Py3 migration worthwhile to developers with heavy investment 
in the 2.x lines (and especially those supporting all the way back to 
2.3 and 2.4), it'd have to have some *really* killer features.  That 
is, be more like a Python 3000, and less like a Python 2.6 with a few 
bells and whistles, hampered by having to relearn some of the basic 
types and a soon-to-be-rebuilt standard library.

Even if, in truth, the cost-benefit ratio right now *is* good for 
migrating to 3.x, nobody's doing a good job at promoting what those 
benefits are.

(And it being easy to port to is NOT a benefit: nobody cares how easy 
it is to do something they don't see a reason to do in the first place.)


From guido at python.org  Thu Nov  5 20:36:23 2009
From: guido at python.org (Guido van Rossum)
Date: Thu, 5 Nov 2009 11:36:23 -0800
Subject: [Python-Dev] Cython as a Python implementation
In-Reply-To: <hcutal$o1s$1@ger.gmane.org>
References: <ca471dc20911030935l25251e0diec64427f5d7bcbf2@mail.gmail.com> 
	<hcu8ct$dog$1@ger.gmane.org> <4AF2A669.4010708@voidspace.org.uk> 
	<hcuagi$kqu$1@ger.gmane.org> <4AF2B949.9030706@voidspace.org.uk> 
	<hcufgp$4ui$1@ger.gmane.org>
	<ea2499da0911050418i498f8f09g5be3d3dd4a5c3f56@mail.gmail.com> 
	<hcuhsq$b0t$1@ger.gmane.org> <loom.20091105T140142-435@post.gmane.org> 
	<hcutal$o1s$1@ger.gmane.org>
Message-ID: <ca471dc20911051136h274c158fm2b21d63894b78487@mail.gmail.com>

Stefan,

I think your attempts to see Cython accepted as one of the major
Python implementations is misguided. It is not self-contained, it is
an add-on tool for CPython (like its ancestor PyRex).

I think Cython is incredibly useful (and I spoke to many very happy
users yesterday at UCB) but trying to present it as a separate Python
implementation is not useful.

Please stop the discussion about this topic, they are distracting.

-- 
--Guido van Rossum (python.org/~guido)

From martin at v.loewis.de  Thu Nov  5 21:08:07 2009
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Thu, 05 Nov 2009 21:08:07 +0100
Subject: [Python-Dev] 2.7 Release? 2.7 == last of the 2.x line?
In-Reply-To: <4AF2C032.4040605@gmail.com>
References: <E077843B-919A-4AC7-95D7-0313E4D0328A@gmail.com>	<4AEFD9AE.7090104@molden.no>	<4AEFF450.9060007@voidspace.org.uk>	<79EADA413EFB44F6864EF71D5B830088@RaymondLaptop1>	<79990c6b0911030323x47a9f008ue7256c56e88c0ee8@mail.gmail.com>	<5CB59A21-D8C9-4BC1-A1F6-4438B86A8BC5@twistedmatrix.com>	<4AF1CFAD.6000305@v.loewis.de>	<da7032ce0911041130h40ebd3adu49cd9b08734ce74f@mail.gmail.com>
	<4AF2C032.4040605@gmail.com>
Message-ID: <4AF330A7.5000809@v.loewis.de>

> Mike Krell wrote:
>>     Well, 3to2 would then be an option for you: use Python 3 as the source
>>     language.
> 
> Making life easier for 3to2 is an *excellent* rationale for backports.
> 

I'm skeptical. If new features get added to 2.7: why would that simplify
3to2? It couldn't *use* these features, since it surely would have to
support 2.6 and earlier as well.

Not sure what 3to2 would do about difficult-to-convert 3.x feature (as,
perhaps, the nonlocal keyword). If it currently gives up, it then may
offer you to restrict your target versions to 2.7+. Not sure whether
users would use that option, though - perhaps they rather stop using
nonlocal in 3.x if 3to2 cannot support it for all 2.x versions they are
interested in.

Perhaps 3to2 has a work-around that still provides a good backport in
most cases. Then, the backport would not make the tool any simpler: if
3to2 would start using the backport, it would actually get more
complicated (not easier), as it now needs to support two cases.

Regards,
Martin

From regebro at gmail.com  Thu Nov  5 21:14:03 2009
From: regebro at gmail.com (Lennart Regebro)
Date: Thu, 5 Nov 2009 21:14:03 +0100
Subject: [Python-Dev] 2.7 Release? 2.7 == last of the 2.x line?
In-Reply-To: <4AF1CFAD.6000305@v.loewis.de>
References: <E077843B-919A-4AC7-95D7-0313E4D0328A@gmail.com>
	<4AEFD9AE.7090104@molden.no> <4AEFF450.9060007@voidspace.org.uk>
	<79EADA413EFB44F6864EF71D5B830088@RaymondLaptop1>
	<79990c6b0911030323x47a9f008ue7256c56e88c0ee8@mail.gmail.com>
	<5CB59A21-D8C9-4BC1-A1F6-4438B86A8BC5@twistedmatrix.com>
	<4AF1CFAD.6000305@v.loewis.de>
Message-ID: <319e029f0911051214k7e5e69bfnddff9b0572765bfe@mail.gmail.com>

2009/11/4 "Martin v. L?wis" <martin at v.loewis.de>:
>> Keep in mind also that the 2.x translation process is extremely slow and
>> results in a clunky development process. ?There's no '2to3
>> --interactive' commandline that lets me type python 2 at a >>> prompt
>> and get python 3 results out so that I can try experiments on the 3.x
>> interpreter; I have to actually put my experiments into my unit tests
>> and wait 10 minutes to see if it works. ?It's like writing C++.
>
> That's not my experience. I see a change in source (say, on Django)
> available for 3.x within 5 seconds.

True, but you need to set up a process that will convert only the
changed files, and before Distribute came with 3.0 support, that was
tedious. Now it's easy, if you want to use distribute. (Except that
there is some bug I promised to look at this week, but haven't....)

-- 
Lennart Regebro: Python, Zope, Plone, Grok
http://regebro.wordpress.com/
+33 661 58 14 64

From fuzzyman at voidspace.org.uk  Thu Nov  5 21:22:43 2009
From: fuzzyman at voidspace.org.uk (Michael Foord)
Date: Thu, 05 Nov 2009 21:22:43 +0100
Subject: [Python-Dev] 2.7 Release? 2.7 == last of the 2.x line?
In-Reply-To: <4AF330A7.5000809@v.loewis.de>
References: <E077843B-919A-4AC7-95D7-0313E4D0328A@gmail.com>	<4AEFD9AE.7090104@molden.no>	<4AEFF450.9060007@voidspace.org.uk>	<79EADA413EFB44F6864EF71D5B830088@RaymondLaptop1>	<79990c6b0911030323x47a9f008ue7256c56e88c0ee8@mail.gmail.com>	<5CB59A21-D8C9-4BC1-A1F6-4438B86A8BC5@twistedmatrix.com>	<4AF1CFAD.6000305@v.loewis.de>	<da7032ce0911041130h40ebd3adu49cd9b08734ce74f@mail.gmail.com>	<4AF2C032.4040605@gmail.com>
	<4AF330A7.5000809@v.loewis.de>
Message-ID: <4AF33413.9060408@voidspace.org.uk>

Martin v. L?wis wrote:
>> Mike Krell wrote:
>>     
>>>     Well, 3to2 would then be an option for you: use Python 3 as the source
>>>     language.
>>>       
>> Making life easier for 3to2 is an *excellent* rationale for backports.
>>
>>     
>
> I'm skeptical. If new features get added to 2.7: why would that simplify
> 3to2? It couldn't *use* these features, since it surely would have to
> support 2.6 and earlier as well.
>
> Not sure what 3to2 would do about difficult-to-convert 3.x feature (as,
> perhaps, the nonlocal keyword). If it currently gives up, it then may
> offer you to restrict your target versions to 2.7+. Not sure whether
> users would use that option, though - perhaps they rather stop using
> nonlocal in 3.x if 3to2 cannot support it for all 2.x versions they are
> interested in.
>   

I would have thought you could translate nonlocal with the following:

Python 3:

def scope():
    name = value
    do_something_with(name)
    def inner():
        nonlocal name
        name = new_value
        do_something_else(name)

Python 2

def scope():
    name = [value]
    do_something_with(name[0])
    def inner():
        name[0] = new_value
        do_something_else(name[0])

I would love to see nonlocal backported to 2.7 as it cleans up a simple 
pattern that I use relatively often for testing.


Suppose you have an class and you want to test that method a calls 
method b, in Python 2 you might write something like this:

def test_method_a_calls_method_b():
   instance = SomeClass()
   was_called = []
   def method_b():
      was_called.append(True)

   instance.method_b = method_b
   instance.method_a()

   assert was_called == [True]

in Python 3 you can replace this with the slightly nicer:


def test_method_a_calls_method_b():
   instance = SomeClass()
   was_called = False
   def method_b():
      nonlocal was_called
      was_called = True

   instance.method_b = method_b
   instance.method_a()

   assert was_called

As to the argument that releasing 2.7 is pointless as few people would 
use it for several years, the success of Python 2.6 shows that although 
*many* people don't / can't use new versions of Python for several years 
many other people are able to and do use new versions of Python.

All the best,

Michael Foord

> Perhaps 3to2 has a work-around that still provides a good backport in
> most cases. Then, the backport would not make the tool any simpler: if
> 3to2 would start using the backport, it would actually get more
> complicated (not easier), as it now needs to support two cases.
>
> Regards,
> Martin
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk
>   


-- 
http://www.ironpythoninaction.com/


From eric at trueblade.com  Thu Nov  5 21:22:42 2009
From: eric at trueblade.com (Eric Smith)
Date: Thu, 05 Nov 2009 15:22:42 -0500
Subject: [Python-Dev] 2.7 Release? 2.7 == last of the 2.x line?
In-Reply-To: <4AF330A7.5000809@v.loewis.de>
References: <E077843B-919A-4AC7-95D7-0313E4D0328A@gmail.com>	<4AEFD9AE.7090104@molden.no>	<4AEFF450.9060007@voidspace.org.uk>	<79EADA413EFB44F6864EF71D5B830088@RaymondLaptop1>	<79990c6b0911030323x47a9f008ue7256c56e88c0ee8@mail.gmail.com>	<5CB59A21-D8C9-4BC1-A1F6-4438B86A8BC5@twistedmatrix.com>	<4AF1CFAD.6000305@v.loewis.de>	<da7032ce0911041130h40ebd3adu49cd9b08734ce74f@mail.gmail.com>	<4AF2C032.4040605@gmail.com>
	<4AF330A7.5000809@v.loewis.de>
Message-ID: <4AF33412.3070104@trueblade.com>

Martin v. L?wis wrote:
>> Mike Krell wrote:
>>>     Well, 3to2 would then be an option for you: use Python 3 as the source
>>>     language.
>> Making life easier for 3to2 is an *excellent* rationale for backports.
>>
> 
> I'm skeptical. If new features get added to 2.7: why would that simplify
> 3to2? It couldn't *use* these features, since it surely would have to
> support 2.6 and earlier as well.
> 
> Not sure what 3to2 would do about difficult-to-convert 3.x feature (as,
> perhaps, the nonlocal keyword). If it currently gives up, it then may
> offer you to restrict your target versions to 2.7+. Not sure whether
> users would use that option, though - perhaps they rather stop using
> nonlocal in 3.x if 3to2 cannot support it for all 2.x versions they are
> interested in.

But surely someday 2.7 will be the oldest targetted 2.x version of 
Python for 3to2 and other tools (regardless of whether there's a 2.8). 
When that day comes, 3to2 can be made simpler, or can increase the 
amount of Python 3.x it can convert (or both) if we add 3.x features to 2.7.

Of course, planning for a time so far in the future is difficult, and 
possibly pointless.

From regebro at gmail.com  Thu Nov  5 21:31:54 2009
From: regebro at gmail.com (Lennart Regebro)
Date: Thu, 5 Nov 2009 21:31:54 +0100
Subject: [Python-Dev] 2.7 Release? 2.7 == last of the 2.x line?
In-Reply-To: <B4DEA4E2-E808-4A07-8A9A-AFDEB5A0CD68@gmail.com>
References: <E077843B-919A-4AC7-95D7-0313E4D0328A@gmail.com>
	<F245110A-2FDF-460B-8E0D-87785F0E6DB0@fuhm.net>
	<BBB79ADA-7E31-4DD8-AAA5-07537FA74C5A@gmail.com>
	<319e029f0911032206y64386499hf4564839ec9acb1a@mail.gmail.com>
	<B4DEA4E2-E808-4A07-8A9A-AFDEB5A0CD68@gmail.com>
Message-ID: <319e029f0911051231i44775539p407e52eb872b9119@mail.gmail.com>

2009/11/4 ssteinerX at gmail.com <ssteinerx at gmail.com>:
> Maybe the 3.x line should just be put out of our misery, merged back to 2.7,
> 2.8, 2.9, and proceed as Glyph suggested in passing with increasing levels
> of deprecation until it just turns into 3.x on its own by running out of
> numbers.

Yeah, maybe. If people haven't moved over to Python 3 in 2015 I think
we should start considering it. Let's discuss this again then.

-- 
Lennart Regebro: Python, Zope, Plone, Grok
http://regebro.wordpress.com/
+33 661 58 14 64

From regebro at gmail.com  Thu Nov  5 21:36:09 2009
From: regebro at gmail.com (Lennart Regebro)
Date: Thu, 5 Nov 2009 21:36:09 +0100
Subject: [Python-Dev] Py3k bytes type in 2.x (Re: nonlocal keyword in
	2.x?)
In-Reply-To: <4AF17FB7.5090708@gmail.com>
References: <da7032ce0910211856g45e45632n550aa5b4a10f9a62@mail.gmail.com>
	<ca471dc20911021333r168e257eja1e5f79dbe4ae234@mail.gmail.com>
	<loom.20091102T223839-797@post.gmane.org>
	<4AEFF3A3.3020103@egenix.com> <4AF06E20.7050307@v.loewis.de>
	<4AF0816A.706@egenix.com>
	<319e029f0911032306j746bc263jcf6d0dfeaba9f24d@mail.gmail.com>
	<4AF16CC7.6000705@gmail.com> <4AF171AD.1000703@egenix.com>
	<4AF17FB7.5090708@gmail.com>
Message-ID: <319e029f0911051236q7219e7bcp68f3892c24081a7d@mail.gmail.com>

2009/11/4 Nick Coghlan <ncoghlan at gmail.com>:
> In writing it up, it occurred to me that having that kind of thing in a
> "py3_compat" compatibility module (to be used as, e.g., "from py3_compat
> import str, bytes") would not only make it easier to use in multiple
> modules, but also easier for 2to3 to remove when forward porting.

Well, when using 2to3 it already handles that stuff. But a module like
that would be very handy if you want to support both 2.6 and 3.x
without 2to3. With such a module it would be quite simple.

In fact, I think the module should be called "timemachine". ;-)

-- 
Lennart Regebro: Python, Zope, Plone, Grok
http://regebro.wordpress.com/
+33 661 58 14 64

From regebro at gmail.com  Thu Nov  5 21:38:32 2009
From: regebro at gmail.com (Lennart Regebro)
Date: Thu, 5 Nov 2009 21:38:32 +0100
Subject: [Python-Dev] Py3k bytes type in 2.x (Re: nonlocal keyword in
	2.x?)
In-Reply-To: <4AF16CC7.6000705@gmail.com>
References: <da7032ce0910211856g45e45632n550aa5b4a10f9a62@mail.gmail.com>
	<4AEF4C53.7090902@v.loewis.de>
	<Pine.LNX.4.64.0911021624110.21420@kimball.webabinitio.net>
	<ca471dc20911021333r168e257eja1e5f79dbe4ae234@mail.gmail.com>
	<loom.20091102T223839-797@post.gmane.org>
	<4AEFF3A3.3020103@egenix.com> <4AF06E20.7050307@v.loewis.de>
	<4AF0816A.706@egenix.com>
	<319e029f0911032306j746bc263jcf6d0dfeaba9f24d@mail.gmail.com>
	<4AF16CC7.6000705@gmail.com>
Message-ID: <319e029f0911051238v232dffffred78247f5011d1a9@mail.gmail.com>

2009/11/4 Nick Coghlan <ncoghlan at gmail.com>:
> Lennart Regebro wrote:
>> I also would really like to see a real port of the bytes class to 2.6,
>> but I have a vague memory that there was some reason that wouldn't
>> work.
>
> Not so much that it wouldn't work, but that the interfaces to support
> using it effectively really aren't there - lots of areas in the standard
> library needed to be tweaked to cope with bytes objects in 3.x.

Ah, right, that was the problem, the standard library. I knew I heard
a good reason against it.

> Generally speaking, the "bytes = str" trick represents a reasonable
> compromise as the APIs that you would pass a bytes object to in 3.x
> expect an 8-bit str instance in 2.x.

Yeah, the problem being that bytes and str has quite different API's. Ah well.

-- 
Lennart Regebro: Python, Zope, Plone, Grok
http://regebro.wordpress.com/
+33 661 58 14 64

From brett at python.org  Thu Nov  5 21:39:08 2009
From: brett at python.org (Brett Cannon)
Date: Thu, 5 Nov 2009 12:39:08 -0800
Subject: [Python-Dev] 2.7 Release? 2.7 == last of the 2.x line?
In-Reply-To: <9d153b7c0911050934k714aca73q62dd78cf9033584e@mail.gmail.com>
References: <E077843B-919A-4AC7-95D7-0313E4D0328A@gmail.com> 
	<F245110A-2FDF-460B-8E0D-87785F0E6DB0@fuhm.net>
	<BBB79ADA-7E31-4DD8-AAA5-07537FA74C5A@gmail.com> 
	<319e029f0911032206y64386499hf4564839ec9acb1a@mail.gmail.com> 
	<B4DEA4E2-E808-4A07-8A9A-AFDEB5A0CD68@gmail.com>
	<426ada670911041039m6df85994v321342231fbd3805@mail.gmail.com> 
	<4AF2C5B8.8070204@gmail.com>
	<9d153b7c0911050934k714aca73q62dd78cf9033584e@mail.gmail.com>
Message-ID: <bbaeab100911051239l11eead6laade84892215a5bf@mail.gmail.com>

On Thu, Nov 5, 2009 at 09:34, Yuvgoog Greenle <ubershmekel at gmail.com> wrote:
> On Thu, Nov 5, 2009 at 2:31 PM, Nick Coghlan <ncoghlan at gmail.com> wrote:
>>
>> While it may be hard to tell without knowing who is and isn't a core
>> developer
>
> Maybe there should be badges or something, hmmm, sounds like making an
> svn-commits-hall-of-fame for python could be a nice project.
>

You might be interested in http://www.ohloh.net/p/python/contributors .

-Brett


> --yuv
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> http://mail.python.org/mailman/options/python-dev/brett%40python.org
>
>

From stefan_ml at behnel.de  Thu Nov  5 21:41:07 2009
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Thu, 05 Nov 2009 21:41:07 +0100
Subject: [Python-Dev] PEP 3003 - Python Language Moratorium
In-Reply-To: <1A472770E042064698CB5ADC83A12ACD04C4A6C6@TK5EX14MBXC118.redmond.corp.microsoft.com>
References: <ca471dc20911030935l25251e0diec64427f5d7bcbf2@mail.gmail.com>	<hctuqq$in0$1@ger.gmane.org>	<4AF29753.3040801@voidspace.org.uk>	<hcu8ct$dog$1@ger.gmane.org>
	<1A472770E042064698CB5ADC83A12ACD04C4A6C6@TK5EX14MBXC118.redmond.corp.microsoft.com>
Message-ID: <hcvd93$i2j$1@ger.gmane.org>

Dino Viehland, 05.11.2009 19:35:
> Stefan wrote:
>> It /does/ make some static assumptions in that it considers builtins
>> true
>> builtins. However, it does not prevent you from replacing them in your
>> code, as long as you do it inside the module. Certainly a restriction
>> compared to Python, where you can import a module into a changed dict
>> environment that redefines 'object', but not a major restriction IMO,
>> and certainly not one that impacts much code.
> 
> To me this is a deal breaker which prevents Cython from being a Python
> implementation.  From a talk given by Colin Winter at the LLVM dev meeting
> (http://llvm.org/devmtg/2009-10/) it seems like Unladen Swallow wanted to
> do something like this as well and Guido said no.  In this case the breaking
> change is so subtle that I'd personally hate to run into something like 
> this porting code to Cython and having to figure out why it's not working.

I assume that this is artificially exaggerated to make a point, as this
behaviour is obviously not a technical requirement but an optimisation,
which could potentially be disabled.

Stefan


From chris at subtlety.com  Thu Nov  5 21:43:32 2009
From: chris at subtlety.com (Chris Bergstresser)
Date: Thu, 5 Nov 2009 14:43:32 -0600
Subject: [Python-Dev] Retrieve an arbitrary element from a
	setwithoutremoving it
In-Reply-To: <99BB308202224D31AA5300223DB59912@RaymondLaptop1>
References: <7b0045ed0910301843y7059d316s8a851142b78b7aeb@mail.gmail.com> 
	<200911040946.15805.steve@pearwood.info>
	<4AF0D0D7.3090403@canterbury.ac.nz> 
	<200911042234.36796.steve@pearwood.info>
	<f3cc57c60911041332v479cc3b0t19d9601e52002128@mail.gmail.com> 
	<99BB308202224D31AA5300223DB59912@RaymondLaptop1>
Message-ID: <7b0045ed0911051243q18f8324ak9abaa98ac26948e9@mail.gmail.com>

On Wed, Nov 4, 2009 at 7:07 PM, Raymond Hettinger <python at rcn.com> wrote:
> [Steven D'Aprano]
>>> Anyway, given the level of opposition to the suggestion, I'm no longer
>>> willing to carry the flag for it. If anyone else -- perhaps the OP --
>>> feels they want to take it any further, be my guest.

   I feel pretty strongly that it's a wart in the language, and a
sufficiently strong one that it should be remedied.  I'm happy to
champion it, but haven't the faintest idea what that entails.

> Summarizing my opposition to a new set method:
> 1) there already are at least two succinct ways to get the same effect
> 2) those ways work with any container, not just sets
> 3) set implementations in other languages show that this isn't needed.
> 4) there is value to keeping the API compact
> 5) isn't needed for optimization (selecting the same value in a loop makes
> no sense)
> 6) absence of real-world code examples that would be meaningfully improved
>
> I would be happy to add an example to the docs so that this thread
> can finally end.

   Adding an example to the docs does not solve the problem, which is
if you come across the following code:

 for x in s:
     break

... it really looks like it does nothing.  It's only because of the
slightly idiosyncratic way Python handles variable scoping that it has
an effect at all, and that effect isn't overtly related to what the
code says, which is "Iterate over all the elements in this set, then
immediately stop after the first one".  s.get() or s.pick() are both
more succinct and more clear, saying "Get me an arbitrary element from
this set".  To make matters worse, "for x in s: break" fails silently
when s is empty, and "x = iter(s).next()" raises a StopIteration
exception.  Neither is clear.
   The obvious way, for newcomers, of achieving the effect is:

 x = s.pop()
 s.add(x)

... and that's simply horrible in terms of efficiency.  So the
"obvious" way of doing it in Python is wrong(TM), and the "correct"
way of doing it is obscure and raises misleading exceptions.

    I suppose, mulling things over, the method should be called
.pick(), which avoids any confusion with .get().  And, as I've stated,
I think it should return a member of the set, with no guarantees what
member of the set is returned.  It could be the same one every time,
or a random one, or the last one placed in the set.
   For cases where people want to cycle through the members of the set
in a predictable order, they can either copy the contents into a list
(sorted or unsorted) *or* subclass set and override the .pick() method
to place stronger guarantees on the API.

So, summarizing my responses:

1) the two succinct ways are unclear and not immediately obvious
2) the existing methods aren't needed for other objects
3) set implementations in other languages are irrelevant
4) this is a small, targeted change which not make the API disordered or unruly
5) could very well be needed for optimization, in cases where
constructing an iterator is expensive
6) there have been several real-world examples posted which would be
improved by this change

-- Chris

From alexander.belopolsky at gmail.com  Thu Nov  5 22:09:11 2009
From: alexander.belopolsky at gmail.com (Alexander Belopolsky)
Date: Thu, 5 Nov 2009 16:09:11 -0500
Subject: [Python-Dev] Retrieve an arbitrary element from a
	setwithoutremoving it
In-Reply-To: <7b0045ed0911051243q18f8324ak9abaa98ac26948e9@mail.gmail.com>
References: <7b0045ed0910301843y7059d316s8a851142b78b7aeb@mail.gmail.com>
	<200911040946.15805.steve@pearwood.info>
	<4AF0D0D7.3090403@canterbury.ac.nz>
	<200911042234.36796.steve@pearwood.info>
	<f3cc57c60911041332v479cc3b0t19d9601e52002128@mail.gmail.com>
	<99BB308202224D31AA5300223DB59912@RaymondLaptop1>
	<7b0045ed0911051243q18f8324ak9abaa98ac26948e9@mail.gmail.com>
Message-ID: <d38f5330911051309l78166534ndf8320e09f9892bc@mail.gmail.com>

On Thu, Nov 5, 2009 at 3:43 PM, Chris Bergstresser <chris at subtlety.com> wrote:
> .. and "x = iter(s).next()" raises a StopIteration
> exception.

And that's why the documented recipe should probably recommend
next(iter(s), default) instead.  Especially because iter(s).next() is
not even valid code in 3.0.

From martin at v.loewis.de  Thu Nov  5 22:21:01 2009
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Thu, 05 Nov 2009 22:21:01 +0100
Subject: [Python-Dev] Retrieve an arbitrary element from
 a	setwithoutremoving it
In-Reply-To: <7b0045ed0911051243q18f8324ak9abaa98ac26948e9@mail.gmail.com>
References: <7b0045ed0910301843y7059d316s8a851142b78b7aeb@mail.gmail.com>
	<200911040946.15805.steve@pearwood.info>	<4AF0D0D7.3090403@canterbury.ac.nz>
	<200911042234.36796.steve@pearwood.info>	<f3cc57c60911041332v479cc3b0t19d9601e52002128@mail.gmail.com>
	<99BB308202224D31AA5300223DB59912@RaymondLaptop1>
	<7b0045ed0911051243q18f8324ak9abaa98ac26948e9@mail.gmail.com>
Message-ID: <4AF341BD.2010203@v.loewis.de>

>    I feel pretty strongly that it's a wart in the language, and a
> sufficiently strong one that it should be remedied.  I'm happy to
> champion it, but haven't the faintest idea what that entails.

There are two ways

a) write a library that provides what you want, publish it on PyPI,
   and report back in a few years of how many users your library has,
   what they use it for, and why it should become builtin
b) write a PEP, wait a few years for the language moratorium to be
   lifted, provide an implementation, and put the PEP for pronouncement.
   Careful reading of the Moratorium PEP may allow shortening of the
   wait.

In any case, it seems that this specific change will see some
opposition. So you will need to convince the opposition, one way or
the other.

>    The obvious way, for newcomers, of achieving the effect is:
> 
>  x = s.pop()
>  s.add(x)
> 
> ... and that's simply horrible in terms of efficiency.  So the
> "obvious" way of doing it in Python is wrong(TM), and the "correct"
> way of doing it is obscure and raises misleading exceptions.

If you chose to write a PEP, include a proof that this approach is
indeed horrible in terms of efficiency; I question that.

Regards,
Martin

From solipsis at pitrou.net  Thu Nov  5 22:46:56 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Thu, 5 Nov 2009 21:46:56 +0000 (UTC)
Subject: [Python-Dev] 2.7 Release? 2.7 == last of the 2.x line?
References: <E077843B-919A-4AC7-95D7-0313E4D0328A@gmail.com>	<4AEFD9AE.7090104@molden.no>	<4AEFF450.9060007@voidspace.org.uk>	<79EADA413EFB44F6864EF71D5B830088@RaymondLaptop1>	<79990c6b0911030323x47a9f008ue7256c56e88c0ee8@mail.gmail.com>	<5CB59A21-D8C9-4BC1-A1F6-4438B86A8BC5@twistedmatrix.com>	<4AF1CFAD.6000305@v.loewis.de>	<da7032ce0911041130h40ebd3adu49cd9b08734ce74f@mail.gmail.com>	<4AF2C032.4040605@gmail.com>
	<4AF330A7.5000809@v.loewis.de> <4AF33413.9060408@voidspace.org.uk>
Message-ID: <loom.20091105T224449-968@post.gmane.org>

Michael Foord <fuzzyman <at> voidspace.org.uk> writes:
> 
> I would love to see nonlocal backported to 2.7 as it cleans up a simple 
> pattern that I use relatively often for testing.

Well you know I'm sure that if someone proposes a proper patch it will
eventually get accepted ;)

Regards

Antoine.



From martin at v.loewis.de  Thu Nov  5 22:51:31 2009
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Thu, 05 Nov 2009 22:51:31 +0100
Subject: [Python-Dev] No buildbot to test wide unicode?
In-Reply-To: <Pine.LNX.4.64.0911051125270.11243@kimball.webabinitio.net>
References: <e27efe130911031557n7839ab25g28fc6dafe3174ab7@mail.gmail.com>	<4AF1D14A.6040007@v.loewis.de>
	<hcut5b$n6b$1@ger.gmane.org>
	<Pine.LNX.4.64.0911051125270.11243@kimball.webabinitio.net>
Message-ID: <4AF348E3.1080200@v.loewis.de>

>> I would certainly agree to setup two slaves on mine. There are ample
>> resources available.
> 
> I could do so as well.  Gentoo is one of the distributions that uses
> the wide build by default, so that would make it a good test candidate
> as well.

I have now setup two slaves, murray-gentoo-wide and pitrou-ubuntu-wide,
same passwords. They configure with the option "--with-wide-unicode"
(if I did that correctly).

I think this means that the 2.x branches will have to grow this option
also (although configure should ignore the option, so it should still
build).

Regards,
Martin

From dinov at microsoft.com  Thu Nov  5 22:55:27 2009
From: dinov at microsoft.com (Dino Viehland)
Date: Thu, 5 Nov 2009 21:55:27 +0000
Subject: [Python-Dev] PEP 3003 - Python Language Moratorium
In-Reply-To: <hcvd93$i2j$1@ger.gmane.org>
References: <ca471dc20911030935l25251e0diec64427f5d7bcbf2@mail.gmail.com>
	<hctuqq$in0$1@ger.gmane.org>	<4AF29753.3040801@voidspace.org.uk>
	<hcu8ct$dog$1@ger.gmane.org>
	<1A472770E042064698CB5ADC83A12ACD04C4A6C6@TK5EX14MBXC118.redmond.corp.microsoft.com>
	<hcvd93$i2j$1@ger.gmane.org>
Message-ID: <1A472770E042064698CB5ADC83A12ACD04C4BACB@TK5EX14MBXC118.redmond.corp.microsoft.com>

Stefan wrote:
> I assume that this is artificially exaggerated to make a point, as this
> behaviour is obviously not a technical requirement but an optimisation,
> which could potentially be disabled.
> 

If there's a way to disable this then that's fine and IMO when it was disabled
you'd still be Python.  But changing behavior in the name of optimization is 
no longer just an optimization.

For what it's worth in IronPython we're going to look at making this faster
in the future as well.  We already cache built-in values locally in a module
and invalidate the cache when the builtins changes.  In the future I'd like to 
even combine loading a global and invoking it into a single call site for even 
better optimizations.  And you could even imagine an extreme scenario
where when built-ins changed all the code in the system that depended upon
them is re-compiled punishing users who actually do this but I doubt we'll go
that far.  

But the important thing is that the optimizations don't change the language 
behavior otherwise I don't think you're still Python.  But of course that's 
what makes it so challenging and fun to try and optimize Python :)




From guido at python.org  Thu Nov  5 23:05:17 2009
From: guido at python.org (Guido van Rossum)
Date: Thu, 5 Nov 2009 14:05:17 -0800
Subject: [Python-Dev] PEP 3003 - Python Language Moratorium
In-Reply-To: <ca471dc20911030935l25251e0diec64427f5d7bcbf2@mail.gmail.com>
References: <ca471dc20911030935l25251e0diec64427f5d7bcbf2@mail.gmail.com>
Message-ID: <ca471dc20911051405o287e0f41m26d38860a200b47f@mail.gmail.com>

On Tue, Nov 3, 2009 at 9:35 AM, Guido van Rossum <guido at python.org> wrote:
> I've checked draft (!) PEP 3003, "Python Language Moratorium", into
> SVN. As authors I've listed Jesse, Brett and myself.

I haven't seen substantial opposition against the PEP -- in fact I
can't recall any, and many people have explicitly posted in support of
it. So unless opposition suddenly appears in the next few days, I'll
move it to the Accepted state next Monday.

-- 
--Guido van Rossum (python.org/~guido)

From python at rcn.com  Thu Nov  5 23:11:20 2009
From: python at rcn.com (Raymond Hettinger)
Date: Thu, 5 Nov 2009 14:11:20 -0800
Subject: [Python-Dev] PEP 3003 - Python Language Moratorium
References: <ca471dc20911030935l25251e0diec64427f5d7bcbf2@mail.gmail.com>
	<ca471dc20911051405o287e0f41m26d38860a200b47f@mail.gmail.com>
Message-ID: <AB6938AB50B747EEB35257E266F810A5@RaymondLaptop1>


[GvR]
> I haven't seen substantial opposition against the PEP -- in fact I
> can't recall any, and many people have explicitly posted in support of
> it. So unless opposition suddenly appears in the next few days, I'll
> move it to the Accepted state next Monday.

But it would have been so much fun to have a never ending python-ideas
thread on BEGIN/END blocks ;-)

C'est la vie,


Raymond

From arfrever.fta at gmail.com  Thu Nov  5 23:20:29 2009
From: arfrever.fta at gmail.com (Arfrever Frehtes Taifersar Arahesis)
Date: Thu, 5 Nov 2009 23:20:29 +0100
Subject: [Python-Dev] PEP 3003 - Python Language Moratorium
In-Reply-To: <ca471dc20911030935l25251e0diec64427f5d7bcbf2@mail.gmail.com>
References: <ca471dc20911030935l25251e0diec64427f5d7bcbf2@mail.gmail.com>
Message-ID: <200911052320.30872.Arfrever.FTA@gmail.com>

2009-11-03 18:35:10 Guido van Rossum napisa?(a):
> I've checked draft (!) PEP 3003, "Python Language Moratorium", into
> SVN. As authors I've listed Jesse, Brett and myself.
> 
> On python-ideas the moratorium idea got fairly positive responses
> (more positive than I'd expected, in fact) but I'm bracing myself for
> fierce discussion here on python-dev. It's important to me that if if
> this is accepted it is a "rough consensus" decision (working code we
> already have plenty of :-), not something enforced by a vocal minority
> or an influential individual such as myself. If there's too much
> opposition I'll withdraw the PEP so as not to waste everybody's time
> with a fruitless discussion.
> 
> The PEP tries to spell out some gray areas but I'm sure there will be
> others; that's life. Do note that the PEP proposes to be *retroactive*
> back to the 3.1 release, i.e. the "frozen" version of the language is
> the state in which it was released as 3.1.

Does moratorium allow to add support for e.g. 'from __future__ import yield_from'
in Python 3.2?

-- 
Arfrever Frehtes Taifersar Arahesis

From brett at python.org  Thu Nov  5 23:18:46 2009
From: brett at python.org (Brett Cannon)
Date: Thu, 5 Nov 2009 14:18:46 -0800
Subject: [Python-Dev] PEP 3003 - Python Language Moratorium
In-Reply-To: <200911052320.30872.Arfrever.FTA@gmail.com>
References: <ca471dc20911030935l25251e0diec64427f5d7bcbf2@mail.gmail.com> 
	<200911052320.30872.Arfrever.FTA@gmail.com>
Message-ID: <bbaeab100911051418r2fb99e23h90b45e3830ddc3bd@mail.gmail.com>

On Thu, Nov 5, 2009 at 14:20, Arfrever Frehtes Taifersar Arahesis
<arfrever.fta at gmail.com> wrote:
> 2009-11-03 18:35:10 Guido van Rossum napisa?(a):
>> I've checked draft (!) PEP 3003, "Python Language Moratorium", into
>> SVN. As authors I've listed Jesse, Brett and myself.
>>
>> On python-ideas the moratorium idea got fairly positive responses
>> (more positive than I'd expected, in fact) but I'm bracing myself for
>> fierce discussion here on python-dev. It's important to me that if if
>> this is accepted it is a "rough consensus" decision (working code we
>> already have plenty of :-), not something enforced by a vocal minority
>> or an influential individual such as myself. If there's too much
>> opposition I'll withdraw the PEP so as not to waste everybody's time
>> with a fruitless discussion.
>>
>> The PEP tries to spell out some gray areas but I'm sure there will be
>> others; that's life. Do note that the PEP proposes to be *retroactive*
>> back to the 3.1 release, i.e. the "frozen" version of the language is
>> the state in which it was released as 3.1.
>
> Does moratorium allow to add support for e.g. 'from __future__ import yield_from'
> in Python 3.2?

No.

-Brett

From chris at subtlety.com  Thu Nov  5 23:19:32 2009
From: chris at subtlety.com (Chris Bergstresser)
Date: Thu, 5 Nov 2009 16:19:32 -0600
Subject: [Python-Dev] Retrieve an arbitrary element from a
	setwithoutremoving it
In-Reply-To: <4AF341BD.2010203@v.loewis.de>
References: <7b0045ed0910301843y7059d316s8a851142b78b7aeb@mail.gmail.com> 
	<200911040946.15805.steve@pearwood.info>
	<4AF0D0D7.3090403@canterbury.ac.nz> 
	<200911042234.36796.steve@pearwood.info>
	<f3cc57c60911041332v479cc3b0t19d9601e52002128@mail.gmail.com> 
	<99BB308202224D31AA5300223DB59912@RaymondLaptop1>
	<7b0045ed0911051243q18f8324ak9abaa98ac26948e9@mail.gmail.com> 
	<4AF341BD.2010203@v.loewis.de>
Message-ID: <7b0045ed0911051419i76f4221dye8ecccc424179ec6@mail.gmail.com>

On Thu, Nov 5, 2009 at 3:21 PM, "Martin v. L?wis" <martin at v.loewis.de> wrote:
> There are two ways
>
> a) write a library that provides what you want, publish it on PyPI,
> ? and report back in a few years of how many users your library has,
> ? what they use it for, and why it should become builtin

    This clearly isn't called for in this case.  We're talking about a
single function on a collection.  In this case, importing an
alternative set API (and maintaining the dependency) is more work than
just writing your own workaround.  The purpose of adding a method is
to prevent the need of everyone writing their own workaround.

> b) write a PEP, wait a few years for the language moratorium to be
> ? lifted, provide an implementation, and put the PEP for pronouncement.
> ? Careful reading of the Moratorium PEP may allow shortening of the
> ? wait.

   Clearly, I'll need to write up the PEP.

> In any case, it seems that this specific change will see some
> opposition. So you will need to convince the opposition, one way or
> the other.

   I doubt some of the people on either side are going to be
convinced.  I'd settle for convincing most of the fence-sitters, along
with a few of the loyal opposition.

-- Chris

From python at rcn.com  Thu Nov  5 23:30:56 2009
From: python at rcn.com (Raymond Hettinger)
Date: Thu, 5 Nov 2009 14:30:56 -0800
Subject: [Python-Dev] Retrieve an arbitrary element from a
	setwithoutremoving it
References: <7b0045ed0910301843y7059d316s8a851142b78b7aeb@mail.gmail.com>
	<200911040946.15805.steve@pearwood.info>
	<4AF0D0D7.3090403@canterbury.ac.nz>
	<200911042234.36796.steve@pearwood.info>
	<f3cc57c60911041332v479cc3b0t19d9601e52002128@mail.gmail.com>
	<99BB308202224D31AA5300223DB59912@RaymondLaptop1>
	<7b0045ed0911051243q18f8324ak9abaa98ac26948e9@mail.gmail.com>
	<4AF341BD.2010203@v.loewis.de>
	<7b0045ed0911051419i76f4221dye8ecccc424179ec6@mail.gmail.com>
Message-ID: <BD050FFAAA19443DA0DA34FD2BD740CF@RaymondLaptop1>


[Chris Bergstresser]
 > Clearly, I'll need to write up the PEP.

Why not write a short, fast get_first() function for your utils directory and be done with it?
That could work with sets, mappings, generators, and other containers and iterators.
No need to fatten the set/frozenset API for something so trivial and so rarely needed.


Raymond

From david.lyon at preisshare.net  Thu Nov  5 23:29:22 2009
From: david.lyon at preisshare.net (David Lyon)
Date: Thu, 05 Nov 2009 17:29:22 -0500
Subject: [Python-Dev] Refactoring installation schemes
In-Reply-To: <94bdd2610911050235l4e851612i46e0392d352eaa20@mail.gmail.com>
References: <94bdd2610910271455s7854e6a9u94e3aeeeacb69c02@mail.gmail.com>
	<4AE8206A.9020108@egenix.com> <4AE824DA.4090701@voidspace.org.uk>
	<ED004FA5-9F77-4215-B06D-6CBECEF6D023@gmail.com>
	<94bdd2610910281105q328bc3a5qaed020ef3afe28df@mail.gmail.com>
	<94bdd2610911040429j50a6df87m51aac2464dd13bd7@mail.gmail.com>
	<c77acedd9a48dd7036cca55fde42de0a@preisshare.net>
	<9cee7ab80911041440j43e51e39qc77a426547d02bcb@mail.gmail.com>
	<39deb0d1eef08419dad65d7354a8ffb1@preisshare.net>
	<94bdd2610911050235l4e851612i46e0392d352eaa20@mail.gmail.com>
Message-ID: <0d223447fa9aebaeb78f5be82eb3aaab@preisshare.net>

On Thu, 5 Nov 2009 11:35:41 +0100, Tarek Ziad? <ziade.tarek at gmail.com>
wrote:
> PEP 376 is working on a default, unified, *installation* format, that
> tries to gather the good ideas of Pip, Setuptools etc.. and propose a
> unified format for our site-packages. This new standard will come with
> APIs in pkgutil to be able to query installed distribution etc. This
> work is also linked to PEP 345 work where we are modifying the
> Metadata, and to PEP 386 that proposes a standard version comparison
> scheme.

Perphaps..

But if you put all these PEPs together, implementing all the new features
can't come to more than 300 lines of code...

Since we hardly got anywhere on them in 2009, it will be interesting to
see how much of it gets done in 2010.

> But there's no plan to include a new *distribution* format in Distutils.

I wasn't suggesting that - at all.

And saying that 'eggs' are a *new* python package format isn't really
really helpful because to my understanding they've been around for
some number of years.

No, i won't raise why we have EGG_INFO directories and a whole lot
of half working egg stuff in standard python... I'm just asking
why it can't be more consistant? while we're on the refactoring
topic.

Be fair...

I'm saying finish what is already there.. or take out the crap ..

It isn't fair to suggest that I am somehow asking for some big
change when I am simply pointing out all the junk that's in
there that's already half built.

> In any case those PEPs are not finished yet, so everyone can help at
> distutils-SIG

True - and False.

But I've been on the list for some twelve months asking for work
to help out with, and haven't been assigned a single task to do
yet.

Seriously, if you won't allocate work out to people then how can
it get done?

Whilst I personally think a lot of the stuff in those PEPs is not
high on quality, why don't we just get them implemented anyway?

I'm a fairly proficient develper, but I can't get assigned a single
work item..

And to me, it doesn't seem any harder than just selecting 'djlyon'
on the python tracker for some work items...

Surely those PEPs all amount to 300+ lines of code. With two people
working on it, that's only surely 150+ lines of code each... That
shouldn't be such a big challenge for 2010..

David


From dirkjan at ochtman.nl  Thu Nov  5 23:53:37 2009
From: dirkjan at ochtman.nl (Dirkjan Ochtman)
Date: Thu, 5 Nov 2009 23:53:37 +0100
Subject: [Python-Dev] PEP 3003 - Python Language Moratorium
In-Reply-To: <ca471dc20911051405o287e0f41m26d38860a200b47f@mail.gmail.com>
References: <ca471dc20911030935l25251e0diec64427f5d7bcbf2@mail.gmail.com>
	<ca471dc20911051405o287e0f41m26d38860a200b47f@mail.gmail.com>
Message-ID: <ea2499da0911051453n3569387aj581ad4acf5b799c8@mail.gmail.com>

On Thu, Nov 5, 2009 at 23:05, Guido van Rossum <guido at python.org> wrote:
> I haven't seen substantial opposition against the PEP -- in fact I
> can't recall any, and many people have explicitly posted in support of
> it. So unless opposition suddenly appears in the next few days, I'll
> move it to the Accepted state next Monday.

Let me state first, I think the PEP is great, and I have no objection
to its current form.

I do have one qualm, where I wonder if the PEP shouldn't be a little
stricter. As a gentoo developer and Mercurial maintainer, most of the
pain in the recent migration towards 2.6 has not been in language
changes, but in the standard library. Unfortunately, it's exempt from
the moratorium in the PEP.

Which makes me wonder, why are we not adding another moratorium, on
deprecations in the standard library? In other words, let's not
deprecate things like md5 or sha or the popen family of functions, but
keep all of that as it is, for both 2.x and 3.x, so people can direct
their energy towards other things (hopefully porting their 2.x
codebase to 3.x).

The standard library has already been through a lot of evolution in
the 2.x to 3.x transition, so one might assume there's not a lot of
stuff in the 3.x stdlib that would need deprecation in the short term.
And for 2.x, well, I'd certainly hope we don't need to deprecate much
more there before it finally gets EOL'ed, so it should be a relatively
light maintenance load to bear.

Is this just crazy talk?

Cheers,

Dirkjan

From robert.kern at gmail.com  Thu Nov  5 23:57:40 2009
From: robert.kern at gmail.com (Robert Kern)
Date: Thu, 05 Nov 2009 16:57:40 -0600
Subject: [Python-Dev] Refactoring installation schemes
In-Reply-To: <0d223447fa9aebaeb78f5be82eb3aaab@preisshare.net>
References: <94bdd2610910271455s7854e6a9u94e3aeeeacb69c02@mail.gmail.com>	<4AE8206A.9020108@egenix.com>
	<4AE824DA.4090701@voidspace.org.uk>	<ED004FA5-9F77-4215-B06D-6CBECEF6D023@gmail.com>	<94bdd2610910281105q328bc3a5qaed020ef3afe28df@mail.gmail.com>	<94bdd2610911040429j50a6df87m51aac2464dd13bd7@mail.gmail.com>	<c77acedd9a48dd7036cca55fde42de0a@preisshare.net>	<9cee7ab80911041440j43e51e39qc77a426547d02bcb@mail.gmail.com>	<39deb0d1eef08419dad65d7354a8ffb1@preisshare.net>	<94bdd2610911050235l4e851612i46e0392d352eaa20@mail.gmail.com>
	<0d223447fa9aebaeb78f5be82eb3aaab@preisshare.net>
Message-ID: <hcvl94$e03$1@ger.gmane.org>

On 2009-11-05 16:29 PM, David Lyon wrote:

> But I've been on the list for some twelve months asking for work
> to help out with, and haven't been assigned a single task to do
> yet.
>
> Seriously, if you won't allocate work out to people then how can
> it get done?

With rare exceptions, open source projects don't really "assign" work. If you 
want to help, help. Don't wait for someone to tell you exactly what to do. No 
one will. Go to the tracker, find something interesting, and do it.

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
  that is made terrible by our own mad attempt to interpret it as though it had
  an underlying truth."
   -- Umberto Eco


From python at rcn.com  Fri Nov  6 00:02:01 2009
From: python at rcn.com (Raymond Hettinger)
Date: Thu, 5 Nov 2009 15:02:01 -0800
Subject: [Python-Dev] Retrieve an arbitrary element from
	asetwithoutremoving it
References: <7b0045ed0910301843y7059d316s8a851142b78b7aeb@mail.gmail.com><200911040946.15805.steve@pearwood.info><4AF0D0D7.3090403@canterbury.ac.nz><200911042234.36796.steve@pearwood.info><f3cc57c60911041332v479cc3b0t19d9601e52002128@mail.gmail.com><99BB308202224D31AA5300223DB59912@RaymondLaptop1><7b0045ed0911051243q18f8324ak9abaa98ac26948e9@mail.gmail.com><4AF341BD.2010203@v.loewis.de><7b0045ed0911051419i76f4221dye8ecccc424179ec6@mail.gmail.com>
	<BD050FFAAA19443DA0DA34FD2BD740CF@RaymondLaptop1>
Message-ID: <97069B15530D4C66AC1D4BCCB70E2FD5@RaymondLaptop1>

[me]
> Why not write a short, fast get_first() function for your utils directory and be done with it?
> That could work with sets, mappings, generators, and other containers and iterators.
> No need to fatten the set/frozenset API for something so trivial and so rarely needed.

Forgot to post the code.  It is short, fast, and easy.  It is explicit about handing the case with an empty input.  And it is 
specific about which value it returns (always the first iterated value; not an arbitrary one).  There's no guessing about what it 
does.  It gets the job done.

def first(iterable):
    'Return the first value from a container or iterable.  If empty, raises LookupError'
    for value in iterable:
        return value
    raise LookupError('no first value; iterable is empty')

If desired, it is not hard to change to the last time to return a default value or some other exception.


Raymond




From ziade.tarek at gmail.com  Fri Nov  6 00:03:43 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Fri, 6 Nov 2009 00:03:43 +0100
Subject: [Python-Dev] Refactoring installation schemes
In-Reply-To: <0d223447fa9aebaeb78f5be82eb3aaab@preisshare.net>
References: <94bdd2610910271455s7854e6a9u94e3aeeeacb69c02@mail.gmail.com>
	<4AE824DA.4090701@voidspace.org.uk>
	<ED004FA5-9F77-4215-B06D-6CBECEF6D023@gmail.com>
	<94bdd2610910281105q328bc3a5qaed020ef3afe28df@mail.gmail.com>
	<94bdd2610911040429j50a6df87m51aac2464dd13bd7@mail.gmail.com>
	<c77acedd9a48dd7036cca55fde42de0a@preisshare.net>
	<9cee7ab80911041440j43e51e39qc77a426547d02bcb@mail.gmail.com>
	<39deb0d1eef08419dad65d7354a8ffb1@preisshare.net>
	<94bdd2610911050235l4e851612i46e0392d352eaa20@mail.gmail.com>
	<0d223447fa9aebaeb78f5be82eb3aaab@preisshare.net>
Message-ID: <94bdd2610911051503t68ef0931p2f6200f3e59c07b4@mail.gmail.com>

On Thu, Nov 5, 2009 at 11:29 PM, David Lyon <david.lyon at preisshare.net> wrote:
[...]
>
> But if you put all these PEPs together, implementing all the new features
> can't come to more than 300 lines of code...
>
> Since we hardly got anywhere on them in 2009, it will be interesting to
> see how much of it gets done in 2010.

The work that was done during the last year was brainstorming on PEPs.
You can't count that work
in SLOCs, but in trying to find a consensus on packaging problems.

And I can fairly say that we are very close to something that can get
accepted by the community,
then turned into code in Distutils/pkgutil.

>
>> But there's no plan to include a new *distribution* format in Distutils.
>
> I wasn't suggesting that - at all.
>
> And saying that 'eggs' are a *new* python package format isn't really
> really helpful because to my understanding they've been around for
> some number of years.

They are new to Distutils. They are not new to the Python packaging
eco-system of course,
and PEP 376 rely on many ideas created in Setuptools.

> No, i won't raise why we have EGG_INFO directories and a whole lot
> of half working egg stuff in standard python... I'm just asking
> why it can't be more consistant? while we're on the refactoring
> topic.

The consistency will come through the standard we are building in PEP 376.

[..]
> But I've been on the list for some twelve months asking for work
> to help out with, and haven't been assigned a single task to do
> yet.
>
> Seriously, if you won't allocate work out to people then how can
> it get done?

I am sorry that you feel that way. We don't allocate coding work to
people in this process.
That's not the way it works.

Rather, people help in building the PEPs by providing their own
feedback/experience.

At the end, we are trying to have PEPs that adresses the wider range of cases.


>
> Whilst I personally think a lot of the stuff in those PEPs is not
> high on quality, why don't we just get them implemented anyway?
>
> I'm a fairly proficient develper, but I can't get assigned a single
> work item..

We have already prototypes for each PEP so people can try them out,
enhance them while the PEPs
are being built. If you want to help in their coding, you are more
than welcome. They are on a DVCS
at bitbucket.

>
> And to me, it doesn't seem any harder than just selecting 'djlyon'
> on the python tracker for some work items...
>
> Surely those PEPs all amount to 300+ lines of code. With two people
> working on it, that's only surely 150+ lines of code each... That
> shouldn't be such a big challenge for 2010..

Again, the big challenge is not on the coding part. If it was,
Distutils would have them already.
The challenge is on the PEPs, and on making sure we collect all PoVs
and feedbacks, before we change
Distutils in Python 2.7/3.2

Regards
Tarek

From debatem1 at gmail.com  Fri Nov  6 00:04:54 2009
From: debatem1 at gmail.com (geremy condra)
Date: Thu, 5 Nov 2009 18:04:54 -0500
Subject: [Python-Dev] Retrieve an arbitrary element from a
	setwithoutremoving it
In-Reply-To: <d38f5330911051309l78166534ndf8320e09f9892bc@mail.gmail.com>
References: <7b0045ed0910301843y7059d316s8a851142b78b7aeb@mail.gmail.com>
	<200911040946.15805.steve@pearwood.info>
	<4AF0D0D7.3090403@canterbury.ac.nz>
	<200911042234.36796.steve@pearwood.info>
	<f3cc57c60911041332v479cc3b0t19d9601e52002128@mail.gmail.com>
	<99BB308202224D31AA5300223DB59912@RaymondLaptop1>
	<7b0045ed0911051243q18f8324ak9abaa98ac26948e9@mail.gmail.com>
	<d38f5330911051309l78166534ndf8320e09f9892bc@mail.gmail.com>
Message-ID: <f3cc57c60911051504l48890d09q74d46580bf1c20ab@mail.gmail.com>

On Thu, Nov 5, 2009 at 4:09 PM, Alexander Belopolsky
<alexander.belopolsky at gmail.com> wrote:
> On Thu, Nov 5, 2009 at 3:43 PM, Chris Bergstresser <chris at subtlety.com> wrote:
>> .. and "x = iter(s).next()" raises a StopIteration
>> exception.
>
> And that's why the documented recipe should probably recommend
> next(iter(s), default) instead. ?Especially because iter(s).next() is
> not even valid code in 3.0.

This seems reasonably legible to you? Strikes me as coding by
incantation. Also, while I've heard people say that the naive
approach is slower, I'm not getting that result. Here's my test:


>>> smrt = timeit.Timer("next(iter(s))", "s=set(range(100))")
>>> smrt.repeat(10)
[1.2845709323883057, 0.60247397422790527, 0.59621405601501465,
0.59133195877075195, 0.58387589454650879, 0.56839084625244141,
0.56839680671691895, 0.56877803802490234, 0.56905913352966309,
0.56846404075622559]

>>> naive = timeit.Timer("x=s.pop();s.add(x)", "s=set(range(100))")
>>> naive.repeat(10)
[0.93139314651489258, 0.53566789627075195, 0.53674602508544922,
0.53608798980712891, 0.53634309768676758, 0.53557991981506348,
0.53578495979309082, 0.53666114807128906, 0.53576493263244629,
0.53491711616516113]


Perhaps my test is flawed in some way?

Geremy Condra

From foom at fuhm.net  Fri Nov  6 00:17:19 2009
From: foom at fuhm.net (James Y Knight)
Date: Thu, 5 Nov 2009 18:17:19 -0500
Subject: [Python-Dev] Retrieve an arbitrary element from a
	setwithoutremoving it
In-Reply-To: <f3cc57c60911051504l48890d09q74d46580bf1c20ab@mail.gmail.com>
References: <7b0045ed0910301843y7059d316s8a851142b78b7aeb@mail.gmail.com>
	<200911040946.15805.steve@pearwood.info>
	<4AF0D0D7.3090403@canterbury.ac.nz>
	<200911042234.36796.steve@pearwood.info>
	<f3cc57c60911041332v479cc3b0t19d9601e52002128@mail.gmail.com>
	<99BB308202224D31AA5300223DB59912@RaymondLaptop1>
	<7b0045ed0911051243q18f8324ak9abaa98ac26948e9@mail.gmail.com>
	<d38f5330911051309l78166534ndf8320e09f9892bc@mail.gmail.com>
	<f3cc57c60911051504l48890d09q74d46580bf1c20ab@mail.gmail.com>
Message-ID: <9CAD2251-C51D-472B-98A4-104E1AFEEDE1@fuhm.net>

On Nov 5, 2009, at 6:04 PM, geremy condra wrote:
> Perhaps my test is flawed in some way?

Yes: you're testing the speed of something that makes absolutely no  
sense to do in a tight loop, so *who the heck cares how fast any way  
of doing it is*!

Is this thread over yet?

James

From guido at python.org  Fri Nov  6 00:18:12 2009
From: guido at python.org (Guido van Rossum)
Date: Thu, 5 Nov 2009 15:18:12 -0800
Subject: [Python-Dev] Refactoring installation schemes
In-Reply-To: <0d223447fa9aebaeb78f5be82eb3aaab@preisshare.net>
References: <94bdd2610910271455s7854e6a9u94e3aeeeacb69c02@mail.gmail.com> 
	<4AE824DA.4090701@voidspace.org.uk>
	<ED004FA5-9F77-4215-B06D-6CBECEF6D023@gmail.com> 
	<94bdd2610910281105q328bc3a5qaed020ef3afe28df@mail.gmail.com> 
	<94bdd2610911040429j50a6df87m51aac2464dd13bd7@mail.gmail.com> 
	<c77acedd9a48dd7036cca55fde42de0a@preisshare.net>
	<9cee7ab80911041440j43e51e39qc77a426547d02bcb@mail.gmail.com> 
	<39deb0d1eef08419dad65d7354a8ffb1@preisshare.net>
	<94bdd2610911050235l4e851612i46e0392d352eaa20@mail.gmail.com> 
	<0d223447fa9aebaeb78f5be82eb3aaab@preisshare.net>
Message-ID: <ca471dc20911051518i75b1a6b9t13f07c5747d65f4a@mail.gmail.com>

David, you have an attitude problem. Your contributions (the post
below is just an example) don't sound healthy to me -- you just
complain and whine and denigrate Tarek's work. In a previous post you
claimed to have had a particular idea first (it doesn't matter which
idea) and you managed to make it sound bad that your idea was
eventually accepted. This is not a productive attitude.

Surely the problem isn't writing 300 lines of code. The problem is
getting everyone to agree on which 300 lines of code should be
written. That is the problem at hand, and claiming that nothing
happened because no code was written and all that was agreed on
amounts to 300 lines of code is outright demeaning.

Stop it. You are wearing out your welcome.

--Guido

PS. Submitting a counter-PEP to the peps editors that hasn't been
discussed on the SIG list at all is also a bad move. You really need
to change the way you try to interact with the SIG.

On Thu, Nov 5, 2009 at 2:29 PM, David Lyon <david.lyon at preisshare.net> wrote:
> On Thu, 5 Nov 2009 11:35:41 +0100, Tarek Ziad? <ziade.tarek at gmail.com>
> wrote:
>> PEP 376 is working on a default, unified, *installation* format, that
>> tries to gather the good ideas of Pip, Setuptools etc.. and propose a
>> unified format for our site-packages. This new standard will come with
>> APIs in pkgutil to be able to query installed distribution etc. This
>> work is also linked to PEP 345 work where we are modifying the
>> Metadata, and to PEP 386 that proposes a standard version comparison
>> scheme.
>
> Perphaps..
>
> But if you put all these PEPs together, implementing all the new features
> can't come to more than 300 lines of code...
>
> Since we hardly got anywhere on them in 2009, it will be interesting to
> see how much of it gets done in 2010.
>
>> But there's no plan to include a new *distribution* format in Distutils.
>
> I wasn't suggesting that - at all.
>
> And saying that 'eggs' are a *new* python package format isn't really
> really helpful because to my understanding they've been around for
> some number of years.
>
> No, i won't raise why we have EGG_INFO directories and a whole lot
> of half working egg stuff in standard python... I'm just asking
> why it can't be more consistant? while we're on the refactoring
> topic.
>
> Be fair...
>
> I'm saying finish what is already there.. or take out the crap ..
>
> It isn't fair to suggest that I am somehow asking for some big
> change when I am simply pointing out all the junk that's in
> there that's already half built.
>
>> In any case those PEPs are not finished yet, so everyone can help at
>> distutils-SIG
>
> True - and False.
>
> But I've been on the list for some twelve months asking for work
> to help out with, and haven't been assigned a single task to do
> yet.
>
> Seriously, if you won't allocate work out to people then how can
> it get done?
>
> Whilst I personally think a lot of the stuff in those PEPs is not
> high on quality, why don't we just get them implemented anyway?
>
> I'm a fairly proficient develper, but I can't get assigned a single
> work item..
>
> And to me, it doesn't seem any harder than just selecting 'djlyon'
> on the python tracker for some work items...
>
> Surely those PEPs all amount to 300+ lines of code. With two people
> working on it, that's only surely 150+ lines of code each... That
> shouldn't be such a big challenge for 2010..
>
> David
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/guido%40python.org
>



-- 
--Guido van Rossum (python.org/~guido)

From jnoller at gmail.com  Fri Nov  6 00:21:42 2009
From: jnoller at gmail.com (Jesse Noller)
Date: Thu, 5 Nov 2009 18:21:42 -0500
Subject: [Python-Dev] PEP 3003 - Python Language Moratorium
In-Reply-To: <ea2499da0911051453n3569387aj581ad4acf5b799c8@mail.gmail.com>
References: <ca471dc20911030935l25251e0diec64427f5d7bcbf2@mail.gmail.com>
	<ca471dc20911051405o287e0f41m26d38860a200b47f@mail.gmail.com>
	<ea2499da0911051453n3569387aj581ad4acf5b799c8@mail.gmail.com>
Message-ID: <4222a8490911051521r1b9c8165id6e0f12d62da0613@mail.gmail.com>

On Thu, Nov 5, 2009 at 5:53 PM, Dirkjan Ochtman <dirkjan at ochtman.nl> wrote:
> On Thu, Nov 5, 2009 at 23:05, Guido van Rossum <guido at python.org> wrote:
>> I haven't seen substantial opposition against the PEP -- in fact I
>> can't recall any, and many people have explicitly posted in support of
>> it. So unless opposition suddenly appears in the next few days, I'll
>> move it to the Accepted state next Monday.
>
> Let me state first, I think the PEP is great, and I have no objection
> to its current form.
>
> I do have one qualm, where I wonder if the PEP shouldn't be a little
> stricter. As a gentoo developer and Mercurial maintainer, most of the
> pain in the recent migration towards 2.6 has not been in language
> changes, but in the standard library. Unfortunately, it's exempt from
> the moratorium in the PEP.
>
> Which makes me wonder, why are we not adding another moratorium, on
> deprecations in the standard library? In other words, let's not
> deprecate things like md5 or sha or the popen family of functions, but
> keep all of that as it is, for both 2.x and 3.x, so people can direct
> their energy towards other things (hopefully porting their 2.x
> codebase to 3.x).
>
> The standard library has already been through a lot of evolution in
> the 2.x to 3.x transition, so one might assume there's not a lot of
> stuff in the 3.x stdlib that would need deprecation in the short term.
> And for 2.x, well, I'd certainly hope we don't need to deprecate much
> more there before it finally gets EOL'ed, so it should be a relatively
> light maintenance load to bear.
>
> Is this just crazy talk?
>
> Cheers,
>
> Dirkjan
>

I'm against restricting deprecation warnings within the stdlib as part
of this. I actually want more things cleaned up and possibly
deprecated. That being said, a deprecation warning just means we will
remove it One Day - anything being deprecated will need a PEP and
follow the long path to actual removal.

So, -1 from me ;)

jesse

From brett at python.org  Fri Nov  6 00:24:11 2009
From: brett at python.org (Brett Cannon)
Date: Thu, 5 Nov 2009 15:24:11 -0800
Subject: [Python-Dev] PEP 3003 - Python Language Moratorium
In-Reply-To: <ea2499da0911051453n3569387aj581ad4acf5b799c8@mail.gmail.com>
References: <ca471dc20911030935l25251e0diec64427f5d7bcbf2@mail.gmail.com> 
	<ca471dc20911051405o287e0f41m26d38860a200b47f@mail.gmail.com> 
	<ea2499da0911051453n3569387aj581ad4acf5b799c8@mail.gmail.com>
Message-ID: <bbaeab100911051524p2eaf12feq1afd04ccdbdd2e58@mail.gmail.com>

On Thu, Nov 5, 2009 at 14:53, Dirkjan Ochtman <dirkjan at ochtman.nl> wrote:
> On Thu, Nov 5, 2009 at 23:05, Guido van Rossum <guido at python.org> wrote:
>> I haven't seen substantial opposition against the PEP -- in fact I
>> can't recall any, and many people have explicitly posted in support of
>> it. So unless opposition suddenly appears in the next few days, I'll
>> move it to the Accepted state next Monday.
>
> Let me state first, I think the PEP is great, and I have no objection
> to its current form.
>
> I do have one qualm, where I wonder if the PEP shouldn't be a little
> stricter. As a gentoo developer and Mercurial maintainer, most of the
> pain in the recent migration towards 2.6 has not been in language
> changes, but in the standard library. Unfortunately, it's exempt from
> the moratorium in the PEP.
>
> Which makes me wonder, why are we not adding another moratorium, on
> deprecations in the standard library? In other words, let's not
> deprecate things like md5 or sha or the popen family of functions, but
> keep all of that as it is, for both 2.x and 3.x, so people can direct
> their energy towards other things (hopefully porting their 2.x
> codebase to 3.x).
>
> The standard library has already been through a lot of evolution in
> the 2.x to 3.x transition, so one might assume there's not a lot of
> stuff in the 3.x stdlib that would need deprecation in the short term.
> And for 2.x, well, I'd certainly hope we don't need to deprecate much
> more there before it finally gets EOL'ed, so it should be a relatively
> light maintenance load to bear.
>
> Is this just crazy talk?

So you are basically asking for a moratorium on stdlib deprecations.
Considering how much was removed in Python 3 from 2.6 this is a really
minor worry. And the only deprecation that I can see potentially
coming down the pipeline is optparse for argparse and cProfile to help
finish PEP 3108.

And, as Jesse said in another reply, it would be nice to take this
time to reflect upon where we want the stdlib to go and to potentially
clear other things out. So I am -0 on this as leaving stuff in for an
extra release isn't going to kill us, but I would prefer to not put it
off.

-Brett

From guido at python.org  Fri Nov  6 00:26:16 2009
From: guido at python.org (Guido van Rossum)
Date: Thu, 5 Nov 2009 15:26:16 -0800
Subject: [Python-Dev] PEP 3003 - Python Language Moratorium
In-Reply-To: <4222a8490911051521r1b9c8165id6e0f12d62da0613@mail.gmail.com>
References: <ca471dc20911030935l25251e0diec64427f5d7bcbf2@mail.gmail.com> 
	<ca471dc20911051405o287e0f41m26d38860a200b47f@mail.gmail.com> 
	<ea2499da0911051453n3569387aj581ad4acf5b799c8@mail.gmail.com> 
	<4222a8490911051521r1b9c8165id6e0f12d62da0613@mail.gmail.com>
Message-ID: <ca471dc20911051526u7f3642ecg19e4eb73bcca1d3a@mail.gmail.com>

On Thu, Nov 5, 2009 at 3:21 PM, Jesse Noller <jnoller at gmail.com> wrote:
> On Thu, Nov 5, 2009 at 5:53 PM, Dirkjan Ochtman <dirkjan at ochtman.nl> wrote:
>> On Thu, Nov 5, 2009 at 23:05, Guido van Rossum <guido at python.org> wrote:
>>> I haven't seen substantial opposition against the PEP -- in fact I
>>> can't recall any, and many people have explicitly posted in support of
>>> it. So unless opposition suddenly appears in the next few days, I'll
>>> move it to the Accepted state next Monday.
>>
>> Let me state first, I think the PEP is great, and I have no objection
>> to its current form.
>>
>> I do have one qualm, where I wonder if the PEP shouldn't be a little
>> stricter. As a gentoo developer and Mercurial maintainer, most of the
>> pain in the recent migration towards 2.6 has not been in language
>> changes, but in the standard library. Unfortunately, it's exempt from
>> the moratorium in the PEP.
>>
>> Which makes me wonder, why are we not adding another moratorium, on
>> deprecations in the standard library? In other words, let's not
>> deprecate things like md5 or sha or the popen family of functions, but
>> keep all of that as it is, for both 2.x and 3.x, so people can direct
>> their energy towards other things (hopefully porting their 2.x
>> codebase to 3.x).
>>
>> The standard library has already been through a lot of evolution in
>> the 2.x to 3.x transition, so one might assume there's not a lot of
>> stuff in the 3.x stdlib that would need deprecation in the short term.
>> And for 2.x, well, I'd certainly hope we don't need to deprecate much
>> more there before it finally gets EOL'ed, so it should be a relatively
>> light maintenance load to bear.
>>
>> Is this just crazy talk?
>>
>> Cheers,
>>
>> Dirkjan
>>
>
> I'm against restricting deprecation warnings within the stdlib as part
> of this. I actually want more things cleaned up and possibly
> deprecated. That being said, a deprecation warning just means we will
> remove it One Day - anything being deprecated will need a PEP and
> follow the long path to actual removal.
>
> So, -1 from me ;)
>
> jesse

Actually, I think Dirkjan has a point. I'm not sure that we need
another moratorium (that's a rather dramatic kind of decision which
should be very rare indeed) but I do agree that deprecations are often
more of a pain than they're worth.

For example, take the deprecation of the md5 and sha modules in Python
2.6. They make it a bit of a pain to write code that *cleanly*
supports Python 2.4 (doesn't have hashlib) through 2.6 (warns when
importing md5 instead of hashlib). You can silence the warning, but
that is in itself not particularly clean, and users really hate having
the warnings.

I have come to the conclusion that there are better ways to
pre-announce that a module is going to disappear instead of
deprecation warnings.

-- 
--Guido van Rossum (python.org/~guido)

From jnoller at gmail.com  Fri Nov  6 00:34:37 2009
From: jnoller at gmail.com (Jesse Noller)
Date: Thu, 5 Nov 2009 18:34:37 -0500
Subject: [Python-Dev] PEP 3003 - Python Language Moratorium
In-Reply-To: <ca471dc20911051526u7f3642ecg19e4eb73bcca1d3a@mail.gmail.com>
References: <ca471dc20911030935l25251e0diec64427f5d7bcbf2@mail.gmail.com>
	<ca471dc20911051405o287e0f41m26d38860a200b47f@mail.gmail.com>
	<ea2499da0911051453n3569387aj581ad4acf5b799c8@mail.gmail.com>
	<4222a8490911051521r1b9c8165id6e0f12d62da0613@mail.gmail.com>
	<ca471dc20911051526u7f3642ecg19e4eb73bcca1d3a@mail.gmail.com>
Message-ID: <4222a8490911051534n504d2a02qd53177662e0c4c70@mail.gmail.com>

On Thu, Nov 5, 2009 at 6:26 PM, Guido van Rossum <guido at python.org> wrote:

>>
>> I'm against restricting deprecation warnings within the stdlib as part
>> of this. I actually want more things cleaned up and possibly
>> deprecated. That being said, a deprecation warning just means we will
>> remove it One Day - anything being deprecated will need a PEP and
>> follow the long path to actual removal.
>>
>> So, -1 from me ;)
>>
>> jesse
>
> Actually, I think Dirkjan has a point. I'm not sure that we need
> another moratorium (that's a rather dramatic kind of decision which
> should be very rare indeed) but I do agree that deprecations are often
> more of a pain than they're worth.
>
> For example, take the deprecation of the md5 and sha modules in Python
> 2.6. They make it a bit of a pain to write code that *cleanly*
> supports Python 2.4 (doesn't have hashlib) through 2.6 (warns when
> importing md5 instead of hashlib). You can silence the warning, but
> that is in itself not particularly clean, and users really hate having
> the warnings.
>
> I have come to the conclusion that there are better ways to
> pre-announce that a module is going to disappear instead of
> deprecation warnings.
>

I'm interested in hearing how to handle "pending removals" other than
deprecation warnings - I guess I'm against the idea that we shouldn't
remove/plan to remove things from the stdlib and signal those
intentions to users during the moratorium.

The mechanics of that can be something other than deprecation
warnings, we can add a line to the current moratorium that puts the
nix on deprecation warnings (because yes, Dirkjan is right - they can
be a pain) but we should outline the alternative process.

jesse

From brett at python.org  Fri Nov  6 00:35:28 2009
From: brett at python.org (Brett Cannon)
Date: Thu, 5 Nov 2009 15:35:28 -0800
Subject: [Python-Dev] PEP 3003 - Python Language Moratorium
In-Reply-To: <ca471dc20911051526u7f3642ecg19e4eb73bcca1d3a@mail.gmail.com>
References: <ca471dc20911030935l25251e0diec64427f5d7bcbf2@mail.gmail.com> 
	<ca471dc20911051405o287e0f41m26d38860a200b47f@mail.gmail.com> 
	<ea2499da0911051453n3569387aj581ad4acf5b799c8@mail.gmail.com> 
	<4222a8490911051521r1b9c8165id6e0f12d62da0613@mail.gmail.com> 
	<ca471dc20911051526u7f3642ecg19e4eb73bcca1d3a@mail.gmail.com>
Message-ID: <bbaeab100911051535g68cefa18kdadea33f78de9495@mail.gmail.com>

On Thu, Nov 5, 2009 at 15:26, Guido van Rossum <guido at python.org> wrote:
> On Thu, Nov 5, 2009 at 3:21 PM, Jesse Noller <jnoller at gmail.com> wrote:
>> On Thu, Nov 5, 2009 at 5:53 PM, Dirkjan Ochtman <dirkjan at ochtman.nl> wrote:
>>> On Thu, Nov 5, 2009 at 23:05, Guido van Rossum <guido at python.org> wrote:
>>>> I haven't seen substantial opposition against the PEP -- in fact I
>>>> can't recall any, and many people have explicitly posted in support of
>>>> it. So unless opposition suddenly appears in the next few days, I'll
>>>> move it to the Accepted state next Monday.
>>>
>>> Let me state first, I think the PEP is great, and I have no objection
>>> to its current form.
>>>
>>> I do have one qualm, where I wonder if the PEP shouldn't be a little
>>> stricter. As a gentoo developer and Mercurial maintainer, most of the
>>> pain in the recent migration towards 2.6 has not been in language
>>> changes, but in the standard library. Unfortunately, it's exempt from
>>> the moratorium in the PEP.
>>>
>>> Which makes me wonder, why are we not adding another moratorium, on
>>> deprecations in the standard library? In other words, let's not
>>> deprecate things like md5 or sha or the popen family of functions, but
>>> keep all of that as it is, for both 2.x and 3.x, so people can direct
>>> their energy towards other things (hopefully porting their 2.x
>>> codebase to 3.x).
>>>
>>> The standard library has already been through a lot of evolution in
>>> the 2.x to 3.x transition, so one might assume there's not a lot of
>>> stuff in the 3.x stdlib that would need deprecation in the short term.
>>> And for 2.x, well, I'd certainly hope we don't need to deprecate much
>>> more there before it finally gets EOL'ed, so it should be a relatively
>>> light maintenance load to bear.
>>>
>>> Is this just crazy talk?
>>>
>>> Cheers,
>>>
>>> Dirkjan
>>>
>>
>> I'm against restricting deprecation warnings within the stdlib as part
>> of this. I actually want more things cleaned up and possibly
>> deprecated. That being said, a deprecation warning just means we will
>> remove it One Day - anything being deprecated will need a PEP and
>> follow the long path to actual removal.
>>
>> So, -1 from me ;)
>>
>> jesse
>
> Actually, I think Dirkjan has a point. I'm not sure that we need
> another moratorium (that's a rather dramatic kind of decision which
> should be very rare indeed) but I do agree that deprecations are often
> more of a pain than they're worth.
>
> For example, take the deprecation of the md5 and sha modules in Python
> 2.6. They make it a bit of a pain to write code that *cleanly*
> supports Python 2.4 (doesn't have hashlib) through 2.6 (warns when
> importing md5 instead of hashlib). You can silence the warning, but
> that is in itself not particularly clean, and users really hate having
> the warnings.
>
> I have come to the conclusion that there are better ways to
> pre-announce that a module is going to disappear instead of
> deprecation warnings.
>

What exactly are those better ways? Document as deprecated only?

-Brett

From ubershmekel at gmail.com  Fri Nov  6 00:52:54 2009
From: ubershmekel at gmail.com (Yuvgoog Greenle)
Date: Fri, 6 Nov 2009 01:52:54 +0200
Subject: [Python-Dev] Retrieve an arbitrary element from a
	setwithoutremoving it
In-Reply-To: <9CAD2251-C51D-472B-98A4-104E1AFEEDE1@fuhm.net>
References: <7b0045ed0910301843y7059d316s8a851142b78b7aeb@mail.gmail.com>
	<200911040946.15805.steve@pearwood.info>
	<4AF0D0D7.3090403@canterbury.ac.nz>
	<200911042234.36796.steve@pearwood.info>
	<f3cc57c60911041332v479cc3b0t19d9601e52002128@mail.gmail.com>
	<99BB308202224D31AA5300223DB59912@RaymondLaptop1>
	<7b0045ed0911051243q18f8324ak9abaa98ac26948e9@mail.gmail.com>
	<d38f5330911051309l78166534ndf8320e09f9892bc@mail.gmail.com>
	<f3cc57c60911051504l48890d09q74d46580bf1c20ab@mail.gmail.com>
	<9CAD2251-C51D-472B-98A4-104E1AFEEDE1@fuhm.net>
Message-ID: <9d153b7c0911051552v7cfabf61ifede37a72ec239d8@mail.gmail.com>

On Fri, Nov 6, 2009 at 1:17 AM, James Y Knight <foom at fuhm.net> wrote:

>
> Is this thread over yet?
>
>
Sorry, I just had to point out that pop/add has a side effect that would be
apparent on a set that multiple threads access - it loses an item and then
gets it back. Sounds like a sleeper race condition that's going to be rare
but extremely hard to find if it does occur. Crooked as a gil.

--yuv
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20091106/11ac81fd/attachment.htm>

From bobbyrward at gmail.com  Fri Nov  6 00:55:34 2009
From: bobbyrward at gmail.com (Bobby R. Ward)
Date: Thu, 5 Nov 2009 17:55:34 -0600
Subject: [Python-Dev] PEP 3003 - Python Language Moratorium
In-Reply-To: <bbaeab100911051535g68cefa18kdadea33f78de9495@mail.gmail.com>
References: <ca471dc20911030935l25251e0diec64427f5d7bcbf2@mail.gmail.com> 
	<ca471dc20911051405o287e0f41m26d38860a200b47f@mail.gmail.com> 
	<ea2499da0911051453n3569387aj581ad4acf5b799c8@mail.gmail.com> 
	<4222a8490911051521r1b9c8165id6e0f12d62da0613@mail.gmail.com> 
	<ca471dc20911051526u7f3642ecg19e4eb73bcca1d3a@mail.gmail.com> 
	<bbaeab100911051535g68cefa18kdadea33f78de9495@mail.gmail.com>
Message-ID: <48c184130911051555r9a5b78cs3a13cb1345d3cbf7@mail.gmail.com>

> What exactly are those better ways? Document as deprecated only?
>
> -Brett

A switch to ENABLE those warnings?


Lord knows I'm sick of filtering them out of logs.

A switch to enable deprecation warnings  would give developers a
chance to see them when migrating to a new version of python.  And it
would prevent users from having to deal with them.



-- 
Bobby R. Ward
------------------------------
bobbyrward at gmail.com
http://github.com/bobbyrward
http://launchpad.net/~bobbyrward

"While many languages can be used to encrypt data, PERL has something
built-in that gives you encryption. Perl calls it `syntax`."
-- http://uncyclopedia.wikia.com/wiki/Perl

From david.lyon at preisshare.net  Fri Nov  6 00:55:01 2009
From: david.lyon at preisshare.net (David Lyon)
Date: Thu, 05 Nov 2009 18:55:01 -0500
Subject: [Python-Dev] Refactoring installation schemes
In-Reply-To: <ca471dc20911051518i75b1a6b9t13f07c5747d65f4a@mail.gmail.com>
References: <94bdd2610910271455s7854e6a9u94e3aeeeacb69c02@mail.gmail.com>
	<4AE824DA.4090701@voidspace.org.uk>
	<ED004FA5-9F77-4215-B06D-6CBECEF6D023@gmail.com>
	<94bdd2610910281105q328bc3a5qaed020ef3afe28df@mail.gmail.com>
	<94bdd2610911040429j50a6df87m51aac2464dd13bd7@mail.gmail.com>
	<c77acedd9a48dd7036cca55fde42de0a@preisshare.net>
	<9cee7ab80911041440j43e51e39qc77a426547d02bcb@mail.gmail.com>
	<39deb0d1eef08419dad65d7354a8ffb1@preisshare.net>
	<94bdd2610911050235l4e851612i46e0392d352eaa20@mail.gmail.com>
	<0d223447fa9aebaeb78f5be82eb3aaab@preisshare.net>
	<ca471dc20911051518i75b1a6b9t13f07c5747d65f4a@mail.gmail.com>
Message-ID: <6d81ac30c331f8b9ca71a75d1049ffd4@preisshare.net>


Tarek, Guido,

Forgive my grumpy tone..

Looking positive, if we now have a rough consensus that the PEPS might
represent some 300+ lines of code... then good - lets get started, that's
all that I meant. I'm glad above all, that you noticed that part foremost.

If it's a simple case that alternative's to the solutions cannot
be considered - because I am grumpy - whatever, then that's something 
I can live with. I didn't know that proposing a different solution 
wasn't a good idea.

> PS. Submitting a counter-PEP to the peps editors that hasn't been
> discussed on the SIG list at all is also a bad move. You really need
> to change the way you try to interact with the SIG.

Here are the references to the discussions from distutils-sig. In these 
references you'll see that my interaction with the list was not grumpy 
but was entirely positive. I feel that I got a fair hearing on distutils
list and I had a number of supporters to my proposal.

The proposal is in the final stages of being wrapped up.

Here are the discussion References that have occurred over the year:

http://mail.python.org/pipermail/distutils-sig/2009-November/thread.html

November 2009 Archives by thread

    * Messages sorted by: [ subject ] [ author ] [ date ]
    * More info on this list...

Starting: Sun Nov 1 09:31:46 CET 2009
Ending: Fri Nov 6 00:09:22 CET 2009
Messages: 65

    * [Distutils] Alternate static metadata PEP submission...   Georg
Brandl
          o [Distutils] Alternate static metadata PEP submission...   David
Lyon
          o [Distutils] Alternate static metadata PEP submission...  
Floris Bruynooghe
                + [Distutils] Alternate static metadata PEP submission...  
Georg Brandl
                + [Distutils] Alternate static metadata PEP submission...  
David Lyon
                      # [Distutils] Alternate static metadata PEP
submission...   Georg Brandl
                      # [Distutils] Alternate static metadata PEP
submission...   David Lyon 


http://mail.python.org/pipermail/distutils-sig/2009-October/thread.html

# [Distutils] Alternate static metadata PEP submission...   David Lyon

    * [Distutils] Alternate static metadata PEP submission...   Tarek
Ziad?
          o [Distutils] Alternate static metadata PEP submission...   David
Lyon
          o [Distutils] Alternate static metadata PEP submission...   Tarek
Ziad?
          o [Distutils] Alternate static metadata PEP submission...   David
Lyon
          o [Distutils] Alternate static metadata PEP submission...   Eric
Smith
          o [Distutils] Alternate static metadata PEP submission...   Tarek
Ziad?
          o [Distutils] Alternate static metadata PEP submission...   David
Lyon 
    * [Distutils] Alternate static metadata PEP submission...   David
Cournapeau
          o [Distutils] Alternate static metadata PEP submission...   David
Lyon
          o [Distutils] Alternate static metadata PEP submission...   David
Cournapeau 
    * [Distutils] Alternate static metadata PEP submission...   Floris
Bruynooghe
          o [Distutils] Alternate static metadata PEP submission...   Tarek
Ziad?
          o [Distutils] Alternate static metadata PEP submission...   David
Lyon
          o [Distutils] Alternate static metadata PEP submission...   Tarek
Ziad?
          o [Distutils] Alternate static metadata PEP submission...   David
Lyon
          o [Distutils] Alternate static metadata PEP submission...  
Floris Bruynooghe
          o [Distutils] Alternate static metadata PEP submission...   David
Lyon
          o [Distutils] Alternate static metadata PEP submission...   Tarek
Ziad?
          o [Distutils] Alternate static metadata PEP submission...   David
Lyon
          o [Distutils] Alternate static metadata PEP submission...   Ian
Bicking
          o [Distutils] Alternate static metadata PEP submission...   David
Cournapeau
          o [Distutils] Alternate static metadata PEP submission...   David
Lyon
          o [Distutils] Alternate static metadata PEP submission...   David
Cournapeau
          o [Distutils] Alternate static metadata PEP submission...   David
Lyon
          o [Distutils] Alternate static metadata PEP submission...   Fred
Drake
          o [Distutils] Alternate static metadata PEP submission...   Kevin
Teague
          o [Distutils] Alternate static metadata PEP submission...   David
Lyon
          o [Distutils] Alternate static metadata PEP submission...   David
Cournapeau
          o [Distutils] Alternate static metadata PEP submission...   David
Lyon
          o [Distutils] Alternate static metadata PEP submission...   David
Cournapeau
          o [Distutils] Alternate static metadata PEP submission...   Fred
Drake
          o [Distutils] Alternate static metadata PEP submission...   David
Cournapeau
          o [Distutils] Alternate static metadata PEP submission...   David
Lyon
          o [Distutils] Alternate static metadata PEP submission...   David
Cournapeau
          o [Distutils] Alternate static metadata PEP submission...   David
Lyon
          o [Distutils] Alternate static metadata PEP submission...   David
Cournapeau
          o [Distutils] Alternate static metadata PEP submission...   David
Lyon
          o [Distutils] Alternate static metadata PEP submission...   David
Cournapeau
          o [Distutils] Alternate static metadata PEP submission...   David
Lyon
          o [Distutils] Alternate static metadata PEP submission...   David
Cournapeau
          o [Distutils] Alternate static metadata PEP submission...   David
Lyon
          o [Distutils] Alternate static metadata PEP submission...   Chris
Withers
          o [Distutils] Alternate static metadata PEP submission...   David
Lyon
          o [Distutils] Alternate static metadata PEP submission...   Chris
Withers
          o [Distutils] Alternate static metadata PEP submission...   David
Lyon
          o [Distutils] Alternate static metadata PEP submission...   Chris
Withers
          o [Distutils] Alternate static metadata PEP submission...   David
Lyon
          o [Distutils] Alternate static metadata PEP submission...   Chris
Withers
          o [Distutils] Alternate static metadata PEP submission...   Piotr
Ozarowski
          o [Distutils] Alternate static metadata PEP submission...   David
Lyon
          o [Distutils] Alternate static metadata PEP submission...   Piotr
Ozarowski
          o [Distutils] Alternate static metadata PEP submission...   David
Lyon
          o [Distutils] Alternate static metadata PEP submission...   David
Lyon
          o [Distutils] Alternate static metadata PEP submission...   Chris
Withers
          o [Distutils] Alternate static metadata PEP submission...   David
Lyon
          o [Distutils] Alternate static metadata PEP submission...   Chris
Withers
          o [Distutils] Alternate static metadata PEP submission...   David
Cournapeau
          o [Distutils] Alternate static metadata PEP submission...   Chris
Withers
          o [Distutils] Alternate static metadata PEP submission...   David
Lyon
          o [Distutils] Alternate static metadata PEP submission...   David
Cournapeau
          o [Distutils] Alternate static metadata PEP submission...   David
Lyon
          o [Distutils] Alternate static metadata PEP submission...   Fred
Drake
          o [Distutils] Alternate static metadata PEP submission...   Chris
Withers
          o [Distutils] Alternate static metadata PEP submission...   Fred
Drake
          o [Distutils] Alternate static metadata PEP submission...   Chris
Withers
          o [Distutils] Alternate static metadata PEP submission...   Eric
Smith
          o [Distutils] Alternate static metadata PEP submission...   Fred
Drake
          o [Distutils] Alternate static metadata PEP submission...   David
Cournapeau
          o [Distutils] Alternate static metadata PEP submission...   David
Lyon
          o [Distutils] Alternate static metadata PEP submission...   David
Cournapeau
          o [Distutils] Alternate static metadata PEP submission...   David
Lyon 

http://mail.python.org/pipermail/distutils-sig/2009-August/thread.html
[Distutils] Static metadata using setup.cfg   Tarek Ziad?

    * [Distutils] Static metadata using setup.cfg   Hanno Schlichting
    * [Distutils] Static metadata using setup.cfg   P.J. Eby
          o [Distutils] Static metadata using setup.cfg   Tarek Ziad?
                + [Distutils] Static metadata using setup.cfg   Tarek
Ziad?
                + [Distutils] Static metadata using setup.cfg   Tarek
Ziad? 
    * [Distutils] Static metadata using setup.cfg   David Lyon
          o [Distutils] Static metadata using setup.cfg   Jeff Rush
                + [Distutils] Static metadata using setup.cfg   Ben Finney
                + [Distutils] Static metadata using setup.cfg   Tarek
Ziad?
                + [Distutils] Static metadata using setup.cfg   David Lyon
                + [Distutils] Static metadata using setup.cfg   Eric Smith
                + [Distutils] Static metadata using setup.cfg   Tarek
Ziad?
                + [Distutils] Static metadata using setup.cfg   Tarek
Ziad?
                + [Distutils] Static metadata using setup.cfg   Chris
Withers
                + [Distutils] Static metadata using setup.cfg   Tarek
Ziad?
                + [Distutils] Static metadata using setup.cfg   Chris
Withers 
    * [Distutils] Static metadata using setup.cfg   David Lyon
          o [Distutils] Static metadata using setup.cfg   Sridhar
Ratnakumar
                + [Distutils] Static metadata using setup.cfg   David Lyon 
          o [Distutils] Static metadata using setup.cfg   Alex Gr?nholm
                + [Distutils] Static metadata using setup.cfg   David Lyon
                + [Distutils] Static metadata using setup.cfg   Floris
Bruynooghe
                + [Distutils] Static metadata using setup.cfg   Ronald
Oussoren
                + [Distutils] Static metadata using setup.cfg   P.J. Eby
                + [Distutils] Static metadata using setup.cfg   Chris
Withers 
          o [Distutils] Static metadata using setup.cfg   Chris Withers
                + [Distutils] Static metadata using setup.cfg   Tarek
Ziad?
                + [Distutils] Static metadata using setup.cfg   Chris
Withers
                + [Distutils] Static metadata using setup.cfg   Tarek
Ziad?
                + [Distutils] Static metadata using setup.cfg   David Lyon
                + [Distutils] Static metadata using setup.cfg   Tarek
Ziad? 

http://mail.python.org/pipermail/distutils-sig/2009-July/thread.html

# [Distutils] "setup.py needs to go away" (was [PEP 376] - Open questions
on python-dev)   David Lyon
# [Distutils] "setup.py needs to go away" (was [PEP 376] - Open questions
on python-dev)   P.J. Eby

    * [Distutils] "setup.py needs to go away" (was [PEP 376] - Open
questions on python-dev)   Eric Smith
    * [Distutils] "setup.py needs to go away" (was [PEP 376] - Open
questions on python-dev)   David Cournapeau
          o [Distutils] "setup.py needs to go away" (was [PEP 376] - Open
questions on python-dev)   Jim Fulton
          o [Distutils] "setup.py needs to go away" (was [PEP 376] - Open
questions on python-dev)   David Cournapeau
          o [Distutils] "setup.py needs to go away" (was [PEP 376] - Open
questions on python-dev)   Jim Fulton
          o [Distutils] "setup.py needs to go away" (was [PEP 376] - Open
questions on python-dev)   David Cournapeau
          o [Distutils] "setup.py needs to go away" (was [PEP 376] - Open
questions on python-dev)   Jim Fulton
          o [Distutils] "setup.py needs to go away" (was [PEP 376] - Open
questions on python-dev)   Tres Seaver
          o [Distutils] "setup.py needs to go away" (was [PEP 376] - Open
questions on python-dev)   David Cournapeau
          o [Distutils] Questions about virtual-python on ubuntu   David
Lyon
          o [Distutils] "setup.py needs to go away" (was [PEP 376] - Open
questions on python-dev)   Paul Moore
          o [Distutils] "setup.py needs to go away" (was [PEP 376] - Open
questions on python-dev)   Jim Fulton
          o [Distutils] "setup.py needs to go away" (was [PEP 376] - Open
questions on python-dev)   David Lyon
          o [Distutils] "setup.py needs to go away" (was [PEP 376] - Open
questions on python-dev)   Jim Fulton
          o [Distutils] "setup.py needs to go away" (was [PEP 376] - Open
questions on python-dev)   David Lyon
          o [Distutils] "setup.py needs to go away" (was [PEP 376] - Open
questions on python-dev)   Jim Fulton
          o [Distutils] "setup.py needs to go away" (was [PEP 376] - Open
questions on python-dev)   David Lyon 

# [Distutils] Removing the hack from setup.py - Setup.py as a Class, not a
Procedure   David Lyon
# [Distutils] Removing the hack from setup.py - Setup.py as a Class, not a
Procedure   Jeff Rush
# [Distutils] Removing the hack from setup.py - Setup.py as a Class, not a
Procedure   David Lyon
# [Distutils] Removing the hack from setup.py - Setup.py as a Class, not a
Procedure   P.J. Eby
# [Distutils] Removing the hack from setup.py - Setup.py as a Class, not a
Procedure   David Lyon
# [Distutils] Removing the hack from setup.py - Setup.py as a Class, not a
Procedure   Jean-Paul Calderone
# [Distutils] Setup.py as a Class, not a Procedure   David Lyon
# [Distutils] Setup.py as a Class, not a Procedure   Jean-Paul Calderone
# [Distutils] Setup.py as a Class, not a Procedure   David Lyon
[Distutils] introspecting package data   Chris Withers

http://mail.python.org/pipermail/distutils-sig/2009-June/thread.html

    * [Distutils] introspecting package data   Tarek Ziad?
          o [Distutils] introspecting package data   Chris Withers
                + [Distutils] introspecting package data   Jim Fulton
                + [Distutils] introspecting package data   Chris Withers
                + [Distutils] introspecting package data   Tarek Ziad?
                + [Distutils] introspecting package data   David Lyon 


On Thu, 5 Nov 2009 15:18:12 -0800, Guido van Rossum <guido at python.org>
wrote:
> David, you have an attitude problem. Your contributions (the post
> below is just an example) don't sound healthy to me -- you just
> complain and whine and denigrate Tarek's work. In a previous post you
> claimed to have had a particular idea first (it doesn't matter which
> idea) and you managed to make it sound bad that your idea was
> eventually accepted. This is not a productive attitude.
> 
> Surely the problem isn't writing 300 lines of code. The problem is
> getting everyone to agree on which 300 lines of code should be
> written. That is the problem at hand, and claiming that nothing
> happened because no code was written and all that was agreed on
> amounts to 300 lines of code is outright demeaning.
> 
> Stop it. You are wearing out your welcome.
> 
> --Guido
> 
> PS. Submitting a counter-PEP to the peps editors that hasn't been
> discussed on the SIG list at all is also a bad move. You really need
> to change the way you try to interact with the SIG.
> 
> On Thu, Nov 5, 2009 at 2:29 PM, David Lyon <david.lyon at preisshare.net>
> wrote:
>> On Thu, 5 Nov 2009 11:35:41 +0100, Tarek Ziad? <ziade.tarek at gmail.com>
>> wrote:
>>> PEP 376 is working on a default, unified, *installation* format, that
>>> tries to gather the good ideas of Pip, Setuptools etc.. and propose a
>>> unified format for our site-packages. This new standard will come with
>>> APIs in pkgutil to be able to query installed distribution etc. This
>>> work is also linked to PEP 345 work where we are modifying the
>>> Metadata, and to PEP 386 that proposes a standard version comparison
>>> scheme.
>>
>> Perphaps..
>>
>> But if you put all these PEPs together, implementing all the new
features
>> can't come to more than 300 lines of code...
>>
>> Since we hardly got anywhere on them in 2009, it will be interesting to
>> see how much of it gets done in 2010.
>>
>>> But there's no plan to include a new *distribution* format in
Distutils.
>>
>> I wasn't suggesting that - at all.
>>
>> And saying that 'eggs' are a *new* python package format isn't really
>> really helpful because to my understanding they've been around for
>> some number of years.
>>
>> No, i won't raise why we have EGG_INFO directories and a whole lot
>> of half working egg stuff in standard python... I'm just asking
>> why it can't be more consistant? while we're on the refactoring
>> topic.
>>
>> Be fair...
>>
>> I'm saying finish what is already there.. or take out the crap ..
>>
>> It isn't fair to suggest that I am somehow asking for some big
>> change when I am simply pointing out all the junk that's in
>> there that's already half built.
>>
>>> In any case those PEPs are not finished yet, so everyone can help at
>>> distutils-SIG
>>
>> True - and False.
>>
>> But I've been on the list for some twelve months asking for work
>> to help out with, and haven't been assigned a single task to do
>> yet.
>>
>> Seriously, if you won't allocate work out to people then how can
>> it get done?
>>
>> Whilst I personally think a lot of the stuff in those PEPs is not
>> high on quality, why don't we just get them implemented anyway?
>>
>> I'm a fairly proficient develper, but I can't get assigned a single
>> work item..
>>
>> And to me, it doesn't seem any harder than just selecting 'djlyon'
>> on the python tracker for some work items...
>>
>> Surely those PEPs all amount to 300+ lines of code. With two people
>> working on it, that's only surely 150+ lines of code each... That
>> shouldn't be such a big challenge for 2010..
>>
>> David
>>
>> _______________________________________________
>> Python-Dev mailing list
>> Python-Dev at python.org
>> http://mail.python.org/mailman/listinfo/python-dev
>> Unsubscribe:
>> http://mail.python.org/mailman/options/python-dev/guido%40python.org
>>

From ubershmekel at gmail.com  Fri Nov  6 01:13:29 2009
From: ubershmekel at gmail.com (Yuvgoog Greenle)
Date: Fri, 6 Nov 2009 02:13:29 +0200
Subject: [Python-Dev] PEP 3003 - Python Language Moratorium
In-Reply-To: <48c184130911051555r9a5b78cs3a13cb1345d3cbf7@mail.gmail.com>
References: <ca471dc20911030935l25251e0diec64427f5d7bcbf2@mail.gmail.com>
	<ca471dc20911051405o287e0f41m26d38860a200b47f@mail.gmail.com>
	<ea2499da0911051453n3569387aj581ad4acf5b799c8@mail.gmail.com>
	<4222a8490911051521r1b9c8165id6e0f12d62da0613@mail.gmail.com>
	<ca471dc20911051526u7f3642ecg19e4eb73bcca1d3a@mail.gmail.com>
	<bbaeab100911051535g68cefa18kdadea33f78de9495@mail.gmail.com>
	<48c184130911051555r9a5b78cs3a13cb1345d3cbf7@mail.gmail.com>
Message-ID: <9d153b7c0911051613o62be5dc9l52102f7805cd4a0e@mail.gmail.com>

On Fri, Nov 6, 2009 at 1:55 AM, Bobby R. Ward <bobbyrward at gmail.com> wrote:

>
> A switch to ENABLE those warnings?
>
>
I think a few people I know would still be raising strings like exceptions
if not for the deprecation warnings. Most people won't turn on the switch
with the extra warnings.

--yuv
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20091106/8068b515/attachment.htm>

From exarkun at twistedmatrix.com  Fri Nov  6 01:14:30 2009
From: exarkun at twistedmatrix.com (exarkun at twistedmatrix.com)
Date: Fri, 06 Nov 2009 00:14:30 -0000
Subject: [Python-Dev] PEP 3003 - Python Language Moratorium
In-Reply-To: <48c184130911051555r9a5b78cs3a13cb1345d3cbf7@mail.gmail.com>
References: <ca471dc20911030935l25251e0diec64427f5d7bcbf2@mail.gmail.com>
	<ca471dc20911051405o287e0f41m26d38860a200b47f@mail.gmail.com>
	<ea2499da0911051453n3569387aj581ad4acf5b799c8@mail.gmail.com>
	<4222a8490911051521r1b9c8165id6e0f12d62da0613@mail.gmail.com>
	<ca471dc20911051526u7f3642ecg19e4eb73bcca1d3a@mail.gmail.com>
	<bbaeab100911051535g68cefa18kdadea33f78de9495@mail.gmail.com>
	<48c184130911051555r9a5b78cs3a13cb1345d3cbf7@mail.gmail.com>
Message-ID: <20091106001430.3229.796306500.divmod.xquotient.180@localhost.localdomain>

On 5 Nov, 11:55 pm, bobbyrward at gmail.com wrote:
>>What exactly are those better ways? Document as deprecated only?
>>
>>-Brett
>
>A switch to ENABLE those warnings?
>
>
>Lord knows I'm sick of filtering them out of logs.
>
>A switch to enable deprecation warnings  would give developers a
>chance to see them when migrating to a new version of python.  And it
>would prevent users from having to deal with them.

PendingDeprecationWarning is silent by default and requires a switch to 
be enabled.

Jean-Paul

From debatem1 at gmail.com  Fri Nov  6 01:30:05 2009
From: debatem1 at gmail.com (geremy condra)
Date: Thu, 5 Nov 2009 19:30:05 -0500
Subject: [Python-Dev] Retrieve an arbitrary element from a
	setwithoutremoving it
In-Reply-To: <9CAD2251-C51D-472B-98A4-104E1AFEEDE1@fuhm.net>
References: <7b0045ed0910301843y7059d316s8a851142b78b7aeb@mail.gmail.com>
	<200911040946.15805.steve@pearwood.info>
	<4AF0D0D7.3090403@canterbury.ac.nz>
	<200911042234.36796.steve@pearwood.info>
	<f3cc57c60911041332v479cc3b0t19d9601e52002128@mail.gmail.com>
	<99BB308202224D31AA5300223DB59912@RaymondLaptop1>
	<7b0045ed0911051243q18f8324ak9abaa98ac26948e9@mail.gmail.com>
	<d38f5330911051309l78166534ndf8320e09f9892bc@mail.gmail.com>
	<f3cc57c60911051504l48890d09q74d46580bf1c20ab@mail.gmail.com>
	<9CAD2251-C51D-472B-98A4-104E1AFEEDE1@fuhm.net>
Message-ID: <f3cc57c60911051630o51f6cf32m4e60a69ceb0ca922@mail.gmail.com>

On Thu, Nov 5, 2009 at 6:17 PM, James Y Knight <foom at fuhm.net> wrote:
> On Nov 5, 2009, at 6:04 PM, geremy condra wrote:
>>
>> Perhaps my test is flawed in some way?
>
> Yes: you're testing the speed of something that makes absolutely no sense to
> do in a tight loop, so *who the heck cares how fast any way of doing it is*!
>
> Is this thread over yet?
>
> James

I'm testing the speed because the claim was made that the pop/add
approach was inefficient. Here's the full quote:

>    The obvious way, for newcomers, of achieving the effect is:
>
>  x = s.pop()
>  s.add(x)
>
> ... and that's simply horrible in terms of efficiency.  So the
> "obvious" way of doing it in Python is wrong(TM), and the "correct"
> way of doing it is obscure and raises misleading exceptions.

Since I use this in a graph theory library that I am currently trying
to scale to handle 300 million node simulations, and this is used in
several relevant algorithms, I was concerned.

Geremy Condra

From python at mrabarnett.plus.com  Fri Nov  6 01:33:54 2009
From: python at mrabarnett.plus.com (MRAB)
Date: Fri, 06 Nov 2009 00:33:54 +0000
Subject: [Python-Dev] PEP 3003 - Python Language Moratorium
In-Reply-To: <ca471dc20911051405o287e0f41m26d38860a200b47f@mail.gmail.com>
References: <ca471dc20911030935l25251e0diec64427f5d7bcbf2@mail.gmail.com>
	<ca471dc20911051405o287e0f41m26d38860a200b47f@mail.gmail.com>
Message-ID: <4AF36EF2.3050706@mrabarnett.plus.com>

Guido van Rossum wrote:
> On Tue, Nov 3, 2009 at 9:35 AM, Guido van Rossum <guido at python.org> wrote:
>> I've checked draft (!) PEP 3003, "Python Language Moratorium", into
>> SVN. As authors I've listed Jesse, Brett and myself.
> 
> I haven't seen substantial opposition against the PEP -- in fact I
> can't recall any, and many people have explicitly posted in support of
> it. So unless opposition suddenly appears in the next few days, I'll
> move it to the Accepted state next Monday.
> 
My view is that instead of there being new language features appearing
in the next few years, what we have in Python 3 _are_ the new language
features. We just need time to digest them before looking for more. :-)

+1

From ziade.tarek at gmail.com  Fri Nov  6 02:09:58 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Fri, 6 Nov 2009 02:09:58 +0100
Subject: [Python-Dev] Refactoring installation schemes
In-Reply-To: <6d81ac30c331f8b9ca71a75d1049ffd4@preisshare.net>
References: <94bdd2610910271455s7854e6a9u94e3aeeeacb69c02@mail.gmail.com>
	<94bdd2610910281105q328bc3a5qaed020ef3afe28df@mail.gmail.com>
	<94bdd2610911040429j50a6df87m51aac2464dd13bd7@mail.gmail.com>
	<c77acedd9a48dd7036cca55fde42de0a@preisshare.net>
	<9cee7ab80911041440j43e51e39qc77a426547d02bcb@mail.gmail.com>
	<39deb0d1eef08419dad65d7354a8ffb1@preisshare.net>
	<94bdd2610911050235l4e851612i46e0392d352eaa20@mail.gmail.com>
	<0d223447fa9aebaeb78f5be82eb3aaab@preisshare.net>
	<ca471dc20911051518i75b1a6b9t13f07c5747d65f4a@mail.gmail.com>
	<6d81ac30c331f8b9ca71a75d1049ffd4@preisshare.net>
Message-ID: <94bdd2610911051709y6050bca1w8e10c44d3598d145@mail.gmail.com>

2009/11/6 David Lyon <david.lyon at preisshare.net>:
[..]
> Here are the references to the discussions from distutils-sig. In these
> references you'll see that my interaction with the list was not grumpy
> but was entirely positive. I feel that I got a fair hearing on distutils
> list and I had a number of supporters to my proposal.
>
> The proposal is in the final stages of being wrapped up.

There are three different topics :

1. a standard for the installation format: PEP 376
2. changes in the metadata format : PEP 345 (-- work in progress here,
some parts are still missing)
3. ways to built the metadata that are shipped into a distribution
statically, using Distutils : PEP 390 was started for that

And as a matter of fact, PEP 390 is not so important, because at the
end, the changes that are made in PEP 345 are
the most important ones. They allow describing metadata fields that
can vary depending on the target platform.

PEP 390 is just the simplest way to make it possible to describe
static metadata using a setup.cfg ConfigParser section,
rather than some code in setup.py. And when we built it, it helped us
understand what we really wanted on PEP 345 side (with a big insights
from people like Marc-Andr? on this)

But PEP 390 limits its scope to the Metadata fields and now waits for
PEP 345 changes.

Your proposal is a partial alternative to PEP 390, and is a quite
interesting work to dig in, and the SIG is discussing it,
but a very important point about it is that it does a lot more. It's a
new *building* system on its own, and does not limit itself to
describing metadata fields.

That's fine, and that's quite interesting, but I doubt we will be able
to push it in Distutils itself any sooner, because
it a whole different system ala scons (which is a great tool don't get
me wrong). I think (as I told you before IIRC) that you need to create
a project to demonstrate it  and I think it can be great to continue
getting some help from Distutils-SIG for this.

Furthermore, building such a tool on the top of Distutils can improve
Distutils itself, because it would be yet another project that would
provide valuable feedback on the APIs and standards we provide. That's
for example what we are getting from projects like PyPM or Enthought
Enstaller.

As a matter of fact, there's already great feedback from Active State
on PEP 376 because they want to be early adopters of the upcoming
standard and make sure we do the right choices in there. Enthought
also started to use the prototype we have written for PEP 386 (version
comparisons) IIRC.

But in the existing PEPs discussions, we need to focus on their
respective scopes.  And they are all targeted to improve the existing
stdlib tool : Distutils. (and a wanted side effect is to make
Distutils smaller if we can, and a robust basis for all third party
tools)


That's why you can't just drop a counter-PEP to any existing PEP we
have started for Distutils. Imho help us build those PEPs, and / or
create your own build system, wether its based or not on Distutils.

Regards
Tarek

From brett at python.org  Fri Nov  6 02:25:20 2009
From: brett at python.org (Brett Cannon)
Date: Thu, 5 Nov 2009 17:25:20 -0800
Subject: [Python-Dev] Refactoring installation schemes
In-Reply-To: <6d81ac30c331f8b9ca71a75d1049ffd4@preisshare.net>
References: <94bdd2610910271455s7854e6a9u94e3aeeeacb69c02@mail.gmail.com> 
	<94bdd2610910281105q328bc3a5qaed020ef3afe28df@mail.gmail.com> 
	<94bdd2610911040429j50a6df87m51aac2464dd13bd7@mail.gmail.com> 
	<c77acedd9a48dd7036cca55fde42de0a@preisshare.net>
	<9cee7ab80911041440j43e51e39qc77a426547d02bcb@mail.gmail.com> 
	<39deb0d1eef08419dad65d7354a8ffb1@preisshare.net>
	<94bdd2610911050235l4e851612i46e0392d352eaa20@mail.gmail.com> 
	<0d223447fa9aebaeb78f5be82eb3aaab@preisshare.net>
	<ca471dc20911051518i75b1a6b9t13f07c5747d65f4a@mail.gmail.com> 
	<6d81ac30c331f8b9ca71a75d1049ffd4@preisshare.net>
Message-ID: <bbaeab100911051725s24e6b457x47696d1465fc6a7b@mail.gmail.com>

On Thu, Nov 5, 2009 at 15:55, David Lyon <david.lyon at preisshare.net> wrote:
>
> Tarek, Guido,
>
> Forgive my grumpy tone..
>
> Looking positive, if we now have a rough consensus that the PEPS might
> represent some 300+ lines of code... then good - lets get started, that's
> all that I meant. I'm glad above all, that you noticed that part foremost.
>
> If it's a simple case that alternative's to the solutions cannot
> be considered - because I am grumpy - whatever, then that's something
> I can live with. I didn't know that proposing a different solution
> wasn't a good idea.

That is passive-aggressive and uncalled for. Proposing an opposing
view is fine when done respectfully which you have not done here on
python-dev.

>
>> PS. Submitting a counter-PEP to the peps editors that hasn't been
>> discussed on the SIG list at all is also a bad move. You really need
>> to change the way you try to interact with the SIG.
>
> Here are the references to the discussions from distutils-sig. In these
> references you'll see that my interaction with the list was not grumpy
> but was entirely positive. I feel that I got a fair hearing on distutils
> list and I had a number of supporters to my proposal.
>

That's fine, but it's Tarek's call and as a group we reached the
conclusion that Tarek is doing a fine job and is being more than fair.
It has been stated as such and yet you keep pushing us like we have
not heard you. We did hear you the first time, but you just seem to
not get the message. Adding to the fact that your tone has been an
issue and that we don't want to listen to it anymore has pushed Guido
to the brink and me over the edge. I honestly don't know how Tarek
keeps his patience over on the distutils SIG if he has to deal with
kind of crap constantly.

Personally, I am now sending your emails to the trash since you can't
seem to even reply to Guido's email w/o some nasty undertone.
Hopefully someday you can learn to communicate in a better, friendlier
manner.

-Brett

From david.lyon at preisshare.net  Fri Nov  6 03:51:09 2009
From: david.lyon at preisshare.net (David Lyon)
Date: Thu, 05 Nov 2009 21:51:09 -0500
Subject: [Python-Dev] Refactoring installation schemes
In-Reply-To: <94bdd2610911051709y6050bca1w8e10c44d3598d145@mail.gmail.com>
References: <94bdd2610910271455s7854e6a9u94e3aeeeacb69c02@mail.gmail.com>
	<94bdd2610910281105q328bc3a5qaed020ef3afe28df@mail.gmail.com>
	<94bdd2610911040429j50a6df87m51aac2464dd13bd7@mail.gmail.com>
	<c77acedd9a48dd7036cca55fde42de0a@preisshare.net>
	<9cee7ab80911041440j43e51e39qc77a426547d02bcb@mail.gmail.com>
	<39deb0d1eef08419dad65d7354a8ffb1@preisshare.net>
	<94bdd2610911050235l4e851612i46e0392d352eaa20@mail.gmail.com>
	<0d223447fa9aebaeb78f5be82eb3aaab@preisshare.net>
	<ca471dc20911051518i75b1a6b9t13f07c5747d65f4a@mail.gmail.com>
	<6d81ac30c331f8b9ca71a75d1049ffd4@preisshare.net>
	<94bdd2610911051709y6050bca1w8e10c44d3598d145@mail.gmail.com>
Message-ID: <4257d43e76eca63ae99c2638a0dc5e54@preisshare.net>


Tarek,

> Your proposal is a partial alternative to PEP 390, and is a quite
> interesting work to dig in, and the SIG is discussing it,
> but a very important point about it is that it does a lot more. It's a
> new *building* system on its own, and does not limit itself to
> describing metadata fields.

Actually no, the proposal is only for a metadata installation of packages
that addresses the security problem of requiring users to run untrusted and

unverified setup.py's.

That was where this discussion originated and that is what I am addressing.

PEP-390 doesn't go there at all...

My proposal uses the static metadata contained within existing package 
formats (PKG_INFO + sources.txt) and therefore doesn't require any 
changes to the build system.

There isn't a new build system. I never proposed that.

However PEP-390 does imply changing the build system.

> That's why you can't just drop a counter-PEP to any existing PEP we
> have started for Distutils. Imho help us build those PEPs, and / or
> create your own build system, wether its based or not on Distutils.

Once again, I can't see why all the fuss.

All I did was have a discussion on distutils-ml about a different way 
of specifying package dependencies that might work across different 
python platforms. 

It was suggested to me that I do a PEP..

So I took the documentation on face value that developers could submit
a PEP-proposal.

> Imho help us build those PEPs, 

This could best apply to PEP-345:

(PKG_INFO)
Requires: sys
Requires(python <= 2.4): lxml
Requires(windows): win32com
Requires(linux): pyodbc
Requires(linux ubuntu gnome python <= 3.4): gix
Requires(windows xp python < 2.2): win32prn

Thank's for reading my alternate-metadata installation proposal
and I accept your feedback. What I can do is clarify that my
proposal is not for a build system but for a metadata installation 
scheme based on a static setup.py, existing metadata and use of 
"python -m setup install" for invocation.

I'm just trying to make this stuff no more complicated than it
needs to be...

David












From chris at subtlety.com  Fri Nov  6 03:59:24 2009
From: chris at subtlety.com (Chris Bergstresser)
Date: Thu, 5 Nov 2009 20:59:24 -0600
Subject: [Python-Dev] Retrieve an arbitrary element from
	asetwithoutremoving it
In-Reply-To: <97069B15530D4C66AC1D4BCCB70E2FD5@RaymondLaptop1>
References: <7b0045ed0910301843y7059d316s8a851142b78b7aeb@mail.gmail.com> 
	<4AF0D0D7.3090403@canterbury.ac.nz>
	<200911042234.36796.steve@pearwood.info> 
	<f3cc57c60911041332v479cc3b0t19d9601e52002128@mail.gmail.com> 
	<99BB308202224D31AA5300223DB59912@RaymondLaptop1>
	<7b0045ed0911051243q18f8324ak9abaa98ac26948e9@mail.gmail.com> 
	<4AF341BD.2010203@v.loewis.de>
	<7b0045ed0911051419i76f4221dye8ecccc424179ec6@mail.gmail.com> 
	<BD050FFAAA19443DA0DA34FD2BD740CF@RaymondLaptop1>
	<97069B15530D4C66AC1D4BCCB70E2FD5@RaymondLaptop1>
Message-ID: <7b0045ed0911051859y13f8abf3u7b5b4549aa2a41fc@mail.gmail.com>

On Thu, Nov 5, 2009 at 5:02 PM, Raymond Hettinger <python at rcn.com> wrote:
> Forgot to post the code. ?It is short, fast, and easy. ?It is explicit about
> handing the case with an empty input. ?And it is specific about which value
> it returns (always the first iterated value; not an arbitrary one). ?There's
> no guessing about what it does. ?It gets the job done.

    I'm trying to take this suggestion in the best possible light,
which is that you honestly think I didn't read past Chapter 3 of the
Python Tutorial, and I am therefore in fact unfamiliar with function
definitions.

-- Chris

From chris at subtlety.com  Fri Nov  6 04:12:45 2009
From: chris at subtlety.com (Chris Bergstresser)
Date: Thu, 5 Nov 2009 21:12:45 -0600
Subject: [Python-Dev] Retrieve an arbitrary element from a
	setwithoutremoving it
In-Reply-To: <f3cc57c60911051630o51f6cf32m4e60a69ceb0ca922@mail.gmail.com>
References: <7b0045ed0910301843y7059d316s8a851142b78b7aeb@mail.gmail.com> 
	<4AF0D0D7.3090403@canterbury.ac.nz>
	<200911042234.36796.steve@pearwood.info> 
	<f3cc57c60911041332v479cc3b0t19d9601e52002128@mail.gmail.com> 
	<99BB308202224D31AA5300223DB59912@RaymondLaptop1>
	<7b0045ed0911051243q18f8324ak9abaa98ac26948e9@mail.gmail.com> 
	<d38f5330911051309l78166534ndf8320e09f9892bc@mail.gmail.com> 
	<f3cc57c60911051504l48890d09q74d46580bf1c20ab@mail.gmail.com> 
	<9CAD2251-C51D-472B-98A4-104E1AFEEDE1@fuhm.net>
	<f3cc57c60911051630o51f6cf32m4e60a69ceb0ca922@mail.gmail.com>
Message-ID: <7b0045ed0911051912x6cb1bd19r7677a10f47d36a45@mail.gmail.com>

On Thu, Nov 5, 2009 at 6:30 PM, geremy condra <debatem1 at gmail.com> wrote:
> I'm testing the speed because the claim was made that the pop/add
> approach was inefficient. Here's the full quote:
>
>> ? ?The obvious way, for newcomers, of achieving the effect is:
>>
>> ?x = s.pop()
>> ?s.add(x)
>>
>> ... and that's simply horrible in terms of efficiency. ?So the
>> "obvious" way of doing it in Python is wrong(TM), and the "correct"
>> way of doing it is obscure and raises misleading exceptions.

   I was talking mainly from a theoretical standpoint, and because the
library I'm working on is designed to work seamlessly over the
network.  In those cases, where the set the user is working with is
actually a proxy object across the wire, the time to acquire the
locks, remove the object, release the locks, reacquire the locks, add
the object, then rerelease the locks is *significantly* more expensive
than just noting the set hasn't changed and returning a cached object
from it.

-- Chris

From guido at python.org  Fri Nov  6 04:23:26 2009
From: guido at python.org (Guido van Rossum)
Date: Thu, 5 Nov 2009 19:23:26 -0800
Subject: [Python-Dev] PEP 3003 - Python Language Moratorium
In-Reply-To: <bbaeab100911051535g68cefa18kdadea33f78de9495@mail.gmail.com>
References: <ca471dc20911030935l25251e0diec64427f5d7bcbf2@mail.gmail.com> 
	<ca471dc20911051405o287e0f41m26d38860a200b47f@mail.gmail.com> 
	<ea2499da0911051453n3569387aj581ad4acf5b799c8@mail.gmail.com> 
	<4222a8490911051521r1b9c8165id6e0f12d62da0613@mail.gmail.com> 
	<ca471dc20911051526u7f3642ecg19e4eb73bcca1d3a@mail.gmail.com> 
	<bbaeab100911051535g68cefa18kdadea33f78de9495@mail.gmail.com>
Message-ID: <ca471dc20911051923j113d4deame8757c5775f040d2@mail.gmail.com>

On Thu, Nov 5, 2009 at 3:35 PM, Brett Cannon <brett at python.org> wrote:
> On Thu, Nov 5, 2009 at 15:26, Guido van Rossum <guido at python.org> wrote:
>> I have come to the conclusion that there are better ways to
>> pre-announce that a module is going to disappear instead of
>> deprecation warnings.
>
> What exactly are those better ways? Document as deprecated only?

Sorry, I have an existence proof, but no construction. :-)

Ideas welcome. Silent deprecations, loud documentation, biweekly home
visits, whatever, as long as it doesn't log a message by default.

The thing is, in practice, people will be testing with a new release
anyway, and that's the earliest time they are likely to take action
(other than silencing the warnings).

-- 
--Guido van Rossum (python.org/~guido)

From rdmurray at bitdance.com  Fri Nov  6 04:53:27 2009
From: rdmurray at bitdance.com (R. David Murray)
Date: Thu, 5 Nov 2009 22:53:27 -0500 (EST)
Subject: [Python-Dev] Status of the Buildbot fleet and related bugs
Message-ID: <Pine.LNX.4.64.0911052209000.12274@kimball.webabinitio.net>

The buildbot waterfall is much greener now.  Thanks to all who have
contributed to making it so (and it hasn't just been Mark and Antoine
and I, though we've been the most directly active (and yes, Mark, you
did contribute several fixes!)).

The 'stable builders' fleet is green now except for:

     (1) issue 7269: occasional 2.6/trunk bsddb3 failures on windows
     (2) issue 6748: random 3.1/3.x failures on most buidbots.
     (3) the klose-debian-ppc builder being offline

Of these, (2) is by _far_ the biggest issue, and the one that causes the
most flap (success-failure-success).  And flap is the thing that most
harms the buildbot usefulness.

Anyone who wants to debug this on a platform where it is consistently
reproducible please email me your public key and I'll give you a shell
account on my buildbot (Antoine already has one).

In the 'unstable' builder fleet, Antoine's new builder seems to be
stable across the board, while mine fails consistently on 3.1 and 3.x
because of the test_telnetlib bug.  Thomas Heller's OS X buildbot is
consistently having lots of test failures (the same ones each time
I've checked).  The master claims the klose-debian-alpha buildbot
doesn't know about branches, which is odd since it was working not
too long ago.  The remaining buildslaves appear to have been offline
for some time.

Open issues here are:

     (1) issue 3864: FreeBSD testing hangs consistently.  According to the
         ticket this is a FreeBSD bug fixed in 6.4, so an OS upgrade
         on the buildslave would probably solve it.
     (2) issue 4970: consistent signal 32 error on the norwitz-x86 Gentoo
         buildslave in 3.1 and 3.x.  This may be due to the box
         running an old threading library, but it does make one wonder what
         changed in 3.x that exposed it.

Another issue that I've seen on the buildbots but that doesn't
seem to be showing up right now (is it fixed?) is issue 7251, which
Mark is working on.

So, overall I think the buildbot fleet is in good shape, and if
we can nail issue 6748 I think it will be back to being an
important resource for sanity checking our checkins.

By the way, Georg set up the IRC interface on the #python-dev channel,
so you can hang out there if you want to get realtime reports of which
buildbots have going from success to failure and vice versa.

--David

From mbk.lists at gmail.com  Fri Nov  6 05:12:25 2009
From: mbk.lists at gmail.com (Mike Krell)
Date: Thu, 5 Nov 2009 21:12:25 -0700
Subject: [Python-Dev] 2.7 Release? 2.7 == last of the 2.x line?
In-Reply-To: <4AF330A7.5000809@v.loewis.de>
References: <E077843B-919A-4AC7-95D7-0313E4D0328A@gmail.com>
	<4AEFD9AE.7090104@molden.no> <4AEFF450.9060007@voidspace.org.uk>
	<79EADA413EFB44F6864EF71D5B830088@RaymondLaptop1>
	<79990c6b0911030323x47a9f008ue7256c56e88c0ee8@mail.gmail.com>
	<5CB59A21-D8C9-4BC1-A1F6-4438B86A8BC5@twistedmatrix.com>
	<4AF1CFAD.6000305@v.loewis.de>
	<da7032ce0911041130h40ebd3adu49cd9b08734ce74f@mail.gmail.com>
	<4AF2C032.4040605@gmail.com> <4AF330A7.5000809@v.loewis.de>
Message-ID: <da7032ce0911052012w6c3c6cc4r447a77e3702fd421@mail.gmail.com>

On Thu, Nov 5, 2009 at 1:08 PM, "Martin v. L?wis" <martin at v.loewis.de>wrote:

> > Mike Krell wrote:
> >>     Well, 3to2 would then be an option for you: use Python 3 as the
> source
> >>     language.
> >
> > Making life easier for 3to2 is an *excellent* rationale for backports.
>

Clarifying a bit of potentially misleading editing:  I wrote neither of the
statements quoted above.  M v L wrote the first and Nick C. wrote the
second.


> I'm skeptical. If new features get added to 2.7: why would that simplify
> 3to2? It couldn't *use* these features, since it surely would have to
> support 2.6 and earlier as well.
>
> Not sure what 3to2 would do about difficult-to-convert 3.x feature (as,
> perhaps, the nonlocal keyword). If it currently gives up, it then may
> offer you to restrict your target versions to 2.7+. Not sure whether
> users would use that option, though - perhaps they rather stop using
> nonlocal in 3.x if 3to2 cannot support it for all 2.x versions they are
> interested in.
>

> Perhaps 3to2 has a work-around that still provides a good backport in
> most cases. Then, the backport would not make the tool any simpler: if
> 3to2 would start using the backport, it would actually get more
> complicated (not easier), as it now needs to support two cases.
>

I basically agree with you here, except perhaps for the likely definition of
"versions they are interested in".  You have suggested on several occasions
that a 2.7 (or 2.8) containing new syntax such as nonlocal would be of
limited value because the vast majority of developers interested in
supporting 2.x would have to support 2.6 as well.  I respectfully suggest
that may not necessarily be the case.  I suspect that there are lots of
small fish out there like me who have the luxury of being able to hop onto
whatever version of the language suits them the best.  Not to mention all of
the new users who will be drawn to Python over the next several years while
the 3.x standard library and third party library situation becomes more
stable and comprehensive.  Why not make the 2.x feature set the best it can
be given the likelihood that 2.x will be the most compelling alternative to
many users until the 3.x libraries mature?

Of course, it's easy for me to ask other people to the hard work.  It might
be fun to take a crack at implementing nonlocal myself, but I know next to
nothing about the implementation of CPython.  By the time I pestered y'all
with enough questions to get up to speed, you'd probably wish you'd just
implemented it yourself -- less work :-)

   Mike
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20091105/aeb19871/attachment-0001.htm>

From john.arbash.meinel at gmail.com  Fri Nov  6 05:21:05 2009
From: john.arbash.meinel at gmail.com (John Arbash Meinel)
Date: Thu, 05 Nov 2009 22:21:05 -0600
Subject: [Python-Dev] Retrieve an arbitrary element from
 a	setwithoutremoving it
In-Reply-To: <f3cc57c60911051504l48890d09q74d46580bf1c20ab@mail.gmail.com>
References: <7b0045ed0910301843y7059d316s8a851142b78b7aeb@mail.gmail.com>	<200911040946.15805.steve@pearwood.info>	<4AF0D0D7.3090403@canterbury.ac.nz>	<200911042234.36796.steve@pearwood.info>	<f3cc57c60911041332v479cc3b0t19d9601e52002128@mail.gmail.com>	<99BB308202224D31AA5300223DB59912@RaymondLaptop1>	<7b0045ed0911051243q18f8324ak9abaa98ac26948e9@mail.gmail.com>	<d38f5330911051309l78166534ndf8320e09f9892bc@mail.gmail.com>
	<f3cc57c60911051504l48890d09q74d46580bf1c20ab@mail.gmail.com>
Message-ID: <4AF3A431.2020607@gmail.com>

geremy condra wrote:
> On Thu, Nov 5, 2009 at 4:09 PM, Alexander Belopolsky
> <alexander.belopolsky at gmail.com> wrote:
>> On Thu, Nov 5, 2009 at 3:43 PM, Chris Bergstresser <chris at subtlety.com> wrote:
>>> .. and "x = iter(s).next()" raises a StopIteration
>>> exception.
>> And that's why the documented recipe should probably recommend
>> next(iter(s), default) instead.  Especially because iter(s).next() is
>> not even valid code in 3.0.
> 
> This seems reasonably legible to you? Strikes me as coding by
> incantation. Also, while I've heard people say that the naive
> approach is slower, I'm not getting that result. Here's my test:
> 
> 
>>>> smrt = timeit.Timer("next(iter(s))", "s=set(range(100))")
>>>> smrt.repeat(10)
> [1.2845709323883057, 0.60247397422790527, 0.59621405601501465,
> 0.59133195877075195, 0.58387589454650879, 0.56839084625244141,
> 0.56839680671691895, 0.56877803802490234, 0.56905913352966309,
> 0.56846404075622559]
> 
>>>> naive = timeit.Timer("x=s.pop();s.add(x)", "s=set(range(100))")
>>>> naive.repeat(10)
> [0.93139314651489258, 0.53566789627075195, 0.53674602508544922,
> 0.53608798980712891, 0.53634309768676758, 0.53557991981506348,
> 0.53578495979309082, 0.53666114807128906, 0.53576493263244629,
> 0.53491711616516113]
> 
> 
> Perhaps my test is flawed in some way?
> 
> Geremy Condra

Well, it also uses a fairly trivial set. 'set(range(100))' is going to
generally have 0 collisions and everything will hash to a unique bucket.
 As such, pop ing an item out of the hash is a single "val = table[int &
mask]; table[int & mask] = _dummy", and then looking it up again
requires 2 table lookups (one finds the _dummy, the next finds that the
chain is broken and can rewrite the _dummy.)

However, if a set is more full, or has more collisions, or ... then
pop() and add() become relatively more expensive.

Surprising to me, is that "next(iter(s))" was actually slower than
.pop() + .add() for 100 node set in my testing:

$ alias TIMEIT="python -m timeit -s 's = set(range(100)'"
$ TIMEIT "x = next(iter(s))"
1000000 loops, best of 3: 0.263 usec per loop

$ TIMEIT "x = s.pop(); s.add(x)"
1000000 loops, best of 3: 0.217 usec per loop

though both are much slower than the fastest we've found:
$ TIMEIT "for x in s: break"
10000000 loops, best of 3: 0.0943 usec per loop


now, what about a set with *lots* of collisions. Create 100 keys that
all hash to the same bucket:
aliase TIMEIT="python -m timeit -s 's = set([x*1024*1024 for x in
range(100)])'"
$ TIMEIT "x = next(iter(s))"
1000000 loops, best of 3: 0.257 usec per loop

$ TIMEIT "x = s.pop(); s.add(x)"
1000000 loops, best of 3: 0.218 usec per loop

I tried a few different ways, and I got the same results, until I did:

$ python -m timeit -s "s = set(range(100000, 1000100))" "next(iter(s))"
1000 loops, best of 3: 255 usec per loop

^^^^ Now something seems terribly wrong here. next(iter(s)) suddenly
jumps up from being < 0.3 us, to being more than 200us. Or ~1000x slower.

I realize the above has 900k keys, which is big. But 'next(iter(s))'
should only touch 1, and, in fact, should always return the *first*
entry. My best guess is just that the *first* entry in the internal set
table is no longer close to offset 0, which means that 'next(iter(s))'
has to evaluate a bunch of table slots before it finds a non-empty entry.


Anyway, none of the proposals will really ever be faster than:
  for x in s: break

It is a bit ugly of a construct, but you don't have an attribute lookup,
etc. As long as you don't do:
  for x in s: pass

Then it stays nice and fast.

John
=:->

From debatem1 at gmail.com  Fri Nov  6 06:04:30 2009
From: debatem1 at gmail.com (geremy condra)
Date: Fri, 6 Nov 2009 00:04:30 -0500
Subject: [Python-Dev] Retrieve an arbitrary element from a
	setwithoutremoving it
In-Reply-To: <4AF3A431.2020607@gmail.com>
References: <7b0045ed0910301843y7059d316s8a851142b78b7aeb@mail.gmail.com>
	<200911040946.15805.steve@pearwood.info>
	<4AF0D0D7.3090403@canterbury.ac.nz>
	<200911042234.36796.steve@pearwood.info>
	<f3cc57c60911041332v479cc3b0t19d9601e52002128@mail.gmail.com>
	<99BB308202224D31AA5300223DB59912@RaymondLaptop1>
	<7b0045ed0911051243q18f8324ak9abaa98ac26948e9@mail.gmail.com>
	<d38f5330911051309l78166534ndf8320e09f9892bc@mail.gmail.com>
	<f3cc57c60911051504l48890d09q74d46580bf1c20ab@mail.gmail.com>
	<4AF3A431.2020607@gmail.com>
Message-ID: <f3cc57c60911052104k514d313fn4780f06b4a392274@mail.gmail.com>

On Thu, Nov 5, 2009 at 11:21 PM, John Arbash Meinel
<john.arbash.meinel at gmail.com> wrote:
> geremy condra wrote:
>> On Thu, Nov 5, 2009 at 4:09 PM, Alexander Belopolsky
>> <alexander.belopolsky at gmail.com> wrote:
>>> On Thu, Nov 5, 2009 at 3:43 PM, Chris Bergstresser <chris at subtlety.com> wrote:
>>>> .. and "x = iter(s).next()" raises a StopIteration
>>>> exception.
>>> And that's why the documented recipe should probably recommend
>>> next(iter(s), default) instead. ?Especially because iter(s).next() is
>>> not even valid code in 3.0.
>>
>> This seems reasonably legible to you? Strikes me as coding by
>> incantation. Also, while I've heard people say that the naive
>> approach is slower, I'm not getting that result. Here's my test:
>>
>>
>>>>> smrt = timeit.Timer("next(iter(s))", "s=set(range(100))")
>>>>> smrt.repeat(10)
>> [1.2845709323883057, 0.60247397422790527, 0.59621405601501465,
>> 0.59133195877075195, 0.58387589454650879, 0.56839084625244141,
>> 0.56839680671691895, 0.56877803802490234, 0.56905913352966309,
>> 0.56846404075622559]
>>
>>>>> naive = timeit.Timer("x=s.pop();s.add(x)", "s=set(range(100))")
>>>>> naive.repeat(10)
>> [0.93139314651489258, 0.53566789627075195, 0.53674602508544922,
>> 0.53608798980712891, 0.53634309768676758, 0.53557991981506348,
>> 0.53578495979309082, 0.53666114807128906, 0.53576493263244629,
>> 0.53491711616516113]
>>
>>
>> Perhaps my test is flawed in some way?
>>
>> Geremy Condra
>
> Well, it also uses a fairly trivial set. 'set(range(100))' is going to
> generally have 0 collisions and everything will hash to a unique bucket.
> ?As such, pop ing an item out of the hash is a single "val = table[int &
> mask]; table[int & mask] = _dummy", and then looking it up again
> requires 2 table lookups (one finds the _dummy, the next finds that the
> chain is broken and can rewrite the _dummy.)
>
> However, if a set is more full, or has more collisions, or ... then
> pop() and add() become relatively more expensive.
>
> Surprising to me, is that "next(iter(s))" was actually slower than
> .pop() + .add() for 100 node set in my testing:
>
> $ alias TIMEIT="python -m timeit -s 's = set(range(100)'"
> $ TIMEIT "x = next(iter(s))"
> 1000000 loops, best of 3: 0.263 usec per loop
>
> $ TIMEIT "x = s.pop(); s.add(x)"
> 1000000 loops, best of 3: 0.217 usec per loop
>
> though both are much slower than the fastest we've found:
> $ TIMEIT "for x in s: break"
> 10000000 loops, best of 3: 0.0943 usec per loop
>
>
> now, what about a set with *lots* of collisions. Create 100 keys that
> all hash to the same bucket:
> aliase TIMEIT="python -m timeit -s 's = set([x*1024*1024 for x in
> range(100)])'"
> $ TIMEIT "x = next(iter(s))"
> 1000000 loops, best of 3: 0.257 usec per loop
>
> $ TIMEIT "x = s.pop(); s.add(x)"
> 1000000 loops, best of 3: 0.218 usec per loop
>
> I tried a few different ways, and I got the same results, until I did:
>
> $ python -m timeit -s "s = set(range(100000, 1000100))" "next(iter(s))"
> 1000 loops, best of 3: 255 usec per loop
>
> ^^^^ Now something seems terribly wrong here. next(iter(s)) suddenly
> jumps up from being < 0.3 us, to being more than 200us. Or ~1000x slower.
>
> I realize the above has 900k keys, which is big. But 'next(iter(s))'
> should only touch 1, and, in fact, should always return the *first*
> entry. My best guess is just that the *first* entry in the internal set
> table is no longer close to offset 0, which means that 'next(iter(s))'
> has to evaluate a bunch of table slots before it finds a non-empty entry.
>
>
> Anyway, none of the proposals will really ever be faster than:
> ?for x in s: break
>
> It is a bit ugly of a construct, but you don't have an attribute lookup,
> etc. As long as you don't do:
> ?for x in s: pass
>
> Then it stays nice and fast.
>
> John
> =:->

Thanks for the info. Doing a bit more digging it appears that taking
the lookup out of the picture puts the pop/add more or less on par
with the for x in s: break solution, with the former behaving more
predictably and the latter averaging marginally faster times. Either
way, the loss in readability isn't worth it to me to get the minor
improvement in performance.

Given that adding a set.pick method would incur half the lookup
cost that pop/add does, I think its reasonable to say that even
using the fastest method proposed to implement it won't differ
all that greatly from the least efficient proposal, and that its
therefore pointless to consider set.pick from an optimisation
standpoint. Aesthetics, of course, are another thing altogether.

Geremy Condra

From steve at pearwood.info  Fri Nov  6 06:18:38 2009
From: steve at pearwood.info (Steven D'Aprano)
Date: Fri, 6 Nov 2009 16:18:38 +1100
Subject: [Python-Dev] PEP 3003 - Python Language Moratorium
In-Reply-To: <4AF29753.3040801@voidspace.org.uk>
References: <ca471dc20911030935l25251e0diec64427f5d7bcbf2@mail.gmail.com>
	<hctuqq$in0$1@ger.gmane.org> <4AF29753.3040801@voidspace.org.uk>
Message-ID: <200911061618.38790.steve@pearwood.info>

On Thu, 5 Nov 2009 08:13:55 pm Michael Foord wrote:

> There are several partial implementations, including Python inspired
> languages, but if we are looking at 'major complete implementations'
> then the current list seems to be: CPython, Jython, IronPython and
> PyPy. Even Unladen Swallow is a fork (sorry - I mean branch) of
> CPython rather than a separate implementation.

I don't know how mature or active it is, so it may not count as either 
major or complete, but there's also CLPython:

http://common-lisp.net/project/clpython/




-- 
Steven D'Aprano

From martin at v.loewis.de  Fri Nov  6 06:43:06 2009
From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=)
Date: Fri, 06 Nov 2009 06:43:06 +0100
Subject: [Python-Dev] Retrieve an arbitrary element from
 asetwithoutremoving it
In-Reply-To: <7b0045ed0911051859y13f8abf3u7b5b4549aa2a41fc@mail.gmail.com>
References: <7b0045ed0910301843y7059d316s8a851142b78b7aeb@mail.gmail.com>
	<4AF0D0D7.3090403@canterbury.ac.nz>
	<200911042234.36796.steve@pearwood.info>
	<f3cc57c60911041332v479cc3b0t19d9601e52002128@mail.gmail.com>
	<99BB308202224D31AA5300223DB59912@RaymondLaptop1>
	<7b0045ed0911051243q18f8324ak9abaa98ac26948e9@mail.gmail.com>
	<4AF341BD.2010203@v.loewis.de>
	<7b0045ed0911051419i76f4221dye8ecccc424179ec6@mail.gmail.com>
	<BD050FFAAA19443DA0DA34FD2BD740CF@RaymondLaptop1>
	<97069B15530D4C66AC1D4BCCB70E2FD5@RaymondLaptop1>
	<7b0045ed0911051859y13f8abf3u7b5b4549aa2a41fc@mail.gmail.com>
Message-ID: <4AF3B76A.1050006@v.loewis.de>

> On Thu, Nov 5, 2009 at 5:02 PM, Raymond Hettinger <python at rcn.com> wrote:
>> Forgot to post the code.  It is short, fast, and easy.  It is explicit about
>> handing the case with an empty input.  And it is specific about which value
>> it returns (always the first iterated value; not an arbitrary one).  There's
>> no guessing about what it does.  It gets the job done.
> 
>     I'm trying to take this suggestion in the best possible light,
> which is that you honestly think I didn't read past Chapter 3 of the
> Python Tutorial, and I am therefore in fact unfamiliar with function
> definitions.

I read Raymond's suggestion rather as a question: why bother with a
tedious, multi-year process, when a three-line function will achieve
exactly the same?

Regards,
Martin

From brett at python.org  Fri Nov  6 06:55:51 2009
From: brett at python.org (Brett Cannon)
Date: Thu, 5 Nov 2009 21:55:51 -0800
Subject: [Python-Dev] PEP 3003 - Python Language Moratorium
In-Reply-To: <ca471dc20911051923j113d4deame8757c5775f040d2@mail.gmail.com>
References: <ca471dc20911030935l25251e0diec64427f5d7bcbf2@mail.gmail.com> 
	<ca471dc20911051405o287e0f41m26d38860a200b47f@mail.gmail.com> 
	<ea2499da0911051453n3569387aj581ad4acf5b799c8@mail.gmail.com> 
	<4222a8490911051521r1b9c8165id6e0f12d62da0613@mail.gmail.com> 
	<ca471dc20911051526u7f3642ecg19e4eb73bcca1d3a@mail.gmail.com> 
	<bbaeab100911051535g68cefa18kdadea33f78de9495@mail.gmail.com> 
	<ca471dc20911051923j113d4deame8757c5775f040d2@mail.gmail.com>
Message-ID: <bbaeab100911052155m3558186xbcd833bc50c9a59f@mail.gmail.com>

On Thu, Nov 5, 2009 at 19:23, Guido van Rossum <guido at python.org> wrote:
> On Thu, Nov 5, 2009 at 3:35 PM, Brett Cannon <brett at python.org> wrote:
>> On Thu, Nov 5, 2009 at 15:26, Guido van Rossum <guido at python.org> wrote:
>>> I have come to the conclusion that there are better ways to
>>> pre-announce that a module is going to disappear instead of
>>> deprecation warnings.
>>
>> What exactly are those better ways? Document as deprecated only?
>
> Sorry, I have an existence proof, but no construction. :-)
>
> Ideas welcome. Silent deprecations, loud documentation, biweekly home
> visits, whatever, as long as it doesn't log a message by default.

Well, one option is to come up with the equivalent of -3, but for all
warnings; the antithesis of -W. And obviously glaring deprecation
warnings in the docs (removal has been discussed but always shot down
as someone who comes across old code might still need docs for it).

The clarification I need is will this in any way influence when
modules are removed. If they stay in for the life of a major version
then I want it made clear that bug fixes for the code take lower
priority over all other code in the standard library.

-Brett

From steve at pearwood.info  Fri Nov  6 07:01:01 2009
From: steve at pearwood.info (Steven D'Aprano)
Date: Fri, 6 Nov 2009 17:01:01 +1100
Subject: [Python-Dev] Retrieve an arbitrary element from a
	setwithoutremoving it
In-Reply-To: <9d153b7c0911051552v7cfabf61ifede37a72ec239d8@mail.gmail.com>
References: <7b0045ed0910301843y7059d316s8a851142b78b7aeb@mail.gmail.com>
	<9CAD2251-C51D-472B-98A4-104E1AFEEDE1@fuhm.net>
	<9d153b7c0911051552v7cfabf61ifede37a72ec239d8@mail.gmail.com>
Message-ID: <200911061701.02292.steve@pearwood.info>

On Fri, 6 Nov 2009 10:52:54 am Yuvgoog Greenle wrote:
> On Fri, Nov 6, 2009 at 1:17 AM, James Y Knight <foom at fuhm.net> wrote:
> > Is this thread over yet?
>
> Sorry, I just had to point out that pop/add has a side effect that
> would be apparent on a set that multiple threads access - it loses an
> item and then gets it back. Sounds like a sleeper race condition
> that's going to be rare but extremely hard to find if it does occur.
> Crooked as a gil.


Surely Raymond's suggestion also suffers from a similar race condition?

for x in set:
    return x

creates a set_iterator. If another thread modifies the original set 
after the set_iterator is created but before the return, you would get 
a mysterious and almost impossible to debug RuntimeError.



-- 
Steven D'Aprano

From chris at subtlety.com  Fri Nov  6 07:19:27 2009
From: chris at subtlety.com (Chris Bergstresser)
Date: Fri, 6 Nov 2009 00:19:27 -0600
Subject: [Python-Dev] Retrieve an arbitrary element from
	asetwithoutremoving it
In-Reply-To: <4AF3B76A.1050006@v.loewis.de>
References: <7b0045ed0910301843y7059d316s8a851142b78b7aeb@mail.gmail.com> 
	<f3cc57c60911041332v479cc3b0t19d9601e52002128@mail.gmail.com> 
	<99BB308202224D31AA5300223DB59912@RaymondLaptop1>
	<7b0045ed0911051243q18f8324ak9abaa98ac26948e9@mail.gmail.com> 
	<4AF341BD.2010203@v.loewis.de>
	<7b0045ed0911051419i76f4221dye8ecccc424179ec6@mail.gmail.com> 
	<BD050FFAAA19443DA0DA34FD2BD740CF@RaymondLaptop1>
	<97069B15530D4C66AC1D4BCCB70E2FD5@RaymondLaptop1> 
	<7b0045ed0911051859y13f8abf3u7b5b4549aa2a41fc@mail.gmail.com> 
	<4AF3B76A.1050006@v.loewis.de>
Message-ID: <7b0045ed0911052219u1893bd62gebf71bf46721014a@mail.gmail.com>

On Thu, Nov 5, 2009 at 11:43 PM, "Martin v. L?wis" <martin at v.loewis.de> wrote:
> I read Raymond's suggestion rather as a question: why bother with a
> tedious, multi-year process, when a three-line function will achieve
> exactly the same?

   Because it doesn't achieve exactly the same.  What I want is a
sane, rational way to describe what I'm doing in the core API, so
other programmers learning the language don't spend the amount of time
I did perplexed that there was a .pop() and a .remove() and a
.discard(), but there wasn't a .pick().  I don't want to have to write
the same little helper function in every project to fill a deficiency
in the library.  I don't want to have to argue about the flaws in
solutions with race conditions, or the fact that cheap functions
become expensive functions when performed over the network, or that
there's a real value in having an atomic operation which throws a sane
exception when it fails, or how it's disturbing to my OCD core to have
an API which believes:

  if x in s:
    s.remove(x)

... is too confusing, so there should be a .discard() method, but ...

  for x in s:
    break

... is self-evident and obvious, so there's no need for a .pick().

-- Chris

From dickinsm at gmail.com  Fri Nov  6 09:27:48 2009
From: dickinsm at gmail.com (Mark Dickinson)
Date: Fri, 6 Nov 2009 08:27:48 +0000
Subject: [Python-Dev] Status of the Buildbot fleet and related bugs
In-Reply-To: <Pine.LNX.4.64.0911052209000.12274@kimball.webabinitio.net>
References: <Pine.LNX.4.64.0911052209000.12274@kimball.webabinitio.net>
Message-ID: <5c6f2a5d0911060027x6152a27ao77df7938a079f622@mail.gmail.com>

On Fri, Nov 6, 2009 at 3:53 AM, R. David Murray <rdmurray at bitdance.com> wrote:

> ? ?(1) issue 3864: FreeBSD testing hangs consistently. ?According to the
> ? ? ? ?ticket this is a FreeBSD bug fixed in 6.4, so an OS upgrade
> ? ? ? ?on the buildslave would probably solve it.

I think the particular issue mentioned in 3864  is fixed, in some sense:
test_signal used to hang, but now just plain fails in a reasonable amount of
time (~15 seconds) instead.  So at least the test_signal failure isn't
preventing us from seeing the results of other tests.

The big problem now on the FreeBSD buildbot is that test_multiprocessing
reliably causes the whole test run to abort with 'Signal 12'.  Solving this
may be as simple as just getting someone to install a copy of FreeBSD 6.2
on an ssh-accessible machine so that the source of the error can be tracked
down.

> ? ?(2) issue 4970: consistent signal 32 error on the norwitz-x86 Gentoo
> ? ? ? ?buildslave in 3.1 and 3.x. ?This may be due to the box
> ? ? ? ?running an old threading library, but it does make one wonder what
> ? ? ? ?changed in 3.x that exposed it.

This error has been happening since well before 3.0 was released.  Asking
for access to Neal's machine is probably the only sensible way to diagnose
it. (A less invasive but slower way to debug would be to create a branch
especially for this bug and do repeated runs to figure out which part of test_os
is causing the failure.)

> Another issue that I've seen on the buildbots but that doesn't
> seem to be showing up right now (is it fixed?) is issue 7251, which
> Mark is working on.

It's not fixed, but I hope to have time to fix it this weekend.  It's just
not showing up on some runs because test_multiprocessing kills the
buildslave first.  :-)

> So, overall I think the buildbot fleet is in good shape, and if
> we can nail issue 6748 I think it will be back to being an
> important resource for sanity checking our checkins.

Wholeheartedly agreed!  Sorting out the FreeBSD test_multiprocessing
Signal 12 would be nice, too.  I'll open an issue for this, unless
someone else gets there first.

Mark

From theller at ctypes.org  Fri Nov  6 10:02:24 2009
From: theller at ctypes.org (Thomas Heller)
Date: Fri, 06 Nov 2009 10:02:24 +0100
Subject: [Python-Dev] Status of the Buildbot fleet and related bugs
In-Reply-To: <Pine.LNX.4.64.0911052209000.12274@kimball.webabinitio.net>
References: <Pine.LNX.4.64.0911052209000.12274@kimball.webabinitio.net>
Message-ID: <hd0omr$1gm$1@ger.gmane.org>

R. David Murray schrieb:
> The buildbot waterfall is much greener now.  Thanks to all who have
> contributed to making it so (and it hasn't just been Mark and Antoine
> and I, though we've been the most directly active (and yes, Mark, you
> did contribute several fixes!)).
[...]
> In the 'unstable' builder fleet, Antoine's new builder seems to be
> stable across the board, while mine fails consistently on 3.1 and 3.x
> because of the test_telnetlib bug.  Thomas Heller's OS X buildbot is
> consistently having lots of test failures (the same ones each time
> I've checked).

My buildbot is behind our companys firewall.

Well, I was able to fix the test_smtpnet test by additionally
opening port 465 in the firewall; however I'm not really sure
I should do that.

I had to open another port already although that one is probably
less critical.

For the other test failures, I have no idea where they come from.

-- 
Thanks,
Thomas


From ncoghlan at gmail.com  Fri Nov  6 10:35:46 2009
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Fri, 06 Nov 2009 19:35:46 +1000
Subject: [Python-Dev] PEP 3003 - Python Language Moratorium
In-Reply-To: <bbaeab100911052155m3558186xbcd833bc50c9a59f@mail.gmail.com>
References: <ca471dc20911030935l25251e0diec64427f5d7bcbf2@mail.gmail.com>
	<ca471dc20911051405o287e0f41m26d38860a200b47f@mail.gmail.com>
	<ea2499da0911051453n3569387aj581ad4acf5b799c8@mail.gmail.com>
	<4222a8490911051521r1b9c8165id6e0f12d62da0613@mail.gmail.com>
	<ca471dc20911051526u7f3642ecg19e4eb73bcca1d3a@mail.gmail.com>
	<bbaeab100911051535g68cefa18kdadea33f78de9495@mail.gmail.com>
	<ca471dc20911051923j113d4deame8757c5775f040d2@mail.gmail.com>
	<bbaeab100911052155m3558186xbcd833bc50c9a59f@mail.gmail.com>
Message-ID: <4AF3EDF2.2010406@gmail.com>

Brett Cannon wrote:
> On Thu, Nov 5, 2009 at 19:23, Guido van Rossum <guido at python.org> wrote:
>> On Thu, Nov 5, 2009 at 3:35 PM, Brett Cannon <brett at python.org> wrote:
>>> On Thu, Nov 5, 2009 at 15:26, Guido van Rossum <guido at python.org> wrote:
>>>> I have come to the conclusion that there are better ways to
>>>> pre-announce that a module is going to disappear instead of
>>>> deprecation warnings.
>>> What exactly are those better ways? Document as deprecated only?
>> Sorry, I have an existence proof, but no construction. :-)
>>
>> Ideas welcome. Silent deprecations, loud documentation, biweekly home
>> visits, whatever, as long as it doesn't log a message by default.
> 
> Well, one option is to come up with the equivalent of -3, but for all
> warnings; the antithesis of -W. And obviously glaring deprecation
> warnings in the docs (removal has been discussed but always shot down
> as someone who comes across old code might still need docs for it).

For a PEP 3003-specific solution, we could adopt a policy that we will
add only Pending Deprecation warnings during the moratorium period. The
actually deprecation clock would then only start in the first release
after the moratorium ends (i.e. full Deprecation Warning in that
release, removal in the following release).

No doubt some third party developers would end up being shocked by a
raft of warnings at that point, but those that are paying more attention
would get a chance to clean up the warnings without alarming their users.

Longer term, a solution may be to extend the standard deprecation period
one release and make pending deprecation warnings required rather than
optional. That way, on the ball developers would have a full release to
quash deprecation warnings before their users encountered them by default.

That is:

  Release X.Y: deprecated in docs, pending deprecation in code
  Release X.Y+1: deprecated in code
  Release X.Y+2: removed in code and docs

(Or we could just make that the policy now and not do anything specific
in relation to the moratorium and the standard library)

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------

From metawilm at gmail.com  Fri Nov  6 10:46:00 2009
From: metawilm at gmail.com (Willem Broekema)
Date: Fri, 6 Nov 2009 10:46:00 +0100
Subject: [Python-Dev] PEP 3003 - Python Language Moratorium
In-Reply-To: <200911061618.38790.steve@pearwood.info>
References: <ca471dc20911030935l25251e0diec64427f5d7bcbf2@mail.gmail.com>
	<hctuqq$in0$1@ger.gmane.org> <4AF29753.3040801@voidspace.org.uk>
	<200911061618.38790.steve@pearwood.info>
Message-ID: <f6bc9b490911060146k39b84f4sbe9f0fe41667e83d@mail.gmail.com>

On Fri, Nov 6, 2009 at 6:18 AM, Steven D'Aprano <steve at pearwood.info> wrote:
> I don't know how mature or active it is, so it may not count as either
> major or complete, but there's also CLPython:
>
> http://common-lisp.net/project/clpython/

CLPython is in steady development, quite complete and stable on the
language level (somewhere between 2.5 and 2.6), but missing most
built-in library functionality. (It reuses the pure-Python parts of
the stdlib.)

As its developer, I don't care much about being mentioned in
discussions or presentations, as CLPython is not as widely used as
IronPython and Jython. But in a PEP like this, which is directly about
alternative implementations, it deserves to be mentioned, IMHO.

- Willem

From dirkjan at ochtman.nl  Fri Nov  6 10:47:14 2009
From: dirkjan at ochtman.nl (Dirkjan Ochtman)
Date: Fri, 6 Nov 2009 10:47:14 +0100
Subject: [Python-Dev] PEP 3003 - Python Language Moratorium
In-Reply-To: <4AF3EDF2.2010406@gmail.com>
References: <ca471dc20911030935l25251e0diec64427f5d7bcbf2@mail.gmail.com>
	<ca471dc20911051405o287e0f41m26d38860a200b47f@mail.gmail.com>
	<ea2499da0911051453n3569387aj581ad4acf5b799c8@mail.gmail.com>
	<4222a8490911051521r1b9c8165id6e0f12d62da0613@mail.gmail.com>
	<ca471dc20911051526u7f3642ecg19e4eb73bcca1d3a@mail.gmail.com>
	<bbaeab100911051535g68cefa18kdadea33f78de9495@mail.gmail.com>
	<ca471dc20911051923j113d4deame8757c5775f040d2@mail.gmail.com>
	<bbaeab100911052155m3558186xbcd833bc50c9a59f@mail.gmail.com>
	<4AF3EDF2.2010406@gmail.com>
Message-ID: <ea2499da0911060147n70755574yf27d573e011df7af@mail.gmail.com>

On Fri, Nov 6, 2009 at 10:35, Nick Coghlan <ncoghlan at gmail.com> wrote:
> Longer term, a solution may be to extend the standard deprecation period
> one release and make pending deprecation warnings required rather than
> optional. That way, on the ball developers would have a full release to
> quash deprecation warnings before their users encountered them by default.
>
> That is:
>
> ?Release X.Y: deprecated in docs, pending deprecation in code
> ?Release X.Y+1: deprecated in code
> ?Release X.Y+2: removed in code and docs
>
> (Or we could just make that the policy now and not do anything specific
> in relation to the moratorium and the standard library)

This sounds like an improvement for things like Mercurial, at least.
We did support 2.3-2.6 until relatively recently, and I think that's
hard to get around for software that actually has to work on the
user's box. This is a bit different from webapps, I suspect, where you
"only" have to support the servers, which you often have more control
over.

But supporting 4 releases in a row has been a bit of a PITA. Luckily,
we finally felt it was time to drop 2.3, so now we can make use of
luxuries such as subprocess... Still, supporting 3 releases seems
relatively common in the Python world (after all, parts of Zope still
require 2.4, I think), and so it would be nice if the stdlib moved a
little bit slower.

Cheers,

Dirkjan

From ncoghlan at gmail.com  Fri Nov  6 11:20:24 2009
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Fri, 06 Nov 2009 20:20:24 +1000
Subject: [Python-Dev] PEP 3003 - Python Language Moratorium
In-Reply-To: <ea2499da0911060147n70755574yf27d573e011df7af@mail.gmail.com>
References: <ca471dc20911030935l25251e0diec64427f5d7bcbf2@mail.gmail.com>	
	<ca471dc20911051405o287e0f41m26d38860a200b47f@mail.gmail.com>	
	<ea2499da0911051453n3569387aj581ad4acf5b799c8@mail.gmail.com>	
	<4222a8490911051521r1b9c8165id6e0f12d62da0613@mail.gmail.com>	
	<ca471dc20911051526u7f3642ecg19e4eb73bcca1d3a@mail.gmail.com>	
	<bbaeab100911051535g68cefa18kdadea33f78de9495@mail.gmail.com>	
	<ca471dc20911051923j113d4deame8757c5775f040d2@mail.gmail.com>	
	<bbaeab100911052155m3558186xbcd833bc50c9a59f@mail.gmail.com>	
	<4AF3EDF2.2010406@gmail.com>
	<ea2499da0911060147n70755574yf27d573e011df7af@mail.gmail.com>
Message-ID: <4AF3F868.5040403@gmail.com>

Dirkjan Ochtman wrote:
> On Fri, Nov 6, 2009 at 10:35, Nick Coghlan <ncoghlan at gmail.com> wrote:
>> Longer term, a solution may be to extend the standard deprecation period
>> one release and make pending deprecation warnings required rather than
>> optional. That way, on the ball developers would have a full release to
>> quash deprecation warnings before their users encountered them by default.
>>
>> That is:
>>
>>  Release X.Y: deprecated in docs, pending deprecation in code
>>  Release X.Y+1: deprecated in code
>>  Release X.Y+2: removed in code and docs
>>
>> (Or we could just make that the policy now and not do anything specific
>> in relation to the moratorium and the standard library)
> 
> This sounds like an improvement for things like Mercurial, at least.
> We did support 2.3-2.6 until relatively recently, and I think that's
> hard to get around for software that actually has to work on the
> user's box. This is a bit different from webapps, I suspect, where you
> "only" have to support the servers, which you often have more control
> over.
> 
> But supporting 4 releases in a row has been a bit of a PITA. Luckily,
> we finally felt it was time to drop 2.3, so now we can make use of
> luxuries such as subprocess... Still, supporting 3 releases seems
> relatively common in the Python world (after all, parts of Zope still
> require 2.4, I think), and so it would be nice if the stdlib moved a
> little bit slower.

I would personally be open to the open to the idea of requiring 2
releases worth of Pending Deprecation warnings. Then you would have the
following situation for warnings-free operation on up to 3 versions of
Python:

Release X.Y-2: old approach only
Release X.Y-1: no change
Release X.Y: new approach added, old approach deprecated in docs,
pending deprecation warning in code
Release X.Y+1: no change
Release X.Y+2: deprecation warning in code
Release X.Y+3: old approach removed from docs and code

Libraries and applications supporting up to 3 Python versions
simultaneously would then have a clear path to follow: use the old
mechanism up to Release X.Y+1, then switch to the new mechanism in
Release X.Y+2

We would be looking at around 5 years to completely kill off a feature
at that point, which actually aligns fairly well with the time period
for which we provide source-only security patches for really old branches.

Making such a system practical would probably require some additional
infrastructure in the warnings module to handle the details though.

Specifically, rather than having to update the code to raise the correct
kind of warning each release, it would be better to instead be able to
write something like "warnings.warn_deprecated(msg, (3, 4))" and have it
generate PendingDeprecationWarning messages in 3.2 and 3.3, and then
DeprecationWarning messages in 3.4 and later. In code, something like:

  def warn_deprecated(msg, version, ref=sys.version, stacklevel=2):
    deprecated = version >= ref
    if deprecated:
      category = DeprecationWarning
    else:
      category = PendingDeprecationWarning
    warn(msg, category, stacklevel)
    return deprecated

Would people be interested in such a helper method (along with a
corresponding C API, naturally)?

With the reference version passed in as an optional argument, it would
even be applicable to more than just the Python core and standard library.

Regards,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------

From stefan_ml at behnel.de  Fri Nov  6 11:42:43 2009
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Fri, 06 Nov 2009 11:42:43 +0100
Subject: [Python-Dev] PEP 3003 - Python Language Moratorium
In-Reply-To: <AB6938AB50B747EEB35257E266F810A5@RaymondLaptop1>
References: <ca471dc20911030935l25251e0diec64427f5d7bcbf2@mail.gmail.com>	<ca471dc20911051405o287e0f41m26d38860a200b47f@mail.gmail.com>
	<AB6938AB50B747EEB35257E266F810A5@RaymondLaptop1>
Message-ID: <hd0uj3$ijq$1@ger.gmane.org>

Raymond Hettinger, 05.11.2009 23:11:
> [GvR]
>> I haven't seen substantial opposition against the PEP -- in fact I
>> can't recall any, and many people have explicitly posted in support of
>> it. So unless opposition suddenly appears in the next few days, I'll
>> move it to the Accepted state next Monday.
> 
> But it would have been so much fun to have a never ending python-ideas
> thread on BEGIN/END blocks ;-)

According to the PEP, you can still have that. It just won't lead to an
implementation being merged (or at least becoming part of a release) within
the next two years.

Stefan


From db3l.net at gmail.com  Fri Nov  6 11:54:51 2009
From: db3l.net at gmail.com (David Bolen)
Date: Fri, 06 Nov 2009 05:54:51 -0500
Subject: [Python-Dev] Status of the Buildbot fleet and related bugs
References: <Pine.LNX.4.64.0911052209000.12274@kimball.webabinitio.net>
	<5c6f2a5d0911060027x6152a27ao77df7938a079f622@mail.gmail.com>
Message-ID: <m24op7apok.fsf@valheru.db3l.homeip.net>

Mark Dickinson <dickinsm at gmail.com> writes:

> On Fri, Nov 6, 2009 at 3:53 AM, R. David Murray <rdmurray at bitdance.com> wrote:
>
>> ? ?(1) issue 3864: FreeBSD testing hangs consistently. ?According to the
>> ? ? ? ?ticket this is a FreeBSD bug fixed in 6.4, so an OS upgrade
>> ? ? ? ?on the buildslave would probably solve it.
>
> I think the particular issue mentioned in 3864  is fixed, in some sense:
> test_signal used to hang, but now just plain fails in a reasonable amount of
> time (~15 seconds) instead.  So at least the test_signal failure isn't
> preventing us from seeing the results of other tests.
>
> The big problem now on the FreeBSD buildbot is that test_multiprocessing
> reliably causes the whole test run to abort with 'Signal 12'.  Solving this
> may be as simple as just getting someone to install a copy of FreeBSD 6.2
> on an ssh-accessible machine so that the source of the error can be tracked
> down.

I could arrange ssh access to the build slave if that would help
anyone who wants to look into that.  Just contact me directly.

In terms of the overall release, I'm also fine with upgrading the
build slave to 6.4.  Or I could jump up to 7.2 instead.  When I first
brought the build slave up, 7.x wasn't finalized yet - not sure now
which release is more prevalent in use.

-- David


From hfuerstenau at gmx.net  Fri Nov  6 08:25:29 2009
From: hfuerstenau at gmx.net (=?ISO-8859-1?Q?Hagen_F=FCrstenau?=)
Date: Fri, 06 Nov 2009 08:25:29 +0100
Subject: [Python-Dev] PEP 3003 - Python Language Moratorium
In-Reply-To: <20091106001430.3229.796306500.divmod.xquotient.180@localhost.localdomain>
References: <ca471dc20911030935l25251e0diec64427f5d7bcbf2@mail.gmail.com>	<ca471dc20911051405o287e0f41m26d38860a200b47f@mail.gmail.com>	<ea2499da0911051453n3569387aj581ad4acf5b799c8@mail.gmail.com>	<4222a8490911051521r1b9c8165id6e0f12d62da0613@mail.gmail.com>	<ca471dc20911051526u7f3642ecg19e4eb73bcca1d3a@mail.gmail.com>	<bbaeab100911051535g68cefa18kdadea33f78de9495@mail.gmail.com>	<48c184130911051555r9a5b78cs3a13cb1345d3cbf7@mail.gmail.com>
	<20091106001430.3229.796306500.divmod.xquotient.180@localhost.localdomain>
Message-ID: <4AF3CF69.6090709@gmx.net>

>> A switch to enable deprecation warnings  would give developers a
>> chance to see them when migrating to a new version of python.  And it
>> would prevent users from having to deal with them.
> 
> PendingDeprecationWarning is silent by default and requires a switch to
> be enabled.

So what we need is perhaps not a stdlib moratorium, but rather strict
adherence to a sequence of PendingDeprecationWarning ->
DeprecationWarning -> Removal? (The first one seems to have been
underused so far.)

That way nothing could get removed without having had at least two
warnings in subsequent minor versions, one silent and one loud. People
who want to avoid loud warnings can check for silent ones before
upgrading, and everyone is guaranteed to have seen a warning before
final removal.

- Hagen

From exarkun at twistedmatrix.com  Fri Nov  6 15:04:12 2009
From: exarkun at twistedmatrix.com (exarkun at twistedmatrix.com)
Date: Fri, 06 Nov 2009 14:04:12 -0000
Subject: [Python-Dev] PEP 3003 - Python Language Moratorium
In-Reply-To: <20091106135058.GA13578@vidar.dreamhost.com>
References: <ca471dc20911030935l25251e0diec64427f5d7bcbf2@mail.gmail.com>
	<ca471dc20911051405o287e0f41m26d38860a200b47f@mail.gmail.com>
	<ea2499da0911051453n3569387aj581ad4acf5b799c8@mail.gmail.com>
	<4222a8490911051521r1b9c8165id6e0f12d62da0613@mail.gmail.com>
	<ca471dc20911051526u7f3642ecg19e4eb73bcca1d3a@mail.gmail.com>
	<bbaeab100911051535g68cefa18kdadea33f78de9495@mail.gmail.com>
	<48c184130911051555r9a5b78cs3a13cb1345d3cbf7@mail.gmail.com>
	<20091106001430.3229.796306500.divmod.xquotient.180@localhost.localdomain>
	<4AF3CF69.6090709@gmx.net>
	<20091106135058.GA13578@vidar.dreamhost.com>
Message-ID: <20091106140412.3229.80002680.divmod.xquotient.193@localhost.localdomain>

On 01:50 pm, dripton at ripton.net wrote:
>On 2009.11.06 08:25:29 +0100, Hagen F?rstenau wrote:
>> >> A switch to enable deprecation warnings  would give developers a
>> >> chance to see them when migrating to a new version of python.  And 
>>it
>> >> would prevent users from having to deal with them.
>> >
>> > PendingDeprecationWarning is silent by default and requires a switch 
>>to
>> > be enabled.
>>
>>So what we need is perhaps not a stdlib moratorium, but rather strict
>>adherence to a sequence of PendingDeprecationWarning ->
>>DeprecationWarning -> Removal? (The first one seems to have been
>>underused so far.)
>
>+1
>
>I really don't like the loud warnings for sha and md5 in 2.6,
>considering that hashlib wasn't even added until 2.5.  It should be
>possible to write code that works across all the versions of Python 2
>that are currently shipping with mainstream Linux distrobutions without
>deprecation warnings or conditional version-dependent code.
>
>To go a step further, maybe follow javac's example and emit just one
>summary deprecation warning per program by default.  Something
>like "this program uses deprecated API calls.  Run it with
>python -deprecation to see the details."

End users don't even need to be told this much, though (and if it's 
javac that does this for Java, then indeed Java end users probably 
aren't seeing this either).  I think it would be great for deprecation 
warnings to be completely silent for a while *and* for the policy to be 
prominently documented so that developers, the people who really need 
this information, know how to get it.

Jean-Paul

From jnoller at gmail.com  Fri Nov  6 15:05:48 2009
From: jnoller at gmail.com (Jesse Noller)
Date: Fri, 6 Nov 2009 09:05:48 -0500
Subject: [Python-Dev] Status of the Buildbot fleet and related bugs
In-Reply-To: <5c6f2a5d0911060027x6152a27ao77df7938a079f622@mail.gmail.com>
References: <Pine.LNX.4.64.0911052209000.12274@kimball.webabinitio.net>
	<5c6f2a5d0911060027x6152a27ao77df7938a079f622@mail.gmail.com>
Message-ID: <4222a8490911060605k17bcd575if927dc6317dabfb0@mail.gmail.com>

On Fri, Nov 6, 2009 at 3:27 AM, Mark Dickinson <dickinsm at gmail.com> wrote:
> On Fri, Nov 6, 2009 at 3:53 AM, R. David Murray <rdmurray at bitdance.com> wrote:
>
>> ? ?(1) issue 3864: FreeBSD testing hangs consistently. ?According to the
>> ? ? ? ?ticket this is a FreeBSD bug fixed in 6.4, so an OS upgrade
>> ? ? ? ?on the buildslave would probably solve it.
>
> I think the particular issue mentioned in 3864 ?is fixed, in some sense:
> test_signal used to hang, but now just plain fails in a reasonable amount of
> time (~15 seconds) instead. ?So at least the test_signal failure isn't
> preventing us from seeing the results of other tests.
>
> The big problem now on the FreeBSD buildbot is that test_multiprocessing
> reliably causes the whole test run to abort with 'Signal 12'. ?Solving this
> may be as simple as just getting someone to install a copy of FreeBSD 6.2
> on an ssh-accessible machine so that the source of the error can be tracked
> down.

Sorry I haven't been watching the bots; the signal 12 is new, I don't
have direct access to a fbsd box to poke at it. I'm fairly alarmed
that it's triggering that. Let me know what I can do to help - fbsd
support of certain lower level shared semaphore stuff has been spotty
up until the most recent versions.

> Wholeheartedly agreed! ?Sorting out the FreeBSD test_multiprocessing
> Signal 12 would be nice, too. ?I'll open an issue for this, unless
> someone else gets there first.

If/when you do, please add me on the noisy list. I remember there
being some patche(s) pending for FBSD around improved shared semaphore
support, but I can't find the bug report right now.

jesse

From daniel at stutzbachenterprises.com  Fri Nov  6 15:17:59 2009
From: daniel at stutzbachenterprises.com (Daniel Stutzbach)
Date: Fri, 6 Nov 2009 08:17:59 -0600
Subject: [Python-Dev] Retrieve an arbitrary element from
	asetwithoutremoving it
In-Reply-To: <97069B15530D4C66AC1D4BCCB70E2FD5@RaymondLaptop1>
References: <7b0045ed0910301843y7059d316s8a851142b78b7aeb@mail.gmail.com>
	<4AF0D0D7.3090403@canterbury.ac.nz>
	<200911042234.36796.steve@pearwood.info>
	<f3cc57c60911041332v479cc3b0t19d9601e52002128@mail.gmail.com>
	<99BB308202224D31AA5300223DB59912@RaymondLaptop1>
	<7b0045ed0911051243q18f8324ak9abaa98ac26948e9@mail.gmail.com>
	<4AF341BD.2010203@v.loewis.de>
	<7b0045ed0911051419i76f4221dye8ecccc424179ec6@mail.gmail.com>
	<BD050FFAAA19443DA0DA34FD2BD740CF@RaymondLaptop1>
	<97069B15530D4C66AC1D4BCCB70E2FD5@RaymondLaptop1>
Message-ID: <eae285400911060617s15d8f0bpaa2c3c4d6d946d41@mail.gmail.com>

On Thu, Nov 5, 2009 at 5:02 PM, Raymond Hettinger <python at rcn.com> wrote:

> def first(iterable):
>   'Return the first value from a container or iterable.  If empty, raises
> LookupError'
>   for value in iterable:
>       return value
>   raise LookupError('no first value; iterable is empty')
>

(This email should not be construed to mean that I support adding a .pick
method; I do not.)

I realized this morning that the "for i in s: break" idiom can be O(n), even
assuming no hash collisions.  Observe this subtly O(n**2) algorithm:

Cashew:~$ cat > test.py
while s:
    for i in s:
        break
    # Imagine a complex algorithm here that depends on i still being in s
    s.remove(i)

Cashew:~$ python2.5 /usr/lib/python2.5/timeit.py -s 's=set(range(20000))'
"`cat test.py`"
10 loops, best of 3: 31.2 msec per loop
Cashew:~$ python2.5 /usr/lib/python2.5/timeit.py -s 's=set(range(40000))'
"`cat test.py`"
10 loops, best of 3: 124 msec per loop
Cashew:~$ python2.5 /usr/lib/python2.5/timeit.py -s 's=set(range(80000))'
"`cat test.py`"
10 loops, best of 3: 491 msec per loop
Cashew:~$ python2.5 /usr/lib/python2.5/timeit.py -s 's=set(range(160000))'
"`cat test.py`"
10 loops, best of 3: 1.96 sec per loop

Doubling n quadruples the run-time, i.e., O(n**2) behavior.

I wonder if those clamoring for a pick() method and complaining about
efficiency are inadvertently running into this?

The problem arises because we're systematically unbalancing the hash table.
The iterator returns the first valid entry in the hash table, which we
remove.  Repeat several times and pretty soon the iterator has to spend O(n)
time scanning through empty entries to find the first remaining valid entry.

On the other hand, the pop/add idiom is O(1), since the .pop implementation
contains some voodoo to remember where it left off.  Consequently, the
following algorithm is O(n) instead of O(n**2):

Cashew:~$ cat > test.py
while s:
    i = s.pop()
    s.add(i)
    # Imagine a complex algorithm here that depends on i still being in s
    s.remove(i)

Cashew:~$ python2.5 /usr/lib/python2.5/timeit.py -s 's=set(range(400000))'
"`cat test.py`"
10 loops, best of 3: 28 msec per loop
Cashew:~$ python2.5 /usr/lib/python2.5/timeit.py -s 's=set(range(800000))'
"`cat test.py`"
10 loops, best of 3: 56.2 msec per loop
Cashew:~$ python2.5 /usr/lib/python2.5/timeit.py -s 's=set(range(1600000))'
"`cat test.py`"
10 loops, best of 3: 113 msec per loop
Cashew:~$ python2.5 /usr/lib/python2.5/timeit.py -s 's=set(range(3200000))'
"`cat test.py`"
10 loops, best of 3: 227 msec per loop

Doubling n doubles the run-time, i.e., O(n) behavior.

--
Daniel Stutzbach, Ph.D.
President, Stutzbach Enterprises, LLC <http://stutzbachenterprises.com>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20091106/0a03f761/attachment.htm>

From dickinsm at gmail.com  Fri Nov  6 16:40:10 2009
From: dickinsm at gmail.com (Mark Dickinson)
Date: Fri, 6 Nov 2009 15:40:10 +0000
Subject: [Python-Dev] Status of the Buildbot fleet and related bugs
In-Reply-To: <m24op7apok.fsf@valheru.db3l.homeip.net>
References: <Pine.LNX.4.64.0911052209000.12274@kimball.webabinitio.net>
	<5c6f2a5d0911060027x6152a27ao77df7938a079f622@mail.gmail.com>
	<m24op7apok.fsf@valheru.db3l.homeip.net>
Message-ID: <5c6f2a5d0911060740g3178b2d0p48e8ef483d6e9c7d@mail.gmail.com>

On Fri, Nov 6, 2009 at 10:54 AM, David Bolen <db3l.net at gmail.com> wrote:
> Mark Dickinson <dickinsm at gmail.com> writes:
>> The big problem now on the FreeBSD buildbot is that test_multiprocessing
>> reliably causes the whole test run to abort with 'Signal 12'. ?Solving this
>> may be as simple as just getting someone to install a copy of FreeBSD 6.2
>> on an ssh-accessible machine so that the source of the error can be tracked
>> down.
>
> I could arrange ssh access to the build slave if that would help
> anyone who wants to look into that.

I suspect it would help a lot!  Thanks for the offer.

Jesse would likely be able to pin down the cause of the failure
faster than I would, but I don't know how many cycles he has
available.

> Just contact me directly.

I'll do that, if/when I find time.

> In terms of the overall release, I'm also fine with upgrading the
> build slave to 6.4. ?Or I could jump up to 7.2 instead. ?When I first
> brought the build slave up, 7.x wasn't finalized yet - not sure now
> which release is more prevalent in use.

Not sure either, but it's certainly useful to be able to test on
FreeBSD 6.x.  Ideally, we'd have buildslaves for both...

Mark

From guido at python.org  Fri Nov  6 17:02:47 2009
From: guido at python.org (Guido van Rossum)
Date: Fri, 6 Nov 2009 08:02:47 -0800
Subject: [Python-Dev] PEP 3003 - Python Language Moratorium
In-Reply-To: <bbaeab100911052155m3558186xbcd833bc50c9a59f@mail.gmail.com>
References: <ca471dc20911030935l25251e0diec64427f5d7bcbf2@mail.gmail.com> 
	<ca471dc20911051405o287e0f41m26d38860a200b47f@mail.gmail.com> 
	<ea2499da0911051453n3569387aj581ad4acf5b799c8@mail.gmail.com> 
	<4222a8490911051521r1b9c8165id6e0f12d62da0613@mail.gmail.com> 
	<ca471dc20911051526u7f3642ecg19e4eb73bcca1d3a@mail.gmail.com> 
	<bbaeab100911051535g68cefa18kdadea33f78de9495@mail.gmail.com> 
	<ca471dc20911051923j113d4deame8757c5775f040d2@mail.gmail.com> 
	<bbaeab100911052155m3558186xbcd833bc50c9a59f@mail.gmail.com>
Message-ID: <ca471dc20911060802v65c2102dt1d8b47440a993045@mail.gmail.com>

On Thu, Nov 5, 2009 at 9:55 PM, Brett Cannon <brett at python.org> wrote:
> The clarification I need is will this in any way influence when
> modules are removed. If they stay in for the life of a major version
> then I want it made clear that bug fixes for the code take lower
> priority over all other code in the standard library.

I think we should be as cautious as ever with removing modules. We've
had our chance for clean-up without abandon with Py3k; now we should
stick with our commitment to backwards compatibility. In fact, we
should probably be *more* conservative than we already were given that
the amount of code written in Python is always increasing and hence
the cumulative pain caused by incompatible changes will increase too.

I'm fine with silent deprecations or requiring a flag to turn on
deprecation warnings (like Java does).

We're not yet at the point where C is, but who wouldn't be next to C
on the TIOBE index? :-)

-- 
--Guido van Rossum (python.org/~guido)

From ubershmekel at gmail.com  Fri Nov  6 17:25:31 2009
From: ubershmekel at gmail.com (Yuvgoog Greenle)
Date: Fri, 6 Nov 2009 18:25:31 +0200
Subject: [Python-Dev] PEP 3003 - Python Language Moratorium
In-Reply-To: <ca471dc20911060802v65c2102dt1d8b47440a993045@mail.gmail.com>
References: <ca471dc20911030935l25251e0diec64427f5d7bcbf2@mail.gmail.com>
	<ca471dc20911051405o287e0f41m26d38860a200b47f@mail.gmail.com>
	<ea2499da0911051453n3569387aj581ad4acf5b799c8@mail.gmail.com>
	<4222a8490911051521r1b9c8165id6e0f12d62da0613@mail.gmail.com>
	<ca471dc20911051526u7f3642ecg19e4eb73bcca1d3a@mail.gmail.com>
	<bbaeab100911051535g68cefa18kdadea33f78de9495@mail.gmail.com>
	<ca471dc20911051923j113d4deame8757c5775f040d2@mail.gmail.com>
	<bbaeab100911052155m3558186xbcd833bc50c9a59f@mail.gmail.com>
	<ca471dc20911060802v65c2102dt1d8b47440a993045@mail.gmail.com>
Message-ID: <9d153b7c0911060825n2ab937catbd576a058e8b5543@mail.gmail.com>

On Fri, Nov 6, 2009 at 6:02 PM, Guido van Rossum <guido at python.org> wrote:

> I'm fine with silent deprecations or requiring a flag to turn on
> deprecation warnings (like Java does).
>
>
It's easier for java because they have compile time for the deprecation
warnings.

Maybe we should somehow silence all warnings when running eggs?

--yuv
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20091106/d16406c9/attachment.htm>

From debatem1 at gmail.com  Fri Nov  6 17:25:59 2009
From: debatem1 at gmail.com (geremy condra)
Date: Fri, 6 Nov 2009 11:25:59 -0500
Subject: [Python-Dev] PEP 3003 - Python Language Moratorium
In-Reply-To: <ca471dc20911060802v65c2102dt1d8b47440a993045@mail.gmail.com>
References: <ca471dc20911030935l25251e0diec64427f5d7bcbf2@mail.gmail.com>
	<ca471dc20911051405o287e0f41m26d38860a200b47f@mail.gmail.com>
	<ea2499da0911051453n3569387aj581ad4acf5b799c8@mail.gmail.com>
	<4222a8490911051521r1b9c8165id6e0f12d62da0613@mail.gmail.com>
	<ca471dc20911051526u7f3642ecg19e4eb73bcca1d3a@mail.gmail.com>
	<bbaeab100911051535g68cefa18kdadea33f78de9495@mail.gmail.com>
	<ca471dc20911051923j113d4deame8757c5775f040d2@mail.gmail.com>
	<bbaeab100911052155m3558186xbcd833bc50c9a59f@mail.gmail.com>
	<ca471dc20911060802v65c2102dt1d8b47440a993045@mail.gmail.com>
Message-ID: <f3cc57c60911060825j608d7bf9q66cc77bd2828252@mail.gmail.com>

On Fri, Nov 6, 2009 at 11:02 AM, Guido van Rossum <guido at python.org> wrote:
> On Thu, Nov 5, 2009 at 9:55 PM, Brett Cannon <brett at python.org> wrote:
>> The clarification I need is will this in any way influence when
>> modules are removed. If they stay in for the life of a major version
>> then I want it made clear that bug fixes for the code take lower
>> priority over all other code in the standard library.
>
> I think we should be as cautious as ever with removing modules. We've
> had our chance for clean-up without abandon with Py3k; now we should
> stick with our commitment to backwards compatibility. In fact, we
> should probably be *more* conservative than we already were given that
> the amount of code written in Python is always increasing and hence
> the cumulative pain caused by incompatible changes will increase too.
>
> I'm fine with silent deprecations or requiring a flag to turn on
> deprecation warnings (like Java does).
>
> We're not yet at the point where C is, but who wouldn't be next to C
> on the TIOBE index? :-)

I'd take being next to lisp if it meant that we didn't have to become
any more like C ;).

And speaking of TIOBE, my impression- and apparently yours- was
that Python was on its way up, but TIOBE lists us as being down
from a little over a year ago. Anybody know anything about their
methodology?

Geremy Condra

From status at bugs.python.org  Fri Nov  6 18:07:13 2009
From: status at bugs.python.org (Python tracker)
Date: Fri,  6 Nov 2009 18:07:13 +0100 (CET)
Subject: [Python-Dev] Summary of Python tracker Issues
Message-ID: <20091106170713.E492C782A4@psf.upfronthosting.co.za>


ACTIVITY SUMMARY (10/30/09 - 11/06/09)
Python tracker at http://bugs.python.org/

To view or respond to any of the issues listed below, click on the issue 
number.  Do NOT respond to this message.


 2479 open (+15) / 16616 closed (+15) / 19095 total (+30)

Open issues with patches:   991

Average duration of open issues: 683 days.
Median duration of open issues: 437 days.

Open Issues Breakdown
   open  2444 (+15)
pending    34 ( +0)

Issues Created Or Reopened (31)
_______________________________

mac binary download link for 2.6.4 broken                        10/30/09
CLOSED http://bugs.python.org/issue7243    reopened beard                         
                                                                               

itertools.zip_longest behaves strangely with an iterable class   10/31/09
CLOSED http://bugs.python.org/issue7244    created  durban                        
       patch                                                                   

better Ctrl-C support in pdb (program can be resumed)            10/31/09
       http://bugs.python.org/issue7245    created  isandler                      
                                                                               

getpass raises IOError when several returns are in stdin before  11/01/09
CLOSED http://bugs.python.org/issue7246    created  avihu                         
                                                                               

test_fcntl_64_bit from test_fcntl.py fails in Python 2.6.4       11/01/09
       http://bugs.python.org/issue7247    created  Chainsaw                      
                                                                               

test_importlib uses a fixed name in /tmp                         11/01/09
CLOSED http://bugs.python.org/issue7248    created  pitrou                        
                                                                               

Consider allowing io.BytesIO sizes to be passed as 'long' in 2.6 11/02/09
CLOSED http://bugs.python.org/issue7249    created  ryles                         
                                                                               

wsgiref.handlers.CGIHandler caches os.environ, leaking info betw 11/02/09
       http://bugs.python.org/issue7250    created  snprbob86                     
       patch, easy, 26backport                                                 

Mark expected failures of test_math, test_cmath and test_round a 11/02/09
       http://bugs.python.org/issue7251    created  mark.dickinson                
       buildbot                                                                

list().index() should provide better error reporting             11/02/09
CLOSED http://bugs.python.org/issue7252    created  djc                           
                                                                               

AssertionError Tree is insane with *args and reduce              11/02/09
CLOSED http://bugs.python.org/issue7253    created  jaraco                        
                                                                               

Class members not properly initialized if declared outside of fu 11/02/09
CLOSED http://bugs.python.org/issue7254    created  vladc6                        
                                                                               

"Default" word boundaries for Unicode data?                      11/03/09
       http://bugs.python.org/issue7255    created  RegEx4All                     
                                                                               

parse_qs and parse_qsl in urlparse are not documented as New in  11/03/09
CLOSED http://bugs.python.org/issue7256    created  VeXocide                      
                                                                               

Improve documentation of list.sort and sorted()                  11/03/09
       http://bugs.python.org/issue7257    created  olau                          
                                                                               

test_multiprocessing leaves a temporary directory                11/04/09
CLOSED http://bugs.python.org/issue7258    created  pitrou                        
                                                                               

Documentation update for operator module                         11/04/09
CLOSED http://bugs.python.org/issue7259    created  gagenellina                   
       patch                                                                   

SyntaxError with a not-existing offset for unicode code          11/04/09
CLOSED http://bugs.python.org/issue7260    created  noam                          
                                                                               

Document 2.x -> 3.x round changes in "What's New" documents.     11/04/09
       http://bugs.python.org/issue7261    created  mark.dickinson                
                                                                               

codecs.open() + eol (windows)                                    11/04/09
       http://bugs.python.org/issue7262    created  shamilbi                      
                                                                               

Fix set.intersection docstring                                   11/04/09
       http://bugs.python.org/issue7263    created  belopolsky                    
       patch                                                                   

test_threading_local sometimes hangs when run with -R            11/04/09
CLOSED http://bugs.python.org/issue7264    created  pitrou                        
       patch                                                                   

IDLE not working                                                 11/05/09
CLOSED http://bugs.python.org/issue7265    created  Carey                         
                                                                               

test_lib2to3 failure under Windows                               11/05/09
       http://bugs.python.org/issue7266    created  pitrou                        
       easy                                                                    

format method: c presentation type broken                        11/05/09
       http://bugs.python.org/issue7267    created  doerwalter                    
                                                                               

2to3 execfile conversion changes behavior                        11/06/09
CLOSED http://bugs.python.org/issue7268    created  flashk                        
                                                                               

Windows buildbot occasional DBFileExistsError failures in test_b 11/06/09
       http://bugs.python.org/issue7269    created  r.david.murray                
       buildbot                                                                

Tests for synchronization primitives                             11/06/09
       http://bugs.python.org/issue7270    created  pitrou                        
       patch                                                                   

Typo in turtle module documentation.                             11/06/09
       http://bugs.python.org/issue7271    created  mark.dickinson                
                                                                               

test_multiprocessing fails consistently with 'signal 12' on Free 11/06/09
       http://bugs.python.org/issue7272    created  mark.dickinson                
       buildbot                                                                

struct.pack adding extra '\x00' character in very specific case  11/06/09
CLOSED http://bugs.python.org/issue7273    created  cmadrigal                     
                                                                               



Issues Now Closed (37)
______________________

Minor Change For Better cross compile                             788 days
       http://bugs.python.org/issue1115    gregory.p.smith               
       patch                                                                   

IDLE fails to launch                                              669 days
       http://bugs.python.org/issue1743    richjtd                       
                                                                               

IDLE won't start in custom directory.                             414 days
       http://bugs.python.org/issue3881    srid                          
       patch                                                                   

Add "key" argument to "bisect" module functions                   348 days
       http://bugs.python.org/issue4356    rhettinger                    
       patch                                                                   

xml.etree does not escape CR, LF and TAB characters within attri  110 days
       http://bugs.python.org/issue6492    ezio.melotti                  
                                                                               

Compilation error if configuref --with-computed-gotos              93 days
       http://bugs.python.org/issue6603    mark.dickinson                
       patch, easy                                                             

fnmatch fails on filenames containing \n character                 86 days
       http://bugs.python.org/issue6665    gregory.p.smith               
                                                                               

Intermittent failures in test_mailbox                              51 days
       http://bugs.python.org/issue6896    pitrou                        
       patch                                                                   

test_signal fails on os x 10.6                                     29 days
       http://bugs.python.org/issue7042    mark.dickinson                
       patch                                                                   

SysLogHandler can't handle Unicode                                 29 days
       http://bugs.python.org/issue7077    vinay.sajip                   
                                                                               

Value error 'path is on drive c: start on drive d:' in os.path.r    8 days
       http://bugs.python.org/issue7195    ncoghlan                      
                                                                               

Getpass echo's password to screen on 2.6, but not on 2.5 or 3.1     6 days
       http://bugs.python.org/issue7208    gregory.p.smith               
       patch                                                                   

Proposed Syntax Checks in Test Suite                                5 days
       http://bugs.python.org/issue7210    ChuckRhode                    
       patch                                                                   

select module - kevent ident field 64 bit issue                     9 days
       http://bugs.python.org/issue7211    pitrou                        
       patch                                                                   

IDLE Subprocess Startup Error {Solved}                              8 days
       http://bugs.python.org/issue7217    amaury.forgeotdarc            
                                                                               

thread reaping is imperfect                                         3 days
       http://bugs.python.org/issue7222    pitrou                        
       patch                                                                   

One obvious way to do interning                                     9 days
       http://bugs.python.org/issue7224    rhettinger                    
       patch                                                                   

test_hotshot fails on solaris                                       2 days
       http://bugs.python.org/issue7230    pitrou                        
                                                                               

subprocess.Popen.stdout.flush fails os OS-X 10.6.1                  5 days
       http://bugs.python.org/issue7240    pjenvey                       
                                                                               

mac binary download link for 2.6.4 broken                           0 days
       http://bugs.python.org/issue7243    beard                         
                                                                               

itertools.zip_longest behaves strangely with an iterable class      1 days
       http://bugs.python.org/issue7244    rhettinger                    
       patch                                                                   

getpass raises IOError when several returns are in stdin before     1 days
       http://bugs.python.org/issue7246    avihu                         
                                                                               

test_importlib uses a fixed name in /tmp                            3 days
       http://bugs.python.org/issue7248    brett.cannon                  
                                                                               

Consider allowing io.BytesIO sizes to be passed as 'long' in 2.6    1 days
       http://bugs.python.org/issue7249    pitrou                        
                                                                               

list().index() should provide better error reporting                0 days
       http://bugs.python.org/issue7252    benjamin.peterson             
                                                                               

AssertionError Tree is insane with *args and reduce                 0 days
       http://bugs.python.org/issue7253    benjamin.peterson             
                                                                               

Class members not properly initialized if declared outside of fu    0 days
       http://bugs.python.org/issue7254    r.david.murray                
                                                                               

parse_qs and parse_qsl in urlparse are not documented as New in     0 days
       http://bugs.python.org/issue7256    georg.brandl                  
                                                                               

test_multiprocessing leaves a temporary directory                   0 days
       http://bugs.python.org/issue7258    pitrou                        
                                                                               

Documentation update for operator module                            0 days
       http://bugs.python.org/issue7259    georg.brandl                  
       patch                                                                   

SyntaxError with a not-existing offset for unicode code             0 days
       http://bugs.python.org/issue7260    ezio.melotti                  
                                                                               

test_threading_local sometimes hangs when run with -R               1 days
       http://bugs.python.org/issue7264    pitrou                        
       patch                                                                   

IDLE not working                                                    1 days
       http://bugs.python.org/issue7265    loewis                        
                                                                               

2to3 execfile conversion changes behavior                           0 days
       http://bugs.python.org/issue7268    flashk                        
                                                                               

struct.pack adding extra '\x00' character in very specific case     0 days
       http://bugs.python.org/issue7273    mark.dickinson                
                                                                               

socket.inet_aton returns 8 bytes on LP64 platforms                265 days
       http://bugs.python.org/issue1008086 gregory.p.smith               
       64bit, easy                                                             

Investigated ref leak report related to thread regrtest.py        866 days
       http://bugs.python.org/issue1739118 ocean-city                    
       patch                                                                   



Top Issues Most Discussed (10)
______________________________

 14 itertools.zip_longest behaves strangely with an iterable class     1 days
closed  http://bugs.python.org/issue7244   

 14 64 bit python fails on Windows 7                                  12 days
open    http://bugs.python.org/issue7206   

 14 Python 2.5 64 bit compile fails on Solaris 10/gcc 4.1.1         1036 days
open    http://bugs.python.org/issue1628484

 11 2to3 execfile conversion changes behavior                          0 days
closed  http://bugs.python.org/issue7268   

 11 test_debuglevel from test_telnetlib.py fails                      78 days
open    http://bugs.python.org/issue6748   

 10 Intermittent failures in test_mailbox                             51 days
closed  http://bugs.python.org/issue6896   

  9 cross compile patch                                             1915 days
open    http://bugs.python.org/issue1006238

  8 One obvious way to do interning                                    9 days
closed  http://bugs.python.org/issue7224   

  7 wsgiref.handlers.CGIHandler caches os.environ, leaking info bet    4 days
open    http://bugs.python.org/issue7250   

  7 Documentation add note about SystemRandom                         30 days
open    http://bugs.python.org/issue7076   




From brett at python.org  Fri Nov  6 18:37:59 2009
From: brett at python.org (Brett Cannon)
Date: Fri, 6 Nov 2009 09:37:59 -0800
Subject: [Python-Dev] PEP 3003 - Python Language Moratorium
In-Reply-To: <ca471dc20911060802v65c2102dt1d8b47440a993045@mail.gmail.com>
References: <ca471dc20911030935l25251e0diec64427f5d7bcbf2@mail.gmail.com> 
	<ca471dc20911051405o287e0f41m26d38860a200b47f@mail.gmail.com> 
	<ea2499da0911051453n3569387aj581ad4acf5b799c8@mail.gmail.com> 
	<4222a8490911051521r1b9c8165id6e0f12d62da0613@mail.gmail.com> 
	<ca471dc20911051526u7f3642ecg19e4eb73bcca1d3a@mail.gmail.com> 
	<bbaeab100911051535g68cefa18kdadea33f78de9495@mail.gmail.com> 
	<ca471dc20911051923j113d4deame8757c5775f040d2@mail.gmail.com> 
	<bbaeab100911052155m3558186xbcd833bc50c9a59f@mail.gmail.com> 
	<ca471dc20911060802v65c2102dt1d8b47440a993045@mail.gmail.com>
Message-ID: <bbaeab100911060937q33f0d1b8rdf5ad80bc6784a0@mail.gmail.com>

On Fri, Nov 6, 2009 at 08:02, Guido van Rossum <guido at python.org> wrote:
> On Thu, Nov 5, 2009 at 9:55 PM, Brett Cannon <brett at python.org> wrote:
>> The clarification I need is will this in any way influence when
>> modules are removed. If they stay in for the life of a major version
>> then I want it made clear that bug fixes for the code take lower
>> priority over all other code in the standard library.
>
> I think we should be as cautious as ever with removing modules. We've
> had our chance for clean-up without abandon with Py3k; now we should
> stick with our commitment to backwards compatibility. In fact, we
> should probably be *more* conservative than we already were given that
> the amount of code written in Python is always increasing and hence
> the cumulative pain caused by incompatible changes will increase too.
>
> I'm fine with silent deprecations or requiring a flag to turn on
> deprecation warnings (like Java does).
>

We can discuss this on the stdlib-sig and come back with a proposal on
how to update PEP 4 with an explicit policy on how to handle
deprecations (either Nick's proposal or some flag, and then how long
to let the deprecation last, i.e. three versions or the life of a
major version of Python).

> We're not yet at the point where C is, but who wouldn't be next to C
> on the TIOBE index? :-)
>
> --
> --Guido van Rossum (python.org/~guido)
>

From stechert at gmail.com  Fri Nov  6 20:31:09 2009
From: stechert at gmail.com (Andre Stechert)
Date: Fri, 6 Nov 2009 11:31:09 -0800
Subject: [Python-Dev] PEP 3003 - Python Language Moratorium
In-Reply-To: <f3cc57c60911060825j608d7bf9q66cc77bd2828252@mail.gmail.com>
References: <ca471dc20911030935l25251e0diec64427f5d7bcbf2@mail.gmail.com> 
	<ca471dc20911051405o287e0f41m26d38860a200b47f@mail.gmail.com> 
	<ea2499da0911051453n3569387aj581ad4acf5b799c8@mail.gmail.com> 
	<4222a8490911051521r1b9c8165id6e0f12d62da0613@mail.gmail.com> 
	<ca471dc20911051526u7f3642ecg19e4eb73bcca1d3a@mail.gmail.com> 
	<bbaeab100911051535g68cefa18kdadea33f78de9495@mail.gmail.com> 
	<ca471dc20911051923j113d4deame8757c5775f040d2@mail.gmail.com> 
	<bbaeab100911052155m3558186xbcd833bc50c9a59f@mail.gmail.com> 
	<ca471dc20911060802v65c2102dt1d8b47440a993045@mail.gmail.com> 
	<f3cc57c60911060825j608d7bf9q66cc77bd2828252@mail.gmail.com>
Message-ID: <12a697af0911061131yaa62530n16d11c78317d8f28@mail.gmail.com>

On Fri, Nov 6, 2009 at 8:25 AM, geremy condra <debatem1 at gmail.com> wrote:
> And speaking of TIOBE, my impression- and apparently yours- was
> that Python was on its way up, but TIOBE lists us as being down
> from a little over a year ago. Anybody know anything about their
> methodology?

There's a detailed explanation at:

http://www.tiobe.com/index.php/content/paperinfo/tpci/tpci_definition.htm

Andre

From tseaver at palladion.com  Fri Nov  6 20:37:44 2009
From: tseaver at palladion.com (Tres Seaver)
Date: Fri, 06 Nov 2009 14:37:44 -0500
Subject: [Python-Dev] PEP 3003 - Python Language Moratorium
In-Reply-To: <ea2499da0911060147n70755574yf27d573e011df7af@mail.gmail.com>
References: <ca471dc20911030935l25251e0diec64427f5d7bcbf2@mail.gmail.com>	<ca471dc20911051405o287e0f41m26d38860a200b47f@mail.gmail.com>	<ea2499da0911051453n3569387aj581ad4acf5b799c8@mail.gmail.com>	<4222a8490911051521r1b9c8165id6e0f12d62da0613@mail.gmail.com>	<ca471dc20911051526u7f3642ecg19e4eb73bcca1d3a@mail.gmail.com>	<bbaeab100911051535g68cefa18kdadea33f78de9495@mail.gmail.com>	<ca471dc20911051923j113d4deame8757c5775f040d2@mail.gmail.com>	<bbaeab100911052155m3558186xbcd833bc50c9a59f@mail.gmail.com>	<4AF3EDF2.2010406@gmail.com>
	<ea2499da0911060147n70755574yf27d573e011df7af@mail.gmail.com>
Message-ID: <4AF47B08.4020009@palladion.com>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Dirkjan Ochtman wrote:

> But supporting 4 releases in a row has been a bit of a PITA. Luckily,
> we finally felt it was time to drop 2.3, so now we can make use of
> luxuries such as subprocess... Still, supporting 3 releases seems
> relatively common in the Python world (after all, parts of Zope still
> require 2.4, I think), and so it would be nice if the stdlib moved a
> little bit slower.

The current release of Zope (finally) requires 2.5 or 2.6:  we have a
"don't intentionally break 2.4" policy in place, however, FBO those
users whose system Python is still 2.4, and who can't / won't build
their own Python.


Tres.
- --
===================================================================
Tres Seaver          +1 540-429-0999          tseaver at palladion.com
Palladion Software   "Excellence by Design"    http://palladion.com
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iEYEARECAAYFAkr0ewgACgkQ+gerLs4ltQ7W2ACeOXPwhpg8Hi4/XZom/B59/7GD
4mwAnRkixQm0A3ZkNRQMPW1JIbEOy1iC
=gTbW
-----END PGP SIGNATURE-----


From guido at python.org  Fri Nov  6 21:22:38 2009
From: guido at python.org (Guido van Rossum)
Date: Fri, 6 Nov 2009 12:22:38 -0800
Subject: [Python-Dev] OpenSSL vulnerability
Message-ID: <ca471dc20911061222v5c380643te7d5a4e84d3496f9@mail.gmail.com>

Now that a new SSL vulnerability is out
(http://extendedsubset.com/?p=8) should we regenerate binary
distributions that include copies of openssl (I think only the Windows
MSIs) ?

Does it affect any of our ssl APIs?

-- 
--Guido van Rossum (python.org/~guido)

From db3l.net at gmail.com  Fri Nov  6 21:34:36 2009
From: db3l.net at gmail.com (David Bolen)
Date: Fri, 06 Nov 2009 15:34:36 -0500
Subject: [Python-Dev] Status of the Buildbot fleet and related bugs
References: <Pine.LNX.4.64.0911052209000.12274@kimball.webabinitio.net>
	<5c6f2a5d0911060027x6152a27ao77df7938a079f622@mail.gmail.com>
	<m24op7apok.fsf@valheru.db3l.homeip.net>
	<5c6f2a5d0911060740g3178b2d0p48e8ef483d6e9c7d@mail.gmail.com>
Message-ID: <m2zl6z8k9v.fsf@valheru.db3l.homeip.net>

Mark Dickinson <dickinsm at gmail.com> writes:

> Not sure either, but it's certainly useful to be able to test on
> FreeBSD 6.x.  Ideally, we'd have buildslaves for both...

Well, let me plan on leaving the current slave at 6.x (but working on
getting it to 6.4).  I could probably see about providing an
additional 7.x slave - the only real issue is that at the moment
they'd both be VMs on the same physical host, which is already shared
with my Windows slave.

I think a recent posting by Martin mentioned being able to interlock
one or more slaves so they didn't execute in parallel, so that could
help clean up the contention, though I must admit to not knowing for
sure if serial execution would yield a faster overall result among all
the slaves, though it feels likely.

-- David


From glyph at twistedmatrix.com  Fri Nov  6 21:48:23 2009
From: glyph at twistedmatrix.com (Glyph Lefkowitz)
Date: Fri, 6 Nov 2009 15:48:23 -0500
Subject: [Python-Dev] PEP 3003 - Python Language Moratorium
In-Reply-To: <20091106140412.3229.80002680.divmod.xquotient.193@localhost.localdomain>
References: <ca471dc20911030935l25251e0diec64427f5d7bcbf2@mail.gmail.com>
	<ca471dc20911051405o287e0f41m26d38860a200b47f@mail.gmail.com>
	<ea2499da0911051453n3569387aj581ad4acf5b799c8@mail.gmail.com>
	<4222a8490911051521r1b9c8165id6e0f12d62da0613@mail.gmail.com>
	<ca471dc20911051526u7f3642ecg19e4eb73bcca1d3a@mail.gmail.com>
	<bbaeab100911051535g68cefa18kdadea33f78de9495@mail.gmail.com>
	<48c184130911051555r9a5b78cs3a13cb1345d3cbf7@mail.gmail.com>
	<20091106001430.3229.796306500.divmod.xquotient.180@localhost.localdomain>
	<4AF3CF69.6090709@gmx.net>
	<20091106135058.GA13578@vidar.dreamhost.com>
	<20091106140412.3229.80002680.divmod.xquotient.193@localhost.localdomain>
Message-ID: <68272E43-D993-4833-8DC9-DB45CB40F368@twistedmatrix.com>

On Nov 6, 2009, at 9:04 AM, exarkun at twistedmatrix.com wrote:

> End users don't even need to be told this much, though (and if it's  
> javac that does this for Java, then indeed Java end users probably  
> aren't seeing this either).  I think it would be great for  
> deprecation warnings to be completely silent for a while *and* for  
> the policy to be prominently documented so that developers, the  
> people who really need this information, know how to get it.

Documentation would be great, but then you have to get people to read  
the documentation and that's kind of tricky.  Better would be for  
every project on PyPI to have a score which listed warnings emitted  
with each version of Python.  People love optimizing for stuff like  
that and comparing it.

I suspect that even if all warnings were completely silent by default,  
developers would suddenly become keenly interested in fixing them if  
there were a metric like that publicly posted somewhere :).


From debatem1 at gmail.com  Fri Nov  6 22:15:52 2009
From: debatem1 at gmail.com (geremy condra)
Date: Fri, 6 Nov 2009 16:15:52 -0500
Subject: [Python-Dev] OpenSSL vulnerability
In-Reply-To: <ca471dc20911061222v5c380643te7d5a4e84d3496f9@mail.gmail.com>
References: <ca471dc20911061222v5c380643te7d5a4e84d3496f9@mail.gmail.com>
Message-ID: <f3cc57c60911061315m7d77ec09o4695fb57d29f8c22@mail.gmail.com>

On Fri, Nov 6, 2009 at 3:22 PM, Guido van Rossum <guido at python.org> wrote:
> Now that a new SSL vulnerability is out
> (http://extendedsubset.com/?p=8) should we regenerate binary
> distributions that include copies of openssl (I think only the Windows
> MSIs) ?
>
> Does it affect any of our ssl APIs?
>
> --
> --Guido van Rossum (python.org/~guido)

The proposal on the table is to add a TLS extension that
takes care of the problem, leave clients unchanged, and
to stop servers from rehandshaking with clients that don't
support the extension. AFAICS, that's all supposed to be
handled by openssl. Certainly the EVP stuff won't need
to be modified.

The version of openssl being distributed should definitely
be brought up to 0.9.8l though.

Geremy Condra

From tjreedy at udel.edu  Fri Nov  6 22:45:39 2009
From: tjreedy at udel.edu (Terry Reedy)
Date: Fri, 06 Nov 2009 16:45:39 -0500
Subject: [Python-Dev] PEP 3003 - Python Language Moratorium
In-Reply-To: <bbaeab100911060937q33f0d1b8rdf5ad80bc6784a0@mail.gmail.com>
References: <ca471dc20911030935l25251e0diec64427f5d7bcbf2@mail.gmail.com>
	<ca471dc20911051405o287e0f41m26d38860a200b47f@mail.gmail.com>
	<ea2499da0911051453n3569387aj581ad4acf5b799c8@mail.gmail.com>
	<4222a8490911051521r1b9c8165id6e0f12d62da0613@mail.gmail.com>
	<ca471dc20911051526u7f3642ecg19e4eb73bcca1d3a@mail.gmail.com>
	<bbaeab100911051535g68cefa18kdadea33f78de9495@mail.gmail.com>
	<ca471dc20911051923j113d4deame8757c5775f040d2@mail.gmail.com>
	<bbaeab100911052155m3558186xbcd833bc50c9a59f@mail.gmail.com>
	<ca471dc20911060802v65c2102dt1d8b47440a993045@mail.gmail.com>
	<bbaeab100911060937q33f0d1b8rdf5ad80bc6784a0@mail.gmail.com>
Message-ID: <hd25dv$oqu$1@ger.gmane.org>

Brett Cannon wrote:
> On Fri, Nov 6, 2009 at 08:02, Guido van Rossum <guido at python.org> wrote:
>> On Thu, Nov 5, 2009 at 9:55 PM, Brett Cannon <brett at python.org> wrote:
>>> The clarification I need is will this in any way influence when
>>> modules are removed. If they stay in for the life of a major version
>>> then I want it made clear that bug fixes for the code take lower
>>> priority over all other code in the standard library.
>> I think we should be as cautious as ever with removing modules. We've
>> had our chance for clean-up without abandon with Py3k; now we should
>> stick with our commitment to backwards compatibility. In fact, we
>> should probably be *more* conservative than we already were given that
>> the amount of code written in Python is always increasing and hence
>> the cumulative pain caused by incompatible changes will increase too.
>>
>> I'm fine with silent deprecations or requiring a flag to turn on
>> deprecation warnings (like Java does).
>>
> 
> We can discuss this on the stdlib-sig and come back with a proposal on
> how to update PEP 4 with an explicit policy on how to handle
> deprecations (either Nick's proposal or some flag, and then how long
> to let the deprecation last, i.e. three versions or the life of a
> major version of Python).

Complete removal of modules during the seems to have caused problems 
especially for people supporting a range of versions. On the other hand, 
you want to be able to replace and cease support of modules. So rather 
that complete removal, perhaps move them to a new package called, for 
instance, 'old'. And move their doc sections to a new chapter call 'Old 
Modules' which would begin with something like "The following modules 
have been replaced but are still present in the *old* package for back 
compatibility. They are no longer maintained and will possibly disappear 
in the future, such as if there is ever a Python4." This suggestion 
based on the "3. Non-essential Built-in Functions" of the 2.x doc, whose 
contents disappeared in 3.0 (even though that seems not to have been 
specified in the doc).

Terry Jan Reedy



From rdmurray at bitdance.com  Fri Nov  6 22:48:37 2009
From: rdmurray at bitdance.com (R. David Murray)
Date: Fri, 6 Nov 2009 16:48:37 -0500 (EST)
Subject: [Python-Dev] PEP 3003 - Python Language Moratorium
In-Reply-To: <68272E43-D993-4833-8DC9-DB45CB40F368@twistedmatrix.com>
References: <ca471dc20911030935l25251e0diec64427f5d7bcbf2@mail.gmail.com>
	<ca471dc20911051405o287e0f41m26d38860a200b47f@mail.gmail.com>
	<ea2499da0911051453n3569387aj581ad4acf5b799c8@mail.gmail.com>
	<4222a8490911051521r1b9c8165id6e0f12d62da0613@mail.gmail.com>
	<ca471dc20911051526u7f3642ecg19e4eb73bcca1d3a@mail.gmail.com>
	<bbaeab100911051535g68cefa18kdadea33f78de9495@mail.gmail.com>
	<48c184130911051555r9a5b78cs3a13cb1345d3cbf7@mail.gmail.com>
	<20091106001430.3229.796306500.divmod.xquotient.180@localhost.localdomain>
	<4AF3CF69.6090709@gmx.net> <20091106135058.GA13578@vidar.dreamhost.com>
	<20091106140412.3229.80002680.divmod.xquotient.193@localhost.localdomain>
	<68272E43-D993-4833-8DC9-DB45CB40F368@twistedmatrix.com>
Message-ID: <Pine.LNX.4.64.0911061647160.12274@kimball.webabinitio.net>

On Fri, 6 Nov 2009 at 15:48, Glyph Lefkowitz wrote:
> On Nov 6, 2009, at 9:04 AM, exarkun at twistedmatrix.com wrote:
>> End users don't even need to be told this much, though (and if it's javac 
>> that does this for Java, then indeed Java end users probably aren't seeing 
>> this either).  I think it would be great for deprecation warnings to be 
>> completely silent for a while *and* for the policy to be prominently 
>> documented so that developers, the people who really need this information, 
>> know how to get it.
>
> Documentation would be great, but then you have to get people to read the 
> documentation and that's kind of tricky.  Better would be for every project 
> on PyPI to have a score which listed warnings emitted with each version of 
> Python.  People love optimizing for stuff like that and comparing it.
>
> I suspect that even if all warnings were completely silent by default, 
> developers would suddenly become keenly interested in fixing them if there 
> were a metric like that publicly posted somewhere :).

+1, but somebody needs to write the code...

--David (RDM)

From exarkun at twistedmatrix.com  Fri Nov  6 22:52:10 2009
From: exarkun at twistedmatrix.com (exarkun at twistedmatrix.com)
Date: Fri, 06 Nov 2009 21:52:10 -0000
Subject: [Python-Dev] PEP 3003 - Python Language Moratorium
In-Reply-To: <Pine.LNX.4.64.0911061647160.12274@kimball.webabinitio.net>
References: <ca471dc20911030935l25251e0diec64427f5d7bcbf2@mail.gmail.com>
	<ca471dc20911051405o287e0f41m26d38860a200b47f@mail.gmail.com>
	<ea2499da0911051453n3569387aj581ad4acf5b799c8@mail.gmail.com>
	<4222a8490911051521r1b9c8165id6e0f12d62da0613@mail.gmail.com>
	<ca471dc20911051526u7f3642ecg19e4eb73bcca1d3a@mail.gmail.com>
	<bbaeab100911051535g68cefa18kdadea33f78de9495@mail.gmail.com>
	<48c184130911051555r9a5b78cs3a13cb1345d3cbf7@mail.gmail.com>
	<20091106001430.3229.796306500.divmod.xquotient.180@localhost.localdomain>
	<4AF3CF69.6090709@gmx.net>
	<20091106135058.GA13578@vidar.dreamhost.com>
	<20091106140412.3229.80002680.divmod.xquotient.193@localhost.localdomain>
	<68272E43-D993-4833-8DC9-DB45CB40F368@twistedmatrix.com>
	<Pine.LNX.4.64.0911061647160.12274@kimball.webabinitio.net>
Message-ID: <20091106215210.3229.1142452245.divmod.xquotient.212@localhost.localdomain>

On 09:48 pm, rdmurray at bitdance.com wrote:
>On Fri, 6 Nov 2009 at 15:48, Glyph Lefkowitz wrote:
>>On Nov 6, 2009, at 9:04 AM, exarkun at twistedmatrix.com wrote:
>>>End users don't even need to be told this much, though (and if it's 
>>>javac that does this for Java, then indeed Java end users probably 
>>>aren't seeing this either).  I think it would be great for 
>>>deprecation warnings to be completely silent for a while *and* for 
>>>the policy to be prominently documented so that developers, the 
>>>people who really need this information, know how to get it.
>>
>>Documentation would be great, but then you have to get people to read 
>>the documentation and that's kind of tricky.  Better would be for 
>>every project on PyPI to have a score which listed warnings emitted 
>>with each version of Python.  People love optimizing for stuff like 
>>that and comparing it.
>>
>>I suspect that even if all warnings were completely silent by default, 
>>developers would suddenly become keenly interested in fixing them if 
>>there were a metric like that publicly posted somewhere :).
>
>+1, but somebody needs to write the code...

How would you collect this information?  Would you run the test suite 
for each project?  This would reward projects with small or absent test 
suites. ;)

Jean-Paul

From janssen at parc.com  Fri Nov  6 23:18:22 2009
From: janssen at parc.com (Bill Janssen)
Date: Fri, 6 Nov 2009 14:18:22 PST
Subject: [Python-Dev] OpenSSL vulnerability
In-Reply-To: <ca471dc20911061222v5c380643te7d5a4e84d3496f9@mail.gmail.com>
References: <ca471dc20911061222v5c380643te7d5a4e84d3496f9@mail.gmail.com>
Message-ID: <40373.1257545902@parc.com>

Guido,

I'm working from <http://extendedsubset.com/Renegotiating_TLS.pdf>.

I believe geremy is right.  The current SSL module does not expose much
of the SSL API, so servers implemented in Python, using it, should
(fortuituously) be immune to the some of the attacks outlined, simply
because there's no way to do an application-initiated renegotiation,
which the first two scenarios presuppose.  On the other hand, there's no
way to do application-directed session resumption, either, which might
be a good add to support new or updated application protocols which
address this problem.

So I think there's not much we can do in Python source code to address
this, unless there's a switch we can throw in the existing OpenSSL API
to turn off renegotiation completely.  I'll look, and I'll talk this
over with our security group.  Building binaries with newer versions of
OpenSSL is pretty much always a good idea, it seems to me.

More generally, this is a nice description of how simply layering TLS
onto existing application protocols like HTTP doesn't always work very
well.

Bill

From g.brandl at gmx.net  Sat Nov  7 00:24:51 2009
From: g.brandl at gmx.net (Georg Brandl)
Date: Sat, 07 Nov 2009 00:24:51 +0100
Subject: [Python-Dev] PEP 3003 - Python Language Moratorium
In-Reply-To: <ca471dc20911051526u7f3642ecg19e4eb73bcca1d3a@mail.gmail.com>
References: <ca471dc20911030935l25251e0diec64427f5d7bcbf2@mail.gmail.com>
	<ca471dc20911051405o287e0f41m26d38860a200b47f@mail.gmail.com>
	<ea2499da0911051453n3569387aj581ad4acf5b799c8@mail.gmail.com>
	<4222a8490911051521r1b9c8165id6e0f12d62da0613@mail.gmail.com>
	<ca471dc20911051526u7f3642ecg19e4eb73bcca1d3a@mail.gmail.com>
Message-ID: <hd27t3$v2d$2@ger.gmane.org>

Guido van Rossum schrieb:

> Actually, I think Dirkjan has a point. I'm not sure that we need
> another moratorium (that's a rather dramatic kind of decision which
> should be very rare indeed) but I do agree that deprecations are often
> more of a pain than they're worth.
> 
> For example, take the deprecation of the md5 and sha modules in Python
> 2.6. They make it a bit of a pain to write code that *cleanly*
> supports Python 2.4 (doesn't have hashlib) through 2.6 (warns when
> importing md5 instead of hashlib). You can silence the warning, but
> that is in itself not particularly clean, and users really hate having
> the warnings.

Trying to import hashlib and importing md5 on ImportError isn't *too* unclean,
is it?

Georg


-- 
Thus spake the Lord: Thou shalt indent with four spaces. No more, no less.
Four shall be the number of spaces thou shalt indent, and the number of thy
indenting shall be four. Eight shalt thou not indent, nor either indent thou
two, excepting that thou then proceed to four. Tabs are right out.


From exarkun at twistedmatrix.com  Fri Nov  6 23:36:34 2009
From: exarkun at twistedmatrix.com (exarkun at twistedmatrix.com)
Date: Fri, 06 Nov 2009 22:36:34 -0000
Subject: [Python-Dev] OpenSSL vulnerability
In-Reply-To: <40373.1257545902@parc.com>
References: <ca471dc20911061222v5c380643te7d5a4e84d3496f9@mail.gmail.com>
	<40373.1257545902@parc.com>
Message-ID: <20091106223634.3229.2049231320.divmod.xquotient.232@localhost.localdomain>

On 10:18 pm, janssen at parc.com wrote:
>Guido,
>
>I'm working from <http://extendedsubset.com/Renegotiating_TLS.pdf>.
>
>I believe geremy is right.  The current SSL module does not expose much
>of the SSL API, so servers implemented in Python, using it, should
>(fortuituously) be immune to the some of the attacks outlined, simply
>because there's no way to do an application-initiated renegotiation,
>which the first two scenarios presuppose.  On the other hand, there's 
>no
>way to do application-directed session resumption, either, which might
>be a good add to support new or updated application protocols which
>address this problem.

Also, for Python 2.5 and earlier, any SSL-based code is vulnerable to a 
MitM anyway, so this can only be an issue for code using the new APIs in 
Python 2.6.

Jean-Paul

From guido at python.org  Fri Nov  6 23:37:10 2009
From: guido at python.org (Guido van Rossum)
Date: Fri, 6 Nov 2009 14:37:10 -0800
Subject: [Python-Dev] PEP 3003 - Python Language Moratorium
In-Reply-To: <hd27t3$v2d$2@ger.gmane.org>
References: <ca471dc20911030935l25251e0diec64427f5d7bcbf2@mail.gmail.com> 
	<ca471dc20911051405o287e0f41m26d38860a200b47f@mail.gmail.com> 
	<ea2499da0911051453n3569387aj581ad4acf5b799c8@mail.gmail.com> 
	<4222a8490911051521r1b9c8165id6e0f12d62da0613@mail.gmail.com> 
	<ca471dc20911051526u7f3642ecg19e4eb73bcca1d3a@mail.gmail.com> 
	<hd27t3$v2d$2@ger.gmane.org>
Message-ID: <ca471dc20911061437g3f524653le92ee788bf8f8a64@mail.gmail.com>

On Fri, Nov 6, 2009 at 3:24 PM, Georg Brandl <g.brandl at gmx.net> wrote:
> Guido van Rossum schrieb:
>
>> Actually, I think Dirkjan has a point. I'm not sure that we need
>> another moratorium (that's a rather dramatic kind of decision which
>> should be very rare indeed) but I do agree that deprecations are often
>> more of a pain than they're worth.
>>
>> For example, take the deprecation of the md5 and sha modules in Python
>> 2.6. They make it a bit of a pain to write code that *cleanly*
>> supports Python 2.4 (doesn't have hashlib) through 2.6 (warns when
>> importing md5 instead of hashlib). You can silence the warning, but
>> that is in itself not particularly clean, and users really hate having
>> the warnings.
>
> Trying to import hashlib and importing md5 on ImportError isn't *too* unclean,
> is it?

That's not all -- you also have to modify the code that uses the
module, unless you use "import as", which has problems of its own.
Plus, I may not care (yet) about supporting 2.7, and yet I am forced
to change my code to cleanly support 2.6. I really don't like it.

-- 
--Guido van Rossum (python.org/~guido)

From g.brandl at gmx.net  Sat Nov  7 00:34:25 2009
From: g.brandl at gmx.net (Georg Brandl)
Date: Sat, 07 Nov 2009 00:34:25 +0100
Subject: [Python-Dev] Status of the Buildbot fleet and related bugs
In-Reply-To: <Pine.LNX.4.64.0911052209000.12274@kimball.webabinitio.net>
References: <Pine.LNX.4.64.0911052209000.12274@kimball.webabinitio.net>
Message-ID: <hd28f1$v2d$3@ger.gmane.org>

R. David Murray schrieb:

> So, overall I think the buildbot fleet is in good shape, and if
> we can nail issue 6748 I think it will be back to being an
> important resource for sanity checking our checkins.

Yay! Thanks to all of you!

> By the way, Georg set up the IRC interface on the #python-dev channel,
> so you can hang out there if you want to get realtime reports of which
> buildbots have going from success to failure and vice versa.

JFTR, I didn't set up the IRC bot (I assume that credit goes to Martin,
even if it's only one line in the buildbot config :).  I just tried to
get it to say something :)

Georg

-- 
Thus spake the Lord: Thou shalt indent with four spaces. No more, no less.
Four shall be the number of spaces thou shalt indent, and the number of thy
indenting shall be four. Eight shalt thou not indent, nor either indent thou
two, excepting that thou then proceed to four. Tabs are right out.


From g.brandl at gmx.net  Sat Nov  7 00:59:10 2009
From: g.brandl at gmx.net (Georg Brandl)
Date: Sat, 07 Nov 2009 00:59:10 +0100
Subject: [Python-Dev] Retrieve an arbitrary element from
	asetwithoutremoving it
In-Reply-To: <7b0045ed0911052219u1893bd62gebf71bf46721014a@mail.gmail.com>
References: <7b0045ed0910301843y7059d316s8a851142b78b7aeb@mail.gmail.com>
	<f3cc57c60911041332v479cc3b0t19d9601e52002128@mail.gmail.com>
	<99BB308202224D31AA5300223DB59912@RaymondLaptop1>	<7b0045ed0911051243q18f8324ak9abaa98ac26948e9@mail.gmail.com>
	<4AF341BD.2010203@v.loewis.de>	<7b0045ed0911051419i76f4221dye8ecccc424179ec6@mail.gmail.com>
	<BD050FFAAA19443DA0DA34FD2BD740CF@RaymondLaptop1>	<97069B15530D4C66AC1D4BCCB70E2FD5@RaymondLaptop1>
	<7b0045ed0911051859y13f8abf3u7b5b4549aa2a41fc@mail.gmail.com>
	<4AF3B76A.1050006@v.loewis.de>
	<7b0045ed0911052219u1893bd62gebf71bf46721014a@mail.gmail.com>
Message-ID: <hd29te$6km$1@ger.gmane.org>

Chris Bergstresser schrieb:
> On Thu, Nov 5, 2009 at 11:43 PM, "Martin v. L?wis" <martin at v.loewis.de> wrote:
>> I read Raymond's suggestion rather as a question: why bother with a
>> tedious, multi-year process, when a three-line function will achieve
>> exactly the same?
> 
>    Because it doesn't achieve exactly the same.  What I want is a
> sane, rational way to describe what I'm doing in the core API, so
> other programmers learning the language don't spend the amount of time
> I did perplexed that there was a .pop() and a .remove() and a
> .discard(), but there wasn't a .pick().  I don't want to have to write
> the same little helper function in every project to fill a deficiency
> in the library.

Why don't you write that little helper function only in those projects that
actually need it?  I suspect that will be a small fraction...

> I don't want to have to argue about the flaws in
> solutions with race conditions, or the fact that cheap functions
> become expensive functions when performed over the network, or that
> there's a real value in having an atomic operation which throws a sane
> exception when it fails, or how it's disturbing to my OCD core to have
> an API which believes:
> 
>   if x in s:
>     s.remove(x)
> 
> ... is too confusing, so there should be a .discard() method, but ...
> 
>   for x in s:
>     break
> 
> ... is self-evident and obvious, so there's no need for a .pick().

It's not just the self-evidence of the required "workaround" that determines
whether an API function is added to a builtin type.  It's whether you need it.
Imagine someone asking for a function on lists that removes duplicates, but
keeps the order of all first occurrences.  This is not a completely unimaginable
method, and writing it by hand is more than a few lines (see e.g.
http://code.activestate.com/recipes/52560/#c1).  Still, it's not needed
frequently enough to require being a list method.

Georg

-- 
Thus spake the Lord: Thou shalt indent with four spaces. No more, no less.
Four shall be the number of spaces thou shalt indent, and the number of thy
indenting shall be four. Eight shalt thou not indent, nor either indent thou
two, excepting that thou then proceed to four. Tabs are right out.


From guido at python.org  Sat Nov  7 00:12:28 2009
From: guido at python.org (Guido van Rossum)
Date: Fri, 6 Nov 2009 15:12:28 -0800
Subject: [Python-Dev] OpenSSL vulnerability
In-Reply-To: <20091106223634.3229.2049231320.divmod.xquotient.232@localhost.localdomain>
References: <ca471dc20911061222v5c380643te7d5a4e84d3496f9@mail.gmail.com> 
	<40373.1257545902@parc.com>
	<20091106223634.3229.2049231320.divmod.xquotient.232@localhost.localdomain>
Message-ID: <ca471dc20911061512j1f5f7e45nb5fd2de0d2734d45@mail.gmail.com>

On Fri, Nov 6, 2009 at 2:36 PM,  <exarkun at twistedmatrix.com> wrote:
> Also, for Python 2.5 and earlier, any SSL-based code is vulnerable to a MitM
> anyway, so this can only be an issue for code using the new APIs in Python
> 2.6.

That's not going to stop the
wannabe-self-proclaimed-so-called-vulnerability-"experts" from whining
about Python not releasing updated binary distributions though. :-(

-- 
--Guido van Rossum (python.org/~guido)

From andymac at bullseye.apana.org.au  Sat Nov  7 01:39:20 2009
From: andymac at bullseye.apana.org.au (Andrew MacIntyre)
Date: Sat, 07 Nov 2009 11:39:20 +1100
Subject: [Python-Dev] Status of the Buildbot fleet and related bugs
In-Reply-To: <5c6f2a5d0911060740g3178b2d0p48e8ef483d6e9c7d@mail.gmail.com>
References: <Pine.LNX.4.64.0911052209000.12274@kimball.webabinitio.net>	<5c6f2a5d0911060027x6152a27ao77df7938a079f622@mail.gmail.com>	<m24op7apok.fsf@valheru.db3l.homeip.net>
	<5c6f2a5d0911060740g3178b2d0p48e8ef483d6e9c7d@mail.gmail.com>
Message-ID: <4AF4C1B8.1090007@bullseye.andymac.org>

Mark Dickinson wrote:
> On Fri, Nov 6, 2009 at 10:54 AM, David Bolen <db3l.net at gmail.com> wrote:
>> 
>> In terms of the overall release, I'm also fine with upgrading the
>> build slave to 6.4.  Or I could jump up to 7.2 instead.  When I first
>> brought the build slave up, 7.x wasn't finalized yet - not sure now
>> which release is more prevalent in use.
> 
> Not sure either, but it's certainly useful to be able to test on
> FreeBSD 6.x.  Ideally, we'd have buildslaves for both...

http://security.freebsd.org/#sup lists the releases currently with 
active security updates and their EOL dates.

Upgrading the 6.2 buildbot to 6.4 would seem a good idea to me.
Unfortunately FreeBSD's binary update capability only became available
with 6.3... :-(

A separate 7.2 buildbot (which could then be binary updated to 7.3
on release) would also seem a good idea.

8.0 is at RC3 and could be expected to be finalised fairly soon.

My understanding is there won't be another 6.x release, and I'm 
inferring that there will be one more 7.x release (7.3).

Andrew.

-- 
-------------------------------------------------------------------------
Andrew I MacIntyre                     "These thoughts are mine alone..."
E-mail: andymac at bullseye.apana.org.au  (pref) | Snail: PO Box 370
        andymac at pcug.org.au             (alt) |        Belconnen ACT 2616
Web:    http://www.andymac.org/               |        Australia

From ncoghlan at gmail.com  Sat Nov  7 03:42:35 2009
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sat, 07 Nov 2009 12:42:35 +1000
Subject: [Python-Dev] PEP 3003 - Python Language Moratorium
In-Reply-To: <hd27t3$v2d$2@ger.gmane.org>
References: <ca471dc20911030935l25251e0diec64427f5d7bcbf2@mail.gmail.com>	<ca471dc20911051405o287e0f41m26d38860a200b47f@mail.gmail.com>	<ea2499da0911051453n3569387aj581ad4acf5b799c8@mail.gmail.com>	<4222a8490911051521r1b9c8165id6e0f12d62da0613@mail.gmail.com>	<ca471dc20911051526u7f3642ecg19e4eb73bcca1d3a@mail.gmail.com>
	<hd27t3$v2d$2@ger.gmane.org>
Message-ID: <4AF4DE9B.6080703@gmail.com>

Georg Brandl wrote:
> Guido van Rossum schrieb:
> 
>> Actually, I think Dirkjan has a point. I'm not sure that we need
>> another moratorium (that's a rather dramatic kind of decision which
>> should be very rare indeed) but I do agree that deprecations are often
>> more of a pain than they're worth.
>>
>> For example, take the deprecation of the md5 and sha modules in Python
>> 2.6. They make it a bit of a pain to write code that *cleanly*
>> supports Python 2.4 (doesn't have hashlib) through 2.6 (warns when
>> importing md5 instead of hashlib). You can silence the warning, but
>> that is in itself not particularly clean, and users really hate having
>> the warnings.
> 
> Trying to import hashlib and importing md5 on ImportError isn't *too* unclean,
> is it?

Having had to do this myself a few times, I was always a little
surprised the "import-or" idea [1] didn't get more support the last time
it was proposed.

Ah well, don't need to worry about that idea again until 3.4 or so :)

Cheers,
Nick

P.S. For anyone unfamiliar with it, "import-or" was a suggestion made
around the time xml.etree was added to the standard library to provide a
"import x or y as z" shorthand for:
  try:
    import x as z
  except ImportError:
    import y as z

(allowing "from a or b import c" was also part of the suggestion, as was
allowing multiple "or" elements in a single import request)

Support for the idea was lukewarm at best, hostile at worst (hence the
lack of PEP).

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------

From db3l.net at gmail.com  Sat Nov  7 03:44:44 2009
From: db3l.net at gmail.com (David Bolen)
Date: Fri, 06 Nov 2009 21:44:44 -0500
Subject: [Python-Dev] Status of the Buildbot fleet and related bugs
References: <Pine.LNX.4.64.0911052209000.12274@kimball.webabinitio.net>
	<5c6f2a5d0911060027x6152a27ao77df7938a079f622@mail.gmail.com>
	<m24op7apok.fsf@valheru.db3l.homeip.net>
	<5c6f2a5d0911060740g3178b2d0p48e8ef483d6e9c7d@mail.gmail.com>
	<4AF4C1B8.1090007@bullseye.andymac.org>
Message-ID: <m2vdhn834z.fsf@valheru.db3l.homeip.net>

Andrew MacIntyre <andymac at bullseye.apana.org.au> writes:

> Upgrading the 6.2 buildbot to 6.4 would seem a good idea to me.
> Unfortunately FreeBSD's binary update capability only became available
> with 6.3... :-(

No biggie - I was just planning on installing a new VM from scratch
anyway, and then just cutting over VMs to switch over.

-- David


From guido at python.org  Sat Nov  7 04:16:10 2009
From: guido at python.org (Guido van Rossum)
Date: Fri, 6 Nov 2009 19:16:10 -0800
Subject: [Python-Dev] PEP 3003 - Python Language Moratorium
In-Reply-To: <4AF4DE9B.6080703@gmail.com>
References: <ca471dc20911030935l25251e0diec64427f5d7bcbf2@mail.gmail.com> 
	<ca471dc20911051405o287e0f41m26d38860a200b47f@mail.gmail.com> 
	<ea2499da0911051453n3569387aj581ad4acf5b799c8@mail.gmail.com> 
	<4222a8490911051521r1b9c8165id6e0f12d62da0613@mail.gmail.com> 
	<ca471dc20911051526u7f3642ecg19e4eb73bcca1d3a@mail.gmail.com> 
	<hd27t3$v2d$2@ger.gmane.org> <4AF4DE9B.6080703@gmail.com>
Message-ID: <ca471dc20911061916h7d4a83a9qf66abaaca5480750@mail.gmail.com>

On Fri, Nov 6, 2009 at 6:42 PM, Nick Coghlan <ncoghlan at gmail.com> wrote:
> Georg Brandl wrote:
>> Guido van Rossum schrieb:
>>
>>> Actually, I think Dirkjan has a point. I'm not sure that we need
>>> another moratorium (that's a rather dramatic kind of decision which
>>> should be very rare indeed) but I do agree that deprecations are often
>>> more of a pain than they're worth.
>>>
>>> For example, take the deprecation of the md5 and sha modules in Python
>>> 2.6. They make it a bit of a pain to write code that *cleanly*
>>> supports Python 2.4 (doesn't have hashlib) through 2.6 (warns when
>>> importing md5 instead of hashlib). You can silence the warning, but
>>> that is in itself not particularly clean, and users really hate having
>>> the warnings.
>>
>> Trying to import hashlib and importing md5 on ImportError isn't *too* unclean,
>> is it?
>
> Having had to do this myself a few times, I was always a little
> surprised the "import-or" idea [1] didn't get more support the last time
> it was proposed.
>
> Ah well, don't need to worry about that idea again until 3.4 or so :)
>
> Cheers,
> Nick
>
> P.S. For anyone unfamiliar with it, "import-or" was a suggestion made
> around the time xml.etree was added to the standard library to provide a
> "import x or y as z" shorthand for:
> ?try:
> ? ?import x as z
> ?except ImportError:
> ? ?import y as z
>
> (allowing "from a or b import c" was also part of the suggestion, as was
> allowing multiple "or" elements in a single import request)
>
> Support for the idea was lukewarm at best, hostile at worst (hence the
> lack of PEP).

I'm still -1 on that -- while it may sound like a good shorthand if
you have to deal with this, it's IMO pretty mysterious if you
encounter this and don't already happen to know what it means. Also,
it makes no sense without the 'as' clause which makes it a bit
awkward. (I could probably come up with a few more arguments against
it, just trying to hint that I'm personally less than lukewarm. :-)

-- 
--Guido van Rossum (python.org/~guido)

From stephen at xemacs.org  Sat Nov  7 09:35:22 2009
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Sat,  7 Nov 2009 17:35:22 +0900 (JST)
Subject: [Python-Dev] PEP 3003 - Python Language Moratorium
In-Reply-To: <20091106215210.3229.1142452245.divmod.xquotient.212@localhost.localdomain>
References: <ca471dc20911030935l25251e0diec64427f5d7bcbf2@mail.gmail.com>
	<ca471dc20911051405o287e0f41m26d38860a200b47f@mail.gmail.com>
Message-ID: <20091107083522.7AC4C1A266C@uwakimon.sk.tsukuba.ac.jp>

~b	<ea2499da0911051453n3569387aj581ad4acf5b799c8 at mail.gmail.com>
	<4222a8490911051521r1b9c8165id6e0f12d62da0613 at mail.gmail.com>
	<ca471dc20911051526u7f3642ecg19e4eb73bcca1d3a at mail.gmail.com>
	<bbaeab100911051535g68cefa18kdadea33f78de9495 at mail.gmail.com>
	<48c184130911051555r9a5b78cs3a13cb1345d3cbf7 at mail.gmail.com>
	<20091106001430.3229.796306500.divmod.xquotient.180 at localhost.localdomain>
	<4AF3CF69.6090709 at gmx.net>
	<20091106135058.GA13578 at vidar.dreamhost.com>
	<20091106140412.3229.80002680.divmod.xquotient.193 at localhost.localdomain>
	<68272E43-D993-4833-8DC9-DB45CB40F368 at twistedmatrix.com>
	<Pine.LNX.4.64.0911061647160.12274 at kimball.webabinitio.net>
	<20091106215210.3229.1142452245.divmod.xquotient.212 at localhost.localdomain>
X-Mailer: VM 8.0.12-devo-585 under 21.5  (beta29) "garbanzo" d20e0a45a4b2 XEmacs Lucid (x86_64-unknown-linux)
Date: Sat, 07 Nov 2009 17:35:21 +0900
Message-ID: <87aayyspfa.fsf at uwakimon.sk.tsukuba.ac.jp>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii

exarkun at twistedmatrix.com writes:

 > How would you collect this information?  Would you run the test suite 
 > for each project?  This would reward projects with small or absent test 
 > suites. ;)

Multiply by the number of lines of code and divide by the number of
tests run.

From glyph at twistedmatrix.com  Sat Nov  7 09:34:21 2009
From: glyph at twistedmatrix.com (Glyph Lefkowitz)
Date: Sat, 7 Nov 2009 03:34:21 -0500
Subject: [Python-Dev] Status of the Buildbot fleet and related bugs
In-Reply-To: <hd28f1$v2d$3@ger.gmane.org>
References: <Pine.LNX.4.64.0911052209000.12274@kimball.webabinitio.net>
	<hd28f1$v2d$3@ger.gmane.org>
Message-ID: <2D95753F-B50F-4F1B-9A8A-B8CA789AB148@twistedmatrix.com>

On Nov 6, 2009, at 6:34 PM, Georg Brandl wrote:

> R. David Murray schrieb:
>
>> So, overall I think the buildbot fleet is in good shape, and if
>> we can nail issue 6748 I think it will be back to being an
>> important resource for sanity checking our checkins.
>
> Yay! Thanks to all of you!

Indeed!  It's great to see so much work going into build and test  
maintenance.  Thanks a lot!


From glyph at twistedmatrix.com  Sat Nov  7 09:36:18 2009
From: glyph at twistedmatrix.com (Glyph Lefkowitz)
Date: Sat, 7 Nov 2009 03:36:18 -0500
Subject: [Python-Dev] PEP 3003 - Python Language Moratorium
In-Reply-To: <20091106215210.3229.1142452245.divmod.xquotient.212@localhost.localdomain>
References: <ca471dc20911030935l25251e0diec64427f5d7bcbf2@mail.gmail.com>
	<ca471dc20911051405o287e0f41m26d38860a200b47f@mail.gmail.com>
	<ea2499da0911051453n3569387aj581ad4acf5b799c8@mail.gmail.com>
	<4222a8490911051521r1b9c8165id6e0f12d62da0613@mail.gmail.com>
	<ca471dc20911051526u7f3642ecg19e4eb73bcca1d3a@mail.gmail.com>
	<bbaeab100911051535g68cefa18kdadea33f78de9495@mail.gmail.com>
	<48c184130911051555r9a5b78cs3a13cb1345d3cbf7@mail.gmail.com>
	<20091106001430.3229.796306500.divmod.xquotient.180@localhost.localdomain>
	<4AF3CF69.6090709@gmx.net>
	<20091106135058.GA13578@vidar.dreamhost.com>
	<20091106140412.3229.80002680.divmod.xquotient.193@localhost.localdomain>
	<68272E43-D993-4833-8DC9-DB45CB40F368@twistedmatrix.com>
	<Pine.LNX.4.64.0911061647160.12274@kimball.webabinitio.net>
	<20091106215210.3229.1142452245.divmod.xquotient.212@localhost.localdomain>
Message-ID: <4A329419-0784-4026-BF77-C13D8DB2A515@twistedmatrix.com>


On Nov 6, 2009, at 4:52 PM, exarkun at twistedmatrix.com wrote:

> On 09:48 pm, rdmurray at bitdance.com wrote:
>> On Fri, 6 Nov 2009 at 15:48, Glyph Lefkowitz wrote:
>>

>>> Documentation would be great, but then you have to get people to  
>>> read the documentation and that's kind of tricky.  Better would be  
>>> for every project on PyPI to have a score which listed warnings  
>>> emitted with each version of Python.  People love optimizing for  
>>> stuff like that and comparing it.
>>>
>>> I suspect that even if all warnings were completely silent by  
>>> default, developers would suddenly become keenly interested in  
>>> fixing them if there were a metric like that publicly posted  
>>> somewhere :).
>>
>> +1, but somebody needs to write the code...
>
> How would you collect this information?  Would you run the test  
> suite for each project?  This would reward projects with small or  
> absent test suites. ;)



*I* would not collect this information, as I am far enough behind on  
other projects ;-) but I if I were to advise someone *else* as to how  
to do it, I'd probably add a feature to the 'warnings' module where  
users could opt-in (sort of like popcon.debian.org) to report warnings  
encountered during normal invocations of any of their Python programs.

I would also advise such a hypothetical data-gathering project to  
start with a buildbot doing coverage runs; any warning during the test  
suite would be 1 demerit, any warning during an actual end-user run of  
the application *not* caught by the test suite would be 1000  
demerits :).

And actually it would make more sense if this were part of an overall  
quality metric, like http://pycheesecake.org/ proposes (although I  
think that cheesecake's current metric is not really that great, the  
idea is wonderful).


From baptiste.lepilleur at gmail.com  Sat Nov  7 11:51:58 2009
From: baptiste.lepilleur at gmail.com (Baptiste Lepilleur)
Date: Sat, 7 Nov 2009 11:51:58 +0100
Subject: [Python-Dev] Reworking the GIL
In-Reply-To: <loom.20091101T123110-331@post.gmane.org>
References: <1256502140.5621.163.camel@localhost>
	<bbaeab100910251457t64616f62x3216ebb4863223f5@mail.gmail.com> 
	<loom.20091101T123110-331@post.gmane.org>
Message-ID: <d14bb4710911070251u13b858d1mdc41e46aa46885fc@mail.gmail.com>

Hi Antoine,
I was finally able to compile py3k and run the benchmark (my compilation
issue was caused by checking out on Windows and compiling on Unix. Some
Makefile templates are missing correct EOL properties in SVN I think).

The benchmark results can be obtained from:
http://gaiacrtn.free.fr/py/benchmark-newgil/benchmark-newgil.tar.bz2
and viewed from:
http://gaiacrtn.free.fr/py/benchmark-newgil/

I ran the benchmark on two platforms:

   - Solaris X86, 16 cores: some python extension are likely missing (see
   config.log)
   - Windows XP SP3, 4 cores: all python extensions but TCL (I didn't bother
   checking why it failed as it is not used in the benchmark). It is a release
   build.

The results look promising but I let you share your conclusion (some latency
results seem a bit strange from my understanding).

Side-note: PCBuild requires nasmw.exe but it no longer exists in the latest
version. I had to rename nasm.exe to nasmw.exe. Would be nice to add this to
the readme to avoid confusion...

Baptiste.

2009/11/1 Antoine Pitrou <solipsis at pitrou.net>

>
> Hello again,
>
> Brett Cannon <brett <at> python.org> writes:
> >
> > I think it's worth it. Removal of the GIL is a totally open-ended problem
> > with no solution in sight. This, on the other hand, is a performance
> benefit
> > now. I say move forward with this. If it happens to be short-lived
> because
> > some actually figures out how to remove the GIL then great, but is that
> > really going to happen between now and Python 3.2? I doubt it.
>
> Based on this whole discussion, I think I am going to merge the new GIL
> work
> into the py3k branch, with priority requests disabled.
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20091107/18a39bdb/attachment-0001.htm>

From skip at pobox.com  Sat Nov  7 13:10:57 2009
From: skip at pobox.com (skip at pobox.com)
Date: Sat, 7 Nov 2009 06:10:57 -0600
Subject: [Python-Dev] PEP 3003 - Python Language Moratorium
In-Reply-To: <ca471dc20911061916h7d4a83a9qf66abaaca5480750@mail.gmail.com>
References: <ca471dc20911030935l25251e0diec64427f5d7bcbf2@mail.gmail.com>
	<ca471dc20911051405o287e0f41m26d38860a200b47f@mail.gmail.com>
	<ea2499da0911051453n3569387aj581ad4acf5b799c8@mail.gmail.com>
	<4222a8490911051521r1b9c8165id6e0f12d62da0613@mail.gmail.com>
	<ca471dc20911051526u7f3642ecg19e4eb73bcca1d3a@mail.gmail.com>
	<hd27t3$v2d$2@ger.gmane.org> <4AF4DE9B.6080703@gmail.com>
	<ca471dc20911061916h7d4a83a9qf66abaaca5480750@mail.gmail.com>
Message-ID: <19189.25553.111384.537110@montanaro.dyndns.org>

    Guido> ... it's IMO pretty mysterious if you encounter this and don't
    Guido> already happen to know what it means.

If you require parens maybe it parses better:

    import (a or b or c) as mod

Given that the or operator shortcuts I think that (a or b or c) terminates
once a module is found isn't too hard to grasp.

Skip

From exarkun at twistedmatrix.com  Sat Nov  7 14:44:46 2009
From: exarkun at twistedmatrix.com (exarkun at twistedmatrix.com)
Date: Sat, 07 Nov 2009 13:44:46 -0000
Subject: [Python-Dev] PEP 3003 - Python Language Moratorium
In-Reply-To: <19189.25553.111384.537110@montanaro.dyndns.org>
References: <ca471dc20911030935l25251e0diec64427f5d7bcbf2@mail.gmail.com>
	<ca471dc20911051405o287e0f41m26d38860a200b47f@mail.gmail.com>
	<ea2499da0911051453n3569387aj581ad4acf5b799c8@mail.gmail.com>
	<4222a8490911051521r1b9c8165id6e0f12d62da0613@mail.gmail.com>
	<ca471dc20911051526u7f3642ecg19e4eb73bcca1d3a@mail.gmail.com>
	<hd27t3$v2d$2@ger.gmane.org> <4AF4DE9B.6080703@gmail.com>
	<ca471dc20911061916h7d4a83a9qf66abaaca5480750@mail.gmail.com>
	<19189.25553.111384.537110@montanaro.dyndns.org>
Message-ID: <20091107134446.3229.1411095315.divmod.xquotient.283@localhost.localdomain>

On 12:10 pm, skip at pobox.com wrote:
>    Guido> ... it's IMO pretty mysterious if you encounter this and 
>don't
>    Guido> already happen to know what it means.
>
>If you require parens maybe it parses better:
>
>    import (a or b or c) as mod
>
>Given that the or operator shortcuts I think that (a or b or c) 
>terminates
>once a module is found isn't too hard to grasp.

Did everyone forget what the subject of this thread is?

Jean-Paul

From ssteinerx at gmail.com  Sat Nov  7 15:30:45 2009
From: ssteinerx at gmail.com (ssteinerX@gmail.com)
Date: Sat, 7 Nov 2009 09:30:45 -0500
Subject: [Python-Dev] People want CPAN :-)
In-Reply-To: <87iqdm7nkp.fsf@benfinney.id.au>
References: <ca471dc20911060953v1db91bbei992dcb964b6ab88b@mail.gmail.com>
	<a21398cfb92acc7d47f8151c249291c4@preisshare.net>
	<ca471dc20911061514q642ad41br9fb02d675fe411db@mail.gmail.com>
	<87iqdm7nkp.fsf@benfinney.id.au>
Message-ID: <B8D9CAC5-DA6D-47FC-AD1D-1FC388FBBC6B@gmail.com>

On Nov 7, 2009, at 3:20 AM, Ben Finney wrote:

> Guido van Rossum <guido at python.org> writes:
>
>> On Fri, Nov 6, 2009 at 2:52 PM, David Lyon  
>> <david.lyon at preisshare.net> wrote:
>>>
> I think buildbot-style test runs for PyPI packages would raise average
> package quality on PyPI.

Please excuse the cross-post but I wanted to make sure that all these  
"CPAN for Python" discussions got this message and I've lost track of  
which list which part of what discussion had occurred on.

We are currently extending our distutils/Distribute test system to  
include installation of a broad range of packages as part of the pre- 
release process for a future release of Distribute and as part of our  
"smoke" test for distutils/Distribute.    Eventually, the goal is to  
integrate this with our buildbot system but that's a ways off.

Our goal is to install a range of packages and, where practicable,  
actually run and record any errors with the packages' individual test  
suites.

Right now, our "smoke" test only does Twisted and numpy.  We've  
discussed how to collect test results from Twisted trial and we'll be  
working on similar things for other test runners (nose et al.).  For  
Twisted, we're going to install and test both the current release  
version and an svn checkout from trunk.

It would be an extension of that concept to install and test *all*  
packages from PyPI but would, obviously, take considerable horsepower  
(and time) to run such an exhaustive test (especially if we're talking  
about 2.4?, 2.5, 2.6, 2.7, and 3.1+.

Right now I'm extending the configuration file for our smoke test to  
allow for various test runners (e.g. nose, twisted trial, etc.) so we  
can "smoke out" more installation problems and/or failed tests after  
installation.

For the first pass, I'm just focusing on Twisted and trial, then  
numpy, then finding packages that support nose so that I can collect  
the data on what ran, what passed, and what didn't.  I'm planning on  
collecting this all in a database and making some simple API so that  
it can be mined by very simple apps later.

At the point where that infrastructure is in place, we could pretty  
easily mine the data to do all kinds of crazy things people have  
mentioned like:

	* A ranking system of test coverage
	* Complexity analysis
	* Test coverage
	* Run pylint, pyflakes, 2to3, whatever automated measurement tools  
over the code
	* Send test failure messages to maintainers (maybe with opt-in in the  
new meta-data).
	* Whatever!

We're actively working on this right now; anyone who wants to lend a  
hand is welcome to contact me off-list and we can talk about what  
types of things we are needing and where we could use a hand.

All in all, I think this could be a big leap forward for the Python  
distribution ecosystem whether or not we eventually write the PyPan I  
wished for as a new Perl refugee.

Thanks,

S


From solipsis at pitrou.net  Sat Nov  7 16:06:28 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Sat, 7 Nov 2009 15:06:28 +0000 (UTC)
Subject: [Python-Dev] Reworking the GIL
References: <1256502140.5621.163.camel@localhost>
	<bbaeab100910251457t64616f62x3216ebb4863223f5@mail.gmail.com>
	<loom.20091101T123110-331@post.gmane.org>
	<d14bb4710911070251u13b858d1mdc41e46aa46885fc@mail.gmail.com>
Message-ID: <loom.20091107T155305-553@post.gmane.org>


Hello,

> Solaris X86, 16 cores: some python extension are likely missing (see 
config.log)
> Windows XP SP3, 4 cores: all python extensions but TCL (I didn't bother
checking why it failed as it is not used in the benchmark). It is a release 
build.
> 
> The results look promising but I let you share your conclusion (some latency
results seem a bit strange from my understanding).

Thank you! The latency results don't look that strange to me.

If you're surprised that py3k shows better latency than newgil on the "pi
calculation" workload, it's easy to understand why: py3k speculatively releases
the GIL so often on that workload (every 100 opcodes) that the latencies are
indeed very good, but if you look at the corresponding throughput numbers they
are dismal (your Solaris box shows it falling to less than 20% with two threads
running compared to the baseline number for single-thread execution, and on your
Windows box the number is hardly better with 45%).

So, to sum it up, the way the current GIL manages to have good latencies is by
issueing an unreasonable number of system calls on a contended lock, and
potentially killing throughput performance (this depends on the OS too, because
numbers under Linux are not so catastrophic).

The new GIL, on the other hand, is much more balanced in that it achieves rather
predictable latencies (especially when you don't overcommit the OS by issueing
more computational threads than you have CPU cores) while preserving throughput
performance.

> Side-note: PCBuild requires nasmw.exe but it no longer exists in the latest
> version. I had to rename nasm.exe to nasmw.exe. Would be nice to add this to
> the readme to avoid confusion...

You should file a bug on http://bugs.python.org

Regards

Antoine.



From jnoller at gmail.com  Sat Nov  7 16:08:55 2009
From: jnoller at gmail.com (Jesse Noller)
Date: Sat, 7 Nov 2009 10:08:55 -0500
Subject: [Python-Dev] People want CPAN :-)
In-Reply-To: <B8D9CAC5-DA6D-47FC-AD1D-1FC388FBBC6B@gmail.com>
References: <ca471dc20911060953v1db91bbei992dcb964b6ab88b@mail.gmail.com>
	<a21398cfb92acc7d47f8151c249291c4@preisshare.net>
	<ca471dc20911061514q642ad41br9fb02d675fe411db@mail.gmail.com>
	<87iqdm7nkp.fsf@benfinney.id.au>
	<B8D9CAC5-DA6D-47FC-AD1D-1FC388FBBC6B@gmail.com>
Message-ID: <4222a8490911070708j70d06be0j19a598fdddcee459@mail.gmail.com>

On Sat, Nov 7, 2009 at 9:30 AM, ssteinerX at gmail.com <ssteinerx at gmail.com> wrote:
> On Nov 7, 2009, at 3:20 AM, Ben Finney wrote:
>
>> Guido van Rossum <guido at python.org> writes:
>>
>>> On Fri, Nov 6, 2009 at 2:52 PM, David Lyon <david.lyon at preisshare.net>
>>> wrote:
>>>>
>> I think buildbot-style test runs for PyPI packages would raise average
>> package quality on PyPI.
>
> Please excuse the cross-post but I wanted to make sure that all these "CPAN
> for Python" discussions got this message and I've lost track of which list
> which part of what discussion had occurred on.
>
> We are currently extending our distutils/Distribute test system to include
> installation of a broad range of packages as part of the pre-release process
> for a future release of Distribute and as part of our "smoke" test for
> distutils/Distribute. ? ?Eventually, the goal is to integrate this with our
> buildbot system but that's a ways off.

Who is "we"?

From ssteinerx at gmail.com  Sat Nov  7 16:10:59 2009
From: ssteinerx at gmail.com (ssteinerX@gmail.com)
Date: Sat, 7 Nov 2009 10:10:59 -0500
Subject: [Python-Dev] People want CPAN :-)
In-Reply-To: <4222a8490911070708j70d06be0j19a598fdddcee459@mail.gmail.com>
References: <ca471dc20911060953v1db91bbei992dcb964b6ab88b@mail.gmail.com>
	<a21398cfb92acc7d47f8151c249291c4@preisshare.net>
	<ca471dc20911061514q642ad41br9fb02d675fe411db@mail.gmail.com>
	<87iqdm7nkp.fsf@benfinney.id.au>
	<B8D9CAC5-DA6D-47FC-AD1D-1FC388FBBC6B@gmail.com>
	<4222a8490911070708j70d06be0j19a598fdddcee459@mail.gmail.com>
Message-ID: <1293335C-B422-472E-ACD8-4E46A756A9C8@gmail.com>


On Nov 7, 2009, at 10:08 AM, Jesse Noller wrote:

> On Sat, Nov 7, 2009 at 9:30 AM, ssteinerX at gmail.com <ssteinerx at gmail.com 
> > wrote:
>> On Nov 7, 2009, at 3:20 AM, Ben Finney wrote:
>>
>>> Guido van Rossum <guido at python.org> writes:
>>>
>>>> On Fri, Nov 6, 2009 at 2:52 PM, David Lyon <david.lyon at preisshare.net 
>>>> >
>>>> wrote:
>>>>>
>>> I think buildbot-style test runs for PyPI packages would raise  
>>> average
>>> package quality on PyPI.
>>
>> Please excuse the cross-post but I wanted to make sure that all  
>> these "CPAN
>> for Python" discussions got this message and I've lost track of  
>> which list
>> which part of what discussion had occurred on.
>>
>> We are currently extending our distutils/Distribute test system to  
>> include
>> installation of a broad range of packages as part of the pre- 
>> release process
>> for a future release of Distribute and as part of our "smoke" test  
>> for
>> distutils/Distribute.    Eventually, the goal is to integrate this  
>> with our
>> buildbot system but that's a ways off.
>
> Who is "we"?


We is the people working on Distribute/distutils.

S



From guido at python.org  Sat Nov  7 17:06:37 2009
From: guido at python.org (Guido van Rossum)
Date: Sat, 7 Nov 2009 08:06:37 -0800
Subject: [Python-Dev] Reworking the GIL
In-Reply-To: <loom.20091107T155305-553@post.gmane.org>
References: <1256502140.5621.163.camel@localhost>
	<bbaeab100910251457t64616f62x3216ebb4863223f5@mail.gmail.com> 
	<loom.20091101T123110-331@post.gmane.org>
	<d14bb4710911070251u13b858d1mdc41e46aa46885fc@mail.gmail.com> 
	<loom.20091107T155305-553@post.gmane.org>
Message-ID: <ca471dc20911070806m244fb9f4j55a332ef25cda551@mail.gmail.com>

Antoine,

How close are you to merging this into the Py3k branch? It looks like
a solid piece of work, that can only get better in the period between
now and the release of 3.2. But I don't want to rush you, and I only
have had a brief look at your code. (I whipped up a small Dave Beazley
example and was impressed by the performance of your code compared to
the original py3k branch -- from 150% to 100% CPU usage according to
top with only 2 threads.)

My only suggestion so far: the description could use more explicit
documentation on the various variables and macros and how they
combine.

I also expect that priority requests aren't important; it somehow
seems strange that if multiple threads are all doing I/O each new
thread whose I/O completes would get to preempt whoever else is active
immediately. (Also the choice of *not* making a priority request when
a read returns no bytes seems strange 00 if I read the code
correctly.)

Anyway, thanks for this work!

-- 
--Guido van Rossum (python.org/~guido)

From g.brandl at gmx.net  Sat Nov  7 18:30:26 2009
From: g.brandl at gmx.net (Georg Brandl)
Date: Sat, 07 Nov 2009 18:30:26 +0100
Subject: [Python-Dev] PEP 3003 - Python Language Moratorium
In-Reply-To: <20091107134446.3229.1411095315.divmod.xquotient.283@localhost.localdomain>
References: <ca471dc20911030935l25251e0diec64427f5d7bcbf2@mail.gmail.com>	<ca471dc20911051405o287e0f41m26d38860a200b47f@mail.gmail.com>	<ea2499da0911051453n3569387aj581ad4acf5b799c8@mail.gmail.com>	<4222a8490911051521r1b9c8165id6e0f12d62da0613@mail.gmail.com>	<ca471dc20911051526u7f3642ecg19e4eb73bcca1d3a@mail.gmail.com>	<hd27t3$v2d$2@ger.gmane.org>
	<4AF4DE9B.6080703@gmail.com>	<ca471dc20911061916h7d4a83a9qf66abaaca5480750@mail.gmail.com>	<19189.25553.111384.537110@montanaro.dyndns.org>
	<20091107134446.3229.1411095315.divmod.xquotient.283@localhost.localdomain>
Message-ID: <hd47gj$k5s$2@ger.gmane.org>

exarkun at twistedmatrix.com schrieb:
> On 12:10 pm, skip at pobox.com wrote:
>>    Guido> ... it's IMO pretty mysterious if you encounter this and 
>>don't
>>    Guido> already happen to know what it means.
>>
>>If you require parens maybe it parses better:
>>
>>    import (a or b or c) as mod
>>
>>Given that the or operator shortcuts I think that (a or b or c) 
>>terminates
>>once a module is found isn't too hard to grasp.
> 
> Did everyone forget what the subject of this thread is?

Why?  The subject of this thread is "Importing module alternatives cleanly".
We just happened to forget adapting the "Subject:" line.

Georg

-- 
Thus spake the Lord: Thou shalt indent with four spaces. No more, no less.
Four shall be the number of spaces thou shalt indent, and the number of thy
indenting shall be four. Eight shalt thou not indent, nor either indent thou
two, excepting that thou then proceed to four. Tabs are right out.


From solipsis at pitrou.net  Sat Nov  7 18:01:57 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Sat, 07 Nov 2009 18:01:57 +0100
Subject: [Python-Dev] Reworking the GIL
In-Reply-To: <ca471dc20911070806m244fb9f4j55a332ef25cda551@mail.gmail.com>
References: <1256502140.5621.163.camel@localhost>
	<bbaeab100910251457t64616f62x3216ebb4863223f5@mail.gmail.com>
	<loom.20091101T123110-331@post.gmane.org>
	<d14bb4710911070251u13b858d1mdc41e46aa46885fc@mail.gmail.com>
	<loom.20091107T155305-553@post.gmane.org>
	<ca471dc20911070806m244fb9f4j55a332ef25cda551@mail.gmail.com>
Message-ID: <1257613317.3437.63.camel@localhost>


Hello Guido,

> How close are you to merging this into the Py3k branch? It looks like
> a solid piece of work, that can only get better in the period between
> now and the release of 3.2. But I don't want to rush you, and I only
> have had a brief look at your code.

The code is ready. Priority requests are already disabled, I'm just
wondering whether to remove them from the code, or leave them there in
case someone thinks they're useful. I suppose removing them is ok.

> My only suggestion so far: the description could use more explicit
> documentation on the various variables and macros and how they
> combine.

Is it before or after
http://mail.python.org/pipermail/python-checkins/2009-November/087482.html ?

Regards

Antoine.



From guido at python.org  Sat Nov  7 20:33:13 2009
From: guido at python.org (Guido van Rossum)
Date: Sat, 7 Nov 2009 11:33:13 -0800
Subject: [Python-Dev] Reworking the GIL
In-Reply-To: <1257613317.3437.63.camel@localhost>
References: <1256502140.5621.163.camel@localhost>
	<bbaeab100910251457t64616f62x3216ebb4863223f5@mail.gmail.com> 
	<loom.20091101T123110-331@post.gmane.org>
	<d14bb4710911070251u13b858d1mdc41e46aa46885fc@mail.gmail.com> 
	<loom.20091107T155305-553@post.gmane.org>
	<ca471dc20911070806m244fb9f4j55a332ef25cda551@mail.gmail.com> 
	<1257613317.3437.63.camel@localhost>
Message-ID: <ca471dc20911071133i56098f5cxf5639554c82e11fc@mail.gmail.com>

On Sat, Nov 7, 2009 at 9:01 AM, Antoine Pitrou <solipsis at pitrou.net> wrote:
>
> Hello Guido,
>
>> How close are you to merging this into the Py3k branch? It looks like
>> a solid piece of work, that can only get better in the period between
>> now and the release of 3.2. But I don't want to rush you, and I only
>> have had a brief look at your code.
>
> The code is ready. Priority requests are already disabled, I'm just
> wondering whether to remove them from the code, or leave them there in
> case someone thinks they're useful. I suppose removing them is ok.

I would remove them -- if and when we find there's a need for
something like them I suspect it won't look like what you currently
have, so it's better not to complexificate your current patch with
them. (I would remove all traces, including the conditions passed into
the end macros.)

>> My only suggestion so far: the description could use more explicit
>> documentation on the various variables and macros and how they
>> combine.
>
> Is it before or after
> http://mail.python.org/pipermail/python-checkins/2009-November/087482.html ?

After. While that is already really helpful, not all the code is
easily linked back to paragraphs in that comment block, and some
variables are not mentioned by name in the block.

-- 
--Guido van Rossum (python.org/~guido)

From baptiste.lepilleur at gmail.com  Sat Nov  7 20:37:30 2009
From: baptiste.lepilleur at gmail.com (Baptiste Lepilleur)
Date: Sat, 7 Nov 2009 20:37:30 +0100
Subject: [Python-Dev] Reworking the GIL
In-Reply-To: <loom.20091107T155305-553@post.gmane.org>
References: <1256502140.5621.163.camel@localhost>
	<bbaeab100910251457t64616f62x3216ebb4863223f5@mail.gmail.com> 
	<loom.20091101T123110-331@post.gmane.org>
	<d14bb4710911070251u13b858d1mdc41e46aa46885fc@mail.gmail.com> 
	<loom.20091107T155305-553@post.gmane.org>
Message-ID: <d14bb4710911071137y4c06e886g53a72236b171a216@mail.gmail.com>

2009/11/7 Antoine Pitrou <solipsis at pitrou.net>

>
> [...]
> So, to sum it up, the way the current GIL manages to have good latencies is
> by
> issueing an unreasonable number of system calls on a contended lock, and
> potentially killing throughput performance (this depends on the OS too,
> because
> numbers under Linux are not so catastrophic).
>
Ah, I remember reading this in the analysis that was published now!

I made another benchmark using one of my script I ported to python 3 (and
simplified a bit). I only test the total execution time. Tests done on
Windows XP SP3. Processor is an Intel Core 2 Quad Q9300 (4 cores).

You can get the script from:
http://gaiacrtn.free.fr/py/benchmark-kwd-newgil/purekeyword-py26-3k.py
Script + test doc (940KB):
http://gaiacrtn.free.fr/py/benchmark-kwd-newgil/benchmark-kwd-newgil.tar.bz2

The threaded loop is:
for match in self.punctuation_pattern.finditer( document ):
    word = document[last_start_index:match.start()]
    if len(word) > 1 and len(word) < MAX_KEYWORD_LENGTH:
        words[word] = words.get(word, 0) + 1
    last_start_index = match.end()
    if word:
        word_count += 1

Here are the results:

-j0 (main thread only)
2.5.2 : 17.991s, 17.947s, 17.780s
2.6.2 : 19.071s, 19.023s, 19.054s
3.1.1 : 46.384s, 46.321s, 46.425s
newgil: 47.483s, 47.605s, 47.512s

-j4 (4 consumer threads, main thread producing/waiting)
2.5.2 : 31.105s, 30.568s
2.6.2 : 31.550s, 30.599s
3.1.1 : 85.114s, 85.185s
newgil: 48.428s, 49.217s

It shows that, on my platform for this specific benchmark:

   -  newgil manage to leverage a significant amount of parallelism (1.7)
   where python 3.1 does not (3.1 is 80% slower)
   - newgil as a small impact on non multi-threaded execution (~1-2%) [may
   be worth investigating]
   - 3.1 is more than 2 times slower than python 2.6 on this benchmark
   - 2.6 is "only" 65% slower when run with multiple threads compared to the
   80% slower of 3.1.

Newgil is a vast improvement as it manages to leverage the short time the
GIL is released by finditer [if I understood correctly in 3.x regex release
the GIL].

What's worry me is the single threaded performance degradation between 2.6
and 3.1 on this test. Could the added GIL release/acquire on each finditer
call explain this?
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20091107/68e38500/attachment-0001.htm>

From solipsis at pitrou.net  Sat Nov  7 20:56:00 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Sat, 07 Nov 2009 20:56:00 +0100
Subject: [Python-Dev] Reworking the GIL
In-Reply-To: <d14bb4710911071137y4c06e886g53a72236b171a216@mail.gmail.com>
References: <1256502140.5621.163.camel@localhost>
	<bbaeab100910251457t64616f62x3216ebb4863223f5@mail.gmail.com>
	<loom.20091101T123110-331@post.gmane.org>
	<d14bb4710911070251u13b858d1mdc41e46aa46885fc@mail.gmail.com>
	<loom.20091107T155305-553@post.gmane.org>
	<d14bb4710911071137y4c06e886g53a72236b171a216@mail.gmail.com>
Message-ID: <1257623760.3437.72.camel@localhost>


Hello again,

> It shows that, on my platform for this specific benchmark:
>       *  newgil manage to leverage a significant amount of parallelism
>         (1.7) where python 3.1 does not (3.1 is 80% slower)

I think you are mistaken:

-j0 (main thread only)
newgil: 47.483s, 47.605s, 47.512s
-j4 (4 consumer threads, main thread producing/waiting)
newgil: 48.428s, 49.217s

The runtimes are actually the same, so newgil doesn't leverage anything.
However, it doesn't degrade performance like 2.x/3.1 does :-)

>       * newgil as a small impact on non multi-threaded execution
>         (~1-2%) [may be worth investigating]

It goes from very slightly slower to very slightly faster and it is
likely to be caused by variations in generated output from the compiler.

>       * 3.1 is more than 2 times slower than python 2.6 on this
>         benchmark

That's the most worrying outcome I'd say. Are you sure the benchmark
really does the same thing? Under 2.6, you should add re.UNICODE to the
regular expression flags so as to match the 3.x semantics.

> [if I understood correctly in 3.x regex release the GIL].

Unless I've missed something it doesn't, no.
This could be a separate opportunity for optimization, if someone wants
to take a look at it.

Regards

Antoine.



From steve at pearwood.info  Sun Nov  8 01:14:59 2009
From: steve at pearwood.info (Steven D'Aprano)
Date: Sun, 8 Nov 2009 11:14:59 +1100
Subject: [Python-Dev] PEP 3003 - Python Language Moratorium
In-Reply-To: <ca471dc20911051405o287e0f41m26d38860a200b47f@mail.gmail.com>
References: <ca471dc20911030935l25251e0diec64427f5d7bcbf2@mail.gmail.com>
	<ca471dc20911051405o287e0f41m26d38860a200b47f@mail.gmail.com>
Message-ID: <200911081115.01803.steve@pearwood.info>

On Fri, 6 Nov 2009 09:05:17 am Guido van Rossum wrote:
> On Tue, Nov 3, 2009 at 9:35 AM, Guido van Rossum <guido at python.org> 
wrote:
> > I've checked draft (!) PEP 3003, "Python Language Moratorium", into
> > SVN. As authors I've listed Jesse, Brett and myself.
>
> I haven't seen substantial opposition against the PEP -- in fact I
> can't recall any, and many people have explicitly posted in support
> of it. So unless opposition suddenly appears in the next few days,
> I'll move it to the Accepted state next Monday.

I'm not a core developer, but I have made some objections on the 
python-ideas list. For what it is worth, I don't think a moratorium is 
needed, and it is not cost free. Whether the cost is worth it is a 
matter of opinion, but I've read posts suggesting that there's no 
downside to a moratorium. I think that's naive.

Firstly, the core developers are very conservative in the features they 
add to the language. Nobody suggests that will change -- there are (as 
far as I know) a grand total of *one* proposed language changes under 
consideration for 3.2, namely "yield from". It isn't like the 
moratorium is needed to stem a flood of vast numbers of incompatible 
language changes, and this is not likely to change.

The given reason for the moratorium is to give the developers of other 
Python implementations a chance to catch up. But as far as I know -- 
and I welcome correction -- only one such developer has spoken up, and 
he says that new language features generally aren't a problem for 
IronPython, but some standard library features are:

http://mail.python.org/pipermail/python-ideas/2009-October/006328.html

(In fairness, I should point out that Dino did not vote against the 
moratorium, he gave a conditional "in favour" vote but asked for some 
wiggle-room. A moratorium with wiggle-room is not a moratorium.)

A moratorium isn't cost-free. With the back-end free to change, patches 
will go stale over 2+ years. People will lose interest or otherwise 
move on. Those with good ideas but little patience will be discouraged. 
I fully expect that, human nature being as it is, those proposing a 
change, good or bad, will be told not to bother wasting their time, 
there's a moratorium on at least as often as they'll be encouraged to 
bide their time while the moratorium is on.

A moratorium turns Python's conservativeness up to 11. If Python already 
has a reputation for being conservative in the features it accepts -- 
and I think it does -- then a moratorium risks giving the impression 
that Python has become the language of choice for old guys sitting on 
their porch yelling at the damn kids to get off the lawn. That's a plus 
for Cobol. I don't think it is a plus for Python.

What one person sees as "stable", another may see as "moribund". 
Perception does matter. Nick Coglan spoke of Jython having "fell by the 
wayside" when CPython introduced new style classes:

http://mail.python.org/pipermail/python-ideas/2009-October/006431.html

I think that speaks to the opposite conclusion that Nick intended. If 
developers craved stability, and were put off by the pace of changes to 
CPython, why weren't they migrating to Jython, which had one new 
production release in six years?

http://www.jython.org/archive/22/news.html

Nick also described C as changing much more slowly over its life that 
Python has. This is misleading: it is true that the C standard has been 
stable, but that doesn't mean that C compilers have been. Changes to 
the standard were driven by changes introduced by the implementations, 
not visa versa. Likewise for Cobol, where the first implementation to 
introduce OOP was released in 1997, but the standard wasn't updated 
until 2002.

Python is nothing like that. Python doesn't have an official standard, 
and Guido has dismissed the need for an ISO standard. The other 
implementations have historically let CPython lead as far as language 
evolution goes. If CPython stops, likely Python stops. Who is going to 
risk adding language features that break compatibility with the most 
popular implementation? It won't be Python anymore.

The PEP says the moratorium will last "at least" two years from the 
release of 3.1. Given the normal slow pace of new language features, 
two years isn't bad -- that's effectively just 3.2. But Guido has 
suggested it could last to 3.3. Could it last beyond that? 3.4? Until 
some distant future Python-4000?

At the very least, I believe, any moratorium should have a clear end 
date. A clear end date will be a powerful counter to the impression 
that Python the language is moribund. It says, this is an exceptional 
pause, not a permanent halt.




-- 
Steven D'Aprano

From steve at pearwood.info  Sun Nov  8 01:16:47 2009
From: steve at pearwood.info (Steven D'Aprano)
Date: Sun, 8 Nov 2009 11:16:47 +1100
Subject: [Python-Dev] PEP 3003 - Python Language Moratorium
In-Reply-To: <f6bc9b490911060146k39b84f4sbe9f0fe41667e83d@mail.gmail.com>
References: <ca471dc20911030935l25251e0diec64427f5d7bcbf2@mail.gmail.com>
	<200911061618.38790.steve@pearwood.info>
	<f6bc9b490911060146k39b84f4sbe9f0fe41667e83d@mail.gmail.com>
Message-ID: <200911081116.47419.steve@pearwood.info>

On Fri, 6 Nov 2009 08:46:00 pm Willem Broekema wrote:

> CLPython is in steady development, quite complete and stable on the
> language level (somewhere between 2.5 and 2.6), but missing most
> built-in library functionality. (It reuses the pure-Python parts of
> the stdlib.)
>
> As its developer, I don't care much about being mentioned in
> discussions or presentations, as CLPython is not as widely used as
> IronPython and Jython. But in a PEP like this, which is directly
> about alternative implementations, it deserves to be mentioned, IMHO.

Willem, the rationale for this PEP is to give alternative 
implementations the chance to catch up with CPython.

Given your statement that CLPython is quite complete on the language 
level, but missing standard library features, how do you think the 
moratorium will help CLPython?



-- 
Steven D'Aprano

From john.arbash.meinel at gmail.com  Sun Nov  8 01:39:18 2009
From: john.arbash.meinel at gmail.com (John Arbash Meinel)
Date: Sat, 07 Nov 2009 18:39:18 -0600
Subject: [Python-Dev] PEP 3003 - Python Language Moratorium
In-Reply-To: <200911081115.01803.steve@pearwood.info>
References: <ca471dc20911030935l25251e0diec64427f5d7bcbf2@mail.gmail.com>	<ca471dc20911051405o287e0f41m26d38860a200b47f@mail.gmail.com>
	<200911081115.01803.steve@pearwood.info>
Message-ID: <4AF61336.5010004@gmail.com>


...

> A moratorium isn't cost-free. With the back-end free to change, patches 
> will go stale over 2+ years. People will lose interest or otherwise 
> move on. Those with good ideas but little patience will be discouraged. 
> I fully expect that, human nature being as it is, those proposing a 
> change, good or bad, will be told not to bother wasting their time, 
> there's a moratorium on at least as often as they'll be encouraged to 
> bide their time while the moratorium is on.
> 

I believe if you go back to the very beginning of this thread, Guido
considers this a *feature* not a *bug*.

He wanted to introduce a moratorium at least partially because he was
tired of endless threads about anonymous code blocks, etc. Which aren't
going to be included in the language anyway, so he may as well make a
point to say "and neither will anything else for a while".

I don't mean to put words into his mouth, so please correct me if I'm wrong.

John
=:->


From metawilm at gmail.com  Sun Nov  8 02:03:50 2009
From: metawilm at gmail.com (Willem Broekema)
Date: Sun, 8 Nov 2009 02:03:50 +0100
Subject: [Python-Dev] PEP 3003 - Python Language Moratorium
In-Reply-To: <200911081116.47419.steve@pearwood.info>
References: <ca471dc20911030935l25251e0diec64427f5d7bcbf2@mail.gmail.com>
	<200911061618.38790.steve@pearwood.info>
	<f6bc9b490911060146k39b84f4sbe9f0fe41667e83d@mail.gmail.com>
	<200911081116.47419.steve@pearwood.info>
Message-ID: <f6bc9b490911071703x72e9be74v76b22d05539ed6ad@mail.gmail.com>

On Sun, Nov 8, 2009 at 1:16 AM, Steven D'Aprano <steve at pearwood.info> wrote:
> Willem, the rationale for this PEP is to give alternative
> implementations the chance to catch up with CPython.
>
> Given your statement that CLPython is quite complete on the language
> level, but missing standard library features, how do you think the
> moratorium will help CLPython?

It would be great to have a pause in Python-the-language, so that
CLPython might at the end of the moratorium finally be up to date with
the latest 2.x and 3.x features.

Supporting standard libraries is a bigger challenge, as the ones
written in C need rewriting. Whether or not there is a pause on that
front does not matter too much to that situation.

- Willem

From baptiste.lepilleur at gmail.com  Sun Nov  8 15:07:22 2009
From: baptiste.lepilleur at gmail.com (Baptiste Lepilleur)
Date: Sun, 8 Nov 2009 15:07:22 +0100
Subject: [Python-Dev] Reworking the GIL
In-Reply-To: <1257623760.3437.72.camel@localhost>
References: <1256502140.5621.163.camel@localhost>
	<bbaeab100910251457t64616f62x3216ebb4863223f5@mail.gmail.com> 
	<loom.20091101T123110-331@post.gmane.org>
	<d14bb4710911070251u13b858d1mdc41e46aa46885fc@mail.gmail.com> 
	<loom.20091107T155305-553@post.gmane.org>
	<d14bb4710911071137y4c06e886g53a72236b171a216@mail.gmail.com> 
	<1257623760.3437.72.camel@localhost>
Message-ID: <d14bb4710911080607l27567f12x72480014bfc07ab4@mail.gmail.com>

2009/11/7 Antoine Pitrou <solipsis at pitrou.net>

>
> Hello again,
>
> > It shows that, on my platform for this specific benchmark:
> >       *  newgil manage to leverage a significant amount of parallelism
> >         (1.7) where python 3.1 does not (3.1 is 80% slower)
>
> I think you are mistaken:
>
> -j0 (main thread only)
> newgil: 47.483s, 47.605s, 47.512s
> -j4 (4 consumer threads, main thread producing/waiting)
> newgil: 48.428s, 49.217s
>
> The runtimes are actually the same, so newgil doesn't leverage anything.
> However, it doesn't degrade performance like 2.x/3.1 does :-)
>

Ooops, I was comparing to 3.1 -j4 times which make no sense. One would think
I wanted to see that result since I though the GIL was released :/. This
greatly reduce the interest of this benchmark...


> >       * 3.1 is more than 2 times slower than python 2.6 on this
> >         benchmark
>
> That's the most worrying outcome I'd say. Are you sure the benchmark
> really does the same thing? Under 2.6, you should add re.UNICODE to the
> regular expression flags so as to match the 3.x semantics.
>

I've tried, but there is no change in result (the regexp does not use \w &
co but specify a lot unicode ranges). All strings are already of unicode
type in 2.6.


>  > [if I understood correctly in 3.x regex release the GIL].
>
> Unless I've missed something it doesn't, no.
>
Hmmm, I was confusing with other modules (bzip2 & hashlib?). Looking back at
the result of your benchmark it's obvious. Is there a place where the list
of functions releasing the GIL is available? I did not see anything in
bz2.compress documentation.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20091108/2c3ea338/attachment.htm>

From solipsis at pitrou.net  Sun Nov  8 15:17:48 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Sun, 8 Nov 2009 14:17:48 +0000 (UTC)
Subject: [Python-Dev] Reworking the GIL
References: <1256502140.5621.163.camel@localhost>
	<bbaeab100910251457t64616f62x3216ebb4863223f5@mail.gmail.com>
	<loom.20091101T123110-331@post.gmane.org>
	<d14bb4710911070251u13b858d1mdc41e46aa46885fc@mail.gmail.com>
	<loom.20091107T155305-553@post.gmane.org>
	<d14bb4710911071137y4c06e886g53a72236b171a216@mail.gmail.com>
	<1257623760.3437.72.camel@localhost>
	<d14bb4710911080607l27567f12x72480014bfc07ab4@mail.gmail.com>
Message-ID: <loom.20091108T151532-103@post.gmane.org>

Baptiste Lepilleur <baptiste.lepilleur <at> gmail.com> writes:
> 
> I've tried, but there is no change in result (the regexp does not use \w &
> co but specify a lot unicode ranges). All strings are already of unicode
> type in 2.6.

No they aren't. You should add "from __future__ import unicode_literals" at the
start of your script and run it again.

> Hmmm, I was confusing with other modules (bzip2 & hashlib?). Looking back at 
> the result of your benchmark it's obvious. Is there a place where the list of 
> functions releasing the GIL is available? I did not see anything in 
> bz2.compress documentation.

No, there isn't. You'd have to test, or read the source code.
But bz2 and zlib, for example, do release the GIL.

Regards

Antoine.



From stefan-usenet at bytereef.org  Sun Nov  8 17:26:48 2009
From: stefan-usenet at bytereef.org (Stefan Krah)
Date: Sun, 8 Nov 2009 17:26:48 +0100
Subject: [Python-Dev] decimal.py: == and != comparisons involving NaNs
In-Reply-To: <1257672530.83.0.836585441467.issue7279@psf.upfronthosting.co.za>
References: <1257590632.12.0.893741146745.issue7279@psf.upfronthosting.co.za>
	<1257672530.83.0.836585441467.issue7279@psf.upfronthosting.co.za>
Message-ID: <20091108162648.GA4099@mail.bytereef.org>

Hi,

in a (misguided) bugreport (http://bugs.python.org/issue7279) I was
questioning the reasons for allowing NaN comparisons with == and !=
rather than raising InvalidOperation.


I think two main issues emerge from the brief discussion:

1. Should the comparison operators follow the 'compare' function
   from the standard?

The standard says:

"An implementation may use this operation [compare] under the covers to
 implement a closed set of comparison operations (greater than, equal, etc.)
 if desired. It need not, in this case, expose the compare operation
 itself."

So, I'd say that this supports following 'compare' as closely as possible.
In fact the operators <, <=, >, >= already follow 'compare-signal' in
their behavior (they raise for any NaN operand).


2. What is the use of == and != outside the decimal scope?

Mark mentions that Python uses == to test for set and dict memberships,
but that you cannot put decimal NaNs into sets:

'TypeError: Cannot hash a NaN value'


I want to add that Decimal('NaN') == Decimal('NaN') gives False, which
should somewhat limit the uses of == for NaNs outside the decimal realm
anyway.


Are there cases where == and != are actually needed to give a result
for NaNs?


Stefan Krah




From martin at v.loewis.de  Sun Nov  8 19:44:42 2009
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sun, 08 Nov 2009 19:44:42 +0100
Subject: [Python-Dev] Status of the Buildbot fleet and related bugs
In-Reply-To: <hd28f1$v2d$3@ger.gmane.org>
References: <Pine.LNX.4.64.0911052209000.12274@kimball.webabinitio.net>
	<hd28f1$v2d$3@ger.gmane.org>
Message-ID: <4AF7119A.2040201@v.loewis.de>

> JFTR, I didn't set up the IRC bot (I assume that credit goes to Martin,
> even if it's only one line in the buildbot config :).  I just tried to
> get it to say something :)

Yes, it was always "on". I don't use IRC regularly, so I don't know
whether it's useful.

Regards,
Martin

From python at rcn.com  Sun Nov  8 19:53:10 2009
From: python at rcn.com (Raymond Hettinger)
Date: Sun, 8 Nov 2009 10:53:10 -0800
Subject: [Python-Dev] decimal.py: == and != comparisons involving NaNs
References: <1257590632.12.0.893741146745.issue7279@psf.upfronthosting.co.za><1257672530.83.0.836585441467.issue7279@psf.upfronthosting.co.za>
	<20091108162648.GA4099@mail.bytereef.org>
Message-ID: <4FC3B5BCE9034A79AFB0530C748F019E@RaymondLaptop1>


[Stefan Krah]
> in a (misguided) bugreport (http://bugs.python.org/issue7279) I was
> questioning the reasons for allowing NaN comparisons with == and !=
> rather than raising InvalidOperation.

Do you have any actual use case issues or are these theoretical musings?
I ask only because a good use case might suggest the best way to adapt
the standard to the regular python api for equality/inequality operators.

NaNs are odd ducks.  They are unique in violating our basic notions
of equality (any relation that is reflexsive, symmetric, and transitive).
Once you use them in a context that goes beyond the decimal spec,
it is no surprise that you run into difficulties where NaNs don't fit
very well (because they violate basic assumptions and invariants in other code).


> Are there cases where == and != are actually needed to give a result
> for NaNs?

I would say that anywhere someone needs the full behaviors specified by
the standard, they need to use the actual compare() method which
allows for a decimal context to be specified and allows for more than
just a true/false return value (i.e. a NaN is a possible result).


Raymond

From martin at v.loewis.de  Sun Nov  8 19:56:23 2009
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sun, 08 Nov 2009 19:56:23 +0100
Subject: [Python-Dev] OpenSSL vulnerability
In-Reply-To: <ca471dc20911061512j1f5f7e45nb5fd2de0d2734d45@mail.gmail.com>
References: <ca471dc20911061222v5c380643te7d5a4e84d3496f9@mail.gmail.com>
	<40373.1257545902@parc.com>	<20091106223634.3229.2049231320.divmod.xquotient.232@localhost.localdomain>
	<ca471dc20911061512j1f5f7e45nb5fd2de0d2734d45@mail.gmail.com>
Message-ID: <4AF71457.3050705@v.loewis.de>

>> Also, for Python 2.5 and earlier, any SSL-based code is vulnerable to a MitM
>> anyway, so this can only be an issue for code using the new APIs in Python
>> 2.6.
> 
> That's not going to stop the
> wannabe-self-proclaimed-so-called-vulnerability-"experts" from whining
> about Python not releasing updated binary distributions though. :-(

The Windows binaries currently build with 0.9.8g. Since changing that
would be a source code change (even though just a single line), I think
a full source release would be necessary (most likely then for both 2.6
and 3.1).

Regards,
Martin

From solipsis at pitrou.net  Sun Nov  8 20:01:01 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Sun, 8 Nov 2009 19:01:01 +0000 (UTC)
Subject: [Python-Dev] decimal.py: == and != comparisons involving NaNs
References: <1257590632.12.0.893741146745.issue7279@psf.upfronthosting.co.za>
	<1257672530.83.0.836585441467.issue7279@psf.upfronthosting.co.za>
	<20091108162648.GA4099@mail.bytereef.org>
Message-ID: <loom.20091108T195834-29@post.gmane.org>

Stefan Krah <stefan-usenet <at> bytereef.org> writes:
> 
> Are there cases where == and != are actually needed to give a result
> for NaNs?

It is a common expectation that == and != always succeed. They return True or
False, but don't raise an exception even on unrelated operands:

>>> b"a" == "a"
False
>>> "5" == 5
False
>>> {} == 0.0
False
>>> None == (lambda x: 1)
False
>>> int == max
False


The only place I know of where this expectation isn't met is when comparing
"naive" and "timezone-aware" datetime objects, which raises a TypeError (IIRC).



From tjreedy at udel.edu  Sun Nov  8 20:33:24 2009
From: tjreedy at udel.edu (Terry Reedy)
Date: Sun, 08 Nov 2009 14:33:24 -0500
Subject: [Python-Dev] PEP 3003 - Python Language Moratorium
In-Reply-To: <4AF61336.5010004@gmail.com>
References: <ca471dc20911030935l25251e0diec64427f5d7bcbf2@mail.gmail.com>	<ca471dc20911051405o287e0f41m26d38860a200b47f@mail.gmail.com>	<200911081115.01803.steve@pearwood.info>
	<4AF61336.5010004@gmail.com>
Message-ID: <hd76e5$rei$1@ger.gmane.org>

John Arbash Meinel wrote:

> He wanted to introduce a moratorium at least partially because he was
> tired of endless threads about anonymous code blocks, etc. Which aren't
> going to be included in the language anyway, so he may as well make a
> point to say "and neither will anything else for a while".

I have not said anything on this topic either way. But I think a general 
moratorium is a false solution to that particular problem. Guido should 
just make a clear pronouncement (again) and let people quote that in 
response to initial posts, before they *appear* to gain steam. Each time 
Python doubles in popularity, the number of people wanting such things 
will approximately double. That is the price of success.

Terry Jan Reedy


From rdmurray at bitdance.com  Sun Nov  8 20:59:05 2009
From: rdmurray at bitdance.com (R. David Murray)
Date: Sun, 8 Nov 2009 14:59:05 -0500 (EST)
Subject: [Python-Dev] Status of the Buildbot fleet and related bugs
In-Reply-To: <4AF7119A.2040201@v.loewis.de>
References: <Pine.LNX.4.64.0911052209000.12274@kimball.webabinitio.net>
	<hd28f1$v2d$3@ger.gmane.org> <4AF7119A.2040201@v.loewis.de>
Message-ID: <Pine.LNX.4.64.0911081458280.12274@kimball.webabinitio.net>

On Sun, 8 Nov 2009 at 19:44, "Martin v. L?wis" wrote:
>> JFTR, I didn't set up the IRC bot (I assume that credit goes to Martin,
>> even if it's only one line in the buildbot config :).  I just tried to
>> get it to say something :)
>
> Yes, it was always "on". I don't use IRC regularly, so I don't know
> whether it's useful.

I think it is.

--David (RDM)

From greg at krypto.org  Sun Nov  8 21:56:34 2009
From: greg at krypto.org (Gregory P. Smith)
Date: Sun, 8 Nov 2009 12:56:34 -0800
Subject: [Python-Dev] PEP 3003 - Python Language Moratorium
In-Reply-To: <9d153b7c0911051613o62be5dc9l52102f7805cd4a0e@mail.gmail.com>
References: <ca471dc20911030935l25251e0diec64427f5d7bcbf2@mail.gmail.com> 
	<ca471dc20911051405o287e0f41m26d38860a200b47f@mail.gmail.com> 
	<ea2499da0911051453n3569387aj581ad4acf5b799c8@mail.gmail.com> 
	<4222a8490911051521r1b9c8165id6e0f12d62da0613@mail.gmail.com> 
	<ca471dc20911051526u7f3642ecg19e4eb73bcca1d3a@mail.gmail.com> 
	<bbaeab100911051535g68cefa18kdadea33f78de9495@mail.gmail.com> 
	<48c184130911051555r9a5b78cs3a13cb1345d3cbf7@mail.gmail.com> 
	<9d153b7c0911051613o62be5dc9l52102f7805cd4a0e@mail.gmail.com>
Message-ID: <52dc1c820911081256q7fc3f7fdk87b2f12932dce11d@mail.gmail.com>

On Thu, Nov 5, 2009 at 4:13 PM, Yuvgoog Greenle <ubershmekel at gmail.com> wrote:
> On Fri, Nov 6, 2009 at 1:55 AM, Bobby R. Ward <bobbyrward at gmail.com> wrote:
>>
>> A switch to ENABLE those warnings?
>>
>
> I think a few people I know would still be raising strings like exceptions
> if not for the deprecation warnings. Most people won't turn on the switch
> with the extra warnings.
> --yuv

Yep and their code will break in the future as a result.  Thats their
problem (or whoever employed them and neglected to do code reviews and
beat them with a stick for using a Python 1.x "feature" in 2009).

SUGGESTION:  Our Whats New document currently has a single "new,
improved and deprecated" section in it.  We should make Deprecations
its own section so that they're easy to find all in one concise place
when upgrading code between releases.

Having seen the fallout from adding deprecating warnings to code: many
annoyed developers whos workflow you've interrupted who will all
eventually turn the warning off by default at all times.  I have to
agree.  The DeprecationWarning being on by default is a failed
experiement.  We should treat it and the rarely noticed
PendingDeprecationWarning as the same thing by default.

-gps

From g.brandl at gmx.net  Sun Nov  8 22:25:38 2009
From: g.brandl at gmx.net (Georg Brandl)
Date: Sun, 08 Nov 2009 22:25:38 +0100
Subject: [Python-Dev] OpenSSL vulnerability
In-Reply-To: <ca471dc20911061512j1f5f7e45nb5fd2de0d2734d45@mail.gmail.com>
References: <ca471dc20911061222v5c380643te7d5a4e84d3496f9@mail.gmail.com>
	<40373.1257545902@parc.com>	<20091106223634.3229.2049231320.divmod.xquotient.232@localhost.localdomain>
	<ca471dc20911061512j1f5f7e45nb5fd2de0d2734d45@mail.gmail.com>
Message-ID: <hd7d0l$ek6$1@ger.gmane.org>

Guido van Rossum schrieb:
> On Fri, Nov 6, 2009 at 2:36 PM,  <exarkun at twistedmatrix.com> wrote:
>> Also, for Python 2.5 and earlier, any SSL-based code is vulnerable to a MitM
>> anyway, so this can only be an issue for code using the new APIs in Python
>> 2.6.
> 
> That's not going to stop the
> wannabe-self-proclaimed-so-called-vulnerability-"experts" from whining
> about Python not releasing updated binary distributions though. :-(

Yet it has been quiet on the Finnish front so far :)

Georg

-- 
Thus spake the Lord: Thou shalt indent with four spaces. No more, no less.
Four shall be the number of spaces thou shalt indent, and the number of thy
indenting shall be four. Eight shalt thou not indent, nor either indent thou
two, excepting that thou then proceed to four. Tabs are right out.


From greg.ewing at canterbury.ac.nz  Sun Nov  8 22:28:35 2009
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Mon, 09 Nov 2009 10:28:35 +1300
Subject: [Python-Dev] PEP 3003 - Python Language Moratorium
In-Reply-To: <4AF61336.5010004@gmail.com>
References: <ca471dc20911030935l25251e0diec64427f5d7bcbf2@mail.gmail.com>
	<ca471dc20911051405o287e0f41m26d38860a200b47f@mail.gmail.com>
	<200911081115.01803.steve@pearwood.info> <4AF61336.5010004@gmail.com>
Message-ID: <4AF73803.9000303@canterbury.ac.nz>

John Arbash Meinel wrote:

> He wanted to introduce a moratorium at least partially because he was
> tired of endless threads about anonymous code blocks, etc. Which aren't
> going to be included in the language anyway, so he may as well make a
> point to say "and neither will anything else for a while".

If anonymous code blocks still get discussed even when
they have no chance of being accepted, this suggests that
a moratorium is *not* going to stop discussion of new
features.

I'm a bit confused about Guido's reasons for wanting a
moratorium. He started out by saying that the intention
wasn't to stop people discussing ideas, then later he
seemed to say that he was finding all these discussions
were distracting, then he explicitly said he wanted to
"crush" people's hopes of getting new features into the
language. That all sounds like he actually *does* want
to discourage such discussions.

-- 
Greg

From g.brandl at gmx.net  Sun Nov  8 22:27:46 2009
From: g.brandl at gmx.net (Georg Brandl)
Date: Sun, 08 Nov 2009 22:27:46 +0100
Subject: [Python-Dev] PEP 3003 - Python Language Moratorium
In-Reply-To: <52dc1c820911081256q7fc3f7fdk87b2f12932dce11d@mail.gmail.com>
References: <ca471dc20911030935l25251e0diec64427f5d7bcbf2@mail.gmail.com>
	<ca471dc20911051405o287e0f41m26d38860a200b47f@mail.gmail.com>
	<ea2499da0911051453n3569387aj581ad4acf5b799c8@mail.gmail.com>
	<4222a8490911051521r1b9c8165id6e0f12d62da0613@mail.gmail.com>
	<ca471dc20911051526u7f3642ecg19e4eb73bcca1d3a@mail.gmail.com>
	<bbaeab100911051535g68cefa18kdadea33f78de9495@mail.gmail.com>
	<48c184130911051555r9a5b78cs3a13cb1345d3cbf7@mail.gmail.com>
	<9d153b7c0911051613o62be5dc9l52102f7805cd4a0e@mail.gmail.com>
	<52dc1c820911081256q7fc3f7fdk87b2f12932dce11d@mail.gmail.com>
Message-ID: <hd7d4k$ek6$2@ger.gmane.org>

Gregory P. Smith schrieb:
> On Thu, Nov 5, 2009 at 4:13 PM, Yuvgoog Greenle <ubershmekel at gmail.com> wrote:
>> On Fri, Nov 6, 2009 at 1:55 AM, Bobby R. Ward <bobbyrward at gmail.com> wrote:
>>>
>>> A switch to ENABLE those warnings?
>>>
>>
>> I think a few people I know would still be raising strings like exceptions
>> if not for the deprecation warnings. Most people won't turn on the switch
>> with the extra warnings.
>> --yuv
> 
> Yep and their code will break in the future as a result.  Thats their
> problem (or whoever employed them and neglected to do code reviews and
> beat them with a stick for using a Python 1.x "feature" in 2009).
> 
> SUGGESTION:  Our Whats New document currently has a single "new,
> improved and deprecated" section in it.  We should make Deprecations
> its own section so that they're easy to find all in one concise place
> when upgrading code between releases.

Good point, I'll make that change if AMK agrees.

Georg

-- 
Thus spake the Lord: Thou shalt indent with four spaces. No more, no less.
Four shall be the number of spaces thou shalt indent, and the number of thy
indenting shall be four. Eight shalt thou not indent, nor either indent thou
two, excepting that thou then proceed to four. Tabs are right out.


From solipsis at pitrou.net  Sun Nov  8 22:38:04 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Sun, 8 Nov 2009 21:38:04 +0000 (UTC)
Subject: [Python-Dev] PEP 3003 - Python Language Moratorium
References: <ca471dc20911030935l25251e0diec64427f5d7bcbf2@mail.gmail.com>
	<ca471dc20911051405o287e0f41m26d38860a200b47f@mail.gmail.com>
	<200911081115.01803.steve@pearwood.info>
	<4AF61336.5010004@gmail.com> <4AF73803.9000303@canterbury.ac.nz>
Message-ID: <loom.20091108T223657-847@post.gmane.org>

Greg Ewing <greg.ewing <at> canterbury.ac.nz> writes:
> 
> If anonymous code blocks still get discussed even when
> they have no chance of being accepted, this suggests that
> a moratorium is *not* going to stop discussion of new
> features.

Well, if they get discussed, it's probably that some people can't help
participating in the discussions, even though they know the idea is doomed to
failure ;)



From steve at pearwood.info  Mon Nov  9 00:06:25 2009
From: steve at pearwood.info (Steven D'Aprano)
Date: Mon, 9 Nov 2009 10:06:25 +1100
Subject: [Python-Dev] PEP 3003 - Python Language Moratorium
In-Reply-To: <200911081115.01803.steve@pearwood.info>
References: <ca471dc20911030935l25251e0diec64427f5d7bcbf2@mail.gmail.com>
	<ca471dc20911051405o287e0f41m26d38860a200b47f@mail.gmail.com>
	<200911081115.01803.steve@pearwood.info>
Message-ID: <200911091006.26433.steve@pearwood.info>

On Sun, 8 Nov 2009 11:14:59 am Steven D'Aprano wrote:
> At the very least, I believe, any moratorium should have a clear end
> date. A clear end date will be a powerful counter to the impression
> that Python the language is moribund. It says, this is an exceptional
> pause, not a permanent halt.

Proposal: 

No new language features in odd-numbered point releases (3.3, 3.5, ...). 
Even-numbered point releases (3.4, 3.6, ...) may include new language 
features provided they meet the usual standards for new features.

3.2 is a special case: as an even-numbered release, it would normally 
allow new features, but in recognition of the special nature of the 2.x 
to 3.1/3.2 migration, no new language features will be allowed.

Advantages:

* It slows down changes to the language while still allowing 
sufficiently high-standard new features.

* Alternate implementations have a stable language version to aim for. 
Assuming point releases come ever 12-18 months, that stable language 
version will last 2-3 years.

* It doesn't have the psychological baggage of an unconditional ban on 
new features for the indefinite future. It gives a fixed, known 
schedule for when new features will be permitted, without the 
uncertainty of "at the BDFL's pleasure".


-- 
Steven D'Aprano

From debatem1 at gmail.com  Mon Nov  9 01:01:26 2009
From: debatem1 at gmail.com (geremy condra)
Date: Sun, 8 Nov 2009 19:01:26 -0500
Subject: [Python-Dev] PEP 3003 - Python Language Moratorium
In-Reply-To: <200911091006.26433.steve@pearwood.info>
References: <ca471dc20911030935l25251e0diec64427f5d7bcbf2@mail.gmail.com>
	<ca471dc20911051405o287e0f41m26d38860a200b47f@mail.gmail.com>
	<200911081115.01803.steve@pearwood.info>
	<200911091006.26433.steve@pearwood.info>
Message-ID: <f3cc57c60911081601l6f35f75ci9249de7b7eeed1a1@mail.gmail.com>

On Sun, Nov 8, 2009 at 6:06 PM, Steven D'Aprano <steve at pearwood.info> wrote:
> On Sun, 8 Nov 2009 11:14:59 am Steven D'Aprano wrote:
>> At the very least, I believe, any moratorium should have a clear end
>> date. A clear end date will be a powerful counter to the impression
>> that Python the language is moribund. It says, this is an exceptional
>> pause, not a permanent halt.
>
> Proposal:
>
> No new language features in odd-numbered point releases (3.3, 3.5, ...).
> Even-numbered point releases (3.4, 3.6, ...) may include new language
> features provided they meet the usual standards for new features.
>
> 3.2 is a special case: as an even-numbered release, it would normally
> allow new features, but in recognition of the special nature of the 2.x
> to 3.1/3.2 migration, no new language features will be allowed.
>
> Advantages:
>
> * It slows down changes to the language while still allowing
> sufficiently high-standard new features.
>
> * Alternate implementations have a stable language version to aim for.
> Assuming point releases come ever 12-18 months, that stable language
> version will last 2-3 years.
>
> * It doesn't have the psychological baggage of an unconditional ban on
> new features for the indefinite future. It gives a fixed, known
> schedule for when new features will be permitted, without the
> uncertainty of "at the BDFL's pleasure".
>
>
> --
> Steven D'Aprano


FWIW, I view a definite end point as a definite plus.

Geremy Condra

From jnoller at gmail.com  Mon Nov  9 02:22:52 2009
From: jnoller at gmail.com (Jesse Noller)
Date: Sun, 8 Nov 2009 20:22:52 -0500
Subject: [Python-Dev] PEP 3003 - Python Language Moratorium
In-Reply-To: <f3cc57c60911081601l6f35f75ci9249de7b7eeed1a1@mail.gmail.com>
References: <ca471dc20911030935l25251e0diec64427f5d7bcbf2@mail.gmail.com>
	<ca471dc20911051405o287e0f41m26d38860a200b47f@mail.gmail.com>
	<200911081115.01803.steve@pearwood.info>
	<200911091006.26433.steve@pearwood.info>
	<f3cc57c60911081601l6f35f75ci9249de7b7eeed1a1@mail.gmail.com>
Message-ID: <F57B38EA-3E44-4920-8BE8-DB0AE1E21BA7@gmail.com>



On Nov 8, 2009, at 7:01 PM, geremy condra <debatem1 at gmail.com> wrote:

> On Sun, Nov 8, 2009 at 6:06 PM, Steven D'Aprano  
> <steve at pearwood.info> wrote:
>> On Sun, 8 Nov 2009 11:14:59 am Steven D'Aprano wrote:
>>> At the very least, I believe, any moratorium should have a clear end
>>> date. A clear end date will be a powerful counter to the impression
>>> that Python the language is moribund. It says, this is an  
>>> exceptional
>>> pause, not a permanent halt.
>>
>> Proposal:
>>
>> No new language features in odd-numbered point releases (3.3,  
>> 3.5, ...).
>> Even-numbered point releases (3.4, 3.6, ...) may include new language
>> features provided they meet the usual standards for new features.
>>
>> 3.2 is a special case: as an even-numbered release, it would normally
>> allow new features, but in recognition of the special nature of the  
>> 2.x
>> to 3.1/3.2 migration, no new language features will be allowed.
>>
>> Advantages:
>>
>> * It slows down changes to the language while still allowing
>> sufficiently high-standard new features.
>>
>> * Alternate implementations have a stable language version to aim  
>> for.
>> Assuming point releases come ever 12-18 months, that stable language
>> version will last 2-3 years.
>>
>> * It doesn't have the psychological baggage of an unconditional ban  
>> on
>> new features for the indefinite future. It gives a fixed, known
>> schedule for when new features will be permitted, without the
>> uncertainty of "at the BDFL's pleasure".
>>
>>
>> --
>> Steven D'Aprano
>
>
> FWIW, I view a definite end point as a definite plus.
>
> Geremy Condra
>

There is a time outlined in the pep.

From skip at pobox.com  Mon Nov  9 02:41:24 2009
From: skip at pobox.com (skip at pobox.com)
Date: Sun, 08 Nov 2009 19:41:24 -0600
Subject: [Python-Dev] raw binary data and 2to3
Message-ID: <m2k4y0sce3.fsf@montanaro.dyndns.org>


SpamBayes has several files which contain raw 8-bit data embedded in
string literals.  Before I do manual work to make them parseable by 2to3
I thought I would ask if there was either a fixer available which I'm
not getting by default or if there is an opportunity to add a new fixer
to 2to3. 

The usage is pretty straightforward.  For example, a string literal
might contain the bytes for a GIF image:

    data = "GIF89a(..."

Is there a potentially automated path from where the code is today to
something Python 3 (and 2to3) will like?

Skip

From debatem1 at gmail.com  Mon Nov  9 02:45:56 2009
From: debatem1 at gmail.com (geremy condra)
Date: Sun, 8 Nov 2009 20:45:56 -0500
Subject: [Python-Dev] PEP 3003 - Python Language Moratorium
In-Reply-To: <F57B38EA-3E44-4920-8BE8-DB0AE1E21BA7@gmail.com>
References: <ca471dc20911030935l25251e0diec64427f5d7bcbf2@mail.gmail.com>
	<ca471dc20911051405o287e0f41m26d38860a200b47f@mail.gmail.com>
	<200911081115.01803.steve@pearwood.info>
	<200911091006.26433.steve@pearwood.info>
	<f3cc57c60911081601l6f35f75ci9249de7b7eeed1a1@mail.gmail.com>
	<F57B38EA-3E44-4920-8BE8-DB0AE1E21BA7@gmail.com>
Message-ID: <f3cc57c60911081745o2ed32e99i7e519f6ad19418d6@mail.gmail.com>

On Sun, Nov 8, 2009 at 8:22 PM, Jesse Noller <jnoller at gmail.com> wrote:
>
>
> On Nov 8, 2009, at 7:01 PM, geremy condra <debatem1 at gmail.com> wrote:
>
>> On Sun, Nov 8, 2009 at 6:06 PM, Steven D'Aprano <steve at pearwood.info>
>> wrote:
>>>
>>> On Sun, 8 Nov 2009 11:14:59 am Steven D'Aprano wrote:
>>>>
>>>> At the very least, I believe, any moratorium should have a clear end
>>>> date. A clear end date will be a powerful counter to the impression
>>>> that Python the language is moribund. It says, this is an exceptional
>>>> pause, not a permanent halt.
>>>
>>> Proposal:
>>>
>>> No new language features in odd-numbered point releases (3.3, 3.5, ...).
>>> Even-numbered point releases (3.4, 3.6, ...) may include new language
>>> features provided they meet the usual standards for new features.
>>>
>>> 3.2 is a special case: as an even-numbered release, it would normally
>>> allow new features, but in recognition of the special nature of the 2.x
>>> to 3.1/3.2 migration, no new language features will be allowed.
>>>
>>> Advantages:
>>>
>>> * It slows down changes to the language while still allowing
>>> sufficiently high-standard new features.
>>>
>>> * Alternate implementations have a stable language version to aim for.
>>> Assuming point releases come ever 12-18 months, that stable language
>>> version will last 2-3 years.
>>>
>>> * It doesn't have the psychological baggage of an unconditional ban on
>>> new features for the indefinite future. It gives a fixed, known
>>> schedule for when new features will be permitted, without the
>>> uncertainty of "at the BDFL's pleasure".
>>>
>>>
>>> --
>>> Steven D'Aprano
>>
>>
>> FWIW, I view a definite end point as a definite plus.
>>
>> Geremy Condra
>>
>
> There is a time outlined in the pep.
>

I quote:

"This PEP proposes a temporary moratorium (suspension) of all changes
to the Python language syntax, semantics, and built-ins for a period
of *at least two years* from the release of Python 3.1."

Emphasis mine.

Like I say, a definite end point would be much preferred to n > 2. If
possible, I'd also like to hear some of Steven's other points addressed.

Geremy Condra

From benjamin at python.org  Mon Nov  9 02:48:46 2009
From: benjamin at python.org (Benjamin Peterson)
Date: Sun, 8 Nov 2009 19:48:46 -0600
Subject: [Python-Dev] raw binary data and 2to3
In-Reply-To: <m2k4y0sce3.fsf@montanaro.dyndns.org>
References: <m2k4y0sce3.fsf@montanaro.dyndns.org>
Message-ID: <1afaf6160911081748u3f47f5dche74b1f0171c5bcff@mail.gmail.com>

2009/11/8  <skip at pobox.com>:
>
> SpamBayes has several files which contain raw 8-bit data embedded in
> string literals. ?Before I do manual work to make them parseable by 2to3
> I thought I would ask if there was either a fixer available which I'm
> not getting by default or if there is an opportunity to add a new fixer
> to 2to3.
>
> The usage is pretty straightforward. ?For example, a string literal
> might contain the bytes for a GIF image:
>
> ? ?data = "GIF89a(..."
>
> Is there a potentially automated path from where the code is today to
> something Python 3 (and 2to3) will like?

Not to my knowledge. I would prefer to not add a fixer for this
directly to 2to3 because it is not correct for most programs. However,
I think 2to3 should grow some sort of plugin system, so custom fixers
can easily be written and used.



-- 
Regards,
Benjamin

From guido at python.org  Mon Nov  9 03:35:53 2009
From: guido at python.org (Guido van Rossum)
Date: Sun, 8 Nov 2009 18:35:53 -0800
Subject: [Python-Dev] PEP 3003 - Python Language Moratorium
In-Reply-To: <200911091006.26433.steve@pearwood.info>
References: <ca471dc20911030935l25251e0diec64427f5d7bcbf2@mail.gmail.com> 
	<ca471dc20911051405o287e0f41m26d38860a200b47f@mail.gmail.com> 
	<200911081115.01803.steve@pearwood.info>
	<200911091006.26433.steve@pearwood.info>
Message-ID: <ca471dc20911081835p3c7dcd8ex4bb57399b102728e@mail.gmail.com>

On Sun, Nov 8, 2009 at 3:06 PM, Steven D'Aprano <steve at pearwood.info> wrote:
> No new language features in odd-numbered point releases (3.3, 3.5, ...).
> Even-numbered point releases (3.4, 3.6, ...) may include new language
> features provided they meet the usual standards for new features.

Oh no, not the eve/odd numbering scheme. Nobody will be able to
remember which is which.

-- 
--Guido van Rossum (python.org/~guido)

From guido at python.org  Mon Nov  9 03:41:32 2009
From: guido at python.org (Guido van Rossum)
Date: Sun, 8 Nov 2009 18:41:32 -0800
Subject: [Python-Dev] PEP 3003 - Python Language Moratorium
In-Reply-To: <f3cc57c60911081745o2ed32e99i7e519f6ad19418d6@mail.gmail.com>
References: <ca471dc20911030935l25251e0diec64427f5d7bcbf2@mail.gmail.com> 
	<ca471dc20911051405o287e0f41m26d38860a200b47f@mail.gmail.com> 
	<200911081115.01803.steve@pearwood.info>
	<200911091006.26433.steve@pearwood.info> 
	<f3cc57c60911081601l6f35f75ci9249de7b7eeed1a1@mail.gmail.com> 
	<F57B38EA-3E44-4920-8BE8-DB0AE1E21BA7@gmail.com>
	<f3cc57c60911081745o2ed32e99i7e519f6ad19418d6@mail.gmail.com>
Message-ID: <ca471dc20911081841u5049344cpc8b7fea78832fa08@mail.gmail.com>

On Sun, Nov 8, 2009 at 5:45 PM, geremy condra <debatem1 at gmail.com> wrote:
> I quote:
>
> "This PEP proposes a temporary moratorium (suspension) of all changes
> to the Python language syntax, semantics, and built-ins for a period
> of *at least two years* from the release of Python 3.1."
>
> Emphasis mine.

I since added this:

"""In particular,
the moratorium would include Python 3.2 (to be released 18-24 months
after 3.1) but (unless explicitly extended) allow Python 3.3 to once
again include language changes."""

> Like I say, a definite end point would be much preferred to n > 2.

My time machine doesn't work very well in the future. So I can't tell
what we'll find necessary 2 years from now. But I would be fine with
defining the time limit to be max(time(3.1) + 2 years, time(3.2)).
I.e. the moratorium (unless explicitly extended) ends as soon as 3.2
has been released *and* at least 2 years have passed since 3.1.

> If possible, I'd also like to hear some of Steven's other points addressed.

They haven't changed my mind.

-- 
--Guido van Rossum (python.org/~guido)

From guido at python.org  Mon Nov  9 03:42:47 2009
From: guido at python.org (Guido van Rossum)
Date: Sun, 8 Nov 2009 18:42:47 -0800
Subject: [Python-Dev] raw binary data and 2to3
In-Reply-To: <1afaf6160911081748u3f47f5dche74b1f0171c5bcff@mail.gmail.com>
References: <m2k4y0sce3.fsf@montanaro.dyndns.org>
	<1afaf6160911081748u3f47f5dche74b1f0171c5bcff@mail.gmail.com>
Message-ID: <ca471dc20911081842i673c7a39ie5e1e9e300328e40@mail.gmail.com>

On Sun, Nov 8, 2009 at 5:48 PM, Benjamin Peterson <benjamin at python.org> wrote:
> 2009/11/8 ?<skip at pobox.com>:
>>
>> SpamBayes has several files which contain raw 8-bit data embedded in
>> string literals. ?Before I do manual work to make them parseable by 2to3
>> I thought I would ask if there was either a fixer available which I'm
>> not getting by default or if there is an opportunity to add a new fixer
>> to 2to3.
>>
>> The usage is pretty straightforward. ?For example, a string literal
>> might contain the bytes for a GIF image:
>>
>> ? ?data = "GIF89a(..."
>>
>> Is there a potentially automated path from where the code is today to
>> something Python 3 (and 2to3) will like?
>
> Not to my knowledge. I would prefer to not add a fixer for this
> directly to 2to3 because it is not correct for most programs. However,
> I think 2to3 should grow some sort of plugin system, so custom fixers
> can easily be written and used.

But if you're happy with only supporting 2.6, you can use b"..." and
the right thing will happen.

-- 
--Guido van Rossum (python.org/~guido)

From skip at pobox.com  Mon Nov  9 03:55:25 2009
From: skip at pobox.com (skip at pobox.com)
Date: Sun, 8 Nov 2009 20:55:25 -0600
Subject: [Python-Dev] raw binary data and 2to3
In-Reply-To: <ca471dc20911081842i673c7a39ie5e1e9e300328e40@mail.gmail.com>
References: <m2k4y0sce3.fsf@montanaro.dyndns.org>
	<1afaf6160911081748u3f47f5dche74b1f0171c5bcff@mail.gmail.com>
	<ca471dc20911081842i673c7a39ie5e1e9e300328e40@mail.gmail.com>
Message-ID: <19191.33949.443509.601813@montanaro.dyndns.org>


    Guido> But if you're happy with only supporting 2.6, you can use b"..." and
    Guido> the right thing will happen.

SpamBayes still supports 2.4...

Thanks for the feedback.  I'll update the source manually, then run 2to3.

S

From scott+python-dev at scottdial.com  Mon Nov  9 03:57:28 2009
From: scott+python-dev at scottdial.com (Scott Dial)
Date: Sun, 08 Nov 2009 21:57:28 -0500
Subject: [Python-Dev] PEP 3003 - Python Language Moratorium
In-Reply-To: <ca471dc20911081835p3c7dcd8ex4bb57399b102728e@mail.gmail.com>
References: <ca471dc20911030935l25251e0diec64427f5d7bcbf2@mail.gmail.com>
	<ca471dc20911051405o287e0f41m26d38860a200b47f@mail.gmail.com>
	<200911081115.01803.steve@pearwood.info>	<200911091006.26433.steve@pearwood.info>
	<ca471dc20911081835p3c7dcd8ex4bb57399b102728e@mail.gmail.com>
Message-ID: <4AF78518.1040709@scottdial.com>

Guido van Rossum wrote:
> On Sun, Nov 8, 2009 at 3:06 PM, Steven D'Aprano <steve at pearwood.info> wrote:
>> No new language features in odd-numbered point releases (3.3, 3.5, ...).
>> Even-numbered point releases (3.4, 3.6, ...) may include new language
>> features provided they meet the usual standards for new features.
> 
> Oh no, not the eve/odd numbering scheme. Nobody will be able to
> remember which is which.
> 

In this case, does it really matter? Usually the confusion is with
stable/unstable being even/odd or odd/even, but in this case the options
are stable/stable (it just happens that certain pairwise versions are
related).

-- 
Scott Dial
scott at scottdial.com
scodial at cs.indiana.edu

From debatem1 at gmail.com  Mon Nov  9 04:50:12 2009
From: debatem1 at gmail.com (geremy condra)
Date: Sun, 8 Nov 2009 22:50:12 -0500
Subject: [Python-Dev] PEP 3003 - Python Language Moratorium
In-Reply-To: <ca471dc20911081841u5049344cpc8b7fea78832fa08@mail.gmail.com>
References: <ca471dc20911030935l25251e0diec64427f5d7bcbf2@mail.gmail.com>
	<ca471dc20911051405o287e0f41m26d38860a200b47f@mail.gmail.com>
	<200911081115.01803.steve@pearwood.info>
	<200911091006.26433.steve@pearwood.info>
	<f3cc57c60911081601l6f35f75ci9249de7b7eeed1a1@mail.gmail.com>
	<F57B38EA-3E44-4920-8BE8-DB0AE1E21BA7@gmail.com>
	<f3cc57c60911081745o2ed32e99i7e519f6ad19418d6@mail.gmail.com>
	<ca471dc20911081841u5049344cpc8b7fea78832fa08@mail.gmail.com>
Message-ID: <f3cc57c60911081950s81095bdq7c758bc74cfece74@mail.gmail.com>

On Sun, Nov 8, 2009 at 9:41 PM, Guido van Rossum <guido at python.org> wrote:
> On Sun, Nov 8, 2009 at 5:45 PM, geremy condra <debatem1 at gmail.com> wrote:
>> I quote:
>>
>> "This PEP proposes a temporary moratorium (suspension) of all changes
>> to the Python language syntax, semantics, and built-ins for a period
>> of *at least two years* from the release of Python 3.1."
>>
>> Emphasis mine.
>
> I since added this:
>
> """In particular,
> the moratorium would include Python 3.2 (to be released 18-24 months
> after 3.1) but (unless explicitly extended) allow Python 3.3 to once
> again include language changes."""
>
>> Like I say, a definite end point would be much preferred to n > 2.
>
> My time machine doesn't work very well in the future. So I can't tell
> what we'll find necessary 2 years from now. But I would be fine with
> defining the time limit to be max(time(3.1) + 2 years, time(3.2)).
> I.e. the moratorium (unless explicitly extended) ends as soon as 3.2
> has been released *and* at least 2 years have passed since 3.1.

Ok, thanks for the clarification. Could you spell out what you would
consider grounds for a future extension?

>> If possible, I'd also like to hear some of Steven's other points addressed.
>
> They haven't changed my mind.

Ok, but the fact that you (or Steven) hold a particular set of beliefs
is a singularly unconvincing argument. Could you explain why you
don't agree, if only for the record?

Geremy Condra

From robert.kern at gmail.com  Mon Nov  9 05:20:35 2009
From: robert.kern at gmail.com (Robert Kern)
Date: Sun, 08 Nov 2009 22:20:35 -0600
Subject: [Python-Dev] decimal.py: == and != comparisons involving NaNs
In-Reply-To: <loom.20091108T195834-29@post.gmane.org>
References: <1257590632.12.0.893741146745.issue7279@psf.upfronthosting.co.za>	<1257672530.83.0.836585441467.issue7279@psf.upfronthosting.co.za>	<20091108162648.GA4099@mail.bytereef.org>
	<loom.20091108T195834-29@post.gmane.org>
Message-ID: <hd85aj$4as$1@ger.gmane.org>

Antoine Pitrou wrote:
> Stefan Krah <stefan-usenet <at> bytereef.org> writes:
>> Are there cases where == and != are actually needed to give a result
>> for NaNs?
> 
> It is a common expectation that == and != always succeed. They return True or
> False, but don't raise an exception even on unrelated operands:

It is a common expectation, but a false one. __eq__ and __ne__ are explicitly 
allowed to return anything, not just bools.

http://www.python.org/dev/peps/pep-0207/

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
  that is made terrible by our own mad attempt to interpret it as though it had
  an underlying truth."
   -- Umberto Eco


From nnorwitz at gmail.com  Mon Nov  9 06:53:40 2009
From: nnorwitz at gmail.com (Neal Norwitz)
Date: Sun, 8 Nov 2009 21:53:40 -0800
Subject: [Python-Dev] Status of the Buildbot fleet and related bugs
In-Reply-To: <5c6f2a5d0911060027x6152a27ao77df7938a079f622@mail.gmail.com>
References: <Pine.LNX.4.64.0911052209000.12274@kimball.webabinitio.net>
	<5c6f2a5d0911060027x6152a27ao77df7938a079f622@mail.gmail.com>
Message-ID: <ee2a432c0911082153k3289b2e3y6eda970ec98ba93a@mail.gmail.com>

On Fri, Nov 6, 2009 at 12:27 AM, Mark Dickinson <dickinsm at gmail.com> wrote:
> On Fri, Nov 6, 2009 at 3:53 AM, R. David Murray <rdmurray at bitdance.com> wrote:
>
>> ? ?(2) issue 4970: consistent signal 32 error on the norwitz-x86 Gentoo
>> ? ? ? ?buildslave in 3.1 and 3.x. ?This may be due to the box
>> ? ? ? ?running an old threading library, but it does make one wonder what
>> ? ? ? ?changed in 3.x that exposed it.
>
> This error has been happening since well before 3.0 was released. ?Asking
> for access to Neal's machine is probably the only sensible way to diagnose
> it. (A less invasive but slower way to debug would be to create a branch
> especially for this bug and do repeated runs to figure out which part of test_os
> is causing the failure.)

IIRC, I spent quite a bit of time trying to nail this down.  I don't
remember finding any useful information on the cause (beyond narrowing
it to some tests).  As Mark said, this has been happening for a long
time.

I'm reticent to provide access to the machine, as it's not really
mine.  I'm not even sure I have access, I haven't logged in for a long
time.

I'd just like to say thanks again to everyone for making the buildbots
more green and also improving the general testing infrastructure for
Python.

n

From techtonik at gmail.com  Mon Nov  9 07:45:30 2009
From: techtonik at gmail.com (anatoly techtonik)
Date: Mon, 9 Nov 2009 08:45:30 +0200
Subject: [Python-Dev] IDLE as default Python editor (Was: [pygame] Python
	IDE for windoz)
Message-ID: <d34314100911082245w79c9233dwad2a0aa867e06355@mail.gmail.com>

Hello,

Quite an interesting question recently popped up in pygame community
that I'd like to ask to Python developers.

How many of you use IDLE?
What's wrong with it?

>From my side I like the idea of having default Python editor that is
small, fast, convenient and extensible/embeddable. IDLE is small and
fast, but I feel really uncomfortable with its. The worst thing - I
can't change it. Does anybody else feel the same to think if we could
replace IDLE with something more maintainable by providing Scintilla
bindings, for example?

Regards,
-- 
anatoly t.



On Sat, Oct 10, 2009 at 8:15 PM, RB[0] <roebros at gmail.com> wrote:
> Most? people just don't like it - personally it's all I use, Windows or
> otherwise...
>
> On Sat, Oct 10, 2009 at 12:23 PM, Kris Schnee <kschnee at xepher.net> wrote:
>>
>> What's wrong with the built-in editor, IDLE?
>
>

From ben+python at benfinney.id.au  Mon Nov  9 08:04:40 2009
From: ben+python at benfinney.id.au (Ben Finney)
Date: Mon, 09 Nov 2009 18:04:40 +1100
Subject: [Python-Dev] IDLE as default Python editor
References: <d34314100911082245w79c9233dwad2a0aa867e06355@mail.gmail.com>
Message-ID: <87iqdk2n7b.fsf@benfinney.id.au>

anatoly techtonik <techtonik at gmail.com> writes:

> Quite an interesting question recently popped up in pygame community
> that I'd like to ask to Python developers.

This forum is specifically about development *of* Python.

You would do better to ask on the discussion forum for Python users
<URL:http://www.python.org/community/lists/#comp-lang-python>.

Or, if you're interested in gauging interest in a new idea for changing
Python, you could raise it on the ?python-ideas? forum
<URL:http://news.gmane.org/gmane.comp.python.ideas>.

-- 
 \       ?The generation of random numbers is too important to be left |
  `\                                    to chance.? ?Robert R. Coveyou |
_o__)                                                                  |
Ben Finney


From db3l.net at gmail.com  Mon Nov  9 08:29:54 2009
From: db3l.net at gmail.com (David Bolen)
Date: Mon, 09 Nov 2009 02:29:54 -0500
Subject: [Python-Dev] Updates to XP-4 and FreeBSD build slaves
Message-ID: <m2fx8o88b1.fsf@valheru.db3l.homeip.net>

I think in a prior discussion, it was suggested that build slave
updates were ok for this list - I apologize to those who may not be
interested.

I've just completed some updates to my two build slaves.

XP-4 is now running XP Pro SP3 (was SP2) and is using the full version
of VS 2008 (was previously using Express).

FreeBSD is now running FreeBSD 6.4-RELEASE (was 6.2-RELEASE).

Both are new VMs, and also now use the latest buildbot (0.7.11p3)
release.  For at least the next few days they are being run on a
different host while I do some long-awaited maintenance to the old
host.  I've done manual builds on both, as well as reissuing a few
builds through the master, but if anyone notices anything strange
please feel free to drop me a note.

-- David

PS: I did discover one interesting bug by starting fresh on
Windows. It turns out that one of the distutils tests assumes Notepad
entries in the registry that aren't actually present unless you run
Notepad at least once.  I hadn't needed to run it, but then noticed
the build failures so ran it just to get the registry populated.


From martin at v.loewis.de  Mon Nov  9 08:46:06 2009
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Mon, 09 Nov 2009 08:46:06 +0100
Subject: [Python-Dev] raw binary data and 2to3
In-Reply-To: <m2k4y0sce3.fsf@montanaro.dyndns.org>
References: <m2k4y0sce3.fsf@montanaro.dyndns.org>
Message-ID: <4AF7C8BE.3090400@v.loewis.de>

>     data = "GIF89a(..."
> 
> Is there a potentially automated path from where the code is today to
> something Python 3 (and 2to3) will like?

Not sure how you'll fix these; I personally always provided a b()
function that will do the right thing in both 2.x and 3.x. This can
get eventually replaced by a b prefix.

Regards,
Martin

From martin at v.loewis.de  Mon Nov  9 08:48:13 2009
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Mon, 09 Nov 2009 08:48:13 +0100
Subject: [Python-Dev] IDLE as default Python editor (Was: [pygame]
 Python IDE for windoz)
In-Reply-To: <d34314100911082245w79c9233dwad2a0aa867e06355@mail.gmail.com>
References: <d34314100911082245w79c9233dwad2a0aa867e06355@mail.gmail.com>
Message-ID: <4AF7C93D.2090505@v.loewis.de>

> Quite an interesting question recently popped up in pygame community
> that I'd like to ask to Python developers.
> 
> How many of you use IDLE?

I do, sometimes.

> What's wrong with it?

AFAICT, nothing.

>>From my side I like the idea of having default Python editor that is
> small, fast, convenient and extensible/embeddable. IDLE is small and
> fast, but I feel really uncomfortable with its. The worst thing - I
> can't change it.

This I cannot understand. I have changed IDLE many times.

Regards,
Martin

From tjreedy at udel.edu  Mon Nov  9 09:25:42 2009
From: tjreedy at udel.edu (Terry Reedy)
Date: Mon, 09 Nov 2009 03:25:42 -0500
Subject: [Python-Dev] IDLE as default Python editor (Was: [pygame]
 Python IDE for windoz)
In-Reply-To: <d34314100911082245w79c9233dwad2a0aa867e06355@mail.gmail.com>
References: <d34314100911082245w79c9233dwad2a0aa867e06355@mail.gmail.com>
Message-ID: <hd8jm6$vit$1@ger.gmane.org>

anatoly techtonik wrote:
> Hello,
> 
> Quite an interesting question recently popped up in pygame community
> that I'd like to ask to Python developers.
> 
> How many of you use IDLE?

I do, both shell and editor (for Python code).

> What's wrong with it?

See tracker.

I agree that discussion of alternatives belongs elsewhere.

tjr



From martin at v.loewis.de  Mon Nov  9 09:42:15 2009
From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=)
Date: Mon, 09 Nov 2009 09:42:15 +0100
Subject: [Python-Dev] Retrieve an arbitrary element from
 asetwithoutremoving it
In-Reply-To: <eae285400911060617s15d8f0bpaa2c3c4d6d946d41@mail.gmail.com>
References: <7b0045ed0910301843y7059d316s8a851142b78b7aeb@mail.gmail.com>	
	<4AF0D0D7.3090403@canterbury.ac.nz>	
	<200911042234.36796.steve@pearwood.info>	
	<f3cc57c60911041332v479cc3b0t19d9601e52002128@mail.gmail.com>	
	<99BB308202224D31AA5300223DB59912@RaymondLaptop1>	
	<7b0045ed0911051243q18f8324ak9abaa98ac26948e9@mail.gmail.com>	
	<4AF341BD.2010203@v.loewis.de>	
	<7b0045ed0911051419i76f4221dye8ecccc424179ec6@mail.gmail.com>	
	<BD050FFAAA19443DA0DA34FD2BD740CF@RaymondLaptop1>	
	<97069B15530D4C66AC1D4BCCB70E2FD5@RaymondLaptop1>
	<eae285400911060617s15d8f0bpaa2c3c4d6d946d41@mail.gmail.com>
Message-ID: <4AF7D5E7.4080901@v.loewis.de>

> The problem arises because we're systematically unbalancing the hash
> table.  The iterator returns the first valid entry in the hash table,
> which we remove.  Repeat several times and pretty soon the iterator has
> to spend O(n) time scanning through empty entries to find the first
> remaining valid entry.

Interesting. Something goes wrong, it seems: if items get removed over
and over again, I think the set should shrink (not sure whether it
actually does). Then, I think you should end up with an amortized O(1)
for selecting an element (assuming that the underlying hashes don't
collide).

Regards,
Martin

From stefan-usenet at bytereef.org  Mon Nov  9 11:42:49 2009
From: stefan-usenet at bytereef.org (Stefan Krah)
Date: Mon, 9 Nov 2009 11:42:49 +0100
Subject: [Python-Dev] decimal.py: == and != comparisons involving NaNs
In-Reply-To: <4FC3B5BCE9034A79AFB0530C748F019E@RaymondLaptop1>
References: <20091108162648.GA4099@mail.bytereef.org>
	<4FC3B5BCE9034A79AFB0530C748F019E@RaymondLaptop1>
Message-ID: <20091109104249.GA3377@mail.bytereef.org>

Raymond Hettinger <python at rcn.com> wrote:
> [Stefan Krah]
> >in a (misguided) bugreport (http://bugs.python.org/issue7279) I was
> >questioning the reasons for allowing NaN comparisons with == and !=
> >rather than raising InvalidOperation.
> 
> Do you have any actual use case issues or are these theoretical musings?
> I ask only because a good use case might suggest the best way to adapt
> the standard to the regular python api for equality/inequality operators.

I think my reasoning goes the opposite way:

The current behavior (raising InvalidOperation) of <, <=, >=, > is sensible
and as close to the standard as one can get.  This behavior was not chosen
for the equality/inequality operators because they _might_ be used for other
purposes.

But since Decimal("NaN") == Decimal("NaN") gives False, these non-decimal
use cases don't work:

>>> d = {0:Decimal("NaN")}
>>> Decimal("NaN") in d.values()
False


So, since non-decimal use cases are limited at best, the equality/inequality
operators might as well have the behavior of the other comparison operators,
which is safer for the user.


I can also give a decimal use case where the current behavior is problematic
A variable initialized to a signaling NaN should always cause an exception.

But this doesn't:

salary = Decimal("sNaN")
minimum_wage = 1000
if (salary == minimum_wage):
    print "do stuff"
else:
    print "do other stuff"



Stefan Krah




From solipsis at pitrou.net  Mon Nov  9 12:21:27 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Mon, 9 Nov 2009 11:21:27 +0000 (UTC)
Subject: [Python-Dev] decimal.py: == and != comparisons involving NaNs
References: <20091108162648.GA4099@mail.bytereef.org>
	<4FC3B5BCE9034A79AFB0530C748F019E@RaymondLaptop1>
	<20091109104249.GA3377@mail.bytereef.org>
Message-ID: <loom.20091109T121728-978@post.gmane.org>

Stefan Krah <stefan-usenet <at> bytereef.org> writes:
> 
> >>> d = {0:Decimal("NaN")}
> >>> Decimal("NaN") in d.values()
> False
> 
> So, since non-decimal use cases are limited at best, the equality/inequality
> operators might as well have the behavior of the other comparison operators,
> which is safer for the user.

The problem is when searching for /another/ object which hashes the same as
Decimal("NaN"). Here is a made-up situation to show you the problem:

>>> class H(object):
...   def __hash__(self): return hash(1)
...   def __eq__(self, other): raise ValueError
... 
>>> h = H()
>>> d = {h: ""}
>>> d[1]
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "<stdin>", line 3, in __eq__
ValueError
>>> d[1] = 2
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "<stdin>", line 3, in __eq__
ValueError




From stefan-usenet at bytereef.org  Mon Nov  9 13:21:59 2009
From: stefan-usenet at bytereef.org (Stefan Krah)
Date: Mon, 9 Nov 2009 13:21:59 +0100
Subject: [Python-Dev] decimal.py: == and != comparisons involving NaNs
In-Reply-To: <loom.20091109T121728-978@post.gmane.org>
References: <20091108162648.GA4099@mail.bytereef.org>
	<4FC3B5BCE9034A79AFB0530C748F019E@RaymondLaptop1>
	<20091109104249.GA3377@mail.bytereef.org>
	<loom.20091109T121728-978@post.gmane.org>
Message-ID: <20091109122159.GA4383@mail.bytereef.org>

Antoine Pitrou <solipsis at pitrou.net> wrote:
> Stefan Krah <stefan-usenet <at> bytereef.org> writes:
> > 
> > >>> d = {0:Decimal("NaN")}
> > >>> Decimal("NaN") in d.values()
> > False
> > 
> > So, since non-decimal use cases are limited at best, the equality/inequality
> > operators might as well have the behavior of the other comparison operators,
> > which is safer for the user.
> 
> The problem is when searching for /another/ object which hashes the same as
> Decimal("NaN"). Here is a made-up situation to show you the problem:
> 
> >>> class H(object):
> ...   def __hash__(self): return hash(1)
> ...   def __eq__(self, other): raise ValueError
> ... 
> >>> h = H()
> >>> d = {h: ""}
> >>> d[1]
> Traceback (most recent call last):
>   File "<stdin>", line 1, in <module>
>   File "<stdin>", line 3, in __eq__
> ValueError
> >>> d[1] = 2
> Traceback (most recent call last):
>   File "<stdin>", line 1, in <module>
>   File "<stdin>", line 3, in __eq__
> ValueError

I see the point, but Decimal("NaN") does not hash:

>>> hash(Decimal("NaN"))
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib/python2.7/decimal.py", line 937, in __hash__
    raise TypeError('Cannot hash a NaN value.')
TypeError: Cannot hash a NaN value.


Also, NaNs cause problems in non-decimal comparisons virtually everywhere:

>>> L = [1, 2, Decimal("NaN")]
>>> L.sort()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib/python2.7/decimal.py", line 877, in __lt__
    ans = self._compare_check_nans(other, context)
  File "/usr/lib/python2.7/decimal.py", line 782, in _compare_check_nans
    self)
  File "/usr/lib/python2.7/decimal.py", line 3755, in _raise_error
    raise error(explanation)
decimal.InvalidOperation: comparison involving NaN
>>> 


I think problems like these would best be avoided by having a separate
__totalorder__ or __lexorder__ method instead of using __eq__, __lt__,
etc., but this is of course outside the scope of this discussion (and
I have no idea how difficult it would be to implement that).



Stefan Krah



From solipsis at pitrou.net  Mon Nov  9 13:34:54 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Mon, 9 Nov 2009 12:34:54 +0000 (UTC)
Subject: [Python-Dev] decimal.py: == and != comparisons involving NaNs
References: <20091108162648.GA4099@mail.bytereef.org>
	<4FC3B5BCE9034A79AFB0530C748F019E@RaymondLaptop1>
	<20091109104249.GA3377@mail.bytereef.org>
	<loom.20091109T121728-978@post.gmane.org>
	<20091109122159.GA4383@mail.bytereef.org>
Message-ID: <loom.20091109T133048-911@post.gmane.org>

Stefan Krah <stefan-usenet <at> bytereef.org> writes:
> 
> I see the point, but Decimal("NaN") does not hash:

Ok but witness again:

>>> L = [1, 2, Decimal("NaN"), 3]
>>> 3 in L
True
>>> class H(object):
...   def __eq__(self, other): raise ValueError
... 
>>> L = [1, 2, H(), 3]
>>> 3 in L
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "<stdin>", line 2, in __eq__
ValueError


(NB: interestingly, float("nan") does hash)

Regards

Antoine.



From ncoghlan at gmail.com  Mon Nov  9 13:39:28 2009
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Mon, 09 Nov 2009 22:39:28 +1000
Subject: [Python-Dev] Status of the Buildbot fleet and related bugs
In-Reply-To: <ee2a432c0911082153k3289b2e3y6eda970ec98ba93a@mail.gmail.com>
References: <Pine.LNX.4.64.0911052209000.12274@kimball.webabinitio.net>	<5c6f2a5d0911060027x6152a27ao77df7938a079f622@mail.gmail.com>
	<ee2a432c0911082153k3289b2e3y6eda970ec98ba93a@mail.gmail.com>
Message-ID: <4AF80D80.9030506@gmail.com>

Neal Norwitz wrote:
> I'd just like to say thanks again to everyone for making the buildbots
> more green and also improving the general testing infrastructure for
> Python.

I'm *really* liking the new assertions in unittest.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------

From dickinsm at gmail.com  Mon Nov  9 13:40:06 2009
From: dickinsm at gmail.com (Mark Dickinson)
Date: Mon, 9 Nov 2009 12:40:06 +0000
Subject: [Python-Dev] decimal.py: == and != comparisons involving NaNs
In-Reply-To: <20091109122159.GA4383@mail.bytereef.org>
References: <20091108162648.GA4099@mail.bytereef.org>
	<4FC3B5BCE9034A79AFB0530C748F019E@RaymondLaptop1>
	<20091109104249.GA3377@mail.bytereef.org>
	<loom.20091109T121728-978@post.gmane.org>
	<20091109122159.GA4383@mail.bytereef.org>
Message-ID: <5c6f2a5d0911090440nc9e0dedsdd46769ec8fca15b@mail.gmail.com>

On Mon, Nov 9, 2009 at 12:21 PM, Stefan Krah <stefan-usenet at bytereef.org> wrote:
> I see the point, but Decimal("NaN") does not hash:
>
>>>> hash(Decimal("NaN"))
> Traceback (most recent call last):
> ?File "<stdin>", line 1, in <module>
> ?File "/usr/lib/python2.7/decimal.py", line 937, in __hash__
> ? ?raise TypeError('Cannot hash a NaN value.')
> TypeError: Cannot hash a NaN value.

I think that may represent an excess of caution.  float nans do hash quite
happily, and I can't see a good reason for preventing Decimal nans from
having a hash.

Mark

From ncoghlan at gmail.com  Mon Nov  9 13:43:31 2009
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Mon, 09 Nov 2009 22:43:31 +1000
Subject: [Python-Dev] IDLE as default Python editor
In-Reply-To: <87iqdk2n7b.fsf@benfinney.id.au>
References: <d34314100911082245w79c9233dwad2a0aa867e06355@mail.gmail.com>
	<87iqdk2n7b.fsf@benfinney.id.au>
Message-ID: <4AF80E73.5090104@gmail.com>

Ben Finney wrote:
> anatoly techtonik <techtonik at gmail.com> writes:
> 
>> Quite an interesting question recently popped up in pygame community
>> that I'd like to ask to Python developers.
> 
> This forum is specifically about development *of* Python.

Anatoly's question is actually a fair one for python-dev - we're the
ones that *ship* Idle, so it is legitimate to ask our reasons for
continuing to do so.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------

From ncoghlan at gmail.com  Mon Nov  9 13:51:09 2009
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Mon, 09 Nov 2009 22:51:09 +1000
Subject: [Python-Dev] IDLE as default Python editor (Was: [pygame]
 Python IDE for windoz)
In-Reply-To: <d34314100911082245w79c9233dwad2a0aa867e06355@mail.gmail.com>
References: <d34314100911082245w79c9233dwad2a0aa867e06355@mail.gmail.com>
Message-ID: <4AF8103D.5050809@gmail.com>

anatoly techtonik wrote:
> Hello,
> 
> Quite an interesting question recently popped up in pygame community
> that I'd like to ask to Python developers.
> 
> How many of you use IDLE?

I use it if it's the only syntax highlighting Python editor on the box
(since machines in the lab usually aren't set up with full dev
environments). Otherwise I'm more likely to use a general syntax
highlighting text editor (e.g. Kate on KDE)

>>From my side I like the idea of having default Python editor that is
> small, fast, convenient and extensible/embeddable. IDLE is small and
> fast, but I feel really uncomfortable with its. The worst thing - I
> can't change it. Does anybody else feel the same to think if we could
> replace IDLE with something more maintainable by providing Scintilla
> bindings, for example?

I don't particularly *like* IDLE, but that's largely because I don't
like Tcl/Tk. However, as far as cross-platform widget sets that are
reasonable to bundle with Python go (both from a licensing and code size
perspective), choices are pretty limited (plus Tcl/Tk has the advantage
of incumbency).

So while I can't claim to like Idle in and of itself, I do like the fact
that the language comes with its own syntax highlighting editor.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------

From dickinsm at gmail.com  Mon Nov  9 14:01:10 2009
From: dickinsm at gmail.com (Mark Dickinson)
Date: Mon, 9 Nov 2009 13:01:10 +0000
Subject: [Python-Dev] decimal.py: == and != comparisons involving NaNs
In-Reply-To: <20091108162648.GA4099@mail.bytereef.org>
References: <1257590632.12.0.893741146745.issue7279@psf.upfronthosting.co.za>
	<1257672530.83.0.836585441467.issue7279@psf.upfronthosting.co.za>
	<20091108162648.GA4099@mail.bytereef.org>
Message-ID: <5c6f2a5d0911090501n23254cc7x8e1eaad34508f8fa@mail.gmail.com>

On Sun, Nov 8, 2009 at 4:26 PM, Stefan Krah <stefan-usenet at bytereef.org> wrote:
> Hi,
>
> in a (misguided) bugreport (http://bugs.python.org/issue7279) I was
> questioning the reasons for allowing NaN comparisons with == and !=
> rather than raising InvalidOperation.

Some quick recent history:

For reference, the current behaviour dates from r60630.  Before this,
comparisons involving nans behaved even less coherently.  See

http://bugs.python.org/issue1979

for details.

Apart from Python's use of __eq__, the other motivation for the
current behaviour comes from the IEEE 854 standard;  given the
absence of helpful information in the Decimal standard, IEEE 854
is an obvious next place to look.  There's an unofficial copy of the
standard available at:

http://754r.ucbtest.org/standards/854.pdf

Section 5.7 describes twenty-six(!) distinct comparison operators.
So it's not immediately clear which of those twenty-six comparison
operators each of Python's six comparison operators should map
to.  However, in the standard, the first six operations in the table
are somewhat distinguished:  they're the ones that are marked
as corresponding to the usual mathematical operations, and to
Fortran's usual comparison operators (.EQ., etc.).  Given this,
and given that this behaviour seemed to fit well with Python's
needs for __eq__, it seemed to make sense at the time to
map Python's six operators to the first 6 operators in table 3.

> I think two main issues emerge from the brief discussion:
>
> 1. Should the comparison operators follow the 'compare' function
> ? from the standard?

That's a possibility.  But see below.

> 2. What is the use of == and != outside the decimal scope?
>
> Mark mentions that Python uses == to test for set and dict memberships,
> but that you cannot put decimal NaNs into sets:
>
> 'TypeError: Cannot hash a NaN value'
>
>
> I want to add that Decimal('NaN') == Decimal('NaN') gives False, which
> should somewhat limit the uses of == for NaNs outside the decimal realm
> anyway.
>
>
> Are there cases where == and != are actually needed to give a result
> for NaNs?

Well, when running in some form of 'non-stop' mode, where (quiet) NaN
results are supposed to be propagated to the end of a computation, you
certainly want equality comparisons with nan just to silently return false.
E.g., in code like:

if x == 0:
    <deal with zero special case>
else:
    <usual algorithm>

nans should just end up in the second branch, without the programmer
having had to think about it too hard.

So I think comparisons with nans should always return either True
or False when InvalidOperation is not trapped. The question is whether
comparisons should always signal when InvalidOperation is trapped
(which is what happens with the default context).

I'm -0.5 on changing the current behaviour:  it may not be exactly right,
and if I were implementing Decimal from scratch I might well do things
differently, but I don't think it's terribly wrong either.  While not based
on the Decimal standard itself, it's based on the next most closely-
related standard.  It works with Python's needs for __eq__ and __ne__.
And it's already out there in Python 2.6;  making minute adjustments
to existing behaviour without a good reason seems like asking for trouble.

Mark

From dickinsm at gmail.com  Mon Nov  9 14:08:41 2009
From: dickinsm at gmail.com (Mark Dickinson)
Date: Mon, 9 Nov 2009 13:08:41 +0000
Subject: [Python-Dev] decimal.py: == and != comparisons involving NaNs
In-Reply-To: <20091109104249.GA3377@mail.bytereef.org>
References: <20091108162648.GA4099@mail.bytereef.org>
	<4FC3B5BCE9034A79AFB0530C748F019E@RaymondLaptop1>
	<20091109104249.GA3377@mail.bytereef.org>
Message-ID: <5c6f2a5d0911090508s1a67dccei734d497b91d08fac@mail.gmail.com>

On Mon, Nov 9, 2009 at 10:42 AM, Stefan Krah <stefan-usenet at bytereef.org> wrote:
> I can also give a decimal use case where the current behavior is problematic
> A variable initialized to a signaling NaN should always cause an exception.
>
> But this doesn't:
>
> salary = Decimal("sNaN")
> minimum_wage = 1000
> if (salary == minimum_wage):
> ? ?print "do stuff"
> else:
> ? ?print "do other stuff"

Hmm.  This does look suspicious.  It's possible we should be raising
for signalling nans here.  For most of what I wrote above I was thinking
of quiet nans.

Mark

From stefan-usenet at bytereef.org  Mon Nov  9 14:21:24 2009
From: stefan-usenet at bytereef.org (Stefan Krah)
Date: Mon, 9 Nov 2009 14:21:24 +0100
Subject: [Python-Dev] decimal.py: == and != comparisons involving NaNs
In-Reply-To: <loom.20091109T133048-911@post.gmane.org>
References: <20091108162648.GA4099@mail.bytereef.org>
	<4FC3B5BCE9034A79AFB0530C748F019E@RaymondLaptop1>
	<20091109104249.GA3377@mail.bytereef.org>
	<loom.20091109T121728-978@post.gmane.org>
	<20091109122159.GA4383@mail.bytereef.org>
	<loom.20091109T133048-911@post.gmane.org>
Message-ID: <20091109132124.GA5063@mail.bytereef.org>

Antoine Pitrou <solipsis at pitrou.net> wrote:
> > I see the point, but Decimal("NaN") does not hash:
> 
> Ok but witness again:
> 
> >>> L = [1, 2, Decimal("NaN"), 3]
> >>> 3 in L
> True
> >>> class H(object):
> ...   def __eq__(self, other): raise ValueError
> ... 
> >>> L = [1, 2, H(), 3]
> >>> 3 in L
> Traceback (most recent call last):
>   File "<stdin>", line 1, in <module>
>   File "<stdin>", line 2, in __eq__
> ValueError


Yes, but the list is already broken in two ways:

>>> L.sort()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib/python2.7/decimal.py", line 877, in __lt__
    ans = self._compare_check_nans(other, context)
  File "/usr/lib/python2.7/decimal.py", line 782, in _compare_check_nans
    self)
  File "/usr/lib/python2.7/decimal.py", line 3755, in _raise_error
    raise error(explanation)
decimal.InvalidOperation: comparison involving NaN

>>> Decimal("NaN") in L
False



> (NB: interestingly, float("nan") does hash)


I wonder if it should:

>>> d = {float('nan'): 10, 0: 20}
>>> 0 in d
True
>>> float('nan') in d
False
>>> d[float('nan')]
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
KeyError: nan
>>> 


I guess my point is that NaNs in lists and dicts are broken in so many
ways that it might be good to discourage this use. (And get the added
benefit of safer mathematical behavior for == and !=.)


Stefan Krah




From dickinsm at gmail.com  Mon Nov  9 14:23:03 2009
From: dickinsm at gmail.com (Mark Dickinson)
Date: Mon, 9 Nov 2009 13:23:03 +0000
Subject: [Python-Dev] decimal.py: == and != comparisons involving NaNs
In-Reply-To: <5c6f2a5d0911090501n23254cc7x8e1eaad34508f8fa@mail.gmail.com>
References: <1257590632.12.0.893741146745.issue7279@psf.upfronthosting.co.za>
	<1257672530.83.0.836585441467.issue7279@psf.upfronthosting.co.za>
	<20091108162648.GA4099@mail.bytereef.org>
	<5c6f2a5d0911090501n23254cc7x8e1eaad34508f8fa@mail.gmail.com>
Message-ID: <5c6f2a5d0911090523tedd4f34geb5530ec10a72ef3@mail.gmail.com>

On Mon, Nov 9, 2009 at 1:01 PM, Mark Dickinson <dickinsm at gmail.com> wrote:
> current behaviour comes from the IEEE 854 standard; ?given the
> absence of helpful information in the Decimal standard, IEEE 854
> is an obvious next place to look. ?There's an unofficial copy of the
> standard available at:
>
> http://754r.ucbtest.org/standards/854.pdf
>
> Section 5.7 describes twenty-six(!) distinct comparison operators.

It's interesting to note that out of the 32 possible different comparison
behaviours (two choices of result for each of {equal, lessthan,
greaterthan, unordered}, together with a choice of whether to signal
or not for unordered in each case), the two interesting operators
that are missing from that IEEE 854 table are precisely the ones
that we're discussing:  signalling __eq__ (i.e., return False for
lessthan, greaterthan, unordered, True for equal, and signal on
unordered), and signalling __ne__ (the reverse of the above, but
still signalling on unordered).

(The other four missing operators are the uninteresting ones that
always return True or always return False.)

Mark

From dickinsm at gmail.com  Mon Nov  9 14:25:42 2009
From: dickinsm at gmail.com (Mark Dickinson)
Date: Mon, 9 Nov 2009 13:25:42 +0000
Subject: [Python-Dev] decimal.py: == and != comparisons involving NaNs
In-Reply-To: <20091109132124.GA5063@mail.bytereef.org>
References: <20091108162648.GA4099@mail.bytereef.org>
	<4FC3B5BCE9034A79AFB0530C748F019E@RaymondLaptop1>
	<20091109104249.GA3377@mail.bytereef.org>
	<loom.20091109T121728-978@post.gmane.org>
	<20091109122159.GA4383@mail.bytereef.org>
	<loom.20091109T133048-911@post.gmane.org>
	<20091109132124.GA5063@mail.bytereef.org>
Message-ID: <5c6f2a5d0911090525oc84f864jb1e336d32d0c6741@mail.gmail.com>

On Mon, Nov 9, 2009 at 1:21 PM, Stefan Krah <stefan-usenet at bytereef.org> wrote:
> Antoine Pitrou <solipsis at pitrou.net> wrote:
>> (NB: interestingly, float("nan") does hash)
>
>
> I wonder if it should:
>
>>>> d = {float('nan'): 10, 0: 20}
>>>> 0 in d
> True
>>>> float('nan') in d
> False
>>>> d[float('nan')]
> Traceback (most recent call last):
> ?File "<stdin>", line 1, in <module>
> KeyError: nan

That's because you're creating two different float nans.
Compare with:

Python 3.2a0 (py3k:76132M, Nov  6 2009, 14:47:39)
[GCC 4.2.1 (SUSE Linux)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> nan = float('nan')
>>> d = {nan: 10, 0: 20}
>>> nan in d
True
>>> d[nan]
10

Mark

From solipsis at pitrou.net  Mon Nov  9 14:29:16 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Mon, 9 Nov 2009 13:29:16 +0000 (UTC)
Subject: [Python-Dev] decimal.py: == and != comparisons involving NaNs
References: <20091108162648.GA4099@mail.bytereef.org>
	<4FC3B5BCE9034A79AFB0530C748F019E@RaymondLaptop1>
	<20091109104249.GA3377@mail.bytereef.org>
	<loom.20091109T121728-978@post.gmane.org>
	<20091109122159.GA4383@mail.bytereef.org>
	<loom.20091109T133048-911@post.gmane.org>
	<20091109132124.GA5063@mail.bytereef.org>
Message-ID: <loom.20091109T142832-846@post.gmane.org>

Stefan Krah <stefan-usenet <at> bytereef.org> writes:
> 
> I guess my point is that NaNs in lists and dicts are broken in so many
> ways that it might be good to discourage this use. (And get the added
> benefit of safer mathematical behavior for == and !=.)

Giving users seemingly random and unexplainable exceptions would not be a good
way to discourage it, though.



From ncoghlan at gmail.com  Mon Nov  9 14:32:15 2009
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Mon, 09 Nov 2009 23:32:15 +1000
Subject: [Python-Dev] Retrieve an arbitrary element from
 asetwithoutremoving it
In-Reply-To: <4AF7D5E7.4080901@v.loewis.de>
References: <7b0045ed0910301843y7059d316s8a851142b78b7aeb@mail.gmail.com>		<4AF0D0D7.3090403@canterbury.ac.nz>		<200911042234.36796.steve@pearwood.info>		<f3cc57c60911041332v479cc3b0t19d9601e52002128@mail.gmail.com>		<99BB308202224D31AA5300223DB59912@RaymondLaptop1>		<7b0045ed0911051243q18f8324ak9abaa98ac26948e9@mail.gmail.com>		<4AF341BD.2010203@v.loewis.de>		<7b0045ed0911051419i76f4221dye8ecccc424179ec6@mail.gmail.com>		<BD050FFAAA19443DA0DA34FD2BD740CF@RaymondLaptop1>		<97069B15530D4C66AC1D4BCCB70E2FD5@RaymondLaptop1>	<eae285400911060617s15d8f0bpaa2c3c4d6d946d41@mail.gmail.com>
	<4AF7D5E7.4080901@v.loewis.de>
Message-ID: <4AF819DF.6030107@gmail.com>

Martin v. L?wis wrote:
>> The problem arises because we're systematically unbalancing the hash
>> table.  The iterator returns the first valid entry in the hash table,
>> which we remove.  Repeat several times and pretty soon the iterator has
>> to spend O(n) time scanning through empty entries to find the first
>> remaining valid entry.
> 
> Interesting. Something goes wrong, it seems: if items get removed over
> and over again, I think the set should shrink (not sure whether it
> actually does). Then, I think you should end up with an amortized O(1)
> for selecting an element (assuming that the underlying hashes don't
> collide).

The behaviour is inherited (in spirit) from dicts. Courtesy of
dictnotes.txt:

"""
* Shrinkage rate upon exceeding maximum sparseness.  The current
    CPython code never even checks sparseness when deleting a
    key.  When a new key is added, it resizes based on the number
    of active keys, so that the addition may trigger shrinkage
    rather than growth.
"""

"""
Dictionary operations involving only a single key can be O(1) unless
resizing is possible.  By checking for a resize only when the
dictionary can grow (and may *require* resizing), other operations
remain O(1), and the odds of resize thrashing or memory fragmentation
are reduced. In particular, an algorithm that empties a dictionary
by repeatedly invoking .pop will see no resizing, which might
not be necessary at all because the dictionary is eventually
discarded entirely.
"""

So the rationale is to ensure that only add operations perform a resize
and so that sequential pop() operations don't incur excessive resizing
costs.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------

From amk at amk.ca  Mon Nov  9 15:16:09 2009
From: amk at amk.ca (A.M. Kuchling)
Date: Mon, 9 Nov 2009 09:16:09 -0500
Subject: [Python-Dev] PEP 3003 - Python Language Moratorium
In-Reply-To: <hd7d4k$ek6$2@ger.gmane.org>
References: <ca471dc20911030935l25251e0diec64427f5d7bcbf2@mail.gmail.com>
	<ca471dc20911051405o287e0f41m26d38860a200b47f@mail.gmail.com>
	<ea2499da0911051453n3569387aj581ad4acf5b799c8@mail.gmail.com>
	<4222a8490911051521r1b9c8165id6e0f12d62da0613@mail.gmail.com>
	<ca471dc20911051526u7f3642ecg19e4eb73bcca1d3a@mail.gmail.com>
	<bbaeab100911051535g68cefa18kdadea33f78de9495@mail.gmail.com>
	<48c184130911051555r9a5b78cs3a13cb1345d3cbf7@mail.gmail.com>
	<9d153b7c0911051613o62be5dc9l52102f7805cd4a0e@mail.gmail.com>
	<52dc1c820911081256q7fc3f7fdk87b2f12932dce11d@mail.gmail.com>
	<hd7d4k$ek6$2@ger.gmane.org>
Message-ID: <20091109141609.GA8499@amk-desktop.matrixgroup.net>

On Sun, Nov 08, 2009 at 10:27:46PM +0100, Georg Brandl wrote:
> Good point, I'll make that change if AMK agrees.

It's certainly fine with me. Do we want to only make that change to
the 2.7 "What's New", or should we also do it for the 2.6 one?

--amk

From daniel at stutzbachenterprises.com  Mon Nov  9 16:09:54 2009
From: daniel at stutzbachenterprises.com (Daniel Stutzbach)
Date: Mon, 9 Nov 2009 09:09:54 -0600
Subject: [Python-Dev] Retrieve an arbitrary element from
	asetwithoutremoving it
In-Reply-To: <4AF7D5E7.4080901@v.loewis.de>
References: <7b0045ed0910301843y7059d316s8a851142b78b7aeb@mail.gmail.com>
	<f3cc57c60911041332v479cc3b0t19d9601e52002128@mail.gmail.com>
	<99BB308202224D31AA5300223DB59912@RaymondLaptop1>
	<7b0045ed0911051243q18f8324ak9abaa98ac26948e9@mail.gmail.com>
	<4AF341BD.2010203@v.loewis.de>
	<7b0045ed0911051419i76f4221dye8ecccc424179ec6@mail.gmail.com>
	<BD050FFAAA19443DA0DA34FD2BD740CF@RaymondLaptop1>
	<97069B15530D4C66AC1D4BCCB70E2FD5@RaymondLaptop1>
	<eae285400911060617s15d8f0bpaa2c3c4d6d946d41@mail.gmail.com>
	<4AF7D5E7.4080901@v.loewis.de>
Message-ID: <eae285400911090709r550bc9d5pf9d3990f603e7eac@mail.gmail.com>

On Mon, Nov 9, 2009 at 2:42 AM, "Martin v. L?wis" <martin at v.loewis.de>wrote:

> Interesting. Something goes wrong, it seems: if items get removed over
> and over again, I think the set should shrink (not sure whether it
> actually does). Then, I think you should end up with an amortized O(1)
> for selecting an element (assuming that the underlying hashes don't
> collide).
>

I'm not sure if Python's implementation shrinks or not, but even if it did
shrink, it would still be amortized O(n).

Imagine a completely full hash table that currently contains n elements in n
slots (I know this cannot happen in Python's implementation but it's useful
for illustrative purposes).  Assume it will shrink when it gets down to n/2
elements.

Here is my pathological algorithm again, for reference purposes:

while s:
    for i in s:
        break
    # Imagine a complex algorithm here that depends on i still being in s
    s.remove(i)

We repeatedly search through the slots sequentially and remove the first
element we find.  The first removal requires checking 1 slot, the second
removal requires checking 2 slots, the third removal requires checking 3
slots, and so forth.  The hash table will shrink after the n/2-th removal,
when we have checked 1 + 2 + 3 + ... + n/2 = O(n**2) slots for n/2 removals
(or amortized O(n) per removal).  It's too late for shrinking to save us;
we've already performed too much work.

--
Daniel Stutzbach, Ph.D.
President, Stutzbach Enterprises, LLC <http://stutzbachenterprises.com>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20091109/e9b61dac/attachment.htm>

From alexander.belopolsky at gmail.com  Mon Nov  9 16:36:27 2009
From: alexander.belopolsky at gmail.com (Alexander Belopolsky)
Date: Mon, 9 Nov 2009 10:36:27 -0500
Subject: [Python-Dev] Retrieve an arbitrary element from
	asetwithoutremoving it
In-Reply-To: <eae285400911090709r550bc9d5pf9d3990f603e7eac@mail.gmail.com>
References: <7b0045ed0910301843y7059d316s8a851142b78b7aeb@mail.gmail.com>
	<99BB308202224D31AA5300223DB59912@RaymondLaptop1>
	<7b0045ed0911051243q18f8324ak9abaa98ac26948e9@mail.gmail.com>
	<4AF341BD.2010203@v.loewis.de>
	<7b0045ed0911051419i76f4221dye8ecccc424179ec6@mail.gmail.com>
	<BD050FFAAA19443DA0DA34FD2BD740CF@RaymondLaptop1>
	<97069B15530D4C66AC1D4BCCB70E2FD5@RaymondLaptop1>
	<eae285400911060617s15d8f0bpaa2c3c4d6d946d41@mail.gmail.com>
	<4AF7D5E7.4080901@v.loewis.de>
	<eae285400911090709r550bc9d5pf9d3990f603e7eac@mail.gmail.com>
Message-ID: <d38f5330911090736m4aaf0f8na6ceb584fb7c2a9a@mail.gmail.com>

On Mon, Nov 9, 2009 at 10:09 AM, Daniel Stutzbach
<daniel at stutzbachenterprises.com> wrote:
> On Mon, Nov 9, 2009 at 2:42 AM, "Martin v. L?wis" <martin at v.loewis.de>
> wrote:
>>
>> Interesting. Something goes wrong, it seems: if items get removed over
>> and over again, I think the set should shrink (not sure whether it
>> actually does). Then, I think you should end up with an amortized O(1)
>> for selecting an element (assuming that the underlying hashes don't
>> collide).
>
> I'm not sure if Python's implementation shrinks or not,

It does not:

>>> s = set(range(100000))
>>> from sys import getsizeof
>>> getsizeof(s)
4194536
>>> while s: x = s.pop()
...
>>> getsizeof(s)
4194536
>>> s.clear()
>>> getsizeof(s)
232

From bjourne at gmail.com  Mon Nov  9 17:40:11 2009
From: bjourne at gmail.com (=?ISO-8859-1?Q?Bj=F6rn_Lindqvist?=)
Date: Mon, 9 Nov 2009 17:40:11 +0100
Subject: [Python-Dev] Retrieve an arbitrary element from
	asetwithoutremoving it
In-Reply-To: <97069B15530D4C66AC1D4BCCB70E2FD5@RaymondLaptop1>
References: <7b0045ed0910301843y7059d316s8a851142b78b7aeb@mail.gmail.com>
	<4AF0D0D7.3090403@canterbury.ac.nz>
	<200911042234.36796.steve@pearwood.info>
	<f3cc57c60911041332v479cc3b0t19d9601e52002128@mail.gmail.com>
	<99BB308202224D31AA5300223DB59912@RaymondLaptop1>
	<7b0045ed0911051243q18f8324ak9abaa98ac26948e9@mail.gmail.com>
	<4AF341BD.2010203@v.loewis.de>
	<7b0045ed0911051419i76f4221dye8ecccc424179ec6@mail.gmail.com>
	<BD050FFAAA19443DA0DA34FD2BD740CF@RaymondLaptop1>
	<97069B15530D4C66AC1D4BCCB70E2FD5@RaymondLaptop1>
Message-ID: <740c3aec0911090840w7d3947a2ocd36b2d4ae5d70ff@mail.gmail.com>

2009/11/6 Raymond Hettinger <python at rcn.com>:
> [me]
>>
>> Why not write a short, fast get_first() function for your utils directory
>> and be done with it?
>> That could work with sets, mappings, generators, and other containers and
>> iterators.
>> No need to fatten the set/frozenset API for something so trivial and so
>> rarely needed.
>
> Forgot to post the code. ?It is short, fast, and easy. ?It is explicit about
> handing the case with an empty input. ?And it is specific about which value
> it returns (always the first iterated value; not an arbitrary one). ?There's
> no guessing about what it does. ?It gets the job done.
>
> def first(iterable):
> ? 'Return the first value from a container or iterable. ?If empty, raises
> LookupError'
> ? for value in iterable:
> ? ? ? return value
> ? raise LookupError('no first value; iterable is empty')
>
> If desired, it is not hard to change to the last time to return a default
> value or some other exception.

That function is very nice and genericly lisp-like. Couldn't that one
be in the builtins? It would both solve the problem and avoid
fattening the set API.


-- 
mvh Bj?rn

From brett at python.org  Mon Nov  9 18:12:28 2009
From: brett at python.org (Brett Cannon)
Date: Mon, 9 Nov 2009 09:12:28 -0800
Subject: [Python-Dev] PEP 3003 - Python Language Moratorium
In-Reply-To: <f3cc57c60911081950s81095bdq7c758bc74cfece74@mail.gmail.com>
References: <ca471dc20911030935l25251e0diec64427f5d7bcbf2@mail.gmail.com> 
	<ca471dc20911051405o287e0f41m26d38860a200b47f@mail.gmail.com> 
	<200911081115.01803.steve@pearwood.info>
	<200911091006.26433.steve@pearwood.info> 
	<f3cc57c60911081601l6f35f75ci9249de7b7eeed1a1@mail.gmail.com> 
	<F57B38EA-3E44-4920-8BE8-DB0AE1E21BA7@gmail.com>
	<f3cc57c60911081745o2ed32e99i7e519f6ad19418d6@mail.gmail.com> 
	<ca471dc20911081841u5049344cpc8b7fea78832fa08@mail.gmail.com> 
	<f3cc57c60911081950s81095bdq7c758bc74cfece74@mail.gmail.com>
Message-ID: <bbaeab100911090912s1d15b509q43c83ec511eda6d@mail.gmail.com>

On Sun, Nov 8, 2009 at 19:50, geremy condra <debatem1 at gmail.com> wrote:
> On Sun, Nov 8, 2009 at 9:41 PM, Guido van Rossum <guido at python.org> wrote:
>> On Sun, Nov 8, 2009 at 5:45 PM, geremy condra <debatem1 at gmail.com> wrote:
>>> I quote:
>>>
>>> "This PEP proposes a temporary moratorium (suspension) of all changes
>>> to the Python language syntax, semantics, and built-ins for a period
>>> of *at least two years* from the release of Python 3.1."
>>>
>>> Emphasis mine.
>>
>> I since added this:
>>
>> """In particular,
>> the moratorium would include Python 3.2 (to be released 18-24 months
>> after 3.1) but (unless explicitly extended) allow Python 3.3 to once
>> again include language changes."""
>>
>>> Like I say, a definite end point would be much preferred to n > 2.
>>
>> My time machine doesn't work very well in the future. So I can't tell
>> what we'll find necessary 2 years from now. But I would be fine with
>> defining the time limit to be max(time(3.1) + 2 years, time(3.2)).
>> I.e. the moratorium (unless explicitly extended) ends as soon as 3.2
>> has been released *and* at least 2 years have passed since 3.1.
>
> Ok, thanks for the clarification. Could you spell out what you would
> consider grounds for a future extension?
>

We feel it's necessary as a group or Guido does, simple as that. You
can't plan it since it's over two years away. If the time comes and
people feel the moratorium has been beneficial and should go longer,
we will extend it. It will most likely be for the same reasons we
started it.

>>> If possible, I'd also like to hear some of Steven's other points addressed.
>>
>> They haven't changed my mind.
>
> Ok, but the fact that you (or Steven) hold a particular set of beliefs
> is a singularly unconvincing argument.

I disagree. Guido is the BDFL so his set of beliefs is enough unless
faced with a huge number of people disagreeing. That has not occurred
on this topic.

>Could you explain why you
> don't agree, if only for the record?

Enough happens on python-dev based on gut feeling that there isn't a
need. If we had to spell out objections to every email we received
while discussing a PEP, threads would never end. Heck, I think this
PEP discussion as gone on long enough and that Guido could pronounce
at this point.

-Brett

From jasper at humppa.nl  Mon Nov  9 18:26:23 2009
From: jasper at humppa.nl (Jasper Lievisse Adriaanse)
Date: Mon, 9 Nov 2009 18:26:23 +0100
Subject: [Python-Dev] Python 2.6: OverflowError: signed integer is greater
	than maximum
Message-ID: <20091109172623.GA15620@gurthang.humppa.nl>

Hi,

while trying to get Python 2.6 working on OpenBSD/sgi (64-bit port)
I ran into the following during build:

OverflowError: signed integer is greater than maximum

I ran the command that triggered this by hand with -v added:

(sgi Python-2.6.3 40)$ export PATH; PATH="`pwd`:$PATH";  export PYTHONPATH; PYTHONPATH="`pwd`/Lib";  export DYLD_FRAMEWORK_PATH; DYLD_FRAMEWORK_PATH="`pwd`";  export EXE; EXE="";  cd ./Lib/plat-openbsd4;  ./regen
python$EXE -v ../../Tools/scripts/h2py.py -i '(u_long)' /usr/include/netinet/in.h
# installing zipimport hook
import zipimport # builtin
# installed zipimport hook
# /usr/obj/ports/Python-2.6.3/Python-2.6.3/Lib/site.pyc matches /usr/obj/ports/Python-2.6.3/Python-2.6.3/Lib/site.py
import site # precompiled from /usr/obj/ports/Python-2.6.3/Python-2.6.3/Lib/site.pyc
'import site' failed; traceback:
Traceback (most recent call last):
  File "/usr/obj/ports/Python-2.6.3/Python-2.6.3/Lib/site.py", line 61, in <module>
    import sys
OverflowError: signed integer is greater than maximum
import encodings # directory /usr/obj/ports/Python-2.6.3/Python-2.6.3/Lib/encodings
# /usr/obj/ports/Python-2.6.3/Python-2.6.3/Lib/encodings/__init__.pyc matches /usr/obj/ports/Python-2.6.3/Python-2.6.3/Lib/encodings/__init__.py
import encodings # precompiled from /usr/obj/ports/Python-2.6.3/Python-2.6.3/Lib/encodings/__init__.pyc
Python 2.6.3 (r263:75183, Nov  6 2009, 09:50:33) 
[GCC 3.3.5 (propolice)] on openbsd4
Type "help", "copyright", "credits" or "license" for more information.
# /usr/obj/ports/Python-2.6.3/Python-2.6.3/Lib/re.pyc matches /usr/obj/ports/Python-2.6.3/Python-2.6.3/Lib/re.py
import re # precompiled from /usr/obj/ports/Python-2.6.3/Python-2.6.3/Lib/re.pyc
Traceback (most recent call last):
  File "../../Tools/scripts/h2py.py", line 24, in <module>
    import sys, re, getopt, os
  File "/usr/obj/ports/Python-2.6.3/Python-2.6.3/Lib/re.py", line 104, in <module>
    import sys
OverflowError: signed integer is greater than maximum
# clear __builtin__._
# clear sys.path
# clear sys.argv
# clear sys.ps1
# clear sys.ps2
# clear sys.exitfunc
# clear sys.exc_type
# clear sys.exc_value
# clear sys.exc_traceback
# clear sys.last_type
# clear sys.last_value
# clear sys.last_traceback
# clear sys.path_hooks
# clear sys.path_importer_cache
# clear sys.meta_path
# clear sys.flags
# clear sys.float_info
# restore sys.stdin
# restore sys.stdout
# restore sys.stderr
# cleanup __main__
# cleanup[1] zipimport
# cleanup[1] signal
# cleanup[1] exceptions
# cleanup[1] _warnings
# cleanup sys
# cleanup __builtin__
# cleanup ints: 3 unfreed ints
# cleanup floats
(sgi plat-openbsd4 41)$ 

Does anyone know what's going on here, or how to fix it?
I can provide a full build and configure log if needed.

Cheers,
Jasper
-- 
"Intelligence should guide our actions, but in harmony with the
  texture of the situation at hand"
        -- Francisco Varela

From guido at python.org  Mon Nov  9 18:56:34 2009
From: guido at python.org (Guido van Rossum)
Date: Mon, 9 Nov 2009 09:56:34 -0800
Subject: [Python-Dev] PEP 3003 - Python Language Moratorium
In-Reply-To: <bbaeab100911090912s1d15b509q43c83ec511eda6d@mail.gmail.com>
References: <ca471dc20911030935l25251e0diec64427f5d7bcbf2@mail.gmail.com> 
	<ca471dc20911051405o287e0f41m26d38860a200b47f@mail.gmail.com> 
	<200911081115.01803.steve@pearwood.info>
	<200911091006.26433.steve@pearwood.info> 
	<f3cc57c60911081601l6f35f75ci9249de7b7eeed1a1@mail.gmail.com> 
	<F57B38EA-3E44-4920-8BE8-DB0AE1E21BA7@gmail.com>
	<f3cc57c60911081745o2ed32e99i7e519f6ad19418d6@mail.gmail.com> 
	<ca471dc20911081841u5049344cpc8b7fea78832fa08@mail.gmail.com> 
	<f3cc57c60911081950s81095bdq7c758bc74cfece74@mail.gmail.com> 
	<bbaeab100911090912s1d15b509q43c83ec511eda6d@mail.gmail.com>
Message-ID: <ca471dc20911090956g2e573ddcp84f06545622aba10@mail.gmail.com>

Thanks Brett.  I've moved the moratorium PEP to Status: Accepted. I've
added the words about inclusion of 3.2 and exclusion of 3.3 (which
were eaten by a svn conflict when I previously tried to add them) and
added a section to th end stating that an extension will require
another PEP.

--Guido

On Mon, Nov 9, 2009 at 9:12 AM, Brett Cannon <brett at python.org> wrote:
> On Sun, Nov 8, 2009 at 19:50, geremy condra <debatem1 at gmail.com> wrote:
>> On Sun, Nov 8, 2009 at 9:41 PM, Guido van Rossum <guido at python.org> wrote:
>>> On Sun, Nov 8, 2009 at 5:45 PM, geremy condra <debatem1 at gmail.com> wrote:
>>>> I quote:
>>>>
>>>> "This PEP proposes a temporary moratorium (suspension) of all changes
>>>> to the Python language syntax, semantics, and built-ins for a period
>>>> of *at least two years* from the release of Python 3.1."
>>>>
>>>> Emphasis mine.
>>>
>>> I since added this:
>>>
>>> """In particular,
>>> the moratorium would include Python 3.2 (to be released 18-24 months
>>> after 3.1) but (unless explicitly extended) allow Python 3.3 to once
>>> again include language changes."""
>>>
>>>> Like I say, a definite end point would be much preferred to n > 2.
>>>
>>> My time machine doesn't work very well in the future. So I can't tell
>>> what we'll find necessary 2 years from now. But I would be fine with
>>> defining the time limit to be max(time(3.1) + 2 years, time(3.2)).
>>> I.e. the moratorium (unless explicitly extended) ends as soon as 3.2
>>> has been released *and* at least 2 years have passed since 3.1.
>>
>> Ok, thanks for the clarification. Could you spell out what you would
>> consider grounds for a future extension?
>>
>
> We feel it's necessary as a group or Guido does, simple as that. You
> can't plan it since it's over two years away. If the time comes and
> people feel the moratorium has been beneficial and should go longer,
> we will extend it. It will most likely be for the same reasons we
> started it.
>
>>>> If possible, I'd also like to hear some of Steven's other points addressed.
>>>
>>> They haven't changed my mind.
>>
>> Ok, but the fact that you (or Steven) hold a particular set of beliefs
>> is a singularly unconvincing argument.
>
> I disagree. Guido is the BDFL so his set of beliefs is enough unless
> faced with a huge number of people disagreeing. That has not occurred
> on this topic.
>
>>Could you explain why you
>> don't agree, if only for the record?
>
> Enough happens on python-dev based on gut feeling that there isn't a
> need. If we had to spell out objections to every email we received
> while discussing a PEP, threads would never end. Heck, I think this
> PEP discussion as gone on long enough and that Guido could pronounce
> at this point.
>
> -Brett
>



-- 
--Guido van Rossum (python.org/~guido)

From dickinsm at gmail.com  Mon Nov  9 20:15:20 2009
From: dickinsm at gmail.com (Mark Dickinson)
Date: Mon, 9 Nov 2009 19:15:20 +0000
Subject: [Python-Dev] Python 2.6: OverflowError: signed integer is
	greater than maximum
In-Reply-To: <20091109172623.GA15620@gurthang.humppa.nl>
References: <20091109172623.GA15620@gurthang.humppa.nl>
Message-ID: <5c6f2a5d0911091115k1ab306ffnc264c70db6dc237a@mail.gmail.com>

On Mon, Nov 9, 2009 at 5:26 PM, Jasper Lievisse Adriaanse
<jasper at humppa.nl> wrote:
> Hi,
>
> while trying to get Python 2.6 working on OpenBSD/sgi (64-bit port)
> I ran into the following during build:
>
> OverflowError: signed integer is greater than maximum
>
> I ran the command that triggered this by hand with -v added:

[traceback snipped]

> Does anyone know what's going on here, or how to fix it?
> I can provide a full build and configure log if needed.

No idea.  Please could you file a report at http://bugs.python.org?
The build and configure log would be useful.  If you can diagnose
the source of the failure yourself any further that would be more
useful;  I suspect that the core developers will have a hard time
figuring the problem out without this.

I may well be barking up the wrong tree here, but as a first guess
it looks as though something in the _PySys_Init function in
Python/sysmodule.c is (directly or indirectly) causing the
OverflowError to be raised.  I'd try adding a bunch of
printf calls to that function that print the value of
PyErr_Occurred(), to see where (if at all) the Python exception
is being set.  (I suggest building with the --with-pydebug configure
option for this.)

Mark

From g.brandl at gmx.net  Mon Nov  9 20:51:39 2009
From: g.brandl at gmx.net (Georg Brandl)
Date: Mon, 09 Nov 2009 19:51:39 +0000
Subject: [Python-Dev] PEP 3003 - Python Language Moratorium
In-Reply-To: <20091109141609.GA8499@amk-desktop.matrixgroup.net>
References: <ca471dc20911030935l25251e0diec64427f5d7bcbf2@mail.gmail.com>	<ca471dc20911051405o287e0f41m26d38860a200b47f@mail.gmail.com>	<ea2499da0911051453n3569387aj581ad4acf5b799c8@mail.gmail.com>	<4222a8490911051521r1b9c8165id6e0f12d62da0613@mail.gmail.com>	<ca471dc20911051526u7f3642ecg19e4eb73bcca1d3a@mail.gmail.com>	<bbaeab100911051535g68cefa18kdadea33f78de9495@mail.gmail.com>	<48c184130911051555r9a5b78cs3a13cb1345d3cbf7@mail.gmail.com>	<9d153b7c0911051613o62be5dc9l52102f7805cd4a0e@mail.gmail.com>	<52dc1c820911081256q7fc3f7fdk87b2f12932dce11d@mail.gmail.com>	<hd7d4k$ek6$2@ger.gmane.org>
	<20091109141609.GA8499@amk-desktop.matrixgroup.net>
Message-ID: <hd9rou$cs4$1@ger.gmane.org>

A.M. Kuchling schrieb:
> On Sun, Nov 08, 2009 at 10:27:46PM +0100, Georg Brandl wrote:
>> Good point, I'll make that change if AMK agrees.
> 
> It's certainly fine with me. Do we want to only make that change to
> the 2.7 "What's New", or should we also do it for the 2.6 one?

Why not for 2.6 as well, it's an immediate benefit, especially since the
list of deprecations in 2.6 is huge.

Georg


From steve at pearwood.info  Mon Nov  9 21:16:56 2009
From: steve at pearwood.info (Steven D'Aprano)
Date: Tue, 10 Nov 2009 07:16:56 +1100
Subject: [Python-Dev] Retrieve an arbitrary element from
	asetwithoutremoving it
In-Reply-To: <740c3aec0911090840w7d3947a2ocd36b2d4ae5d70ff@mail.gmail.com>
References: <7b0045ed0910301843y7059d316s8a851142b78b7aeb@mail.gmail.com>
	<97069B15530D4C66AC1D4BCCB70E2FD5@RaymondLaptop1>
	<740c3aec0911090840w7d3947a2ocd36b2d4ae5d70ff@mail.gmail.com>
Message-ID: <200911100716.57045.steve@pearwood.info>

On Tue, 10 Nov 2009 03:40:11 am Bj?rn Lindqvist wrote:
> 2009/11/6 Raymond Hettinger <python at rcn.com>:
> > [me]
> >
> >> Why not write a short, fast get_first() function for your utils
> >> directory and be done with it?
> >> That could work with sets, mappings, generators, and other
> >> containers and iterators.
> >> No need to fatten the set/frozenset API for something so trivial
> >> and so rarely needed.
> >
> > Forgot to post the code. ?It is short, fast, and easy. ?It is
> > explicit about handing the case with an empty input. ?And it is
> > specific about which value it returns (always the first iterated
> > value; not an arbitrary one). ?There's no guessing about what it
> > does. ?It gets the job done.
> >
> > def first(iterable):
> > ? 'Return the first value from a container or iterable. ?If empty,
> > raises LookupError'
> > ? for value in iterable:
> > ? ? ? return value
> > ? raise LookupError('no first value; iterable is empty')
> >
> > If desired, it is not hard to change to the last time to return a
> > default value or some other exception.
>
> That function is very nice and genericly lisp-like. Couldn't that one
> be in the builtins? It would both solve the problem and avoid
> fattening the set API.


I'm not sure, but isn't that thread-unsafe?

Because sets aren't directly iterable, `for value in iterable` builds a 
set_iterator operator. If another thread modifies the set after the 
set_iterator is built, but before the value is looked up, you will get 
a mysterious RuntimeError almost impossible to debug.


>>> def first():  # simplified
...     for value in iterator:
...             return value
...
>>> dis.dis(first)
  2           0 SETUP_LOOP              15 (to 18)
              3 LOAD_GLOBAL              0 (iterator)
              6 GET_ITER
              7 FOR_ITER                 7 (to 17)
             10 STORE_FAST               0 (value)

  3          13 LOAD_FAST                0 (value)
             16 RETURN_VALUE
        >>   17 POP_BLOCK
        >>   18 LOAD_CONST               0 (None)
             21 RETURN_VALUE

 
Is it possible for another thread to be called between the GET_ITER and 
STORE_FAST? 




-- 
Steven D'Aprano

From greg.ewing at canterbury.ac.nz  Mon Nov  9 22:22:51 2009
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Tue, 10 Nov 2009 10:22:51 +1300
Subject: [Python-Dev] decimal.py: == and != comparisons involving NaNs
In-Reply-To: <loom.20091109T121728-978@post.gmane.org>
References: <20091108162648.GA4099@mail.bytereef.org>
	<4FC3B5BCE9034A79AFB0530C748F019E@RaymondLaptop1>
	<20091109104249.GA3377@mail.bytereef.org>
	<loom.20091109T121728-978@post.gmane.org>
Message-ID: <4AF8882B.4040200@canterbury.ac.nz>

Antoine Pitrou wrote:

> The problem is when searching for /another/ object which hashes the same as
> Decimal("NaN").

Maybe decimal NaNs should be unhashable, so that you can't
put them in a dictionary in the first place.

-- 
Greg

From ncoghlan at gmail.com  Mon Nov  9 22:44:10 2009
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Tue, 10 Nov 2009 07:44:10 +1000
Subject: [Python-Dev] Retrieve an arbitrary element
 from	asetwithoutremoving it
In-Reply-To: <d38f5330911090736m4aaf0f8na6ceb584fb7c2a9a@mail.gmail.com>
References: <7b0045ed0910301843y7059d316s8a851142b78b7aeb@mail.gmail.com>	<99BB308202224D31AA5300223DB59912@RaymondLaptop1>	<7b0045ed0911051243q18f8324ak9abaa98ac26948e9@mail.gmail.com>	<4AF341BD.2010203@v.loewis.de>	<7b0045ed0911051419i76f4221dye8ecccc424179ec6@mail.gmail.com>	<BD050FFAAA19443DA0DA34FD2BD740CF@RaymondLaptop1>	<97069B15530D4C66AC1D4BCCB70E2FD5@RaymondLaptop1>	<eae285400911060617s15d8f0bpaa2c3c4d6d946d41@mail.gmail.com>	<4AF7D5E7.4080901@v.loewis.de>	<eae285400911090709r550bc9d5pf9d3990f603e7eac@mail.gmail.com>
	<d38f5330911090736m4aaf0f8na6ceb584fb7c2a9a@mail.gmail.com>
Message-ID: <4AF88D2A.1020306@gmail.com>

Alexander Belopolsky wrote:
> On Mon, Nov 9, 2009 at 10:09 AM, Daniel Stutzbach
> <daniel at stutzbachenterprises.com> wrote:
>> On Mon, Nov 9, 2009 at 2:42 AM, "Martin v. L?wis" <martin at v.loewis.de>
>> wrote:
>>> Interesting. Something goes wrong, it seems: if items get removed over
>>> and over again, I think the set should shrink (not sure whether it
>>> actually does). Then, I think you should end up with an amortized O(1)
>>> for selecting an element (assuming that the underlying hashes don't
>>> collide).
>> I'm not sure if Python's implementation shrinks or not,
> 
> It does not:
> 
>>>> s = set(range(100000))
>>>> from sys import getsizeof
>>>> getsizeof(s)
> 4194536
>>>> while s: x = s.pop()
> ...
>>>> getsizeof(s)
> 4194536
>>>> s.clear()
>>>> getsizeof(s)
> 232

Interestingly, it looks like there the sparseness check isn't triggering
on addition of items either (when dictnotes.txt says it should):

>>> from sys import getsizeof
>>> s = set(range(100000))
>>> getsizeof(s)
2097264
>>> while s: x = s.pop()
...
>>> getsizeof(s)
2097264
>>> s.add(1)
>>> getsizeof(s)
2097264

I did a similar test with dict.fromkeys and that also didn't resize
until .clear() was invoked. I don't know the current criteria settings
for the sparseness check, but it seems odd that going from 100k active
keys to none wouldn't cause a resize...

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------

From martin at v.loewis.de  Mon Nov  9 22:50:11 2009
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Mon, 09 Nov 2009 22:50:11 +0100
Subject: [Python-Dev] Retrieve an arbitrary element from
 asetwithoutremoving it
In-Reply-To: <4AF819DF.6030107@gmail.com>
References: <7b0045ed0910301843y7059d316s8a851142b78b7aeb@mail.gmail.com>		<4AF0D0D7.3090403@canterbury.ac.nz>		<200911042234.36796.steve@pearwood.info>		<f3cc57c60911041332v479cc3b0t19d9601e52002128@mail.gmail.com>		<99BB308202224D31AA5300223DB59912@RaymondLaptop1>		<7b0045ed0911051243q18f8324ak9abaa98ac26948e9@mail.gmail.com>		<4AF341BD.2010203@v.loewis.de>		<7b0045ed0911051419i76f4221dye8ecccc424179ec6@mail.gmail.com>		<BD050FFAAA19443DA0DA34FD2BD740CF@RaymondLaptop1>		<97069B15530D4C66AC1D4BCCB70E2FD5@RaymondLaptop1>	<eae285400911060617s15d8f0bpaa2c3c4d6d946d41@mail.gmail.com>
	<4AF7D5E7.4080901@v.loewis.de> <4AF819DF.6030107@gmail.com>
Message-ID: <4AF88E93.4010905@v.loewis.de>

> So the rationale is to ensure that only add operations perform a resize
> and so that sequential pop() operations don't incur excessive resizing
> costs.

I agree that the use case of repeated .pop() operations is reasonable,
and (IIUC) that case is also special-cased using a finger/index.

I think for regular removal, the same logic should not apply: if a
series of removals is performed, then further (non-pop) removals
see increasing costs, as do regular lookups. So I think that a removal
should trigger shrinking (with appropriate thresholds) unless it's a
.pop.

Regards,
Martin

From martin at v.loewis.de  Mon Nov  9 23:10:56 2009
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Mon, 09 Nov 2009 23:10:56 +0100
Subject: [Python-Dev] IDLE as default Python editor
In-Reply-To: <4AF80E73.5090104@gmail.com>
References: <d34314100911082245w79c9233dwad2a0aa867e06355@mail.gmail.com>	<87iqdk2n7b.fsf@benfinney.id.au>
	<4AF80E73.5090104@gmail.com>
Message-ID: <4AF89370.2040206@v.loewis.de>

Nick Coghlan wrote:
> Ben Finney wrote:
>> anatoly techtonik <techtonik at gmail.com> writes:
>>
>>> Quite an interesting question recently popped up in pygame community
>>> that I'd like to ask to Python developers.
>> This forum is specifically about development *of* Python.
> 
> Anatoly's question is actually a fair one for python-dev - we're the
> ones that *ship* Idle, so it is legitimate to ask our reasons for
> continuing to do so.

But he didn't - instead, he asked how many of us use it. Usage by
committers is not (or shouldn't be) a primary criterion for including
or not including something. Instead, usage in the community should be,
and python-dev is indeed the wrong place to estimate that.

OTOH, the second (or, rather, third) question (does anybody think it
should be replaced) is indeed on-topic for python-dev.

I didn't really answer that question before, so I do now: I have not
personally plans to replace it, and I'm skeptical wrt. anybody else's
plans unless there is specific code in existence that IDLE could be
replaced *with*.

Regards,
Martin

From martin at v.loewis.de  Mon Nov  9 23:19:13 2009
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Mon, 09 Nov 2009 23:19:13 +0100
Subject: [Python-Dev] Retrieve an arbitrary element
 from	asetwithoutremoving it
In-Reply-To: <200911100716.57045.steve@pearwood.info>
References: <7b0045ed0910301843y7059d316s8a851142b78b7aeb@mail.gmail.com>	<97069B15530D4C66AC1D4BCCB70E2FD5@RaymondLaptop1>	<740c3aec0911090840w7d3947a2ocd36b2d4ae5d70ff@mail.gmail.com>
	<200911100716.57045.steve@pearwood.info>
Message-ID: <4AF89561.9010408@v.loewis.de>

> I'm not sure, but isn't that thread-unsafe?

You are right; it's thread-unsafe.

I would fix it by catching the RuntimeError, and retrying. Given the
current GIL strategy (including proposed changes to it), it won't happen
two times in a row, so the number of retries would be bounded.

Regards,
Martin

From martin at v.loewis.de  Mon Nov  9 23:23:11 2009
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Mon, 09 Nov 2009 23:23:11 +0100
Subject: [Python-Dev] Python 2.6: OverflowError: signed integer
 is	greater than maximum
In-Reply-To: <5c6f2a5d0911091115k1ab306ffnc264c70db6dc237a@mail.gmail.com>
References: <20091109172623.GA15620@gurthang.humppa.nl>
	<5c6f2a5d0911091115k1ab306ffnc264c70db6dc237a@mail.gmail.com>
Message-ID: <4AF8964F.7020104@v.loewis.de>

> I may well be barking up the wrong tree here, but as a first guess
> it looks as though something in the _PySys_Init function in
> Python/sysmodule.c is (directly or indirectly) causing the
> OverflowError to be raised. 

My theory would be different. There is a pending unchecked OverflowError
before the import, and the err-occurred check after the import picks it
up.

Of course, your guess is as good as mine.

While barking up trees: My guess is that it's a compiler bug (i.e. the
compiler generating bad code).

Regards,
Martin

From martin at v.loewis.de  Mon Nov  9 23:39:04 2009
From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=)
Date: Mon, 09 Nov 2009 23:39:04 +0100
Subject: [Python-Dev] Retrieve an arbitrary element from
 asetwithoutremoving it
In-Reply-To: <eae285400911090709r550bc9d5pf9d3990f603e7eac@mail.gmail.com>
References: <7b0045ed0910301843y7059d316s8a851142b78b7aeb@mail.gmail.com>	
	<f3cc57c60911041332v479cc3b0t19d9601e52002128@mail.gmail.com>	
	<99BB308202224D31AA5300223DB59912@RaymondLaptop1>	
	<7b0045ed0911051243q18f8324ak9abaa98ac26948e9@mail.gmail.com>	
	<4AF341BD.2010203@v.loewis.de>	
	<7b0045ed0911051419i76f4221dye8ecccc424179ec6@mail.gmail.com>	
	<BD050FFAAA19443DA0DA34FD2BD740CF@RaymondLaptop1>	
	<97069B15530D4C66AC1D4BCCB70E2FD5@RaymondLaptop1>	
	<eae285400911060617s15d8f0bpaa2c3c4d6d946d41@mail.gmail.com>	
	<4AF7D5E7.4080901@v.loewis.de>
	<eae285400911090709r550bc9d5pf9d3990f603e7eac@mail.gmail.com>
Message-ID: <4AF89A08.5070106@v.loewis.de>

> We repeatedly search through the slots sequentially and remove the first
> element we find.  The first removal requires checking 1 slot, the second
> removal requires checking 2 slots, the third removal requires checking 3
> slots, and so forth.  The hash table will shrink after the n/2-th
> removal, when we have checked 1 + 2 + 3 + ... + n/2 = O(n**2) slots for
> n/2 removals (or amortized O(n) per removal).  It's too late for
> shrinking to save us; we've already performed too much work.

Ah, right.

Perhaps people writing the loop the way you proposed deserve to get bad
performance, as they should use .pop in the first place.

Regards,
Martin

From daniel at stutzbachenterprises.com  Mon Nov  9 23:44:53 2009
From: daniel at stutzbachenterprises.com (Daniel Stutzbach)
Date: Mon, 9 Nov 2009 16:44:53 -0600
Subject: [Python-Dev] Retrieve an arbitrary element from
	asetwithoutremoving it
In-Reply-To: <4AF88E93.4010905@v.loewis.de>
References: <7b0045ed0910301843y7059d316s8a851142b78b7aeb@mail.gmail.com>
	<7b0045ed0911051243q18f8324ak9abaa98ac26948e9@mail.gmail.com>
	<4AF341BD.2010203@v.loewis.de>
	<7b0045ed0911051419i76f4221dye8ecccc424179ec6@mail.gmail.com>
	<BD050FFAAA19443DA0DA34FD2BD740CF@RaymondLaptop1>
	<97069B15530D4C66AC1D4BCCB70E2FD5@RaymondLaptop1>
	<eae285400911060617s15d8f0bpaa2c3c4d6d946d41@mail.gmail.com>
	<4AF7D5E7.4080901@v.loewis.de> <4AF819DF.6030107@gmail.com>
	<4AF88E93.4010905@v.loewis.de>
Message-ID: <eae285400911091444l491f12afk6d5e359b45efaf96@mail.gmail.com>

On Mon, Nov 9, 2009 at 3:50 PM, "Martin v. L?wis" <martin at v.loewis.de>wrote:

> I think for regular removal, the same logic should not apply: if a
> series of removals is performed, then further (non-pop) removals
> see increasing costs, as do regular lookups. So I think that a removal
> should trigger shrinking (with appropriate thresholds) unless it's a
> .pop.
>

Minor technicality, but it's the next() operation of the iterator that has
increasing cost as the set/dict becomes sparse. Removals are always O(1).
The removal uses the hash to find the item quickly.  The iterator has to
scan through the table for non-empty entries.

(the above assumes a good hash function with few collisions, of course)

--
Daniel Stutzbach, Ph.D.
President, Stutzbach Enterprises, LLC <http://stutzbachenterprises.com>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20091109/3736443f/attachment-0001.htm>

From tjreedy at udel.edu  Mon Nov  9 23:51:09 2009
From: tjreedy at udel.edu (Terry Reedy)
Date: Mon, 09 Nov 2009 17:51:09 -0500
Subject: [Python-Dev] decimal.py: == and != comparisons involving NaNs
In-Reply-To: <5c6f2a5d0911090525oc84f864jb1e336d32d0c6741@mail.gmail.com>
References: <20091108162648.GA4099@mail.bytereef.org>	<4FC3B5BCE9034A79AFB0530C748F019E@RaymondLaptop1>	<20091109104249.GA3377@mail.bytereef.org>	<loom.20091109T121728-978@post.gmane.org>	<20091109122159.GA4383@mail.bytereef.org>	<loom.20091109T133048-911@post.gmane.org>	<20091109132124.GA5063@mail.bytereef.org>
	<5c6f2a5d0911090525oc84f864jb1e336d32d0c6741@mail.gmail.com>
Message-ID: <hda6ct$f7c$1@ger.gmane.org>

Mark Dickinson wrote:

> That's because you're creating two different float nans.
> Compare with:
> 
> Python 3.2a0 (py3k:76132M, Nov  6 2009, 14:47:39)
> [GCC 4.2.1 (SUSE Linux)] on linux2
> Type "help", "copyright", "credits" or "license" for more information.
>>>> nan = float('nan')
>>>> d = {nan: 10, 0: 20}
>>>> nan in d
> True
>>>> d[nan]
> 10

This also suggests to me that nan should be a singleton, or at least 
that the doc should recommend that programs should make it be such for 
the program.

tjr


From jasper at humppa.nl  Tue Nov 10 00:05:27 2009
From: jasper at humppa.nl (Jasper Lievisse Adriaanse)
Date: Tue, 10 Nov 2009 00:05:27 +0100
Subject: [Python-Dev] Python 2.6: OverflowError: signed integer is
 greater than maximum
In-Reply-To: <5c6f2a5d0911091115k1ab306ffnc264c70db6dc237a@mail.gmail.com>
References: <20091109172623.GA15620@gurthang.humppa.nl>
	<5c6f2a5d0911091115k1ab306ffnc264c70db6dc237a@mail.gmail.com>
Message-ID: <20091109230527.GA7091@gurthang.humppa.nl>

On Mon, Nov 09, 2009 at 07:15:20PM +0000, Mark Dickinson wrote:
> On Mon, Nov 9, 2009 at 5:26 PM, Jasper Lievisse Adriaanse
> <jasper at humppa.nl> wrote:
> > Hi,
> >
> > while trying to get Python 2.6 working on OpenBSD/sgi (64-bit port)
> > I ran into the following during build:
> >
> > OverflowError: signed integer is greater than maximum
> >
> > I ran the command that triggered this by hand with -v added:
> 
> [traceback snipped]
> 
> > Does anyone know what's going on here, or how to fix it?
> > I can provide a full build and configure log if needed.
> 
> No idea.  Please could you file a report at http://bugs.python.org?
> The build and configure log would be useful.  If you can diagnose
> the source of the failure yourself any further that would be more
> useful;  I suspect that the core developers will have a hard time
> figuring the problem out without this.
> 
> I may well be barking up the wrong tree here, but as a first guess
> it looks as though something in the _PySys_Init function in
> Python/sysmodule.c is (directly or indirectly) causing the
> OverflowError to be raised.  I'd try adding a bunch of
> printf calls to that function that print the value of
> PyErr_Occurred(), to see where (if at all) the Python exception
> is being set.  (I suggest building with the --with-pydebug configure
> option for this.)
> 
> Mark
Hi,

for the record, I filed the bug as: http://bugs.python.org/issue7296
In the meantime I'll try digging into this issue.

  Cheers,
  Jasper

-- 
"Intelligence should guide our actions, but in harmony with the
  texture of the situation at hand"
        -- Francisco Varela

From guido at python.org  Tue Nov 10 00:20:29 2009
From: guido at python.org (Guido van Rossum)
Date: Mon, 9 Nov 2009 15:20:29 -0800
Subject: [Python-Dev] decimal.py: == and != comparisons involving NaNs
In-Reply-To: <hda6ct$f7c$1@ger.gmane.org>
References: <20091108162648.GA4099@mail.bytereef.org>
	<4FC3B5BCE9034A79AFB0530C748F019E@RaymondLaptop1> 
	<20091109104249.GA3377@mail.bytereef.org>
	<loom.20091109T121728-978@post.gmane.org> 
	<20091109122159.GA4383@mail.bytereef.org>
	<loom.20091109T133048-911@post.gmane.org> 
	<20091109132124.GA5063@mail.bytereef.org>
	<5c6f2a5d0911090525oc84f864jb1e336d32d0c6741@mail.gmail.com> 
	<hda6ct$f7c$1@ger.gmane.org>
Message-ID: <ca471dc20911091520s5994f19ag446eca18c00083fe@mail.gmail.com>

On Mon, Nov 9, 2009 at 2:51 PM, Terry Reedy <tjreedy at udel.edu> wrote:
> This also suggests to me that nan should be a singleton, or at least that
> the doc should recommend that programs should make it be such for the
> program.

The IEEE std disagreed -- there's extra info hidden in the mantissa
bits. And the Python float implementation makes it pretty impractical
to do this at the application level since x+y will generate a new
NaN-valued float object each time it is called (if the outcome is
NaN).

-- 
--Guido van Rossum (python.org/~guido)

From sparks.m at gmail.com  Tue Nov 10 02:12:13 2009
From: sparks.m at gmail.com (Michael Sparks)
Date: Tue, 10 Nov 2009 01:12:13 +0000
Subject: [Python-Dev] People want CPAN :-)
In-Reply-To: <B8D9CAC5-DA6D-47FC-AD1D-1FC388FBBC6B@gmail.com>
References: <ca471dc20911060953v1db91bbei992dcb964b6ab88b@mail.gmail.com>
	<a21398cfb92acc7d47f8151c249291c4@preisshare.net>
	<ca471dc20911061514q642ad41br9fb02d675fe411db@mail.gmail.com>
	<87iqdm7nkp.fsf@benfinney.id.au>
	<B8D9CAC5-DA6D-47FC-AD1D-1FC388FBBC6B@gmail.com>
Message-ID: <20e5c9660911091712y3548145yf0b48a0fa075220f@mail.gmail.com>

[ I'm posting this comment in reply to seeing this thread:
    * http://thread.gmane.org/gmane.comp.python.distutils.devel/11359
Which has been reposted around - and I've read that thread. I lurk on
this list, in case anything comes up that I'd hope to be able to say
something useful to. I don't know if this will be, but that's my
reason for posting. If this is the wrong place, my apologies, I don't
sub to distutils-sig :-/ ]

On Sat, Nov 7, 2009 at 2:30 PM, ssteinerX at gmail.com <ssteinerx at gmail.com> wrote:
> On Nov 7, 2009, at 3:20 AM, Ben Finney wrote:
>
>> Guido van Rossum <guido at python.org> writes:
>>
>>> On Fri, Nov 6, 2009 at 2:52 PM, David Lyon <david.lyon at preisshare.net>
>>> wrote:
>>>>
[ lots of snippage ]
...
> All in all, I think this could be a big leap forward for the Python
> distribution ecosystem whether or not we eventually write the PyPan I wished
> for as a new Perl refugee.

Speaking as someone who left the perl world for the python world, many
years ago now, primarily due to working on one project, the thing I
really miss about Perl is CPAN. It's not the fact that you know you do
perl Makefile.PL && make && make test && make install. Nor the fact
that it's trivial to set up a skeleton package setup that makes that
work for you. It's not the fact that there's an installer than can
download & track dependencies.

The thing that makes the difference IMHO is two points:
    * In a language which has a core ethos "There's more than one way
to do it", packaging is the one place where there is one, and only one
obvious way to do it. (Oddly with python, with packaging this is
flipped - do I as a random project use distutils? pip? setuptools?
distribute? virtualenv?)
    * It has a managed namespace or perhaps better - a co-ordinated namespace.

CPAN may have lots of ills, and bad aspects about it (I've never
really trusted the auto installer due to seeing one too many people
having their perl installation as a whole upgraded due to a bug that
was squashed 6-8 years ago), but these two points are pretty much
killer.

All the other aspects like auto download, upload, dependency tracking,
auto doc extraction for the website etc, really follow from the
managed namespace really. I realise that various efforts like
easy_install & distribute & friends make that sort of step implicitly
- since there can only be one http://pypi.python.org/pypi/flibble .
But it's not quite the same - due to externally hosted packages.

For more detail about this aspect:
   * http://www.cpan.org/modules/04pause.html#namespace

I'm really mentioning this because I didn't see it listed, and I
really think that it's very easy to underestimate this aspect of CPAN.

IMHO, it is what matters the most about CPAN. The fact that they've
nabbed the CTAN idea of having an archive network for storing,
mirroring and grabbing stuff from is by comparison /almost/ irrelevant
IMHO. It is the sort of thing that leads to the DBI::DBD type stuff
that is being simple to use, because of the encouragement to talk and
share a namespace.

The biggest issue with this is retrofitting this to an existing world.

Personal opinion, I hope it's useful, and going back into lurk mode (I
hope :). If this annoys you, please just ignore it.


Michael.

From arfrever.fta at gmail.com  Tue Nov 10 11:41:40 2009
From: arfrever.fta at gmail.com (Arfrever Frehtes Taifersar Arahesis)
Date: Tue, 10 Nov 2009 11:41:40 +0100
Subject: [Python-Dev] 2.7/3.2 release schedule
In-Reply-To: <1afaf6160911021200m1bcf27cck4a78db471db038a7@mail.gmail.com>
References: <1afaf6160911021200m1bcf27cck4a78db471db038a7@mail.gmail.com>
Message-ID: <71ac8a310911100241x2567c396pc2e5ef4653ced917@mail.gmail.com>

2009-11-02 21:00 Benjamin Peterson <benjamin at python.org> napisa?(a):
> I've updated PEP 373 with my proposed release schedule:
>
> - 2.7/3.2 alpha 1 2009-12-05
> - 2.7/3.2 alpha 2 2010-01-09
> - 2.7/3.2 alpha 3 2010-02-06
> - 2.7/3.2 alpha 4 2010-03-06
> - 2.7/3.2 beta 1 2010-04-03
> - 2.7/3.2 beta 2 2010-05-01
> - 2.7/3.2 rc1 2010-05-29
> - 2.7/3.2 rc2 2010-06-12
> - 2.7/3.2 final 2010-06-26

PEP 3003 states that Python 3.2 will be released 18-24 months after
Python 3.1. Python 3.1 was released on June 2009-06-27 [1], so
theoretically Python 3.2 should be released not before 2010-12-19 [2].

[1] http://python.org/download/releases/3.1/
[2] datetime.date(2009, 6, 27) + datetime.timedelta(18 * 30)

--
Arfrever Frehtes Taifersar Arahesis

From ncoghlan at gmail.com  Tue Nov 10 11:47:05 2009
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Tue, 10 Nov 2009 20:47:05 +1000
Subject: [Python-Dev] IDLE as default Python editor
In-Reply-To: <4AF89370.2040206@v.loewis.de>
References: <d34314100911082245w79c9233dwad2a0aa867e06355@mail.gmail.com>	<87iqdk2n7b.fsf@benfinney.id.au>
	<4AF80E73.5090104@gmail.com> <4AF89370.2040206@v.loewis.de>
Message-ID: <4AF944A9.2030108@gmail.com>

Martin v. L?wis wrote:
> Nick Coghlan wrote:
>> Ben Finney wrote:
>>> anatoly techtonik <techtonik at gmail.com> writes:
>>>
>>>> Quite an interesting question recently popped up in pygame community
>>>> that I'd like to ask to Python developers.
>>> This forum is specifically about development *of* Python.
>> Anatoly's question is actually a fair one for python-dev - we're the
>> ones that *ship* Idle, so it is legitimate to ask our reasons for
>> continuing to do so.
> 
> But he didn't - instead, he asked how many of us use it. Usage by
> committers is not (or shouldn't be) a primary criterion for including
> or not including something. Instead, usage in the community should be,
> and python-dev is indeed the wrong place to estimate that.

I took it as being a somewhat relevant leading question. If the response
had been "No, none of us use it" then the obvious follow-up question
would be "Why do you ship something that you don't consider worth using?".

I agree I was responding to a question that wasn't actually written in
the email though (and as I said in my other message, agree that it can't
be replaced without someone putting up a serious alternative for
consideration that ticks all the same boxes that IDLE currently ticks
and then offers significant improvements over and above that).

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------

From ncoghlan at gmail.com  Tue Nov 10 11:51:19 2009
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Tue, 10 Nov 2009 20:51:19 +1000
Subject: [Python-Dev] Retrieve an arbitrary element
 from	asetwithoutremoving it
In-Reply-To: <4AF89561.9010408@v.loewis.de>
References: <7b0045ed0910301843y7059d316s8a851142b78b7aeb@mail.gmail.com>	<97069B15530D4C66AC1D4BCCB70E2FD5@RaymondLaptop1>	<740c3aec0911090840w7d3947a2ocd36b2d4ae5d70ff@mail.gmail.com>	<200911100716.57045.steve@pearwood.info>
	<4AF89561.9010408@v.loewis.de>
Message-ID: <4AF945A7.5090904@gmail.com>

Martin v. L?wis wrote:
>> I'm not sure, but isn't that thread-unsafe?
> 
> You are right; it's thread-unsafe.
> 
> I would fix it by catching the RuntimeError, and retrying. Given the
> current GIL strategy (including proposed changes to it), it won't happen
> two times in a row, so the number of retries would be bounded.

It's also one of the major reasons for not sharing mutable containers
between threads if you can avoid it (and serialising access to them if
you can't)

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------

From wa2n39 at gmail.com  Wed Nov 11 03:17:19 2009
From: wa2n39 at gmail.com (wa2n39 at gmail.com)
Date: Wed, 11 Nov 2009 02:17:19 +0000
Subject: [Python-Dev] Python-Dev Digest, Vol 76, Issue 83
Message-ID: <258427219-1257851199-cardhu_decombobulator_blackberry.rim.net-1355761493-@bda056.bisx.prodap.on.blackberry>

Unreg
------Pesan Asli------
Dari:python-dev-request at python.org
Pengirim:python-dev-bounces+wa2n39=gmail.com at python.org
Ke:python-dev at python.org
Balas Ke:python-dev at python.org
Perihal:Python-Dev Digest, Vol 76, Issue 83
Terkirim:10 Nov 2009 03:00

Send Python-Dev mailing list submissions to
	python-dev at python.org

To subscribe or unsubscribe via the World Wide Web, visit
	http://mail.python.org/mailman/listinfo/python-dev
or, via email, send a message with subject or body 'help' to
	python-dev-request at python.org

You can reach the person managing the list at
	python-dev-owner at python.org

When replying, please edit your Subject line so it is more specific
than "Re: Contents of Python-Dev digest..."


Today's Topics:

   1. Re: Retrieve an arbitrary element from	asetwithoutremoving it
      (Nick Coghlan)


----------------------------------------------------------------------

Message: 1
Date: Tue, 10 Nov 2009 20:51:19 +1000
From: Nick Coghlan <ncoghlan at gmail.com>
To: "Martin v. L?wis" <martin at v.loewis.de>
Cc: python-dev at python.org
Subject: Re: [Python-Dev] Retrieve an arbitrary element from
	asetwithoutremoving it
Message-ID: <4AF945A7.5090904 at gmail.com>
Content-Type: text/plain; charset=ISO-8859-1

Martin v. L?wis wrote:
>> I'm not sure, but isn't that thread-unsafe?
> 
> You are right; it's thread-unsafe.
> 
> I would fix it by catching the RuntimeError, and retrying. Given the
> current GIL strategy (including proposed changes to it), it won't happen
> two times in a row, so the number of retries would be bounded.

It's also one of the major reasons for not sharing mutable containers
between threads if you can avoid it (and serialising access to them if
you can't)

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------


------------------------------

_______________________________________________
Python-Dev mailing list
Python-Dev at python.org
http://mail.python.org/mailman/listinfo/python-dev


End of Python-Dev Digest, Vol 76, Issue 83
******************************************


Sent from my BlackBerry?
powered by Sinyal Kuat INDOSAT

From ncoghlan at gmail.com  Tue Nov 10 14:13:28 2009
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Tue, 10 Nov 2009 23:13:28 +1000
Subject: [Python-Dev] 2.7/3.2 release schedule
In-Reply-To: <71ac8a310911100241x2567c396pc2e5ef4653ced917@mail.gmail.com>
References: <1afaf6160911021200m1bcf27cck4a78db471db038a7@mail.gmail.com>
	<71ac8a310911100241x2567c396pc2e5ef4653ced917@mail.gmail.com>
Message-ID: <4AF966F8.5040008@gmail.com>

Arfrever Frehtes Taifersar Arahesis wrote:
> 2009-11-02 21:00 Benjamin Peterson <benjamin at python.org> napisa?(a):
>> I've updated PEP 373 with my proposed release schedule:
>>
>> - 2.7/3.2 alpha 1 2009-12-05
>> - 2.7/3.2 alpha 2 2010-01-09
>> - 2.7/3.2 alpha 3 2010-02-06
>> - 2.7/3.2 alpha 4 2010-03-06
>> - 2.7/3.2 beta 1 2010-04-03
>> - 2.7/3.2 beta 2 2010-05-01
>> - 2.7/3.2 rc1 2010-05-29
>> - 2.7/3.2 rc2 2010-06-12
>> - 2.7/3.2 final 2010-06-26
> 
> PEP 3003 states that Python 3.2 will be released 18-24 months after
> Python 3.1. Python 3.1 was released on June 2009-06-27 [1], so
> theoretically Python 3.2 should be released not before 2010-12-19 [2].

The PEP 3003 text isn't allowing for the fact that 3.1 is "3.0 as it
should have been", so the starting point for the 18-24 month rule of
thumb is actually back when 3.0 was released in Dec 2008. This was
discussed a fair bit back when the decision was made to do the short
release cycle between 3.0 and 3.1 in order to address some of the more
glaring shortcomings of the 3.0 release.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------

From solipsis at pitrou.net  Tue Nov 10 14:55:48 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Tue, 10 Nov 2009 13:55:48 +0000 (UTC)
Subject: [Python-Dev]
	=?utf-8?q?Retrieve_an_arbitrary_element_from=09asetw?=
	=?utf-8?q?ithoutremoving_it?=
References: <7b0045ed0910301843y7059d316s8a851142b78b7aeb@mail.gmail.com>	<97069B15530D4C66AC1D4BCCB70E2FD5@RaymondLaptop1>	<740c3aec0911090840w7d3947a2ocd36b2d4ae5d70ff@mail.gmail.com>	<200911100716.57045.steve@pearwood.info>
	<4AF89561.9010408@v.loewis.de> <4AF945A7.5090904@gmail.com>
Message-ID: <loom.20091110T145503-189@post.gmane.org>

Nick Coghlan <ncoghlan <at> gmail.com> writes:
> 
> It's also one of the major reasons for not sharing mutable containers
> between threads if you can avoid it (and serialising access to them if
> you can't)

Not necessarily, for example it is common to rely on the fact that list.append()
is atomic.

Regards

Antoine.



From barry at python.org  Mon Nov  9 15:12:02 2009
From: barry at python.org (Barry Warsaw)
Date: Mon, 9 Nov 2009 08:12:02 -0600
Subject: [Python-Dev] OpenSSL vulnerability
In-Reply-To: <4AF71457.3050705@v.loewis.de>
References: <ca471dc20911061222v5c380643te7d5a4e84d3496f9@mail.gmail.com>
	<40373.1257545902@parc.com>	<20091106223634.3229.2049231320.divmod.xquotient.232@localhost.localdomain>
	<ca471dc20911061512j1f5f7e45nb5fd2de0d2734d45@mail.gmail.com>
	<4AF71457.3050705@v.loewis.de>
Message-ID: <3BC6D0BD-9677-40E2-9CF8-2353E55278FD@python.org>

On Nov 8, 2009, at 12:56 PM, Martin v. L?wis wrote:

>>> Also, for Python 2.5 and earlier, any SSL-based code is vulnerable  
>>> to a MitM
>>> anyway, so this can only be an issue for code using the new APIs  
>>> in Python
>>> 2.6.
>>
>> That's not going to stop the
>> wannabe-self-proclaimed-so-called-vulnerability-"experts" from  
>> whining
>> about Python not releasing updated binary distributions though. :-(
>
> The Windows binaries currently build with 0.9.8g. Since changing that
> would be a source code change (even though just a single line), I  
> think
> a full source release would be necessary (most likely then for both  
> 2.6
> and 3.1).

I don't think it's worth making a quick 2.6.5 release for this if it's  
primary intent is to produce new Windows binaries.  I'm okay with  
making the changes to the tree, but we'll release 2.6.5 on a "normal"  
schedule.

-Barry

-------------- next part --------------
A non-text attachment was scrubbed...
Name: PGP.sig
Type: application/pgp-signature
Size: 194 bytes
Desc: This is a digitally signed message part
URL: <http://mail.python.org/pipermail/python-dev/attachments/20091109/38db296d/attachment.pgp>

From ncoghlan at gmail.com  Tue Nov 10 15:27:30 2009
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Wed, 11 Nov 2009 00:27:30 +1000
Subject: [Python-Dev] Retrieve an arbitrary element from
 asetwithoutremoving it
In-Reply-To: <loom.20091110T145503-189@post.gmane.org>
References: <7b0045ed0910301843y7059d316s8a851142b78b7aeb@mail.gmail.com>	<97069B15530D4C66AC1D4BCCB70E2FD5@RaymondLaptop1>	<740c3aec0911090840w7d3947a2ocd36b2d4ae5d70ff@mail.gmail.com>	<200911100716.57045.steve@pearwood.info>	<4AF89561.9010408@v.loewis.de>
	<4AF945A7.5090904@gmail.com>
	<loom.20091110T145503-189@post.gmane.org>
Message-ID: <4AF97852.9020100@gmail.com>

Antoine Pitrou wrote:
> Nick Coghlan <ncoghlan <at> gmail.com> writes:
>> It's also one of the major reasons for not sharing mutable containers
>> between threads if you can avoid it (and serialising access to them if
>> you can't)
> 
> Not necessarily, for example it is common to rely on the fact that list.append()
> is atomic.

s/"mutable
containers"/"mutable-containers-that-object-loudly-to-their-size-changing-during-iteration
-like-sets-and-dicts" :)

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------

From ncoghlan at gmail.com  Tue Nov 10 15:28:50 2009
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Wed, 11 Nov 2009 00:28:50 +1000
Subject: [Python-Dev] OpenSSL vulnerability
In-Reply-To: <3BC6D0BD-9677-40E2-9CF8-2353E55278FD@python.org>
References: <ca471dc20911061222v5c380643te7d5a4e84d3496f9@mail.gmail.com>	<40373.1257545902@parc.com>	<20091106223634.3229.2049231320.divmod.xquotient.232@localhost.localdomain>	<ca471dc20911061512j1f5f7e45nb5fd2de0d2734d45@mail.gmail.com>	<4AF71457.3050705@v.loewis.de>
	<3BC6D0BD-9677-40E2-9CF8-2353E55278FD@python.org>
Message-ID: <4AF978A2.2090304@gmail.com>

Barry Warsaw wrote:
> I don't think it's worth making a quick 2.6.5 release for this if it's
> primary intent is to produce new Windows binaries.  I'm okay with making
> the changes to the tree, but we'll release 2.6.5 on a "normal" schedule.

Perhaps publish a source patch relative to 2.6.4 for people that would
like to rebuild their own Windows binaries with just that change?

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------

From barry at python.org  Tue Nov 10 15:34:40 2009
From: barry at python.org (Barry Warsaw)
Date: Tue, 10 Nov 2009 08:34:40 -0600
Subject: [Python-Dev] OpenSSL vulnerability
In-Reply-To: <4AF978A2.2090304@gmail.com>
References: <ca471dc20911061222v5c380643te7d5a4e84d3496f9@mail.gmail.com>	<40373.1257545902@parc.com>	<20091106223634.3229.2049231320.divmod.xquotient.232@localhost.localdomain>	<ca471dc20911061512j1f5f7e45nb5fd2de0d2734d45@mail.gmail.com>	<4AF71457.3050705@v.loewis.de>
	<3BC6D0BD-9677-40E2-9CF8-2353E55278FD@python.org>
	<4AF978A2.2090304@gmail.com>
Message-ID: <4A6F4151-3083-4745-B76A-F16921966ED9@python.org>

On Nov 10, 2009, at 8:28 AM, Nick Coghlan wrote:

> Barry Warsaw wrote:
>> I don't think it's worth making a quick 2.6.5 release for this if  
>> it's
>> primary intent is to produce new Windows binaries.  I'm okay with  
>> making
>> the changes to the tree, but we'll release 2.6.5 on a "normal"  
>> schedule.
>
> Perhaps publish a source patch relative to 2.6.4 for people that would
> like to rebuild their own Windows binaries with just that change?

+1.  A link to that could easily go on the 2.6.4 page.  Perhaps the  
right way to do it is to put the blessed patch in roundup and add a  
link to that patch file on the website.

-Barry

-------------- next part --------------
A non-text attachment was scrubbed...
Name: PGP.sig
Type: application/pgp-signature
Size: 194 bytes
Desc: This is a digitally signed message part
URL: <http://mail.python.org/pipermail/python-dev/attachments/20091110/6c2ff832/attachment.pgp>

From guido at python.org  Tue Nov 10 16:55:11 2009
From: guido at python.org (Guido van Rossum)
Date: Tue, 10 Nov 2009 07:55:11 -0800
Subject: [Python-Dev] 2.7/3.2 release schedule
In-Reply-To: <4AF966F8.5040008@gmail.com>
References: <1afaf6160911021200m1bcf27cck4a78db471db038a7@mail.gmail.com> 
	<71ac8a310911100241x2567c396pc2e5ef4653ced917@mail.gmail.com> 
	<4AF966F8.5040008@gmail.com>
Message-ID: <ca471dc20911100755m6c4bb37ei6375228e4fa3e962@mail.gmail.com>

On Tue, Nov 10, 2009 at 5:13 AM, Nick Coghlan <ncoghlan at gmail.com> wrote:
> Arfrever Frehtes Taifersar Arahesis wrote:
>> 2009-11-02 21:00 Benjamin Peterson <benjamin at python.org> napisa?(a):
>>> I've updated PEP 373 with my proposed release schedule:
>>>
>>> - 2.7/3.2 alpha 1 2009-12-05
>>> - 2.7/3.2 alpha 2 2010-01-09
>>> - 2.7/3.2 alpha 3 2010-02-06
>>> - 2.7/3.2 alpha 4 2010-03-06
>>> - 2.7/3.2 beta 1 2010-04-03
>>> - 2.7/3.2 beta 2 2010-05-01
>>> - 2.7/3.2 rc1 2010-05-29
>>> - 2.7/3.2 rc2 2010-06-12
>>> - 2.7/3.2 final 2010-06-26
>>
>> PEP 3003 states that Python 3.2 will be released 18-24 months after
>> Python 3.1. Python 3.1 was released on June 2009-06-27 [1], so
>> theoretically Python 3.2 should be released not before 2010-12-19 [2].
>
> The PEP 3003 text isn't allowing for the fact that 3.1 is "3.0 as it
> should have been", so the starting point for the 18-24 month rule of
> thumb is actually back when 3.0 was released in Dec 2008. This was
> discussed a fair bit back when the decision was made to do the short
> release cycle between 3.0 and 3.1 in order to address some of the more
> glaring shortcomings of the 3.0 release.

Was this discussed somewhere? When I agreed to an early 3.1 release
(or did I propose it?) I'm quite sure that I expected 3.2 to come the
usual time (i.e., 18-24 months) after 3.1. I think I said something to
the extent of "we'll treat 3.1 the same way we treat any release"
which IMO implies a lifetime of 18-24 months.

-- 
--Guido van Rossum (python.org/~guido)

From solipsis at pitrou.net  Tue Nov 10 17:09:53 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Tue, 10 Nov 2009 16:09:53 +0000 (UTC)
Subject: [Python-Dev] 2.7/3.2 release schedule
References: <1afaf6160911021200m1bcf27cck4a78db471db038a7@mail.gmail.com>
	<71ac8a310911100241x2567c396pc2e5ef4653ced917@mail.gmail.com>
	<4AF966F8.5040008@gmail.com>
	<ca471dc20911100755m6c4bb37ei6375228e4fa3e962@mail.gmail.com>
Message-ID: <loom.20091110T170843-458@post.gmane.org>

Guido van Rossum <guido <at> python.org> writes:
> 
> Was this discussed somewhere?

I don't remember so, except for a short subthread on python-ideas where you
indeed mentioned (to my disappointment :-)) that you were against a one-year
release period.

Regards

Antoine.



From techtonik at gmail.com  Tue Nov 10 17:20:58 2009
From: techtonik at gmail.com (anatoly techtonik)
Date: Tue, 10 Nov 2009 18:20:58 +0200
Subject: [Python-Dev] Replacing IDLE
Message-ID: <d34314100911100820u7a9e6f7bp6dde4ec8d2ff2c4@mail.gmail.com>

On Tue, Nov 10, 2009 at 12:10 AM, "Martin v. L?wis" <martin at v.loewis.de> wrote:
>>
>> Anatoly's question is actually a fair one for python-dev - we're the
>> ones that *ship* Idle, so it is legitimate to ask our reasons for
>> continuing to do so.
>
> OTOH, the second (or, rather, third) question (does anybody think it
> should be replaced) is indeed on-topic for python-dev.

That is the exact question I have in my mind. Sorry for not being able
to state it clearly. I'll be more straightforward next time.

> I didn't really answer that question before, so I do now: I have not
> personally plans to replace it, and I'm skeptical wrt. anybody else's
> plans unless there is specific code in existence that IDLE could be
> replaced *with*.

There are a lot of Python editors. It is unlikely that they implement
all IDLE features, but I believe that 20% of IDLE features that is
used by 80% of users are guaranteed by any editor out of the box.
Assuming that they are more extensible than IDLE anyway. (I would say
"perfection is achieved when there is nothing more to add...", but I
promised to be straightforward).

If we filter list of http://wiki.python.org/moin/PythonEditors by
language/license/framework, we will be able to see if there is any
suitable open source Python code to replace IDLE's.


Then there will be another issue - all editors are based upon some
frameworks - I didn't see any popular cross-platform GUI toolkits in
Python, so we will inevitably face the need to replace Tkinter with
other default GUI toolkit.

Should we place an editor first and combine the list a of unique
features of IDLE that we (as developers) will surely miss  OR  should
we think what GUI framework will be more beneficial for Python users
if shipped by default (keeping in mind that we might have to add
missing features to whatever editor we choose later)?

-- 
anatoly t.

From ronaldoussoren at mac.com  Tue Nov 10 17:44:29 2009
From: ronaldoussoren at mac.com (Ronald Oussoren)
Date: Tue, 10 Nov 2009 17:44:29 +0100
Subject: [Python-Dev] Replacing IDLE
In-Reply-To: <d34314100911100820u7a9e6f7bp6dde4ec8d2ff2c4@mail.gmail.com>
References: <d34314100911100820u7a9e6f7bp6dde4ec8d2ff2c4@mail.gmail.com>
Message-ID: <59691AA2-995E-452B-B21D-0C7F8861C7A6@mac.com>


On 10 Nov, 2009, at 17:20, anatoly techtonik wrote:

> On Tue, Nov 10, 2009 at 12:10 AM, "Martin v. L?wis" <martin at v.loewis.de> wrote:
>>> 
>>> Anatoly's question is actually a fair one for python-dev - we're the
>>> ones that *ship* Idle, so it is legitimate to ask our reasons for
>>> continuing to do so.
>> 
>> OTOH, the second (or, rather, third) question (does anybody think it
>> should be replaced) is indeed on-topic for python-dev.
> 
> That is the exact question I have in my mind. Sorry for not being able
> to state it clearly. I'll be more straightforward next time.

What's so bad about IDLE that you'd like to replace it? The obvious alternative to replacing IDLE is to fix whatever is wrong with it. 

I sometimes use IDLE and it's not too bad as a simple Python editor.  A major reason for not using it more often is that the keystrokes for my prefered text-editor are in my muscle memory and IDLE is not that much of an improvement over my current habits that I'm willing the re-educate myself.

Ronald
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 3567 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/python-dev/attachments/20091110/e5b58163/attachment-0001.bin>

From guido at python.org  Tue Nov 10 18:26:31 2009
From: guido at python.org (Guido van Rossum)
Date: Tue, 10 Nov 2009 09:26:31 -0800
Subject: [Python-Dev] 2.7/3.2 release schedule
In-Reply-To: <loom.20091110T170843-458@post.gmane.org>
References: <1afaf6160911021200m1bcf27cck4a78db471db038a7@mail.gmail.com> 
	<71ac8a310911100241x2567c396pc2e5ef4653ced917@mail.gmail.com> 
	<4AF966F8.5040008@gmail.com>
	<ca471dc20911100755m6c4bb37ei6375228e4fa3e962@mail.gmail.com> 
	<loom.20091110T170843-458@post.gmane.org>
Message-ID: <ca471dc20911100926wb5106c4j1b81290df2e60da9@mail.gmail.com>

On Tue, Nov 10, 2009 at 8:09 AM, Antoine Pitrou <solipsis at pitrou.net> wrote:
> Guido van Rossum <guido <at> python.org> writes:
>>
>> Was this discussed somewhere?
>
> I don't remember so, except for a short subthread on python-ideas where you
> indeed mentioned (to my disappointment :-)) that you were against a one-year
> release period.

Trust me on this too, please. We used to have releases once a year and
we got really big serious feedback from our biggest users that the
release cycle was going too fast. We discussed it amply and agreed on
a minimum time of 18 months between releases. This was quite a while
ago (I recall it being somewhere between 2000 and 2004) but I don't
see that the situation has really changed -- if anything, we need to
slow down more. We should really have a PEP for this, like we do for
bugfix releases (PEP 6), but at the time we weren't so anal about PEPs
for process.

I realize this can be frustrating for developers who want to see their
code released. I had the same feeling at the time. But when it was
explained to me what a version upgrade looks like for a typical large
company who have 100,000 or more lines of Python, often with
insufficient tests, written by people who no longer work for the
company or expensive consultants, I realized that releasing once a
year was a break-neck pace for such users. There was wide discussion
and community agreement on the 18 months minimum, as a compromise
between the needs of large users (who would have been happier with two
years) and the desires of developers (who, like you, preferred shorter
release cycles).

I really don't think we should change this "contract with our users"
now. If necessary, we'll have to write a PEP to describe and explain
it.

-- 
--Guido van Rossum (python.org/~guido)

From regebro at gmail.com  Tue Nov 10 18:27:00 2009
From: regebro at gmail.com (Lennart Regebro)
Date: Tue, 10 Nov 2009 18:27:00 +0100
Subject: [Python-Dev] raw binary data and 2to3
In-Reply-To: <1afaf6160911081748u3f47f5dche74b1f0171c5bcff@mail.gmail.com>
References: <m2k4y0sce3.fsf@montanaro.dyndns.org>
	<1afaf6160911081748u3f47f5dche74b1f0171c5bcff@mail.gmail.com>
Message-ID: <319e029f0911100927k78a05be4o1d3e82ba5fb12873@mail.gmail.com>

2009/11/9 Benjamin Peterson <benjamin at python.org>:
> Not to my knowledge. I would prefer to not add a fixer for this
> directly to 2to3 because it is not correct for most programs. However,
> I think 2to3 should grow some sort of plugin system, so custom fixers
> can easily be written and used.

Well, 2to3 is only plugins, so. :) The difficulty in writing fixers is
understanding the abstract parse tree or whatever it's called and how
to use it. Documentation on that for people who don't have a doctorate
in computer linguistics would probably be a good idea. ;)

-- 
Lennart Regebro: Python, Zope, Plone, Grok
http://regebro.wordpress.com/
+33 661 58 14 64

From python at rcn.com  Tue Nov 10 19:26:13 2009
From: python at rcn.com (Raymond Hettinger)
Date: Tue, 10 Nov 2009 10:26:13 -0800
Subject: [Python-Dev] 2.7/3.2 release schedule
References: <1afaf6160911021200m1bcf27cck4a78db471db038a7@mail.gmail.com>
	<71ac8a310911100241x2567c396pc2e5ef4653ced917@mail.gmail.com>
	<4AF966F8.5040008@gmail.com><ca471dc20911100755m6c4bb37ei6375228e4fa3e962@mail.gmail.com>
	<loom.20091110T170843-458@post.gmane.org>
	<ca471dc20911100926wb5106c4j1b81290df2e60da9@mail.gmail.com>
Message-ID: <5FFB0E2375804AD28576A8C3EE953A9B@RaymondLaptop1>


[Guido van Rossum]
>. We used to have releases once a year and
> we got really big serious feedback from our biggest users that the
> release cycle was going too fast. We discussed it amply and agreed on
> a minimum time of 18 months between releases. 

If the language moratorium goes into effect, would shorter release cycles
still have a negative impact?  Do people possibly want slower changes
to the language and faster updates to the library?

I don't know the answer.  Just asking how this matches up with the
feedback you have gotten previously.


Raymond

From tseaver at palladion.com  Tue Nov 10 19:42:58 2009
From: tseaver at palladion.com (Tres Seaver)
Date: Tue, 10 Nov 2009 13:42:58 -0500
Subject: [Python-Dev] 2.7/3.2 release schedule
In-Reply-To: <5FFB0E2375804AD28576A8C3EE953A9B@RaymondLaptop1>
References: <1afaf6160911021200m1bcf27cck4a78db471db038a7@mail.gmail.com>	<71ac8a310911100241x2567c396pc2e5ef4653ced917@mail.gmail.com>	<4AF966F8.5040008@gmail.com><ca471dc20911100755m6c4bb37ei6375228e4fa3e962@mail.gmail.com>	<loom.20091110T170843-458@post.gmane.org>	<ca471dc20911100926wb5106c4j1b81290df2e60da9@mail.gmail.com>
	<5FFB0E2375804AD28576A8C3EE953A9B@RaymondLaptop1>
Message-ID: <hdcc7h$p58$1@ger.gmane.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Raymond Hettinger wrote:
> [Guido van Rossum]
>> . We used to have releases once a year and
>> we got really big serious feedback from our biggest users that the
>> release cycle was going too fast. We discussed it amply and agreed on
>> a minimum time of 18 months between releases. 
> 
> If the language moratorium goes into effect, would shorter release cycles
> still have a negative impact?  Do people possibly want slower changes
> to the language and faster updates to the library?
> 
> I don't know the answer.  Just asking how this matches up with the
> feedback you have gotten previously.

Depends on the kind of changes you are talking about:  backward-
incompatible ones (like the hashlib / medusa ones in 2.6, for instance)
are likely going to be unpopular with the "Python with a tie" crowd (I
think that was the name of the IPC BoF from which the original feedback
coalesced).


Tres.
- --
===================================================================
Tres Seaver          +1 540-429-0999          tseaver at palladion.com
Palladion Software   "Excellence by Design"    http://palladion.com
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iEYEARECAAYFAkr5tC0ACgkQ+gerLs4ltQ4QzQCg1eUsdnAEyIficxjeFQevR9ul
pv0An2qpDswkGhqAHkM2REtGE9Zpz8+V
=t5kK
-----END PGP SIGNATURE-----


From fuzzyman at voidspace.org.uk  Tue Nov 10 20:26:47 2009
From: fuzzyman at voidspace.org.uk (Michael Foord)
Date: Tue, 10 Nov 2009 20:26:47 +0100
Subject: [Python-Dev] Replacing IDLE
In-Reply-To: <d34314100911100820u7a9e6f7bp6dde4ec8d2ff2c4@mail.gmail.com>
References: <d34314100911100820u7a9e6f7bp6dde4ec8d2ff2c4@mail.gmail.com>
Message-ID: <4AF9BE77.5050100@voidspace.org.uk>

anatoly techtonik wrote:
> On Tue, Nov 10, 2009 at 12:10 AM, "Martin v. L?wis" <martin at v.loewis.de> wrote:
>   
>>> Anatoly's question is actually a fair one for python-dev - we're the
>>> ones that *ship* Idle, so it is legitimate to ask our reasons for
>>> continuing to do so.
>>>       
>> OTOH, the second (or, rather, third) question (does anybody think it
>> should be replaced) is indeed on-topic for python-dev.
>>     
>
> That is the exact question I have in my mind. Sorry for not being able
> to state it clearly. I'll be more straightforward next time.
>
>   
>> I didn't really answer that question before, so I do now: I have not
>> personally plans to replace it, and I'm skeptical wrt. anybody else's
>> plans unless there is specific code in existence that IDLE could be
>> replaced *with*.
>>     
>
> There are a lot of Python editors. It is unlikely that they implement
> all IDLE features, but I believe that 20% of IDLE features that is
> used by 80% of users are guaranteed by any editor out of the box.
> Assuming that they are more extensible than IDLE anyway. (I would say
> "perfection is achieved when there is nothing more to add...", but I
> promised to be straightforward).
>
> If we filter list of http://wiki.python.org/moin/PythonEditors by
> language/license/framework, we will be able to see if there is any
> suitable open source Python code to replace IDLE's.
>
>   

Why replace IDLE?

> Then there will be another issue - all editors are based upon some
> frameworks - I didn't see any popular cross-platform GUI toolkits in
> Python, so we will inevitably face the need to replace Tkinter with
> other default GUI toolkit.
>
>   
This just *won't* happen. (At least not in the forseeable future.) 
Better filter your list to only those IDEs that use Tkinter or have a 
GUI not dependent on packages that aren't already in the standard library.


Michael

-- 
http://www.ironpythoninaction.com/


From martin at v.loewis.de  Tue Nov 10 21:49:28 2009
From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=)
Date: Tue, 10 Nov 2009 21:49:28 +0100
Subject: [Python-Dev] Replacing IDLE
In-Reply-To: <d34314100911100820u7a9e6f7bp6dde4ec8d2ff2c4@mail.gmail.com>
References: <d34314100911100820u7a9e6f7bp6dde4ec8d2ff2c4@mail.gmail.com>
Message-ID: <4AF9D1D8.3060406@v.loewis.de>

> If we filter list of http://wiki.python.org/moin/PythonEditors by
> language/license/framework, we will be able to see if there is any
> suitable open source Python code to replace IDLE's.

This is not how it works. We cannot incorporate something into Python
without explicit consent and support from the author(s). So for any
editor to be incorporated as a replacement (along with all libraries
it depends on) we would need a commitment from the author(s) that they
support the fork that we create (and eventually abandon their
stand-alone release in favor of the fork).

> Should we place an editor first and combine the list a of unique
> features of IDLE that we (as developers) will surely miss  OR  should
> we think what GUI framework will be more beneficial for Python users
> if shipped by default (keeping in mind that we might have to add
> missing features to whatever editor we choose later)?

I think neither approach can possibly work. It's always the authors of
contribution who need to act first.

Regards,
Martin

From martin at v.loewis.de  Tue Nov 10 22:05:41 2009
From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=)
Date: Tue, 10 Nov 2009 22:05:41 +0100
Subject: [Python-Dev] 2.7/3.2 release schedule
In-Reply-To: <4AF966F8.5040008@gmail.com>
References: <1afaf6160911021200m1bcf27cck4a78db471db038a7@mail.gmail.com>	<71ac8a310911100241x2567c396pc2e5ef4653ced917@mail.gmail.com>
	<4AF966F8.5040008@gmail.com>
Message-ID: <4AF9D5A5.5000707@v.loewis.de>

>> PEP 3003 states that Python 3.2 will be released 18-24 months after
>> Python 3.1. Python 3.1 was released on June 2009-06-27 [1], so
>> theoretically Python 3.2 should be released not before 2010-12-19 [2].
> 
> The PEP 3003 text isn't allowing for the fact that 3.1 is "3.0 as it
> should have been", so the starting point for the 18-24 month rule of
> thumb is actually back when 3.0 was released in Dec 2008. This was
> discussed a fair bit back when the decision was made to do the short
> release cycle between 3.0 and 3.1 in order to address some of the more
> glaring shortcomings of the 3.0 release.

I agree with everybody else who said that

a) there was *no* consensus that the release cycle for 3.2 should be
   shortened because of 3.1 being released early. I also remember the
   opposite.

b) whatever past discussion may have been, it would be a mistake to
   release 3.2 earlier than 18 months after 3.1.

Of course, 2.7 *could* be released by the proposed schedule; it just
would have to be decoupled from 3.2 (just as 2.6 eventually got
decoupled from 3.0).

I personally think that decoupling the releases would be best, i.e.
not start thinking about 3.2 for another 6 months.

Regards,
Martin

From ncoghlan at gmail.com  Tue Nov 10 22:16:50 2009
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Wed, 11 Nov 2009 07:16:50 +1000
Subject: [Python-Dev] 2.7/3.2 release schedule
In-Reply-To: <4AF9D5A5.5000707@v.loewis.de>
References: <1afaf6160911021200m1bcf27cck4a78db471db038a7@mail.gmail.com>	<71ac8a310911100241x2567c396pc2e5ef4653ced917@mail.gmail.com>
	<4AF966F8.5040008@gmail.com> <4AF9D5A5.5000707@v.loewis.de>
Message-ID: <4AF9D842.5000808@gmail.com>

Martin v. L?wis wrote:
>>> PEP 3003 states that Python 3.2 will be released 18-24 months after
>>> Python 3.1. Python 3.1 was released on June 2009-06-27 [1], so
>>> theoretically Python 3.2 should be released not before 2010-12-19 [2].
>> The PEP 3003 text isn't allowing for the fact that 3.1 is "3.0 as it
>> should have been", so the starting point for the 18-24 month rule of
>> thumb is actually back when 3.0 was released in Dec 2008. This was
>> discussed a fair bit back when the decision was made to do the short
>> release cycle between 3.0 and 3.1 in order to address some of the more
>> glaring shortcomings of the 3.0 release.
> 
> I agree with everybody else who said that
> 
> a) there was *no* consensus that the release cycle for 3.2 should be
>    shortened because of 3.1 being released early. I also remember the
>    opposite.
> 
> b) whatever past discussion may have been, it would be a mistake to
>    release 3.2 earlier than 18 months after 3.1.

Fair enough - I didn't remember that discussion all that well, but
assumed it had reached that consensus due to the lack of comment when
Benjamin originally posted his proposed schedule. Your response and
Guido's clearly indicate otherwise :)

If that wasn't the consensus, then all the dates should slide back 6
months (i.e. no alphas until June 2010). (I actually prefer that since
it gives me a chance to find a cleaner approach to the runpy.run_path
problem, but didn't want to rehash a discussion I thought we had already
had)

> Of course, 2.7 *could* be released by the proposed schedule; it just
> would have to be decoupled from 3.2 (just as 2.6 eventually got
> decoupled from 3.0).

That leads to a 2.x version with features that aren't yet available in a
3.x version though. I thought we weren't planning on doing that anymore.

> I personally think that decoupling the releases would be best, i.e.
> not start thinking about 3.2 for another 6 months.

Or just have the timing be 18 months for 3.2 and 24 months for 2.7 (i.e.
push the first alpha of both back to June next year).

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------

From solipsis at pitrou.net  Tue Nov 10 22:26:24 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Tue, 10 Nov 2009 21:26:24 +0000 (UTC)
Subject: [Python-Dev] Reworking the GIL
References: <1256502140.5621.163.camel@localhost>
	<bbaeab100910251457t64616f62x3216ebb4863223f5@mail.gmail.com>
	<loom.20091101T123110-331@post.gmane.org>
	<d14bb4710911070251u13b858d1mdc41e46aa46885fc@mail.gmail.com>
	<loom.20091107T155305-553@post.gmane.org>
	<ca471dc20911070806m244fb9f4j55a332ef25cda551@mail.gmail.com>
	<1257613317.3437.63.camel@localhost>
	<ca471dc20911071133i56098f5cxf5639554c82e11fc@mail.gmail.com>
Message-ID: <hdclq0$tov$1@ger.gmane.org>


Hello again,

I've now removed priority requests, tried to improve the internal doc a 
bit, and merged the changes into py3k.

Afterwards, the new Windows 7 buildbot has hung in test_multiprocessing, 
but I don't know whether it's related.

Regards

Antoine.


Guido van Rossum <guido <at> python.org> writes:
> 
> 
> I would remove them -- if and when we find there's a need for
> something like them I suspect it won't look like what you currently
> have, so it's better not to complexificate your current patch with
> them. (I would remove all traces, including the conditions passed into
> the end macros.)
> 
> >> My only suggestion so far: the description could use more explicit
> >> documentation on the various variables and macros and how they
> >> combine.
> >
> > Is it before or after
> > http://mail.python.org/pipermail/python-checkins/2009-
November/087482.html ?
> 
> After. While that is already really helpful, not all the code is
> easily linked back to paragraphs in that comment block, and some
> variables are not mentioned by name in the block.
>



From solipsis at pitrou.net  Tue Nov 10 22:50:26 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Tue, 10 Nov 2009 21:50:26 +0000 (UTC)
Subject: [Python-Dev] Status of the Buildbot fleet and related bugs
References: <Pine.LNX.4.64.0911052209000.12274@kimball.webabinitio.net>
Message-ID: <hdcn72$2rl$1@ger.gmane.org>


Le Thu, 05 Nov 2009 22:53:27 -0500, R. David Murray a ?crit?:
> The buildbot waterfall is much greener now.  Thanks to all who have
> contributed to making it so (and it hasn't just been Mark and Antoine
> and I, though we've been the most directly active (and yes, Mark, you
> did contribute several fixes!)).

The buildbots still show occasional oddities. For example, right now in 
the page "http://www.python.org/dev/buildbot/3.x/", some results have 
disappeared (the columns for "AMD64 Ubuntu" builders have become empty). 

Moreover, some buildslaves have gone back in time (they are building 
r76188 after having built and tested r76195)... I swear the new GIL 
doesn't include a time machine.

Regards

Antoine.



From martin at v.loewis.de  Tue Nov 10 23:06:14 2009
From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=)
Date: Tue, 10 Nov 2009 23:06:14 +0100
Subject: [Python-Dev] Status of the Buildbot fleet and related bugs
In-Reply-To: <hdcn72$2rl$1@ger.gmane.org>
References: <Pine.LNX.4.64.0911052209000.12274@kimball.webabinitio.net>
	<hdcn72$2rl$1@ger.gmane.org>
Message-ID: <4AF9E3D6.2000807@v.loewis.de>

>> The buildbot waterfall is much greener now.  Thanks to all who have
>> contributed to making it so (and it hasn't just been Mark and Antoine
>> and I, though we've been the most directly active (and yes, Mark, you
>> did contribute several fixes!)).
> 
> The buildbots still show occasional oddities. For example, right now in 
> the page "http://www.python.org/dev/buildbot/3.x/", some results have 
> disappeared (the columns for "AMD64 Ubuntu" builders have become empty). 

Yes, I noticed it too. It will go away after some page reloads.

> Moreover, some buildslaves have gone back in time (they are building 
> r76188 after having built and tested r76195)... I swear the new GIL 
> doesn't include a time machine.

That's because I resubmitted these changes after restarting the master.

Regards,
Martin

From benjamin at python.org  Tue Nov 10 23:06:46 2009
From: benjamin at python.org (Benjamin Peterson)
Date: Tue, 10 Nov 2009 16:06:46 -0600
Subject: [Python-Dev] 2.7/3.2 release schedule
In-Reply-To: <4AF9D5A5.5000707@v.loewis.de>
References: <1afaf6160911021200m1bcf27cck4a78db471db038a7@mail.gmail.com>
	<71ac8a310911100241x2567c396pc2e5ef4653ced917@mail.gmail.com>
	<4AF966F8.5040008@gmail.com> <4AF9D5A5.5000707@v.loewis.de>
Message-ID: <1afaf6160911101406t6da78accq8918eb4ceb1adca5@mail.gmail.com>

2009/11/10 "Martin v. L?wis" <martin at v.loewis.de>:
> I personally think that decoupling the releases would be best, i.e.
> not start thinking about 3.2 for another 6 months.

The problem with that is that there is a period of time where 2.x has
features which 3.x doesn't. My preference is to move back the whole
schedule 6 months.



-- 
Regards,
Benjamin

From martin at v.loewis.de  Tue Nov 10 23:09:42 2009
From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=)
Date: Tue, 10 Nov 2009 23:09:42 +0100
Subject: [Python-Dev] 2.7/3.2 release schedule
In-Reply-To: <1afaf6160911101406t6da78accq8918eb4ceb1adca5@mail.gmail.com>
References: <1afaf6160911021200m1bcf27cck4a78db471db038a7@mail.gmail.com>	
	<71ac8a310911100241x2567c396pc2e5ef4653ced917@mail.gmail.com>	
	<4AF966F8.5040008@gmail.com> <4AF9D5A5.5000707@v.loewis.de>
	<1afaf6160911101406t6da78accq8918eb4ceb1adca5@mail.gmail.com>
Message-ID: <4AF9E4A6.8070304@v.loewis.de>

Benjamin Peterson wrote:
> 2009/11/10 "Martin v. L?wis" <martin at v.loewis.de>:
>> I personally think that decoupling the releases would be best, i.e.
>> not start thinking about 3.2 for another 6 months.
> 
> The problem with that is that there is a period of time where 2.x has
> features which 3.x doesn't. My preference is to move back the whole
> schedule 6 months.

Ok, fine with me as well. FWIW, I don't see that (2.7 released before
3.2) as a problem (but I understand that others might, for what I would
consider formal reasons).

Regards,
Martin

From g.brandl at gmx.net  Tue Nov 10 23:27:09 2009
From: g.brandl at gmx.net (Georg Brandl)
Date: Tue, 10 Nov 2009 22:27:09 +0000
Subject: [Python-Dev] IDLE as default Python editor
In-Reply-To: <4AF944A9.2030108@gmail.com>
References: <d34314100911082245w79c9233dwad2a0aa867e06355@mail.gmail.com>	<87iqdk2n7b.fsf@benfinney.id.au>	<4AF80E73.5090104@gmail.com>
	<4AF89370.2040206@v.loewis.de> <4AF944A9.2030108@gmail.com>
Message-ID: <hdcp8i$9q4$1@ger.gmane.org>

Nick Coghlan schrieb:
> Martin v. L?wis wrote:
>> Nick Coghlan wrote:
>>> Ben Finney wrote:
>>>> anatoly techtonik <techtonik at gmail.com> writes:
>>>>
>>>>> Quite an interesting question recently popped up in pygame community
>>>>> that I'd like to ask to Python developers.
>>>> This forum is specifically about development *of* Python.
>>> Anatoly's question is actually a fair one for python-dev - we're the
>>> ones that *ship* Idle, so it is legitimate to ask our reasons for
>>> continuing to do so.
>> 
>> But he didn't - instead, he asked how many of us use it. Usage by
>> committers is not (or shouldn't be) a primary criterion for including
>> or not including something. Instead, usage in the community should be,
>> and python-dev is indeed the wrong place to estimate that.
> 
> I took it as being a somewhat relevant leading question. If the response
> had been "No, none of us use it" then the obvious follow-up question
> would be "Why do you ship something that you don't consider worth using?".

Quite simple: because we can't possibly ship Emacs.

cheers,
Georg


From wa2n39 at gmail.com  Wed Nov 11 14:36:29 2009
From: wa2n39 at gmail.com (wa2n39 at gmail.com)
Date: Wed, 11 Nov 2009 13:36:29 +0000
Subject: [Python-Dev] Python-Dev Digest, Vol 76, Issue 87
In-Reply-To: <mailman.753.1257890985.2872.python-dev@python.org>
References: <mailman.753.1257890985.2872.python-dev@python.org>
Message-ID: <873739028-1257892142-cardhu_decombobulator_blackberry.rim.net-1348400255-@bda056.bisx.prodap.on.blackberry>

Stop email
Sent from my BlackBerry?
powered by Sinyal Kuat INDOSAT

-----Original Message-----
From: python-dev-request at python.org
Date: Tue, 10 Nov 2009 23:09:45 
To: <python-dev at python.org>
Subject: Python-Dev Digest, Vol 76, Issue 87

Send Python-Dev mailing list submissions to
	python-dev at python.org

To subscribe or unsubscribe via the World Wide Web, visit
	http://mail.python.org/mailman/listinfo/python-dev
or, via email, send a message with subject or body 'help' to
	python-dev-request at python.org

You can reach the person managing the list at
	python-dev-owner at python.org

When replying, please edit your Subject line so it is more specific
than "Re: Contents of Python-Dev digest..."


Today's Topics:

   1. Re: 2.7/3.2 release schedule (Martin v. L?wis)
   2. Re: 2.7/3.2 release schedule (Nick Coghlan)
   3. Re: Reworking the GIL (Antoine Pitrou)
   4. Re: Status of the Buildbot fleet and related bugs (Antoine Pitrou)
   5. Re: Status of the Buildbot fleet and related bugs
      (Martin v. L?wis)
   6. Re: 2.7/3.2 release schedule (Benjamin Peterson)
   7. Re: 2.7/3.2 release schedule (Martin v. L?wis)


----------------------------------------------------------------------

Message: 1
Date: Tue, 10 Nov 2009 22:05:41 +0100
From: "Martin v. L?wis" <martin at v.loewis.de>
To: Nick Coghlan <ncoghlan at gmail.com>
Cc: Python Development <python-dev at python.org>
Subject: Re: [Python-Dev] 2.7/3.2 release schedule
Message-ID: <4AF9D5A5.5000707 at v.loewis.de>
Content-Type: text/plain; charset=UTF-8

>> PEP 3003 states that Python 3.2 will be released 18-24 months after
>> Python 3.1. Python 3.1 was released on June 2009-06-27 [1], so
>> theoretically Python 3.2 should be released not before 2010-12-19 [2].
> 
> The PEP 3003 text isn't allowing for the fact that 3.1 is "3.0 as it
> should have been", so the starting point for the 18-24 month rule of
> thumb is actually back when 3.0 was released in Dec 2008. This was
> discussed a fair bit back when the decision was made to do the short
> release cycle between 3.0 and 3.1 in order to address some of the more
> glaring shortcomings of the 3.0 release.

I agree with everybody else who said that

a) there was *no* consensus that the release cycle for 3.2 should be
   shortened because of 3.1 being released early. I also remember the
   opposite.

b) whatever past discussion may have been, it would be a mistake to
   release 3.2 earlier than 18 months after 3.1.

Of course, 2.7 *could* be released by the proposed schedule; it just
would have to be decoupled from 3.2 (just as 2.6 eventually got
decoupled from 3.0).

I personally think that decoupling the releases would be best, i.e.
not start thinking about 3.2 for another 6 months.

Regards,
Martin


------------------------------

Message: 2
Date: Wed, 11 Nov 2009 07:16:50 +1000
From: Nick Coghlan <ncoghlan at gmail.com>
To: "Martin v. L?wis" <martin at v.loewis.de>
Cc: Python Development <python-dev at python.org>
Subject: Re: [Python-Dev] 2.7/3.2 release schedule
Message-ID: <4AF9D842.5000808 at gmail.com>
Content-Type: text/plain; charset=UTF-8

Martin v. L?wis wrote:
>>> PEP 3003 states that Python 3.2 will be released 18-24 months after
>>> Python 3.1. Python 3.1 was released on June 2009-06-27 [1], so
>>> theoretically Python 3.2 should be released not before 2010-12-19 [2].
>> The PEP 3003 text isn't allowing for the fact that 3.1 is "3.0 as it
>> should have been", so the starting point for the 18-24 month rule of
>> thumb is actually back when 3.0 was released in Dec 2008. This was
>> discussed a fair bit back when the decision was made to do the short
>> release cycle between 3.0 and 3.1 in order to address some of the more
>> glaring shortcomings of the 3.0 release.
> 
> I agree with everybody else who said that
> 
> a) there was *no* consensus that the release cycle for 3.2 should be
>    shortened because of 3.1 being released early. I also remember the
>    opposite.
> 
> b) whatever past discussion may have been, it would be a mistake to
>    release 3.2 earlier than 18 months after 3.1.

Fair enough - I didn't remember that discussion all that well, but
assumed it had reached that consensus due to the lack of comment when
Benjamin originally posted his proposed schedule. Your response and
Guido's clearly indicate otherwise :)

If that wasn't the consensus, then all the dates should slide back 6
months (i.e. no alphas until June 2010). (I actually prefer that since
it gives me a chance to find a cleaner approach to the runpy.run_path
problem, but didn't want to rehash a discussion I thought we had already
had)

> Of course, 2.7 *could* be released by the proposed schedule; it just
> would have to be decoupled from 3.2 (just as 2.6 eventually got
> decoupled from 3.0).

That leads to a 2.x version with features that aren't yet available in a
3.x version though. I thought we weren't planning on doing that anymore.

> I personally think that decoupling the releases would be best, i.e.
> not start thinking about 3.2 for another 6 months.

Or just have the timing be 18 months for 3.2 and 24 months for 2.7 (i.e.
push the first alpha of both back to June next year).

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------


------------------------------

Message: 3
Date: Tue, 10 Nov 2009 21:26:24 +0000 (UTC)
From: Antoine Pitrou <solipsis at pitrou.net>
To: python-dev at python.org
Subject: Re: [Python-Dev] Reworking the GIL
Message-ID: <hdclq0$tov$1 at ger.gmane.org>
Content-Type: text/plain; charset=UTF-8


Hello again,

I've now removed priority requests, tried to improve the internal doc a 
bit, and merged the changes into py3k.

Afterwards, the new Windows 7 buildbot has hung in test_multiprocessing, 
but I don't know whether it's related.

Regards

Antoine.


Guido van Rossum <guido <at> python.org> writes:
> 
> 
> I would remove them -- if and when we find there's a need for
> something like them I suspect it won't look like what you currently
> have, so it's better not to complexificate your current patch with
> them. (I would remove all traces, including the conditions passed into
> the end macros.)
> 
> >> My only suggestion so far: the description could use more explicit
> >> documentation on the various variables and macros and how they
> >> combine.
> >
> > Is it before or after
> > http://mail.python.org/pipermail/python-checkins/2009-
November/087482.html ?
> 
> After. While that is already really helpful, not all the code is
> easily linked back to paragraphs in that comment block, and some
> variables are not mentioned by name in the block.
>




------------------------------

Message: 4
Date: Tue, 10 Nov 2009 21:50:26 +0000 (UTC)
From: Antoine Pitrou <solipsis at pitrou.net>
To: python-dev at python.org
Subject: Re: [Python-Dev] Status of the Buildbot fleet and related
	bugs
Message-ID: <hdcn72$2rl$1 at ger.gmane.org>
Content-Type: text/plain; charset=UTF-8


Le Thu, 05 Nov 2009 22:53:27 -0500, R. David Murray a ?crit?:
> The buildbot waterfall is much greener now.  Thanks to all who have
> contributed to making it so (and it hasn't just been Mark and Antoine
> and I, though we've been the most directly active (and yes, Mark, you
> did contribute several fixes!)).

The buildbots still show occasional oddities. For example, right now in 
the page "http://www.python.org/dev/buildbot/3.x/", some results have 
disappeared (the columns for "AMD64 Ubuntu" builders have become empty). 

Moreover, some buildslaves have gone back in time (they are building 
r76188 after having built and tested r76195)... I swear the new GIL 
doesn't include a time machine.

Regards

Antoine.




------------------------------

Message: 5
Date: Tue, 10 Nov 2009 23:06:14 +0100
From: "Martin v. L?wis" <martin at v.loewis.de>
To: Antoine Pitrou <solipsis at pitrou.net>
Cc: python-dev at python.org
Subject: Re: [Python-Dev] Status of the Buildbot fleet and related
	bugs
Message-ID: <4AF9E3D6.2000807 at v.loewis.de>
Content-Type: text/plain; charset=UTF-8

>> The buildbot waterfall is much greener now.  Thanks to all who have
>> contributed to making it so (and it hasn't just been Mark and Antoine
>> and I, though we've been the most directly active (and yes, Mark, you
>> did contribute several fixes!)).
> 
> The buildbots still show occasional oddities. For example, right now in 
> the page "http://www.python.org/dev/buildbot/3.x/", some results have 
> disappeared (the columns for "AMD64 Ubuntu" builders have become empty). 

Yes, I noticed it too. It will go away after some page reloads.

> Moreover, some buildslaves have gone back in time (they are building 
> r76188 after having built and tested r76195)... I swear the new GIL 
> doesn't include a time machine.

That's because I resubmitted these changes after restarting the master.

Regards,
Martin


------------------------------

Message: 6
Date: Tue, 10 Nov 2009 16:06:46 -0600
From: Benjamin Peterson <benjamin at python.org>
To: Martin v. L?wis <martin at v.loewis.de>
Cc: Nick Coghlan <ncoghlan at gmail.com>,	Python Development
	<python-dev at python.org>
Subject: Re: [Python-Dev] 2.7/3.2 release schedule
Message-ID:
	<1afaf6160911101406t6da78accq8918eb4ceb1adca5 at mail.gmail.com>
Content-Type: text/plain; charset=UTF-8

2009/11/10 "Martin v. L?wis" <martin at v.loewis.de>:
> I personally think that decoupling the releases would be best, i.e.
> not start thinking about 3.2 for another 6 months.

The problem with that is that there is a period of time where 2.x has
features which 3.x doesn't. My preference is to move back the whole
schedule 6 months.



-- 
Regards,
Benjamin


------------------------------

Message: 7
Date: Tue, 10 Nov 2009 23:09:42 +0100
From: "Martin v. L?wis" <martin at v.loewis.de>
To: Benjamin Peterson <benjamin at python.org>
Cc: Nick Coghlan <ncoghlan at gmail.com>,	Python Development
	<python-dev at python.org>
Subject: Re: [Python-Dev] 2.7/3.2 release schedule
Message-ID: <4AF9E4A6.8070304 at v.loewis.de>
Content-Type: text/plain; charset=UTF-8

Benjamin Peterson wrote:
> 2009/11/10 "Martin v. L?wis" <martin at v.loewis.de>:
>> I personally think that decoupling the releases would be best, i.e.
>> not start thinking about 3.2 for another 6 months.
> 
> The problem with that is that there is a period of time where 2.x has
> features which 3.x doesn't. My preference is to move back the whole
> schedule 6 months.

Ok, fine with me as well. FWIW, I don't see that (2.7 released before
3.2) as a problem (but I understand that others might, for what I would
consider formal reasons).

Regards,
Martin


------------------------------

_______________________________________________
Python-Dev mailing list
Python-Dev at python.org
http://mail.python.org/mailman/listinfo/python-dev


End of Python-Dev Digest, Vol 76, Issue 87
******************************************

From python at rcn.com  Tue Nov 10 23:30:22 2009
From: python at rcn.com (Raymond Hettinger)
Date: Tue, 10 Nov 2009 14:30:22 -0800
Subject: [Python-Dev] 2.7/3.2 release schedule
References: <1afaf6160911021200m1bcf27cck4a78db471db038a7@mail.gmail.com><71ac8a310911100241x2567c396pc2e5ef4653ced917@mail.gmail.com><4AF966F8.5040008@gmail.com>
	<4AF9D5A5.5000707@v.loewis.de>
	<1afaf6160911101406t6da78accq8918eb4ceb1adca5@mail.gmail.com>
Message-ID: <DBD8F06E720A4BD18EA555B4710088CB@RaymondLaptop1>


[MvL]
>> I personally think that decoupling the releases would be best, i.e.
>> not start thinking about 3.2 for another 6 months.

[Benjamin] 
> The problem with that is that there is a period of time where 2.x has
> features which 3.x doesn't. My preference is to move back the whole
> schedule 6 months.

My preference is to decouple and let 2.7 go out 18 months after 2.6.
In general, 2.x users should not have to pay a price for whatever
we do with 3.x.


Raymond

From techtonik at gmail.com  Wed Nov 11 02:48:19 2009
From: techtonik at gmail.com (anatoly techtonik)
Date: Wed, 11 Nov 2009 03:48:19 +0200
Subject: [Python-Dev] Replacing IDLE
In-Reply-To: <59691AA2-995E-452B-B21D-0C7F8861C7A6@mac.com>
References: <d34314100911100820u7a9e6f7bp6dde4ec8d2ff2c4@mail.gmail.com> 
	<59691AA2-995E-452B-B21D-0C7F8861C7A6@mac.com>
Message-ID: <d34314100911101748y202335a4td05f38880c0d694d@mail.gmail.com>

On Tue, Nov 10, 2009 at 6:44 PM, Ronald Oussoren <ronaldoussoren at mac.com> wrote:
>
> What's so bad about IDLE that you'd like to replace it?

That was exactly my previous question. You don't use IDLE either, so
why not to replace it with something that you can actually use, with
something that is at least extensible? So people will be interested to
learn and contribute.

> The obvious alternative to replacing IDLE is to fix whatever is wrong with it.

It is also obvious that developer would better download a decent
editor than bog their mind with once perspective, but now dead
technology IDLE was build upon.

-- 
anatoly t.

From techtonik at gmail.com  Wed Nov 11 03:33:20 2009
From: techtonik at gmail.com (anatoly techtonik)
Date: Wed, 11 Nov 2009 04:33:20 +0200
Subject: [Python-Dev] Replacing IDLE
In-Reply-To: <4AF9BE77.5050100@voidspace.org.uk>
References: <d34314100911100820u7a9e6f7bp6dde4ec8d2ff2c4@mail.gmail.com> 
	<4AF9BE77.5050100@voidspace.org.uk>
Message-ID: <d34314100911101833l4e6c3ba9qd988700dd57fcb3d@mail.gmail.com>

On Tue, Nov 10, 2009 at 9:26 PM, Michael Foord
<fuzzyman at voidspace.org.uk> wrote:
>
>> Then there will be another issue - all editors are based upon some
>> frameworks - I didn't see any popular cross-platform GUI toolkits in
>> Python, so we will inevitably face the need to replace Tkinter with
>> other default GUI toolkit.
>
> This just *won't* happen. (At least not in the forseeable future.) Better
> filter your list to only those IDEs that use Tkinter or have a GUI not
> dependent on packages that aren't already in the standard library.

That is a real stumbling block. There aren't any GUI toolkits in pure
Python to be included alongside the editor, and those Tkinter bindings
that compiled for every platform are just useless. If there is a
reason that we just *won't* let any GUI framework (except Tkinter)
slip into Python distribution then of course, any talk about editor is
just a waste of time. But I'd like to see is that reason is real to
know when to expect the changes.

I've done a small research about Tkinter editors to investigate the
solution you proposed. http://wiki.python.org/moin/PythonEditors shows
that these were made with Tk:

Pye - not true, either switched or wrong project, but PyQT4 -
http://code.google.com/p/pye-editor/
xRope - GPL and probably worse than IDLE -
http://sourceforge.net/projects/xrope/
IDEStudio - enhanced IDLE, dead - http://wiki.python.org/moin/IDEStudio
SourceNavigator - GPL, C and not really an editor -
http://sourcenav.sourceforge.net/

There is only one perspective project called Leo (with MIT license)
that was historically written in Tkinter with the goal (I believe) to
construct pure Python editor. It is editor with outlines unique in the
way that it is able to rewrite itself literally at runtime. I believe
only using such advanced tool one can cope with Tkinter
inconsistencies and limitations. Since the beginning Leo has also
developed PyQT4 interface.

-- 
anatoly t.

From guido at python.org  Wed Nov 11 03:46:50 2009
From: guido at python.org (Guido van Rossum)
Date: Tue, 10 Nov 2009 18:46:50 -0800
Subject: [Python-Dev] 2.7/3.2 release schedule
In-Reply-To: <5FFB0E2375804AD28576A8C3EE953A9B@RaymondLaptop1>
References: <1afaf6160911021200m1bcf27cck4a78db471db038a7@mail.gmail.com> 
	<71ac8a310911100241x2567c396pc2e5ef4653ced917@mail.gmail.com> 
	<4AF966F8.5040008@gmail.com>
	<ca471dc20911100755m6c4bb37ei6375228e4fa3e962@mail.gmail.com> 
	<loom.20091110T170843-458@post.gmane.org>
	<ca471dc20911100926wb5106c4j1b81290df2e60da9@mail.gmail.com> 
	<5FFB0E2375804AD28576A8C3EE953A9B@RaymondLaptop1>
Message-ID: <ca471dc20911101846n5258e99aw318f2d910b45ba12@mail.gmail.com>

On Tue, Nov 10, 2009 at 10:26 AM, Raymond Hettinger <python at rcn.com> wrote:
>
> [Guido van Rossum]
>>
>> . We used to have releases once a year and
>> we got really big serious feedback from our biggest users that the
>> release cycle was going too fast. We discussed it amply and agreed on
>> a minimum time of 18 months between releases.
>
> If the language moratorium goes into effect,

"If"? It's already Approved. :-)

> would shorter release cycles
> still have a negative impact? ?Do people possibly want slower changes
> to the language and faster updates to the library?
>
> I don't know the answer. ?Just asking how this matches up with the
> feedback you have gotten previously.

I expect it wouldn't make a difference since backwards compatibility
testing involves a lot more than language syntax.

-- 
--Guido van Rossum (python.org/~guido)

From techtonik at gmail.com  Wed Nov 11 03:50:03 2009
From: techtonik at gmail.com (anatoly techtonik)
Date: Wed, 11 Nov 2009 04:50:03 +0200
Subject: [Python-Dev] Replacing IDLE
In-Reply-To: <4AF9D1D8.3060406@v.loewis.de>
References: <d34314100911100820u7a9e6f7bp6dde4ec8d2ff2c4@mail.gmail.com> 
	<4AF9D1D8.3060406@v.loewis.de>
Message-ID: <d34314100911101850weee529aw442cfefdb161d3c5@mail.gmail.com>

On Tue, Nov 10, 2009 at 10:49 PM, "Martin v. L?wis" <martin at v.loewis.de> wrote:
>
> This is not how it works. We cannot incorporate something into Python
> without explicit consent and support from the author(s). So for any
> editor to be incorporated as a replacement (along with all libraries
> it depends on) we would need a commitment from the author(s) that they
> support the fork that we create (and eventually abandon their
> stand-alone release in favor of the fork).
>

Does that mean even if authors of some imaginary editor agree to
incorporate their code into Python, the framework that it is built
upon will have to be incorporated into Python also (and eventually
abandoned at original location)?

In this situation I see no way we can provide any decent tools for GUI
like for default editor.

Thanks for support.
-- 
anatoly t.

From stephen at xemacs.org  Wed Nov 11 05:06:36 2009
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Wed, 11 Nov 2009 13:06:36 +0900
Subject: [Python-Dev] Replacing IDLE
In-Reply-To: <d34314100911101850weee529aw442cfefdb161d3c5@mail.gmail.com>
References: <d34314100911100820u7a9e6f7bp6dde4ec8d2ff2c4@mail.gmail.com>
	<4AF9D1D8.3060406@v.loewis.de>
	<d34314100911101850weee529aw442cfefdb161d3c5@mail.gmail.com>
Message-ID: <87639h4sdv.fsf@uwakimon.sk.tsukuba.ac.jp>

anatoly techtonik writes:

 > Does that mean even if authors of some imaginary editor agree to
 > incorporate their code into Python, the framework that it is built
 > upon will have to be incorporated into Python also (and eventually
 > abandoned at original location)?

I would assume so.  How else can Python guarantee its maintenance?

 > In this situation I see no way we can provide any decent tools for GUI
 > like for default editor.

IDLE *is* a "decent tool".  You don't have to like it, but it's good
enough to get started.  By the time you realize its limitations, you
should have a handle on easy_install.

The promise is "batteries included."  Nobody promised you a nickel
metal hydride battery that you can use as a replacement in your Prius.

From greg.ewing at canterbury.ac.nz  Wed Nov 11 07:07:56 2009
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Wed, 11 Nov 2009 19:07:56 +1300
Subject: [Python-Dev] Replacing IDLE
In-Reply-To: <d34314100911101748y202335a4td05f38880c0d694d@mail.gmail.com>
References: <d34314100911100820u7a9e6f7bp6dde4ec8d2ff2c4@mail.gmail.com>
	<59691AA2-995E-452B-B21D-0C7F8861C7A6@mac.com>
	<d34314100911101748y202335a4td05f38880c0d694d@mail.gmail.com>
Message-ID: <4AFA54BC.90401@canterbury.ac.nz>

anatoly techtonik wrote:

> why not to replace it with something that you can actually use, with
> something that is at least extensible? So people will be interested to
> learn and contribute.

IDLE is written in Python, so it's about as extensible
as you can get.

Seems to me the only kind of IDE that it makes sense to
ship with Python is one that is written in Python and
maintained by the core developers. Anything else is best
left as a third party package for download by those
who want to use it.

Tkinter is arguably not a very good basis for it from a
technical point of view, but so far all the alternatives
are too bloated and/or nonportable to consider bundling
with the standard distribution.

So the only realistic alternatives at the moment seem
to be either IDLE+Tkinter or nothing.

Quite likely most experienced developers use something
else, and wouldn't care if IDLE were removed and not
replaced with anything. But there are benefits to having
a standard, lightweight IDE present in most Python
installations. Books and tutorials can be designed around
it, for example, with fairly high assurance that it will
be available to beginners who aren't comfortable finding
their way around text editors and command shells.

So, I'd say that, like democracy, it's not very good, but
it's better than any of the alternatives. :-)

-- 
Greg

From martin at v.loewis.de  Wed Nov 11 07:17:20 2009
From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=)
Date: Wed, 11 Nov 2009 07:17:20 +0100
Subject: [Python-Dev] Replacing IDLE
In-Reply-To: <d34314100911101850weee529aw442cfefdb161d3c5@mail.gmail.com>
References: <d34314100911100820u7a9e6f7bp6dde4ec8d2ff2c4@mail.gmail.com>
	<4AF9D1D8.3060406@v.loewis.de>
	<d34314100911101850weee529aw442cfefdb161d3c5@mail.gmail.com>
Message-ID: <4AFA56F0.1090704@v.loewis.de>

> Does that mean even if authors of some imaginary editor agree to
> incorporate their code into Python, the framework that it is built
> upon will have to be incorporated into Python also (and eventually
> abandoned at original location)?

It depends. It should work the same way as IDLE: it's ok that Tcl/Tk
itself is not included in Python, however, Tkinter is.

If that imaginary editor has lots of build dependencies that make
it really difficult to use it, I would be opposed to including it.
If it requires one library that is typically already available on
a Linux system, it would be fine with me.

Regards,
Martin


From greg.ewing at canterbury.ac.nz  Wed Nov 11 08:01:38 2009
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Wed, 11 Nov 2009 20:01:38 +1300
Subject: [Python-Dev] Replacing IDLE
In-Reply-To: <4AFA56F0.1090704@v.loewis.de>
References: <d34314100911100820u7a9e6f7bp6dde4ec8d2ff2c4@mail.gmail.com>
	<4AF9D1D8.3060406@v.loewis.de>
	<d34314100911101850weee529aw442cfefdb161d3c5@mail.gmail.com>
	<4AFA56F0.1090704@v.loewis.de>
Message-ID: <4AFA6152.50900@canterbury.ac.nz>

Martin v. L?wis wrote:

> If that imaginary editor has lots of build dependencies that make
> it really difficult to use it, I would be opposed to including it.
> If it requires one library that is typically already available on
> a Linux system, it would be fine with me.

If I manage to get PyGUI into a practically usable
state and it ever gets included in the core distribution,
then it might be a viable basis for a modernised version
of IDLE. That's probably a fairly long way off, though.

-- 
Greg

From vmanis at telus.net  Wed Nov 11 07:30:06 2009
From: vmanis at telus.net (Vincent Manis)
Date: Tue, 10 Nov 2009 22:30:06 -0800
Subject: [Python-Dev] Replacing IDLE
In-Reply-To: <4AFA54BC.90401@canterbury.ac.nz>
References: <d34314100911100820u7a9e6f7bp6dde4ec8d2ff2c4@mail.gmail.com>
	<59691AA2-995E-452B-B21D-0C7F8861C7A6@mac.com>
	<d34314100911101748y202335a4td05f38880c0d694d@mail.gmail.com>
	<4AFA54BC.90401@canterbury.ac.nz>
Message-ID: <BDA278C5-924D-411A-8E56-8EEDE772C8F3@telus.net>

On 2009-11-10, at 22:07, Greg Ewing wrote:
> So, I'd say that, like democracy, [IDLE is] not very good, but
> it's better than any of the alternatives. :-)


Speaking purely as a Python user, I am very happy that IDLE is part of the Python distribution. Personally, I use and like emacs too much, and have no use for IDLE for my own programming at all. But I have taught a lot of Python classes, and being able to tell the IT staff `please install Python x.y on the machines in the lab', and then knowing that students have everything they need to do their lab work, is a major plus. (I found that many of the students stayed with IDLE even after their course was over, too.)

As a side observation, I think the main reasons why Tk is so resistant to removal from Python (and several other dynamic languages) is that its developers took very seriously the idea of being lightweight and portable, while many other toolkits grew more and more complex. Not all GUI toolkits are like that; I have nice things to say about FLTK, for example. Perhaps one day somebody will do a revision of Tk (sans Tcl) for the 21st century, but until then, Greg's Winston Churchill quote applies not just to IDLE but also to Tk. 

[`This horse is dead, please stop flogging it.' `No it's not, it's just observing the moratorium']

-- v

From vmanis at telus.net  Wed Nov 11 07:32:25 2009
From: vmanis at telus.net (Vincent Manis)
Date: Tue, 10 Nov 2009 22:32:25 -0800
Subject: [Python-Dev] Replacing IDLE
In-Reply-To: <4AFA56F0.1090704@v.loewis.de>
References: <d34314100911100820u7a9e6f7bp6dde4ec8d2ff2c4@mail.gmail.com>
	<4AF9D1D8.3060406@v.loewis.de>
	<d34314100911101850weee529aw442cfefdb161d3c5@mail.gmail.com>
	<4AFA56F0.1090704@v.loewis.de>
Message-ID: <C1B87069-9BA4-4100-97ED-D6F01DA11FFF@telus.net>

On 2009-11-10, at 22:17, Martin v. L?wis wrote:
> If that imaginary editor has lots of build dependencies that make
> it really difficult to use it, I would be opposed to including it.
> If it requires one library that is typically already available on
> a Linux system, it would be fine with me.
s/Linux/Linux, Mac (Aqua, not X11), and Windows/

-- v

From ronaldoussoren at mac.com  Wed Nov 11 08:27:06 2009
From: ronaldoussoren at mac.com (Ronald Oussoren)
Date: Wed, 11 Nov 2009 08:27:06 +0100
Subject: [Python-Dev] Replacing IDLE
In-Reply-To: <d34314100911101748y202335a4td05f38880c0d694d@mail.gmail.com>
References: <d34314100911100820u7a9e6f7bp6dde4ec8d2ff2c4@mail.gmail.com>
	<59691AA2-995E-452B-B21D-0C7F8861C7A6@mac.com>
	<d34314100911101748y202335a4td05f38880c0d694d@mail.gmail.com>
Message-ID: <E8CACA81-795D-41E7-8E0A-B2212E581415@mac.com>


On 11 Nov, 2009, at 2:48, anatoly techtonik wrote:

> On Tue, Nov 10, 2009 at 6:44 PM, Ronald Oussoren <ronaldoussoren at mac.com> wrote:
>> 
>> What's so bad about IDLE that you'd like to replace it?
> 
> That was exactly my previous question. You don't use IDLE either, so
> why not to replace it with something that you can actually use, with
> something that is at least extensible? So people will be interested to
> learn and contribute.

I don't use IDLE because I'm too used to my current editor, which btw. is the cmd-line version of vim.  That offers everything I need from a Python editor.  That said, I have used IDLE occasionaly and it works fine for a basic Python environment.

IDLE is extensible, I've seen a number of IDLE extensions on the IDLE-dev mailinglist (<http://mail.python.org/mailman/listinfo/idle-dev>). That list would be a better location to discus this.

BTW. You still haven't mentioned why you think IDLE should be replaced.
> 
>> The obvious alternative to replacing IDLE is to fix whatever is wrong with it.
> 
> It is also obvious that developer would better download a decent
> editor than bog their mind with once perspective, but now dead
> technology IDLE was build upon.

Tk is actively developed, it is not dead technology.  Replacing IDLE by an editor which uses another toolkit would either mean that the default editor uses libraries not in the stdlib, or the inclusion of that toolkit in the stdlib. 

Ronald

-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 3567 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/python-dev/attachments/20091111/def86a9f/attachment.bin>

From stefan-usenet at bytereef.org  Wed Nov 11 12:36:13 2009
From: stefan-usenet at bytereef.org (Stefan Krah)
Date: Wed, 11 Nov 2009 12:36:13 +0100
Subject: [Python-Dev] [format] Restrict fill characters for numerical values
Message-ID: <20091111113613.GA4802@mail.bytereef.org>

Hi,

I think http://www.python.org/dev/peps/pep-3101/ is too liberal with the
choice of fill characters for numerical values. As far as I can see, this
is quite legal:


Python 2.7a0 (trunk:76132M, Nov  6 2009, 15:20:35) 
[GCC 4.1.3 20080623 (prerelease) (Ubuntu 4.1.2-23ubuntu3)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> from decimal import *
>>> format(Decimal(4), "9<10")
'4999999999'
>>> format(Decimal(4), "9>10")
'9999999994'
>>> format(Decimal(4), "->2")
'-4'
>>> 


I propose to disallow digits and '+-' fill characters for numerical values,
except for the combination '0, left-padding'. Actually, I'd prefer to allow
only whitespace and '0, left-padding'.


See also:

http://bugs.python.org/issue6871
http://bugs.python.org/issue6902



Stefan Krah



From eric at trueblade.com  Wed Nov 11 13:39:26 2009
From: eric at trueblade.com (Eric Smith)
Date: Wed, 11 Nov 2009 07:39:26 -0500
Subject: [Python-Dev] [format] Restrict fill characters for numerical
 values
In-Reply-To: <20091111113613.GA4802@mail.bytereef.org>
References: <20091111113613.GA4802@mail.bytereef.org>
Message-ID: <4AFAB07E.2030707@trueblade.com>

Stefan Krah wrote:
> Hi,
> 
> I think http://www.python.org/dev/peps/pep-3101/ is too liberal with the
> choice of fill characters for numerical values. As far as I can see, this
> is quite legal:
> 
> 
> Python 2.7a0 (trunk:76132M, Nov  6 2009, 15:20:35) 
> [GCC 4.1.3 20080623 (prerelease) (Ubuntu 4.1.2-23ubuntu3)] on linux2
> Type "help", "copyright", "credits" or "license" for more information.
>>>> from decimal import *
>>>> format(Decimal(4), "9<10")
> '4999999999'
>>>> format(Decimal(4), "9>10")
> '9999999994'
>>>> format(Decimal(4), "->2")
> '-4'
> 
> 
> I propose to disallow digits and '+-' fill characters for numerical values,
> except for the combination '0, left-padding'. Actually, I'd prefer to allow
> only whitespace and '0, left-padding'.

Why?

What problem are you trying to prevent that would justify the extra code 
to implement this and the extra documentation to explain it?

Eric.

From stefan-usenet at bytereef.org  Wed Nov 11 14:06:34 2009
From: stefan-usenet at bytereef.org (Stefan Krah)
Date: Wed, 11 Nov 2009 14:06:34 +0100
Subject: [Python-Dev] [format] Restrict fill characters for numerical
	values
In-Reply-To: <4AFAB07E.2030707@trueblade.com>
References: <20091111113613.GA4802@mail.bytereef.org>
	<4AFAB07E.2030707@trueblade.com>
Message-ID: <20091111130634.GA618@yoda.bytereef.org>

Eric Smith <eric at trueblade.com> wrote:
> Stefan Krah wrote:
> >Hi,
> >
> >I think http://www.python.org/dev/peps/pep-3101/ is too liberal with the
> >choice of fill characters for numerical values. As far as I can see, this
> >is quite legal:
> >
> >
> >Python 2.7a0 (trunk:76132M, Nov  6 2009, 15:20:35) 
> >[GCC 4.1.3 20080623 (prerelease) (Ubuntu 4.1.2-23ubuntu3)] on linux2
> >Type "help", "copyright", "credits" or "license" for more information.
> >>>>from decimal import *
> >>>>format(Decimal(4), "9<10")
> >'4999999999'
> >>>>format(Decimal(4), "9>10")
> >'9999999994'
> >>>>format(Decimal(4), "->2")
> >'-4'
> >
> >
> >I propose to disallow digits and '+-' fill characters for numerical values,
> >except for the combination '0, left-padding'. Actually, I'd prefer to allow
> >only whitespace and '0, left-padding'.
> 
> Why?
> 
> What problem are you trying to prevent that would justify the extra code
> to implement this and the extra documentation to explain it?


I simply think that apart from rounding, the output of format should not
change the numerical value of its argument. The format functions in C do
not allow this to happen. Are there other languages where this is possible?


Stefan Krah



From ziade.tarek at gmail.com  Wed Nov 11 14:36:06 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Wed, 11 Nov 2009 14:36:06 +0100
Subject: [Python-Dev] PEP 382 status
Message-ID: <94bdd2610911110536n60a882acj8b2cd63ced3af25b@mail.gmail.com>

Hi,

I was wondering what's the status of PEP 382. Is anyone (MvL?) is
going to start to work on its implementation for Python 2.7/3.2
inclusion ?

Regards
Tarek

From stefan-usenet at bytereef.org  Wed Nov 11 14:34:33 2009
From: stefan-usenet at bytereef.org (Stefan Krah)
Date: Wed, 11 Nov 2009 14:34:33 +0100
Subject: [Python-Dev] [format] Restrict fill characters for numerical
	values
In-Reply-To: <20091111130634.GA618@yoda.bytereef.org>
References: <20091111113613.GA4802@mail.bytereef.org>
	<4AFAB07E.2030707@trueblade.com>
	<20091111130634.GA618@yoda.bytereef.org>
Message-ID: <20091111133433.GA2645@yoda.bytereef.org>

Stefan Krah <stefan-usenet at bytereef.org> wrote:
> I simply think that apart from rounding, the output of format should not
> change the numerical value of its argument. The format functions in C do
> not allow this to happen. Are there other languages where this is possible?

Actually there are other cases: strfmon() allows this, too. Let's drop it.


Stefan Krah



From drobinow at gmail.com  Wed Nov 11 15:00:41 2009
From: drobinow at gmail.com (David Robinow)
Date: Wed, 11 Nov 2009 06:00:41 -0800
Subject: [Python-Dev] Replacing IDLE
In-Reply-To: <4AFA54BC.90401@canterbury.ac.nz>
References: <d34314100911100820u7a9e6f7bp6dde4ec8d2ff2c4@mail.gmail.com>
	<59691AA2-995E-452B-B21D-0C7F8861C7A6@mac.com>
	<d34314100911101748y202335a4td05f38880c0d694d@mail.gmail.com>
	<4AFA54BC.90401@canterbury.ac.nz>
Message-ID: <4eb0089f0911110600l6c4a895j5da0c7bdb367d708@mail.gmail.com>

On Tue, Nov 10, 2009 at 10:07 PM, Greg Ewing
<greg.ewing at canterbury.ac.nz> wrote:
> Seems to me the only kind of IDE that it makes sense to
> ship with Python is one that is written in Python and
> maintained by the core developers. Anything else is best
> left as a third party package for download by those
> who want to use it.
>
> Tkinter is arguably not a very good basis for it from a
> technical point of view, but so far all the alternatives
> are too bloated and/or nonportable to consider bundling
> with the standard distribution.
>
> So the only realistic alternatives at the moment seem
> to be either IDLE+Tkinter or nothing.
>
 My immediate reaction? What about stdwin?
 Then I found this:
http://mail.python.org/pipermail/python-dev/2000-October/010113.html

From eric at trueblade.com  Wed Nov 11 15:11:48 2009
From: eric at trueblade.com (Eric Smith)
Date: Wed, 11 Nov 2009 09:11:48 -0500
Subject: [Python-Dev] [format] Restrict fill characters for numerical
 values
In-Reply-To: <20091111130634.GA618@yoda.bytereef.org>
References: <20091111113613.GA4802@mail.bytereef.org>	<4AFAB07E.2030707@trueblade.com>
	<20091111130634.GA618@yoda.bytereef.org>
Message-ID: <4AFAC624.8070007@trueblade.com>

Stefan Krah wrote:
> Eric Smith <eric at trueblade.com> wrote:
>> Stefan Krah wrote:
>>> Hi,
>>>
>>> I think http://www.python.org/dev/peps/pep-3101/ is too liberal with the
>>> choice of fill characters for numerical values. As far as I can see, this
>>> is quite legal:
>>>
>>>
>>> Python 2.7a0 (trunk:76132M, Nov  6 2009, 15:20:35) 
>>> [GCC 4.1.3 20080623 (prerelease) (Ubuntu 4.1.2-23ubuntu3)] on linux2
>>> Type "help", "copyright", "credits" or "license" for more information.
>>>>> >from decimal import *
>>>>>> format(Decimal(4), "9<10")
>>> '4999999999'
>>>>>> format(Decimal(4), "9>10")
>>> '9999999994'
>>>>>> format(Decimal(4), "->2")
>>> '-4'
>>>
>>>
>>> I propose to disallow digits and '+-' fill characters for numerical values,
>>> except for the combination '0, left-padding'. Actually, I'd prefer to allow
>>> only whitespace and '0, left-padding'.
>> Why?
>>
>> What problem are you trying to prevent that would justify the extra code
>> to implement this and the extra documentation to explain it?
> 
> 
> I simply think that apart from rounding, the output of format should not
> change the numerical value of its argument. The format functions in C do
> not allow this to happen. Are there other languages where this is possible?

I don't know if there are other languages that allow it. But I 
definitely have use cases for padding with '*', for example. I don't see 
the point in just disallowing numbers 1 through 9. If you don't want 
your numbers padded with 9's, don't put a 9 in your format string.

Eric.


From eric at trueblade.com  Wed Nov 11 15:18:11 2009
From: eric at trueblade.com (Eric Smith)
Date: Wed, 11 Nov 2009 09:18:11 -0500
Subject: [Python-Dev] PEP 382 status
In-Reply-To: <94bdd2610911110536n60a882acj8b2cd63ced3af25b@mail.gmail.com>
References: <94bdd2610911110536n60a882acj8b2cd63ced3af25b@mail.gmail.com>
Message-ID: <4AFAC7A3.4000801@trueblade.com>

Tarek Ziad? wrote:
> I was wondering what's the status of PEP 382. Is anyone (MvL?) is
> going to start to work on its implementation for Python 2.7/3.2
> inclusion ?

If Martin isn't interested in doing it, I'll take a try at it. But I'll 
need some rough guidance on the implementation approach.

Eric.

From pluskid at gmail.com  Wed Nov 11 17:16:43 2009
From: pluskid at gmail.com (Zhang Chiyuan)
Date: Thu, 12 Nov 2009 00:16:43 +0800
Subject: [Python-Dev] Is this a bug of the HTMLParser?
Message-ID: <b6f0e9dd0911110816n16258c03se8d6bb6da745e2ed@mail.gmail.com>

Hi all,

I'm using BeautifulSoup to parsing an HTML page and find it refused to
parse the page. By looking at the backtrace, I found it is a problem
with the python built-in HTMLParser.py. In fact, the web page I'm
parsing is with some Chinese characters. there is a tag like <img
src=/foo/bar.png alt=??> , note this is legacy html page where the
attributes are not quoted. However, the regexp defined in
HTMLParser.py is :

 attrfind = re.compile(
     r'\s*([a-zA-Z_][-.:a-zA-Z_0-9]*)(\s*=\s*'
     r'(\'[^\']*\'|"[^"]*"|[-a-zA-Z0-9./,:;+*%?!&$\(\)_#=~@]*))?')

Note that the Chinese character (also any other non-english
characters), so it fire an error parsing this. I'm not sure whether
the HTML standard allow un-quoted non-ASCII characters in the
attributes. If it allows, this seems to be a bug. and the regexp to
better be [^>\s] IMHO.

BTW: It seems something like :

<script>
var st = "<a></";
</script>

can not be parsed. :-/

--
pluskid
http://blog.pluskid.org

From fuzzyman at voidspace.org.uk  Wed Nov 11 17:24:02 2009
From: fuzzyman at voidspace.org.uk (Michael Foord)
Date: Wed, 11 Nov 2009 17:24:02 +0100
Subject: [Python-Dev] Is this a bug of the HTMLParser?
In-Reply-To: <b6f0e9dd0911110816n16258c03se8d6bb6da745e2ed@mail.gmail.com>
References: <b6f0e9dd0911110816n16258c03se8d6bb6da745e2ed@mail.gmail.com>
Message-ID: <4AFAE522.2090403@voidspace.org.uk>

Hello Zhang Chiyuan,

Can you file a bug on the Python issue tracker please:

    http://bugs.python.org

Thanks

Michael Foord

Zhang Chiyuan wrote:
> Hi all,
>
> I'm using BeautifulSoup to parsing an HTML page and find it refused to
> parse the page. By looking at the backtrace, I found it is a problem
> with the python built-in HTMLParser.py. In fact, the web page I'm
> parsing is with some Chinese characters. there is a tag like <img
> src=/foo/bar.png alt=??> , note this is legacy html page where the
> attributes are not quoted. However, the regexp defined in
> HTMLParser.py is :
>
>  attrfind = re.compile(
>      r'\s*([a-zA-Z_][-.:a-zA-Z_0-9]*)(\s*=\s*'
>      r'(\'[^\']*\'|"[^"]*"|[-a-zA-Z0-9./,:;+*%?!&$\(\)_#=~@]*))?')
>
> Note that the Chinese character (also any other non-english
> characters), so it fire an error parsing this. I'm not sure whether
> the HTML standard allow un-quoted non-ASCII characters in the
> attributes. If it allows, this seems to be a bug. and the regexp to
> better be [^>\s] IMHO.
>
> BTW: It seems something like :
>
> <script>
> var st = "<a></";
> </script>
>
> can not be parsed. :-/
>
> --
> pluskid
> http://blog.pluskid.org
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk
>   


-- 
http://www.ironpythoninaction.com/


From oshecho at gmail.com  Wed Nov 11 18:59:47 2009
From: oshecho at gmail.com (Echo)
Date: Wed, 11 Nov 2009 12:59:47 -0500
Subject: [Python-Dev] IDLE as default Python editor
In-Reply-To: <hdcp8i$9q4$1@ger.gmane.org>
References: <d34314100911082245w79c9233dwad2a0aa867e06355@mail.gmail.com>
	<87iqdk2n7b.fsf@benfinney.id.au> <4AF80E73.5090104@gmail.com>
	<4AF89370.2040206@v.loewis.de> <4AF944A9.2030108@gmail.com>
	<hdcp8i$9q4$1@ger.gmane.org>
Message-ID: <910885da0911110959m6c86edbepf15934902982c96a@mail.gmail.com>

On Tue, Nov 10, 2009 at 5:27 PM, Georg Brandl <g.brandl at gmx.net> wrote:
>
> Quite simple: because we can't possibly ship Emacs.
>
> cheers,
> Georg
>

We just need a PyEmacs. Written in python, extensible in elist and
python. Nice and simple ;-D


-- 
-Brandon Singer

From fdrake at acm.org  Wed Nov 11 19:50:05 2009
From: fdrake at acm.org (Fred Drake)
Date: Wed, 11 Nov 2009 13:50:05 -0500
Subject: [Python-Dev] IDLE as default Python editor
In-Reply-To: <910885da0911110959m6c86edbepf15934902982c96a@mail.gmail.com>
References: <d34314100911082245w79c9233dwad2a0aa867e06355@mail.gmail.com> 
	<87iqdk2n7b.fsf@benfinney.id.au> <4AF80E73.5090104@gmail.com> 
	<4AF89370.2040206@v.loewis.de> <4AF944A9.2030108@gmail.com> 
	<hdcp8i$9q4$1@ger.gmane.org>
	<910885da0911110959m6c86edbepf15934902982c96a@mail.gmail.com>
Message-ID: <9cee7ab80911111050k5f2a077en36d41a134d2dbfb2@mail.gmail.com>

On Wed, Nov 11, 2009 at 12:59 PM, Echo <oshecho at gmail.com> wrote:
> We just need a PyEmacs. Written in python, extensible in elist and
> python. Nice and simple ;-D

I'd even give up the elisp support if I could have Python in my Emacs.


  -Fred

-- 
Fred L. Drake, Jr.    <fdrake at gmail.com>
"Chaos is the score upon which reality is written." --Henry Miller

From martin at v.loewis.de  Wed Nov 11 21:04:00 2009
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Wed, 11 Nov 2009 21:04:00 +0100
Subject: [Python-Dev] PEP 382 status
In-Reply-To: <94bdd2610911110536n60a882acj8b2cd63ced3af25b@mail.gmail.com>
References: <94bdd2610911110536n60a882acj8b2cd63ced3af25b@mail.gmail.com>
Message-ID: <4AFB18B0.3050800@v.loewis.de>

> I was wondering what's the status of PEP 382. Is anyone (MvL?) is
> going to start to work on its implementation for Python 2.7/3.2
> inclusion ?

I'll be working on an implementation, but contributions are welcome.
Unfortunately, I'm really short on free software time recently (and
keep hoping that this will improve).

One thing that I always wondered about is the status of importlib.
Will that be used in 3.2?

In addition, I still owe a few answers to comments from the previous
discussion.

Regards,
Martin

From drobinow at gmail.com  Wed Nov 11 21:17:44 2009
From: drobinow at gmail.com (David Robinow)
Date: Wed, 11 Nov 2009 15:17:44 -0500
Subject: [Python-Dev] IDLE as default Python editor
In-Reply-To: <9cee7ab80911111050k5f2a077en36d41a134d2dbfb2@mail.gmail.com>
References: <d34314100911082245w79c9233dwad2a0aa867e06355@mail.gmail.com>
	<87iqdk2n7b.fsf@benfinney.id.au> <4AF80E73.5090104@gmail.com>
	<4AF89370.2040206@v.loewis.de> <4AF944A9.2030108@gmail.com>
	<hdcp8i$9q4$1@ger.gmane.org>
	<910885da0911110959m6c86edbepf15934902982c96a@mail.gmail.com>
	<9cee7ab80911111050k5f2a077en36d41a134d2dbfb2@mail.gmail.com>
Message-ID: <4eb0089f0911111217n688cc830x5ba768dec9df53a@mail.gmail.com>

On Wed, Nov 11, 2009 at 1:50 PM, Fred Drake <fdrake at acm.org> wrote:
> On Wed, Nov 11, 2009 at 12:59 PM, Echo <oshecho at gmail.com> wrote:
>> We just need a PyEmacs. Written in python, extensible in elist and
>> python. Nice and simple ;-D
> I'd even give up the elisp support if I could have Python in my Emacs.
Have you tried Pymacs?

From ncoghlan at gmail.com  Wed Nov 11 22:18:23 2009
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Thu, 12 Nov 2009 07:18:23 +1000
Subject: [Python-Dev] 2.7/3.2 release schedule
In-Reply-To: <DBD8F06E720A4BD18EA555B4710088CB@RaymondLaptop1>
References: <1afaf6160911021200m1bcf27cck4a78db471db038a7@mail.gmail.com><71ac8a310911100241x2567c396pc2e5ef4653ced917@mail.gmail.com><4AF966F8.5040008@gmail.com>
	<4AF9D5A5.5000707@v.loewis.de>
	<1afaf6160911101406t6da78accq8918eb4ceb1adca5@mail.gmail.com>
	<DBD8F06E720A4BD18EA555B4710088CB@RaymondLaptop1>
Message-ID: <4AFB2A1F.5060307@gmail.com>

Raymond Hettinger wrote:
> 
> [MvL]
>>> I personally think that decoupling the releases would be best, i.e.
>>> not start thinking about 3.2 for another 6 months.
> 
> [Benjamin]
>> The problem with that is that there is a period of time where 2.x has
>> features which 3.x doesn't. My preference is to move back the whole
>> schedule 6 months.
> 
> My preference is to decouple and let 2.7 go out 18 months after 2.6.
> In general, 2.x users should not have to pay a price for whatever
> we do with 3.x.

And I guess anyone doing forward ports is likely going to be maintaining
2.6 compatibility anyway, so waiting until 3.2 comes out before using
new 2.7 features wouldn't be a major burden.

I think I've decided I don't mind either way, so I'm fine with whichever
approach is easier for Benjamin and the platform installer builders to
manage.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------

From guido at python.org  Wed Nov 11 22:23:47 2009
From: guido at python.org (Guido van Rossum)
Date: Wed, 11 Nov 2009 13:23:47 -0800
Subject: [Python-Dev] 2.7/3.2 release schedule
In-Reply-To: <4AFB2A1F.5060307@gmail.com>
References: <1afaf6160911021200m1bcf27cck4a78db471db038a7@mail.gmail.com> 
	<71ac8a310911100241x2567c396pc2e5ef4653ced917@mail.gmail.com> 
	<4AF966F8.5040008@gmail.com> <4AF9D5A5.5000707@v.loewis.de> 
	<1afaf6160911101406t6da78accq8918eb4ceb1adca5@mail.gmail.com> 
	<DBD8F06E720A4BD18EA555B4710088CB@RaymondLaptop1>
	<4AFB2A1F.5060307@gmail.com>
Message-ID: <ca471dc20911111323r6a12be2cofa41a590c3d8af1@mail.gmail.com>

On Wed, Nov 11, 2009 at 1:18 PM, Nick Coghlan <ncoghlan at gmail.com> wrote:
> I think I've decided I don't mind either way, so I'm fine with whichever
> approach is easier for Benjamin and the platform installer builders to
> manage.

+1

-- 
--Guido van Rossum (python.org/~guido)

From naughtysriram at gmail.com  Thu Nov 12 10:23:50 2009
From: naughtysriram at gmail.com (Sriram Srinivasan)
Date: Thu, 12 Nov 2009 14:23:50 +0500
Subject: [Python-Dev] standard libraries don't behave like standard
	'libraries'
Message-ID: <c910262d0911120123i7c7e9dla918fff50cb5aecd@mail.gmail.com>

I guess why every programming language has some kind of a 'standard
library' built in within it. In my view it must not be called as a 'library'
at all. what it does
is like a 'bunch of built-in programs ready-made to do stuff'.

Lets see what a 'library' does:

1. offers books for customers
 1.1 lets user select a book by genre, etc
 1.2 lets user to use different books of same genre, etc
 1.3 lets user to use books by same author, etc for different genre

2. keeps track of all the books + their genre
 2.1 first knows what all books it has at present
 2.2 when new book comes it is added to the particular shelf sorted by
genre,author,edition, etc.
 2.3 when books become old they are kept separately for future reference
 2.4 very old books can be sent to a museum/discarded

I guess no standard library does the minimum of this but wants to be
called a library.

As a python user I always wanted the standard library to have such
features so the user/developer decides to use what set of libraries he
want.

consider the libraries for 2.5 ,2.6, 3K are all available to the user,
the user selects what he wants with something like.

use library 2.5 or use library 2.6 etc.

The 2 main things that the library management interface has to do is
intra library management and inter library management.

intra library mgmt- consider books to be different libraries
(standard, commercial, private, hobby, etc)
inter library mgmt- consider books to be modules inside a library
( standard, commercial, private, hobby, etc)

if somehow we could accomplish this kind of mother of a all plugin/ad-
hoc system that is a real breakthrough.

Advantages:

1. new modules can be added to the stream quickly
2. let the user select what he want to do
3. modules (that interdepend on each other) can be packed into small
distribution and added to the stream quickly without waiting for new
releases
4. solution to problems like py 2.x and 3.x
5. users can be up to date
6. documentation becomes easy + elaborate to users
7. bug managing is easy too
8. more feed back
9. testing also becomes easy
10. many more , i don't know.. you have to find.

Python already has some thing like that __future__ stuff. but my
question is how many people know that? and how many use that? most of
them wait until old crust gets totally removed. that is bad for user
and python. that is why problems like py2.x py3.x originate. If there
is a newer book collection it must always be available at the library.
i must not go to another library to get that book.
I am not an expert, I am just another python learner. These are just my
views on the state of the standard libraries and to
make them state-of-the-art..! ;)

-- 
Regards,
Sriram.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20091112/5166f7e9/attachment-0001.htm>

From ludvig at lericson.se  Thu Nov 12 10:44:32 2009
From: ludvig at lericson.se (Ludvig Ericson)
Date: Thu, 12 Nov 2009 10:44:32 +0100
Subject: [Python-Dev] PyPI comments and ratings, *really*?
Message-ID: <D7E23A6D-C91C-46D8-A8FE-1EB16F3B2101@lericson.se>

Why are there comments on PyPI? Moreso, why are there comments which I  
cannot control as a package author on my very own packages? That's  
just absurd.

It's *my* package, and so should be *my* choice if I want user input  
or not.

And ratings? I thought it was the Python Package Index, not the Python  
Anonymous Package Tribunal.

As I see it, there are only two ways to fix these misguided steps of  
development: throw them out, or make them opt-in settings.

The comments I simply do not understand. Why not instead provide a  
form for mailing the package author?
The ratings are just not what PyPI should be doing, is about, or what  
I signed up for.

-L

From fuzzyman at voidspace.org.uk  Thu Nov 12 11:15:11 2009
From: fuzzyman at voidspace.org.uk (Michael Foord)
Date: Thu, 12 Nov 2009 11:15:11 +0100
Subject: [Python-Dev] standard libraries don't behave like
	standard	'libraries'
In-Reply-To: <c910262d0911120123i7c7e9dla918fff50cb5aecd@mail.gmail.com>
References: <c910262d0911120123i7c7e9dla918fff50cb5aecd@mail.gmail.com>
Message-ID: <4AFBE02F.6020302@voidspace.org.uk>

Sriram Srinivasan wrote:
> I guess why every programming language has some kind of a 'standard
> library' built in within it. In my view it must not be called as a 
> 'library' at all. what it does
> is like a 'bunch of built-in programs ready-made to do stuff'.
>
> Lets see what a 'library' does:
>
> 1. offers books for customers
>  1.1 lets user select a book by genre, etc
>  1.2 lets user to use different books of same genre, etc
>  1.3 lets user to use books by same author, etc for different genre
>
> 2. keeps track of all the books + their genre
>  2.1 first knows what all books it has at present
>  2.2 when new book comes it is added to the particular shelf sorted by 
> genre,author,edition, etc.
>  2.3 when books become old they are kept separately for future reference
>  2.4 very old books can be sent to a museum/discarded
>
> I guess no standard library does the minimum of this but wants to be
> called a library.
>

I don't really understand your requirements, but it sound like you want 
a package management system. The standard library just provides a 
standard set of tools (it is the books not the book management system - 
although part of what you want is in the standard library in the form of 
distutils which is currently receiving a radical overhaul).

You should look at Distribute and virtualenv, which gets you pretty much 
what you are suggesting (as far as I can tell):

    http://pypi.python.org/pypi/virtualenv
    http://pypi.python.org/pypi/distribute

All the best,

Michael Foord


> As a python user I always wanted the standard library to have such
> features so the user/developer decides to use what set of libraries he
> want.
>
> consider the libraries for 2.5 ,2.6, 3K are all available to the user,
> the user selects what he wants with something like.
>
> use library 2.5 or use library 2.6 etc.
>
> The 2 main things that the library management interface has to do is
> intra library management and inter library management.
>
> intra library mgmt- consider books to be different libraries
> (standard, commercial, private, hobby, etc)
> inter library mgmt- consider books to be modules inside a library
> ( standard, commercial, private, hobby, etc)
>
> if somehow we could accomplish this kind of mother of a all plugin/ad-
> hoc system that is a real breakthrough.
>
> Advantages:
>
> 1. new modules can be added to the stream quickly
> 2. let the user select what he want to do
> 3. modules (that interdepend on each other) can be packed into small
> distribution and added to the stream quickly without waiting for new
> releases
> 4. solution to problems like py 2.x and 3.x
> 5. users can be up to date
> 6. documentation becomes easy + elaborate to users
> 7. bug managing is easy too
> 8. more feed back
> 9. testing also becomes easy
> 10. many more , i don't know.. you have to find.
>
> Python already has some thing like that __future__ stuff. but my
> question is how many people know that? and how many use that? most of
> them wait until old crust gets totally removed. that is bad for user
> and python. that is why problems like py2.x py3.x originate. If there
> is a newer book collection it must always be available at the library.
> i must not go to another library to get that book.
>
> I am not an expert, I am just another python learner. These are just 
> my views on the state of the standard libraries and to
> make them state-of-the-art..! ;)
>
> -- 
> Regards,
> Sriram.
> ------------------------------------------------------------------------
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk
>   


-- 
http://www.ironpythoninaction.com/


From naughtysriram at gmail.com  Thu Nov 12 11:54:58 2009
From: naughtysriram at gmail.com (Sriram Srinivasan)
Date: Thu, 12 Nov 2009 15:54:58 +0500
Subject: [Python-Dev] standard libraries don't behave like standard
	'libraries'
In-Reply-To: <4AFBE02F.6020302@voidspace.org.uk>
References: <c910262d0911120123i7c7e9dla918fff50cb5aecd@mail.gmail.com>
	<4AFBE02F.6020302@voidspace.org.uk>
Message-ID: <c910262d0911120254o1a900338u50b442701e9d7c75@mail.gmail.com>

>
> I don't really understand your requirements, but it sound like you want a
> package management system. The standard library just provides a standard set
> of tools (it is the books not the book management system - although part of
> what you want is in the standard library in the form of distutils which is
> currently receiving a radical overhaul).
>
> You should look at Distribute and virtualenv, which gets you pretty much
> what you are suggesting (as far as I can tell):
>
>   http://pypi.python.org/pypi/virtualenv
>   http://pypi.python.org/pypi/distribute
>
> All the best,
>
> Michael Foord
>


I don't know if you have used
Dev-C++<http://www.bloodshed.net/dev/index.html%20>. It has a 'package
management' mechanism for the standard libraries.
please see the http://devpaks.org/ webpage where all the packaged libraries
are stored.

In python we have the PyPI which is equivalent to the
http://devpacks.orgbut in PyPI the packages are all user made
applications.
What I want is similar to PyPI but for the python standard libraries, so
that they (libraries) are as add-on as possible.

I guess you understand what I am thinking... and do pardon my english too..

-- 

Regards,
Sriram.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20091112/62e27d22/attachment.htm>

From eckhardt at satorlaser.com  Thu Nov 12 12:14:09 2009
From: eckhardt at satorlaser.com (Ulrich Eckhardt)
Date: Thu, 12 Nov 2009 12:14:09 +0100
Subject: [Python-Dev] standard libraries don't behave like standard
 'libraries'
In-Reply-To: <c910262d0911120254o1a900338u50b442701e9d7c75@mail.gmail.com>
References: <c910262d0911120123i7c7e9dla918fff50cb5aecd@mail.gmail.com> 
	<4AFBE02F.6020302@voidspace.org.uk> 
	<c910262d0911120254o1a900338u50b442701e9d7c75@mail.gmail.com>
Message-ID: <200911121214.09958.eckhardt@satorlaser.com>

On Thursday 12 November 2009, Sriram Srinivasan wrote:
> I don't know if you have used Dev-C++. It has a 'package management'
> mechanism for the standard libraries.

I disagree. It has a package management system for libraries, not for the 
standard libraries. The point is that the Python standard library is supplied 
as part of Python itself, as is e.g. the C++ standard library as part of a 
C++ compiler.

> In python we have the PyPI which is equivalent to the
> http://devpacks.org but in PyPI the packages are all user made
> applications.

...as in devpacks.org. Please scroll down on that page and read the warning 
that mentions than any person can upload a package possibly containing 
malware or broken code or whatever.

> What I want is similar to PyPI but for the python standard libraries, so
> that they (libraries) are as add-on as possible.

So, you want to make libs that are currently part of the standard library 
external libs available via PyPI? If that is what you want, it would be 
helpful if you provided a reason and also which libraries you're talking 
about.

Uli

-- 
Sator Laser GmbH, Fangdieckstra?e 75a, 22547 Hamburg, Deutschland
Gesch?ftsf?hrer: Thorsten F?cking, Amtsgericht Hamburg HR B62 932

**************************************************************************************
Sator Laser GmbH, Fangdieckstra?e 75a, 22547 Hamburg, Deutschland
Gesch?ftsf?hrer: Thorsten F?cking, Amtsgericht Hamburg HR B62 932
**************************************************************************************
           Visit our website at <http://www.satorlaser.de/>
**************************************************************************************
Diese E-Mail einschlie?lich s?mtlicher Anh?nge ist nur f?r den Adressaten bestimmt und kann vertrauliche Informationen enthalten. Bitte benachrichtigen Sie den Absender umgehend, falls Sie nicht der beabsichtigte Empf?nger sein sollten. Die E-Mail ist in diesem Fall zu l?schen und darf weder gelesen, weitergeleitet, ver?ffentlicht oder anderweitig benutzt werden.
E-Mails k?nnen durch Dritte gelesen werden und Viren sowie nichtautorisierte ?nderungen enthalten. Sator Laser GmbH ist f?r diese Folgen nicht verantwortlich.
**************************************************************************************


From naughtysriram at gmail.com  Thu Nov 12 12:36:30 2009
From: naughtysriram at gmail.com (Sriram Srinivasan)
Date: Thu, 12 Nov 2009 16:36:30 +0500
Subject: [Python-Dev] standard libraries don't behave like standard
	'libraries'
In-Reply-To: <200911121214.09958.eckhardt@satorlaser.com>
References: <c910262d0911120123i7c7e9dla918fff50cb5aecd@mail.gmail.com>
	<4AFBE02F.6020302@voidspace.org.uk>
	<c910262d0911120254o1a900338u50b442701e9d7c75@mail.gmail.com>
	<200911121214.09958.eckhardt@satorlaser.com>
Message-ID: <c910262d0911120336q5784509er759443d0b5c95a2@mail.gmail.com>

> I disagree. It has a package management system for libraries, not for the
> standard libraries. The point is that the Python standard library is
> supplied
> as part of Python itself, as is e.g. the C++ standard library as part of a
> C++ compiler.
>
>
standard libraries i meant the standard libraries used.
and what i asked for is for(in python) both the standard-libraries and the
standard libraries used.
c the term (intra and inter library management) which includes the default
standard libraries and other standard libraries


>
> ...as in devpacks.org. Please scroll down on that page and read the
> warning
> that mentions than any person can upload a package possibly containing
> malware or broken code or whatever.
>
>
if you talk about security then nothing is secure for sure!
in PyPI ppl can upload their application as egg.

what if there is another site where only eggs of python.org containing
add-on standard libraries are distributed
 and noone is allowed to upload stuff except the python.org or PSF.

>
> So, you want to make libs that are currently part of the standard library
> external libs available via PyPI? If that is what you want, it would be
> helpful if you provided a reason and also which libraries you're talking
> about.
>
>
>
Not exactly via PyPI but something similar to it.
and don't compare C++ with python. both have their own + & -

-- 

Regards,
Sriram.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20091112/b4716f50/attachment.htm>

From matthew at matthewwilkes.co.uk  Thu Nov 12 13:03:27 2009
From: matthew at matthewwilkes.co.uk (Matthew Wilkes)
Date: Thu, 12 Nov 2009 12:03:27 +0000
Subject: [Python-Dev] standard libraries don't behave like standard
	'libraries'
In-Reply-To: <c910262d0911120336q5784509er759443d0b5c95a2@mail.gmail.com>
References: <c910262d0911120123i7c7e9dla918fff50cb5aecd@mail.gmail.com>
	<4AFBE02F.6020302@voidspace.org.uk>
	<c910262d0911120254o1a900338u50b442701e9d7c75@mail.gmail.com>
	<200911121214.09958.eckhardt@satorlaser.com>
	<c910262d0911120336q5784509er759443d0b5c95a2@mail.gmail.com>
Message-ID: <BBC376A4-44C7-4E23-A69C-0FFB36AF3638@matthewwilkes.co.uk>


On 2009-11-12, at 1136, Sriram Srinivasan wrote:

> standard libraries i meant the standard libraries used.
> and what i asked for is for(in python) both the standard-libraries  
> and the standard libraries used.
> c the term (intra and inter library management) which includes the  
> default standard libraries and other standard libraries

I don't think you're using the correct terminology.  The standard  
library is what is shipped with the interpreter.  It's the only thing  
that's common between all users of that interpreter, i.e. it's standard.

> what if there is another site where only eggs of python.org  
> containing add-on standard libraries are distributed
>  and noone is allowed to upload stuff except the python.org or PSF.

The PSF does not maintain add-on libraries, nor can it make guarantees  
about other add-ons libraries.

>> So, you want to make libs that are currently part of the standard  
>> library
>> external libs available via PyPI?
> Not exactly via PyPI but something similar to it.

Again, why?

> and don't compare C++ with python. both have their own + & -

Frankly, comparing and contrasting with other languages is very  
useful, I have little to no idea what you're talking about, and doubt  
it's got anything to do with the development of the Python language.

Matthew


From naughtysriram at gmail.com  Thu Nov 12 13:08:22 2009
From: naughtysriram at gmail.com (Sriram Srinivasan)
Date: Thu, 12 Nov 2009 17:08:22 +0500
Subject: [Python-Dev] standard libraries don't behave like standard
	'libraries'
In-Reply-To: <BBC376A4-44C7-4E23-A69C-0FFB36AF3638@matthewwilkes.co.uk>
References: <c910262d0911120123i7c7e9dla918fff50cb5aecd@mail.gmail.com>
	<4AFBE02F.6020302@voidspace.org.uk>
	<c910262d0911120254o1a900338u50b442701e9d7c75@mail.gmail.com>
	<200911121214.09958.eckhardt@satorlaser.com>
	<c910262d0911120336q5784509er759443d0b5c95a2@mail.gmail.com>
	<BBC376A4-44C7-4E23-A69C-0FFB36AF3638@matthewwilkes.co.uk>
Message-ID: <c910262d0911120408s16389bb5p4a73bc543e880655@mail.gmail.com>

ok let me make it more clear..
forget how you use python now.. i am talking about __futuristic__
python programming.

these are just my ideas.. more over i can say imaginations.
there is no more python2.x or python3.x or python y.x releases. there
is only updates of python and standard library say 1.1.5 and 1.1.6.
let the difference be an old xml library updated with a new regex
support.

i am coding my program now.
i want my application to be compatible with 1.1.5 library

use library 1.1.5
import blah form blahblah
...
...

i cannot use regex feature of xml in this application
i then update my library in the evening
now both the libraries are present in my system.
now i try using the new feature

use library 1.1.6 #thats all now i get all features
import blah from blahblah
...
...

-- 

Regards,
Sriram.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20091112/fae16dc1/attachment.htm>

From matthew at matthewwilkes.co.uk  Thu Nov 12 13:13:39 2009
From: matthew at matthewwilkes.co.uk (Matthew Wilkes)
Date: Thu, 12 Nov 2009 12:13:39 +0000
Subject: [Python-Dev] standard libraries don't behave like standard
	'libraries'
In-Reply-To: <c910262d0911120408s16389bb5p4a73bc543e880655@mail.gmail.com>
References: <c910262d0911120123i7c7e9dla918fff50cb5aecd@mail.gmail.com>
	<4AFBE02F.6020302@voidspace.org.uk>
	<c910262d0911120254o1a900338u50b442701e9d7c75@mail.gmail.com>
	<200911121214.09958.eckhardt@satorlaser.com>
	<c910262d0911120336q5784509er759443d0b5c95a2@mail.gmail.com>
	<BBC376A4-44C7-4E23-A69C-0FFB36AF3638@matthewwilkes.co.uk>
	<c910262d0911120408s16389bb5p4a73bc543e880655@mail.gmail.com>
Message-ID: <DB74C2BF-6F40-42DC-8B92-54823EC0863C@matthewwilkes.co.uk>

Oh, I see.

> use library 1.1.5

versus

> use library 1.1.6 #thats all now i get all features

That's part of pkg_resources.  It looks like this:

pkg_resources.require("mylibrary==1.1.6")
import mylibrary

There are plenty of other ways to manage this, most people use systems  
like virtualenv or buildout, but that's a discussion for the normal  
python mailing list, not the development one.

Hope that helps,

Matt

From ncoghlan at gmail.com  Thu Nov 12 13:26:56 2009
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Thu, 12 Nov 2009 22:26:56 +1000
Subject: [Python-Dev] standard libraries don't behave like
	standard	'libraries'
In-Reply-To: <DB74C2BF-6F40-42DC-8B92-54823EC0863C@matthewwilkes.co.uk>
References: <c910262d0911120123i7c7e9dla918fff50cb5aecd@mail.gmail.com>	<4AFBE02F.6020302@voidspace.org.uk>	<c910262d0911120254o1a900338u50b442701e9d7c75@mail.gmail.com>	<200911121214.09958.eckhardt@satorlaser.com>	<c910262d0911120336q5784509er759443d0b5c95a2@mail.gmail.com>	<BBC376A4-44C7-4E23-A69C-0FFB36AF3638@matthewwilkes.co.uk>	<c910262d0911120408s16389bb5p4a73bc543e880655@mail.gmail.com>
	<DB74C2BF-6F40-42DC-8B92-54823EC0863C@matthewwilkes.co.uk>
Message-ID: <4AFBFF10.7010800@gmail.com>

Matthew Wilkes wrote:
> There are plenty of other ways to manage this, most people use systems
> like virtualenv or buildout, but that's a discussion for the normal
> python mailing list, not the development one.

Indeed. Please take this to python-list (it isn't even a coherent enough
suggestion for python-ideas at this point - there are way too many
mistaken assumptions and/or unconventional uses of terminology)

Regards,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------

From exarkun at twistedmatrix.com  Thu Nov 12 14:06:57 2009
From: exarkun at twistedmatrix.com (exarkun at twistedmatrix.com)
Date: Thu, 12 Nov 2009 13:06:57 -0000
Subject: [Python-Dev] PyPI comments and ratings, *really*?
In-Reply-To: <D7E23A6D-C91C-46D8-A8FE-1EB16F3B2101@lericson.se>
References: <D7E23A6D-C91C-46D8-A8FE-1EB16F3B2101@lericson.se>
Message-ID: <20091112130657.3229.1629790699.divmod.xquotient.653@localhost.localdomain>

On 09:44 am, ludvig at lericson.se wrote:
>Why are there comments on PyPI? Moreso, why are there comments which I 
>cannot control as a package author on my very own packages? That's 
>just absurd.
>
>It's *my* package, and so should be *my* choice if I want user input 
>or not.
>
>And ratings? I thought it was the Python Package Index, not the Python 
>Anonymous Package Tribunal.
>
>As I see it, there are only two ways to fix these misguided steps of 
>development: throw them out, or make them opt-in settings.
>
>The comments I simply do not understand. Why not instead provide a 
>form for mailing the package author?
>The ratings are just not what PyPI should be doing, is about, or what 
>I signed up for.

See the various catalog-sig threads on this topic.

Jean-Paul

From steve at pearwood.info  Thu Nov 12 14:38:03 2009
From: steve at pearwood.info (Steven D'Aprano)
Date: Fri, 13 Nov 2009 00:38:03 +1100
Subject: [Python-Dev] PyPI comments and ratings, *really*?
In-Reply-To: <D7E23A6D-C91C-46D8-A8FE-1EB16F3B2101@lericson.se>
References: <D7E23A6D-C91C-46D8-A8FE-1EB16F3B2101@lericson.se>
Message-ID: <200911130038.05516.steve@pearwood.info>

On Thu, 12 Nov 2009 08:44:32 pm Ludvig Ericson wrote:
> Why are there comments on PyPI? Moreso, why are there comments which
> I cannot control as a package author on my very own packages? That's
> just absurd.

No, what's absurd is thinking that the act of publishing software 
somehow gives you the right to demand control over what others say 
about your software.

I don't suppose that this rant of yours has something to do with the 
comment posted today?

http://pypi.python.org/pypi/spypam/1.0




-- 
Steven D'Aprano

From jnoller at gmail.com  Thu Nov 12 15:06:31 2009
From: jnoller at gmail.com (Jesse Noller)
Date: Thu, 12 Nov 2009 09:06:31 -0500
Subject: [Python-Dev] PyPI comments and ratings, *really*?
In-Reply-To: <200911130038.05516.steve@pearwood.info>
References: <D7E23A6D-C91C-46D8-A8FE-1EB16F3B2101@lericson.se>
	<200911130038.05516.steve@pearwood.info>
Message-ID: <4222a8490911120606t3e138957kcdb21104261a1ead@mail.gmail.com>

On Thu, Nov 12, 2009 at 8:38 AM, Steven D'Aprano <steve at pearwood.info> wrote:
> On Thu, 12 Nov 2009 08:44:32 pm Ludvig Ericson wrote:
>> Why are there comments on PyPI? Moreso, why are there comments which
>> I cannot control as a package author on my very own packages? That's
>> just absurd.
>
> No, what's absurd is thinking that the act of publishing software
> somehow gives you the right to demand control over what others say
> about your software.
>
> I don't suppose that this rant of yours has something to do with the
> comment posted today?

Frankly, I agree with him. As implemented, I *and others* think this
is broken. I've taken the stance of not publishing things to PyPi
until A> I find the time to contribute to make it better or B> It
changes.

jesse

From barry at python.org  Thu Nov 12 15:25:56 2009
From: barry at python.org (Barry Warsaw)
Date: Thu, 12 Nov 2009 08:25:56 -0600
Subject: [Python-Dev] PyPI comments and ratings, *really*?
In-Reply-To: <4222a8490911120606t3e138957kcdb21104261a1ead@mail.gmail.com>
References: <D7E23A6D-C91C-46D8-A8FE-1EB16F3B2101@lericson.se>
	<200911130038.05516.steve@pearwood.info>
	<4222a8490911120606t3e138957kcdb21104261a1ead@mail.gmail.com>
Message-ID: <B10AA20C-1C2F-44F0-B8B5-64F26691CC83@python.org>

On Nov 12, 2009, at 8:06 AM, Jesse Noller wrote:

> Frankly, I agree with him. As implemented, I *and others* think this
> is broken. I've taken the stance of not publishing things to PyPi
> until A> I find the time to contribute to make it better or B> It
> changes.

That's distressing.  For better or worse PyPI is the central  
repository of 3rd party packages.  It should be easy, desirable, fun  
and socially encouraged to get your packages there.

I personally think a ratings system can be useful, but you should be  
able to opt-out of it if you want.  Or just write such awesome  
software that the bogus bad reviews will be buried by an avalanche of  
kudos.

-Barry

-------------- next part --------------
A non-text attachment was scrubbed...
Name: PGP.sig
Type: application/pgp-signature
Size: 194 bytes
Desc: This is a digitally signed message part
URL: <http://mail.python.org/pipermail/python-dev/attachments/20091112/2397c01f/attachment.pgp>

From regebro at gmail.com  Thu Nov 12 15:28:38 2009
From: regebro at gmail.com (Lennart Regebro)
Date: Thu, 12 Nov 2009 15:28:38 +0100
Subject: [Python-Dev] standard libraries don't behave like standard
	'libraries'
In-Reply-To: <c910262d0911120254o1a900338u50b442701e9d7c75@mail.gmail.com>
References: <c910262d0911120123i7c7e9dla918fff50cb5aecd@mail.gmail.com>
	<4AFBE02F.6020302@voidspace.org.uk>
	<c910262d0911120254o1a900338u50b442701e9d7c75@mail.gmail.com>
Message-ID: <319e029f0911120628o1ec4aed1qba81b29e190bcac3@mail.gmail.com>

2009/11/12 Sriram Srinivasan <naughtysriram at gmail.com>:
> In python we have the PyPI which is equivalent to the http://devpacks.org
> but in PyPI the packages are all user made applications.
> What I want is similar to PyPI but for the python standard libraries, so
> that they (libraries) are as add-on as possible.

They are. The mechanism for deciding if you want to use them is called
"importing".

Now of you don't want them installed, the easiest is to remove the
ones you don't want. This is useful on embedded systems, etc, but not
generally useful otehrwise.

-- 
Lennart Regebro: Python, Zope, Plone, Grok
http://regebro.wordpress.com/
+33 661 58 14 64

From amnorvend at gmail.com  Thu Nov 12 15:29:17 2009
From: amnorvend at gmail.com (Jason Baker)
Date: Thu, 12 Nov 2009 08:29:17 -0600
Subject: [Python-Dev] PyPI comments and ratings, *really*?
In-Reply-To: <4222a8490911120606t3e138957kcdb21104261a1ead@mail.gmail.com>
References: <D7E23A6D-C91C-46D8-A8FE-1EB16F3B2101@lericson.se> 
	<200911130038.05516.steve@pearwood.info>
	<4222a8490911120606t3e138957kcdb21104261a1ead@mail.gmail.com>
Message-ID: <816c4abb0911120629i479d05d2i5d331854ff90b55d@mail.gmail.com>

On Thu, Nov 12, 2009 at 8:06 AM, Jesse Noller <jnoller at gmail.com> wrote:
> On Thu, Nov 12, 2009 at 8:38 AM, Steven D'Aprano <steve at pearwood.info> wrote:
>> On Thu, 12 Nov 2009 08:44:32 pm Ludvig Ericson wrote:
>>> Why are there comments on PyPI? Moreso, why are there comments which
>>> I cannot control as a package author on my very own packages? That's
>>> just absurd.
>>
>> No, what's absurd is thinking that the act of publishing software
>> somehow gives you the right to demand control over what others say
>> about your software.
>>
>> I don't suppose that this rant of yours has something to do with the
>> comment posted today?
>
> Frankly, I agree with him. As implemented, I *and others* think this
> is broken. I've taken the stance of not publishing things to PyPi
> until A> I find the time to contribute to make it better or B> It
> changes.

I'm not sure I see the utility of ratings, but I think comments can be
useful as long as they don't carry over from release to release.  For
instance, suppose there's a bug in my package and someone leaves a
comment about it.  I don't want that comment still hanging around 3
years after I've already fixed the bug.

From jnoller at gmail.com  Thu Nov 12 15:32:11 2009
From: jnoller at gmail.com (Jesse Noller)
Date: Thu, 12 Nov 2009 09:32:11 -0500
Subject: [Python-Dev] PyPI comments and ratings, *really*?
In-Reply-To: <B10AA20C-1C2F-44F0-B8B5-64F26691CC83@python.org>
References: <D7E23A6D-C91C-46D8-A8FE-1EB16F3B2101@lericson.se>
	<200911130038.05516.steve@pearwood.info>
	<4222a8490911120606t3e138957kcdb21104261a1ead@mail.gmail.com>
	<B10AA20C-1C2F-44F0-B8B5-64F26691CC83@python.org>
Message-ID: <4222a8490911120632j43035c3ndd08292de5ada60a@mail.gmail.com>

On Thu, Nov 12, 2009 at 9:25 AM, Barry Warsaw <barry at python.org> wrote:
> On Nov 12, 2009, at 8:06 AM, Jesse Noller wrote:
>
>> Frankly, I agree with him. As implemented, I *and others* think this
>> is broken. I've taken the stance of not publishing things to PyPi
>> until A> I find the time to contribute to make it better or B> It
>> changes.
>
> That's distressing. ?For better or worse PyPI is the central repository of
> 3rd party packages. ?It should be easy, desirable, fun and socially
> encouraged to get your packages there.
>
> I personally think a ratings system can be useful, but you should be able to
> opt-out of it if you want. ?Or just write such awesome software that the
> bogus bad reviews will be buried by an avalanche of kudos.
>
> -Barry

I completely and totally agree with you. That's why it's a
self-imposed thing for me, I want to help, but don't have the time. In
the same breath, I don't want to support it as-is.

PyPI isn't a place to file bugs, complain something didn't work for
you if you didn't even have the common decency in some cases to email
them. Being unable, as an author, to remove, moderate, or even respond
to issues there bothers me quite a bit.

Heck, I would even be for requiring packages have a mailing list and /
or bug tracker to even upload, rather than comments. At least then the
authors or maintainers have a fighting chance.

From dalcinl at gmail.com  Thu Nov 12 16:01:19 2009
From: dalcinl at gmail.com (Lisandro Dalcin)
Date: Thu, 12 Nov 2009 13:01:19 -0200
Subject: [Python-Dev] PyPI comments and ratings, *really*?
In-Reply-To: <4222a8490911120632j43035c3ndd08292de5ada60a@mail.gmail.com>
References: <D7E23A6D-C91C-46D8-A8FE-1EB16F3B2101@lericson.se>
	<200911130038.05516.steve@pearwood.info>
	<4222a8490911120606t3e138957kcdb21104261a1ead@mail.gmail.com>
	<B10AA20C-1C2F-44F0-B8B5-64F26691CC83@python.org>
	<4222a8490911120632j43035c3ndd08292de5ada60a@mail.gmail.com>
Message-ID: <e7ba66e40911120701w415b478di7e85cacdac81ed53@mail.gmail.com>

On Thu, Nov 12, 2009 at 12:32 PM, Jesse Noller <jnoller at gmail.com> wrote:
> On Thu, Nov 12, 2009 at 9:25 AM, Barry Warsaw <barry at python.org> wrote:
>> On Nov 12, 2009, at 8:06 AM, Jesse Noller wrote:
>>
>>> Frankly, I agree with him. As implemented, I *and others* think this
>>> is broken. I've taken the stance of not publishing things to PyPi
>>> until A> I find the time to contribute to make it better or B> It
>>> changes.
>>
>> That's distressing. ?For better or worse PyPI is the central repository of
>> 3rd party packages. ?It should be easy, desirable, fun and socially
>> encouraged to get your packages there.
>>
>
> PyPI isn't a place to file bugs, complain something didn't work for
> you if you didn't even have the common decency in some cases to email
> them. Being unable, as an author, to remove, moderate, or even respond
> to issues there bothers me quite a bit.
>

I also agree with you. I do not see the point to make PyPI yet another
social network.



-- 
Lisandro Dalc?n
---------------
Centro Internacional de M?todos Computacionales en Ingenier?a (CIMEC)
Instituto de Desarrollo Tecnol?gico para la Industria Qu?mica (INTEC)
Consejo Nacional de Investigaciones Cient?ficas y T?cnicas (CONICET)
PTLC - G?emes 3450, (3000) Santa Fe, Argentina
Tel/Fax: +54-(0)342-451.1594

From exarkun at twistedmatrix.com  Thu Nov 12 16:23:01 2009
From: exarkun at twistedmatrix.com (exarkun at twistedmatrix.com)
Date: Thu, 12 Nov 2009 15:23:01 -0000
Subject: [Python-Dev] PyPI comments and ratings, *really*?
In-Reply-To: <e7ba66e40911120701w415b478di7e85cacdac81ed53@mail.gmail.com>
References: <D7E23A6D-C91C-46D8-A8FE-1EB16F3B2101@lericson.se>
	<200911130038.05516.steve@pearwood.info>
	<4222a8490911120606t3e138957kcdb21104261a1ead@mail.gmail.com>
	<B10AA20C-1C2F-44F0-B8B5-64F26691CC83@python.org>
	<4222a8490911120632j43035c3ndd08292de5ada60a@mail.gmail.com>
	<e7ba66e40911120701w415b478di7e85cacdac81ed53@mail.gmail.com>
Message-ID: <20091112152301.3229.485210589.divmod.xquotient.654@localhost.localdomain>

On 03:01 pm, dalcinl at gmail.com wrote:
>On Thu, Nov 12, 2009 at 12:32 PM, Jesse Noller <jnoller at gmail.com> 
>wrote:
>>On Thu, Nov 12, 2009 at 9:25 AM, Barry Warsaw <barry at python.org> 
>>wrote:
>>>On Nov 12, 2009, at 8:06 AM, Jesse Noller wrote:
>>>>Frankly, I agree with him. As implemented, I *and others* think this
>>>>is broken. I've taken the stance of not publishing things to PyPi
>>>>until A> I find the time to contribute to make it better or B> It
>>>>changes.
>>>
>>>That's distressing. ?For better or worse PyPI is the central 
>>>repository of
>>>3rd party packages. ?It should be easy, desirable, fun and socially
>>>encouraged to get your packages there.
>>
>>PyPI isn't a place to file bugs, complain something didn't work for
>>you if you didn't even have the common decency in some cases to email
>>them. Being unable, as an author, to remove, moderate, or even respond
>>to issues there bothers me quite a bit.
>
>I also agree with you. I do not see the point to make PyPI yet another
>social network.

+1

Jean-Paul

From olemis at gmail.com  Thu Nov 12 16:33:55 2009
From: olemis at gmail.com (Olemis Lang)
Date: Thu, 12 Nov 2009 10:33:55 -0500
Subject: [Python-Dev] PyPI comments and ratings, *really*?
In-Reply-To: <200911130038.05516.steve@pearwood.info>
References: <D7E23A6D-C91C-46D8-A8FE-1EB16F3B2101@lericson.se>
	<200911130038.05516.steve@pearwood.info>
Message-ID: <24ea26600911120733n58e22403hbed0afa289e3fd60@mail.gmail.com>

On Thu, Nov 12, 2009 at 8:38 AM, Steven D'Aprano <steve at pearwood.info> wrote:
> On Thu, 12 Nov 2009 08:44:32 pm Ludvig Ericson wrote:
>> Why are there comments on PyPI? Moreso, why are there comments which
>> I cannot control as a package author on my very own packages? That's
>> just absurd.
>
> No, what's absurd is thinking that the act of publishing software
> somehow gives you the right to demand control over what others say
> about your software.
>

Both of you are right , but I agree with Mr. Ludvig Ericson opinion (>
90%). The package author probably has no ?right to demand control over
what others say about her/his software? , but he has the right to
decide where such comments should be posted and also if he/she wants
to focus on (opinions | comments | ... ) or more useful feedback like
issues or support request .

For example, in my case, I prefer to have either custom ticket types
in the project's Trac environment or a plugin to receive this kind of
feedback *in the project's site* . IMHO PyPI is not the right place .
/me probably wrong

IMO -0.1 for votes and comments in PyPI and therefore

+1 for including settings to let coders decide (somehow ;o) whether to
allow this or not

-- 
Regards,

Olemis.

Blog ES: http://simelo-es.blogspot.com/
Blog EN: http://simelo-en.blogspot.com/

Featured article:

From olemis at gmail.com  Thu Nov 12 16:50:23 2009
From: olemis at gmail.com (Olemis Lang)
Date: Thu, 12 Nov 2009 10:50:23 -0500
Subject: [Python-Dev] PyPI comments and ratings, *really*?
In-Reply-To: <4222a8490911120632j43035c3ndd08292de5ada60a@mail.gmail.com>
References: <D7E23A6D-C91C-46D8-A8FE-1EB16F3B2101@lericson.se>
	<200911130038.05516.steve@pearwood.info>
	<4222a8490911120606t3e138957kcdb21104261a1ead@mail.gmail.com>
	<B10AA20C-1C2F-44F0-B8B5-64F26691CC83@python.org>
	<4222a8490911120632j43035c3ndd08292de5ada60a@mail.gmail.com>
Message-ID: <24ea26600911120750k427ffcbj4a10afd0ff288747@mail.gmail.com>

On Thu, Nov 12, 2009 at 9:32 AM, Jesse Noller <jnoller at gmail.com> wrote:
> On Thu, Nov 12, 2009 at 9:25 AM, Barry Warsaw <barry at python.org> wrote:
>> On Nov 12, 2009, at 8:06 AM, Jesse Noller wrote:
>>
>>> Frankly, I agree with him. As implemented, I *and others* think this
>>> is broken. I've taken the stance of not publishing things to PyPi
>>> until A> I find the time to contribute to make it better or B> It
>>> changes.
>>
>> That's distressing. ?For better or worse PyPI is the central repository of
>> 3rd party packages. ?It should be easy, desirable, fun and socially
>> encouraged to get your packages there.
>>
>> I personally think a ratings system can be useful, but you should be able to
>> opt-out of it if you want. ?Or just write such awesome software that the
>> bogus bad reviews will be buried by an avalanche of kudos.
>>
>> -Barry
>
> I completely and totally agree with you. That's why it's a
> self-imposed thing for me, I want to help, but don't have the time. In
> the same breath, I don't want to support it as-is.
>
> PyPI isn't a place to file bugs, complain something didn't work for
> you if you didn't even have the common decency in some cases to email
> them. Being unable, as an author, to remove, moderate, or even respond
> to issues there bothers me quite a bit.
>

+1

> Heck, I would even be for requiring packages have a mailing list and /
> or bug tracker to even upload, rather than comments. At least then the
> authors or maintainers have a fighting chance.
>

Intention = suggestion + crazy idea => for a better PyPI

But there's probably a chance to display what people said in the
project's site . If PyPI would be able to retrieve that information
from the project's site (e.g. that'd be possible for Trac and other
PMS via RPC ) and also some of the aforementioned (Jesse's) issues
might be solved with added benefits (data being cached and discarded
from time to time, better performance, less DB space, ...) or not .

IMO, what's missing in my reasoning is whether there is a common std
API for e.g. issues . But there's a popular API for wikis (i.e.
WikiRPC) so probably there's something std (I repeat, that I don't
know :-/ ) out there . At least Trac's TicketRPC is very simple (i.e.
two simple methods) and extensible (e.g. custom ticket fields ;o)

PS: I don't really know the exact details of the impl of votes and
comments in PyPI.

-- 
Regards,

Olemis.

Blog ES: http://simelo-es.blogspot.com/
Blog EN: http://simelo-en.blogspot.com/

Featured article:

From guido at python.org  Thu Nov 12 17:02:46 2009
From: guido at python.org (Guido van Rossum)
Date: Thu, 12 Nov 2009 08:02:46 -0800
Subject: [Python-Dev] PyPI comments and ratings, *really*?
In-Reply-To: <200911130038.05516.steve@pearwood.info>
References: <D7E23A6D-C91C-46D8-A8FE-1EB16F3B2101@lericson.se> 
	<200911130038.05516.steve@pearwood.info>
Message-ID: <ca471dc20911120802r4c765f78h669e56335a38f11c@mail.gmail.com>

On Thu, Nov 12, 2009 at 5:38 AM, Steven D'Aprano <steve at pearwood.info> wrote:
> On Thu, 12 Nov 2009 08:44:32 pm Ludvig Ericson wrote:
>> Why are there comments on PyPI? Moreso, why are there comments which
>> I cannot control as a package author on my very own packages? That's
>> just absurd.
>
> No, what's absurd is thinking that the act of publishing software
> somehow gives you the right to demand control over what others say
> about your software.
>
> I don't suppose that this rant of yours has something to do with the
> comment posted today?
>
> http://pypi.python.org/pypi/spypam/1.0

If you were to ask me, the people arguing against ratings and user
comments are fighting a losing battle. If they had an iPhone or
Android phone (or some other device with an "app store" kind of place
to find downloads) they'd know the value (for prospective downloaders)
of ratings and comments. Now, I think PyPI can use some (perhaps a lot
of) improvement in the details of how it works, e.g. there should be a
way to flag inappropriate messages (and users who post many
inappropriate messages) and the software author should be able to talk
back, but the general idea is here and won't go away by wishing it
away.

-- 
--Guido van Rossum (python.org/~guido)

From solipsis at pitrou.net  Thu Nov 12 17:19:09 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Thu, 12 Nov 2009 16:19:09 +0000 (UTC)
Subject: [Python-Dev] PyPI comments and ratings, *really*?
References: <D7E23A6D-C91C-46D8-A8FE-1EB16F3B2101@lericson.se>
	<200911130038.05516.steve@pearwood.info>
	<ca471dc20911120802r4c765f78h669e56335a38f11c@mail.gmail.com>
Message-ID: <loom.20091112T171636-892@post.gmane.org>

Guido van Rossum <guido <at> python.org> writes:
> 
> If you were to ask me, the people arguing against ratings and user
> comments are fighting a losing battle. If they had an iPhone or
> Android phone (or some other device with an "app store" kind of place
> to find downloads) they'd know the value (for prospective downloaders)
> of ratings and comments. Now, I think PyPI can use some (perhaps a lot
> of) improvement in the details of how it works, e.g. there should be a
> way to flag inappropriate messages (and users who post many
> inappropriate messages) and the software author should be able to talk
> back, but the general idea is here and won't go away by wishing it
> away.

Perhaps the suggestions and appreciations about the PyPI ratings and comment
system can go to http://pypi.python.org/pypi/pypi/ :-)

(more seriously, the problem with a comment system is that once it takes off,
you need a whole array of functionalities to maintain a good S/N ratio. Just
allowing people to "comment" without any sort of moderation, filtering or
community building doesn't work)

Regards

Antoine.



From pluskid at gmail.com  Thu Nov 12 17:27:07 2009
From: pluskid at gmail.com (Zhang Chiyuan)
Date: Fri, 13 Nov 2009 00:27:07 +0800
Subject: [Python-Dev] Is this a bug of the HTMLParser?
In-Reply-To: <4AFAE522.2090403@voidspace.org.uk>
References: <b6f0e9dd0911110816n16258c03se8d6bb6da745e2ed@mail.gmail.com> 
	<4AFAE522.2090403@voidspace.org.uk>
Message-ID: <b6f0e9dd0911120827q6bcd9f52lf89a3f53ce238438@mail.gmail.com>

filed: http://bugs.python.org/issue7311

On Thu, Nov 12, 2009 at 12:24 AM, Michael Foord
<fuzzyman at voidspace.org.uk>wrote:

> Hello Zhang Chiyuan,
>
> Can you file a bug on the Python issue tracker please:
>
>   http://bugs.python.org
>
> Thanks
>
> Michael Foord
>
> Zhang Chiyuan wrote:
>
>> Hi all,
>>
>> I'm using BeautifulSoup to parsing an HTML page and find it refused to
>> parse the page. By looking at the backtrace, I found it is a problem
>> with the python built-in HTMLParser.py. In fact, the web page I'm
>> parsing is with some Chinese characters. there is a tag like <img
>> src=/foo/bar.png alt=??> , note this is legacy html page where the
>> attributes are not quoted. However, the regexp defined in
>> HTMLParser.py is :
>>
>>  attrfind = re.compile(
>>     r'\s*([a-zA-Z_][-.:a-zA-Z_0-9]*)(\s*=\s*'
>>     r'(\'[^\']*\'|"[^"]*"|[-a-zA-Z0-9./,:;+*%?!&$\(\)_#=~@]*))?')
>>
>> Note that the Chinese character (also any other non-english
>> characters), so it fire an error parsing this. I'm not sure whether
>> the HTML standard allow un-quoted non-ASCII characters in the
>> attributes. If it allows, this seems to be a bug. and the regexp to
>> better be [^>\s] IMHO.
>>
>> BTW: It seems something like :
>>
>> <script>
>> var st = "<a></";
>> </script>
>>
>> can not be parsed. :-/
>>
>> --
>> pluskid
>> http://blog.pluskid.org
>> _______________________________________________
>> Python-Dev mailing list
>> Python-Dev at python.org
>> http://mail.python.org/mailman/listinfo/python-dev
>> Unsubscribe:
>> http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk
>>
>>
>
>
> --
> http://www.ironpythoninaction.com/
>
>


-- 
pluskid
http://blog.pluskid.org
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20091113/edf08e69/attachment.htm>

From jnoller at gmail.com  Thu Nov 12 17:31:29 2009
From: jnoller at gmail.com (Jesse Noller)
Date: Thu, 12 Nov 2009 11:31:29 -0500
Subject: [Python-Dev] PyPI comments and ratings, *really*?
In-Reply-To: <ca471dc20911120802r4c765f78h669e56335a38f11c@mail.gmail.com>
References: <D7E23A6D-C91C-46D8-A8FE-1EB16F3B2101@lericson.se>
	<200911130038.05516.steve@pearwood.info>
	<ca471dc20911120802r4c765f78h669e56335a38f11c@mail.gmail.com>
Message-ID: <4222a8490911120831y5233e547pbfd22de60bcf436e@mail.gmail.com>

On Thu, Nov 12, 2009 at 11:02 AM, Guido van Rossum <guido at python.org> wrote:

> If you were to ask me, the people arguing against ratings and user
> comments are fighting a losing battle. If they had an iPhone or
> Android phone (or some other device with an "app store" kind of place
> to find downloads) they'd know the value (for prospective downloaders)
> of ratings and comments. Now, I think PyPI can use some (perhaps a lot
> of) improvement in the details of how it works, e.g. there should be a
> way to flag inappropriate messages (and users who post many
> inappropriate messages) and the software author should be able to talk
> back, but the general idea is here and won't go away by wishing it
> away.
>
> --
> --Guido van Rossum (python.org/~guido)


I don't want us to impersonate the mindless, sub-useful drivel found
in App store/YouTube/etc comments 99% of the time or the broken "5
star ratings" systems, etc. It's too easy to game.

I'm not arguing something like this *shouldn't* exist; but that the
current implementation is a far cry from something actually *good and
useful*. If we want forums, let's put in forums. If we want a real
review system, then do that.

But before we even did those; why not have mandatory links for entries
to bug trackers, mailing lists, source repositories, etc? I'm saying
saying this doesn't seem well thought out, and the current
implementation is broken by design. Of course, as I said earlier;
since I don't have time to patch it; I'll simply just not participate.

jesse

From amnorvend at gmail.com  Thu Nov 12 17:59:27 2009
From: amnorvend at gmail.com (Jason Baker)
Date: Thu, 12 Nov 2009 10:59:27 -0600
Subject: [Python-Dev] PyPI comments and ratings, *really*?
In-Reply-To: <loom.20091112T171636-892@post.gmane.org>
References: <D7E23A6D-C91C-46D8-A8FE-1EB16F3B2101@lericson.se> 
	<200911130038.05516.steve@pearwood.info>
	<ca471dc20911120802r4c765f78h669e56335a38f11c@mail.gmail.com> 
	<loom.20091112T171636-892@post.gmane.org>
Message-ID: <816c4abb0911120859x482fdfe8r53715730e6f05ec5@mail.gmail.com>

On Thu, Nov 12, 2009 at 10:19 AM, Antoine Pitrou <solipsis at pitrou.net> wrote:
> (more seriously, the problem with a comment system is that once it takes off,
> you need a whole array of functionalities to maintain a good S/N ratio. Just
> allowing people to "comment" without any sort of moderation, filtering or
> community building doesn't work)

Why not allow ratings on comments as well?

From dstanek at dstanek.com  Thu Nov 12 18:02:59 2009
From: dstanek at dstanek.com (David Stanek)
Date: Thu, 12 Nov 2009 12:02:59 -0500
Subject: [Python-Dev] PyPI comments and ratings, *really*?
In-Reply-To: <4222a8490911120606t3e138957kcdb21104261a1ead@mail.gmail.com>
References: <D7E23A6D-C91C-46D8-A8FE-1EB16F3B2101@lericson.se>
	<200911130038.05516.steve@pearwood.info>
	<4222a8490911120606t3e138957kcdb21104261a1ead@mail.gmail.com>
Message-ID: <de32cc030911120902i1a1a9b74sf948683364def926@mail.gmail.com>

On Thu, Nov 12, 2009 at 9:06 AM, Jesse Noller <jnoller at gmail.com> wrote:

> On Thu, Nov 12, 2009 at 8:38 AM, Steven D'Aprano <steve at pearwood.info>
> wrote:
> > On Thu, 12 Nov 2009 08:44:32 pm Ludvig Ericson wrote:
> >> Why are there comments on PyPI? Moreso, why are there comments which
> >> I cannot control as a package author on my very own packages? That's
> >> just absurd.
> >
> > No, what's absurd is thinking that the act of publishing software
> > somehow gives you the right to demand control over what others say
> > about your software.
> >
> > I don't suppose that this rant of yours has something to do with the
> > comment posted today?
>
> Frankly, I agree with him. As implemented, I *and others* think this
> is broken. I've taken the stance of not publishing things to PyPi
> until A> I find the time to contribute to make it better or B> It
> changes.
>

Where is the code for PyPi? I took a quick look and didn't turn up anything.

-- 
David
blog: http://www.traceback.org
twitter: http://twitter.com/dstanek
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20091112/d00decf9/attachment.htm>

From jacob at jacobian.org  Thu Nov 12 18:06:36 2009
From: jacob at jacobian.org (Jacob Kaplan-Moss)
Date: Thu, 12 Nov 2009 11:06:36 -0600
Subject: [Python-Dev] PyPI comments and ratings, *really*?
In-Reply-To: <de32cc030911120902i1a1a9b74sf948683364def926@mail.gmail.com>
References: <D7E23A6D-C91C-46D8-A8FE-1EB16F3B2101@lericson.se> 
	<200911130038.05516.steve@pearwood.info>
	<4222a8490911120606t3e138957kcdb21104261a1ead@mail.gmail.com> 
	<de32cc030911120902i1a1a9b74sf948683364def926@mail.gmail.com>
Message-ID: <bc8ec34e0911120906r6fe98947sf8b6d915acdf7f34@mail.gmail.com>

On Thu, Nov 12, 2009 at 11:02 AM, David Stanek <dstanek at dstanek.com> wrote:
> Where is the code for PyPi? I took a quick look and didn't turn up anything.

https://svn.python.org/packages/trunk/pypi/

I've already started on a patch to make comments an option that
package maintainers could turn on or off, but I don't want to waste
any more time fighting this code unless I have some assurance it'll be
checked in.

Jacob

From masklinn at masklinn.net  Thu Nov 12 18:08:27 2009
From: masklinn at masklinn.net (Masklinn)
Date: Thu, 12 Nov 2009 18:08:27 +0100
Subject: [Python-Dev] PyPI comments and ratings, *really*?
In-Reply-To: <4222a8490911120831y5233e547pbfd22de60bcf436e@mail.gmail.com>
References: <D7E23A6D-C91C-46D8-A8FE-1EB16F3B2101@lericson.se>
	<200911130038.05516.steve@pearwood.info>
	<ca471dc20911120802r4c765f78h669e56335a38f11c@mail.gmail.com>
	<4222a8490911120831y5233e547pbfd22de60bcf436e@mail.gmail.com>
Message-ID: <7AF4743F-4602-4E4C-8B8F-2FBC34D3945F@masklinn.net>

On 12 Nov 2009, at 17:31 , Jesse Noller wrote:
> 
> But before we even did those; why not have mandatory links for entries
> to bug trackers, mailing lists, source repositories, etc? I'm saying
> saying this doesn't seem well thought out, and the current
> implementation is broken by design. Of course, as I said earlier;
> since I don't have time to patch it; I'll simply just not participate.
I think having links to those is a very good idea and more important
than a comment/notation system. They shouldn't be mandatory though, not
every library has a mailing list, or even a (public anyway) bug tracker.

Giving users and easy way to contact the author would be a must as well
("package index owner" and "package index maintainer" should link to
the users's profiles, and users should be able to setup contact
informations e.g. a mail address or a website). See how search.cpan
does it, the information is worth a lot to users. Currently, finding
how to send feedback to a package owner usually requires wading through
the description text itself, which in some cases (e.g. distribute)
amounts to pages of document.

On 12 Nov 2009, at 17:31 , Jesse Noller wrote:
> I don't want us to impersonate the mindless, sub-useful drivel found
> in App store/YouTube/etc comments 99% of the time or the broken "5
> star ratings" systems, etc. It's too easy to game.
I think I read a paper on that subject a short time ago (but can't
seem to find it right now), basically saying 5-star systems weren't
very useful, a simple +1/0/-1 was just as good (if not better) and
0/+1 worked as well.

From dstanek at dstanek.com  Thu Nov 12 18:14:11 2009
From: dstanek at dstanek.com (David Stanek)
Date: Thu, 12 Nov 2009 12:14:11 -0500
Subject: [Python-Dev] PyPI comments and ratings, *really*?
In-Reply-To: <bc8ec34e0911120906r6fe98947sf8b6d915acdf7f34@mail.gmail.com>
References: <D7E23A6D-C91C-46D8-A8FE-1EB16F3B2101@lericson.se>
	<200911130038.05516.steve@pearwood.info>
	<4222a8490911120606t3e138957kcdb21104261a1ead@mail.gmail.com>
	<de32cc030911120902i1a1a9b74sf948683364def926@mail.gmail.com>
	<bc8ec34e0911120906r6fe98947sf8b6d915acdf7f34@mail.gmail.com>
Message-ID: <de32cc030911120914h2963e66dg3e8aefb2ff94249e@mail.gmail.com>

On Thu, Nov 12, 2009 at 12:06 PM, Jacob Kaplan-Moss <jacob at jacobian.org>wrote:

> On Thu, Nov 12, 2009 at 11:02 AM, David Stanek <dstanek at dstanek.com>
> wrote:
> > Where is the code for PyPi? I took a quick look and didn't turn up
> anything.
>
> https://svn.python.org/packages/trunk/pypi/
>
> I've already started on a patch to make comments an option that
> package maintainers could turn on or off, but I don't want to waste
> any more time fighting this code unless I have some assurance it'll be
> checked in.
>

Thanks. If I have some spare time I'm going take a look. Should I post
patches to the regular Python bug tracker?

-- 
David
blog: http://www.traceback.org
twitter: http://twitter.com/dstanek
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20091112/a680f2fc/attachment.htm>

From arcriley at gmail.com  Thu Nov 12 18:27:19 2009
From: arcriley at gmail.com (Arc Riley)
Date: Thu, 12 Nov 2009 12:27:19 -0500
Subject: [Python-Dev] PyPI comments and ratings, *really*?
In-Reply-To: <200911130038.05516.steve@pearwood.info>
References: <D7E23A6D-C91C-46D8-A8FE-1EB16F3B2101@lericson.se> 
	<200911130038.05516.steve@pearwood.info>
Message-ID: <bad82a80911120927vfadcabcia63920d56c2f6430@mail.gmail.com>

Nobody is claiming right to censor what people say about their software.

This is the Internet.  There are blogs.  Google and other search engines
find blogs quickly, and people who agree with the viewpoints expressed link
to them thus making the blog postings more visible.  There are countless
other social networks and outlets for people to flame and slander (or praise
and promote, in a much less common case) software.

It would be more useful to provide a PyPI mechanism to publish a link to
file bugs on the project's own website and leave project ratings the work of
other sites such as Ohloh.

On Thu, Nov 12, 2009 at 8:38 AM, Steven D'Aprano <steve at pearwood.info>wrote:

>
> No, what's absurd is thinking that the act of publishing software
> somehow gives you the right to demand control over what others say
> about your software.
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20091112/11159dae/attachment.htm>

From python at rcn.com  Thu Nov 12 18:32:15 2009
From: python at rcn.com (Raymond Hettinger)
Date: Thu, 12 Nov 2009 09:32:15 -0800
Subject: [Python-Dev] PyPI comments and ratings, *really*?
References: <D7E23A6D-C91C-46D8-A8FE-1EB16F3B2101@lericson.se>
	<200911130038.05516.steve@pearwood.info><4222a8490911120606t3e138957kcdb21104261a1ead@mail.gmail.com>
	<de32cc030911120902i1a1a9b74sf948683364def926@mail.gmail.com>
	<bc8ec34e0911120906r6fe98947sf8b6d915acdf7f34@mail.gmail.com>
Message-ID: <6B1D2192FBED4006B98E6BBEE952097E@RaymondLaptop1>


[Jacob Kaplan-Moss]
> I've already started on a patch to make comments an option that
> package maintainers could turn on or off, but I don't want to waste
> any more time fighting this code unless I have some assurance it'll be
> checked in.

I support your efforts.


Raymond

From amk at amk.ca  Thu Nov 12 18:40:10 2009
From: amk at amk.ca (A.M. Kuchling)
Date: Thu, 12 Nov 2009 12:40:10 -0500
Subject: [Python-Dev] FYI: LWN article on 2.x, 3.x, and the moratorium
Message-ID: <20091112174010.GA11309@amk-desktop.matrixgroup.net>

FYI: I've written an article for Linux Weekly News on the moratorium &
related issues.

The article is subscribers-only for a week, but here's a free link:

http://lwn.net/SubscriberLink/361266/ef88bdbed5369800/

If you find this sort of thing useful/interesting, please consider
subscribing to LWN.

--amk

From phd at phd.pp.ru  Thu Nov 12 19:03:33 2009
From: phd at phd.pp.ru (Oleg Broytman)
Date: Thu, 12 Nov 2009 21:03:33 +0300
Subject: [Python-Dev] FYI: LWN article on 2.x, 3.x, and the moratorium
In-Reply-To: <20091112174010.GA11309@amk-desktop.matrixgroup.net>
References: <20091112174010.GA11309@amk-desktop.matrixgroup.net>
Message-ID: <20091112180333.GD5080@phd.pp.ru>

On Thu, Nov 12, 2009 at 12:40:10PM -0500, A.M. Kuchling wrote:
> FYI: I've written an article for Linux Weekly News on the moratorium &
> related issues.
> 
> The article is subscribers-only for a week, but here's a free link:
> 
> http://lwn.net/SubscriberLink/361266/ef88bdbed5369800/
> 
> If you find this sort of thing useful/interesting, please consider
> subscribing to LWN.

   +1. I've found LWN the best magazine (among both printed and electronic
magazines) in its area, and despite the fact its articles are freed after a
week (and many articles are published free from the very beginning) I
subscribed to it to support the magazine ("Patronize" business model).
   They say their site is written in Python using Quixote as the web
framework; they promised to publish the code but didn't publish it yet -
the fact I consider the biggest drawback of the site.

Oleg.
-- 
     Oleg Broytman            http://phd.pp.ru/            phd at phd.pp.ru
           Programmers don't die, they just GOSUB without RETURN.

From tjreedy at udel.edu  Thu Nov 12 19:30:15 2009
From: tjreedy at udel.edu (Terry Reedy)
Date: Thu, 12 Nov 2009 13:30:15 -0500
Subject: [Python-Dev] PyPI comments and ratings, *really*?
In-Reply-To: <B10AA20C-1C2F-44F0-B8B5-64F26691CC83@python.org>
References: <D7E23A6D-C91C-46D8-A8FE-1EB16F3B2101@lericson.se>	<200911130038.05516.steve@pearwood.info>	<4222a8490911120606t3e138957kcdb21104261a1ead@mail.gmail.com>
	<B10AA20C-1C2F-44F0-B8B5-64F26691CC83@python.org>
Message-ID: <hdhk7l$tru$1@ger.gmane.org>

Barry Warsaw wrote:
> On Nov 12, 2009, at 8:06 AM, Jesse Noller wrote:
> 
>> Frankly, I agree with him. As implemented, I *and others* think this
>> is broken. I've taken the stance of not publishing things to PyPi
>> until A> I find the time to contribute to make it better or B> It
>> changes.
> 
> That's distressing.  For better or worse PyPI is the central repository 
> of 3rd party packages.  It should be easy, desirable, fun and socially 
> encouraged to get your packages there.

I think his point is that a new book announcement servive is different 
from a book review and rating service.  And that mixing the two is 
'socially discouraging'. I do not know what the answer is

> I personally think a ratings system can be useful, but you should be 
> able to opt-out of it if you want.  Or just write such awesome software 
> that the bogus bad reviews will be buried by an avalanche of kudos.


From guido at python.org  Thu Nov 12 19:54:24 2009
From: guido at python.org (Guido van Rossum)
Date: Thu, 12 Nov 2009 10:54:24 -0800
Subject: [Python-Dev] PyPI comments and ratings, *really*?
In-Reply-To: <hdhk7l$tru$1@ger.gmane.org>
References: <D7E23A6D-C91C-46D8-A8FE-1EB16F3B2101@lericson.se> 
	<200911130038.05516.steve@pearwood.info>
	<4222a8490911120606t3e138957kcdb21104261a1ead@mail.gmail.com> 
	<B10AA20C-1C2F-44F0-B8B5-64F26691CC83@python.org>
	<hdhk7l$tru$1@ger.gmane.org>
Message-ID: <ca471dc20911121054t76eb5bfbsbd6a9e47a9e71d8a@mail.gmail.com>

On Thu, Nov 12, 2009 at 10:30 AM, Terry Reedy <tjreedy at udel.edu> wrote:
> Barry Warsaw wrote:
>>
>> On Nov 12, 2009, at 8:06 AM, Jesse Noller wrote:
>>
>>> Frankly, I agree with him. As implemented, I *and others* think this
>>> is broken. I've taken the stance of not publishing things to PyPi
>>> until A> I find the time to contribute to make it better or B> It
>>> changes.
>>
>> That's distressing. ?For better or worse PyPI is the central repository of
>> 3rd party packages. ?It should be easy, desirable, fun and socially
>> encouraged to get your packages there.
>
> I think his point is that a new book announcement servive is different from
> a book review and rating service. ?And that mixing the two is 'socially
> discouraging'. I do not know what the answer is

I would say that publishers disagree -- they seem to really like
adding "social" stuff to their book announcement service. See e.g.
Amazon (which combines all functions: announcement/promotion,
ordering/download, review/comments/rate/popularity).

I agree that creating a good social app is not easy, and if we can't
improve the social app embedded in PyPI quickly enough, we should at
least give authors the option to disable comments. Of course, as a
user, I might not trust a module that has no reviews or ratings.

-- 
--Guido van Rossum (python.org/~guido)

From olemis at gmail.com  Thu Nov 12 20:21:57 2009
From: olemis at gmail.com (Olemis Lang)
Date: Thu, 12 Nov 2009 14:21:57 -0500
Subject: [Python-Dev] PyPI comments and ratings, *really*?
In-Reply-To: <ca471dc20911121054t76eb5bfbsbd6a9e47a9e71d8a@mail.gmail.com>
References: <D7E23A6D-C91C-46D8-A8FE-1EB16F3B2101@lericson.se>
	<200911130038.05516.steve@pearwood.info>
	<4222a8490911120606t3e138957kcdb21104261a1ead@mail.gmail.com>
	<B10AA20C-1C2F-44F0-B8B5-64F26691CC83@python.org>
	<hdhk7l$tru$1@ger.gmane.org>
	<ca471dc20911121054t76eb5bfbsbd6a9e47a9e71d8a@mail.gmail.com>
Message-ID: <24ea26600911121121w122c1ea3lfc2a5fc0e6fd059a@mail.gmail.com>

Intention = precision => for a better PyPI

On Thu, Nov 12, 2009 at 1:54 PM, Guido van Rossum <guido at python.org> wrote:
> On Thu, Nov 12, 2009 at 10:30 AM, Terry Reedy <tjreedy at udel.edu> wrote:
>> Barry Warsaw wrote:
>>>
>>> On Nov 12, 2009, at 8:06 AM, Jesse Noller wrote:
>>>
>>>> Frankly, I agree with him. As implemented, I *and others* think this
>>>> is broken. I've taken the stance of not publishing things to PyPi
>>>> until A> I find the time to contribute to make it better or B> It
>>>> changes.
>>>
>>> That's distressing. ?For better or worse PyPI is the central repository of
>>> 3rd party packages. ?It should be easy, desirable, fun and socially
>>> encouraged to get your packages there.
>>
>> I think his point is that a new book announcement servive is different from
>> a book review and rating service. ?And that mixing the two is 'socially
>> discouraging'. I do not know what the answer is
>
> I would say that publishers disagree -- they seem to really like
> adding "social" stuff to their book announcement service. See e.g.
> Amazon (which combines all functions: announcement/promotion,
> ordering/download, review/comments/rate/popularity).
>

... but (most) book writers don't use an issue tracker to manage and
get *useful* feedback from their readers (I know there are exceptions
to the rule ;o) and fix the book chapters or anything else . Besides
there are some differences between software and books and the way both
of them are created, used and enhanced . What I don't like (today)
about comments + votes is that I have to do the same thing in two
different places (especially because one of the sources is *very*
noisy). If there's a way to integrate both and ?reduce? the noise ,
that would be nice .

;o)

> I agree that creating a good social app is not easy, and if we can't
> improve the social app embedded in PyPI quickly enough, we should at
> least give authors the option to disable comments.

+1

> Of course, as a
> user, I might not trust a module that has no reviews or ratings.
>

Not really sure. For example, if a user access the page for setuptools
(just an example ;o) soon she/he will realize that other people use it
very often and also has a high kwalitee score, therefore it is quite
unlikely that such package be ?irrelevant? or ?untrusted? (this is
IMHO) .

-- 
Regards,

Olemis.

Blog ES: http://simelo-es.blogspot.com/
Blog EN: http://simelo-en.blogspot.com/

Featured article:

From jnoller at gmail.com  Thu Nov 12 20:25:42 2009
From: jnoller at gmail.com (Jesse Noller)
Date: Thu, 12 Nov 2009 14:25:42 -0500
Subject: [Python-Dev] PyPI comments and ratings, *really*?
In-Reply-To: <ca471dc20911121054t76eb5bfbsbd6a9e47a9e71d8a@mail.gmail.com>
References: <D7E23A6D-C91C-46D8-A8FE-1EB16F3B2101@lericson.se>
	<200911130038.05516.steve@pearwood.info>
	<4222a8490911120606t3e138957kcdb21104261a1ead@mail.gmail.com>
	<B10AA20C-1C2F-44F0-B8B5-64F26691CC83@python.org>
	<hdhk7l$tru$1@ger.gmane.org>
	<ca471dc20911121054t76eb5bfbsbd6a9e47a9e71d8a@mail.gmail.com>
Message-ID: <4222a8490911121125w67ffacfct9989db242d7c00f5@mail.gmail.com>

On Thu, Nov 12, 2009 at 1:54 PM, Guido van Rossum <guido at python.org> wrote:

>
> I would say that publishers disagree -- they seem to really like
> adding "social" stuff to their book announcement service. See e.g.
> Amazon (which combines all functions: announcement/promotion,
> ordering/download, review/comments/rate/popularity).
>
> I agree that creating a good social app is not easy, and if we can't
> improve the social app embedded in PyPI quickly enough, we should at
> least give authors the option to disable comments. Of course, as a
> user, I might not trust a module that has no reviews or ratings.
>
> --
> --Guido van Rossum (python.org/~guido)


I'd not trust a package without a bug tracker, mailing list or link to
the source a lot sooner than something without comments and ratings.
Especially with ratings like milk and wolf shirts get:

http://www.amazon.com/Tuscan-Whole-Milk-Gallon-128/dp/B00032G1S0/ref=sr_1_13?ie=UTF8&s=grocery&qid=1258053581&sr=1-13

http://www.amazon.com/Mountain-Mens-Short-Sleeve-Large/dp/B001VMZFPQ/ref=sr_1_1?ie=UTF8&s=apparel&qid=1258053663&sr=8-1

What about astroturfing? What's to stop me from writing a script to
create a pile of accounts and then bumping packages I like with
glowing ratings and reviews? Who is going to be the moderator, and how
to decide between spam, incorrect comment, etc?

From guido at python.org  Thu Nov 12 20:30:27 2009
From: guido at python.org (Guido van Rossum)
Date: Thu, 12 Nov 2009 11:30:27 -0800
Subject: [Python-Dev] PyPI comments and ratings, *really*?
In-Reply-To: <4222a8490911121125w67ffacfct9989db242d7c00f5@mail.gmail.com>
References: <D7E23A6D-C91C-46D8-A8FE-1EB16F3B2101@lericson.se> 
	<200911130038.05516.steve@pearwood.info>
	<4222a8490911120606t3e138957kcdb21104261a1ead@mail.gmail.com> 
	<B10AA20C-1C2F-44F0-B8B5-64F26691CC83@python.org>
	<hdhk7l$tru$1@ger.gmane.org> 
	<ca471dc20911121054t76eb5bfbsbd6a9e47a9e71d8a@mail.gmail.com> 
	<4222a8490911121125w67ffacfct9989db242d7c00f5@mail.gmail.com>
Message-ID: <ca471dc20911121130r35c47256q75b284bf36313ab7@mail.gmail.com>

On Thu, Nov 12, 2009 at 11:25 AM, Jesse Noller <jnoller at gmail.com> wrote:
> I'd not trust a package without a bug tracker, mailing list or link to
> the source a lot sooner than something without comments and ratings.

Yeah, but you're not exactly an average user. Most users don't know
how to use a bug tracker. Also, there's a large variety of packages on
PyPI. Not every developer has the same attitude, but they all live
happily together on PyPI. (Or did you want someone to start a separate
CPAN "for the rest of them" ? :-)

[...]
> What about astroturfing? What's to stop me from writing a script to
> create a pile of accounts and then bumping packages I like with
> glowing ratings and reviews? Who is going to be the moderator, and how
> to decide between spam, incorrect comment, etc?

Those are all hard problems, though all of them have at least partial
solutions in the other worlds (Amazon, Wikipedia, Apple app store,
etc.). Maybe there should be a standard "social app" that you can just
customize for a specific purpose. Sounds like an interesting project,
actually.

-- 
--Guido van Rossum (python.org/~guido)

From jnoller at gmail.com  Thu Nov 12 20:38:21 2009
From: jnoller at gmail.com (Jesse Noller)
Date: Thu, 12 Nov 2009 14:38:21 -0500
Subject: [Python-Dev] PyPI comments and ratings, *really*?
In-Reply-To: <ca471dc20911121130r35c47256q75b284bf36313ab7@mail.gmail.com>
References: <D7E23A6D-C91C-46D8-A8FE-1EB16F3B2101@lericson.se>
	<200911130038.05516.steve@pearwood.info>
	<4222a8490911120606t3e138957kcdb21104261a1ead@mail.gmail.com>
	<B10AA20C-1C2F-44F0-B8B5-64F26691CC83@python.org>
	<hdhk7l$tru$1@ger.gmane.org>
	<ca471dc20911121054t76eb5bfbsbd6a9e47a9e71d8a@mail.gmail.com>
	<4222a8490911121125w67ffacfct9989db242d7c00f5@mail.gmail.com>
	<ca471dc20911121130r35c47256q75b284bf36313ab7@mail.gmail.com>
Message-ID: <4222a8490911121138s2b7a26f5w6f68a03a9067c927@mail.gmail.com>

On Thu, Nov 12, 2009 at 2:30 PM, Guido van Rossum <guido at python.org> wrote:
> On Thu, Nov 12, 2009 at 11:25 AM, Jesse Noller <jnoller at gmail.com> wrote:
>> I'd not trust a package without a bug tracker, mailing list or link to
>> the source a lot sooner than something without comments and ratings.
>
> Yeah, but you're not exactly an average user. Most users don't know
> how to use a bug tracker. Also, there's a large variety of packages on
> PyPI. Not every developer has the same attitude, but they all live
> happily together on PyPI. (Or did you want someone to start a separate
> CPAN "for the rest of them" ? :-)

True, but if you make entries for them mandatory (bug trackers,
source, etc), and you encourage users to use them, you begin being to
be the change you want to be, which is making PyPi *less* of an "app
store" where the consumer doesn't collaborate with the authors.

Or maybe rather than *putting* this stuff into Pypi; pypi allows
plugins to allow authors to link in RSS feeds to their bug trackers,
wiki streams, what have you.

I think everyone can co exist, just not one at the cost of another ;)

> [...]
>> What about astroturfing? What's to stop me from writing a script to
>> create a pile of accounts and then bumping packages I like with
>> glowing ratings and reviews? Who is going to be the moderator, and how
>> to decide between spam, incorrect comment, etc?
>
> Those are all hard problems, though all of them have at least partial
> solutions in the other worlds (Amazon, Wikipedia, Apple app store,
> etc.). Maybe there should be a standard "social app" that you can just
> customize for a specific purpose. Sounds like an interesting project,
> actually.

Yup, reddit's source is out there, and I think there's lots of
possibilities, I guess for me I'd rather have nothing than 50% of
something that doesn't account for the various problems and the pretty
valid opinions of authors and maintainers.

jesse

From martin at v.loewis.de  Thu Nov 12 20:38:40 2009
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Thu, 12 Nov 2009 20:38:40 +0100
Subject: [Python-Dev] standard libraries don't behave like
	standard	'libraries'
In-Reply-To: <c910262d0911120123i7c7e9dla918fff50cb5aecd@mail.gmail.com>
References: <c910262d0911120123i7c7e9dla918fff50cb5aecd@mail.gmail.com>
Message-ID: <4AFC6440.1040008@v.loewis.de>

> I am not an expert, I am just another python learner. These are just my
> views on the state of the standard libraries and to
> make them state-of-the-art..! ;)

If I understand correctly, you want the (current) standard library to be
separated from the Python implementation, and available separately.

Interestingly enough, people are very much split over whether that would
be a good thing or not. Some like it the way Python does, some dislike
it (and some quite strongly so).

In any case, many Python users consider it a good thing that it comes
"with batteries included", ie. with no need to add extra stuff for many
tasks.

Some of the Python maintainers have recently started objecting to this
setup, asking that the standard library should be split into separate
packages that are released and distributed independent of Python. Others
of us feel strongly that such a change should not be made.

So don't expect any immediate change to happen.

Regards,
Martin

From martin at v.loewis.de  Thu Nov 12 20:44:41 2009
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Thu, 12 Nov 2009 20:44:41 +0100
Subject: [Python-Dev] PyPI comments and ratings, *really*?
In-Reply-To: <ca471dc20911120802r4c765f78h669e56335a38f11c@mail.gmail.com>
References: <D7E23A6D-C91C-46D8-A8FE-1EB16F3B2101@lericson.se>
	<200911130038.05516.steve@pearwood.info>
	<ca471dc20911120802r4c765f78h669e56335a38f11c@mail.gmail.com>
Message-ID: <4AFC65A9.8060304@v.loewis.de>

> If you were to ask me, the people arguing against ratings and user
> comments are fighting a losing battle. If they had an iPhone or
> Android phone (or some other device with an "app store" kind of place
> to find downloads) they'd know the value (for prospective downloaders)
> of ratings and comments. Now, I think PyPI can use some (perhaps a lot
> of) improvement in the details of how it works, e.g. there should be a
> way to flag inappropriate messages (and users who post many
> inappropriate messages)

There is, although it's admittedly tedious: users can file a support
request (follow the link "Get Help").

> and the software author should be able to talk
> back

That is actually the case: the author *can* talk back.

> but the general idea is here and won't go away by wishing it
> away.

I'm going to take a poll RSN, and see what the majority of users
think (rather than their vocal fraction). Then we can see what to do
about it.

Regards,
Martin




From jacob at jacobian.org  Thu Nov 12 20:45:45 2009
From: jacob at jacobian.org (Jacob Kaplan-Moss)
Date: Thu, 12 Nov 2009 13:45:45 -0600
Subject: [Python-Dev] PyPI comments and ratings, *really*?
In-Reply-To: <ca471dc20911121130r35c47256q75b284bf36313ab7@mail.gmail.com>
References: <D7E23A6D-C91C-46D8-A8FE-1EB16F3B2101@lericson.se> 
	<200911130038.05516.steve@pearwood.info>
	<4222a8490911120606t3e138957kcdb21104261a1ead@mail.gmail.com> 
	<B10AA20C-1C2F-44F0-B8B5-64F26691CC83@python.org>
	<hdhk7l$tru$1@ger.gmane.org> 
	<ca471dc20911121054t76eb5bfbsbd6a9e47a9e71d8a@mail.gmail.com> 
	<4222a8490911121125w67ffacfct9989db242d7c00f5@mail.gmail.com> 
	<ca471dc20911121130r35c47256q75b284bf36313ab7@mail.gmail.com>
Message-ID: <bc8ec34e0911121145w795b081ft2e27168edd205ef@mail.gmail.com>

On Thu, Nov 12, 2009 at 1:30 PM, Guido van Rossum <guido at python.org> wrote:
> Yeah, but you're not exactly an average user. Most users don't know
> how to use a bug tracker.

But they do know how to use mailing lists. Or IRC chats. Or support forums.

Those places have (for many projects) tens, hundreds, or even
thousands of peers who are able and willing to help new users get
started. Only the package maintainers see comments on PyPI, meaning
we've got to deal with requests for support there manually.

This isn't academic; just this morning a user asked a question on
Django's PyPI listing that would have been better asked on any of the
support channels we provide. I have no way of directing him there
besides lamely commenting after the fact, and then it just seems like
I'm giving him the runaround.

Look, nobody's asking to kill the feature. We're asking to *make it
optional*, and to allow us to link to a more appropriate support forum
instead. Can you please explain to me what's wrong with that?

Jacob

From martin at v.loewis.de  Thu Nov 12 20:46:44 2009
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Thu, 12 Nov 2009 20:46:44 +0100
Subject: [Python-Dev] PyPI comments and ratings, *really*?
In-Reply-To: <loom.20091112T171636-892@post.gmane.org>
References: <D7E23A6D-C91C-46D8-A8FE-1EB16F3B2101@lericson.se>	<200911130038.05516.steve@pearwood.info>	<ca471dc20911120802r4c765f78h669e56335a38f11c@mail.gmail.com>
	<loom.20091112T171636-892@post.gmane.org>
Message-ID: <4AFC6624.2070802@v.loewis.de>

> (more seriously, the problem with a comment system is that once it takes off,
> you need a whole array of functionalities to maintain a good S/N ratio. Just
> allowing people to "comment" without any sort of moderation, filtering or
> community building doesn't work)

The current rate is roughly 1 comment per day (with peaks of 5
comments), so it takes of rather slowly.

Regards,
Martin

From g.brandl at gmx.net  Thu Nov 12 20:50:43 2009
From: g.brandl at gmx.net (Georg Brandl)
Date: Thu, 12 Nov 2009 20:50:43 +0100
Subject: [Python-Dev] standard libraries don't behave like standard
	'libraries'
In-Reply-To: <c910262d0911120123i7c7e9dla918fff50cb5aecd@mail.gmail.com>
References: <c910262d0911120123i7c7e9dla918fff50cb5aecd@mail.gmail.com>
Message-ID: <hdhoum$fh1$1@ger.gmane.org>

Sriram Srinivasan schrieb:
> I guess why every programming language has some kind of a 'standard
> library' built in within it. In my view it must not be called as a
> 'library' at all. what it does
> is like a 'bunch of built-in programs ready-made to do stuff'.
> 
> Lets see what a 'library' does:
> 
> 1. offers books for customers
>  1.1 lets user select a book by genre, etc
>  1.2 lets user to use different books of same genre, etc
>  1.3 lets user to use books by same author, etc for different genre
> 
> 2. keeps track of all the books + their genre
>  2.1 first knows what all books it has at present
>  2.2 when new book comes it is added to the particular shelf sorted by
> genre,author,edition, etc..
>  2.3 when books become old they are kept separately for future reference
>  2.4 very old books can be sent to a museum/discarded
>
> I guess no standard library does the minimum of this but wants to be
> called a library.

I guess you're simply stretching the "library" metaphor far beyond its
usefulness :)

Georg


From jnoller at gmail.com  Thu Nov 12 20:57:45 2009
From: jnoller at gmail.com (Jesse Noller)
Date: Thu, 12 Nov 2009 14:57:45 -0500
Subject: [Python-Dev] standard libraries don't behave like standard
	'libraries'
In-Reply-To: <4AFC6440.1040008@v.loewis.de>
References: <c910262d0911120123i7c7e9dla918fff50cb5aecd@mail.gmail.com>
	<4AFC6440.1040008@v.loewis.de>
Message-ID: <4222a8490911121157i622fbab8q5fe82d32fe9ea8f@mail.gmail.com>

On Thu, Nov 12, 2009 at 2:38 PM, "Martin v. L?wis" <martin at v.loewis.de> wrote:
>> I am not an expert, I am just another python learner. These are just my
>> views on the state of the standard libraries and to
>> make them state-of-the-art..! ;)
>
> If I understand correctly, you want the (current) standard library to be
> separated from the Python implementation, and available separately.
>
> Interestingly enough, people are very much split over whether that would
> be a good thing or not. Some like it the way Python does, some dislike
> it (and some quite strongly so).
>
> In any case, many Python users consider it a good thing that it comes
> "with batteries included", ie. with no need to add extra stuff for many
> tasks.
>
> Some of the Python maintainers have recently started objecting to this
> setup, asking that the standard library should be split into separate
> packages that are released and distributed independent of Python. Others
> of us feel strongly that such a change should not be made.
>
> So don't expect any immediate change to happen.
>
> Regards,
> Martin

Martin is correct; this came up on distutils-sig a month or so ago; I
proposed offering multiple downloads "with batteries" and "without
batteries (with the batteries on the side)". We decided as a group to
hold off on that until further in the future.

jesse

From guido at python.org  Thu Nov 12 21:01:08 2009
From: guido at python.org (Guido van Rossum)
Date: Thu, 12 Nov 2009 12:01:08 -0800
Subject: [Python-Dev] PyPI comments and ratings, *really*?
In-Reply-To: <4AFC65A9.8060304@v.loewis.de>
References: <D7E23A6D-C91C-46D8-A8FE-1EB16F3B2101@lericson.se> 
	<200911130038.05516.steve@pearwood.info>
	<ca471dc20911120802r4c765f78h669e56335a38f11c@mail.gmail.com> 
	<4AFC65A9.8060304@v.loewis.de>
Message-ID: <ca471dc20911121201k28230c2fo3b6a16345a6733fc@mail.gmail.com>

On Thu, Nov 12, 2009 at 11:44 AM, "Martin v. L?wis" <martin at v.loewis.de> wrote:
> I'm going to take a poll RSN, and see what the majority of users
> think (rather than their vocal fraction). Then we can see what to do
> about it.

Or (ironically) the vocal fraction can write scripts to stuff the ballot. :-)

-- 
--Guido van Rossum (python.org/~guido)

From guido at python.org  Thu Nov 12 21:04:01 2009
From: guido at python.org (Guido van Rossum)
Date: Thu, 12 Nov 2009 12:04:01 -0800
Subject: [Python-Dev] PyPI comments and ratings, *really*?
In-Reply-To: <bc8ec34e0911121145w795b081ft2e27168edd205ef@mail.gmail.com>
References: <D7E23A6D-C91C-46D8-A8FE-1EB16F3B2101@lericson.se> 
	<200911130038.05516.steve@pearwood.info>
	<4222a8490911120606t3e138957kcdb21104261a1ead@mail.gmail.com> 
	<B10AA20C-1C2F-44F0-B8B5-64F26691CC83@python.org>
	<hdhk7l$tru$1@ger.gmane.org> 
	<ca471dc20911121054t76eb5bfbsbd6a9e47a9e71d8a@mail.gmail.com> 
	<4222a8490911121125w67ffacfct9989db242d7c00f5@mail.gmail.com> 
	<ca471dc20911121130r35c47256q75b284bf36313ab7@mail.gmail.com> 
	<bc8ec34e0911121145w795b081ft2e27168edd205ef@mail.gmail.com>
Message-ID: <ca471dc20911121204p2fd8d643xf5f8c99f874f8e0d@mail.gmail.com>

On Thu, Nov 12, 2009 at 11:45 AM, Jacob Kaplan-Moss <jacob at jacobian.org> wrote:
> On Thu, Nov 12, 2009 at 1:30 PM, Guido van Rossum <guido at python.org> wrote:
>> Yeah, but you're not exactly an average user. Most users don't know
>> how to use a bug tracker.
>
> But they do know how to use mailing lists. Or IRC chats. Or support forums.
>
> Those places have (for many projects) tens, hundreds, or even
> thousands of peers who are able and willing to help new users get
> started. Only the package maintainers see comments on PyPI, meaning
> we've got to deal with requests for support there manually.
>
> This isn't academic; just this morning a user asked a question on
> Django's PyPI listing that would have been better asked on any of the
> support channels we provide. I have no way of directing him there
> besides lamely commenting after the fact, and then it just seems like
> I'm giving him the runaround.

Maybe that's an example of a user who doesn't know how to use those
support channels? I know I wouldn't bother with IRC even if it was the
only way to get in touch with users, I hate it with a vengeance.
(Though arguably I'm a special case -- whenever I show up everyone
goes "ooooh, Guido is here." :-) And I might not want to sign up for a
mailing list for a casual question. And what exactly is a "forum"?

> Look, nobody's asking to kill the feature. We're asking to *make it
> optional*, and to allow us to link to a more appropriate support forum
> instead. Can you please explain to me what's wrong with that?

I already said it was fine to make it opt-out. What more do you want?

-- 
--Guido van Rossum (python.org/~guido)

From dalcinl at gmail.com  Thu Nov 12 21:13:48 2009
From: dalcinl at gmail.com (Lisandro Dalcin)
Date: Thu, 12 Nov 2009 18:13:48 -0200
Subject: [Python-Dev] PyPI comments and ratings, *really*?
In-Reply-To: <4AFC6624.2070802@v.loewis.de>
References: <D7E23A6D-C91C-46D8-A8FE-1EB16F3B2101@lericson.se>
	<200911130038.05516.steve@pearwood.info>
	<ca471dc20911120802r4c765f78h669e56335a38f11c@mail.gmail.com>
	<loom.20091112T171636-892@post.gmane.org>
	<4AFC6624.2070802@v.loewis.de>
Message-ID: <e7ba66e40911121213j23d16841i1955743994426f2@mail.gmail.com>

On Thu, Nov 12, 2009 at 5:46 PM, "Martin v. L?wis" <martin at v.loewis.de> wrote:
>> (more seriously, the problem with a comment system is that once it takes off,
>> you need a whole array of functionalities to maintain a good S/N ratio. Just
>> allowing people to "comment" without any sort of moderation, filtering or
>> community building doesn't work)
>
> The current rate is roughly 1 comment per day (with peaks of 5
> comments), so it takes of rather slowly.
>

Until spammers decide to attack...


-- 
Lisandro Dalc?n
---------------
Centro Internacional de M?todos Computacionales en Ingenier?a (CIMEC)
Instituto de Desarrollo Tecnol?gico para la Industria Qu?mica (INTEC)
Consejo Nacional de Investigaciones Cient?ficas y T?cnicas (CONICET)
PTLC - G?emes 3450, (3000) Santa Fe, Argentina
Tel/Fax: +54-(0)342-451.1594

From martin at v.loewis.de  Thu Nov 12 21:19:33 2009
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Thu, 12 Nov 2009 21:19:33 +0100
Subject: [Python-Dev] PyPI comments and ratings, *really*?
In-Reply-To: <e7ba66e40911121213j23d16841i1955743994426f2@mail.gmail.com>
References: <D7E23A6D-C91C-46D8-A8FE-1EB16F3B2101@lericson.se>	
	<200911130038.05516.steve@pearwood.info>	
	<ca471dc20911120802r4c765f78h669e56335a38f11c@mail.gmail.com>	
	<loom.20091112T171636-892@post.gmane.org>	
	<4AFC6624.2070802@v.loewis.de>
	<e7ba66e40911121213j23d16841i1955743994426f2@mail.gmail.com>
Message-ID: <4AFC6DD5.7070702@v.loewis.de>

>> The current rate is roughly 1 comment per day (with peaks of 5
>> comments), so it takes of rather slowly.
>>
> 
> Until spammers decide to attack...

Sure. However, spambots have avoided PyPI so far, and manual spamming
only had one incident (of somebody creating dozens of packages on a
single day).

My understanding is that spambots target systems where a comment form
is present even in the unloggedin state. In PyPI, you need to create
the account before you can comment.

Regards,
Martin


From olemis at gmail.com  Thu Nov 12 21:30:17 2009
From: olemis at gmail.com (Olemis Lang)
Date: Thu, 12 Nov 2009 15:30:17 -0500
Subject: [Python-Dev] PyPI comments and ratings, *really*?
In-Reply-To: <4222a8490911121138s2b7a26f5w6f68a03a9067c927@mail.gmail.com>
References: <D7E23A6D-C91C-46D8-A8FE-1EB16F3B2101@lericson.se>
	<200911130038.05516.steve@pearwood.info>
	<4222a8490911120606t3e138957kcdb21104261a1ead@mail.gmail.com>
	<B10AA20C-1C2F-44F0-B8B5-64F26691CC83@python.org>
	<hdhk7l$tru$1@ger.gmane.org>
	<ca471dc20911121054t76eb5bfbsbd6a9e47a9e71d8a@mail.gmail.com>
	<4222a8490911121125w67ffacfct9989db242d7c00f5@mail.gmail.com>
	<ca471dc20911121130r35c47256q75b284bf36313ab7@mail.gmail.com>
	<4222a8490911121138s2b7a26f5w6f68a03a9067c927@mail.gmail.com>
Message-ID: <24ea26600911121230n536a50a5n56effad77fdba35e@mail.gmail.com>

Intention = personal opinion => for a better  PyPI

On Thu, Nov 12, 2009 at 2:38 PM, Jesse Noller <jnoller at gmail.com> wrote:
> On Thu, Nov 12, 2009 at 2:30 PM, Guido van Rossum <guido at python.org> wrote:
>> On Thu, Nov 12, 2009 at 11:25 AM, Jesse Noller <jnoller at gmail.com> wrote:
>>> I'd not trust a package without a bug tracker, mailing list or link to
>>> the source a lot sooner than something without comments and ratings.
>>
>> Yeah, but you're not exactly an average user. Most users don't know
>> how to use a bug tracker. Also, there's a large variety of packages on
>> PyPI. Not every developer has the same attitude, but they all live
>> happily together on PyPI. (Or did you want someone to start a separate
>> CPAN "for the rest of them" ? :-)
>
> True, but if you make entries for them mandatory (bug trackers,
> source, etc), and you encourage users to use them, you begin being to
> be the change you want to be, which is making PyPi *less* of an "app
> store" where the consumer doesn't collaborate with the authors.
>
> Or maybe rather than *putting* this stuff into Pypi; pypi allows
> plugins to allow authors to link in RSS feeds to their bug trackers,
> wiki streams, what have you.
>

IMO plugins could be a little bit complicated because PyPI would need
to be extended, and there's also the problem of installing, upgrading
and maintaining each plugin . OTOH if PyPI relies on a single API
based on open standards (e.g. RPC or something RESTful ;o) then that
would represent less overhead for PyPI maintainers .

Instead of votes + comments I'd prefer a similar user interface but
doing things as follows (feel free to filter things; besides I'll
mention how it should work using Trac XmlRpcPlugin , but should be
similar for other PMS ;o) :

  - Previous comments retrieved from third party site
    and shown (e.g. no more than 5 and only most recent shown like TH.org)
    as well as a link to navigate to third party site in order to look
for further
    issues . (ticket.getAll + ticket.get)
  - Text input would be the description of a ticket (ticket.create)
  - A combobox to select either comment | defect | support (ticket.create)
  - Ratings could be interpreted as a priority or severity ...
    (labels = ticket.priority.getAll + ticket.priority.getAll ,
     values for single item = ticket.get )

Implementing all this might require to add more information to the
index (I am not sure) and also config options in the site for package
maintainers, but since it'd be more useful (for me) probable (me and
others) will prefer something like that : *and users won't even notice
the changes*

;o)

Nonetheless plugins approach is more general and flexible, and it is
also possible to develop a plugin to support the RPC-based integration
with external issue trackers . The main difference is maintenance
effort once it's up and running .

;o)

> I think everyone can co exist, just not one at the cost of another ;)
>

+1 ... keeping relevant data in single place

-- 
Regards,

Olemis.

Blog ES: http://simelo-es.blogspot.com/
Blog EN: http://simelo-en.blogspot.com/

Featured article:

From amk at amk.ca  Thu Nov 12 21:30:57 2009
From: amk at amk.ca (A.M. Kuchling)
Date: Thu, 12 Nov 2009 15:30:57 -0500
Subject: [Python-Dev] PyPI comments and ratings, *really*?
In-Reply-To: <ca471dc20911121130r35c47256q75b284bf36313ab7@mail.gmail.com>
References: <D7E23A6D-C91C-46D8-A8FE-1EB16F3B2101@lericson.se>
	<200911130038.05516.steve@pearwood.info>
	<4222a8490911120606t3e138957kcdb21104261a1ead@mail.gmail.com>
	<B10AA20C-1C2F-44F0-B8B5-64F26691CC83@python.org>
	<hdhk7l$tru$1@ger.gmane.org>
	<ca471dc20911121054t76eb5bfbsbd6a9e47a9e71d8a@mail.gmail.com>
	<4222a8490911121125w67ffacfct9989db242d7c00f5@mail.gmail.com>
	<ca471dc20911121130r35c47256q75b284bf36313ab7@mail.gmail.com>
Message-ID: <20091112203057.GA14396@amk-desktop.matrixgroup.net>

On Thu, Nov 12, 2009 at 11:30:27AM -0800, Guido van Rossum wrote:
> etc.). Maybe there should be a standard "social app" that you can just
> customize for a specific purpose. Sounds like an interesting project,
> actually.

For comments, haloscan and disqus are third-party comment-hosting
services; http://redalt.com/blog/comment-services has a longer list.
I don't know if any of them support rating of the posts or objects
being commented on (as opposed to rating other comments, which is
supported by some of them).  Or if any of them can delegate moderation
to the module authors, as opposed to the PyPI admins.

PyPI's REST-style URLs also work nicely as keys or RDF identifiers, so
it would be straightforward to use them for identifying ratings or
comment threads.

--amk

From robertc at robertcollins.net  Thu Nov 12 21:34:20 2009
From: robertc at robertcollins.net (Robert Collins)
Date: Fri, 13 Nov 2009 07:34:20 +1100
Subject: [Python-Dev] PyPI comments and ratings, *really*?
In-Reply-To: <B10AA20C-1C2F-44F0-B8B5-64F26691CC83@python.org>
References: <D7E23A6D-C91C-46D8-A8FE-1EB16F3B2101@lericson.se>
	<200911130038.05516.steve@pearwood.info>
	<4222a8490911120606t3e138957kcdb21104261a1ead@mail.gmail.com>
	<B10AA20C-1C2F-44F0-B8B5-64F26691CC83@python.org>
Message-ID: <1258058060.25002.4.camel@lifeless-64>

On Thu, 2009-11-12 at 08:25 -0600, Barry Warsaw wrote:
> 
> 
> That's distressing.  For better or worse PyPI is the central  
> repository of 3rd party packages.  It should be easy, desirable, fun  
> and socially encouraged to get your packages there.

Its already socially encouraged: heck, if package foo is not on PyPI, it
doesn't exist.

As an author who hosts code elsewhere, I really want to get feedback on
my packages. I don't really see the point of having a comment system in
PyPI ((*for me*), but I would really like to be able to have a link to
the appropriate web page for folk that want to ask questions. To be
clear, I mean a specific link, not just the 'link to a website'.

-Rob
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 197 bytes
Desc: This is a digitally signed message part
URL: <http://mail.python.org/pipermail/python-dev/attachments/20091113/2adbf9cc/attachment.pgp>

From tjreedy at udel.edu  Thu Nov 12 21:42:37 2009
From: tjreedy at udel.edu (Terry Reedy)
Date: Thu, 12 Nov 2009 15:42:37 -0500
Subject: [Python-Dev] PyPI comments and ratings, *really*?
In-Reply-To: <ca471dc20911121054t76eb5bfbsbd6a9e47a9e71d8a@mail.gmail.com>
References: <D7E23A6D-C91C-46D8-A8FE-1EB16F3B2101@lericson.se>
	<200911130038.05516.steve@pearwood.info>	<4222a8490911120606t3e138957kcdb21104261a1ead@mail.gmail.com>
	<B10AA20C-1C2F-44F0-B8B5-64F26691CC83@python.org>	<hdhk7l$tru$1@ger.gmane.org>
	<ca471dc20911121054t76eb5bfbsbd6a9e47a9e71d8a@mail.gmail.com>
Message-ID: <hdhrvs$qs8$1@ger.gmane.org>

Guido van Rossum wrote:
> On Thu, Nov 12, 2009 at 10:30 AM, Terry Reedy <tjreedy at udel.edu> wrote:
>> Barry Warsaw wrote:
>>> On Nov 12, 2009, at 8:06 AM, Jesse Noller wrote:
>>>
>>>> Frankly, I agree with him. As implemented, I *and others* think this
>>>> is broken. I've taken the stance of not publishing things to PyPi
>>>> until A> I find the time to contribute to make it better or B> It
>>>> changes.
>>> That's distressing.  For better or worse PyPI is the central repository of
>>> 3rd party packages.  It should be easy, desirable, fun and socially
>>> encouraged to get your packages there.
>> I think his point is that a new book announcement servive is different from
>> a book review and rating service.  And that mixing the two is 'socially
>> discouraging'. I do not know what the answer is
> 
> I would say that publishers disagree -- they seem to really like
> adding "social" stuff to their book announcement service. See e.g.
> Amazon (which combines all functions: announcement/promotion,
> ordering/download, review/comments/rate/popularity).

I use user reviews on both Amazon and Netflix. I notice that both let 
people rate the reviews (helpful or not), and I sometimes look at those 
also. Both list items without the say-so of creators, though most will 
tolerate possible bad reviews in exchange for sales. I very seldom see 
an item with only one review, so there is usually a mix. There are also 
ratings averaged across a lot more people.

Part of the pypi problem is a startup problem of initially low numbers. 
If the only people who bother to log in to rate are the disgruntled, 
then the ratings/reviews will be biased. I wonder how many of the people 
promoting the new feature have themselves logged in to systematically 
rate and possibly comment on every package they have looked at, and 
thereby kickstart the system with fair responses.

Authors can often respond to magazine/journal reviews in Letters to the 
Editor. Publishers tend to exercise some editorial control over reviews 
so as to not make the publication look bad with grossly bad reviews. 
Even on Amazon, an author could, I presume, add a response to a 
factually incorrect review.

Terry Jan Reedy


From ben+python at benfinney.id.au  Thu Nov 12 21:51:06 2009
From: ben+python at benfinney.id.au (Ben Finney)
Date: Fri, 13 Nov 2009 07:51:06 +1100
Subject: [Python-Dev] PyPI comments and ratings, *really*?
References: <D7E23A6D-C91C-46D8-A8FE-1EB16F3B2101@lericson.se>
	<200911130038.05516.steve@pearwood.info>
	<4222a8490911120606t3e138957kcdb21104261a1ead@mail.gmail.com>
	<B10AA20C-1C2F-44F0-B8B5-64F26691CC83@python.org>
Message-ID: <87zl6rwjph.fsf@benfinney.id.au>

Barry Warsaw <barry at python.org> writes:

> On Nov 12, 2009, at 8:06 AM, Jesse Noller wrote:
>
> > Frankly, I agree with him. As implemented, I *and others* think this
> > is broken. I've taken the stance of not publishing things to PyPi
> > until A> I find the time to contribute to make it better or B> It
> > changes.

I'm updating my existing packages, but I have zero interest in
publishing new packages to PyPI while it's trying to be a popularity
rating and comment system.

> That's distressing. For better or worse PyPI is the central repository
> of 3rd party packages.

That's exactly what we want it to be. The problem is the current
implementation is diluting that focus and making it much less attractive
to have one's packages there.

-- 
 \          ?Rightful liberty is unobstructed action, according to our |
  `\        will, within limits drawn around us by the equal rights of |
_o__)                                       others.? ?Thomas Jefferson |
Ben Finney
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 197 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/python-dev/attachments/20091113/bce9dfab/attachment-0001.pgp>

From ncoghlan at gmail.com  Thu Nov 12 21:59:08 2009
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Fri, 13 Nov 2009 06:59:08 +1000
Subject: [Python-Dev] PyPI comments and ratings, *really*?
In-Reply-To: <B10AA20C-1C2F-44F0-B8B5-64F26691CC83@python.org>
References: <D7E23A6D-C91C-46D8-A8FE-1EB16F3B2101@lericson.se>	<200911130038.05516.steve@pearwood.info>	<4222a8490911120606t3e138957kcdb21104261a1ead@mail.gmail.com>
	<B10AA20C-1C2F-44F0-B8B5-64F26691CC83@python.org>
Message-ID: <4AFC771C.50608@gmail.com>

Barry Warsaw wrote:
> I personally think a ratings system can be useful, but you should be
> able to opt-out of it if you want.  Or just write such awesome software
> that the bogus bad reviews will be buried by an avalanche of kudos.

One of the problems I have with online rating/comment systems for
software are I see them as inherently biased. Happy users are more
likely to be busy coding, unhappy users are more likely to be frustrated
and looking for somewhere to vent.

If the package author can't even *respond* to mistaken or misguided
comments then the review system is fundamentally broken. Better not to
have one at all - let people vent on their own blogs and other sites,
and let potential users let Google do its thing (and if Google turns up
nothing, then that in itself is a data point).

Regards,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------

From chris at simplistix.co.uk  Thu Nov 12 22:01:16 2009
From: chris at simplistix.co.uk (Chris Withers)
Date: Thu, 12 Nov 2009 21:01:16 +0000
Subject: [Python-Dev] PyPI comments and ratings, *really*?
In-Reply-To: <4AFC65A9.8060304@v.loewis.de>
References: <D7E23A6D-C91C-46D8-A8FE-1EB16F3B2101@lericson.se>	<200911130038.05516.steve@pearwood.info>	<ca471dc20911120802r4c765f78h669e56335a38f11c@mail.gmail.com>
	<4AFC65A9.8060304@v.loewis.de>
Message-ID: <4AFC779C.6040608@simplistix.co.uk>

Martin v. L?wis wrote:
> I'm going to take a poll RSN, and see what the majority of users
> think (rather than their vocal fraction). Then we can see what to do
> about it.

Yes please! I've been silently waiting for this and have (surprisingly 
for me!) managed to resist joining in the rant.

I'm of the crowd who just wants to make comments (I don't care about 
ratings one way or another, and I can't, sadly, see anyone giving me 
access as to who-voted-less-than-5 so I can follow up with them and see 
what needs to be improved) optional. I'm quite okay with having a banner 
saying "This package has opted not to receive comments". I believe the 
software I release on PyPI is strong enough on its own merits that 
people will use it ;-)

Chris

-- 
Simplistix - Content Management, Batch Processing & Python Consulting
            - http://www.simplistix.co.uk

From chris at simplistix.co.uk  Thu Nov 12 22:03:58 2009
From: chris at simplistix.co.uk (Chris Withers)
Date: Thu, 12 Nov 2009 21:03:58 +0000
Subject: [Python-Dev] PyPI comments and ratings, *really*?
In-Reply-To: <bad82a80911120927vfadcabcia63920d56c2f6430@mail.gmail.com>
References: <D7E23A6D-C91C-46D8-A8FE-1EB16F3B2101@lericson.se>
	<200911130038.05516.steve@pearwood.info>
	<bad82a80911120927vfadcabcia63920d56c2f6430@mail.gmail.com>
Message-ID: <4AFC783E.5020006@simplistix.co.uk>

Arc Riley wrote:
> It would be more useful to provide a PyPI mechanism to publish a link to 
> file bugs on the project's own website and leave project ratings the 
> work of other sites such as Ohloh.

Yes, I really wish I could include all the links in the sections on, 
say, http://www.simplistix.co.uk/software/python/errorhandler in my 
package metadata, and have that show on PyPI.

Sadly, there are many PEP battles and much gnashing of teeth before that 
will ever happen :-(

Chris

-- 
Simplistix - Content Management, Batch Processing & Python Consulting
            - http://www.simplistix.co.uk

From ben+python at benfinney.id.au  Thu Nov 12 22:11:03 2009
From: ben+python at benfinney.id.au (Ben Finney)
Date: Fri, 13 Nov 2009 08:11:03 +1100
Subject: [Python-Dev] PyPI comments and ratings, *really*?
References: <D7E23A6D-C91C-46D8-A8FE-1EB16F3B2101@lericson.se>
	<200911130038.05516.steve@pearwood.info>
	<ca471dc20911120802r4c765f78h669e56335a38f11c@mail.gmail.com>
	<4222a8490911120831y5233e547pbfd22de60bcf436e@mail.gmail.com>
	<7AF4743F-4602-4E4C-8B8F-2FBC34D3945F@masklinn.net>
Message-ID: <87vdhfwis8.fsf@benfinney.id.au>

Masklinn <masklinn at masklinn.net> writes:

> On 12 Nov 2009, at 17:31 , Jesse Noller wrote:
> > But before we even did those; why not have mandatory links for
> > entries to bug trackers, mailing lists, source repositories, etc?
> > I'm saying saying this doesn't seem well thought out, and the
> > current implementation is broken by design. Of course, as I said
> > earlier; since I don't have time to patch it; I'll simply just not
> > participate.
> I think having links to those is a very good idea and more important
> than a comment/notation system. They shouldn't be mandatory though,
> not every library has a mailing list, or even a (public anyway) bug
> tracker.

I think Jesse's point (or, if he's not willing to claim it, my point) is
that, compared to the mandatory comment system, it makes much *more*
sense to have a mandatory field for ?URL to the BTS for this project?.

At least it can be expected that in many cases project maintainers will
*want* to use a conventional BTS, VCS, discussion forum, etc. So that
route makes more sense than a mandatory comment system outside the
project maintainer's control, while providing the user-participation
that is the purported motivation.

-- 
 \        ?Spam will be a thing of the past in two years' time.? ?Bill |
  `\                                                 Gates, 2004-01-24 |
_o__)                                                                  |
Ben Finney


From solipsis at pitrou.net  Thu Nov 12 22:26:34 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Thu, 12 Nov 2009 21:26:34 +0000 (UTC)
Subject: [Python-Dev] PyPI comments and ratings, *really*?
References: <D7E23A6D-C91C-46D8-A8FE-1EB16F3B2101@lericson.se>
	<200911130038.05516.steve@pearwood.info>
	<4222a8490911120606t3e138957kcdb21104261a1ead@mail.gmail.com>
	<B10AA20C-1C2F-44F0-B8B5-64F26691CC83@python.org>
	<hdhk7l$tru$1@ger.gmane.org>
	<ca471dc20911121054t76eb5bfbsbd6a9e47a9e71d8a@mail.gmail.com>
	<4222a8490911121125w67ffacfct9989db242d7c00f5@mail.gmail.com>
	<ca471dc20911121130r35c47256q75b284bf36313ab7@mail.gmail.com>
	<20091112203057.GA14396@amk-desktop.matrixgroup.net>
Message-ID: <loom.20091112T222535-253@post.gmane.org>

A.M. Kuchling <amk <at> amk.ca> writes:
> 
> For comments, haloscan and disqus are third-party comment-hosting
> services; http://redalt.com/blog/comment-services has a longer list.

They are horrible for page loading times; and besides, I don't know how you can
trust such third-party to provide an important function of your Web site, /and/
manage its data.




From ncoghlan at gmail.com  Thu Nov 12 22:31:17 2009
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Fri, 13 Nov 2009 07:31:17 +1000
Subject: [Python-Dev] PyPI comments and ratings, *really*?
In-Reply-To: <4AFC779C.6040608@simplistix.co.uk>
References: <D7E23A6D-C91C-46D8-A8FE-1EB16F3B2101@lericson.se>	<200911130038.05516.steve@pearwood.info>	<ca471dc20911120802r4c765f78h669e56335a38f11c@mail.gmail.com>	<4AFC65A9.8060304@v.loewis.de>
	<4AFC779C.6040608@simplistix.co.uk>
Message-ID: <4AFC7EA5.5080700@gmail.com>

Chris Withers wrote:
> I'm quite okay with having a banner
> saying "This package has opted not to receive comments".

Particularly if the developer is able to add a prominent link to the
project's own support site or mailing list.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------

From rdmurray at bitdance.com  Thu Nov 12 22:53:00 2009
From: rdmurray at bitdance.com (R. David Murray)
Date: Thu, 12 Nov 2009 16:53:00 -0500 (EST)
Subject: [Python-Dev] PyPI comments and ratings, *really*?
In-Reply-To: <hdhrvs$qs8$1@ger.gmane.org>
References: <D7E23A6D-C91C-46D8-A8FE-1EB16F3B2101@lericson.se>
	<200911130038.05516.steve@pearwood.info>
	<4222a8490911120606t3e138957kcdb21104261a1ead@mail.gmail.com>
	<B10AA20C-1C2F-44F0-B8B5-64F26691CC83@python.org>
	<hdhk7l$tru$1@ger.gmane.org>
	<ca471dc20911121054t76eb5bfbsbd6a9e47a9e71d8a@mail.gmail.com>
	<hdhrvs$qs8$1@ger.gmane.org>
Message-ID: <Pine.LNX.4.64.0911121648480.12274@kimball.webabinitio.net>

On Thu, 12 Nov 2009 at 15:42, Terry Reedy wrote:
> Part of the pypi problem is a startup problem of initially low numbers. If 
> the only people who bother to log in to rate are the disgruntled, then the 
> ratings/reviews will be biased. I wonder how many of the people promoting the 
> new feature have themselves logged in to systematically rate and possibly 
> comment on every package they have looked at, and thereby kickstart the 
> system with fair responses.

For what it's worth, I never look at PyPI.  I get my packages either
through Gentoo's portage or, if it isn't there (yet), by finding the
package's home site through Google.  So I'm a happy user of a number of
packages, whose comments will never show up on PyPI.

--David (RDM)

From foom at fuhm.net  Thu Nov 12 22:53:47 2009
From: foom at fuhm.net (James Y Knight)
Date: Thu, 12 Nov 2009 16:53:47 -0500
Subject: [Python-Dev] PyPI comments and ratings, *really*?
In-Reply-To: <87vdhfwis8.fsf@benfinney.id.au>
References: <D7E23A6D-C91C-46D8-A8FE-1EB16F3B2101@lericson.se>
	<200911130038.05516.steve@pearwood.info>
	<ca471dc20911120802r4c765f78h669e56335a38f11c@mail.gmail.com>
	<4222a8490911120831y5233e547pbfd22de60bcf436e@mail.gmail.com>
	<7AF4743F-4602-4E4C-8B8F-2FBC34D3945F@masklinn.net>
	<87vdhfwis8.fsf@benfinney.id.au>
Message-ID: <3132DFCC-BA06-4AF6-863F-534C715A13E2@fuhm.net>


On Nov 12, 2009, at 4:11 PM, Ben Finney wrote:
> I think Jesse's point (or, if he's not willing to claim it, my  
> point) is
> that, compared to the mandatory comment system, it makes much *more*
> sense to have a mandatory field for ?URL to the BTS for this project?.

One might look at the "competition" for inspiration. Looking at CPAN.  
There's no "comments" feature, but there is a "CPAN RT" bug-tracker  
which appears to be a way for users to submit comments/problems about  
packages in a way common to all packages in CPAN, but distinct from  
upstream's bug trackers/lists/etc. I'd assume that gets emailed to the  
listed maintainer of the package as well as being accessible to other  
users, although I don't really have any idea.

e.g.
http://search.cpan.org/~capttofu/DBD-mysql/lib/DBD/mysql.pm

There might be something to be said for providing users a way to  
provide feedback that doesn't require making a accounts in a bazillion  
separate bugtrackers.

*shrug*

James

From martin at v.loewis.de  Thu Nov 12 22:58:02 2009
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Thu, 12 Nov 2009 22:58:02 +0100
Subject: [Python-Dev] PyPI comments and ratings, *really*?
In-Reply-To: <4AFC7EA5.5080700@gmail.com>
References: <D7E23A6D-C91C-46D8-A8FE-1EB16F3B2101@lericson.se>	<200911130038.05516.steve@pearwood.info>	<ca471dc20911120802r4c765f78h669e56335a38f11c@mail.gmail.com>	<4AFC65A9.8060304@v.loewis.de>
	<4AFC779C.6040608@simplistix.co.uk> <4AFC7EA5.5080700@gmail.com>
Message-ID: <4AFC84EA.9070003@v.loewis.de>

Nick Coghlan wrote:
> Chris Withers wrote:
>> I'm quite okay with having a banner
>> saying "This package has opted not to receive comments".
> 
> Particularly if the developer is able to add a prominent link to the
> project's own support site or mailing list.

It's really puzzling that people always assume that people would use
comments primarily to get help, or to report problems. It appears that
nobody expects users to merely comment, voicing their opinion - yet
the comment that triggered this particular thread was precisely that.

Regards,
Martin

From martin at v.loewis.de  Thu Nov 12 23:08:35 2009
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Thu, 12 Nov 2009 23:08:35 +0100
Subject: [Python-Dev] PyPI comments and ratings, *really*?
In-Reply-To: <hdhrvs$qs8$1@ger.gmane.org>
References: <D7E23A6D-C91C-46D8-A8FE-1EB16F3B2101@lericson.se>	<200911130038.05516.steve@pearwood.info>	<4222a8490911120606t3e138957kcdb21104261a1ead@mail.gmail.com>	<B10AA20C-1C2F-44F0-B8B5-64F26691CC83@python.org>	<hdhk7l$tru$1@ger.gmane.org>	<ca471dc20911121054t76eb5bfbsbd6a9e47a9e71d8a@mail.gmail.com>
	<hdhrvs$qs8$1@ger.gmane.org>
Message-ID: <4AFC8763.9080300@v.loewis.de>

> Part of the pypi problem is a startup problem of initially low numbers.
> If the only people who bother to log in to rate are the disgruntled,
> then the ratings/reviews will be biased.

Fortunately, that isn't actually the case. The majority of comments is
positive (from scanning the full list of comments, without actually
counting).

> Even on Amazon, an author could, I presume, add a response to a
> factually incorrect review.

And so they can on PyPI.

Regards,
Martin

From martin at v.loewis.de  Thu Nov 12 23:12:18 2009
From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=)
Date: Thu, 12 Nov 2009 23:12:18 +0100
Subject: [Python-Dev] PyPI comments and ratings, *really*?
In-Reply-To: <87vdhfwis8.fsf@benfinney.id.au>
References: <D7E23A6D-C91C-46D8-A8FE-1EB16F3B2101@lericson.se>	<200911130038.05516.steve@pearwood.info>	<ca471dc20911120802r4c765f78h669e56335a38f11c@mail.gmail.com>	<4222a8490911120831y5233e547pbfd22de60bcf436e@mail.gmail.com>	<7AF4743F-4602-4E4C-8B8F-2FBC34D3945F@masklinn.net>
	<87vdhfwis8.fsf@benfinney.id.au>
Message-ID: <4AFC8842.5050904@v.loewis.de>

> At least it can be expected that in many cases project maintainers will
> *want* to use a conventional BTS, VCS, discussion forum, etc. So that
> route makes more sense than a mandatory comment system outside the
> project maintainer's control, while providing the user-participation
> that is the purported motivation.

I think you are missing the point of the commenting system: these
comments are *not* directed towards the package author. Instead, they
are directed towards fellow users of the package. For this kind of
message, a bugtracker is completely inappropriate, as is a mailing list,
or any other kind of support forum.

I agree that users asking for support should not use the PyPI commenting
system.

Regards,
Martin


From g.brandl at gmx.net  Thu Nov 12 23:12:02 2009
From: g.brandl at gmx.net (Georg Brandl)
Date: Thu, 12 Nov 2009 23:12:02 +0100
Subject: [Python-Dev] PyPI comments and ratings, *really*?
In-Reply-To: <Pine.LNX.4.64.0911121648480.12274@kimball.webabinitio.net>
References: <D7E23A6D-C91C-46D8-A8FE-1EB16F3B2101@lericson.se>	<200911130038.05516.steve@pearwood.info>	<4222a8490911120606t3e138957kcdb21104261a1ead@mail.gmail.com>	<B10AA20C-1C2F-44F0-B8B5-64F26691CC83@python.org>	<hdhk7l$tru$1@ger.gmane.org>	<ca471dc20911121054t76eb5bfbsbd6a9e47a9e71d8a@mail.gmail.com>	<hdhrvs$qs8$1@ger.gmane.org>
	<Pine.LNX.4.64.0911121648480.12274@kimball.webabinitio.net>
Message-ID: <hdi17l$cli$1@ger.gmane.org>

R. David Murray schrieb:
> On Thu, 12 Nov 2009 at 15:42, Terry Reedy wrote:
>> Part of the pypi problem is a startup problem of initially low numbers. If 
>> the only people who bother to log in to rate are the disgruntled, then the 
>> ratings/reviews will be biased. I wonder how many of the people promoting the 
>> new feature have themselves logged in to systematically rate and possibly 
>> comment on every package they have looked at, and thereby kickstart the 
>> system with fair responses.
> 
> For what it's worth, I never look at PyPI.  I get my packages either
> through Gentoo's portage or, if it isn't there (yet), by finding the
> package's home site through Google.  So I'm a happy user of a number of
> packages, whose comments will never show up on PyPI.

The other large group of easy_install users will also never get to the
individual package pages on PyPI.

Except when they have a problem, and then they are likely to only complain
through the comments.

Georg


From masklinn at masklinn.net  Thu Nov 12 23:23:09 2009
From: masklinn at masklinn.net (Masklinn)
Date: Thu, 12 Nov 2009 23:23:09 +0100
Subject: [Python-Dev] PyPI comments and ratings, *really*?
In-Reply-To: <3132DFCC-BA06-4AF6-863F-534C715A13E2@fuhm.net>
References: <D7E23A6D-C91C-46D8-A8FE-1EB16F3B2101@lericson.se>
	<200911130038.05516.steve@pearwood.info>
	<ca471dc20911120802r4c765f78h669e56335a38f11c@mail.gmail.com>
	<4222a8490911120831y5233e547pbfd22de60bcf436e@mail.gmail.com>
	<7AF4743F-4602-4E4C-8B8F-2FBC34D3945F@masklinn.net>
	<87vdhfwis8.fsf@benfinney.id.au>
	<3132DFCC-BA06-4AF6-863F-534C715A13E2@fuhm.net>
Message-ID: <524B006B-E69A-4697-A75A-F22C98BFE441@masklinn.net>

On 12 Nov 2009, at 22:53 , James Y Knight wrote:
> On Nov 12, 2009, at 4:11 PM, Ben Finney wrote:
>> I think Jesse's point (or, if he's not willing to claim it, my point) is
>> that, compared to the mandatory comment system, it makes much *more*
>> sense to have a mandatory field for ?URL to the BTS for this project?.
> 
> One might look at the "competition" for inspiration. Looking at CPAN. There's no "comments" feature
There is, on search.cpan.org. See http://search.cpan.org/~petdance/ack/ for instance, the link leads to http://cpanratings.perl.org/ (a pretty interesting example of the "distributed" nature of cpan in fact).


From ben+python at benfinney.id.au  Thu Nov 12 23:39:01 2009
From: ben+python at benfinney.id.au (Ben Finney)
Date: Fri, 13 Nov 2009 09:39:01 +1100
Subject: [Python-Dev] PyPI comments and ratings, *really*?
References: <D7E23A6D-C91C-46D8-A8FE-1EB16F3B2101@lericson.se>
	<200911130038.05516.steve@pearwood.info>
	<4222a8490911120606t3e138957kcdb21104261a1ead@mail.gmail.com>
	<B10AA20C-1C2F-44F0-B8B5-64F26691CC83@python.org>
	<hdhk7l$tru$1@ger.gmane.org>
	<ca471dc20911121054t76eb5bfbsbd6a9e47a9e71d8a@mail.gmail.com>
	<4222a8490911121125w67ffacfct9989db242d7c00f5@mail.gmail.com>
	<ca471dc20911121130r35c47256q75b284bf36313ab7@mail.gmail.com>
	<bc8ec34e0911121145w795b081ft2e27168edd205ef@mail.gmail.com>
	<ca471dc20911121204p2fd8d643xf5f8c99f874f8e0d@mail.gmail.com>
Message-ID: <87my2rwepm.fsf@benfinney.id.au>

Guido van Rossum <guido at python.org> writes:

> Maybe that's an example of a user who doesn't know how to use those
> support channels? I know I wouldn't bother with IRC even if it was the
> only way to get in touch with users, I hate it with a vengeance.
> (Though arguably I'm a special case -- whenever I show up everyone
> goes "ooooh, Guido is here." :-) And I might not want to sign up for a
> mailing list for a casual question. And what exactly is a "forum"?

By my understanding of English, a ?forum? is any assembly for open
discussion of a topic. Mailing lists, IRC channels, Usenet groups, and
gatherings in a pub all merit the term ?forum?; and no crappy web
applications get to usurp its meaning exclusively.

(Nothing to do with PyPI, just an appeal to keeping terms useful.)

-- 
 \     ?I have had a perfectly wonderful evening, but this wasn't it.? |
  `\                                                     ?Groucho Marx |
_o__)                                                                  |
Ben Finney


From foom at fuhm.net  Thu Nov 12 23:44:16 2009
From: foom at fuhm.net (James Y Knight)
Date: Thu, 12 Nov 2009 17:44:16 -0500
Subject: [Python-Dev] PyPI comments and ratings, *really*?
In-Reply-To: <524B006B-E69A-4697-A75A-F22C98BFE441@masklinn.net>
References: <D7E23A6D-C91C-46D8-A8FE-1EB16F3B2101@lericson.se>
	<200911130038.05516.steve@pearwood.info>
	<ca471dc20911120802r4c765f78h669e56335a38f11c@mail.gmail.com>
	<4222a8490911120831y5233e547pbfd22de60bcf436e@mail.gmail.com>
	<7AF4743F-4602-4E4C-8B8F-2FBC34D3945F@masklinn.net>
	<87vdhfwis8.fsf@benfinney.id.au>
	<3132DFCC-BA06-4AF6-863F-534C715A13E2@fuhm.net>
	<524B006B-E69A-4697-A75A-F22C98BFE441@masklinn.net>
Message-ID: <111E097E-53C3-4F91-BDD6-2AAFC2E7AC4E@fuhm.net>


On Nov 12, 2009, at 5:23 PM, Masklinn wrote:

> On 12 Nov 2009, at 22:53 , James Y Knight wrote:
>> On Nov 12, 2009, at 4:11 PM, Ben Finney wrote:
>>> I think Jesse's point (or, if he's not willing to claim it, my  
>>> point) is
>>> that, compared to the mandatory comment system, it makes much *more*
>>> sense to have a mandatory field for ?URL to the BTS for this  
>>> project?.
>>
>> One might look at the "competition" for inspiration. Looking at  
>> CPAN. There's no "comments" feature
> There is, on search.cpan.org. See http://search.cpan.org/~petdance/ack/ 
>  for instance, the link leads to http://cpanratings.perl.org/ (a  
> pretty interesting example of the "distributed" nature of cpan in  
> fact).

Ah, I see. I totally managed to miss that...I guess that's an  
interesting example of a bad web ui. :)

James


From martin at v.loewis.de  Thu Nov 12 23:47:00 2009
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Thu, 12 Nov 2009 23:47:00 +0100
Subject: [Python-Dev] PyPI comments and ratings, *really*?
In-Reply-To: <hdi17l$cli$1@ger.gmane.org>
References: <D7E23A6D-C91C-46D8-A8FE-1EB16F3B2101@lericson.se>	<200911130038.05516.steve@pearwood.info>	<4222a8490911120606t3e138957kcdb21104261a1ead@mail.gmail.com>	<B10AA20C-1C2F-44F0-B8B5-64F26691CC83@python.org>	<hdhk7l$tru$1@ger.gmane.org>	<ca471dc20911121054t76eb5bfbsbd6a9e47a9e71d8a@mail.gmail.com>	<hdhrvs$qs8$1@ger.gmane.org>	<Pine.LNX.4.64.0911121648480.12274@kimball.webabinitio.net>
	<hdi17l$cli$1@ger.gmane.org>
Message-ID: <4AFC9064.20700@v.loewis.de>

> Except when they have a problem, and then they are likely to only complain
> through the comments.

As this theory has been repeated often here, I decided to go through all
comments and classify them, as:
- good: (overall) positive evaluation (possibly including minor
  criticism/wishes)
- bad: negative evaluation, complaint, bug report
- neutral: statement of fact (typically in response to some help
  request)

Here is what I got:
- good:   20
- bad:    11
- neutral: 9

So far, the theory that only complainers will comment cannot be
substantiated by facts. Of course, it could be that all the negative
comments come from easy_install users, and all the positive comments
from people who browse through PyPI... Notice however that such
easy_install users often would have to create a PyPI account first.

Regards,
Martin

From ben+python at benfinney.id.au  Thu Nov 12 23:49:50 2009
From: ben+python at benfinney.id.au (Ben Finney)
Date: Fri, 13 Nov 2009 09:49:50 +1100
Subject: [Python-Dev] PyPI comments and ratings, *really*?
References: <D7E23A6D-C91C-46D8-A8FE-1EB16F3B2101@lericson.se>
	<200911130038.05516.steve@pearwood.info>
	<ca471dc20911120802r4c765f78h669e56335a38f11c@mail.gmail.com>
	<4AFC65A9.8060304@v.loewis.de> <4AFC779C.6040608@simplistix.co.uk>
	<4AFC7EA5.5080700@gmail.com> <4AFC84EA.9070003@v.loewis.de>
Message-ID: <87iqdfwe7l.fsf@benfinney.id.au>

"Martin v. L?wis" <martin at v.loewis.de> writes:

> Nick Coghlan wrote:
> > Particularly if the developer is able to add a prominent link to the
> > project's own support site or mailing list.
>
> It's really puzzling that people always assume that people would use
> comments primarily to get help, or to report problems. It appears that
> nobody expects users to merely comment, voicing their opinion - yet
> the comment that triggered this particular thread was precisely that.

I'm reading ?the project's own mailing list? as a fine place to do
exactly that: discuss the project.

-- 
 \     ?If nothing changes, everything will remain the same.? ?Barne's |
  `\                                                               Law |
_o__)                                                                  |
Ben Finney


From masklinn at masklinn.net  Thu Nov 12 23:50:25 2009
From: masklinn at masklinn.net (Masklinn)
Date: Thu, 12 Nov 2009 23:50:25 +0100
Subject: [Python-Dev] PyPI comments and ratings, *really*?
In-Reply-To: <111E097E-53C3-4F91-BDD6-2AAFC2E7AC4E@fuhm.net>
References: <D7E23A6D-C91C-46D8-A8FE-1EB16F3B2101@lericson.se>
	<200911130038.05516.steve@pearwood.info>
	<ca471dc20911120802r4c765f78h669e56335a38f11c@mail.gmail.com>
	<4222a8490911120831y5233e547pbfd22de60bcf436e@mail.gmail.com>
	<7AF4743F-4602-4E4C-8B8F-2FBC34D3945F@masklinn.net>
	<87vdhfwis8.fsf@benfinney.id.au>
	<3132DFCC-BA06-4AF6-863F-534C715A13E2@fuhm.net>
	<524B006B-E69A-4697-A75A-F22C98BFE441@masklinn.net>
	<111E097E-53C3-4F91-BDD6-2AAFC2E7AC4E@fuhm.net>
Message-ID: <37F67F35-31EF-4DC9-A2D6-85FDA7A279E9@masklinn.net>

On 12 Nov 2009, at 23:44 , James Y Knight wrote:
> On Nov 12, 2009, at 5:23 PM, Masklinn wrote:
>> On 12 Nov 2009, at 22:53 , James Y Knight wrote:
>>> On Nov 12, 2009, at 4:11 PM, Ben Finney wrote:
>>>> I think Jesse's point (or, if he's not willing to claim it, my point) is
>>>> that, compared to the mandatory comment system, it makes much *more*
>>>> sense to have a mandatory field for ?URL to the BTS for this project?.
>>> 
>>> One might look at the "competition" for inspiration. Looking at CPAN. There's no "comments" feature
>> There is, on search.cpan.org. See http://search.cpan.org/~petdance/ack/ for instance, the link leads to http://cpanratings.perl.org/ (a pretty interesting example of the "distributed" nature of cpan in fact).
> 
> Ah, I see. I totally managed to miss that...I guess that's an interesting example of a bad web ui. :)
I'm not sure it's so bad, it's just that it's at the root of the "cpan package" rather than in the POD (just click on "BDB-mysql" in the breadcrumb trail, landing at http://search.cpan.org/~capttofu/DBD-mysql/).

Interestingly, the link to cpanratings from BDB-mysql is broken and yields a 404, even though its CPAN page lists 5 reviews and a score of ~3.5.

From tseaver at palladion.com  Thu Nov 12 23:52:08 2009
From: tseaver at palladion.com (Tres Seaver)
Date: Thu, 12 Nov 2009 17:52:08 -0500
Subject: [Python-Dev] PyPI comments and ratings, *really*?
In-Reply-To: <ca471dc20911121130r35c47256q75b284bf36313ab7@mail.gmail.com>
References: <D7E23A6D-C91C-46D8-A8FE-1EB16F3B2101@lericson.se>
	<200911130038.05516.steve@pearwood.info>	<4222a8490911120606t3e138957kcdb21104261a1ead@mail.gmail.com>
	<B10AA20C-1C2F-44F0-B8B5-64F26691CC83@python.org>	<hdhk7l$tru$1@ger.gmane.org>
	<ca471dc20911121054t76eb5bfbsbd6a9e47a9e71d8a@mail.gmail.com>
	<4222a8490911121125w67ffacfct9989db242d7c00f5@mail.gmail.com>
	<ca471dc20911121130r35c47256q75b284bf36313ab7@mail.gmail.com>
Message-ID: <hdi3io$khr$1@ger.gmane.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Guido van Rossum wrote:
> On Thu, Nov 12, 2009 at 11:25 AM, Jesse Noller <jnoller at gmail.com> wrote:
>> I'd not trust a package without a bug tracker, mailing list or link to
>> the source a lot sooner than something without comments and ratings.
> 
> Yeah, but you're not exactly an average user. Most users don't know
> how to use a bug tracker. Also, there's a large variety of packages on
> PyPI. Not every developer has the same attitude, but they all live
> happily together on PyPI. (Or did you want someone to start a separate
> CPAN "for the rest of them" ? :-)
> 
> [...]
>> What about astroturfing? What's to stop me from writing a script to
>> create a pile of accounts and then bumping packages I like with
>> glowing ratings and reviews? Who is going to be the moderator, and how
>> to decide between spam, incorrect comment, etc?
> 
> Those are all hard problems, though all of them have at least partial
> solutions in the other worlds (Amazon, Wikipedia, Apple app store,
> etc.). Maybe there should be a standard "social app" that you can just
> customize for a specific purpose. Sounds like an interesting project,
> actually.

The appstore analogy actually helps Jesse's case"  "iFart in your
general direction."  (iFart is the top-rated app).  Popularity and
quality aren't related in any direct fashion.


Tres.
- --
===================================================================
Tres Seaver          +1 540-429-0999          tseaver at palladion.com
Palladion Software   "Excellence by Design"    http://palladion.com
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iEYEARECAAYFAkr8kZMACgkQ+gerLs4ltQ4ScQCeJtYU9KkAq2K1Dkk0jK9ffHvB
IuwAoNBWpMPFR1YsdhQN31oS1L5m91UL
=qmmQ
-----END PGP SIGNATURE-----


From martin at v.loewis.de  Thu Nov 12 23:57:50 2009
From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=)
Date: Thu, 12 Nov 2009 23:57:50 +0100
Subject: [Python-Dev] PyPI comments and ratings, *really*?
In-Reply-To: <87iqdfwe7l.fsf@benfinney.id.au>
References: <D7E23A6D-C91C-46D8-A8FE-1EB16F3B2101@lericson.se>	<200911130038.05516.steve@pearwood.info>	<ca471dc20911120802r4c765f78h669e56335a38f11c@mail.gmail.com>	<4AFC65A9.8060304@v.loewis.de>
	<4AFC779C.6040608@simplistix.co.uk>	<4AFC7EA5.5080700@gmail.com>
	<4AFC84EA.9070003@v.loewis.de> <87iqdfwe7l.fsf@benfinney.id.au>
Message-ID: <4AFC92EE.3080409@v.loewis.de>

Ben Finney wrote:
> "Martin v. L?wis" <martin at v.loewis.de> writes:
> 
>> Nick Coghlan wrote:
>>> Particularly if the developer is able to add a prominent link to the
>>> project's own support site or mailing list.
>> It's really puzzling that people always assume that people would use
>> comments primarily to get help, or to report problems. It appears that
>> nobody expects users to merely comment, voicing their opinion - yet
>> the comment that triggered this particular thread was precisely that.
> 
> I'm reading ?the project's own mailing list? as a fine place to do
> exactly that: discuss the project.

No no no: discuss with whom? The commenter may not want to discuss
anything about the project, let alone with the package author. If it's
a positive comment, you may likely only get "me too", anyway; if it's
a negative comment, perhaps the commenter has already given up on the
package, and just wants to warn other users. It's those other users that
then will have problems finding reviews on the mailing list.

Regards,
Martin

From solipsis at pitrou.net  Fri Nov 13 00:00:52 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Thu, 12 Nov 2009 23:00:52 +0000 (UTC)
Subject: [Python-Dev] PyPI comments and ratings, *really*?
References: <D7E23A6D-C91C-46D8-A8FE-1EB16F3B2101@lericson.se>	<200911130038.05516.steve@pearwood.info>	<ca471dc20911120802r4c765f78h669e56335a38f11c@mail.gmail.com>	<4222a8490911120831y5233e547pbfd22de60bcf436e@mail.gmail.com>	<7AF4743F-4602-4E4C-8B8F-2FBC34D3945F@masklinn.net>
	<87vdhfwis8.fsf@benfinney.id.au> <4AFC8842.5050904@v.loewis.de>
Message-ID: <loom.20091113T000003-485@post.gmane.org>

Martin v. L?wis <martin <at> v.loewis.de> writes:
> 
> I think you are missing the point of the commenting system: these
> comments are *not* directed towards the package author. Instead, they
> are directed towards fellow users of the package. For this kind of
> message, a bugtracker is completely inappropriate, as is a mailing list,
> or any other kind of support forum.

Then why not simply add a sentence or two before the comment form warning that
the comment system is not meant to ask for help, support or debugging about the
package?

Regards

Antoine.



From masklinn at masklinn.net  Fri Nov 13 00:10:05 2009
From: masklinn at masklinn.net (Masklinn)
Date: Fri, 13 Nov 2009 00:10:05 +0100
Subject: [Python-Dev] PyPI comments and ratings, *really*?
In-Reply-To: <loom.20091113T000003-485@post.gmane.org>
References: <D7E23A6D-C91C-46D8-A8FE-1EB16F3B2101@lericson.se>	<200911130038.05516.steve@pearwood.info>	<ca471dc20911120802r4c765f78h669e56335a38f11c@mail.gmail.com>	<4222a8490911120831y5233e547pbfd22de60bcf436e@mail.gmail.com>	<7AF4743F-4602-4E4C-8B8F-2FBC34D3945F@masklinn.net>
	<87vdhfwis8.fsf@benfinney.id.au> <4AFC8842.5050904@v.loewis.de>
	<loom.20091113T000003-485@post.gmane.org>
Message-ID: <805B874E-6FD5-44DB-A908-922C93866458@masklinn.net>

On 13 Nov 2009, at 00:00 , Antoine Pitrou wrote:
> 
> Then why not simply add a sentence or two before the comment form warning that
> the comment system is not meant to ask for help, support or debugging about the
> package?
Because users don't read warnings. The warning will therefore be promptly ignored, and then the aforementioned user will start ripping on the package because he didn't get help following his comment.

From jnoller at gmail.com  Fri Nov 13 00:11:23 2009
From: jnoller at gmail.com (Jesse Noller)
Date: Thu, 12 Nov 2009 18:11:23 -0500
Subject: [Python-Dev] PyPI comments and ratings, *really*?
In-Reply-To: <4AFC9064.20700@v.loewis.de>
References: <D7E23A6D-C91C-46D8-A8FE-1EB16F3B2101@lericson.se>
	<200911130038.05516.steve@pearwood.info>
	<4222a8490911120606t3e138957kcdb21104261a1ead@mail.gmail.com>
	<B10AA20C-1C2F-44F0-B8B5-64F26691CC83@python.org>
	<hdhk7l$tru$1@ger.gmane.org>
	<ca471dc20911121054t76eb5bfbsbd6a9e47a9e71d8a@mail.gmail.com>
	<hdhrvs$qs8$1@ger.gmane.org>
	<Pine.LNX.4.64.0911121648480.12274@kimball.webabinitio.net>
	<hdi17l$cli$1@ger.gmane.org> <4AFC9064.20700@v.loewis.de>
Message-ID: <4222a8490911121511s40eba937n2c7b1fde6f695c6c@mail.gmail.com>

On Thu, Nov 12, 2009 at 5:47 PM, "Martin v. L?wis" <martin at v.loewis.de> wrote:
>> Except when they have a problem, and then they are likely to only complain
>> through the comments.
>
> As this theory has been repeated often here, I decided to go through all
> comments and classify them, as:
> - good: (overall) positive evaluation (possibly including minor
> ?criticism/wishes)
> - bad: negative evaluation, complaint, bug report
> - neutral: statement of fact (typically in response to some help
> ?request)
>
> Here is what I got:
> - good: ? 20
> - bad: ? ?11
> - neutral: 9
>
> So far, the theory that only complainers will comment cannot be
> substantiated by facts. Of course, it could be that all the negative
> comments come from easy_install users, and all the positive comments
> from people who browse through PyPI... Notice however that such
> easy_install users often would have to create a PyPI account first.
>
> Regards,
> Martin

And how many of the "good" comments are astroturfers?

What's so bad about package maintainers from having an opt-out? I'd
rather run a pypi competitor at this point.

From solipsis at pitrou.net  Fri Nov 13 00:15:08 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Thu, 12 Nov 2009 23:15:08 +0000 (UTC)
Subject: [Python-Dev] PyPI comments and ratings, *really*?
References: <D7E23A6D-C91C-46D8-A8FE-1EB16F3B2101@lericson.se>	<200911130038.05516.steve@pearwood.info>	<ca471dc20911120802r4c765f78h669e56335a38f11c@mail.gmail.com>	<4222a8490911120831y5233e547pbfd22de60bcf436e@mail.gmail.com>	<7AF4743F-4602-4E4C-8B8F-2FBC34D3945F@masklinn.net>
	<87vdhfwis8.fsf@benfinney.id.au> <4AFC8842.5050904@v.loewis.de>
	<loom.20091113T000003-485@post.gmane.org>
	<805B874E-6FD5-44DB-A908-922C93866458@masklinn.net>
Message-ID: <loom.20091113T001402-725@post.gmane.org>

Masklinn <masklinn <at> masklinn.net> writes:
> 
> On 13 Nov 2009, at 00:00 , Antoine Pitrou wrote:
> > 
> > Then why not simply add a sentence or two before the comment form warning
that
> > the comment system is not meant to ask for help, support or debugging about 
the
> > package?
> Because users don't read warnings.

I don't like assuming users are idiots.

> The warning will therefore be promptly ignored, and then the
> aforementioned user will start ripping on the package because he didn't get
> help following his comment.

And then it's easy to point out that he was wrong if there was a warning in the
first place.



From martin at v.loewis.de  Fri Nov 13 00:25:41 2009
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Fri, 13 Nov 2009 00:25:41 +0100
Subject: [Python-Dev] PyPI comments and ratings, *really*?
In-Reply-To: <4222a8490911121511s40eba937n2c7b1fde6f695c6c@mail.gmail.com>
References: <D7E23A6D-C91C-46D8-A8FE-1EB16F3B2101@lericson.se>	
	<200911130038.05516.steve@pearwood.info>	
	<4222a8490911120606t3e138957kcdb21104261a1ead@mail.gmail.com>	
	<B10AA20C-1C2F-44F0-B8B5-64F26691CC83@python.org>	
	<hdhk7l$tru$1@ger.gmane.org>	
	<ca471dc20911121054t76eb5bfbsbd6a9e47a9e71d8a@mail.gmail.com>	
	<hdhrvs$qs8$1@ger.gmane.org>	
	<Pine.LNX.4.64.0911121648480.12274@kimball.webabinitio.net>	
	<hdi17l$cli$1@ger.gmane.org> <4AFC9064.20700@v.loewis.de>
	<4222a8490911121511s40eba937n2c7b1fde6f695c6c@mail.gmail.com>
Message-ID: <4AFC9975.7040204@v.loewis.de>

> And how many of the "good" comments are astroturfers?

If I understand that term correctly, it's about disguise: how would
I be able to answer that question?

> What's so bad about package maintainers from having an opt-out?

PyPI is not just (and perhaps not even primarily) there for the package
authors, but for the package users (and not surprisingly, it's
primarily the package authors who ask for banning the user opinions).

I'm just not willing to submit to one side; hence the poll.

Regards,
Martin



From ben+python at benfinney.id.au  Fri Nov 13 00:29:30 2009
From: ben+python at benfinney.id.au (Ben Finney)
Date: Fri, 13 Nov 2009 10:29:30 +1100
Subject: [Python-Dev] PyPI comments and ratings, *really*?
References: <D7E23A6D-C91C-46D8-A8FE-1EB16F3B2101@lericson.se>
	<200911130038.05516.steve@pearwood.info>
	<ca471dc20911120802r4c765f78h669e56335a38f11c@mail.gmail.com>
	<4222a8490911120831y5233e547pbfd22de60bcf436e@mail.gmail.com>
	<7AF4743F-4602-4E4C-8B8F-2FBC34D3945F@masklinn.net>
	<87vdhfwis8.fsf@benfinney.id.au> <4AFC8842.5050904@v.loewis.de>
	<loom.20091113T000003-485@post.gmane.org>
	<805B874E-6FD5-44DB-A908-922C93866458@masklinn.net>
	<loom.20091113T001402-725@post.gmane.org>
Message-ID: <87eio3wcdh.fsf@benfinney.id.au>

Antoine Pitrou <solipsis at pitrou.net> writes:

> Masklinn <masklinn <at> masklinn.net> writes:
> > Because users don't read warnings.
>
> I don't like assuming users are idiots.

You don't have to. You need only assume that users are busy, focussed on
a task (?leave feedback?), and will therefore unconsciously filter out
*anything* that is not the simplest path to complete that task.

> > The warning will therefore be promptly ignored, and then the
> > aforementioned user will start ripping on the package because he
> > didn't get help following his comment.
>
> And then it's easy to point out that he was wrong if there was a
> warning in the first place.

I don't like having systems which make it easier to do the wrong thing
than do the right thing, then blame those users for taking the obvious
path to their goal.

-- 
 \                         ?I'm a great lover, I'll bet.? ?Emo Philips |
  `\                                                                   |
_o__)                                                                  |
Ben Finney


From masklinn at masklinn.net  Fri Nov 13 00:30:59 2009
From: masklinn at masklinn.net (Masklinn)
Date: Fri, 13 Nov 2009 00:30:59 +0100
Subject: [Python-Dev] PyPI comments and ratings, *really*?
In-Reply-To: <loom.20091113T001402-725@post.gmane.org>
References: <D7E23A6D-C91C-46D8-A8FE-1EB16F3B2101@lericson.se>	<200911130038.05516.steve@pearwood.info>	<ca471dc20911120802r4c765f78h669e56335a38f11c@mail.gmail.com>	<4222a8490911120831y5233e547pbfd22de60bcf436e@mail.gmail.com>	<7AF4743F-4602-4E4C-8B8F-2FBC34D3945F@masklinn.net>
	<87vdhfwis8.fsf@benfinney.id.au> <4AFC8842.5050904@v.loewis.de>
	<loom.20091113T000003-485@post.gmane.org>
	<805B874E-6FD5-44DB-A908-922C93866458@masklinn.net>
	<loom.20091113T001402-725@post.gmane.org>
Message-ID: <FBB5BF49-8EE6-4407-B733-D929A9385457@masklinn.net>

On 13 Nov 2009, at 00:15 , Antoine Pitrou wrote:
> Masklinn <masklinn <at> masklinn.net> writes:
>> 
>> On 13 Nov 2009, at 00:00 , Antoine Pitrou wrote:
>>> 
>>> Then why not simply add a sentence or two before the comment form warning
> that
>>> the comment system is not meant to ask for help, support or debugging about 
> the
>>> package?
>> Because users don't read warnings.
> I don't like assuming users are idiots.
You don't have to. It's about expediency and care (or lack thereof) rather than idiocy. User(*) wants a solution, user finds place where he could ask, user asks.

Users (which includes e.g. language users) tend to be lazy, rather than stupid.

>> The warning will therefore be promptly ignored, and then the
>> aforementioned user will start ripping on the package because he didn't get
>> help following his comment.
> And then it's easy to point out that he was wrong if there was a warning in the
> first place.
And then user will probably ask why you're not answering the question since you're here anyway, or might go as far as telling you that if you're not going to help you might as well not answer.

*: not every user, but I believe a significant minority at least.

From nyamatongwe at gmail.com  Fri Nov 13 00:32:51 2009
From: nyamatongwe at gmail.com (Neil Hodgson)
Date: Fri, 13 Nov 2009 10:32:51 +1100
Subject: [Python-Dev] PyPI comments and ratings, *really*?
In-Reply-To: <loom.20091113T001402-725@post.gmane.org>
References: <D7E23A6D-C91C-46D8-A8FE-1EB16F3B2101@lericson.se>
	<200911130038.05516.steve@pearwood.info>
	<ca471dc20911120802r4c765f78h669e56335a38f11c@mail.gmail.com>
	<4222a8490911120831y5233e547pbfd22de60bcf436e@mail.gmail.com>
	<7AF4743F-4602-4E4C-8B8F-2FBC34D3945F@masklinn.net>
	<87vdhfwis8.fsf@benfinney.id.au> <4AFC8842.5050904@v.loewis.de>
	<loom.20091113T000003-485@post.gmane.org>
	<805B874E-6FD5-44DB-A908-922C93866458@masklinn.net>
	<loom.20091113T001402-725@post.gmane.org>
Message-ID: <50862ebd0911121532l3fb20c60p5672ab1747dea491@mail.gmail.com>

   When SourceForge started having comments and ratings, I was a
little upset at having poor negative comments there (like "not
work!"). But after it has been running for a while it appears useful.
I suppose it helps that Scintilla has 88% thumbs up from 134
respondents. Because there is voting on comments, the more useful
comments have bubbled onto the front page.

   As the system is used more, you'll see a wider range of comments on
projects and you'll be able to tell more from them. It should be seen
as a completely separate thing to the existing fora and trackers that
each project has. While you want people to become involved in your
project, many are just having a quick look and don't want to sign up
for mailing lists or to interact with project members. They may just
want to quickly comment about whether it was suitable or not.

   Neil

From martin at v.loewis.de  Fri Nov 13 00:33:46 2009
From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=)
Date: Fri, 13 Nov 2009 00:33:46 +0100
Subject: [Python-Dev] PyPI comments and ratings, *really*?
In-Reply-To: <loom.20091113T000003-485@post.gmane.org>
References: <D7E23A6D-C91C-46D8-A8FE-1EB16F3B2101@lericson.se>	<200911130038.05516.steve@pearwood.info>	<ca471dc20911120802r4c765f78h669e56335a38f11c@mail.gmail.com>	<4222a8490911120831y5233e547pbfd22de60bcf436e@mail.gmail.com>	<7AF4743F-4602-4E4C-8B8F-2FBC34D3945F@masklinn.net>	<87vdhfwis8.fsf@benfinney.id.au>
	<4AFC8842.5050904@v.loewis.de>
	<loom.20091113T000003-485@post.gmane.org>
Message-ID: <4AFC9B5A.2070406@v.loewis.de>

Antoine Pitrou wrote:
> Martin v. L?wis <martin <at> v.loewis.de> writes:
>> I think you are missing the point of the commenting system: these
>> comments are *not* directed towards the package author. Instead, they
>> are directed towards fellow users of the package. For this kind of
>> message, a bugtracker is completely inappropriate, as is a mailing list,
>> or any other kind of support forum.
> 
> Then why not simply add a sentence or two before the comment form warning that
> the comment system is not meant to ask for help, support or debugging about the
> package?

Done!

Martin

From jnoller at gmail.com  Fri Nov 13 00:34:17 2009
From: jnoller at gmail.com (Jesse Noller)
Date: Thu, 12 Nov 2009 18:34:17 -0500
Subject: [Python-Dev] PyPI comments and ratings, *really*?
In-Reply-To: <4AFC9975.7040204@v.loewis.de>
References: <D7E23A6D-C91C-46D8-A8FE-1EB16F3B2101@lericson.se>
	<B10AA20C-1C2F-44F0-B8B5-64F26691CC83@python.org>
	<hdhk7l$tru$1@ger.gmane.org>
	<ca471dc20911121054t76eb5bfbsbd6a9e47a9e71d8a@mail.gmail.com>
	<hdhrvs$qs8$1@ger.gmane.org>
	<Pine.LNX.4.64.0911121648480.12274@kimball.webabinitio.net>
	<hdi17l$cli$1@ger.gmane.org> <4AFC9064.20700@v.loewis.de>
	<4222a8490911121511s40eba937n2c7b1fde6f695c6c@mail.gmail.com>
	<4AFC9975.7040204@v.loewis.de>
Message-ID: <4222a8490911121534h30572eabv73ae58649c77155c@mail.gmail.com>

On Thu, Nov 12, 2009 at 6:25 PM, "Martin v. L?wis" <martin at v.loewis.de> wrote:
>> And how many of the "good" comments are astroturfers?
>
> If I understand that term correctly, it's about disguise: how would
> I be able to answer that question?

It's unprovable. But I could see a group of people easily coordinating
large amounts of negative, or positive feedback targeting particular
packages, that looks legit.

I know any "end user" rating and feedback system can be gamed. Just
look at the reviews of milk on amazon.

>> What's so bad about package maintainers from having an opt-out?
>
> PyPI is not just (and perhaps not even primarily) there for the package
> authors, but for the package users (and not surprisingly, it's
> primarily the package authors who ask for banning the user opinions).
>
> I'm just not willing to submit to one side; hence the poll.

That's because as an author/maintainer - we have methods of giving
feedback and communication. Why not rate ( or auto-rate) packages on
objective criteria?

E.g.: tests and test coverage, docs, installs on python version X, Y,
Z, works on windows, etc?

Quality can be measured. Me being a total failure and not reading the
docs, and failing to install something is subjective. I don't disagree
with the goal of giving *users* a voice, but is PyPI the right place
for that? How many moderators do we have to watch comments? Can other
users down vote comments by people which are simply *wrong*?

Why can't we just disable it until we can come up with a better system
that finds a balance between the rights of maintainers, and those of
the user?

From solipsis at pitrou.net  Fri Nov 13 00:35:28 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Thu, 12 Nov 2009 23:35:28 +0000 (UTC)
Subject: [Python-Dev] PyPI comments and ratings, *really*?
References: <D7E23A6D-C91C-46D8-A8FE-1EB16F3B2101@lericson.se>	<200911130038.05516.steve@pearwood.info>	<ca471dc20911120802r4c765f78h669e56335a38f11c@mail.gmail.com>	<4222a8490911120831y5233e547pbfd22de60bcf436e@mail.gmail.com>	<7AF4743F-4602-4E4C-8B8F-2FBC34D3945F@masklinn.net>
	<87vdhfwis8.fsf@benfinney.id.au> <4AFC8842.5050904@v.loewis.de>
	<loom.20091113T000003-485@post.gmane.org>
	<805B874E-6FD5-44DB-A908-922C93866458@masklinn.net>
	<loom.20091113T001402-725@post.gmane.org>
	<FBB5BF49-8EE6-4407-B733-D929A9385457@masklinn.net>
Message-ID: <loom.20091113T003433-735@post.gmane.org>

Masklinn <masklinn <at> masklinn.net> writes:
> 
> And then user will probably ask why you're not answering the question since
> you're here anyway, or might go
> as far as telling you that if you're not going to help you might as well not
> answer.

As I said, you are regarding the user as an idiot or as a troll.



From martin at v.loewis.de  Fri Nov 13 00:37:16 2009
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Fri, 13 Nov 2009 00:37:16 +0100
Subject: [Python-Dev] PyPI comments and ratings, *really*?
In-Reply-To: <FBB5BF49-8EE6-4407-B733-D929A9385457@masklinn.net>
References: <D7E23A6D-C91C-46D8-A8FE-1EB16F3B2101@lericson.se>	<200911130038.05516.steve@pearwood.info>	<ca471dc20911120802r4c765f78h669e56335a38f11c@mail.gmail.com>	<4222a8490911120831y5233e547pbfd22de60bcf436e@mail.gmail.com>	<7AF4743F-4602-4E4C-8B8F-2FBC34D3945F@masklinn.net>	<87vdhfwis8.fsf@benfinney.id.au>
	<4AFC8842.5050904@v.loewis.de>	<loom.20091113T000003-485@post.gmane.org>	<805B874E-6FD5-44DB-A908-922C93866458@masklinn.net>	<loom.20091113T001402-725@post.gmane.org>
	<FBB5BF49-8EE6-4407-B733-D929A9385457@masklinn.net>
Message-ID: <4AFC9C2C.9090408@v.loewis.de>

> Users (which includes e.g. language users) tend to be lazy, rather than stupid.

Then they likely won't comment on PyPI. To do so, they have to setup an
account (which most don't have). They can't post comments without an
account.

Regards,
Martin

From jacob at jacobian.org  Fri Nov 13 00:37:30 2009
From: jacob at jacobian.org (Jacob Kaplan-Moss)
Date: Thu, 12 Nov 2009 17:37:30 -0600
Subject: [Python-Dev] PyPI comments and ratings, *really*?
In-Reply-To: <4AFC9975.7040204@v.loewis.de>
References: <D7E23A6D-C91C-46D8-A8FE-1EB16F3B2101@lericson.se> 
	<B10AA20C-1C2F-44F0-B8B5-64F26691CC83@python.org>
	<hdhk7l$tru$1@ger.gmane.org> 
	<ca471dc20911121054t76eb5bfbsbd6a9e47a9e71d8a@mail.gmail.com> 
	<hdhrvs$qs8$1@ger.gmane.org>
	<Pine.LNX.4.64.0911121648480.12274@kimball.webabinitio.net> 
	<hdi17l$cli$1@ger.gmane.org> <4AFC9064.20700@v.loewis.de> 
	<4222a8490911121511s40eba937n2c7b1fde6f695c6c@mail.gmail.com> 
	<4AFC9975.7040204@v.loewis.de>
Message-ID: <bc8ec34e0911121537k510c26f8gfe8b54b94953aeed@mail.gmail.com>

On Thu, Nov 12, 2009 at 5:25 PM, "Martin v. L?wis" <martin at v.loewis.de> wrote:
> I'm just not willing to submit to one side; hence the poll.

Nobody's asking you to "submit" to anything! We're asking for the
control to decide ourselves.

Look, there's already a large faction of people who just want to write
off PyPI and launch our own package server instead. I'm nearly on
board, but we've had enough fragmentation in the packaging world
lately, and I don't want to make the project worse.

But you can bet your ass that if PyPI isn't made a good, neutral,
central resource I'm going to leave for one that is. Do you really
want a flood of package maintainers de-listing their packages just so
that things work the way you think they should?

I should clarify that I'm speaking personally and not in any official
"Django capacity." I don't have personal control over whether or not
Django would de-list from PyPI. Django's run by a community process,
and I'd listen to the voice of the community before doing anything
unilaterally. It's a good idea, this community process. We might want
to apply it to PyPI one of these days.

Jacob

From martin at v.loewis.de  Fri Nov 13 00:41:00 2009
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Fri, 13 Nov 2009 00:41:00 +0100
Subject: [Python-Dev] PyPI comments and ratings, *really*?
In-Reply-To: <4222a8490911121534h30572eabv73ae58649c77155c@mail.gmail.com>
References: <D7E23A6D-C91C-46D8-A8FE-1EB16F3B2101@lericson.se>	
	<B10AA20C-1C2F-44F0-B8B5-64F26691CC83@python.org>	
	<hdhk7l$tru$1@ger.gmane.org>	
	<ca471dc20911121054t76eb5bfbsbd6a9e47a9e71d8a@mail.gmail.com>	
	<hdhrvs$qs8$1@ger.gmane.org>	
	<Pine.LNX.4.64.0911121648480.12274@kimball.webabinitio.net>	
	<hdi17l$cli$1@ger.gmane.org> <4AFC9064.20700@v.loewis.de>	
	<4222a8490911121511s40eba937n2c7b1fde6f695c6c@mail.gmail.com>	
	<4AFC9975.7040204@v.loewis.de>
	<4222a8490911121534h30572eabv73ae58649c77155c@mail.gmail.com>
Message-ID: <4AFC9D0C.5030500@v.loewis.de>

> Why can't we just disable it until we can come up with a better system
> that finds a balance between the rights of maintainers, and those of
> the user?

Because I want to wait for the outcome of the poll first.

Regards,
Martin


From martin at v.loewis.de  Fri Nov 13 00:42:32 2009
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Fri, 13 Nov 2009 00:42:32 +0100
Subject: [Python-Dev] PyPI comments and ratings, *really*?
In-Reply-To: <bc8ec34e0911121537k510c26f8gfe8b54b94953aeed@mail.gmail.com>
References: <D7E23A6D-C91C-46D8-A8FE-1EB16F3B2101@lericson.se>
	<B10AA20C-1C2F-44F0-B8B5-64F26691CC83@python.org>
	<hdhk7l$tru$1@ger.gmane.org>
	<ca471dc20911121054t76eb5bfbsbd6a9e47a9e71d8a@mail.gmail.com>
	<hdhrvs$qs8$1@ger.gmane.org>
	<Pine.LNX.4.64.0911121648480.12274@kimball.webabinitio.net>
	<hdi17l$cli$1@ger.gmane.org> <4AFC9064.20700@v.loewis.de>
	<4222a8490911121511s40eba937n2c7b1fde6f695c6c@mail.gmail.com>
	<4AFC9975.7040204@v.loewis.de>
	<bc8ec34e0911121537k510c26f8gfe8b54b94953aeed@mail.gmail.com>
Message-ID: <4AFC9D68.9010904@v.loewis.de>

> But you can bet your ass that if PyPI isn't made a good, neutral,
> central resource I'm going to leave for one that is. Do you really
> want a flood of package maintainers de-listing their packages just so
> that things work the way you think they should?
> 
> I should clarify that I'm speaking personally and not in any official
> "Django capacity." I don't have personal control over whether or not
> Django would de-list from PyPI. Django's run by a community process,
> and I'd listen to the voice of the community before doing anything
> unilaterally. It's a good idea, this community process. We might want
> to apply it to PyPI one of these days.

And indeed, I do: feel free to participate in the poll.

Regards,
Martin


From catch-all at masklinn.net  Fri Nov 13 00:44:30 2009
From: catch-all at masklinn.net (Xavier Morel)
Date: Fri, 13 Nov 2009 00:44:30 +0100
Subject: [Python-Dev] PyPI comments and ratings, *really*?
In-Reply-To: <4222a8490911121534h30572eabv73ae58649c77155c@mail.gmail.com>
References: <D7E23A6D-C91C-46D8-A8FE-1EB16F3B2101@lericson.se>
	<B10AA20C-1C2F-44F0-B8B5-64F26691CC83@python.org>
	<hdhk7l$tru$1@ger.gmane.org>
	<ca471dc20911121054t76eb5bfbsbd6a9e47a9e71d8a@mail.gmail.com>
	<hdhrvs$qs8$1@ger.gmane.org>
	<Pine.LNX.4.64.0911121648480.12274@kimball.webabinitio.net>
	<hdi17l$cli$1@ger.gmane.org> <4AFC9064.20700@v.loewis.de>
	<4222a8490911121511s40eba937n2c7b1fde6f695c6c@mail.gmail.com>
	<4AFC9975.7040204@v.loewis.de>
	<4222a8490911121534h30572eabv73ae58649c77155c@mail.gmail.com>
Message-ID: <7C2A33E9-7E5A-4588-BA87-DBB0F8F8EF0F@masklinn.net>

On 13 Nov 2009, at 00:34 , Jesse Noller wrote:
> That's because as an author/maintainer - we have methods of giving
> feedback and communication. Why not rate ( or auto-rate) packages on
> objective criteria?
> 
> E.g.: tests and test coverage, docs, installs on python version X, Y,
> Z, works on windows, etc?
Because there are lots of subjective criteria which are still very useful to users? The feeling of the API, the completeness of the library or its flexibility, etc??

If pypi one day has a CPAN-style buildbot farm allowing it to test the package on any platform under the sun, that can be included, the tests can be included as well but given the number of testing solutions (and coverage discovery associated) that would be quite an undertaking.

And as far as docs go, what would be the criterion? "Has documentation"? How do you define "has documentation"? Has auto-extracted documentation from the docstrings? Has a README? Has a complete sphinx package? I don't think there's much that you can rate "objectively" about documentation.


From catch-all at masklinn.net  Fri Nov 13 00:45:40 2009
From: catch-all at masklinn.net (Xavier Morel)
Date: Fri, 13 Nov 2009 00:45:40 +0100
Subject: [Python-Dev] PyPI comments and ratings, *really*?
In-Reply-To: <loom.20091113T003433-735@post.gmane.org>
References: <D7E23A6D-C91C-46D8-A8FE-1EB16F3B2101@lericson.se>	<200911130038.05516.steve@pearwood.info>	<ca471dc20911120802r4c765f78h669e56335a38f11c@mail.gmail.com>	<4222a8490911120831y5233e547pbfd22de60bcf436e@mail.gmail.com>	<7AF4743F-4602-4E4C-8B8F-2FBC34D3945F@masklinn.net>
	<87vdhfwis8.fsf@benfinney.id.au> <4AFC8842.5050904@v.loewis.de>
	<loom.20091113T000003-485@post.gmane.org>
	<805B874E-6FD5-44DB-A908-922C93866458@masklinn.net>
	<loom.20091113T001402-725@post.gmane.org>
	<FBB5BF49-8EE6-4407-B733-D929A9385457@masklinn.net>
	<loom.20091113T003433-735@post.gmane.org>
Message-ID: <C7578E12-5C3D-452E-846A-7C8D1280D4F1@masklinn.net>

On 13 Nov 2009, at 00:35 , Antoine Pitrou wrote:
> Masklinn <masklinn <at> masklinn.net> writes:
>> 
>> And then user will probably ask why you're not answering the question since
>> you're here anyway, or might go
>> as far as telling you that if you're not going to help you might as well not
>> answer.
> As I said, you are regarding the user as an idiot or as a troll.
I don't think so, but we might disagree on either definition, or both.

From masklinn at masklinn.net  Fri Nov 13 00:46:08 2009
From: masklinn at masklinn.net (Masklinn)
Date: Fri, 13 Nov 2009 00:46:08 +0100
Subject: [Python-Dev] PyPI comments and ratings, *really*?
In-Reply-To: <4AFC9C2C.9090408@v.loewis.de>
References: <D7E23A6D-C91C-46D8-A8FE-1EB16F3B2101@lericson.se>	<200911130038.05516.steve@pearwood.info>	<ca471dc20911120802r4c765f78h669e56335a38f11c@mail.gmail.com>	<4222a8490911120831y5233e547pbfd22de60bcf436e@mail.gmail.com>	<7AF4743F-4602-4E4C-8B8F-2FBC34D3945F@masklinn.net>	<87vdhfwis8.fsf@benfinney.id.au>
	<4AFC8842.5050904@v.loewis.de>	<loom.20091113T000003-485@post.gmane.org>	<805B874E-6FD5-44DB-A908-922C93866458@masklinn.net>	<loom.20091113T001402-725@post.gmane.org>
	<FBB5BF49-8EE6-4407-B733-D929A9385457@masklinn.net>
	<4AFC9C2C.9090408@v.loewis.de>
Message-ID: <E0C3E560-159E-4377-A6D3-1A120F7FDE34@masklinn.net>

On 13 Nov 2009, at 00:37 , Martin v. L?wis wrote:
>> Users (which includes e.g. language users) tend to be lazy, rather than stupid.
> Then they likely won't comment on PyPI. To do so, they have to setup an
> account (which most don't have). They can't post comments without an
> account.
Fair point

From ben+python at benfinney.id.au  Fri Nov 13 00:54:50 2009
From: ben+python at benfinney.id.au (Ben Finney)
Date: Fri, 13 Nov 2009 10:54:50 +1100
Subject: [Python-Dev] PyPI comments and ratings, *really*?
References: <D7E23A6D-C91C-46D8-A8FE-1EB16F3B2101@lericson.se>
	<200911130038.05516.steve@pearwood.info>
	<4222a8490911120606t3e138957kcdb21104261a1ead@mail.gmail.com>
	<B10AA20C-1C2F-44F0-B8B5-64F26691CC83@python.org>
	<hdhk7l$tru$1@ger.gmane.org>
	<ca471dc20911121054t76eb5bfbsbd6a9e47a9e71d8a@mail.gmail.com>
	<hdhrvs$qs8$1@ger.gmane.org>
	<Pine.LNX.4.64.0911121648480.12274@kimball.webabinitio.net>
	<hdi17l$cli$1@ger.gmane.org> <4AFC9064.20700@v.loewis.de>
	<4222a8490911121511s40eba937n2c7b1fde6f695c6c@mail.gmail.com>
	<4AFC9975.7040204@v.loewis.de>
Message-ID: <87aayrwb79.fsf@benfinney.id.au>

"Martin v. L?wis" <martin at v.loewis.de> writes:

> PyPI is not just (and perhaps not even primarily) there for the
> package authors, but for the package users (and not surprisingly, it's
> primarily the package authors who ask for banning the user opinions).

No-one here is asking for ?banning the user opinions?. As already
pointed out, users are not mute; there are plenty of places for them to
have a voice.

I understand the mood here to be, not that user feedback is not wanted,
but rather that PyPI in particular should be a place for *objective*
data about a package.

> I'm just not willing to submit to one side; hence the poll.

I hope the poll question will not be as biased as the above
mischaracterisation. If you ask ?do you want feedback from users?? or
something similar, that's missing the point entirely, and a ?yes? answer
doesn't speak to this discussion at all.

-- 
 \        ?Always code as if the guy who ends up maintaining your code |
  `\     will be a violent psychopath who knows where you live.? ?John |
_o__)                                                         F. Woods |
Ben Finney


From ben+python at benfinney.id.au  Fri Nov 13 01:00:00 2009
From: ben+python at benfinney.id.au (Ben Finney)
Date: Fri, 13 Nov 2009 11:00:00 +1100
Subject: [Python-Dev] PyPI comments and ratings, *really*?
References: <D7E23A6D-C91C-46D8-A8FE-1EB16F3B2101@lericson.se>
	<200911130038.05516.steve@pearwood.info>
	<ca471dc20911120802r4c765f78h669e56335a38f11c@mail.gmail.com>
	<4222a8490911120831y5233e547pbfd22de60bcf436e@mail.gmail.com>
	<7AF4743F-4602-4E4C-8B8F-2FBC34D3945F@masklinn.net>
	<87vdhfwis8.fsf@benfinney.id.au> <4AFC8842.5050904@v.loewis.de>
	<loom.20091113T000003-485@post.gmane.org>
	<805B874E-6FD5-44DB-A908-922C93866458@masklinn.net>
	<loom.20091113T001402-725@post.gmane.org>
	<FBB5BF49-8EE6-4407-B733-D929A9385457@masklinn.net>
Message-ID: <87639fwayn.fsf@benfinney.id.au>

Masklinn <masklinn at masklinn.net> writes:

> Users (which includes e.g. language users) tend to be lazy, rather
> than stupid.

I've found it useful to realise that, from the perspective of a
program/website/feedback form, etc., the user has a tiny brain: but
that's only because the user's big brain is *not* solely dedicated to
the program/website/feedback form, etc.

When designing a UI, one must realise that, though users are generally
possessed of big brains, only a *tiny* portion of that brain can be
assumed to be available for the UI; the rest is focussed on stuff the
user actually cares about at the time.

Assuming the user is stupid or lazy is bad, since it's false most of the
time. Assuming that they're only willing to put forth as much attention
or effort as absolutely necessary to complete the task, is *good*, since
that turns out to be true most of the time.

-- 
 \       ?It's easy to play any musical instrument: all you have to do |
  `\       is touch the right key at the right time and the instrument |
_o__)                        will play itself.? ?Johann Sebastian Bach |
Ben Finney


From david.lyon at preisshare.net  Fri Nov 13 01:01:53 2009
From: david.lyon at preisshare.net (David Lyon)
Date: Thu, 12 Nov 2009 19:01:53 -0500
Subject: [Python-Dev]
	=?utf-8?q?PyPI_comments_and_ratings=2C_*really*=3F?=
In-Reply-To: <7C2A33E9-7E5A-4588-BA87-DBB0F8F8EF0F@masklinn.net>
References: <D7E23A6D-C91C-46D8-A8FE-1EB16F3B2101@lericson.se>
	<B10AA20C-1C2F-44F0-B8B5-64F26691CC83@python.org>
	<hdhk7l$tru$1@ger.gmane.org>
	<ca471dc20911121054t76eb5bfbsbd6a9e47a9e71d8a@mail.gmail.com>
	<hdhrvs$qs8$1@ger.gmane.org>
	<Pine.LNX.4.64.0911121648480.12274@kimball.webabinitio.net>
	<hdi17l$cli$1@ger.gmane.org> <4AFC9064.20700@v.loewis.de>
	<4222a8490911121511s40eba937n2c7b1fde6f695c6c@mail.gmail.com>
	<4AFC9975.7040204@v.loewis.de>
	<4222a8490911121534h30572eabv73ae58649c77155c@mail.gmail.com>
	<7C2A33E9-7E5A-4588-BA87-DBB0F8F8EF0F@masklinn.net>
Message-ID: <1b4a243d79db54095040c7e068414474@preisshare.net>

On Fri, 13 Nov 2009 00:44:30 +0100, Xavier Morel <catch-all at masklinn.net>
wrote:
> If pypi one day has a CPAN-style buildbot farm allowing it to test the
> package on any platform under the sun, that can be included, the tests
can
> be included as well but given the number of testing solutions (and
coverage
> discovery associated) that would be quite an undertaking.

I'm working on such a thing in my spare time. Yep, it's a big time
commitment.

http://bitbucket.org/djlyon/pypi-package-testbot/

> And as far as docs go, what would be the criterion?  "Has documentation"?

Yes.

> How do you define "has documentation"? Has auto-extracted documentation
> from the docstrings? 

Yes.

> Has a README? 

Yes

Has a complete sphinx package? I don't think there's much that you can 
rate "objectively" about documentation.

You can't do it objectively, but you can use a computer to count the
number of lines and come up with a score.

Daivd

From martin at v.loewis.de  Fri Nov 13 01:14:54 2009
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Fri, 13 Nov 2009 01:14:54 +0100
Subject: [Python-Dev] PyPI comments and ratings, *really*?
In-Reply-To: <1b4a243d79db54095040c7e068414474@preisshare.net>
References: <D7E23A6D-C91C-46D8-A8FE-1EB16F3B2101@lericson.se>	<B10AA20C-1C2F-44F0-B8B5-64F26691CC83@python.org>	<hdhk7l$tru$1@ger.gmane.org>	<ca471dc20911121054t76eb5bfbsbd6a9e47a9e71d8a@mail.gmail.com>	<hdhrvs$qs8$1@ger.gmane.org>	<Pine.LNX.4.64.0911121648480.12274@kimball.webabinitio.net>	<hdi17l$cli$1@ger.gmane.org>
	<4AFC9064.20700@v.loewis.de>	<4222a8490911121511s40eba937n2c7b1fde6f695c6c@mail.gmail.com>	<4AFC9975.7040204@v.loewis.de>	<4222a8490911121534h30572eabv73ae58649c77155c@mail.gmail.com>	<7C2A33E9-7E5A-4588-BA87-DBB0F8F8EF0F@masklinn.net>
	<1b4a243d79db54095040c7e068414474@preisshare.net>
Message-ID: <4AFCA4FE.3090106@v.loewis.de>

David Lyon wrote:
> On Fri, 13 Nov 2009 00:44:30 +0100, Xavier Morel <catch-all at masklinn.net>
> wrote:
>> If pypi one day has a CPAN-style buildbot farm allowing it to test the
>> package on any platform under the sun, that can be included, the tests
> can
>> be included as well but given the number of testing solutions (and
> coverage
>> discovery associated) that would be quite an undertaking.
> 
> I'm working on such a thing in my spare time. Yep, it's a big time
> commitment.
> 
> http://bitbucket.org/djlyon/pypi-package-testbot/

See also

http://pycheesecake.org/

Regards,
Martin

From guido at python.org  Fri Nov 13 01:17:50 2009
From: guido at python.org (Guido van Rossum)
Date: Thu, 12 Nov 2009 16:17:50 -0800
Subject: [Python-Dev] PyPI comments and ratings, *really*?
In-Reply-To: <87639fwayn.fsf@benfinney.id.au>
References: <D7E23A6D-C91C-46D8-A8FE-1EB16F3B2101@lericson.se> 
	<4222a8490911120831y5233e547pbfd22de60bcf436e@mail.gmail.com> 
	<7AF4743F-4602-4E4C-8B8F-2FBC34D3945F@masklinn.net>
	<87vdhfwis8.fsf@benfinney.id.au> <4AFC8842.5050904@v.loewis.de>
	<loom.20091113T000003-485@post.gmane.org> 
	<805B874E-6FD5-44DB-A908-922C93866458@masklinn.net>
	<loom.20091113T001402-725@post.gmane.org> 
	<FBB5BF49-8EE6-4407-B733-D929A9385457@masklinn.net>
	<87639fwayn.fsf@benfinney.id.au>
Message-ID: <ca471dc20911121617r306fbfddq35c195f5257275a0@mail.gmail.com>

On Thu, Nov 12, 2009 at 4:00 PM, Ben Finney <ben+python at benfinney.id.au> wrote:
> I've found it useful to realise that, from the perspective of a
> program/website/feedback form, etc., the user has a tiny brain: [...]

Actually it's the other way around. It's the program that has the tiny
brain. :-)

-- 
--Guido van Rossum (python.org/~guido)

From david.lyon at preisshare.net  Fri Nov 13 01:13:14 2009
From: david.lyon at preisshare.net (David Lyon)
Date: Thu, 12 Nov 2009 19:13:14 -0500
Subject: [Python-Dev]
	=?utf-8?q?PyPI_comments_and_ratings=2C_*really*=3F?=
In-Reply-To: <4222a8490911121534h30572eabv73ae58649c77155c@mail.gmail.com>
References: <D7E23A6D-C91C-46D8-A8FE-1EB16F3B2101@lericson.se>
	<B10AA20C-1C2F-44F0-B8B5-64F26691CC83@python.org>
	<hdhk7l$tru$1@ger.gmane.org>
	<ca471dc20911121054t76eb5bfbsbd6a9e47a9e71d8a@mail.gmail.com>
	<hdhrvs$qs8$1@ger.gmane.org>
	<Pine.LNX.4.64.0911121648480.12274@kimball.webabinitio.net>
	<hdi17l$cli$1@ger.gmane.org> <4AFC9064.20700@v.loewis.de>
	<4222a8490911121511s40eba937n2c7b1fde6f695c6c@mail.gmail.com>
	<4AFC9975.7040204@v.loewis.de>
	<4222a8490911121534h30572eabv73ae58649c77155c@mail.gmail.com>
Message-ID: <3c0a8eeb1d731b5c7f05c841442cadc1@preisshare.net>

Hi All,

What do people think about this idea? I've actually started writing
something to try to to do this and create sn automated scoring system
for the packages on pypi.

It was started last week based on Guido's comments on the distutils
mailing list.

> Why not rate ( or auto-rate) packages on
> objective criteria?
> 
> E.g.: tests and test coverage, docs, installs on python version X, Y,
> Z, works on windows, etc?
> 
> Quality can be measured. Me being a total failure and not reading the
> docs, and failing to install something is subjective. I don't disagree
> with the goal of giving *users* a voice, but is PyPI the right place
> for that? How many moderators do we have to watch comments? Can other
> users down vote comments by people which are simply *wrong*?
> 
> Why can't we just disable it until we can come up with a better system
> that finds a balance between the rights of maintainers, and those of
> the user?


From david.lyon at preisshare.net  Fri Nov 13 01:15:57 2009
From: david.lyon at preisshare.net (David Lyon)
Date: Thu, 12 Nov 2009 19:15:57 -0500
Subject: [Python-Dev]
	=?utf-8?q?PyPI_comments_and_ratings=2C_*really*=3F?=
In-Reply-To: <4AFCA4FE.3090106@v.loewis.de>
References: <D7E23A6D-C91C-46D8-A8FE-1EB16F3B2101@lericson.se>	<B10AA20C-1C2F-44F0-B8B5-64F26691CC83@python.org>	<hdhk7l$tru$1@ger.gmane.org>	<ca471dc20911121054t76eb5bfbsbd6a9e47a9e71d8a@mail.gmail.com>	<hdhrvs$qs8$1@ger.gmane.org>	<Pine.LNX.4.64.0911121648480.12274@kimball.webabinitio.net>	<hdi17l$cli$1@ger.gmane.org>
	<4AFC9064.20700@v.loewis.de>	<4222a8490911121511s40eba937n2c7b1fde6f695c6c@mail.gmail.com>	<4AFC9975.7040204@v.loewis.de>	<4222a8490911121534h30572eabv73ae58649c77155c@mail.gmail.com>	<7C2A33E9-7E5A-4588-BA87-DBB0F8F8EF0F@masklinn.net>
	<1b4a243d79db54095040c7e068414474@preisshare.net>
	<4AFCA4FE.3090106@v.loewis.de>
Message-ID: <5b8b56c7fd501914bec8f92f18cabebe@preisshare.net>

On Fri, 13 Nov 2009 01:14:54 +0100, "Martin v. L?wis" <martin at v.loewis.de>
wrote:

> http://pycheesecake.org/

Ok, so what is the current status on it?

David

From martin at v.loewis.de  Fri Nov 13 01:27:47 2009
From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=)
Date: Fri, 13 Nov 2009 01:27:47 +0100
Subject: [Python-Dev] PyPI comments and ratings, *really*?
In-Reply-To: <5b8b56c7fd501914bec8f92f18cabebe@preisshare.net>
References: <D7E23A6D-C91C-46D8-A8FE-1EB16F3B2101@lericson.se>	<B10AA20C-1C2F-44F0-B8B5-64F26691CC83@python.org>	<hdhk7l$tru$1@ger.gmane.org>	<ca471dc20911121054t76eb5bfbsbd6a9e47a9e71d8a@mail.gmail.com>	<hdhrvs$qs8$1@ger.gmane.org>	<Pine.LNX.4.64.0911121648480.12274@kimball.webabinitio.net>	<hdi17l$cli$1@ger.gmane.org>
	<4AFC9064.20700@v.loewis.de>	<4222a8490911121511s40eba937n2c7b1fde6f695c6c@mail.gmail.com>	<4AFC9975.7040204@v.loewis.de>	<4222a8490911121534h30572eabv73ae58649c77155c@mail.gmail.com>	<7C2A33E9-7E5A-4588-BA87-DBB0F8F8EF0F@masklinn.net>
	<1b4a243d79db54095040c7e068414474@preisshare.net>
	<4AFCA4FE.3090106@v.loewis.de>
	<5b8b56c7fd501914bec8f92f18cabebe@preisshare.net>
Message-ID: <4AFCA803.6040408@v.loewis.de>

David Lyon wrote:
> On Fri, 13 Nov 2009 01:14:54 +0100, "Martin v. L?wis" <martin at v.loewis.de>
> wrote:
> 
>> http://pycheesecake.org/
> 
> Ok, so what is the current status on it?

Not sure; you would have to ask Grig. Apparently, there is a service
running somewhere that computes cheesecake data for PyPI packages;
it also sends them to PyPI. People have expressed to concerns that any
kind of ranking based on kwalitee sounds fairly useless.

Regards,
Martin

From ben+python at benfinney.id.au  Fri Nov 13 01:44:42 2009
From: ben+python at benfinney.id.au (Ben Finney)
Date: Fri, 13 Nov 2009 11:44:42 +1100
Subject: [Python-Dev] PyPI comments and ratings, *really*?
References: <D7E23A6D-C91C-46D8-A8FE-1EB16F3B2101@lericson.se>
	<B10AA20C-1C2F-44F0-B8B5-64F26691CC83@python.org>
	<hdhk7l$tru$1@ger.gmane.org>
	<ca471dc20911121054t76eb5bfbsbd6a9e47a9e71d8a@mail.gmail.com>
	<hdhrvs$qs8$1@ger.gmane.org>
	<Pine.LNX.4.64.0911121648480.12274@kimball.webabinitio.net>
	<hdi17l$cli$1@ger.gmane.org> <4AFC9064.20700@v.loewis.de>
	<4222a8490911121511s40eba937n2c7b1fde6f695c6c@mail.gmail.com>
	<4AFC9975.7040204@v.loewis.de>
	<4222a8490911121534h30572eabv73ae58649c77155c@mail.gmail.com>
	<4AFC9D0C.5030500@v.loewis.de>
Message-ID: <87y6mbuubp.fsf@benfinney.id.au>

"Martin v. L?wis" <martin at v.loewis.de> writes:

> > Why can't we just disable it until we can come up with a better
> > system that finds a balance between the rights of maintainers, and
> > those of the user?
>
> Because I want to wait for the outcome of the poll first.

There's a problem with the poll's placement: on the front page of the
PyPI website.

I only know about the poll because you said there was one, and I went
hunting for it. The front page of PyPI is not one I ever visit, as a
package maintainer; I'll visit the pages for the packages I maintain, or
make a specific search of packages I'm looking for.

So, the poll's audience is limited to those who visit the front page
(which is hardly ever necessary for package maintainers), and those who
already know it exists (e.g. through this discussion thread). You'll be
missing the opinions of those maintainers who, like the OP of this
thread, only discovered the behaviour much later.

-- 
 \           ?I do not believe in forgiveness as it is preached by the |
  `\        church. We do not need the forgiveness of God, but of each |
_o__)                    other and of ourselves.? ?Robert G. Ingersoll |
Ben Finney


From sparks.m at gmail.com  Fri Nov 13 01:52:44 2009
From: sparks.m at gmail.com (Michael Sparks)
Date: Fri, 13 Nov 2009 00:52:44 +0000
Subject: [Python-Dev] PyPI comments and ratings, *really*?
In-Reply-To: <4222a8490911120606t3e138957kcdb21104261a1ead@mail.gmail.com>
References: <D7E23A6D-C91C-46D8-A8FE-1EB16F3B2101@lericson.se>
	<200911130038.05516.steve@pearwood.info>
	<4222a8490911120606t3e138957kcdb21104261a1ead@mail.gmail.com>
Message-ID: <20e5c9660911121652s6c9c25caqf8856eb6c1685986@mail.gmail.com>

On Thu, Nov 12, 2009 at 2:06 PM, Jesse Noller <jnoller at gmail.com> wrote:
...
> Frankly, I agree with him. As implemented, I *and others* think this
> is broken. I've taken the stance of not publishing things to PyPi
> until A> I find the time to contribute to make it better or B> It
> changes.

Ditto, but maybe for different reasons.

Personally, I'm interested in feedback - good and bad. That's the
reason I choose odd names for projects, since it means I can create a
google alert to find out random comments in bizarre locations (hence
why when you wrote a blog entry, I responded). However, the reason I
released anything onto PyPI (and relunctantly at that) was due to a
random complaint that a user couldn't go "easy_install <foo>" and have
it pick up code from PyPI.

Going along with comments made elsewhere (by Guido I think) saying
"but user's like reviews and rating when someone publishes a book",
probably using Amazon, B&N & similar as examples, I agree they do.

The closest equivalent here though IMO is somewhere like lulu.com -
where people self-publish. Like PyPI that has a ratings system and
comments, so you could say if it "works" there it should work for
PyPI.

The problem though is that software is much more mutable that a book.

Taking the example listed - a comment added here:
  http://pypi.python.org/pypi/spypam/1.0

There's a note:
   "Inadequate docstrings give no clue about function arguments. Dumps
core when I call it after reading the source to figure out the API.
Cannot recommend."

That's useful from a user perspective. Or is it? It's useful from a
user perspective, until that issue is fixed. Then what? Is it still
useful? Can the commenter remove it? Can they get notified it's
changed? Can the maintainer say "this is fixed/changed?"

I never look at the PyPI pages for stuff I create. Which means if
someone is using it for support, they're wasting their time. (Why
would I look at it? I know what the project is for and where to get
it! :) (and also PyPI isn't the prime download for it either - so the
download stats are irrelevant to me) I doubt I'm alone, so how many
people's time are wasted asking questions there ?

*shrug*

I suppose, personally, I'm dubious about the idea of having unchanging
comments and ratings associated with projects which are changing and
improving - that feels like a mismatch. (Unlike a book, which
generally is unchanging or has a separate edition and separate set of
ratings and reviews)

Incidentally, and perhaps probably more relevant to the discussion
than my random opinion - some time back I created the twitter id
http://twitter.com/pypi - using twitterfeed - since I wanted an easier
way of following additions to pypi. There's currently 774 people
following that.

If there's interest, and if there's a survey to be done, I could
forward a link to a survey through that twitterfeed - which I suspect
is a mix of users of PyPI and uploaders to PyPI.

(On a secondary note, if there's someone else who thinks they should
own it, please let me know - it was a random convenience that people
seem to find useful :-)


Michael.

From solipsis at pitrou.net  Fri Nov 13 01:52:38 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Fri, 13 Nov 2009 00:52:38 +0000 (UTC)
Subject: [Python-Dev] PyPI front page
References: <D7E23A6D-C91C-46D8-A8FE-1EB16F3B2101@lericson.se>
	<B10AA20C-1C2F-44F0-B8B5-64F26691CC83@python.org>
	<hdhk7l$tru$1@ger.gmane.org>
	<ca471dc20911121054t76eb5bfbsbd6a9e47a9e71d8a@mail.gmail.com>
	<hdhrvs$qs8$1@ger.gmane.org>
	<Pine.LNX.4.64.0911121648480.12274@kimball.webabinitio.net>
	<hdi17l$cli$1@ger.gmane.org> <4AFC9064.20700@v.loewis.de>
	<4222a8490911121511s40eba937n2c7b1fde6f695c6c@mail.gmail.com>
	<4AFC9975.7040204@v.loewis.de>
	<4222a8490911121534h30572eabv73ae58649c77155c@mail.gmail.com>
	<4AFC9D0C.5030500@v.loewis.de> <87y6mbuubp.fsf@benfinney.id.au>
Message-ID: <loom.20091113T015130-137@post.gmane.org>

Ben Finney <ben+python <at> benfinney.id.au> writes:
> 
> There's a problem with the poll's placement: on the front page of the
> PyPI website.

Speaking of which, why is it that http://pypi.python.org/pypi and
http://pypi.python.org/pypi/ (note the ending slash) return different contents
(the latter being very voluminous)? I always mistake one for the other when
entering the URL directly.

cheers

Antoine.



From david.lyon at preisshare.net  Fri Nov 13 01:51:08 2009
From: david.lyon at preisshare.net (David Lyon)
Date: Thu, 12 Nov 2009 19:51:08 -0500
Subject: [Python-Dev]
	=?utf-8?q?PyPI_comments_and_ratings=2C_*really*=3F?=
In-Reply-To: <4AFCA803.6040408@v.loewis.de>
References: <D7E23A6D-C91C-46D8-A8FE-1EB16F3B2101@lericson.se>	<B10AA20C-1C2F-44F0-B8B5-64F26691CC83@python.org>	<hdhk7l$tru$1@ger.gmane.org>	<ca471dc20911121054t76eb5bfbsbd6a9e47a9e71d8a@mail.gmail.com>	<hdhrvs$qs8$1@ger.gmane.org>	<Pine.LNX.4.64.0911121648480.12274@kimball.webabinitio.net>	<hdi17l$cli$1@ger.gmane.org>
	<4AFC9064.20700@v.loewis.de>	<4222a8490911121511s40eba937n2c7b1fde6f695c6c@mail.gmail.com>	<4AFC9975.7040204@v.loewis.de>	<4222a8490911121534h30572eabv73ae58649c77155c@mail.gmail.com>	<7C2A33E9-7E5A-4588-BA87-DBB0F8F8EF0F@masklinn.net>
	<1b4a243d79db54095040c7e068414474@preisshare.net>
	<4AFCA4FE.3090106@v.loewis.de>
	<5b8b56c7fd501914bec8f92f18cabebe@preisshare.net>
	<4AFCA803.6040408@v.loewis.de>
Message-ID: <3df661fb01b68d13cc5bd01eaab5c935@preisshare.net>

On Fri, 13 Nov 2009 01:27:47 +0100, "Martin v. L?wis" <martin at v.loewis.de>
wrote:
> Not sure; you would have to ask Grig. Apparently, there is a service
> running somewhere that computes cheesecake data for PyPI packages;
> it also sends them to PyPI. People have expressed to concerns that any
> kind of ranking based on kwalitee sounds fairly useless.

Of course.

"Package Quality Metrics" would be a much better term.

Still, the ideas they had were good.

CPAN runs such a bot on all packages daily. Obviously they do it
on seperate machines.

Introducing any change is going to have people complain. Checking
package authors packages is much like a dentist check. It mightn't
be totally pleasant while its happening.

But then if it isn't done, a user can then reflect and ask why
nothing is being done to look at overall package quality. Which
is currently the case.

Processing so many packages for so many platforms is a monstrous 
task. Nobody should get the idea it can be done by the weekend.

It will take a few months... well at the rate I am going
anyway..

David






From sparks.m at gmail.com  Fri Nov 13 02:02:20 2009
From: sparks.m at gmail.com (Michael Sparks)
Date: Fri, 13 Nov 2009 01:02:20 +0000
Subject: [Python-Dev] PyPI comments and ratings, *really*?
In-Reply-To: <87y6mbuubp.fsf@benfinney.id.au>
References: <D7E23A6D-C91C-46D8-A8FE-1EB16F3B2101@lericson.se>
	<hdhrvs$qs8$1@ger.gmane.org>
	<Pine.LNX.4.64.0911121648480.12274@kimball.webabinitio.net>
	<hdi17l$cli$1@ger.gmane.org> <4AFC9064.20700@v.loewis.de>
	<4222a8490911121511s40eba937n2c7b1fde6f695c6c@mail.gmail.com>
	<4AFC9975.7040204@v.loewis.de>
	<4222a8490911121534h30572eabv73ae58649c77155c@mail.gmail.com>
	<4AFC9D0C.5030500@v.loewis.de> <87y6mbuubp.fsf@benfinney.id.au>
Message-ID: <20e5c9660911121702r162caf28jb6d656383147d909@mail.gmail.com>

On Fri, Nov 13, 2009 at 12:44 AM, Ben Finney <ben+python at benfinney.id.au> wrote:
> "Martin v. L?wis" <martin at v.loewis.de> writes:
>
>> > Why can't we just disable it until we can come up with a better
>> > system that finds a balance between the rights of maintainers, and
>> > those of the user?
>>
>> Because I want to wait for the outcome of the poll first.
>
> There's a problem with the poll's placement: on the front page of the
> PyPI website.
>
> I only know about the poll because you said there was one, and I went
> hunting for it. The front page of PyPI is not one I ever visit, as a
> package maintainer; I'll visit the pages for the packages I maintain, or
> make a specific search of packages I'm looking for.
>
> So, the poll's audience is limited to those who visit the front page
> (which is hardly ever necessary for package maintainers), and those who
> already know it exists (e.g. through this discussion thread). You'll be
> missing the opinions of those maintainers who, like the OP of this
> thread, only discovered the behaviour much later.

This poll is only visible if you're logged into PyPI. This strikes me
as a mistake. I went looking for a poll and didn't see it.

I only found the poll by accident by wondering randomly what might
change if I hit the login using open id button. So you can only vote
in the poll if you a) get told about it b) realise you need to create
an account to login and use in order to vote. I realise there's good
reasons for that, but I think it's a mistake. (There's no guidance
that you need to log in to see the poll for example)


Michael.

From amk at amk.ca  Fri Nov 13 02:18:12 2009
From: amk at amk.ca (A.M. Kuchling)
Date: Thu, 12 Nov 2009 20:18:12 -0500
Subject: [Python-Dev] PyPI comments and ratings, *really*?
In-Reply-To: <87y6mbuubp.fsf@benfinney.id.au>
References: <ca471dc20911121054t76eb5bfbsbd6a9e47a9e71d8a@mail.gmail.com>
	<hdhrvs$qs8$1@ger.gmane.org>
	<Pine.LNX.4.64.0911121648480.12274@kimball.webabinitio.net>
	<hdi17l$cli$1@ger.gmane.org> <4AFC9064.20700@v.loewis.de>
	<4222a8490911121511s40eba937n2c7b1fde6f695c6c@mail.gmail.com>
	<4AFC9975.7040204@v.loewis.de>
	<4222a8490911121534h30572eabv73ae58649c77155c@mail.gmail.com>
	<4AFC9D0C.5030500@v.loewis.de> <87y6mbuubp.fsf@benfinney.id.au>
Message-ID: <20091113011812.GA75517@andrew-kuchlings-macbook.local>

On Fri, Nov 13, 2009 at 11:44:42AM +1100, Ben Finney wrote:
> There's a problem with the poll's placement: on the front page of the
> PyPI website.

I've posted a tweet to the ThePSF account about the poll.  If the poll
runs for a week or two, that would provide time for word of the poll
to propagate through Twitter, blogs, etc.

--amk

From ianb at colorstudy.com  Fri Nov 13 02:33:27 2009
From: ianb at colorstudy.com (Ian Bicking)
Date: Thu, 12 Nov 2009 20:33:27 -0500
Subject: [Python-Dev] PyPI front page
In-Reply-To: <loom.20091113T015130-137@post.gmane.org>
References: <D7E23A6D-C91C-46D8-A8FE-1EB16F3B2101@lericson.se> 
	<Pine.LNX.4.64.0911121648480.12274@kimball.webabinitio.net> 
	<hdi17l$cli$1@ger.gmane.org> <4AFC9064.20700@v.loewis.de> 
	<4222a8490911121511s40eba937n2c7b1fde6f695c6c@mail.gmail.com> 
	<4AFC9975.7040204@v.loewis.de>
	<4222a8490911121534h30572eabv73ae58649c77155c@mail.gmail.com> 
	<4AFC9D0C.5030500@v.loewis.de> <87y6mbuubp.fsf@benfinney.id.au> 
	<loom.20091113T015130-137@post.gmane.org>
Message-ID: <b654cd2e0911121733v4b5667e0j8837fe825f925086@mail.gmail.com>

On Thu, Nov 12, 2009 at 7:52 PM, Antoine Pitrou <solipsis at pitrou.net> wrote:

> Ben Finney <ben+python <at> benfinney.id.au> writes:
> >
> > There's a problem with the poll's placement: on the front page of the
> > PyPI website.
>
> Speaking of which, why is it that http://pypi.python.org/pypi and
> http://pypi.python.org/pypi/ (note the ending slash) return different
> contents
> (the latter being very voluminous)? I always mistake one for the other when
> entering the URL directly.
>

easy_install replied on the behavior of /pypi/ (it uses the long list to do
case-insensitive searches).  Someone changed it, easy_install broke, and a
compromise was to keep /pypi/ the way it was (but not /pypi).

Probably this could be removed, as the /simple/ index is already
case-insensitive, so easy_install shouldn't have to hit /pypi/ at all.

-- 
Ian Bicking  |  http://blog.ianbicking.org  |
http://topplabs.org/civichacker
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20091112/2a5b74fe/attachment.htm>

From ben+python at benfinney.id.au  Fri Nov 13 02:39:14 2009
From: ben+python at benfinney.id.au (Ben Finney)
Date: Fri, 13 Nov 2009 12:39:14 +1100
Subject: [Python-Dev] PyPI comments and ratings, *really*?
References: <D7E23A6D-C91C-46D8-A8FE-1EB16F3B2101@lericson.se>
	<hdhrvs$qs8$1@ger.gmane.org>
	<Pine.LNX.4.64.0911121648480.12274@kimball.webabinitio.net>
	<hdi17l$cli$1@ger.gmane.org> <4AFC9064.20700@v.loewis.de>
	<4222a8490911121511s40eba937n2c7b1fde6f695c6c@mail.gmail.com>
	<4AFC9975.7040204@v.loewis.de>
	<4222a8490911121534h30572eabv73ae58649c77155c@mail.gmail.com>
	<4AFC9D0C.5030500@v.loewis.de> <87y6mbuubp.fsf@benfinney.id.au>
	<20e5c9660911121702r162caf28jb6d656383147d909@mail.gmail.com>
Message-ID: <87tywzurst.fsf@benfinney.id.au>

Michael Sparks <sparks.m at gmail.com> writes:

> On Fri, Nov 13, 2009 at 12:44 AM, Ben Finney <ben+python at benfinney.id.au> wrote:
> > So, the poll's audience is limited to those who visit the front page
> > (which is hardly ever necessary for package maintainers), and those
> > who already know it exists (e.g. through this discussion thread).
> > You'll be missing the opinions of those maintainers who, like the OP
> > of this thread, only discovered the behaviour much later.
>
> This poll is only visible if you're logged into PyPI. This strikes me
> as a mistake. I went looking for a poll and didn't see it.

The mistake, I think, is having a poll basically asking ?what should the
PyPI maintainers do??, instead of weighing evidence and reasoned
arguments. A poll may be good for gathering preferences and opinion, but
it's a poor way to make a *decision*.

-- 
 \      ?I busted a mirror and got seven years bad luck, but my lawyer |
  `\                        thinks he can get me five.? ?Steven Wright |
_o__)                                                                  |
Ben Finney


From brett at python.org  Fri Nov 13 02:43:02 2009
From: brett at python.org (Brett Cannon)
Date: Thu, 12 Nov 2009 17:43:02 -0800
Subject: [Python-Dev] PEP 382 status
In-Reply-To: <4AFB18B0.3050800@v.loewis.de>
References: <94bdd2610911110536n60a882acj8b2cd63ced3af25b@mail.gmail.com> 
	<4AFB18B0.3050800@v.loewis.de>
Message-ID: <bbaeab100911121743u6bc675c7r95aad40ca9fe0735@mail.gmail.com>

2009/11/11 "Martin v. L?wis" <martin at v.loewis.de>:
>> I was wondering what's the status of PEP 382. Is anyone (MvL?) is
>> going to start to work on its implementation for Python 2.7/3.2
>> inclusion ?
>
> I'll be working on an implementation, but contributions are welcome.
> Unfortunately, I'm really short on free software time recently (and
> keep hoping that this will improve).
>
> One thing that I always wondered about is the status of importlib.
> Will that be used in 3.2?
>

I doubt I will get to that as I suspect most people would rather I
spent my time on getting the Hg conversion to happen or trying to help
ramp up 3to2. So importlib will more than likely stay just a standard
library package for 3.2 (unfortunately as that means making this work
requires changes in two places, although prototyping in importlib
should be the easiest thing to get working).

-Brett

> In addition, I still owe a few answers to comments from the previous
> discussion.
>
> Regards,
> Martin
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/brett%40python.org
>

From robert.kern at gmail.com  Fri Nov 13 03:29:42 2009
From: robert.kern at gmail.com (Robert Kern)
Date: Thu, 12 Nov 2009 20:29:42 -0600
Subject: [Python-Dev] PyPI comments and ratings, *really*?
In-Reply-To: <20091113011812.GA75517@andrew-kuchlings-macbook.local>
References: <ca471dc20911121054t76eb5bfbsbd6a9e47a9e71d8a@mail.gmail.com>	<hdhrvs$qs8$1@ger.gmane.org>	<Pine.LNX.4.64.0911121648480.12274@kimball.webabinitio.net>	<hdi17l$cli$1@ger.gmane.org>
	<4AFC9064.20700@v.loewis.de>	<4222a8490911121511s40eba937n2c7b1fde6f695c6c@mail.gmail.com>	<4AFC9975.7040204@v.loewis.de>	<4222a8490911121534h30572eabv73ae58649c77155c@mail.gmail.com>	<4AFC9D0C.5030500@v.loewis.de>
	<87y6mbuubp.fsf@benfinney.id.au>
	<20091113011812.GA75517@andrew-kuchlings-macbook.local>
Message-ID: <hdigan$j6d$1@ger.gmane.org>

A.M. Kuchling wrote:
> On Fri, Nov 13, 2009 at 11:44:42AM +1100, Ben Finney wrote:
>> There's a problem with the poll's placement: on the front page of the
>> PyPI website.
> 
> I've posted a tweet to the ThePSF account about the poll.  If the poll
> runs for a week or two, that would provide time for word of the poll
> to propagate through Twitter, blogs, etc.

You should also make an announcement on python-announce.

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
  that is made terrible by our own mad attempt to interpret it as though it had
  an underlying truth."
   -- Umberto Eco


From tseaver at palladion.com  Fri Nov 13 05:08:04 2009
From: tseaver at palladion.com (Tres Seaver)
Date: Thu, 12 Nov 2009 23:08:04 -0500
Subject: [Python-Dev] PyPI comments and ratings, *really*?
In-Reply-To: <20e5c9660911121702r162caf28jb6d656383147d909@mail.gmail.com>
References: <D7E23A6D-C91C-46D8-A8FE-1EB16F3B2101@lericson.se>	<hdhrvs$qs8$1@ger.gmane.org>	<Pine.LNX.4.64.0911121648480.12274@kimball.webabinitio.net>	<hdi17l$cli$1@ger.gmane.org>
	<4AFC9064.20700@v.loewis.de>	<4222a8490911121511s40eba937n2c7b1fde6f695c6c@mail.gmail.com>	<4AFC9975.7040204@v.loewis.de>	<4222a8490911121534h30572eabv73ae58649c77155c@mail.gmail.com>	<4AFC9D0C.5030500@v.loewis.de>
	<87y6mbuubp.fsf@benfinney.id.au>
	<20e5c9660911121702r162caf28jb6d656383147d909@mail.gmail.com>
Message-ID: <hdim34$utf$1@ger.gmane.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Michael Sparks wrote:
> On Fri, Nov 13, 2009 at 12:44 AM, Ben Finney <ben+python at benfinney.id.au> wrote:
>> "Martin v. L?wis" <martin at v.loewis.de> writes:
>>
>>>> Why can't we just disable it until we can come up with a better
>>>> system that finds a balance between the rights of maintainers, and
>>>> those of the user?
>>> Because I want to wait for the outcome of the poll first.
>> There's a problem with the poll's placement: on the front page of the
>> PyPI website.
>>
>> I only know about the poll because you said there was one, and I went
>> hunting for it. The front page of PyPI is not one I ever visit, as a
>> package maintainer; I'll visit the pages for the packages I maintain, or
>> make a specific search of packages I'm looking for.
>>
>> So, the poll's audience is limited to those who visit the front page
>> (which is hardly ever necessary for package maintainers), and those who
>> already know it exists (e.g. through this discussion thread). You'll be
>> missing the opinions of those maintainers who, like the OP of this
>> thread, only discovered the behaviour much later.
> 
> This poll is only visible if you're logged into PyPI. This strikes me
> as a mistake. I went looking for a poll and didn't see it.
> 
> I only found the poll by accident by wondering randomly what might
> change if I hit the login using open id button. So you can only vote
> in the poll if you a) get told about it b) realise you need to create
> an account to login and use in order to vote. I realise there's good
> reasons for that, but I think it's a mistake. (There's no guidance
> that you need to log in to see the poll for example)

I can see the information about the poll, and a link to view the
results, without logging in.

  http://pypi.python.org/pypi

(second paragraph there).  That paragraph tells you that you need to log
in to vote in the poll.


Tres.
- --
===================================================================
Tres Seaver          +1 540-429-0999          tseaver at palladion.com
Palladion Software   "Excellence by Design"    http://palladion.com
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iEYEARECAAYFAkr825UACgkQ+gerLs4ltQ74YwCgsMU6UvoF6DZQP0If5yDV8j6J
GQIAnRVGlH0onr5nQwBFYUfGl/ni0SaR
=Ba00
-----END PGP SIGNATURE-----


From regebro at gmail.com  Fri Nov 13 05:47:39 2009
From: regebro at gmail.com (Lennart Regebro)
Date: Fri, 13 Nov 2009 05:47:39 +0100
Subject: [Python-Dev] PyPI comments and ratings, *really*?
In-Reply-To: <ca471dc20911120802r4c765f78h669e56335a38f11c@mail.gmail.com>
References: <D7E23A6D-C91C-46D8-A8FE-1EB16F3B2101@lericson.se>
	<200911130038.05516.steve@pearwood.info>
	<ca471dc20911120802r4c765f78h669e56335a38f11c@mail.gmail.com>
Message-ID: <319e029f0911122047q4da642c7x53e217e8f6d4d682@mail.gmail.com>

2009/11/12 Guido van Rossum <guido at python.org>:
> If you were to ask me, the people arguing against ratings and user
> comments are fighting a losing battle. If they had an iPhone or
> Android phone (or some other device with an "app store" kind of place
> to find downloads) they'd know the value (for prospective downloaders)
> of ratings and comments. Now, I think PyPI can use some (perhaps a lot
> of) improvement in the details of how it works, e.g. there should be a
> way to flag inappropriate messages (and users who post many
> inappropriate messages) and the software author should be able to talk
> back, but the general idea is here and won't go away by wishing it
> away.

Sure. But then we need moderation, spam filtering, flagging, etc. We
need to implement a whole discussion forum basically. We also need to
mark comments with the version they are relevant for and allow authors
to mark that comments are not longer valid. In fact, we get a
bug-tracker as well.

Maybe we should just install a trac per package? :-)

-- 
Lennart Regebro: Python, Zope, Plone, Grok
http://regebro.wordpress.com/
+33 661 58 14 64

From jacob at jacobian.org  Fri Nov 13 05:55:08 2009
From: jacob at jacobian.org (Jacob Kaplan-Moss)
Date: Thu, 12 Nov 2009 22:55:08 -0600
Subject: [Python-Dev] PyPI comments and ratings, *really*?
In-Reply-To: <4AFC9D0C.5030500@v.loewis.de>
References: <D7E23A6D-C91C-46D8-A8FE-1EB16F3B2101@lericson.se> 
	<ca471dc20911121054t76eb5bfbsbd6a9e47a9e71d8a@mail.gmail.com> 
	<hdhrvs$qs8$1@ger.gmane.org>
	<Pine.LNX.4.64.0911121648480.12274@kimball.webabinitio.net> 
	<hdi17l$cli$1@ger.gmane.org> <4AFC9064.20700@v.loewis.de> 
	<4222a8490911121511s40eba937n2c7b1fde6f695c6c@mail.gmail.com> 
	<4AFC9975.7040204@v.loewis.de>
	<4222a8490911121534h30572eabv73ae58649c77155c@mail.gmail.com> 
	<4AFC9D0C.5030500@v.loewis.de>
Message-ID: <bc8ec34e0911122055r7cd42337wde2effe4faa2204@mail.gmail.com>

On Thu, Nov 12, 2009 at 5:41 PM, "Martin v. L?wis" <martin at v.loewis.de> wrote:
> Because I want to wait for the outcome of the poll first.

I'm curious: what criteria will you use to judge the outcome of the
poll? That is, how will you translate the results of the poll into
action? Right now, the results stand as

Allow ratings and comments on all packages (status quo) 13
Allow package owners to disallow comments (ratings unmodified). 17
Allow comments, but only send them to package owners (ratings
unmodified).      2
Disallow comments (ratings unmodified). 11
Disallow ratings and comments (status three months ago).        36

If the poll ended this moment, how would you judge? Would it just be
mob rule (no comments)? Or some sort of spectrum -- there's 32 for
comments in some capacity and 47 against, so does somehow translate to
ratings but not comments? Or a weighted average? The average is 3.51
(1 being "allow comments" and 5 being "no comments")... what does
*that* mean?

On a deeper level, why are we voting at all? When else in the history
of Python have we used popular vote to decide questions of this
nature?

Jacob

[Martin, sorry for the repeat; I sent this privately first by mistake.]

From henning.vonbargen at arcor.de  Fri Nov 13 05:56:03 2009
From: henning.vonbargen at arcor.de (Henning von Bargen)
Date: Fri, 13 Nov 2009 05:56:03 +0100
Subject: [Python-Dev] PyPI comments and ratings, *really*?
In-Reply-To: <mailman.1265.1258076385.2872.python-dev@python.org>
References: <mailman.1265.1258076385.2872.python-dev@python.org>
Message-ID: <4AFCE6E3.4070209@arcor.de>

mode = "python-dev reader"

Please excuse me if I'm wrong here,
but I think python-dev just isn't the right place to discuss this topic,
because it's about 3rd party packages and it's got nothing to do with 
the development *of the python language itself*, but generated a lot of
traffic.

mode = "package developer"

I personally added a PyPI entry for my open source project just because 
it's there and it's a more or less standard, like CPAN in the Perl 
world, so I hoped others would perhaps find it there. On the other hand,
if I am searching for software myself, I don't look at PyPI - I just use 
G**gle.

I don't think PyPI needs any new features.
For the "developer's social web" that support the feature, there are 
other good places, for example take a look at http://www.ohloh.net

Henning


From ben+python at benfinney.id.au  Fri Nov 13 06:35:28 2009
From: ben+python at benfinney.id.au (Ben Finney)
Date: Fri, 13 Nov 2009 16:35:28 +1100
Subject: [Python-Dev] PyPI comments and ratings, *really*?
References: <D7E23A6D-C91C-46D8-A8FE-1EB16F3B2101@lericson.se>
	<ca471dc20911121054t76eb5bfbsbd6a9e47a9e71d8a@mail.gmail.com>
	<hdhrvs$qs8$1@ger.gmane.org>
	<Pine.LNX.4.64.0911121648480.12274@kimball.webabinitio.net>
	<hdi17l$cli$1@ger.gmane.org> <4AFC9064.20700@v.loewis.de>
	<4222a8490911121511s40eba937n2c7b1fde6f695c6c@mail.gmail.com>
	<4AFC9975.7040204@v.loewis.de>
	<4222a8490911121534h30572eabv73ae58649c77155c@mail.gmail.com>
	<4AFC9D0C.5030500@v.loewis.de>
	<bc8ec34e0911122055r7cd42337wde2effe4faa2204@mail.gmail.com>
Message-ID: <87ljibugv3.fsf@benfinney.id.au>

Jacob Kaplan-Moss <jacob at jacobian.org> writes:

> If the poll ended this moment, how would you judge? Would it just be
> mob rule (no comments)? [?]

Even though that's my preferred option, I *don't* want it chosen on the
basis of a poll result, but on the basis of evidence and reasoned
argument.

> On a deeper level, why are we voting at all? When else in the history
> of Python have we used popular vote to decide questions of this
> nature?

+1

-- 
 \           ?I do not believe in immortality of the individual, and I |
  `\        consider ethics to be an exclusively human concern with no |
_o__)  superhuman authority behind it.? ?Albert Einstein, letter, 1953 |
Ben Finney


From steve at pearwood.info  Fri Nov 13 06:38:18 2009
From: steve at pearwood.info (Steven D'Aprano)
Date: Fri, 13 Nov 2009 16:38:18 +1100
Subject: [Python-Dev] PyPI comments and ratings, *really*?
In-Reply-To: <ca471dc20911121054t76eb5bfbsbd6a9e47a9e71d8a@mail.gmail.com>
References: <D7E23A6D-C91C-46D8-A8FE-1EB16F3B2101@lericson.se>
	<hdhk7l$tru$1@ger.gmane.org>
	<ca471dc20911121054t76eb5bfbsbd6a9e47a9e71d8a@mail.gmail.com>
Message-ID: <200911131638.18786.steve@pearwood.info>

On Fri, 13 Nov 2009 05:54:24 am Guido van Rossum wrote:

> I agree that creating a good social app is not easy, and if we can't
> improve the social app embedded in PyPI quickly enough, we should at
> least give authors the option to disable comments. Of course, as a
> user, I might not trust a module that has no reviews or ratings.

As a user, I'd be more likely to trust a module with no reviews/ratings 
than one where the author disabled reviews/ratings. The first 
says "nobody hated it enough to complain", the second one says "the 
author is trying to hide something".



-- 
Steven D'Aprano

From ben+python at benfinney.id.au  Fri Nov 13 06:54:28 2009
From: ben+python at benfinney.id.au (Ben Finney)
Date: Fri, 13 Nov 2009 16:54:28 +1100
Subject: [Python-Dev] PyPI comments and ratings, *really*?
References: <D7E23A6D-C91C-46D8-A8FE-1EB16F3B2101@lericson.se>
	<hdhk7l$tru$1@ger.gmane.org>
	<ca471dc20911121054t76eb5bfbsbd6a9e47a9e71d8a@mail.gmail.com>
	<200911131638.18786.steve@pearwood.info>
Message-ID: <87hbszufzf.fsf@benfinney.id.au>

Steven D'Aprano <steve at pearwood.info> writes:

> On Fri, 13 Nov 2009 05:54:24 am Guido van Rossum wrote:
>
> > I agree that creating a good social app is not easy, and if we can't
> > improve the social app embedded in PyPI quickly enough, we should at
> > least give authors the option to disable comments. Of course, as a
> > user, I might not trust a module that has no reviews or ratings.
>
> As a user, I'd be more likely to trust a module with no reviews/ratings 
> than one where the author disabled reviews/ratings. The first 
> says "nobody hated it enough to complain", the second one says "the 
> author is trying to hide something".

Agreed, that's how I'd feel (and it's important to note that this would
be an emotional, not necessarily entirely rational, reaction) as a user
also.

Package maintainers who also see that users would feel that way, and who
agree with the purpose of PyPI as a common repository of all third-party
packages, but who *don't* want to deal with PyPI's implementation of
comments (whatever that may be at any time), have a clear option: to
avoid hosting the package at PyPI at all. That's harmful, and I don't
want it; but I don't see an alternative for such a maintainer.

-- 
 \              ?Dvorak users of the world flgkd!? ?Kirsten Chevalier, |
  `\                                                rec.humor.oracle.d |
_o__)                                                                  |
Ben Finney


From steve at pearwood.info  Fri Nov 13 07:22:38 2009
From: steve at pearwood.info (Steven D'Aprano)
Date: Fri, 13 Nov 2009 17:22:38 +1100
Subject: [Python-Dev] PyPI comments and ratings, *really*?
In-Reply-To: <7EAD464A-45AC-41EF-9E11-A9BAA3461DB9@lericson.se>
References: <D7E23A6D-C91C-46D8-A8FE-1EB16F3B2101@lericson.se>
	<200911130038.05516.steve@pearwood.info>
	<7EAD464A-45AC-41EF-9E11-A9BAA3461DB9@lericson.se>
Message-ID: <200911131722.38598.steve@pearwood.info>

On Fri, 13 Nov 2009 04:27:48 am Ludvig Ericson wrote:
> On 12 nov 2009, at 14:38, Steven D'Aprano wrote:
> > On Thu, 12 Nov 2009 08:44:32 pm Ludvig Ericson wrote:
> >> Why are there comments on PyPI? Moreso, why are there comments
> >> which I cannot control as a package author on my very own
> >> packages? That's just absurd.
> >
> > No, what's absurd is thinking that the act of publishing software
> > somehow gives you the right to demand control over what others say
> > about your software.
>
> ... on my own package's page. 

It's your package. It's the community's page about your package.

I think of PyPI as a community-owned noticeboard. Its primary purpose is 
to allow the community to find good packages -- the benefit to the 
community is why it exists, not the benefit to the author of the 
package.

In my opinion, the community is best served by a good comment/review 
system, one which avoids the worst trolling, and allows authors the 
right of reply, but does not allow authors to censor inconvenient but 
honest reviews. 



-- 
Steven D'Aprano

From steve at pearwood.info  Fri Nov 13 07:23:08 2009
From: steve at pearwood.info (Steven D'Aprano)
Date: Fri, 13 Nov 2009 17:23:08 +1100
Subject: [Python-Dev] PyPI comments and ratings, *really*?
In-Reply-To: <hdhrvs$qs8$1@ger.gmane.org>
References: <D7E23A6D-C91C-46D8-A8FE-1EB16F3B2101@lericson.se>
	<ca471dc20911121054t76eb5bfbsbd6a9e47a9e71d8a@mail.gmail.com>
	<hdhrvs$qs8$1@ger.gmane.org>
Message-ID: <200911131723.08584.steve@pearwood.info>

On Fri, 13 Nov 2009 07:42:37 am Terry Reedy wrote:
> Part of the pypi problem is a startup problem of initially low
> numbers. If the only people who bother to log in to rate are the
> disgruntled, then the ratings/reviews will be biased.

The package author who started this thread, Ludvig Ericson, had (as of 
last night) two comments, one positive, one negative.

This admittedly tiny sample doesn't suggest to me that PyPI is suffering 
from the problem of biased ratings.



-- 
Steven D'Aprano

From ben+python at benfinney.id.au  Fri Nov 13 08:00:08 2009
From: ben+python at benfinney.id.au (Ben Finney)
Date: Fri, 13 Nov 2009 18:00:08 +1100
Subject: [Python-Dev] PyPI comments and ratings, *really*?
References: <D7E23A6D-C91C-46D8-A8FE-1EB16F3B2101@lericson.se>
	<200911130038.05516.steve@pearwood.info>
	<7EAD464A-45AC-41EF-9E11-A9BAA3461DB9@lericson.se>
	<200911131722.38598.steve@pearwood.info>
Message-ID: <87aayqvrif.fsf@benfinney.id.au>

Steven D'Aprano <steve at pearwood.info> writes:

> In my opinion, the community is best served by a good comment/review 
> system, one which avoids the worst trolling, and allows authors the 
> right of reply, but does not allow authors to censor inconvenient but 
> honest reviews. 

I think you're right.

I also think, though, that the community is best served by an objective
repository of third-party Python packages, with information derived only
directly from the package itself and objective data. That allows the
least barrier to having a package maintainer want to register their
package with such a service, which is in the interest of having it be as
complete a registry of packages as can be.

A community forum, on the other hand, has many characteristics that will
be *disincentives* to a package manager for having their package appear
there. It's never going to attract as many package maintainers as an
impartial, objective registry; the many reasons already given here as to
why some package maintainers *don't* want their packages in such a
system are evidence of that.

Those two purposes ? community forum, impartial registry ? are in
conflict. I think PyPI has clearly already been serving the role of the
registry, and that any community forum should be quite separate to
encourage those who don't like it to still register their packages at
PyPI.

-- 
 \        ?I'd take the awe of understanding over the awe of ignorance |
  `\                                          any day.? ?Douglas Adams |
_o__)                                                                  |
Ben Finney


From steve at pearwood.info  Fri Nov 13 08:07:05 2009
From: steve at pearwood.info (Steven D'Aprano)
Date: Fri, 13 Nov 2009 18:07:05 +1100
Subject: [Python-Dev] PyPI comments and ratings, *really*?
In-Reply-To: <87aayrwb79.fsf@benfinney.id.au>
References: <D7E23A6D-C91C-46D8-A8FE-1EB16F3B2101@lericson.se>
	<4AFC9975.7040204@v.loewis.de> <87aayrwb79.fsf@benfinney.id.au>
Message-ID: <200911131807.06132.steve@pearwood.info>

On Fri, 13 Nov 2009 10:54:50 am Ben Finney wrote:
> "Martin v. L?wis" <martin at v.loewis.de> writes:
> > PyPI is not just (and perhaps not even primarily) there for the
> > package authors, but for the package users (and not surprisingly,
> > it's primarily the package authors who ask for banning the user
> > opinions).
>
> No-one here is asking for ?banning the user opinions?. 

That's EXACTLY what people have asked for: the ability to ban user 
comments. Sure, some of them say they want to "opt-out", or "disable 
comments", but that's just a polite way of saying the same thing. Some 
of them -- 44% of the people who have responded to the poll as of a 
minute ago -- want to prohibit *all* comments, even for packages whose 
authors wants to receive comments. WTF? The original poster (Ludvig 
Ericson) started this thread insisting that there were only two 
acceptable options: prohibit all comments on PyPI, or make it opt-in.

I quote:

"As I see it, there are only two ways to fix these misguided steps of  
development: throw them out, or make them opt-in settings."


> As already 
> pointed out, users are not mute; there are plenty of places for them
> to have a voice.

Ben, you've been talking recently about the dangers of fragmenting the 
Python community into multiple forums, but that's what you're arguing 
for on PyPI.

I'm a user, I'm interested in a package, so I look for it on PyPI. I 
want to know what others think about it, before I commit to downloading 
it, installing it and seeing if it meets my needs. Instead of looking 
in the One Obvious place, the PyPI page, you want me to go off looking 
for blogs, mailing lists, or other forums, which might not even exist.

Part of the value of a centralised place for comments on a package is 
that I can see whether other users agree with the comment or not. If I 
see a comment "This doesn't even work", and ten other comments 
saying "No, this is great, but you need to transmogrify the phlogiston 
first to get it to work", I've learned something useful. But if I 
stumble across a single blog that says "This doesn't work", I've 
actually been given negative knowledge: I've learned something that 
just isn't so.


What *actual* problem are we trying to solve here? Is there a problem 
with spam on PyPI? No. Is there a problem with the comments being 
wildly biased, either for or against? Apparently not. Is there a 
problem with people using the comment system as a help forum? No. So 
what's the problem we're trying to solve?



> I understand the mood here to be, not that user feedback is not
> wanted, but rather that PyPI in particular should be a place for
> *objective* data about a package.

Who is responsible for gather this "objective" data? How do we decide 
what is objective and what isn't? People can't even agree on the 
validity of benchmarks!

This is open source. The power of the bazaar, remember? I'm amazed at 
how many people are not just disinterested in, but actively hostile, to 
even *useful* comments from users. That's fine. If you, the package 
author, don't care about comments from users, don't read them. But 
they're not there for *your* benefit, they're there for the benefit of 
other users.



-- 
Steven D'Aprano

From martin at v.loewis.de  Fri Nov 13 08:34:05 2009
From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=)
Date: Fri, 13 Nov 2009 08:34:05 +0100
Subject: [Python-Dev] PyPI comments and ratings, *really*?
In-Reply-To: <87y6mbuubp.fsf@benfinney.id.au>
References: <D7E23A6D-C91C-46D8-A8FE-1EB16F3B2101@lericson.se>	<B10AA20C-1C2F-44F0-B8B5-64F26691CC83@python.org>	<hdhk7l$tru$1@ger.gmane.org>	<ca471dc20911121054t76eb5bfbsbd6a9e47a9e71d8a@mail.gmail.com>	<hdhrvs$qs8$1@ger.gmane.org>	<Pine.LNX.4.64.0911121648480.12274@kimball.webabinitio.net>	<hdi17l$cli$1@ger.gmane.org>
	<4AFC9064.20700@v.loewis.de>	<4222a8490911121511s40eba937n2c7b1fde6f695c6c@mail.gmail.com>	<4AFC9975.7040204@v.loewis.de>	<4222a8490911121534h30572eabv73ae58649c77155c@mail.gmail.com>	<4AFC9D0C.5030500@v.loewis.de>
	<87y6mbuubp.fsf@benfinney.id.au>
Message-ID: <4AFD0BED.4040700@v.loewis.de>

> There's a problem with the poll's placement: on the front page of the
> PyPI website.

You really should participate in the proper forum for the discussion of
PyPI: catalog-sig. Then you would have noticed that I said I'll announce
the poll later (i.e. today), which I'm doing right now.

Feel free to announce it places where I didn't.

Regards,
Martin

From kevin at bud.ca  Fri Nov 13 08:35:09 2009
From: kevin at bud.ca (Kevin Teague)
Date: Thu, 12 Nov 2009 23:35:09 -0800
Subject: [Python-Dev] standard libraries don't behave like standard
	'libraries'
In-Reply-To: <4222a8490911121157i622fbab8q5fe82d32fe9ea8f@mail.gmail.com>
References: <c910262d0911120123i7c7e9dla918fff50cb5aecd@mail.gmail.com>
	<4AFC6440.1040008@v.loewis.de>
	<4222a8490911121157i622fbab8q5fe82d32fe9ea8f@mail.gmail.com>
Message-ID: <56195B13-39DF-44A3-9A79-5BDAFF244F40@bud.ca>


On Nov 12, 2009, at 11:57 AM, Jesse Noller wrote:

> On Thu, Nov 12, 2009 at 2:38 PM, "Martin v. L?wis"  
> <martin at v.loewis.de> wrote:
>>> I am not an expert, I am just another python learner. These are  
>>> just my
>>> views on the state of the standard libraries and to
>>> make them state-of-the-art..! ;)
>>
>> If I understand correctly, you want the (current) standard library  
>> to be
>> separated from the Python implementation, and available separately.
>>
>> Interestingly enough, people are very much split over whether that  
>> would
>> be a good thing or not. Some like it the way Python does, some  
>> dislike
>> it (and some quite strongly so).
>>
>> In any case, many Python users consider it a good thing that it comes
>> "with batteries included", ie. with no need to add extra stuff for  
>> many
>> tasks.
>>
>> Some of the Python maintainers have recently started objecting to  
>> this
>> setup, asking that the standard library should be split into separate
>> packages that are released and distributed independent of Python.  
>> Others
>> of us feel strongly that such a change should not be made.
>>
>> So don't expect any immediate change to happen.
>>
>> Regards,
>> Martin
>
> Martin is correct; this came up on distutils-sig a month or so ago; I
> proposed offering multiple downloads "with batteries" and "without
> batteries (with the batteries on the side)". We decided as a group to
> hold off on that until further in the future.
>
> jesse


It's also worth noting that there are three issues with respect to  
standard library packaging which are all orthologous:

  * Packaging for metadata: The standard library could be packaged so  
that there is consistent metadata about the distributions which  
compromise the standard library (version, author, maintainer, etc).  
This could be useful so that it would be easier to see at a glance  
which packages changed between any two Python releases. For example, I  
have a script which compares two working sets of packages and  
generates a web page with a colour-coded list of differences between  
those two working sets, based on the size of the version number bump,  
e.g. major.medium.minor (doesn't work with packages which use odd  
version numbering schemes). I've used this to compare working sets  
between different Grok releases or Plone releases to help see where  
the activity or major API changes might be happening inside any given  
Zope-based release. However, this script only works on 80% of the  
code, the other 20% is in the standard library, which is currently a  
black-box.

     Also, if PEP 376 is accepted, then the standard library will be  
inviolation of the stanard metadata for installed packages unless this  
happens.

  * Packaging for release:  The standard library could be packaged so  
that releases happen on PyPI. This could be useful so that it's easier  
to consume newer versions of a package before those packages are  
included in the next Python release. It could also make Python  
upgrades easier, as you could hold-back packages which might otherwise  
break an application upon upgrade. I happened to find a bug in a  
standard library package today, and currently my option is to: Submit  
bug report to PyPI, wait for Python 2.7 release, then wait for Plone  
to update to Python 2.7. I'm looking at many years before this fix  
goes into my production application, so in the meantime I need to  
maintain a work-around for the bug as well account for the day when  
this bug is fixed. It makes a mess of the code. If it was possible to  
release this package on PyPI, this could reasonably happen in a few  
days or at worst a couple months. The setup.py for that package could  
be updated to require the newer version of the package, and the  
workaround could be removed from the code.

  * Packaging for resizing: The standard library could be packaged so  
that it could be made smaller or larger, or there could be different  
sized standard libraries made with different Python releases.

So there are benefits to packaging the standard library which would be  
useful, for reasons which would have no impact on the "batteries  
included" aspect of Python.

Plus, if the standard library was packaged and releases were made on  
PyPI (in addition to including them normally in the Python releases),  
we *might* be able to use PyPI's commenting infrastructure on this  
packages (~evil grin~).



From martin at v.loewis.de  Fri Nov 13 08:35:48 2009
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Fri, 13 Nov 2009 08:35:48 +0100
Subject: [Python-Dev] PyPI front page
In-Reply-To: <loom.20091113T015130-137@post.gmane.org>
References: <D7E23A6D-C91C-46D8-A8FE-1EB16F3B2101@lericson.se>	<B10AA20C-1C2F-44F0-B8B5-64F26691CC83@python.org>	<hdhk7l$tru$1@ger.gmane.org>	<ca471dc20911121054t76eb5bfbsbd6a9e47a9e71d8a@mail.gmail.com>	<hdhrvs$qs8$1@ger.gmane.org>	<Pine.LNX.4.64.0911121648480.12274@kimball.webabinitio.net>	<hdi17l$cli$1@ger.gmane.org>
	<4AFC9064.20700@v.loewis.de>	<4222a8490911121511s40eba937n2c7b1fde6f695c6c@mail.gmail.com>	<4AFC9975.7040204@v.loewis.de>	<4222a8490911121534h30572eabv73ae58649c77155c@mail.gmail.com>	<4AFC9D0C.5030500@v.loewis.de>
	<87y6mbuubp.fsf@benfinney.id.au>
	<loom.20091113T015130-137@post.gmane.org>
Message-ID: <4AFD0C54.1020709@v.loewis.de>

> Ben Finney <ben+python <at> benfinney.id.au> writes:
>> There's a problem with the poll's placement: on the front page of the
>> PyPI website.
> 
> Speaking of which, why is it that http://pypi.python.org/pypi and
> http://pypi.python.org/pypi/ (note the ending slash) return different contents
> (the latter being very voluminous)? I always mistake one for the other when
> entering the URL directly.

As Ian says: setuptools relies on it. It's part of the specification of
the package index.

Regards,
Martin

From martin at v.loewis.de  Fri Nov 13 08:37:54 2009
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Fri, 13 Nov 2009 08:37:54 +0100
Subject: [Python-Dev] PyPI comments and ratings, *really*?
In-Reply-To: <20e5c9660911121702r162caf28jb6d656383147d909@mail.gmail.com>
References: <D7E23A6D-C91C-46D8-A8FE-1EB16F3B2101@lericson.se>	<hdhrvs$qs8$1@ger.gmane.org>	<Pine.LNX.4.64.0911121648480.12274@kimball.webabinitio.net>	<hdi17l$cli$1@ger.gmane.org>
	<4AFC9064.20700@v.loewis.de>	<4222a8490911121511s40eba937n2c7b1fde6f695c6c@mail.gmail.com>	<4AFC9975.7040204@v.loewis.de>	<4222a8490911121534h30572eabv73ae58649c77155c@mail.gmail.com>	<4AFC9D0C.5030500@v.loewis.de>
	<87y6mbuubp.fsf@benfinney.id.au>
	<20e5c9660911121702r162caf28jb6d656383147d909@mail.gmail.com>
Message-ID: <4AFD0CD2.6000304@v.loewis.de>

> I only found the poll by accident by wondering randomly what might
> change if I hit the login using open id button. So you can only vote
> in the poll if you a) get told about it b) realise you need to create
> an account to login and use in order to vote. I realise there's good
> reasons for that, but I think it's a mistake. (There's no guidance
> that you need to log in to see the poll for example)

Why do you say that? "If you want to vote, you need to login to PyPI (on
the right)."

If that isn't clear to you, please propose a rephrasing (English is not
my native language, nor the native language of MAL who proposed this
specific wording)

Regards,
Martin

From martin at v.loewis.de  Fri Nov 13 08:39:59 2009
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Fri, 13 Nov 2009 08:39:59 +0100
Subject: [Python-Dev] PyPI comments and ratings, *really*?
In-Reply-To: <hdigan$j6d$1@ger.gmane.org>
References: <ca471dc20911121054t76eb5bfbsbd6a9e47a9e71d8a@mail.gmail.com>	<hdhrvs$qs8$1@ger.gmane.org>	<Pine.LNX.4.64.0911121648480.12274@kimball.webabinitio.net>	<hdi17l$cli$1@ger.gmane.org>	<4AFC9064.20700@v.loewis.de>	<4222a8490911121511s40eba937n2c7b1fde6f695c6c@mail.gmail.com>	<4AFC9975.7040204@v.loewis.de>	<4222a8490911121534h30572eabv73ae58649c77155c@mail.gmail.com>	<4AFC9D0C.5030500@v.loewis.de>	<87y6mbuubp.fsf@benfinney.id.au>	<20091113011812.GA75517@andrew-kuchlings-macbook.local>
	<hdigan$j6d$1@ger.gmane.org>
Message-ID: <4AFD0D4F.3090904@v.loewis.de>

>> I've posted a tweet to the ThePSF account about the poll.  If the poll
>> runs for a week or two, that would provide time for word of the poll
>> to propagate through Twitter, blogs, etc.
> 
> You should also make an announcement on python-announce.

On catalog-sig (the place where PyPI was discussed in recent years),
I mentioned that I would announce it today, which I now have.

Regards,
Martin


From martin at v.loewis.de  Fri Nov 13 08:44:16 2009
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Fri, 13 Nov 2009 08:44:16 +0100
Subject: [Python-Dev] PyPI comments and ratings, *really*?
In-Reply-To: <20e5c9660911121652s6c9c25caqf8856eb6c1685986@mail.gmail.com>
References: <D7E23A6D-C91C-46D8-A8FE-1EB16F3B2101@lericson.se>	<200911130038.05516.steve@pearwood.info>	<4222a8490911120606t3e138957kcdb21104261a1ead@mail.gmail.com>
	<20e5c9660911121652s6c9c25caqf8856eb6c1685986@mail.gmail.com>
Message-ID: <4AFD0E50.5090309@v.loewis.de>

> That's useful from a user perspective. Or is it? It's useful from a
> user perspective, until that issue is fixed. Then what? Is it still
> useful? Can the commenter remove it?

Yes.

> Can they get notified it's changed?

Yes.

> Can the maintainer say "this is fixed/changed?"

Yes.

> I never look at the PyPI pages for stuff I create. Which means if
> someone is using it for support, they're wasting their time. (Why
> would I look at it? I know what the project is for and where to get
> it! :) (and also PyPI isn't the prime download for it either - so the
> download stats are irrelevant to me) I doubt I'm alone, so how many
> people's time are wasted asking questions there ?

You'll get an email when someone comments, so you don't have to
look at the page.

> I suppose, personally, I'm dubious about the idea of having unchanging
> comments and ratings associated with projects which are changing and
> improving - that feels like a mismatch.

That's why the comments are per-release.

> If there's interest, and if there's a survey to be done, I could
> forward a link to a survey through that twitterfeed - which I suspect
> is a mix of users of PyPI and uploaders to PyPI.

Feel free to annouce it on Twitter - I don't use that system myself.

Regards,
Martin

From martin at v.loewis.de  Fri Nov 13 08:45:39 2009
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Fri, 13 Nov 2009 08:45:39 +0100
Subject: [Python-Dev] PyPI comments and ratings, *really*?
In-Reply-To: <4AFCE6E3.4070209@arcor.de>
References: <mailman.1265.1258076385.2872.python-dev@python.org>
	<4AFCE6E3.4070209@arcor.de>
Message-ID: <4AFD0EA3.5050007@v.loewis.de>

> mode = "python-dev reader"
> 
> Please excuse me if I'm wrong here,
> but I think python-dev just isn't the right place to discuss this topic,
> because it's about 3rd party packages and it's got nothing to do with
> the development *of the python language itself*, but generated a lot of
> traffic.

+1. The place to discuss pypi is catalog-sig.

Regards,
Martin

From fuzzyman at voidspace.org.uk  Fri Nov 13 09:47:30 2009
From: fuzzyman at voidspace.org.uk (Michael Foord)
Date: Fri, 13 Nov 2009 09:47:30 +0100
Subject: [Python-Dev] PyPI comments and ratings, *really*?
In-Reply-To: <200911131722.38598.steve@pearwood.info>
References: <D7E23A6D-C91C-46D8-A8FE-1EB16F3B2101@lericson.se>	<200911130038.05516.steve@pearwood.info>	<7EAD464A-45AC-41EF-9E11-A9BAA3461DB9@lericson.se>
	<200911131722.38598.steve@pearwood.info>
Message-ID: <4AFD1D22.3050708@voidspace.org.uk>

Steven D'Aprano wrote:
> On Fri, 13 Nov 2009 04:27:48 am Ludvig Ericson wrote:
>   
>> On 12 nov 2009, at 14:38, Steven D'Aprano wrote:
>>     
>>> On Thu, 12 Nov 2009 08:44:32 pm Ludvig Ericson wrote:
>>>       
>>>> Why are there comments on PyPI? Moreso, why are there comments
>>>> which I cannot control as a package author on my very own
>>>> packages? That's just absurd.
>>>>         
>>> No, what's absurd is thinking that the act of publishing software
>>> somehow gives you the right to demand control over what others say
>>> about your software.
>>>       
>> ... on my own package's page. 
>>     
>
> It's your package. It's the community's page about your package.
>
> I think of PyPI as a community-owned noticeboard. Its primary purpose is 
> to allow the community to find good packages -- the benefit to the 
> community is why it exists, not the benefit to the author of the 
> package.
>
> In my opinion, the community is best served by a good comment/review 
> system, one which avoids the worst trolling, and allows authors the 
> right of reply, but does not allow authors to censor inconvenient but 
> honest reviews. 
>
>   
+1 :-)

Michael

-- 
http://www.ironpythoninaction.com/


From naughtysriram at gmail.com  Fri Nov 13 09:59:46 2009
From: naughtysriram at gmail.com (Sriram Srinivasan)
Date: Fri, 13 Nov 2009 13:59:46 +0500
Subject: [Python-Dev] standard libraries don't behave like standard
	'libraries'
In-Reply-To: <56195B13-39DF-44A3-9A79-5BDAFF244F40@bud.ca>
References: <c910262d0911120123i7c7e9dla918fff50cb5aecd@mail.gmail.com>
	<4AFC6440.1040008@v.loewis.de>
	<4222a8490911121157i622fbab8q5fe82d32fe9ea8f@mail.gmail.com>
	<56195B13-39DF-44A3-9A79-5BDAFF244F40@bud.ca>
Message-ID: <c910262d0911130059g590624c0ybf74e2f32e850637@mail.gmail.com>

you were thinking wrong. If suppose this feature is introduced it doesn't
mean python will become batteries removed!
you can ship the python release with the 'standard library packages' already
installed.
so what we get here is batteries included and ability to be changed after it
is discharged! ;) users did not get this feature before. what we experience
now is like 'use and throw' not the batteries but the whole python
distribution (like torchlight/flashlight) itself.
also users might get the ability to include other batteries too.

also say for example some X company wants its own python distribution
(including the standard libraries).
they may have the standard libraries removed or another custom library on
top of it (forget performance here). Now if the standard libraries were
available as packages a new version may be installed and tested at no time.
but if they get it by every release, there comes the problem.

this not only makes things easier on both sides (releasing side and user
side) it also makes the whole system transparent.
even patches for standard libraries may be available as soon as a bug has
been found, but not with the current system as Kevin mentioned.

say for example we take an opensouce project, the linux kernel as example.(
dont compare kernel with python each work differently and both scales are
not equal) just think it was another opensource software. i can say mostly
where there linux there is python (not talking about windows,embedded etc,
both software go hand in hand, so i took it for comparison). if there is a
bug/security issue found on the linux the patches are available in
days/mostly by the end of next week. but for python it must take more time.
may be some backward incompatible issue also comes along with it(eg. py3)
then they have to wait for the dependencies being updated. i am only
comparing the packaging and distributing system here not the softwares.
althought the rate at which the kernel development is at a very high scale,
the system is stable. and that is how open source works. every body can have
look into everything. that is what makes open source software best. if it
were (libraries) released as packages, by the time i install the patch
package, i may also get the patch package of the dependency too.


-- 

Regards,
Sriram.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20091113/85207500/attachment.htm>

From ubershmekel at gmail.com  Fri Nov 13 10:30:19 2009
From: ubershmekel at gmail.com (Yuval Greenfield)
Date: Fri, 13 Nov 2009 11:30:19 +0200
Subject: [Python-Dev] standard libraries don't behave like standard
	'libraries'
In-Reply-To: <56195B13-39DF-44A3-9A79-5BDAFF244F40@bud.ca>
References: <c910262d0911120123i7c7e9dla918fff50cb5aecd@mail.gmail.com>
	<4AFC6440.1040008@v.loewis.de>
	<4222a8490911121157i622fbab8q5fe82d32fe9ea8f@mail.gmail.com>
	<56195B13-39DF-44A3-9A79-5BDAFF244F40@bud.ca>
Message-ID: <9d153b7c0911130130v35ab44b7n5c98727505009409@mail.gmail.com>

On Fri, Nov 13, 2009 at 9:35 AM, Kevin Teague <kevin at bud.ca> wrote:
>
> It's also worth noting that there are three issues with respect to standard
> library packaging which are all orthologous:
>
> ?* Packaging for metadata: The standard library could be packaged so that
> there is consistent metadata about the distributions which compromise the
> standard library (version, author, maintainer, etc). This could be useful so
> that it would be easier to see at a glance which packages changed between
> any two Python releases. For example, I have a script which compares two
> working sets of packages and generates a web page with a colour-coded list
> of differences between those two working sets, based on the size of the
> version number bump, e.g. major.medium.minor (doesn't work with packages
> which use odd version numbering schemes). I've used this to compare working
> sets between different Grok releases or Plone releases to help see where the
> activity or major API changes might be happening inside any given Zope-based
> release. However, this script only works on 80% of the code, the other 20%
> is in the standard library, which is currently a black-box.
>
> ? ?Also, if PEP 376 is accepted, then the standard library will be
> inviolation of the stanard metadata for installed packages unless this
> happens.
>
> ?* Packaging for release: ?The standard library could be packaged so that
> releases happen on PyPI. This could be useful so that it's easier to consume
> newer versions of a package before those packages are included in the next
> Python release. It could also make Python upgrades easier, as you could
> hold-back packages which might otherwise break an application upon upgrade.
> I happened to find a bug in a standard library package today, and currently
> my option is to: Submit bug report to PyPI, wait for Python 2.7 release,
> then wait for Plone to update to Python 2.7. I'm looking at many years
> before this fix goes into my production application, so in the meantime I
> need to maintain a work-around for the bug as well account for the day when
> this bug is fixed. It makes a mess of the code. If it was possible to
> release this package on PyPI, this could reasonably happen in a few days or
> at worst a couple months. The setup.py for that package could be updated to
> require the newer version of the package, and the workaround could be
> removed from the code.
>
> ?* Packaging for resizing: The standard library could be packaged so that it
> could be made smaller or larger, or there could be different sized standard
> libraries made with different Python releases.
>
> So there are benefits to packaging the standard library which would be
> useful, for reasons which would have no impact on the "batteries included"
> aspect of Python.
>
> Plus, if the standard library was packaged and releases were made on PyPI
> (in addition to including them normally in the Python releases), we *might*
> be able to use PyPI's commenting infrastructure on this packages (~evil
> grin~).
>
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> http://mail.python.org/mailman/options/python-dev/ubershmekel%40gmail.com
>

I kinda like the idea that the only packages included in the python
trunk would be distribute and pip (maybe just svn:external). Upon
packaging python for distribution (by installer or by source) the
relevant batteries would be included. Moving packages in and out of
the standard library just becomes so standard.

But of course this really should be decided by the people who work
most with the trunk and as I understand
http://mail.python.org/pipermail/stdlib-sig/2009-September/000609.html
this discussion's already run its course.

--yuv

From abpillai at gmail.com  Fri Nov 13 11:36:10 2009
From: abpillai at gmail.com (Anand Balachandran Pillai)
Date: Fri, 13 Nov 2009 16:06:10 +0530
Subject: [Python-Dev] standard libraries don't behave like standard
	'libraries'
In-Reply-To: <4AFC6440.1040008@v.loewis.de>
References: <c910262d0911120123i7c7e9dla918fff50cb5aecd@mail.gmail.com>
	<4AFC6440.1040008@v.loewis.de>
Message-ID: <8548c5f30911130236j72d6b1a4m21d5bc714f451c43@mail.gmail.com>

On Fri, Nov 13, 2009 at 1:08 AM, "Martin v. L?wis" <martin at v.loewis.de>wrote:

> > I am not an expert, I am just another python learner. These are just my
> > views on the state of the standard libraries and to
> > make them state-of-the-art..! ;)
>
> If I understand correctly, you want the (current) standard library to be
> separated from the Python implementation, and available separately.
>
> Interestingly enough, people are very much split over whether that would
> be a good thing or not. Some like it the way Python does, some dislike
> it (and some quite strongly so).
>
> In any case, many Python users consider it a good thing that it comes
> "with batteries included", ie. with no need to add extra stuff for many
> tasks.
>
> Some of the Python maintainers have recently started objecting to this
> setup, asking that the standard library should be split into separate
> packages that are released and distributed independent of Python. Others
> of us feel strongly that such a change should not be made.
>
> So don't expect any immediate change to happen.
>

 I think the recent postings on "CPAN for Python"  in this list also stems
 from similar thoughts. As the size of the Python std library increases,
 and when it reaches a kind of implosion point (perhaps it is there
already),
 it does make sense to split it into some thing like a "core" group of
 modules and add-ons which could be updated dynamically as and when
 required by a  tool.

 It could be something as simple as a "require" keyword which could
 pull in the depdencies if not found. Perhaps at the top of your module,

 require (stuff, '1.27')
 import stuff

 Anyone who takes a quick look at the Python std library now is
 sure to feel that there is an overkill of stuff there, which could be
 classified and packaged better than dumping into the language
 build.


> Regards,
> Martin
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> http://mail.python.org/mailman/options/python-dev/abpillai%40gmail.com
>



-- 
--Anand
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20091113/2c4ac635/attachment-0001.htm>

From masklinn at masklinn.net  Fri Nov 13 11:50:49 2009
From: masklinn at masklinn.net (Masklinn)
Date: Fri, 13 Nov 2009 11:50:49 +0100
Subject: [Python-Dev] standard libraries don't behave like standard
	'libraries'
In-Reply-To: <8548c5f30911130236j72d6b1a4m21d5bc714f451c43@mail.gmail.com>
References: <c910262d0911120123i7c7e9dla918fff50cb5aecd@mail.gmail.com>
	<4AFC6440.1040008@v.loewis.de>
	<8548c5f30911130236j72d6b1a4m21d5bc714f451c43@mail.gmail.com>
Message-ID: <45186ACD-29BA-4D5E-9ADB-25632021A539@masklinn.net>

On 13 Nov 2009, at 11:36 , Anand Balachandran Pillai wrote:
> I think the recent postings on "CPAN for Python"  in this list also stems
> from similar thoughts.
No. The CPAN for Python messages (in and out of the list) are about third-party packages and a better way to manage (and a better experience of installing) those.

Think Ruby, it has a significant stdlib (though not as extensive as Python's), but it also has gems (and the `gem` CLI util). Or Haskell, which also has a significant stdlib (GHC does anyway) but also has cabal and cabal-install.

From wentland at cl.uni-heidelberg.de  Fri Nov 13 11:19:06 2009
From: wentland at cl.uni-heidelberg.de (Wolodja Wentland)
Date: Fri, 13 Nov 2009 11:19:06 +0100
Subject: [Python-Dev] PyPI comments and ratings, *really*?
In-Reply-To: <87aayqvrif.fsf@benfinney.id.au>
References: <D7E23A6D-C91C-46D8-A8FE-1EB16F3B2101@lericson.se>
	<200911130038.05516.steve@pearwood.info>
	<7EAD464A-45AC-41EF-9E11-A9BAA3461DB9@lericson.se>
	<200911131722.38598.steve@pearwood.info>
	<87aayqvrif.fsf@benfinney.id.au>
Message-ID: <20091113101906.GA3246@kinakuta.local>

On Fri, Nov 13, 2009 at 18:00 +1100, Ben Finney wrote:
> Steven D'Aprano <steve at pearwood.info> writes:
> 
> > In my opinion, the community is best served by a good comment/review 
> > system, one which avoids the worst trolling, and allows authors the 
> > right of reply, but does not allow authors to censor inconvenient but 
> > honest reviews. 

> I also think, though, that the community is best served by an objective
> repository of third-party Python packages, with information derived only
> directly from the package itself and objective data. That allows the
> least barrier to having a package maintainer want to register their
> package with such a service, which is in the interest of having it be as
> complete a registry of packages as can be.
V> 
> A community forum, on the other hand, has many characteristics that will
> be *disincentives* to a package manager for having their package appear
> there. It's never going to attract as many package maintainers as an
> impartial, objective registry; the many reasons already given here as to
> why some package maintainers *don't* want their packages in such a
> system are evidence of that.
+1

I think that there are a lot of things that could be done to make PyPi a
place where users can find *high wuality* packages and one that makes it
desirerably for developers to have their packages listed there.
Unfortunately comments *on the package main page* is not among them.

I think the PyPi/Python developers should consider the following points:

* PyPi is the de facto *official* repository of third party packages
  that form the "enhanced" stdlib

* I as a developer chose PyPi to be the *official* website of my
  projects and I want to have control over this website.

I could name a plethora of things I would do to enhance PyPi and here
are some ideas:

Better UI
---------

Right now the package consists of a single page and all information is
cramped in there. I would suggest having a "tab-style" bar containing
things like:

    * Package description/Home (default)
    * Documentation

      Right now I have to place a link to the documentation I uploaded
      to PyPi in the package description myself? Why? Include that
      automatically on the package site as soon as it is uploaded and
      tell the developers about that feature!

    * Issue tracker (either hosted on PyPi or somewhere else)
    * Downloads (for different releases)
    * Comments

      Place comments on a different page, give developers the possibilty
      to enable this feature and react to comments. Maybe include a way
      to rate comments and those with a higher rating will bubble up
      reddit style.

    * Changelog
    * ...

Automatic Package testing
-------------------------

What I really like on PyPi is that my packages are tested automatically
with Cheesecake and the order of packages when searching is determined
by this rating. This gives a huge incentive to work on the package
quality, which is a great thing IMHO. Ratings by users could influence
this ranking as well...

I would take this idea even further.

All packages should be tested automatically and *REJECTED* if they don't
meet certain quality criteria. PyPi should become a place where high
quality packages can be found, not a dump for every single "Hello World"
ever developed.

Debian will run lintian (a Debian package checker) automatically against
newly uploaded packages and reject those that don't meet a basic set of
tests.

Tests could reject packages that:

    * FTBFS
    * do not install/uninstall cleanly
    * miss important files (README, CHANGELOG, .. )
      
      This could leverage platform specific information as in: If it is
      a *nix package and includes scripts, it should have man pages for
      them, ....

    * rate poorly on automatic tests (pep8, cheesecake, pylint, ...)
    * are not GPG signed
    * have not specified a license
    * ...

I admit that this makes PyPi a much stricter place and suggest that
sections are introduced that define different rules for inclusion (like
Debian's stable, testing, unstable - main, contrib, non-free). There
could be sections like this:

    * main - manually checked packages of high quality
      This would comprise packages of a high enough quality to be deemed
      acceptable in the stdlib 

    * tested - packages not in main that rank high on automatic tests
    * ...
    * junk (better name required)

Users could then decide which parts they want pip to retrieve packages
from. We could even encourage github, bitbucket, sourceforge, ... to
provide ways to automatically query their repositories from pip so they
could be included in pip's "sources.list" and used to install
dev-versions, packages of por quality.

*IF* Python plans to develop a fully fledged platform agnostic package
manager for Python programs/libraries it would be wise to learn from the
solutions people came up with until now (aptitude, yum, pkg_add, emerge,
...)

kind regards

    Wolodja Wentland
-- 
  .''`.     Wolodja Wentland    <wentland at cl.uni-heidelberg.de> 
 : :'  :    
 `. `'`     4096R/CAF14EFC 
   `-       081C B7CD FF04 2BA9 94EA  36B2 8B7F 7D30 CAF1 4EFC
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 836 bytes
Desc: Digital signature
URL: <http://mail.python.org/pipermail/python-dev/attachments/20091113/ce03fba5/attachment.pgp>

From wa2n39 at gmail.com  Sat Nov 14 03:44:14 2009
From: wa2n39 at gmail.com (wa2n39 at gmail.com)
Date: Sat, 14 Nov 2009 02:44:14 +0000
Subject: [Python-Dev] Python-Dev Digest, Vol 76, Issue 114
Message-ID: <1038341212-1258112004-cardhu_decombobulator_blackberry.rim.net-1218332849-@bda056.bisx.prodap.on.blackberry>

Please don't send me again this email


------Pesan Asli------
Dari:python-dev-request at python.org
Pengirim:python-dev-bounces+wa2n39=gmail.com at python.org
Ke:python-dev at python.org
Balas Ke:python-dev at python.org
Perihal:Python-Dev Digest, Vol 76, Issue 114
Terkirim:13 Nov 2009 03:00

Send Python-Dev mailing list submissions to
	python-dev at python.org

To subscribe or unsubscribe via the World Wide Web, visit
	http://mail.python.org/mailman/listinfo/python-dev
or, via email, send a message with subject or body 'help' to
	python-dev-request at python.org

You can reach the person managing the list at
	python-dev-owner at python.org

When replying, please edit your Subject line so it is more specific
than "Re: Contents of Python-Dev digest..."


Today's Topics:

   1. Re: standard libraries don't behave like standard	'libraries'
      (Masklinn)


----------------------------------------------------------------------

Message: 1
Date: Fri, 13 Nov 2009 11:50:49 +0100
From: Masklinn <masklinn at masklinn.net>
To: Python-Dev <python-dev at python.org>
Subject: Re: [Python-Dev] standard libraries don't behave like
	standard	'libraries'
Message-ID: <45186ACD-29BA-4D5E-9ADB-25632021A539 at masklinn.net>
Content-Type: text/plain; charset=us-ascii

On 13 Nov 2009, at 11:36 , Anand Balachandran Pillai wrote:
> I think the recent postings on "CPAN for Python"  in this list also stems
> from similar thoughts.
No. The CPAN for Python messages (in and out of the list) are about third-party packages and a better way to manage (and a better experience of installing) those.

Think Ruby, it has a significant stdlib (though not as extensive as Python's), but it also has gems (and the `gem` CLI util). Or Haskell, which also has a significant stdlib (GHC does anyway) but also has cabal and cabal-install.

------------------------------

_______________________________________________
Python-Dev mailing list
Python-Dev at python.org
http://mail.python.org/mailman/listinfo/python-dev


End of Python-Dev Digest, Vol 76, Issue 114
*******************************************


Sent from my BlackBerry?
powered by Sinyal Kuat INDOSAT

From naughtysriram at gmail.com  Fri Nov 13 13:15:17 2009
From: naughtysriram at gmail.com (Sriram Srinivasan)
Date: Fri, 13 Nov 2009 17:15:17 +0500
Subject: [Python-Dev] standard libraries don't behave like standard
	'libraries'
In-Reply-To: <8548c5f30911130236j72d6b1a4m21d5bc714f451c43@mail.gmail.com>
References: <c910262d0911120123i7c7e9dla918fff50cb5aecd@mail.gmail.com>
	<4AFC6440.1040008@v.loewis.de>
	<8548c5f30911130236j72d6b1a4m21d5bc714f451c43@mail.gmail.com>
Message-ID: <c910262d0911130415s4f3c7ccfn2b6f0b481697756d@mail.gmail.com>

as people like to compare languages take ruby for example (i am confident
that there will be no flame war here ;) )

we have PyPI
they have RAA

we have ?
they have rubyforge

i am seeing the rubyforge site now on my other tab, i find

Communications (365 projects)
Database (245 projects)
Desktop Environment (80 projects)
Education (81 projects)
Games/Entertainment (225 projects)
Internet (1339 projects)
Multimedia (296 projects)
Office/Business (226 projects)
Other/Nonlisted Topic (82 projects)
Printing (23 projects)
Religion (7 projects)
Scientific/Engineering (278 projects)
Security (86 projects)
Sociology (16 projects)
Software Development (1164 projects)
System (548 projects)
Terminals (37 projects)
Text Editors (136 projects)

these are the ongoing library development. although they are small in number
but it is put very clear.

in python the only way i have found about the libraries we have is by using
the doc (docs is great at this).
but i find less/no details about standalone/libraries that are not part of
python stdlib. (google is great at this)

lets take RAA now,

By Category
 Application
  114 subcategories
  553 projects
 Documentation
  14 subcategories
  31 projects
 Library
  150 subcategories
  1179 projects
 Ports
  8 subcategories
  10 projects

* By Project(ALL): 286 subcategories, 1773 projects
* By Owner: 905 owners

first of all they can never ever beat python's record of packages.

its funny i found a ruby project in pypi. on further inspection [rython
0.0.1] a library module for coding in ruby inside python code
transparently... how awesome..! its happy to see that libraries already has
taken the add-on feature. sad to see libraries mixed up with software
packages (no one will find about  the library amidst those 1000's of
software!

packaging libraries has some other advantages too, like accumulation of
ideas happen. what if i design a *library* and no one uses it? whats the
point in creating? but if i could upload that in pypi or something like
that, also one python core active developer has had a look in to that, and
might one day be integrated with the standard modules..!

the best thing for now i propose is that segregating libraries and
applications in pypi. if thats a first stage bring about an awesome library
for distribution of packages (already in progress). third is chunk the
standard library and start providing the packages. seems simple but already
the python community is waiting for a long time.

-- 
Regards,
Sriram.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20091113/a8c60446/attachment.htm>

From naughtysriram at gmail.com  Fri Nov 13 13:33:38 2009
From: naughtysriram at gmail.com (Sriram Srinivasan)
Date: Fri, 13 Nov 2009 17:33:38 +0500
Subject: [Python-Dev] standard libraries don't behave like standard
	'libraries'
In-Reply-To: <c910262d0911130415s4f3c7ccfn2b6f0b481697756d@mail.gmail.com>
References: <c910262d0911120123i7c7e9dla918fff50cb5aecd@mail.gmail.com>
	<4AFC6440.1040008@v.loewis.de>
	<8548c5f30911130236j72d6b1a4m21d5bc714f451c43@mail.gmail.com>
	<c910262d0911130415s4f3c7ccfn2b6f0b481697756d@mail.gmail.com>
Message-ID: <c910262d0911130433y5c8d7e26scef96723e92b2584@mail.gmail.com>

http://pypi.python.org/pypi?%3Aaction=search&term=library&submit=search

this lists all the packages with the term 'library' in it.

-- 

Regards,
Sriram.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20091113/0a3954e5/attachment-0001.htm>

From ncoghlan at gmail.com  Fri Nov 13 14:11:59 2009
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Fri, 13 Nov 2009 23:11:59 +1000
Subject: [Python-Dev] standard libraries don't behave like
	standard	'libraries'
In-Reply-To: <c910262d0911130433y5c8d7e26scef96723e92b2584@mail.gmail.com>
References: <c910262d0911120123i7c7e9dla918fff50cb5aecd@mail.gmail.com>	<4AFC6440.1040008@v.loewis.de>	<8548c5f30911130236j72d6b1a4m21d5bc714f451c43@mail.gmail.com>	<c910262d0911130415s4f3c7ccfn2b6f0b481697756d@mail.gmail.com>
	<c910262d0911130433y5c8d7e26scef96723e92b2584@mail.gmail.com>
Message-ID: <4AFD5B1F.1060900@gmail.com>

Sriram Srinivasan wrote:
> http://pypi.python.org/pypi?%3Aaction=search&term=library&submit=search
> <http://pypi.python.org/pypi?%3Aaction=search&term=library&submit=search>
> 
> this lists all the packages with the term 'library' in it.

Sriram,

Please take this discussion to catalog-sig - python-dev isn't the place
(the fact that many of us didn't immediately know the *right* place for
the discussion indicates where PyPI sits on our personal active level of
interest).

You should find more interested (and knowledgable) folks and catalog-sig.

Regards,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------

From steve at pearwood.info  Fri Nov 13 14:14:43 2009
From: steve at pearwood.info (Steven D'Aprano)
Date: Sat, 14 Nov 2009 00:14:43 +1100
Subject: [Python-Dev] standard libraries don't behave like standard
	'libraries'
In-Reply-To: <8548c5f30911130236j72d6b1a4m21d5bc714f451c43@mail.gmail.com>
References: <c910262d0911120123i7c7e9dla918fff50cb5aecd@mail.gmail.com>
	<4AFC6440.1040008@v.loewis.de>
	<8548c5f30911130236j72d6b1a4m21d5bc714f451c43@mail.gmail.com>
Message-ID: <200911140014.43627.steve@pearwood.info>

On Fri, 13 Nov 2009 09:36:10 pm Anand Balachandran Pillai wrote:

> It could be something as simple as a "require" keyword which could
> ?pull in the depdencies if not found. Perhaps at the top of your
> module,
>
> ?require (stuff, '1.27')
> ?import stuff

So Python, the *language*, has to become a package management system as 
well as a programming language?

No thank you.

At most, perhaps there could be a package management tool in the 
standard library:

try:
  import stuff
except ImportError:
  import packman
  found = packman.requires(stuff, '1.27')
  if found:
    import stuff
  else:
    fail()



> Anyone who takes a quick look at the Python std library now is
> ?sure to feel that there is an overkill of stuff there, which could
> be classified and packaged better than dumping into the language
> build.

I don't.

I think the std library could possibly be organised better, but just 
because something isn't useful to me right now, doesn't mean it isn't 
useful to someone, and may be useful to me tomorrow.



-- 
Steven D'Aprano

From abpillai at gmail.com  Fri Nov 13 14:27:17 2009
From: abpillai at gmail.com (Anand Balachandran Pillai)
Date: Fri, 13 Nov 2009 18:57:17 +0530
Subject: [Python-Dev] standard libraries don't behave like standard
	'libraries'
In-Reply-To: <200911140014.43627.steve@pearwood.info>
References: <c910262d0911120123i7c7e9dla918fff50cb5aecd@mail.gmail.com>
	<4AFC6440.1040008@v.loewis.de>
	<8548c5f30911130236j72d6b1a4m21d5bc714f451c43@mail.gmail.com>
	<200911140014.43627.steve@pearwood.info>
Message-ID: <8548c5f30911130527g5494b825y357acd95abac3090@mail.gmail.com>

On Fri, Nov 13, 2009 at 6:44 PM, Steven D'Aprano <steve at pearwood.info>wrote:

> On Fri, 13 Nov 2009 09:36:10 pm Anand Balachandran Pillai wrote:
>
> > It could be something as simple as a "require" keyword which could
> >  pull in the depdencies if not found. Perhaps at the top of your
> > module,
> >
> >  require (stuff, '1.27')
> >  import stuff
>
> So Python, the *language*, has to become a package management system as
> well as a programming language?
>
> No thank you.
>
> At most, perhaps there could be a package management tool in the
> standard library:
>
> try:
>  import stuff
> except ImportError:
>  import packman
>  found = packman.requires(stuff, '1.27')
>  if found:
>    import stuff
>  else:
>    fail()
>
>
>
 That was just a suggestion, don't take it literally. My intention
was something similar to the stuff you have written above.
 It would be however useful to standardize the pkg manager
 "packman" to something in the standard library however.


>
> > Anyone who takes a quick look at the Python std library now is
> >  sure to feel that there is an overkill of stuff there, which could
> > be classified and packaged better than dumping into the language
> > build.
>
> I don't.
>
> I think the std library could possibly be organised better, but just
> because something isn't useful to me right now, doesn't mean it isn't
> useful to someone, and may be useful to me tomorrow.
>
>
>
 Ah yes, the only problem is that for any updates/fixes to your
 module, you need to wait for the language release which is not
 very much an ideal situation IMHO. Especially now when we are
 in discussions about end of life of 2.x etc...

 W.r.t the original analogy, batteries v 1.1 are included with the torch
1.1,
 but if you want battery upgrade to 1.2, you need to buy torch 1.2..


>
> --
> Steven D'Aprano
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> http://mail.python.org/mailman/options/python-dev/abpillai%40gmail.com
>



-- 
--Anand
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20091113/6f428e84/attachment.htm>

From naughtysriram at gmail.com  Fri Nov 13 15:17:04 2009
From: naughtysriram at gmail.com (Sriram Srinivasan)
Date: Fri, 13 Nov 2009 19:17:04 +0500
Subject: [Python-Dev] Fwd: standard libraries don't behave like standard
	'libraries'
In-Reply-To: <c910262d0911130613l3284d76j993497cc41439173@mail.gmail.com>
References: <c910262d0911120123i7c7e9dla918fff50cb5aecd@mail.gmail.com>
	<4AFC6440.1040008@v.loewis.de>
	<8548c5f30911130236j72d6b1a4m21d5bc714f451c43@mail.gmail.com>
	<c910262d0911130415s4f3c7ccfn2b6f0b481697756d@mail.gmail.com>
	<c910262d0911130433y5c8d7e26scef96723e92b2584@mail.gmail.com>
	<4AFD5B1F.1060900@gmail.com>
	<c910262d0911130613l3284d76j993497cc41439173@mail.gmail.com>
Message-ID: <c910262d0911130617s1c58d308o39043c5096627edd@mail.gmail.com>

Sriram,
>
> Please take this discussion to catalog-sig - python-dev isn't the place
> (the fact that many of us didn't immediately know the *right* place for
> the discussion indicates where PyPI sits on our personal active level of
> interest).
>
> You should find more interested (and knowledgable) folks and catalog-sig.
>
> Regards,
> Nick.
>


sorry for posting the link in a lone post, i must have posted it with my
previous post, and the theme is how to make the standard libraries
state-of-the-art and i many feel it must be in py-dev as it is directly
connected with the development of python. As it required some comparison, i
compared pypi with the ruby's *rubyforge*. in fact what i found astonishing
is that there are a lot of libraries (3rd party) available in packaged
forms. which is a very good sign, the problem is there is no sorted out list
for libraries and software in pypi.

*rubyforge* is like a pypi for *libraries* (standard and 3rd party)

i guess some misplaced topics such as [Python-Dev] PyPI comments and
ratings, *really*? , [Python-Dev] PyPI front page
are also getting alarming responses in the py-dev list. including yours, ;)


As Anand says requiring a new update of battery v1.5 requires update of
python/flashlight which only occurs every release is the manin senario.

I would also like to add that what if a company doesn't need the batteries
that you give at the first place, they have developed their own batteries
and want to use them(or use some from the default with theit own), also
wants to use the same flashlight/python system? - it would be *easy* only if
the batteries included had a feature of being removed.

-- 

Regards,
Sriram.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20091113/b68248b9/attachment.htm>

From naughtysriram at gmail.com  Fri Nov 13 15:18:54 2009
From: naughtysriram at gmail.com (Sriram Srinivasan)
Date: Fri, 13 Nov 2009 19:18:54 +0500
Subject: [Python-Dev] standard libraries don't behave like standard
	'libraries'
In-Reply-To: <c910262d0911130613l3284d76j993497cc41439173@mail.gmail.com>
References: <c910262d0911120123i7c7e9dla918fff50cb5aecd@mail.gmail.com>
	<4AFC6440.1040008@v.loewis.de>
	<8548c5f30911130236j72d6b1a4m21d5bc714f451c43@mail.gmail.com>
	<c910262d0911130415s4f3c7ccfn2b6f0b481697756d@mail.gmail.com>
	<c910262d0911130433y5c8d7e26scef96723e92b2584@mail.gmail.com>
	<4AFD5B1F.1060900@gmail.com>
	<c910262d0911130613l3284d76j993497cc41439173@mail.gmail.com>
Message-ID: <c910262d0911130618y53b2f1a1l5f3be5d1adbd3677@mail.gmail.com>

> Sriram,
>
> Please take this discussion to catalog-sig - python-dev isn't the place
> (the fact that many of us didn't immediately know the *right* place for
> the discussion indicates where PyPI sits on our personal active level of
> interest).
>
> You should find more interested (and knowledgable) folks and catalog-sig.
>
> Regards,
> Nick.
>


sorry for posting the link in a lone post, i must have posted it with my
previous post, and the theme is how to make the standard libraries
state-of-the-art and i many feel it must be in py-dev as it is directly
connected with the development of python. As it required some comparison, i
compared pypi with the ruby's *rubyforge*. in fact what i found astonishing
is that there are a lot of libraries (3rd party) available in packaged
forms. which is a very good sign, the problem is there is no sorted out list
for libraries and software in pypi.

*rubyforge* is like a pypi for *libraries* (standard and 3rd party)

i guess some misplaced topics such as [Python-Dev] PyPI comments and
ratings, *really*? , [Python-Dev] PyPI front page
are also getting alarming responses in the py-dev list. including yours, ;)


As Anand says requiring a new update of battery v1.5 requires update of
python/flashlight which only occurs every release is the manin senario.

I would also like to add that what if a company doesn't need the batteries
that you give at the first place, they have developed their own batteries
and want to use them(or use some from the default with theit own), also
wants to use the same flashlight/python system? - it would be *easy* only if
the batteries included had a feature of being removed.

-- 

Regards,
Sriram
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20091113/4e01e033/attachment-0001.htm>

From techtonik at gmail.com  Fri Nov 13 17:14:37 2009
From: techtonik at gmail.com (anatoly techtonik)
Date: Fri, 13 Nov 2009 18:14:37 +0200
Subject: [Python-Dev] PyPI front page
In-Reply-To: <4AFD0C54.1020709@v.loewis.de>
References: <D7E23A6D-C91C-46D8-A8FE-1EB16F3B2101@lericson.se> 
	<hdi17l$cli$1@ger.gmane.org> <4AFC9064.20700@v.loewis.de> 
	<4222a8490911121511s40eba937n2c7b1fde6f695c6c@mail.gmail.com> 
	<4AFC9975.7040204@v.loewis.de>
	<4222a8490911121534h30572eabv73ae58649c77155c@mail.gmail.com> 
	<4AFC9D0C.5030500@v.loewis.de> <87y6mbuubp.fsf@benfinney.id.au> 
	<loom.20091113T015130-137@post.gmane.org>
	<4AFD0C54.1020709@v.loewis.de>
Message-ID: <d34314100911130814m20acf9f3ta4a9d319f1a1a6ce@mail.gmail.com>

On Fri, Nov 13, 2009 at 9:35 AM, "Martin v. L?wis" <martin at v.loewis.de> wrote:
>> Ben Finney <ben+python <at> benfinney.id.au> writes:
>>> There's a problem with the poll's placement: on the front page of the
>>> PyPI website.
>>
>> Speaking of which, why is it that http://pypi.python.org/pypi and
>> http://pypi.python.org/pypi/ (note the ending slash) return different contents
>> (the latter being very voluminous)? I always mistake one for the other when
>> entering the URL directly.
>
> As Ian says: setuptools relies on it. It's part of the specification of
> the package index.

Where is that specification? Does it require only
http://pypi.python.org/pypi/ to be voluminous or also
http://pypi.python.org/pypi to provide some search interface for
automatic processing? There should be no problem in moving search page
to the root http://pypi.python.org/ if spec is silent about the
latter.

-- 
anatoly t.

From status at bugs.python.org  Fri Nov 13 18:07:41 2009
From: status at bugs.python.org (Python tracker)
Date: Fri, 13 Nov 2009 18:07:41 +0100 (CET)
Subject: [Python-Dev] Summary of Python tracker Issues
Message-ID: <20091113170741.A325B78364@psf.upfronthosting.co.za>


ACTIVITY SUMMARY (11/06/09 - 11/13/09)
Python tracker at http://bugs.python.org/

To view or respond to any of the issues listed below, click on the issue 
number.  Do NOT respond to this message.


 2491 open (+27) / 16645 closed (+14) / 19136 total (+41)

Open issues with patches:   993

Average duration of open issues: 684 days.
Median duration of open issues: 442 days.

Open Issues Breakdown
   open  2455 (+27)
pending    35 ( +0)

Issues Created Or Reopened (42)
_______________________________

test_debuglevel from test_telnetlib.py fails                     11/06/09
CLOSED http://bugs.python.org/issue6748    reopened r.david.murray                
       patch, buildbot                                                         

json module apparently fails regression found at http://json.org 11/06/09
CLOSED http://bugs.python.org/issue7274    created  DougShawhan                   
                                                                               

CoverageResult fails to merge input file with non-empty callers  11/06/09
       http://bugs.python.org/issue7275    created  glinsvad                      
       patch                                                                   

UnboundLocalError scoping problem with nested functions          11/06/09
CLOSED http://bugs.python.org/issue7276    created  olau                          
                                                                               

online docs out-of-sync with sources and distributed .chm files  11/07/09
CLOSED http://bugs.python.org/issue7277    created  gagenellina                   
                                                                               

decimal.py: New instance vs. new reference                       11/07/09
CLOSED http://bugs.python.org/issue7278    created  skrah                         
                                                                               

decimal.py: == and != comparisons involving NaNs                 11/07/09
CLOSED http://bugs.python.org/issue7279    created  skrah                         
                                                                               

PCBuild instruction says to use nasmw.exe but it no longer exist 11/07/09
       http://bugs.python.org/issue7280    created  blep                          
       patch                                                                   

copysign() with NaN arguments on OpenSolaris                     11/07/09
CLOSED http://bugs.python.org/issue7281    created  skrah                         
                                                                               

RLocks leak references when used in raw threads                  11/08/09
CLOSED http://bugs.python.org/issue7282    created  pitrou                        
       patch                                                                   

test_site failure when .local/lib/pythonX.Y/site-packages hasn't 11/08/09
       http://bugs.python.org/issue7283    created  pitrou                        
                                                                               

optparse - display version in usage by default                   11/08/09
       http://bugs.python.org/issue7284    created  techtonik                     
                                                                               

multiprocessing module, example code error                       11/08/09
       http://bugs.python.org/issue7285    created  hsmtkk                        
       patch                                                                   

odd exec() behavior or documentation                             11/08/09
       http://bugs.python.org/issue7286    created  pmawhorter                    
                                                                               

import hook demo does not work                                   11/08/09
       http://bugs.python.org/issue7287    created  blep                          
                                                                               

Detect improper leading whitespace in C files for Vim            11/08/09
       http://bugs.python.org/issue7288    created  brett.cannon                  
       easy                                                                    

bad import in redemo.py script in v3.1                           11/08/09
CLOSED http://bugs.python.org/issue7289    created  alan.gauld at btinternet.com     
                                                                               

Update 'how do I set a global variable' faq entry                11/09/09
CLOSED http://bugs.python.org/issue7290    created  r.david.murray                
       patch                                                                   

urllib2 cannot handle https with proxy requiring auth            11/09/09
       http://bugs.python.org/issue7291    created  tsujikawa                     
       patch                                                                   

Multiprocessing Joinable race condition?                         11/09/09
       http://bugs.python.org/issue7292    created  jonathan                      
                                                                               

test_msvc9compiler test_reg_class failure on new Windows box     11/09/09
       http://bugs.python.org/issue7293    created  db3l                          
                                                                               

MSDN URL in subprocess.py is incorrect                           11/09/09
CLOSED http://bugs.python.org/issue7294    created  eric.smith                    
       easy                                                                    

test_tarfile uses a hardcoded file name                          11/09/09
CLOSED http://bugs.python.org/issue7295    created  pitrou                        
       patch                                                                   

OverflowError: signed integer is greater than maximum on mips64  11/09/09
       http://bugs.python.org/issue7296    created  jasper                        
                                                                               

--spam--                                                         11/10/09
CLOSED http://bugs.python.org/issue7297    created  brandon.dixon                 
                                                                               

reversed(range(x, -1, -1)) is empty when x > 1                   11/10/09
       http://bugs.python.org/issue7298    created  ledave123                     
       patch                                                                   

setup.py install doesn't honor PYTHONUSERBASE                    11/10/09
       http://bugs.python.org/issue7299    created  doko                          
       patch, patch                                                            

Unicode arguments in str.format()                                11/10/09
       http://bugs.python.org/issue7300    created  doerwalter                    
                                                                               

Add environment variable $PYTHONWARNINGS                         11/10/09
       http://bugs.python.org/issue7301    created  barry                         
                                                                               

Wrong link to Python Language Mapping Specification              11/10/09
       http://bugs.python.org/issue7302    created  efrerich                      
                                                                               

pkgutil lacks documentation for useful functions                 11/10/09
       http://bugs.python.org/issue7303    created  englabenny                    
                                                                               

email.message.Message.set_payload and as_string given charset 'u 11/10/09
       http://bugs.python.org/issue7304    created  r.david.murray                
                                                                               

urllib2.urlopen() segfault using SSL on Solaris                  11/11/09
       http://bugs.python.org/issue7305    created  rtarnell                      
                                                                               

Patch - skip winsound tests if no default sound is configured    11/11/09
       http://bugs.python.org/issue7306    created  brian.curtin                  
       patch                                                                   

test_distutils failure under Windows 7                           11/11/09
CLOSED http://bugs.python.org/issue7307    created  pitrou                        
                                                                               

Named group regex error                                          11/11/09
CLOSED http://bugs.python.org/issue7308    created  pythonmeister                 
                                                                               

crasher in str(Exception())                                      11/12/09
       http://bugs.python.org/issue7309    created  arigo                         
                                                                               

Unhelpful __repr__() in os.environ                               11/12/09
       http://bugs.python.org/issue7310    created  kjohnson                      
                                                                               

Bug on regexp of HTMLParser                                      11/12/09
       http://bugs.python.org/issue7311    created  pluskid                       
                                                                               

Run some tests in a loop until failure                           11/12/09
       http://bugs.python.org/issue7312    created  pitrou                        
       patch                                                                   

2to3 fails when source file starts with BOM                      11/13/09
       http://bugs.python.org/issue7313    created  gagenellina                   
                                                                               

test_multiprocessing hangs on Windows 7 64 bit                   11/13/09
       http://bugs.python.org/issue7314    created  krisvale                      
       64bit                                                                   



Issues Now Closed (31)
______________________

CGIHTTPRequestHandler.run_cgi() does not run on Windows if sys.e  769 days
       http://bugs.python.org/issue1235    orsenthil                     
       patch                                                                   

Management of KeyboardInterrupt in cmd.py                         710 days
       http://bugs.python.org/issue1294    gvanrossum                    
       patch                                                                   

RLock's are SLOW                                                  530 days
       http://bugs.python.org/issue3001    pitrou                        
       patch                                                                   

Real segmentation fault handler                                   406 days
       http://bugs.python.org/issue3999    Rhamphoryncus                 
       patch                                                                   

No universal newline support for compile() when using bytes       337 days
       http://bugs.python.org/issue4628    benjamin.peterson             
                                                                               

Auto-detect indentation in C source in vimrc                      223 days
       http://bugs.python.org/issue5611    brett.cannon                  
       patch                                                                   

Subclassing property doesn't preserve the auto __doc__ behavior   195 days
       http://bugs.python.org/issue5890    moriyoshi                     
       patch, needs review                                                     

test_codecs fails when ran after test_zipimport and test_mailbox  114 days
       http://bugs.python.org/issue6551    pitrou                        
       patch                                                                   

test_debuglevel from test_telnetlib.py fails                        0 days
       http://bugs.python.org/issue6748    jackdied                      
       patch, buildbot                                                         

round(x) gives wrong result for large odd integers                 34 days
       http://bugs.python.org/issue7070    mark.dickinson                
       easy                                                                    

Add looping capability to regrtest                                 29 days
       http://bugs.python.org/issue7134    pitrou                        
                                                                               

importlib/_bootstrap.py write_bytecode raises an IOError if it c   16 days
       http://bugs.python.org/issue7187    brett.cannon                  
       patch                                                                   

test_multiprocessing crashes under Windows when run in verbose m   18 days
       http://bugs.python.org/issue7197    pitrou                        
       patch                                                                   

2to3 execfile conversion changes behavior                           1 days
       http://bugs.python.org/issue7268    gagenellina                   
       patch                                                                   

Tests for synchronization primitives                                0 days
       http://bugs.python.org/issue7270    pitrou                        
       patch                                                                   

Typo in turtle module documentation.                                1 days
       http://bugs.python.org/issue7271    georg.brandl                  
                                                                               

json module apparently fails regression found at http://json.org    2 days
       http://bugs.python.org/issue7274    DougShawhan                   
                                                                               

UnboundLocalError scoping problem with nested functions             3 days
       http://bugs.python.org/issue7276    r.david.murray                
                                                                               

online docs out-of-sync with sources and distributed .chm files     0 days
       http://bugs.python.org/issue7277    r.david.murray                
                                                                               

decimal.py: New instance vs. new reference                          1 days
       http://bugs.python.org/issue7278    rhettinger                    
                                                                               

decimal.py: == and != comparisons involving NaNs                    1 days
       http://bugs.python.org/issue7279    mark.dickinson                
                                                                               

copysign() with NaN arguments on OpenSolaris                        2 days
       http://bugs.python.org/issue7281    mark.dickinson                
                                                                               

RLocks leak references when used in raw threads                     2 days
       http://bugs.python.org/issue7282    pitrou                        
       patch                                                                   

bad import in redemo.py script in v3.1                              0 days
       http://bugs.python.org/issue7289    benjamin.peterson             
                                                                               

Update 'how do I set a global variable' faq entry                   2 days
       http://bugs.python.org/issue7290    r.david.murray                
       patch                                                                   

MSDN URL in subprocess.py is incorrect                              0 days
       http://bugs.python.org/issue7294    eric.smith                    
       easy                                                                    

test_tarfile uses a hardcoded file name                             2 days
       http://bugs.python.org/issue7295    pitrou                        
       patch                                                                   

--spam--                                                            0 days
       http://bugs.python.org/issue7297    rhettinger                    
                                                                               

test_distutils failure under Windows 7                              1 days
       http://bugs.python.org/issue7307    amaury.forgeotdarc            
                                                                               

Named group regex error                                             0 days
       http://bugs.python.org/issue7308    mark.dickinson                
                                                                               

'compile' built-in function failures when missing EOL            1293 days
       http://bugs.python.org/issue1479099 benjamin.peterson             
                                                                               



Top Issues Most Discussed (10)
______________________________

 19 Enable short float repr() on Solaris/x86                         208 days
open    http://bugs.python.org/issue5792   

 10 RLock's are SLOW                                                 530 days
closed  http://bugs.python.org/issue3001   

  7 reversed(range(x, -1, -1)) is empty when x > 1                     3 days
open    http://bugs.python.org/issue7298   

  7 test_multiprocessing dictionary changed size errors and hang      40 days
open    http://bugs.python.org/issue7060   

  6 OverflowError: signed integer is greater than maximum on mips64    4 days
open    http://bugs.python.org/issue7296   

  6 CGIHTTPRequestHandler.run_cgi() does not run on Windows if sys.  769 days
closed  http://bugs.python.org/issue1235   

  5 Management of KeyboardInterrupt in cmd.py                        710 days
closed  http://bugs.python.org/issue1294   

  4 Run some tests in a loop until failure                             1 days
open    http://bugs.python.org/issue7312   

  4 copysign() with NaN arguments on OpenSolaris                       2 days
closed  http://bugs.python.org/issue7281   

  4 test_debuglevel from test_telnetlib.py fails                       0 days
closed  http://bugs.python.org/issue6748   




From rhamph at gmail.com  Fri Nov 13 19:18:45 2009
From: rhamph at gmail.com (Adam Olsen)
Date: Fri, 13 Nov 2009 11:18:45 -0700
Subject: [Python-Dev] decimal.py: == and != comparisons involving NaNs
In-Reply-To: <5c6f2a5d0911090501n23254cc7x8e1eaad34508f8fa@mail.gmail.com>
References: <1257590632.12.0.893741146745.issue7279@psf.upfronthosting.co.za>
	<1257672530.83.0.836585441467.issue7279@psf.upfronthosting.co.za>
	<20091108162648.GA4099@mail.bytereef.org>
	<5c6f2a5d0911090501n23254cc7x8e1eaad34508f8fa@mail.gmail.com>
Message-ID: <aac2c7cb0911131018s43b95428o2599d65ae47bf99c@mail.gmail.com>

On Mon, Nov 9, 2009 at 06:01, Mark Dickinson <dickinsm at gmail.com> wrote:
> Well, when running in some form of 'non-stop' mode, where (quiet) NaN
> results are supposed to be propagated to the end of a computation, you
> certainly want equality comparisons with nan just to silently return false.
> E.g., in code like:
>
> if x == 0:
> ? ?<deal with zero special case>
> else:
> ? ?<usual algorithm>
>
> nans should just end up in the second branch, without the programmer
> having had to think about it too hard.

if x != 0:
    <usual algorithm>
else:
    <deal with zero special case>

nans should just end up in the first branch, without the programmer
having had to think about it too hard.

There is a more consistent alternative: have all comparisons involving
NaN also return NaN, signifying they're unordered.  Let bool coercion
raise the exception.  Thus, both examples would raise an exception,
but a programmer who wants to handle NaN could do so explicitly:

temp = x == 0
if temp.isnan() or temp:
    <usual algorithm>
else:
    <deal with zero special case>

IEEE 754 is intended for a very different context.  I don't think it
makes sense to attempt literal conformance to it.


-- 
Adam Olsen, aka Rhamphoryncus

From p.f.moore at gmail.com  Fri Nov 13 20:33:18 2009
From: p.f.moore at gmail.com (Paul Moore)
Date: Fri, 13 Nov 2009 19:33:18 +0000
Subject: [Python-Dev] PyPI comments and ratings, *really*?
In-Reply-To: <hdim34$utf$1@ger.gmane.org>
References: <D7E23A6D-C91C-46D8-A8FE-1EB16F3B2101@lericson.se>
	<hdi17l$cli$1@ger.gmane.org> <4AFC9064.20700@v.loewis.de>
	<4222a8490911121511s40eba937n2c7b1fde6f695c6c@mail.gmail.com>
	<4AFC9975.7040204@v.loewis.de>
	<4222a8490911121534h30572eabv73ae58649c77155c@mail.gmail.com>
	<4AFC9D0C.5030500@v.loewis.de> <87y6mbuubp.fsf@benfinney.id.au>
	<20e5c9660911121702r162caf28jb6d656383147d909@mail.gmail.com>
	<hdim34$utf$1@ger.gmane.org>
Message-ID: <79990c6b0911131133i2987a5d4s5c4ef780a527732@mail.gmail.com>

2009/11/13 Tres Seaver <tseaver at palladion.com>:
> I can see the information about the poll, and a link to view the
> results, without logging in.
>
> ?http://pypi.python.org/pypi
>
> (second paragraph there). ?That paragraph tells you that you need to log
> in to vote in the poll.

I don't want to create a PyPI account (more account details I'll
rarely use to remember) just to vote. I can see why anonymous votes
are bad, but the sample's going to be self-selecting - people like me
who don't want to create an account will be excluded.

By the way, there's no way for me to vote that I don't care what
option ends up being chosen, but I strongly oppose choosing something
that would tend to make developers avoid using PyPI.

The distutils/setuptools/distribute/... saga has fragmented Python's
package ecosystem enough already. Let's not have PyPI go the same way
:-(

Paul.

From g.brandl at gmx.net  Fri Nov 13 23:04:25 2009
From: g.brandl at gmx.net (Georg Brandl)
Date: Fri, 13 Nov 2009 23:04:25 +0100
Subject: [Python-Dev] PyPI comments and ratings, *really*?
In-Reply-To: <87tywzurst.fsf@benfinney.id.au>
References: <D7E23A6D-C91C-46D8-A8FE-1EB16F3B2101@lericson.se>	<hdhrvs$qs8$1@ger.gmane.org>	<Pine.LNX.4.64.0911121648480.12274@kimball.webabinitio.net>	<hdi17l$cli$1@ger.gmane.org>
	<4AFC9064.20700@v.loewis.de>	<4222a8490911121511s40eba937n2c7b1fde6f695c6c@mail.gmail.com>	<4AFC9975.7040204@v.loewis.de>	<4222a8490911121534h30572eabv73ae58649c77155c@mail.gmail.com>	<4AFC9D0C.5030500@v.loewis.de>
	<87y6mbuubp.fsf@benfinney.id.au>	<20e5c9660911121702r162caf28jb6d656383147d909@mail.gmail.com>
	<87tywzurst.fsf@benfinney.id.au>
Message-ID: <hdkhqc$k7k$1@ger.gmane.org>

Ben Finney schrieb:
> Michael Sparks <sparks.m at gmail.com> writes:
> 
>> On Fri, Nov 13, 2009 at 12:44 AM, Ben Finney <ben+python at benfinney.id.au> wrote:
>> > So, the poll's audience is limited to those who visit the front page
>> > (which is hardly ever necessary for package maintainers), and those
>> > who already know it exists (e.g. through this discussion thread).
>> > You'll be missing the opinions of those maintainers who, like the OP
>> > of this thread, only discovered the behaviour much later.
>>
>> This poll is only visible if you're logged into PyPI. This strikes me
>> as a mistake. I went looking for a poll and didn't see it.
> 
> The mistake, I think, is having a poll basically asking ?what should the
> PyPI maintainers do??, instead of weighing evidence and reasoned
> arguments. A poll may be good for gathering preferences and opinion, but
> it's a poor way to make a *decision*.

I haven't seen Martin commit to doing whatever the poll's preferred outcome
was.  I think it's just to see if there is really so much silent resistance
or support as is claimed by both arguing sides.

Georg

-- 
Thus spake the Lord: Thou shalt indent with four spaces. No more, no less.
Four shall be the number of spaces thou shalt indent, and the number of thy
indenting shall be four. Eight shalt thou not indent, nor either indent thou
two, excepting that thou then proceed to four. Tabs are right out.


From matthew at woodcraft.me.uk  Fri Nov 13 22:34:32 2009
From: matthew at woodcraft.me.uk (Matthew Woodcraft)
Date: Fri, 13 Nov 2009 21:34:32 +0000 (UTC)
Subject: [Python-Dev] PyPI comments and ratings, *really*?
References: <D7E23A6D-C91C-46D8-A8FE-1EB16F3B2101@lericson.se>
	<4AFC9975.7040204@v.loewis.de>
	<4222a8490911121534h30572eabv73ae58649c77155c@mail.gmail.com>
	<4AFC9D0C.5030500@v.loewis.de>
Message-ID: <hdkjd8$psr$1@ger.gmane.org>

Martin v. L?wis <martin at v.loewis.de> wrote:
> Because I want to wait for the outcome of the poll first.

The pypi front page says:

| The poll will be closed on December 1, 2009. The option that receives
| most votes will be implemented.


As I write, the option with the most votes is /Allow both ratings and
comments/.

But between them, /Disallow comments/ and /Disallow both ratings and
comments/ have more votes.

It seems clear to me that, given those figures, allowing comments would
not be following the result of the vote.

So I think that simply implementing the option that receives most votes
is not a wise plan.


(There are two other options; I don't think their presence makes the
plan any wiser.)

-M-


From solipsis at pitrou.net  Fri Nov 13 22:38:31 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Fri, 13 Nov 2009 21:38:31 +0000 (UTC)
Subject: [Python-Dev] PyPI comments and ratings, *really*?
References: <D7E23A6D-C91C-46D8-A8FE-1EB16F3B2101@lericson.se>
	<4AFC9975.7040204@v.loewis.de>
	<4222a8490911121534h30572eabv73ae58649c77155c@mail.gmail.com>
	<4AFC9D0C.5030500@v.loewis.de> <hdkjd8$psr$1@ger.gmane.org>
Message-ID: <loom.20091113T223757-463@post.gmane.org>

Matthew Woodcraft <matthew <at> woodcraft.me.uk> writes:
> 
> It seems clear to me that, given those figures, allowing comments would
> not be following the result of the vote.
> 
> So I think that simply implementing the option that receives most votes
> is not a wise plan.

Can we defer any further discussion about this until the poll is closed?

Thank you

Antoine.



From ben+python at benfinney.id.au  Fri Nov 13 22:43:25 2009
From: ben+python at benfinney.id.au (Ben Finney)
Date: Sat, 14 Nov 2009 08:43:25 +1100
Subject: [Python-Dev] PyPI comments and ratings, *really*?
References: <D7E23A6D-C91C-46D8-A8FE-1EB16F3B2101@lericson.se>
	<hdi17l$cli$1@ger.gmane.org> <4AFC9064.20700@v.loewis.de>
	<4222a8490911121511s40eba937n2c7b1fde6f695c6c@mail.gmail.com>
	<4AFC9975.7040204@v.loewis.de>
	<4222a8490911121534h30572eabv73ae58649c77155c@mail.gmail.com>
	<4AFC9D0C.5030500@v.loewis.de> <87y6mbuubp.fsf@benfinney.id.au>
	<20e5c9660911121702r162caf28jb6d656383147d909@mail.gmail.com>
	<hdim34$utf$1@ger.gmane.org>
	<79990c6b0911131133i2987a5d4s5c4ef780a527732@mail.gmail.com>
Message-ID: <87k4xut81u.fsf@benfinney.id.au>

Paul Moore <p.f.moore at gmail.com> writes:

> By the way, there's no way for me to vote that I don't care what
> option ends up being chosen, but I strongly oppose choosing something
> that would tend to make developers avoid using PyPI.

There's also no option to vote that decisions on how to manage Python
infrastructure (like PyPI) shouldn't be made by popular vote.

-- 
 \          ?I like to reminisce with people I don't know. Granted, it |
  `\                                     takes longer.? ?Steven Wright |
_o__)                                                                  |
Ben Finney


From dickinsm at gmail.com  Fri Nov 13 22:50:57 2009
From: dickinsm at gmail.com (Mark Dickinson)
Date: Fri, 13 Nov 2009 21:50:57 +0000
Subject: [Python-Dev] decimal.py: == and != comparisons involving NaNs
In-Reply-To: <aac2c7cb0911131018s43b95428o2599d65ae47bf99c@mail.gmail.com>
References: <1257590632.12.0.893741146745.issue7279@psf.upfronthosting.co.za>
	<1257672530.83.0.836585441467.issue7279@psf.upfronthosting.co.za>
	<20091108162648.GA4099@mail.bytereef.org>
	<5c6f2a5d0911090501n23254cc7x8e1eaad34508f8fa@mail.gmail.com>
	<aac2c7cb0911131018s43b95428o2599d65ae47bf99c@mail.gmail.com>
Message-ID: <5c6f2a5d0911131350r1e07af33pca49ebb1d794bdd3@mail.gmail.com>

On Fri, Nov 13, 2009 at 6:18 PM, Adam Olsen <rhamph at gmail.com> wrote:
> On Mon, Nov 9, 2009 at 06:01, Mark Dickinson <dickinsm at gmail.com> wrote:
>> Well, when running in some form of 'non-stop' mode, where (quiet) NaN
>> results are supposed to be propagated to the end of a computation, you
>> certainly want equality comparisons with nan just to silently return false.
>> E.g., in code like:
>>
>> if x == 0:
>> ? ?<deal with zero special case>
>> else:
>> ? ?<usual algorithm>
>>
>> nans should just end up in the second branch, without the programmer
>> having had to think about it too hard.
>
> if x != 0:
> ? ?<usual algorithm>
> else:
> ? ?<deal with zero special case>
>
> nans should just end up in the first branch, without the programmer
> having had to think about it too hard.

And they do:  nan != 0 returns False.  Maybe I'm missing your point
here?

> IEEE 754 is intended for a very different context. ?I don't think it
> makes sense to attempt literal conformance to it.

I disagree.  The decimal specification is tied closely to
IEEE 754 (the 2008 version), to the extent that even minor changes
made by the 754r working group were mirrored in the decimal
specification as it evolved;  it only stopped evolving after IEEE
754-2008 was complete.

IEEE 754-2008 also makes a point of targeting languages, rather than
just floating-point hardware;  to me it seems very much applicable
to decimal.py.

Mark

From dickinsm at gmail.com  Fri Nov 13 22:52:10 2009
From: dickinsm at gmail.com (Mark Dickinson)
Date: Fri, 13 Nov 2009 21:52:10 +0000
Subject: [Python-Dev] decimal.py: == and != comparisons involving NaNs
In-Reply-To: <5c6f2a5d0911131350r1e07af33pca49ebb1d794bdd3@mail.gmail.com>
References: <1257590632.12.0.893741146745.issue7279@psf.upfronthosting.co.za>
	<1257672530.83.0.836585441467.issue7279@psf.upfronthosting.co.za>
	<20091108162648.GA4099@mail.bytereef.org>
	<5c6f2a5d0911090501n23254cc7x8e1eaad34508f8fa@mail.gmail.com>
	<aac2c7cb0911131018s43b95428o2599d65ae47bf99c@mail.gmail.com>
	<5c6f2a5d0911131350r1e07af33pca49ebb1d794bdd3@mail.gmail.com>
Message-ID: <5c6f2a5d0911131352w47d46fffo3b54b8fa8995a332@mail.gmail.com>

On Fri, Nov 13, 2009 at 9:50 PM, Mark Dickinson <dickinsm at gmail.com> wrote:
> And they do: ?nan != 0 returns False. ?Maybe I'm missing your point
> here?

Aargh!  True!  I meant to say True!

Mark

From rhamph at gmail.com  Fri Nov 13 23:08:59 2009
From: rhamph at gmail.com (Adam Olsen)
Date: Fri, 13 Nov 2009 15:08:59 -0700
Subject: [Python-Dev] decimal.py: == and != comparisons involving NaNs
In-Reply-To: <5c6f2a5d0911131352w47d46fffo3b54b8fa8995a332@mail.gmail.com>
References: <1257590632.12.0.893741146745.issue7279@psf.upfronthosting.co.za>
	<1257672530.83.0.836585441467.issue7279@psf.upfronthosting.co.za>
	<20091108162648.GA4099@mail.bytereef.org>
	<5c6f2a5d0911090501n23254cc7x8e1eaad34508f8fa@mail.gmail.com>
	<aac2c7cb0911131018s43b95428o2599d65ae47bf99c@mail.gmail.com>
	<5c6f2a5d0911131350r1e07af33pca49ebb1d794bdd3@mail.gmail.com>
	<5c6f2a5d0911131352w47d46fffo3b54b8fa8995a332@mail.gmail.com>
Message-ID: <aac2c7cb0911131408p6d8ec666j896cd794a98c6840@mail.gmail.com>

On Fri, Nov 13, 2009 at 14:52, Mark Dickinson <dickinsm at gmail.com> wrote:
> On Fri, Nov 13, 2009 at 9:50 PM, Mark Dickinson <dickinsm at gmail.com> wrote:
>> And they do: ?nan != 0 returns False. ?Maybe I'm missing your point
>> here?
>
> Aargh! ?True! ?I meant to say True!

Huh.  Somewhere along the line I lost track of how python handled NaN.
 I thought "comparisons always evaluate to false" was the rule.


-- 
Adam Olsen, aka Rhamphoryncus

From amk at amk.ca  Fri Nov 13 23:12:34 2009
From: amk at amk.ca (A.M. Kuchling)
Date: Fri, 13 Nov 2009 17:12:34 -0500
Subject: [Python-Dev] PyPI comments and ratings, *really*?
In-Reply-To: <87k4xut81u.fsf@benfinney.id.au>
References: <4AFC9064.20700@v.loewis.de>
	<4222a8490911121511s40eba937n2c7b1fde6f695c6c@mail.gmail.com>
	<4AFC9975.7040204@v.loewis.de>
	<4222a8490911121534h30572eabv73ae58649c77155c@mail.gmail.com>
	<4AFC9D0C.5030500@v.loewis.de> <87y6mbuubp.fsf@benfinney.id.au>
	<20e5c9660911121702r162caf28jb6d656383147d909@mail.gmail.com>
	<hdim34$utf$1@ger.gmane.org>
	<79990c6b0911131133i2987a5d4s5c4ef780a527732@mail.gmail.com>
	<87k4xut81u.fsf@benfinney.id.au>
Message-ID: <20091113221234.GA15732@amk-desktop.matrixgroup.net>

On Sat, Nov 14, 2009 at 08:43:25AM +1100, Ben Finney wrote:
> There's also no option to vote that decisions on how to manage Python
> infrastructure (like PyPI) shouldn't be made by popular vote.

If the open source approach of 'the maintainer decides' is followed,
well, both the maintainer of the code and the admin of the site is
Martin.  Comments stay, then.

If 'BDFL decides' is followed, GvR thinks the idea is reasonable
(http://mail.python.org/pipermail/python-dev/2009-November/094058.html),
though the implementation might be improved.  Again, comments stay.

If popular vote is ruled out, I don't see who else could possibly make
the decision to disable comments and/or ratings.

--amk

From jacob at jacobian.org  Fri Nov 13 23:17:40 2009
From: jacob at jacobian.org (Jacob Kaplan-Moss)
Date: Fri, 13 Nov 2009 16:17:40 -0600
Subject: [Python-Dev] PyPI comments and ratings, *really*?
In-Reply-To: <79990c6b0911131133i2987a5d4s5c4ef780a527732@mail.gmail.com>
References: <D7E23A6D-C91C-46D8-A8FE-1EB16F3B2101@lericson.se> 
	<4AFC9064.20700@v.loewis.de>
	<4222a8490911121511s40eba937n2c7b1fde6f695c6c@mail.gmail.com> 
	<4AFC9975.7040204@v.loewis.de>
	<4222a8490911121534h30572eabv73ae58649c77155c@mail.gmail.com> 
	<4AFC9D0C.5030500@v.loewis.de> <87y6mbuubp.fsf@benfinney.id.au> 
	<20e5c9660911121702r162caf28jb6d656383147d909@mail.gmail.com> 
	<hdim34$utf$1@ger.gmane.org>
	<79990c6b0911131133i2987a5d4s5c4ef780a527732@mail.gmail.com>
Message-ID: <bc8ec34e0911131417y47315a7bp7a4fb1dfde7056bd@mail.gmail.com>

On Fri, Nov 13, 2009 at 1:33 PM, Paul Moore <p.f.moore at gmail.com> wrote:
> By the way, there's no way for me to vote that I don't care what
> option ends up being chosen, but I strongly oppose choosing something
> that would tend to make developers avoid using PyPI.

I think the damage has already been done. There are enough people --
I'm one -- who feel this has revealed huge flaws in PyPI's
stewardship. I'm pretty convinced, results of the poll or not, that
the way PyPI is being run is fatally flawed.

This "poll" business is just smoke and mirrors, anyway -- notice the
way the "no comments" votors are split among three choices, while the
"pro comments" voters have just two choices. It's also worded in a way
that obscures the real debate. Regardless of the outcome, the poll's
not going to change anyone's mind, and it certainly won't change the
fact that PyPI's being run as a one-man show, not as a community
resource.

I'm just not comfortable with that. Unless PyPI changes its governance
radically, I'm not going to invest any more effort in it.

Jacob

From theller at ctypes.org  Fri Nov 13 23:55:24 2009
From: theller at ctypes.org (Thomas Heller)
Date: Fri, 13 Nov 2009 23:55:24 +0100
Subject: [Python-Dev] PyPI comments and ratings, *really*?
In-Reply-To: <20091113221234.GA15732@amk-desktop.matrixgroup.net>
References: <4AFC9064.20700@v.loewis.de>	<4222a8490911121511s40eba937n2c7b1fde6f695c6c@mail.gmail.com>	<4AFC9975.7040204@v.loewis.de>	<4222a8490911121534h30572eabv73ae58649c77155c@mail.gmail.com>	<4AFC9D0C.5030500@v.loewis.de>
	<87y6mbuubp.fsf@benfinney.id.au>	<20e5c9660911121702r162caf28jb6d656383147d909@mail.gmail.com>	<hdim34$utf$1@ger.gmane.org>	<79990c6b0911131133i2987a5d4s5c4ef780a527732@mail.gmail.com>	<87k4xut81u.fsf@benfinney.id.au>
	<20091113221234.GA15732@amk-desktop.matrixgroup.net>
Message-ID: <hdko4s$7r1$1@ger.gmane.org>

A.M. Kuchling schrieb:

> If the open source approach of 'the maintainer decides' is followed,
> well, both the maintainer of the code and the admin of the site is
> Martin.  Comments stay, then.
> 
> If 'BDFL decides' is followed, GvR thinks the idea is reasonable
> (http://mail.python.org/pipermail/python-dev/2009-November/094058.html),
> though the implementation might be improved.  Again, comments stay.
> 
> If popular vote is ruled out, I don't see who else could possibly make
> the decision to disable comments and/or ratings.

+1

-- 
Thanks,
Thomas


From ben+python at benfinney.id.au  Fri Nov 13 23:57:18 2009
From: ben+python at benfinney.id.au (Ben Finney)
Date: Sat, 14 Nov 2009 09:57:18 +1100
Subject: [Python-Dev] PyPI comments and ratings, *really*?
References: <4AFC9064.20700@v.loewis.de>
	<4222a8490911121511s40eba937n2c7b1fde6f695c6c@mail.gmail.com>
	<4AFC9975.7040204@v.loewis.de>
	<4222a8490911121534h30572eabv73ae58649c77155c@mail.gmail.com>
	<4AFC9D0C.5030500@v.loewis.de> <87y6mbuubp.fsf@benfinney.id.au>
	<20e5c9660911121702r162caf28jb6d656383147d909@mail.gmail.com>
	<hdim34$utf$1@ger.gmane.org>
	<79990c6b0911131133i2987a5d4s5c4ef780a527732@mail.gmail.com>
	<87k4xut81u.fsf@benfinney.id.au>
	<20091113221234.GA15732@amk-desktop.matrixgroup.net>
Message-ID: <87bpj6t4mp.fsf@benfinney.id.au>

"A.M. Kuchling" <amk at amk.ca> writes:

> If popular vote is ruled out, I don't see who else could possibly make
> the decision to disable comments and/or ratings.

Reasoned argument with the person who decides. A bad idea with many
people's support is still a bad idea; a good idea with few people's
support is still a good idea.

-- 
 \        ?I have a microwave fireplace in my house. The other night I |
  `\   laid down in front of the fire for the evening in two minutes.? |
_o__)                                                   ?Steven Wright |
Ben Finney


From steve at pearwood.info  Sat Nov 14 00:18:12 2009
From: steve at pearwood.info (Steven D'Aprano)
Date: Sat, 14 Nov 2009 10:18:12 +1100
Subject: [Python-Dev] PyPI comments and ratings, *really*?
In-Reply-To: <87bpj6t4mp.fsf@benfinney.id.au>
References: <4AFC9064.20700@v.loewis.de>
	<20091113221234.GA15732@amk-desktop.matrixgroup.net>
	<87bpj6t4mp.fsf@benfinney.id.au>
Message-ID: <200911141018.13536.steve@pearwood.info>

On Sat, 14 Nov 2009 09:57:18 am Ben Finney wrote:
> "A.M. Kuchling" <amk at amk.ca> writes:
> > If popular vote is ruled out, I don't see who else could possibly
> > make the decision to disable comments and/or ratings.
>
> Reasoned argument with the person who decides. A bad idea with many
> people's support is still a bad idea; a good idea with few people's
> support is still a good idea.


Okay, let's talk reasoned debate.

I understand the reason for making comments compulsory: they're for the 
benefit of the users, not the package owner. It helps prevent 
information about the package from being fragmented: there is One 
Obvious place to find out about a package on PyPI, which is the PyPI 
page, instead of having to search for blogs where people may or may not 
have made comments about the package. If individual package owners 
don't like this, too bad, because PyPI is run for the benefit of the 
community, not individual package owners.

I understand the reason for making comments optional: personal choice of 
the package owner is valuable in and of itself, even if it is against 
the best interests of the community.

But for the life of me, I can't understand the 1/3 of the votes that 
have been cast in favour of prohibiting comments for everybody, even 
those who want comments.


-- 
Steven D'Aprano

From steve at pearwood.info  Sat Nov 14 00:19:17 2009
From: steve at pearwood.info (Steven D'Aprano)
Date: Sat, 14 Nov 2009 10:19:17 +1100
Subject: [Python-Dev] PyPI comments and ratings, *really*?
In-Reply-To: <bc8ec34e0911131417y47315a7bp7a4fb1dfde7056bd@mail.gmail.com>
References: <D7E23A6D-C91C-46D8-A8FE-1EB16F3B2101@lericson.se>
	<79990c6b0911131133i2987a5d4s5c4ef780a527732@mail.gmail.com>
	<bc8ec34e0911131417y47315a7bp7a4fb1dfde7056bd@mail.gmail.com>
Message-ID: <200911141019.17917.steve@pearwood.info>

On Sat, 14 Nov 2009 09:17:40 am Jacob Kaplan-Moss wrote:

> This "poll" business is just smoke and mirrors, anyway -- notice the
> way the "no comments" votors are split among three choices, while the
> "pro comments" voters have just two choices.

What?

Allow ratings and comments on all packages (status quo)
Allow package owners to disallow comments (ratings unmodified)
Allow comments, but only send them to package owners (ratings 
unmodified)
Disallow comments (ratings unmodified)
Disallow ratings and comments (status three months ago)

That's three choices for allowing comments, two for disallowing.




-- 
Steven D'Aprano

From steve at pearwood.info  Sat Nov 14 00:20:43 2009
From: steve at pearwood.info (Steven D'Aprano)
Date: Sat, 14 Nov 2009 10:20:43 +1100
Subject: [Python-Dev] PyPI comments and ratings, *really*?
In-Reply-To: <hdkjd8$psr$1@ger.gmane.org>
References: <D7E23A6D-C91C-46D8-A8FE-1EB16F3B2101@lericson.se>
	<4AFC9D0C.5030500@v.loewis.de> <hdkjd8$psr$1@ger.gmane.org>
Message-ID: <200911141020.43496.steve@pearwood.info>

On Sat, 14 Nov 2009 08:34:32 am Matthew Woodcraft wrote:

> As I write, the option with the most votes is /Allow both ratings and
> comments/.
>
> But between them, /Disallow comments/ and /Disallow both ratings and
> comments/ have more votes.

If that's an accurate characterisation, then things must have changed 
*radically* in just a few minutes, because the voting at 8:38am 
(Melbourne time) was running 2:1 in favour of allowing comments versus 
disallowing them. At 10:17 it's still running at the same ratio, 
although the individual votes have changed slightly.


> It seems clear to me that, given those figures, allowing comments
> would not be following the result of the vote.

Why do you think it is okay to combine the Disallow vote, without also 
combining the Allow vote? Less than a third of the total votes are in 
favour of disallowing comments, with two-thirds in favour of allowing 
them.




-- 
Steven D'Aprano

From steve at pearwood.info  Sat Nov 14 00:30:14 2009
From: steve at pearwood.info (Steven D'Aprano)
Date: Sat, 14 Nov 2009 10:30:14 +1100
Subject: [Python-Dev] PyPI comments and ratings, *really*?
In-Reply-To: <200911141019.17917.steve@pearwood.info>
References: <D7E23A6D-C91C-46D8-A8FE-1EB16F3B2101@lericson.se>
	<bc8ec34e0911131417y47315a7bp7a4fb1dfde7056bd@mail.gmail.com>
	<200911141019.17917.steve@pearwood.info>
Message-ID: <200911141030.15071.steve@pearwood.info>

On Sat, 14 Nov 2009 10:19:17 am Steven D'Aprano wrote:
> On Sat, 14 Nov 2009 09:17:40 am Jacob Kaplan-Moss wrote:
> > This "poll" business is just smoke and mirrors, anyway -- notice
> > the way the "no comments" votors are split among three choices,
> > while the "pro comments" voters have just two choices.
>
> What?
>
> Allow ratings and comments on all packages (status quo)
> Allow package owners to disallow comments (ratings unmodified)
> Allow comments, but only send them to package owners (ratings
> unmodified)
> Disallow comments (ratings unmodified)
> Disallow ratings and comments (status three months ago)
>
> That's three choices for allowing comments, two for disallowing.

Sorry, I should have been more explicit:

The first choice makes comments compulsory. In other words, comments are 
allowed.

The second choice makes comments optional. Whether it is opt-in or 
opt-out, PyPI still allows comments as an option.

The third choice makes comments available, but private. That's still 
allowing comments.

Only the fourth and fifth options disallow comments. If you don't want 
there to be comments on PyPI, you only have two choices: 4 or 5.



-- 
Steven D'Aprano

From matthew at woodcraft.me.uk  Sat Nov 14 01:00:52 2009
From: matthew at woodcraft.me.uk (Matthew Woodcraft)
Date: Sat, 14 Nov 2009 00:00:52 +0000 (UTC)
Subject: [Python-Dev] PyPI comments and ratings, *really*?
References: <D7E23A6D-C91C-46D8-A8FE-1EB16F3B2101@lericson.se>
	<4AFC9D0C.5030500@v.loewis.de> <hdkjd8$psr$1@ger.gmane.org>
	<200911141020.43496.steve@pearwood.info>
Message-ID: <hdkrvk$i6i$1@ger.gmane.org>

Steven D'Aprano  <steve at pearwood.info> wrote:
> Why do you think it is okay to combine the Disallow vote, without also
> combining the Allow vote? Less than a third of the total votes are in
> favour of disallowing comments, with two-thirds in favour of allowing
> them.

I don't. I was giving one example of the problem with the plan of not
combining options at all.

I wasn't writing to take sides. I was trying to say that using 'first
past the post' voting with five options which are combinations of
different issues (comments, ratings, whether to allow package owners to
opt out) can't be expected to give a good decision.

I think there is no safe way to pick a winner from this vote (unless
more than half the voters choose a single option).


(As a general point, in any discussion it tends to be unhelpful to ask
"Why do you think X" unless the person you're asking has clearly
indicated that they do in fact think X.)

-M-


From solipsis at pitrou.net  Sat Nov 14 01:09:19 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Sat, 14 Nov 2009 00:09:19 +0000 (UTC)
Subject: [Python-Dev] Status of the Buildbot fleet and related bugs
References: <Pine.LNX.4.64.0911052209000.12274@kimball.webabinitio.net>
	<hdcn72$2rl$1@ger.gmane.org> <4AF9E3D6.2000807@v.loewis.de>
Message-ID: <loom.20091114T010914-556@post.gmane.org>

Martin v. L?wis <martin <at> v.loewis.de> writes:
> 
> > The buildbots still show occasional oddities. For example, right now in 
> > the page "http://www.python.org/dev/buildbot/3.x/", some results have 
> > disappeared (the columns for "AMD64 Ubuntu" builders have become empty). 
> 
> Yes, I noticed it too. It will go away after some page reloads.

It is still happening more or less randomly unfortunately.
http://www.python.org/dev/buildbot/3.x/



From robert.kern at gmail.com  Sat Nov 14 01:10:16 2009
From: robert.kern at gmail.com (Robert Kern)
Date: Fri, 13 Nov 2009 18:10:16 -0600
Subject: [Python-Dev] PyPI comments and ratings, *really*?
In-Reply-To: <200911141018.13536.steve@pearwood.info>
References: <4AFC9064.20700@v.loewis.de>	<20091113221234.GA15732@amk-desktop.matrixgroup.net>	<87bpj6t4mp.fsf@benfinney.id.au>
	<200911141018.13536.steve@pearwood.info>
Message-ID: <hdksh9$jkt$1@ger.gmane.org>

[We really should be discussing this on catalog-sig, but moving things 
mid-thread never works, so here we go. I apologize to python-dev. I also 
apologize for the length.]

On 2009-11-13 17:18 PM, Steven D'Aprano wrote:
> On Sat, 14 Nov 2009 09:57:18 am Ben Finney wrote:
>> "A.M. Kuchling"<amk at amk.ca>  writes:
>>> If popular vote is ruled out, I don't see who else could possibly
>>> make the decision to disable comments and/or ratings.
>>
>> Reasoned argument with the person who decides. A bad idea with many
>> people's support is still a bad idea; a good idea with few people's
>> support is still a good idea.
>
> Okay, let's talk reasoned debate.
>
> I understand the reason for making comments compulsory: they're for the
> benefit of the users, not the package owner. It helps prevent
> information about the package from being fragmented: there is One
> Obvious place to find out about a package on PyPI, which is the PyPI
> page, instead of having to search for blogs where people may or may not
> have made comments about the package. If individual package owners
> don't like this, too bad, because PyPI is run for the benefit of the
> community, not individual package owners.
>
> I understand the reason for making comments optional: personal choice of
> the package owner is valuable in and of itself, even if it is against
> the best interests of the community.
>
> But for the life of me, I can't understand the 1/3 of the votes that
> have been cast in favour of prohibiting comments for everybody, even
> those who want comments.

While I do have a couple of packages on PyPI, I use PyPI as a consumer of 
packages much more frequently, every day in fact. I voted against comments and 
ratings because I think it is a detriment to my experience of PyPI as a user (I 
also think that they will be a detriment to the community, but that's a 
prediction, not a fact). Short form comments are simply not useful to me. 
Ratings are worse. They do not carry reliable information; they carry short 
statements of opinion from a wide variety of experiences, most of which are 
entirely unrelated to my own needs.

To make an example, I have a lot of experience making ornery stuff build. A lot 
of other people don't. Their personal experience of not managing to install a 
package correctly turns into a weird, objective-seeming statement that the 
"package is difficult to build". People have different thresholds, different 
experiences, and different standards.

When such opinions get numerically aggregated by a monolithic rating system, 
that's even worse. Even with short-form comments, they have the ability, though 
often not the propensity, to give a little bit of information ("I had trouble 
building it on Windows.") that can help me tease out whether their experiences 
will be relevant to mine, but one star is just one star.

I *do* like to read long-form reviews where people relate what their needs were, 
what they tried to use the package for, and exactly where the package succeeded 
and where it failed. I can compare that context with my own needs and extract 
the relevant parts of their experience. Blogs are great for that.

Now I do appreciate ratings and comments on shopping sites if they don't provide 
the capability for long-form reviews. But that's because the product is locked 
behind the barrier of payment and often shipping. There is no such barrier on 
PyPI. If I can get to a web view of their repository, thirty seconds with it 
will give a much better idea of whether the package is worth trying than any 
amount of comments I could read in that time. I can easily see how much 
documentation they have, if they have funny build requirements, if they are just 
prototyping, etc. without needing to trust that everyone else has needs and 
standards like mine. That's the key difference between comments and ratings on 
shopping sites and why I don't think that their success in that field translates 
to here.

If you want one idea that would make my use of PyPI much more pleasant and 
informative, it would be to add a "Repository-URL" entry to the recommended PyPI 
metadata so that I could always start looking at the code in one click. Or 
integrate the source code browsing into PyPI itself; it could open up the sdists 
and eggs and show a WebVCS-like browser interface.

Now, these are all reasons why commenting and rating are not beneficial to me. 
Where I think they are harmful is that when one is exposed to them, one cannot 
just ignore them. They have an emotional, unreasonable impact whether I want 
them to or not. And that's why I do not want to see them. Give me access to 
information, not opinions. If authors do want comments and ratings on their 
packages, let them use the services that already exist for *just* that purpose 
like Ohloh. They have the time and energy to implement these mechanisms with the 
care and attention that they need.

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
  that is made terrible by our own mad attempt to interpret it as though it had
  an underlying truth."
   -- Umberto Eco


From chris at simplistix.co.uk  Sat Nov 14 01:24:00 2009
From: chris at simplistix.co.uk (Chris Withers)
Date: Sat, 14 Nov 2009 00:24:00 +0000
Subject: [Python-Dev] PyPI comments and ratings, *really*?
In-Reply-To: <200911141030.15071.steve@pearwood.info>
References: <D7E23A6D-C91C-46D8-A8FE-1EB16F3B2101@lericson.se>	<bc8ec34e0911131417y47315a7bp7a4fb1dfde7056bd@mail.gmail.com>	<200911141019.17917.steve@pearwood.info>
	<200911141030.15071.steve@pearwood.info>
Message-ID: <4AFDF8A0.1050100@simplistix.co.uk>

Steven D'Aprano wrote:
>> That's three choices for allowing comments, two for disallowing.
> 
> Sorry, I should have been more explicit:
> 
> The first choice makes comments compulsory. In other words, comments are 
> allowed.
> 
> The second choice makes comments optional. Whether it is opt-in or 
> opt-out, PyPI still allows comments as an option.
> 
> The third choice makes comments available, but private. That's still 
> allowing comments.
> 
> Only the fourth and fifth options disallow comments. If you don't want 
> there to be comments on PyPI, you only have two choices: 4 or 5.

The wording isn't ideal, unless you're in agreement with Martin:

http://mail.python.org/pipermail/catalog-sig/2009-November/002288.html

Chris

-- 
Simplistix - Content Management, Batch Processing & Python Consulting
            - http://www.simplistix.co.uk

From skip at pobox.com  Sat Nov 14 01:33:00 2009
From: skip at pobox.com (skip at pobox.com)
Date: Fri, 13 Nov 2009 18:33:00 -0600
Subject: [Python-Dev] PyPI comments and ratings, *really*?
In-Reply-To: <B10AA20C-1C2F-44F0-B8B5-64F26691CC83@python.org>
References: <D7E23A6D-C91C-46D8-A8FE-1EB16F3B2101@lericson.se>
	<200911130038.05516.steve@pearwood.info>
	<4222a8490911120606t3e138957kcdb21104261a1ead@mail.gmail.com>
	<B10AA20C-1C2F-44F0-B8B5-64F26691CC83@python.org>
Message-ID: <19197.64188.94002.797250@montanaro.dyndns.org>


    >> Frankly, I agree with him. As implemented, I *and others* think this
    >> is broken. I've taken the stance of not publishing things to PyPi
    >> until A> I find the time to contribute to make it better or B> It
    >> changes.

    Barry> That's distressing.  For better or worse PyPI is the central
    Barry> repository of 3rd party packages.  It should be easy, desirable,
    Barry> fun and socially encouraged to get your packages there.

I only realized people could comment on my packages when I got the poll
email from Martin.  It then took me awhile to figure out how to actually see
comments about my packages.

    https://sourceforge.net/tracker/?func=detail&aid=2897527&group_id=66150&atid=513503

At the very least I think the feature needs to be easier for package
authors to use.

Skip

From python at mrabarnett.plus.com  Sat Nov 14 01:38:18 2009
From: python at mrabarnett.plus.com (MRAB)
Date: Sat, 14 Nov 2009 00:38:18 +0000
Subject: [Python-Dev] PyPI comments and ratings, *really*?
In-Reply-To: <hdkrvk$i6i$1@ger.gmane.org>
References: <D7E23A6D-C91C-46D8-A8FE-1EB16F3B2101@lericson.se>	<4AFC9D0C.5030500@v.loewis.de>
	<hdkjd8$psr$1@ger.gmane.org>	<200911141020.43496.steve@pearwood.info>
	<hdkrvk$i6i$1@ger.gmane.org>
Message-ID: <4AFDFBFA.9030809@mrabarnett.plus.com>

Matthew Woodcraft wrote:
> Steven D'Aprano  <steve at pearwood.info> wrote:
>> Why do you think it is okay to combine the Disallow vote, without also
>> combining the Allow vote? Less than a third of the total votes are in
>> favour of disallowing comments, with two-thirds in favour of allowing
>> them.
> 
> I don't. I was giving one example of the problem with the plan of not
> combining options at all.
> 
> I wasn't writing to take sides. I was trying to say that using 'first
> past the post' voting with five options which are combinations of
> different issues (comments, ratings, whether to allow package owners to
> opt out) can't be expected to give a good decision.
> 
> I think there is no safe way to pick a winner from this vote (unless
> more than half the voters choose a single option).
> 
> 
> (As a general point, in any discussion it tends to be unhelpful to ask
> "Why do you think X" unless the person you're asking has clearly
> indicated that they do in fact think X.)
> 
Perhaps Approval voting should be used:

http://en.wikipedia.org/wiki/Approval_voting

From chris at simplistix.co.uk  Sat Nov 14 01:44:02 2009
From: chris at simplistix.co.uk (Chris Withers)
Date: Sat, 14 Nov 2009 00:44:02 +0000
Subject: [Python-Dev] PyPI governance
In-Reply-To: <bc8ec34e0911131417y47315a7bp7a4fb1dfde7056bd@mail.gmail.com>
References: <D7E23A6D-C91C-46D8-A8FE-1EB16F3B2101@lericson.se>
	<4AFC9064.20700@v.loewis.de>	<4222a8490911121511s40eba937n2c7b1fde6f695c6c@mail.gmail.com>
	<4AFC9975.7040204@v.loewis.de>	<4222a8490911121534h30572eabv73ae58649c77155c@mail.gmail.com>
	<4AFC9D0C.5030500@v.loewis.de> <87y6mbuubp.fsf@benfinney.id.au>
	<20e5c9660911121702r162caf28jb6d656383147d909@mail.gmail.com>
	<hdim34$utf$1@ger.gmane.org>	<79990c6b0911131133i2987a5d4s5c4ef780a527732@mail.gmail.com>
	<bc8ec34e0911131417y47315a7bp7a4fb1dfde7056bd@mail.gmail.com>
Message-ID: <4AFDFD52.4070401@simplistix.co.uk>

Jacob Kaplan-Moss wrote:
> that obscures the real debate. Regardless of the outcome, the poll's
> not going to change anyone's mind, and it certainly won't change the
> fact that PyPI's being run as a one-man show, not as a community
> resource.

While I may not agree on his choices regarding comments and ratings, I 
don't feel this is fair to Martin. No one else, as far as I know, has 
offered to step up and help with the maintenance of PyPI. I don't know 
if Richard is still active, but the lack of response from him would 
suggest not.

A test on your hypothesis would be to offer to help. I don't know who 
has access to the server the software runs on, but ultimately any core 
python committer can change the code in the subversion repository.

I did just actually have a look over the code and was happy to find it 
was a lot simpler than I had feared but a bit dismayed at the style of 
coding and total lack of unit tests ;-)

I might well be up for helping with the maintenance of PyPI if I'm 
welcome, especially as I'm already familiar with a lot of the components 
used, although I don't know how much time I can commit :-(
Martin, are you interested in help?

Chris

PS: While I'm sure a lot of python-dev people are interested in this 
topic, I'm pretty sure this whole huge sprawling thread belongs on 
catalog-sig...

-- 
Simplistix - Content Management, Batch Processing & Python Consulting
            - http://www.simplistix.co.uk

From skip at pobox.com  Sat Nov 14 01:45:17 2009
From: skip at pobox.com (skip at pobox.com)
Date: Fri, 13 Nov 2009 18:45:17 -0600
Subject: [Python-Dev] PyPI comments and ratings, *really*?
In-Reply-To: <4222a8490911120831y5233e547pbfd22de60bcf436e@mail.gmail.com>
References: <D7E23A6D-C91C-46D8-A8FE-1EB16F3B2101@lericson.se>
	<200911130038.05516.steve@pearwood.info>
	<ca471dc20911120802r4c765f78h669e56335a38f11c@mail.gmail.com>
	<4222a8490911120831y5233e547pbfd22de60bcf436e@mail.gmail.com>
Message-ID: <19197.64925.119992.882456@montanaro.dyndns.org>


    Jesse> I don't want us to impersonate the mindless, sub-useful drivel
    Jesse> found in App store/YouTube/etc comments 99% of the time or the
    Jesse> broken "5 star ratings" systems, etc. It's too easy to game.

I implemented a "5-star" system for Musi-Cal back in the day.  Now,
admittedly, rating musicians is a bit different than rating software, but
you would probably be surprised at how much offense one musician can take
when they discover that some other musician got a 4.7 rating and they only
got a 4.2.  :rolleyes:

It did take some effort to reduce the chance that the system would be
subverted.  Similar to a free software site such as PyPI (and unlike a
for-profit system such as the iTunes App Store), I think almost all people
realized that the rating system was there to help the community and that by
polluting the database they only hurt themselves.

Skip

From solipsis at pitrou.net  Sat Nov 14 01:47:29 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Sat, 14 Nov 2009 00:47:29 +0000 (UTC)
Subject: [Python-Dev] PyPI governance
References: <D7E23A6D-C91C-46D8-A8FE-1EB16F3B2101@lericson.se>
	<4AFC9064.20700@v.loewis.de>	<4222a8490911121511s40eba937n2c7b1fde6f695c6c@mail.gmail.com>
	<4AFC9975.7040204@v.loewis.de>	<4222a8490911121534h30572eabv73ae58649c77155c@mail.gmail.com>
	<4AFC9D0C.5030500@v.loewis.de> <87y6mbuubp.fsf@benfinney.id.au>
	<20e5c9660911121702r162caf28jb6d656383147d909@mail.gmail.com>
	<hdim34$utf$1@ger.gmane.org>	<79990c6b0911131133i2987a5d4s5c4ef780a527732@mail.gmail.com>
	<bc8ec34e0911131417y47315a7bp7a4fb1dfde7056bd@mail.gmail.com>
	<4AFDFD52.4070401@simplistix.co.uk>
Message-ID: <loom.20091114T014713-392@post.gmane.org>

Chris Withers <chris <at> simplistix.co.uk> writes:
> 
> PS: While I'm sure a lot of python-dev people are interested in this 
> topic, I'm pretty sure this whole huge sprawling thread belongs on 
> catalog-sig...

Yes, please :)


Antoine.



From db3l.net at gmail.com  Sat Nov 14 02:51:07 2009
From: db3l.net at gmail.com (David Bolen)
Date: Fri, 13 Nov 2009 20:51:07 -0500
Subject: [Python-Dev] External module tests skipping on Windows build slaves?
Message-ID: <m2zl6p7u2c.fsf@valheru.db3l.homeip.net>

While working with the Windows (and now Windows 7) build slaves I've
been noticing that a number of "unexpected" test skips are occurring,
mostly for external modules like ssl, bz2, tk, etc...

However, I know for a fact that those libraries are being fetched and
built, so I was a little perplexed that they were being skipped for
testing.  I don't think it's a regression due to the recent slave work,
but something that may have been happening for a while, if not always.

Some manual testing I did would seem to indicate that there's a
problem related to loading the right MS CRT when loading these
modules.  But they seem to reference both debug and non-debug
versions, so even if I copy the CRT DLLs (from the SxS tre) into
pcbuild manually, then I get a CRT assertion about a bad
initialization.  I suspect this wouldn't be a problem when everything
is built in release mode.

I would have sworn they used to get run, but now I'm not so sure.
Perhaps I'm remembering older Python releases with VS.NET 2003, since
the MSVC9 versions of the CRT and the SXS stuff was new with VS 2008 I
think.

Does anyone happen to know if these tests have ever run recently on
the Windows build slaves?

-- David


From skip at pobox.com  Sat Nov 14 02:54:31 2009
From: skip at pobox.com (skip at pobox.com)
Date: Fri, 13 Nov 2009 19:54:31 -0600
Subject: [Python-Dev] PyPI comments and ratings, *really*?
In-Reply-To: <ca471dc20911121054t76eb5bfbsbd6a9e47a9e71d8a@mail.gmail.com>
References: <D7E23A6D-C91C-46D8-A8FE-1EB16F3B2101@lericson.se>
	<200911130038.05516.steve@pearwood.info>
	<4222a8490911120606t3e138957kcdb21104261a1ead@mail.gmail.com>
	<B10AA20C-1C2F-44F0-B8B5-64F26691CC83@python.org>
	<hdhk7l$tru$1@ger.gmane.org>
	<ca471dc20911121054t76eb5bfbsbd6a9e47a9e71d8a@mail.gmail.com>
Message-ID: <19198.3543.344446.709885@montanaro.dyndns.org>


    Guido> Of course, as a user, I might not trust a module that has no
    Guido> reviews or ratings.

I suspect the vast majority of projects will never acquire ratings or
reviews.

Skip

From amnorvend at gmail.com  Sat Nov 14 02:54:29 2009
From: amnorvend at gmail.com (Jason Baker)
Date: Fri, 13 Nov 2009 19:54:29 -0600
Subject: [Python-Dev] PyPI governance
In-Reply-To: <4AFDFD52.4070401@simplistix.co.uk>
References: <D7E23A6D-C91C-46D8-A8FE-1EB16F3B2101@lericson.se> 
	<4AFC9975.7040204@v.loewis.de>
	<4222a8490911121534h30572eabv73ae58649c77155c@mail.gmail.com> 
	<4AFC9D0C.5030500@v.loewis.de> <87y6mbuubp.fsf@benfinney.id.au> 
	<20e5c9660911121702r162caf28jb6d656383147d909@mail.gmail.com> 
	<hdim34$utf$1@ger.gmane.org>
	<79990c6b0911131133i2987a5d4s5c4ef780a527732@mail.gmail.com> 
	<bc8ec34e0911131417y47315a7bp7a4fb1dfde7056bd@mail.gmail.com> 
	<4AFDFD52.4070401@simplistix.co.uk>
Message-ID: <816c4abb0911131754q38711732rbb9c3bfe43b3d4eb@mail.gmail.com>

On Fri, Nov 13, 2009 at 6:44 PM, Chris Withers <chris at simplistix.co.uk> wrote:
> PS: While I'm sure a lot of python-dev people are interested in this topic,
> I'm pretty sure this whole huge sprawling thread belongs on catalog-sig...

+100

From greg.ewing at canterbury.ac.nz  Sat Nov 14 03:23:14 2009
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Sat, 14 Nov 2009 15:23:14 +1300
Subject: [Python-Dev] standard libraries don't behave like	standard
 'libraries'
In-Reply-To: <4AFC6440.1040008@v.loewis.de>
References: <c910262d0911120123i7c7e9dla918fff50cb5aecd@mail.gmail.com>
	<4AFC6440.1040008@v.loewis.de>
Message-ID: <4AFE1492.7050401@canterbury.ac.nz>

Martin v. L?wis wrote:

 > Some of the Python maintainers have recently started objecting to this
 > setup, asking that the standard library should be split into separate
 > packages that are released and distributed independent of Python. Others
 > of us feel strongly that such a change should not be made.

I'd be worried, because I would no longer be able to
release an app or package and say "requires Python x.y".
I'd have to list the version numbers of all the micro
packages making up the standard library that I use.

Worse, I'd have to be aware of which ones I actually
*do* use so I could mantain that list, something I don't
have to think about at the moment.

It mightn't be so bad if the whole stdlib were a *single*
package, but I don't see much advantage in that.

-- 
Greg

From ben+python at benfinney.id.au  Sat Nov 14 05:00:47 2009
From: ben+python at benfinney.id.au (Ben Finney)
Date: Sat, 14 Nov 2009 15:00:47 +1100
Subject: [Python-Dev] PyPI comments and ratings, *really*?
References: <4AFC9064.20700@v.loewis.de>
	<20091113221234.GA15732@amk-desktop.matrixgroup.net>
	<87bpj6t4mp.fsf@benfinney.id.au>
	<200911141018.13536.steve@pearwood.info>
Message-ID: <873a4hu55c.fsf@benfinney.id.au>

Steven D'Aprano <steve at pearwood.info> writes:

> But for the life of me, I can't understand the 1/3 of the votes that
> have been cast in favour of prohibiting comments for everybody, even
> those who want comments.

You gave reason (and I agree with you) for why, on a service that
allows comments and/or ratings, that users might think less of a
project that chooses to be on the service but then chooses not to have
comments and/or ratings.

Thus, there's a good reason on such a system to *not* participate in the
service at all. A barrier to entry, on something that should be *the*
place to look for third-party Python packages.

To encourage as many third-party packages to register on PyPI, it seems
barriers like that should be removed.

-- 
 \                ?Please to bathe inside the tub.? ?hotel room, Japan |
  `\                                                                   |
_o__)                                                                  |
Ben Finney


From rdmurray at bitdance.com  Sat Nov 14 05:19:31 2009
From: rdmurray at bitdance.com (R. David Murray)
Date: Fri, 13 Nov 2009 23:19:31 -0500 (EST)
Subject: [Python-Dev] Status of the Buildbot fleet and related bugs
In-Reply-To: <loom.20091114T010914-556@post.gmane.org>
References: <Pine.LNX.4.64.0911052209000.12274@kimball.webabinitio.net>
	<hdcn72$2rl$1@ger.gmane.org> <4AF9E3D6.2000807@v.loewis.de>
	<loom.20091114T010914-556@post.gmane.org>
Message-ID: <Pine.LNX.4.64.0911132317040.12274@kimball.webabinitio.net>

On Sat, 14 Nov 2009 at 00:09, Antoine Pitrou wrote:
> Martin v. L??wis <martin <at> v.loewis.de> writes:
>>
>>> The buildbots still show occasional oddities. For example, right now in
>>> the page "http://www.python.org/dev/buildbot/3.x/", some results have
>>> disappeared (the columns for "AMD64 Ubuntu" builders have become empty).
>>
>> Yes, I noticed it too. It will go away after some page reloads.
>
> It is still happening more or less randomly unfortunately.
> http://www.python.org/dev/buildbot/3.x/

The buildbot pages appear to be pretty messed up now.  I get many 404s
(ex: the above url, the all stable builders page), although some seem to
work (ex: the all builders page), and if I stick an 'all' into the URL
for my buildbot page I can get to it, though that's is not the version
of the URL linked from the 'all builders' table header.

--David (RDM)

From db3l.net at gmail.com  Sat Nov 14 05:34:29 2009
From: db3l.net at gmail.com (David Bolen)
Date: Fri, 13 Nov 2009 23:34:29 -0500
Subject: [Python-Dev] External module tests skipping on Windows build
	slaves?
References: <m2zl6p7u2c.fsf@valheru.db3l.homeip.net>
Message-ID: <m2tywx7mi2.fsf@valheru.db3l.homeip.net>

David Bolen <db3l.net at gmail.com> writes:

> (...)
> I would have sworn they used to get run, but now I'm not so sure.
> Perhaps I'm remembering older Python releases with VS.NET 2003, since
> the MSVC9 versions of the CRT and the SXS stuff was new with VS 2008 I
> think.
>
> Does anyone happen to know if these tests have ever run recently on
> the Windows build slaves?

Just to follow up - after finding an old 2.6a0 tree I had manually
built that worked fine (so clearly it wasn't just VS 2008), I worked
my way through some 2.6 tags and this broke as of 2.6.1, so I think
the tests have been getting skipped since then.

The root problem appears to be the change (r67121 for 2.6 merged from
r67120 in trunk) that removed embedded manifests.

I'll follow up on the issue tracker.

-- David


From db3l.net at gmail.com  Sat Nov 14 05:40:12 2009
From: db3l.net at gmail.com (David Bolen)
Date: Fri, 13 Nov 2009 23:40:12 -0500
Subject: [Python-Dev] Status of the Buildbot fleet and related bugs
References: <Pine.LNX.4.64.0911052209000.12274@kimball.webabinitio.net>
	<hdcn72$2rl$1@ger.gmane.org> <4AF9E3D6.2000807@v.loewis.de>
	<loom.20091114T010914-556@post.gmane.org>
	<Pine.LNX.4.64.0911132317040.12274@kimball.webabinitio.net>
Message-ID: <m2pr7l7m8j.fsf@valheru.db3l.homeip.net>

"R. David Murray" <rdmurray at bitdance.com> writes:

> The buildbot pages appear to be pretty messed up now.  I get many 404s
> (ex: the above url, the all stable builders page), although some seem to
> work (ex: the all builders page), and if I stick an 'all' into the URL
> for my buildbot page I can get to it, though that's is not the version
> of the URL linked from the 'all builders' table header.

Yes, I think this just started happening. I'm guessing that the main
site proxies the buildbot URL requests to the buildbot master process,
and when it's down you get the 404s from the main server.

I figured someone might be working on the master, though perhaps it
just burped on its own :-)

-- David


From martin at v.loewis.de  Sat Nov 14 09:43:50 2009
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sat, 14 Nov 2009 09:43:50 +0100
Subject: [Python-Dev] PyPI comments and ratings, *really*?
In-Reply-To: <20091113101906.GA3246@kinakuta.local>
References: <D7E23A6D-C91C-46D8-A8FE-1EB16F3B2101@lericson.se>	<200911130038.05516.steve@pearwood.info>	<7EAD464A-45AC-41EF-9E11-A9BAA3461DB9@lericson.se>	<200911131722.38598.steve@pearwood.info>	<87aayqvrif.fsf@benfinney.id.au>
	<20091113101906.GA3246@kinakuta.local>
Message-ID: <4AFE6DC6.60308@v.loewis.de>

> What I really like on PyPi is that my packages are tested automatically
> with Cheesecake and the order of packages when searching is determined
> by this rating.

That is an (apparently widespread) myth. The Cheesecake rating is not
considered in ranking search results; it's just the relevance of the
search term that is.

Regards,
Martin

From martin at v.loewis.de  Sat Nov 14 09:45:04 2009
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sat, 14 Nov 2009 09:45:04 +0100
Subject: [Python-Dev] PyPI comments and ratings, *really*?
In-Reply-To: <79990c6b0911131133i2987a5d4s5c4ef780a527732@mail.gmail.com>
References: <D7E23A6D-C91C-46D8-A8FE-1EB16F3B2101@lericson.se>	<hdi17l$cli$1@ger.gmane.org>
	<4AFC9064.20700@v.loewis.de>	<4222a8490911121511s40eba937n2c7b1fde6f695c6c@mail.gmail.com>	<4AFC9975.7040204@v.loewis.de>	<4222a8490911121534h30572eabv73ae58649c77155c@mail.gmail.com>	<4AFC9D0C.5030500@v.loewis.de>
	<87y6mbuubp.fsf@benfinney.id.au>	<20e5c9660911121702r162caf28jb6d656383147d909@mail.gmail.com>	<hdim34$utf$1@ger.gmane.org>
	<79990c6b0911131133i2987a5d4s5c4ef780a527732@mail.gmail.com>
Message-ID: <4AFE6E10.3050307@v.loewis.de>

Paul Moore wrote:
> 2009/11/13 Tres Seaver <tseaver at palladion.com>:
>> I can see the information about the poll, and a link to view the
>> results, without logging in.
>>
>>  http://pypi.python.org/pypi
>>
>> (second paragraph there).  That paragraph tells you that you need to log
>> in to vote in the poll.
> 
> I don't want to create a PyPI account (more account details I'll
> rarely use to remember) just to vote. I can see why anonymous votes
> are bad, but the sample's going to be self-selecting - people like me
> who don't want to create an account will be excluded.

This is indeed intentional: people like you won't upload packages to
PyPI, nor will they take part in the rating system, as both require
a PyPI account.

Regards,
Martin

From martin at v.loewis.de  Sat Nov 14 09:52:58 2009
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sat, 14 Nov 2009 09:52:58 +0100
Subject: [Python-Dev] PyPI governance
In-Reply-To: <4AFDFD52.4070401@simplistix.co.uk>
References: <D7E23A6D-C91C-46D8-A8FE-1EB16F3B2101@lericson.se>	<4AFC9064.20700@v.loewis.de>	<4222a8490911121511s40eba937n2c7b1fde6f695c6c@mail.gmail.com>	<4AFC9975.7040204@v.loewis.de>	<4222a8490911121534h30572eabv73ae58649c77155c@mail.gmail.com>	<4AFC9D0C.5030500@v.loewis.de>
	<87y6mbuubp.fsf@benfinney.id.au>	<20e5c9660911121702r162caf28jb6d656383147d909@mail.gmail.com>	<hdim34$utf$1@ger.gmane.org>	<79990c6b0911131133i2987a5d4s5c4ef780a527732@mail.gmail.com>	<bc8ec34e0911131417y47315a7bp7a4fb1dfde7056bd@mail.gmail.com>
	<4AFDFD52.4070401@simplistix.co.uk>
Message-ID: <4AFE6FEA.5070704@v.loewis.de>

> Martin, are you interested in help?

Certainly, yes. For the specific feature in question, I'd like to wait
for the outcome of the poll. Otherwise, contributions are welcome.

Regards,
Martin

From martin at v.loewis.de  Sat Nov 14 10:09:44 2009
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sat, 14 Nov 2009 10:09:44 +0100
Subject: [Python-Dev] Status of the Buildbot fleet and related bugs
In-Reply-To: <m2pr7l7m8j.fsf@valheru.db3l.homeip.net>
References: <Pine.LNX.4.64.0911052209000.12274@kimball.webabinitio.net>	<hdcn72$2rl$1@ger.gmane.org>
	<4AF9E3D6.2000807@v.loewis.de>	<loom.20091114T010914-556@post.gmane.org>	<Pine.LNX.4.64.0911132317040.12274@kimball.webabinitio.net>
	<m2pr7l7m8j.fsf@valheru.db3l.homeip.net>
Message-ID: <4AFE73D8.9020509@v.loewis.de>

> Yes, I think this just started happening. I'm guessing that the main
> site proxies the buildbot URL requests to the buildbot master process,
> and when it's down you get the 404s from the main server.
> 
> I figured someone might be working on the master, though perhaps it
> just burped on its own :-)

It was actually an Apache misconfiguration (the wrong virtual host would
pick up requests, missing the reverse proxy configuration). I have fixed
that now.

Regards,
Martin

From wentland at cl.uni-heidelberg.de  Sat Nov 14 14:18:13 2009
From: wentland at cl.uni-heidelberg.de (Wolodja Wentland)
Date: Sat, 14 Nov 2009 14:18:13 +0100
Subject: [Python-Dev] PyPI comments and ratings, *really*?
In-Reply-To: <4AFE6DC6.60308@v.loewis.de>
References: <D7E23A6D-C91C-46D8-A8FE-1EB16F3B2101@lericson.se>
	<200911130038.05516.steve@pearwood.info>
	<7EAD464A-45AC-41EF-9E11-A9BAA3461DB9@lericson.se>
	<200911131722.38598.steve@pearwood.info>
	<87aayqvrif.fsf@benfinney.id.au>
	<20091113101906.GA3246@kinakuta.local> <4AFE6DC6.60308@v.loewis.de>
Message-ID: <20091114131813.GA3798@kinakuta.local>

On Sat, Nov 14, 2009 at 09:43 +0100, "Martin v. L?wis" wrote:
> > What I really like on PyPi is that my packages are tested automatically
> > with Cheesecake and the order of packages when searching is determined
> > by this rating.

> That is an (apparently widespread) myth. The Cheesecake rating is not
> considered in ranking search results; it's just the relevance of the
> search term that is.

Thanks for clarifying this!

What about the other points?

Crossposted to catalog-sig to move the discussion there.
-- 
  .''`.     Wolodja Wentland    <wentland at cl.uni-heidelberg.de> 
 : :'  :    
 `. `'`     4096R/CAF14EFC 
   `-       081C B7CD FF04 2BA9 94EA  36B2 8B7F 7D30 CAF1 4EFC
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 836 bytes
Desc: Digital signature
URL: <http://mail.python.org/pipermail/python-dev/attachments/20091114/551494bc/attachment.pgp>

From martin at v.loewis.de  Sat Nov 14 14:22:22 2009
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sat, 14 Nov 2009 14:22:22 +0100
Subject: [Python-Dev] PyPI comments and ratings, *really*?
In-Reply-To: <20091114131813.GA3798@kinakuta.local>
References: <D7E23A6D-C91C-46D8-A8FE-1EB16F3B2101@lericson.se>	<200911130038.05516.steve@pearwood.info>	<7EAD464A-45AC-41EF-9E11-A9BAA3461DB9@lericson.se>	<200911131722.38598.steve@pearwood.info>	<87aayqvrif.fsf@benfinney.id.au>	<20091113101906.GA3246@kinakuta.local>
	<4AFE6DC6.60308@v.loewis.de> <20091114131813.GA3798@kinakuta.local>
Message-ID: <4AFEAF0E.1040808@v.loewis.de>

> 
>> That is an (apparently widespread) myth. The Cheesecake rating is not
>> considered in ranking search results; it's just the relevance of the
>> search term that is.
> 
> Thanks for clarifying this!
> 
> What about the other points?

They are really off-topic for python-dev; PyPI discussion should take
place on catalog-sig.

I personally had problems following your message, as it seemed to mix
too many aspects into one message.

Regards,
Martin

From pje at telecommunity.com  Sat Nov 14 15:39:24 2009
From: pje at telecommunity.com (P.J. Eby)
Date: Sat, 14 Nov 2009 09:39:24 -0500
Subject: [Python-Dev] PyPI front page
In-Reply-To: <b654cd2e0911121733v4b5667e0j8837fe825f925086@mail.gmail.co
 m>
References: <D7E23A6D-C91C-46D8-A8FE-1EB16F3B2101@lericson.se>
	<Pine.LNX.4.64.0911121648480.12274@kimball.webabinitio.net>
	<hdi17l$cli$1@ger.gmane.org> <4AFC9064.20700@v.loewis.de>
	<4222a8490911121511s40eba937n2c7b1fde6f695c6c@mail.gmail.com>
	<4AFC9975.7040204@v.loewis.de>
	<4222a8490911121534h30572eabv73ae58649c77155c@mail.gmail.com>
	<4AFC9D0C.5030500@v.loewis.de> <87y6mbuubp.fsf@benfinney.id.au>
	<loom.20091113T015130-137@post.gmane.org>
	<b654cd2e0911121733v4b5667e0j8837fe825f925086@mail.gmail.com>
Message-ID: <20091114143931.507E33A411A@sparrow.telecommunity.com>

At 08:33 PM 11/12/2009 -0500, Ian Bicking wrote:
>On Thu, Nov 12, 2009 at 7:52 PM, Antoine Pitrou 
><<mailto:solipsis at pitrou.net>solipsis at pitrou.net> wrote:
>Ben Finney <ben+python <at> <http://benfinney.id.au>benfinney.id.au> writes:
> >
> > There's a problem with the poll's placement: on the front page of the
> > PyPI website.
>
>Speaking of which, why is it that 
><http://pypi.python.org/pypi>http://pypi.python.org/pypi and
><http://pypi.python.org/pypi/>http://pypi.python.org/pypi/ (note the 
>ending slash) return different contents
>(the latter being very voluminous)? I always mistake one for the other when
>entering the URL directly.
>
>
>easy_install replied on the behavior of /pypi/ (it uses the long 
>list to do case-insensitive searches).?  Someone changed it, 
>easy_install broke, and a compromise was to keep /pypi/ the way it 
>was (but not /pypi).
>
>Probably this could be removed, as the /simple/ index is already 
>case-insensitive, so easy_install shouldn't have to hit /pypi/ at all.

This was changed over a year ago; easy_install *does* use /simple by 
default.  I would guess enough people are upgraded by now that PyPI 
need no longer continue to support it.


From pje at telecommunity.com  Sat Nov 14 15:52:21 2009
From: pje at telecommunity.com (P.J. Eby)
Date: Sat, 14 Nov 2009 09:52:21 -0500
Subject: [Python-Dev] PyPI comments and ratings, *really*?
In-Reply-To: <4AFE6E10.3050307@v.loewis.de>
References: <D7E23A6D-C91C-46D8-A8FE-1EB16F3B2101@lericson.se>
	<hdi17l$cli$1@ger.gmane.org> <4AFC9064.20700@v.loewis.de>
	<4222a8490911121511s40eba937n2c7b1fde6f695c6c@mail.gmail.com>
	<4AFC9975.7040204@v.loewis.de>
	<4222a8490911121534h30572eabv73ae58649c77155c@mail.gmail.com>
	<4AFC9D0C.5030500@v.loewis.de> <87y6mbuubp.fsf@benfinney.id.au>
	<20e5c9660911121702r162caf28jb6d656383147d909@mail.gmail.com>
	<hdim34$utf$1@ger.gmane.org>
	<79990c6b0911131133i2987a5d4s5c4ef780a527732@mail.gmail.com>
	<4AFE6E10.3050307@v.loewis.de>
Message-ID: <20091114145222.084583A411A@sparrow.telecommunity.com>

At 09:45 AM 11/14/2009 +0100, Martin v. L?wis wrote:
>Paul Moore wrote:
> > 2009/11/13 Tres Seaver <tseaver at palladion.com>:
> >> I can see the information about the poll, and a link to view the
> >> results, without logging in.
> >>
> >>  http://pypi.python.org/pypi
> >>
> >> (second paragraph there).  That paragraph tells you that you need to log
> >> in to vote in the poll.
> >
> > I don't want to create a PyPI account (more account details I'll
> > rarely use to remember) just to vote. I can see why anonymous votes
> > are bad, but the sample's going to be self-selecting - people like me
> > who don't want to create an account will be excluded.
>
>This is indeed intentional: people like you won't upload packages to
>PyPI, nor will they take part in the rating system, as both require
>a PyPI account.

Which is bizarre, since Paul belongs to the group of people you say 
you care most about - i.e., those people browsing the index and 
looking for packages.  The *consumers* of the comments, in other words.


From martin at v.loewis.de  Sat Nov 14 16:19:22 2009
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sat, 14 Nov 2009 16:19:22 +0100
Subject: [Python-Dev] PyPI comments and ratings, *really*?
In-Reply-To: <20091114145222.084583A411A@sparrow.telecommunity.com>
References: <D7E23A6D-C91C-46D8-A8FE-1EB16F3B2101@lericson.se>
	<hdi17l$cli$1@ger.gmane.org> <4AFC9064.20700@v.loewis.de>
	<4222a8490911121511s40eba937n2c7b1fde6f695c6c@mail.gmail.com>
	<4AFC9975.7040204@v.loewis.de>
	<4222a8490911121534h30572eabv73ae58649c77155c@mail.gmail.com>
	<4AFC9D0C.5030500@v.loewis.de> <87y6mbuubp.fsf@benfinney.id.au>
	<20e5c9660911121702r162caf28jb6d656383147d909@mail.gmail.com>
	<hdim34$utf$1@ger.gmane.org>
	<79990c6b0911131133i2987a5d4s5c4ef780a527732@mail.gmail.com>
	<4AFE6E10.3050307@v.loewis.de>
	<20091114145222.084583A411A@sparrow.telecommunity.com>
Message-ID: <4AFECA7A.8070400@v.loewis.de>

>> > I don't want to create a PyPI account (more account details I'll
>> > rarely use to remember) just to vote. I can see why anonymous votes
>> > are bad, but the sample's going to be self-selecting - people like me
>> > who don't want to create an account will be excluded.
>>
>> This is indeed intentional: people like you won't upload packages to
>> PyPI, nor will they take part in the rating system, as both require
>> a PyPI account.
> 
> Which is bizarre, since Paul belongs to the group of people you say you
> care most about - i.e., those people browsing the index and looking for
> packages.  The *consumers* of the comments, in other words.

Ok: "intentional" is not the right attribute. It's more that I don't
consider it too harmful.

Regards,
Martin

From steve at pearwood.info  Sat Nov 14 16:25:54 2009
From: steve at pearwood.info (Steven D'Aprano)
Date: Sun, 15 Nov 2009 02:25:54 +1100
Subject: [Python-Dev] PyPI comments and ratings, *really*?
In-Reply-To: <20091114145222.084583A411A@sparrow.telecommunity.com>
References: <D7E23A6D-C91C-46D8-A8FE-1EB16F3B2101@lericson.se>
	<4AFE6E10.3050307@v.loewis.de>
	<20091114145222.084583A411A@sparrow.telecommunity.com>
Message-ID: <200911150225.55412.steve@pearwood.info>

On Sun, 15 Nov 2009 01:52:21 am P.J. Eby wrote:

> > > I don't want to create a PyPI account (more account details I'll
> > > rarely use to remember) just to vote. I can see why anonymous
> > > votes are bad, but the sample's going to be self-selecting -
> > > people like me who don't want to create an account will be
> > > excluded.
> >
> >This is indeed intentional: people like you won't upload packages to
> >PyPI, nor will they take part in the rating system, as both require
> >a PyPI account.
>
> Which is bizarre, since Paul belongs to the group of people you say
> you care most about - i.e., those people browsing the index and
> looking for packages.  The *consumers* of the comments, in other
> words.

Not bizarre at all, practical. Without authenticated votes, gaming the 
system goes from technically challenging to simple enough anyone can do 
it. Martin may have the best interests of "consumers" in mind, but he 
can't force them to act in their best interest. If they choose to not 
vote, that's their choice to make.

Think of it as voting registration. Even in countries like Australia 
with compulsory voting, you have to register first, to ensure that 
people make a single vote.

Since Paul doesn't have an account, he can't make comments, or rate 
packages, or cast a vote. This is by design, not out of a desire to 
defranchise people like Paul, but because the alternatives -- easy 
comment spam, people voting multiple times -- are worse than some 
proportion of users being unable to vote.




-- 
Steven D'Aprano

From floris.bruynooghe at gmail.com  Sat Nov 14 16:29:21 2009
From: floris.bruynooghe at gmail.com (Floris Bruynooghe)
Date: Sat, 14 Nov 2009 15:29:21 +0000
Subject: [Python-Dev] PyPI comments and ratings, *really*?
In-Reply-To: <hdksh9$jkt$1@ger.gmane.org>
References: <4AFC9064.20700@v.loewis.de>
	<20091113221234.GA15732@amk-desktop.matrixgroup.net>
	<87bpj6t4mp.fsf@benfinney.id.au>
	<200911141018.13536.steve@pearwood.info>
	<hdksh9$jkt$1@ger.gmane.org>
Message-ID: <20091114152921.GA28003@laurie.devork>

On Fri, Nov 13, 2009 at 06:10:16PM -0600, Robert Kern wrote:
> If you want one idea that would make my use of PyPI much more
> pleasant and informative, it would be to add a "Repository-URL"
> entry to the recommended PyPI metadata so that I could always start
> looking at the code in one click.

+1

Having a "Repository-URL", "Repository-Browse-URL" and a
"Bug-Tracker-URL" field in PyPI would be a lot more usefule then
comments and ratings.


Regards
Floris

From arcriley at gmail.com  Sat Nov 14 16:34:30 2009
From: arcriley at gmail.com (Arc Riley)
Date: Sat, 14 Nov 2009 10:34:30 -0500
Subject: [Python-Dev] PyPI comments and ratings, *really*?
In-Reply-To: <20091114152921.GA28003@laurie.devork>
References: <4AFC9064.20700@v.loewis.de>
	<20091113221234.GA15732@amk-desktop.matrixgroup.net> 
	<87bpj6t4mp.fsf@benfinney.id.au>
	<200911141018.13536.steve@pearwood.info> 
	<hdksh9$jkt$1@ger.gmane.org> <20091114152921.GA28003@laurie.devork>
Message-ID: <bad82a80911140734i822938fm8b57d15db5364c66@mail.gmail.com>

> +1
>
> Having a "Repository-URL", "Repository-Browse-URL" and a
> "Bug-Tracker-URL" field in PyPI would be a lot more usefule then
> comments and ratings.
>
>
+1
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20091114/b1cfb0a2/attachment.htm>

From baptiste13z at free.fr  Sat Nov 14 17:51:59 2009
From: baptiste13z at free.fr (Baptiste Carvello)
Date: Sat, 14 Nov 2009 17:51:59 +0100
Subject: [Python-Dev] PyPI comments and ratings, *really*?
In-Reply-To: <hdksh9$jkt$1@ger.gmane.org>
References: <4AFC9064.20700@v.loewis.de>	<20091113221234.GA15732@amk-desktop.matrixgroup.net>	<87bpj6t4mp.fsf@benfinney.id.au>	<200911141018.13536.steve@pearwood.info>
	<hdksh9$jkt$1@ger.gmane.org>
Message-ID: <hdmn7m$niv$1@ger.gmane.org>

Robert Kern a ?crit :

> While I do have a couple of packages on PyPI, I use PyPI as a consumer 
> of packages much more frequently, every day in fact. 

Another "consumer" opinion: when investigating a new package, I usually look for 
the following things, in that order:

1) the "big picture" description: is there a coherent design? does it fit my 
needs? On PyPI, the description field should provide that.

2) the changelog: is the project still alive? are bugs fixed, or just features 
added? is the code rewritten from scratch for each release? Well, every project 
on PyPI should have a public changelog, so a link to that fits my need.

3) documentation: I don't necessarily care for the number of lines, but more 
whether it is understandable and goes into sufficient detail to not leave me 
guessing. Again, a link to the docs fits my needs. A single number (number of 
documentation lines,...) does not.

4) a mailing list archive (or newsgroup, or web forum), where I'm looking for 
signs of a healthy community. I usually go for the -devel list, but a -user list 
will do as well if the committers keep an eye on it.

If a healthy community exists around a project, I will completely disregard 
comments, if present: time invested in the community speaks louder than the 
opinion of random bystanders. Only for small projects with no real community 
will I look at the comments (+ answers from the author) in order to make an 
opinion on the developpement process. I always disregard ratings.

So as a conclusion about comments, they can have their use for projects without 
a publicly archived mailing list, but can otherwise be *replaced* by a direct 
link to a list archive. This could be a reasonable default for PyPI: disable 
comments when a link to a list archive is provided.

While my experience may not be that of a typical user, I believe users will 
ultimately make use of all information they are provided. So it is important to 
provide the most relevant information, and not just what naive users ask for.

Cheers,
B.


From tjreedy at udel.edu  Sat Nov 14 21:31:00 2009
From: tjreedy at udel.edu (Terry Reedy)
Date: Sat, 14 Nov 2009 15:31:00 -0500
Subject: [Python-Dev] Too many Python accounts (was Re: PyPI comments and
	ratings, *really*?)
In-Reply-To: <4AFECA7A.8070400@v.loewis.de>
References: <D7E23A6D-C91C-46D8-A8FE-1EB16F3B2101@lericson.se>	<hdi17l$cli$1@ger.gmane.org>
	<4AFC9064.20700@v.loewis.de>	<4222a8490911121511s40eba937n2c7b1fde6f695c6c@mail.gmail.com>	<4AFC9975.7040204@v.loewis.de>	<4222a8490911121534h30572eabv73ae58649c77155c@mail.gmail.com>	<4AFC9D0C.5030500@v.loewis.de>
	<87y6mbuubp.fsf@benfinney.id.au>	<20e5c9660911121702r162caf28jb6d656383147d909@mail.gmail.com>	<hdim34$utf$1@ger.gmane.org>	<79990c6b0911131133i2987a5d4s5c4ef780a527732@mail.gmail.com>	<4AFE6E10.3050307@v.loewis.de>	<20091114145222.084583A411A@sparrow.telecommunity.com>
	<4AFECA7A.8070400@v.loewis.de>
Message-ID: <hdn42a$r76$1@ger.gmane.org>

Martin v. L?wis wrote:

>>> This is indeed intentional: people like you won't upload packages to
>>> PyPI, nor will they take part in the rating system, as both require
>>> a PyPI account.

Why? I already have python tracker account and a python wiki account 
which is already 2 too many. The latter was a pain because the site lost 
my registration and as of a year ago, the registration process was 
broken. I would much prefer just one python site registration.

Terry Jan Reedy


From ben+python at benfinney.id.au  Sat Nov 14 23:23:55 2009
From: ben+python at benfinney.id.au (Ben Finney)
Date: Sun, 15 Nov 2009 09:23:55 +1100
Subject: [Python-Dev] Too many Python accounts
References: <D7E23A6D-C91C-46D8-A8FE-1EB16F3B2101@lericson.se>
	<hdi17l$cli$1@ger.gmane.org> <4AFC9064.20700@v.loewis.de>
	<4222a8490911121511s40eba937n2c7b1fde6f695c6c@mail.gmail.com>
	<4AFC9975.7040204@v.loewis.de>
	<4222a8490911121534h30572eabv73ae58649c77155c@mail.gmail.com>
	<4AFC9D0C.5030500@v.loewis.de> <87y6mbuubp.fsf@benfinney.id.au>
	<20e5c9660911121702r162caf28jb6d656383147d909@mail.gmail.com>
	<hdim34$utf$1@ger.gmane.org>
	<79990c6b0911131133i2987a5d4s5c4ef780a527732@mail.gmail.com>
	<4AFE6E10.3050307@v.loewis.de>
	<20091114145222.084583A411A@sparrow.telecommunity.com>
	<4AFECA7A.8070400@v.loewis.de> <hdn42a$r76$1@ger.gmane.org>
Message-ID: <87fx8gsq2s.fsf@benfinney.id.au>

Terry Reedy <tjreedy at udel.edu> writes:

> Martin v. L?wis wrote:
>
> >>> This is indeed intentional: people like you won't upload packages
> >>> to PyPI, nor will they take part in the rating system, as both
> >>> require a PyPI account.
>
> Why? I already have python tracker account and a python wiki account
> which is already 2 too many. The latter was a pain because the site
> lost my registration and as of a year ago, the registration process
> was broken. I would much prefer just one python site registration.

And that registration should be using any OpenID, so that I don't need
any new identities to participate on the Python sites: I can re-use
existing identities.

I would make a bug report for that, but the Python bug tracker also
requires yet-another-identity. It's lunacy.

-- 
 \           ?Value your freedom or you will lose it, teaches history. |
  `\     ?Don't bother us with politics,? respond those who don't want |
_o__)                               to learn.? ?Richard Stallman, 2002 |
Ben Finney


From stephen at xemacs.org  Sun Nov 15 00:52:34 2009
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Sun, 15 Nov 2009 08:52:34 +0900
Subject: [Python-Dev] Too many Python accounts
In-Reply-To: <87fx8gsq2s.fsf@benfinney.id.au>
References: <D7E23A6D-C91C-46D8-A8FE-1EB16F3B2101@lericson.se>
	<hdi17l$cli$1@ger.gmane.org> <4AFC9064.20700@v.loewis.de>
	<4222a8490911121511s40eba937n2c7b1fde6f695c6c@mail.gmail.com>
	<4AFC9975.7040204@v.loewis.de>
	<4222a8490911121534h30572eabv73ae58649c77155c@mail.gmail.com>
	<4AFC9D0C.5030500@v.loewis.de> <87y6mbuubp.fsf@benfinney.id.au>
	<20e5c9660911121702r162caf28jb6d656383147d909@mail.gmail.com>
	<hdim34$utf$1@ger.gmane.org>
	<79990c6b0911131133i2987a5d4s5c4ef780a527732@mail.gmail.com>
	<4AFE6E10.3050307@v.loewis.de>
	<20091114145222.084583A411A@sparrow.telecommunity.com>
	<4AFECA7A.8070400@v.loewis.de> <hdn42a$r76$1@ger.gmane.org>
	<87fx8gsq2s.fsf@benfinney.id.au>
Message-ID: <874oowbr5p.fsf@uwakimon.sk.tsukuba.ac.jp>

Ben Finney writes:

 > I would make a bug report for that, but the Python bug tracker also
 > requires yet-another-identity. It's lunacy.

No, it's legacy.  The software predates the availability of OpenID.

If you'd like to integrate Open ID authentication into Roundup, I will
personally be happy to do the followup required to get the patch on
the radar of the Python site and Roundup software maintainers.

From ziade.tarek at gmail.com  Sun Nov 15 02:21:41 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Sun, 15 Nov 2009 02:21:41 +0100
Subject: [Python-Dev] buildtime vs runtime in Distutils
Message-ID: <94bdd2610911141721n6700dcds5a489f1972176e48@mail.gmail.com>

Hello,

http://bugs.python.org/issue4359 reminds me that Distutils reads build
files like Makefile or pyconfig.h to get some environment
variables through the sysconfig module at *runtime*.

This cannot work on all platforms, when our Makefile is not shipped
with python but python-devel. (like Fedora)

Since I am already working on the refactoring of this
distutils/sysconfig module, so it lands in a stdlib module together
with some
elements from site.py (see a previous mail on the topic, and my
tarek_sysconfig branch - work in progress),

I was wondering if we couldn't make this module a template, and inject
at its beginning new values extracted from Makefile and pyconfig.h.

This could happen when "configure" is called,

Regards,
Tarek

-- 
Tarek Ziad? | http://ziade.org |????????????! |????????????

From martin at v.loewis.de  Sun Nov 15 02:36:26 2009
From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=)
Date: Sun, 15 Nov 2009 02:36:26 +0100
Subject: [Python-Dev] Too many Python accounts (was Re: PyPI comments
 and	ratings, *really*?)
In-Reply-To: <hdn42a$r76$1@ger.gmane.org>
References: <D7E23A6D-C91C-46D8-A8FE-1EB16F3B2101@lericson.se>	<hdi17l$cli$1@ger.gmane.org>	<4AFC9064.20700@v.loewis.de>	<4222a8490911121511s40eba937n2c7b1fde6f695c6c@mail.gmail.com>	<4AFC9975.7040204@v.loewis.de>	<4222a8490911121534h30572eabv73ae58649c77155c@mail.gmail.com>	<4AFC9D0C.5030500@v.loewis.de>	<87y6mbuubp.fsf@benfinney.id.au>	<20e5c9660911121702r162caf28jb6d656383147d909@mail.gmail.com>	<hdim34$utf$1@ger.gmane.org>	<79990c6b0911131133i2987a5d4s5c4ef780a527732@mail.gmail.com>	<4AFE6E10.3050307@v.loewis.de>	<20091114145222.084583A411A@sparrow.telecommunity.com>	<4AFECA7A.8070400@v.loewis.de>
	<hdn42a$r76$1@ger.gmane.org>
Message-ID: <4AFF5B1A.8090105@v.loewis.de>

> Why? I already have python tracker account and a python wiki account
> which is already 2 too many. The latter was a pain because the site lost
> my registration and as of a year ago, the registration process was
> broken. I would much prefer just one python site registration.

Then I recommend that you get a google account for your email address,
and register to PyPI using OpenID.

Regards,
Martin

From martin at v.loewis.de  Sun Nov 15 02:36:54 2009
From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=)
Date: Sun, 15 Nov 2009 02:36:54 +0100
Subject: [Python-Dev] Too many Python accounts
In-Reply-To: <87fx8gsq2s.fsf@benfinney.id.au>
References: <D7E23A6D-C91C-46D8-A8FE-1EB16F3B2101@lericson.se>	<hdi17l$cli$1@ger.gmane.org>
	<4AFC9064.20700@v.loewis.de>	<4222a8490911121511s40eba937n2c7b1fde6f695c6c@mail.gmail.com>	<4AFC9975.7040204@v.loewis.de>	<4222a8490911121534h30572eabv73ae58649c77155c@mail.gmail.com>	<4AFC9D0C.5030500@v.loewis.de>
	<87y6mbuubp.fsf@benfinney.id.au>	<20e5c9660911121702r162caf28jb6d656383147d909@mail.gmail.com>	<hdim34$utf$1@ger.gmane.org>	<79990c6b0911131133i2987a5d4s5c4ef780a527732@mail.gmail.com>	<4AFE6E10.3050307@v.loewis.de>	<20091114145222.084583A411A@sparrow.telecommunity.com>	<4AFECA7A.8070400@v.loewis.de>
	<hdn42a$r76$1@ger.gmane.org> <87fx8gsq2s.fsf@benfinney.id.au>
Message-ID: <4AFF5B36.9030502@v.loewis.de>

> And that registration should be using any OpenID, so that I don't need
> any new identities to participate on the Python sites: I can re-use
> existing identities.

PyPI actually does support OpenID.

Regards,
Martin

From martin at v.loewis.de  Sun Nov 15 02:38:30 2009
From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=)
Date: Sun, 15 Nov 2009 02:38:30 +0100
Subject: [Python-Dev] buildtime vs runtime in Distutils
In-Reply-To: <94bdd2610911141721n6700dcds5a489f1972176e48@mail.gmail.com>
References: <94bdd2610911141721n6700dcds5a489f1972176e48@mail.gmail.com>
Message-ID: <4AFF5B96.5030703@v.loewis.de>

> http://bugs.python.org/issue4359 reminds me that Distutils reads build
> files like Makefile or pyconfig.h to get some environment
> variables through the sysconfig module at *runtime*.
> 
> This cannot work on all platforms, when our Makefile is not shipped
> with python but python-devel. (like Fedora)

I don't see a problem with that: you'll need the python-devel package
*anyway* when running distutils, for many packages.

Regards,
Martin

From arfrever.fta at gmail.com  Sun Nov 15 02:41:18 2009
From: arfrever.fta at gmail.com (Arfrever Frehtes Taifersar Arahesis)
Date: Sun, 15 Nov 2009 02:41:18 +0100
Subject: [Python-Dev] buildtime vs runtime in Distutils
In-Reply-To: <94bdd2610911141721n6700dcds5a489f1972176e48@mail.gmail.com>
References: <94bdd2610911141721n6700dcds5a489f1972176e48@mail.gmail.com>
Message-ID: <200911150241.26083.Arfrever.FTA@gmail.com>

2009-11-15 02:21:41 Tarek Ziad? napisa?(a):
> Hello,
> 
> http://bugs.python.org/issue4359 reminds me that Distutils reads build
> files like Makefile or pyconfig.h to get some environment
> variables through the sysconfig module at *runtime*.
> 
> This cannot work on all platforms, when our Makefile is not shipped
> with python but python-devel. (like Fedora)
> 
> Since I am already working on the refactoring of this
> distutils/sysconfig module, so it lands in a stdlib module together
> with some
> elements from site.py (see a previous mail on the topic, and my
> tarek_sysconfig branch - work in progress),
> 
> I was wondering if we couldn't make this module a template, and inject
> at its beginning new values extracted from Makefile and pyconfig.h.
> 
> This could happen when "configure" is called,

It seems to be a good idea. You should create a .py.in file and use
AC_SUBST and AC_CONFIG_FILES macros in configure.in.

-- 
Arfrever Frehtes Taifersar Arahesis
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 836 bytes
Desc: This is a digitally signed message part.
URL: <http://mail.python.org/pipermail/python-dev/attachments/20091115/3fd81593/attachment.pgp>

From ben+python at benfinney.id.au  Sun Nov 15 02:44:11 2009
From: ben+python at benfinney.id.au (Ben Finney)
Date: Sun, 15 Nov 2009 12:44:11 +1100
Subject: [Python-Dev] Too many Python accounts
References: <D7E23A6D-C91C-46D8-A8FE-1EB16F3B2101@lericson.se>
	<hdi17l$cli$1@ger.gmane.org> <4AFC9064.20700@v.loewis.de>
	<4222a8490911121511s40eba937n2c7b1fde6f695c6c@mail.gmail.com>
	<4AFC9975.7040204@v.loewis.de>
	<4222a8490911121534h30572eabv73ae58649c77155c@mail.gmail.com>
	<4AFC9D0C.5030500@v.loewis.de> <87y6mbuubp.fsf@benfinney.id.au>
	<20e5c9660911121702r162caf28jb6d656383147d909@mail.gmail.com>
	<hdim34$utf$1@ger.gmane.org>
	<79990c6b0911131133i2987a5d4s5c4ef780a527732@mail.gmail.com>
	<4AFE6E10.3050307@v.loewis.de>
	<20091114145222.084583A411A@sparrow.telecommunity.com>
	<4AFECA7A.8070400@v.loewis.de> <hdn42a$r76$1@ger.gmane.org>
	<87fx8gsq2s.fsf@benfinney.id.au>
	<874oowbr5p.fsf@uwakimon.sk.tsukuba.ac.jp>
Message-ID: <87y6m8r28k.fsf@benfinney.id.au>

"Stephen J. Turnbull" <stephen at xemacs.org> writes:

> Ben Finney writes:
>
>  > I would make a bug report for that, but the Python bug tracker also
>  > requires yet-another-identity. It's lunacy.
>
> No, it's legacy.  The software predates the availability of OpenID.

I don't agree that's a ?no?; it's a ?yes, and? :-)

> If you'd like to integrate Open ID authentication into Roundup, I will
> personally be happy to do the followup required to get the patch on
> the radar of the Python site and Roundup software maintainers.

I'll give the usual caveat of ?this is but one of the many services I
would like to be able to authenticate in with OpenID?, but it's a caveat
rather than a rejection. I'll keep your kind offer in mind in case I get
the bandwidth to take that on.

-- 
 \          ?Our products just aren't engineered for security.? ?Brian |
  `\             Valentine, senior vice-president of Microsoft Windows |
_o__)                                                development, 2002 |
Ben Finney


From ziade.tarek at gmail.com  Sun Nov 15 02:54:43 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Sun, 15 Nov 2009 02:54:43 +0100
Subject: [Python-Dev] buildtime vs runtime in Distutils
In-Reply-To: <4AFF5B96.5030703@v.loewis.de>
References: <94bdd2610911141721n6700dcds5a489f1972176e48@mail.gmail.com>
	<4AFF5B96.5030703@v.loewis.de>
Message-ID: <94bdd2610911141754o37f45987ofa989b5267d282ba@mail.gmail.com>

2009/11/15 "Martin v. L?wis" <martin at v.loewis.de>:
>> http://bugs.python.org/issue4359 reminds me that Distutils reads build
>> files like Makefile or pyconfig.h to get some environment
>> variables through the sysconfig module at *runtime*.
>>
>> This cannot work on all platforms, when our Makefile is not shipped
>> with python but python-devel. (like Fedora)
>
> I don't see a problem with that: you'll need the python-devel package
> *anyway* when running distutils, for many packages.

The problem is that the main python distribution ("python") is not
working as advertised since
it contains distutils, which requires "python-devel" to work.

This implies that "python" has a dependency on "python-devel", which
does not make sense
anymore for linux distros to have two distinct packages for Python.

Having some of the makefile vars stored in stdlib solve this problem.

Tarek

From ziade.tarek at gmail.com  Sun Nov 15 02:56:04 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Sun, 15 Nov 2009 02:56:04 +0100
Subject: [Python-Dev] buildtime vs runtime in Distutils
In-Reply-To: <200911150241.26083.Arfrever.FTA@gmail.com>
References: <94bdd2610911141721n6700dcds5a489f1972176e48@mail.gmail.com>
	<200911150241.26083.Arfrever.FTA@gmail.com>
Message-ID: <94bdd2610911141756o230e7dd9gd6abd385522454f0@mail.gmail.com>

On Sun, Nov 15, 2009 at 2:41 AM, Arfrever Frehtes Taifersar Arahesis
<arfrever.fta at gmail.com> wrote:
[..]
>>
>> This could happen when "configure" is called,
>
> It seems to be a good idea. You should create a .py.in file and use
> AC_SUBST and AC_CONFIG_FILES macros in configure.in.

I need to investigate on these, thx

Tarek

From martin at v.loewis.de  Sun Nov 15 02:57:12 2009
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sun, 15 Nov 2009 02:57:12 +0100
Subject: [Python-Dev] buildtime vs runtime in Distutils
In-Reply-To: <94bdd2610911141754o37f45987ofa989b5267d282ba@mail.gmail.com>
References: <94bdd2610911141721n6700dcds5a489f1972176e48@mail.gmail.com>	
	<4AFF5B96.5030703@v.loewis.de>
	<94bdd2610911141754o37f45987ofa989b5267d282ba@mail.gmail.com>
Message-ID: <4AFF5FF8.7040708@v.loewis.de>

> The problem is that the main python distribution ("python") is not
> working as advertised since
> it contains distutils, which requires "python-devel" to work.
> 
> This implies that "python" has a dependency on "python-devel", which
> does not make sense
> anymore for linux distros to have two distinct packages for Python.
> 
> Having some of the makefile vars stored in stdlib solve this problem.

I don't see how this can solve the problem. Distutils contains the
build_ext command, which definitely will require the devel package.

Regards,
Martin

From ben+python at benfinney.id.au  Sun Nov 15 03:11:26 2009
From: ben+python at benfinney.id.au (Ben Finney)
Date: Sun, 15 Nov 2009 13:11:26 +1100
Subject: [Python-Dev] Too many Python accounts
References: <D7E23A6D-C91C-46D8-A8FE-1EB16F3B2101@lericson.se>
	<hdi17l$cli$1@ger.gmane.org> <4AFC9064.20700@v.loewis.de>
	<4222a8490911121511s40eba937n2c7b1fde6f695c6c@mail.gmail.com>
	<4AFC9975.7040204@v.loewis.de>
	<4222a8490911121534h30572eabv73ae58649c77155c@mail.gmail.com>
	<4AFC9D0C.5030500@v.loewis.de> <87y6mbuubp.fsf@benfinney.id.au>
	<20e5c9660911121702r162caf28jb6d656383147d909@mail.gmail.com>
	<hdim34$utf$1@ger.gmane.org>
	<79990c6b0911131133i2987a5d4s5c4ef780a527732@mail.gmail.com>
	<4AFE6E10.3050307@v.loewis.de>
	<20091114145222.084583A411A@sparrow.telecommunity.com>
	<4AFECA7A.8070400@v.loewis.de> <hdn42a$r76$1@ger.gmane.org>
	<87fx8gsq2s.fsf@benfinney.id.au> <4AFF5B36.9030502@v.loewis.de>
Message-ID: <87tywwr0z5.fsf@benfinney.id.au>

"Martin v. L?wis" <martin at v.loewis.de> writes:

> > And that registration should be using any OpenID, so that I don't
> > need any new identities to participate on the Python sites: I can
> > re-use existing identities.
>
> PyPI actually does support OpenID.

I commend the PyPI administrators for this step, but the implementation
is currently insufficient: it conflates a user's OpenID (their identity,
as a URL) with their OpenID provider (the service which the person has
chosen to do the authentication check and serve the data). That's what I
meant by ?should be using any OpenID?.

One of the best features of the OpenID system is identity delegation:
that one's identity can be decoupled from the service behind the scenes
which provides that identity. This is important, because it means I am
not tied to a particular provider to maintain my identity; if they no
longer provide my identity in a way I like, I can switch to a different
provider while keeping the same identity.

Fred can use his own OpenID ?fred.example.org?, initially set up behind
the scenes to delegate to ?bigcorp.example.com? as the provider. Any
time he likes, Fred can *change* which provider is actually used for
authentication, without changing his OpenID. PyPI gets to find out which
provider Fred is using for the identity ?fred.example.org? each time it
performs discovery on that identity, not before.

So, PyPI should not be asking the user ?what is your provider?? since
that's (a) a detail irrelevant to the user if they just know their
OpenID, (b) liable to change independent of the OpenID, and (c)
discoverable from the OpenID auth process anyway.

PyPI should instead ask the user for their OpenID (?fred.example.org?),
without discussing providers. Then, attempt to authenticate that user,
at which point PyPI automatically gets to find out which provider is in
use (?bigcorp.example.com?). If you *then* want to be picky and complain
that PyPI refuses identities provided by ?bigcorp.example.com?, that's
the time to do it.

I hope that makes more sense.

-- 
 \         ?Geeks like to think that they can ignore politics. You can |
  `\        leave politics alone, but politics won't leave you alone.? |
_o__)                                    ?Richard Stallman, 2002-07-26 |
Ben Finney


From ziade.tarek at gmail.com  Sun Nov 15 03:13:53 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Sun, 15 Nov 2009 03:13:53 +0100
Subject: [Python-Dev] buildtime vs runtime in Distutils
In-Reply-To: <4AFF5FF8.7040708@v.loewis.de>
References: <94bdd2610911141721n6700dcds5a489f1972176e48@mail.gmail.com>
	<4AFF5B96.5030703@v.loewis.de>
	<94bdd2610911141754o37f45987ofa989b5267d282ba@mail.gmail.com>
	<4AFF5FF8.7040708@v.loewis.de>
Message-ID: <94bdd2610911141813qfc1fa43tb33b19e256819050@mail.gmail.com>

2009/11/15 "Martin v. L?wis" <martin at v.loewis.de>:
>> The problem is that the main python distribution ("python") is not
>> working as advertised since
>> it contains distutils, which requires "python-devel" to work.
>>
>> This implies that "python" has a dependency on "python-devel", which
>> does not make sense
>> anymore for linux distros to have two distinct packages for Python.
>>
>> Having some of the makefile vars stored in stdlib solve this problem.
>
> I don't see how this can solve the problem. Distutils contains the
> build_ext command, which definitely will require the devel package.

Sure that's the edge case where files like Python.h is required for
compiling C extensions.
And compiler errors can make it pretty clear at this level, that the
devel package is required.
Not because of the need to read a Makefile, but to get the header
files to compile.

And it should be possible to compile extensions that don't require
python-devel at all,
when these extensions are not CPython extensions.

Last, commands like "install" also requires using sysconfig, (thus
reading Makefile) to get some values.
I can change this for "install" it's not hard,  but we would still
provide some public APIs that are reading Makefile from within the
stlib.

right now, importing "disutils.sysconfig" triggers this behavior.

That makes several use case of requiring python-devel just to read a
few values that could be injected
in the stdlib at build time.

Regards
Tarek

From martin at v.loewis.de  Sun Nov 15 03:31:04 2009
From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=)
Date: Sun, 15 Nov 2009 03:31:04 +0100
Subject: [Python-Dev] Too many Python accounts
In-Reply-To: <87tywwr0z5.fsf@benfinney.id.au>
References: <D7E23A6D-C91C-46D8-A8FE-1EB16F3B2101@lericson.se>	<hdi17l$cli$1@ger.gmane.org>
	<4AFC9064.20700@v.loewis.de>	<4222a8490911121511s40eba937n2c7b1fde6f695c6c@mail.gmail.com>	<4AFC9975.7040204@v.loewis.de>	<4222a8490911121534h30572eabv73ae58649c77155c@mail.gmail.com>	<4AFC9D0C.5030500@v.loewis.de>
	<87y6mbuubp.fsf@benfinney.id.au>	<20e5c9660911121702r162caf28jb6d656383147d909@mail.gmail.com>	<hdim34$utf$1@ger.gmane.org>	<79990c6b0911131133i2987a5d4s5c4ef780a527732@mail.gmail.com>	<4AFE6E10.3050307@v.loewis.de>	<20091114145222.084583A411A@sparrow.telecommunity.com>	<4AFECA7A.8070400@v.loewis.de>
	<hdn42a$r76$1@ger.gmane.org>	<87fx8gsq2s.fsf@benfinney.id.au>
	<4AFF5B36.9030502@v.loewis.de> <87tywwr0z5.fsf@benfinney.id.au>
Message-ID: <4AFF67E8.5060209@v.loewis.de>

> Fred can use his own OpenID ?fred.example.org?, initially set up behind
> the scenes to delegate to ?bigcorp.example.com? as the provider. Any
> time he likes, Fred can *change* which provider is actually used for
> authentication, without changing his OpenID. PyPI gets to find out which
> provider Fred is using for the identity ?fred.example.org? each time it
> performs discovery on that identity, not before.

Does that actually work? What actual OpenID provider allows me to claim
'fred.example.org' as my OpenID? Sure, one can use authentication
delegation, by means of the openid.delegate link. However, that still
doesn't make the claimed identifier fred.example.com, but
bigcorp.example.com/fred.

So the only thing users gain with delegation is that they don't need
to remember the tedious URL that their provider assigns them. When they
switch providers, their claimed ID will still change, and they'll have
to reregister in all services they use.

Please correct me if I'm wrong.

Regards,
Martin

From lists at cheimes.de  Sun Nov 15 03:35:00 2009
From: lists at cheimes.de (Christian Heimes)
Date: Sun, 15 Nov 2009 03:35:00 +0100
Subject: [Python-Dev] buildtime vs runtime in Distutils
In-Reply-To: <94bdd2610911141721n6700dcds5a489f1972176e48@mail.gmail.com>
References: <94bdd2610911141721n6700dcds5a489f1972176e48@mail.gmail.com>
Message-ID: <4AFF68D4.9070103@cheimes.de>

Tarek Ziad? wrote:
> Hello,
> 
> http://bugs.python.org/issue4359 reminds me that Distutils reads build
> files like Makefile or pyconfig.h to get some environment
> variables through the sysconfig module at *runtime*.
> 
> This cannot work on all platforms, when our Makefile is not shipped
> with python but python-devel. (like Fedora)

Do we really want to change distutils to solve a problem of a third
party packaging system when the problem was created by the very same
third party in the first place? In other words: Should you spend your
precious development time with fixing a problem that isn't our fault?
The decision to split the header files into a separate package, that
isn't installed by default, has already created tons of bad user
experience in the past. Do you want to endorse the split even further?

-0 from me

Christian

From tjreedy at udel.edu  Sun Nov 15 04:02:32 2009
From: tjreedy at udel.edu (Terry Reedy)
Date: Sat, 14 Nov 2009 22:02:32 -0500
Subject: [Python-Dev] buildtime vs runtime in Distutils
In-Reply-To: <4AFF68D4.9070103@cheimes.de>
References: <94bdd2610911141721n6700dcds5a489f1972176e48@mail.gmail.com>
	<4AFF68D4.9070103@cheimes.de>
Message-ID: <hdnr06$eqq$1@ger.gmane.org>

Christian Heimes wrote:
> Tarek Ziad? wrote:
>> Hello,
>>
>> http://bugs.python.org/issue4359 reminds me that Distutils reads build
>> files like Makefile or pyconfig.h to get some environment
>> variables through the sysconfig module at *runtime*.
>>
>> This cannot work on all platforms, when our Makefile is not shipped
>> with python but python-devel. (like Fedora)
> 
> Do we really want to change distutils to solve a problem of a third
> party packaging system when the problem was created by the very same
> third party in the first place? In other words: Should you spend your
> precious development time with fixing a problem that isn't our fault?
> The decision to split the header files into a separate package, that
> isn't installed by default, has already created tons of bad user
> experience in the past. Do you want to endorse the split even further?
> 
> -0 from me

If the third party disables distutils by removing some of the things it 
(sometimes) needs, then it seems to me that they should also remove 
disutils into the separate package. If *that* creates a problem, that 
should be *their* problem.



From ncoghlan at gmail.com  Sun Nov 15 04:06:48 2009
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sun, 15 Nov 2009 13:06:48 +1000
Subject: [Python-Dev] Using itertools in modules that are part of the build
 chain (Re: [Python-checkins] r76264 - python/branches/py3k/Lib/tokenize.py)
In-Reply-To: <4afeda77.0702d00a.4fa1.488cSMTPIN_ADDED@mx.google.com>
References: <4afeda77.0702d00a.4fa1.488cSMTPIN_ADDED@mx.google.com>
Message-ID: <4AFF7048.7070701@gmail.com>

benjamin.peterson wrote:
> Modified: python/branches/py3k/Lib/tokenize.py
> ==============================================================================
> --- python/branches/py3k/Lib/tokenize.py	(original)
> +++ python/branches/py3k/Lib/tokenize.py	Sat Nov 14 17:27:26 2009
> @@ -377,17 +377,12 @@
>      The first token sequence will always be an ENCODING token
>      which tells you which encoding was used to decode the bytes stream.
>      """
> +    # This import is here to avoid problems when the itertools module is not
> +    # built yet and tokenize is imported.
> +    from itertools import chain

This is probably a bad idea - calling tokenize.tokenize() from a thread
started as a side effect of importing a module will now deadlock on the
import lock if the module import waits for that thread to finish.

We tell people not to do that (starting and then waiting on threads as
part of module import) for exactly this reason, but it is also the
reason we avoid embedding import statements inside functions in the
standard library (not to mention encouraging third party developers to
also avoid embedding import statements inside functions).

This does constrain where we can use itertools - if we want carte
blanche to use it anywhere in the standard library, even those parts
that are imported as part of the build chain, we'll need to bite the
bullet and make it a builtin module rather than a separately built
extension module.

Cheers,
Nick.

P.S. The problem is easy to demonstrate on the current Py3k branch:

1. Put this in a module file in your py3k directory (e.g. "deadlock.py"):
-----------
import threading
import tokenize
f = open(__file__, 'rU')
def _deadlocks():
  tokenize.tokenize(f.readline)
t = threading.Thread(target=_deadlocks)
t.start()
t.join()
-----------

2. Then run: ./python -c "import deadlock"

It will, as advertised, deadlock and you'll need to use Ctrl-Brk or kill
-9 to get rid of it. (Note that preventing this kind of thing is one of
the major reasons why direct execution and even the -m switch *don't*
hang onto the import lock while running the __main__ module)

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------

From tjreedy at udel.edu  Sun Nov 15 04:09:54 2009
From: tjreedy at udel.edu (Terry Reedy)
Date: Sat, 14 Nov 2009 22:09:54 -0500
Subject: [Python-Dev] Too many Python accounts (was Re: PyPI comments
 and ratings, *really*?)
In-Reply-To: <4AFF5B1A.8090105@v.loewis.de>
References: <D7E23A6D-C91C-46D8-A8FE-1EB16F3B2101@lericson.se>	<hdi17l$cli$1@ger.gmane.org>	<4AFC9064.20700@v.loewis.de>	<4222a8490911121511s40eba937n2c7b1fde6f695c6c@mail.gmail.com>	<4AFC9975.7040204@v.loewis.de>	<4222a8490911121534h30572eabv73ae58649c77155c@mail.gmail.com>	<4AFC9D0C.5030500@v.loewis.de>	<87y6mbuubp.fsf@benfinney.id.au>	<20e5c9660911121702r162caf28jb6d656383147d909@mail.gmail.com>	<hdim34$utf$1@ger.gmane.org>	<79990c6b0911131133i2987a5d4s5c4ef780a527732@mail.gmail.com>	<4AFE6E10.3050307@v.loewis.de>	<20091114145222.084583A411A@sparrow.telecommunity.com>	<4AFECA7A.8070400@v.loewis.de>	<hdn42a$r76$1@ger.gmane.org>
	<4AFF5B1A.8090105@v.loewis.de>
Message-ID: <hdnrdv$fd9$1@ger.gmane.org>

Martin v. L?wis wrote:
>> Why? I already have python tracker account and a python wiki account
>> which is already 2 too many. The latter was a pain because the site lost
>> my registration and as of a year ago, the registration process was
>> broken. I would much prefer just one python site registration.
> 
> Then I recommend that you get a google account for your email address,
> and register to PyPI using OpenID.

OK, that worked after I hit the button a second time. Good enough.


From ben+python at benfinney.id.au  Sun Nov 15 04:53:57 2009
From: ben+python at benfinney.id.au (Ben Finney)
Date: Sun, 15 Nov 2009 14:53:57 +1100
Subject: [Python-Dev] Too many Python accounts
References: <D7E23A6D-C91C-46D8-A8FE-1EB16F3B2101@lericson.se>
	<hdi17l$cli$1@ger.gmane.org> <4AFC9064.20700@v.loewis.de>
	<4222a8490911121511s40eba937n2c7b1fde6f695c6c@mail.gmail.com>
	<4AFC9975.7040204@v.loewis.de>
	<4222a8490911121534h30572eabv73ae58649c77155c@mail.gmail.com>
	<4AFC9D0C.5030500@v.loewis.de> <87y6mbuubp.fsf@benfinney.id.au>
	<20e5c9660911121702r162caf28jb6d656383147d909@mail.gmail.com>
	<hdim34$utf$1@ger.gmane.org>
	<79990c6b0911131133i2987a5d4s5c4ef780a527732@mail.gmail.com>
	<4AFE6E10.3050307@v.loewis.de>
	<20091114145222.084583A411A@sparrow.telecommunity.com>
	<4AFECA7A.8070400@v.loewis.de> <hdn42a$r76$1@ger.gmane.org>
	<87fx8gsq2s.fsf@benfinney.id.au> <4AFF5B36.9030502@v.loewis.de>
	<87tywwr0z5.fsf@benfinney.id.au> <4AFF67E8.5060209@v.loewis.de>
Message-ID: <87my2oqw8a.fsf@benfinney.id.au>

"Martin v. L?wis" <martin at v.loewis.de> writes:

> > Fred can use his own OpenID ?fred.example.org?, initially set up
> > behind the scenes to delegate to ?bigcorp.example.com? as the
> > provider. Any time he likes, Fred can *change* which provider is
> > actually used for authentication, without changing his OpenID. PyPI
> > gets to find out which provider Fred is using for the identity
> > ?fred.example.org? each time it performs discovery on that identity,
> > not before.
>
> Does that actually work? What actual OpenID provider allows me to
> claim 'fred.example.org' as my OpenID?

It's inherent in the protocol, and any standards-conformant provider
supports it. If Fred controls ?fred.example.org? (i.e. can cause that
URL to serve a document of his choosing), then Fred gets to delegate to
any OpenID provider, and can change it at any time while keeping the
same identity URL.

If Albert, on the other hand, doesn't want to set up a URL under his
control but instead just use an identity managed by someone else, that's
a perfectly valid option: he can use ?bigcorp.example.com/albert? as his
OpenID, and BigCorp will take care of being the endpoint as well. Albert
is not getting any benefit from identity delegation (he's going to have
a transition difficulty at some later date when he decides BigCorp isn't
serving his identity needs any more), but neither is he required to know
about delegation or other protocol details.

> Sure, one can use authentication delegation, by means of the
> openid.delegate link. However, that still doesn't make the claimed
> identifier fred.example.com, but bigcorp.example.com/fred.

That's not right (this is what I mean by unnecessarily conflating ?who
provides the identity this time? with ?what is the identity?). The
?bigcorp.example.com/fred? URL is *not* Fred's identity in our example,
it is only the ?endpoint URL?: where Fred has configured his delegation
to go. Having control over ?fred.example.org?, Fred configures
delegation, and then can happily forget the endpoint URL in his daily
activities. When asked for his OpenID, Fred uses ?fred.example.org?.

Albert, on the other hand, has punted on the whole issue of delegation;
he uses ?bigcorp.example.com/albert? as his OpenID, because BigCorp
causes that URL to be both an OpenID and an endpoint. That's still fine
as far as the authentication process is concerned.

But the relying party (the party requesting an OpenID from the user; in
this case the relying party is PyPI) should not ask the user what their
provider is. The user is asked only for their OpenID: Fred will provide
?fred.example.org?, Albert will provide ?bigcorp.example.com/albert?.
PyPI discovers automatically, each time they use those identities, where
the provider is this time by using the OpenID discovery step on the
identity they provide.

Fred is very happy for this, since he can keep using ?fred.example.org?
even if he later changes which provider he's delegating to. Albert is
very happy for this, because he doesn't have to know anything about
?provider? or ?delegation? or ?endpoint?, he just needs to remember his
OpenID. PyPI's user interface becomes simpler, because it only needs to
ask the user ?what is your OpenID?? and the rest is taken care of by the
OpenID protocol implementation.

> Please correct me if I'm wrong.

I hope that helps. The documents at <URL:http://openid.net/developers/>
may be useful if you need more specifics.

-- 
 \          ?Computer perspective on Moore's Law: Human effort becomes |
  `\           twice as expensive roughly every two years.? ?anonymous |
_o__)                                                                  |
Ben Finney


From benjamin at python.org  Sun Nov 15 05:01:07 2009
From: benjamin at python.org (Benjamin Peterson)
Date: Sat, 14 Nov 2009 22:01:07 -0600
Subject: [Python-Dev] [Python-checkins] Using itertools in modules that
	are part of the build chain (Re: r76264 -
	python/branches/py3k/Lib/tokenize.py)
In-Reply-To: <4AFF7048.7070701@gmail.com>
References: <4afeda77.0702d00a.4fa1.488cSMTPIN_ADDED@mx.google.com>
	<4AFF7048.7070701@gmail.com>
Message-ID: <1afaf6160911142001m39d0f0ceyd3646e922d3bf703@mail.gmail.com>

2009/11/14 Nick Coghlan <ncoghlan at gmail.com>:
> This does constrain where we can use itertools - if we want carte
> blanche to use it anywhere in the standard library, even those parts
> that are imported as part of the build chain, we'll need to bite the
> bullet and make it a builtin module rather than a separately built
> extension module.

I have another unpleasant but slightly less hacky solution. We put
detect_encoding in linecache where it is actually used.


-- 
Regards,
Benjamin

From ncoghlan at gmail.com  Sun Nov 15 07:04:36 2009
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sun, 15 Nov 2009 16:04:36 +1000
Subject: [Python-Dev] Modules that run other scripts
Message-ID: <4AFF99F4.9000405@gmail.com>

I'm in the process of adding a "run_path" function to the runpy module
that allows Python code to execute scripts using the same rules as the
CPython command line (i.e. accepting both source and compiled Python
files in addition to zipfiles, directories and other valid sys.path
entries containing a __main__.py file).

After that I was considering looking at the standard library to see
which modules can be used to execute other scripts (e.g. pdb, profile)
and determine what would be involved in updating them to support the
same flexibility as the main command line (possibly including adding
their own "-m" option to run modules by name rather than file path
location).

For that second part:
1. Is it even worth doing at this stage? I'm not sure to what degree the
new command line flexibility has even been adopted by third party
application packagers, so I have no idea how large the pool of potential
users might be. Should I instead wait until we start seeing complaints
that these tools can't be used with script references that the main
command line will handle quite happily?

2. Aside from runpy itself, pdb and profile are the only examples that
spring to mind when I try to think of standard library modules that
execute other Python scripts. Are there any others that I'm missing?
(Even if we decide not to do anything at this stage, collating such a
list may still be handy for future reference)

Regards,
Nick.

P.S. pdb in particular may be messy to update, since the interaction
between the way it provides exception post-mortem debugging support and
the way the runpy module tries to avoid mutating the application's own
__main__ module will likely require careful handling.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------

From robert.kern at gmail.com  Sun Nov 15 07:37:00 2009
From: robert.kern at gmail.com (Robert Kern)
Date: Sun, 15 Nov 2009 00:37:00 -0600
Subject: [Python-Dev] Modules that run other scripts
In-Reply-To: <4AFF99F4.9000405@gmail.com>
References: <4AFF99F4.9000405@gmail.com>
Message-ID: <hdo7id$3hm$1@ger.gmane.org>

Nick Coghlan wrote:

> For that second part:
> 1. Is it even worth doing at this stage? I'm not sure to what degree the
> new command line flexibility has even been adopted by third party
> application packagers, so I have no idea how large the pool of potential
> users might be. Should I instead wait until we start seeing complaints
> that these tools can't be used with script references that the main
> command line will handle quite happily?

There is a small, but important class of "scripts that run scripts", which are 
mostly all development tools (e.g. coverage, my line_profiler, etc.). Doing this 
correctly in all of the corner cases is reasonably tricky, so I think this is a 
perfect case for having the functionality implemented once in the standard library.

For what its worth, I think Ned Batchelder did the most thorough job of 
implementing this in the latest version of coverage:

   http://bitbucket.org/ned/coveragepy/src/tip/coverage/execfile.py

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
  that is made terrible by our own mad attempt to interpret it as though it had
  an underlying truth."
   -- Umberto Eco


From ncoghlan at gmail.com  Sun Nov 15 08:02:47 2009
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sun, 15 Nov 2009 17:02:47 +1000
Subject: [Python-Dev] Modules that run other scripts
In-Reply-To: <hdo7id$3hm$1@ger.gmane.org>
References: <4AFF99F4.9000405@gmail.com> <hdo7id$3hm$1@ger.gmane.org>
Message-ID: <4AFFA797.4040907@gmail.com>

Robert Kern wrote:
> Nick Coghlan wrote:
> 
>> For that second part:
>> 1. Is it even worth doing at this stage? I'm not sure to what degree the
>> new command line flexibility has even been adopted by third party
>> application packagers, so I have no idea how large the pool of potential
>> users might be. Should I instead wait until we start seeing complaints
>> that these tools can't be used with script references that the main
>> command line will handle quite happily?
> 
> There is a small, but important class of "scripts that run scripts",
> which are mostly all development tools (e.g. coverage, my line_profiler,
> etc.). Doing this correctly in all of the corner cases is reasonably
> tricky, so I think this is a perfect case for having the functionality
> implemented once in the standard library.

Yep, that part I'm convinced of the need for - hence runpy.run_path.

It's whether the immediate demand is there to justify tinkering with the
existing tools in the standard library that I'm not sure about
(particularly when there is a risk of altering behaviour in some corner
cases).

> For what its worth, I think Ned Batchelder did the most thorough job of
> implementing this in the latest version of coverage:
> 
>   http://bitbucket.org/ned/coveragepy/src/tip/coverage/execfile.py

That's actually fairly similar to what run_path will do in the simple
script case (zipfile and directory execution is a fair bit messier in
practice, but based on a similar concept in principle).

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------

From fuzzyman at voidspace.org.uk  Sun Nov 15 12:41:23 2009
From: fuzzyman at voidspace.org.uk (Michael Foord)
Date: Sun, 15 Nov 2009 11:41:23 +0000
Subject: [Python-Dev] Too many Python accounts
In-Reply-To: <4AFF67E8.5060209@v.loewis.de>
References: <D7E23A6D-C91C-46D8-A8FE-1EB16F3B2101@lericson.se>	<hdi17l$cli$1@ger.gmane.org>	<4AFC9064.20700@v.loewis.de>	<4222a8490911121511s40eba937n2c7b1fde6f695c6c@mail.gmail.com>	<4AFC9975.7040204@v.loewis.de>	<4222a8490911121534h30572eabv73ae58649c77155c@mail.gmail.com>	<4AFC9D0C.5030500@v.loewis.de>	<87y6mbuubp.fsf@benfinney.id.au>	<20e5c9660911121702r162caf28jb6d656383147d909@mail.gmail.com>	<hdim34$utf$1@ger.gmane.org>	<79990c6b0911131133i2987a5d4s5c4ef780a527732@mail.gmail.com>	<4AFE6E10.3050307@v.loewis.de>	<20091114145222.084583A411A@sparrow.telecommunity.com>	<4AFECA7A.8070400@v.loewis.de>	<hdn42a$r76$1@ger.gmane.org>	<87fx8gsq2s.fsf@benfinney.id.au>	<4AFF5B36.9030502@v.loewis.de>
	<87tywwr0z5.fsf@benfinney.id.au> <4AFF67E8.5060209@v.loewis.de>
Message-ID: <4AFFE8E3.5020508@voidspace.org.uk>

Martin v. L?wis wrote:
>> Fred can use his own OpenID ?fred.example.org?, initially set up behind
>> the scenes to delegate to ?bigcorp.example.com? as the provider. Any
>> time he likes, Fred can *change* which provider is actually used for
>> authentication, without changing his OpenID. PyPI gets to find out which
>> provider Fred is using for the identity ?fred.example.org? each time it
>> performs discovery on that identity, not before.
>>     
>
> Does that actually work? What actual OpenID provider allows me to claim
> 'fred.example.org' as my OpenID? Sure, one can use authentication
> delegation, by means of the openid.delegate link. However, that still
> doesn't make the claimed identifier fred.example.com, but
> bigcorp.example.com/fred.
>
> So the only thing users gain with delegation is that they don't need
> to remember the tedious URL that their provider assigns them. When they
> switch providers, their claimed ID will still change, and they'll have
> to reregister in all services they use.
>
>   
No, the whole point of delegation is that I can use voidspace.org.uk as 
my openid - backed with any provider I want. I can then use 
voidspace.org.uk as my openid and not be tied to any provider.

I'm afraid the PyPI implementation of openid is useless to me too - I 
want to use voidspace.org.uk as my openid but it doesn't let me.

All the best,

Michael

> Please correct me if I'm wrong.
>
> Regards,
> Martin
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk
>   


-- 
http://www.ironpythoninaction.com/
http://www.voidspace.org.uk/blog



From solipsis at pitrou.net  Sun Nov 15 13:42:53 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Sun, 15 Nov 2009 12:42:53 +0000 (UTC)
Subject: [Python-Dev] buildtime vs runtime in Distutils
References: <94bdd2610911141721n6700dcds5a489f1972176e48@mail.gmail.com>
Message-ID: <loom.20091115T133752-257@post.gmane.org>

Tarek Ziad? <ziade.tarek <at> gmail.com> writes:
> 
> This cannot work on all platforms, when our Makefile is not shipped
> with python but python-devel. (like Fedora)

This practice is stupid anyway, because it means you have to install
python-devel even to install pure Python packages with setuptools/distribute.
Just ask Fedora, Mandriva and friends to change their packaging practice
(Mandriva already has a bug open for that by the way).

In Debian/Ubuntu, the Makefile is correctly part of the main Python package:

$ dpkg -S /usr/lib/python2.5/config/Makefile
python2.5: /usr/lib/python2.5/config/Makefile


Regards

Antoine.



From ziade.tarek at gmail.com  Sun Nov 15 14:30:34 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Sun, 15 Nov 2009 14:30:34 +0100
Subject: [Python-Dev] buildtime vs runtime in Distutils
In-Reply-To: <4AFF68D4.9070103@cheimes.de>
References: <94bdd2610911141721n6700dcds5a489f1972176e48@mail.gmail.com>
	<4AFF68D4.9070103@cheimes.de>
Message-ID: <94bdd2610911150530y3191a192mb00d910e30e01c2e@mail.gmail.com>

On Sun, Nov 15, 2009 at 3:35 AM, Christian Heimes <lists at cheimes.de> wrote:
[..]
> Do we really want to change distutils to solve a problem of a third
> party packaging system when the problem was created by the very same
> third party in the first place? In other words: Should you spend your
> precious development time with fixing a problem that isn't our fault?
> The decision to split the header files into a separate package, that
> isn't installed by default, has already created tons of bad user
> experience in the past. Do you want to endorse the split even further?

I didn't know the split story went like this. I took it like the
"natural" split everyone
agreed on, and I saw this distutils <-> Makefile link like something to fix.

So, it sounds like a bad idea now :)

Tarek.

From g.brandl at gmx.net  Sun Nov 15 14:31:45 2009
From: g.brandl at gmx.net (Georg Brandl)
Date: Sun, 15 Nov 2009 14:31:45 +0100
Subject: [Python-Dev] buildtime vs runtime in Distutils
In-Reply-To: <loom.20091115T133752-257@post.gmane.org>
References: <94bdd2610911141721n6700dcds5a489f1972176e48@mail.gmail.com>
	<loom.20091115T133752-257@post.gmane.org>
Message-ID: <hdovs4$n83$1@ger.gmane.org>

Antoine Pitrou schrieb:
> Tarek Ziad? <ziade.tarek <at> gmail.com> writes:
>> 
>> This cannot work on all platforms, when our Makefile is not shipped
>> with python but python-devel. (like Fedora)
> 
> This practice is stupid anyway, because it means you have to install
> python-devel even to install pure Python packages with setuptools/distribute.
> Just ask Fedora, Mandriva and friends to change their packaging practice
> (Mandriva already has a bug open for that by the way).

+1.  They are the ones splitting what "make install" installs into several
packages, so they are the ones who have to fix the resulting dependency
problems.

Georg

-- 
Thus spake the Lord: Thou shalt indent with four spaces. No more, no less.
Four shall be the number of spaces thou shalt indent, and the number of thy
indenting shall be four. Eight shalt thou not indent, nor either indent thou
two, excepting that thou then proceed to four. Tabs are right out.


From ziade.tarek at gmail.com  Sun Nov 15 14:32:19 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Sun, 15 Nov 2009 14:32:19 +0100
Subject: [Python-Dev] buildtime vs runtime in Distutils
In-Reply-To: <hdnr06$eqq$1@ger.gmane.org>
References: <94bdd2610911141721n6700dcds5a489f1972176e48@mail.gmail.com>
	<4AFF68D4.9070103@cheimes.de> <hdnr06$eqq$1@ger.gmane.org>
Message-ID: <94bdd2610911150532w2a91d0c3u61106231025bd6cd@mail.gmail.com>

On Sun, Nov 15, 2009 at 4:02 AM, Terry Reedy <tjreedy at udel.edu> wrote:
> Christian Heimes wrote:
>>
>> Tarek Ziad? wrote:
>>>
>>> Hello,
>>>
>>> http://bugs.python.org/issue4359 reminds me that Distutils reads build
>>> files like Makefile or pyconfig.h to get some environment
>>> variables through the sysconfig module at *runtime*.
>>>
>>> This cannot work on all platforms, when our Makefile is not shipped
>>> with python but python-devel. (like Fedora)
>>
>> Do we really want to change distutils to solve a problem of a third
>> party packaging system when the problem was created by the very same
>> third party in the first place? In other words: Should you spend your
>> precious development time with fixing a problem that isn't our fault?
>> The decision to split the header files into a separate package, that
>> isn't installed by default, has already created tons of bad user
>> experience in the past. Do you want to endorse the split even further?
>>
>> -0 from me
>
> If the third party disables distutils by removing some of the things it
> (sometimes) needs, then it seems to me that they should also remove disutils
> into the separate package. If *that* creates a problem, that should be
> *their* problem.

Ok. Fair enough, I'll work with them this way.

Regards
Tarek

From g.brandl at gmx.net  Sun Nov 15 14:33:34 2009
From: g.brandl at gmx.net (Georg Brandl)
Date: Sun, 15 Nov 2009 14:33:34 +0100
Subject: [Python-Dev] Status of the Buildbot fleet and related bugs
In-Reply-To: <4AFE73D8.9020509@v.loewis.de>
References: <Pine.LNX.4.64.0911052209000.12274@kimball.webabinitio.net>	<hdcn72$2rl$1@ger.gmane.org>	<4AF9E3D6.2000807@v.loewis.de>	<loom.20091114T010914-556@post.gmane.org>	<Pine.LNX.4.64.0911132317040.12274@kimball.webabinitio.net>	<m2pr7l7m8j.fsf@valheru.db3l.homeip.net>
	<4AFE73D8.9020509@v.loewis.de>
Message-ID: <hdovvh$n83$2@ger.gmane.org>

Martin v. L?wis schrieb:
>> Yes, I think this just started happening. I'm guessing that the main
>> site proxies the buildbot URL requests to the buildbot master process,
>> and when it's down you get the 404s from the main server.
>> 
>> I figured someone might be working on the master, though perhaps it
>> just burped on its own :-)
> 
> It was actually an Apache misconfiguration (the wrong virtual host would
> pick up requests, missing the reverse proxy configuration). I have fixed
> that now.

BTW, I noticed that the "cancel build" and "cancel all builds" buttons result
in unhandled exceptions.  (One build is cancelled nevertheless, so for
"cancel build" it's just an inconvenience.)

Georg

-- 
Thus spake the Lord: Thou shalt indent with four spaces. No more, no less.
Four shall be the number of spaces thou shalt indent, and the number of thy
indenting shall be four. Eight shalt thou not indent, nor either indent thou
two, excepting that thou then proceed to four. Tabs are right out.


From cournape at gmail.com  Sun Nov 15 14:38:37 2009
From: cournape at gmail.com (David Cournapeau)
Date: Sun, 15 Nov 2009 22:38:37 +0900
Subject: [Python-Dev] buildtime vs runtime in Distutils
In-Reply-To: <94bdd2610911150532w2a91d0c3u61106231025bd6cd@mail.gmail.com>
References: <94bdd2610911141721n6700dcds5a489f1972176e48@mail.gmail.com>
	<4AFF68D4.9070103@cheimes.de> <hdnr06$eqq$1@ger.gmane.org>
	<94bdd2610911150532w2a91d0c3u61106231025bd6cd@mail.gmail.com>
Message-ID: <5b8d13220911150538r380c24d1ic107c69b1e68526c@mail.gmail.com>

On Sun, Nov 15, 2009 at 10:32 PM, Tarek Ziad? <ziade.tarek at gmail.com> wrote:

>
> Ok. Fair enough, I'll work with them this way.

Although packagers should certainly fix the problems they introduce in
the first place, the second suggestion in the bug report would be
useful, independently on how linux distributions package things.

Especially if the data can be obtained for every build (autoconf and
VS-based), this would help packages which use something else than
distutils for their build.

David

From martin at v.loewis.de  Sun Nov 15 19:57:04 2009
From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=)
Date: Sun, 15 Nov 2009 19:57:04 +0100
Subject: [Python-Dev] Too many Python accounts
In-Reply-To: <4AFFE8E3.5020508@voidspace.org.uk>
References: <D7E23A6D-C91C-46D8-A8FE-1EB16F3B2101@lericson.se>	<hdi17l$cli$1@ger.gmane.org>	<4AFC9064.20700@v.loewis.de>	<4222a8490911121511s40eba937n2c7b1fde6f695c6c@mail.gmail.com>	<4AFC9975.7040204@v.loewis.de>	<4222a8490911121534h30572eabv73ae58649c77155c@mail.gmail.com>	<4AFC9D0C.5030500@v.loewis.de>	<87y6mbuubp.fsf@benfinney.id.au>	<20e5c9660911121702r162caf28jb6d656383147d909@mail.gmail.com>	<hdim34$utf$1@ger.gmane.org>	<79990c6b0911131133i2987a5d4s5c4ef780a527732@mail.gmail.com>	<4AFE6E10.3050307@v.loewis.de>	<20091114145222.084583A411A@sparrow.telecommunity.com>	<4AFECA7A.8070400@v.loewis.de>	<hdn42a$r76$1@ger.gmane.org>	<87fx8gsq2s.fsf@benfinney.id.au>	<4AFF5B36.9030502@v.loewis.de>
	<87tywwr0z5.fsf@benfinney.id.au> <4AFF67E8.5060209@v.loewis.de>
	<4AFFE8E3.5020508@voidspace.org.uk>
Message-ID: <4B004F00.7080405@v.loewis.de>

>> So the only thing users gain with delegation is that they don't need
>> to remember the tedious URL that their provider assigns them. When they
>> switch providers, their claimed ID will still change, and they'll have
>> to reregister in all services they use.
>>
>>   
> No, the whole point of delegation is that I can use voidspace.org.uk as
> my openid - backed with any provider I want. I can then use
> voidspace.org.uk as my openid and not be tied to any provider.

I'm fairly skeptical that you actually do use voidspace.org.uk.
I can believe that this is what you type into the openid field -
but the ID that is validated to the relying party is
http://fuzzyman.myopenid.com/.

So when you change the provider (and hence the openid.delegate
value), you will have to create new accounts for all services
that you use, right?

> I'm afraid the PyPI implementation of openid is useless to me too - I
> want to use voidspace.org.uk as my openid but it doesn't let me.

See above - the services may let you *enter* that string, but it
isn't your openid.

Regards,
Martin

From fuzzyman at voidspace.org.uk  Sun Nov 15 20:18:43 2009
From: fuzzyman at voidspace.org.uk (Michael Foord)
Date: Sun, 15 Nov 2009 19:18:43 +0000
Subject: [Python-Dev] Too many Python accounts
In-Reply-To: <4B004F00.7080405@v.loewis.de>
References: <D7E23A6D-C91C-46D8-A8FE-1EB16F3B2101@lericson.se>	<hdi17l$cli$1@ger.gmane.org>	<4AFC9064.20700@v.loewis.de>	<4222a8490911121511s40eba937n2c7b1fde6f695c6c@mail.gmail.com>	<4AFC9975.7040204@v.loewis.de>	<4222a8490911121534h30572eabv73ae58649c77155c@mail.gmail.com>	<4AFC9D0C.5030500@v.loewis.de>	<87y6mbuubp.fsf@benfinney.id.au>	<20e5c9660911121702r162caf28jb6d656383147d909@mail.gmail.com>	<hdim34$utf$1@ger.gmane.org>	<79990c6b0911131133i2987a5d4s5c4ef780a527732@mail.gmail.com>	<4AFE6E10.3050307@v.loewis.de>	<20091114145222.084583A411A@sparrow.telecommunity.com>	<4AFECA7A.8070400@v.loewis.de>	<hdn42a$r76$1@ger.gmane.org>	<87fx8gsq2s.fsf@benfinney.id.au>	<4AFF5B36.9030502@v.loewis.de>
	<87tywwr0z5.fsf@benfinney.id.au> <4AFF67E8.5060209@v.loewis.de>
	<4AFFE8E3.5020508@voidspace.org.uk> <4B004F00.7080405@v.loewis.de>
Message-ID: <4B005413.3010702@voidspace.org.uk>

Martin v. L?wis wrote:
>>> So the only thing users gain with delegation is that they don't need
>>> to remember the tedious URL that their provider assigns them. When they
>>> switch providers, their claimed ID will still change, and they'll have
>>> to reregister in all services they use.
>>>
>>>   
>>>       
>> No, the whole point of delegation is that I can use voidspace.org.uk as
>> my openid - backed with any provider I want. I can then use
>> voidspace.org.uk as my openid and not be tied to any provider.
>>     
>
> I'm fairly skeptical that you actually do use voidspace.org.uk.
> I can believe that this is what you type into the openid field -
> but the ID that is validated to the relying party is
> http://fuzzyman.myopenid.com/.
>
> So when you change the provider (and hence the openid.delegate
> value), you will have to create new accounts for all services
> that you use, right?
>   

Well, when I login my registered ID is www.voidspace.org.uk and *not* 
fuzzyman.myopenid.com - so I believe you are incorrect (and in fact this 
very point was touted as one of the advantages of openid - that your 
account is independent of your provider and that you *can* change 
provider whilst retaining the same id).

myopenid *does* the validation, but my registered openid is 
www.voidspace.org.uk and I *should* be able to change provider without 
creating a new account.

Michael

>   
>> I'm afraid the PyPI implementation of openid is useless to me too - I
>> want to use voidspace.org.uk as my openid but it doesn't let me.
>>     
>
> See above - the services may let you *enter* that string, but it
> isn't your openid.
>
> Regards,
> Martin
>   


-- 
http://www.ironpythoninaction.com/
http://www.voidspace.org.uk/blog



From martin at v.loewis.de  Sun Nov 15 20:31:51 2009
From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=)
Date: Sun, 15 Nov 2009 20:31:51 +0100
Subject: [Python-Dev] Too many Python accounts
In-Reply-To: <4B005413.3010702@voidspace.org.uk>
References: <D7E23A6D-C91C-46D8-A8FE-1EB16F3B2101@lericson.se>	<hdi17l$cli$1@ger.gmane.org>	<4AFC9064.20700@v.loewis.de>	<4222a8490911121511s40eba937n2c7b1fde6f695c6c@mail.gmail.com>	<4AFC9975.7040204@v.loewis.de>	<4222a8490911121534h30572eabv73ae58649c77155c@mail.gmail.com>	<4AFC9D0C.5030500@v.loewis.de>	<87y6mbuubp.fsf@benfinney.id.au>	<20e5c9660911121702r162caf28jb6d656383147d909@mail.gmail.com>	<hdim34$utf$1@ger.gmane.org>	<79990c6b0911131133i2987a5d4s5c4ef780a527732@mail.gmail.com>	<4AFE6E10.3050307@v.loewis.de>	<20091114145222.084583A411A@sparrow.telecommunity.com>	<4AFECA7A.8070400@v.loewis.de>	<hdn42a$r76$1@ger.gmane.org>	<87fx8gsq2s.fsf@benfinney.id.au>	<4AFF5B36.9030502@v.loewis.de>
	<87tywwr0z5.fsf@benfinney.id.au> <4AFF67E8.5060209@v.loewis.de>
	<4AFFE8E3.5020508@voidspace.org.uk> <4B004F00.7080405@v.loewis.de>
	<4B005413.3010702@voidspace.org.uk>
Message-ID: <4B005727.4050004@v.loewis.de>

> Well, when I login my registered ID is www.voidspace.org.uk and *not*
> fuzzyman.myopenid.com - so I believe you are incorrect (and in fact this
> very point was touted as one of the advantages of openid - that your
> account is independent of your provider and that you *can* change
> provider whilst retaining the same id).

On the wire (between relying party and provider), voidspace.org.co.uk
does never appear. From the OpenID 1.1 specification:

# Now, when a Consumer sees that, it'll talk to
# http://www.livejournal.com/openid/server.bml and ask if the End User
# is exampleuser.livejournal.com, never mentioning www.example.com
# anywhere on the wire.

So all I (as a relying party) get verifyied is fuzzyman.myopenid.com.
Why should I trust that voidspace.org.uk is actually a valid ID?
Can't you then produce hundreds of IDs, all delegating to the same
identity?

IOW, why should I (as a relying party) pay any attention to the ID
that you entered, rather than to what I get actually validated?

Regards,
Martin

From ezio.melotti at gmail.com  Sun Nov 15 20:55:00 2009
From: ezio.melotti at gmail.com (Ezio Melotti)
Date: Sun, 15 Nov 2009 21:55:00 +0200
Subject: [Python-Dev] global statements outside functions/methods should
	raise SyntaxError
Message-ID: <4B005C94.8060707@gmail.com>

Python currently accepts global statements at the top level:
>>> global foo
>>>

Beside being a meaningless operation, this might lead unexperienced
user to make mistakes like:
>>> foo = 5
>>> global foo # make foo global
>>> def func():
...   print foo # access the global foo
...
>>> func()
5
>>> # it works!

"global foo" should raise a SyntaxError, similarly to what already
happens with "return":
>>> return foo
  File "<stdin>", line 1
SyntaxError: 'return' outside function


I opened an issue on the tracker (http://bugs.python.org/issue7329)
and Benjamin suggested to discuss this here.
The test he mentioned is in test_global.py:

    def test4(self):
        prog_text_4 = """\
global x
x = 2
"""
        # this should work
        compile(prog_text_4, "<test string>", "exec")

It just says that "it should work" but it doesn't say /why/.

Any thoughts?


From fuzzyman at voidspace.org.uk  Sun Nov 15 20:59:43 2009
From: fuzzyman at voidspace.org.uk (Michael Foord)
Date: Sun, 15 Nov 2009 19:59:43 +0000
Subject: [Python-Dev] global statements outside functions/methods should
 raise SyntaxError
In-Reply-To: <4B005C94.8060707@gmail.com>
References: <4B005C94.8060707@gmail.com>
Message-ID: <4B005DAF.8050904@voidspace.org.uk>

Ezio Melotti wrote:
> Python currently accepts global statements at the top level:
>>>> global foo
>>>>
>
> Beside being a meaningless operation, this might lead unexperienced
> user to make mistakes like:
>>>> foo = 5
>>>> global foo # make foo global
>>>> def func():
> ...   print foo # access the global foo
> ...
>>>> func()
> 5
>>>> # it works!
>
> "global foo" should raise a SyntaxError, similarly to what already
> happens with "return":
>>>> return foo
>  File "<stdin>", line 1
> SyntaxError: 'return' outside function
>
>
> I opened an issue on the tracker (http://bugs.python.org/issue7329)
> and Benjamin suggested to discuss this here.
> The test he mentioned is in test_global.py:
>
>    def test4(self):
>        prog_text_4 = """\
> global x
> x = 2
> """
>        # this should work
>        compile(prog_text_4, "<test string>", "exec")
>
> It just says that "it should work" but it doesn't say /why/.

Well, personally I think it would be a good thing if this raised an 
exception during bytecode compilation - but it would fall under the 
moratorium and have to wait a few years.

On the other hand it should be easy to get PyLint to include a checker 
for this and they are very responsive to feature requests.

All the best,

Michael

>
> Any thoughts?
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: 
> http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk 
>


-- 
http://www.ironpythoninaction.com/
http://www.voidspace.org.uk/blog



From benjamin at python.org  Sun Nov 15 21:08:49 2009
From: benjamin at python.org (Benjamin Peterson)
Date: Sun, 15 Nov 2009 14:08:49 -0600
Subject: [Python-Dev] global statements outside functions/methods should
	raise SyntaxError
In-Reply-To: <4B005DAF.8050904@voidspace.org.uk>
References: <4B005C94.8060707@gmail.com> <4B005DAF.8050904@voidspace.org.uk>
Message-ID: <1afaf6160911151208h6ff65337vca988b0a1e2f0707@mail.gmail.com>

2009/11/15 Michael Foord <fuzzyman at voidspace.org.uk>:
> Well, personally I think it would be a good thing if this raised an
> exception during bytecode compilation - but it would fall under the
> moratorium and have to wait a few years.

It could probably be considered a bug, though, since the global
statement in that case silently has absolutely no effect.


-- 
Regards,
Benjamin

From carey.tilden at gmail.com  Sun Nov 15 21:20:05 2009
From: carey.tilden at gmail.com (Carey Tilden)
Date: Sun, 15 Nov 2009 12:20:05 -0800
Subject: [Python-Dev] Too many Python accounts
In-Reply-To: <4B005727.4050004@v.loewis.de>
References: <D7E23A6D-C91C-46D8-A8FE-1EB16F3B2101@lericson.se>
	<hdn42a$r76$1@ger.gmane.org> <87fx8gsq2s.fsf@benfinney.id.au>
	<4AFF5B36.9030502@v.loewis.de> <87tywwr0z5.fsf@benfinney.id.au>
	<4AFF67E8.5060209@v.loewis.de> <4AFFE8E3.5020508@voidspace.org.uk>
	<4B004F00.7080405@v.loewis.de> <4B005413.3010702@voidspace.org.uk>
	<4B005727.4050004@v.loewis.de>
Message-ID: <bfd8702d0911151220u2068fa04p61d2b3d615d219a9@mail.gmail.com>

On Sun, Nov 15, 2009 at 11:31 AM, "Martin v. L?wis" <martin at v.loewis.de> wrote:
>
> > Well, when I login my registered ID is www.voidspace.org.uk and *not*
> > fuzzyman.myopenid.com - so I believe you are incorrect (and in fact this
> > very point was touted as one of the advantages of openid - that your
> > account is independent of your provider and that you *can* change
> > provider whilst retaining the same id).
>
> On the wire (between relying party and provider), voidspace.org.co.uk
> does never appear. From the OpenID 1.1 specification:
>
> # Now, when a Consumer sees that, it'll talk to
> # http://www.livejournal.com/openid/server.bml and ask if the End User
> # is exampleuser.livejournal.com, never mentioning www.example.com
> # anywhere on the wire.
>
> So all I (as a relying party) get verifyied is fuzzyman.myopenid.com.
> Why should I trust that voidspace.org.uk is actually a valid ID?

Since the user entered voidspace.org.uk, they presumably believe it's an
address they control.  You have to assume they delegated to another
provider on purpose.

> Can't you then produce hundreds of IDs, all delegating to the same
> identity?

Yes.

> IOW, why should I (as a relying party) pay any attention to the ID
> that you entered, rather than to what I get actually validated?

Because the user entered the value they wanted as their identity.  This is
the reason delegation even exists in the spec.  In fact, the very next line
after the section you quoted is:

# The main advantage of this is that an End User can keep their Identifier
# over many years, even as services come and go; they'll just keep
# changing who they delegate to.

If the provider dictates the identity, as you keep insisting, that sentence
makes no sense whatsoever.  The value entered as the identifier is the
identifier you should use.  Otherwise, what's the point of delegation at all?

> Regards,
> Martin
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/carey.tilden%40gmail.com

From solipsis at pitrou.net  Sun Nov 15 21:23:43 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Sun, 15 Nov 2009 20:23:43 +0000 (UTC)
Subject: [Python-Dev]
	=?utf-8?q?global_statements_outside_functions/method?=
	=?utf-8?q?s_should=09raise_SyntaxError?=
References: <4B005C94.8060707@gmail.com> <4B005DAF.8050904@voidspace.org.uk>
	<1afaf6160911151208h6ff65337vca988b0a1e2f0707@mail.gmail.com>
Message-ID: <loom.20091115T212239-344@post.gmane.org>

Benjamin Peterson <benjamin <at> python.org> writes:
> 
> 2009/11/15 Michael Foord <fuzzyman <at> voidspace.org.uk>:
> > Well, personally I think it would be a good thing if this raised an
> > exception during bytecode compilation - but it would fall under the
> > moratorium and have to wait a few years.
> 
> It could probably be considered a bug, though, since the global
> statement in that case silently has absolutely no effect.

Indeed, and it's not like other implementations would be at a disadvantage if
they didn't implement this error.

Warning that the construct is meaningless can be helpful, especially for
refugees from other languages.

Regards

Antoine.



From skip at pobox.com  Sun Nov 15 21:27:28 2009
From: skip at pobox.com (skip at pobox.com)
Date: Sun, 15 Nov 2009 14:27:28 -0600
Subject: [Python-Dev] Too many Python accounts (was Re: PyPI comments
 and ratings, *really*?)
In-Reply-To: <4AFF5B1A.8090105@v.loewis.de>
References: <D7E23A6D-C91C-46D8-A8FE-1EB16F3B2101@lericson.se>
	<hdi17l$cli$1@ger.gmane.org> <4AFC9064.20700@v.loewis.de>
	<4222a8490911121511s40eba937n2c7b1fde6f695c6c@mail.gmail.com>
	<4AFC9975.7040204@v.loewis.de>
	<4222a8490911121534h30572eabv73ae58649c77155c@mail.gmail.com>
	<4AFC9D0C.5030500@v.loewis.de> <87y6mbuubp.fsf@benfinney.id.au>
	<20e5c9660911121702r162caf28jb6d656383147d909@mail.gmail.com>
	<hdim34$utf$1@ger.gmane.org>
	<79990c6b0911131133i2987a5d4s5c4ef780a527732@mail.gmail.com>
	<4AFE6E10.3050307@v.loewis.de>
	<20091114145222.084583A411A@sparrow.telecommunity.com>
	<4AFECA7A.8070400@v.loewis.de> <hdn42a$r76$1@ger.gmane.org>
	<4AFF5B1A.8090105@v.loewis.de>
Message-ID: <19200.25648.998776.238480@montanaro.dyndns.org>


    Martin> Then I recommend that you get a google account for your email
    Martin> address, and register to PyPI using OpenID.

I've never found OpenID at all intuitive to use.  Are there instructions on
pypi.python.org which detail the steps necessary to use OpenID to login?  I
saw the "Claim OpenID" link on my PyPI profile page.  So now I have an
OpenID URL.  What am I supposed to do with that?  If I visit that URL it
downloads a small bit of XML.

I've tried using my Yahoo! and Luanchpad OpenIDs for other sites in the
past.  I've never successfully logged into any website with them, at least
not as far as I can recall.  I realize that maybe this is something that
just doesn't click with me (maybe I'm an OpenID Luddite), but it seems to me
that OpenID needs to be a bit easier (or obvious?) to use if it's to become
some sort of de facto standard login mechanism.

Skip

From python at mrabarnett.plus.com  Sun Nov 15 21:31:39 2009
From: python at mrabarnett.plus.com (MRAB)
Date: Sun, 15 Nov 2009 20:31:39 +0000
Subject: [Python-Dev] global statements outside functions/methods should
 raise SyntaxError
In-Reply-To: <1afaf6160911151208h6ff65337vca988b0a1e2f0707@mail.gmail.com>
References: <4B005C94.8060707@gmail.com> <4B005DAF.8050904@voidspace.org.uk>
	<1afaf6160911151208h6ff65337vca988b0a1e2f0707@mail.gmail.com>
Message-ID: <4B00652B.6010008@mrabarnett.plus.com>

Benjamin Peterson wrote:
> 2009/11/15 Michael Foord <fuzzyman at voidspace.org.uk>:
>> Well, personally I think it would be a good thing if this raised an
>> exception during bytecode compilation - but it would fall under the
>> moratorium and have to wait a few years.
> 
> It could probably be considered a bug, though, since the global
> statement in that case silently has absolutely no effect.
> 
It wouldn't be adding a new feature, but it would be changing the
behaviour of programs that currently have 'global' outside a function.

Perhaps, due to the moratorium, there should be a warning before it's
actually corrected.

From tjreedy at udel.edu  Sun Nov 15 21:38:26 2009
From: tjreedy at udel.edu (Terry Reedy)
Date: Sun, 15 Nov 2009 15:38:26 -0500
Subject: [Python-Dev] global statements outside functions/methods should
	raise SyntaxError
In-Reply-To: <4B005C94.8060707@gmail.com>
References: <4B005C94.8060707@gmail.com>
Message-ID: <hdporv$476$1@ger.gmane.org>

Ezio Melotti wrote:
> Python currently accepts global statements at the top level:
>
> I opened an issue on the tracker (http://bugs.python.org/issue7329)
> and Benjamin suggested to discuss this here.
> The test he mentioned is in test_global.py:
> 
>    def test4(self):
>        prog_text_4 = """\
> global x
> x = 2
> """
>        # this should work
>        compile(prog_text_4, "<test string>", "exec")
> 
> It just says that "it should work" but it doesn't say /why/.
> 
> Any thoughts?

I make the same suggestion a couple of years ago, either this list or 
Py3k list, after newby reported 'problem' on python-list expecting 
module-level global to do something. Guido rejected it on the basis that 
he wanted to minimized differences between module code and function code.


From brett at python.org  Sun Nov 15 21:40:32 2009
From: brett at python.org (Brett Cannon)
Date: Sun, 15 Nov 2009 12:40:32 -0800
Subject: [Python-Dev] [Python-checkins] Using itertools in modules that
	are part of the build chain (Re: r76264 -
	python/branches/py3k/Lib/tokenize.py)
In-Reply-To: <1afaf6160911142001m39d0f0ceyd3646e922d3bf703@mail.gmail.com>
References: <4afeda77.0702d00a.4fa1.488cSMTPIN_ADDED@mx.google.com> 
	<4AFF7048.7070701@gmail.com>
	<1afaf6160911142001m39d0f0ceyd3646e922d3bf703@mail.gmail.com>
Message-ID: <bbaeab100911151240w1f2d8df1ke4d7c2107a1a9deb@mail.gmail.com>

On Sat, Nov 14, 2009 at 20:01, Benjamin Peterson <benjamin at python.org> wrote:
> 2009/11/14 Nick Coghlan <ncoghlan at gmail.com>:
>> This does constrain where we can use itertools - if we want carte
>> blanche to use it anywhere in the standard library, even those parts
>> that are imported as part of the build chain, we'll need to bite the
>> bullet and make it a builtin module rather than a separately built
>> extension module.
>
> I have another unpleasant but slightly less hacky solution. We put
> detect_encoding in linecache where it is actually used.

Well, it happens to be used by the standard library in linecache, but
not all external uses of it necessarily tie into linecache (e.g.
importlib uses detect_encoding() in some non-critical code). Might
just have to live with sub-optimal code.

-Brett

From benjamin at python.org  Sun Nov 15 21:43:10 2009
From: benjamin at python.org (Benjamin Peterson)
Date: Sun, 15 Nov 2009 14:43:10 -0600
Subject: [Python-Dev] [Python-checkins] Using itertools in modules that
	are part of the build chain (Re: r76264 -
	python/branches/py3k/Lib/tokenize.py)
In-Reply-To: <bbaeab100911151240w1f2d8df1ke4d7c2107a1a9deb@mail.gmail.com>
References: <4afeda77.0702d00a.4fa1.488cSMTPIN_ADDED@mx.google.com>
	<4AFF7048.7070701@gmail.com>
	<1afaf6160911142001m39d0f0ceyd3646e922d3bf703@mail.gmail.com>
	<bbaeab100911151240w1f2d8df1ke4d7c2107a1a9deb@mail.gmail.com>
Message-ID: <1afaf6160911151243s23c56722lfa00010f14a0d6b3@mail.gmail.com>

2009/11/15 Brett Cannon <brett at python.org>:
> On Sat, Nov 14, 2009 at 20:01, Benjamin Peterson <benjamin at python.org> wrote:
>> 2009/11/14 Nick Coghlan <ncoghlan at gmail.com>:
>>> This does constrain where we can use itertools - if we want carte
>>> blanche to use it anywhere in the standard library, even those parts
>>> that are imported as part of the build chain, we'll need to bite the
>>> bullet and make it a builtin module rather than a separately built
>>> extension module.
>>
>> I have another unpleasant but slightly less hacky solution. We put
>> detect_encoding in linecache where it is actually used.
>
> Well, it happens to be used by the standard library in linecache, but
> not all external uses of it necessarily tie into linecache (e.g.
> importlib uses detect_encoding() in some non-critical code). Might
> just have to live with sub-optimal code.

Well, what I mean is that we'd do:

def _detect_encoding():

in linecache and then "from linecache import _detect_encoding as
detect_encoding" in tokenize.py.



-- 
Regards,
Benjamin

From martin at v.loewis.de  Sun Nov 15 21:44:43 2009
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sun, 15 Nov 2009 21:44:43 +0100
Subject: [Python-Dev] Too many Python accounts
In-Reply-To: <bfd8702d0911151220u2068fa04p61d2b3d615d219a9@mail.gmail.com>
References: <D7E23A6D-C91C-46D8-A8FE-1EB16F3B2101@lericson.se>	
	<hdn42a$r76$1@ger.gmane.org> <87fx8gsq2s.fsf@benfinney.id.au>	
	<4AFF5B36.9030502@v.loewis.de> <87tywwr0z5.fsf@benfinney.id.au>	
	<4AFF67E8.5060209@v.loewis.de> <4AFFE8E3.5020508@voidspace.org.uk>	
	<4B004F00.7080405@v.loewis.de> <4B005413.3010702@voidspace.org.uk>	
	<4B005727.4050004@v.loewis.de>
	<bfd8702d0911151220u2068fa04p61d2b3d615d219a9@mail.gmail.com>
Message-ID: <4B00683B.6090902@v.loewis.de>

>> Can't you then produce hundreds of IDs, all delegating to the same
>> identity?
> 
> Yes.

But then, users can easily create as many fake accounts as they want to.
This is not something I want to happen (it's still possible to setup
fake accounts, but it should be more difficult for the average script
kiddy).

> If the provider dictates the identity, as you keep insisting, that sentence
> makes no sense whatsoever.  The value entered as the identifier is the
> identifier you should use.  Otherwise, what's the point of delegation at all?

It may help users to remember their openid more easily, and always fill
in the same text into the login box.

Regards,
Martin

From martin at v.loewis.de  Sun Nov 15 21:46:29 2009
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sun, 15 Nov 2009 21:46:29 +0100
Subject: [Python-Dev] Too many Python accounts (was Re: PyPI comments
 and ratings, *really*?)
In-Reply-To: <19200.25648.998776.238480@montanaro.dyndns.org>
References: <D7E23A6D-C91C-46D8-A8FE-1EB16F3B2101@lericson.se>
	<hdi17l$cli$1@ger.gmane.org> <4AFC9064.20700@v.loewis.de>
	<4222a8490911121511s40eba937n2c7b1fde6f695c6c@mail.gmail.com>
	<4AFC9975.7040204@v.loewis.de>
	<4222a8490911121534h30572eabv73ae58649c77155c@mail.gmail.com>
	<4AFC9D0C.5030500@v.loewis.de> <87y6mbuubp.fsf@benfinney.id.au>
	<20e5c9660911121702r162caf28jb6d656383147d909@mail.gmail.com>
	<hdim34$utf$1@ger.gmane.org>
	<79990c6b0911131133i2987a5d4s5c4ef780a527732@mail.gmail.com>
	<4AFE6E10.3050307@v.loewis.de>
	<20091114145222.084583A411A@sparrow.telecommunity.com>
	<4AFECA7A.8070400@v.loewis.de> <hdn42a$r76$1@ger.gmane.org>
	<4AFF5B1A.8090105@v.loewis.de>
	<19200.25648.998776.238480@montanaro.dyndns.org>
Message-ID: <4B0068A5.2040605@v.loewis.de>

> I've never found OpenID at all intuitive to use.  Are there instructions on
> pypi.python.org which detail the steps necessary to use OpenID to login?  I
> saw the "Claim OpenID" link on my PyPI profile page.  So now I have an
> OpenID URL.  What am I supposed to do with that?  If I visit that URL it
> downloads a small bit of XML.
> 
> I've tried using my Yahoo! and Luanchpad OpenIDs for other sites in the
> past.  I've never successfully logged into any website with them, at least
> not as far as I can recall.  I realize that maybe this is something that
> just doesn't click with me (maybe I'm an OpenID Luddite), but it seems to me
> that OpenID needs to be a bit easier (or obvious?) to use if it's to become
> some sort of de facto standard login mechanism.

That's indeed what PyPI attempts to do. At the "claim openid" place,
follow the Launchpad link. It should guide you through the procedure.

Then, when you want to login, again follow the Launchpad link on the
front page.

Regards,
Martin

From guido at python.org  Sun Nov 15 21:46:26 2009
From: guido at python.org (Guido van Rossum)
Date: Sun, 15 Nov 2009 12:46:26 -0800
Subject: [Python-Dev] global statements outside functions/methods should
	raise SyntaxError
In-Reply-To: <hdporv$476$1@ger.gmane.org>
References: <4B005C94.8060707@gmail.com> <hdporv$476$1@ger.gmane.org>
Message-ID: <ca471dc20911151246wd4db8fakc0c40103fb3e06f4@mail.gmail.com>

On Sun, Nov 15, 2009 at 12:38 PM, Terry Reedy <tjreedy at udel.edu> wrote:
> Ezio Melotti wrote:
>>
>> Python currently accepts global statements at the top level:
>>
>> I opened an issue on the tracker (http://bugs.python.org/issue7329)
>> and Benjamin suggested to discuss this here.
>> The test he mentioned is in test_global.py:
>>
>> ? def test4(self):
>> ? ? ? prog_text_4 = """\
>> global x
>> x = 2
>> """
>> ? ? ? # this should work
>> ? ? ? compile(prog_text_4, "<test string>", "exec")
>>
>> It just says that "it should work" but it doesn't say /why/.
>>
>> Any thoughts?
>
> I make the same suggestion a couple of years ago, either this list or Py3k
> list, after newby reported 'problem' on python-list expecting module-level
> global to do something. Guido rejected it on the basis that he wanted to
> minimized differences between module code and function code.

That example should work because you could pass it to exec()/eval()
with separate dicts for locals and globals:

$ python3.0
Python 3.0 (py3k:67506, Dec  3 2008, 10:12:04)
[GCC 4.0.1 (Apple Inc. build 5484)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> glo = {}
>>> lo = {}
>>> so = 'global x; x = 2'
>>> co = compile(so, '', 'exec')
>>> exec(co, glo, lo)
>>> glo['x']
['x']
>>> exec('x = 3', glo, lo)
>>> glo['x']
2
>>> lo['x']
3
>>>

-- 
--Guido van Rossum (python.org/~guido)

From brett at python.org  Sun Nov 15 21:51:10 2009
From: brett at python.org (Brett Cannon)
Date: Sun, 15 Nov 2009 12:51:10 -0800
Subject: [Python-Dev] buildtime vs runtime in Distutils
In-Reply-To: <hdovs4$n83$1@ger.gmane.org>
References: <94bdd2610911141721n6700dcds5a489f1972176e48@mail.gmail.com> 
	<loom.20091115T133752-257@post.gmane.org> <hdovs4$n83$1@ger.gmane.org>
Message-ID: <bbaeab100911151251i2b9fa613n3c81d70efc6acfad@mail.gmail.com>

On Sun, Nov 15, 2009 at 05:31, Georg Brandl <g.brandl at gmx.net> wrote:
> Antoine Pitrou schrieb:
>> Tarek Ziad? <ziade.tarek <at> gmail.com> writes:
>>>
>>> This cannot work on all platforms, when our Makefile is not shipped
>>> with python but python-devel. (like Fedora)
>>
>> This practice is stupid anyway, because it means you have to install
>> python-devel even to install pure Python packages with setuptools/distribute.
>> Just ask Fedora, Mandriva and friends to change their packaging practice
>> (Mandriva already has a bug open for that by the way).
>
> +1. ?They are the ones splitting what "make install" installs into several
> packages, so they are the ones who have to fix the resulting dependency
> problems.

+1 from me as well. Python is designed to run as a whole. If they
choose to ignore our design decisions they do so at their own peril.

Now if you want to put the time in, Tarek, to make sure distutils can
be removed from the stdlib and have everything still work so the linux
distros can add distutils to python-devel that's fine, but once again,
they are the ones mucking around with things in a way we did not
design for, so only do it if you really want to.

-Brett

From daniel at stutzbachenterprises.com  Sun Nov 15 22:25:36 2009
From: daniel at stutzbachenterprises.com (Daniel Stutzbach)
Date: Sun, 15 Nov 2009 15:25:36 -0600
Subject: [Python-Dev] Too many Python accounts
In-Reply-To: <4B00683B.6090902@v.loewis.de>
References: <D7E23A6D-C91C-46D8-A8FE-1EB16F3B2101@lericson.se>
	<4AFF5B36.9030502@v.loewis.de> <87tywwr0z5.fsf@benfinney.id.au>
	<4AFF67E8.5060209@v.loewis.de> <4AFFE8E3.5020508@voidspace.org.uk>
	<4B004F00.7080405@v.loewis.de> <4B005413.3010702@voidspace.org.uk>
	<4B005727.4050004@v.loewis.de>
	<bfd8702d0911151220u2068fa04p61d2b3d615d219a9@mail.gmail.com>
	<4B00683B.6090902@v.loewis.de>
Message-ID: <eae285400911151325m423ce7a2r6f0712f97cbe035@mail.gmail.com>

On Sun, Nov 15, 2009 at 2:44 PM, "Martin v. L?wis" <martin at v.loewis.de>wrote:

> But then, users can easily create as many fake accounts as they want to.
>

Why not do something more robust, then?  For example, when a user enters an
OpenID that hasn't been seen by PyPi before, make them enter a CAPTCHA.

--
Daniel Stutzbach, Ph.D.
President, Stutzbach Enterprises, LLC <http://stutzbachenterprises.com>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20091115/e85c5124/attachment.htm>

From fuzzyman at voidspace.org.uk  Sun Nov 15 22:27:36 2009
From: fuzzyman at voidspace.org.uk (Michael Foord)
Date: Sun, 15 Nov 2009 21:27:36 +0000
Subject: [Python-Dev] Too many Python accounts
In-Reply-To: <4B00683B.6090902@v.loewis.de>
References: <D7E23A6D-C91C-46D8-A8FE-1EB16F3B2101@lericson.se>	
	<hdn42a$r76$1@ger.gmane.org> <87fx8gsq2s.fsf@benfinney.id.au>	
	<4AFF5B36.9030502@v.loewis.de> <87tywwr0z5.fsf@benfinney.id.au>	
	<4AFF67E8.5060209@v.loewis.de> <4AFFE8E3.5020508@voidspace.org.uk>	
	<4B004F00.7080405@v.loewis.de> <4B005413.3010702@voidspace.org.uk>	
	<4B005727.4050004@v.loewis.de>
	<bfd8702d0911151220u2068fa04p61d2b3d615d219a9@mail.gmail.com>
	<4B00683B.6090902@v.loewis.de>
Message-ID: <4B007248.2020201@voidspace.org.uk>

Martin v. L?wis wrote:
>>> Can't you then produce hundreds of IDs, all delegating to the same
>>> identity?
>>>       
>> Yes.
>>     
>
> But then, users can easily create as many fake accounts as they want to.
> This is not something I want to happen (it's still possible to setup
> fake accounts, but it should be more difficult for the average script
> kiddy).
>
>   

This doesn't seem to be a problem for all the other sites I use my 
openid with. Why not allow users to login with their own openid, but 
only allow one account to refer back to the same delegated account?

Michael

>> If the provider dictates the identity, as you keep insisting, that sentence
>> makes no sense whatsoever.  The value entered as the identifier is the
>> identifier you should use.  Otherwise, what's the point of delegation at all?
>>     
>
> It may help users to remember their openid more easily, and always fill
> in the same text into the login box.
>
> Regards,
> Martin
>   


-- 
http://www.ironpythoninaction.com/
http://www.voidspace.org.uk/blog



From skip at pobox.com  Sun Nov 15 22:34:40 2009
From: skip at pobox.com (skip at pobox.com)
Date: Sun, 15 Nov 2009 15:34:40 -0600
Subject: [Python-Dev] Too many Python accounts (was Re: PyPI comments
 and ratings, *really*?)
In-Reply-To: <4B0068A5.2040605@v.loewis.de>
References: <D7E23A6D-C91C-46D8-A8FE-1EB16F3B2101@lericson.se>
	<hdi17l$cli$1@ger.gmane.org> <4AFC9064.20700@v.loewis.de>
	<4222a8490911121511s40eba937n2c7b1fde6f695c6c@mail.gmail.com>
	<4AFC9975.7040204@v.loewis.de>
	<4222a8490911121534h30572eabv73ae58649c77155c@mail.gmail.com>
	<4AFC9D0C.5030500@v.loewis.de> <87y6mbuubp.fsf@benfinney.id.au>
	<20e5c9660911121702r162caf28jb6d656383147d909@mail.gmail.com>
	<hdim34$utf$1@ger.gmane.org>
	<79990c6b0911131133i2987a5d4s5c4ef780a527732@mail.gmail.com>
	<4AFE6E10.3050307@v.loewis.de>
	<20091114145222.084583A411A@sparrow.telecommunity.com>
	<4AFECA7A.8070400@v.loewis.de> <hdn42a$r76$1@ger.gmane.org>
	<4AFF5B1A.8090105@v.loewis.de>
	<19200.25648.998776.238480@montanaro.dyndns.org>
	<4B0068A5.2040605@v.loewis.de>
Message-ID: <19200.29680.613310.668341@montanaro.dyndns.org>


    Martin> That's indeed what PyPI attempts to do. At the "claim openid"
    Martin> place, follow the Launchpad link. It should guide you through
    Martin> the procedure.

Well, since I use Google a lot more I'd prefer to use that.  If I click the
Google OpenID link I now get

    OpenID is already claimed

    Martin> Then, when you want to login, again follow the Launchpad link on
    Martin> the front page.

That seems to work, but I'm not sure how.  Doesn't seem to use cookies.  The
Google OpenID link leads to

    http://pypi.python.org/pypi?:action=login&provider=Google

which contains nothing about me.  I saw a pypi.python.org cookie which
expires "On Quit", so I restarted Camino and verified there were no
pypi.python.org cookies, then clicked the Google OpenID link.  It still
works.  I must be missing something obvious...

S

From martin at v.loewis.de  Sun Nov 15 22:58:23 2009
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sun, 15 Nov 2009 22:58:23 +0100
Subject: [Python-Dev] Too many Python accounts
In-Reply-To: <4B007248.2020201@voidspace.org.uk>
References: <D7E23A6D-C91C-46D8-A8FE-1EB16F3B2101@lericson.se>	
	<hdn42a$r76$1@ger.gmane.org> <87fx8gsq2s.fsf@benfinney.id.au>	
	<4AFF5B36.9030502@v.loewis.de> <87tywwr0z5.fsf@benfinney.id.au>	
	<4AFF67E8.5060209@v.loewis.de> <4AFFE8E3.5020508@voidspace.org.uk>	
	<4B004F00.7080405@v.loewis.de> <4B005413.3010702@voidspace.org.uk>	
	<4B005727.4050004@v.loewis.de>
	<bfd8702d0911151220u2068fa04p61d2b3d615d219a9@mail.gmail.com>
	<4B00683B.6090902@v.loewis.de> <4B007248.2020201@voidspace.org.uk>
Message-ID: <4B00797F.2080003@v.loewis.de>

> This doesn't seem to be a problem for all the other sites I use my
> openid with.

Perhaps they don't care about fake accounts at all? That would, in
particular, be the case if they accept arbitrary OpenID providers
(which means that essentially no real authentication happens).

> Why not allow users to login with their own openid, but
> only allow one account to refer back to the same delegated account?

That sounds good. I'm not sure how to implement a provider change
in that scenario, though.

Regards,
Martin

From ben+python at benfinney.id.au  Sun Nov 15 22:58:50 2009
From: ben+python at benfinney.id.au (Ben Finney)
Date: Mon, 16 Nov 2009 08:58:50 +1100
Subject: [Python-Dev] =?utf-8?q?OpenID_on_PyPI=3A_please_discuss_on_?=
 =?utf-8?q?=E2=80=98catalog-sig=E2=80=99_=28was=3A_Too_many_Python_account?=
 =?utf-8?q?s=29?=
References: <D7E23A6D-C91C-46D8-A8FE-1EB16F3B2101@lericson.se>
	<4AFC9064.20700@v.loewis.de>
	<4222a8490911121511s40eba937n2c7b1fde6f695c6c@mail.gmail.com>
	<4AFC9975.7040204@v.loewis.de>
	<4222a8490911121534h30572eabv73ae58649c77155c@mail.gmail.com>
	<4AFC9D0C.5030500@v.loewis.de> <87y6mbuubp.fsf@benfinney.id.au>
	<20e5c9660911121702r162caf28jb6d656383147d909@mail.gmail.com>
	<hdim34$utf$1@ger.gmane.org>
	<79990c6b0911131133i2987a5d4s5c4ef780a527732@mail.gmail.com>
	<4AFE6E10.3050307@v.loewis.de>
	<20091114145222.084583A411A@sparrow.telecommunity.com>
	<4AFECA7A.8070400@v.loewis.de> <hdn42a$r76$1@ger.gmane.org>
	<87fx8gsq2s.fsf@benfinney.id.au> <4AFF5B36.9030502@v.loewis.de>
	<87tywwr0z5.fsf@benfinney.id.au> <4AFF67E8.5060209@v.loewis.de>
	<4AFFE8E3.5020508@voidspace.org.uk> <4B004F00.7080405@v.loewis.de>
	<4B005413.3010702@voidspace.org.uk>
Message-ID: <87vdhbpi05.fsf_-_@benfinney.id.au>

Michael Foord <fuzzyman at voidspace.org.uk> writes:

> myopenid *does* the validation, but my registered openid is
> www.voidspace.org.uk and I *should* be able to change provider without
> creating a new account.

I'm very glad this issue is being discussed, but it's unfortunately
taking place both here (where it's off-topic) and in private email
(where it's not as visible as it should be), which surely doesn't make
much sense.

I have started a new sub-thread over on ?catalog-sig? for this topic,
starting with Message-ID: <87zl6npiky.fsf_-_ at benfinney.id.au>
<URL:http://mail.python.org/pipermail/catalog-sig/2009-November/002308.html>.
Please continue the discussion there; I'll be gathering together under
that thread some of the salient messages I've seen.

-- 
 \     ?Anyone can do any amount of work provided it isn't the work he |
  `\          is supposed to be doing at the moment.? ?Robert Benchley |
_o__)                                                                  |
Ben Finney


From rdmurray at bitdance.com  Sun Nov 15 23:09:19 2009
From: rdmurray at bitdance.com (R. David Murray)
Date: Sun, 15 Nov 2009 17:09:19 -0500 (EST)
Subject: [Python-Dev] Status of the Buildbot fleet and related bugs
In-Reply-To: <m24op7apok.fsf@valheru.db3l.homeip.net>
References: <Pine.LNX.4.64.0911052209000.12274@kimball.webabinitio.net>
	<5c6f2a5d0911060027x6152a27ao77df7938a079f622@mail.gmail.com>
	<m24op7apok.fsf@valheru.db3l.homeip.net>
Message-ID: <Pine.LNX.4.64.0911151651570.12274@kimball.webabinitio.net>

There are non-stable buildbots that are failing consistently, but this
message is about something else.  Now that the biggest stability
issues have been addressed some less-noisy stability issues are visible.
The two that I have noticed most often are test_httpsservers, which
hangs occasionally, and test_multiprocessing, which fails in various
assertions occasionally.

Since the buildbots are often slow and/or heavily loaded, I tried
increasing DELTA in test_multiprocessing, but while it did seem to help
some as evidenced by how how many times it ran in my buildbot using -F
before and after, it did not prevent failures.

--David (RDM)

From martin at v.loewis.de  Sun Nov 15 23:11:36 2009
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sun, 15 Nov 2009 23:11:36 +0100
Subject: [Python-Dev] Too many Python accounts (was Re: PyPI comments
 and ratings, *really*?)
In-Reply-To: <19200.29680.613310.668341@montanaro.dyndns.org>
References: <D7E23A6D-C91C-46D8-A8FE-1EB16F3B2101@lericson.se>
	<hdi17l$cli$1@ger.gmane.org> <4AFC9064.20700@v.loewis.de>
	<4222a8490911121511s40eba937n2c7b1fde6f695c6c@mail.gmail.com>
	<4AFC9975.7040204@v.loewis.de>
	<4222a8490911121534h30572eabv73ae58649c77155c@mail.gmail.com>
	<4AFC9D0C.5030500@v.loewis.de> <87y6mbuubp.fsf@benfinney.id.au>
	<20e5c9660911121702r162caf28jb6d656383147d909@mail.gmail.com>
	<hdim34$utf$1@ger.gmane.org>
	<79990c6b0911131133i2987a5d4s5c4ef780a527732@mail.gmail.com>
	<4AFE6E10.3050307@v.loewis.de>
	<20091114145222.084583A411A@sparrow.telecommunity.com>
	<4AFECA7A.8070400@v.loewis.de> <hdn42a$r76$1@ger.gmane.org>
	<4AFF5B1A.8090105@v.loewis.de>
	<19200.25648.998776.238480@montanaro.dyndns.org>
	<4B0068A5.2040605@v.loewis.de>
	<19200.29680.613310.668341@montanaro.dyndns.org>
Message-ID: <4B007C98.8010901@v.loewis.de>

skip at pobox.com wrote:
>     Martin> That's indeed what PyPI attempts to do. At the "claim openid"
>     Martin> place, follow the Launchpad link. It should guide you through
>     Martin> the procedure.
> 
> Well, since I use Google a lot more I'd prefer to use that.  If I click the
> Google OpenID link I now get
> 
>     OpenID is already claimed

That means that you already did it. You can associate each OpenID only
with one account (and it doesn't special-case that you are trying to do
it again).

>     Martin> Then, when you want to login, again follow the Launchpad link on
>     Martin> the front page.
> 
> That seems to work, but I'm not sure how.

If it logs you in, it works.

> Doesn't seem to use cookies.  The
> Google OpenID link leads to
> 
>     http://pypi.python.org/pypi?:action=login&provider=Google
> 
> which contains nothing about me.

Don't worry about that. That's how OpenID is supposed to work.

> I saw a pypi.python.org cookie which
> expires "On Quit", so I restarted Camino and verified there were no
> pypi.python.org cookies, then clicked the Google OpenID link.  It still
> works.  I must be missing something obvious...

It's far from obvious. It's called "provider-driven identifier
selection". PyPI redirects your browser to Google. Google looks at
the Google cookie, and finds your identity; they also see that you
have opted to automatically log into PyPI. So without further questions,
they redirect you back to PyPI. PyPI finds your account, and displays
a logged-in page.

Look Ma, no ugly login box with tons of characters to type.

Regards,
Martin

From fuzzyman at voidspace.org.uk  Sun Nov 15 23:20:25 2009
From: fuzzyman at voidspace.org.uk (Michael Foord)
Date: Sun, 15 Nov 2009 22:20:25 +0000
Subject: [Python-Dev] Too many Python accounts
In-Reply-To: <4B00797F.2080003@v.loewis.de>
References: <D7E23A6D-C91C-46D8-A8FE-1EB16F3B2101@lericson.se>	
	<hdn42a$r76$1@ger.gmane.org> <87fx8gsq2s.fsf@benfinney.id.au>	
	<4AFF5B36.9030502@v.loewis.de> <87tywwr0z5.fsf@benfinney.id.au>	
	<4AFF67E8.5060209@v.loewis.de> <4AFFE8E3.5020508@voidspace.org.uk>	
	<4B004F00.7080405@v.loewis.de> <4B005413.3010702@voidspace.org.uk>	
	<4B005727.4050004@v.loewis.de>
	<bfd8702d0911151220u2068fa04p61d2b3d615d219a9@mail.gmail.com>
	<4B00683B.6090902@v.loewis.de> <4B007248.2020201@voidspace.org.uk>
	<4B00797F.2080003@v.loewis.de>
Message-ID: <4B007EA9.1020702@voidspace.org.uk>

Martin v. L?wis wrote:
> [snip...]
>> Why not allow users to login with their own openid, but
>> only allow one account to refer back to the same delegated account?
>>     
>
> That sounds good. I'm not sure how to implement a provider change
> in that scenario, though.
>
>   
Even not having provider changes supported would still allow me to use 
my openid with PyPI which would be great. The only problem with changing 
provider that I can see is when the provider a user changes to would 
then be a duplicate - in which case I think just refusing to allow the 
login would be fine (rather than changing the provider for that account).

All the best,

Michael
> Regards,
> Martin
>   


-- 
http://www.ironpythoninaction.com/
http://www.voidspace.org.uk/blog



From martin at v.loewis.de  Sun Nov 15 23:26:19 2009
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sun, 15 Nov 2009 23:26:19 +0100
Subject: [Python-Dev] Too many Python accounts
In-Reply-To: <4B007EA9.1020702@voidspace.org.uk>
References: <D7E23A6D-C91C-46D8-A8FE-1EB16F3B2101@lericson.se>	
	<hdn42a$r76$1@ger.gmane.org> <87fx8gsq2s.fsf@benfinney.id.au>	
	<4AFF5B36.9030502@v.loewis.de> <87tywwr0z5.fsf@benfinney.id.au>	
	<4AFF67E8.5060209@v.loewis.de> <4AFFE8E3.5020508@voidspace.org.uk>	
	<4B004F00.7080405@v.loewis.de> <4B005413.3010702@voidspace.org.uk>	
	<4B005727.4050004@v.loewis.de>
	<bfd8702d0911151220u2068fa04p61d2b3d615d219a9@mail.gmail.com>
	<4B00683B.6090902@v.loewis.de> <4B007248.2020201@voidspace.org.uk>
	<4B00797F.2080003@v.loewis.de> <4B007EA9.1020702@voidspace.org.uk>
Message-ID: <4B00800B.7050302@v.loewis.de>

> Even not having provider changes supported would still allow me to use
> my openid with PyPI which would be great. The only problem with changing
> provider that I can see is when the provider a user changes to would
> then be a duplicate - in which case I think just refusing to allow the
> login would be fine (rather than changing the provider for that account).

In that case, it would be much easier to record your true openid (i.e.
the one that your provider is able to validate). You can then enter the
alias that you are used to, and PyPI would map that right away to the
verifiable ID, and log you in with that.

For this to work, you would have to upgrade your web page to OpenID 2,
as this is the only protocol that PyPI supports.

Regards,
Martin

From skip at pobox.com  Mon Nov 16 03:20:19 2009
From: skip at pobox.com (skip at pobox.com)
Date: Sun, 15 Nov 2009 20:20:19 -0600
Subject: [Python-Dev] Too many Python accounts (was Re: PyPI comments
 and ratings, *really*?)
In-Reply-To: <4B007C98.8010901@v.loewis.de>
References: <D7E23A6D-C91C-46D8-A8FE-1EB16F3B2101@lericson.se>
	<hdi17l$cli$1@ger.gmane.org> <4AFC9064.20700@v.loewis.de>
	<4222a8490911121511s40eba937n2c7b1fde6f695c6c@mail.gmail.com>
	<4AFC9975.7040204@v.loewis.de>
	<4222a8490911121534h30572eabv73ae58649c77155c@mail.gmail.com>
	<4AFC9D0C.5030500@v.loewis.de> <87y6mbuubp.fsf@benfinney.id.au>
	<20e5c9660911121702r162caf28jb6d656383147d909@mail.gmail.com>
	<hdim34$utf$1@ger.gmane.org>
	<79990c6b0911131133i2987a5d4s5c4ef780a527732@mail.gmail.com>
	<4AFE6E10.3050307@v.loewis.de>
	<20091114145222.084583A411A@sparrow.telecommunity.com>
	<4AFECA7A.8070400@v.loewis.de> <hdn42a$r76$1@ger.gmane.org>
	<4AFF5B1A.8090105@v.loewis.de>
	<19200.25648.998776.238480@montanaro.dyndns.org>
	<4B0068A5.2040605@v.loewis.de>
	<19200.29680.613310.668341@montanaro.dyndns.org>
	<4B007C98.8010901@v.loewis.de>
Message-ID: <19200.46819.688129.940631@montanaro.dyndns.org>


    Martin> It's far from obvious. It's called "provider-driven identifier
    Martin> selection". PyPI redirects your browser to Google. Google looks
    Martin> at the Google cookie, and finds your identity; they also see
    Martin> that you have opted to automatically log into PyPI. So without
    Martin> further questions, they redirect you back to PyPI. PyPI finds
    Martin> your account, and displays a logged-in page.

Thanks.  Makes sense now...

Skip

From tseaver at palladion.com  Mon Nov 16 05:14:30 2009
From: tseaver at palladion.com (Tres Seaver)
Date: Sun, 15 Nov 2009 23:14:30 -0500
Subject: [Python-Dev] PyPI comments and ratings, *really*?
In-Reply-To: <20091114145222.084583A411A@sparrow.telecommunity.com>
References: <D7E23A6D-C91C-46D8-A8FE-1EB16F3B2101@lericson.se>	<hdi17l$cli$1@ger.gmane.org>
	<4AFC9064.20700@v.loewis.de>	<4222a8490911121511s40eba937n2c7b1fde6f695c6c@mail.gmail.com>	<4AFC9975.7040204@v.loewis.de>	<4222a8490911121534h30572eabv73ae58649c77155c@mail.gmail.com>	<4AFC9D0C.5030500@v.loewis.de>
	<87y6mbuubp.fsf@benfinney.id.au>	<20e5c9660911121702r162caf28jb6d656383147d909@mail.gmail.com>	<hdim34$utf$1@ger.gmane.org>	<79990c6b0911131133i2987a5d4s5c4ef780a527732@mail.gmail.com>	<4AFE6E10.3050307@v.loewis.de>
	<20091114145222.084583A411A@sparrow.telecommunity.com>
Message-ID: <hdqjj6$5at$1@ger.gmane.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

P.J. Eby wrote:
> At 09:45 AM 11/14/2009 +0100, Martin v. L?wis wrote:
>> Paul Moore wrote:
>>> 2009/11/13 Tres Seaver <tseaver at palladion.com>:
>>>> I can see the information about the poll, and a link to view the
>>>> results, without logging in.
>>>>
>>>>  http://pypi.python.org/pypi
>>>>
>>>> (second paragraph there).  That paragraph tells you that you need to log
>>>> in to vote in the poll.
>>> I don't want to create a PyPI account (more account details I'll
>>> rarely use to remember) just to vote. I can see why anonymous votes
>>> are bad, but the sample's going to be self-selecting - people like me
>>> who don't want to create an account will be excluded.
>> This is indeed intentional: people like you won't upload packages to
>> PyPI, nor will they take part in the rating system, as both require
>> a PyPI account.
> 
> Which is bizarre, since Paul belongs to the group of people you say 
> you care most about - i.e., those people browsing the index and 
> looking for packages.  The *consumers* of the comments, in other words.

I agree with Martin that anonymous votes, like anonymous comments, are
process-harmful here.


Tres.
- --
===================================================================
Tres Seaver          +1 540-429-0999          tseaver at palladion.com
Palladion Software   "Excellence by Design"    http://palladion.com
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iEYEARECAAYFAksA0aYACgkQ+gerLs4ltQ4WEACfUJmt5LEFujv5xZNCl1tbDGzR
CrIAoKUtK10tQVIiEbDljaHhyTssr4r5
=+UqM
-----END PGP SIGNATURE-----


From kevin at bud.ca  Mon Nov 16 05:19:24 2009
From: kevin at bud.ca (Kevin Teague)
Date: Sun, 15 Nov 2009 20:19:24 -0800
Subject: [Python-Dev] standard libraries don't behave like	standard
	'libraries'
In-Reply-To: <4AFE1492.7050401@canterbury.ac.nz>
References: <c910262d0911120123i7c7e9dla918fff50cb5aecd@mail.gmail.com>
	<4AFC6440.1040008@v.loewis.de> <4AFE1492.7050401@canterbury.ac.nz>
Message-ID: <DDD26F03-F106-4374-A465-FF980042BD64@bud.ca>


On Nov 13, 2009, at 6:23 PM, Greg Ewing wrote:

> Martin v. L?wis wrote:
>
> > Some of the Python maintainers have recently started objecting to  
> this
> > setup, asking that the standard library should be split into  
> separate
> > packages that are released and distributed independent of Python.  
> Others
> > of us feel strongly that such a change should not be made.
>
> I'd be worried, because I would no longer be able to
> release an app or package and say "requires Python x.y".
> I'd have to list the version numbers of all the micro
> packages making up the standard library that I use.
>
> Worse, I'd have to be aware of which ones I actually
> *do* use so I could mantain that list, something I don't
> have to think about at the moment.

"requires Python x.y" would imply a dependency on the working set of  
micro-packages which were shipped with that version of Python (or more  
specifically, any working set that was released within a particular  
Python release series). You would only need to list packages from the  
standard library as dependencies in special-case circumstances where  
you required a version higher or lower than what shipped with a  
particular release series of Python.

It would perhaps increase the potential for people to get into  
situations where they update a Python with newer packages which makes  
it incompatibe with other Python applications. This problem would be  
mitgated by the fact that the standard library tends to be very API  
stable, so usually newer releases only contain minor bug fixes or  
performance enhancements and are unlikely to cause breakage. Package  
installation tools would also still continue to install into site- 
packages, or ideally in a virtualenv or script-generation environement  
like Buildout does. So installing updates to the standard library  
could be done only to support those applications which require them,  
but leave the default working set untouched for any other Python  
applications. Conversely, it may also help the standard library  
compatability in some situations, as I've seen people copy newer files  
into the standard library locations as a method of applying bug fixes,  
and given a better set of metadata it would then be more natural to  
use tools which allowed updates to happen in an orderly fashion.

That's all given that splitting the standard library into individual  
packages also continues to release a single standard library. I don't  
really think releases small/medium/large sized standard libraries as  
was also discussed is a good idea.

Maybe usage of tools such as virtualenv and buildout aren't yet  
widespread enough to alleviate people mucking up their installations  
in such a way that causes them pain. And this would also make it  
easier for people to develop applications which would be harder to  
pakcage into linux distributions or other package managers which don't  
allow for non-global updates. However, these are only theoretical  
concerns. It's concrete issue such as this one:

http://stackoverflow.com/questions/1734373/including-package-data-with-distribute/

Where a developer uses something in the standard library, and a python- 
dev commiter provides a fix very quickly (yay Tarek!). But then that  
developer has to be told to wait a year until the next Python release,  
then wait until you've got the time to migrate the rest of your  
application to the new Python release, then you can finally use that  
fix, and in the meantime even though the issue has been solved you  
still need to workaround the problem! It's issues like this where it's  
hard not to want to avoid using standard library packages (beyond  
"core" modules which stable and only change very rarely lke os, sys,  
re, etc.) because there are unneccessary roadblocks between developer  
and user.


From tseaver at palladion.com  Mon Nov 16 05:27:29 2009
From: tseaver at palladion.com (Tres Seaver)
Date: Sun, 15 Nov 2009 23:27:29 -0500
Subject: [Python-Dev] buildtime vs runtime in Distutils
In-Reply-To: <94bdd2610911150530y3191a192mb00d910e30e01c2e@mail.gmail.com>
References: <94bdd2610911141721n6700dcds5a489f1972176e48@mail.gmail.com>	<4AFF68D4.9070103@cheimes.de>
	<94bdd2610911150530y3191a192mb00d910e30e01c2e@mail.gmail.com>
Message-ID: <hdqkbh$5at$2@ger.gmane.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Tarek Ziad? wrote:
> On Sun, Nov 15, 2009 at 3:35 AM, Christian Heimes <lists at cheimes.de> wrote:
> [..]
>> Do we really want to change distutils to solve a problem of a third
>> party packaging system when the problem was created by the very same
>> third party in the first place? In other words: Should you spend your
>> precious development time with fixing a problem that isn't our fault?
>> The decision to split the header files into a separate package, that
>> isn't installed by default, has already created tons of bad user
>> experience in the past. Do you want to endorse the split even further?
> 
> I didn't know the split story went like this. I took it like the
> "natural" split everyone
> agreed on, and I saw this distutils <-> Makefile link like something to fix.
> 
> So, it sounds like a bad idea now :)

Parsing the Makefile at runtime seems like an insane choice anyway to
me:  +1 for your new module having constants generated at ./configure time.


Tres.
- --
===================================================================
Tres Seaver          +1 540-429-0999          tseaver at palladion.com
Palladion Software   "Excellence by Design"    http://palladion.com
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iEYEARECAAYFAksA1LEACgkQ+gerLs4ltQ5gCQCgtYpBewlnocHJf5hp33TfkLjG
72IAnRW1d9n2CO8S2V+9ewcMb81oWNL+
=GG45
-----END PGP SIGNATURE-----


From ncoghlan at gmail.com  Mon Nov 16 05:43:05 2009
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Mon, 16 Nov 2009 14:43:05 +1000
Subject: [Python-Dev] buildtime vs runtime in Distutils
In-Reply-To: <hdqkbh$5at$2@ger.gmane.org>
References: <94bdd2610911141721n6700dcds5a489f1972176e48@mail.gmail.com>	<4AFF68D4.9070103@cheimes.de>	<94bdd2610911150530y3191a192mb00d910e30e01c2e@mail.gmail.com>
	<hdqkbh$5at$2@ger.gmane.org>
Message-ID: <4B00D859.7090202@gmail.com>

Tres Seaver wrote:
> Tarek Ziad? wrote:
>> I didn't know the split story went like this. I took it like the
>> "natural" split everyone
>> agreed on, and I saw this distutils <-> Makefile link like something to fix.
> 
>> So, it sounds like a bad idea now :)
> 
> Parsing the Makefile at runtime seems like an insane choice anyway to
> me:  +1 for your new module having constants generated at ./configure time.

I'm with Tres here - having distutils (aside from build_ext) depend on
non-Python parts of the source tree seem a little strange.

However, given that the recommended packaging includes the needed files,
putting an RFE on the tracker to reduce the runtime dependency on the
source code would be an acceptable response if there is something else
you'd rather be working on.

Cheers,
Nick.

_______________________________________________
Python-Dev mailing list
Python-Dev at python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/ncoghlan%40gmail.com



-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------

From ziade.tarek at gmail.com  Mon Nov 16 10:38:11 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Mon, 16 Nov 2009 10:38:11 +0100
Subject: [Python-Dev] PyPI comments and ratings, *really*?
In-Reply-To: <bad82a80911140734i822938fm8b57d15db5364c66@mail.gmail.com>
References: <4AFC9064.20700@v.loewis.de>
	<20091113221234.GA15732@amk-desktop.matrixgroup.net>
	<87bpj6t4mp.fsf@benfinney.id.au>
	<200911141018.13536.steve@pearwood.info> <hdksh9$jkt$1@ger.gmane.org>
	<20091114152921.GA28003@laurie.devork>
	<bad82a80911140734i822938fm8b57d15db5364c66@mail.gmail.com>
Message-ID: <94bdd2610911160138k3f973a15i78d49cd36534df23@mail.gmail.com>

On Sat, Nov 14, 2009 at 4:34 PM, Arc Riley <arcriley at gmail.com> wrote:
>
>> +1
>>
>> Having a "Repository-URL", "Repository-Browse-URL" and a
>> "Bug-Tracker-URL" field in PyPI would be a lot more usefule then
>> comments and ratings.
>>
>
> +1

You mean in the Metadata ? We are currenty working on adding fields in them,
for an upcoming 1.2 version (see PEP 345)

I am crossposting to distutils-sig so we can discuss this point over
there, because I think that would
be a great triple of fields to add in 1.2

Tarek

From fuzzyman at voidspace.org.uk  Mon Nov 16 14:41:38 2009
From: fuzzyman at voidspace.org.uk (Michael Foord)
Date: Mon, 16 Nov 2009 13:41:38 +0000
Subject: [Python-Dev] Too many Python accounts
In-Reply-To: <4B00800B.7050302@v.loewis.de>
References: <D7E23A6D-C91C-46D8-A8FE-1EB16F3B2101@lericson.se>	
	<hdn42a$r76$1@ger.gmane.org> <87fx8gsq2s.fsf@benfinney.id.au>	
	<4AFF5B36.9030502@v.loewis.de> <87tywwr0z5.fsf@benfinney.id.au>	
	<4AFF67E8.5060209@v.loewis.de> <4AFFE8E3.5020508@voidspace.org.uk>	
	<4B004F00.7080405@v.loewis.de> <4B005413.3010702@voidspace.org.uk>	
	<4B005727.4050004@v.loewis.de>
	<bfd8702d0911151220u2068fa04p61d2b3d615d219a9@mail.gmail.com>
	<4B00683B.6090902@v.loewis.de> <4B007248.2020201@voidspace.org.uk>
	<4B00797F.2080003@v.loewis.de> <4B007EA9.1020702@voidspace.org.uk>
	<4B00800B.7050302@v.loewis.de>
Message-ID: <4B015692.5020501@voidspace.org.uk>

Martin v. L?wis wrote:
>> Even not having provider changes supported would still allow me to use
>> my openid with PyPI which would be great. The only problem with changing
>> provider that I can see is when the provider a user changes to would
>> then be a duplicate - in which case I think just refusing to allow the
>> login would be fine (rather than changing the provider for that account).
>>     
>
> In that case, it would be much easier to record your true openid (i.e.
> the one that your provider is able to validate). You can then enter the
> alias that you are used to, and PyPI would map that right away to the
> verifiable ID, and log you in with that.
>
>   
Would it be possible to detect a change of provider and then offer the 
option to migrate the account to the new provider (so long as it does 
not conflict with another account)?

> For this to work, you would have to upgrade your web page to OpenID 2,
> as this is the only protocol that PyPI supports.
>   

This I can do.

Thanks

Michael

> Regards,
> Martin
>   


-- 
http://www.ironpythoninaction.com/
http://www.voidspace.org.uk/blog



From martin at v.loewis.de  Mon Nov 16 19:46:14 2009
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Mon, 16 Nov 2009 19:46:14 +0100
Subject: [Python-Dev] Too many Python accounts
In-Reply-To: <4B015692.5020501@voidspace.org.uk>
References: <D7E23A6D-C91C-46D8-A8FE-1EB16F3B2101@lericson.se>	
	<hdn42a$r76$1@ger.gmane.org> <87fx8gsq2s.fsf@benfinney.id.au>	
	<4AFF5B36.9030502@v.loewis.de> <87tywwr0z5.fsf@benfinney.id.au>	
	<4AFF67E8.5060209@v.loewis.de> <4AFFE8E3.5020508@voidspace.org.uk>	
	<4B004F00.7080405@v.loewis.de> <4B005413.3010702@voidspace.org.uk>	
	<4B005727.4050004@v.loewis.de>
	<bfd8702d0911151220u2068fa04p61d2b3d615d219a9@mail.gmail.com>
	<4B00683B.6090902@v.loewis.de> <4B007248.2020201@voidspace.org.uk>
	<4B00797F.2080003@v.loewis.de> <4B007EA9.1020702@voidspace.org.uk>
	<4B00800B.7050302@v.loewis.de> <4B015692.5020501@voidspace.org.uk>
Message-ID: <4B019DF6.9090903@v.loewis.de>

> Would it be possible to detect a change of provider and then offer the
> option to migrate the account to the new provider (so long as it does
> not conflict with another account)?

That would be possible - but again complicate the UI.

Regards,
Martin

From fuzzyman at voidspace.org.uk  Mon Nov 16 20:01:13 2009
From: fuzzyman at voidspace.org.uk (Michael Foord)
Date: Mon, 16 Nov 2009 19:01:13 +0000
Subject: [Python-Dev] Too many Python accounts
In-Reply-To: <4B019DF6.9090903@v.loewis.de>
References: <D7E23A6D-C91C-46D8-A8FE-1EB16F3B2101@lericson.se>	
	<hdn42a$r76$1@ger.gmane.org> <87fx8gsq2s.fsf@benfinney.id.au>	
	<4AFF5B36.9030502@v.loewis.de> <87tywwr0z5.fsf@benfinney.id.au>	
	<4AFF67E8.5060209@v.loewis.de> <4AFFE8E3.5020508@voidspace.org.uk>	
	<4B004F00.7080405@v.loewis.de> <4B005413.3010702@voidspace.org.uk>	
	<4B005727.4050004@v.loewis.de>
	<bfd8702d0911151220u2068fa04p61d2b3d615d219a9@mail.gmail.com>
	<4B00683B.6090902@v.loewis.de> <4B007248.2020201@voidspace.org.uk>
	<4B00797F.2080003@v.loewis.de> <4B007EA9.1020702@voidspace.org.uk>
	<4B00800B.7050302@v.loewis.de> <4B015692.5020501@voidspace.org.uk>
	<4B019DF6.9090903@v.loewis.de>
Message-ID: <4B01A179.1040704@voidspace.org.uk>

Martin v. L?wis wrote:
>> Would it be possible to detect a change of provider and then offer the
>> option to migrate the account to the new provider (so long as it does
>> not conflict with another account)?
>>     
>
> That would be possible - but again complicate the UI.
>   

Why? You should only need to present this option *after* the user has 
authenticated with the a different provider to the one associated with 
their account (or perhaps on detecting that the provider is different - 
but either way no UI changes until / unless the user actually has 
changed provider). That complicates the UI code perhaps, but shouldn't 
change the standard UI.

I don't think it should be given as an option to the user before they 
have changed provider, given that one of the major use cases is to allow 
the user to switch provider if their current one becomes unavailable. A 
UI that only allows them to switch provider if they can first login via 
their current provider would lock them out if their provider goes down.

Michael

> Regards,
> Martin
>   


-- 
http://www.ironpythoninaction.com/
http://www.voidspace.org.uk/blog



From martin at v.loewis.de  Mon Nov 16 20:13:10 2009
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Mon, 16 Nov 2009 20:13:10 +0100
Subject: [Python-Dev] Too many Python accounts
In-Reply-To: <4B01A179.1040704@voidspace.org.uk>
References: <D7E23A6D-C91C-46D8-A8FE-1EB16F3B2101@lericson.se>	
	<hdn42a$r76$1@ger.gmane.org> <87fx8gsq2s.fsf@benfinney.id.au>	
	<4AFF5B36.9030502@v.loewis.de> <87tywwr0z5.fsf@benfinney.id.au>	
	<4AFF67E8.5060209@v.loewis.de> <4AFFE8E3.5020508@voidspace.org.uk>	
	<4B004F00.7080405@v.loewis.de> <4B005413.3010702@voidspace.org.uk>	
	<4B005727.4050004@v.loewis.de>
	<bfd8702d0911151220u2068fa04p61d2b3d615d219a9@mail.gmail.com>
	<4B00683B.6090902@v.loewis.de> <4B007248.2020201@voidspace.org.uk>
	<4B00797F.2080003@v.loewis.de> <4B007EA9.1020702@voidspace.org.uk>
	<4B00800B.7050302@v.loewis.de> <4B015692.5020501@voidspace.org.uk>
	<4B019DF6.9090903@v.loewis.de> <4B01A179.1040704@voidspace.org.uk>
Message-ID: <4B01A446.4020708@v.loewis.de>

Michael Foord wrote:
> Martin v. L?wis wrote:
>>> Would it be possible to detect a change of provider and then offer the
>>> option to migrate the account to the new provider (so long as it does
>>> not conflict with another account)?
>>>     
>>
>> That would be possible - but again complicate the UI.
>>   
> 
> Why? You should only need to present this option *after* the user has
> authenticated 

That's what I mean - it's another HTML form to present, in certain
cases.

It would be easier to change the underlying ID silently, and
refuse login if that then would conflict with an existing account.

Regards,
Martin

From a.badger at gmail.com  Mon Nov 16 20:15:04 2009
From: a.badger at gmail.com (Toshio Kuratomi)
Date: Mon, 16 Nov 2009 11:15:04 -0800
Subject: [Python-Dev] buildtime vs runtime in Distutils
In-Reply-To: <hdovs4$n83$1@ger.gmane.org>
References: <94bdd2610911141721n6700dcds5a489f1972176e48@mail.gmail.com>
	<loom.20091115T133752-257@post.gmane.org>
	<hdovs4$n83$1@ger.gmane.org>
Message-ID: <20091116191504.GS25960@clingman.lan>

On Sun, Nov 15, 2009 at 02:31:45PM +0100, Georg Brandl wrote:
> Antoine Pitrou schrieb:
> > Tarek Ziad? <ziade.tarek <at> gmail.com> writes:
> >> 
> >> This cannot work on all platforms, when our Makefile is not shipped
> >> with python but python-devel. (like Fedora)
> > 
> > This practice is stupid anyway, because it means you have to install
> > python-devel even to install pure Python packages with setuptools/distribute.
> > Just ask Fedora, Mandriva and friends to change their packaging practice
> > (Mandriva already has a bug open for that by the way).
> 
> +1.  They are the ones splitting what "make install" installs into several
> packages, so they are the ones who have to fix the resulting dependency
> problems.
> 
I agree with this, however, my point on the bug was more akin to this:

Tres Seaver wrote:
> Parsing the Makefile at runtime seems like an insane choice anyway to
> me:  +1 for your new module having constants generated at ./configure
> time.

Makefiles and C header files are not intended as general purpose data
formats.  Using them as such has a variety of disadvantages:

* If someone else wants to get at the data, they have to go through the API
  in distutils.  Any data that's not exposed by the API is unavailable.
* Since disturils doesn't implement a full parser for the make and C syntax
  it is possible to break distutils when making legitimate changes to those
  build files.

These are the reasons I opened the bug to get that information into a real
data file rather than parsing the Makefile and header files.

I'll also mention two further things:

The reason that python-devel was split off was to make it more useful for
livecds, olpc, embedded systems, and other places where disk space is at a
premimum.  Being able to combine an operating system that is used by people
beyond your immediate community is great for finding and fixing bugs before
your users run into them.  Being able to program in a high level language on
these platforms has benefits that I'm sure everyone here can appreciate.

I've brought the issue of Makefile and pyconfig.h being needed for distutils
to the attention of every new Fedora python maintainer since the package
split was made.  The current maintainer, David Malcolm, agrees that
distutils.sysconfig needs to be able to use this data and he has moved the
Makefile and header files into the main python package.  This doesn't change
the problems with using a Makefile and C header files as a data format for
python.

-Toshio
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 197 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/python-dev/attachments/20091116/79dc1383/attachment.pgp>

From fuzzyman at voidspace.org.uk  Mon Nov 16 20:18:08 2009
From: fuzzyman at voidspace.org.uk (Michael Foord)
Date: Mon, 16 Nov 2009 19:18:08 +0000
Subject: [Python-Dev] Too many Python accounts
In-Reply-To: <4B01A446.4020708@v.loewis.de>
References: <D7E23A6D-C91C-46D8-A8FE-1EB16F3B2101@lericson.se>	
	<hdn42a$r76$1@ger.gmane.org> <87fx8gsq2s.fsf@benfinney.id.au>	
	<4AFF5B36.9030502@v.loewis.de> <87tywwr0z5.fsf@benfinney.id.au>	
	<4AFF67E8.5060209@v.loewis.de> <4AFFE8E3.5020508@voidspace.org.uk>	
	<4B004F00.7080405@v.loewis.de> <4B005413.3010702@voidspace.org.uk>	
	<4B005727.4050004@v.loewis.de>
	<bfd8702d0911151220u2068fa04p61d2b3d615d219a9@mail.gmail.com>
	<4B00683B.6090902@v.loewis.de> <4B007248.2020201@voidspace.org.uk>
	<4B00797F.2080003@v.loewis.de> <4B007EA9.1020702@voidspace.org.uk>
	<4B00800B.7050302@v.loewis.de> <4B015692.5020501@voidspace.org.uk>
	<4B019DF6.9090903@v.loewis.de> <4B01A179.1040704@voidspace.org.uk>
	<4B01A446.4020708@v.loewis.de>
Message-ID: <4B01A570.8000103@voidspace.org.uk>

Martin v. L?wis wrote:
> Michael Foord wrote:
>   
>> Martin v. L?wis wrote:
>>     
>>>> Would it be possible to detect a change of provider and then offer the
>>>> option to migrate the account to the new provider (so long as it does
>>>> not conflict with another account)?
>>>>     
>>>>         
>>> That would be possible - but again complicate the UI.
>>>   
>>>       
>> Why? You should only need to present this option *after* the user has
>> authenticated 
>>     
>
> That's what I mean - it's another HTML form to present, in certain
> cases.
>
> It would be easier to change the underlying ID silently, and
> refuse login if that then would conflict with an existing account.
>
>   

Sounds good to me.

Michael

> Regards,
> Martin
>   


-- 
http://www.ironpythoninaction.com/
http://www.voidspace.org.uk/blog



From floris.bruynooghe at gmail.com  Mon Nov 16 20:35:05 2009
From: floris.bruynooghe at gmail.com (Floris Bruynooghe)
Date: Mon, 16 Nov 2009 19:35:05 +0000
Subject: [Python-Dev] buildtime vs runtime in Distutils
In-Reply-To: <hdqkbh$5at$2@ger.gmane.org>
References: <94bdd2610911141721n6700dcds5a489f1972176e48@mail.gmail.com>
	<4AFF68D4.9070103@cheimes.de>
	<94bdd2610911150530y3191a192mb00d910e30e01c2e@mail.gmail.com>
	<hdqkbh$5at$2@ger.gmane.org>
Message-ID: <20091116193505.GA29976@laurie.devork>

On Sun, Nov 15, 2009 at 11:27:29PM -0500, Tres Seaver wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
> 
> Tarek Ziad? wrote:
> > On Sun, Nov 15, 2009 at 3:35 AM, Christian Heimes <lists at cheimes.de> wrote:
> > [..]
> >> Do we really want to change distutils to solve a problem of a third
> >> party packaging system when the problem was created by the very same
> >> third party in the first place? In other words: Should you spend your
> >> precious development time with fixing a problem that isn't our fault?
> >> The decision to split the header files into a separate package, that
> >> isn't installed by default, has already created tons of bad user
> >> experience in the past. Do you want to endorse the split even further?
> > 
> > I didn't know the split story went like this. I took it like the
> > "natural" split everyone
> > agreed on, and I saw this distutils <-> Makefile link like something to fix.
> > 
> > So, it sounds like a bad idea now :)
> 
> Parsing the Makefile at runtime seems like an insane choice anyway to
> me:  +1 for your new module having constants generated at ./configure time.

+1

There have been bugs in the past about
distutils.sysconfig.parse_makefile() not knowing the full make syntax
and there will no doubt still be bugs like this.  Substituting these
variables at ./configure time into a .py.in file would be much safer.

Regards
Floris

-- 
Debian GNU/Linux -- The Power of Freedom
www.debian.org | www.gnu.org | www.kernel.org

From p.f.moore at gmail.com  Mon Nov 16 21:39:09 2009
From: p.f.moore at gmail.com (Paul Moore)
Date: Mon, 16 Nov 2009 20:39:09 +0000
Subject: [Python-Dev] PyPI comments and ratings, *really*?
In-Reply-To: <hdqjj6$5at$1@ger.gmane.org>
References: <D7E23A6D-C91C-46D8-A8FE-1EB16F3B2101@lericson.se>
	<4222a8490911121534h30572eabv73ae58649c77155c@mail.gmail.com>
	<4AFC9D0C.5030500@v.loewis.de> <87y6mbuubp.fsf@benfinney.id.au>
	<20e5c9660911121702r162caf28jb6d656383147d909@mail.gmail.com>
	<hdim34$utf$1@ger.gmane.org>
	<79990c6b0911131133i2987a5d4s5c4ef780a527732@mail.gmail.com>
	<4AFE6E10.3050307@v.loewis.de>
	<20091114145222.084583A411A@sparrow.telecommunity.com>
	<hdqjj6$5at$1@ger.gmane.org>
Message-ID: <79990c6b0911161239y75760654yb70bcde05761da25@mail.gmail.com>

2009/11/16 Tres Seaver <tseaver at palladion.com>:
>> Which is bizarre, since Paul belongs to the group of people you say
>> you care most about - i.e., those people browsing the index and
>> looking for packages. ?The *consumers* of the comments, in other words.
>
> I agree with Martin that anonymous votes, like anonymous comments, are
> process-harmful here.

FWIW, having started this, I've no problem with this. As long as
people are aware of the self-selecting nature of the sample (and
Martin clearly is) that's fine.

Paul.

From amcnabb at mcnabbs.org  Mon Nov 16 21:10:09 2009
From: amcnabb at mcnabbs.org (Andrew McNabb)
Date: Mon, 16 Nov 2009 13:10:09 -0700
Subject: [Python-Dev] buildtime vs runtime in Distutils
In-Reply-To: <4AFF68D4.9070103@cheimes.de>
References: <94bdd2610911141721n6700dcds5a489f1972176e48@mail.gmail.com>
	<4AFF68D4.9070103@cheimes.de>
Message-ID: <20091116201009.GB9127@mcnabbs.org>

On Sun, Nov 15, 2009 at 03:35:00AM +0100, Christian Heimes wrote:
> 
> Do we really want to change distutils to solve a problem of a third
> party packaging system when the problem was created by the very same
> third party in the first place? In other words: Should you spend your
> precious development time with fixing a problem that isn't our fault?
> The decision to split the header files into a separate package, that
> isn't installed by default, has already created tons of bad user
> experience in the past. Do you want to endorse the split even further?

The problem being addressed is that distutils is using an artefact of
the build system (a Makefile) as a data file.  I think the issue of
whether or not to "endorse the split" is a red herring.

-- 
Andrew McNabb
http://www.mcnabbs.org/andrew/
PGP Fingerprint: 8A17 B57C 6879 1863 DE55  8012 AB4D 6098 8826 6868

From dmalcolm at redhat.com  Mon Nov 16 21:27:53 2009
From: dmalcolm at redhat.com (David Malcolm)
Date: Mon, 16 Nov 2009 15:27:53 -0500
Subject: [Python-Dev] Static analysis of CPython using coccinelle/spatch
Message-ID: <1258403275.31627.98.camel@brick>

Has anyone else looked at using Coccinelle/spatch[1] on CPython source
code?

It's a GPL-licensed tool for matching semantic patterns in C source
code. It's been used on the Linux kernel for detecting and fixing
problems, and for autogenerating patches when refactoring
(http://coccinelle.lip6.fr/impact_linux.php).  Although it's implemented
in OCaml, it is scriptable using Python.

I've been experimenting with using it on CPython code, both on the core
implementation, and on C extension modules.

As a test, I've written a validator for the mini-language used by
PyArg_ParseTuple and its variants.  My code examines the types of the
variables passed as varargs, and attempts to check that they are
correct, according to the rules here
http://docs.python.org/c-api/arg.html (and in Python/getargs.c)

It can detect this old error (fixed in svn r34931):
buggy.c:12:socket_htons:Mismatching type of argument 1 in ""i:htons"":
expected "int *" but got "unsigned long *"

Similarly, it finds the deliberate error in xxmodule.c:
xxmodule.c:207:xx_roj:unknown format char in "O#:roj": '#'

(Unfortunately, when run on the full source tree, I see numerous
messages, and as far as I can tell, the others are false positives)

You can see the code here:
http://fedorapeople.org/gitweb?p=dmalcolm/public_git/check-cpython.git;a=tree
and download using anonymous git in this manner:
git clone git://fedorapeople.org/home/fedora/dmalcolm/public_git/check-cpython.git

The .cocci file detects invocations of PyArg_ParseTuple and determines
the types of the arguments.  At each matching call site it invokes
python code, passing the type information to validate.py's
validate_types.

(I suspect it's possible to use spatch to detect reference counting
antipatterns; I've also attempted 2to3 refactoring of c code using
semantic patches, but so far macros tend to get in the way).

Alternatively, are there any other non-proprietary static analysis tools
for CPython?

Thoughts?
Dave
 
[1] http://coccinelle.lip6.fr/


From dmalcolm at redhat.com  Mon Nov 16 22:09:33 2009
From: dmalcolm at redhat.com (David Malcolm)
Date: Mon, 16 Nov 2009 16:09:33 -0500
Subject: [Python-Dev] buildtime vs runtime in Distutils
In-Reply-To: <loom.20091115T133752-257@post.gmane.org>
References: <94bdd2610911141721n6700dcds5a489f1972176e48@mail.gmail.com>
	<loom.20091115T133752-257@post.gmane.org>
Message-ID: <1258405773.3763.3.camel@brick>

On Sun, 2009-11-15 at 12:42 +0000, Antoine Pitrou wrote:
> Tarek Ziad? <ziade.tarek <at> gmail.com> writes:
> > 
> > This cannot work on all platforms, when our Makefile is not shipped
> > with python but python-devel. (like Fedora)
> 
> This practice is stupid anyway, because it means you have to install
> python-devel even to install pure Python packages with setuptools/distribute.
> Just ask Fedora, Mandriva and friends to change their packaging practice
> (Mandriva already has a bug open for that by the way).
> 
> In Debian/Ubuntu, the Makefile is correctly part of the main Python package:
> 
> $ dpkg -S /usr/lib/python2.5/config/Makefile
> python2.5: /usr/lib/python2.5/config/Makefile
For what it's worth, this is fixed in Fedora as of python-2.6.4-2.fc13 ;
the bug tracking it was:
https://bugzilla.redhat.com/show_bug.cgi?id=531901




From benjamin at python.org  Tue Nov 17 00:11:25 2009
From: benjamin at python.org (Benjamin Peterson)
Date: Mon, 16 Nov 2009 17:11:25 -0600
Subject: [Python-Dev] 2.7 and 3.2 release schedules
Message-ID: <1afaf6160911161511v5c79f861xe7e4c84728d920af@mail.gmail.com>

After more thought, I think that separating the 2.7 and 3.2 releases
is not as big of an issue as I once thought. Therefore, I'd like to
adopt the schedule I posted a few weeks back for 2.7 only.

This only means some other lucky victi... I mean volunteer can do 3.2. :)

-- 
Regards,
Benjamin

From greg at hewgill.com  Tue Nov 17 11:31:07 2009
From: greg at hewgill.com (Greg Hewgill)
Date: Tue, 17 Nov 2009 10:31:07 +0000
Subject: [Python-Dev] new unbounded memory leak in exception handling?
Message-ID: <20091117103107.GO49760@occam.hewgill.net>

I've constructed an example program that does not leak memory in Python
2.x, but causes unbounded memory allocation in Python 3.1. Here is the
code:

import gc
import sys

class E(Exception):
    def __init__(self, fn):
        self.fn = fn
    def call(self):
        self.fn()

def f():
    raise E(f)

a = E(f)
while True:
    print(len(gc.get_objects()))
    try:
        a.call()
    except E:
        # get exception value in a python2/3 portable way
        a = sys.exc_info()[1]

Every time through the loop, the length of gc.get_objects() increases
by 7 under Python 3.1. I believe this is a regression error, since
Python 2.x does not exhibit the same behaviour.

Can anybody confirm this?

Greg Hewgill
http://hewgill.com

From solipsis at pitrou.net  Tue Nov 17 12:46:18 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Tue, 17 Nov 2009 11:46:18 +0000 (UTC)
Subject: [Python-Dev] Add an optional timeout to lock operations
Message-ID: <loom.20091117T124412-892@post.gmane.org>


Hello,

I've submitted a patch (*) to add an optional timeout to locking operations
(Lock.acquire() etc.). Since it's a pretty basic functionality, I would like to
know if there was any good reason for not doing it.

(*) http://bugs.python.org/issue7316

Thank you

Antoine.



From rdmurray at bitdance.com  Tue Nov 17 14:40:37 2009
From: rdmurray at bitdance.com (R. David Murray)
Date: Tue, 17 Nov 2009 08:40:37 -0500 (EST)
Subject: [Python-Dev] new unbounded memory leak in exception handling?
In-Reply-To: <20091117103107.GO49760@occam.hewgill.net>
References: <20091117103107.GO49760@occam.hewgill.net>
Message-ID: <Pine.LNX.4.64.0911170832270.12274@kimball.webabinitio.net>

On Tue, 17 Nov 2009 at 10:31, Greg Hewgill wrote:
> I've constructed an example program that does not leak memory in Python
> 2.x, but causes unbounded memory allocation in Python 3.1. Here is the
> code:
>
> import gc
> import sys
>
> class E(Exception):
>    def __init__(self, fn):
>        self.fn = fn
>    def call(self):
>        self.fn()
>
> def f():
>    raise E(f)
>
> a = E(f)
> while True:
>    print(len(gc.get_objects()))
>    try:
>        a.call()
>    except E:
>        # get exception value in a python2/3 portable way
>        a = sys.exc_info()[1]
>
> Every time through the loop, the length of gc.get_objects() increases
> by 7 under Python 3.1. I believe this is a regression error, since
> Python 2.x does not exhibit the same behaviour.
>
> Can anybody confirm this?

I think you want to take a look at PEP 3134.  And then please file a doc
bug to have someone update the documentation of sys.exc_info, since the
advice in the warning box is no longer valid in Python 3.

--David (RDM)

From guido at python.org  Tue Nov 17 17:01:49 2009
From: guido at python.org (Guido van Rossum)
Date: Tue, 17 Nov 2009 08:01:49 -0800
Subject: [Python-Dev] Add an optional timeout to lock operations
In-Reply-To: <loom.20091117T124412-892@post.gmane.org>
References: <loom.20091117T124412-892@post.gmane.org>
Message-ID: <ca471dc20911170801m54be779fq87ca11a9797b2db8@mail.gmail.com>

I think I can answer the "why" question: thread.c is *very* old code,
in fact it predates the posix threads standard. When we (actually
Sjoerd Mullender) wrote it, we had a number of OS-specific locking
APIs to work with and the API was designed to fit all of them. I don't
even recall the initial set, but I think it included SGI Irix and and
old version of SunOS. So then over time new platforms were added, but
adding a new method or parameter to the API was nearly impossible
because someone would have to find out how to implement the new
feature on all supported platforms. I think the number of platforms
has dwindled to two or three (Posix, Windows, and maybe one minority
OS?), so now's the time to do it. (IOW I think the idea of the patch
is fine.)

Will locks be interruptible with ^C? That is an oft-requested feature
which also wasn't supported at that time; what's the situation
nowadays?

--Guido

On Tue, Nov 17, 2009 at 3:46 AM, Antoine Pitrou <solipsis at pitrou.net> wrote:
>
> Hello,
>
> I've submitted a patch (*) to add an optional timeout to locking operations
> (Lock.acquire() etc.). Since it's a pretty basic functionality, I would like to
> know if there was any good reason for not doing it.
>
> (*) http://bugs.python.org/issue7316
>
> Thank you
>
> Antoine.
>
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/guido%40python.org
>



-- 
--Guido van Rossum (python.org/~guido)

From solipsis at pitrou.net  Tue Nov 17 17:21:41 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Tue, 17 Nov 2009 16:21:41 +0000 (UTC)
Subject: [Python-Dev] Add an optional timeout to lock operations
References: <loom.20091117T124412-892@post.gmane.org>
	<ca471dc20911170801m54be779fq87ca11a9797b2db8@mail.gmail.com>
Message-ID: <loom.20091117T171910-149@post.gmane.org>

Guido van Rossum <guido <at> python.org> writes:
> 
> I think the number of platforms
> has dwindled to two or three (Posix, Windows, and maybe one minority
> OS?), so now's the time to do it. (IOW I think the idea of the patch
> is fine.)

Thanks. (the minority OS would be OS/2, I think)

> Will locks be interruptible with ^C? That is an oft-requested feature
> which also wasn't supported at that time; what's the situation
> nowadays?

They still aren't interruptible. From what I can read it may be possible to make
them interruptible in the POSIX semaphore-based implementation, not in the POSIX
condition variable-based implementation (which is used as a fallback when POSIX
semaphores are not available, but I don't know whether this fallback is still
useful). 
As for Windows I have absolutely no idea.

Regards

Antoine.



From greg at hewgill.com  Tue Nov 17 19:14:02 2009
From: greg at hewgill.com (Greg Hewgill)
Date: Tue, 17 Nov 2009 18:14:02 +0000
Subject: [Python-Dev] new unbounded memory leak in exception handling?
In-Reply-To: <Pine.LNX.4.64.0911170832270.12274@kimball.webabinitio.net>
References: <20091117103107.GO49760@occam.hewgill.net>
	<Pine.LNX.4.64.0911170832270.12274@kimball.webabinitio.net>
Message-ID: <20091117181402.GQ49760@occam.hewgill.net>

On Tue, Nov 17, 2009 at 08:40:37AM -0500, R. David Murray wrote:
> I think you want to take a look at PEP 3134.  And then please file a doc
> bug to have someone update the documentation of sys.exc_info, since the
> advice in the warning box is no longer valid in Python 3.

Thanks for the pointer, I understand the cause now. I had read the
warning in the description of sys.exc_info, but I hadn't realised that
Python was (now) implicitly saving the traceback for me. I was able to
solve this memory leak by explicitly clearing __traceback__:

    except E:
        # get exception value in a python2/3 portable way
        a = sys.exc_info()[1]
        a.__traceback__ = None

I have added a doc bug issue for this: http://bugs.python.org/issue7340

Greg Hewgill
http://hewgill.com

From brett at python.org  Tue Nov 17 22:03:23 2009
From: brett at python.org (Brett Cannon)
Date: Tue, 17 Nov 2009 13:03:23 -0800
Subject: [Python-Dev] Static analysis of CPython using coccinelle/spatch
In-Reply-To: <1258403275.31627.98.camel@brick>
References: <1258403275.31627.98.camel@brick>
Message-ID: <bbaeab100911171303g21f58e1fsa219e95150d678a0@mail.gmail.com>

On Mon, Nov 16, 2009 at 12:27, David Malcolm <dmalcolm at redhat.com> wrote:
> Has anyone else looked at using Coccinelle/spatch[1] on CPython source
> code?

Not that has been mentioned on the list before.

>
> It's a GPL-licensed tool for matching semantic patterns in C source
> code. It's been used on the Linux kernel for detecting and fixing
> problems, and for autogenerating patches when refactoring
> (http://coccinelle.lip6.fr/impact_linux.php). ?Although it's implemented
> in OCaml, it is scriptable using Python.
>
> I've been experimenting with using it on CPython code, both on the core
> implementation, and on C extension modules.
>
> As a test, I've written a validator for the mini-language used by
> PyArg_ParseTuple and its variants. ?My code examines the types of the
> variables passed as varargs, and attempts to check that they are
> correct, according to the rules here
> http://docs.python.org/c-api/arg.html (and in Python/getargs.c)
>
> It can detect this old error (fixed in svn r34931):
> buggy.c:12:socket_htons:Mismatching type of argument 1 in ""i:htons"":
> expected "int *" but got "unsigned long *"
>
> Similarly, it finds the deliberate error in xxmodule.c:
> xxmodule.c:207:xx_roj:unknown format char in "O#:roj": '#'
>
> (Unfortunately, when run on the full source tree, I see numerous
> messages, and as far as I can tell, the others are false positives)
>
> You can see the code here:
> http://fedorapeople.org/gitweb?p=dmalcolm/public_git/check-cpython.git;a=tree
> and download using anonymous git in this manner:
> git clone git://fedorapeople.org/home/fedora/dmalcolm/public_git/check-cpython.git
>
> The .cocci file detects invocations of PyArg_ParseTuple and determines
> the types of the arguments. ?At each matching call site it invokes
> python code, passing the type information to validate.py's
> validate_types.
>
> (I suspect it's possible to use spatch to detect reference counting
> antipatterns; I've also attempted 2to3 refactoring of c code using
> semantic patches, but so far macros tend to get in the way).
>
> Alternatively, are there any other non-proprietary static analysis tools
> for CPython?

Specific to CPython? No. But I had a chance to run practically every
major commercial static analysis tool over the code base back on 2006.
We also occasionally run valgrind over the code. But thanks to have we
have structured the code and taken performance shortcuts static
analysis tools easily get tripped up by CPython (as you have
discovered).

>
> Thoughts?

Running the tool over the code base and reporting the found bugs would
be appreciated.

-Brett


> Dave
>
> [1] http://coccinelle.lip6.fr/
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/brett%40python.org
>

From amk at amk.ca  Tue Nov 17 22:41:27 2009
From: amk at amk.ca (A.M. Kuchling)
Date: Tue, 17 Nov 2009 16:41:27 -0500
Subject: [Python-Dev] Static analysis of CPython using coccinelle/spatch
In-Reply-To: <1258403275.31627.98.camel@brick>
References: <1258403275.31627.98.camel@brick>
Message-ID: <20091117214127.GA15423@amk-desktop.matrixgroup.net>

On Mon, Nov 16, 2009 at 03:27:53PM -0500, David Malcolm wrote:
> Has anyone else looked at using Coccinelle/spatch[1] on CPython source
> code?

For an excellent explanation of Coccinelle, see
<http://lwn.net/Articles/315686/>.

--amk

From g.brandl at gmx.net  Tue Nov 17 23:55:51 2009
From: g.brandl at gmx.net (Georg Brandl)
Date: Tue, 17 Nov 2009 23:55:51 +0100
Subject: [Python-Dev] 2.7 and 3.2 release schedules
In-Reply-To: <1afaf6160911161511v5c79f861xe7e4c84728d920af@mail.gmail.com>
References: <1afaf6160911161511v5c79f861xe7e4c84728d920af@mail.gmail.com>
Message-ID: <hdv9i7$k63$1@ger.gmane.org>

Benjamin Peterson schrieb:
> After more thought, I think that separating the 2.7 and 3.2 releases
> is not as big of an issue as I once thought. Therefore, I'd like to
> adopt the schedule I posted a few weeks back for 2.7 only.
> 
> This only means some other lucky victi... I mean volunteer can do 3.2. :)

If no one else wants to try and ruin Python 3, I'll do it <wink>.

In the regular 18-month schedule, it is due around Jan 2011, when I will
have finished my diploma and have enough time while looking for PhD
opportunities.

cheers,
Georg


From barry at python.org  Wed Nov 18 00:12:44 2009
From: barry at python.org (Barry Warsaw)
Date: Tue, 17 Nov 2009 17:12:44 -0600
Subject: [Python-Dev] 2.7 and 3.2 release schedules
In-Reply-To: <hdv9i7$k63$1@ger.gmane.org>
References: <1afaf6160911161511v5c79f861xe7e4c84728d920af@mail.gmail.com>
	<hdv9i7$k63$1@ger.gmane.org>
Message-ID: <EF55A70E-20B8-42E7-9BF9-D224DA2014B3@python.org>

On Nov 17, 2009, at 4:55 PM, Georg Brandl wrote:

> Benjamin Peterson schrieb:
>> After more thought, I think that separating the 2.7 and 3.2 releases
>> is not as big of an issue as I once thought. Therefore, I'd like to
>> adopt the schedule I posted a few weeks back for 2.7 only.
>>
>> This only means some other lucky victi... I mean volunteer can do  
>> 3.2. :)
>
> If no one else wants to try and ruin Python 3, I'll do it <wink>.

Ha ha ha^H^H^H^H^H^H^H^H^HGreat!  Benjamin's the expert now, but I'm  
here to help if needed.

-Barry

-------------- next part --------------
A non-text attachment was scrubbed...
Name: PGP.sig
Type: application/pgp-signature
Size: 194 bytes
Desc: This is a digitally signed message part
URL: <http://mail.python.org/pipermail/python-dev/attachments/20091117/fc7d6354/attachment.pgp>

From solipsis at pitrou.net  Wed Nov 18 01:09:01 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Wed, 18 Nov 2009 00:09:01 +0000 (UTC)
Subject: [Python-Dev] 2.7 and 3.2 release schedules
References: <1afaf6160911161511v5c79f861xe7e4c84728d920af@mail.gmail.com>
	<hdv9i7$k63$1@ger.gmane.org>
	<EF55A70E-20B8-42E7-9BF9-D224DA2014B3@python.org>
Message-ID: <loom.20091118T010128-5@post.gmane.org>

Barry Warsaw <barry <at> python.org> writes:
> 
> > If no one else wants to try and ruin Python 3, I'll do it <wink>.
> 
> Ha ha ha^H^H^H^H^H^H^H^H^HGreat!  Benjamin's the expert now, but I'm  
> here to help if needed.

Well Georg isn't a novice when it comes to ruining things, especially
documentation and commit statistics. Although I'm sure guidance from a veteran
can be useful if we want to reach that ultimate degree of perfection which helps
attract an endless stream of anonymous function syntax proposals.

Regards

Antoine, who is improving as well.



From tjreedy at udel.edu  Wed Nov 18 01:45:20 2009
From: tjreedy at udel.edu (Terry Reedy)
Date: Tue, 17 Nov 2009 19:45:20 -0500
Subject: [Python-Dev] Static analysis of CPython using coccinelle/spatch
In-Reply-To: <20091117214127.GA15423@amk-desktop.matrixgroup.net>
References: <1258403275.31627.98.camel@brick>
	<20091117214127.GA15423@amk-desktop.matrixgroup.net>
Message-ID: <hdvg31$5lv$1@ger.gmane.org>

A.M. Kuchling wrote:
> On Mon, Nov 16, 2009 at 03:27:53PM -0500, David Malcolm wrote:
>> Has anyone else looked at using Coccinelle/spatch[1] on CPython source
>> code?
> 
> For an excellent explanation of Coccinelle, see
> <http://lwn.net/Articles/315686/>.

For those who have not looked, Coccinelle means ladybug (a bug-eating 
bug ;-) in French. Its principle use to to take C code and a SmPl file 
of high-level patch descriptions (fixers, in 2to3 talk) and produce a 
standard diff file. I wonder if this could be used to help people 
migrate C extensions to 3.1, by developing a SmPl file with the needed 
changes dictated by API changes. This is similar to its motivating 
application to Linux. From

http://coccinelle.lip6.fr/

"Coccinelle is a program matching and transformation engine which 
provides the language SmPL (Semantic Patch Language) for specifying 
desired matches and transformations in C code. Coccinelle was initially 
targeted towards performing collateral evolutions in Linux. Such 
evolutions comprise the changes that are needed in client code in 
response to evolutions in library APIs, and may include modifications 
such as renaming a function, adding a function argument whose value is 
somehow context-dependent, and reorganizing a data structure. "

As I understand it, the problem with C extensions and 3.1 is the current 
lack of a "collateral evolution" tool like 2to3 for Python code.

Terry Jan Reedy





From ncoghlan at gmail.com  Wed Nov 18 11:38:06 2009
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Wed, 18 Nov 2009 20:38:06 +1000
Subject: [Python-Dev] Add an optional timeout to lock operations
In-Reply-To: <loom.20091117T171910-149@post.gmane.org>
References: <loom.20091117T124412-892@post.gmane.org>	<ca471dc20911170801m54be779fq87ca11a9797b2db8@mail.gmail.com>
	<loom.20091117T171910-149@post.gmane.org>
Message-ID: <4B03CE8E.4070303@gmail.com>

Antoine Pitrou wrote:
> Guido van Rossum <guido <at> python.org> writes:
>> Will locks be interruptible with ^C? That is an oft-requested feature
>> which also wasn't supported at that time; what's the situation
>> nowadays?
> 
> They still aren't interruptible. From what I can read it may be possible to make
> them interruptible in the POSIX semaphore-based implementation, not in the POSIX
> condition variable-based implementation (which is used as a fallback when POSIX
> semaphores are not available, but I don't know whether this fallback is still
> useful).

I'm pretty sure at least some variants of *BSD still don't have OS level
semaphores - their lack is the reason multiprocessing doesn't
necessarily work everywhere that the threading module works (since mp
needs semaphores in order to work its magic).

Jesse would probably know the gory details.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------

From jnoller at gmail.com  Wed Nov 18 13:45:42 2009
From: jnoller at gmail.com (Jesse Noller)
Date: Wed, 18 Nov 2009 07:45:42 -0500
Subject: [Python-Dev] Add an optional timeout to lock operations
In-Reply-To: <4B03CE8E.4070303@gmail.com>
References: <loom.20091117T124412-892@post.gmane.org>
	<ca471dc20911170801m54be779fq87ca11a9797b2db8@mail.gmail.com>
	<loom.20091117T171910-149@post.gmane.org>
	<4B03CE8E.4070303@gmail.com>
Message-ID: <7D64D4D0-1061-4494-BF98-B3C56B6CCF05@gmail.com>



On Nov 18, 2009, at 5:38 AM, Nick Coghlan <ncoghlan at gmail.com> wrote:

> Antoine Pitrou wrote:
>> Guido van Rossum <guido <at> python.org> writes:
>>> Will locks be interruptible with ^C? That is an oft-requested  
>>> feature
>>> which also wasn't supported at that time; what's the situation
>>> nowadays?
>>
>> They still aren't interruptible. From what I can read it may be  
>> possible to make
>> them interruptible in the POSIX semaphore-based implementation, not  
>> in the POSIX
>> condition variable-based implementation (which is used as a  
>> fallback when POSIX
>> semaphores are not available, but I don't know whether this  
>> fallback is still
>> useful).
>
> I'm pretty sure at least some variants of *BSD still don't have OS  
> level
> semaphores - their lack is the reason multiprocessing doesn't
> necessarily work everywhere that the threading module works (since mp
> needs semaphores in order to work its magic).
>
> Jesse would probably know the gory details.
>
> Cheers,
> Nick.
>

Nick is right, many of the BSDs and FreeBSD up until fairly recently  
did not have named shared semaphore support. Still yet, the behavior  
is broken on some OSes such as OS X which you have to work around.

I wouldn't recommend using them for threading right now, there's an  
assumption that threading "just works" unlike multiprocessing, which  
people understand has caveats.

Jesse

From solipsis at pitrou.net  Wed Nov 18 14:50:25 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Wed, 18 Nov 2009 13:50:25 +0000 (UTC)
Subject: [Python-Dev] Add an optional timeout to lock operations
References: <loom.20091117T124412-892@post.gmane.org>
	<ca471dc20911170801m54be779fq87ca11a9797b2db8@mail.gmail.com>
	<loom.20091117T171910-149@post.gmane.org>
	<4B03CE8E.4070303@gmail.com>
	<7D64D4D0-1061-4494-BF98-B3C56B6CCF05@gmail.com>
Message-ID: <loom.20091118T144603-49@post.gmane.org>

Jesse Noller <jnoller <at> gmail.com> writes:
> 
> Nick is right, many of the BSDs and FreeBSD up until fairly recently  
> did not have named shared semaphore support. Still yet, the behavior  
> is broken on some OSes such as OS X which you have to work around.

The core locking support only uses anonymous semaphores (they don't need to be
shared accross processes obviously).

Regards

Antoine.



From jnoller at gmail.com  Wed Nov 18 15:27:36 2009
From: jnoller at gmail.com (Jesse Noller)
Date: Wed, 18 Nov 2009 09:27:36 -0500
Subject: [Python-Dev] Add an optional timeout to lock operations
In-Reply-To: <loom.20091118T144603-49@post.gmane.org>
References: <loom.20091117T124412-892@post.gmane.org>
	<ca471dc20911170801m54be779fq87ca11a9797b2db8@mail.gmail.com>
	<loom.20091117T171910-149@post.gmane.org> <4B03CE8E.4070303@gmail.com>
	<7D64D4D0-1061-4494-BF98-B3C56B6CCF05@gmail.com>
	<loom.20091118T144603-49@post.gmane.org>
Message-ID: <4222a8490911180627q432bb2f8o81f711adf396283a@mail.gmail.com>

On Wed, Nov 18, 2009 at 8:50 AM, Antoine Pitrou <solipsis at pitrou.net> wrote:
> Jesse Noller <jnoller <at> gmail.com> writes:
>>
>> Nick is right, many of the BSDs and FreeBSD up until fairly recently
>> did not have named shared semaphore support. Still yet, the behavior
>> is broken on some OSes such as OS X which you have to work around.
>
> The core locking support only uses anonymous semaphores (they don't need to be
> shared accross processes obviously).
>
> Regards
>
> Antoine.

That's what I get for replying from an iphone over morning coffee. See
also http://semanchuk.com/philip/sysv_ipc/ for Philip Semanchuk
semaphore/IPC work as well.

Given both multiprocessing and Philip's work are concerned about the
shared aspects, I think you're right - FBSD and others support the
posix semaphores for non-ipc stuff ok AFAIK (recent benchmark
http://www.ioremap.net/node/216).

jesse

From dmalcolm at redhat.com  Wed Nov 18 20:09:52 2009
From: dmalcolm at redhat.com (David Malcolm)
Date: Wed, 18 Nov 2009 14:09:52 -0500
Subject: [Python-Dev] Static analysis of CPython using coccinelle/spatch
In-Reply-To: <bbaeab100911171303g21f58e1fsa219e95150d678a0@mail.gmail.com>
References: <1258403275.31627.98.camel@brick>
	<bbaeab100911171303g21f58e1fsa219e95150d678a0@mail.gmail.com>
Message-ID: <1258571393.3005.28.camel@brick>

On Tue, 2009-11-17 at 13:03 -0800, Brett Cannon wrote:
> On Mon, Nov 16, 2009 at 12:27, David Malcolm <dmalcolm at redhat.com> wrote:
> > Has anyone else looked at using Coccinelle/spatch[1] on CPython source
> > code?
[snip]

> Running the tool over the code base and reporting the found bugs would
> be appreciated.

Discounting the false positives, the only issue it finds in python
itself (trunk) is the deliberate mistake in Modules/xxmodule.c

I also ran it on a random sample of extension modules and found some
real bugs (only reported downstream so far, within Fedora's bug
tracker):
    - DBus python bindings assume in one place that "unsigned long" is
32 bits wide: https://bugzilla.redhat.com/show_bug.cgi?id=538225
    - MySQL-python assumes in one place that sizeof(int) ==
sizeof(long):
https://bugzilla.redhat.com/show_bug.cgi?id=538234
    - rpm.ps.append() uses unrecognized 'N' format specifier:
https://bugzilla.redhat.com/show_bug.cgi?id=538218



From dmalcolm at redhat.com  Wed Nov 18 20:26:23 2009
From: dmalcolm at redhat.com (David Malcolm)
Date: Wed, 18 Nov 2009 14:26:23 -0500
Subject: [Python-Dev] 2to3 for .c code (was Static analysis of CPython using
 coccinelle/spatch)
In-Reply-To: <hdvg31$5lv$1@ger.gmane.org>
References: <1258403275.31627.98.camel@brick>
	<20091117214127.GA15423@amk-desktop.matrixgroup.net>
	<hdvg31$5lv$1@ger.gmane.org>
Message-ID: <1258572383.3005.38.camel@brick>

On Tue, 2009-11-17 at 19:45 -0500, Terry Reedy wrote:
> A.M. Kuchling wrote:
> > On Mon, Nov 16, 2009 at 03:27:53PM -0500, David Malcolm wrote:
> >> Has anyone else looked at using Coccinelle/spatch[1] on CPython source
> >> code?
> > 
> > For an excellent explanation of Coccinelle, see
> > <http://lwn.net/Articles/315686/>.
> 
> For those who have not looked, Coccinelle means ladybug (a bug-eating 
> bug ;-) in French. Its principle use to to take C code and a SmPl file 
> of high-level patch descriptions (fixers, in 2to3 talk) and produce a 
> standard diff file. I wonder if this could be used to help people 
> migrate C extensions to 3.1, by developing a SmPl file with the needed 
> changes dictated by API changes. This is similar to its motivating 
> application to Linux. From
> 
> http://coccinelle.lip6.fr/
> 
> "Coccinelle is a program matching and transformation engine which 
> provides the language SmPL (Semantic Patch Language) for specifying 
> desired matches and transformations in C code. Coccinelle was initially 
> targeted towards performing collateral evolutions in Linux. Such 
> evolutions comprise the changes that are needed in client code in 
> response to evolutions in library APIs, and may include modifications 
> such as renaming a function, adding a function argument whose value is 
> somehow context-dependent, and reorganizing a data structure. "
> 
> As I understand it, the problem with C extensions and 3.1 is the current 
> lack of a "collateral evolution" tool like 2to3 for Python code.
Indeed; I think it may be possible to use Coccinelle for this.  

Here's a .cocci semantic patch to convert non-PyObject* dereferences of
an "ob_type" field to use Py_TYPE macro instead. 

@@
PyObject *py_obj_ptr;
type T; 
T non_py_obj_ptr;
@@
(
  py_obj_ptr->ob_type
|
- non_py_obj_ptr->ob_type
+ Py_TYPE(non_py_obj_ptr) 
)


I was able to use this to generate the attached patch for the DBus
python bindings.  Note that it leaves dereferences of a PyObject*
untouched, and works inside sub-expressions.  (There's some noise at the
typedef of Server; I don't know why).

Hope this is helpful
Dave
-------------- next part --------------
A non-text attachment was scrubbed...
Name: example-resultant-ob_type.patch
Type: text/x-patch
Size: 1927 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/python-dev/attachments/20091118/2c172797/attachment.bin>

From db3l.net at gmail.com  Wed Nov 18 22:28:39 2009
From: db3l.net at gmail.com (David Bolen)
Date: Wed, 18 Nov 2009 16:28:39 -0500
Subject: [Python-Dev] Add an optional timeout to lock operations
References: <loom.20091117T124412-892@post.gmane.org>
Message-ID: <m2vdh75xq0.fsf@valheru.db3l.homeip.net>

Antoine Pitrou <solipsis at pitrou.net> writes:

> I've submitted a patch (*) to add an optional timeout to locking
> operations (Lock.acquire() etc.). Since it's a pretty basic
> functionality, I would like to know if there was any good reason for
> not doing it.

I always assumed it was because as a least-common-denominator set of
functionality, some platforms didn't have the necessary support.

Providing the discussion on this ends up with the an implementation
being accepted, I'd absolutely love to see this then leveraged by
threading.Condition.wait() rather rather than the current
poll-with-timed-sleep approach.

The overhead (CPU, but most notably latency) of that approach (which
also directly impacts Queues) has historically been my top reason in
various applications on Windows to have to implement my own Queue or
Condition structures using native Windows calls.

-- David


From solipsis at pitrou.net  Wed Nov 18 23:20:08 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Wed, 18 Nov 2009 22:20:08 +0000 (UTC)
Subject: [Python-Dev] Add an optional timeout to lock operations
References: <loom.20091117T124412-892@post.gmane.org>
	<m2vdh75xq0.fsf@valheru.db3l.homeip.net>
Message-ID: <loom.20091118T231503-307@post.gmane.org>


Hello,

> > I've submitted a patch (*) to add an optional timeout to locking
> > operations (Lock.acquire() etc.). Since it's a pretty basic
> > functionality, I would like to know if there was any good reason for
> > not doing it.
> 
> I always assumed it was because as a least-common-denominator set of
> functionality, some platforms didn't have the necessary support.

Guido's answer says so indeed. Now py3k only needs to support POSIX and Windows
(and, provided Andrew MacIntyre maintains the port, OS/2), which both have
standard support for waiting-with-timeout.

> Providing the discussion on this ends up with the an implementation
> being accepted, I'd absolutely love to see this then leveraged by
> threading.Condition.wait() rather rather than the current
> poll-with-timed-sleep approach.

Agreed. The current patch (as proposed on http://bugs.python.org/issue7316)
includes exactly that.
Poll-with-timed-sleep is especially sub-optimal on laptops where short but
frequent wakeups can cause a significant decrease in battery life.
(the Linux community has been chasing this using powertop:
http://www.lesswatts.org/projects/powertop/)

Feel free to test or review if you're interested.

Regards

Antoine.



From ziade.tarek at gmail.com  Thu Nov 19 10:30:52 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Thu, 19 Nov 2009 10:30:52 +0100
Subject: [Python-Dev] buildtime vs runtime in Distutils
In-Reply-To: <20091116191504.GS25960@clingman.lan>
References: <94bdd2610911141721n6700dcds5a489f1972176e48@mail.gmail.com>
	<loom.20091115T133752-257@post.gmane.org> <hdovs4$n83$1@ger.gmane.org>
	<20091116191504.GS25960@clingman.lan>
Message-ID: <94bdd2610911190130h5006773exf7c43a166ebe9dd9@mail.gmail.com>

On Mon, Nov 16, 2009 at 8:15 PM, Toshio Kuratomi <a.badger at gmail.com> wrote:
[..]
> I've brought the issue of Makefile and pyconfig.h being needed for distutils
> to the attention of every new Fedora python maintainer since the package
> split was made. ?The current maintainer, David Malcolm, agrees that
> distutils.sysconfig needs to be able to use this data and he has moved the
> Makefile and header files into the main python package. ?This doesn't change
> the problems with using a Makefile and C header files as a data format for
> python.

Great News !

Now for the format problem, I agree that it seems more robust to
pre-process the variables
and inject them in the stdlib when ./configure is run.

I am not sure what is the best strategy here, but I would rather not
add yet another
configuration file (wether its an xml format or an ini-like format).

So what I am proposing is to inject those values in a private dict in
the new sysconfig.py module,
that can be read through the get_config_vars / get_config_var APIs.

This means that sysconfig.py will be added as "sysconfig.py.in"

Regards
Tarek

From ziade.tarek at gmail.com  Thu Nov 19 10:32:08 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Thu, 19 Nov 2009 10:32:08 +0100
Subject: [Python-Dev] buildtime vs runtime in Distutils
In-Reply-To: <1258405773.3763.3.camel@brick>
References: <94bdd2610911141721n6700dcds5a489f1972176e48@mail.gmail.com>
	<loom.20091115T133752-257@post.gmane.org>
	<1258405773.3763.3.camel@brick>
Message-ID: <94bdd2610911190132n216eb7b2o7542e4a503907b0@mail.gmail.com>

On Mon, Nov 16, 2009 at 10:09 PM, David Malcolm <dmalcolm at redhat.com> wrote:
[..]
>>
>> $ dpkg -S /usr/lib/python2.5/config/Makefile
>> python2.5: /usr/lib/python2.5/config/Makefile
> For what it's worth, this is fixed in Fedora as of python-2.6.4-2.fc13 ;
> the bug tracking it was:
> https://bugzilla.redhat.com/show_bug.cgi?id=531901

Thanks David !

From lists at cheimes.de  Thu Nov 19 10:55:32 2009
From: lists at cheimes.de (Christian Heimes)
Date: Thu, 19 Nov 2009 10:55:32 +0100
Subject: [Python-Dev] buildtime vs runtime in Distutils
In-Reply-To: <94bdd2610911190130h5006773exf7c43a166ebe9dd9@mail.gmail.com>
References: <94bdd2610911141721n6700dcds5a489f1972176e48@mail.gmail.com>	<loom.20091115T133752-257@post.gmane.org>
	<hdovs4$n83$1@ger.gmane.org>	<20091116191504.GS25960@clingman.lan>
	<94bdd2610911190130h5006773exf7c43a166ebe9dd9@mail.gmail.com>
Message-ID: <4B051614.3000506@cheimes.de>

Tarek Ziad? wrote:
> Now for the format problem, I agree that it seems more robust to
> pre-process the variables
> and inject them in the stdlib when ./configure is run.
> 
> I am not sure what is the best strategy here, but I would rather not
> add yet another
> configuration file (wether its an xml format or an ini-like format).
> 
> So what I am proposing is to inject those values in a private dict in
> the new sysconfig.py module,
> that can be read through the get_config_vars / get_config_var APIs.
> 
> This means that sysconfig.py will be added as "sysconfig.py.in"

Are you planing to implement the proposal for Windows, too? There may be
no need to change the code because we have full control over the
official installers. Also the VisualStudio Windows build doesn't use
./configure but hand crafted header files. You'd have to fall back to a
hand crafter sysconfig.py.

Christian

From solipsis at pitrou.net  Thu Nov 19 11:04:03 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Thu, 19 Nov 2009 10:04:03 +0000 (UTC)
Subject: [Python-Dev] buildtime vs runtime in Distutils
References: <94bdd2610911141721n6700dcds5a489f1972176e48@mail.gmail.com>
	<loom.20091115T133752-257@post.gmane.org>
	<hdovs4$n83$1@ger.gmane.org> <20091116191504.GS25960@clingman.lan>
	<94bdd2610911190130h5006773exf7c43a166ebe9dd9@mail.gmail.com>
Message-ID: <loom.20091119T110210-92@post.gmane.org>

Tarek Ziad? <ziade.tarek <at> gmail.com> writes:
> 
> So what I am proposing is to inject those values in a private dict in
> the new sysconfig.py module,
> that can be read through the get_config_vars / get_config_var APIs.
> 
> This means that sysconfig.py will be added as "sysconfig.py.in"

This means you have to relaunch the whole configure thing each time you make a
change to sysconfig? This doesn't sound like a good idea to me. You could make
the private dict a dedicated module instead, e.g. "_sysconfig_data.py.in".

As for Windows, if people have the ability to choose the installation directory
when installing, I'm not sure how you're gonna handle it.



From mal at egenix.com  Thu Nov 19 12:04:44 2009
From: mal at egenix.com (M.-A. Lemburg)
Date: Thu, 19 Nov 2009 12:04:44 +0100
Subject: [Python-Dev] buildtime vs runtime in Distutils
In-Reply-To: <94bdd2610911190130h5006773exf7c43a166ebe9dd9@mail.gmail.com>
References: <94bdd2610911141721n6700dcds5a489f1972176e48@mail.gmail.com>	<loom.20091115T133752-257@post.gmane.org>
	<hdovs4$n83$1@ger.gmane.org>	<20091116191504.GS25960@clingman.lan>
	<94bdd2610911190130h5006773exf7c43a166ebe9dd9@mail.gmail.com>
Message-ID: <4B05264C.4030203@egenix.com>

Tarek Ziad? wrote:
> On Mon, Nov 16, 2009 at 8:15 PM, Toshio Kuratomi <a.badger at gmail.com> wrote:
> [..]
>> I've brought the issue of Makefile and pyconfig.h being needed for distutils
>> to the attention of every new Fedora python maintainer since the package
>> split was made.  The current maintainer, David Malcolm, agrees that
>> distutils.sysconfig needs to be able to use this data and he has moved the
>> Makefile and header files into the main python package.  This doesn't change
>> the problems with using a Makefile and C header files as a data format for
>> python.
> 
> Great News !
> 
> Now for the format problem, I agree that it seems more robust to
> pre-process the variables
> and inject them in the stdlib when ./configure is run.
> 
> I am not sure what is the best strategy here, but I would rather not
> add yet another
> configuration file (wether its an xml format or an ini-like format).
> 
> So what I am proposing is to inject those values in a private dict in
> the new sysconfig.py module,
> that can be read through the get_config_vars / get_config_var APIs.
> 
> This means that sysconfig.py will be added as "sysconfig.py.in"

Wouldn't it be cleaner to place the extracted information into
a new module _sysconfig_settings.py which gets created by the
configure script or the Makefile later during the build ?

This new module could then be imported into sysconfig.py at
run-time.

BTW: There's already a Tools/scripts/h2py.py we could use for
converting pyconfig.h into a Python file. For the Makefile
we could use the distutils Makefile parser to add the extra
values.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Nov 19 2009)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try our new mxODBC.Connect Python Database Interface for free ! ::::


   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
           Registered at Amtsgericht Duesseldorf: HRB 46611
               http://www.egenix.com/company/contact/

From ziade.tarek at gmail.com  Thu Nov 19 17:14:16 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Thu, 19 Nov 2009 17:14:16 +0100
Subject: [Python-Dev] buildtime vs runtime in Distutils
In-Reply-To: <loom.20091119T110210-92@post.gmane.org>
References: <94bdd2610911141721n6700dcds5a489f1972176e48@mail.gmail.com>
	<loom.20091115T133752-257@post.gmane.org> <hdovs4$n83$1@ger.gmane.org>
	<20091116191504.GS25960@clingman.lan>
	<94bdd2610911190130h5006773exf7c43a166ebe9dd9@mail.gmail.com>
	<loom.20091119T110210-92@post.gmane.org>
Message-ID: <94bdd2610911190814m710ecc66ve5fd085650b1299a@mail.gmail.com>

On Thu, Nov 19, 2009 at 11:04 AM, Antoine Pitrou <solipsis at pitrou.net> wrote:
> Tarek Ziad? <ziade.tarek <at> gmail.com> writes:
>>
>> So what I am proposing is to inject those values in a private dict in
>> the new sysconfig.py module,
>> that can be read through the get_config_vars / get_config_var APIs.
>>
>> This means that sysconfig.py will be added as "sysconfig.py.in"
>
> This means you have to relaunch the whole configure thing each time you make a
> change to sysconfig? This doesn't sound like a good idea to me. You could make
> the private dict a dedicated module instead, e.g. "_sysconfig_data.py.in".

That's much better indeed.

>
> As for Windows, if people have the ability to choose the installation directory
> when installing, I'm not sure how you're gonna handle it.

I don't know. I am investigating with what Christian has explained on
Windows installers, to
see how it can be hooked in the installation process, and a fallback
for the win32 build version.

If it gets too complex, I'll probably postpone this and finish a first
version of sysconfig that does
use Makefile.

Tarek

From ziade.tarek at gmail.com  Thu Nov 19 17:19:10 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Thu, 19 Nov 2009 17:19:10 +0100
Subject: [Python-Dev] buildtime vs runtime in Distutils
In-Reply-To: <4B05264C.4030203@egenix.com>
References: <94bdd2610911141721n6700dcds5a489f1972176e48@mail.gmail.com>
	<loom.20091115T133752-257@post.gmane.org> <hdovs4$n83$1@ger.gmane.org>
	<20091116191504.GS25960@clingman.lan>
	<94bdd2610911190130h5006773exf7c43a166ebe9dd9@mail.gmail.com>
	<4B05264C.4030203@egenix.com>
Message-ID: <94bdd2610911190819xad416cl37ed8564d18d529d@mail.gmail.com>

On Thu, Nov 19, 2009 at 12:04 PM, M.-A. Lemburg <mal at egenix.com> wrote:
[..]
> BTW: There's already a Tools/scripts/h2py.py we could use for
> converting pyconfig.h into a Python file.

Good to know,

> For the Makefile
> we could use the distutils Makefile parser to add the extra
> values.

Yes, there's also a pyconfig.h parser function. It seems though, for Makefile,
that the tools available during the build phase are simpler/better at
extracting values.
(currently experimenting those)

Tarek

From benjamin at python.org  Fri Nov 20 00:43:30 2009
From: benjamin at python.org (Benjamin Peterson)
Date: Thu, 19 Nov 2009 17:43:30 -0600
Subject: [Python-Dev] Meet your 3.2 release manager
Message-ID: <1afaf6160911191543g34a746b9xf5395c5278791c7e@mail.gmail.com>

I'm pleased to announce that Georg has (naively) volunteered to
shepherd the 3.2 release.

-- 
Regards,
Benjamin

From status at bugs.python.org  Fri Nov 20 18:07:15 2009
From: status at bugs.python.org (Python tracker)
Date: Fri, 20 Nov 2009 18:07:15 +0100 (CET)
Subject: [Python-Dev] Summary of Python tracker Issues
Message-ID: <20091120170715.A321478364@psf.upfronthosting.co.za>


ACTIVITY SUMMARY (11/13/09 - 11/20/09)
Python tracker at http://bugs.python.org/

To view or respond to any of the issues listed below, click on the issue 
number.  Do NOT respond to this message.


 2506 open (+41) / 16686 closed (+15) / 19192 total (+56)

Open issues with patches:  1000

Average duration of open issues: 685 days.
Median duration of open issues: 440 days.

Open Issues Breakdown
   open  2471 (+41)
pending    34 ( +0)

Issues Created Or Reopened (59)
_______________________________

textwrap doesn't linebreak on "\n"                               11/19/09
       http://bugs.python.org/issue1859    reopened gvanrossum                    
                                                                               

float compared to decimal is silently incorrect.                 11/14/09
       http://bugs.python.org/issue2531    reopened mark.dickinson                
       patch                                                                   

2to3 execfile conversion changes behavior                        11/16/09
       http://bugs.python.org/issue7268    reopened benjamin.peterson             
       patch                                                                   

os.path.normpath doesn't normalize ../path/something.py          11/13/09
CLOSED http://bugs.python.org/issue7315    created  purpleidea                    
                                                                               

Add a timeout functionality to common locking operations         11/13/09
       http://bugs.python.org/issue7316    created  pitrou                        
       patch                                                                   

Display full tracebacks when an error occurs asynchronously      11/13/09
       http://bugs.python.org/issue7317    created  pitrou                        
                                                                               

multiprocessing should not wait endlessly                        11/13/09
CLOSED http://bugs.python.org/issue7318    created  pitrou                        
       patch                                                                   

Silence DeprecationWarning by default                            11/14/09
       http://bugs.python.org/issue7319    created  brett.cannon                  
       patch                                                                   

Unable to load external modules on build slave with debug python 11/14/09
       http://bugs.python.org/issue7320    created  db3l                          
                                                                               

PyIter_Check(obj) fails when obj is of type PySetType            11/14/09
CLOSED http://bugs.python.org/issue7321    created  damianeads                    
                                                                               

Socket timeout can cause file-like readline() method to lose dat 11/14/09
       http://bugs.python.org/issue7322    created  beazley                       
                                                                               

decimal.Decimal greater than/less than sometimes gives wrong ans 11/14/09
CLOSED http://bugs.python.org/issue7323    created  adamtj                        
                                                                               

Add sanity-check else case to regrtest option parsing            11/14/09
CLOSED http://bugs.python.org/issue7324    created  r.david.murray                
       patch, patch                                                            

tempfile.mkdtemp() does not return absolute pathname when dir is 11/14/09
       http://bugs.python.org/issue7325    created  roysmith                      
       easy                                                                    

SOLUTION pls? /usr/lib/python2.6/dist-packages/visual/__init__.p 11/15/09
CLOSED http://bugs.python.org/issue7326    created  pveloz                        
                                                                               

format: minimum width: UTF-8 separators and decimal points       11/15/09
       http://bugs.python.org/issue7327    created  skrah                         
                                                                               

pydoc doesn't work from the command line                         11/15/09
CLOSED http://bugs.python.org/issue7328    created  pitrou                        
                                                                               

global statements outside functions/methods should raise SyntaxE 11/15/09
       http://bugs.python.org/issue7329    created  ezio.melotti                  
                                                                               

PyUnicode_FromFormat segfault when using widths.                 11/15/09
       http://bugs.python.org/issue7330    created  mark.dickinson                
                                                                               

Command line testing consistency between 2.x and 3.x             11/16/09
       http://bugs.python.org/issue7331    created  ncoghlan                      
       easy                                                                    

python script segment fault at PyMarshal_ReadLastObjectFromFile  11/16/09
       http://bugs.python.org/issue7332    created  liang                         
                                                                               

Add initgroups to the posix/os modules                           11/16/09
       http://bugs.python.org/issue7333    created  exarkun                       
       patch                                                                   

XML file locking in Jython 2.5 (OSError on Windows)              11/16/09
       http://bugs.python.org/issue7334    created  gdoutch                       
                                                                               

int/long discrepancy when formatting zero with "%.0d"            11/16/09
       http://bugs.python.org/issue7335    created  mark.dickinson                
                                                                               

traceback module not properly printing exceptions on interpreter 11/16/09
       http://bugs.python.org/issue7336    created  rtyler                        
                                                                               

Add lossy queue to queue library module                          11/17/09
       http://bugs.python.org/issue7337    created  bpb                           
                                                                               

recursive __attribute__ -> Fatal Python error: Cannot recover fr 11/17/09
       http://bugs.python.org/issue7338    created  kaizhu                        
                                                                               

bsddb crashes when transaction limit occurs                      11/17/09
       http://bugs.python.org/issue7339    created  pc                            
                                                                               

Doc for sys.exc_info has warning that is no longer valid         11/17/09
       http://bugs.python.org/issue7340    created  ghewgill                      
                                                                               

test_tarfile failing (file in use when deleting) on Windows buil 11/17/09
CLOSED http://bugs.python.org/issue7341    created  pmoore                        
       patch, buildbot                                                         

str(datetime_obj) doesn't include microseconds if their value is 11/17/09
       http://bugs.python.org/issue7342    created  ezio.melotti                  
                                                                               

"What's new in 3.0" says % operator will be deprecated in 3.1    11/18/09
CLOSED http://bugs.python.org/issue7343    created  jaredgrubb                    
                                                                               

wsgiref tests failing on Windows 7 buildbot                      11/18/09
CLOSED http://bugs.python.org/issue7344    created  ncoghlan                      
       buildbot                                                                

calendar.TextCalendar().formatyear wrong argument list           11/18/09
       http://bugs.python.org/issue7345    created  dingle                        
                                                                               

Redirected stdout fires [Errno 9]                                11/18/09
       http://bugs.python.org/issue7346    created  ecir.hana                     
                                                                               

Patch - add RegCreateKeyEx to _winreg                            11/18/09
       http://bugs.python.org/issue7347    created  brian.curtin                  
       patch                                                                   

StringIO.StringIO.readline(-1)  returns the wrong result compare 11/18/09
       http://bugs.python.org/issue7348    created  exarkun                       
                                                                               

StringIO.StringIO, io.BytesIO, and io.StringIO accept None in pl 11/18/09
       http://bugs.python.org/issue7349    created  exarkun                       
                                                                               

csv doesn't handle escaped characters properly                   11/18/09
       http://bugs.python.org/issue7350    created  est_python_tracker            
                                                                               

Documentation typos found in "zipfile ??? Work with ZIP archives 11/18/09
       http://bugs.python.org/issue7351    reopened Retro                         
       patch                                                                   

python2.6-config --ldflags out of /usr and missing -L<install_li 11/18/09
       http://bugs.python.org/issue7352    created  mauger                        
                                                                               

cporting docs recommend using Include/intobject.h, which was rem 11/18/09
       http://bugs.python.org/issue7353    created  dmalcolm                      
                                                                               

distutils.tests.test_msvc9compiler - dragfullwindows can have va 11/18/09
CLOSED http://bugs.python.org/issue7354    created  pmoore                        
       patch, patch, buildbot                                                  

Struct incorrectly compiles format strings                       11/19/09
       http://bugs.python.org/issue7355    reopened mark.dickinson                
       easy                                                                    

parsing of ldconfig output in ctypes/utils.py depends on the loc 11/19/09
       http://bugs.python.org/issue7356    created  doko                          
       patch                                                                   

tarfile doesn't detect disk full error on extraction             11/19/09
       http://bugs.python.org/issue7357    created  eliv                          
                                                                               

cPickle crash on failed assertion                                11/19/09
       http://bugs.python.org/issue7358    created  rybesh                        
                                                                               

mailbox cannot modify mailboxes in system mail spool             11/19/09
       http://bugs.python.org/issue7359    created  doko                          
                                                                               

[mailbox] race: mbox may lose data with concurrent access        11/19/09
       http://bugs.python.org/issue7360    created  doko                          
                                                                               

Strange importlib error during test_multiprocessing failure      11/19/09
       http://bugs.python.org/issue7361    created  r.david.murray                
                                                                               

Incorrect error message with def((x)=0)                          11/19/09
CLOSED http://bugs.python.org/issue7362    created  james.lingard                 
                                                                               

Inconsistent indentation in SocketServer docs                    11/20/09
CLOSED http://bugs.python.org/issue7363    created  ubershmekel                   
       patch                                                                   

Traceback contains incorrect line number for duplicate argument  11/20/09
CLOSED http://bugs.python.org/issue7364    created  james.lingard                 
                                                                               

grp and pwd should treat uid and gid as unsigned                 11/20/09
       http://bugs.python.org/issue7365    created  eplese                        
       patch                                                                   

weakref module example relies on behaviour not guaranteed by id( 11/20/09
       http://bugs.python.org/issue7366    created  jamesh                        
                                                                               

IDLE OSError [errno 13] permission denied accessing help files   11/20/09
       http://bugs.python.org/issue7367    created  Quik_Foot                     
                                                                               

Internal error in threading.py                                   11/20/09
CLOSED http://bugs.python.org/issue7368    created  3Jane                         
                                                                               

Fibonacci example does not include 0; section 4.6                11/20/09
       http://bugs.python.org/issue7369    created  prm225                        
                                                                               

patch: BaseHTTPServer reinventing rfc822                         11/20/09
       http://bugs.python.org/issue7370    created  schmiddy                      
       patch                                                                   



Issues Now Closed (50)
______________________

py3k os.popen result is not iterable, patch attached              806 days
       http://bugs.python.org/issue1087    ezio.melotti                  
       patch                                                                   

"python -m pydoc -g"  fails                                       650 days
       http://bugs.python.org/issue2029    ncoghlan                      
                                                                               

add ftp-tls support to ftplib - RFC 4217                          648 days
       http://bugs.python.org/issue2054    pitrou                        
       patch                                                                   

Fold unary + and not on constants                                 601 days
       http://bugs.python.org/issue2499    rhettinger                    
       patch                                                                   

IDLE won't start in custom directory.                             426 days
       http://bugs.python.org/issue3881    gagenellina                   
       patch                                                                   

single architecture framework build fails on OS X 10.5            418 days
       http://bugs.python.org/issue3962    ronaldoussoren                
                                                                               

pprint._safe_repr is not general enough in one instance           418 days
       http://bugs.python.org/issue3976    rhettinger                    
                                                                               

IDLE does not open too                                            405 days
       http://bugs.python.org/issue4049    loewis                        
       patch                                                                   

mimetypes on Windows should read MIME database from registry (w/  302 days
       http://bugs.python.org/issue4969    pitrou                        
       patch                                                                   

unicode(x) for weakref.proxy objects invokes __str__ instead of   299 days
       http://bugs.python.org/issue5037    benjamin.peterson             
       patch, needs review                                                     

Malloc errors in test_io                                          232 days
       http://bugs.python.org/issue5614    ronaldoussoren                
                                                                               

Enable short float repr() on Solaris/x86                          210 days
       http://bugs.python.org/issue5792    mark.dickinson                
       patch, easy                                                             

Subclassing property doesn't preserve the auto __doc__ behavior   202 days
       http://bugs.python.org/issue5890    moriyoshi                     
       patch, needs review                                                     

normpath() sometimes maps unicode to str                          127 days
       http://bugs.python.org/issue6450    ezio.melotti                  
                                                                               

Wrong doc strings in itertools                                     91 days
       http://bugs.python.org/issue6738    rhettinger                    
       patch                                                                   

Provide CPython command line functionality via runpy module        79 days
       http://bugs.python.org/issue6816    cjw296                        
                                                                               

enable compilation of readline module on Mac OS X 10.5 and 10.6    68 days
       http://bugs.python.org/issue6877    srid                          
       patch, 26backport, needs review                                         

Redundant calls made to comparison methods.                        54 days
       http://bugs.python.org/issue6970    mark.dickinson                
       needs review                                                            

strptime problem                                                   42 days
       http://bugs.python.org/issue7085    ronaldoussoren                
                                                                               

imp.load_module in thread causes core dump on OSX 10.6             35 days
       http://bugs.python.org/issue7144    ronaldoussoren                
                                                                               

Unportable test(1) construct                                       30 days
       http://bugs.python.org/issue7179    ronaldoussoren                
       patch                                                                   

"pydoc -k" can generate AttributeError on Mac OS X                 29 days
       http://bugs.python.org/issue7180    ronaldoussoren                
                                                                               

64 bit python fails on Windows 7                                   24 days
       http://bugs.python.org/issue7206    richo                         
                                                                               

Mark expected failures of test_math, test_cmath and test_round a   13 days
       http://bugs.python.org/issue7251    mark.dickinson                
       buildbot                                                                

Fix set.intersection docstring                                     14 days
       http://bugs.python.org/issue7263    rhettinger                    
       patch                                                                   

odd exec() behavior or documentation                                8 days
       http://bugs.python.org/issue7286    georg.brandl                  
                                                                               

test_msvc9compiler test_reg_class failure on new Windows box        9 days
       http://bugs.python.org/issue7293    tarek                         
                                                                               

reversed(range(x, -1, -1)) is empty when x > 1                      6 days
       http://bugs.python.org/issue7298    mark.dickinson                
       patch                                                                   

Wrong link to Python Language Mapping Specification                 6 days
       http://bugs.python.org/issue7302    georg.brandl                  
                                                                               

Run some tests in a loop until failure                              2 days
       http://bugs.python.org/issue7312    r.david.murray                
       patch                                                                   

2to3 fails when source file starts with BOM                         0 days
       http://bugs.python.org/issue7313    benjamin.peterson             
                                                                               

test_multiprocessing hangs in TestZZZNumberOfObjects after unitt    4 days
       http://bugs.python.org/issue7314    krisvale                      
       buildbot                                                                

os.path.normpath doesn't normalize ../path/something.py             1 days
       http://bugs.python.org/issue7315    georg.brandl                  
                                                                               

multiprocessing should not wait endlessly                           4 days
       http://bugs.python.org/issue7318    krisvale                      
       patch                                                                   

PyIter_Check(obj) fails when obj is of type PySetType               0 days
       http://bugs.python.org/issue7321    pitrou                        
                                                                               

decimal.Decimal greater than/less than sometimes gives wrong ans    0 days
       http://bugs.python.org/issue7323    mark.dickinson                
                                                                               

Add sanity-check else case to regrtest option parsing               0 days
       http://bugs.python.org/issue7324    r.david.murray                
       patch, patch                                                            

SOLUTION pls? /usr/lib/python2.6/dist-packages/visual/__init__.p    0 days
       http://bugs.python.org/issue7326    Arfrever                      
                                                                               

pydoc doesn't work from the command line                            0 days
       http://bugs.python.org/issue7328    ncoghlan                      
                                                                               

test_tarfile failing (file in use when deleting) on Windows buil    1 days
       http://bugs.python.org/issue7341    lars.gustaebel                
       patch, buildbot                                                         

"What's new in 3.0" says % operator will be deprecated in 3.1       1 days
       http://bugs.python.org/issue7343    rhettinger                    
                                                                               

wsgiref tests failing on Windows 7 buildbot                         0 days
       http://bugs.python.org/issue7344    amaury.forgeotdarc            
       buildbot                                                                

distutils.tests.test_msvc9compiler - dragfullwindows can have va    0 days
       http://bugs.python.org/issue7354    tarek                         
       patch, patch, buildbot                                                  

Incorrect error message with def((x)=0)                             0 days
       http://bugs.python.org/issue7362    benjamin.peterson             
                                                                               

Inconsistent indentation in SocketServer docs                       1 days
       http://bugs.python.org/issue7363    r.david.murray                
       patch                                                                   

Traceback contains incorrect line number for duplicate argument     0 days
       http://bugs.python.org/issue7364    benjamin.peterson             
                                                                               

Internal error in threading.py                                      0 days
       http://bugs.python.org/issue7368    r.david.murray                
                                                                               

fix for scheme identification in urllib2?                        1448 days
       http://bugs.python.org/issue1368312 orsenthil                     
       easy                                                                    

cp720 encoding map                                               1066 days
       http://bugs.python.org/issue1616979 bialix                        
       patch, needs review                                                     

[PATCH] Fix ptcp154 encoding cyrillic_asian alias                 851 days
       http://bugs.python.org/issue1757126 pjenvey                       
       patch                                                                   



Top Issues Most Discussed (10)
______________________________

 19 crasher in str(Exception())                                        8 days
open    http://bugs.python.org/issue7309   

 14 test_msvc9compiler test_reg_class failure on new Windows box       9 days
closed  http://bugs.python.org/issue7293   

 13 Provide CPython command line functionality via runpy module       79 days
closed  http://bugs.python.org/issue6816   

 12 test_multiprocessing dictionary changed size errors and hang      47 days
open    http://bugs.python.org/issue7060   

 11 test_tarfile failing (file in use when deleting) on Windows bui    1 days
closed  http://bugs.python.org/issue7341   

  9 reversed(range(x, -1, -1)) is empty when x > 1                     6 days
closed  http://bugs.python.org/issue7298   

  9 add ftp-tls support to ftplib - RFC 4217                         648 days
closed  http://bugs.python.org/issue2054   

  7 Struct incorrectly compiles format strings                         1 days
open    http://bugs.python.org/issue7355   

  7 Documentation typos found in "zipfile ??? Work with ZIP archive    2 days
open    http://bugs.python.org/issue7351   

  6 Add initgroups to the posix/os modules                             4 days
open    http://bugs.python.org/issue7333   




From glenn at nevcal.com  Fri Nov 20 21:31:00 2009
From: glenn at nevcal.com (Glenn Linderman)
Date: Fri, 20 Nov 2009 12:31:00 -0800
Subject: [Python-Dev] PyPI comments and ratings, *really*?
In-Reply-To: <20091114152921.GA28003@laurie.devork>
References: <4AFC9064.20700@v.loewis.de>	<20091113221234.GA15732@amk-desktop.matrixgroup.net>	<87bpj6t4mp.fsf@benfinney.id.au>	<200911141018.13536.steve@pearwood.info>	<hdksh9$jkt$1@ger.gmane.org>
	<20091114152921.GA28003@laurie.devork>
Message-ID: <4B06FC84.80302@nevcal.com>

On approximately 11/14/2009 7:29 AM, came the following characters from 
the keyboard of Floris Bruynooghe:
> Having a "Repository-URL", "Repository-Browse-URL" and a
> "Bug-Tracker-URL" field in PyPI would be a lot more usefule then
> comments and ratings.
>    

+1


Here's a thought... if the author supplies the above URLs, then comments 
would be prevented, in favor of using the author-supplied system.

If the author doesn't supply the above URL, or someone reports that the 
URLs don't work, then comments on PyPI would be enabled.

-- 
Glenn
------------------------------------------------------------------------
?Everyone is entitled to their own opinion, but not their own facts. In 
turn, everyone is entitled to their own opinions of the facts, but not 
their own facts based on their opinions.? -- Guy Rocha, retiring NV 
state archivist

From ben+python at benfinney.id.au  Sat Nov 21 00:06:56 2009
From: ben+python at benfinney.id.au (Ben Finney)
Date: Sat, 21 Nov 2009 10:06:56 +1100
Subject: [Python-Dev] PyPI comments and ratings, *really*?
References: <4AFC9064.20700@v.loewis.de>
	<20091113221234.GA15732@amk-desktop.matrixgroup.net>
	<87bpj6t4mp.fsf@benfinney.id.au>
	<200911141018.13536.steve@pearwood.info> <hdksh9$jkt$1@ger.gmane.org>
	<20091114152921.GA28003@laurie.devork> <4B06FC84.80302@nevcal.com>
Message-ID: <87my2gdcdr.fsf@benfinney.id.au>

Glenn Linderman <glenn at nevcal.com> writes:

> On approximately 11/14/2009 7:29 AM, came the following characters
> from the keyboard of Floris Bruynooghe:
> > Having a "Repository-URL", "Repository-Browse-URL" and a
> > "Bug-Tracker-URL" field in PyPI would be a lot more usefule then
> > comments and ratings.
>
> +1

Agreed, the above would be very useful.

> Here's a thought... if the author supplies the above URLs, then
> comments would be prevented, in favor of using the author-supplied
> system.

Others in this thread have pointed out that comments on the package
don't serve the same purpose as a bug tracker for the project. I don't
think you'll get that ?either/or? suggestion to fly.

-- 
 \              ?Programs must be written for people to read, and only |
  `\        incidentally for machines to execute.? ?Abelson & Sussman, |
_o__)              _Structure and Interpretation of Computer Programs_ |
Ben Finney


From regebro at gmail.com  Sat Nov 21 13:40:13 2009
From: regebro at gmail.com (Lennart Regebro)
Date: Sat, 21 Nov 2009 13:40:13 +0100
Subject: [Python-Dev] standard libraries don't behave like standard
	'libraries'
In-Reply-To: <c910262d0911130059g590624c0ybf74e2f32e850637@mail.gmail.com>
References: <c910262d0911120123i7c7e9dla918fff50cb5aecd@mail.gmail.com>
	<4AFC6440.1040008@v.loewis.de>
	<4222a8490911121157i622fbab8q5fe82d32fe9ea8f@mail.gmail.com>
	<56195B13-39DF-44A3-9A79-5BDAFF244F40@bud.ca>
	<c910262d0911130059g590624c0ybf74e2f32e850637@mail.gmail.com>
Message-ID: <319e029f0911210440g19472242ld274c66f3940b820@mail.gmail.com>

On Fri, Nov 13, 2009 at 09:59, Sriram Srinivasan
<naughtysriram at gmail.com> wrote:
> you were thinking wrong. If suppose this feature is introduced it doesn't
> mean python will become batteries removed!
> you can ship the python release with the 'standard library packages' already
> installed.
> so what we get here is batteries included and ability to be changed after it
> is discharged! ;)

You can. You can simply delete any files in the standard library that
you don't want.

What is your usecase?

-- 
Lennart Regebro: Python, Zope, Plone, Grok
http://regebro.wordpress.com/
+33 661 58 14 64

From mal at egenix.com  Sat Nov 21 16:06:47 2009
From: mal at egenix.com (M.-A. Lemburg)
Date: Sat, 21 Nov 2009 16:06:47 +0100
Subject: [Python-Dev] Removal of intobject.h in 3.1
Message-ID: <4B080207.2020808@egenix.com>

The wiki page for porting to 3.x says:

http://wiki.python.org/moin/PortingExtensionModulesToPy3k
"""
long/int Unification

In Python 3.0, there is only one integer type. It is called int on the Python level, but actually
corresponds to 2.x's long type. In the C-API, PyInt_* functions are replaced by their PyLong_*
neighbors. The best course of action here is using the PyInt_* functions aliased to PyLong_* found
in intobject.h.
"""

However, intobject.h is no longer available. The checkin message
for the removal says:

"""
r71697 | mark.dickinson | 2009-04-18 12:12:16 +0200 (Sat, 18 Apr 2009) | 3 lines

The comments at the top of intobject.h say that it will be removed in 3.1.
Make it so.
"""

Since package developers are just starting to port things to 3.x and
many appear to be considering supporting both 2.7 and 3.1 (including
myself), I find it a bit strange that such an import aliasing header
was removed in 3.1.

IMHO, that's not really a good way to encourage people to try to provide
a smooth upgrade to the 3.x branch. Much to the contrary. 3.x should make
it easier for developers by providing more standard helpers like
the removed intobject.h header file.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Nov 21 2009)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try our new mxODBC.Connect Python Database Interface for free ! ::::


   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
           Registered at Amtsgericht Duesseldorf: HRB 46611
               http://www.egenix.com/company/contact/

From eric at trueblade.com  Sat Nov 21 17:37:54 2009
From: eric at trueblade.com (Eric Smith)
Date: Sat, 21 Nov 2009 11:37:54 -0500
Subject: [Python-Dev] Removal of intobject.h in 3.1
In-Reply-To: <4B080207.2020808@egenix.com>
References: <4B080207.2020808@egenix.com>
Message-ID: <4B081762.8060007@trueblade.com>

M.-A. Lemburg wrote:
> Since package developers are just starting to port things to 3.x and
> many appear to be considering supporting both 2.7 and 3.1 (including
> myself), I find it a bit strange that such an import aliasing header
> was removed in 3.1.

There's some discussion of this at http://bugs.python.org/issue7353

You might want to comment there.

From sshock3 at gmail.com  Sat Nov 21 19:31:33 2009
From: sshock3 at gmail.com (Phillip Hellewell)
Date: Sat, 21 Nov 2009 11:31:33 -0700
Subject: [Python-Dev] Issue 1488943 - differ lib missing tab hinting
Message-ID: <d7d4e8f50911211031n757e755bx8d54fe7e50e186bc@mail.gmail.com>

Is there anything holding up fixing issue 1488943?  The bug was found and a
patch submitted 3 1/2 years ago.  The patch is just a single line of code
(and some comment changes):

    common = min(common, _count_leading(btags[:common], " "))

http://bugs.python.org/issue1488943

Thanks,
Phillip Hellewell
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20091121/52a036c3/attachment.htm>

From martin at v.loewis.de  Sat Nov 21 20:05:57 2009
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sat, 21 Nov 2009 20:05:57 +0100
Subject: [Python-Dev] Removal of intobject.h in 3.1
In-Reply-To: <4B080207.2020808@egenix.com>
References: <4B080207.2020808@egenix.com>
Message-ID: <4B083A15.4020901@v.loewis.de>

> IMHO, that's not really a good way to encourage people to try to provide
> a smooth upgrade to the 3.x branch. Much to the contrary. 3.x should make
> it easier for developers by providing more standard helpers like
> the removed intobject.h header file.

I think it's better than it sounds. The macros (unfortunately) allowed
to make non-obvious mistakes. Now that they are gone, people need to
really think of what precisely they want to do.

For example, consider

if (PyInt_Check(o)){
  long val = PyInt_AsLong(o);
  // process
} else if (PyLong_Check(o)) {
  long long val = PyLong_AsLongLong(o);
  // check for overflow
  // process
}

With intobject.h, this code would continue to compile, but work
incorrectly, as the second case will never be executed. It would
be better to port this as

#if Py2.x
if (PyInt_Check(o)){
  long val = PyInt_AsLong(o);
  // process
} else
#endif
if (PyLong_Check(o)) {

i.e. eliminating the int case altogether. For another example,

long foo = PyInt_AsLong(Foo);

has a hidden error in 3.x, with intobject: PyLong_AsLong might
overflow, which the 2.x case doesn't.

So eliminating intobject.h likely helps avoiding subtle errors.

Regards,
Martin

From casevh at gmail.com  Sat Nov 21 21:00:41 2009
From: casevh at gmail.com (Case Vanhorsen)
Date: Sat, 21 Nov 2009 12:00:41 -0800
Subject: [Python-Dev] Removal of intobject.h in 3.1
In-Reply-To: <4B083A15.4020901@v.loewis.de>
References: <4B080207.2020808@egenix.com> <4B083A15.4020901@v.loewis.de>
Message-ID: <99e0b9530911211200g1c4473e1s31597d519184af3@mail.gmail.com>

On Sat, Nov 21, 2009 at 11:05 AM, "Martin v. L?wis" <martin at v.loewis.de> wrote:
>> IMHO, that's not really a good way to encourage people to try to provide
>> a smooth upgrade to the 3.x branch. Much to the contrary. 3.x should make
>> it easier for developers by providing more standard helpers like
>> the removed intobject.h header file.
>
> I think it's better than it sounds. The macros (unfortunately) allowed
> to make non-obvious mistakes. Now that they are gone, people need to
> really think of what precisely they want to do.
>
> For example, consider
>
> if (PyInt_Check(o)){
> ?long val = PyInt_AsLong(o);
> ?// process
> } else if (PyLong_Check(o)) {
> ?long long val = PyLong_AsLongLong(o);
> ?// check for overflow
> ?// process
> }
>
> With intobject.h, this code would continue to compile, but work
> incorrectly, as the second case will never be executed. It would
> be better to port this as
>
> #if Py2.x
> if (PyInt_Check(o)){
> ?long val = PyInt_AsLong(o);
> ?// process
> } else
> #endif
> if (PyLong_Check(o)) {
>
> i.e. eliminating the int case altogether. For another example,
>
> long foo = PyInt_AsLong(Foo);
>
> has a hidden error in 3.x, with intobject: PyLong_AsLong might
> overflow, which the 2.x case doesn't.
>
> So eliminating intobject.h likely helps avoiding subtle errors.

FWIW, I ported gmpy to Python 3.x without using intobject.h. I'm now
using the #if Py2.x  ... #endif approach (almost) everywhere.  The
same source compiles successfully with Python 2.4 to 3.2.

Case
>
> Regards,
> Martin
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/casevh%40gmail.com
>

From s.r at visotech.at  Mon Nov 23 09:54:56 2009
From: s.r at visotech.at (Stefan Ring)
Date: Mon, 23 Nov 2009 08:54:56 +0000 (UTC)
Subject: [Python-Dev] Reworking the GIL
References: <1256502140.5621.163.camel@localhost>
Message-ID: <loom.20091123T095336-325@post.gmane.org>

Hello,

I built something very similar for my company last year, and it?s been running
flawlessly in production at a few customer sites since, with avg. CPU usage ~50%
around the clock. I even posted about it on the Python mailing list [1] where
there was almost no resonance at that time. I never posted code, though --
nobody seemed to be too interested.

I am well aware that your current work is a lot more far-reaching than what I?ve
done, which is basically just a FIFO scheduler. I even added scheduling
priorities later which don?t work too great because the amount of time used for
a "tick" can vary by several orders of magnitude, as you know.

Thought you might be interested.

Regards
Stefan

[1] http://mail.python.org/pipermail/python-dev/2008-March/077814.html
[2] http://www.bestinclass.dk/index.php/2009/10/python-vs-clojure-evolving/
[3] www.dabeaz.com/python/GIL.pdf

PS On a slightly different note, I came across some Python bashing [2] yesterday
and somehow from there to David Beazley?s presentation about the GIL [3]. While
I don?t mind the bashing, the observations about the GIL seem quite unfair to me
because David?s measurements have been made on Mac OS X with its horribly slow
pthreads functions. I was not able to measure any slowdown on Linux.



From ncoghlan at gmail.com  Mon Nov 23 11:29:58 2009
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Mon, 23 Nov 2009 20:29:58 +1000
Subject: [Python-Dev] Reworking the GIL
In-Reply-To: <loom.20091123T095336-325@post.gmane.org>
References: <1256502140.5621.163.camel@localhost>
	<loom.20091123T095336-325@post.gmane.org>
Message-ID: <4B0A6426.8030103@gmail.com>

Stefan Ring wrote:
> [2] http://www.bestinclass.dk/index.php/2009/10/python-vs-clojure-evolving/
> [3] www.dabeaz.com/python/GIL.pdf
> 
> PS On a slightly different note, I came across some Python bashing [2] yesterday
> and somehow from there to David Beazley?s presentation about the GIL [3]. While
> I don?t mind the bashing, the observations about the GIL seem quite unfair to me
> because David?s measurements have been made on Mac OS X with its horribly slow
> pthreads functions. I was not able to measure any slowdown on Linux.

We care about Mac OS X though, so even if the contention wasn't as bad
on a different OS, the Mac downsides matter.

With the GIL updates in place, it would be interesting to see that
analysis redone for 2.7/3.2 though.

Regards,
Nick.

P.S. As far as interest in the idea goes, the GIL is one of those areas
where it takes a fairly rare combination of interest, expertise and
established credibility to propose a change and get assent to it. You'll
notice that even Antoine had to resort to the "if nobody objects soon,
I'm checking this in" tactic to garner any responses. It's an area where
even those with relevant expertise still have to put aside a fair chunk
of time in order to properly review a proposed change :)

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------

From mal at egenix.com  Mon Nov 23 11:44:36 2009
From: mal at egenix.com (M.-A. Lemburg)
Date: Mon, 23 Nov 2009 11:44:36 +0100
Subject: [Python-Dev] Removal of intobject.h in 3.1
In-Reply-To: <4B081762.8060007@trueblade.com>
References: <4B080207.2020808@egenix.com> <4B081762.8060007@trueblade.com>
Message-ID: <4B0A6794.9010108@egenix.com>

Eric Smith wrote:
> M.-A. Lemburg wrote:
>> Since package developers are just starting to port things to 3.x and
>> many appear to be considering supporting both 2.7 and 3.1 (including
>> myself), I find it a bit strange that such an import aliasing header
>> was removed in 3.1.
> 
> There's some discussion of this at http://bugs.python.org/issue7353
> 
> You might want to comment there.

Thanks for pointing me to the that ticket.

Looks like Guido already commented on this, so intobject.h could
be revived in some form.

Since this file would only be used by extension modules and possibly
also include other helper macros, perhaps it'd be better to to rename
the file to py2compat.h or something along those lines ?!

We could then also have a py2compat.c to hold corresponding
C code, e.g. to provide compatibility wrappers of new APIs that
implement different semantics in 3.x.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Nov 23 2009)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try our new mxODBC.Connect Python Database Interface for free ! ::::


   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
           Registered at Amtsgericht Duesseldorf: HRB 46611
               http://www.egenix.com/company/contact/

From mal at egenix.com  Mon Nov 23 12:05:51 2009
From: mal at egenix.com (M.-A. Lemburg)
Date: Mon, 23 Nov 2009 12:05:51 +0100
Subject: [Python-Dev] Removal of intobject.h in 3.1
In-Reply-To: <4B083A15.4020901@v.loewis.de>
References: <4B080207.2020808@egenix.com> <4B083A15.4020901@v.loewis.de>
Message-ID: <4B0A6C8F.1060803@egenix.com>

"Martin v. L?wis" wrote:
>> IMHO, that's not really a good way to encourage people to try to provide
>> a smooth upgrade to the 3.x branch. Much to the contrary. 3.x should make
>> it easier for developers by providing more standard helpers like
>> the removed intobject.h header file.
> 
> I think it's better than it sounds. The macros (unfortunately) allowed
> to make non-obvious mistakes. Now that they are gone, people need to
> really think of what precisely they want to do.
> 
> For example, consider
> 
> if (PyInt_Check(o)){
>   long val = PyInt_AsLong(o);
>   // process
> } else if (PyLong_Check(o)) {
>   long long val = PyLong_AsLongLong(o);
>   // check for overflow
>   // process
> }
> 
> With intobject.h, this code would continue to compile, but work
> incorrectly, as the second case will never be executed. It would
> be better to port this as
> 
> #if Py2.x
> if (PyInt_Check(o)){
>   long val = PyInt_AsLong(o);
>   // process
> } else
> #endif
> if (PyLong_Check(o)) {
> 
> i.e. eliminating the int case altogether.

Sure, but that assumes that the original code already had support
for Python longs, which a lot of code doesn't.

In an ideal world, developers would add that code to their
extensions right away. In the real world, where developers only
have limited resources available, you'll get more 3.x ports
by making such ports as painless as possible while at the
same time not forcing them to alienate their 2.x user base.

The long support could then be added in later releases
of the extensions, giving the developers more time adapt.

> For another example,
> 
> long foo = PyInt_AsLong(Foo);
> 
> has a hidden error in 3.x, with intobject: PyLong_AsLong might
> overflow, which the 2.x case doesn't.

That's not quite true: PyInt_AsLong(obj) will try the
nb_int slot on non-integer objects which can return errors
(it returns -1 and sets the error message).

> So eliminating intobject.h likely helps avoiding subtle errors.

In the long run, yes. In the short run, other criteria are
more important, IMHO.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Nov 23 2009)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try our new mxODBC.Connect Python Database Interface for free ! ::::


   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
           Registered at Amtsgericht Duesseldorf: HRB 46611
               http://www.egenix.com/company/contact/

From s.r at visotech.at  Mon Nov 23 12:25:39 2009
From: s.r at visotech.at (Stefan Ring)
Date: Mon, 23 Nov 2009 12:25:39 +0100
Subject: [Python-Dev] Reworking the GIL
In-Reply-To: <loom.20091123T095336-325@post.gmane.org>
References: <1256502140.5621.163.camel@localhost>
	<loom.20091123T095336-325@post.gmane.org>
Message-ID: <53e878390911230325p7da0f767qfdc069c03d0357c2@mail.gmail.com>

> I built something very similar for my company last year, and it?s been running
> flawlessly in production at a few customer sites since, with avg. CPU usage ~50%
> around the clock. I even posted about it on the Python mailing list [1] where
> there was almost no resonance at that time. I never posted code, though --
> nobody seemed to be too interested.

I've never bothered to make this tidy and nice, especially the
function naming (PySpecial_*) leaves some things to be desired. It's
not too bad, though; it just doesn't have commit-ready quality. I
don't worry about this anymore, so I just post what I have. Maybe
someone can make use of it.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: py252-scheduling.patch
Type: text/x-patch
Size: 7507 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/python-dev/attachments/20091123/64ff3a4f/attachment.bin>

From martin at v.loewis.de  Mon Nov 23 13:46:07 2009
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Mon, 23 Nov 2009 13:46:07 +0100
Subject: [Python-Dev] Removal of intobject.h in 3.1
In-Reply-To: <4B0A6C8F.1060803@egenix.com>
References: <4B080207.2020808@egenix.com> <4B083A15.4020901@v.loewis.de>
	<4B0A6C8F.1060803@egenix.com>
Message-ID: <4B0A840F.5000706@v.loewis.de>

> In an ideal world, developers would add that code to their
> extensions right away. In the real world, where developers only
> have limited resources available, you'll get more 3.x ports
> by making such ports as painless as possible while at the
> same time not forcing them to alienate their 2.x user base.

Unfortunately, such 3.x code would be full of bugs.

>> For another example,
>>
>> long foo = PyInt_AsLong(Foo);
>>
>> has a hidden error in 3.x, with intobject: PyLong_AsLong might
>> overflow, which the 2.x case doesn't.
> 
> That's not quite true: PyInt_AsLong(obj) will try the
> nb_int slot on non-integer objects which can return errors
> (it returns -1 and sets the error message).

However, the 2.x code will typically assume that the object
is an int object, in which case PyInt_AsLong can never fail.
So with intobject.h, the code will happily compile, but then
fail to detect an exception at run-time - causing other
difficult-to-find bugs.

Regards,
Martin

From dickinsm at gmail.com  Mon Nov 23 18:12:14 2009
From: dickinsm at gmail.com (Mark Dickinson)
Date: Mon, 23 Nov 2009 17:12:14 +0000
Subject: [Python-Dev] Removal of intobject.h in 3.1
In-Reply-To: <4B0A6794.9010108@egenix.com>
References: <4B080207.2020808@egenix.com> <4B081762.8060007@trueblade.com>
	<4B0A6794.9010108@egenix.com>
Message-ID: <5c6f2a5d0911230912t6d3560f8m7994d990bf36fcc7@mail.gmail.com>

On Mon, Nov 23, 2009 at 10:44 AM, M.-A. Lemburg <mal at egenix.com> wrote:
> Thanks for pointing me to the that ticket.
>
> Looks like Guido already commented on this, so intobject.h could
> be revived in some form.

I'm wondering how a resurrected intobject.h should be used:  would
Linux distributors (for example) package up intobject.h with the rest
of the python-devel RPM/package/whatever so that Python extension
modules could just include it directly, or would it be better to
encourage the extension writers to make a copy of intobject.h to add
to the source for their extension?

In the first case, intobject.h would become a requirement for those
extensions, so we'd presumably end up committed to either
maintaining intobject.h for the lifetime of Python 3.x, or causing
some pain when it does eventually get deleted.

I quite like Benjamin's suggestion (in the issue tracker) of putting
it in Doc/includes.

> Since this file would only be used by extension modules and possibly
> also include other helper macros, perhaps it'd be better to to rename
> the file to py2compat.h or something along those lines ?!
>
> We could then also have a py2compat.c to hold corresponding
> C code, e.g. to provide compatibility wrappers of new APIs that
> implement different semantics in 3.x.

This is also an interesting idea.

Mark

From mal at egenix.com  Mon Nov 23 19:27:33 2009
From: mal at egenix.com (M.-A. Lemburg)
Date: Mon, 23 Nov 2009 19:27:33 +0100
Subject: [Python-Dev] Removal of intobject.h in 3.1
In-Reply-To: <5c6f2a5d0911230912t6d3560f8m7994d990bf36fcc7@mail.gmail.com>
References: <4B080207.2020808@egenix.com>
	<4B081762.8060007@trueblade.com>	<4B0A6794.9010108@egenix.com>
	<5c6f2a5d0911230912t6d3560f8m7994d990bf36fcc7@mail.gmail.com>
Message-ID: <4B0AD415.6020508@egenix.com>

Mark Dickinson wrote:
> On Mon, Nov 23, 2009 at 10:44 AM, M.-A. Lemburg <mal at egenix.com> wrote:
>> Thanks for pointing me to the that ticket.
>>
>> Looks like Guido already commented on this, so intobject.h could
>> be revived in some form.
> 
> I'm wondering how a resurrected intobject.h should be used:  would
> Linux distributors (for example) package up intobject.h with the rest
> of the python-devel RPM/package/whatever so that Python extension
> modules could just include it directly, or would it be better to
> encourage the extension writers to make a copy of intobject.h to add
> to the source for their extension?
> 
> In the first case, intobject.h would become a requirement for those
> extensions, so we'd presumably end up committed to either
> maintaining intobject.h for the lifetime of Python 3.x, or causing
> some pain when it does eventually get deleted.
> 
> I quite like Benjamin's suggestion (in the issue tracker) of putting
> it in Doc/includes.

I'm not sure whether Doc/ is a good location for such things.

In the past, we've put such files in Misc/, e.g. the old Makefile.pre.in
based extension mechanism was shipped that way:

Extension writers would simply copy the file to their extension
directory, create a Setup.in file and then have the users
run "make -f Makefile.pre.in boot" to have a Makefile created
which then built the extension.

However, I'm not sure whether using the copy mechanism really
works out that well: it's easier to just have the file(s) in
the standard Python include file locations and then have
extension writers use

#include "py2compat.h"

where necessary. Extension writer can then still use a private
copy if needed, but most of the time they are probably better
served by using the Python provided standard one.

>> Since this file would only be used by extension modules and possibly
>> also include other helper macros, perhaps it'd be better to to rename
>> the file to py2compat.h or something along those lines ?!
>>
>> We could then also have a py2compat.c to hold corresponding
>> C code, e.g. to provide compatibility wrappers of new APIs that
>> implement different semantics in 3.x.
> 
> This is also an interesting idea.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Nov 23 2009)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try our new mxODBC.Connect Python Database Interface for free ! ::::


   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
           Registered at Amtsgericht Duesseldorf: HRB 46611
               http://www.egenix.com/company/contact/

From solipsis at pitrou.net  Mon Nov 23 19:47:46 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Mon, 23 Nov 2009 18:47:46 +0000 (UTC)
Subject: [Python-Dev] Removal of intobject.h in 3.1
References: <4B080207.2020808@egenix.com> <4B081762.8060007@trueblade.com>
	<4B0A6794.9010108@egenix.com>
Message-ID: <loom.20091123T194448-613@post.gmane.org>

M.-A. Lemburg <mal <at> egenix.com> writes:
> 
> We could then also have a py2compat.c to hold corresponding
> C code, e.g. to provide compatibility wrappers of new APIs that
> implement different semantics in 3.x.

If the semantic differences are embodied in the builtin object types I wonder
how you can make a compatibility wrapper (short of reimplementing the 2.x
types). An int is not a long, a bytes is not a str is not a unicode.

I'm not sure concealing the differences between 2.x and 3.x behind such a
wrapper is a good idea. It would be better if people became aware of / learnt
about the new semantics.

Regards

Antoine.



From martin at v.loewis.de  Mon Nov 23 21:12:25 2009
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Mon, 23 Nov 2009 21:12:25 +0100
Subject: [Python-Dev] Removal of intobject.h in 3.1
In-Reply-To: <loom.20091123T194448-613@post.gmane.org>
References: <4B080207.2020808@egenix.com>
	<4B081762.8060007@trueblade.com>	<4B0A6794.9010108@egenix.com>
	<loom.20091123T194448-613@post.gmane.org>
Message-ID: <4B0AECA9.9080109@v.loewis.de>

> I'm not sure concealing the differences between 2.x and 3.x behind such a
> wrapper is a good idea. It would be better if people became aware of / learnt
> about the new semantics.

+1. When porting a larger code base, I always came up with a custom set
of macros, specific to the way the modules were written in the package I
ported. In particular for PyString_, some of the code meant to deal with
bytes, other code with text. There was no way to tell, short of reading
and understanding the code. As you'll have to review the complete code,
anyway, the savings you gain by using macros somebody else wrote are
really minor.

Regards,
Martin

From p.f.moore at gmail.com  Tue Nov 24 22:00:51 2009
From: p.f.moore at gmail.com (Paul Moore)
Date: Tue, 24 Nov 2009 21:00:51 +0000
Subject: [Python-Dev] Buildslave gets intermittent errors in the svn step
Message-ID: <79990c6b0911241300m5e472b28j819aef69537c0836@mail.gmail.com>

I'm getting occasional odd errors in the subversion step on my
buildslave (http://www.python.org/dev/buildbot/buildslaves/moore-windows):

Traceback (most recent call last):
  File "/data/buildbot/lib/python/buildbot/process/buildstep.py", line
690, in startStep
    d.addCallback(self._startStep_2)
  File "/usr/lib/python2.5/site-packages/twisted/internet/defer.py",
line 195, in addCallback
    callbackKeywords=kw)
  File "/usr/lib/python2.5/site-packages/twisted/internet/defer.py",
line 186, in addCallbacks
    self._runCallbacks()
  File "/usr/lib/python2.5/site-packages/twisted/internet/defer.py",
line 328, in _runCallbacks
    self.result = callback(self.result, *args, **kw)
--- <exception caught here> ---
  File "/data/buildbot/lib/python/buildbot/process/buildstep.py", line
721, in _startStep_2
    skip = self.start()
  File "/data/buildbot/lib/python/buildbot/steps/source.py", line 184, in start
    self.startVC(branch, revision, patch)
  File "/data/buildbot/lib/python/buildbot/steps/source.py", line 445,
in startVC
    raise BuildSlaveTooOldError(m)
buildbot.interfaces.BuildSlaveTooOldError: This buildslave
(moore-windows) does not know about multiple branches, and using
mode=update would probably build the wrong tree. Refusing to build.
Please upgrade the buildslave to buildbot-0.7.0 or newer.

This is very odd, as (a) a simple rerun fixes the issue, and (b) I'm
already on buildbot 0.7.0 or newer (can't recall the exact version).

It's a Windows XP system, if that helps.

Does anyone have any suggestions?

Thanks,
Paul.

From db3l.net at gmail.com  Tue Nov 24 23:35:17 2009
From: db3l.net at gmail.com (David Bolen)
Date: Tue, 24 Nov 2009 17:35:17 -0500
Subject: [Python-Dev] Buildslave gets intermittent errors in the svn step
References: <79990c6b0911241300m5e472b28j819aef69537c0836@mail.gmail.com>
Message-ID: <m2einn5z6i.fsf@valheru.db3l.homeip.net>

Paul Moore <p.f.moore at gmail.com> writes:

> buildbot.interfaces.BuildSlaveTooOldError: This buildslave
> (moore-windows) does not know about multiple branches, and using
> mode=update would probably build the wrong tree. Refusing to build.
> Please upgrade the buildslave to buildbot-0.7.0 or newer.
>
> This is very odd, as (a) a simple rerun fixes the issue, and (b) I'm
> already on buildbot 0.7.0 or newer (can't recall the exact version).
>
> It's a Windows XP system, if that helps.
>
> Does anyone have any suggestions?

In my experience this error just means that someone tried to manually
trigger a build but put the wrong branch information (or left it
blank) on the submission form.

-- David


From p.f.moore at gmail.com  Tue Nov 24 23:45:03 2009
From: p.f.moore at gmail.com (Paul Moore)
Date: Tue, 24 Nov 2009 22:45:03 +0000
Subject: [Python-Dev] Buildslave gets intermittent errors in the svn step
In-Reply-To: <m2einn5z6i.fsf@valheru.db3l.homeip.net>
References: <79990c6b0911241300m5e472b28j819aef69537c0836@mail.gmail.com>
	<m2einn5z6i.fsf@valheru.db3l.homeip.net>
Message-ID: <79990c6b0911241445n61e31007id7e9d19e4303a2e7@mail.gmail.com>

2009/11/24 David Bolen <db3l.net at gmail.com>:
> Paul Moore <p.f.moore at gmail.com> writes:
>
>> buildbot.interfaces.BuildSlaveTooOldError: This buildslave
>> (moore-windows) does not know about multiple branches, and using
>> mode=update would probably build the wrong tree. Refusing to build.
>> Please upgrade the buildslave to buildbot-0.7.0 or newer.
>>
>> This is very odd, as (a) a simple rerun fixes the issue, and (b) I'm
>> already on buildbot 0.7.0 or newer (can't recall the exact version).
>>
>> It's a Windows XP system, if that helps.
>>
>> Does anyone have any suggestions?
>
> In my experience this error just means that someone tried to manually
> trigger a build but put the wrong branch information (or left it
> blank) on the submission form.

Hmm, wasn't me :-) The reason field on the build is blank, so this
looks like it's an automatic build.

I'm getting odd failures with some of the tests as well, at the moment
- test_smtplib and others failing but when "Re-running failed tests in
verbose mode" there were no failures.

I wonder if it could be a flaky network connection...?

Thanks for the pointer.
Paul.

PS Is there a better list for this type of question? I'm not sure how
interesting it is to general python-dev readers.

From solipsis at pitrou.net  Tue Nov 24 23:54:52 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Tue, 24 Nov 2009 22:54:52 +0000 (UTC)
Subject: [Python-Dev] Buildslave gets intermittent errors in the svn step
References: <79990c6b0911241300m5e472b28j819aef69537c0836@mail.gmail.com>
	<m2einn5z6i.fsf@valheru.db3l.homeip.net>
	<79990c6b0911241445n61e31007id7e9d19e4303a2e7@mail.gmail.com>
Message-ID: <loom.20091124T235329-608@post.gmane.org>

Paul Moore <p.f.moore <at> gmail.com> writes:
> 
> I'm getting odd failures with some of the tests as well, at the moment
> - test_smtplib and others failing but when "Re-running failed tests in
> verbose mode" there were no failures.
> 
> I wonder if it could be a flaky network connection...?

test_smtplib is notoriously flaky.

> PS Is there a better list for this type of question? I'm not sure how
> interesting it is to general python-dev readers.

IMHO it's ok for python-dev. It's a pity buildbot issues don't generate more
interest though.

Regards

Antoine.



From db3l.net at gmail.com  Wed Nov 25 01:24:35 2009
From: db3l.net at gmail.com (David Bolen)
Date: Tue, 24 Nov 2009 19:24:35 -0500
Subject: [Python-Dev] Buildslave gets intermittent errors in the svn step
References: <79990c6b0911241300m5e472b28j819aef69537c0836@mail.gmail.com>
	<m2einn5z6i.fsf@valheru.db3l.homeip.net>
	<79990c6b0911241445n61e31007id7e9d19e4303a2e7@mail.gmail.com>
Message-ID: <m2aayb5u4c.fsf@valheru.db3l.homeip.net>

Paul Moore <p.f.moore at gmail.com> writes:

> Hmm, wasn't me :-) The reason field on the build is blank, so this
> looks like it's an automatic build.

If it's got an SVN tag indicated, and you can checkout that tag on
your slave, then it's probably not the scenario that I was thinking
of.

In my case, it's almost always me trying to initiate a test build, and
forgetting the right branch tag or what its relative to (historically
forgetting to put "branches/" in it).

> I'm getting odd failures with some of the tests as well, at the moment
> - test_smtplib and others failing but when "Re-running failed tests in
> verbose mode" there were no failures.
>
> I wonder if it could be a flaky network connection...?

I suppose.  Certainly, for this specific error, it's pretty clear that
it's not that precise, and various SVN checkout problems can probably
be trapped by the error, so it's pretty unlikely it's actually a
buildbot version error.

What you might try is locating the buildbot log entries (twistd.log)
on your slave that correspond to the failed build.  It'll show the
exact svn command being issued, and you can try it manually in a test
tree.  If it succeeds then you'll know it was some sort of transient
issue - if it fails, it ought to provide useful data as well.

> PS Is there a better list for this type of question? I'm not sure how
> interesting it is to general python-dev readers.

I don't think there's anywhere separate, and generally (recent history
being a bit different) buildbot related stuff has been very low
volume.  So this list is probably appropriate (given what the
buildbots are being used for).  I expect if it gets too noisy we'll
hear about it :-)

-- David


From martin at v.loewis.de  Wed Nov 25 07:45:11 2009
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Wed, 25 Nov 2009 07:45:11 +0100
Subject: [Python-Dev] Buildslave gets intermittent errors in the svn step
In-Reply-To: <79990c6b0911241300m5e472b28j819aef69537c0836@mail.gmail.com>
References: <79990c6b0911241300m5e472b28j819aef69537c0836@mail.gmail.com>
Message-ID: <4B0CD277.3090703@v.loewis.de>

> Does anyone have any suggestions?

I guess that's a buildbot bug. At 2009-11-23 22:14:23, your
buildslave disconnected. The master recognized it it as offline.
For some reason, it then still decided to start a build, at
22:14:24. With the slave disconnected, all master-side information
gets discarded, and it wasn't possible anymore to find out what version
of the svn command the slave implements, hence the exception.

Regards,
Martin

From g.brandl at gmx.net  Wed Nov 25 09:36:31 2009
From: g.brandl at gmx.net (Georg Brandl)
Date: Wed, 25 Nov 2009 09:36:31 +0100
Subject: [Python-Dev] Making loggerClass an attribute of the logger manager?
Message-ID: <heiq76$6ds$1@ger.gmane.org>

Hi,

recently I had a use case where I wanted to use logging in two
completely separate parts of the same process.  One of them
needs to create instances a specific Logger subclass, while the
other is fine with the default loggers.

I got around the problem of the unique root node by using two
Managers (and then using Manager.getLogger() instead of
getLogger()), but I can only set the loggerClass globally.

Making the loggerClass configurable per manager would solve the
problem for me, and AFAICS since most applications don't use
different managers anyway, there should not be any detrimental
effects.  What do you think?

cheers,
Georg


From vinay_sajip at yahoo.co.uk  Wed Nov 25 11:32:15 2009
From: vinay_sajip at yahoo.co.uk (Vinay Sajip)
Date: Wed, 25 Nov 2009 10:32:15 +0000 (UTC)
Subject: [Python-Dev] Making loggerClass an attribute of the logger
	manager?
References: <heiq76$6ds$1@ger.gmane.org>
Message-ID: <loom.20091125T112612-113@post.gmane.org>

Georg Brandl <g.brandl <at> gmx.net> writes:

> Making the loggerClass configurable per manager would solve the
> problem for me, and AFAICS since most applications don't use
> different managers anyway, there should not be any detrimental
> effects.  What do you think?

Seems reasonable. Apart from the API to set/get, _loggerClass is only used by
the manager when instantiating a new logger.

Regards,

Vinay Sajip


From vinay_sajip at yahoo.co.uk  Wed Nov 25 22:00:25 2009
From: vinay_sajip at yahoo.co.uk (Vinay Sajip)
Date: Wed, 25 Nov 2009 21:00:25 +0000 (GMT)
Subject: [Python-Dev] PEP 391 ready for review
Message-ID: <904696.84625.qm@web25805.mail.ukl.yahoo.com>

Full View
PEP 391 is, I believe, ready for review.

http://www.python.org/dev/peps/pep-0391/

I've
also worked up an implementation, though not yet checked it in: it's
available as a module "dictconfig.py" with accompanying unit tests in
"test_dictconfig.py", at:

http://bitbucket.org/vinay.sajip/dictconfig

I
believe I have incorporated into the PEP comments from people both here
and on python-list. There have been comments in favour of the proposal, no objections to the proposal as a
whole, and some questions and objections about specific details. I believe
I have addressed these with changes to the PEP.

Discussion threads on python-dev:

http://mail.python.org/pipermail/python-dev/2009-October/092695.html
http://mail.python.org/pipermail/python-dev/2009-October/092782.html
http://mail.python.org/pipermail/python-dev/2009-October/093062.html

And on python-list:

http://mail.python.org/pipermail/python-list/2009-October/1223658.html
http://mail.python.org/pipermail/python-list/2009-October/1224228.html

There hasn't been a *huge* amount of discussion, but as it's a niche topic I suppose that's fair enough.

I'm
ready to actually integrate these changes in the core logging package.
As I understand it, the next step is to put the PEP forward for review,
which is why I'm sending this mail. I've already sent an equivalent mail to the PEP editors: David Goodger suggested I incorporate the above links (done) and Brett Cannon suggested I send this to python-dev - also, now, done.

Looking forward to the feedback!

Regards,

Vinay Sajip


      

From digitalxero at gmail.com  Wed Nov 25 23:28:30 2009
From: digitalxero at gmail.com (Dj Gilcrease)
Date: Wed, 25 Nov 2009 15:28:30 -0700
Subject: [Python-Dev] PEP 391 ready for review
In-Reply-To: <904696.84625.qm@web25805.mail.ukl.yahoo.com>
References: <904696.84625.qm@web25805.mail.ukl.yahoo.com>
Message-ID: <e9764b730911251428q4319371duee3adfd56c5ea8d4@mail.gmail.com>

I would recommend removing the class keyword and replacing it with the
() as you have in the custom examples or replacing () with class so it
is uniform across all config options

handlers:
 file:
   class : logging.handlers.RotatingFileHandler
   formatter: precise
   filename: logconfig.log
   maxBytes: 1024
   backupCount: 3

 custom:
   (): my.package.MyHandler
   alternate: cfg://handlers.file

This just strikes me as odd to have to remember for built in handlers
I need to use the class keyword and for my custom handlers I need to
use (). My preference would be the class keyword vs the odd (), as
that is what I am defining, the class to be used for this handler.



The ext:// I think should be py:// when defining objects to be access
via pythons normal import mechanisms and ext:// for resolving external
processes or system calls

From tjreedy at udel.edu  Thu Nov 26 00:20:08 2009
From: tjreedy at udel.edu (Terry Reedy)
Date: Wed, 25 Nov 2009 18:20:08 -0500
Subject: [Python-Dev] PEP 391 ready for review
In-Reply-To: <904696.84625.qm@web25805.mail.ukl.yahoo.com>
References: <904696.84625.qm@web25805.mail.ukl.yahoo.com>
Message-ID: <heke3h$mh0$1@ger.gmane.org>

Vinay Sajip wrote:
> Full View
> PEP 391 is, I believe, ready for review.
> 
> http://www.python.org/dev/peps/pep-0391/

This is my first reading of this. Comments:

nit: I believe "both of these serialization formats allow 
deserialization of Python dictionaries." should be "... to Python 
dictionaries."

"bespoke configuration methods" I had to look up this mostly 
non-American British word, which I thought meant 'engaged (to be 
married, as in spoken for)' to guess that you probably mean the much 
more obscure 'custom configuration methods'. If so, please say so (in 
plain American ;-)

"The logging.config module will have the following additions:"
There is currently only one addition.

"Nothing will be returned, though exceptions will be raised "
Technically incorrect. "Returns None; exceptions ..." or just 
"Exceptions ...". The doc for print(), for instance, just says what is 
does and does not mention the default None return.

"particular logger has an attached to it a particular handler."
delete 'an'

" '()' : 'my.package.customFormatterFactory', "

This at first appeared to be a typo. You earlier wrote "the user will 
need to provide a 'factory' - a callable which is called with a 
configuration dictionary and which returns the instantiated object. This 
will be signalled by the factory being made available under the special 
key '()'. " A string is not a callable. So I thought you perhaps meant
" '()' : my.package.customFormatterFactory, " without the quotes, or
" '()' : <<my.package.customFormatterFactory>>, ", where <<object type>> 
is understood to be a place filler for an object of the type specified.

Then I see "the specified factory callable will be located using normal 
import mechanisms" If you stick with this, then
"This will be signalled by the factory being made available under the 
special key '()'."
would be clearer as
"This will be signalled by an absolute import path to the factory being 
made available under the special key '()'."

However, if the code that constructs the dict passed to dictConfig() has 
the factory in hand, and possibly no absolute import path, why not let 
it pass the function directly instead of indirectly. (I am here using 
'absolute import path' to mean one that logging can resolve, rather than 
being only resolvable from the dictConfig caller.) Being able to do so 
is one of the great features of Python. The code for dictConfig could 
easily wrap the current import.

if not hasattr(factory, '__call__'): factory = __import__(factory)


"The '()' also serves as a mnemonic that the corresponding value is a 
callable." More than just that, parentheses preceeded by an expression 
*are* a call operator, mapped to '__call__', just as the addition 
operator '+' is mapped to '__add__'. Great choice to me.

...
"If serialization is successful, then dictConfig() will be called to 
process the resulting dictionary"
deserialization


Terry Jan Reedy


From ctb at msu.edu  Thu Nov 26 04:39:22 2009
From: ctb at msu.edu (C. Titus Brown)
Date: Wed, 25 Nov 2009 19:39:22 -0800
Subject: [Python-Dev] Building a Windows MSI for Python /trunk
Message-ID: <20091126033922.GA15605@idyll.org>

Hi all,

I got an MSI build working on my WinXP VM just now, and I wanted to
touch base with whomever it is that is maintaining this (wonderful!)
set of scripts...

I ran into three problems, and I managed to figure out two of them; the third
wasn't fatal.  Note, the diff of my fixed checkout is attached.

First, the script that finds & builds the external dependencies has two
minor problems.

 * it puts Tcl in tcl-8.*, and Tk in tk-8.*, but msi.py looks for them in
   tcl8.* and tk8.* to grab the license text.  I changed the glob strings
   appropriately and that seemed to work.

 * Tix isn't downloaded/installed/built automatically like everything else,
   and msi.py looks for its license file, too.  I just removed the
   Tix reference.  I can't figure out how to build Tix appropriately; any
   tips?

Second, the buildmsi.bat file refers to python26a3.hhp instead of
python27a0.hhp.

Third, I could not get _tkinter to build properly, although it wasn't fatal
to the endeavor.  It couldn't find ..\..\tcltk\lib\tcl85.lib, although
tcl85g.lib existed.

Oh, and there were a bunch of missing commands that (as a non-Windows xpert) I
had to figure out with google -- things like nasm/nasmw, for example.  Are
these documented somewhere, or would it be helpful to document them?  I think I
had to install:

 - Microsoft HTML Help Compiler
 - cygwin with make and python2.5 to build the docs
 - nasm (and copy nasm.exe to nasmw.exe)
 - cabarc

Errm, and the 'buildmsi.bat' file has 'build' misspelled as 'buold' ;)

I'd love to get this build process working completely automatically and
100% correctly, too.

Hat tip to Trent Nelson, who helped me figure out where the scripts are
and what other things I needed...

cheers,
--titus
-- 
C. Titus Brown, ctb at msu.edu
-------------- next part --------------
A non-text attachment was scrubbed...
Name: msi-build.diff
Type: text/x-diff
Size: 1345 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/python-dev/attachments/20091125/d53834f1/attachment.diff>

From vinay_sajip at yahoo.co.uk  Thu Nov 26 09:01:53 2009
From: vinay_sajip at yahoo.co.uk (Vinay Sajip)
Date: Thu, 26 Nov 2009 08:01:53 +0000 (UTC)
Subject: [Python-Dev] PEP 391 ready for review
References: <904696.84625.qm@web25805.mail.ukl.yahoo.com>
	<heke3h$mh0$1@ger.gmane.org>
Message-ID: <loom.20091126T085925-130@post.gmane.org>

Terry Reedy <tjreedy <at> udel.edu> writes:

> 
[snip]

Terry,

Thanks for your helpful comments. I have checked in some changes to the PEP
(r76533) which take into account the comments you made.

Regards,

Vinay Sajip


From vinay_sajip at yahoo.co.uk  Thu Nov 26 09:09:56 2009
From: vinay_sajip at yahoo.co.uk (Vinay Sajip)
Date: Thu, 26 Nov 2009 08:09:56 +0000 (UTC)
Subject: [Python-Dev] PEP 391 ready for review
References: <904696.84625.qm@web25805.mail.ukl.yahoo.com>
	<e9764b730911251428q4319371duee3adfd56c5ea8d4@mail.gmail.com>
Message-ID: <loom.20091126T090204-703@post.gmane.org>

Dj Gilcrease <digitalxero <at> gmail.com> writes:

> 
> I would recommend removing the class keyword and replacing it with the
> () as you have in the custom examples or replacing () with class so it
> is uniform across all config options
[snip]
> 
> This just strikes me as odd to have to remember for built in handlers
> I need to use the class keyword and for my custom handlers I need to
> use (). My preference would be the class keyword vs the odd (), as
> that is what I am defining, the class to be used for this handler.

If you are using the built in handlers, you will be using a class - hence the
key 'class' is used. If you are using a custom factory, then the system does not
force you to use a class - you can use any callable (e.g. a function which
constructs an instance and sets some attributes of it before returning it), and
the use of '()' indicates that you're not being forced to use a class. Of course
you can, since a class is a callable, but you're not restricted to classes here.

> The ext:// I think should be py:// when defining objects to be access
> via pythons normal import mechanisms and ext:// for resolving external
> processes or system calls
> 

The mechanism I proposed can be extended or changed as you suggest according to
the norms of a specific developer environment (e.g. organization policies), but
the basic system as I see it doesn't (and shouldn't) care about whether a
specific value resolves to an internal (e.g. provided by stdlib) value or an
external (e.g. provided by 3rd-party lib) value. The resolution process would be
exactly the same in either case. Or perhaps I misunderstood what you meant?



From martin at v.loewis.de  Thu Nov 26 09:54:35 2009
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Thu, 26 Nov 2009 09:54:35 +0100
Subject: [Python-Dev] Building a Windows MSI for Python /trunk
In-Reply-To: <20091126033922.GA15605@idyll.org>
References: <20091126033922.GA15605@idyll.org>
Message-ID: <4B0E424B.9000204@v.loewis.de>

> First, the script that finds & builds the external dependencies has two
> minor problems.
> 
>  * it puts Tcl in tcl-8.*, and Tk in tk-8.*, but msi.py looks for them in
>    tcl8.* and tk8.* to grab the license text.  I changed the glob strings
>    appropriately and that seemed to work.

Not sure what "it" is that put it there; perhaps you are referring to
the Tools/buildbot scripts?

Unfortunately, the naming convention that these scripts establish
doesn't quite work, as Tix would fail to build under these conventions.
So in my own checkout, I manually renamed the trees, and that's what
msi.py refers to.

>  * Tix isn't downloaded/installed/built automatically like everything else,
>    and msi.py looks for its license file, too.  I just removed the
>    Tix reference.  I can't figure out how to build Tix appropriately; any
>    tips?

See above. I keep forgetting how to build Tix; one set of command lines
is in PCbuild/build_tkinter.py.

> Second, the buildmsi.bat file refers to python26a3.hhp instead of
> python27a0.hhp.

Yes, that script hasn't been updated for a long time, ever since we
stopped having automated builds.

Ideally, the script would find the Python version on its own.

> Third, I could not get _tkinter to build properly, although it wasn't fatal
> to the endeavor.  It couldn't find ..\..\tcltk\lib\tcl85.lib, although
> tcl85g.lib existed.

You'll need to build release versions of Tcl/Tk/Tix.

> Oh, and there were a bunch of missing commands that (as a non-Windows xpert) I
> had to figure out with google -- things like nasm/nasmw, for example.  Are
> these documented somewhere, or would it be helpful to document them?

See PCbuild/readme.txt.

> Errm, and the 'buildmsi.bat' file has 'build' misspelled as 'buold' ;)

Thanks, fixed.

> I'd love to get this build process working completely automatically and
> 100% correctly, too.
> 
> Hat tip to Trent Nelson, who helped me figure out where the scripts are
> and what other things I needed...

Feel free to submit patches. There may be a misunderstanding, though, in
that buildmsi.bat isn't actually used for anything, and isn't "meant" to
be run by users, but instead by build slaves.

One consequence is that these scripts build in debug mode (perhaps
except for buildmsi), whereas end users would typically want to build in
release mode.

Another consequence is that different committers actually use different
procedures. Trent created much of the Tools/buildbot setup, so he is
obviously in favor of that strategy. Christian Heimes created
PCbuild/build_tkinter, so he would probably prefer to use that instead.
When I took over Windows builds from Tim Peters, PCbuild/readme.txt
was the official reference, and I try to stick to that.

Regards,
Martin

From ncoghlan at gmail.com  Thu Nov 26 12:28:49 2009
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Thu, 26 Nov 2009 21:28:49 +1000
Subject: [Python-Dev] PEP 391 ready for review
In-Reply-To: <heke3h$mh0$1@ger.gmane.org>
References: <904696.84625.qm@web25805.mail.ukl.yahoo.com>
	<heke3h$mh0$1@ger.gmane.org>
Message-ID: <4B0E6671.4080604@gmail.com>

Terry Reedy wrote:
> if not hasattr(factory, '__call__'): factory = __import__(factory)

That won't quite work since the string generally isn't referring to a
module, and due to the quirk of __import__ returning the top level
module since that is what the interpreter needs to bind to a name as
part of a normal import statement.

This would need to look more like:

if not hasattr(factory, '__call__'):
  mod_name, dot, attr = factory.rpartition('.')
  if not dot:
    raise ValueError("<something meaningful>")
  module = imputil.import_module(mod_name)
  factory = getattr(module, attr)

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------

From ncoghlan at gmail.com  Thu Nov 26 13:11:46 2009
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Thu, 26 Nov 2009 22:11:46 +1000
Subject: [Python-Dev] PEP 391 ready for review
In-Reply-To: <4B0E6671.4080604@gmail.com>
References: <904696.84625.qm@web25805.mail.ukl.yahoo.com>
	<heke3h$mh0$1@ger.gmane.org> <4B0E6671.4080604@gmail.com>
Message-ID: <4B0E7082.8010704@gmail.com>

Nick Coghlan wrote:
> Terry Reedy wrote:
>> if not hasattr(factory, '__call__'): factory = __import__(factory)
> 
> That won't quite work since the string generally isn't referring to a
> module, and due to the quirk of __import__ returning the top level
> module since that is what the interpreter needs to bind to a name as
> part of a normal import statement.
> 
> This would need to look more like:
> 
> if not hasattr(factory, '__call__'):
>   mod_name, dot, attr = factory.rpartition('.')
>   if not dot:
>     raise ValueError("<something meaningful>")
>   module = imputil.import_module(mod_name)
>   factory = getattr(module, attr)

s/imputil/importlib/ (Oops...)

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------

From ctb at msu.edu  Thu Nov 26 15:03:27 2009
From: ctb at msu.edu (C. Titus Brown)
Date: Thu, 26 Nov 2009 06:03:27 -0800
Subject: [Python-Dev] Building a Windows MSI for Python /trunk
In-Reply-To: <4B0E424B.9000204@v.loewis.de>
References: <20091126033922.GA15605@idyll.org> <4B0E424B.9000204@v.loewis.de>
Message-ID: <20091126140327.GB7370@idyll.org>

On Thu, Nov 26, 2009 at 09:54:35AM +0100, "Martin v. L?wis" wrote:
> > First, the script that finds & builds the external dependencies has two
> > minor problems.
> > 
> >  * it puts Tcl in tcl-8.*, and Tk in tk-8.*, but msi.py looks for them in
> >    tcl8.* and tk8.* to grab the license text.  I changed the glob strings
> >    appropriately and that seemed to work.
> 
> Not sure what "it" is that put it there; perhaps you are referring to
> the Tools/buildbot scripts?

Yes, sorry, Tools/buildbot/external*.bat, as in the diff.

> Unfortunately, the naming convention that these scripts establish
> doesn't quite work, as Tix would fail to build under these conventions.
> So in my own checkout, I manually renamed the trees, and that's what
> msi.py refers to.

OK, thanks!

> >  * Tix isn't downloaded/installed/built automatically like everything else,
> >    and msi.py looks for its license file, too.  I just removed the
> >    Tix reference.  I can't figure out how to build Tix appropriately; any
> >    tips?
> 
> See above. I keep forgetting how to build Tix; one set of command lines
> is in PCbuild/build_tkinter.py.

Ok, thanks!

> > Second, the buildmsi.bat file refers to python26a3.hhp instead of
> > python27a0.hhp.
> 
> Yes, that script hasn't been updated for a long time, ever since we
> stopped having automated builds.
> 
> Ideally, the script would find the Python version on its own.

Right...

> > Third, I could not get _tkinter to build properly, although it wasn't fatal
> > to the endeavor.  It couldn't find ..\..\tcltk\lib\tcl85.lib, although
> > tcl85g.lib existed.
> 
> You'll need to build release versions of Tcl/Tk/Tix.

Yes, I saw a reference to that online, but I wasn't sure if that was the
problem -- is the naming convention really that 'tcl85g.lib' is the debug
lib!?

> > Oh, and there were a bunch of missing commands that (as a non-Windows xpert) I
> > had to figure out with google -- things like nasm/nasmw, for example.  Are
> > these documented somewhere, or would it be helpful to document them?
> 
> See PCbuild/readme.txt.

OK, thanks.

> > I'd love to get this build process working completely automatically and
> > 100% correctly, too.
> > 
> > Hat tip to Trent Nelson, who helped me figure out where the scripts are
> > and what other things I needed...
> 
> Feel free to submit patches. There may be a misunderstanding, though, in
> that buildmsi.bat isn't actually used for anything, and isn't "meant" to
> be run by users, but instead by build slaves.

Well, yes, although wouldn't we want the scripts to run without any user
editing, for the build slaves?

> One consequence is that these scripts build in debug mode (perhaps
> except for buildmsi), whereas end users would typically want to build in
> release mode.

Right.

Is there a reason the buildbot builds were stopped? I've added binary
file uploads to pony-build, so that platform-specific files can be
uploaded to build results -- see, e.g.

	http://lyorn.idyll.org/ctb/pb-dev/p/python/10768/files/

or

	http://lyorn.idyll.org/ctb/pb-dev/p/pygr/10805/files/

and I thought daily -latest builds for Python latest on Windows and Mac would
be a good demo...

Incidentally, the pony-build results & file upload scripts can be used from
within buildbot, if people are interested.

> Another consequence is that different committers actually use different
> procedures. Trent created much of the Tools/buildbot setup, so he is
> obviously in favor of that strategy. Christian Heimes created
> PCbuild/build_tkinter, so he would probably prefer to use that instead.
> When I took over Windows builds from Tim Peters, PCbuild/readme.txt
> was the official reference, and I try to stick to that.

OK, I see.  Thanks!  So if I can bring the scripts into concordance with
that it might be valuable?

--titus
-- 
C. Titus Brown, ctb at msu.edu

From martin at v.loewis.de  Thu Nov 26 16:07:25 2009
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Thu, 26 Nov 2009 16:07:25 +0100
Subject: [Python-Dev] Building a Windows MSI for Python /trunk
In-Reply-To: <20091126140327.GB7370@idyll.org>
References: <20091126033922.GA15605@idyll.org> <4B0E424B.9000204@v.loewis.de>
	<20091126140327.GB7370@idyll.org>
Message-ID: <4B0E99AD.707@v.loewis.de>

>> You'll need to build release versions of Tcl/Tk/Tix.
> 
> Yes, I saw a reference to that online, but I wasn't sure if that was the
> problem -- is the naming convention really that 'tcl85g.lib' is the debug
> lib!?

Yes; Tcl apparently follows Unix conventions here rather than Windows
conventions (where the debug build would end with "d").

>> Feel free to submit patches. There may be a misunderstanding, though, in
>> that buildmsi.bat isn't actually used for anything, and isn't "meant" to
>> be run by users, but instead by build slaves.
> 
> Well, yes, although wouldn't we want the scripts to run without any user
> editing, for the build slaves?

Sure. For the scripts that are in actual use, that's also the case. I
stopped working on buildmsi because it was just too tedious to get
right.

> Is there a reason the buildbot builds were stopped?

It was too difficult to get right, plus nobody was interested in
using the daily MSI files.

At some point, it was broken for several months (IIRC), with nobody
reporting that breakage. So when we noticed, we just turned the service
off.

> and I thought daily -latest builds for Python latest on Windows and Mac would
> be a good demo...

I'll guess that you will find the same: nobody is really interested
in *using* daily builds of Python.

> OK, I see.  Thanks!  So if I can bring the scripts into concordance with
> that it might be valuable?

Most certainly, yes.

Regards,
Martin


From cesare.di.mauro at gmail.com  Thu Nov 26 22:50:42 2009
From: cesare.di.mauro at gmail.com (Cesare Di Mauro)
Date: Thu, 26 Nov 2009 22:50:42 +0100
Subject: [Python-Dev] wpython is back
Message-ID: <c184dbd10911261350v1e776c9r4a58b403288f9367@mail.gmail.com>

Hi Mart

I'm back with some news about wpython. I completed all the work that I was
committed to do till the end of the year. I made a lot of changes to the
code, that I'll report here.

First, I added several conditional compilation sections that enable or
disable almost every optimization I introduced into the project. Everything
is controlled by a new include file, wpython.h, which holds a lot of
#DEFINEs for each one of them.
Every #DEFINE has a brief explanation, and some report an example with
Python code disassembled, showing what happens.
It can be useful both to document the code (also to access to the interested
parts), and to let people test the effect of all optimizations. There are
also a couple of #DEFINEs which are useful to enable or disable all
superinstructions, or to make wpython work like CPython (with all new
optimizations and superinstructions disabled).

Full tracing support required a big effort, due to the missing
SETUP_LOOP/POP_BLOCK instructions used in FOR_ITER blocks. It was a pain in
the neck to let them work, but I think I have found a good solution fot it.
If I remember correctly, Collin asked in the past about performance with
testing enabled. I believe that speed is comparable to CPython, since I can
trace FOR_ITER blocks enter/exit with very little time spent intercepting
them; stack unrolling (for forward jumps cases) is fast too.

Restoring Python object model required much of the work. I reverted all the
changes that I made to many PyObjects, and just added some accessory code
only to a few of them. There are no more hacks, and code is quite polite;
only CodeObject required one line of code change in the hash function, to
let it calculate hash correctly for the constants tuple (because it can hold
lists and dictionaries now, which usally aren't hashable).
Every file in Include/ and Objects/ that I modified has only 1 diff (except
frameobject.c, for tracing code), so it's easy so see what is changed and
the extra helper functions that I added to introduce lists and dictionaries
in the consts tuple.

In the meanwhile I've added a little optimization for lists and dictionaries
used in for loops. Writing this:

def f():
    for x in ['a', 'b', 'c']: print x

generates the following (word)code with the previous wpython:

LOAD_CONST (['a', 'b', 'c'])
DEEP_LIST_COPY
GET_ITER
FOR_ITER

because ['a', 'b', 'c'] is a mutable object, and a copy must be made before
using it.

Now it'll be:

LOAD_CONST (['a', 'b', 'c'])
GET_ITER
FOR_ITER

So code is reduced and memory consumption too, because there's no need clone
the list. The trick works only for lists and dictionaries that holds
non-mutable objects, but I found it's a common pattern in Python code.

I've also updated the source to the latest Python 2.x version, 2.6.4.

All tests pass, both with Debug and Release code, on Visual Studio Express
with 32 bit code (I can't compile 64 bits versions with it).

There are only a few open issues.

test_syntax.py required some changes in the doctest (adding full filesystem
path) to let them pass correctly. It's really strange, but... works now!

test_compile.py has 2 tests disabled in test_compile_ast:

#['<forblock>', """for n in [1, 2, 3]:\n print n\n"""],
#[fname, fcontents],

that's because there's no support for constants (except Num_kind and
Str_kind) in the current ASTs code. However code compiles well, except that
it cannot make use of the new constant folding code.

I haven't updated Doc/library/dis.rst, which is exactly the same of CPython.
I'll do it when I stop introducing or changing opcodes.

Right now wpython requires manual patching of Include/Python-ast.h, with the
following lines:

enum _expr_kind {BoolOp_kind=1, BinOp_kind=2, UnaryOp_kind=3, Lambda_kind=4,
[...]
List_kind=18, Tuple_kind=19, Const_kind=20};

enum _expr_const {no_const=0, mutable_const=1, content_const=3,
pure_const=7};

struct _expr {

    enum _expr_kind kind;
    union {
[...]

        struct {
            object c;
            enum _expr_const constant;
        } Const;

    } v;
    int lineno;
    int col_offset;
};

They are need to let ast.c handle constants for the new constant folding
code.
I greatly appreciate any help to let it be generated automatically with ASDL
grammar.


That's all about the new code. Now the weird and stupid part. A few days I
got a new gmail account, but accidentally I removed the google account that
I've used to create the wpython at Google Code. I definitely lost project
ownership, so I can't tag the old code and put the new one in trunk.
I'll thank very much if someone that works or has contacts with Google can
ask for moving ownership from my old account (cesare at pronto do it) to my
new (the one which I've using now to write this mail), so I'll commit ASAP.
Alternatively, I need to create a new project at Google Code.

I hope that the community will appreciate the work (when I'll upload it :-).
I know that it's a young project, but I think it's mature enough to take a
look at it.
Last but not least, think about it like a starting point. I have many ideas
on how to optimize several other parts of Python, and the wordcode structure
gives me rooms to do it in an elegant and efficient way thanks to the
superinstructions (when needed).

For the next release I plan to cleanup opcode.h and ceval.c, grouping some
instructions into single superinstructions (CALL_FUNCTIONs and IMPORT_NAME),
adding a few opcodes, and tweaking a bit the VM main loop (primarily
targeted to reduce the jump-table that compilers produce for the big switch
statement).

Then I'll consider porting it to python 2.7 and/or python 3.1/3.2 if
there'll be interest and feedbacks about it.

I'm also at your disposal to discuss any detail about wpython source code,
since I know that it isn't a simple patch to apply to some files.

Cheers,
Cesare

2009/11/4 Mart S?mermaa <mrts.pydev at gmail.com>

> Thanks for the recap and for the good work on wpython!
>
> Best, eagerly waiting for the results of your work to land in mainline
> python,
> MS
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20091126/8108b7b2/attachment-0001.htm>

From db3l.net at gmail.com  Fri Nov 27 00:14:03 2009
From: db3l.net at gmail.com (David Bolen)
Date: Thu, 26 Nov 2009 18:14:03 -0500
Subject: [Python-Dev] Building a Windows MSI for Python /trunk
References: <20091126033922.GA15605@idyll.org> <4B0E424B.9000204@v.loewis.de>
	<20091126140327.GB7370@idyll.org> <4B0E99AD.707@v.loewis.de>
Message-ID: <m2638w6fr8.fsf@valheru.db3l.homeip.net>

"Martin v. L?wis" <martin at v.loewis.de> writes:

> It was too difficult to get right, plus nobody was interested in
> using the daily MSI files.
>
> At some point, it was broken for several months (IIRC), with nobody
> reporting that breakage. So when we noticed, we just turned the service
> off.

Sounds right - the daily MSIs were built on my slave from Sep 2007
into 2008 somewhere, but it had been failing for 6 months or more when
the task was turned off last December.  I'm not sure precisely when
the failures started (the buildbot status page was also down for a
while starting in March), but it had been failing for a long time.

I believe the failures at the time were tcl related - maybe some of
the MSI buildbot-related scripts getting out of date with ongoing
development or something.  Sounds like it might be the same stuff
Titus is running into.  It didn't seem worth the effort to track down
at the time, given that nobody had seemed to notice the lack of
builds.

-- David


From g.brandl at gmx.net  Fri Nov 27 00:41:56 2009
From: g.brandl at gmx.net (Georg Brandl)
Date: Fri, 27 Nov 2009 00:41:56 +0100
Subject: [Python-Dev] wpython is back
In-Reply-To: <c184dbd10911261350v1e776c9r4a58b403288f9367@mail.gmail.com>
References: <c184dbd10911261350v1e776c9r4a58b403288f9367@mail.gmail.com>
Message-ID: <hen3l6$sgn$1@ger.gmane.org>

Cesare Di Mauro schrieb:
> Hi Mart
> 
> I'm back with some news about wpython. I completed all the work that I
> was committed to do till the end of the year. I made a lot of changes to
> the code, that I'll report here..

For those of us without an elephant's memory, could you please also recap
what exactly wpython is, and maybe what the goals and intentions are?

cheers,
Georg


From guido at python.org  Fri Nov 27 00:55:01 2009
From: guido at python.org (Guido van Rossum)
Date: Thu, 26 Nov 2009 15:55:01 -0800
Subject: [Python-Dev] wpython is back
In-Reply-To: <hen3l6$sgn$1@ger.gmane.org>
References: <c184dbd10911261350v1e776c9r4a58b403288f9367@mail.gmail.com> 
	<hen3l6$sgn$1@ger.gmane.org>
Message-ID: <ca471dc20911261555q7798631u923b038411ed6456@mail.gmail.com>

On Thu, Nov 26, 2009 at 3:41 PM, Georg Brandl <g.brandl at gmx.net> wrote:
> Cesare Di Mauro schrieb:
>> Hi Mart
>>
>> I'm back with some news about wpython. I completed all the work that I
>> was committed to do till the end of the year. I made a lot of changes to
>> the code, that I'll report here..
>
> For those of us without an elephant's memory, could you please also recap
> what exactly wpython is, and maybe what the goals and intentions are?

It's a Python implementation that uses wordcode instead of bytecode.

http://code.google.com/p/wpython/

I don't see any benchmarks though.

-- 
--Guido van Rossum (python.org/~guido)

From cesare.di.mauro at gmail.com  Fri Nov 27 07:01:17 2009
From: cesare.di.mauro at gmail.com (Cesare Di Mauro)
Date: Fri, 27 Nov 2009 07:01:17 +0100
Subject: [Python-Dev] wpython is back
In-Reply-To: <ca471dc20911261555q7798631u923b038411ed6456@mail.gmail.com>
References: <c184dbd10911261350v1e776c9r4a58b403288f9367@mail.gmail.com>
	<hen3l6$sgn$1@ger.gmane.org>
	<ca471dc20911261555q7798631u923b038411ed6456@mail.gmail.com>
Message-ID: <c184dbd10911262201y285e35e0j34573251410a4a97@mail.gmail.com>

2009/11/27 Guido van Rossum <guido at python.org>

> It's a Python implementation that uses wordcode instead of bytecode.
>
> http://code.google.com/p/wpython/
>
> I don't see any benchmarks though.
>

You'll find some at page 28
here<http://wpython.googlecode.com/files/Beyond%20Bytecode%20-%20A%20Wordcode-based%20Python.pdf>
.

Mart made more interesting
ones<http://www.mail-archive.com/python-dev at python.org/msg43282.html>with
Unladen benchmarks.

Cesare
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20091127/086b7cbe/attachment.htm>

From lists at cheimes.de  Fri Nov 27 15:09:55 2009
From: lists at cheimes.de (Christian Heimes)
Date: Fri, 27 Nov 2009 15:09:55 +0100
Subject: [Python-Dev] wpython is back
In-Reply-To: <c184dbd10911262201y285e35e0j34573251410a4a97@mail.gmail.com>
References: <c184dbd10911261350v1e776c9r4a58b403288f9367@mail.gmail.com>	<hen3l6$sgn$1@ger.gmane.org>	<ca471dc20911261555q7798631u923b038411ed6456@mail.gmail.com>
	<c184dbd10911262201y285e35e0j34573251410a4a97@mail.gmail.com>
Message-ID: <4B0FDDB3.9010605@cheimes.de>

Cesare Di Mauro wrote:
> 
> You'll find some at page 28
> here<http://wpython.googlecode.com/files/Beyond%20Bytecode%20-%20A%20Wordcode-based%20Python.pdf>
> ..
> 
> Mart made more interesting
> ones<http://www.mail-archive.com/python-dev at python.org/msg43282.html>with
> Unladen benchmarks.

The PDF document sounded interesting and I was tempted to test WPython.
Unfortunately it doesn't compile on my box:

$ make
gcc -pthread -c -fno-strict-aliasing -DNDEBUG -g -fwrapv -O3 -Wall
-Wstrict-prototypes  -I. -IInclude -I./Include   -DPy_BUILD_CORE -o
Python/ast.o Python/ast.c


Python/ast.c:30: warning: ?enum _expr_const? declared inside parameter
list
Python/ast.c:30: warning: its scope is only this definition or
declaration, which is probably not what you want

Python/ast.c:335: warning: ?enum _expr_const? declared inside parameter
list
Python/ast.c:335: error: parameter 2 (?constant?) has incomplete type

Python/ast.c: In function ?Const?:

Python/ast.c:341: error: ?Const_kind? undeclared (first use in this
function)

Python/ast.c:341: error: (Each undeclared identifier is reported only
once
Python/ast.c:341: error: for each function it appears in.)

Python/ast.c:342: error: ?union <anonymous>? has no member named ?Const?

Python/ast.c:343: error: ?union <anonymous>? has no member named ?Const?

Python/ast.c: In function ?set_context?:

Python/ast.c:457: error: ?Const_kind? undeclared (first use in this
function)

Python/ast.c: At top level:

Python/ast.c:591: warning: ?enum _expr_const? declared inside parameter
list
Python/ast.c:590: error: conflicting types for ?seq_for_testlist?

Python/ast.c:29: note: previous declaration of ?seq_for_testlist? was here
[...]

$ gcc --version
gcc (Ubuntu 4.4.1-4ubuntu8) 4.4.1
$ uname -a
Linux hamiller 2.6.31-14-generic #48-Ubuntu SMP Fri Oct 16 14:05:01 UTC
2009 x86_64 GNU/Linux


From cesare.di.mauro at gmail.com  Fri Nov 27 15:18:24 2009
From: cesare.di.mauro at gmail.com (Cesare Di Mauro)
Date: Fri, 27 Nov 2009 15:18:24 +0100
Subject: [Python-Dev] wpython is back
In-Reply-To: <4B0FDDB3.9010605@cheimes.de>
References: <c184dbd10911261350v1e776c9r4a58b403288f9367@mail.gmail.com>
	<hen3l6$sgn$1@ger.gmane.org>
	<ca471dc20911261555q7798631u923b038411ed6456@mail.gmail.com>
	<c184dbd10911262201y285e35e0j34573251410a4a97@mail.gmail.com>
	<4B0FDDB3.9010605@cheimes.de>
Message-ID: <c184dbd10911270618w5823bf1era63f1b2d1cf3443d@mail.gmail.com>

2009/11/27 Christian Heimes <lists at cheimes.de>

> Cesare Di Mauro wrote:
> >
> > You'll find some at page 28
> > here<
> http://wpython.googlecode.com/files/Beyond%20Bytecode%20-%20A%20Wordcode-based%20Python.pdf
> >
> > ..
> >
> > Mart made more interesting
> > ones<http://www.mail-archive.com/python-dev at python.org/msg43282.html
> >with
> > Unladen benchmarks.
>
> The PDF document sounded interesting and I was tempted to test WPython.
> Unfortunately it doesn't compile on my box:
>
>  [...]


That's because Include/Python-ast.h file is autogenerated from the ASDL
grammar file the first time that you try to compile wpython.

You need to replace it with the one bundled with wpython.

It's a known problem that I'll address ASAP.

Cesare
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20091127/c5682b78/attachment.htm>

From status at bugs.python.org  Fri Nov 27 18:07:14 2009
From: status at bugs.python.org (Python tracker)
Date: Fri, 27 Nov 2009 18:07:14 +0100 (CET)
Subject: [Python-Dev] Summary of Python tracker Issues
Message-ID: <20091127170714.1913878608@psf.upfronthosting.co.za>


ACTIVITY SUMMARY (11/20/09 - 11/27/09)
Python tracker at http://bugs.python.org/

To view or respond to any of the issues listed below, click on the issue 
number.  Do NOT respond to this message.


 2506 open (+19) / 16719 closed (+14) / 19225 total (+33)

Open issues with patches:   999

Average duration of open issues: 689 days.
Median duration of open issues: 444 days.

Open Issues Breakdown
   open  2471 (+19)
pending    34 ( +0)

Issues Created Or Reopened (34)
_______________________________

multiprocessing logging support test                             11/24/09
       http://bugs.python.org/issue6615    reopened jnoller                       
       patch                                                                   

Windows 64-bit build "skips" several projects in VS 2008         11/20/09
CLOSED http://bugs.python.org/issue7371    created  srid                          
                                                                               

Regression in pstats                                             11/21/09
       http://bugs.python.org/issue7372    created  belopolsky                    
       patch                                                                   

Use PyModule_AddIntMacro() in Modules/gcmodule.c                 11/21/09
CLOSED http://bugs.python.org/issue7373    created  jon                           
       patch                                                                   

Property accessor/getter called twice                            11/22/09
CLOSED http://bugs.python.org/issue7374    created  mliddle                       
                                                                               

2to3 - does not translate urllib2 to urllib.request correctly fo 11/22/09
CLOSED http://bugs.python.org/issue7375    created  orsenthil                     
                                                                               

"python -m doctest" results in FAIL: Doctest: __main__.DebugRunn 11/22/09
       http://bugs.python.org/issue7376    created  flox                          
       patch                                                                   

Slight difference: math.floor returns an Integral                11/22/09
CLOSED http://bugs.python.org/issue7377    created  flox                          
                                                                               

unexpected truncation of traceback                               11/23/09
       http://bugs.python.org/issue7378    created  ghewgill                      
       patch                                                                   

Docstring for fractions.Fraction.limit_denominator fails         11/23/09
CLOSED http://bugs.python.org/issue7379    created  flox                          
                                                                               

uuid.UUID.bytes gives a bytearray() instead of bytes             11/23/09
       http://bugs.python.org/issue7380    created  flox                          
       patch                                                                   

subprocess.check_output: "docstring has inconsistent leading whi 11/23/09
       http://bugs.python.org/issue7381    created  flox                          
       patch                                                                   

bytes.__getnewargs__ is broken; copy.copy() therefore doesn't wo 11/23/09
       http://bugs.python.org/issue7382    created  sh                            
       patch, easy                                                             

test_multiprocessing leaks                                       11/23/09
CLOSED http://bugs.python.org/issue7383    created  pitrou                        
                                                                               

test_curses crash on FreeBSD buildbots                           11/23/09
       http://bugs.python.org/issue7384    created  mark.dickinson                
       buildbot                                                                

MemoryView_FromObject crashes if PyBuffer_GetBuffer fails        11/23/09
       http://bugs.python.org/issue7385    created  pv                            
                                                                               

More precise document on os.path.normpath()                      11/24/09
       http://bugs.python.org/issue7386    created  pengyu.ut                     
                                                                               

More precise document on os.path.normpath()                      11/24/09
CLOSED http://bugs.python.org/issue7387    created  pengyu.ut                     
                                                                               

Documentation: capitalizations of the word 'python' needed when  11/24/09
       http://bugs.python.org/issue7388    created  Retro                         
       easy                                                                    

Make decimal floating point be default, remove binary floating p 11/24/09
CLOSED http://bugs.python.org/issue7389    created  Retro                         
                                                                               

inconsistent type return                                         11/24/09
CLOSED http://bugs.python.org/issue7390    created  surkamp                       
                                                                               

Re-title the "Using Backslash to Continue Statements" anti-idiom 11/24/09
       http://bugs.python.org/issue7391    created  Horpner                       
                                                                               

cPickle test failure on release26-maint branch                   11/24/09
       http://bugs.python.org/issue7392    created  alexandre.vassalotti          
                                                                               

Executing Python program of sum of 2 nos.                        11/25/09
CLOSED http://bugs.python.org/issue7393    created  mayur78                       
                                                                               

sqlite3: some OperationalError exceptions should be ProgrammingE 11/25/09
       http://bugs.python.org/issue7394    created  dontbugme                     
                                                                               

pstats add command raises unhandled exception                    11/25/09
       http://bugs.python.org/issue7395    created  exarkun                       
                                                                               

regrtest single: iterator not subscriptable                      11/26/09
       http://bugs.python.org/issue7396    created  jgsack                        
                                                                               

__import__ docs should reference importlib.import_module         11/26/09
       http://bugs.python.org/issue7397    created  ncoghlan                      
       easy                                                                    

Issue with _PyLong_Format while trying to build Python on HP-UX  11/26/09
CLOSED http://bugs.python.org/issue7398    created  paulh_irl                     
                                                                               

Possible bug in Python Tutorial                                  11/26/09
       http://bugs.python.org/issue7399    created  dh                            
                                                                               

Small typo in Python's library documentation: set                11/26/09
CLOSED http://bugs.python.org/issue7400    created  trovao                        
       patch                                                                   

os.write and os.close on pipe from separate threads hangs on Mac 11/26/09
       http://bugs.python.org/issue7401    created  hughsw                        
                                                                               

reduce() is an anti-example in "Idioms and Anti-Idioms"          11/27/09
       http://bugs.python.org/issue7402    created  cito                          
                                                                               

Race condition in logging._acquireLock()?                        11/27/09
CLOSED http://bugs.python.org/issue7403    created  allenap                       
                                                                               



Issues Now Closed (34)
______________________

Traceback doesn't run back all the way                            412 days
       http://bugs.python.org/issue4097    benjamin.peterson             
                                                                               

compilation of sqlite3 fails                                      400 days
       http://bugs.python.org/issue4158    brett.cannon                  
                                                                               

urllib2 HTTPS connection failure (BadStatusLine Exception)        307 days
       http://bugs.python.org/issue5007    orsenthil                     
       easy                                                                    

multiprocessing example wrong                                     224 days
       http://bugs.python.org/issue5738    jnoller                       
       patch                                                                   

datetime.timedelta is inconvenient to use...                      222 days
       http://bugs.python.org/issue5788    pitrou                        
       patch                                                                   

tarfile: opening an empty tar file fails                          179 days
       http://bugs.python.org/issue6123    lars.gustaebel                
       patch                                                                   

OS X: python3 from python-3.1.dmg crashes at startup              146 days
       http://bugs.python.org/issue6393    ronaldoussoren                
       patch, needs review                                                     

expose setresuid                                                  133 days
       http://bugs.python.org/issue6508    loewis                        
       patch                                                                   

Restart support in binary upload for ftplib                        83 days
       http://bugs.python.org/issue6845    pitrou                        
       patch                                                                   

Backport py3k float repr to trunk                                  43 days
       http://bugs.python.org/issue7117    mark.dickinson                
       patch                                                                   

cPickle looking for non-existent package copyreg                   41 days
       http://bugs.python.org/issue7128    alexandre.vassalotti          
       patch, easy                                                             

%lld for PyErr_Format (Modules/_io/bufferedio.c)                   28 days
       http://bugs.python.org/issue7228    mark.dickinson                
       patch                                                                   

global statements outside functions/methods should raise SyntaxE    6 days
       http://bugs.python.org/issue7329    rhettinger                    
                                                                               

calendar.TextCalendar().formatyear wrong argument list              5 days
       http://bugs.python.org/issue7345    georg.brandl                  
                                                                               

csv doesn't handle escaped characters properly                      2 days
       http://bugs.python.org/issue7350    tjreedy                       
                                                                               

Fibonacci example does not include 0; section 4.6                   3 days
       http://bugs.python.org/issue7369    mark.dickinson                
                                                                               

Windows 64-bit build "skips" several projects in VS 2008            5 days
       http://bugs.python.org/issue7371    srid                          
                                                                               

Use PyModule_AddIntMacro() in Modules/gcmodule.c                    1 days
       http://bugs.python.org/issue7373    benjamin.peterson             
       patch                                                                   

Property accessor/getter called twice                               0 days
       http://bugs.python.org/issue7374    mliddle                       
                                                                               

2to3 - does not translate urllib2 to urllib.request correctly fo    0 days
       http://bugs.python.org/issue7375    benjamin.peterson             
                                                                               

Slight difference: math.floor returns an Integral                   0 days
       http://bugs.python.org/issue7377    r.david.murray                
                                                                               

Docstring for fractions.Fraction.limit_denominator fails            0 days
       http://bugs.python.org/issue7379    mark.dickinson                
                                                                               

test_multiprocessing leaks                                          1 days
       http://bugs.python.org/issue7383    r.david.murray                
                                                                               

More precise document on os.path.normpath()                         0 days
       http://bugs.python.org/issue7387    benjamin.peterson             
                                                                               

Make decimal floating point be default, remove binary floating p    0 days
       http://bugs.python.org/issue7389    mark.dickinson                
                                                                               

inconsistent type return                                            0 days
       http://bugs.python.org/issue7390    pitrou                        
                                                                               

Executing Python program of sum of 2 nos.                           0 days
       http://bugs.python.org/issue7393    rhettinger                    
                                                                               

Issue with _PyLong_Format while trying to build Python on HP-UX     0 days
       http://bugs.python.org/issue7398    eric.smith                    
                                                                               

Small typo in Python's library documentation: set                   0 days
       http://bugs.python.org/issue7400    georg.brandl                  
       patch                                                                   

Race condition in logging._acquireLock()?                           0 days
       http://bugs.python.org/issue7403    vinay.sajip                   
                                                                               

Handle ungzipped man pages in rpm builds correctly               1705 days
       http://bugs.python.org/issue1169193 dmalcolm                      
       patch                                                                   

difflib.Differ() doesn't always add hints for tab characters     1288 days
       http://bugs.python.org/issue1488943 orsenthil                     
       patch                                                                   

urllib2 proxy does not work in 2.4.3                             1233 days
       http://bugs.python.org/issue1519816 orsenthil                     
                                                                               

csv module broken for unicode                                    1091 days
       http://bugs.python.org/issue1606092 r.david.murray                
                                                                               



Top Issues Most Discussed (10)
______________________________

 21 multiprocessing logging support test                               3 days
open    http://bugs.python.org/issue6615   

  8 test_multiprocessing fails consistently with 'signal 12' on Fre   21 days
open    http://bugs.python.org/issue7272   

  6 test_multiprocessing leaks                                         1 days
closed  http://bugs.python.org/issue7383   

  6 Cython compiler run crashes CPython 3.1.1 and 3.2                 38 days
open    http://bugs.python.org/issue7173   

  5 "python -m doctest" results in FAIL: Doctest: __main__.DebugRun    5 days
open    http://bugs.python.org/issue7376   

  5 Fibonacci example does not include 0; section 4.6                  3 days
closed  http://bugs.python.org/issue7369   

  5 cPickle looking for non-existent package copyreg                  41 days
closed  http://bugs.python.org/issue7128   

  5 Backport py3k float repr to trunk                                 43 days
closed  http://bugs.python.org/issue7117   

  5 bdist_rpm fails when installing man pages                       2557 days
open    http://bugs.python.org/issue644744 

  5 deepcopy doesn't copy instance methods                           730 days
open    http://bugs.python.org/issue1515   




From ctb at msu.edu  Sun Nov 29 18:32:59 2009
From: ctb at msu.edu (C. Titus Brown)
Date: Sun, 29 Nov 2009 09:32:59 -0800
Subject: [Python-Dev] /trunk test_distutils failing on Mac OS X 10.5
Message-ID: <20091129173259.GA26611@idyll.org>

Hi all,

/trunk test_distutils is failing with the following error on Mac OS X 10.5:

Here's the error:

test_distutils
test test_distutils failed -- Traceback (most recent call last):
File
"/private/tmp/tmp8UfLPT/python27/Lib/distutils/tests/test_sdist.py",
line 342, in test_make_distribution_owner_group 
    self.assertEquals(member.gid, os.getgid())
AssertionError: 0 != 20

It has been a problem for over a week, perhaps longer.  I've filed it as:

	http://bugs.python.org/issue7408

... So why am I posting this to python-dev?

I went to double-check this on the buildbots and noticed that there
aren't any Mac OS X buildbots.  I would be happy to give one or two
people remote access to my Mac OS X 10.5 iMac if someone wanted to
set up a buildbot and/or debug this issue further.

Tarek, I can give you access immediately through your lyorn account, too.

cheers,
--titus
-- 
C. Titus Brown, ctb at msu.edu

From ziade.tarek at gmail.com  Sun Nov 29 18:44:36 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Sun, 29 Nov 2009 18:44:36 +0100
Subject: [Python-Dev] /trunk test_distutils failing on Mac OS X 10.5
In-Reply-To: <20091129173259.GA26611@idyll.org>
References: <20091129173259.GA26611@idyll.org>
Message-ID: <94bdd2610911290944v10a41bf1te83b96b3bc6c6ad7@mail.gmail.com>

On Sun, Nov 29, 2009 at 6:32 PM, C. Titus Brown <ctb at msu.edu> wrote:
> Hi all,
>
> /trunk test_distutils is failing with the following error on Mac OS X 10.5:
>
> Here's the error:
>
> test_distutils
> test test_distutils failed -- Traceback (most recent call last):
> File
> "/private/tmp/tmp8UfLPT/python27/Lib/distutils/tests/test_sdist.py",
> line 342, in test_make_distribution_owner_group
> ? ?self.assertEquals(member.gid, os.getgid())
> AssertionError: 0 != 20
>
> It has been a problem for over a week, perhaps longer. ?I've filed it as:
>
> ? ? ? ?http://bugs.python.org/issue7408


I don't have this problem on my own Mac OS 10.5, I am investigating,

>
> ... So why am I posting this to python-dev?
>
> I went to double-check this on the buildbots and noticed that there
> aren't any Mac OS X buildbots.

I thought we had one at some point,

>?I would be happy to give one or two
> people remote access to my Mac OS X 10.5 iMac if someone wanted to
> set up a buildbot and/or debug this issue further.
>
> Tarek, I can give you access immediately through your lyorn account, too.

Please, yes that would be awesome

Thank you

From martin at v.loewis.de  Sun Nov 29 19:28:50 2009
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sun, 29 Nov 2009 19:28:50 +0100
Subject: [Python-Dev] /trunk test_distutils failing on Mac OS X 10.5
In-Reply-To: <20091129173259.GA26611@idyll.org>
References: <20091129173259.GA26611@idyll.org>
Message-ID: <4B12BD62.9040907@v.loewis.de>

> I went to double-check this on the buildbots and noticed that there
> aren't any Mac OS X buildbots. 

That's not true, see

http://www.python.org/dev/buildbot/builders/x86 osx.5 trunk

Regards,
Martin

From ctb at msu.edu  Sun Nov 29 19:36:40 2009
From: ctb at msu.edu (C. Titus Brown)
Date: Sun, 29 Nov 2009 10:36:40 -0800
Subject: [Python-Dev] /trunk test_distutils failing on Mac OS X 10.5
In-Reply-To: <4B12BD62.9040907@v.loewis.de>
References: <20091129173259.GA26611@idyll.org> <4B12BD62.9040907@v.loewis.de>
Message-ID: <20091129183636.GA18249@idyll.org>

On Sun, Nov 29, 2009 at 07:28:50PM +0100, "Martin v. L?wis" wrote:
> > I went to double-check this on the buildbots and noticed that there
> > aren't any Mac OS X buildbots. 
> 
> That's not true, see
> 
> http://www.python.org/dev/buildbot/builders/x86 osx.5 trunk

Ahh!  Sorry, was searching for 'mac' somewhere in the string.

Those tests are also broken but in different areas from mine.  Again,
if people would like shell access, just ask.

best,
--titus
-- 
C. Titus Brown, ctb at msu.edu

From dickinsm at gmail.com  Sun Nov 29 19:43:50 2009
From: dickinsm at gmail.com (Mark Dickinson)
Date: Sun, 29 Nov 2009 18:43:50 +0000
Subject: [Python-Dev] /trunk test_distutils failing on Mac OS X 10.5
In-Reply-To: <20091129183636.GA18249@idyll.org>
References: <20091129173259.GA26611@idyll.org> <4B12BD62.9040907@v.loewis.de>
	<20091129183636.GA18249@idyll.org>
Message-ID: <5c6f2a5d0911291043x95f8a10n24f95926a96f8076@mail.gmail.com>

On Sun, Nov 29, 2009 at 6:36 PM, C. Titus Brown <ctb at msu.edu> wrote:
> On Sun, Nov 29, 2009 at 07:28:50PM +0100, "Martin v. L?wis" wrote:
>> That's not true, see
>>
>> http://www.python.org/dev/buildbot/builders/x86 osx.5 trunk
>
> Ahh! ?Sorry, was searching for 'mac' somewhere in the string.
>
> Those tests are also broken but in different areas from mine. ?Again,
> if people would like shell access, just ask.

I think that buildbot is running OS X 10.6 (despite its name).  It
would definitely be useful to have an OS X 10.5 buildbot as well,
since there are significant differences:  most obviously, the
default build on 10.6 is an LP64 build.

I'm not seeing the test_distutils failure you report on my own
10.5 machine, for some reason.

Mark

From ctb at msu.edu  Sun Nov 29 19:47:22 2009
From: ctb at msu.edu (C. Titus Brown)
Date: Sun, 29 Nov 2009 10:47:22 -0800
Subject: [Python-Dev] /trunk test_distutils failing on Mac OS X 10.5
In-Reply-To: <5c6f2a5d0911291043x95f8a10n24f95926a96f8076@mail.gmail.com>
References: <20091129173259.GA26611@idyll.org> <4B12BD62.9040907@v.loewis.de>
	<20091129183636.GA18249@idyll.org>
	<5c6f2a5d0911291043x95f8a10n24f95926a96f8076@mail.gmail.com>
Message-ID: <20091129184722.GA32263@idyll.org>

On Sun, Nov 29, 2009 at 06:43:50PM +0000, Mark Dickinson wrote:
> On Sun, Nov 29, 2009 at 6:36 PM, C. Titus Brown <ctb at msu.edu> wrote:
> > On Sun, Nov 29, 2009 at 07:28:50PM +0100, "Martin v. L?wis" wrote:
> >> That's not true, see
> >>
> >> http://www.python.org/dev/buildbot/builders/x86 osx.5 trunk
> >
> > Ahh! ??Sorry, was searching for 'mac' somewhere in the string.
> >
> > Those tests are also broken but in different areas from mine. ??Again,
> > if people would like shell access, just ask.
> 
> I think that buildbot is running OS X 10.6 (despite its name).  It
> would definitely be useful to have an OS X 10.5 buildbot as well,
> since there are significant differences:  most obviously, the
> default build on 10.6 is an LP64 build.
> 
> I'm not seeing the test_distutils failure you report on my own
> 10.5 machine, for some reason.

Yes, neither is Tarek... I'll see if I can track down why.  I don't
think I installed anything other than the minimum necessary to build
Python, but that's a bit worrisome.

cheers,
--titus
-- 
C. Titus Brown, ctb at msu.edu

From nad at acm.org  Sun Nov 29 20:55:46 2009
From: nad at acm.org (Ned Deily)
Date: Sun, 29 Nov 2009 11:55:46 -0800
Subject: [Python-Dev] /trunk test_distutils failing on Mac OS X 10.5
References: <20091129173259.GA26611@idyll.org> <4B12BD62.9040907@v.loewis.de>
	<20091129183636.GA18249@idyll.org>
	<5c6f2a5d0911291043x95f8a10n24f95926a96f8076@mail.gmail.com>
	<20091129184722.GA32263@idyll.org>
Message-ID: <nad-2C2DDE.11554629112009@news.gmane.org>

In article <20091129184722.GA32263 at idyll.org>,
 "C. Titus Brown" <ctb at msu.edu> wrote:
> On Sun, Nov 29, 2009 at 06:43:50PM +0000, Mark Dickinson wrote:
> > I'm not seeing the test_distutils failure you report on my own
> > 10.5 machine, for some reason.
> 
> Yes, neither is Tarek... I'll see if I can track down why.  I don't
> think I installed anything other than the minimum necessary to build
> Python, but that's a bit worrisome.

I'm also seeing it on 10.5.  Investigating...

-- 
 Ned Deily,
 nad at acm.org


From ziade.tarek at gmail.com  Sun Nov 29 21:33:45 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Sun, 29 Nov 2009 21:33:45 +0100
Subject: [Python-Dev] /trunk test_distutils failing on Mac OS X 10.5
In-Reply-To: <nad-2C2DDE.11554629112009@news.gmane.org>
References: <20091129173259.GA26611@idyll.org> <4B12BD62.9040907@v.loewis.de>
	<20091129183636.GA18249@idyll.org>
	<5c6f2a5d0911291043x95f8a10n24f95926a96f8076@mail.gmail.com>
	<20091129184722.GA32263@idyll.org>
	<nad-2C2DDE.11554629112009@news.gmane.org>
Message-ID: <94bdd2610911291233h434f129dvcb7976ac87b0ae5b@mail.gmail.com>

On Sun, Nov 29, 2009 at 8:55 PM, Ned Deily <nad at acm.org> wrote:
> In article <20091129184722.GA32263 at idyll.org>,
> ?"C. Titus Brown" <ctb at msu.edu> wrote:
>> On Sun, Nov 29, 2009 at 06:43:50PM +0000, Mark Dickinson wrote:
>> > I'm not seeing the test_distutils failure you report on my own
>> > 10.5 machine, for some reason.
>>
>> Yes, neither is Tarek... I'll see if I can track down why. ?I don't
>> think I installed anything other than the minimum necessary to build
>> Python, but that's a bit worrisome.
>
> I'm also seeing it on 10.5. ?Investigating...
>

Following-up in the issue;

From db3l.net at gmail.com  Sun Nov 29 22:04:06 2009
From: db3l.net at gmail.com (David Bolen)
Date: Sun, 29 Nov 2009 16:04:06 -0500
Subject: [Python-Dev] /trunk test_distutils failing on Mac OS X 10.5
References: <20091129173259.GA26611@idyll.org> <4B12BD62.9040907@v.loewis.de>
	<20091129183636.GA18249@idyll.org>
	<5c6f2a5d0911291043x95f8a10n24f95926a96f8076@mail.gmail.com>
Message-ID: <m21vjh59h5.fsf@valheru.db3l.homeip.net>

Mark Dickinson <dickinsm at gmail.com> writes:

> I'm not seeing the test_distutils failure you report on my own
> 10.5 machine, for some reason.

I don't think it's OSX specific.  My FreeBSD slaves (both 6.4 and 7.2)
have been getting the same error.

It seems related to the sticky bit on /tmp and the fact that the
account running the build slave is in the wheel(0) group, so
directories/files created beneath /tmp get a group of 0 rather than
the user's gid.  That affects the temporary files in the distutils
test, which is expecting a default of os.getgid().

-- David


From aahz at pythoncraft.com  Mon Nov 30 17:13:28 2009
From: aahz at pythoncraft.com (Aahz)
Date: Mon, 30 Nov 2009 08:13:28 -0800
Subject: [Python-Dev] PyCon 2010 registration open!  (Early-bird Jan 6)
Message-ID: <20091130161328.GA12794@panix.com>

PyCon 2010 registration has opened!  Register by January 6 for the best
rates!

http://us.pycon.org/2010/registration/

Registering early gets you early-bird registration rates, guarantees you
the tutorials you want, and helps the PyCon volunteers plan better.

Scheduled talk and tutorial lists:
http://us.pycon.org/2010/conference/talks/
http://us.pycon.org/2010/tutorials/

We'll see you in Atlanta!  Spread the word!
-- 
Aahz (aahz at pythoncraft.com)           <*>         http://www.pythoncraft.com/

The best way to get information on Usenet is not to ask a question, but
to post the wrong information.