From martin at v.loewis.de  Wed Dec  1 00:06:47 2010
From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=)
Date: Wed, 01 Dec 2010 00:06:47 +0100
Subject: [Python-Dev] Python and the Unicode Character Database
In-Reply-To: <87r5e236hj.fsf@benfinney.id.au>
References: <AANLkTinVpySuRFkjf-Sc+7a91zrxhuQpqPQL5nM9u1w+@mail.gmail.com>	<AANLkTinQLK7+5NfEp-2nMiJeDEKRxKN9Zfs4ihv-NMyH@mail.gmail.com>	<87wrnv43v5.fsf@uwakimon.sk.tsukuba.ac.jp>	<AANLkTinAf6CstywehOGRd4G4ms6K5LtSuq4Lj0z5wFBW@mail.gmail.com>
	<87r5e236hj.fsf@benfinney.id.au>
Message-ID: <4CF58387.1000609@v.loewis.de>

Am 30.11.2010 21:24, schrieb Ben Finney:
> haiyang kang <cornsea at gmail.com> writes:
> 
>>   I think it is a little ugly to have code like this: num =
>> float("?.?"), expected result is: num = 1.1
> 
> That's a straw man, though. The string need not be a literal in the
> program; it can be input to the program.
> 
>     num = float(input_from_the_external_world)
> 
> Does that change your assessment of whether non-ASCII digits are used?

I think the OP (haiyang kang) already indicated that he finds it quite
unlikely that anybody would possibly want to enter that. You would need
a number of key strokes to enter each individual ideograph, plus you
have to press the keys for keyboard layout switching to enter the Latin
decimal separator (which you normally wouldn't use along with the Han
numerals).

Regards,
Martin

From martin at v.loewis.de  Wed Dec  1 00:09:18 2010
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Wed, 01 Dec 2010 00:09:18 +0100
Subject: [Python-Dev] Python and the Unicode Character Database
In-Reply-To: <id3umc$ps8$1@dough.gmane.org>
References: <AANLkTinVpySuRFkjf-Sc+7a91zrxhuQpqPQL5nM9u1w+@mail.gmail.com>	<AANLkTinQLK7+5NfEp-2nMiJeDEKRxKN9Zfs4ihv-NMyH@mail.gmail.com>	<87wrnv43v5.fsf@uwakimon.sk.tsukuba.ac.jp>
	<id3umc$ps8$1@dough.gmane.org>
Message-ID: <4CF5841E.6020401@v.loewis.de>

Am 30.11.2010 23:43, schrieb Terry Reedy:
> On 11/30/2010 3:23 AM, Stephen J. Turnbull wrote:
> 
>> I see no reason not to make a similar promise for numeric literals.  I
>> see no good reason to allow compatibility full-width Japanese "ASCII"
>> numerals or Arabic cursive numerals in "for i in range(...)" for
>> example.
> 
> I do not think that anyone, at least not me, has argued for anything
> other than 0-9 digits (or 0-f for hex) in literals in program code. The
> only issue is whether non-programmer *users* should be able to use their
> native digits in applications in response to input prompts.

And here, my observation stands: if they wanted to, they currently
couldn't - at least not for real numbers (and also not for integers
if they want to use grouping). So the presumed application of this
feature doesn't actually work, despite the presence of the feature it
was supposedly meant to enable.

Regards,
Martin

From tjreedy at udel.edu  Wed Dec  1 00:19:30 2010
From: tjreedy at udel.edu (Terry Reedy)
Date: Tue, 30 Nov 2010 18:19:30 -0500
Subject: [Python-Dev] Python and the Unicode Character Database
In-Reply-To: <AANLkTim+5D4L20cya2pvJdnCNPyewdXX+VBovQJq99JB@mail.gmail.com>
References: <AANLkTinVpySuRFkjf-Sc+7a91zrxhuQpqPQL5nM9u1w+@mail.gmail.com>	<20101128214311.092abd35@pitrou.net>
	<4CF2D4E9.3060607@v.loewis.de>	<4CF2F067.5020705@pearwood.info>
	<4CF354C6.9020302@v.loewis.de>	<20101129193302.115dbcd5@pitrou.net>	<AANLkTinodHhm3xfq75eM=_e7uCHcHqC7N_8VMAVbSeJ8@mail.gmail.com>	<4CF41785.5020807@v.loewis.de>
	<AANLkTim+5D4L20cya2pvJdnCNPyewdXX+VBovQJq99JB@mail.gmail.com>
Message-ID: <id40q4$3di$1@dough.gmane.org>

On 11/30/2010 10:05 AM, Alexander Belopolsky wrote:

My general answers to the questions you have raised are as follows:

1. Each new feature release should use the latest version of the UCD as 
of the first beta release (or perhaps a week or so before). New chars 
are new features and the beta period can be used to (hopefully) iron out 
any bugs introduced by a new UCD version.

2. The language specification should not be UCD version specific. Martin 
pointed out that the definition of identifiers was intentionally written 
to not be, bu referring to 'current version' or some such. On the other 
hand, the UCD version used should be programatically discoverable, 
perhaps as an attribute of sys or str.

3.. The UCD should not change in bugfix releases. New chars are new 
features. Adding them in bugfix releases will introduce gratuitous 
imcompatibilities between releases. People who want the latest Unicode 
should either upgrade to the latest Python version or patch an older 
version (but not expect core support for any problems that creates).

> Given that 2.7 will be maintained for 5 years and arguably Unicode
> Consortium takes backward compatibility very seriously, wouldn't it
> make sense to consider a backport at some point?
>
> I am sure we will soon see a bug report that the following does not
> work in 2.7: :-)
>>>> ord('\N{CAT FACE WITH WRY SMILE}')
> 128572

3 (cont). 2.7 is no different in that regard. It is feature frozen just 
like all other x.y releases. And that is the answer to any such report. 
If that code became valid in 2.7.2, for instance, it would still not 
work in 2.7 and 2.7.1. Not working is not a bug; working is a new 
feature introduced after 2.7 was released.

>>> - How specific should library reference manual be in defining methods
>>> affected by UCD such as str.upper()?
>>
>> It should specify what this actually does in Unicode terminology
>> (probably in addition to a layman's rephrase of that)
>>
>
> I opened an issue for this:
>
> http://bugs.python.org/issue10587

1,2 (cont). Good idea in general.

> I was more concerned about wide an narrow unicode CPython builds.  Is
> it a bug that   '\UXXXXXXXX'.isalpha() may disagree even when the two
> implementations are based on the same version of UCD?


4. While the difference between narrow/wide builds of (CPython) x.y 
(which should have once constant UCD) cannot be completely masked, I 
appreciate and generally agree with  your efforts to minimize them. In 
some cases, there will be a conflict/tradeoff between eliminating this 
difference versus that.

-- 
Terry Jan Reedy


From ben+python at benfinney.id.au  Wed Dec  1 01:13:23 2010
From: ben+python at benfinney.id.au (Ben Finney)
Date: Wed, 01 Dec 2010 11:13:23 +1100
Subject: [Python-Dev] Python and the Unicode Character Database
References: <AANLkTinVpySuRFkjf-Sc+7a91zrxhuQpqPQL5nM9u1w+@mail.gmail.com>
	<AANLkTinQLK7+5NfEp-2nMiJeDEKRxKN9Zfs4ihv-NMyH@mail.gmail.com>
	<87wrnv43v5.fsf@uwakimon.sk.tsukuba.ac.jp>
	<AANLkTinAf6CstywehOGRd4G4ms6K5LtSuq4Lj0z5wFBW@mail.gmail.com>
	<87r5e236hj.fsf@benfinney.id.au> <4CF58387.1000609@v.loewis.de>
Message-ID: <87k4ju2vvg.fsf@benfinney.id.au>

"Martin v. L?wis" <martin at v.loewis.de> writes:

> Am 30.11.2010 21:24, schrieb Ben Finney:
> > The string need not be a literal in the program; it can be input to
> > the program.
> > 
> >     num = float(input_from_the_external_world)
> > 
> > Does that change your assessment of whether non-ASCII digits are
> > used?
>
> I think the OP (haiyang kang) already indicated that he finds it quite
> unlikely that anybody would possibly want to enter that.

Who's talking about *entering* it into the program at a keyboard
directly, though? Input to a program can come from all kinds of crazy
sources. Just because it wasn't typed by the person at the keyboard
using this program doesn't stop it being input to the program.

A concrete example, but certainly not the only possible case: non-ASCII
digit characters representing integers, stored as text in a file.

Note that I'm not saying this is common. Nor am I saying it's a
desirable situation. I'm saying it is a feasible use case, to be
dismissed only if there is strong evidence that it's not used by
existing Python code.

-- 
 \       ?When a well-packaged web of lies has been sold to the masses |
  `\    over generations, the truth will seem utterly preposterous and |
_o__)                    its speaker a raving lunatic.? ?Dresden James |
Ben Finney


From stutzbach at google.com  Wed Dec  1 02:11:30 2010
From: stutzbach at google.com (Daniel Stutzbach)
Date: Tue, 30 Nov 2010 17:11:30 -0800
Subject: [Python-Dev] I/O ABCs
Message-ID: <AANLkTikn7-cK4ktf81gDzNh3zW5EFsNaK+q1fAM37D2s@mail.gmail.com>

The documentation for the collections Abstract Base Classes (ABCs) [1]
contains a table listing all of the collections ABCs, their parent classes,
their abstract methods, and the methods they provide.  This table makes it
very easy to figure out which methods I must override when I derive from one
of the ABCs, as well as which methods will be provided for me.

I'm working on a similar table for the I/O ABCs (
http://bugs.python.org/issue10589).  The existing documentation [2]
describes the methods of each class but doesn't describe which methods
provide a meaningful implementation and which methods a user should
override.  If I want to inherit from one of the I/O ABCs, I have to go
poking into Lib/_pyio.py to figure out which methods I need to override.

While starting to examine the I/O ABCs, I discovered that there are
some inconsistencies.  For example, RawIOBase provides .read() if the
subclass provides .readinto().  BufferedIOBase does the opposite; it
provides .readinto() if the subclass provides .read() [3].

I would like to fix some of these inconsistencies.  However, this will be a
backwards-incompatible change.  A Google Code Search suggests that the ABCs
are currently only used within the standard library [4].  Just to be clear,
the changes would NOT impact code that merely uses I/O objects; they would
only impact code that implements I/O by subclassing one of the I/O ABCs and
depending on features that are currently undocumented.

Does anyone have any categorical objections?

[1]:
http://docs.python.org/py3k/library/collections.html#abcs-abstract-base-classes
 [2]: http://docs.python.org/py3k/library/io.html#class-hierarchy
[3]: Possibly hurting performance by forcing .readinto() to perform the
extra allocations, copies, and deallocations required by .read().
[4]:
http://www.google.com/codesearch?hl=en&sa=N&q=BufferedIOBase++lang:python&ct=rr&cs_r=lang:python

-- 
Daniel Stutzbach
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20101130/a216d560/attachment.html>

From ncoghlan at gmail.com  Wed Dec  1 02:19:52 2010
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Wed, 1 Dec 2010 11:19:52 +1000
Subject: [Python-Dev] python3k : imp.find_module raises SyntaxError
In-Reply-To: <id3v0p$r97$1@dough.gmane.org>
References: <201011251530.23947.emile.anclin@logilab>
	<4CEE9B72.1070002@ronadam.com>
	<20101129115311.GD18888@lupus.logilab.fr>
	<AANLkTi=9a=Z-6SSCtx5Men071uo28-8bD8xsZHdXtY6B@mail.gmail.com>
	<id11vk$9d$1@dough.gmane.org>
	<AANLkTik1KC0wtF_Y8EPfVZ5cbLiRSeRJVCUFECzWd-hT@mail.gmail.com>
	<id3v0p$r97$1@dough.gmane.org>
Message-ID: <AANLkTi=aPBPvsYMPfjd8RvC68ucEUa0_TcPZJD9aG_xn@mail.gmail.com>

On Wed, Dec 1, 2010 at 8:48 AM, Ron Adam <rrr at ronadam.com> wrote:
> * It almost seems like the concept of a sub-module (in a package) is flawed.
> ?I'm not sure I can explain what causes me to feel that way at the moment
> though.

It isn't flawed, it is just a *lot* more complicated than most people
realise (cf. PEP 302).

In this case, the signature of find_module (returning an already open
file) is unfortunate, but probably necessary given the way the import
internals currently work. As Brett says, returning a loader would be
preferable, but the builtin import machinery doesn't have proper
loaders defined (and won't until we manage to get to the point where
importlib *is* the import machinery).

Cheers,
Nick.

-- 
Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia

From songofacandy at gmail.com  Wed Dec  1 03:30:43 2010
From: songofacandy at gmail.com (INADA Naoki)
Date: Wed, 1 Dec 2010 11:30:43 +0900
Subject: [Python-Dev] What is PyBuffer_SizeFromFormat?
Message-ID: <AANLkTik_0MCdMH8g5gK8Xb7rH8rzjDH+uGR355bFWVsh@mail.gmail.com>

PyBuffer_SizeFromFormat is documented and defined in abstract.h.
But I can't find an implementation of the function.
Do I overlook anything?

-- 
INADA Naoki? <songofacandy at gmail.com>

From ncoghlan at gmail.com  Wed Dec  1 04:51:11 2010
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Wed, 1 Dec 2010 13:51:11 +1000
Subject: [Python-Dev] What is PyBuffer_SizeFromFormat?
In-Reply-To: <AANLkTik_0MCdMH8g5gK8Xb7rH8rzjDH+uGR355bFWVsh@mail.gmail.com>
References: <AANLkTik_0MCdMH8g5gK8Xb7rH8rzjDH+uGR355bFWVsh@mail.gmail.com>
Message-ID: <AANLkTin74Du3_JJ9DP2sBAw4N93GO-xE-CcMsLg7QpNz@mail.gmail.com>

On Wed, Dec 1, 2010 at 12:30 PM, INADA Naoki <songofacandy at gmail.com> wrote:
> PyBuffer_SizeFromFormat is documented and defined in abstract.h.
> But I can't find an implementation of the function.
> Do I overlook anything?

PEP 3118 describes what it is *meant* to do. Looks like it might be
yet another thing that was missed in the implementation of that PEP
though :P

Regards,
Nick.

-- 
Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia

From rrr at ronadam.com  Wed Dec  1 06:59:18 2010
From: rrr at ronadam.com (Ron Adam)
Date: Tue, 30 Nov 2010 23:59:18 -0600
Subject: [Python-Dev] python3k : imp.find_module raises SyntaxError
In-Reply-To: <AANLkTi=aPBPvsYMPfjd8RvC68ucEUa0_TcPZJD9aG_xn@mail.gmail.com>
References: <201011251530.23947.emile.anclin@logilab>	<4CEE9B72.1070002@ronadam.com>	<20101129115311.GD18888@lupus.logilab.fr>	<AANLkTi=9a=Z-6SSCtx5Men071uo28-8bD8xsZHdXtY6B@mail.gmail.com>	<id11vk$9d$1@dough.gmane.org>	<AANLkTik1KC0wtF_Y8EPfVZ5cbLiRSeRJVCUFECzWd-hT@mail.gmail.com>	<id3v0p$r97$1@dough.gmane.org>
	<AANLkTi=aPBPvsYMPfjd8RvC68ucEUa0_TcPZJD9aG_xn@mail.gmail.com>
Message-ID: <4CF5E436.3070503@ronadam.com>



On 11/30/2010 07:19 PM, Nick Coghlan wrote:
> On Wed, Dec 1, 2010 at 8:48 AM, Ron Adam<rrr at ronadam.com>  wrote:
>> * It almost seems like the concept of a sub-module (in a package) is flawed.
>>   I'm not sure I can explain what causes me to feel that way at the moment
>> though.
>
> It isn't flawed, it is just a *lot* more complicated than most people
> realise (cf. PEP 302).

Yes, it's realising that it is a *lot* more *complicated*, that gets me. 
Flawed isn't the right word, it's rather a feeling things could have been 
simpler if perhaps some things were done differently.

Here is the gist of ideas I got from these feelings.  (Food for thought and 
YMMV and all that.)

Python doesn't have a nice way to define a collection of modules that isn't 
also a package.  So we have packages used to organise modules, and packages 
inside other packages.  A collection of modules wouldn't require importing 
a package before importing a module in it.

Another idea is, to have a way to split a large module into files, and have 
it still *be* a module, and not a package.  And also be able to tell what 
is what, by looking at the directory structure.

The train of thought these things came from is, how can we get back to 
having the directory tree have enough info in it so it's clear what is 
what?  And how can we avoid some of the *interdependent* nesting?


> In this case, the signature of find_module (returning an already open
> file) is unfortunate, but probably necessary given the way the import
> internals currently work. As Brett says, returning a loader would be
> preferable, but the builtin import machinery doesn't have proper
> loaders defined (and won't until we manage to get to the point where
> importlib *is* the import machinery).

I'll be looking forward to the new loaders. :-)

Cheers,

Ron


From rrr at ronadam.com  Wed Dec  1 06:59:18 2010
From: rrr at ronadam.com (Ron Adam)
Date: Tue, 30 Nov 2010 23:59:18 -0600
Subject: [Python-Dev] python3k : imp.find_module raises SyntaxError
In-Reply-To: <AANLkTi=aPBPvsYMPfjd8RvC68ucEUa0_TcPZJD9aG_xn@mail.gmail.com>
References: <201011251530.23947.emile.anclin@logilab>	<4CEE9B72.1070002@ronadam.com>	<20101129115311.GD18888@lupus.logilab.fr>	<AANLkTi=9a=Z-6SSCtx5Men071uo28-8bD8xsZHdXtY6B@mail.gmail.com>	<id11vk$9d$1@dough.gmane.org>	<AANLkTik1KC0wtF_Y8EPfVZ5cbLiRSeRJVCUFECzWd-hT@mail.gmail.com>	<id3v0p$r97$1@dough.gmane.org>
	<AANLkTi=aPBPvsYMPfjd8RvC68ucEUa0_TcPZJD9aG_xn@mail.gmail.com>
Message-ID: <4CF5E436.3070503@ronadam.com>



On 11/30/2010 07:19 PM, Nick Coghlan wrote:
> On Wed, Dec 1, 2010 at 8:48 AM, Ron Adam<rrr at ronadam.com>  wrote:
>> * It almost seems like the concept of a sub-module (in a package) is flawed.
>>   I'm not sure I can explain what causes me to feel that way at the moment
>> though.
>
> It isn't flawed, it is just a *lot* more complicated than most people
> realise (cf. PEP 302).

Yes, it's realising that it is a *lot* more *complicated*, that gets me. 
Flawed isn't the right word, it's rather a feeling things could have been 
simpler if perhaps some things were done differently.

Here is the gist of ideas I got from these feelings.  (Food for thought and 
YMMV and all that.)

Python doesn't have a nice way to define a collection of modules that isn't 
also a package.  So we have packages used to organise modules, and packages 
inside other packages.  A collection of modules wouldn't require importing 
a package before importing a module in it.

Another idea is, to have a way to split a large module into files, and have 
it still *be* a module, and not a package.  And also be able to tell what 
is what, by looking at the directory structure.

The train of thought these things came from is, how can we get back to 
having the directory tree have enough info in it so it's clear what is 
what?  And how can we avoid some of the *interdependent* nesting?


> In this case, the signature of find_module (returning an already open
> file) is unfortunate, but probably necessary given the way the import
> internals currently work. As Brett says, returning a loader would be
> preferable, but the builtin import machinery doesn't have proper
> loaders defined (and won't until we manage to get to the point where
> importlib *is* the import machinery).

I'll be looking forward to the new loaders. :-)

Cheers,

Ron

From ncoghlan at gmail.com  Wed Dec  1 08:41:07 2010
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Wed, 1 Dec 2010 17:41:07 +1000
Subject: [Python-Dev] python3k : imp.find_module raises SyntaxError
In-Reply-To: <4CF5E436.3070503@ronadam.com>
References: <201011251530.23947.emile.anclin@logilab>
	<4CEE9B72.1070002@ronadam.com>
	<20101129115311.GD18888@lupus.logilab.fr>
	<AANLkTi=9a=Z-6SSCtx5Men071uo28-8bD8xsZHdXtY6B@mail.gmail.com>
	<id11vk$9d$1@dough.gmane.org>
	<AANLkTik1KC0wtF_Y8EPfVZ5cbLiRSeRJVCUFECzWd-hT@mail.gmail.com>
	<id3v0p$r97$1@dough.gmane.org>
	<AANLkTi=aPBPvsYMPfjd8RvC68ucEUa0_TcPZJD9aG_xn@mail.gmail.com>
	<4CF5E436.3070503@ronadam.com>
Message-ID: <AANLkTiniDZbp-eaGMuOgn6=GACkzZwckdOKb4FpTsmBv@mail.gmail.com>

On Wed, Dec 1, 2010 at 3:59 PM, Ron Adam <rrr at ronadam.com> wrote:
> Yes, it's realising that it is a *lot* more *complicated*, that gets me.
> Flawed isn't the right word, it's rather a feeling things could have been
> simpler if perhaps some things were done differently.

*That* feeling I can understand. The import system has steadily
acquired features over time, with each addition constrained by
backwards compatibility concerns with all the past additions,
including the exotic hacks people were using to fake features that
were added more cleanly later.

For the directory-as-module-not-package idea, you could probably
implement a PEP 302 importer/loader that did that (independent of the
stdlib). It would have the advantage of avoiding a lot of the pickle
compatibility problems that a "flat package" like the new unittest
layout can cause. However, you would need to be very careful with it,
since all the files would be sharing a common globals() namespace.

Cheers,
Nick.

-- 
Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia

From mal at egenix.com  Wed Dec  1 10:15:57 2010
From: mal at egenix.com (M.-A. Lemburg)
Date: Wed, 01 Dec 2010 10:15:57 +0100
Subject: [Python-Dev] Python and the Unicode Character Database
In-Reply-To: <id40q4$3di$1@dough.gmane.org>
References: <AANLkTinVpySuRFkjf-Sc+7a91zrxhuQpqPQL5nM9u1w+@mail.gmail.com>	<20101128214311.092abd35@pitrou.net>	<4CF2D4E9.3060607@v.loewis.de>	<4CF2F067.5020705@pearwood.info>	<4CF354C6.9020302@v.loewis.de>	<20101129193302.115dbcd5@pitrou.net>	<AANLkTinodHhm3xfq75eM=_e7uCHcHqC7N_8VMAVbSeJ8@mail.gmail.com>	<4CF41785.5020807@v.loewis.de>	<AANLkTim+5D4L20cya2pvJdnCNPyewdXX+VBovQJq99JB@mail.gmail.com>
	<id40q4$3di$1@dough.gmane.org>
Message-ID: <4CF6124D.6000904@egenix.com>

Terry Reedy wrote:
> On 11/30/2010 10:05 AM, Alexander Belopolsky wrote:
> 
> My general answers to the questions you have raised are as follows:
> 
> 1. Each new feature release should use the latest version of the UCD as
> of the first beta release (or perhaps a week or so before). New chars
> are new features and the beta period can be used to (hopefully) iron out
> any bugs introduced by a new UCD version.

The UCD is versioned just like Python is, so if the Unicode Consortium
decides to ship a 5.2.1 version of the UCD, we can add that to Python 2.7.x,
since Python 2.7 started out with 5.2.0.

> 2. The language specification should not be UCD version specific. Martin
> pointed out that the definition of identifiers was intentionally written
> to not be, bu referring to 'current version' or some such. On the other
> hand, the UCD version used should be programatically discoverable,
> perhaps as an attribute of sys or str.

It already is and has been for while, e.g.

Python 2.5:
>>> import unicodedata
>>> unicodedata.unidata_version
'4.1.0'

> 3.. The UCD should not change in bugfix releases. New chars are new
> features. Adding them in bugfix releases will introduce gratuitous
> imcompatibilities between releases. People who want the latest Unicode
> should either upgrade to the latest Python version or patch an older
> version (but not expect core support for any problems that creates).

See above. Patch level revisions of the UCD are fine for patch level
releases of Python, since those patch level revisions of the UCD fix
bugs just like we do in Python.

Note that each new UCD major.minor version is a new standard on its
own, so it's perfectly ok to stick with one such standard version
per Python version.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Dec 01 2010)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try our new mxODBC.Connect Python Database Interface for free ! ::::


   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
           Registered at Amtsgericht Duesseldorf: HRB 46611
               http://www.egenix.com/company/contact/

From mal at egenix.com  Wed Dec  1 10:28:10 2010
From: mal at egenix.com (M.-A. Lemburg)
Date: Wed, 01 Dec 2010 10:28:10 +0100
Subject: [Python-Dev] Python and the Unicode Character Database
In-Reply-To: <4CF58387.1000609@v.loewis.de>
References: <AANLkTinVpySuRFkjf-Sc+7a91zrxhuQpqPQL5nM9u1w+@mail.gmail.com>	<AANLkTinQLK7+5NfEp-2nMiJeDEKRxKN9Zfs4ihv-NMyH@mail.gmail.com>	<87wrnv43v5.fsf@uwakimon.sk.tsukuba.ac.jp>	<AANLkTinAf6CstywehOGRd4G4ms6K5LtSuq4Lj0z5wFBW@mail.gmail.com>	<87r5e236hj.fsf@benfinney.id.au>
	<4CF58387.1000609@v.loewis.de>
Message-ID: <4CF6152A.3050400@egenix.com>

"Martin v. L?wis" wrote:
> Am 30.11.2010 21:24, schrieb Ben Finney:
>> haiyang kang <cornsea at gmail.com> writes:
>>
>>>   I think it is a little ugly to have code like this: num =
>>> float("?.?"), expected result is: num = 1.1
>>
>> That's a straw man, though. The string need not be a literal in the
>> program; it can be input to the program.
>>
>>     num = float(input_from_the_external_world)
>>
>> Does that change your assessment of whether non-ASCII digits are used?
> 
> I think the OP (haiyang kang) already indicated that he finds it quite
> unlikely that anybody would possibly want to enter that. You would need
> a number of key strokes to enter each individual ideograph, plus you
> have to press the keys for keyboard layout switching to enter the Latin
> decimal separator (which you normally wouldn't use along with the Han
> numerals).

That's a somewhat limited view, IMHO. Numbers are not always entered
using a computer keyboard, you have tool like cash registries, special
numeric keypads, scanners, OCR, etc. for external entry, and you also
have other programs producing such output, e.g. MS Office if configured
that way.

The argument with the decimal point doesn't work well either, since
it's obvious that float() and int() do not support localized input.

E.g. in Germany we write 3,141 instead of 3.141:

>>> float('3,141')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ValueError: invalid literal for float(): 3,141

No surprise there. The localization of the input data, e.g. removal
of thousands separators and conversion of decimal marks to the dot,
have to be done by the application, just like you have to now for
German floating point number literals.

The locale module already has locale.atof() and locale.atoi() for
just this purpose.

FYI, here's a list of decimal digits supported by Python 2.7:

http://www.unicode.org/Public/5.2.0/ucd/extracted/DerivedNumericType.txt:
"""
0030..0039    ; Decimal # Nd  [10] DIGIT ZERO..DIGIT NINE
0660..0669    ; Decimal # Nd  [10] ARABIC-INDIC DIGIT ZERO..ARABIC-INDIC DIGIT NINE
06F0..06F9    ; Decimal # Nd  [10] EXTENDED ARABIC-INDIC DIGIT ZERO..EXTENDED ARABIC-INDIC DIGIT NINE
07C0..07C9    ; Decimal # Nd  [10] NKO DIGIT ZERO..NKO DIGIT NINE
0966..096F    ; Decimal # Nd  [10] DEVANAGARI DIGIT ZERO..DEVANAGARI DIGIT NINE
09E6..09EF    ; Decimal # Nd  [10] BENGALI DIGIT ZERO..BENGALI DIGIT NINE
0A66..0A6F    ; Decimal # Nd  [10] GURMUKHI DIGIT ZERO..GURMUKHI DIGIT NINE
0AE6..0AEF    ; Decimal # Nd  [10] GUJARATI DIGIT ZERO..GUJARATI DIGIT NINE
0B66..0B6F    ; Decimal # Nd  [10] ORIYA DIGIT ZERO..ORIYA DIGIT NINE
0BE6..0BEF    ; Decimal # Nd  [10] TAMIL DIGIT ZERO..TAMIL DIGIT NINE
0C66..0C6F    ; Decimal # Nd  [10] TELUGU DIGIT ZERO..TELUGU DIGIT NINE
0CE6..0CEF    ; Decimal # Nd  [10] KANNADA DIGIT ZERO..KANNADA DIGIT NINE
0D66..0D6F    ; Decimal # Nd  [10] MALAYALAM DIGIT ZERO..MALAYALAM DIGIT NINE
0E50..0E59    ; Decimal # Nd  [10] THAI DIGIT ZERO..THAI DIGIT NINE
0ED0..0ED9    ; Decimal # Nd  [10] LAO DIGIT ZERO..LAO DIGIT NINE
0F20..0F29    ; Decimal # Nd  [10] TIBETAN DIGIT ZERO..TIBETAN DIGIT NINE
1040..1049    ; Decimal # Nd  [10] MYANMAR DIGIT ZERO..MYANMAR DIGIT NINE
1090..1099    ; Decimal # Nd  [10] MYANMAR SHAN DIGIT ZERO..MYANMAR SHAN DIGIT NINE
17E0..17E9    ; Decimal # Nd  [10] KHMER DIGIT ZERO..KHMER DIGIT NINE
1810..1819    ; Decimal # Nd  [10] MONGOLIAN DIGIT ZERO..MONGOLIAN DIGIT NINE
1946..194F    ; Decimal # Nd  [10] LIMBU DIGIT ZERO..LIMBU DIGIT NINE
19D0..19DA    ; Decimal # Nd  [11] NEW TAI LUE DIGIT ZERO..NEW TAI LUE THAM DIGIT ONE
1A80..1A89    ; Decimal # Nd  [10] TAI THAM HORA DIGIT ZERO..TAI THAM HORA DIGIT NINE
1A90..1A99    ; Decimal # Nd  [10] TAI THAM THAM DIGIT ZERO..TAI THAM THAM DIGIT NINE
1B50..1B59    ; Decimal # Nd  [10] BALINESE DIGIT ZERO..BALINESE DIGIT NINE
1BB0..1BB9    ; Decimal # Nd  [10] SUNDANESE DIGIT ZERO..SUNDANESE DIGIT NINE
1C40..1C49    ; Decimal # Nd  [10] LEPCHA DIGIT ZERO..LEPCHA DIGIT NINE
1C50..1C59    ; Decimal # Nd  [10] OL CHIKI DIGIT ZERO..OL CHIKI DIGIT NINE
A620..A629    ; Decimal # Nd  [10] VAI DIGIT ZERO..VAI DIGIT NINE
A8D0..A8D9    ; Decimal # Nd  [10] SAURASHTRA DIGIT ZERO..SAURASHTRA DIGIT NINE
A900..A909    ; Decimal # Nd  [10] KAYAH LI DIGIT ZERO..KAYAH LI DIGIT NINE
A9D0..A9D9    ; Decimal # Nd  [10] JAVANESE DIGIT ZERO..JAVANESE DIGIT NINE
AA50..AA59    ; Decimal # Nd  [10] CHAM DIGIT ZERO..CHAM DIGIT NINE
ABF0..ABF9    ; Decimal # Nd  [10] MEETEI MAYEK DIGIT ZERO..MEETEI MAYEK DIGIT NINE
FF10..FF19    ; Decimal # Nd  [10] FULLWIDTH DIGIT ZERO..FULLWIDTH DIGIT NINE
104A0..104A9  ; Decimal # Nd  [10] OSMANYA DIGIT ZERO..OSMANYA DIGIT NINE
1D7CE..1D7FF  ; Decimal # Nd  [50] MATHEMATICAL BOLD DIGIT ZERO..MATHEMATICAL MONOSPACE DIGIT NINE
"""

The Chinese and Japanese ideographs are not supported because of the
way they are defined in the Unihan database. I'm currently
investigating how we could support them as well.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Dec 01 2010)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try our new mxODBC.Connect Python Database Interface for free ! ::::


   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
           Registered at Amtsgericht Duesseldorf: HRB 46611
               http://www.egenix.com/company/contact/

From mal at egenix.com  Wed Dec  1 10:29:26 2010
From: mal at egenix.com (M.-A. Lemburg)
Date: Wed, 01 Dec 2010 10:29:26 +0100
Subject: [Python-Dev] Python and the Unicode Character Database
In-Reply-To: <id3umc$ps8$1@dough.gmane.org>
References: <AANLkTinVpySuRFkjf-Sc+7a91zrxhuQpqPQL5nM9u1w+@mail.gmail.com>	<AANLkTinQLK7+5NfEp-2nMiJeDEKRxKN9Zfs4ihv-NMyH@mail.gmail.com>	<87wrnv43v5.fsf@uwakimon.sk.tsukuba.ac.jp>
	<id3umc$ps8$1@dough.gmane.org>
Message-ID: <4CF61576.8030607@egenix.com>

Terry Reedy wrote:
> On 11/30/2010 3:23 AM, Stephen J. Turnbull wrote:
> 
>> I see no reason not to make a similar promise for numeric literals.  I
>> see no good reason to allow compatibility full-width Japanese "ASCII"
>> numerals or Arabic cursive numerals in "for i in range(...)" for
>> example.
> 
> I do not think that anyone, at least not me, has argued for anything
> other than 0-9 digits (or 0-f for hex) in literals in program code. The
> only issue is whether non-programmer *users* should be able to use their
> native digits in applications in response to input prompts.

Me neither. This is solely about Python being able to parse numeric
input in the float(), int() and complex() constructors.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Dec 01 2010)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try our new mxODBC.Connect Python Database Interface for free ! ::::


   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
           Registered at Amtsgericht Duesseldorf: HRB 46611
               http://www.egenix.com/company/contact/

From steve at pearwood.info  Wed Dec  1 10:47:45 2010
From: steve at pearwood.info (Steven D'Aprano)
Date: Wed, 01 Dec 2010 20:47:45 +1100
Subject: [Python-Dev] Python and the Unicode Character Database
In-Reply-To: <4CF5841E.6020401@v.loewis.de>
References: <AANLkTinVpySuRFkjf-Sc+7a91zrxhuQpqPQL5nM9u1w+@mail.gmail.com>	<AANLkTinQLK7+5NfEp-2nMiJeDEKRxKN9Zfs4ihv-NMyH@mail.gmail.com>	<87wrnv43v5.fsf@uwakimon.sk.tsukuba.ac.jp>	<id3umc$ps8$1@dough.gmane.org>
	<4CF5841E.6020401@v.loewis.de>
Message-ID: <4CF619C1.8030403@pearwood.info>

Martin v. L?wis wrote:
> Am 30.11.2010 23:43, schrieb Terry Reedy:
>> On 11/30/2010 3:23 AM, Stephen J. Turnbull wrote:
>>
>>> I see no reason not to make a similar promise for numeric literals.  I
>>> see no good reason to allow compatibility full-width Japanese "ASCII"
>>> numerals or Arabic cursive numerals in "for i in range(...)" for
>>> example.
>> I do not think that anyone, at least not me, has argued for anything
>> other than 0-9 digits (or 0-f for hex) in literals in program code. The
>> only issue is whether non-programmer *users* should be able to use their
>> native digits in applications in response to input prompts.
> 
> And here, my observation stands: if they wanted to, they currently
> couldn't - at least not for real numbers (and also not for integers
> if they want to use grouping). So the presumed application of this
> feature doesn't actually work, despite the presence of the feature it
> was supposedly meant to enable.

By that argument, English speakers wanting to enter integers using 
Arabic numerals can't either! I'd like to use grouping for large 
literals, if only I could think of a half-decent syntax, and if only 
Python supported it. This fails on both counts:

x = 123_456_789_012_345

The lack of grouping and the lack of a native decimal point doesn't mean 
that the feature "doesn't work" -- it merely means the feature requires 
some compromise before it can be used.

In the same way, if I wanted to enter a number using non-Arabic digits, 
it works provided I compromise by using the Anglo-American decimal point 
instead of the European comma or the native decimal point I might prefer.

The lack of support for non-dot decimal points is arguably a bug that 
should be fixed, not a reason to remove functionality.


-- 
Steven


From greg.ewing at canterbury.ac.nz  Wed Dec  1 11:22:42 2010
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Wed, 01 Dec 2010 23:22:42 +1300
Subject: [Python-Dev] python3k : imp.find_module raises SyntaxError
In-Reply-To: <AANLkTiniDZbp-eaGMuOgn6=GACkzZwckdOKb4FpTsmBv@mail.gmail.com>
References: <201011251530.23947.emile.anclin@logilab>
	<4CEE9B72.1070002@ronadam.com>
	<20101129115311.GD18888@lupus.logilab.fr>
	<AANLkTi=9a=Z-6SSCtx5Men071uo28-8bD8xsZHdXtY6B@mail.gmail.com>
	<id11vk$9d$1@dough.gmane.org>
	<AANLkTik1KC0wtF_Y8EPfVZ5cbLiRSeRJVCUFECzWd-hT@mail.gmail.com>
	<id3v0p$r97$1@dough.gmane.org>
	<AANLkTi=aPBPvsYMPfjd8RvC68ucEUa0_TcPZJD9aG_xn@mail.gmail.com>
	<4CF5E436.3070503@ronadam.com>
	<AANLkTiniDZbp-eaGMuOgn6=GACkzZwckdOKb4FpTsmBv@mail.gmail.com>
Message-ID: <4CF621F2.3060203@canterbury.ac.nz>

Nick Coghlan wrote:

> For the directory-as-module-not-package idea ...
 > you would need to be very careful with it,
> since all the files would be sharing a common globals() namespace.

One of the things I like about Python's module system
is that once I know which module a name was imported
from, I also know which file to look in for its
definition. If a module can be spread over several
files, that feature would be lost.

-- 
Greg


From ncoghlan at gmail.com  Wed Dec  1 11:39:47 2010
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Wed, 1 Dec 2010 20:39:47 +1000
Subject: [Python-Dev] python3k : imp.find_module raises SyntaxError
In-Reply-To: <4CF621F2.3060203@canterbury.ac.nz>
References: <201011251530.23947.emile.anclin@logilab>
	<4CEE9B72.1070002@ronadam.com>
	<20101129115311.GD18888@lupus.logilab.fr>
	<AANLkTi=9a=Z-6SSCtx5Men071uo28-8bD8xsZHdXtY6B@mail.gmail.com>
	<id11vk$9d$1@dough.gmane.org>
	<AANLkTik1KC0wtF_Y8EPfVZ5cbLiRSeRJVCUFECzWd-hT@mail.gmail.com>
	<id3v0p$r97$1@dough.gmane.org>
	<AANLkTi=aPBPvsYMPfjd8RvC68ucEUa0_TcPZJD9aG_xn@mail.gmail.com>
	<4CF5E436.3070503@ronadam.com>
	<AANLkTiniDZbp-eaGMuOgn6=GACkzZwckdOKb4FpTsmBv@mail.gmail.com>
	<4CF621F2.3060203@canterbury.ac.nz>
Message-ID: <AANLkTikQ8pp-HjcVWs97796sf===nJ1F-VqttT+DCjtu@mail.gmail.com>

On Wed, Dec 1, 2010 at 8:22 PM, Greg Ewing <greg.ewing at canterbury.ac.nz> wrote:
> Nick Coghlan wrote:
>
>> For the directory-as-module-not-package idea ...
>> you would need to be very careful with it,
>> since all the files would be sharing a common globals() namespace.
>
> One of the things I like about Python's module system
> is that once I know which module a name was imported
> from, I also know which file to look in for its
> definition. If a module can be spread over several
> files, that feature would be lost.

There are many potential problems with the idea, I just chose to
mention one of the ones that could easily make the affected code
*break* :)

Cheers,
Nick.

-- 
Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia

From regebro at gmail.com  Wed Dec  1 13:58:01 2010
From: regebro at gmail.com (Lennart Regebro)
Date: Wed, 1 Dec 2010 13:58:01 +0100
Subject: [Python-Dev] Python and the Unicode Character Database
In-Reply-To: <87wrnv43v5.fsf@uwakimon.sk.tsukuba.ac.jp>
References: <AANLkTinVpySuRFkjf-Sc+7a91zrxhuQpqPQL5nM9u1w+@mail.gmail.com>
	<AANLkTinQLK7+5NfEp-2nMiJeDEKRxKN9Zfs4ihv-NMyH@mail.gmail.com>
	<87wrnv43v5.fsf@uwakimon.sk.tsukuba.ac.jp>
Message-ID: <AANLkTimnar7CuHKXtvFtspTphmB3GrNMiA59zfFhTeoq@mail.gmail.com>

On Tue, Nov 30, 2010 at 09:23, Stephen J. Turnbull <stephen at xemacs.org> wrote:
> Sure you can. ?In Python program text, all keywords will be ASCII

Yes, yes, sure, but not the contents of variables,

> I see no reason not to make a similar promise for numeric literals.

Wait what, literas? The example was

>>> float('????.??')

Which doesn't have any numeric literals in them at all. Do that work?
Nope, it's a syntax error. Too badm that would have been cool, but whatever.

Why would this be a problem:

>>> T1234 = float('????.??')
>>> T1234
1234.56

But this OK?

>>> T???? = float('1234.56')
>>> T????
1234.56

I don't see that.


Should we bother to implement ????.?? as a literal equivalent to
1234.56? Well, not unless somebody askes for it, or it turns out to be
easy. :-) But that's another question.

From rrr at ronadam.com  Wed Dec  1 16:20:12 2010
From: rrr at ronadam.com (Ron Adam)
Date: Wed, 01 Dec 2010 09:20:12 -0600
Subject: [Python-Dev] python3k : imp.find_module raises SyntaxError
In-Reply-To: <AANLkTikQ8pp-HjcVWs97796sf===nJ1F-VqttT+DCjtu@mail.gmail.com>
References: <201011251530.23947.emile.anclin@logilab>	<4CEE9B72.1070002@ronadam.com>	<20101129115311.GD18888@lupus.logilab.fr>	<AANLkTi=9a=Z-6SSCtx5Men071uo28-8bD8xsZHdXtY6B@mail.gmail.com>	<id11vk$9d$1@dough.gmane.org>	<AANLkTik1KC0wtF_Y8EPfVZ5cbLiRSeRJVCUFECzWd-hT@mail.gmail.com>	<id3v0p$r97$1@dough.gmane.org>	<AANLkTi=aPBPvsYMPfjd8RvC68ucEUa0_TcPZJD9aG_xn@mail.gmail.com>	<4CF5E436.3070503@ronadam.com>	<AANLkTiniDZbp-eaGMuOgn6=GACkzZwckdOKb4FpTsmBv@mail.gmail.com>	<4CF621F2.3060203@canterbury.ac.nz>
	<AANLkTikQ8pp-HjcVWs97796sf===nJ1F-VqttT+DCjtu@mail.gmail.com>
Message-ID: <id5p3e$8d4$1@dough.gmane.org>



On 12/01/2010 04:39 AM, Nick Coghlan wrote:
> On Wed, Dec 1, 2010 at 8:22 PM, Greg Ewing<greg.ewing at canterbury.ac.nz>  wrote:
>> Nick Coghlan wrote:
>>
>>> For the directory-as-module-not-package idea ...
>>> you would need to be very careful with it,
>>> since all the files would be sharing a common globals() namespace.
>>
>> One of the things I like about Python's module system
>> is that once I know which module a name was imported
>> from, I also know which file to look in for its
>> definition. If a module can be spread over several
>> files, that feature would be lost.
 >
> There are many potential problems with the idea, I just chose to
> mention one of the ones that could easily make the affected code
> *break* :)

Right.  It would require additional pieces as well.

Ron :-)


From alexander.belopolsky at gmail.com  Wed Dec  1 18:55:28 2010
From: alexander.belopolsky at gmail.com (Alexander Belopolsky)
Date: Wed, 1 Dec 2010 12:55:28 -0500
Subject: [Python-Dev] Python and the Unicode Character Database
In-Reply-To: <4CF2DC5B.4020702@egenix.com>
References: <AANLkTinVpySuRFkjf-Sc+7a91zrxhuQpqPQL5nM9u1w+@mail.gmail.com>
	<4CF2DC5B.4020702@egenix.com>
Message-ID: <AANLkTimB+FkPxHyC_udiCr-oVmSG4jnZ6USUHk14_Vh-@mail.gmail.com>

On Sun, Nov 28, 2010 at 5:48 PM, M.-A. Lemburg <mal at egenix.com> wrote:
..
>> With Python 3.1:
>>
>>>>> exec('\u0CF1 = 1')
>> Traceback (most recent call last):
>> ?File "<stdin>", line 1, in <module>
>> ?File "<string>", line 1
>> ? ?? = 1
>> ? ? ?^
>> SyntaxError: invalid character in identifier
>>
>> but with Python 3.2a4:
>>
>>>>> exec('\u0CF1 = 1')
>>>>> eval('\u0CF1')
>> 1
>
> Such changes are not new, but I agree that they should probably
> be highlighted in the "What's new in Python x.x".
>

As of today, "What?s New In Python 3.2" [1] does not even mention the
unicodedata upgrade to 6.0.0.  Here are the features form the
unicode.org summary [2] that I think should be reflected in Python's
"What's New" document:

"""
* adds 2,088 characters, including over 1,000 additional symbols?chief
among them the additional emoji symbols, which are especially
important for mobile phones;

* corrects character properties for existing characters including
 - a general category change to two Kannada characters (U+0CF1,
U+0CF2), which has the effect of making them newly eligible for
inclusion in identifiers;

 - a general category change to one New Tai Lue numeric character
(U+19DA), which would have the effect of disqualifying it from
inclusion in identifiers unless grandfathering measures are in place
for the defining identifier syntax.
"""

The above may be too verbose for inclusion to "What?s New In Python
3.2", but I think we should add a possibly shorter summary with a link
to unicode.org for details.

PS: Yes, I think everyone should know about the Python 3.2 killer
feature: ('\N{CAT FACE WITH WRY SMILE}'!

[1] http://docs.python.org/dev/whatsnew/3.2.html
[2] http://www.unicode.org/versions/Unicode6.0.0/

From tjreedy at udel.edu  Wed Dec  1 19:29:06 2010
From: tjreedy at udel.edu (Terry Reedy)
Date: Wed, 01 Dec 2010 13:29:06 -0500
Subject: [Python-Dev] Python and the Unicode Character Database
In-Reply-To: <AANLkTimB+FkPxHyC_udiCr-oVmSG4jnZ6USUHk14_Vh-@mail.gmail.com>
References: <AANLkTinVpySuRFkjf-Sc+7a91zrxhuQpqPQL5nM9u1w+@mail.gmail.com>	<4CF2DC5B.4020702@egenix.com>
	<AANLkTimB+FkPxHyC_udiCr-oVmSG4jnZ6USUHk14_Vh-@mail.gmail.com>
Message-ID: <id645j$5di$1@dough.gmane.org>

On 12/1/2010 12:55 PM, Alexander Belopolsky wrote:
> On Sun, Nov 28, 2010 at 5:48 PM, M.-A. Lemburg<mal at egenix.com>  wrote:
> ..
>>> With Python 3.1:
>>>
>>>>>> exec('\u0CF1 = 1')
>>> Traceback (most recent call last):
>>>   File "<stdin>", line 1, in<module>
>>>   File "<string>", line 1
>>>     ? = 1
>>>       ^
>>> SyntaxError: invalid character in identifier
>>>
>>> but with Python 3.2a4:
>>>
>>>>>> exec('\u0CF1 = 1')
>>>>>> eval('\u0CF1')
>>> 1
>>
>> Such changes are not new, but I agree that they should probably
>> be highlighted in the "What's new in Python x.x".
>>
>
> As of today, "What?s New In Python 3.2" [1] does not even mention the
> unicodedata upgrade to 6.0.0.  Here are the features form the
> unicode.org summary [2] that I think should be reflected in Python's
> "What's New" document:
>
> """
> * adds 2,088 characters, including over 1,000 additional symbols?chief
> among them the additional emoji symbols, which are especially
> important for mobile phones;
>
> * corrects character properties for existing characters including
>   - a general category change to two Kannada characters (U+0CF1,
> U+0CF2), which has the effect of making them newly eligible for
> inclusion in identifiers;
>
>   - a general category change to one New Tai Lue numeric character
> (U+19DA), which would have the effect of disqualifying it from
> inclusion in identifiers unless grandfathering measures are in place
> for the defining identifier syntax.
> """

> The above may be too verbose for inclusion to "What?s New In Python
> 3.2",

I think those 11 lines are pretty good. Put them in
('\N{CAT FACE WITH WRY SMILE}'!

Plus give a link to Unicode site (Issue numbers are implicit links).

-- 
Terry Jan Reedy



From contactprashantat at gmail.com  Wed Dec  1 19:51:03 2010
From: contactprashantat at gmail.com (Prashant Kumar)
Date: Thu, 2 Dec 2010 00:21:03 +0530
Subject: [Python-Dev] Porting Ideas
Message-ID: <AANLkTing_iT7Y6FOZPWh=CUwtK8owvYtSdE7aXf0W181@mail.gmail.com>

Hello everyone. My name is Prashant. I and my friend Zubin recently
ported 'Configobj'. It would be great if somebody can suggest about
any utilities or scripts that are being widely used and need to be
ported.

From brian.curtin at gmail.com  Wed Dec  1 20:02:00 2010
From: brian.curtin at gmail.com (Brian Curtin)
Date: Wed, 1 Dec 2010 13:02:00 -0600
Subject: [Python-Dev] Porting Ideas
In-Reply-To: <AANLkTing_iT7Y6FOZPWh=CUwtK8owvYtSdE7aXf0W181@mail.gmail.com>
References: <AANLkTing_iT7Y6FOZPWh=CUwtK8owvYtSdE7aXf0W181@mail.gmail.com>
Message-ID: <AANLkTi=6rvXDEfzmMC=vE5jaCaTzAwnDddxHrZ2GAB1b@mail.gmail.com>

On Wed, Dec 1, 2010 at 12:51, Prashant Kumar <contactprashantat at gmail.com>wrote:

> Hello everyone. My name is Prashant. I and my friend Zubin recently
> ported 'Configobj'. It would be great if somebody can suggest about
> any utilities or scripts that are being widely used and need to be
> ported.


http://onpython3yet.com/ might be helpful to you. It orders the projects on
PyPI with the most dependencies which are not yet ported to 3.x.

Note that there are a number of false positives, e.g., the first result --
NumPy, since people don't seem to keep their classifiers up-to-date.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20101201/d8f3036b/attachment.html>

From chef at ghum.de  Wed Dec  1 20:17:08 2010
From: chef at ghum.de (Massa, Harald Armin)
Date: Wed, 1 Dec 2010 20:17:08 +0100
Subject: [Python-Dev] Porting Ideas
In-Reply-To: <AANLkTi=6rvXDEfzmMC=vE5jaCaTzAwnDddxHrZ2GAB1b@mail.gmail.com>
References: <AANLkTing_iT7Y6FOZPWh=CUwtK8owvYtSdE7aXf0W181@mail.gmail.com>
	<AANLkTi=6rvXDEfzmMC=vE5jaCaTzAwnDddxHrZ2GAB1b@mail.gmail.com>
Message-ID: <AANLkTimORORhW5-QBykKaMv-3h=DLdGOtKLu3UOjfazo@mail.gmail.com>

>
> http://onpython3yet.com/ might be helpful to you. It orders the projects
> on PyPI with the most dependencies which are not yet ported to 3.x.
>
> Note that there are a number of false positives, e.g., the first result --
> NumPy, since people don't seem to keep their classifiers up-to-date.
>
> That could be a nice list. But quite disturbing content, as "Python, the
programming language" is stated as "not being ported to 3.0". Does not
really provoke trust.


Harald


-- 
GHUM GmbH
Harald Armin Massa
Spielberger Stra?e 49
70435 Stuttgart
0173/9409607

Amtsgericht Stuttgart, HRB 734971
-
persuadere.
et programmare
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20101201/3e118352/attachment.html>

From solipsis at pitrou.net  Wed Dec  1 20:17:16 2010
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Wed, 1 Dec 2010 20:17:16 +0100
Subject: [Python-Dev] Porting Ideas
References: <AANLkTing_iT7Y6FOZPWh=CUwtK8owvYtSdE7aXf0W181@mail.gmail.com>
	<AANLkTi=6rvXDEfzmMC=vE5jaCaTzAwnDddxHrZ2GAB1b@mail.gmail.com>
Message-ID: <20101201201716.4357a316@pitrou.net>

On Wed, 1 Dec 2010 13:02:00 -0600
Brian Curtin <brian.curtin at gmail.com> wrote:
> On Wed, Dec 1, 2010 at 12:51, Prashant Kumar <contactprashantat at gmail.com>wrote:
> 
> > Hello everyone. My name is Prashant. I and my friend Zubin recently
> > ported 'Configobj'. It would be great if somebody can suggest about
> > any utilities or scripts that are being widely used and need to be
> > ported.
> 
> http://onpython3yet.com/ might be helpful to you. It orders the projects on
> PyPI with the most dependencies which are not yet ported to 3.x.

I don't know who did that page but it seems like there's some FUD there.

simplejson, ctypes, pysqlite and others are available in the 3.x
stdlib. Mercurial is a command-line tool and doesn't need to be ported
to be used for Python 3 projects. setuptools is supplanted by
distribute which should Python 3 compatible.

And I'm not sure what this package called "Python" is (?a high-level
object-oriented programming language?? like Java?), but I'm pretty sure
I've heard there's a Python 3 compatible version.

(granted, it's probably less FUD than stupid automation)

Regards

Antoine.



From brian.curtin at gmail.com  Wed Dec  1 20:39:43 2010
From: brian.curtin at gmail.com (Brian Curtin)
Date: Wed, 1 Dec 2010 13:39:43 -0600
Subject: [Python-Dev] Porting Ideas
In-Reply-To: <20101201201716.4357a316@pitrou.net>
References: <AANLkTing_iT7Y6FOZPWh=CUwtK8owvYtSdE7aXf0W181@mail.gmail.com>
	<AANLkTi=6rvXDEfzmMC=vE5jaCaTzAwnDddxHrZ2GAB1b@mail.gmail.com>
	<20101201201716.4357a316@pitrou.net>
Message-ID: <AANLkTikaYKGH6OhS7GxFN=5bofQgk7cT5z0jar_ng9ww@mail.gmail.com>

On Wed, Dec 1, 2010 at 13:17, Antoine Pitrou <solipsis at pitrou.net> wrote:

> On Wed, 1 Dec 2010 13:02:00 -0600
> Brian Curtin <brian.curtin at gmail.com> wrote:
> > On Wed, Dec 1, 2010 at 12:51, Prashant Kumar <
> contactprashantat at gmail.com>wrote:
> >
> > > Hello everyone. My name is Prashant. I and my friend Zubin recently
> > > ported 'Configobj'. It would be great if somebody can suggest about
> > > any utilities or scripts that are being widely used and need to be
> > > ported.
> >
> > http://onpython3yet.com/ might be helpful to you. It orders the projects
> on
> > PyPI with the most dependencies which are not yet ported to 3.x.
>
> I don't know who did that page but it seems like there's some FUD there.
>
> simplejson, ctypes, pysqlite and others are available in the 3.x
> stdlib.


It grabs the info from their PyPI pages, which are probably not kept
up-to-date.

This was brought up at a local user group meeting and I think it can be a
useful tool, but as you can see it requires good input data which isn't
always the case for some packages.


Package authors: if you spent time making your project work on 3.x -- let
the world know, update your classifiers.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20101201/0cf1c095/attachment.html>

From tjreedy at udel.edu  Wed Dec  1 21:23:03 2010
From: tjreedy at udel.edu (Terry Reedy)
Date: Wed, 01 Dec 2010 15:23:03 -0500
Subject: [Python-Dev] Deprecating undocumented, unused functions in difflib.
Message-ID: <id6ar8$9is$1@dough.gmane.org>

Difflib.SequenceMatcher object currently get two feature attributes:
     self.isbjunk = junk.__contains__
     self.isbpopular = popular.__contains__

Tim Peters agrees that the junk and popular sets should be directly 
exposed and documented as part of the api, thereby making the functions 
redundant. The two functions are not currently documented (and should 
not be now). A google codesearch of 'isbjunk' and 'isbpopular' only 
returns hits in difflib.py itself (and its predecessor, ndiff.py).

It would be easiest to just remove the two lines above.
Or should I define functions _xxx names that issue a deprecation warning 
and attach them as attributes to each object? (Defining instance methods 
would not be the same).

There is only one internal use of one of the two functions which is 
easily edited.

-- 
Terry Jan Reedy


From ddasilva at umd.edu  Wed Dec  1 22:15:04 2010
From: ddasilva at umd.edu (Daniel da Silva)
Date: Wed, 1 Dec 2010 16:15:04 -0500
Subject: [Python-Dev] [Preview] Comments and change proposals on
	documentation
In-Reply-To: <AANLkTik9b++fPvRnQDxi1W5mF4kHVH1vp2VHrUAjfGNj@mail.gmail.com>
References: <icjsal$eqk$1@dough.gmane.org>
	<AANLkTinQ03rEMzsKfjma1p7O8hKfC3rXfuSX2F5cwrwL@mail.gmail.com>
	<4CF18220.7000202@pearwood.info> <ics17i$che$1@dough.gmane.org>
	<4CF19B3C.2000308@pearwood.info> <icvpmv$ikr$1@dough.gmane.org>
	<AANLkTik9b++fPvRnQDxi1W5mF4kHVH1vp2VHrUAjfGNj@mail.gmail.com>
Message-ID: <AANLkTi=1mf4YPrdopjJKq+rZY2hZBmh1ekcNwkKx1_Qb@mail.gmail.com>

I think it looks great.!

If you are looking for some suggestions to make it a little more elegant:
1. If I delete a comment that has no children, it should remove it
completely (currently, it just replaces it with [deleted]). If there are
children, I think it is doing the right thing.
2. When I post a comment, it should automatically vote that comment up. I
wouldn't have posted it if I didn't like it.
3. As far as text formatting, I personally think there should be some
hilighting support for code spans/blocks (IMO that should match the idle
colors).

Also, I seemed to manage to trigger a visible system warning in my badly
formatted comment on math.fabs(x), :)


-Daniel
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20101201/1bdb6a48/attachment.html>

From martin at v.loewis.de  Wed Dec  1 23:32:25 2010
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Wed, 01 Dec 2010 23:32:25 +0100
Subject: [Python-Dev] Python and the Unicode Character Database
In-Reply-To: <4CF619C1.8030403@pearwood.info>
References: <AANLkTinVpySuRFkjf-Sc+7a91zrxhuQpqPQL5nM9u1w+@mail.gmail.com>	<AANLkTinQLK7+5NfEp-2nMiJeDEKRxKN9Zfs4ihv-NMyH@mail.gmail.com>	<87wrnv43v5.fsf@uwakimon.sk.tsukuba.ac.jp>	<id3umc$ps8$1@dough.gmane.org>	<4CF5841E.6020401@v.loewis.de>
	<4CF619C1.8030403@pearwood.info>
Message-ID: <4CF6CCF9.6000100@v.loewis.de>

>> And here, my observation stands: if they wanted to, they currently
>> couldn't - at least not for real numbers (and also not for integers
>> if they want to use grouping). So the presumed application of this
>> feature doesn't actually work, despite the presence of the feature it
>> was supposedly meant to enable.
> 
> By that argument, English speakers wanting to enter integers using
> Arabic numerals can't either!

That's correct, and the key point here for the argument. It's just not
*meant* to support localized number forms, but deliberately constrains
them to a formal grammar which users using it must be aware of in order
to use it.

> I'd like to use grouping for large
> literals, if only I could think of a half-decent syntax, and if only
> Python supported it. This fails on both counts:
> 
> x = 123_456_789_012_345

Here you are confusing issues, though: this fragment uses the syntax of
the Python programming language. Whether or not the syntax of the
float() constructor arguments matches that syntax is also a subject of
the debate.

I take it that you speak in favor of the float syntax also being used
for the float() constructor.

> The lack of grouping and the lack of a native decimal point doesn't mean
> that the feature "doesn't work" -- it merely means the feature requires
> some compromise before it can be used.

No, it means that the Python programming language syntax for floating
point numbers just doesn't take local notation into account *at all*.
This is not a flaw - it just means that this feature is non-existent.

Now, for the float() constructor, some people in this thread have
claimed that it *is* aimed at people who want to enter numbers in their
local spellings. I claim that this feature either doesn't work, or is
absent also.

> In the same way, if I wanted to enter a number using non-Arabic digits,
> it works provided I compromise by using the Anglo-American decimal point
> instead of the European comma or the native decimal point I might prefer.

Why would you want that, if, what you really wanted, could not be
done. There certainly *is* a way to convert strings into floats,
and there would be a way if that restricted itself to the digits 0..9.
So it can't be the mere desire to convert strings to float that make
you ask for non-ASCII digits.

> The lack of support for non-dot decimal points is arguably a bug that
> should be fixed, not a reason to remove functionality.

I keep repeating my two concerns:
a) if that was a feature, it is not specified at all in the
   documentation. In fact, the documentation was recently clarified
   to deny existence of that feature.
b) fixing it will be much more difficult than you apparently think.

Regards,
Martin


From martin at v.loewis.de  Wed Dec  1 23:36:13 2010
From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=)
Date: Wed, 01 Dec 2010 23:36:13 +0100
Subject: [Python-Dev] Python and the Unicode Character Database
In-Reply-To: <87k4ju2vvg.fsf@benfinney.id.au>
References: <AANLkTinVpySuRFkjf-Sc+7a91zrxhuQpqPQL5nM9u1w+@mail.gmail.com>	<AANLkTinQLK7+5NfEp-2nMiJeDEKRxKN9Zfs4ihv-NMyH@mail.gmail.com>	<87wrnv43v5.fsf@uwakimon.sk.tsukuba.ac.jp>	<AANLkTinAf6CstywehOGRd4G4ms6K5LtSuq4Lj0z5wFBW@mail.gmail.com>	<87r5e236hj.fsf@benfinney.id.au>
	<4CF58387.1000609@v.loewis.de> <87k4ju2vvg.fsf@benfinney.id.au>
Message-ID: <4CF6CDDD.4060109@v.loewis.de>

>> I think the OP (haiyang kang) already indicated that he finds it quite
>> unlikely that anybody would possibly want to enter that.
> 
> Who's talking about *entering* it into the program at a keyboard
> directly, though? Input to a program can come from all kinds of crazy
> sources. Just because it wasn't typed by the person at the keyboard
> using this program doesn't stop it being input to the program.

I think haiyang kang claimed exactly that - it won't ever be input to a
program. I trust him on that - and so should you, unless you have
sufficient experience with the Chinese language and writing system.

> Note that I'm not saying this is common. Nor am I saying it's a
> desirable situation. I'm saying it is a feasible use case, to be
> dismissed only if there is strong evidence that it's not used by
> existing Python code.

And indeed, for the Chinese numerals, we have such strong evidence.

Regards,
Martin


From martin at v.loewis.de  Wed Dec  1 23:39:17 2010
From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=)
Date: Wed, 01 Dec 2010 23:39:17 +0100
Subject: [Python-Dev] Python and the Unicode Character Database
In-Reply-To: <AANLkTimB+FkPxHyC_udiCr-oVmSG4jnZ6USUHk14_Vh-@mail.gmail.com>
References: <AANLkTinVpySuRFkjf-Sc+7a91zrxhuQpqPQL5nM9u1w+@mail.gmail.com>	<4CF2DC5B.4020702@egenix.com>
	<AANLkTimB+FkPxHyC_udiCr-oVmSG4jnZ6USUHk14_Vh-@mail.gmail.com>
Message-ID: <4CF6CE95.1060707@v.loewis.de>

> As of today, "What?s New In Python 3.2" [1] does not even mention the
> unicodedata upgrade to 6.0.0.

One reason was that I was instructed not to change "What's New" a few
years ago.

Regards,
Martin

From martin at v.loewis.de  Wed Dec  1 23:48:29 2010
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Wed, 01 Dec 2010 23:48:29 +0100
Subject: [Python-Dev] Porting Ideas
In-Reply-To: <AANLkTi=6rvXDEfzmMC=vE5jaCaTzAwnDddxHrZ2GAB1b@mail.gmail.com>
References: <AANLkTing_iT7Y6FOZPWh=CUwtK8owvYtSdE7aXf0W181@mail.gmail.com>
	<AANLkTi=6rvXDEfzmMC=vE5jaCaTzAwnDddxHrZ2GAB1b@mail.gmail.com>
Message-ID: <4CF6D0BD.6010204@v.loewis.de>

Am 01.12.2010 20:02, schrieb Brian Curtin:
> On Wed, Dec 1, 2010 at 12:51, Prashant Kumar
> <contactprashantat at gmail.com <mailto:contactprashantat at gmail.com>> wrote:
> 
>     Hello everyone. My name is Prashant. I and my friend Zubin recently
>     ported 'Configobj'. It would be great if somebody can suggest about
>     any utilities or scripts that are being widely used and need to be
>     ported.
> 
> 
> http://onpython3yet.com/ might be helpful to you.

Another such list is at

http://www.python.org/3kpoll

Regards,
Martin

From steve at pearwood.info  Thu Dec  2 00:18:10 2010
From: steve at pearwood.info (Steven D'Aprano)
Date: Thu, 02 Dec 2010 10:18:10 +1100
Subject: [Python-Dev] Python and the Unicode Character Database
In-Reply-To: <4CF6CDDD.4060109@v.loewis.de>
References: <AANLkTinVpySuRFkjf-Sc+7a91zrxhuQpqPQL5nM9u1w+@mail.gmail.com>	<AANLkTinQLK7+5NfEp-2nMiJeDEKRxKN9Zfs4ihv-NMyH@mail.gmail.com>	<87wrnv43v5.fsf@uwakimon.sk.tsukuba.ac.jp>	<AANLkTinAf6CstywehOGRd4G4ms6K5LtSuq4Lj0z5wFBW@mail.gmail.com>	<87r5e236hj.fsf@benfinney.id.au>	<4CF58387.1000609@v.loewis.de>
	<87k4ju2vvg.fsf@benfinney.id.au> <4CF6CDDD.4060109@v.loewis.de>
Message-ID: <4CF6D7B2.6030506@pearwood.info>

Martin v. L?wis wrote:
>>> I think the OP (haiyang kang) already indicated that he finds it quite
>>> unlikely that anybody would possibly want to enter that.
>> Who's talking about *entering* it into the program at a keyboard
>> directly, though? Input to a program can come from all kinds of crazy
>> sources. Just because it wasn't typed by the person at the keyboard
>> using this program doesn't stop it being input to the program.
> 
> I think haiyang kang claimed exactly that - it won't ever be input to a
> program. I trust him on that - and so should you, unless you have
> sufficient experience with the Chinese language and writing system.
> 
>> Note that I'm not saying this is common. Nor am I saying it's a
>> desirable situation. I'm saying it is a feasible use case, to be
>> dismissed only if there is strong evidence that it's not used by
>> existing Python code.
> 
> And indeed, for the Chinese numerals, we have such strong evidence.

With full respect to haiyang kang, hear-say from one person can hardly 
be described as "strong" evidence -- particularly, as Alexander 
Belopolsky pointed out, the use-case described isn't currently supported 
by Python. Given that what haiyang kang describes *can't* be done, the 
fact that people don't do it is hardly surprising -- nor is it a good 
reason for taking away functionality that does exist.



-- 
Steven


From alexander.belopolsky at gmail.com  Thu Dec  2 00:32:32 2010
From: alexander.belopolsky at gmail.com (Alexander Belopolsky)
Date: Wed, 1 Dec 2010 18:32:32 -0500
Subject: [Python-Dev] Python and the Unicode Character Database
In-Reply-To: <4CF6CDDD.4060109@v.loewis.de>
References: <AANLkTinVpySuRFkjf-Sc+7a91zrxhuQpqPQL5nM9u1w+@mail.gmail.com>
	<AANLkTinQLK7+5NfEp-2nMiJeDEKRxKN9Zfs4ihv-NMyH@mail.gmail.com>
	<87wrnv43v5.fsf@uwakimon.sk.tsukuba.ac.jp>
	<AANLkTinAf6CstywehOGRd4G4ms6K5LtSuq4Lj0z5wFBW@mail.gmail.com>
	<87r5e236hj.fsf@benfinney.id.au> <4CF58387.1000609@v.loewis.de>
	<87k4ju2vvg.fsf@benfinney.id.au> <4CF6CDDD.4060109@v.loewis.de>
Message-ID: <AANLkTim=BP4ESMCLmN0CNLsMVxN3bKETdQ0sf29s5Cw5@mail.gmail.com>

On Wed, Dec 1, 2010 at 5:36 PM, "Martin v. L?wis" <martin at v.loewis.de> wrote:
..
>> Note that I'm not saying this is common. Nor am I saying it's a
>> desirable situation. I'm saying it is a feasible use case, to be
>> dismissed only if there is strong evidence that it's not used by
>> existing Python code.
>
> And indeed, for the Chinese numerals, we have such strong evidence.
>

Indeed: it over 10 years that Python's int() accepted Arabic-Indic
numerals, nobody has complained that it *did not* accept Chinese.

From steve at pearwood.info  Thu Dec  2 01:17:51 2010
From: steve at pearwood.info (Steven D'Aprano)
Date: Thu, 02 Dec 2010 11:17:51 +1100
Subject: [Python-Dev] Python and the Unicode Character Database
In-Reply-To: <4CF6CCF9.6000100@v.loewis.de>
References: <AANLkTinVpySuRFkjf-Sc+7a91zrxhuQpqPQL5nM9u1w+@mail.gmail.com>	<AANLkTinQLK7+5NfEp-2nMiJeDEKRxKN9Zfs4ihv-NMyH@mail.gmail.com>	<87wrnv43v5.fsf@uwakimon.sk.tsukuba.ac.jp>	<id3umc$ps8$1@dough.gmane.org>	<4CF5841E.6020401@v.loewis.de>
	<4CF619C1.8030403@pearwood.info> <4CF6CCF9.6000100@v.loewis.de>
Message-ID: <4CF6E5AF.7080308@pearwood.info>

Martin v. L?wis wrote:
>>> And here, my observation stands: if they wanted to, they currently
>>> couldn't - at least not for real numbers (and also not for integers
>>> if they want to use grouping). So the presumed application of this
>>> feature doesn't actually work, despite the presence of the feature it
>>> was supposedly meant to enable.
>> By that argument, English speakers wanting to enter integers using
>> Arabic numerals can't either!
> 
> That's correct, and the key point here for the argument. It's just not
> *meant* to support localized number forms, but deliberately constrains
> them to a formal grammar which users using it must be aware of in order
> to use it.

You're *agreeing* that English speakers can't enter integers using 
Arabic numerals? What do you think I'm doing when I do this?

 >>> int("1234")
1234

Ah wait... did you think I meant Arabic numerals in the sense of digits 
used by Arabs in Arabia? I meant Arabic numerals as opposed to Roman 
numerals. Sorry for the confusion.

Your argument was that even though Python's int() supports many 
non-ASCII digits, the lack of grouping means that it "doesn't actually 
work". If that argument were correct, then it applies equally to ASCII 
digits as well.

It's clearly nonsense to say that int("1234") "doesn't work" just 
because of the lack of grouping. It's equally nonsense to say that
int("????") "doesn't work" because of the lack of grouping.


[...]
> I take it that you speak in favor of the float syntax also being used
> for the float() constructor.

I'm sorry, I don't understand what you mean here. I've repeatedly said 
that the syntax for numeric literals should remain constrained to the 
ASCII digits, as it currently is.

n = ????

gives a SyntaxError, and I don't want to see that change.

But I've also argued that the float constructor currently accepts 
non-ASCII strings:

n = int("????")

we should continue to support the existing behaviour. None of the 
arguments against it seem convincing to me, particularly since the 
opponents of the current behaviour admit that there is a use-case for 
it, but they just want it to move elsewhere, such as the locale module.

We've even heard from one person -- I forget who, sorry -- who claimed 
that C++ has the same behaviour, and if you want ASCII-only digits, you 
have to explicitly ask for it.

For what it's worth, Microsoft warns developers not to assume users will 
enter numeric data using ASCII digits:

"Number representation can also use non-ASCII native digits, so your 
application may encounter characters other than 0-9 as inputs. Avoid 
filtering on U+0030 through U+0039 to prevent frustration for users who 
are trying to enter data using non-ASCII digits."

http://msdn.microsoft.com/en-us/magazine/cc163506.aspx


There was a similar discussion going on in Perl-land recently:

http://www.nntp.perl.org/group/perl.perl5.porters/2010/07/msg162400.html

although, being Perl, the discussion was dominated by concerns about 
regexes and implicit conversions, rather than an explicit call to 
float() or int() as we are discussing here.


[...]
>> In the same way, if I wanted to enter a number using non-Arabic digits,
>> it works provided I compromise by using the Anglo-American decimal point
>> instead of the European comma or the native decimal point I might prefer.
> 
> Why would you want that, if, what you really wanted, could not be
> done. There certainly *is* a way to convert strings into floats,
> and there would be a way if that restricted itself to the digits 0..9.
> So it can't be the mere desire to convert strings to float that make
> you ask for non-ASCII digits.

Why do Europeans use programming languages that force them to use a dot 
instead of a comma for the decimal place? Why do I misspell 
string.centre as string.center? Because if you want to get something 
done, you use the tools you have and not the tools you'd like to have.




-- 
Steven

From stephen at xemacs.org  Thu Dec  2 01:31:54 2010
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Thu, 02 Dec 2010 09:31:54 +0900
Subject: [Python-Dev] Python and the Unicode Character Database
In-Reply-To: <AANLkTimnar7CuHKXtvFtspTphmB3GrNMiA59zfFhTeoq@mail.gmail.com>
References: <AANLkTinVpySuRFkjf-Sc+7a91zrxhuQpqPQL5nM9u1w+@mail.gmail.com>
	<AANLkTinQLK7+5NfEp-2nMiJeDEKRxKN9Zfs4ihv-NMyH@mail.gmail.com>
	<87wrnv43v5.fsf@uwakimon.sk.tsukuba.ac.jp>
	<AANLkTimnar7CuHKXtvFtspTphmB3GrNMiA59zfFhTeoq@mail.gmail.com>
Message-ID: <87tyixxbet.fsf@uwakimon.sk.tsukuba.ac.jp>

Lennart Regebro writes:
 > On Tue, Nov 30, 2010 at 09:23, Stephen J. Turnbull <stephen at xemacs.org> wrote:
 > > Sure you can. ?In Python program text, all keywords will be ASCII
 > 
 > Yes, yes, sure, but not the contents of variables,

Irrelevant, you're not converting these to a string representation.
If you're generating numerals for internal use, I don't see why you
would want to do arithmetic on them; conversion is a YAGNI.  This is
only interesting to allow naive users to input in a comfortable way.

As yet there is no evidence that there are *any* such naive users, 1.3
billion of "possibles" are shut out, and at least two cultures which
use non-ASCII numerals every day, representing 1.3 billion naive users
(the coincidence of numbers is no coincidence), have reported that
nobody in their right mind would would *input* the numbers that way,
and at least for Japanese, the use cases are not really numeric anyway.

 > > I see no reason not to make a similar promise for numeric literals.
 > 
 > Wait what, literas?

Sorry, my bad.
-------------- next part --------------

 > Why would this be a problem:
 > 
 > >>> T1234 = float('~~~~.~~')
 > >>> T1234
 > 1234.56
 > 
 > But this OK?
 > 
 > >>> T~~~~ = float('1234.56')
 > >>> T~~~~
 > 1234.56

(Sorry, the Arabic is going to get munged, my mailer is beta and
somebody screwed up.)

Because the characters in the identifier are uninterpreted and have no
syntactic content other than their identity.  They're arbitrary.
That's not true of numerics.

Because that works, but

print(T1234)

doesn't (it prints ASCII).  You can't round-trip, but users will
want/expect that.

Because that works but this doesn't:
-------------- next part --------------

T1000 = float('?.???')

Violates TOOWTDI.

If you're proposing to fix the numeric parsers, I still don't like it
but I could go to -0 on it.  However as Alexander points out and MAL
admits, it's apparently not so easy to do that.

From alexander.belopolsky at gmail.com  Thu Dec  2 01:44:28 2010
From: alexander.belopolsky at gmail.com (Alexander Belopolsky)
Date: Wed, 1 Dec 2010 19:44:28 -0500
Subject: [Python-Dev] Python and the Unicode Character Database
In-Reply-To: <4CF6E5AF.7080308@pearwood.info>
References: <AANLkTinVpySuRFkjf-Sc+7a91zrxhuQpqPQL5nM9u1w+@mail.gmail.com>
	<AANLkTinQLK7+5NfEp-2nMiJeDEKRxKN9Zfs4ihv-NMyH@mail.gmail.com>
	<87wrnv43v5.fsf@uwakimon.sk.tsukuba.ac.jp>
	<id3umc$ps8$1@dough.gmane.org> <4CF5841E.6020401@v.loewis.de>
	<4CF619C1.8030403@pearwood.info> <4CF6CCF9.6000100@v.loewis.de>
	<4CF6E5AF.7080308@pearwood.info>
Message-ID: <AANLkTikuUki5LgisVHrSMGZtmWZxxy9A8TOwny7dVCMe@mail.gmail.com>

On Wed, Dec 1, 2010 at 7:17 PM, Steven D'Aprano <steve at pearwood.info> wrote:
..
> we should continue to support the existing behaviour. None of the arguments
> against it seem convincing to me, particularly since the opponents of the
> current behaviour admit that there is a use-case for it, but they just want
> it to move elsewhere, such as the locale module.
>

I don't remember who made this argument, but I think you misunderstood
it.  The argument was that if there was a use case for parsing Eastern
Arabic numerals, it would be better served by a module written by
someone who speaks one of the Arabic languages and knows the details
of how  Eastern Arabic numerals are written.  So far nobody has even
claimed to know conclusively that Arabic-Indic digits are always
written left-to-right.

>>> unicodedata.bidirectional('?')
'AN'

is not very helpful because it means "any Arabic-Indic digit"
according to unicode.org.  (To me, a special category hints that it
may be written in either direction and the proper interpretation may
depend on context.)   I have not seen a real use case reported in this
thread and for theoretical use cases, the current implementation is
either outright wrong or does not solve the problem completely. Given
that a function that replaces all Unicode digits in a string with 0-9
can be written in 3 lines of Python code, it is very unlikely that
anyone would prefer to rely on undocumented behavior of Python
builtins instead of having explicit control over parsing of their
data.

From stephen at xemacs.org  Thu Dec  2 01:45:55 2010
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Thu, 02 Dec 2010 09:45:55 +0900
Subject: [Python-Dev] Python and the Unicode Character Database
In-Reply-To: <4CF6D7B2.6030506@pearwood.info>
References: <AANLkTinVpySuRFkjf-Sc+7a91zrxhuQpqPQL5nM9u1w+@mail.gmail.com>
	<AANLkTinQLK7+5NfEp-2nMiJeDEKRxKN9Zfs4ihv-NMyH@mail.gmail.com>
	<87wrnv43v5.fsf@uwakimon.sk.tsukuba.ac.jp>
	<AANLkTinAf6CstywehOGRd4G4ms6K5LtSuq4Lj0z5wFBW@mail.gmail.com>
	<87r5e236hj.fsf@benfinney.id.au> <4CF58387.1000609@v.loewis.de>
	<87k4ju2vvg.fsf@benfinney.id.au> <4CF6CDDD.4060109@v.loewis.de>
	<4CF6D7B2.6030506@pearwood.info>
Message-ID: <87sjyhxarg.fsf@uwakimon.sk.tsukuba.ac.jp>

Steven D'Aprano writes:

 > With full respect to haiyang kang, hear-say from one person can hardly 
 > be described as "strong" evidence

That's *disrespectful* nonsense.  What Haiyang reported was not
hearsay, it's direct observation of what he sees around him and
personal experience, plus extrapolation.  Look up "hearsay," please.

Furthermore, he provided good *objective* reason (excessive cost, to
which I can also testify, in several different input methods for
Japanese) why numbers simply would not be input that way.

What's left is copy/paste via the mouse.  I assure you, every day I
see dozens of Japanese copy/pasting *only* ASCII numerals, and the
sales figures for Microsoft Excel (not to mention the download numbers
for Open Office) strongly suggest that 30 million Japanese salarymen
are similarly dedicated to ASCII.  (That's not "hearsay" either,
that's direct observation and extrapolation, which is more than the
"we need float to translate Arabic" supporters can offer.)

I have seen only *one* use case: it's a toy for sophisticated
programmers who want to think of themselves as broadminded.  We've
seen several examples of that in this thread, so I can't deny that is
a real use case.

Please, give us just *one* more real use case that isn't "somebody
might".

From fuzzyman at voidspace.org.uk  Thu Dec  2 02:17:43 2010
From: fuzzyman at voidspace.org.uk (Michael Foord)
Date: Thu, 02 Dec 2010 01:17:43 +0000
Subject: [Python-Dev] Porting Ideas
In-Reply-To: <20101201201716.4357a316@pitrou.net>
References: <AANLkTing_iT7Y6FOZPWh=CUwtK8owvYtSdE7aXf0W181@mail.gmail.com>	<AANLkTi=6rvXDEfzmMC=vE5jaCaTzAwnDddxHrZ2GAB1b@mail.gmail.com>
	<20101201201716.4357a316@pitrou.net>
Message-ID: <4CF6F3B7.1040805@voidspace.org.uk>

On 01/12/2010 19:17, Antoine Pitrou wrote:
> On Wed, 1 Dec 2010 13:02:00 -0600
> Brian Curtin<brian.curtin at gmail.com>  wrote:
>> On Wed, Dec 1, 2010 at 12:51, Prashant Kumar<contactprashantat at gmail.com>wrote:
>>
>>> Hello everyone. My name is Prashant. I and my friend Zubin recently
>>> ported 'Configobj'. It would be great if somebody can suggest about
>>> any utilities or scripts that are being widely used and need to be
>>> ported.
>> http://onpython3yet.com/ might be helpful to you. It orders the projects on
>> PyPI with the most dependencies which are not yet ported to 3.x.
> I don't know who did that page but it seems like there's some FUD there.
>
> simplejson, ctypes, pysqlite and others are available in the 3.x
> stdlib. Mercurial is a command-line tool and doesn't need to be ported
> to be used for Python 3 projects. setuptools is supplanted by
> distribute which should Python 3 compatible.
>
> And I'm not sure what this package called "Python" is (?a high-level
> object-oriented programming language?? like Java?), but I'm pretty sure
> I've heard there's a Python 3 compatible version.
>
> (granted, it's probably less FUD than stupid automation)

 From what I can tell it simply looks at dependencies and availability 
of those dependencies with a "Python 3" trove classification. Some 
manual filtering may well be useful.

It is well *possible* that there are packages with a runtime dependency 
on libraries in mercurial however. Those would need mercurial porting to 
Python 3 if they are to run on Python 3. If they simply shell out to 
mercurial that wouldn't be the case.

Michael

> Regards
>
> Antoine.
>
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk


-- 

http://www.voidspace.org.uk/

READ CAREFULLY. By accepting and reading this email you agree,
on behalf of your employer, to release me from all obligations
and waivers arising from any and all NON-NEGOTIATED agreements,
licenses, terms-of-service, shrinkwrap, clickwrap, browsewrap,
confidentiality, non-disclosure, non-compete and acceptable use
policies (?BOGUS AGREEMENTS?) that I have entered into with your
employer, its partners, licensors, agents and assigns, in
perpetuity, without prejudice to my ongoing rights and privileges.
You further represent that you have the authority to release me
from any BOGUS AGREEMENTS on behalf of your employer.


From ncoghlan at gmail.com  Thu Dec  2 02:22:15 2010
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Thu, 2 Dec 2010 11:22:15 +1000
Subject: [Python-Dev] Deprecating undocumented,
	unused functions in difflib.
In-Reply-To: <id6ar8$9is$1@dough.gmane.org>
References: <id6ar8$9is$1@dough.gmane.org>
Message-ID: <AANLkTimLJQdJQO0vfx9zoJjAAEa=sNtVsWEaXdD-5+2p@mail.gmail.com>

On Thu, Dec 2, 2010 at 6:23 AM, Terry Reedy <tjreedy at udel.edu> wrote:
> It would be easiest to just remove the two lines above.
> Or should I define functions _xxx names that issue a deprecation warning and
> attach them as attributes to each object? (Defining instance methods would
> not be the same).

Given that functions are converted to bound methods only on retrieval
from an instance, why wouldn't it be the same?

But yes, if you want to get rid of them, then deprecation for 3.2 and
removal in 3.3 is the way to go.

Alternatively, not deprecating them at all and just leaving them
undocumented with a comment in the source to say they have been
deliberately omitted from the docs would also be fine.

Cheers,
Nick.

-- 
Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia

From fuzzyman at voidspace.org.uk  Thu Dec  2 02:22:18 2010
From: fuzzyman at voidspace.org.uk (Michael Foord)
Date: Thu, 02 Dec 2010 01:22:18 +0000
Subject: [Python-Dev] Deprecating undocumented,
	unused functions in difflib.
In-Reply-To: <id6ar8$9is$1@dough.gmane.org>
References: <id6ar8$9is$1@dough.gmane.org>
Message-ID: <4CF6F4CA.4030005@voidspace.org.uk>

On 01/12/2010 20:23, Terry Reedy wrote:
> Difflib.SequenceMatcher object currently get two feature attributes:
> self.isbjunk = junk.__contains__
> self.isbpopular = popular.__contains__
>
> Tim Peters agrees that the junk and popular sets should be directly 
> exposed and documented as part of the api, thereby making the 
> functions redundant. The two functions are not currently documented 
> (and should not be now). A google codesearch of 'isbjunk' and 
> 'isbpopular' only returns hits in difflib.py itself (and its 
> predecessor, ndiff.py).
>
> It would be easiest to just remove the two lines above.
> Or should I define functions _xxx names that issue a deprecation 
> warning and attach them as attributes to each object? (Defining 
> instance methods would not be the same).
>
> There is only one internal use of one of the two functions which is 
> easily edited.
>
I would still be tempted to go through a single release of deprecation. 
You can add a test that the names are gone if the version of Python is 
3.3. When the tests start failing the code and the tests can be ripped out.

All the best,

Michael

-- 

http://www.voidspace.org.uk/

READ CAREFULLY. By accepting and reading this email you agree,
on behalf of your employer, to release me from all obligations
and waivers arising from any and all NON-NEGOTIATED agreements,
licenses, terms-of-service, shrinkwrap, clickwrap, browsewrap,
confidentiality, non-disclosure, non-compete and acceptable use
policies (?BOGUS AGREEMENTS?) that I have entered into with your
employer, its partners, licensors, agents and assigns, in
perpetuity, without prejudice to my ongoing rights and privileges.
You further represent that you have the authority to release me
from any BOGUS AGREEMENTS on behalf of your employer.


From ben+python at benfinney.id.au  Thu Dec  2 03:01:29 2010
From: ben+python at benfinney.id.au (Ben Finney)
Date: Thu, 02 Dec 2010 13:01:29 +1100
Subject: [Python-Dev] Python and the Unicode Character Database
References: <AANLkTinVpySuRFkjf-Sc+7a91zrxhuQpqPQL5nM9u1w+@mail.gmail.com>
	<AANLkTinQLK7+5NfEp-2nMiJeDEKRxKN9Zfs4ihv-NMyH@mail.gmail.com>
	<87wrnv43v5.fsf@uwakimon.sk.tsukuba.ac.jp>
	<AANLkTinAf6CstywehOGRd4G4ms6K5LtSuq4Lj0z5wFBW@mail.gmail.com>
	<87r5e236hj.fsf@benfinney.id.au> <4CF58387.1000609@v.loewis.de>
	<87k4ju2vvg.fsf@benfinney.id.au> <4CF6CDDD.4060109@v.loewis.de>
	<4CF6D7B2.6030506@pearwood.info>
	<87sjyhxarg.fsf@uwakimon.sk.tsukuba.ac.jp>
Message-ID: <87bp552arq.fsf@benfinney.id.au>

"Stephen J. Turnbull" <stephen at xemacs.org> writes:

> Furthermore, he provided good *objective* reason (excessive cost, to
> which I can also testify, in several different input methods for
> Japanese) why numbers simply would not be input that way.
>
> What's left is copy/paste via the mouse.

For direct entry by an interactive user, yes. Why are some people in
this discussion thinking only of direct entry by an interactive user?

Input to a program comes from various sources other than direct entry by
the interactive user, as has been pointed out many times.

> Please, give us just *one* more real use case that isn't "somebody
> might".

Input from an existing text file, as I said earlier. Or any other way of
text data making its way into a Python program.

Direct entry at the console is a red herring.

-- 
 \       ?First things first, but not necessarily in that order.? ?The |
  `\                                              Doctor, _Doctor Who_ |
_o__)                                                                  |
Ben Finney


From tjreedy at udel.edu  Thu Dec  2 03:42:03 2010
From: tjreedy at udel.edu (Terry Reedy)
Date: Wed, 01 Dec 2010 21:42:03 -0500
Subject: [Python-Dev] Deprecating undocumented,
	unused functions in difflib.
In-Reply-To: <4CF6F4CA.4030005@voidspace.org.uk>
References: <id6ar8$9is$1@dough.gmane.org> <4CF6F4CA.4030005@voidspace.org.uk>
Message-ID: <id711t$7ds$1@dough.gmane.org>

On 12/1/2010 8:22 PM, Michael Foord wrote:

> I would still be tempted to go through a single release of deprecation.
> You can add a test that the names are gone if the version of Python is
> 3.3. When the tests start failing the code and the tests can be ripped out.

I was wondering how people remember...
It would be nice is there were instead a central place to 'deposit' 
simple future patches that just consist of removals

-- 
Terry Jan Reedy


From tjreedy at udel.edu  Thu Dec  2 03:53:47 2010
From: tjreedy at udel.edu (Terry Reedy)
Date: Wed, 01 Dec 2010 21:53:47 -0500
Subject: [Python-Dev] Porting Ideas
In-Reply-To: <4CF6F3B7.1040805@voidspace.org.uk>
References: <AANLkTing_iT7Y6FOZPWh=CUwtK8owvYtSdE7aXf0W181@mail.gmail.com>	<AANLkTi=6rvXDEfzmMC=vE5jaCaTzAwnDddxHrZ2GAB1b@mail.gmail.com>	<20101201201716.4357a316@pitrou.net>
	<4CF6F3B7.1040805@voidspace.org.uk>
Message-ID: <id71ns$9g5$1@dough.gmane.org>

On 12/1/2010 8:17 PM, Michael Foord wrote:

> It is well *possible* that there are packages with a runtime dependency
> on libraries in mercurial however. Those would need mercurial porting to
> Python 3 if they are to run on Python 3. If they simply shell out to
> mercurial that wouldn't be the case.

It would be nice is all the Python-coded tools needed to work on Python3 
ran on Python3, so one did not have to install 2.x just for that purpose 
;-).

Does Sphinx run on PY3 yet?

-- 
Terry Jan Reedy


From alexander.belopolsky at gmail.com  Thu Dec  2 04:06:24 2010
From: alexander.belopolsky at gmail.com (Alexander Belopolsky)
Date: Wed, 1 Dec 2010 22:06:24 -0500
Subject: [Python-Dev] Porting Ideas
In-Reply-To: <id71ns$9g5$1@dough.gmane.org>
References: <AANLkTing_iT7Y6FOZPWh=CUwtK8owvYtSdE7aXf0W181@mail.gmail.com>
	<AANLkTi=6rvXDEfzmMC=vE5jaCaTzAwnDddxHrZ2GAB1b@mail.gmail.com>
	<20101201201716.4357a316@pitrou.net>
	<4CF6F3B7.1040805@voidspace.org.uk> <id71ns$9g5$1@dough.gmane.org>
Message-ID: <AANLkTik1sj89c41VusTJajsEtQ6a7CD08xs=uprzBYqy@mail.gmail.com>

On Wed, Dec 1, 2010 at 9:53 PM, Terry Reedy <tjreedy at udel.edu> wrote:
..
> Does Sphinx run on PY3 yet?

It does, but see issue10224 for details.

 http://bugs.python.org/issue10224

From tjreedy at udel.edu  Thu Dec  2 04:09:14 2010
From: tjreedy at udel.edu (Terry Reedy)
Date: Wed, 01 Dec 2010 22:09:14 -0500
Subject: [Python-Dev] Deprecating undocumented,
	unused functions in difflib.
In-Reply-To: <AANLkTimLJQdJQO0vfx9zoJjAAEa=sNtVsWEaXdD-5+2p@mail.gmail.com>
References: <id6ar8$9is$1@dough.gmane.org>
	<AANLkTimLJQdJQO0vfx9zoJjAAEa=sNtVsWEaXdD-5+2p@mail.gmail.com>
Message-ID: <id72kr$cdp$1@dough.gmane.org>

On 12/1/2010 8:22 PM, Nick Coghlan wrote:
> On Thu, Dec 2, 2010 at 6:23 AM, Terry Reedy<tjreedy at udel.edu>  wrote:
>> It would be easiest to just remove the two lines above.
>> Or should I define functions _xxx names that issue a deprecation warning and
>> attach them as attributes to each object? (Defining instance methods would
>> not be the same).
>
> Given that functions are converted to bound methods only on retrieval
> from an instance, why wouldn't it be the same?

The two SequenceMatcher instance attributes are bound functions of the 
two sets, not of the instance. But you are right in the sense that the 
usage would be the same. Since, as of a week ago, the sets were 
implemented as dicts, any code depending on the class of the underlying 
instance is already broken. So I will go with S-M methods and add a doc 
string like "Undocumented, deprecated method that will disappear in 3.3. 
Do not use!" to show up in a help() listing.

> But yes, if you want to get rid of them, then deprecation for 3.2 and
> removal in 3.3 is the way to go.

OK.

> Alternatively, not deprecating them at all and just leaving them
> undocumented with a comment in the source to say they have been
> deliberately omitted from the docs would also be fine.

Too messy and too useless ;-).

-- 
Terry Jan Reedy


From ncoghlan at gmail.com  Thu Dec  2 04:14:12 2010
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Thu, 2 Dec 2010 13:14:12 +1000
Subject: [Python-Dev] [Python-checkins] r86924 -
	python/branches/py3k/Doc/library/random.rst
In-Reply-To: <20101202024133.CCAF1EE985@mail.python.org>
References: <20101202024133.CCAF1EE985@mail.python.org>
Message-ID: <AANLkTi=UKUpBOHRko9i-XQj_ErWUqPNgx3reEXAAxCo_@mail.gmail.com>

On Thu, Dec 2, 2010 at 12:41 PM, raymond.hettinger
<python-checkins at python.org> wrote:
> +A more general approach is to arrange the weights in a cumulative probability
> +distribution with :func:`itertools.accumulate`, and then locate the random value
> +with :func:`bisect.bisect`::
> +
> + ? ?>>> choices, weights = zip(*weighted_choices)
> + ? ?>>> cumdist = list(itertools.accumulate(weights))
> + ? ?>>> x = random.random() * cumdist[-1]
> + ? ?>>> choices[bisect.bisect(cumdist, x)]
> + ? ?'Blue'

Neat example, although it would be easier to follow if you broke that
last line into two pieces:

.>>> random_index = bisect.bisect(cumdist, x)
.>>> choices[random_index]
'Blue'

It took me a moment to remember how bisect.bisect worked, but it would
have been instant if the return value was assigned to an appropriately
named variable.

Cheers,
Nick.

-- 
Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia

From tjreedy at udel.edu  Thu Dec  2 04:11:21 2010
From: tjreedy at udel.edu (Terry Reedy)
Date: Wed, 01 Dec 2010 22:11:21 -0500
Subject: [Python-Dev] Python and the Unicode Character Database
In-Reply-To: <AANLkTikuUki5LgisVHrSMGZtmWZxxy9A8TOwny7dVCMe@mail.gmail.com>
References: <AANLkTinVpySuRFkjf-Sc+7a91zrxhuQpqPQL5nM9u1w+@mail.gmail.com>	<AANLkTinQLK7+5NfEp-2nMiJeDEKRxKN9Zfs4ihv-NMyH@mail.gmail.com>	<87wrnv43v5.fsf@uwakimon.sk.tsukuba.ac.jp>	<id3umc$ps8$1@dough.gmane.org>
	<4CF5841E.6020401@v.loewis.de>	<4CF619C1.8030403@pearwood.info>
	<4CF6CCF9.6000100@v.loewis.de>	<4CF6E5AF.7080308@pearwood.info>
	<AANLkTikuUki5LgisVHrSMGZtmWZxxy9A8TOwny7dVCMe@mail.gmail.com>
Message-ID: <id72oo$cdp$2@dough.gmane.org>

On 12/1/2010 7:44 PM, Alexander Belopolsky wrote:

> it.  The argument was that if there was a use case for parsing Eastern
> Arabic numerals, it would be better served by a module written by
> someone who speaks one of the Arabic languages and knows the details
> of how  Eastern Arabic numerals are written.  So far nobody has even
> claimed to know conclusively that Arabic-Indic digits are always
> written left-to-right.

Both my personal observations when travelling from Turkey to India and 
Wikipedia say yes. "When representing a number in Arabic, the 
lowest-valued position is placed on the right, so the order of positions 
is the same as in left-to-right scripts."
https://secure.wikimedia.org/wikipedia/en/wiki/Arabic_language#Numerals

-- 
Terry Jan Reedy


From alexander.belopolsky at gmail.com  Thu Dec  2 04:28:49 2010
From: alexander.belopolsky at gmail.com (Alexander Belopolsky)
Date: Wed, 1 Dec 2010 22:28:49 -0500
Subject: [Python-Dev] Python and the Unicode Character Database
In-Reply-To: <id72oo$cdp$2@dough.gmane.org>
References: <AANLkTinVpySuRFkjf-Sc+7a91zrxhuQpqPQL5nM9u1w+@mail.gmail.com>
	<AANLkTinQLK7+5NfEp-2nMiJeDEKRxKN9Zfs4ihv-NMyH@mail.gmail.com>
	<87wrnv43v5.fsf@uwakimon.sk.tsukuba.ac.jp>
	<id3umc$ps8$1@dough.gmane.org> <4CF5841E.6020401@v.loewis.de>
	<4CF619C1.8030403@pearwood.info> <4CF6CCF9.6000100@v.loewis.de>
	<4CF6E5AF.7080308@pearwood.info>
	<AANLkTikuUki5LgisVHrSMGZtmWZxxy9A8TOwny7dVCMe@mail.gmail.com>
	<id72oo$cdp$2@dough.gmane.org>
Message-ID: <AANLkTikd0zDjr2GQ3_4C7BLZZstePQAiHee9m2ZpgCEh@mail.gmail.com>

On Wed, Dec 1, 2010 at 10:11 PM, Terry Reedy <tjreedy at udel.edu> wrote:
> On 12/1/2010 7:44 PM, Alexander Belopolsky wrote:
>
>> it. ?The argument was that if there was a use case for parsing Eastern
>> Arabic numerals, it would be better served by a module written by
>> someone who speaks one of the Arabic languages and knows the details
>> of how ?Eastern Arabic numerals are written. ?So far nobody has even
>> claimed to know conclusively that Arabic-Indic digits are always
>> written left-to-right.
>
> Both my personal observations when travelling from Turkey to India and
> Wikipedia say yes. "When representing a number in Arabic, the lowest-valued
> position is placed on the right, so the order of positions is the same as in
> left-to-right scripts."
> https://secure.wikimedia.org/wikipedia/en/wiki/Arabic_language#Numerals

This matches my limited research on this topic as well.  However, I am
not sure that when these codes are embedded in Arabic text, their
logical order always matches their display order.  It seems to me that
it can go either way depending on the surrounding text and/or presence
of explicit formatting codes.  Also, I don't understand why Eastern
Arabic-Indic digits have the same Bidi-Class as European digits, but
Arabic-Indic digits, Arabic decimal and thousands separators have
Bidi-Class "AN".

http://www.unicode.org/reports/tr9/tr9-23.html#Bidirectional_Character_Types

From alexander.belopolsky at gmail.com  Thu Dec  2 05:45:26 2010
From: alexander.belopolsky at gmail.com (Alexander Belopolsky)
Date: Wed, 1 Dec 2010 23:45:26 -0500
Subject: [Python-Dev] ICU
In-Reply-To: <1291147987.8628.13.camel@localhost.localdomain>
References: <AANLkTinVpySuRFkjf-Sc+7a91zrxhuQpqPQL5nM9u1w+@mail.gmail.com>
	<20101128214311.092abd35@pitrou.net> <4CF2D4E9.3060607@v.loewis.de>
	<4CF2F067.5020705@pearwood.info> <4CF354C6.9020302@v.loewis.de>
	<20101129193302.115dbcd5@pitrou.net>
	<AANLkTinodHhm3xfq75eM=_e7uCHcHqC7N_8VMAVbSeJ8@mail.gmail.com>
	<AANLkTi=uBgqpuKwPDB0bvRQv_86GQ4D3sZbrEhiWXUd3@mail.gmail.com>
	<4CF53724.8090000@voidspace.org.uk> <4CF54DA1.5080900@v.loewis.de>
	<1291144993.8628.1.camel@localhost.localdomain>
	<4CF55346.1040108@v.loewis.de>
	<1291146254.8628.4.camel@localhost.localdomain>
	<4CF556C8.9010704@v.loewis.de>
	<1291147987.8628.13.camel@localhost.localdomain>
Message-ID: <AANLkTimrT-S-gSzC0bu4m223EnSB59chWY64xuj7YEbc@mail.gmail.com>

On Tue, Nov 30, 2010 at 3:13 PM, Antoine Pitrou <solipsis at pitrou.net> wrote:
>
> Oh, about ICU:
>
>> > Actually, I remember you saying that locale should ideally be replaced
>> > with a wrapper around the ICU library.
>>
>> By that, I stand - however, I have given up the hope that this will
>> happen anytime soon.
>
> Perhaps this could be made a GSOC topic.
>

Incidentally, this may also address another Python's Achilles' heel:
the timezone support.

http://icu-project.org/download/icutzu.html

From a.badger at gmail.com  Thu Dec  2 06:11:27 2010
From: a.badger at gmail.com (Toshio Kuratomi)
Date: Wed, 1 Dec 2010 21:11:27 -0800
Subject: [Python-Dev] Porting Ideas
In-Reply-To: <AANLkTik1sj89c41VusTJajsEtQ6a7CD08xs=uprzBYqy@mail.gmail.com>
References: <AANLkTing_iT7Y6FOZPWh=CUwtK8owvYtSdE7aXf0W181@mail.gmail.com>
	<AANLkTi=6rvXDEfzmMC=vE5jaCaTzAwnDddxHrZ2GAB1b@mail.gmail.com>
	<20101201201716.4357a316@pitrou.net>
	<4CF6F3B7.1040805@voidspace.org.uk> <id71ns$9g5$1@dough.gmane.org>
	<AANLkTik1sj89c41VusTJajsEtQ6a7CD08xs=uprzBYqy@mail.gmail.com>
Message-ID: <20101202051127.GG6965@unaka.lan>

On Wed, Dec 01, 2010 at 10:06:24PM -0500, Alexander Belopolsky wrote:
> On Wed, Dec 1, 2010 at 9:53 PM, Terry Reedy <tjreedy at udel.edu> wrote:
> ..
> > Does Sphinx run on PY3 yet?
> 
> It does, but see issue10224 for details.
> 
>  http://bugs.python.org/issue10224
>
Also, docutils has an unported module.

/me needs to write a bug report for that as he really doesn't have the time
he thought he did to perform the port.

-Toshio
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 198 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/python-dev/attachments/20101201/2f4af6c8/attachment.pgp>

From stephen at xemacs.org  Thu Dec  2 08:49:24 2010
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Thu, 02 Dec 2010 16:49:24 +0900
Subject: [Python-Dev] Python and the Unicode Character Database
In-Reply-To: <87bp552arq.fsf@benfinney.id.au>
References: <AANLkTinVpySuRFkjf-Sc+7a91zrxhuQpqPQL5nM9u1w+@mail.gmail.com>
	<AANLkTinQLK7+5NfEp-2nMiJeDEKRxKN9Zfs4ihv-NMyH@mail.gmail.com>
	<87wrnv43v5.fsf@uwakimon.sk.tsukuba.ac.jp>
	<AANLkTinAf6CstywehOGRd4G4ms6K5LtSuq4Lj0z5wFBW@mail.gmail.com>
	<87r5e236hj.fsf@benfinney.id.au> <4CF58387.1000609@v.loewis.de>
	<87k4ju2vvg.fsf@benfinney.id.au> <4CF6CDDD.4060109@v.loewis.de>
	<4CF6D7B2.6030506@pearwood.info>
	<87sjyhxarg.fsf@uwakimon.sk.tsukuba.ac.jp>
	<87bp552arq.fsf@benfinney.id.au>
Message-ID: <87oc94y5q3.fsf@uwakimon.sk.tsukuba.ac.jp>

Ben Finney writes:

 > Input from an existing text file, as I said earlier. Or any other way of
 > text data making its way into a Python program.

 > Direct entry at the console is a red herring.

I don't think it is.  Not at all.  Here's why: '''print "%d" %
some_integer''' doesn't now, and never will (unless Kristan gets his
Python 2.8<wink>), produce Arabic or Han numerals.  Not in any
language I know of, not in Microsoft Excel, and definitely not in
Python 2.  *Somebody* typed that text at some point.  If it's Han,
that somebody had *way* too much time on his hands, not a working
accountant nor a graduate assistant in a research lab for sure.

How about old archived texts, copied and recopied?  At least for
Japanese, old archival (text) data will *all* be in ASCII, because the
earliest implementations of Japanese language text used JIS X 0201 (or
its predecessor), which doesn't have Han digits (and kana digits don't
exist even if you write with a brush and ink AFAIK).  Ditto Arabic, I
would imagine; ISO 8859/6 (aka Latin/Arabic) does not contain the
Arabic digits that have been presented here earlier AFAICT.  Note that
there's plenty of space for them in that code table (eg, 0xB0-0xB9 is
empty).  Apparently nobody *ever* thought it was useful to have them!

So, which culture, using which script and in which application, inputs
numeric data in other than ASCII digits?  Or would want to, if only
somebody would tell them they can do it in Python?  Hearsay will do,
for starters.

From ncoghlan at gmail.com  Thu Dec  2 10:32:19 2010
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Thu, 2 Dec 2010 19:32:19 +1000
Subject: [Python-Dev] [Python-checkins] r86930 - in
 python/branches/py3k: Doc/library/os.rst Lib/os.py Lib/test/test_os.py
 Misc/ACKS Misc/NEWS
In-Reply-To: <20101202070557.1D971EE98C@mail.python.org>
References: <20101202070557.1D971EE98C@mail.python.org>
Message-ID: <AANLkTikZf_cvFF99w9YJ7jVmSLpJ_rtyiy14GE_Dhu+D@mail.gmail.com>

On Thu, Dec 2, 2010 at 5:05 PM, terry.reedy <python-checkins at python.org> wrote:
> +   If
> + ? the target directory with the same mode as we specified already exists,
> + ? raises an :exc:`OSError` exception if *exist_ok* is False, otherwise no
> + ? exception is raised. ?If the directory cannot be created in other cases,
> + ? raises an :exc:`OSError` exception.

I would suggest being explicit here that "directory exists, but has a
mode other than the one requested" always triggers an exception.
Perhaps something like the following:

"Raises an :exc:`OSError` exception if the target directory already
exists, unless *exist_ok* is True and the existing directory has the
same mode as is specified in the current call.  Also raises an
:exc:`OSError` exception if the directory cannot be created for any
other reason."

Cheers,
Nick.

-- 
Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia

From sable at users.sourceforge.net  Thu Dec  2 11:13:44 2010
From: sable at users.sourceforge.net (=?ISO-8859-1?Q?S=E9bastien_Sabl=E9?=)
Date: Thu, 02 Dec 2010 11:13:44 +0100
Subject: [Python-Dev] AIX 5.3 - Enabling Shared Library Support Vs
	Extensions
In-Reply-To: <AANLkTikkr7mmqgj5VKCFSmQcH--uJZiARQNLxq2vbtwV@mail.gmail.com>
References: <AANLkTikkr7mmqgj5VKCFSmQcH--uJZiARQNLxq2vbtwV@mail.gmail.com>
Message-ID: <4CF77158.3010005@users.sourceforge.net>


Hi Anurag,

Le 25/11/2010 10:24, Anurag Chourasia a ?crit :
 > All,
 >
 > When I configure python to enable shared libraries, none of the 
extensions are getting built during the make step due to this error.
 >

you may want to take a look at the following issue:

http://bugs.python.org/issue941346

Python compiled with shared libraries was broken on AIX until recently. 
There are some patches there to get it to work, or you may want to test 
the latest 2.7 or 3.x releases.

regards

--
S?bastien Sabl?

From merwok at netwok.org  Thu Dec  2 13:18:53 2010
From: merwok at netwok.org (=?UTF-8?B?w4lyaWMgQXJhdWpv?=)
Date: Thu, 02 Dec 2010 13:18:53 +0100
Subject: [Python-Dev] [Python-checkins] r86924 -
	python/branches/py3k/Doc/library/random.rst
In-Reply-To: <AANLkTi=UKUpBOHRko9i-XQj_ErWUqPNgx3reEXAAxCo_@mail.gmail.com>
References: <20101202024133.CCAF1EE985@mail.python.org>
	<AANLkTi=UKUpBOHRko9i-XQj_ErWUqPNgx3reEXAAxCo_@mail.gmail.com>
Message-ID: <4CF78EAD.1060408@netwok.org>

> On Thu, Dec 2, 2010 at 12:41 PM, raymond.hettinger
> <python-checkins at python.org> wrote:
>> +A more general approach is to arrange the weights in a cumulative probability
>> +distribution with :func:`itertools.accumulate`, and then locate the random value
>> +with :func:`bisect.bisect`::
>> +
>> +    >>> choices, weights = zip(*weighted_choices)
>> +    >>> cumdist = list(itertools.accumulate(weights))
>> +    >>> x = random.random() * cumdist[-1]
>> +    >>> choices[bisect.bisect(cumdist, x)]
>> +    'Blue'

?pydoc bisect.bisect? is empty (?Alias for bisect_right()?); in the
code, bisect.bisect is noted as compatibility alias.  Wouldn?t it be
more helpful to use the newer name?

Regards


From nyamatongwe at gmail.com  Thu Dec  2 13:40:45 2010
From: nyamatongwe at gmail.com (Neil Hodgson)
Date: Thu, 2 Dec 2010 23:40:45 +1100
Subject: [Python-Dev] Python and the Unicode Character Database
In-Reply-To: <87oc94y5q3.fsf@uwakimon.sk.tsukuba.ac.jp>
References: <AANLkTinVpySuRFkjf-Sc+7a91zrxhuQpqPQL5nM9u1w+@mail.gmail.com>
	<AANLkTinQLK7+5NfEp-2nMiJeDEKRxKN9Zfs4ihv-NMyH@mail.gmail.com>
	<87wrnv43v5.fsf@uwakimon.sk.tsukuba.ac.jp>
	<AANLkTinAf6CstywehOGRd4G4ms6K5LtSuq4Lj0z5wFBW@mail.gmail.com>
	<87r5e236hj.fsf@benfinney.id.au> <4CF58387.1000609@v.loewis.de>
	<87k4ju2vvg.fsf@benfinney.id.au> <4CF6CDDD.4060109@v.loewis.de>
	<4CF6D7B2.6030506@pearwood.info>
	<87sjyhxarg.fsf@uwakimon.sk.tsukuba.ac.jp>
	<87bp552arq.fsf@benfinney.id.au>
	<87oc94y5q3.fsf@uwakimon.sk.tsukuba.ac.jp>
Message-ID: <AANLkTikuFdtbqXF0tj0mRENBciHwra1iR6ee7fTr79oL@mail.gmail.com>

Stephen J. Turnbull:

> Here's why: '''print "%d" %
> some_integer''' doesn't now, and never will (unless Kristan gets his
> Python 2.8<wink>), produce Arabic or Han numerals. ?Not in any
> language I know of, not in Microsoft Excel, and definitely not in
> Python 2.

   While I don't have Excel to test with, OpenOffice.org Calc will
display in Arabic or Han numerals using the NatNum format codes.
http://www.scintilla.org/ArabicNumbers.png

> Ditto Arabic, I
> would imagine; ISO 8859/6 (aka Latin/Arabic) does not contain the
> Arabic digits that have been presented here earlier AFAICT. ?Note that
> there's plenty of space for them in that code table (eg, 0xB0-0xB9 is
> empty). ?Apparently nobody *ever* thought it was useful to have them!

   DOS code page 864 does use 0xB0-0xB9 for ? .. ?.
http://www.ascii.ca/cp864.htm

   Neil

From g.brandl at gmx.net  Thu Dec  2 13:57:41 2010
From: g.brandl at gmx.net (Georg Brandl)
Date: Thu, 02 Dec 2010 13:57:41 +0100
Subject: [Python-Dev] Python and the Unicode Character Database
In-Reply-To: <4CF6CE95.1060707@v.loewis.de>
References: <AANLkTinVpySuRFkjf-Sc+7a91zrxhuQpqPQL5nM9u1w+@mail.gmail.com>	<4CF2DC5B.4020702@egenix.com>	<AANLkTimB+FkPxHyC_udiCr-oVmSG4jnZ6USUHk14_Vh-@mail.gmail.com>
	<4CF6CE95.1060707@v.loewis.de>
Message-ID: <id857u$m91$1@dough.gmane.org>

Am 01.12.2010 23:39, schrieb "Martin v. L?wis":
>> As of today, "What?s New In Python 3.2" [1] does not even mention the
>> unicodedata upgrade to 6.0.0.
> 
> One reason was that I was instructed not to change "What's New" a few
> years ago.

Maybe all past, present and future whatsnew maintainers can agree on these
rules, which I copied directly from whatsnew/3.2.rst?

   Rules for maintenance:

   * Anyone can add text to this document.  Do not spend very much time
   on the wording of your changes, because your text will probably
   get rewritten to some degree.

   * The maintainer will go through Misc/NEWS periodically and add
   changes; it's therefore more important to add your changes to
   Misc/NEWS than to this file.

   * This is not a complete list of every single change; completeness
   is the purpose of Misc/NEWS.  Some changes I consider too small
   or esoteric to include.  If such a change is added to the text,
   I'll just remove it.  (This is another reason you shouldn't spend
   too much time on writing your addition.)

   * If you want to draw your new text to the attention of the
   maintainer, add 'XXX' to the beginning of the paragraph or
   section.

   * It's OK to just add a fragmentary note about a change.  For
   example: "XXX Describe the transmogrify() function added to the
   socket module."  The maintainer will research the change and
   write the necessary text.

   * You can comment out your additions if you like, but it's not
   necessary (especially when a final release is some months away).

   * Credit the author of a patch or bugfix.   Just the name is
   sufficient; the e-mail address isn't necessary.  It's helpful to
   add the issue number:

     XXX Describe the transmogrify() function added to the socket
     module.

     (Contributed by P.Y. Developer; :issue:`12345`.)

   This saves the maintainer the effort of going through the SVN log
   when researching a change.

Georg


From ziade.tarek at gmail.com  Thu Dec  2 14:08:37 2010
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Thu, 2 Dec 2010 14:08:37 +0100
Subject: [Python-Dev] Change to the Distutils / Distutils2 workflow
Message-ID: <AANLkTi=njm=wX8++HH+YGsZE5vDP6YGbwNf+0fyp+u3-@mail.gmail.com>

Hey

We discussed with Eric about the debugging workflow and we agreed that
our life would be easier if every bug fix would land first in
Distutils2 when it makes sense, then get backported to Distutils1.

For other core-devs that would mean that your patches should be done
against hg.python.org/distutils2, which uses unittest2. Then Eric and
I would take care of the backporting.

I am planning to set up a wiki page with the workflow as soon as I get a chance.

Thanks
Tarek

-- 
Tarek Ziad? | http://ziade.org

From regebro at gmail.com  Thu Dec  2 14:28:33 2010
From: regebro at gmail.com (Lennart Regebro)
Date: Thu, 2 Dec 2010 14:28:33 +0100
Subject: [Python-Dev] Python and the Unicode Character Database
In-Reply-To: <87tyixxbet.fsf@uwakimon.sk.tsukuba.ac.jp>
References: <AANLkTinVpySuRFkjf-Sc+7a91zrxhuQpqPQL5nM9u1w+@mail.gmail.com>
	<AANLkTinQLK7+5NfEp-2nMiJeDEKRxKN9Zfs4ihv-NMyH@mail.gmail.com>
	<87wrnv43v5.fsf@uwakimon.sk.tsukuba.ac.jp>
	<AANLkTimnar7CuHKXtvFtspTphmB3GrNMiA59zfFhTeoq@mail.gmail.com>
	<87tyixxbet.fsf@uwakimon.sk.tsukuba.ac.jp>
Message-ID: <AANLkTik68+CgkVimhL2saVPYem6Fvt5emM_eYwJr7gqz@mail.gmail.com>

2010/12/2 Stephen J. Turnbull <stephen at xemacs.org>:
> Because that works, but
>
> print(T1234)
>
> doesn't (it prints ASCII). ?You can't round-trip, but users will
> want/expect that.

You should be able to round-trip, absolutely. I don't think you should
expect print() to do that. str(56) possibly. :)
That's an argument for it to be in a module, as you then would need to
send in a parameter on which decimal characters you want.

> T1000 = float('?.???')

That was already discussed here, and it's clear that unicode does not
consider these characters to be something you can use in a decimal
number, and hence it's not broken.

From solipsis at pitrou.net  Thu Dec  2 14:36:33 2010
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Thu, 2 Dec 2010 14:36:33 +0100
Subject: [Python-Dev] Python and the Unicode Character Database
References: <AANLkTinVpySuRFkjf-Sc+7a91zrxhuQpqPQL5nM9u1w+@mail.gmail.com>
	<AANLkTinQLK7+5NfEp-2nMiJeDEKRxKN9Zfs4ihv-NMyH@mail.gmail.com>
	<87wrnv43v5.fsf@uwakimon.sk.tsukuba.ac.jp>
	<id3umc$ps8$1@dough.gmane.org> <4CF5841E.6020401@v.loewis.de>
	<4CF619C1.8030403@pearwood.info> <4CF6CCF9.6000100@v.loewis.de>
	<4CF6E5AF.7080308@pearwood.info>
	<AANLkTikuUki5LgisVHrSMGZtmWZxxy9A8TOwny7dVCMe@mail.gmail.com>
	<id72oo$cdp$2@dough.gmane.org>
	<AANLkTikd0zDjr2GQ3_4C7BLZZstePQAiHee9m2ZpgCEh@mail.gmail.com>
Message-ID: <20101202143633.6eb17b8d@pitrou.net>

On Wed, 1 Dec 2010 22:28:49 -0500
Alexander Belopolsky <alexander.belopolsky at gmail.com> wrote:
> >
> > Both my personal observations when travelling from Turkey to India and
> > Wikipedia say yes. "When representing a number in Arabic, the lowest-valued
> > position is placed on the right, so the order of positions is the same as in
> > left-to-right scripts."
> > https://secure.wikimedia.org/wikipedia/en/wiki/Arabic_language#Numerals
> 
> This matches my limited research on this topic as well.  However, I am
> not sure that when these codes are embedded in Arabic text, their
> logical order always matches their display order.

That shouldn't matter, since unicode text follows logical order. The
display order is up to the graphical representation library.

Regards

Antoine.



From regebro at gmail.com  Thu Dec  2 14:36:33 2010
From: regebro at gmail.com (Lennart Regebro)
Date: Thu, 2 Dec 2010 14:36:33 +0100
Subject: [Python-Dev] Porting Ideas
In-Reply-To: <20101201201716.4357a316@pitrou.net>
References: <AANLkTing_iT7Y6FOZPWh=CUwtK8owvYtSdE7aXf0W181@mail.gmail.com>
	<AANLkTi=6rvXDEfzmMC=vE5jaCaTzAwnDddxHrZ2GAB1b@mail.gmail.com>
	<20101201201716.4357a316@pitrou.net>
Message-ID: <AANLkTi=u-YAuEkNcBmkwUzCScoeedFSYno_RC2YRMvEy@mail.gmail.com>

On Wed, Dec 1, 2010 at 20:17, Antoine Pitrou <solipsis at pitrou.net> wrote:
> And I'm not sure what this package called "Python" is (?a high-level
> object-oriented programming language?? like Java?), but I'm pretty sure
> I've heard there's a Python 3 compatible version.

Uhm... http://pypi.python.org/pypi/Python

Anybody wanna remove that, or update it or something? :-)

From anurag.chourasia at gmail.com  Thu Dec  2 15:11:07 2010
From: anurag.chourasia at gmail.com (Anurag Chourasia)
Date: Thu, 2 Dec 2010 19:41:07 +0530
Subject: [Python-Dev] AIX 5.3 - Enabling Shared Library Support Vs
	Extensions
In-Reply-To: <4CF77158.3010005@users.sourceforge.net>
References: <AANLkTikkr7mmqgj5VKCFSmQcH--uJZiARQNLxq2vbtwV@mail.gmail.com>
	<4CF77158.3010005@users.sourceforge.net>
Message-ID: <AANLkTin7W4JKCGizs0NQq=BxN0ivPqUbr+zopzsrYH=3@mail.gmail.com>

Hi Sebastian,

Thanks for your response.

I  looked at http://bugs.python.org/issue941346 earlier. I was referred to
this link by Stefan Krah through another bug that i created at
http://bugs.python.org/issue10555 for this issue.

I confirm that my problem is solved with the Python 2.7.1 release which
contains the changes done by you.

Great work done by you and other folks for enabling the Shared Library build
on AIX. Hats Off !!!

Regards,
Anurag

2010/12/2 S?bastien Sabl? <sable at users.sourceforge.net>

>
> Hi Anurag,
>
> Le 25/11/2010 10:24, Anurag Chourasia a ?crit :
>
> > All,
> >
> > When I configure python to enable shared libraries, none of the
> extensions are getting built during the make step due to this error.
> >
>
> you may want to take a look at the following issue:
>
> http://bugs.python.org/issue941346
>
> Python compiled with shared libraries was broken on AIX until recently.
> There are some patches there to get it to work, or you may want to test the
> latest 2.7 or 3.x releases.
>
> regards
>
> --
> S?bastien Sabl?
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20101202/d442dbf0/attachment.html>

From guido at python.org  Thu Dec  2 16:47:08 2010
From: guido at python.org (Guido van Rossum)
Date: Thu, 2 Dec 2010 07:47:08 -0800
Subject: [Python-Dev] ICU
In-Reply-To: <AANLkTimrT-S-gSzC0bu4m223EnSB59chWY64xuj7YEbc@mail.gmail.com>
References: <AANLkTinVpySuRFkjf-Sc+7a91zrxhuQpqPQL5nM9u1w+@mail.gmail.com>
	<20101128214311.092abd35@pitrou.net> <4CF2D4E9.3060607@v.loewis.de>
	<4CF2F067.5020705@pearwood.info> <4CF354C6.9020302@v.loewis.de>
	<20101129193302.115dbcd5@pitrou.net>
	<AANLkTinodHhm3xfq75eM=_e7uCHcHqC7N_8VMAVbSeJ8@mail.gmail.com>
	<AANLkTi=uBgqpuKwPDB0bvRQv_86GQ4D3sZbrEhiWXUd3@mail.gmail.com>
	<4CF53724.8090000@voidspace.org.uk> <4CF54DA1.5080900@v.loewis.de>
	<1291144993.8628.1.camel@localhost.localdomain>
	<4CF55346.1040108@v.loewis.de>
	<1291146254.8628.4.camel@localhost.localdomain>
	<4CF556C8.9010704@v.loewis.de>
	<1291147987.8628.13.camel@localhost.localdomain>
	<AANLkTimrT-S-gSzC0bu4m223EnSB59chWY64xuj7YEbc@mail.gmail.com>
Message-ID: <AANLkTin6_xNQ5xw6LHWWSbhA_a-YgjAnKDucPVrHOndR@mail.gmail.com>

On Wed, Dec 1, 2010 at 8:45 PM, Alexander Belopolsky
<alexander.belopolsky at gmail.com> wrote:
> On Tue, Nov 30, 2010 at 3:13 PM, Antoine Pitrou <solipsis at pitrou.net> wrote:
>>
>> Oh, about ICU:
>>
>>> > Actually, I remember you saying that locale should ideally be replaced
>>> > with a wrapper around the ICU library.
>>>
>>> By that, I stand - however, I have given up the hope that this will
>>> happen anytime soon.
>>
>> Perhaps this could be made a GSOC topic.
>>
>
> Incidentally, this may also address another Python's Achilles' heel:
> the timezone support.
>
> http://icu-project.org/download/icutzu.html

I work with people who speak highly of ICU, so I want to encourage
work in this area.

At the same time, I'm skeptical -- IIRC, ICU is a large amount of C++
code. I don't know how easy it will be to integrate this into our
build processes for various platforms, nor how "Pythonic" the
resulting APIs will look to the experienced Python user.

Still, those are not roadblocks, the benefits are potentially great,
so it's definitely worth investigating!

-- 
--Guido van Rossum (python.org/~guido)

From ocean-city at m2.ccsnet.ne.jp  Thu Dec  2 16:50:51 2010
From: ocean-city at m2.ccsnet.ne.jp (Hirokazu Yamamoto)
Date: Fri, 03 Dec 2010 00:50:51 +0900
Subject: [Python-Dev] [Python-checkins] r86817 -
	python/branches/py3k-stat-on-windows/Lib/test/test_shutil.py
In-Reply-To: <AANLkTim8Mume09AOR8jtosHtXMBTKCNqC0=_o0UsW1DG@mail.gmail.com>
References: <20101126184428.E04A0EE984@mail.python.org>	<AANLkTimqKxXMe+H4tf9Luca2MKTyAyNAFYWE4oc_oPhS@mail.gmail.com>	<4CF00E4E.6030507@m2.ccsnet.ne.jp>	<AANLkTin3bWRB7aycviHG9X8vvOKMk8HFtoiPvuXZ7bge@mail.gmail.com>	<4CF01632.8070504@m2.ccsnet.ne.jp>
	<AANLkTim8Mume09AOR8jtosHtXMBTKCNqC0=_o0UsW1DG@mail.gmail.com>
Message-ID: <4CF7C05B.7030909@m2.ccsnet.ne.jp>

On 2010/11/27 5:31, Brian Curtin wrote:
> On Fri, Nov 26, 2010 at 14:18, Hirokazu Yamamoto<ocean-city at m2.ccsnet.ne.jp
>> wrote:
>
>> On 2010/11/27 5:02, Brian Curtin wrote:
>>
>>> We briefly chatted about this on the os.link
>>> feature issue, but I never found a way around it.
>>>
>>
>> How about implementing os.path.samefile in
>> Modules/posixmodule.c like this?
>>
>> http://bugs.python.org/file19262/py3k_fix_kill_python_for_short_path.patch
>>
>> # I hope this works.
>>
>
> That's almost identical to what the current os.path.sameopenfile is.
>
> Lib/ntpath.py opens both files, then compares them via _getfileinformation.
> That function is implemented to take in a file descriptor, call
> GetFileInformationByHandle with it, then returns a tuple
> of dwVolumeSerialNumber, nFileIndexHigh, and nFileIndexLow.
>

Yes. Difference is, file object cannot represent directory,
and probably FILE_FLAG_BACKUP_SEMANTICS makes it faster to open file.

From foom at fuhm.net  Thu Dec  2 17:15:54 2010
From: foom at fuhm.net (James Y Knight)
Date: Thu, 2 Dec 2010 11:15:54 -0500
Subject: [Python-Dev] ICU
In-Reply-To: <AANLkTimrT-S-gSzC0bu4m223EnSB59chWY64xuj7YEbc@mail.gmail.com>
References: <AANLkTinVpySuRFkjf-Sc+7a91zrxhuQpqPQL5nM9u1w+@mail.gmail.com>
	<20101128214311.092abd35@pitrou.net>
	<4CF2D4E9.3060607@v.loewis.de> <4CF2F067.5020705@pearwood.info>
	<4CF354C6.9020302@v.loewis.de> <20101129193302.115dbcd5@pitrou.net>
	<AANLkTinodHhm3xfq75eM=_e7uCHcHqC7N_8VMAVbSeJ8@mail.gmail.com>
	<AANLkTi=uBgqpuKwPDB0bvRQv_86GQ4D3sZbrEhiWXUd3@mail.gmail.com>
	<4CF53724.8090000@voidspace.org.uk> <4CF54DA1.5080900@v.loewis.de>
	<1291144993.8628.1.camel@localhost.localdomain>
	<4CF55346.1040108@v.loewis.de>
	<1291146254.8628.4.camel@localhost.localdomain>
	<4CF556C8.9010704@v.loewis.de>
	<1291147987.8628.13.camel@localhost.localdomain>
	<AANLkTimrT-S-gSzC0bu4m223EnSB59chWY64xuj7YEbc@mail.gmail.com>
Message-ID: <96F96B19-67B5-4526-9553-B517C1D0AE38@fuhm.net>


On Dec 1, 2010, at 11:45 PM, Alexander Belopolsky wrote:

> On Tue, Nov 30, 2010 at 3:13 PM, Antoine Pitrou <solipsis at pitrou.net> wrote:
>> 
>> Oh, about ICU:
>> 
>>>> Actually, I remember you saying that locale should ideally be replaced
>>>> with a wrapper around the ICU library.
>>> 
>>> By that, I stand - however, I have given up the hope that this will
>>> happen anytime soon.
>> 
>> Perhaps this could be made a GSOC topic.
>> 
> 
> Incidentally, this may also address another Python's Achilles' heel:
> the timezone support.
> 
> http://icu-project.org/download/icutzu.html

Does ICU do anything regarding timezones that datetime + pytz doesn't already do? Wouldn't it make more sense to integrate the already-existing-and-pythonic pytz into Python than to make a new wrapper based on ICU?

James


From alexander.belopolsky at gmail.com  Thu Dec  2 17:41:11 2010
From: alexander.belopolsky at gmail.com (Alexander Belopolsky)
Date: Thu, 2 Dec 2010 11:41:11 -0500
Subject: [Python-Dev] Python and the Unicode Character Database
In-Reply-To: <20101202143633.6eb17b8d@pitrou.net>
References: <AANLkTinVpySuRFkjf-Sc+7a91zrxhuQpqPQL5nM9u1w+@mail.gmail.com>
	<AANLkTinQLK7+5NfEp-2nMiJeDEKRxKN9Zfs4ihv-NMyH@mail.gmail.com>
	<87wrnv43v5.fsf@uwakimon.sk.tsukuba.ac.jp>
	<id3umc$ps8$1@dough.gmane.org> <4CF5841E.6020401@v.loewis.de>
	<4CF619C1.8030403@pearwood.info> <4CF6CCF9.6000100@v.loewis.de>
	<4CF6E5AF.7080308@pearwood.info>
	<AANLkTikuUki5LgisVHrSMGZtmWZxxy9A8TOwny7dVCMe@mail.gmail.com>
	<id72oo$cdp$2@dough.gmane.org>
	<AANLkTikd0zDjr2GQ3_4C7BLZZstePQAiHee9m2ZpgCEh@mail.gmail.com>
	<20101202143633.6eb17b8d@pitrou.net>
Message-ID: <AANLkTikiziBN87-7gM3rweODnXjEuK3DfhEni1sTDYtJ@mail.gmail.com>

On Thu, Dec 2, 2010 at 8:36 AM, Antoine Pitrou <solipsis at pitrou.net> wrote:
> On Wed, 1 Dec 2010 22:28:49 -0500
> Alexander Belopolsky <alexander.belopolsky at gmail.com> wrote:
..
>> This matches my limited research on this topic as well. ?However, I am
>> not sure that when these codes are embedded in Arabic text, their
>> logical order always matches their display order.
>
> That shouldn't matter, since unicode text follows logical order. The
> display order is up to the graphical representation library.
>

I am not so sure.  On my Mac, U+200F (RIGHT-TO-LEFT MARK) affects 0-9
and Arabic-Indic decimals differently:

>>> print('\u200F123')
?123
>>> print('\u200F\u0661\u0662\u0663')
231

I replaced Arabic-Indic decimals with 0-9 in the output to demonstrate
the point.  Cut-n-paste does not work well in the presence of RTL
directives.

and U+202E (RIGHT-TO-LEFT OVERRIDE) reverts the display order for both:

>>> print('\u202E123')
321
>>> print('\u202E\u0661\u0662\u0663')
321

(again, the output display is simulated not copied.)  I don't know if
explicit RTL directives are ever used in Arabic texts, but it is quite
possible that texts converted from older formats would use them for
efficiency.

Note that my point is not to find the correct answer here, but to
demonstrate that we as a group don't have the expertise to get parsing
of Arabic text right.  If we've got it right for Arabic, it is by
chance and not by design.  This still leaves us with 41 other types of
digits for at least 30 different languages.  Nobody will ever assume
that python builtins are suitable for use with all these variants.
This "feature" is only good for nefarious purposes such as hiding
extra digits in innocent-looking files or smuggling binary data
through naive interfaces.

PS: BTW, shouldn't int('\u0661\u0662\u06DD') be valid? or is it
int('\u06DD\u0661\u0662')?

From solipsis at pitrou.net  Thu Dec  2 17:56:00 2010
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Thu, 02 Dec 2010 17:56:00 +0100
Subject: [Python-Dev] Python and the Unicode Character Database
In-Reply-To: <AANLkTikiziBN87-7gM3rweODnXjEuK3DfhEni1sTDYtJ@mail.gmail.com>
References: <AANLkTinVpySuRFkjf-Sc+7a91zrxhuQpqPQL5nM9u1w+@mail.gmail.com>
	<AANLkTinQLK7+5NfEp-2nMiJeDEKRxKN9Zfs4ihv-NMyH@mail.gmail.com>
	<87wrnv43v5.fsf@uwakimon.sk.tsukuba.ac.jp>
	<id3umc$ps8$1@dough.gmane.org>
	<4CF5841E.6020401@v.loewis.de> <4CF619C1.8030403@pearwood.info>
	<4CF6CCF9.6000100@v.loewis.de> <4CF6E5AF.7080308@pearwood.info>
	<AANLkTikuUki5LgisVHrSMGZtmWZxxy9A8TOwny7dVCMe@mail.gmail.com>
	<id72oo$cdp$2@dough.gmane.org>
	<AANLkTikd0zDjr2GQ3_4C7BLZZstePQAiHee9m2ZpgCEh@mail.gmail.com>
	<20101202143633.6eb17b8d@pitrou.net>
	<AANLkTikiziBN87-7gM3rweODnXjEuK3DfhEni1sTDYtJ@mail.gmail.com>
Message-ID: <1291308960.3576.28.camel@localhost.localdomain>

Le jeudi 02 d?cembre 2010 ? 11:41 -0500, Alexander Belopolsky a ?crit :
> 
> Note that my point is not to find the correct answer here, but to
> demonstrate that we as a group don't have the expertise to get parsing
> of Arabic text right.

I don't understand why you think Arabic or Hebrew text is any different
from Western text. Surely right-to-left isn't more conceptually
complicated than left-to-right, is it?

The fact that mixed rtl + ltr can render bizarrely or is awkward to cut
and paste is quite off-topic for our discussion.

> If we've got it right for Arabic, it is by
> chance and not by design.  This still leaves us with 41 other types of
> digits for at least 30 different languages.

So why do you trust the Unicode standard on other things and not on this
one?

Regards

Antoine.



From benjamin at python.org  Thu Dec  2 18:03:57 2010
From: benjamin at python.org (Benjamin Peterson)
Date: Thu, 2 Dec 2010 11:03:57 -0600
Subject: [Python-Dev] ICU
In-Reply-To: <AANLkTin6_xNQ5xw6LHWWSbhA_a-YgjAnKDucPVrHOndR@mail.gmail.com>
References: <AANLkTinVpySuRFkjf-Sc+7a91zrxhuQpqPQL5nM9u1w+@mail.gmail.com>
	<20101128214311.092abd35@pitrou.net> <4CF2D4E9.3060607@v.loewis.de>
	<4CF2F067.5020705@pearwood.info> <4CF354C6.9020302@v.loewis.de>
	<20101129193302.115dbcd5@pitrou.net>
	<AANLkTinodHhm3xfq75eM=_e7uCHcHqC7N_8VMAVbSeJ8@mail.gmail.com>
	<AANLkTi=uBgqpuKwPDB0bvRQv_86GQ4D3sZbrEhiWXUd3@mail.gmail.com>
	<4CF53724.8090000@voidspace.org.uk> <4CF54DA1.5080900@v.loewis.de>
	<1291144993.8628.1.camel@localhost.localdomain>
	<4CF55346.1040108@v.loewis.de>
	<1291146254.8628.4.camel@localhost.localdomain>
	<4CF556C8.9010704@v.loewis.de>
	<1291147987.8628.13.camel@localhost.localdomain>
	<AANLkTimrT-S-gSzC0bu4m223EnSB59chWY64xuj7YEbc@mail.gmail.com>
	<AANLkTin6_xNQ5xw6LHWWSbhA_a-YgjAnKDucPVrHOndR@mail.gmail.com>
Message-ID: <AANLkTi=SOr2O9YVvAJhW7B+6iwzoTu++jt7bL3tS+yeg@mail.gmail.com>

2010/12/2 Guido van Rossum <guido at python.org>:
> On Wed, Dec 1, 2010 at 8:45 PM, Alexander Belopolsky
> <alexander.belopolsky at gmail.com> wrote:
>> On Tue, Nov 30, 2010 at 3:13 PM, Antoine Pitrou <solipsis at pitrou.net> wrote:
>>>
>>> Oh, about ICU:
>>>
>>>> > Actually, I remember you saying that locale should ideally be replaced
>>>> > with a wrapper around the ICU library.
>>>>
>>>> By that, I stand - however, I have given up the hope that this will
>>>> happen anytime soon.
>>>
>>> Perhaps this could be made a GSOC topic.
>>>
>>
>> Incidentally, this may also address another Python's Achilles' heel:
>> the timezone support.
>>
>> http://icu-project.org/download/icutzu.html
>
> I work with people who speak highly of ICU, so I want to encourage
> work in this area.
>
> At the same time, I'm skeptical -- IIRC, ICU is a large amount of C++
> code. I don't know how easy it will be to integrate this into our
> build processes for various platforms, nor how "Pythonic" the
> resulting APIs will look to the experienced Python user.

There's a nice C-API.



-- 
Regards,
Benjamin

From tjreedy at udel.edu  Thu Dec  2 18:34:27 2010
From: tjreedy at udel.edu (Terry Reedy)
Date: Thu, 02 Dec 2010 12:34:27 -0500
Subject: [Python-Dev] [Python-checkins] r86930 - in
 python/branches/py3k: Doc/library/os.rst Lib/os.py Lib/test/test_os.py
 Misc/ACKS Misc/NEWS
In-Reply-To: <AANLkTikZf_cvFF99w9YJ7jVmSLpJ_rtyiy14GE_Dhu+D@mail.gmail.com>
References: <20101202070557.1D971EE98C@mail.python.org>
	<AANLkTikZf_cvFF99w9YJ7jVmSLpJ_rtyiy14GE_Dhu+D@mail.gmail.com>
Message-ID: <4CF7D8A3.3050800@udel.edu>



On 12/2/2010 4:32 AM, Nick Coghlan wrote:
> On Thu, Dec 2, 2010 at 5:05 PM, terry.reedy<python-checkins at python.org>  wrote:
(except I did not write most of the patch)

>> +   If
>> +   the target directory with the same mode as we specified already exists,
>> +   raises an :exc:`OSError` exception if *exist_ok* is False, otherwise no
>> +   exception is raised.  If the directory cannot be created in other cases,
>> +   raises an :exc:`OSError` exception.
>
> I would suggest being explicit here that "directory exists, but has a
> mode other than the one requested" always triggers an exception.
> Perhaps something like the following:
>
> "Raises an :exc:`OSError` exception if the target directory already
> exists, unless *exist_ok* is True and the existing directory has the
> same mode as is specified in the current call.  Also raises an
> :exc:`OSError` exception if the directory cannot be created for any
> other reason."

Georg has already patched that paragraph. I will let him decide if any 
further change is needed.

Terry

From pje at telecommunity.com  Thu Dec  2 18:16:33 2010
From: pje at telecommunity.com (P.J. Eby)
Date: Thu, 02 Dec 2010 12:16:33 -0500
Subject: [Python-Dev] ICU
In-Reply-To: <AANLkTin6_xNQ5xw6LHWWSbhA_a-YgjAnKDucPVrHOndR@mail.gmail.c
 om>
References: <AANLkTinVpySuRFkjf-Sc+7a91zrxhuQpqPQL5nM9u1w+@mail.gmail.com>
	<20101128214311.092abd35@pitrou.net> <4CF2D4E9.3060607@v.loewis.de>
	<4CF2F067.5020705@pearwood.info> <4CF354C6.9020302@v.loewis.de>
	<20101129193302.115dbcd5@pitrou.net>
	<AANLkTinodHhm3xfq75eM=_e7uCHcHqC7N_8VMAVbSeJ8@mail.gmail.com>
	<AANLkTi=uBgqpuKwPDB0bvRQv_86GQ4D3sZbrEhiWXUd3@mail.gmail.com>
	<4CF53724.8090000@voidspace.org.uk> <4CF54DA1.5080900@v.loewis.de>
	<1291144993.8628.1.camel@localhost.localdomain>
	<4CF55346.1040108@v.loewis.de>
	<1291146254.8628.4.camel@localhost.localdomain>
	<4CF556C8.9010704@v.loewis.de>
	<1291147987.8628.13.camel@localhost.localdomain>
	<AANLkTimrT-S-gSzC0bu4m223EnSB59chWY64xuj7YEbc@mail.gmail.com>
	<AANLkTin6_xNQ5xw6LHWWSbhA_a-YgjAnKDucPVrHOndR@mail.gmail.com>
Message-ID: <20101202171652.024943A4119@sparrow.telecommunity.com>

At 07:47 AM 12/2/2010 -0800, Guido van Rossum wrote:
>On Wed, Dec 1, 2010 at 8:45 PM, Alexander Belopolsky
><alexander.belopolsky at gmail.com> wrote:
> > On Tue, Nov 30, 2010 at 3:13 PM, Antoine Pitrou 
> <solipsis at pitrou.net> wrote:
> >>
> >> Oh, about ICU:
> >>
> >>> > Actually, I remember you saying that locale should ideally be replaced
> >>> > with a wrapper around the ICU library.
> >>>
> >>> By that, I stand - however, I have given up the hope that this will
> >>> happen anytime soon.
> >>
> >> Perhaps this could be made a GSOC topic.
> >>
> >
> > Incidentally, this may also address another Python's Achilles' heel:
> > the timezone support.
> >
> > http://icu-project.org/download/icutzu.html
>
>I work with people who speak highly of ICU, so I want to encourage
>work in this area.
>
>At the same time, I'm skeptical -- IIRC, ICU is a large amount of C++
>code. I don't know how easy it will be to integrate this into our
>build processes for various platforms, nor how "Pythonic" the
>resulting APIs will look to the experienced Python user.
>
>Still, those are not roadblocks, the benefits are potentially great,
>so it's definitely worth investigating!

FWIW, OSAF did a wrapping for Chandler, though I personally haven't used it:

    http://pyicu.osafoundation.org/

The README explains the mapping from the ICU APIs to Python ones, 
including iteration, string conversion, and timezone mapping for use 
with the datetime type.


>--
>--Guido van Rossum (python.org/~guido)
>_______________________________________________
>Python-Dev mailing list
>Python-Dev at python.org
>http://mail.python.org/mailman/listinfo/python-dev
>Unsubscribe: 
>http://mail.python.org/mailman/options/python-dev/pje%40telecommunity.com


From tjreedy at udel.edu  Thu Dec  2 18:59:00 2010
From: tjreedy at udel.edu (Terry Reedy)
Date: Thu, 02 Dec 2010 12:59:00 -0500
Subject: [Python-Dev] Porting Ideas
In-Reply-To: <AANLkTi=u-YAuEkNcBmkwUzCScoeedFSYno_RC2YRMvEy@mail.gmail.com>
References: <AANLkTing_iT7Y6FOZPWh=CUwtK8owvYtSdE7aXf0W181@mail.gmail.com>	<AANLkTi=6rvXDEfzmMC=vE5jaCaTzAwnDddxHrZ2GAB1b@mail.gmail.com>	<20101201201716.4357a316@pitrou.net>
	<AANLkTi=u-YAuEkNcBmkwUzCScoeedFSYno_RC2YRMvEy@mail.gmail.com>
Message-ID: <id8mp3$76u$1@dough.gmane.org>

On 12/2/2010 8:36 AM, Lennart Regebro wrote:
> On Wed, Dec 1, 2010 at 20:17, Antoine Pitrou<solipsis at pitrou.net>  wrote:
>> And I'm not sure what this package called "Python" is (?a high-level
>> object-oriented programming language?? like Java?), but I'm pretty sure
>> I've heard there's a Python 3 compatible version.
>
> Uhm... http://pypi.python.org/pypi/Python
>
> Anybody wanna remove that, or update it or something? :-)

Entry is for Python 2.5.

# Package Index Owner: guido, anthony, barry

-- 
Terry Jan Reedy



From alexander.belopolsky at gmail.com  Thu Dec  2 19:14:29 2010
From: alexander.belopolsky at gmail.com (Alexander Belopolsky)
Date: Thu, 2 Dec 2010 13:14:29 -0500
Subject: [Python-Dev] Python and the Unicode Character Database
In-Reply-To: <1291308960.3576.28.camel@localhost.localdomain>
References: <AANLkTinVpySuRFkjf-Sc+7a91zrxhuQpqPQL5nM9u1w+@mail.gmail.com>
	<AANLkTinQLK7+5NfEp-2nMiJeDEKRxKN9Zfs4ihv-NMyH@mail.gmail.com>
	<87wrnv43v5.fsf@uwakimon.sk.tsukuba.ac.jp>
	<id3umc$ps8$1@dough.gmane.org> <4CF5841E.6020401@v.loewis.de>
	<4CF619C1.8030403@pearwood.info> <4CF6CCF9.6000100@v.loewis.de>
	<4CF6E5AF.7080308@pearwood.info>
	<AANLkTikuUki5LgisVHrSMGZtmWZxxy9A8TOwny7dVCMe@mail.gmail.com>
	<id72oo$cdp$2@dough.gmane.org>
	<AANLkTikd0zDjr2GQ3_4C7BLZZstePQAiHee9m2ZpgCEh@mail.gmail.com>
	<20101202143633.6eb17b8d@pitrou.net>
	<AANLkTikiziBN87-7gM3rweODnXjEuK3DfhEni1sTDYtJ@mail.gmail.com>
	<1291308960.3576.28.camel@localhost.localdomain>
Message-ID: <AANLkTinBzCQ4gi42iNXkSqWAEkdPWZzs=+C5sr0f0vpB@mail.gmail.com>

On Thu, Dec 2, 2010 at 11:56 AM, Antoine Pitrou <solipsis at pitrou.net> wrote:
> Le jeudi 02 d?cembre 2010 ? 11:41 -0500, Alexander Belopolsky a ?crit :
>>
>> Note that my point is not to find the correct answer here, but to
>> demonstrate that we as a group don't have the expertise to get parsing
>> of Arabic text right.
>
> I don't understand why you think Arabic or Hebrew text is any different
> from Western text. Surely right-to-left isn't more conceptually
> complicated than left-to-right, is it?
>

No, but a mix of LTR and RTL is certainly more difficult that either
of the two.  I invite you to digest Unicode Standard Annex #9 before
we continue this discussion.

See <http://unicode.org/reports/tr9/>.


> The fact that mixed rtl + ltr can render bizarrely or is awkward to cut
> and paste is quite off-topic for our discussion.
>

No, it is not.  One of the invented use cases in this thread was naive
users' desire to enter numbers using their preferred local decimals.
Same users may want to be able to cut and paste their decimals as
well.  More importantly, however, legacy formats may not have support
for mixed-direction text and may require that "John is 41" be stored
as "41 si nhoJ" and Unicode converter would turn it into "[RTL]John is
14"  that will still display as  "41 si nhoJ", but int(s[-2:]) will
return 14, not 41.

>> If we've got it right for Arabic, it is by
>> chance and not by design. ?This still leaves us with 41 other types of
>> digits for at least 30 different languages.
>
> So why do you trust the Unicode standard on other things and not on this
> one?

What other things? As far as I understand the only str method that was
designed to comply with Unicode recomendations was str.isidentifier().
 And we have some really bizarre results:


>>> '\u2164'.isidentifier()
True
>>> '\u2164'.isalpha()
False

and can you describe the difference between str.isdigit() and
str.isdecimal()?  According to the reference manual,

"""
str.isdecimal()
Return true if all characters in the string are decimal characters and
there is at least one character, false otherwise. Decimal characters
include digit characters, and all characters that that can be used to
form decimal-radix numbers, e.g. U+0660, ARABIC-INDIC DIGIT ZERO.

str.isdigit()
Return true if all characters in the string are digits and there is at
least one character, false otherwise.
""" http://docs.python.org/dev/library/stdtypes.html#str.isdecimal

Since U+0660 is mentioned in the first definition and not in the
second, I may conclude that it is not a digit, but

>>> '\u0660'.isdigit()
True

If you know the correct answer, please contribute it here:
<http://bugs.python.org/issue10587>.

From solipsis at pitrou.net  Thu Dec  2 19:55:31 2010
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Thu, 02 Dec 2010 19:55:31 +0100
Subject: [Python-Dev] Python and the Unicode Character Database
In-Reply-To: <AANLkTinBzCQ4gi42iNXkSqWAEkdPWZzs=+C5sr0f0vpB@mail.gmail.com>
References: <AANLkTinVpySuRFkjf-Sc+7a91zrxhuQpqPQL5nM9u1w+@mail.gmail.com>
	<AANLkTinQLK7+5NfEp-2nMiJeDEKRxKN9Zfs4ihv-NMyH@mail.gmail.com>
	<87wrnv43v5.fsf@uwakimon.sk.tsukuba.ac.jp>
	<id3umc$ps8$1@dough.gmane.org>
	<4CF5841E.6020401@v.loewis.de> <4CF619C1.8030403@pearwood.info>
	<4CF6CCF9.6000100@v.loewis.de> <4CF6E5AF.7080308@pearwood.info>
	<AANLkTikuUki5LgisVHrSMGZtmWZxxy9A8TOwny7dVCMe@mail.gmail.com>
	<id72oo$cdp$2@dough.gmane.org>
	<AANLkTikd0zDjr2GQ3_4C7BLZZstePQAiHee9m2ZpgCEh@mail.gmail.com>
	<20101202143633.6eb17b8d@pitrou.net>
	<AANLkTikiziBN87-7gM3rweODnXjEuK3DfhEni1sTDYtJ@mail.gmail.com>
	<1291308960.3576.28.camel@localhost.localdomain>
	<AANLkTinBzCQ4gi42iNXkSqWAEkdPWZzs=+C5sr0f0vpB@mail.gmail.com>
Message-ID: <1291316131.3576.43.camel@localhost.localdomain>

Le jeudi 02 d?cembre 2010 ? 13:14 -0500, Alexander Belopolsky a ?crit :
> > I don't understand why you think Arabic or Hebrew text is any different
> > from Western text. Surely right-to-left isn't more conceptually
> > complicated than left-to-right, is it?
> >
> 
> No, but a mix of LTR and RTL is certainly more difficult that either
> of the two.  I invite you to digest Unicode Standard Annex #9 before
> we continue this discussion.
> 
> See <http://unicode.org/reports/tr9/>.

?This annex describes specifications for the *positioning* of characters
flowing from right to left? (emphasis mine)

Looks like something for implementors of rendering engines, which
python-dev is not AFAICT.

> Same users may want to be able to cut and paste their decimals as
> well.  More importantly, however, legacy formats may not have support
> for mixed-direction text and may require that "John is 41" be stored
> as "41 si nhoJ" and Unicode converter would turn it into "[RTL]John is
> 14"  that will still display as  "41 si nhoJ", but int(s[-2:]) will
> return 14, not 41.

The legacy format argument looks like a red herring to me. When
converting from a format to another it is the programmer's job to
his/her job right.

> >> If we've got it right for Arabic, it is by
> >> chance and not by design.  This still leaves us with 41 other types of
> >> digits for at least 30 different languages.
> >
> > So why do you trust the Unicode standard on other things and not on this
> > one?
> 
> What other things?

Everything which the Unicode database stores and that we already rely
on.

> As far as I understand the only str method that was
> designed to comply with Unicode recomendations was str.isidentifier().

I don't think so.  str.split() and str.splitlines() are also defined in
conformance to the SPEC, AFAIK.  They certainly try to.
And, outside of str itself, the re module tries to follow Unicode
categories as well (for example, "\d" should match non-ASCII digits).

Regards

Antoine.



From barry at python.org  Thu Dec  2 20:06:50 2010
From: barry at python.org (Barry Warsaw)
Date: Thu, 2 Dec 2010 14:06:50 -0500
Subject: [Python-Dev] Porting Ideas
In-Reply-To: <id8mp3$76u$1@dough.gmane.org>
References: <AANLkTing_iT7Y6FOZPWh=CUwtK8owvYtSdE7aXf0W181@mail.gmail.com>
	<AANLkTi=6rvXDEfzmMC=vE5jaCaTzAwnDddxHrZ2GAB1b@mail.gmail.com>
	<20101201201716.4357a316@pitrou.net>
	<AANLkTi=u-YAuEkNcBmkwUzCScoeedFSYno_RC2YRMvEy@mail.gmail.com>
	<id8mp3$76u$1@dough.gmane.org>
Message-ID: <20101202140650.11d96108@mission>

On Dec 02, 2010, at 12:59 PM, Terry Reedy wrote:

>On 12/2/2010 8:36 AM, Lennart Regebro wrote:
>> On Wed, Dec 1, 2010 at 20:17, Antoine Pitrou<solipsis at pitrou.net>  wrote:
>>> And I'm not sure what this package called "Python" is (?a high-level
>>> object-oriented programming language?? like Java?), but I'm pretty sure
>>> I've heard there's a Python 3 compatible version.
>>
>> Uhm... http://pypi.python.org/pypi/Python
>>
>> Anybody wanna remove that, or update it or something? :-)
>
>Entry is for Python 2.5.
>
># Package Index Owner: guido, anthony, barry

Well, I definitely can't remember ever seeing that before.  Of course, that
doesn't mean I haven't. ;)

-Barry

Aside: how does one log into the Cheeseshop with your Launchpad OpenID?  When
I try to do it I end up on a "Manual user registration" page.  I fill out the
username with what I think my PyPI user name is, and add my python.org email
address, but then it tells me 'barry' is already taken.  Do I need some kind
of back door linking of my lp openid and my pypi user id?
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 836 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/python-dev/attachments/20101202/db040c36/attachment.pgp>

From sridharr at activestate.com  Thu Dec  2 20:24:36 2010
From: sridharr at activestate.com (Sridhar Ratnakumar)
Date: Thu, 2 Dec 2010 11:24:36 -0800
Subject: [Python-Dev] Porting Ideas
In-Reply-To: <AANLkTi=6rvXDEfzmMC=vE5jaCaTzAwnDddxHrZ2GAB1b@mail.gmail.com>
References: <AANLkTing_iT7Y6FOZPWh=CUwtK8owvYtSdE7aXf0W181@mail.gmail.com>
	<AANLkTi=6rvXDEfzmMC=vE5jaCaTzAwnDddxHrZ2GAB1b@mail.gmail.com>
Message-ID: <F66CD4A4-D672-4D56-B8DD-34BCD102F9E3@activestate.com>


On 2010-12-01, at 11:02 AM, Brian Curtin wrote:

> http://onpython3yet.com/ might be helpful to you. It orders the projects on PyPI with the most dependencies which are not yet ported to 3.x.
> 
> Note that there are a number of false positives, e.g., the first result -- NumPy, since people don't seem to keep their classifiers up-to-date.

Also note that the dependency information is incomplete. For instance, onpython3yet.com shows just 14 packages depending on "Twisted",

  http://onpython3yet.com/packages/show/Twisted

while, in reality, there are 68 of them,

  http://code.activestate.com/pypm/twisted/#requiredby
  (see the right sidebar)

-srid

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20101202/926b2c67/attachment.html>

From martin at v.loewis.de  Thu Dec  2 20:37:29 2010
From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=)
Date: Thu, 02 Dec 2010 20:37:29 +0100
Subject: [Python-Dev] Python and the Unicode Character Database
In-Reply-To: <87bp552arq.fsf@benfinney.id.au>
References: <AANLkTinVpySuRFkjf-Sc+7a91zrxhuQpqPQL5nM9u1w+@mail.gmail.com>	<AANLkTinQLK7+5NfEp-2nMiJeDEKRxKN9Zfs4ihv-NMyH@mail.gmail.com>	<87wrnv43v5.fsf@uwakimon.sk.tsukuba.ac.jp>	<AANLkTinAf6CstywehOGRd4G4ms6K5LtSuq4Lj0z5wFBW@mail.gmail.com>	<87r5e236hj.fsf@benfinney.id.au>
	<4CF58387.1000609@v.loewis.de>	<87k4ju2vvg.fsf@benfinney.id.au>
	<4CF6CDDD.4060109@v.loewis.de>	<4CF6D7B2.6030506@pearwood.info>	<87sjyhxarg.fsf@uwakimon.sk.tsukuba.ac.jp>
	<87bp552arq.fsf@benfinney.id.au>
Message-ID: <4CF7F579.5050603@v.loewis.de>

Am 02.12.2010 03:01, schrieb Ben Finney:
> "Stephen J. Turnbull" <stephen at xemacs.org> writes:
> 
>> Furthermore, he provided good *objective* reason (excessive cost, to
>> which I can also testify, in several different input methods for
>> Japanese) why numbers simply would not be input that way.
>>
>> What's left is copy/paste via the mouse.
> 
> For direct entry by an interactive user, yes. Why are some people in
> this discussion thinking only of direct entry by an interactive user?

Ultimately, somebody will have entered the data.

> Input from an existing text file, as I said earlier.

Which *specific* existing text file? Have you actually *seen* such a
text file?

> Direct entry at the console is a red herring.

And we don't need powerhouses because power comes out of the socket.

Regards,
Martin

From martin at v.loewis.de  Thu Dec  2 20:40:52 2010
From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=)
Date: Thu, 02 Dec 2010 20:40:52 +0100
Subject: [Python-Dev] Python and the Unicode Character Database
In-Reply-To: <id857u$m91$1@dough.gmane.org>
References: <AANLkTinVpySuRFkjf-Sc+7a91zrxhuQpqPQL5nM9u1w+@mail.gmail.com>	<4CF2DC5B.4020702@egenix.com>	<AANLkTimB+FkPxHyC_udiCr-oVmSG4jnZ6USUHk14_Vh-@mail.gmail.com>	<4CF6CE95.1060707@v.loewis.de>
	<id857u$m91$1@dough.gmane.org>
Message-ID: <4CF7F644.8020207@v.loewis.de>

> Maybe all past, present and future whatsnew maintainers can agree on these
> rules, which I copied directly from whatsnew/3.2.rst?

I don't think all past maintainers can (I'm pretty certain that AMK
would disagree), but if that's the current policy, I can certainly try
following it (I didn't know it exists because I never look at the file).

Regards,
Martin

From martin at v.loewis.de  Thu Dec  2 20:44:42 2010
From: martin at v.loewis.de (=?ISO-8859-15?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Thu, 02 Dec 2010 20:44:42 +0100
Subject: [Python-Dev] Porting Ideas
In-Reply-To: <20101202140650.11d96108@mission>
References: <AANLkTing_iT7Y6FOZPWh=CUwtK8owvYtSdE7aXf0W181@mail.gmail.com>	<AANLkTi=6rvXDEfzmMC=vE5jaCaTzAwnDddxHrZ2GAB1b@mail.gmail.com>	<20101201201716.4357a316@pitrou.net>	<AANLkTi=u-YAuEkNcBmkwUzCScoeedFSYno_RC2YRMvEy@mail.gmail.com>	<id8mp3$76u$1@dough.gmane.org>
	<20101202140650.11d96108@mission>
Message-ID: <4CF7F72A.6090406@v.loewis.de>

> Aside: how does one log into the Cheeseshop with your Launchpad OpenID?  When
> I try to do it I end up on a "Manual user registration" page.  I fill out the
> username with what I think my PyPI user name is, and add my python.org email
> address, but then it tells me 'barry' is already taken.  Do I need some kind
> of back door linking of my lp openid and my pypi user id?

Since the "barry" account already exists, you first need to log into
that (likely using a password). You can then claim the LP OpenID as
being associated with that account, and use LP in the future.

Regards,
Martin

From barry at python.org  Thu Dec  2 20:55:51 2010
From: barry at python.org (Barry Warsaw)
Date: Thu, 2 Dec 2010 14:55:51 -0500
Subject: [Python-Dev] Porting Ideas
In-Reply-To: <4CF7F72A.6090406@v.loewis.de>
References: <AANLkTing_iT7Y6FOZPWh=CUwtK8owvYtSdE7aXf0W181@mail.gmail.com>
	<AANLkTi=6rvXDEfzmMC=vE5jaCaTzAwnDddxHrZ2GAB1b@mail.gmail.com>
	<20101201201716.4357a316@pitrou.net>
	<AANLkTi=u-YAuEkNcBmkwUzCScoeedFSYno_RC2YRMvEy@mail.gmail.com>
	<id8mp3$76u$1@dough.gmane.org> <20101202140650.11d96108@mission>
	<4CF7F72A.6090406@v.loewis.de>
Message-ID: <20101202145551.06643709@mission>

On Dec 02, 2010, at 08:44 PM, Martin v. L?wis wrote:

>Since the "barry" account already exists, you first need to log into
>that (likely using a password). You can then claim the LP OpenID as
>being associated with that account, and use LP in the future.

Thanks Martin.
-Barry
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 836 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/python-dev/attachments/20101202/2847bf6d/attachment.pgp>

From benjamin at python.org  Thu Dec  2 20:59:42 2010
From: benjamin at python.org (Benjamin Peterson)
Date: Thu, 2 Dec 2010 13:59:42 -0600
Subject: [Python-Dev] PEP 384 accepted
Message-ID: <AANLkTimQsiU+nyX6CdqQ8JBybJdnx=1EMxX9mefHbx9H@mail.gmail.com>

Hi,
Since discussion has trailed off without any blocking objections, I'm
accepting PEP 384. Martin, you may mark the PEP accepted and proceed
with merging the implementation for the beta on Saturday.

-- 
Regards,
Benjamin

From mal at egenix.com  Thu Dec  2 21:05:21 2010
From: mal at egenix.com (M.-A. Lemburg)
Date: Thu, 02 Dec 2010 21:05:21 +0100
Subject: [Python-Dev] Python and the Unicode Character Database
In-Reply-To: <4CF2DFD1.10901@v.loewis.de>
References: <AANLkTinVpySuRFkjf-Sc+7a91zrxhuQpqPQL5nM9u1w+@mail.gmail.com>	<20101128214311.092abd35@pitrou.net>	<4CF2D4E9.3060607@v.loewis.de>
	<4CF2DAD7.2000408@egenix.com> <4CF2DFD1.10901@v.loewis.de>
Message-ID: <4CF7FC01.80307@egenix.com>

"Martin v. L?wis" wrote:
>>> Now, one may wonder what precisely a "possibly signed floating point
>>> number" is, but most likely, this refers to
>>>
>>> floatnumber   ::=  pointfloat | exponentfloat
>>> pointfloat    ::=  [intpart] fraction | intpart "."
>>> exponentfloat ::=  (intpart | pointfloat) exponent
>>> intpart       ::=  digit+
>>> fraction      ::=  "." digit+
>>> exponent      ::=  ("e" | "E") ["+" | "-"] digit+
>>> digit          ::=  "0"..."9"
>>
>> I don't see why the language spec should limit the wealth of number
>> formats supported by float().
> 
> If it doesn't, there should be some other specification of what
> is correct and what is not. It must not be unspecified.

True.

>> It is not uncommon for Asians and other non-Latin script users to
>> use their own native script symbols for numbers. Just because these
>> digits may look strange to someone doesn't mean that they are
>> meaningless or should be discarded.
> 
> Then these users should speak up and indicate their need, or somebody
> should speak up and confirm that there are users who actually want
> '????.??' to denote 1234.56. To my knowledge, there is no writing
> system in which '????.??e4' means 12345600.0.

I'm not sure what you're after here.

>> Please also remember that Python3 now allows Unicode names for
>> identifiers for much the same reasons.
> 
> No no no. Addition of Unicode identifiers has a well-designed,
> deliberate specification, with a PEP and all. The support for
> non-ASCII digits in float appears to be ad-hoc, and not founded
> on actual needs of actual users.

Please note that we didn't have PEPs and the PEP process at the
time. The Unicode proposal predates and in some respects inspired
the PEP process.

The decision to add this support was deliberate based on the desire
to support as much of the nice features of Unicode in Python as
we could. At least that was what was driving me at the time.

Regarding actual needs of actual users: I don't buy that as an
argument when it comes to supporting a standard that is meant
to attract users with non-ASCII origins.

Some references you may want to read up on:

http://en.wikipedia.org/wiki/Numbers_in_Chinese_culture
http://en.wikipedia.org/wiki/Vietnamese_numerals
http://en.wikipedia.org/wiki/Korean_numerals
http://en.wikipedia.org/wiki/Japanese_numerals

Even MS Office supports them:

http://languages.siuc.edu/Chinese/Language_Settings.html

>> Note that the support in float() (and the other numeric constructors)
>> to work with Unicode code points was explicitly added when Unicode
>> support was added to Python and has been available since Python 1.6.
> 
> That doesn't necessarily make it useful. Alexander's complaint is that
> it makes Python unstable (i.e. changing as the UCD changes).

If that were true, then all Unicode database (UCD) changes would make
Python unstable. However, most changes to existing code points in the UCS
are bug fixes, so they actually have a stabilizing quality more than
a destabilizing one.

>> It is not a bug by any definition of "bug"
> 
> Most certainly it is: the documentation is either underspecified,
> or deviates from the implementation (when taking the most plausible
> interpretation). This is the very definition of "bug".

The implementation is not a bug and neither was this a bug in the
2.x series of the Python documentation. The Python 3.x docs apparently
introduced a reference to the language spec which is clearly not
capturing the wealth of possible inputs.

So, yes, we're talking about a documentation bug, but not an
implementation bug.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Nov 29 2010)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try our new mxODBC.Connect Python Database Interface for free ! ::::


   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
           Registered at Amtsgericht Duesseldorf: HRB 46611
               http://www.egenix.com/company/contact/

From g.brandl at gmx.net  Thu Dec  2 21:05:05 2010
From: g.brandl at gmx.net (Georg Brandl)
Date: Thu, 02 Dec 2010 21:05:05 +0100
Subject: [Python-Dev] Python and the Unicode Character Database
In-Reply-To: <4CF7F644.8020207@v.loewis.de>
References: <AANLkTinVpySuRFkjf-Sc+7a91zrxhuQpqPQL5nM9u1w+@mail.gmail.com>	<4CF2DC5B.4020702@egenix.com>	<AANLkTimB+FkPxHyC_udiCr-oVmSG4jnZ6USUHk14_Vh-@mail.gmail.com>	<4CF6CE95.1060707@v.loewis.de>	<id857u$m91$1@dough.gmane.org>
	<4CF7F644.8020207@v.loewis.de>
Message-ID: <id8u9d$6cq$1@dough.gmane.org>

Am 02.12.2010 20:40, schrieb "Martin v. L?wis":
>> Maybe all past, present and future whatsnew maintainers can agree on these
>> rules, which I copied directly from whatsnew/3.2.rst?
> 
> I don't think all past maintainers can

Yes, and the same goes for the future ones, since they may not even know yet
that they will be whatsnew maintainers.  Or maybe they aren't born yet (let's
hope for a long life of Python 3...).

> (I'm pretty certain that AMK
> would disagree), but if that's the current policy, I can certainly try
> following it (I didn't know it exists because I never look at the file).

The large chunk of rules appeared in 2.6, where AMK still was maintainer.
But even in the whatsnew for 2.4, there is this:

.. Don't write extensive text for new sections; I'll do that.
.. Feel free to add commented-out reminders of things that need
.. to be covered.  --amk

But in any case, they are certainly valid for the current whatsnew -- even
if Raymond likes to grumble about too expansive commits :)

Georg


From g.brandl at gmx.net  Thu Dec  2 21:09:02 2010
From: g.brandl at gmx.net (Georg Brandl)
Date: Thu, 02 Dec 2010 21:09:02 +0100
Subject: [Python-Dev] Porting Ideas
In-Reply-To: <20101202140650.11d96108@mission>
References: <AANLkTing_iT7Y6FOZPWh=CUwtK8owvYtSdE7aXf0W181@mail.gmail.com>	<AANLkTi=6rvXDEfzmMC=vE5jaCaTzAwnDddxHrZ2GAB1b@mail.gmail.com>	<20101201201716.4357a316@pitrou.net>	<AANLkTi=u-YAuEkNcBmkwUzCScoeedFSYno_RC2YRMvEy@mail.gmail.com>	<id8mp3$76u$1@dough.gmane.org>
	<20101202140650.11d96108@mission>
Message-ID: <id8ugq$6cq$2@dough.gmane.org>

Am 02.12.2010 20:06, schrieb Barry Warsaw:
> On Dec 02, 2010, at 12:59 PM, Terry Reedy wrote:
> 
>>On 12/2/2010 8:36 AM, Lennart Regebro wrote:
>>> On Wed, Dec 1, 2010 at 20:17, Antoine Pitrou<solipsis at pitrou.net>  wrote:
>>>> And I'm not sure what this package called "Python" is (?a high-level
>>>> object-oriented programming language?? like Java?), but I'm pretty sure
>>>> I've heard there's a Python 3 compatible version.
>>>
>>> Uhm... http://pypi.python.org/pypi/Python
>>>
>>> Anybody wanna remove that, or update it or something? :-)
>>
>>Entry is for Python 2.5.
>>
>># Package Index Owner: guido, anthony, barry
> 
> Well, I definitely can't remember ever seeing that before.  Of course, that
> doesn't mean I haven't. ;)

No idea what that entry is about.

    * Development Status :: 3 - Alpha
    * Development Status :: 6 - Mature

Aha.  Let's just delete it.

> Aside: how does one log into the Cheeseshop with your Launchpad OpenID?  When
> I try to do it I end up on a "Manual user registration" page.  I fill out the
> username with what I think my PyPI user name is, and add my python.org email
> address, but then it tells me 'barry' is already taken.  Do I need some kind
> of back door linking of my lp openid and my pypi user id?

In addition to what Martin said, the "Claim OpenID" form is on the "Your
Details" page.

Georg


From martin at v.loewis.de  Thu Dec  2 21:23:41 2010
From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=)
Date: Thu, 02 Dec 2010 21:23:41 +0100
Subject: [Python-Dev] Python and the Unicode Character Database
In-Reply-To: <4CF7FC01.80307@egenix.com>
References: <AANLkTinVpySuRFkjf-Sc+7a91zrxhuQpqPQL5nM9u1w+@mail.gmail.com>	<20101128214311.092abd35@pitrou.net>	<4CF2D4E9.3060607@v.loewis.de>
	<4CF2DAD7.2000408@egenix.com> <4CF2DFD1.10901@v.loewis.de>
	<4CF7FC01.80307@egenix.com>
Message-ID: <4CF8004D.1080408@v.loewis.de>

>> Then these users should speak up and indicate their need, or somebody
>> should speak up and confirm that there are users who actually want
>> '????.??' to denote 1234.56. To my knowledge, there is no writing
>> system in which '????.??e4' means 12345600.0.
> 
> I'm not sure what you're after here.

That the current float() constructor accepts tons of bogus character
strings and accepts them as numbers, and that it should stop doing so.

> The decision to add this support was deliberate based on the desire
> to support as much of the nice features of Unicode in Python as
> we could. At least that was what was driving me at the time.

At the time, this may have been the right thing to do. With the
experience gained, we should now conclude to revert this particular aspect.

> Some references you may want to read up on:
> 
> http://en.wikipedia.org/wiki/Numbers_in_Chinese_culture
> http://en.wikipedia.org/wiki/Vietnamese_numerals
> http://en.wikipedia.org/wiki/Korean_numerals
> http://en.wikipedia.org/wiki/Japanese_numerals

I don't question that people use non-ASCII characters to
denote numbers. I claim that the specific support in Python for that
has no connection to reality. I further claim that the use of non-ASCII
numbers is a local convention, and that if you provide a library to
parse numbers, users (of that library) will somehow have to specify
which notational convention(s) is reasonable for the input they have.

> Even MS Office supports them:
> 
> http://languages.siuc.edu/Chinese/Language_Settings.html

That's printing, though, not parsing.

Notice that Python does *not* currently support printing numbers in
other scripts - even though this may actually be more useful than
parsing.

>>> Note that the support in float() (and the other numeric constructors)
>>> to work with Unicode code points was explicitly added when Unicode
>>> support was added to Python and has been available since Python 1.6.
>>
>> That doesn't necessarily make it useful. Alexander's complaint is that
>> it makes Python unstable (i.e. changing as the UCD changes).
> 
> If that were true, then all Unicode database (UCD) changes would make
> Python unstable.

That's indeed the case - they do (see the recent bug report on white
space processing). However, any change makes Python unstable (in the
sense that it can potentially break existing applications), and, in
many cases, the risk of breaking something is well worth it.

In the case of number parsing, I think Python would be better if
float() rejected non-ASCII strings, and any support for such parsing
should be redone correctly in a different place (preferably along with
printing of numbers).

>> Most certainly it is: the documentation is either underspecified,
>> or deviates from the implementation (when taking the most plausible
>> interpretation). This is the very definition of "bug".
> 
> The implementation is not a bug and neither was this a bug in the
> 2.x series of the Python documentation.

Of course the 2.x documentation is wrong, in that it is severely
underspecified, and the most straight-forward interpretation of the
specific wording gives an incorrect impression of the implementation.

> The Python 3.x docs apparently
> introduced a reference to the language spec which is clearly not
> capturing the wealth of possible inputs.

Right - but only because the 2.x documentation *already* suggested that
the supported syntax matches the literal syntax - as that's the most
natural thing to assume.

Regards,
Martin

From martin at v.loewis.de  Thu Dec  2 21:24:43 2010
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Thu, 02 Dec 2010 21:24:43 +0100
Subject: [Python-Dev] PEP 384 accepted
In-Reply-To: <AANLkTimQsiU+nyX6CdqQ8JBybJdnx=1EMxX9mefHbx9H@mail.gmail.com>
References: <AANLkTimQsiU+nyX6CdqQ8JBybJdnx=1EMxX9mefHbx9H@mail.gmail.com>
Message-ID: <4CF8008B.60402@v.loewis.de>

> Since discussion has trailed off without any blocking objections, I'm
> accepting PEP 384. Martin, you may mark the PEP accepted and proceed
> with merging the implementation for the beta on Saturday.

Thanks! will do (I'll also take into consideration the proposed changes).

Regards,
Martin

From dirkjan at ochtman.nl  Thu Dec  2 21:29:52 2010
From: dirkjan at ochtman.nl (Dirkjan Ochtman)
Date: Thu, 2 Dec 2010 21:29:52 +0100
Subject: [Python-Dev] Porting Ideas
In-Reply-To: <F66CD4A4-D672-4D56-B8DD-34BCD102F9E3@activestate.com>
References: <AANLkTing_iT7Y6FOZPWh=CUwtK8owvYtSdE7aXf0W181@mail.gmail.com>
	<AANLkTi=6rvXDEfzmMC=vE5jaCaTzAwnDddxHrZ2GAB1b@mail.gmail.com>
	<F66CD4A4-D672-4D56-B8DD-34BCD102F9E3@activestate.com>
Message-ID: <AANLkTiknHZboLkyzL=3SNw9PCoZ9egHzZfuh0NR1b_mQ@mail.gmail.com>

On Thu, Dec 2, 2010 at 20:24, Sridhar Ratnakumar
<sridharr at activestate.com> wrote:
> Also note that the dependency information is incomplete.

Also, a python3 version of chardet is available (from the website
only, looks like).

Cheers,

Dirkjan

From ziade.tarek at gmail.com  Thu Dec  2 21:48:00 2010
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Thu, 2 Dec 2010 21:48:00 +0100
Subject: [Python-Dev] PEP 384 accepted
In-Reply-To: <4CF8008B.60402@v.loewis.de>
References: <AANLkTimQsiU+nyX6CdqQ8JBybJdnx=1EMxX9mefHbx9H@mail.gmail.com>
	<4CF8008B.60402@v.loewis.de>
Message-ID: <AANLkTikbRJEFYE_TvN++GYhmvSUu3hdhf4znOti7oty1@mail.gmail.com>

On Thu, Dec 2, 2010 at 9:24 PM, "Martin v. L?wis" <martin at v.loewis.de> wrote:
>> Since discussion has trailed off without any blocking objections, I'm
>> accepting PEP 384. Martin, you may mark the PEP accepted and proceed
>> with merging the implementation for the beta on Saturday.
>
> Thanks! will do (I'll also take into consideration the proposed changes).

I did not get an answer to my last mail about distutils / distutils2

>
> Regards,
> Martin
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/ziade.tarek%40gmail.com
>



-- 
Tarek Ziad? | http://ziade.org

From martin at v.loewis.de  Thu Dec  2 22:02:28 2010
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Thu, 02 Dec 2010 22:02:28 +0100
Subject: [Python-Dev] PEP 384 accepted
In-Reply-To: <AANLkTikbRJEFYE_TvN++GYhmvSUu3hdhf4znOti7oty1@mail.gmail.com>
References: <AANLkTimQsiU+nyX6CdqQ8JBybJdnx=1EMxX9mefHbx9H@mail.gmail.com>	<4CF8008B.60402@v.loewis.de>
	<AANLkTikbRJEFYE_TvN++GYhmvSUu3hdhf4znOti7oty1@mail.gmail.com>
Message-ID: <4CF80964.4070300@v.loewis.de>

Am 02.12.2010 21:48, schrieb Tarek Ziad?:
> On Thu, Dec 2, 2010 at 9:24 PM, "Martin v. L?wis" <martin at v.loewis.de> wrote:
>>> Since discussion has trailed off without any blocking objections, I'm
>>> accepting PEP 384. Martin, you may mark the PEP accepted and proceed
>>> with merging the implementation for the beta on Saturday.
>>
>> Thanks! will do (I'll also take into consideration the proposed changes).
> 
> I did not get an answer to my last mail about distutils / distutils2

What was the question again, and whom did you want an answer from?

Regards,
Martin

From ziade.tarek at gmail.com  Thu Dec  2 22:11:20 2010
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Thu, 2 Dec 2010 22:11:20 +0100
Subject: [Python-Dev] PEP 384 accepted
In-Reply-To: <4CF80964.4070300@v.loewis.de>
References: <AANLkTimQsiU+nyX6CdqQ8JBybJdnx=1EMxX9mefHbx9H@mail.gmail.com>
	<4CF8008B.60402@v.loewis.de>
	<AANLkTikbRJEFYE_TvN++GYhmvSUu3hdhf4znOti7oty1@mail.gmail.com>
	<4CF80964.4070300@v.loewis.de>
Message-ID: <AANLkTikcT0WGo3FhFVq=N6ktToNenBDNNcC4eSpefcFF@mail.gmail.com>

2010/12/2 "Martin v. L?wis" <martin at v.loewis.de>:
> Am 02.12.2010 21:48, schrieb Tarek Ziad?:
>> On Thu, Dec 2, 2010 at 9:24 PM, "Martin v. L?wis" <martin at v.loewis.de> wrote:
>>>> Since discussion has trailed off without any blocking objections, I'm
>>>> accepting PEP 384. Martin, you may mark the PEP accepted and proceed
>>>> with merging the implementation for the beta on Saturday.
>>>
>>> Thanks! will do (I'll also take into consideration the proposed changes).
>>
>> I did not get an answer to my last mail about distutils / distutils2
>
> What was the question again, and whom did you want an answer from?

You can read it in the archives here:
http://mail.python.org/pipermail/python-dev/2010-November/106138.html

tldr:

The question was "Why not implementing this in Distutils2 ?"
Your answer was "No, PEP 3149 was accepted, I will do this in Distutils1"
My answer was "Having an accepted PEP does not imply your code lands
in the sdtlib (like PEP 376 and 345)"

So the question still stands: "Why not implementing this in Distutils2 ?"

Regards
Tarek

>
> Regards,
> Martin
>



-- 
Tarek Ziad? | http://ziade.org

From mal at egenix.com  Thu Dec  2 22:14:34 2010
From: mal at egenix.com (M.-A. Lemburg)
Date: Thu, 02 Dec 2010 22:14:34 +0100
Subject: [Python-Dev] Python and the Unicode Character Database
In-Reply-To: <4CF7F579.5050603@v.loewis.de>
References: <AANLkTinVpySuRFkjf-Sc+7a91zrxhuQpqPQL5nM9u1w+@mail.gmail.com>	<AANLkTinQLK7+5NfEp-2nMiJeDEKRxKN9Zfs4ihv-NMyH@mail.gmail.com>	<87wrnv43v5.fsf@uwakimon.sk.tsukuba.ac.jp>	<AANLkTinAf6CstywehOGRd4G4ms6K5LtSuq4Lj0z5wFBW@mail.gmail.com>	<87r5e236hj.fsf@benfinney.id.au>	<4CF58387.1000609@v.loewis.de>	<87k4ju2vvg.fsf@benfinney.id.au>	<4CF6CDDD.4060109@v.loewis.de>	<4CF6D7B2.6030506@pearwood.info>	<87sjyhxarg.fsf@uwakimon.sk.tsukuba.ac.jp>	<87bp552arq.fsf@benfinney.id.au>
	<4CF7F579.5050603@v.loewis.de>
Message-ID: <4CF80C3A.9000705@egenix.com>

"Martin v. L?wis" wrote:
>> [...]
>> For direct entry by an interactive user, yes. Why are some people in
>> this discussion thinking only of direct entry by an interactive user?
> 
> Ultimately, somebody will have entered the data.

I don't think you really believe that all data processed by a
computer was eventually manually entered by a someone :-)

I already gave you a couple of examples of how such data can
end up being input for Python number constructors. If you are
still curious, please see the Wikipedia pages I linked to,
or have a look at these keyboards:

http://en.wikipedia.org/wiki/File:KB_Arabic_MAC.svg
http://en.wikipedia.org/wiki/File:Keyboard_Layout_Sanskrit.png
http://en.wikipedia.org/wiki/File:800px-KB_Thai_Kedmanee.png
http://en.wikipedia.org/wiki/File:Tibetan_Keyboard.png
http://en.wikipedia.org/wiki/File:KBD-DZ-noshift-2009.png

(all referenced on http://en.wikipedia.org/wiki/Keyboard_layout)

and then compare these to:

http://www.unicode.org/Public/5.2.0/ucd/extracted/DerivedNumericType.txt

Arabic numerals are being used a lot nowadays in Asian countries,
but that doesn't mean that the native script versions are not
being used anymore.

Furthermore, data can well originate from texts that were written
hundreds or even thousands of years ago, so there is plenty of
material available for processing.

Even if not entered directly, there are plenty of ways to convert
Arabic numerals (or other numeral systems) to the above forms,
e.g. in MS Office for Thai:

http://office.microsoft.com/en-us/excel-help/convert-arabic-numbers-to-thai-text-format-HP003074364.aspx

Anyway, as mentioned before: all this is really besides the point:

If we want to support Unicode in Python, we have to also support
conversion of numerals declared in Unicode into a form that can
be processed by Python. Regardless of where such data originates.

If we were not to follow this approach, we could just as well
decide not support support reading Egyptian Hieroglyphs based
on the argument that there's no keyboard to enter them...

http://www.unicode.org/charts/PDF/U13000.pdf  :-)

(from http://www.unicode.org/charts/)

>> Input from an existing text file, as I said earlier.
> 
> Which *specific* existing text file? Have you actually *seen* such a
> text file?

Have you tried Google ?

http://www.google.com/search?q=???
http://www.google.com/search?q=?+site%3Agov.lb

Some examples:

http://www.bdl.gov.lb/circ/intpdf/int123.pdf
http://www.cdr.gov.lb/study/sdatl/Arabic/Chapter3.PDF
http://www.batroun.gov.lb/PDF/Waredat2006.pdf

(these all use http://en.wikipedia.org/wiki/Eastern_Arabic_numerals)

>> Direct entry at the console is a red herring.
> 
> And we don't need powerhouses because power comes out of the socket.

Martin, the argument simply doesn't fit well with the discussion
about Python and Unicode.

We introduced Unicode in Python not because there was a need
for each and every code point in Unicode, but because we wanted
to adopt a standard which doesn't prefer any one way of writing
things over another.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Dec 02 2010)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try our new mxODBC.Connect Python Database Interface for free ! ::::


   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
           Registered at Amtsgericht Duesseldorf: HRB 46611
               http://www.egenix.com/company/contact/

From martin at v.loewis.de  Thu Dec  2 22:19:40 2010
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Thu, 02 Dec 2010 22:19:40 +0100
Subject: [Python-Dev] PEP 384 accepted
In-Reply-To: <AANLkTikcT0WGo3FhFVq=N6ktToNenBDNNcC4eSpefcFF@mail.gmail.com>
References: <AANLkTimQsiU+nyX6CdqQ8JBybJdnx=1EMxX9mefHbx9H@mail.gmail.com>	<4CF8008B.60402@v.loewis.de>	<AANLkTikbRJEFYE_TvN++GYhmvSUu3hdhf4znOti7oty1@mail.gmail.com>	<4CF80964.4070300@v.loewis.de>
	<AANLkTikcT0WGo3FhFVq=N6ktToNenBDNNcC4eSpefcFF@mail.gmail.com>
Message-ID: <4CF80D6C.6050703@v.loewis.de>


> So the question still stands: "Why not implementing this in Distutils2 ?"

Because it then wouldn't be available in Python 3.2, which is the target
release of the PEP.

If that really causes too much pain, I'll refrain from making any
changes to distutils; PEP 384 doesn't specify any changes, anyway.

Regards,
Martin

From martin at v.loewis.de  Thu Dec  2 22:27:39 2010
From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=)
Date: Thu, 02 Dec 2010 22:27:39 +0100
Subject: [Python-Dev] Python and the Unicode Character Database
In-Reply-To: <4CF80C3A.9000705@egenix.com>
References: <AANLkTinVpySuRFkjf-Sc+7a91zrxhuQpqPQL5nM9u1w+@mail.gmail.com>	<AANLkTinQLK7+5NfEp-2nMiJeDEKRxKN9Zfs4ihv-NMyH@mail.gmail.com>	<87wrnv43v5.fsf@uwakimon.sk.tsukuba.ac.jp>	<AANLkTinAf6CstywehOGRd4G4ms6K5LtSuq4Lj0z5wFBW@mail.gmail.com>	<87r5e236hj.fsf@benfinney.id.au>	<4CF58387.1000609@v.loewis.de>	<87k4ju2vvg.fsf@benfinney.id.au>	<4CF6CDDD.4060109@v.loewis.de>	<4CF6D7B2.6030506@pearwood.info>	<87sjyhxarg.fsf@uwakimon.sk.tsukuba.ac.jp>	<87bp552arq.fsf@benfinney.id.au>
	<4CF7F579.5050603@v.loewis.de> <4CF80C3A.9000705@egenix.com>
Message-ID: <4CF80F4B.1020905@v.loewis.de>

> Arabic numerals are being used a lot nowadays in Asian countries,
> but that doesn't mean that the native script versions are not
> being used anymore.

I never claimed that people are not using their local scripts to enter
numbers. However, none of your examples is about Chinese numerals using
an ASCII full stop as a decimal point. The only thing I claimed about
usage (actually only repeating haiyang kang's earlier claim) is that
nobody would enter Chinese numerals with a keyboard and then use full
stop as the decimal separator.

So all your counter-examples just don't apply - I don't deny them.

Regards,
Martin

From steve at pearwood.info  Thu Dec  2 22:30:38 2010
From: steve at pearwood.info (Steven D'Aprano)
Date: Fri, 03 Dec 2010 08:30:38 +1100
Subject: [Python-Dev] Python and the Unicode Character Database
In-Reply-To: <4CF8004D.1080408@v.loewis.de>
References: <AANLkTinVpySuRFkjf-Sc+7a91zrxhuQpqPQL5nM9u1w+@mail.gmail.com>	<20101128214311.092abd35@pitrou.net>	<4CF2D4E9.3060607@v.loewis.de>	<4CF2DAD7.2000408@egenix.com>
	<4CF2DFD1.10901@v.loewis.de>	<4CF7FC01.80307@egenix.com>
	<4CF8004D.1080408@v.loewis.de>
Message-ID: <4CF80FFE.8020505@pearwood.info>

Martin v. L?wis wrote:
>>> Then these users should speak up and indicate their need, or somebody
>>> should speak up and confirm that there are users who actually want
>>> '????.??' to denote 1234.56. To my knowledge, there is no writing
>>> system in which '????.??e4' means 12345600.0.
>> I'm not sure what you're after here.
> 
> That the current float() constructor accepts tons of bogus character
> strings and accepts them as numbers, and that it should stop doing so.

What bogus characters do the float() and int() constructors accept? As 
far as I can see, they only accepts numerals.


[...]
> Notice that Python does *not* currently support printing numbers in
> other scripts - even though this may actually be more useful than
> parsing.

Lack of one function, even if more useful, does not imply that an 
existing function should be removed.

[...]
> In the case of number parsing, I think Python would be better if
> float() rejected non-ASCII strings, and any support for such parsing
> should be redone correctly in a different place (preferably along with
> printing of numbers).

So your problems with the current behaviour are:

(1) in some unspecified way, it's not done correctly;

(2) it belongs somewhere other than float() and int().

That second is awfully close to bike-shedding. Since you accept that 
Python *should* have the current behaviour, and Python *already* has the 
current behaviour, it seems strange that you are kicking up such a fuss 
merely to *move* the implementation of that behaviour out of the numeric 
constructors into some unspecified "different place".

I think it would be constructive to explain:

- how the current behaviour is incorrect;
- your suggestions for correcting it; and
- a concrete suggestion for where you would like to see the behaviour 
moved to, and why that would be better than where it currently is.



-- 
Steven


From ziade.tarek at gmail.com  Thu Dec  2 22:33:24 2010
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Thu, 2 Dec 2010 22:33:24 +0100
Subject: [Python-Dev] PEP 384 accepted
In-Reply-To: <4CF80D6C.6050703@v.loewis.de>
References: <AANLkTimQsiU+nyX6CdqQ8JBybJdnx=1EMxX9mefHbx9H@mail.gmail.com>
	<4CF8008B.60402@v.loewis.de>
	<AANLkTikbRJEFYE_TvN++GYhmvSUu3hdhf4znOti7oty1@mail.gmail.com>
	<4CF80964.4070300@v.loewis.de>
	<AANLkTikcT0WGo3FhFVq=N6ktToNenBDNNcC4eSpefcFF@mail.gmail.com>
	<4CF80D6C.6050703@v.loewis.de>
Message-ID: <AANLkTi=s5NMxjPdRQRcupNBzdqjMhLO6TbFWtQzi0TEw@mail.gmail.com>

2010/12/2 "Martin v. L?wis" <martin at v.loewis.de>:
>
>> So the question still stands: "Why not implementing this in Distutils2 ?"
>
> Because it then wouldn't be available in Python 3.2, which is the target
> release of the PEP.

The exact feature I am mentioning is the ability to compile extensions
with new options, so I am not sure which PEP is involved since
distutils changes refer to PEP 384 in the other PEP.

I was told not to touch to Distutils code to avoid any regression
since it's patched to the bones in third party products. So we decided
to freeze distutils and add all new features in Distutils2, which is
at alpha stage now.  So this move seems contradictory to me.

Grouping all new features in the new version and keep Distutils1 in
maintenance mode seems to make more sense to me, if we want to make
Distutils die and push forward Distutils2 for its new features etc. Or
we might get back into backward hell again :)

So, I am +1 on a patch on distutils2 and -1 on de-freezing Distutils
for any new feature.

> If that really causes too much pain, I'll refrain from making any
> changes to distutils; PEP 384 doesn't specify any changes, anyway.

That would be awesome, and we can work on a patch for distutils2 to
provide that abi option.


>
> Regards,
> Martin
>



-- 
Tarek Ziad? | http://ziade.org

From alexander.belopolsky at gmail.com  Thu Dec  2 22:34:34 2010
From: alexander.belopolsky at gmail.com (Alexander Belopolsky)
Date: Thu, 2 Dec 2010 16:34:34 -0500
Subject: [Python-Dev] Python and the Unicode Character Database
In-Reply-To: <1291316131.3576.43.camel@localhost.localdomain>
References: <AANLkTinVpySuRFkjf-Sc+7a91zrxhuQpqPQL5nM9u1w+@mail.gmail.com>
	<AANLkTinQLK7+5NfEp-2nMiJeDEKRxKN9Zfs4ihv-NMyH@mail.gmail.com>
	<87wrnv43v5.fsf@uwakimon.sk.tsukuba.ac.jp>
	<id3umc$ps8$1@dough.gmane.org> <4CF5841E.6020401@v.loewis.de>
	<4CF619C1.8030403@pearwood.info> <4CF6CCF9.6000100@v.loewis.de>
	<4CF6E5AF.7080308@pearwood.info>
	<AANLkTikuUki5LgisVHrSMGZtmWZxxy9A8TOwny7dVCMe@mail.gmail.com>
	<id72oo$cdp$2@dough.gmane.org>
	<AANLkTikd0zDjr2GQ3_4C7BLZZstePQAiHee9m2ZpgCEh@mail.gmail.com>
	<20101202143633.6eb17b8d@pitrou.net>
	<AANLkTikiziBN87-7gM3rweODnXjEuK3DfhEni1sTDYtJ@mail.gmail.com>
	<1291308960.3576.28.camel@localhost.localdomain>
	<AANLkTinBzCQ4gi42iNXkSqWAEkdPWZzs=+C5sr0f0vpB@mail.gmail.com>
	<1291316131.3576.43.camel@localhost.localdomain>
Message-ID: <AANLkTi=W7_WpX7CS2znW-tYsGyv2qcZYq7VFRmrQwAko@mail.gmail.com>

On Thu, Dec 2, 2010 at 1:55 PM, Antoine Pitrou <solipsis at pitrou.net> wrote:
..
> I don't think so. ?str.split() and str.splitlines() are also defined in
> conformance to the SPEC, AFAIK. ?They certainly try to.

You are joking, right?  Where exactly does Unicode specify something like this:

>>> ''.join('???'.split('\udf00\ud800'))
'??'
?

OK, splitting on a given separator has very little to do with Unicode
or UCD, but str.splitlines()  makes absolutely no attempt to conform
to Unicode Standard Annex #14 ("Unicode line breaking algorithm").
Wait, UAX #14 is actually relevant to textwrap module which saw very
little change since 2.x days.  So, what exactly does str.splitlines()
do?   And which part of the Unicode standard defines how it is
different from str.split(.., '\n')?  Reference manual does not help me
here either:

"""
str.splitlines([keepends])

Return a list of the lines in the string, breaking at line boundaries.
Line breaks are not included in the resulting list unless keepends is
given and true.
""" http://docs.python.org/dev/library/stdtypes.html#str.splitlines

From solipsis at pitrou.net  Thu Dec  2 22:36:58 2010
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Thu, 02 Dec 2010 22:36:58 +0100
Subject: [Python-Dev] Python and the Unicode Character Database
In-Reply-To: <AANLkTi=W7_WpX7CS2znW-tYsGyv2qcZYq7VFRmrQwAko@mail.gmail.com>
References: <AANLkTinVpySuRFkjf-Sc+7a91zrxhuQpqPQL5nM9u1w+@mail.gmail.com>
	<AANLkTinQLK7+5NfEp-2nMiJeDEKRxKN9Zfs4ihv-NMyH@mail.gmail.com>
	<87wrnv43v5.fsf@uwakimon.sk.tsukuba.ac.jp>
	<id3umc$ps8$1@dough.gmane.org>
	<4CF5841E.6020401@v.loewis.de> <4CF619C1.8030403@pearwood.info>
	<4CF6CCF9.6000100@v.loewis.de> <4CF6E5AF.7080308@pearwood.info>
	<AANLkTikuUki5LgisVHrSMGZtmWZxxy9A8TOwny7dVCMe@mail.gmail.com>
	<id72oo$cdp$2@dough.gmane.org>
	<AANLkTikd0zDjr2GQ3_4C7BLZZstePQAiHee9m2ZpgCEh@mail.gmail.com>
	<20101202143633.6eb17b8d@pitrou.net>
	<AANLkTikiziBN87-7gM3rweODnXjEuK3DfhEni1sTDYtJ@mail.gmail.com>
	<1291308960.3576.28.camel@localhost.localdomain>
	<AANLkTinBzCQ4gi42iNXkSqWAEkdPWZzs=+C5sr0f0vpB@mail.gmail.com>
	<1291316131.3576.43.camel@localhost.localdomain>
	<AANLkTi=W7_WpX7CS2znW-tYsGyv2qcZYq7VFRmrQwAko@mail.gmail.com>
Message-ID: <1291325818.3576.46.camel@localhost.localdomain>

Le jeudi 02 d?cembre 2010 ? 16:34 -0500, Alexander Belopolsky a ?crit :
> On Thu, Dec 2, 2010 at 1:55 PM, Antoine Pitrou <solipsis at pitrou.net> wrote:
> ..
> > I don't think so.  str.split() and str.splitlines() are also defined in
> > conformance to the SPEC, AFAIK.  They certainly try to.
> 
> You are joking, right?

Perhaps you could look at the implementation.




From martin at v.loewis.de  Thu Dec  2 22:39:20 2010
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Thu, 02 Dec 2010 22:39:20 +0100
Subject: [Python-Dev] PEP 384 accepted
In-Reply-To: <AANLkTi=s5NMxjPdRQRcupNBzdqjMhLO6TbFWtQzi0TEw@mail.gmail.com>
References: <AANLkTimQsiU+nyX6CdqQ8JBybJdnx=1EMxX9mefHbx9H@mail.gmail.com>	<4CF8008B.60402@v.loewis.de>	<AANLkTikbRJEFYE_TvN++GYhmvSUu3hdhf4znOti7oty1@mail.gmail.com>	<4CF80964.4070300@v.loewis.de>	<AANLkTikcT0WGo3FhFVq=N6ktToNenBDNNcC4eSpefcFF@mail.gmail.com>	<4CF80D6C.6050703@v.loewis.de>
	<AANLkTi=s5NMxjPdRQRcupNBzdqjMhLO6TbFWtQzi0TEw@mail.gmail.com>
Message-ID: <4CF81208.8090705@v.loewis.de>

> I was told not to touch to Distutils code to avoid any regression
> since it's patched to the bones in third party products. So we decided
> to freeze distutils and add all new features in Distutils2, which is
> at alpha stage now.  So this move seems contradictory to me.

I think it was a bad decision to freeze distutils, and "we" certainly
didn't make that (not any we that includes me, that is). This freeze
made the situation worse.

IIRC, it was really the incompatible changes that made people ask you to
stop changing distutils.

Regards,
Martin

From martin at v.loewis.de  Thu Dec  2 22:48:33 2010
From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=)
Date: Thu, 02 Dec 2010 22:48:33 +0100
Subject: [Python-Dev] Python and the Unicode Character Database
In-Reply-To: <4CF80FFE.8020505@pearwood.info>
References: <AANLkTinVpySuRFkjf-Sc+7a91zrxhuQpqPQL5nM9u1w+@mail.gmail.com>	<20101128214311.092abd35@pitrou.net>	<4CF2D4E9.3060607@v.loewis.de>	<4CF2DAD7.2000408@egenix.com>	<4CF2DFD1.10901@v.loewis.de>	<4CF7FC01.80307@egenix.com>	<4CF8004D.1080408@v.loewis.de>
	<4CF80FFE.8020505@pearwood.info>
Message-ID: <4CF81431.2040900@v.loewis.de>

Am 02.12.2010 22:30, schrieb Steven D'Aprano:
> Martin v. L?wis wrote:
>>>> Then these users should speak up and indicate their need, or somebody
>>>> should speak up and confirm that there are users who actually want
>>>> '????.??' to denote 1234.56. To my knowledge, there is no writing
>>>> system in which '????.??e4' means 12345600.0.
>>> I'm not sure what you're after here.
>>
>> That the current float() constructor accepts tons of bogus character
>> strings and accepts them as numbers, and that it should stop doing so.
> 
> What bogus characters do the float() and int() constructors accept? As
> far as I can see, they only accepts numerals.

Not bogus characters, but bogus character strings. E.g. strings that mix
digits from different scripts, and mix them with the Python decimal
separator.

>> Notice that Python does *not* currently support printing numbers in
>> other scripts - even though this may actually be more useful than
>> parsing.
> 
> Lack of one function, even if more useful, does not imply that an
> existing function should be removed.

No. But if the specific function(ality) is not useful and
underspecified, it should be removed.

> So your problems with the current behaviour are:
> 
> (1) in some unspecified way, it's not done correctly;

No. My main concern is that it is not properly specified. If it was
specified, I could then tell you what precisely is wrong about it.
Right now, I can only give examples for input that it should not accept,
and examples of input that it should, but does not accept.

> (2) it belongs somewhere other than float() and int().

That's only because it also needs a parameter to specify what syntax to
follow, somehow. That parameter could be explicit or implicit, and it
could be to float or to some other function. But it must be available,
and is not.

> That second is awfully close to bike-shedding. Since you accept that
> Python *should* have the current behaviour

No, I don't. I think it behaves incorrectly, accepting garbage input and
guessing some meaning out of it.

> - how the current behaviour is incorrect;

See above: it accepts strings that do not denote real numbers in any
writing system, and, despite the claim that the feature is there to
support other writing systems, actually does not truly support other
writing systems.

> - your suggestions for correcting it; and

Make the current implementation exactly match the current documentation.
I think the documentation is correct; the implementation is wrong.

> - a concrete suggestion for where you would like to see the behaviour
> moved to, and why that would be better than where it currently is.

The current behavior should go nowhere; it is not useful. Something very
similar to the current behavior (but done correctly) should go into the
locale module.

Regards,
Martin

From alexander.belopolsky at gmail.com  Thu Dec  2 22:50:47 2010
From: alexander.belopolsky at gmail.com (Alexander Belopolsky)
Date: Thu, 2 Dec 2010 16:50:47 -0500
Subject: [Python-Dev] Python and the Unicode Character Database
In-Reply-To: <4CF80C3A.9000705@egenix.com>
References: <AANLkTinVpySuRFkjf-Sc+7a91zrxhuQpqPQL5nM9u1w+@mail.gmail.com>
	<AANLkTinQLK7+5NfEp-2nMiJeDEKRxKN9Zfs4ihv-NMyH@mail.gmail.com>
	<87wrnv43v5.fsf@uwakimon.sk.tsukuba.ac.jp>
	<AANLkTinAf6CstywehOGRd4G4ms6K5LtSuq4Lj0z5wFBW@mail.gmail.com>
	<87r5e236hj.fsf@benfinney.id.au> <4CF58387.1000609@v.loewis.de>
	<87k4ju2vvg.fsf@benfinney.id.au> <4CF6CDDD.4060109@v.loewis.de>
	<4CF6D7B2.6030506@pearwood.info>
	<87sjyhxarg.fsf@uwakimon.sk.tsukuba.ac.jp>
	<87bp552arq.fsf@benfinney.id.au> <4CF7F579.5050603@v.loewis.de>
	<4CF80C3A.9000705@egenix.com>
Message-ID: <AANLkTi=QApgGxCH0ph-_QjMMu5scH97vSdsBw_Da5Fvv@mail.gmail.com>

On Thu, Dec 2, 2010 at 4:14 PM, M.-A. Lemburg <mal at egenix.com> wrote:
..
> Have you tried Google ?
>

I tried google at I could not find any plain text or HTML file that
would use Arabic-Indic numerals.  What was interesting, though that a
search for "quran unicode" (without quotes).  Brought me to
http://www.sacred-texts.com which says that they've been using unicode
since 2002 in their archives.  Interestingly enough, their version of
Qur'an uses ordinary digits for ayah numbers.  See, for example
<http://www.sacred-texts.com/isl/uq/050.htm>.

I will change my mind on this issue when you present a
machine-readable file with Arabic-Indic numerals and a program capable
of reading it and show that this program uses the same number parsing
algorithm as Python's int() or float().

From ziade.tarek at gmail.com  Thu Dec  2 22:54:50 2010
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Thu, 2 Dec 2010 22:54:50 +0100
Subject: [Python-Dev] PEP 384 accepted
In-Reply-To: <4CF81208.8090705@v.loewis.de>
References: <AANLkTimQsiU+nyX6CdqQ8JBybJdnx=1EMxX9mefHbx9H@mail.gmail.com>
	<4CF8008B.60402@v.loewis.de>
	<AANLkTikbRJEFYE_TvN++GYhmvSUu3hdhf4znOti7oty1@mail.gmail.com>
	<4CF80964.4070300@v.loewis.de>
	<AANLkTikcT0WGo3FhFVq=N6ktToNenBDNNcC4eSpefcFF@mail.gmail.com>
	<4CF80D6C.6050703@v.loewis.de>
	<AANLkTi=s5NMxjPdRQRcupNBzdqjMhLO6TbFWtQzi0TEw@mail.gmail.com>
	<4CF81208.8090705@v.loewis.de>
Message-ID: <AANLkTinspUo88oUnd3cX776mMRMU6bqyOaq9Og7JLwhU@mail.gmail.com>

2010/12/2 "Martin v. L?wis" <martin at v.loewis.de>:
>> I was told not to touch to Distutils code to avoid any regression
>> since it's patched to the bones in third party products. So we decided
>> to freeze distutils and add all new features in Distutils2, which is
>> at alpha stage now. ?So this move seems contradictory to me.
>
> I think it was a bad decision to freeze distutils, and "we" certainly
> didn't make that (not any we that includes me, that is).

"We" is the people at the last language summit. Sorry if I used such a
vague word.

> This freeze made the situation worse.

Can you extend on this and explains why it makes it worse ?

If we (as you included) don't agree it's the best solution, I would
not want to be pushed back to square one at the next summit..

I happily reverted all my changes last year when asked, and started to
work on Distutils2. But I'll get out of steam if the direction changes
again, with you stating that it makes the situation worse.

>
> IIRC, it was really the incompatible changes that made people ask you to
> stop changing distutils.

Who is "people" ? Are you suggesting that we could have added all the
new features in Distutils in the stdlib ?

The decision was because we had a mix of:

- incompatible changes in private parts  -- and some packages where
patching distutils internals
- changes on public APIs behavior, whith a behavior that was not
clearly documented and suggest to interpretation
- some mistakes I made as well

But that's what you would expect for a project that needs to evolve a
lot. Thus the freezing.

So how would you make the situation better, if not by doing the work
in distutils2 ?

> Regards,
> Martin
>



-- 
Tarek Ziad? | http://ziade.org

From fuzzyman at voidspace.org.uk  Thu Dec  2 22:54:54 2010
From: fuzzyman at voidspace.org.uk (Michael Foord)
Date: Thu, 02 Dec 2010 21:54:54 +0000
Subject: [Python-Dev] PEP 384 accepted
In-Reply-To: <4CF81208.8090705@v.loewis.de>
References: <AANLkTimQsiU+nyX6CdqQ8JBybJdnx=1EMxX9mefHbx9H@mail.gmail.com>	<4CF8008B.60402@v.loewis.de>	<AANLkTikbRJEFYE_TvN++GYhmvSUu3hdhf4znOti7oty1@mail.gmail.com>	<4CF80964.4070300@v.loewis.de>	<AANLkTikcT0WGo3FhFVq=N6ktToNenBDNNcC4eSpefcFF@mail.gmail.com>	<4CF80D6C.6050703@v.loewis.de>	<AANLkTi=s5NMxjPdRQRcupNBzdqjMhLO6TbFWtQzi0TEw@mail.gmail.com>
	<4CF81208.8090705@v.loewis.de>
Message-ID: <4CF815AE.1040404@voidspace.org.uk>

On 02/12/2010 21:39, "Martin v. L?wis" wrote:
>> I was told not to touch to Distutils code to avoid any regression
>> since it's patched to the bones in third party products. So we decided
>> to freeze distutils and add all new features in Distutils2, which is
>> at alpha stage now.  So this move seems contradictory to me.
> I think it was a bad decision to freeze distutils, and "we" certainly
> didn't make that (not any we that includes me, that is). This freeze
> made the situation worse.

What situation worse?

"We" certainly did ask Tarek to become bdfl of distutils and fix/improve 
it (at a language summit 2 years ago). "We" then asked him to revert 
distutils and do the work in a new package instead of inside distutils 
(at the language summit this year).

I would perhaps argue for a case by case exception on PEPs that 
*required* distutils support that are being accepted and implemented 
prior to distutils2 moving into the standard library. It doesn't sound 
like your changes are *required* by the PEP though.

As I recall Tarek thought it was a bad idea to freeze distutils as well, 
but "we" insisted. :-)
> IIRC, it was really the incompatible changes that made people ask you to
> stop changing distutils.
>
Which included virtually any change to even private APIs. Given the 
issues freezing the distutils APIs except for essential bugfixes is a 
reasonable response. I don't know of any situation it has made worse. 
Things are getting very much better, but happening in distutils2 not 
distutils.

All the best,

Michael Foord

> Regards,
> Martin
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk


-- 

http://www.voidspace.org.uk/

READ CAREFULLY. By accepting and reading this email you agree,
on behalf of your employer, to release me from all obligations
and waivers arising from any and all NON-NEGOTIATED agreements,
licenses, terms-of-service, shrinkwrap, clickwrap, browsewrap,
confidentiality, non-disclosure, non-compete and acceptable use
policies (?BOGUS AGREEMENTS?) that I have entered into with your
employer, its partners, licensors, agents and assigns, in
perpetuity, without prejudice to my ongoing rights and privileges.
You further represent that you have the authority to release me
from any BOGUS AGREEMENTS on behalf of your employer.


From dickinsm at gmail.com  Thu Dec  2 22:57:45 2010
From: dickinsm at gmail.com (Mark Dickinson)
Date: Thu, 2 Dec 2010 21:57:45 +0000
Subject: [Python-Dev] Python and the Unicode Character Database
In-Reply-To: <4CF8004D.1080408@v.loewis.de>
References: <AANLkTinVpySuRFkjf-Sc+7a91zrxhuQpqPQL5nM9u1w+@mail.gmail.com>
	<20101128214311.092abd35@pitrou.net> <4CF2D4E9.3060607@v.loewis.de>
	<4CF2DAD7.2000408@egenix.com> <4CF2DFD1.10901@v.loewis.de>
	<4CF7FC01.80307@egenix.com> <4CF8004D.1080408@v.loewis.de>
Message-ID: <AANLkTik6LKOWrwRTrQn+tFETU5eWfuinf+=T=_yeOA8u@mail.gmail.com>

On Thu, Dec 2, 2010 at 8:23 PM, "Martin v. L?wis" <martin at v.loewis.de> wrote:
> In the case of number parsing, I think Python would be better if
> float() rejected non-ASCII strings, and any support for such parsing
> should be redone correctly in a different place (preferably along with
> printing of numbers).

+1.  The set of strings currently accepted by the float constructor
just seems too ad hoc to be at all useful.  Apart from the decimal
separator issue, and the question of exactly which decimal digits are
accepted and which aren't, there are issues like this one:

>>> x = '\uff11\uff25\uff0b\uff11\uff10'
>>> x
'?????'
>>> float(x)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
UnicodeEncodeError: 'decimal' codec can't encode character '\uff25' in
position 1: invalid decimal Unicode string
>>> y = '\uff11E+\uff11\uff10'
>>> y
'?E+??'
>>> float(y)
10000000000.0

That is, fullwidth *digits* are allowed, but none of the other
characters can be fullwidth variants.  Unfortunately, a float string
doesn't consist solely of digits, and it seems to me to make little
sense to allow variation in the digits without allowing corresponding
variations in the other characters that might appear ('.', 'e', 'E',
'+', '-').

A couple of slightly trickier decisions: (1) the float constructor
currently does accept leading and trailing whitespace;  should it
allow any Unicode whitespace characters here? I'd say yes. (2) For
int() rather than float(), there's a bit more value in allowing the
variant digits, since it provides an easy way to interpret those
digits.  The decimal module currently makes use of this, for example
(the decimal spec requires that non-European digits be accepted).  I'd
be happier if this functionality were moved elsewhere, though.  The
int constructor is, if anything, currently worse off than float,
thanks to its attempts to support non-decimal bases.

There's value in having an easy-to-specify, easy-to-maintain API for
these basic builtin functions.  For one thing, it helps non-CPython
implementations.

[MAL]
>> The Python 3.x docs apparently
>> introduced a reference to the language spec which is clearly not
>> capturing the wealth of possible inputs.

That documentation update was my fault;  I was motivated to make the
update by issues unrelated to this one (mostly to do with Python 3's
more consistent handling of inf and nan, as a result of all the new
float<->string conversion code).  If I'd been thinking harder, I would
have remembered that float accepted the non-European digits and added
a note to that effect.  This (unintentional) omission does underline
the point that it's difficult right now to document and understand
exactly what the float constructor does or doesn't accept.

Mark

From eric at trueblade.com  Thu Dec  2 22:57:45 2010
From: eric at trueblade.com (Eric Smith)
Date: Thu, 02 Dec 2010 16:57:45 -0500
Subject: [Python-Dev] Python and the Unicode Character Database
In-Reply-To: <4CF81431.2040900@v.loewis.de>
References: <AANLkTinVpySuRFkjf-Sc+7a91zrxhuQpqPQL5nM9u1w+@mail.gmail.com>	<20101128214311.092abd35@pitrou.net>	<4CF2D4E9.3060607@v.loewis.de>	<4CF2DAD7.2000408@egenix.com>	<4CF2DFD1.10901@v.loewis.de>	<4CF7FC01.80307@egenix.com>	<4CF8004D.1080408@v.loewis.de>	<4CF80FFE.8020505@pearwood.info>
	<4CF81431.2040900@v.loewis.de>
Message-ID: <4CF81659.4030302@trueblade.com>

On 12/2/2010 4:48 PM, "Martin v. L?wis" wrote:
> Am 02.12.2010 22:30, schrieb Steven D'Aprano:
>> Martin v. L?wis wrote:
>>>>> Then these users should speak up and indicate their need, or somebody
>>>>> should speak up and confirm that there are users who actually want
>>>>> '????.??' to denote 1234.56. To my knowledge, there is no writing
>>>>> system in which '????.??e4' means 12345600.0.
>>>> I'm not sure what you're after here.
>>>
>>> That the current float() constructor accepts tons of bogus character
>>> strings and accepts them as numbers, and that it should stop doing so.
>>
>> What bogus characters do the float() and int() constructors accept? As
>> far as I can see, they only accepts numerals.
>
> Not bogus characters, but bogus character strings. E.g. strings that mix
> digits from different scripts, and mix them with the Python decimal
> separator.
>
>>> Notice that Python does *not* currently support printing numbers in
>>> other scripts - even though this may actually be more useful than
>>> parsing.
>>
>> Lack of one function, even if more useful, does not imply that an
>> existing function should be removed.
>
> No. But if the specific function(ality) is not useful and
> underspecified, it should be removed.
>
>> So your problems with the current behaviour are:
>>
>> (1) in some unspecified way, it's not done correctly;
>
> No. My main concern is that it is not properly specified. If it was
> specified, I could then tell you what precisely is wrong about it.
> Right now, I can only give examples for input that it should not accept,
> and examples of input that it should, but does not accept.
>
>> (2) it belongs somewhere other than float() and int().
>
> That's only because it also needs a parameter to specify what syntax to
> follow, somehow. That parameter could be explicit or implicit, and it
> could be to float or to some other function. But it must be available,
> and is not.
>
>> That second is awfully close to bike-shedding. Since you accept that
>> Python *should* have the current behaviour
>
> No, I don't. I think it behaves incorrectly, accepting garbage input and
> guessing some meaning out of it.
>
>> - how the current behaviour is incorrect;
>
> See above: it accepts strings that do not denote real numbers in any
> writing system, and, despite the claim that the feature is there to
> support other writing systems, actually does not truly support other
> writing systems.
>
>> - your suggestions for correcting it; and
>
> Make the current implementation exactly match the current documentation.
> I think the documentation is correct; the implementation is wrong.
>
>> - a concrete suggestion for where you would like to see the behaviour
>> moved to, and why that would be better than where it currently is.
>
> The current behavior should go nowhere; it is not useful. Something very
> similar to the current behavior (but done correctly) should go into the
> locale module.

I agree with everything Martin says here. I think the basic premise is: 
you won't find strings "in the wild" that use non-ASCII digits but do 
use the ASCII dot as a decimal point. And that's what float() is looking 
for. (And that doesn't even begin to address what it expects for an 
exponent 'e'.)

Eric.

From martin at v.loewis.de  Thu Dec  2 23:17:05 2010
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Thu, 02 Dec 2010 23:17:05 +0100
Subject: [Python-Dev] PEP 384 accepted
In-Reply-To: <AANLkTinspUo88oUnd3cX776mMRMU6bqyOaq9Og7JLwhU@mail.gmail.com>
References: <AANLkTimQsiU+nyX6CdqQ8JBybJdnx=1EMxX9mefHbx9H@mail.gmail.com>	<4CF8008B.60402@v.loewis.de>	<AANLkTikbRJEFYE_TvN++GYhmvSUu3hdhf4znOti7oty1@mail.gmail.com>	<4CF80964.4070300@v.loewis.de>	<AANLkTikcT0WGo3FhFVq=N6ktToNenBDNNcC4eSpefcFF@mail.gmail.com>	<4CF80D6C.6050703@v.loewis.de>	<AANLkTi=s5NMxjPdRQRcupNBzdqjMhLO6TbFWtQzi0TEw@mail.gmail.com>	<4CF81208.8090705@v.loewis.de>
	<AANLkTinspUo88oUnd3cX776mMRMU6bqyOaq9Og7JLwhU@mail.gmail.com>
Message-ID: <4CF81AE1.5050900@v.loewis.de>

>> This freeze made the situation worse.
> 
> Can you extend on this and explains why it makes it worse ?

Before the freeze, distutils was unmaintained (i.e. before you started
maintaining it), but people who want to improve it gradually atleast
could. Now gradual improvements are also banned, so it's not only
unmaintained, but I can't even provide support for the PEP in Python
that was just accepted.

>> IIRC, it was really the incompatible changes that made people ask you to
>> stop changing distutils.
> 
> Who is "people" ? Are you suggesting that we could have added all the
> new features in Distutils in the stdlib ?

No, only the ones that didn't cause backwards incompatibilities,
and broke existing packages.

> But that's what you would expect for a project that needs to evolve a
> lot. Thus the freezing.

Instead of evolving a lot, and instead of freezing, I would have
preferred "evolve a little".

> So how would you make the situation better, if not by doing the work
> in distutils2 ?

Lift the freeze. I'm all for replacing distutils with distutils2, but
I'm not sure whether you will declare distutils2 ready tomorrow, next
year, or ten years from now.

Regards,
Martin

From martin at v.loewis.de  Thu Dec  2 23:21:25 2010
From: martin at v.loewis.de (=?windows-1252?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Thu, 02 Dec 2010 23:21:25 +0100
Subject: [Python-Dev] PEP 384 accepted
In-Reply-To: <4CF815AE.1040404@voidspace.org.uk>
References: <AANLkTimQsiU+nyX6CdqQ8JBybJdnx=1EMxX9mefHbx9H@mail.gmail.com>	<4CF8008B.60402@v.loewis.de>	<AANLkTikbRJEFYE_TvN++GYhmvSUu3hdhf4znOti7oty1@mail.gmail.com>	<4CF80964.4070300@v.loewis.de>	<AANLkTikcT0WGo3FhFVq=N6ktToNenBDNNcC4eSpefcFF@mail.gmail.com>	<4CF80D6C.6050703@v.loewis.de>	<AANLkTi=s5NMxjPdRQRcupNBzdqjMhLO6TbFWtQzi0TEw@mail.gmail.com>
	<4CF81208.8090705@v.loewis.de> <4CF815AE.1040404@voidspace.org.uk>
Message-ID: <4CF81BE5.1060003@v.loewis.de>

Am 02.12.2010 22:54, schrieb Michael Foord:
> On 02/12/2010 21:39, "Martin v. L?wis" wrote:
>>> I was told not to touch to Distutils code to avoid any regression
>>> since it's patched to the bones in third party products. So we decided
>>> to freeze distutils and add all new features in Distutils2, which is
>>> at alpha stage now.  So this move seems contradictory to me.
>> I think it was a bad decision to freeze distutils, and "we" certainly
>> didn't make that (not any we that includes me, that is). This freeze
>> made the situation worse.
> 
> What situation worse?

The "distutils is unmaintained" situation. It's not only unmaintained
now, but proposed improvements are rejected without consideration, on
the grounds that they are changes.

> I would perhaps argue for a case by case exception on PEPs that
> *required* distutils support that are being accepted and implemented
> prior to distutils2 moving into the standard library. It doesn't sound
> like your changes are *required* by the PEP though.

Well, the PEP 384 text in PEP 3149 specifies a change. It's not clear
whether this change was accepted when PEP 3149 was accepted, or whether
it was accepted when PEP 384 was accepted, or whether it was not
accepted at all, or whether it was just proposed.

In any case, without the change, you won't naturally get extension
modules that use the abi3 tag proposed in 3149.

Regards,
Martin

From ziade.tarek at gmail.com  Thu Dec  2 23:23:21 2010
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Thu, 2 Dec 2010 23:23:21 +0100
Subject: [Python-Dev] PEP 384 accepted
In-Reply-To: <4CF81BE5.1060003@v.loewis.de>
References: <AANLkTimQsiU+nyX6CdqQ8JBybJdnx=1EMxX9mefHbx9H@mail.gmail.com>
	<4CF8008B.60402@v.loewis.de>
	<AANLkTikbRJEFYE_TvN++GYhmvSUu3hdhf4znOti7oty1@mail.gmail.com>
	<4CF80964.4070300@v.loewis.de>
	<AANLkTikcT0WGo3FhFVq=N6ktToNenBDNNcC4eSpefcFF@mail.gmail.com>
	<4CF80D6C.6050703@v.loewis.de>
	<AANLkTi=s5NMxjPdRQRcupNBzdqjMhLO6TbFWtQzi0TEw@mail.gmail.com>
	<4CF81208.8090705@v.loewis.de> <4CF815AE.1040404@voidspace.org.uk>
	<4CF81BE5.1060003@v.loewis.de>
Message-ID: <AANLkTimfVCf5fjQQu5Xj4xko4UKKy+nLuou-EH-_8inx@mail.gmail.com>

2010/12/2 "Martin v. L?wis" <martin at v.loewis.de>:
> Am 02.12.2010 22:54, schrieb Michael Foord:
>> On 02/12/2010 21:39, "Martin v. L?wis" wrote:
>>>> I was told not to touch to Distutils code to avoid any regression
>>>> since it's patched to the bones in third party products. So we decided
>>>> to freeze distutils and add all new features in Distutils2, which is
>>>> at alpha stage now. ?So this move seems contradictory to me.
>>> I think it was a bad decision to freeze distutils, and "we" certainly
>>> didn't make that (not any we that includes me, that is). This freeze
>>> made the situation worse.
>>
>> What situation worse?
>
> The "distutils is unmaintained" situation. It's not only unmaintained
> now, but proposed improvements are rejected without consideration, on
> the grounds that they are changes.

I welcome those changes in Distutils2. That's the whole point.

From martin at v.loewis.de  Thu Dec  2 23:29:10 2010
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Thu, 02 Dec 2010 23:29:10 +0100
Subject: [Python-Dev] PEP 384 accepted
In-Reply-To: <AANLkTimfVCf5fjQQu5Xj4xko4UKKy+nLuou-EH-_8inx@mail.gmail.com>
References: <AANLkTimQsiU+nyX6CdqQ8JBybJdnx=1EMxX9mefHbx9H@mail.gmail.com>	<4CF8008B.60402@v.loewis.de>	<AANLkTikbRJEFYE_TvN++GYhmvSUu3hdhf4znOti7oty1@mail.gmail.com>	<4CF80964.4070300@v.loewis.de>	<AANLkTikcT0WGo3FhFVq=N6ktToNenBDNNcC4eSpefcFF@mail.gmail.com>	<4CF80D6C.6050703@v.loewis.de>	<AANLkTi=s5NMxjPdRQRcupNBzdqjMhLO6TbFWtQzi0TEw@mail.gmail.com>	<4CF81208.8090705@v.loewis.de>	<4CF815AE.1040404@voidspace.org.uk>	<4CF81BE5.1060003@v.loewis.de>
	<AANLkTimfVCf5fjQQu5Xj4xko4UKKy+nLuou-EH-_8inx@mail.gmail.com>
Message-ID: <4CF81DB6.7040209@v.loewis.de>

>> The "distutils is unmaintained" situation. It's not only unmaintained
>> now, but proposed improvements are rejected without consideration, on
>> the grounds that they are changes.
> 
> I welcome those changes in Distutils2. That's the whole point.

That would be useful if there was a clear vision of when distutils2
will be released. Please understand that I'm not blaming you for not
releasing it (it *is* too much for a single person), but please
understand that it's also not helpful to submit changes to a codebase
that is not going to be released in a foreseeable future.

Regards,
Martin

From mal at egenix.com  Thu Dec  2 23:43:12 2010
From: mal at egenix.com (M.-A. Lemburg)
Date: Thu, 02 Dec 2010 23:43:12 +0100
Subject: [Python-Dev] Python and the Unicode Character Database
In-Reply-To: <4CF81659.4030302@trueblade.com>
References: <AANLkTinVpySuRFkjf-Sc+7a91zrxhuQpqPQL5nM9u1w+@mail.gmail.com>	<20101128214311.092abd35@pitrou.net>	<4CF2D4E9.3060607@v.loewis.de>	<4CF2DAD7.2000408@egenix.com>	<4CF2DFD1.10901@v.loewis.de>	<4CF7FC01.80307@egenix.com>	<4CF8004D.1080408@v.loewis.de>	<4CF80FFE.8020505@pearwood.info>	<4CF81431.2040900@v.loewis.de>
	<4CF81659.4030302@trueblade.com>
Message-ID: <4CF82100.5080308@egenix.com>

Eric Smith wrote:
>> The current behavior should go nowhere; it is not useful. Something very
>> similar to the current behavior (but done correctly) should go into the
>> locale module.
> 
> I agree with everything Martin says here. I think the basic premise is:
> you won't find strings "in the wild" that use non-ASCII digits but do
> use the ASCII dot as a decimal point. And that's what float() is looking
> for. (And that doesn't even begin to address what it expects for an
> exponent 'e'.)

http://en.wikipedia.org/wiki/Decimal_mark

"In China, comma and space are used to mark digit groups because dot is used as decimal mark."

Note that float() can also parse integers, it just returns them as
floats :-)

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Dec 02 2010)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try our new mxODBC.Connect Python Database Interface for free ! ::::


   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
           Registered at Amtsgericht Duesseldorf: HRB 46611
               http://www.egenix.com/company/contact/

From ziade.tarek at gmail.com  Thu Dec  2 23:44:57 2010
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Thu, 2 Dec 2010 23:44:57 +0100
Subject: [Python-Dev] PEP 384 accepted
In-Reply-To: <4CF81AE1.5050900@v.loewis.de>
References: <AANLkTimQsiU+nyX6CdqQ8JBybJdnx=1EMxX9mefHbx9H@mail.gmail.com>
	<4CF8008B.60402@v.loewis.de>
	<AANLkTikbRJEFYE_TvN++GYhmvSUu3hdhf4znOti7oty1@mail.gmail.com>
	<4CF80964.4070300@v.loewis.de>
	<AANLkTikcT0WGo3FhFVq=N6ktToNenBDNNcC4eSpefcFF@mail.gmail.com>
	<4CF80D6C.6050703@v.loewis.de>
	<AANLkTi=s5NMxjPdRQRcupNBzdqjMhLO6TbFWtQzi0TEw@mail.gmail.com>
	<4CF81208.8090705@v.loewis.de>
	<AANLkTinspUo88oUnd3cX776mMRMU6bqyOaq9Og7JLwhU@mail.gmail.com>
	<4CF81AE1.5050900@v.loewis.de>
Message-ID: <AANLkTi=FDOAgf9DCQ6N5EpMCJs6DCvsC3bQR6bbL=n+_@mail.gmail.com>

2010/12/2 "Martin v. L?wis" <martin at v.loewis.de>:
>>> This freeze made the situation worse.
>>
>> Can you extend on this and explains why it makes it worse ?
>
> Before the freeze, distutils was unmaintained (i.e. before you started
> maintaining it), but people who want to improve it gradually atleast
> could. Now gradual improvements are also banned, so it's not only
> unmaintained, but I can't even provide support for the PEP in Python
> that was just accepted.
>
>>> IIRC, it was really the incompatible changes that made people ask you to
>>> stop changing distutils.
>>
>> Who is "people" ? Are you suggesting that we could have added all the
>> new features in Distutils in the stdlib ?
>
> No, only the ones that didn't cause backwards incompatibilities,
> and broke existing packages.

This is impossible. I can point you to some third party project that
can break if you touch some distutils internals, like setuptools.
Setuptools also uses some privates global variables in some other
modules in the stdlib FYI.

The right answer was maybe back then: make setuptools and other
projects evolve with distutils.

But it did not happen. So we left the status quo and moved forward in
distutils2. Because we knew distutils needed deeper changes anyways,
and we knew setuptools was used everywhere and unfortunately not
evolving at the same pace. (note: I am not blaming PJE or anyone when
I say this -- the way distutils worked and was poorly maintained was
the main reason)

>
>> But that's what you would expect for a project that needs to evolve a
>> lot. Thus the freezing.
>
> Instead of evolving a lot, and instead of freezing, I would have
> preferred "evolve a little".
>
>> So how would you make the situation better, if not by doing the work
>> in distutils2 ?
>
> Lift the freeze. I'm all for replacing distutils with distutils2, but
> I'm not sure whether you will declare distutils2 ready tomorrow, next
> year, or ten years from now.

Depends on what "ready" means.

If by ready you mean it can be used to replace Distutils1 in a
project, I declare Distutils2 ready for usage NOW.  It's in alpha
stage. I want a solid beta before Pycon.

I would even remove Distutils from 3.x altogether at some point since
setuptools is not Python 3 compatible, and just put distutils2.

3.3 sounds like a good target.

Regards
Tarek

-- 
Tarek Ziad? | http://ziade.org

From solipsis at pitrou.net  Thu Dec  2 23:51:14 2010
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Thu, 2 Dec 2010 23:51:14 +0100
Subject: [Python-Dev] PEP 384 accepted
References: <AANLkTimQsiU+nyX6CdqQ8JBybJdnx=1EMxX9mefHbx9H@mail.gmail.com>
	<4CF8008B.60402@v.loewis.de>
	<AANLkTikbRJEFYE_TvN++GYhmvSUu3hdhf4znOti7oty1@mail.gmail.com>
	<4CF80964.4070300@v.loewis.de>
	<AANLkTikcT0WGo3FhFVq=N6ktToNenBDNNcC4eSpefcFF@mail.gmail.com>
	<4CF80D6C.6050703@v.loewis.de>
	<AANLkTi=s5NMxjPdRQRcupNBzdqjMhLO6TbFWtQzi0TEw@mail.gmail.com>
	<4CF81208.8090705@v.loewis.de> <4CF815AE.1040404@voidspace.org.uk>
	<4CF81BE5.1060003@v.loewis.de>
Message-ID: <20101202235114.1dc77cdc@pitrou.net>

On Thu, 02 Dec 2010 23:21:25 +0100
"Martin v. L?wis" <martin at v.loewis.de> wrote:
> Am 02.12.2010 22:54, schrieb Michael Foord:
> > On 02/12/2010 21:39, "Martin v. L?wis" wrote:
> >>> I was told not to touch to Distutils code to avoid any regression
> >>> since it's patched to the bones in third party products. So we decided
> >>> to freeze distutils and add all new features in Distutils2, which is
> >>> at alpha stage now.  So this move seems contradictory to me.
> >> I think it was a bad decision to freeze distutils, and "we" certainly
> >> didn't make that (not any we that includes me, that is). This freeze
> >> made the situation worse.
> > 
> > What situation worse?
> 
> The "distutils is unmaintained" situation. It's not only unmaintained
> now, but proposed improvements are rejected without consideration, on
> the grounds that they are changes.

I think distutils is simply a bugfix branch for distutils2. Similarly
as how we don't commit improvements in e.g. 2.7 or 3.1, neither do we
commit improvements to distutils.

(and I think that's how Guido wanted it anyway)

Regards

Antoine.



From ziade.tarek at gmail.com  Thu Dec  2 23:53:08 2010
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Thu, 2 Dec 2010 23:53:08 +0100
Subject: [Python-Dev] PEP 384 accepted
In-Reply-To: <4CF81DB6.7040209@v.loewis.de>
References: <AANLkTimQsiU+nyX6CdqQ8JBybJdnx=1EMxX9mefHbx9H@mail.gmail.com>
	<4CF8008B.60402@v.loewis.de>
	<AANLkTikbRJEFYE_TvN++GYhmvSUu3hdhf4znOti7oty1@mail.gmail.com>
	<4CF80964.4070300@v.loewis.de>
	<AANLkTikcT0WGo3FhFVq=N6ktToNenBDNNcC4eSpefcFF@mail.gmail.com>
	<4CF80D6C.6050703@v.loewis.de>
	<AANLkTi=s5NMxjPdRQRcupNBzdqjMhLO6TbFWtQzi0TEw@mail.gmail.com>
	<4CF81208.8090705@v.loewis.de> <4CF815AE.1040404@voidspace.org.uk>
	<4CF81BE5.1060003@v.loewis.de>
	<AANLkTimfVCf5fjQQu5Xj4xko4UKKy+nLuou-EH-_8inx@mail.gmail.com>
	<4CF81DB6.7040209@v.loewis.de>
Message-ID: <AANLkTi==eWeCx9-VpPtWhpX-XNF8-QLxvDrkS5QbRMxf@mail.gmail.com>

2010/12/2 "Martin v. L?wis" <martin at v.loewis.de>:
>>> The "distutils is unmaintained" situation. It's not only unmaintained
>>> now, but proposed improvements are rejected without consideration, on
>>> the grounds that they are changes.
>>
>> I welcome those changes in Distutils2. That's the whole point.
>
> That would be useful if there was a clear vision of when distutils2
> will be released.   Please understand that I'm not blaming you for not
> releasing it (it *is* too much for a single person), but please
> understand that it's also not helpful to submit changes to a codebase
> that is not going to be released in a foreseeable future.

I know you're not blaming me.

Distutils 2 alpha3 is currently released and available at PyPI. I use
it in some of my professional projects FWIW.

alpha4 was postponed but should be out this month. It contains major
features, people from the GSOC worked on.

The initial roadmap was to have a final by the time 3.2 final is out,
but that'll be too short. So the target is to have a beta release for
Pycon, and to sync the final release with 3.3, with lots of feedback
in the meantime hopefully, and people using it from 2.4 onward.


>
> Regards,
> Martin
>



-- 
Tarek Ziad? | http://ziade.org

From barry at python.org  Thu Dec  2 23:54:01 2010
From: barry at python.org (Barry Warsaw)
Date: Thu, 2 Dec 2010 17:54:01 -0500
Subject: [Python-Dev] PEP 384 accepted
In-Reply-To: <4CF81BE5.1060003@v.loewis.de>
References: <AANLkTimQsiU+nyX6CdqQ8JBybJdnx=1EMxX9mefHbx9H@mail.gmail.com>
	<4CF8008B.60402@v.loewis.de>
	<AANLkTikbRJEFYE_TvN++GYhmvSUu3hdhf4znOti7oty1@mail.gmail.com>
	<4CF80964.4070300@v.loewis.de>
	<AANLkTikcT0WGo3FhFVq=N6ktToNenBDNNcC4eSpefcFF@mail.gmail.com>
	<4CF80D6C.6050703@v.loewis.de>
	<AANLkTi=s5NMxjPdRQRcupNBzdqjMhLO6TbFWtQzi0TEw@mail.gmail.com>
	<4CF81208.8090705@v.loewis.de> <4CF815AE.1040404@voidspace.org.uk>
	<4CF81BE5.1060003@v.loewis.de>
Message-ID: <20101202175401.1a94a5f4@mission>

On Dec 02, 2010, at 11:21 PM, Martin v. L?wis wrote:

>Well, the PEP 384 text in PEP 3149 specifies a change. It's not clear
>whether this change was accepted when PEP 3149 was accepted, or whether
>it was accepted when PEP 384 was accepted, or whether it was not
>accepted at all, or whether it was just proposed.

From my point of view, the PEP 3149 text is just a proposal.  It leaves the
final decision to PEP 384, but tries to address some of the issues raised
during the PEP 3149 discussion.  I think it is within PEP 384's scope to make
the final decisions about it.

>In any case, without the change, you won't naturally get extension
>modules that use the abi3 tag proposed in 3149.

I would favor changing distutils, if it can be done in a way that reasonably
preserves backward compatibility.  I suppose it's impossible to know all the
ways 3rd party code has reached into distutils, but I think you can make
fairly good judgements about whether a change is backward compatible or not.

-Barry
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 836 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/python-dev/attachments/20101202/3964c8ad/attachment.pgp>

From martin at v.loewis.de  Thu Dec  2 23:54:56 2010
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Thu, 02 Dec 2010 23:54:56 +0100
Subject: [Python-Dev] PEP 384 accepted
In-Reply-To: <AANLkTi=FDOAgf9DCQ6N5EpMCJs6DCvsC3bQR6bbL=n+_@mail.gmail.com>
References: <AANLkTimQsiU+nyX6CdqQ8JBybJdnx=1EMxX9mefHbx9H@mail.gmail.com>	<4CF8008B.60402@v.loewis.de>	<AANLkTikbRJEFYE_TvN++GYhmvSUu3hdhf4znOti7oty1@mail.gmail.com>	<4CF80964.4070300@v.loewis.de>	<AANLkTikcT0WGo3FhFVq=N6ktToNenBDNNcC4eSpefcFF@mail.gmail.com>	<4CF80D6C.6050703@v.loewis.de>	<AANLkTi=s5NMxjPdRQRcupNBzdqjMhLO6TbFWtQzi0TEw@mail.gmail.com>	<4CF81208.8090705@v.loewis.de>	<AANLkTinspUo88oUnd3cX776mMRMU6bqyOaq9Og7JLwhU@mail.gmail.com>	<4CF81AE1.5050900@v.loewis.de>
	<AANLkTi=FDOAgf9DCQ6N5EpMCJs6DCvsC3bQR6bbL=n+_@mail.gmail.com>
Message-ID: <4CF823C0.7070002@v.loewis.de>

>> No, only the ones that didn't cause backwards incompatibilities,
>> and broke existing packages.
> 
> This is impossible. I can point you to some third party project that
> can break if you touch some distutils internals, like setuptools.
> Setuptools also uses some privates global variables in some other
> modules in the stdlib FYI.

So what would break if Extension accepted an abi= keyword parameter?

>> Lift the freeze. I'm all for replacing distutils with distutils2, but
>> I'm not sure whether you will declare distutils2 ready tomorrow, next
>> year, or ten years from now.
> 
> Depends on what "ready" means.

Included in Python, so that changes become possible again.

> If by ready you mean it can be used to replace Distutils1 in a
> project, I declare Distutils2 ready for usage NOW.  It's in alpha
> stage. I want a solid beta before Pycon.
> 
> I would even remove Distutils from 3.x altogether at some point since
> setuptools is not Python 3 compatible, and just put distutils2.
> 
> 3.3 sounds like a good target.

So will distuils2 be released before that? If so, when?

Regards,
Martin

From mal at egenix.com  Thu Dec  2 23:58:07 2010
From: mal at egenix.com (M.-A. Lemburg)
Date: Thu, 02 Dec 2010 23:58:07 +0100
Subject: [Python-Dev] Python and the Unicode Character Database
In-Reply-To: <AANLkTi=QApgGxCH0ph-_QjMMu5scH97vSdsBw_Da5Fvv@mail.gmail.com>
References: <AANLkTinVpySuRFkjf-Sc+7a91zrxhuQpqPQL5nM9u1w+@mail.gmail.com>	<AANLkTinQLK7+5NfEp-2nMiJeDEKRxKN9Zfs4ihv-NMyH@mail.gmail.com>	<87wrnv43v5.fsf@uwakimon.sk.tsukuba.ac.jp>	<AANLkTinAf6CstywehOGRd4G4ms6K5LtSuq4Lj0z5wFBW@mail.gmail.com>	<87r5e236hj.fsf@benfinney.id.au>
	<4CF58387.1000609@v.loewis.de>	<87k4ju2vvg.fsf@benfinney.id.au>
	<4CF6CDDD.4060109@v.loewis.de>	<4CF6D7B2.6030506@pearwood.info>	<87sjyhxarg.fsf@uwakimon.sk.tsukuba.ac.jp>	<87bp552arq.fsf@benfinney.id.au>
	<4CF7F579.5050603@v.loewis.de>	<4CF80C3A.9000705@egenix.com>
	<AANLkTi=QApgGxCH0ph-_QjMMu5scH97vSdsBw_Da5Fvv@mail.gmail.com>
Message-ID: <4CF8247F.4000603@egenix.com>

Alexander Belopolsky wrote:
> On Thu, Dec 2, 2010 at 4:14 PM, M.-A. Lemburg <mal at egenix.com> wrote:
> ..
>> Have you tried Google ?
>>
> 
> I tried google at I could not find any plain text or HTML file that
> would use Arabic-Indic numerals.  What was interesting, though that a
> search for "quran unicode" (without quotes).  Brought me to
> http://www.sacred-texts.com which says that they've been using unicode
> since 2002 in their archives.  Interestingly enough, their version of
> Qur'an uses ordinary digits for ayah numbers.  See, for example
> <http://www.sacred-texts.com/isl/uq/050.htm>.
> 
> I will change my mind on this issue when you present a
> machine-readable file with Arabic-Indic numerals and a program capable
> of reading it and show that this program uses the same number parsing
> algorithm as Python's int() or float().

Have you had a look at the examples I posted ? They include texts
and tables with numbers written using east asian arabic numerals.

Here's an example of a a famous Chinese text using Chinese numerals:

http://ctext.org/nine-chapters

Unfortunately, the Chinese numerals are not listed in the Category "Nd",
so Python won't be able to parse them. This has various reasons, it
seems, one of them being that the numeral code points were not defined
as range of code points.

I'm sure you can find other books on mathematics in sanscrit or
arabic scripts as well.

But this whole branch of the discussion is not going to go anywhere.

The point is that we support all of Unicode in Python, not just a fragment,
and therefore the numeric constructors support all of Unicode.

Using them, it's very easy to support numbers in all kinds of variants,
whether bound to a locale or not.

Adding more locale aware numeric parsers and formatters to the
locale module, based on these APIs is certainly a good idea,
but orthogonal to the ongoing discussion, IMO.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Dec 02 2010)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try our new mxODBC.Connect Python Database Interface for free ! ::::


   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
           Registered at Amtsgericht Duesseldorf: HRB 46611
               http://www.egenix.com/company/contact/

From mal at egenix.com  Fri Dec  3 00:01:24 2010
From: mal at egenix.com (M.-A. Lemburg)
Date: Fri, 03 Dec 2010 00:01:24 +0100
Subject: [Python-Dev] Python and the Unicode Character Database
In-Reply-To: <id0ujb$d5s$1@dough.gmane.org>
References: <AANLkTinVpySuRFkjf-Sc+7a91zrxhuQpqPQL5nM9u1w+@mail.gmail.com>	<20101128214311.092abd35@pitrou.net>	<4CF2D4E9.3060607@v.loewis.de>	<4CF2DAD7.2000408@egenix.com>	<87pqto6bnv.fsf@uwakimon.sk.tsukuba.ac.jp>	<AANLkTi=97wAXyxD_1nJsGJknXL3WdsNvQH-sKRaz-KR4@mail.gmail.com>	<4CF38861.5090309@egenix.com>	<AANLkTim9Othk-AGA2SjTmiWEsfyHowLsEpZ8ux-eNTLw@mail.gmail.com>	<4CF3C477.1020007@egenix.com>
	<id0ujb$d5s$1@dough.gmane.org>
Message-ID: <4CF82544.2080506@egenix.com>

Terry Reedy wrote:
> On 11/29/2010 10:19 AM, M.-A. Lemburg wrote:
>> Nick Coghlan wrote:
>>> On Mon, Nov 29, 2010 at 9:02 PM, M.-A. Lemburg<mal at egenix.com>  wrote:
>>>> If we would go down that road, we would also have to disable other
>>>> Unicode features based on locale, e.g. whether to apply non-ASCII
>>>> case mappings, what to consider whitespace, etc.
>>>>
>>>> We don't do that for a good reason: Unicode is supposed to be
>>>> universal and not limited to a single locale.
>>>
>>> Because parsing numbers is about more than just the characters used
>>> for the individual digits. There are additional semantics associated
>>> with digit ordering (for any number) and decimal separators and
>>> exponential notation (for floating point numbers) and those vary by
>>> locale. We deliberately chose to make the builtin numeric parsers
>>> unaware of all of those things, and assuming that we can simply parse
>>> other digits as if they were their ASCII equivalents and otherwise
>>> assume a C locale seems questionable.
>>
>> Sure, and those additional semantics are locale dependent, even
>> between ASCII-only locales. However, that does not apply to the
>> basic building blocks, the decimal digits themselves.
>>
>>> If the existing semantics can be adequately defined, documented and
>>> defended, then retaining them would be fine. However, the language
>>> reference needs to define the behaviour properly so that other
>>> implementations know what they need to support and what can be chalked
>>> up as being just an implementation accident of CPython. (As a point in
>>> the plus column, both decimal.Decimal and fractions.Fraction were able
>>> to handle the '????.??' example in a manner consistent with the int
>>> and float handling)
>>
>> The support is built into the C API, so there's not really much
>> surprise there.
>>
>> Regarding documentation, we'd just have to add that numbers may
>> be made up of an Unicode code point in the category "Nd".
>>
>> See http://www.unicode.org/versions/Unicode5.2.0/ch04.pdf, section
>> 4.6 for details....
>>
>> """
>> Decimal digits form a large subcategory of numbers consisting of those
>> digits that can be
>> used to form decimal-radix numbers. They include script-specific
>> digits, but exclude char-
>> acters such as Roman numerals and Greek acrophonic numerals. (Note
>> that<1, 5>  = 15 =
>> fifteen, but<I, V>  = IV = four.) Decimal digits also exclude the
>> compatibility subscript or
>> superscript digits to prevent simplistic parsers from misinterpreting
>> their values in context.
>> """
>>
>> int(), float() and long() (in Python2) are such simplistic
>> parsers.
> 
> Since you are the knowledgable advocate of the current behavior, perhaps
> you could open an issue and propose a doc patch, even if not .rst
> formatted.

Good suggestion. I tried to collect as much context as possible:

http://bugs.python.org/issue10610

I'll leave the rst-magic to someone else, but will certainly help
if you have more questions about the details.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Dec 02 2010)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try our new mxODBC.Connect Python Database Interface for free ! ::::


   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
           Registered at Amtsgericht Duesseldorf: HRB 46611
               http://www.egenix.com/company/contact/

From martin at v.loewis.de  Fri Dec  3 00:01:52 2010
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Fri, 03 Dec 2010 00:01:52 +0100
Subject: [Python-Dev] PEP 384 accepted
In-Reply-To: <20101202235114.1dc77cdc@pitrou.net>
References: <AANLkTimQsiU+nyX6CdqQ8JBybJdnx=1EMxX9mefHbx9H@mail.gmail.com>	<4CF8008B.60402@v.loewis.de>	<AANLkTikbRJEFYE_TvN++GYhmvSUu3hdhf4znOti7oty1@mail.gmail.com>	<4CF80964.4070300@v.loewis.de>	<AANLkTikcT0WGo3FhFVq=N6ktToNenBDNNcC4eSpefcFF@mail.gmail.com>	<4CF80D6C.6050703@v.loewis.de>	<AANLkTi=s5NMxjPdRQRcupNBzdqjMhLO6TbFWtQzi0TEw@mail.gmail.com>	<4CF81208.8090705@v.loewis.de>
	<4CF815AE.1040404@voidspace.org.uk>	<4CF81BE5.1060003@v.loewis.de>
	<20101202235114.1dc77cdc@pitrou.net>
Message-ID: <4CF82560.7020001@v.loewis.de>

> I think distutils is simply a bugfix branch for distutils2. Similarly
> as how we don't commit improvements in e.g. 2.7 or 3.1, neither do we
> commit improvements to distutils.

It's different, though, in the sense that Python has a release schedule
and multiple committers working on it, and that it normally gets
released even if some changes don't get included in a specific release
yet.

All this seems not to be true for distutils2. So my motivation to
contribute changes to it is *much* lower than my desire to contribute
to distutils, and it is also provably lower than my motivation to
contribute to distribute (say). I'm just getting tired having to talk to
five projects just to make a single change to the build infrastructure
available to the Python community.

Regards,
Martin

From alexander.belopolsky at gmail.com  Fri Dec  3 00:11:24 2010
From: alexander.belopolsky at gmail.com (Alexander Belopolsky)
Date: Thu, 2 Dec 2010 18:11:24 -0500
Subject: [Python-Dev] Python and the Unicode Character Database
In-Reply-To: <4CF8247F.4000603@egenix.com>
References: <AANLkTinVpySuRFkjf-Sc+7a91zrxhuQpqPQL5nM9u1w+@mail.gmail.com>
	<AANLkTinQLK7+5NfEp-2nMiJeDEKRxKN9Zfs4ihv-NMyH@mail.gmail.com>
	<87wrnv43v5.fsf@uwakimon.sk.tsukuba.ac.jp>
	<AANLkTinAf6CstywehOGRd4G4ms6K5LtSuq4Lj0z5wFBW@mail.gmail.com>
	<87r5e236hj.fsf@benfinney.id.au> <4CF58387.1000609@v.loewis.de>
	<87k4ju2vvg.fsf@benfinney.id.au> <4CF6CDDD.4060109@v.loewis.de>
	<4CF6D7B2.6030506@pearwood.info>
	<87sjyhxarg.fsf@uwakimon.sk.tsukuba.ac.jp>
	<87bp552arq.fsf@benfinney.id.au> <4CF7F579.5050603@v.loewis.de>
	<4CF80C3A.9000705@egenix.com>
	<AANLkTi=QApgGxCH0ph-_QjMMu5scH97vSdsBw_Da5Fvv@mail.gmail.com>
	<4CF8247F.4000603@egenix.com>
Message-ID: <AANLkTim8Uy9yDM8AFeho03dpEUmrgRdOyrUa1QdeKS5z@mail.gmail.com>

On Thu, Dec 2, 2010 at 5:58 PM, M.-A. Lemburg <mal at egenix.com> wrote:
..
>> I will change my mind on this issue when you present a
>> machine-readable file with Arabic-Indic numerals and a program capable
>> of reading it and show that this program uses the same number parsing
>> algorithm as Python's int() or float().
>
> Have you had a look at the examples I posted ? They include texts
> and tables with numbers written using east asian arabic numerals.

Yes, but this was all about output.  I am pretty sure TeX was able to
typeset Qur'an in all its glory long before Unicode was invented.
Yet, in machine readable form it would be something like {\quran 1}
(invented directive).   I have asked for a file that is intended for
machine processing, not for human enjoyment in print or on a display.
 I claim that if such file exists, the program that reads it does not
use the same rules as Python and converting non-ascii digits would be
a tiny portion of what that program does.

From martin at v.loewis.de  Fri Dec  3 00:19:20 2010
From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=)
Date: Fri, 03 Dec 2010 00:19:20 +0100
Subject: [Python-Dev] Python and the Unicode Character Database
In-Reply-To: <4CF82100.5080308@egenix.com>
References: <AANLkTinVpySuRFkjf-Sc+7a91zrxhuQpqPQL5nM9u1w+@mail.gmail.com>	<20101128214311.092abd35@pitrou.net>	<4CF2D4E9.3060607@v.loewis.de>	<4CF2DAD7.2000408@egenix.com>	<4CF2DFD1.10901@v.loewis.de>	<4CF7FC01.80307@egenix.com>	<4CF8004D.1080408@v.loewis.de>	<4CF80FFE.8020505@pearwood.info>	<4CF81431.2040900@v.loewis.de>	<4CF81659.4030302@trueblade.com>
	<4CF82100.5080308@egenix.com>
Message-ID: <4CF82978.1060406@v.loewis.de>

Am 02.12.2010 23:43, schrieb M.-A. Lemburg:
> Eric Smith wrote:
>>> The current behavior should go nowhere; it is not useful. Something very
>>> similar to the current behavior (but done correctly) should go into the
>>> locale module.
>>
>> I agree with everything Martin says here. I think the basic premise is:
>> you won't find strings "in the wild" that use non-ASCII digits but do
>> use the ASCII dot as a decimal point. And that's what float() is looking
>> for. (And that doesn't even begin to address what it expects for an
>> exponent 'e'.)
> 
> http://en.wikipedia.org/wiki/Decimal_mark
> 
> "In China, comma and space are used to mark digit groups because dot is used as decimal mark."

I may be misinterpreting that, but I think that refers to the case of
writing numbers using Arabic digits.

"Chinese" digits are, e.g., used in the Suzhou numerals

http://en.wikipedia.org/wiki/Suzhou_numerals

This doesn't have a decimal point at all. Instead, the second line
(below or left to the actual digits) describes the power of ten and
the unit of measurement (i.e. similar to scientific notation,
but with ideographs for the powers of ten).

In another writing system, they use ? (U+70B9) as the decimal
separator, see

http://en.wikipedia.org/wiki/Chinese_numerals#Fractional_values

In the same system, the integral part uses multipliers, i.e.
12345 is [1][10000][2][1000][3][100][4][10][5]; the fractional
part uses regular digits.

Regards,
Martin


From eric at trueblade.com  Thu Dec  2 23:45:19 2010
From: eric at trueblade.com (Eric Smith)
Date: Thu, 02 Dec 2010 17:45:19 -0500
Subject: [Python-Dev] Python and the Unicode Character Database
In-Reply-To: <4CF82100.5080308@egenix.com>
References: <AANLkTinVpySuRFkjf-Sc+7a91zrxhuQpqPQL5nM9u1w+@mail.gmail.com>	<20101128214311.092abd35@pitrou.net>	<4CF2D4E9.3060607@v.loewis.de>	<4CF2DAD7.2000408@egenix.com>	<4CF2DFD1.10901@v.loewis.de>	<4CF7FC01.80307@egenix.com>	<4CF8004D.1080408@v.loewis.de>	<4CF80FFE.8020505@pearwood.info>	<4CF81431.2040900@v.loewis.de>
	<4CF81659.4030302@trueblade.com> <4CF82100.5080308@egenix.com>
Message-ID: <4CF8217F.2010005@trueblade.com>

On 12/2/2010 5:43 PM, M.-A. Lemburg wrote:
> Eric Smith wrote:
>>> The current behavior should go nowhere; it is not useful. Something very
>>> similar to the current behavior (but done correctly) should go into the
>>> locale module.
>>
>> I agree with everything Martin says here. I think the basic premise is:
>> you won't find strings "in the wild" that use non-ASCII digits but do
>> use the ASCII dot as a decimal point. And that's what float() is looking
>> for. (And that doesn't even begin to address what it expects for an
>> exponent 'e'.)
>
> http://en.wikipedia.org/wiki/Decimal_mark
>
> "In China, comma and space are used to mark digit groups because dot is used as decimal mark."

Is that an ASCII dot? That page doesn't say.

> Note that float() can also parse integers, it just returns them as
> floats :-)

:)



From fuzzyman at voidspace.org.uk  Fri Dec  3 00:23:46 2010
From: fuzzyman at voidspace.org.uk (Michael Foord)
Date: Thu, 02 Dec 2010 23:23:46 +0000
Subject: [Python-Dev] PEP 384 accepted
In-Reply-To: <4CF82560.7020001@v.loewis.de>
References: <AANLkTimQsiU+nyX6CdqQ8JBybJdnx=1EMxX9mefHbx9H@mail.gmail.com>	<4CF8008B.60402@v.loewis.de>	<AANLkTikbRJEFYE_TvN++GYhmvSUu3hdhf4znOti7oty1@mail.gmail.com>	<4CF80964.4070300@v.loewis.de>	<AANLkTikcT0WGo3FhFVq=N6ktToNenBDNNcC4eSpefcFF@mail.gmail.com>	<4CF80D6C.6050703@v.loewis.de>	<AANLkTi=s5NMxjPdRQRcupNBzdqjMhLO6TbFWtQzi0TEw@mail.gmail.com>	<4CF81208.8090705@v.loewis.de>	<4CF815AE.1040404@voidspace.org.uk>	<4CF81BE5.1060003@v.loewis.de>	<20101202235114.1dc77cdc@pitrou.net>
	<4CF82560.7020001@v.loewis.de>
Message-ID: <4CF82A82.2020804@voidspace.org.uk>

On 02/12/2010 23:01, "Martin v. L?wis" wrote:
>> [snip...]
> I'm just getting tired having to talk to
> five projects just to make a single change to the build infrastructure
> available to the Python community.
>

The very best hope of resolving that particular problem is distutils2. :-)

distutils2 is *already* available to the Python community, and whether 
or not there is a fixed release date it will have betas and then a 1.0 
release in the foreseeable future. The team working on it has made an 
enormous amount of progress. We're much better off as a development 
community putting our support and energy into distutils2 rather than 
pining for evolution of distutils.

Michael

> Regards,
> Martin
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk


-- 

http://www.voidspace.org.uk/

READ CAREFULLY. By accepting and reading this email you agree,
on behalf of your employer, to release me from all obligations
and waivers arising from any and all NON-NEGOTIATED agreements,
licenses, terms-of-service, shrinkwrap, clickwrap, browsewrap,
confidentiality, non-disclosure, non-compete and acceptable use
policies (?BOGUS AGREEMENTS?) that I have entered into with your
employer, its partners, licensors, agents and assigns, in
perpetuity, without prejudice to my ongoing rights and privileges.
You further represent that you have the authority to release me
from any BOGUS AGREEMENTS on behalf of your employer.


From ziade.tarek at gmail.com  Fri Dec  3 00:25:48 2010
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Fri, 3 Dec 2010 00:25:48 +0100
Subject: [Python-Dev] PEP 384 accepted
In-Reply-To: <4CF823C0.7070002@v.loewis.de>
References: <AANLkTimQsiU+nyX6CdqQ8JBybJdnx=1EMxX9mefHbx9H@mail.gmail.com>
	<4CF8008B.60402@v.loewis.de>
	<AANLkTikbRJEFYE_TvN++GYhmvSUu3hdhf4znOti7oty1@mail.gmail.com>
	<4CF80964.4070300@v.loewis.de>
	<AANLkTikcT0WGo3FhFVq=N6ktToNenBDNNcC4eSpefcFF@mail.gmail.com>
	<4CF80D6C.6050703@v.loewis.de>
	<AANLkTi=s5NMxjPdRQRcupNBzdqjMhLO6TbFWtQzi0TEw@mail.gmail.com>
	<4CF81208.8090705@v.loewis.de>
	<AANLkTinspUo88oUnd3cX776mMRMU6bqyOaq9Og7JLwhU@mail.gmail.com>
	<4CF81AE1.5050900@v.loewis.de>
	<AANLkTi=FDOAgf9DCQ6N5EpMCJs6DCvsC3bQR6bbL=n+_@mail.gmail.com>
	<4CF823C0.7070002@v.loewis.de>
Message-ID: <AANLkTinOwVj3iLbYT4kkoh0dPmW34WQx1HsZJPfnepT4@mail.gmail.com>

2010/12/2 "Martin v. L?wis" <martin at v.loewis.de>:
>>> No, only the ones that didn't cause backwards incompatibilities,
>>> and broke existing packages.
>>
>> This is impossible. I can point you to some third party project that
>> can break if you touch some distutils internals, like setuptools.
>> Setuptools also uses some privates global variables in some other
>> modules in the stdlib FYI.
>
> So what would break if Extension accepted an abi= keyword parameter?

I suppose you have code behind this, that will be in build_ext and in
the compilers. So you will need to try out ALL projects out there that
customize build_ext, like numpy or setuptools, etc, But you won't be
able to try out all projects because they are not listed somewhere.

For starters, the Extension class is replaced by another one in
setuptools, that patches the constructor if Pyrex is installed,
which is unlikely I guess, so no big deal. But you will also get a
replaced version of the Distribution class that uses a private method
from distutils, and another version of build_ext with custom compiling
flags.

Now depending on how you do your thing it could work if you are
careful at doing things on the top of setuptools.

And then, if numpy.distutils is installed, it relies on distutils
build_ext and tries to rely on setuptools one's too, so it gets in the
mix of the patched classes, and you get an horrible mix and possible
bad interactions.

So I am not saying it's impossible to add the feature, but it is
impossible to be sure nothing gets broken in third party.

So the freeze seems wise indeed

>>> Lift the freeze. I'm all for replacing distutils with distutils2, but
>>> I'm not sure whether you will declare distutils2 ready tomorrow, next
>>> year, or ten years from now.
>>
>> Depends on what "ready" means.
>
> Included in Python, so that changes become possible again.
>
>> If by ready you mean it can be used to replace Distutils1 in a
>> project, I declare Distutils2 ready for usage NOW. ?It's in alpha
>> stage. I want a solid beta before Pycon.
>>
>> I would even remove Distutils from 3.x altogether at some point since
>> setuptools is not Python 3 compatible, and just put distutils2.
>>
>> 3.3 sounds like a good target.
>
> So will distuils2 be released before that? If so, when?

An alpha is already released. A beta will be released for Pycon (I
need it for my talk :) )   Then hopefully the final before 3.2


> Regards,
> Martin
>



-- 
Tarek Ziad? | http://ziade.org

From martin at v.loewis.de  Fri Dec  3 00:27:16 2010
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Fri, 03 Dec 2010 00:27:16 +0100
Subject: [Python-Dev] Python and the Unicode Character Database
In-Reply-To: <4CF8247F.4000603@egenix.com>
References: <AANLkTinVpySuRFkjf-Sc+7a91zrxhuQpqPQL5nM9u1w+@mail.gmail.com>	<AANLkTinQLK7+5NfEp-2nMiJeDEKRxKN9Zfs4ihv-NMyH@mail.gmail.com>	<87wrnv43v5.fsf@uwakimon.sk.tsukuba.ac.jp>	<AANLkTinAf6CstywehOGRd4G4ms6K5LtSuq4Lj0z5wFBW@mail.gmail.com>	<87r5e236hj.fsf@benfinney.id.au>	<4CF58387.1000609@v.loewis.de>	<87k4ju2vvg.fsf@benfinney.id.au>	<4CF6CDDD.4060109@v.loewis.de>	<4CF6D7B2.6030506@pearwood.info>	<87sjyhxarg.fsf@uwakimon.sk.tsukuba.ac.jp>	<87bp552arq.fsf@benfinney.id.au>	<4CF7F579.5050603@v.loewis.de>	<4CF80C3A.9000705@egenix.com>	<AANLkTi=QApgGxCH0ph-_QjMMu5scH97vSdsBw_Da5Fvv@mail.gmail.com>
	<4CF8247F.4000603@egenix.com>
Message-ID: <4CF82B54.8090000@v.loewis.de>

> The point is that we support all of Unicode in Python, not just a fragment,
> and therefore the numeric constructors support all of Unicode.

That conclusion is as false today as it was in Python 1.6, but only now
people start caring about that.

a) we don't support all of Unicode in numeric constructors. There are
   lots of things that you can write down that readers would recognize
   as a real/rational/integral number that float() won't parse.
b) if float() would restrict itself to the scientific notation of
   real numbers (as it should), Python could well continue to claim all
   of Unicode.

> Adding more locale aware numeric parsers and formatters to the
> locale module, based on these APIs is certainly a good idea,
> but orthogonal to the ongoing discussion, IMO.

Not at all. The concept of "Unicode numbers" is flawed: Unicode does
*not* prescribe any specific way to denote numbers. Unicode is about
characters, and Python supports the Unicode characters for digits as
well as it supports all the other Unicode characters.

Instead, support for non-scientific notation of real numbers should
be based on user needs, which probably can be approximated by looking
at actual scripts. This, in turn, is inherently locale-dependent.

Regards,
Martin


From mal at egenix.com  Fri Dec  3 00:37:13 2010
From: mal at egenix.com (M.-A. Lemburg)
Date: Fri, 03 Dec 2010 00:37:13 +0100
Subject: [Python-Dev] Python and the Unicode Character Database
In-Reply-To: <4CF8217F.2010005@trueblade.com>
References: <AANLkTinVpySuRFkjf-Sc+7a91zrxhuQpqPQL5nM9u1w+@mail.gmail.com>	<20101128214311.092abd35@pitrou.net>	<4CF2D4E9.3060607@v.loewis.de>	<4CF2DAD7.2000408@egenix.com>	<4CF2DFD1.10901@v.loewis.de>	<4CF7FC01.80307@egenix.com>	<4CF8004D.1080408@v.loewis.de>	<4CF80FFE.8020505@pearwood.info>	<4CF81431.2040900@v.loewis.de>	<4CF81659.4030302@trueblade.com>
	<4CF82100.5080308@egenix.com> <4CF8217F.2010005@trueblade.com>
Message-ID: <4CF82DA9.4070100@egenix.com>

Eric Smith wrote:
> On 12/2/2010 5:43 PM, M.-A. Lemburg wrote:
>> Eric Smith wrote:
>>>> The current behavior should go nowhere; it is not useful. Something
>>>> very
>>>> similar to the current behavior (but done correctly) should go into the
>>>> locale module.
>>>
>>> I agree with everything Martin says here. I think the basic premise is:
>>> you won't find strings "in the wild" that use non-ASCII digits but do
>>> use the ASCII dot as a decimal point. And that's what float() is looking
>>> for. (And that doesn't even begin to address what it expects for an
>>> exponent 'e'.)
>>
>> http://en.wikipedia.org/wiki/Decimal_mark
>>
>> "In China, comma and space are used to mark digit groups because dot
>> is used as decimal mark."
> 
> Is that an ASCII dot? That page doesn't say.

Yes, but to be fair: I think that the page actually refers to the
use of the Arabic numeral format in China, rather than with their
own script symbols.

>> Note that float() can also parse integers, it just returns them as
>> floats :-)
> 
> :)

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Dec 02 2010)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try our new mxODBC.Connect Python Database Interface for free ! ::::


   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
           Registered at Amtsgericht Duesseldorf: HRB 46611
               http://www.egenix.com/company/contact/

From martin at v.loewis.de  Fri Dec  3 00:38:36 2010
From: martin at v.loewis.de (=?ISO-8859-15?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Fri, 03 Dec 2010 00:38:36 +0100
Subject: [Python-Dev] PEP 384 accepted
In-Reply-To: <20101202175401.1a94a5f4@mission>
References: <AANLkTimQsiU+nyX6CdqQ8JBybJdnx=1EMxX9mefHbx9H@mail.gmail.com>	<4CF8008B.60402@v.loewis.de>	<AANLkTikbRJEFYE_TvN++GYhmvSUu3hdhf4znOti7oty1@mail.gmail.com>	<4CF80964.4070300@v.loewis.de>	<AANLkTikcT0WGo3FhFVq=N6ktToNenBDNNcC4eSpefcFF@mail.gmail.com>	<4CF80D6C.6050703@v.loewis.de>	<AANLkTi=s5NMxjPdRQRcupNBzdqjMhLO6TbFWtQzi0TEw@mail.gmail.com>	<4CF81208.8090705@v.loewis.de>
	<4CF815AE.1040404@voidspace.org.uk>	<4CF81BE5.1060003@v.loewis.de>
	<20101202175401.1a94a5f4@mission>
Message-ID: <4CF82DFC.8090301@v.loewis.de>

> From my point of view, the PEP 3149 text is just a proposal.  It leaves the
> final decision to PEP 384, but tries to address some of the issues raised
> during the PEP 3149 discussion.  I think it is within PEP 384's scope to make
> the final decisions about it.

Ok, then it looks like there just won't be any support for module
tagging of ABI-conforming modules. It might be possible to support
something like this in the import code, but I would consider this
pointless without accompanying distutils support.

Then, by default, the modules just use the ABI tag that distutils
assigns to them by default. It's interesting to note that #9807
got into distutils despite it being frozen (but this is not about
ABI tags, right - so does distutils in 3.2 actually assign any
ABI tag at all?)

> I would favor changing distutils, if it can be done in a way that reasonably
> preserves backward compatibility.

It seems this is right out for policy reasons.

Regards,
Martin

From martin at v.loewis.de  Fri Dec  3 00:42:10 2010
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Fri, 03 Dec 2010 00:42:10 +0100
Subject: [Python-Dev] PEP 384 accepted
In-Reply-To: <AANLkTinOwVj3iLbYT4kkoh0dPmW34WQx1HsZJPfnepT4@mail.gmail.com>
References: <AANLkTimQsiU+nyX6CdqQ8JBybJdnx=1EMxX9mefHbx9H@mail.gmail.com>	<4CF8008B.60402@v.loewis.de>	<AANLkTikbRJEFYE_TvN++GYhmvSUu3hdhf4znOti7oty1@mail.gmail.com>	<4CF80964.4070300@v.loewis.de>	<AANLkTikcT0WGo3FhFVq=N6ktToNenBDNNcC4eSpefcFF@mail.gmail.com>	<4CF80D6C.6050703@v.loewis.de>	<AANLkTi=s5NMxjPdRQRcupNBzdqjMhLO6TbFWtQzi0TEw@mail.gmail.com>	<4CF81208.8090705@v.loewis.de>	<AANLkTinspUo88oUnd3cX776mMRMU6bqyOaq9Og7JLwhU@mail.gmail.com>	<4CF81AE1.5050900@v.loewis.de>	<AANLkTi=FDOAgf9DCQ6N5EpMCJs6DCvsC3bQR6bbL=n+_@mail.gmail.com>	<4CF823C0.7070002@v.loewis.de>
	<AANLkTinOwVj3iLbYT4kkoh0dPmW34WQx1HsZJPfnepT4@mail.gmail.com>
Message-ID: <4CF82ED2.5020304@v.loewis.de>

> An alpha is already released. A beta will be released for Pycon (I
> need it for my talk :) )   Then hopefully the final before 3.2

Ok, that's promising.

Regards,
Martin

From ziade.tarek at gmail.com  Fri Dec  3 00:50:19 2010
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Fri, 3 Dec 2010 00:50:19 +0100
Subject: [Python-Dev] PEP 384 accepted
In-Reply-To: <4CF82560.7020001@v.loewis.de>
References: <AANLkTimQsiU+nyX6CdqQ8JBybJdnx=1EMxX9mefHbx9H@mail.gmail.com>
	<4CF8008B.60402@v.loewis.de>
	<AANLkTikbRJEFYE_TvN++GYhmvSUu3hdhf4znOti7oty1@mail.gmail.com>
	<4CF80964.4070300@v.loewis.de>
	<AANLkTikcT0WGo3FhFVq=N6ktToNenBDNNcC4eSpefcFF@mail.gmail.com>
	<4CF80D6C.6050703@v.loewis.de>
	<AANLkTi=s5NMxjPdRQRcupNBzdqjMhLO6TbFWtQzi0TEw@mail.gmail.com>
	<4CF81208.8090705@v.loewis.de> <4CF815AE.1040404@voidspace.org.uk>
	<4CF81BE5.1060003@v.loewis.de> <20101202235114.1dc77cdc@pitrou.net>
	<4CF82560.7020001@v.loewis.de>
Message-ID: <AANLkTinR9A2B+iuWsJkroD0A98=Nq+NSiAbtMtXCSr1y@mail.gmail.com>

On Fri, Dec 3, 2010 at 12:01 AM, "Martin v. L?wis" <martin at v.loewis.de> wrote:
>> I think distutils is simply a bugfix branch for distutils2. Similarly
>> as how we don't commit improvements in e.g. 2.7 or 3.1, neither do we
>> commit improvements to distutils.
>
> It's different, though, in the sense that Python has a release schedule
> and multiple committers working on it, and that it normally gets
> released even if some changes don't get included in a specific release
> yet.
>
> All this seems not to be true for distutils2.

We have 3 or 4 regular contributors. That's not a lot for sure.

> So my motivation to
> contribute changes to it is *much* lower than my desire to contribute
> to distutils, and it is also provably lower than my motivation to
> contribute to distribute (say). I'm just getting tired having to talk to
> five projects just to make a single change to the build infrastructure
> available to the Python community.

I am not trying to motivate you to contribute to Distutils2. I am
trying to make sure we are all on the same page for what's good for
Python.

So if we work in Distutils2 and you work in Distutils saying publicly
that you don't want to contribute to Distutils2, that's a total
nonsense.

We took some decisions, and you want to go against them.  So I want to
have a consensus here for the packaging eco-system and make sure we
are still on track.

I am sorry if you get tired of it, but I don't want to be told at the
next summit: sorry Tarek, now we need to do changes little by little
in distutils1

From amauryfa at gmail.com  Fri Dec  3 00:51:37 2010
From: amauryfa at gmail.com (Amaury Forgeot d'Arc)
Date: Fri, 3 Dec 2010 00:51:37 +0100
Subject: [Python-Dev] PEP 384 accepted
In-Reply-To: <4CF82A82.2020804@voidspace.org.uk>
References: <AANLkTimQsiU+nyX6CdqQ8JBybJdnx=1EMxX9mefHbx9H@mail.gmail.com>
	<4CF8008B.60402@v.loewis.de>
	<AANLkTikbRJEFYE_TvN++GYhmvSUu3hdhf4znOti7oty1@mail.gmail.com>
	<4CF80964.4070300@v.loewis.de>
	<AANLkTikcT0WGo3FhFVq=N6ktToNenBDNNcC4eSpefcFF@mail.gmail.com>
	<4CF80D6C.6050703@v.loewis.de>
	<AANLkTi=s5NMxjPdRQRcupNBzdqjMhLO6TbFWtQzi0TEw@mail.gmail.com>
	<4CF81208.8090705@v.loewis.de> <4CF815AE.1040404@voidspace.org.uk>
	<4CF81BE5.1060003@v.loewis.de> <20101202235114.1dc77cdc@pitrou.net>
	<4CF82560.7020001@v.loewis.de> <4CF82A82.2020804@voidspace.org.uk>
Message-ID: <AANLkTimve7cKamsZox1ryNDHMZU6cPoSkTbwB291TCR=@mail.gmail.com>

Hi,

2010/12/3 Michael Foord <fuzzyman at voidspace.org.uk>

> On 02/12/2010 23:01, "Martin v. L?wis" wrote:
>
>> [snip...]
>>>
>> I'm just getting tired having to talk to
>> five projects just to make a single change to the build infrastructure
>> available to the Python community.
>>
>>
> The very best hope of resolving that particular problem is distutils2. :-)
>
> distutils2 is *already* available to the Python community, and whether or
> not there is a fixed release date it will have betas and then a 1.0 release
> in the foreseeable future. The team working on it has made an enormous
> amount of progress. We're much better off as a development community putting
> our support and energy into distutils2 rather than pining for evolution of
> distutils.
>

Sure. But today (before 3.2b1) we want to merge PEP3149 and PEP384;
they change the paths and filenames used by python.
Either we modify distutils to comply with the new names,
or defer these PEPs until distutils2 is ready.

-- 
Amaury Forgeot d'Arc
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20101203/1497db6d/attachment.html>

From alexander.belopolsky at gmail.com  Fri Dec  3 00:54:10 2010
From: alexander.belopolsky at gmail.com (Alexander Belopolsky)
Date: Thu, 2 Dec 2010 18:54:10 -0500
Subject: [Python-Dev] Python and the Unicode Character Database
In-Reply-To: <4CF80C3A.9000705@egenix.com>
References: <AANLkTinVpySuRFkjf-Sc+7a91zrxhuQpqPQL5nM9u1w+@mail.gmail.com>
	<AANLkTinQLK7+5NfEp-2nMiJeDEKRxKN9Zfs4ihv-NMyH@mail.gmail.com>
	<87wrnv43v5.fsf@uwakimon.sk.tsukuba.ac.jp>
	<AANLkTinAf6CstywehOGRd4G4ms6K5LtSuq4Lj0z5wFBW@mail.gmail.com>
	<87r5e236hj.fsf@benfinney.id.au> <4CF58387.1000609@v.loewis.de>
	<87k4ju2vvg.fsf@benfinney.id.au> <4CF6CDDD.4060109@v.loewis.de>
	<4CF6D7B2.6030506@pearwood.info>
	<87sjyhxarg.fsf@uwakimon.sk.tsukuba.ac.jp>
	<87bp552arq.fsf@benfinney.id.au> <4CF7F579.5050603@v.loewis.de>
	<4CF80C3A.9000705@egenix.com>
Message-ID: <AANLkTinKP6=u4X2W2Qhwoyrm2hoKTkBeqcKgR+g93eS0@mail.gmail.com>

On Thu, Dec 2, 2010 at 4:14 PM, M.-A. Lemburg <mal at egenix.com> wrote:
..
> Some examples:
>
> http://www.bdl.gov.lb/circ/intpdf/int123.pdf

I looked at this one more closely.  While I cannot understand what it
says, It appears that Arabic numerals are used in dates.   It looks
like Python want be able to deal with those:

>>> datetime.strptime('????/??/??', '%Y/%m/%d')
..
ValueError: time data '????/??/??' does not match format '%Y/%m/%d'

Interestingly,

>>> datetime.strptime('????', '%Y')
datetime.datetime(1999, 1, 1, 0, 0)

which further suggests that support of such numerals is accidental.

As I think more about it, though I am becoming less avert to accepting
these numerals for base 10 integers.  Integers can be easily extracted
from text using simple regex and '\d' accepts all category Nd
characters.  I would require though that all digits be from the same
block, which is not hard because Unicode now promises to only have
them in contiguous blocks of 10.   This rule seems to address some of
security issues because it is unlikely that a system that can display
some of the local digits would not be able to display all of them
properly.

I still don't think it makes any sense to accept them in float().

From steve at pearwood.info  Fri Dec  3 01:12:30 2010
From: steve at pearwood.info (Steven D'Aprano)
Date: Fri, 03 Dec 2010 11:12:30 +1100
Subject: [Python-Dev] Python and the Unicode Character Database
In-Reply-To: <87sjyhxarg.fsf@uwakimon.sk.tsukuba.ac.jp>
References: <AANLkTinVpySuRFkjf-Sc+7a91zrxhuQpqPQL5nM9u1w+@mail.gmail.com>	<AANLkTinQLK7+5NfEp-2nMiJeDEKRxKN9Zfs4ihv-NMyH@mail.gmail.com>	<87wrnv43v5.fsf@uwakimon.sk.tsukuba.ac.jp>	<AANLkTinAf6CstywehOGRd4G4ms6K5LtSuq4Lj0z5wFBW@mail.gmail.com>	<87r5e236hj.fsf@benfinney.id.au>	<4CF58387.1000609@v.loewis.de>	<87k4ju2vvg.fsf@benfinney.id.au>	<4CF6CDDD.4060109@v.loewis.de>	<4CF6D7B2.6030506@pearwood.info>
	<87sjyhxarg.fsf@uwakimon.sk.tsukuba.ac.jp>
Message-ID: <4CF835EE.1060604@pearwood.info>

Stephen J. Turnbull wrote:
> Steven D'Aprano writes:
> 
>  > With full respect to haiyang kang, hear-say from one person can hardly 
>  > be described as "strong" evidence
> 
> That's *disrespectful* nonsense.  What Haiyang reported was not
> hearsay, it's direct observation of what he sees around him and
> personal experience, plus extrapolation.  Look up "hearsay," please.

Fair enough. I choose my words poorly and apologise. A better 
description would be anecdotal evidence.


-- 
Steven

From fuzzyman at voidspace.org.uk  Fri Dec  3 01:35:51 2010
From: fuzzyman at voidspace.org.uk (Michael Foord)
Date: Fri, 03 Dec 2010 00:35:51 +0000
Subject: [Python-Dev] PEP 384 accepted
In-Reply-To: <AANLkTimve7cKamsZox1ryNDHMZU6cPoSkTbwB291TCR=@mail.gmail.com>
References: <AANLkTimQsiU+nyX6CdqQ8JBybJdnx=1EMxX9mefHbx9H@mail.gmail.com>	<4CF8008B.60402@v.loewis.de>	<AANLkTikbRJEFYE_TvN++GYhmvSUu3hdhf4znOti7oty1@mail.gmail.com>	<4CF80964.4070300@v.loewis.de>	<AANLkTikcT0WGo3FhFVq=N6ktToNenBDNNcC4eSpefcFF@mail.gmail.com>	<4CF80D6C.6050703@v.loewis.de>	<AANLkTi=s5NMxjPdRQRcupNBzdqjMhLO6TbFWtQzi0TEw@mail.gmail.com>	<4CF81208.8090705@v.loewis.de>
	<4CF815AE.1040404@voidspace.org.uk>	<4CF81BE5.1060003@v.loewis.de>
	<20101202235114.1dc77cdc@pitrou.net>	<4CF82560.7020001@v.loewis.de>
	<4CF82A82.2020804@voidspace.org.uk>
	<AANLkTimve7cKamsZox1ryNDHMZU6cPoSkTbwB291TCR=@mail.gmail.com>
Message-ID: <4CF83B67.10809@voidspace.org.uk>

On 02/12/2010 23:51, Amaury Forgeot d'Arc wrote:
> Hi,
>
> 2010/12/3 Michael Foord <fuzzyman at voidspace.org.uk 
> <mailto:fuzzyman at voidspace.org.uk>>
>
>     On 02/12/2010 23:01, "Martin v. L?wis" wrote:
>
>             [snip...]
>
>         I'm just getting tired having to talk to
>         five projects just to make a single change to the build
>         infrastructure
>         available to the Python community.
>
>
>     The very best hope of resolving that particular problem is
>     distutils2. :-)
>
>     distutils2 is *already* available to the Python community, and
>     whether or not there is a fixed release date it will have betas
>     and then a 1.0 release in the foreseeable future. The team working
>     on it has made an enormous amount of progress. We're much better
>     off as a development community putting our support and energy into
>     distutils2 rather than pining for evolution of distutils.
>
>
> Sure. But today (before 3.2b1) we want to merge PEP3149 and PEP384;
> they change the paths and filenames used by python.
> Either we modify distutils to comply with the new names,
> or defer these PEPs until distutils2 is ready.

Or put support for them into distutils2 now?

Michael

>
> -- 
> Amaury Forgeot d'Arc
>
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk


-- 

http://www.voidspace.org.uk/

READ CAREFULLY. By accepting and reading this email you agree,
on behalf of your employer, to release me from all obligations
and waivers arising from any and all NON-NEGOTIATED agreements,
licenses, terms-of-service, shrinkwrap, clickwrap, browsewrap,
confidentiality, non-disclosure, non-compete and acceptable use
policies ("BOGUS AGREEMENTS") that I have entered into with your
employer, its partners, licensors, agents and assigns, in
perpetuity, without prejudice to my ongoing rights and privileges.
You further represent that you have the authority to release me
from any BOGUS AGREEMENTS on behalf of your employer.

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20101203/64a0b903/attachment.html>

From barry at python.org  Fri Dec  3 02:07:34 2010
From: barry at python.org (Barry Warsaw)
Date: Thu, 2 Dec 2010 20:07:34 -0500
Subject: [Python-Dev] PEP 384 accepted
In-Reply-To: <AANLkTimve7cKamsZox1ryNDHMZU6cPoSkTbwB291TCR=@mail.gmail.com>
References: <AANLkTimQsiU+nyX6CdqQ8JBybJdnx=1EMxX9mefHbx9H@mail.gmail.com>
	<4CF8008B.60402@v.loewis.de>
	<AANLkTikbRJEFYE_TvN++GYhmvSUu3hdhf4znOti7oty1@mail.gmail.com>
	<4CF80964.4070300@v.loewis.de>
	<AANLkTikcT0WGo3FhFVq=N6ktToNenBDNNcC4eSpefcFF@mail.gmail.com>
	<4CF80D6C.6050703@v.loewis.de>
	<AANLkTi=s5NMxjPdRQRcupNBzdqjMhLO6TbFWtQzi0TEw@mail.gmail.com>
	<4CF81208.8090705@v.loewis.de> <4CF815AE.1040404@voidspace.org.uk>
	<4CF81BE5.1060003@v.loewis.de> <20101202235114.1dc77cdc@pitrou.net>
	<4CF82560.7020001@v.loewis.de> <4CF82A82.2020804@voidspace.org.uk>
	<AANLkTimve7cKamsZox1ryNDHMZU6cPoSkTbwB291TCR=@mail.gmail.com>
Message-ID: <20101202200734.60649f05@mission>

On Dec 03, 2010, at 12:51 AM, Amaury Forgeot d'Arc wrote:

>Sure. But today (before 3.2b1) we want to merge PEP3149 and PEP384;
>they change the paths and filenames used by python.
>Either we modify distutils to comply with the new names,
>or defer these PEPs until distutils2 is ready.

I do not think it would be a good idea to revert PEP 3149.

-Barry
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 836 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/python-dev/attachments/20101202/44b168f7/attachment.pgp>

From doko at ubuntu.com  Fri Dec  3 02:48:25 2010
From: doko at ubuntu.com (Matthias Klose)
Date: Fri, 03 Dec 2010 02:48:25 +0100
Subject: [Python-Dev] PEP 384 accepted
In-Reply-To: <AANLkTinOwVj3iLbYT4kkoh0dPmW34WQx1HsZJPfnepT4@mail.gmail.com>
References: <AANLkTimQsiU+nyX6CdqQ8JBybJdnx=1EMxX9mefHbx9H@mail.gmail.com>	<4CF8008B.60402@v.loewis.de>	<AANLkTikbRJEFYE_TvN++GYhmvSUu3hdhf4znOti7oty1@mail.gmail.com>	<4CF80964.4070300@v.loewis.de>	<AANLkTikcT0WGo3FhFVq=N6ktToNenBDNNcC4eSpefcFF@mail.gmail.com>	<4CF80D6C.6050703@v.loewis.de>	<AANLkTi=s5NMxjPdRQRcupNBzdqjMhLO6TbFWtQzi0TEw@mail.gmail.com>	<4CF81208.8090705@v.loewis.de>	<AANLkTinspUo88oUnd3cX776mMRMU6bqyOaq9Og7JLwhU@mail.gmail.com>	<4CF81AE1.5050900@v.loewis.de>	<AANLkTi=FDOAgf9DCQ6N5EpMCJs6DCvsC3bQR6bbL=n+_@mail.gmail.com>	<4CF823C0.7070002@v.loewis.de>
	<AANLkTinOwVj3iLbYT4kkoh0dPmW34WQx1HsZJPfnepT4@mail.gmail.com>
Message-ID: <4CF84C69.9020105@ubuntu.com>

On 03.12.2010 00:25, Tarek Ziad? wrote:
> 2010/12/2 "Martin v. L?wis"<martin at v.loewis.de>:
>>>> No, only the ones that didn't cause backwards incompatibilities,
>>>> and broke existing packages.
>>>
>>> This is impossible. I can point you to some third party project that
>>> can break if you touch some distutils internals, like setuptools.
>>> Setuptools also uses some privates global variables in some other
>>> modules in the stdlib FYI.
>>
>> So what would break if Extension accepted an abi= keyword parameter?
>
> I suppose you have code behind this, that will be in build_ext and in
> the compilers. So you will need to try out ALL projects out there that
> customize build_ext, like numpy or setuptools, etc, But you won't be
> able to try out all projects because they are not listed somewhere.

is this necessary?  are all these projects known to work with 3.2, without 
having changes compared to 3.1 *without* this pep?  hardly ...

how many extensions will use this restricted api at all?  Is it a legitimate 
solution to back up building an extension in the "default" mode?

even without having any changes in distutils it would make sense to know if an 
extension can be built with the restricted ABI, so maybe it is better to defer 
any changes to the extension soname, and provide a check for an extension if it 
conforms to the restricted ABI, even if the extension still uses the python 
version specific soname.

I did not mean to block this pep by choosing any installation names.

   Matthias

From tjreedy at udel.edu  Fri Dec  3 02:52:23 2010
From: tjreedy at udel.edu (Terry Reedy)
Date: Thu, 02 Dec 2010 20:52:23 -0500
Subject: [Python-Dev] Python and the Unicode Character Database
In-Reply-To: <AANLkTinKP6=u4X2W2Qhwoyrm2hoKTkBeqcKgR+g93eS0@mail.gmail.com>
References: <AANLkTinVpySuRFkjf-Sc+7a91zrxhuQpqPQL5nM9u1w+@mail.gmail.com>	<AANLkTinQLK7+5NfEp-2nMiJeDEKRxKN9Zfs4ihv-NMyH@mail.gmail.com>	<87wrnv43v5.fsf@uwakimon.sk.tsukuba.ac.jp>	<AANLkTinAf6CstywehOGRd4G4ms6K5LtSuq4Lj0z5wFBW@mail.gmail.com>	<87r5e236hj.fsf@benfinney.id.au>
	<4CF58387.1000609@v.loewis.de>	<87k4ju2vvg.fsf@benfinney.id.au>
	<4CF6CDDD.4060109@v.loewis.de>	<4CF6D7B2.6030506@pearwood.info>	<87sjyhxarg.fsf@uwakimon.sk.tsukuba.ac.jp>	<87bp552arq.fsf@benfinney.id.au>
	<4CF7F579.5050603@v.loewis.de>	<4CF80C3A.9000705@egenix.com>
	<AANLkTinKP6=u4X2W2Qhwoyrm2hoKTkBeqcKgR+g93eS0@mail.gmail.com>
Message-ID: <id9igp$8nn$1@dough.gmane.org>

On 12/2/2010 6:54 PM, Alexander Belopolsky wrote:
> On Thu, Dec 2, 2010 at 4:14 PM, M.-A. Lemburg<mal at egenix.com>  wrote:
> ..
>> Some examples:
>>
>> http://www.bdl.gov.lb/circ/intpdf/int123.pdf
>
> I looked at this one more closely.  While I cannot understand what it
> says, It appears that Arabic numerals are used in dates.   It looks
> like Python want be able to deal with those:

When I travelled in S. Asia around 25 years ago, arabic and indic 
numerals were in obvious use in stores, road signs, and banks (as with 
money exchange receipts). I learned the digits partly for 
self-protestions ;-). I have no real idea of what is done *now* in 
computerized business, but I assume the native digits are used.

It may well be that there is no Python software yet that operates with 
native digits. The lack of direct output capability would hinder that. 
Of course, someone could run both input and output through 
language-specific str.translate digit translators.

>>>> datetime.strptime('????/??/??', '%Y/%m/%d')

Googling ???? gets about 83,000 hits.
> ..
> ValueError: time data '????/??/??' does not match format '%Y/%m/%d'
>
> Interestingly,
>
>>>> datetime.strptime('????', '%Y')
> datetime.datetime(1999, 1, 1, 0, 0)
>
> which further suggests that support of such numerals is accidental.
>
> As I think more about it, though I am becoming less avert to accepting
> these numerals for base 10 integers.

Both input and output are needed for educational programming, though 
translation tables might be enough.

 >  Integers can be easily extracted
> from text using simple regex and '\d' accepts all category Nd
> characters.  I would require though that all digits be from the same
> block, which is not hard because Unicode now promises to only have
> them in contiguous blocks of 10.

That seems sensible.

 > This rule seems to address some of
> security issues because it is unlikely that a system that can display
> some of the local digits would not be able to display all of them
> properly.
>
> I still don't think it makes any sense to accept them in float().

For the present, I would pretty well agree with that, at least until we 
know more.

You have raised an important issue. It is a bit of a chicken and egg 
problem though. We will not really know what is needed until Python is 
used more in non-english/non-euro contexts, while such usage may await 
better support.

-- 
Terry Jan Reedy



From merwok at netwok.org  Fri Dec  3 03:14:21 2010
From: merwok at netwok.org (=?UTF-8?B?w4lyaWMgQXJhdWpv?=)
Date: Fri, 03 Dec 2010 03:14:21 +0100
Subject: [Python-Dev] PEP 384 accepted
In-Reply-To: <4CF84C69.9020105@ubuntu.com>
References: <AANLkTimQsiU+nyX6CdqQ8JBybJdnx=1EMxX9mefHbx9H@mail.gmail.com>	<4CF8008B.60402@v.loewis.de>	<AANLkTikbRJEFYE_TvN++GYhmvSUu3hdhf4znOti7oty1@mail.gmail.com>	<4CF80964.4070300@v.loewis.de>	<AANLkTikcT0WGo3FhFVq=N6ktToNenBDNNcC4eSpefcFF@mail.gmail.com>	<4CF80D6C.6050703@v.loewis.de>	<AANLkTi=s5NMxjPdRQRcupNBzdqjMhLO6TbFWtQzi0TEw@mail.gmail.com>	<4CF81208.8090705@v.loewis.de>	<AANLkTinspUo88oUnd3cX776mMRMU6bqyOaq9Og7JLwhU@mail.gmail.com>	<4CF81AE1.5050900@v.loewis.de>	<AANLkTi=FDOAgf9DCQ6N5EpMCJs6DCvsC3bQR6bbL=n+_@mail.gmail.com>	<4CF823C0.7070002@v.loewis.de>	<AANLkTinOwVj3iLbYT4kkoh0dPmW34WQx1HsZJPfnepT4@mail.gmail.com>
	<4CF84C69.9020105@ubuntu.com>
Message-ID: <4CF8527D.9030503@netwok.org>

> even without having any changes in distutils it would make sense to know if an 
> extension can be built with the restricted ABI, so maybe it is better to defer 
> any changes to the extension soname, and provide a check for an extension if it 
> conforms to the restricted ABI, even if the extension still uses the python 
> version specific soname.

Python?s setup.py has an example in Martin?s branch:

  ext = Extension('xxlimited', ['xxlimited.c'],
                  define_macros=[('Py_LIMITED_API', 1)])

<http://codereview.appspot.com/3262043/patch/1/68>

This is possible with today?s distutils.  I don?t know if it?s enough to
build stable-ABI-conformant extension modules.

Regards


From stephen at xemacs.org  Fri Dec  3 03:20:19 2010
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Fri, 03 Dec 2010 11:20:19 +0900
Subject: [Python-Dev] Porting Ideas
In-Reply-To: <4CF7F72A.6090406@v.loewis.de>
References: <AANLkTing_iT7Y6FOZPWh=CUwtK8owvYtSdE7aXf0W181@mail.gmail.com>
	<AANLkTi=6rvXDEfzmMC=vE5jaCaTzAwnDddxHrZ2GAB1b@mail.gmail.com>
	<20101201201716.4357a316@pitrou.net>
	<AANLkTi=u-YAuEkNcBmkwUzCScoeedFSYno_RC2YRMvEy@mail.gmail.com>
	<id8mp3$76u$1@dough.gmane.org> <20101202140650.11d96108@mission>
	<4CF7F72A.6090406@v.loewis.de>
Message-ID: <87lj47y4v0.fsf@uwakimon.sk.tsukuba.ac.jp>

"Martin v. L?wis" writes:
 > > Aside: how does one log into the Cheeseshop with your Launchpad OpenID?  When
 > > I try to do it I end up on a "Manual user registration" page.  I fill out the
 > > username with what I think my PyPI user name is, and add my python.org email
 > > address, but then it tells me 'barry' is already taken.  Do I need some kind
 > > of back door linking of my lp openid and my pypi user id?
 > 
 > Since the "barry" account already exists, you first need to log into
 > that (likely using a password). You can then claim the LP OpenID as
 > being associated with that account, and use LP in the future.

It would be nice if the UI told users that, and offered an opportunity
to log in.

Better yet would be a option for an OpenID to claim a user name by
giving the password for it (ie, automatically on a successful login
from that page).



From stephen at xemacs.org  Fri Dec  3 03:27:45 2010
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Fri, 03 Dec 2010 11:27:45 +0900
Subject: [Python-Dev] Python and the Unicode Character Database
In-Reply-To: <AANLkTik68+CgkVimhL2saVPYem6Fvt5emM_eYwJr7gqz@mail.gmail.com>
References: <AANLkTinVpySuRFkjf-Sc+7a91zrxhuQpqPQL5nM9u1w+@mail.gmail.com>
	<AANLkTinQLK7+5NfEp-2nMiJeDEKRxKN9Zfs4ihv-NMyH@mail.gmail.com>
	<87wrnv43v5.fsf@uwakimon.sk.tsukuba.ac.jp>
	<AANLkTimnar7CuHKXtvFtspTphmB3GrNMiA59zfFhTeoq@mail.gmail.com>
	<87tyixxbet.fsf@uwakimon.sk.tsukuba.ac.jp>
	<AANLkTik68+CgkVimhL2saVPYem6Fvt5emM_eYwJr7gqz@mail.gmail.com>
Message-ID: <87k4jry4im.fsf@uwakimon.sk.tsukuba.ac.jp>

Lennart Regebro writes:
 > 2010/12/2 Stephen J. Turnbull <stephen at xemacs.org>:

 > > T1000 = float('?.???')
 > 
 > That was already discussed here, and it's clear that unicode does not
 > consider these characters to be something you can use in a decimal
 > number, and hence it's not broken.

Huh?  IOW, use Unicode features just because they're there, what the
users want and use doesn't matter?

The only evidence I've seen so far that this feature is anything but a
a toy for a small faction of developers is Neil Hodgson's information
that OOo will generate these kinds of digits (note that it *will* do
Han! so the evidence is as good for users demanding Han numerals as
for any other kind, Unicode.org definitions notwithstanding), and that
DOS CP 864 contains the Indo/Arabic versions.

Of course, it's quite possible that those were toys for the developers
of those software packages too.



From merwok at netwok.org  Fri Dec  3 03:39:12 2010
From: merwok at netwok.org (=?UTF-8?B?w4lyaWMgQXJhdWpv?=)
Date: Fri, 03 Dec 2010 03:39:12 +0100
Subject: [Python-Dev] Porting Ideas
In-Reply-To: <AANLkTing_iT7Y6FOZPWh=CUwtK8owvYtSdE7aXf0W181@mail.gmail.com>
References: <AANLkTing_iT7Y6FOZPWh=CUwtK8owvYtSdE7aXf0W181@mail.gmail.com>
Message-ID: <4CF85850.2070305@netwok.org>

Hi Prashant,

Python 3 support in distutils2 is not entirely finished, it?s an
interesting and challenging task.

Another idea: convert the python.org internal scripts to use Python 3,
for example starting with patches for http://code.python.org/hg/peps/ .
This would not have any impact on the community, but it?s easy work
that?d help the Python developers to eat their own dogfood.

Regards


From cornsea at gmail.com  Fri Dec  3 04:18:43 2010
From: cornsea at gmail.com (haiyang kang)
Date: Fri, 3 Dec 2010 11:18:43 +0800
Subject: [Python-Dev] Python and the Unicode Character Database
In-Reply-To: <4CF80C3A.9000705@egenix.com>
References: <AANLkTinVpySuRFkjf-Sc+7a91zrxhuQpqPQL5nM9u1w+@mail.gmail.com>
	<AANLkTinQLK7+5NfEp-2nMiJeDEKRxKN9Zfs4ihv-NMyH@mail.gmail.com>
	<87wrnv43v5.fsf@uwakimon.sk.tsukuba.ac.jp>
	<AANLkTinAf6CstywehOGRd4G4ms6K5LtSuq4Lj0z5wFBW@mail.gmail.com>
	<87r5e236hj.fsf@benfinney.id.au> <4CF58387.1000609@v.loewis.de>
	<87k4ju2vvg.fsf@benfinney.id.au> <4CF6CDDD.4060109@v.loewis.de>
	<4CF6D7B2.6030506@pearwood.info>
	<87sjyhxarg.fsf@uwakimon.sk.tsukuba.ac.jp>
	<87bp552arq.fsf@benfinney.id.au> <4CF7F579.5050603@v.loewis.de>
	<4CF80C3A.9000705@egenix.com>
Message-ID: <AANLkTi=LTpj2TJBN8iGP9O+Q-2=K7VqUzbXdRGXXrY1X@mail.gmail.com>

> Furthermore, data can well originate from texts that were written
> hundreds or even thousands of years ago, so there is plenty of
> material available for processing.

humm...,  for this, i think we need a special tuned language
processing system to handle this, and one subsystem for one language :)...
(sometimes a single word is not enough, we also need context)

Take pi for example, in modern math, it is wrote as: 3.1415...;
 in old China, it is sometimes wrote as: ????? or
 ?????? or ??????;

And if these texts are extracted through scanner
 (OCR or other image processing tech),  in my POV,
it is the job of this image processing subsystem
 (or some other subsystem between the image processing and database)
to do the mapping between number and raw text data, example table in DB:
text      | raw data                    |raw image data
-----------|---------------------------------|-----------------------
3.1415 | ?????                | image...

br,
khy

From stephen at xemacs.org  Fri Dec  3 04:54:05 2010
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Fri, 03 Dec 2010 12:54:05 +0900
Subject: [Python-Dev] Python and the Unicode Character Database
In-Reply-To: <AANLkTikuFdtbqXF0tj0mRENBciHwra1iR6ee7fTr79oL@mail.gmail.com>
References: <AANLkTinVpySuRFkjf-Sc+7a91zrxhuQpqPQL5nM9u1w+@mail.gmail.com>
	<AANLkTinQLK7+5NfEp-2nMiJeDEKRxKN9Zfs4ihv-NMyH@mail.gmail.com>
	<87wrnv43v5.fsf@uwakimon.sk.tsukuba.ac.jp>
	<AANLkTinAf6CstywehOGRd4G4ms6K5LtSuq4Lj0z5wFBW@mail.gmail.com>
	<87r5e236hj.fsf@benfinney.id.au> <4CF58387.1000609@v.loewis.de>
	<87k4ju2vvg.fsf@benfinney.id.au> <4CF6CDDD.4060109@v.loewis.de>
	<4CF6D7B2.6030506@pearwood.info>
	<87sjyhxarg.fsf@uwakimon.sk.tsukuba.ac.jp>
	<87bp552arq.fsf@benfinney.id.au>
	<87oc94y5q3.fsf@uwakimon.sk.tsukuba.ac.jp>
	<AANLkTikuFdtbqXF0tj0mRENBciHwra1iR6ee7fTr79oL@mail.gmail.com>
Message-ID: <87hbevy0iq.fsf@uwakimon.sk.tsukuba.ac.jp>

Neil Hodgson writes:

 >    While I don't have Excel to test with, OpenOffice.org Calc will
 > display in Arabic or Han numerals using the NatNum format codes.

Display is different from input, but at least this is concrete
evidence.

Will it accept Arabic on input?  (Han might be too much to ask for
since Unicode considers Han digits to be "impure".)

 > > Ditto Arabic, I would imagine; ISO 8859/6 (aka Latin/Arabic) does
 > > not contain the Arabic digits that have been presented here
 > > earlier AFAICT.
 > 
 >    DOS code page 864 does use 0xB0-0xB9

OK, Microsoft thought it would be useful.

I'd still like to know whether people actually use them for input (or
output, for that matter -- anybody have a corpus of Arabic Form 10-Ks
to grep through?), but that's more concrete evidence than we've seen
before.  Thank you!


From stephen at xemacs.org  Fri Dec  3 05:58:55 2010
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Fri, 03 Dec 2010 13:58:55 +0900
Subject: [Python-Dev] Python and the Unicode Character Database
In-Reply-To: <1291316131.3576.43.camel@localhost.localdomain>
References: <AANLkTinVpySuRFkjf-Sc+7a91zrxhuQpqPQL5nM9u1w+@mail.gmail.com>
	<AANLkTinQLK7+5NfEp-2nMiJeDEKRxKN9Zfs4ihv-NMyH@mail.gmail.com>
	<87wrnv43v5.fsf@uwakimon.sk.tsukuba.ac.jp>
	<id3umc$ps8$1@dough.gmane.org> <4CF5841E.6020401@v.loewis.de>
	<4CF619C1.8030403@pearwood.info> <4CF6CCF9.6000100@v.loewis.de>
	<4CF6E5AF.7080308@pearwood.info>
	<AANLkTikuUki5LgisVHrSMGZtmWZxxy9A8TOwny7dVCMe@mail.gmail.com>
	<id72oo$cdp$2@dough.gmane.org>
	<AANLkTikd0zDjr2GQ3_4C7BLZZstePQAiHee9m2ZpgCEh@mail.gmail.com>
	<20101202143633.6eb17b8d@pitrou.net>
	<AANLkTikiziBN87-7gM3rweODnXjEuK3DfhEni1sTDYtJ@mail.gmail.com>
	<1291308960.3576.28.camel@localhost.localdomain>
	<AANLkTinBzCQ4gi42iNXkSqWAEkdPWZzs=+C5sr0f0vpB@mail.gmail.com>
	<1291316131.3576.43.camel@localhost.localdomain>
Message-ID: <87fwufxxio.fsf@uwakimon.sk.tsukuba.ac.jp>

Antoine Pitrou writes:

 > The legacy format argument looks like a red herring to me. When
 > converting from a format to another it is the programmer's job to
 > his/her job right.

Uhmmmmmm, the argument *for* this "feature" proposed by several people
is that Python's numeric constructors do it (right) so that the
programmer doesn't have to.

If Python *doesn't* do it right, why should Python do it at all?


From alexander.belopolsky at gmail.com  Fri Dec  3 06:10:29 2010
From: alexander.belopolsky at gmail.com (Alexander Belopolsky)
Date: Fri, 3 Dec 2010 00:10:29 -0500
Subject: [Python-Dev] Python and the Unicode Character Database
In-Reply-To: <AANLkTik6LKOWrwRTrQn+tFETU5eWfuinf+=T=_yeOA8u@mail.gmail.com>
References: <AANLkTinVpySuRFkjf-Sc+7a91zrxhuQpqPQL5nM9u1w+@mail.gmail.com>
	<20101128214311.092abd35@pitrou.net> <4CF2D4E9.3060607@v.loewis.de>
	<4CF2DAD7.2000408@egenix.com> <4CF2DFD1.10901@v.loewis.de>
	<4CF7FC01.80307@egenix.com> <4CF8004D.1080408@v.loewis.de>
	<AANLkTik6LKOWrwRTrQn+tFETU5eWfuinf+=T=_yeOA8u@mail.gmail.com>
Message-ID: <AANLkTinXv1QSGb=oRUJutwsrWjHBppYn8UcbEeczEGFn@mail.gmail.com>

On Thu, Dec 2, 2010 at 4:57 PM, Mark Dickinson <dickinsm at gmail.com> wrote:
..
> (the decimal spec requires that non-European digits be accepted).

Mark,

I think *requires* is too strong of a word to describe what the spec
says.   The decimal module documentation refers to two authorities:

1. IBM?s General Decimal Arithmetic Specification
2. IEEE standard 854-1987

The IEEE standards predates Unicode and unsurprisingly does not have
anything related to the issue.  the IBM's spec says the following in
the Conversions section:

"""
It is recommended that implementations also provide additional number
formatting routines (including some which are locale-dependent), and
if available should accept non-European decimal digits in strings.
""" http://speleotrove.com/decimal/daconvs.html

This cannot possibly be interpreted as normative text.  The emphasis
is clearly on "formatting routines" with "non-European decimal digits"
added as an afterthought.  This recommendation can reasonably be
interpreted as a requirement that conversion routines should accept
what formatting routines can produce.  In Python there are no
formatting routines to produce non-European numerals, so there is no
requirement to accept them in conversions.

I don't think decimal module should support non-European decimal
digits.  The only place where it can make some sense is in int()
because here we have a fighting chance of producing a reasonable
definition.   The motivating use case is conversion of numerical data
extracted from text using simple '\d+'  regex matches.

Here is how I would do it:

1.  String x of non-European decimal digits is only accepted in
int(x), but not by int(x, 0) or int(x, 10).
2.  If x contains one or more non-European digits, then

    (a)  all digits must be from the same block:

      def basepoint(c):
            return ord(c) - unicodedata.digit(c)
      all(basepoint(c) == basepoint(x[0]) for c in x) -> True

     (b) and '+' or '-' sign is not alowed.

3. A character c is a digit if it matches '\d' regex.  I think this
means unicodedata.category(c) -> 'Nd'.

Condition 2(b) is important because there is no clear way to define
what is acceptable as '+' or '-' using Unicode character properties
and not all number systems even support local form of negation.  (It
is also YAGNI.)

From merwok at netwok.org  Fri Dec  3 06:32:39 2010
From: merwok at netwok.org (=?UTF-8?B?w4lyaWMgQXJhdWpv?=)
Date: Fri, 03 Dec 2010 06:32:39 +0100
Subject: [Python-Dev] PEP 384 accepted
In-Reply-To: <4CF81AE1.5050900@v.loewis.de>
References: <AANLkTimQsiU+nyX6CdqQ8JBybJdnx=1EMxX9mefHbx9H@mail.gmail.com>	<4CF8008B.60402@v.loewis.de>	<AANLkTikbRJEFYE_TvN++GYhmvSUu3hdhf4znOti7oty1@mail.gmail.com>	<4CF80964.4070300@v.loewis.de>	<AANLkTikcT0WGo3FhFVq=N6ktToNenBDNNcC4eSpefcFF@mail.gmail.com>	<4CF80D6C.6050703@v.loewis.de>	<AANLkTi=s5NMxjPdRQRcupNBzdqjMhLO6TbFWtQzi0TEw@mail.gmail.com>	<4CF81208.8090705@v.loewis.de>	<AANLkTinspUo88oUnd3cX776mMRMU6bqyOaq9Og7JLwhU@mail.gmail.com>
	<4CF81AE1.5050900@v.loewis.de>
Message-ID: <4CF880F7.9000800@netwok.org>

Le 02/12/2010 23:17, Martin v. L?wis a ?crit :
> Before the freeze, distutils was unmaintained (i.e. before you started
> maintaining it), but people who want to improve it gradually atleast
> could. Now gradual improvements are also banned, so it's not only
> unmaintained, but I can't even provide support for the PEP in Python
> that was just accepted.

I wonder what your definition of ?unmaintained? is.  Tarek has been
fixing bugs for two years, and recently I have been made a committer to
assist him.  It?s true that I?ve not been as active as I would have
liked*, but I did fix some bugs, as I think you know, given that you?ve
helped me in some reports.

Sure, distutils is not as well-maintained as other modules, but a dozen
bugs have been fixed by five or six of us since the revert.  I do feel
responsible for all 116 remaining bugs, and intend to address all of them.

 * This is partly normal, since I had warned before I was accepted as a
   committer that my time would be scarce for a year, partly due to
   the fact that I also do bug triage, doc work and patch reviews, and
   partly due to some personal problems with focusing.


On the matter of freeze exceptions, there have been two:
- reading the makefile with surogateescape error handler so that python
can build with an ASCII locale in a non-ASCII path (haypo, #6011)
- handle soabiflags (barry, #9807).
I took part in the discussion before those changes and did not object to
them: they are very small changes that enable a new feature of Python
3.2.  Maybe I should have requested Tarek?s approval for those changes;
he knows better than me how third parties may break because of changes
that don?t seem to break anything.


Regards


From merwok at netwok.org  Fri Dec  3 07:34:21 2010
From: merwok at netwok.org (=?UTF-8?B?w4lyaWMgQXJhdWpv?=)
Date: Fri, 03 Dec 2010 07:34:21 +0100
Subject: [Python-Dev] Change to the Distutils / Distutils2 workflow
In-Reply-To: <AANLkTi=njm=wX8++HH+YGsZE5vDP6YGbwNf+0fyp+u3-@mail.gmail.com>
References: <AANLkTi=njm=wX8++HH+YGsZE5vDP6YGbwNf+0fyp+u3-@mail.gmail.com>
Message-ID: <4CF88F6D.7000804@netwok.org>

Hi everyone,

I have sketched a workflow guide on
http://wiki.python.org/moin/Distutils/FixingBugs

Cheers


From martin at v.loewis.de  Fri Dec  3 08:16:35 2010
From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=)
Date: Fri, 03 Dec 2010 08:16:35 +0100
Subject: [Python-Dev] PEP 384 accepted
In-Reply-To: <4CF8527D.9030503@netwok.org>
References: <AANLkTimQsiU+nyX6CdqQ8JBybJdnx=1EMxX9mefHbx9H@mail.gmail.com>	<4CF8008B.60402@v.loewis.de>	<AANLkTikbRJEFYE_TvN++GYhmvSUu3hdhf4znOti7oty1@mail.gmail.com>	<4CF80964.4070300@v.loewis.de>	<AANLkTikcT0WGo3FhFVq=N6ktToNenBDNNcC4eSpefcFF@mail.gmail.com>	<4CF80D6C.6050703@v.loewis.de>	<AANLkTi=s5NMxjPdRQRcupNBzdqjMhLO6TbFWtQzi0TEw@mail.gmail.com>	<4CF81208.8090705@v.loewis.de>	<AANLkTinspUo88oUnd3cX776mMRMU6bqyOaq9Og7JLwhU@mail.gmail.com>	<4CF81AE1.5050900@v.loewis.de>	<AANLkTi=FDOAgf9DCQ6N5EpMCJs6DCvsC3bQR6bbL=n+_@mail.gmail.com>	<4CF823C0.7070002@v.loewis.de>	<AANLkTinOwVj3iLbYT4kkoh0dPmW34WQx1HsZJPfnepT4@mail.gmail.com>	<4CF84C69.9020105@ubuntu.com>
	<4CF8527D.9030503@netwok.org>
Message-ID: <4CF89953.3070906@v.loewis.de>

> Python?s setup.py has an example in Martin?s branch:
> 
>   ext = Extension('xxlimited', ['xxlimited.c'],
>                   define_macros=[('Py_LIMITED_API', 1)])
> 
> <http://codereview.appspot.com/3262043/patch/1/68>
> 
> This is possible with today?s distutils.  I don?t know if it?s enough to
> build stable-ABI-conformant extension modules.

It is. However, there is also the proposal that they use an ABI tag in
the SO name; having that generated automatically would require a
distutils change.

Regards,
Martin

From martin at v.loewis.de  Fri Dec  3 08:31:27 2010
From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=)
Date: Fri, 03 Dec 2010 08:31:27 +0100
Subject: [Python-Dev] PEP 384 accepted
In-Reply-To: <4CF880F7.9000800@netwok.org>
References: <AANLkTimQsiU+nyX6CdqQ8JBybJdnx=1EMxX9mefHbx9H@mail.gmail.com>	<4CF8008B.60402@v.loewis.de>	<AANLkTikbRJEFYE_TvN++GYhmvSUu3hdhf4znOti7oty1@mail.gmail.com>	<4CF80964.4070300@v.loewis.de>	<AANLkTikcT0WGo3FhFVq=N6ktToNenBDNNcC4eSpefcFF@mail.gmail.com>	<4CF80D6C.6050703@v.loewis.de>	<AANLkTi=s5NMxjPdRQRcupNBzdqjMhLO6TbFWtQzi0TEw@mail.gmail.com>	<4CF81208.8090705@v.loewis.de>	<AANLkTinspUo88oUnd3cX776mMRMU6bqyOaq9Og7JLwhU@mail.gmail.com>	<4CF81AE1.5050900@v.loewis.de>
	<4CF880F7.9000800@netwok.org>
Message-ID: <4CF89CCF.20307@v.loewis.de>

> I wonder what your definition of ?unmaintained? is.

In this specific case: doesn't get feature requests acted upon.
I'm well aware that you are fixing bugs, and that is appreciated.

> Sure, distutils is not as well-maintained as other modules, but a dozen
> bugs have been fixed by five or six of us since the revert.  I do feel
> responsible for all 116 remaining bugs, and intend to address all of them.

But if the resolution of the bug would require a new feature, your
answer will be "this is going to be fixed in distutils2 (if at all),
it's out of scope for distutils". Before, if the submitter contributed
a patch, the patch was just unreviewed for a long time, unless one
of the committers picked it up. Now, the patch will be rejected, which
I consider worse - because the patch is not being rejected on its own
merits, but just because of a policy decision to not improve distutils
anymore.

For example, I keep running into the issue that distutils doesn't
currently support parallel builds. I have been pondering supporting
"-j" for building extensions, using both unbounded "-j" and the GNU make
style -jN build server. However, I know that the patch will be rejected,
so I don't even start working on it.

> On the matter of freeze exceptions, there have been two:
> - reading the makefile with surogateescape error handler so that python
> can build with an ASCII locale in a non-ASCII path (haypo, #6011)
> - handle soabiflags (barry, #9807).
> I took part in the discussion before those changes and did not object to
> them: they are very small changes that enable a new feature of Python
> 3.2.  Maybe I should have requested Tarek?s approval for those changes;
> he knows better than me how third parties may break because of changes
> that don?t seem to break anything.

I see. Now, I'd claim that the reasoning as to why an abi= parameter
on Extension may break things also applies to the soabiflags:
to support soabiflags, the INSTALL_SCHEMES syntax was modified.
If the install command is subclassed, that could lead to funny
interactions, e.g. where the subclass fails to put abiflags into
config_vars. IIUC, subst_vars will then eventually raise a ValueError.

I'm not saying that this is a likely scenario - only that the
reasoning "if a change can possibly affect existing code, it
should not be made" applies to essentially any change. So if you
want to avoid breaking things with certainty, not even bug
fixes would be acceptable.

Regards,
Martin

From martin at v.loewis.de  Fri Dec  3 08:33:23 2010
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Fri, 03 Dec 2010 08:33:23 +0100
Subject: [Python-Dev] Porting Ideas
In-Reply-To: <87lj47y4v0.fsf@uwakimon.sk.tsukuba.ac.jp>
References: <AANLkTing_iT7Y6FOZPWh=CUwtK8owvYtSdE7aXf0W181@mail.gmail.com>	<AANLkTi=6rvXDEfzmMC=vE5jaCaTzAwnDddxHrZ2GAB1b@mail.gmail.com>	<20101201201716.4357a316@pitrou.net>	<AANLkTi=u-YAuEkNcBmkwUzCScoeedFSYno_RC2YRMvEy@mail.gmail.com>	<id8mp3$76u$1@dough.gmane.org>	<20101202140650.11d96108@mission>	<4CF7F72A.6090406@v.loewis.de>
	<87lj47y4v0.fsf@uwakimon.sk.tsukuba.ac.jp>
Message-ID: <4CF89D43.5030805@v.loewis.de>

> It would be nice if the UI told users that, and offered an opportunity
> to log in.
> 
> Better yet would be a option for an OpenID to claim a user name by
> giving the password for it (ie, automatically on a successful login
> from that page).

So many projects, so little time. Contributions are welcome.
IOW, it's easier for me to educate users.

Regards,
Martin

From victor.stinner at haypocalc.com  Fri Dec  3 10:16:04 2010
From: victor.stinner at haypocalc.com (Victor Stinner)
Date: Fri, 3 Dec 2010 10:16:04 +0100
Subject: [Python-Dev] transform() and untransform() methods,
	and the codec registry
In-Reply-To: <20101202180651.F2AB0EE9CD@mail.python.org>
References: <20101202180651.F2AB0EE9CD@mail.python.org>
Message-ID: <201012031016.05063.victor.stinner@haypocalc.com>

On Thursday 02 December 2010 19:06:51 georg.brandl wrote:
> Author: georg.brandl
> Date: Thu Dec  2 19:06:51 2010
> New Revision: 86934
> 
> Log:
> #7475: add (un)transform method to bytes/bytearray and str, add back codecs
> that can be used with them from Python 2.

Oh no, someone did it. Was it really needed to reintroduce rot13 and friends?

I'm not strongly opposed to .transform()/.untranform() if it can be complelty 
separated to text encodings (ascii, latin9, utf-8 & cie). But str.encode() and 
bytes.decode() do accept transform codec names and raise strange error 
messages. Quote of Martin von L?wis (#7475):

"If the codecs are restored, one half of them becomes available to
.encode/.decode methods, since the codec registry cannot tell which
ones implement real character encodings, and which ones are other
conversion methods. So adding them would be really confusing."

>>> 'abc'.transform('hex')
TypeError: 'str' does not support the buffer interface
>>> b'abc'.transform('rot13')
TypeError: expected an object with the buffer interface

>>> b'abcd'.decode('hex')
TypeError: decoder did not return a str object (type=bytes)
>>> 'abc'.encode('rot13')
TypeError: encoder did not return a bytes object (type=str)

I don't like transform() and untransform() because I think that we should not 
add too much operations to the base types (bytes and str), and they do 
implicit module import. I prefer explicit module import (eg. import binascii; 
binascii.hexlify(b'to hex')). It remembers me PHP and it's ugly namespace with 
+5000 functions. I prefer Python because it uses smaller and more namespaces 
which are more specific and well defined. If we add email and compression 
functions to bytes, why not adding a web browser to the str?

Victor

From nyamatongwe at gmail.com  Fri Dec  3 11:07:32 2010
From: nyamatongwe at gmail.com (Neil Hodgson)
Date: Fri, 3 Dec 2010 21:07:32 +1100
Subject: [Python-Dev] Python and the Unicode Character Database
In-Reply-To: <87hbevy0iq.fsf@uwakimon.sk.tsukuba.ac.jp>
References: <AANLkTinVpySuRFkjf-Sc+7a91zrxhuQpqPQL5nM9u1w+@mail.gmail.com>
	<AANLkTinQLK7+5NfEp-2nMiJeDEKRxKN9Zfs4ihv-NMyH@mail.gmail.com>
	<87wrnv43v5.fsf@uwakimon.sk.tsukuba.ac.jp>
	<AANLkTinAf6CstywehOGRd4G4ms6K5LtSuq4Lj0z5wFBW@mail.gmail.com>
	<87r5e236hj.fsf@benfinney.id.au> <4CF58387.1000609@v.loewis.de>
	<87k4ju2vvg.fsf@benfinney.id.au> <4CF6CDDD.4060109@v.loewis.de>
	<4CF6D7B2.6030506@pearwood.info>
	<87sjyhxarg.fsf@uwakimon.sk.tsukuba.ac.jp>
	<87bp552arq.fsf@benfinney.id.au>
	<87oc94y5q3.fsf@uwakimon.sk.tsukuba.ac.jp>
	<AANLkTikuFdtbqXF0tj0mRENBciHwra1iR6ee7fTr79oL@mail.gmail.com>
	<87hbevy0iq.fsf@uwakimon.sk.tsukuba.ac.jp>
Message-ID: <AANLkTi=6n_Rg+KK8L2pJNWSeUhYhO9gBuUDLJtHaN0LP@mail.gmail.com>

Stephen J. Turnbull:

> Will it accept Arabic on input? ?(Han might be too much to ask for
> since Unicode considers Han digits to be "impure".)

   I couldn't find a direct way to input Arabic digits into OO Calc,
the normal use of Alt+number didn't work in Calc although it did in
WordPad where Alt+1632 is ? and so on.

   OO Calc does have settings in the Complex Text Layout section for
choosing different numerals but I don't understand the interaction of
choices here.

   Neil

From mal at egenix.com  Fri Dec  3 11:15:51 2010
From: mal at egenix.com (M.-A. Lemburg)
Date: Fri, 03 Dec 2010 11:15:51 +0100
Subject: [Python-Dev] Python and the Unicode Character Database
In-Reply-To: <AANLkTim8Uy9yDM8AFeho03dpEUmrgRdOyrUa1QdeKS5z@mail.gmail.com>
References: <AANLkTinVpySuRFkjf-Sc+7a91zrxhuQpqPQL5nM9u1w+@mail.gmail.com>	<AANLkTinQLK7+5NfEp-2nMiJeDEKRxKN9Zfs4ihv-NMyH@mail.gmail.com>	<87wrnv43v5.fsf@uwakimon.sk.tsukuba.ac.jp>	<AANLkTinAf6CstywehOGRd4G4ms6K5LtSuq4Lj0z5wFBW@mail.gmail.com>	<87r5e236hj.fsf@benfinney.id.au>
	<4CF58387.1000609@v.loewis.de>	<87k4ju2vvg.fsf@benfinney.id.au>
	<4CF6CDDD.4060109@v.loewis.de>	<4CF6D7B2.6030506@pearwood.info>	<87sjyhxarg.fsf@uwakimon.sk.tsukuba.ac.jp>	<87bp552arq.fsf@benfinney.id.au>
	<4CF7F579.5050603@v.loewis.de>	<4CF80C3A.9000705@egenix.com>	<AANLkTi=QApgGxCH0ph-_QjMMu5scH97vSdsBw_Da5Fvv@mail.gmail.com>	<4CF8247F.4000603@egenix.com>
	<AANLkTim8Uy9yDM8AFeho03dpEUmrgRdOyrUa1QdeKS5z@mail.gmail.com>
Message-ID: <4CF8C357.4020608@egenix.com>

Alexander Belopolsky wrote:
> On Thu, Dec 2, 2010 at 5:58 PM, M.-A. Lemburg <mal at egenix.com> wrote:
> ..
>>> I will change my mind on this issue when you present a
>>> machine-readable file with Arabic-Indic numerals and a program capable
>>> of reading it and show that this program uses the same number parsing
>>> algorithm as Python's int() or float().
>>
>> Have you had a look at the examples I posted ? They include texts
>> and tables with numbers written using east asian arabic numerals.
> 
> Yes, but this was all about output.  I am pretty sure TeX was able to
> typeset Qur'an in all its glory long before Unicode was invented.
> Yet, in machine readable form it would be something like {\quran 1}
> (invented directive).   I have asked for a file that is intended for
> machine processing, not for human enjoyment in print or on a display.
>  I claim that if such file exists, the program that reads it does not
> use the same rules as Python and converting non-ascii digits would be
> a tiny portion of what that program does.

Well, programs that take input from the keyboards I posted in this
thread will have to deal with the digits. Since Python's input()
accepts keyboard input, you have your use case :-)

Seriously, I find the distinction between input and output forms
of numerals somewhat misguided. Any output can also serve as input.
For books and other printed material, images, etc. you have scanners
and OCR. For screen output you have screen readers. For spreadsheets
and data, you have CSV, TSV, XML, etc. etc. etc.

Just for the fun of it, I created a CSV file with Thai and Dzongkha
numerals (in addition to Arabic ones) using OpenOffice. Here's the
cut and paste version:

"""
Numbers in various scripts		
		
Arabic	Thai	Dzongkha
1	?	?
2	?	?
3	?	?
4	?	?
5	?	?
6	?	?
7	?	?
8	?	?
9	?	?
10	??	??
11	??	??
12	??	??
13	??	??
14	??	??
15	??	??
16	??	??
17	??	??
18	??	??
19	??	??
20	??	??
"""

And here's the script that goes with it:

import csv
c = csv.reader(open('Numbers-in-various-scripts.csv'))
headers = [c.next() for i in range(3)]
while c:
    print [int(unicode(x, 'utf-8')) for x in c.next()]

and the output using Python 2.7:

[1, 1, 1]
[2, 2, 2]
[3, 3, 3]
[4, 4, 4]
[5, 5, 5]
[6, 6, 6]
[7, 7, 7]
[8, 8, 8]
[9, 9, 9]
[10, 10, 10]
[11, 11, 11]
[12, 12, 12]
[13, 13, 13]
[14, 14, 14]
[15, 15, 15]
[16, 16, 16]
[17, 17, 17]
[18, 18, 18]
[19, 19, 19]
[20, 20, 20]

If you need more such files, I can generate as many as you like ;-)
I can send the OOo file as well, if you like to play around with it.

I'd say: case closed :-)

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Dec 03 2010)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try our new mxODBC.Connect Python Database Interface for free ! ::::


   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
           Registered at Amtsgericht Duesseldorf: HRB 46611
               http://www.egenix.com/company/contact/
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: Numbers-in-various-scripts.csv
URL: <http://mail.python.org/pipermail/python-dev/attachments/20101203/0f4a8bee/attachment.ksh>

From ncoghlan at gmail.com  Fri Dec  3 11:53:04 2010
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Fri, 3 Dec 2010 20:53:04 +1000
Subject: [Python-Dev] [Python-checkins] r86965 -
	python/branches/py3k/Lib/test/__main__.py
In-Reply-To: <20101203104203.E5461EE982@mail.python.org>
References: <20101203104203.E5461EE982@mail.python.org>
Message-ID: <AANLkTimE3FFcgRGPUP9Y7L6xboZKrYe6+zr32vJnR8jv@mail.gmail.com>

On Fri, Dec 3, 2010 at 8:42 PM, michael.foord
<python-checkins at python.org> wrote:
> +# When tests are run from the Python build directory, it is best practice
> +# to keep the test files in a subfolder. ?It eases the cleanup of leftover
> +# files using command "make distclean".
> +if sysconfig.is_python_build():
> + ? ?TEMPDIR = os.path.join(sysconfig.get_config_var('srcdir'), 'build')
> + ? ?TEMPDIR = os.path.abspath(TEMPDIR)
> + ? ?if not os.path.exists(TEMPDIR):
> + ? ? ? ?os.mkdir(TEMPDIR)
> + ? ?regrtest.TEMPDIR = TEMPDIR

If syconfig.is_python_build() returns False...

> +# Define a writable temp dir that will be used as cwd while running
> +# the tests. The name of the dir includes the pid to allow parallel
> +# testing (see the -j option).
> +TESTCWD = 'test_python_{}'.format(os.getpid())
> +
> +TESTCWD = os.path.join(TEMPDIR, TESTCWD)

... this line is going to blow up with a NameError.

I would suggest putting this common setup code into a _make_test_dir()
helper function in regrtest, then have both regrtest and test.__main__
call it.

Cheers,
Nick.

-- 
Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia

From fuzzyman at voidspace.org.uk  Fri Dec  3 11:56:21 2010
From: fuzzyman at voidspace.org.uk (Michael Foord)
Date: Fri, 03 Dec 2010 10:56:21 +0000
Subject: [Python-Dev] [Python-checkins] r86965
	-	python/branches/py3k/Lib/test/__main__.py
In-Reply-To: <AANLkTimE3FFcgRGPUP9Y7L6xboZKrYe6+zr32vJnR8jv@mail.gmail.com>
References: <20101203104203.E5461EE982@mail.python.org>
	<AANLkTimE3FFcgRGPUP9Y7L6xboZKrYe6+zr32vJnR8jv@mail.gmail.com>
Message-ID: <4CF8CCD5.6000905@voidspace.org.uk>

On 03/12/2010 10:53, Nick Coghlan wrote:
> On Fri, Dec 3, 2010 at 8:42 PM, michael.foord
> <python-checkins at python.org>  wrote:
>> +# When tests are run from the Python build directory, it is best practice
>> +# to keep the test files in a subfolder.  It eases the cleanup of leftover
>> +# files using command "make distclean".
>> +if sysconfig.is_python_build():
>> +    TEMPDIR = os.path.join(sysconfig.get_config_var('srcdir'), 'build')
>> +    TEMPDIR = os.path.abspath(TEMPDIR)
>> +    if not os.path.exists(TEMPDIR):
>> +        os.mkdir(TEMPDIR)
>> +    regrtest.TEMPDIR = TEMPDIR
> If syconfig.is_python_build() returns False...
>
>> +# Define a writable temp dir that will be used as cwd while running
>> +# the tests. The name of the dir includes the pid to allow parallel
>> +# testing (see the -j option).
>> +TESTCWD = 'test_python_{}'.format(os.getpid())
>> +
>> +TESTCWD = os.path.join(TEMPDIR, TESTCWD)
> ... this line is going to blow up with a NameError.
>
> I would suggest putting this common setup code into a _make_test_dir()
> helper function in regrtest, then have both regrtest and test.__main__
> call it.
>

Ok, good suggestion. Thanks

Michael

> Cheers,
> Nick.
>


-- 

http://www.voidspace.org.uk/

READ CAREFULLY. By accepting and reading this email you agree,
on behalf of your employer, to release me from all obligations
and waivers arising from any and all NON-NEGOTIATED agreements,
licenses, terms-of-service, shrinkwrap, clickwrap, browsewrap,
confidentiality, non-disclosure, non-compete and acceptable use
policies (?BOGUS AGREEMENTS?) that I have entered into with your
employer, its partners, licensors, agents and assigns, in
perpetuity, without prejudice to my ongoing rights and privileges.
You further represent that you have the authority to release me
from any BOGUS AGREEMENTS on behalf of your employer.


From merwok at netwok.org  Fri Dec  3 13:27:23 2010
From: merwok at netwok.org (=?UTF-8?B?w4lyaWMgQXJhdWpv?=)
Date: Fri, 03 Dec 2010 13:27:23 +0100
Subject: [Python-Dev] PEP 384 accepted
In-Reply-To: <4CF89CCF.20307@v.loewis.de>
References: <AANLkTimQsiU+nyX6CdqQ8JBybJdnx=1EMxX9mefHbx9H@mail.gmail.com>	<4CF8008B.60402@v.loewis.de>	<AANLkTikbRJEFYE_TvN++GYhmvSUu3hdhf4znOti7oty1@mail.gmail.com>	<4CF80964.4070300@v.loewis.de>	<AANLkTikcT0WGo3FhFVq=N6ktToNenBDNNcC4eSpefcFF@mail.gmail.com>	<4CF80D6C.6050703@v.loewis.de>	<AANLkTi=s5NMxjPdRQRcupNBzdqjMhLO6TbFWtQzi0TEw@mail.gmail.com>	<4CF81208.8090705@v.loewis.de>	<AANLkTinspUo88oUnd3cX776mMRMU6bqyOaq9Og7JLwhU@mail.gmail.com>	<4CF81AE1.5050900@v.loewis.de>
	<4CF880F7.9000800@netwok.org> <4CF89CCF.20307@v.loewis.de>
Message-ID: <4CF8E22B.7050005@netwok.org>

Le 03/12/2010 08:31, Martin v. L?wis a ?crit :
>> I wonder what your definition of ?unmaintained? is. 
> In this specific case: doesn't get feature requests acted upon.
Thanks for clarifying.  I think that?s a stretch, but I see your meaning
now.

>> Sure, distutils is not as well-maintained as other modules, but a dozen
>> bugs have been fixed by five or six of us since the revert.  I do feel
>> responsible for all 116 remaining bugs, and intend to address all of them.
> 
> But if the resolution of the bug would require a new feature, your
> answer will be "this is going to be fixed in distutils2 (if at all),
> it's out of scope for distutils". Before, if the submitter contributed
> a patch, the patch was just unreviewed for a long time, unless one
> of the committers picked it up. Now, the patch will be rejected, which
> I consider worse - because the patch is not being rejected on its own
> merits, but just because of a policy decision to not improve distutils
> anymore.
The patch would not be rejected, but assigned to a different version.
It?s not different than replying to an old bug with a patch for Python
2.5 and requesting that it be updated for py3k.  It?s also not uncommon
to have another contributor or a core dev updating the patch if the
original poster does not reply.

> For example, I keep running into the issue that distutils doesn't
> currently support parallel builds. I have been pondering supporting
> "-j" for building extensions, using both unbounded "-j" and the GNU make
> style -jN build server. However, I know that the patch will be rejected,
> so I don't even start working on it.
This would be a very useful feature for distutils2.

>> On the matter of freeze exceptions, there have been two: [snip]
> I see. Now, I'd claim that the reasoning as to why an abi= parameter
> on Extension may break things also applies to the soabiflags:
> to support soabiflags, the INSTALL_SCHEMES syntax was modified.
> If the install command is subclassed, that could lead to funny
> interactions, e.g. where the subclass fails to put abiflags into
> config_vars. IIUC, subst_vars will then eventually raise a ValueError.
This is a concern.

> I'm not saying that this is a likely scenario - only that the
> reasoning "if a change can possibly affect existing code, it
> should not be made" applies to essentially any change. So if you
> want to avoid breaking things with certainty, not even bug
> fixes would be acceptable.
If we wanted 100% certainty, yes.

Regards


From solipsis at pitrou.net  Fri Dec  3 13:51:06 2010
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Fri, 03 Dec 2010 13:51:06 +0100
Subject: [Python-Dev] Python and the Unicode Character Database
In-Reply-To: <87fwufxxio.fsf@uwakimon.sk.tsukuba.ac.jp>
References: <AANLkTinVpySuRFkjf-Sc+7a91zrxhuQpqPQL5nM9u1w+@mail.gmail.com>
	<AANLkTinQLK7+5NfEp-2nMiJeDEKRxKN9Zfs4ihv-NMyH@mail.gmail.com>
	<87wrnv43v5.fsf@uwakimon.sk.tsukuba.ac.jp>
	<id3umc$ps8$1@dough.gmane.org>
	<4CF5841E.6020401@v.loewis.de> <4CF619C1.8030403@pearwood.info>
	<4CF6CCF9.6000100@v.loewis.de> <4CF6E5AF.7080308@pearwood.info>
	<AANLkTikuUki5LgisVHrSMGZtmWZxxy9A8TOwny7dVCMe@mail.gmail.com>
	<id72oo$cdp$2@dough.gmane.org>
	<AANLkTikd0zDjr2GQ3_4C7BLZZstePQAiHee9m2ZpgCEh@mail.gmail.com>
	<20101202143633.6eb17b8d@pitrou.net>
	<AANLkTikiziBN87-7gM3rweODnXjEuK3DfhEni1sTDYtJ@mail.gmail.com>
	<1291308960.3576.28.camel@localhost.localdomain>
	<AANLkTinBzCQ4gi42iNXkSqWAEkdPWZzs=+C5sr0f0vpB@mail.gmail.com>
	<1291316131.3576.43.camel@localhost.localdomain>
	<87fwufxxio.fsf@uwakimon.sk.tsukuba.ac.jp>
Message-ID: <1291380666.3624.0.camel@localhost.localdomain>

Le vendredi 03 d?cembre 2010 ? 13:58 +0900, Stephen J. Turnbull a
?crit :
> Antoine Pitrou writes:
> 
>  > The legacy format argument looks like a red herring to me. When
>  > converting from a format to another it is the programmer's job to
>  > his/her job right.
> 
> Uhmmmmmm, the argument *for* this "feature" proposed by several people
> is that Python's numeric constructors do it (right) so that the
> programmer doesn't have to.

As far as I understand, Alexander was talking about a legacy pre-unicode
text format. We don't have to support this.

Regards

Antoine.



From solipsis at pitrou.net  Fri Dec  3 15:00:15 2010
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Fri, 3 Dec 2010 15:00:15 +0100
Subject: [Python-Dev] r86962 - in python/branches/py3k:
 Doc/library/pydoc.rst Doc/whatsnew/3.2.rst Lib/pydoc.py
 Lib/test/test_pydoc.py Misc/ACKS Misc/NEWS
References: <20101203092912.3D2A0EE981@mail.python.org>
Message-ID: <20101203150015.62d89632@pitrou.net>

On Fri,  3 Dec 2010 10:29:12 +0100 (CET)
nick.coghlan <python-checkins at python.org> wrote:
> Author: nick.coghlan
> Date: Fri Dec  3 10:29:11 2010
> New Revision: 86962
> 
> Log:
> Improve Pydoc interactive browsing (#2001).  Patch by Ron Adam.

Tests seem to fail under Windows:

======================================================================
FAIL: test_url_requests (test.test_pydoc.PyDocUrlHandlerTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "D:\cygwin\home\db3l\buildarea\3.x.bolen-windows\build\lib\test\test_pydoc.py", line 456, in test_url_requests
    self.assertEqual(result, title)
AssertionError: 'Python: Read Error' != 'Python: getfile /D:\\cygwin\\home\\db3l\\buildarea\\3.x.bolen-windows\\build\\l [truncated]...
- Python: Read Error
+ Python: getfile /D:\cygwin\home\db3l\buildarea\3.x.bolen-windows\build\lib\string.py


(http://www.python.org/dev/buildbot/all/builders/x86%20XP-4%203.x/builds/3708/steps/test/logs/stdio)




From rdmurray at bitdance.com  Fri Dec  3 16:11:29 2010
From: rdmurray at bitdance.com (R. David Murray)
Date: Fri, 03 Dec 2010 10:11:29 -0500
Subject: [Python-Dev] transform() and untransform() methods,
	and the codec registry
In-Reply-To: <201012031016.05063.victor.stinner@haypocalc.com>
References: <20101202180651.F2AB0EE9CD@mail.python.org>
	<201012031016.05063.victor.stinner@haypocalc.com>
Message-ID: <20101203151129.74C5222A88F@kimball.webabinitio.net>

On Fri, 03 Dec 2010 10:16:04 +0100, Victor Stinner <victor.stinner at haypocalc.com> wrote:
> On Thursday 02 December 2010 19:06:51 georg.brandl wrote:
> > Author: georg.brandl
> > Date: Thu Dec  2 19:06:51 2010
> > New Revision: 86934
> >
> > Log:
> > #7475: add (un)transform method to bytes/bytearray and str, add back codecs
> > that can be used with them from Python 2.
> 
> Oh no, someone did it. Was it really needed to reintroduce rot13 and friends?
> 
> I'm not strongly opposed to .transform()/.untranform() if it can be complelty
> separated to text encodings (ascii, latin9, utf-8 & cie). But str.encode() and
> bytes.decode() do accept transform codec names and raise strange error
> messages. Quote of Martin von L??wis (#7475):
> 
> "If the codecs are restored, one half of them becomes available to
> .encode/.decode methods, since the codec registry cannot tell which
> ones implement real character encodings, and which ones are other
> conversion methods. So adding them would be really confusing."
> 
> >>> 'abc'.transform('hex')
> TypeError: 'str' does not support the buffer interface
> >>> b'abc'.transform('rot13')
> TypeError: expected an object with the buffer interface

I find these 'buffer interface' error messages to be the most confusing
error message I get out of Python3 no matter what context they show up
in.  I have no idea what they are telling me.  That issue is more
general than transform/untransform, but perhaps it could be fixed
for transform/untransform in particular.

> >>> b'abcd'.decode('hex')
> TypeError: decoder did not return a str object (type=bytes)
> >>> 'abc'.encode('rot13')
> TypeError: encoder did not return a bytes object (type=str)

These error messages make perfect sense to me.  I think it
is called "duck typing" :)

> I don't like transform() and untransform() because I think that we should not
> add too much operations to the base types (bytes and str), and they do
> implicit module import. I prefer explicit module import (eg. import binascii;
> binascii.hexlify(b'to hex')). It remembers me PHP and it's ugly namespace with
> +5000 functions. I prefer Python because it uses smaller and more namespaces
> which are more specific and well defined. If we add email and compression
> functions to bytes, why not adding a web browser to the str?

As MAL says, the codec machinery is a general purpose tool.  I think
it, and the transform methods, are a useful level of abstraction over
a general class of problems.

Please also recall that transform/untransform was discussed before
the release of Python 3.0 and was approved at the time, but it just
did not get implemented before the 3.0 release.

--
R. David Murray                                      www.bitdance.com

From alexander.belopolsky at gmail.com  Fri Dec  3 17:14:56 2010
From: alexander.belopolsky at gmail.com (Alexander Belopolsky)
Date: Fri, 3 Dec 2010 11:14:56 -0500
Subject: [Python-Dev] transform() and untransform() methods,
	and the codec registry
In-Reply-To: <20101203151129.74C5222A88F@kimball.webabinitio.net>
References: <20101202180651.F2AB0EE9CD@mail.python.org>
	<201012031016.05063.victor.stinner@haypocalc.com>
	<20101203151129.74C5222A88F@kimball.webabinitio.net>
Message-ID: <AANLkTiktmxr_eH-S9ihcwFoqd6yJeN03Hes_urH7Qt3A@mail.gmail.com>

On Fri, Dec 3, 2010 at 10:11 AM, R. David Murray <rdmurray at bitdance.com> wrote:
..
> Please also recall that transform/untransform was discussed before
> the release of Python 3.0 and was approved at the time, but it just
> did not get implemented before the 3.0 release.
>

Can you provide a link?  My search for transform on python-dev came out with

http://mail.python.org/pipermail/python-dev/2010-June/100564.html

where you seem to oppose these methods.  Also, new methods to builtins
fall under the language moratorium (but can be approved on a
case-by-case basis):

http://www.python.org/dev/peps/pep-3003/#case-by-case-exemptions

Is there an effort to document these exceptions?  I expected such
approvals to be added to PEP 3003, but apparently this was not the
case.

From solipsis at pitrou.net  Fri Dec  3 17:28:14 2010
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Fri, 3 Dec 2010 17:28:14 +0100
Subject: [Python-Dev] "buffer interface" messages
References: <20101202180651.F2AB0EE9CD@mail.python.org>
	<201012031016.05063.victor.stinner@haypocalc.com>
	<20101203151129.74C5222A88F@kimball.webabinitio.net>
Message-ID: <20101203172814.6aa8ba0e@pitrou.net>

On Fri, 03 Dec 2010 10:11:29 -0500
"R. David Murray" <rdmurray at bitdance.com> wrote:
> > 
> > >>> 'abc'.transform('hex')
> > TypeError: 'str' does not support the buffer interface
> > >>> b'abc'.transform('rot13')
> > TypeError: expected an object with the buffer interface
> 
> I find these 'buffer interface' error messages to be the most confusing
> error message I get out of Python3 no matter what context they show up
> in.  I have no idea what they are telling me.  That issue is more
> general than transform/untransform, but perhaps it could be fixed
> for transform/untransform in particular.

I agree. "buffer interface" is a technicality that the Python user
doesn't do about (unless (s)he also writes C extensions). How about
"expected a bytes-compatible object"?

Regards

Antoine.



From ncoghlan at gmail.com  Fri Dec  3 17:28:41 2010
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sat, 4 Dec 2010 02:28:41 +1000
Subject: [Python-Dev] r86962 - in python/branches/py3k:
 Doc/library/pydoc.rst Doc/whatsnew/3.2.rst Lib/pydoc.py
 Lib/test/test_pydoc.py Misc/ACKS Misc/NEWS
In-Reply-To: <20101203150015.62d89632@pitrou.net>
References: <20101203092912.3D2A0EE981@mail.python.org>
	<20101203150015.62d89632@pitrou.net>
Message-ID: <AANLkTinRQDrsU+Rztjq9UwwSFL3ySZ4Lc9ZXiBnkMhBE@mail.gmail.com>

On Sat, Dec 4, 2010 at 12:00 AM, Antoine Pitrou <solipsis at pitrou.net> wrote:
> On Fri, ?3 Dec 2010 10:29:12 +0100 (CET)
> nick.coghlan <python-checkins at python.org> wrote:
>> Author: nick.coghlan
>> Date: Fri Dec ?3 10:29:11 2010
>> New Revision: 86962
>>
>> Log:
>> Improve Pydoc interactive browsing (#2001). ?Patch by Ron Adam.
>
> Tests seem to fail under Windows:
>
> ======================================================================
> FAIL: test_url_requests (test.test_pydoc.PyDocUrlHandlerTest)
> ----------------------------------------------------------------------
> Traceback (most recent call last):
> ?File "D:\cygwin\home\db3l\buildarea\3.x.bolen-windows\build\lib\test\test_pydoc.py", line 456, in test_url_requests
> ? ?self.assertEqual(result, title)
> AssertionError: 'Python: Read Error' != 'Python: getfile /D:\\cygwin\\home\\db3l\\buildarea\\3.x.bolen-windows\\build\\l [truncated]...
> - Python: Read Error
> + Python: getfile /D:\cygwin\home\db3l\buildarea\3.x.bolen-windows\build\lib\string.py
>
>
> (http://www.python.org/dev/buildbot/all/builders/x86%20XP-4%203.x/builds/3708/steps/test/logs/stdio)

Yeah, Georg pointed that one out. My latest checkin *should* have
fixed it (but we'll see what the buildbots have to say).

Cheers,
Nick.

-- 
Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia

From victor.stinner at haypocalc.com  Fri Dec  3 17:44:17 2010
From: victor.stinner at haypocalc.com (Victor Stinner)
Date: Fri, 3 Dec 2010 17:44:17 +0100
Subject: [Python-Dev] transform() and untransform() methods,
	and the codec registry
In-Reply-To: <20101203151129.74C5222A88F@kimball.webabinitio.net>
References: <20101202180651.F2AB0EE9CD@mail.python.org>
	<201012031016.05063.victor.stinner@haypocalc.com>
	<20101203151129.74C5222A88F@kimball.webabinitio.net>
Message-ID: <201012031744.17152.victor.stinner@haypocalc.com>

On Friday 03 December 2010 16:11:29 R. David Murray wrote:
> > >>> 'abc'.transform('hex')
> > TypeError: 'str' does not support the buffer interface
> > 
> > >>> b'abc'.transform('rot13')
> > TypeError: expected an object with the buffer interface
> 
> I find these 'buffer interface' error messages to be the most confusing
> error message I get out of Python3 no matter what context they show up
> in.  I have no idea what they are telling me.  That issue is more
> general than transform/untransform, but perhaps it could be fixed
> for transform/untransform in particular.

If it's more general, let's open an issue for that:
http://bugs.python.org/issue10616

> > >>> b'abcd'.decode('hex')
> > TypeError: decoder did not return a str object (type=bytes)
> > 
> > >>> 'abc'.encode('rot13')
> > TypeError: encoder did not return a bytes object (type=str)
> 
> These error messages make perfect sense to me.  I think it
> is called "duck typing" :)
>
> (...) 
>
> Please also recall that transform/untransform was discussed before
> the release of Python 3.0 and was approved at the time, but it just
> did not get implemented before the 3.0 release.

Ok.

Victor

From ncoghlan at gmail.com  Fri Dec  3 17:45:42 2010
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sat, 4 Dec 2010 02:45:42 +1000
Subject: [Python-Dev] "buffer interface" messages
In-Reply-To: <20101203172814.6aa8ba0e@pitrou.net>
References: <20101202180651.F2AB0EE9CD@mail.python.org>
	<201012031016.05063.victor.stinner@haypocalc.com>
	<20101203151129.74C5222A88F@kimball.webabinitio.net>
	<20101203172814.6aa8ba0e@pitrou.net>
Message-ID: <AANLkTikn8KpffwYKOxHS_kCazah05hfztJ7CDBh9iJW=@mail.gmail.com>

On Sat, Dec 4, 2010 at 2:28 AM, Antoine Pitrou <solipsis at pitrou.net> wrote:
> On Fri, 03 Dec 2010 10:11:29 -0500
> "R. David Murray" <rdmurray at bitdance.com> wrote:
>> >
>> > >>> 'abc'.transform('hex')
>> > TypeError: 'str' does not support the buffer interface
>> > >>> b'abc'.transform('rot13')
>> > TypeError: expected an object with the buffer interface
>>
>> I find these 'buffer interface' error messages to be the most confusing
>> error message I get out of Python3 no matter what context they show up
>> in. ?I have no idea what they are telling me. ?That issue is more
>> general than transform/untransform, but perhaps it could be fixed
>> for transform/untransform in particular.
>
> I agree. "buffer interface" is a technicality that the Python user
> doesn't do about (unless (s)he also writes C extensions). How about
> "expected a bytes-compatible object"?

Why not "binary data interface"? That's what they're actually looking for.

It seems odd for 'rot13' to be throwing that error though.

Cheers,
Nick.

-- 
Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia

From solipsis at pitrou.net  Fri Dec  3 17:57:22 2010
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Fri, 3 Dec 2010 17:57:22 +0100
Subject: [Python-Dev] "buffer interface" messages
In-Reply-To: <AANLkTikn8KpffwYKOxHS_kCazah05hfztJ7CDBh9iJW=@mail.gmail.com>
References: <20101202180651.F2AB0EE9CD@mail.python.org>
	<201012031016.05063.victor.stinner@haypocalc.com>
	<20101203151129.74C5222A88F@kimball.webabinitio.net>
	<20101203172814.6aa8ba0e@pitrou.net>
	<AANLkTikn8KpffwYKOxHS_kCazah05hfztJ7CDBh9iJW=@mail.gmail.com>
Message-ID: <20101203175722.6124d0a6@pitrou.net>

On Sat, 4 Dec 2010 02:45:42 +1000
Nick Coghlan <ncoghlan at gmail.com> wrote:
> On Sat, Dec 4, 2010 at 2:28 AM, Antoine Pitrou <solipsis at pitrou.net> wrote:
> > On Fri, 03 Dec 2010 10:11:29 -0500
> > "R. David Murray" <rdmurray at bitdance.com> wrote:
> >> >
> >> > >>> 'abc'.transform('hex')
> >> > TypeError: 'str' does not support the buffer interface
> >> > >>> b'abc'.transform('rot13')
> >> > TypeError: expected an object with the buffer interface
> >>
> >> I find these 'buffer interface' error messages to be the most confusing
> >> error message I get out of Python3 no matter what context they show up
> >> in. ?I have no idea what they are telling me. ?That issue is more
> >> general than transform/untransform, but perhaps it could be fixed
> >> for transform/untransform in particular.
> >
> > I agree. "buffer interface" is a technicality that the Python user
> > doesn't do about (unless (s)he also writes C extensions). How about
> > "expected a bytes-compatible object"?
> 
> Why not "binary data interface"? That's what they're actually looking for.

I don't think it's more understandable, since it's not a well-known
Python concept.
(and in this specific case, the transform() method is on str and bytes,
not on arbitrary bytes-like objects)

> It seems odd for 'rot13' to be throwing that error though.

Agreed.

Regards

Antoine.

From ncoghlan at gmail.com  Fri Dec  3 18:05:32 2010
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sat, 4 Dec 2010 03:05:32 +1000
Subject: [Python-Dev] "buffer interface" messages
In-Reply-To: <20101203175722.6124d0a6@pitrou.net>
References: <20101202180651.F2AB0EE9CD@mail.python.org>
	<201012031016.05063.victor.stinner@haypocalc.com>
	<20101203151129.74C5222A88F@kimball.webabinitio.net>
	<20101203172814.6aa8ba0e@pitrou.net>
	<AANLkTikn8KpffwYKOxHS_kCazah05hfztJ7CDBh9iJW=@mail.gmail.com>
	<20101203175722.6124d0a6@pitrou.net>
Message-ID: <AANLkTinAwT3m7PQUBEV-uWOR3JSbYgS22CQXCcCR_ekS@mail.gmail.com>

On Sat, Dec 4, 2010 at 2:57 AM, Antoine Pitrou <solipsis at pitrou.net> wrote:
> On Sat, 4 Dec 2010 02:45:42 +1000
> Nick Coghlan <ncoghlan at gmail.com> wrote:
>> On Sat, Dec 4, 2010 at 2:28 AM, Antoine Pitrou <solipsis at pitrou.net> wrote:
>> > On Fri, 03 Dec 2010 10:11:29 -0500
>> > "R. David Murray" <rdmurray at bitdance.com> wrote:
>> >> >
>> >> > >>> 'abc'.transform('hex')
>> >> > TypeError: 'str' does not support the buffer interface
>> >> > >>> b'abc'.transform('rot13')
>> >> > TypeError: expected an object with the buffer interface
>> >>
>> >> I find these 'buffer interface' error messages to be the most confusing
>> >> error message I get out of Python3 no matter what context they show up
>> >> in. ?I have no idea what they are telling me. ?That issue is more
>> >> general than transform/untransform, but perhaps it could be fixed
>> >> for transform/untransform in particular.
>> >
>> > I agree. "buffer interface" is a technicality that the Python user
>> > doesn't do about (unless (s)he also writes C extensions). How about
>> > "expected a bytes-compatible object"?
>>
>> Why not "binary data interface"? That's what they're actually looking for.
>
> I don't think it's more understandable, since it's not a well-known
> Python concept.
> (and in this specific case, the transform() method is on str and bytes,
> not on arbitrary bytes-like objects)

Indeed, "bytes-compatible" is likely to be the most meaningful phrase
(and then we can have the bytes docs explain further as to what
"bytes-compatible" means, likely by reference to memoryview).

Although, as Victor's mod to the complex constructor error message
shows, the default error message is still going to be overly specific
in many cases that also accept non-binary data.

Cheers,
Nick.

-- 
Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia

From status at bugs.python.org  Fri Dec  3 18:07:01 2010
From: status at bugs.python.org (Python tracker)
Date: Fri,  3 Dec 2010 18:07:01 +0100 (CET)
Subject: [Python-Dev] Summary of Python tracker Issues
Message-ID: <20101203170701.A2821104031@psf.upfronthosting.co.za>


ACTIVITY SUMMARY (2010-11-26 - 2010-12-03)
Python tracker at http://bugs.python.org/

To view or respond to any of the issues listed below, click on the issue.
Do NOT respond to this message.

Issues counts and deltas:
  open    2537 ( +4)
  closed 19861 (+69)
  total  22398 (+73)

Open issues with patches: 1080 


Issues opened (47)
==================

#2690: Precompute range length and enhance range subscript support
http://bugs.python.org/issue2690  reopened by ncoghlan

#9333: Expose a way to enable os.symlink on Windows
http://bugs.python.org/issue9333  reopened by brian.curtin

#10544: yield expression inside generator expression does nothing
http://bugs.python.org/issue10544  opened by Inyeol.Lee

#10545: remove or rewrite "Using Backslash to Continue Statements" ant
http://bugs.python.org/issue10545  opened by rurpy2

#10546: UTF-16-LE and UTF-16-BE support non-BMP characters
http://bugs.python.org/issue10546  opened by haypo

#10548: Error in setUp not reported as expectedFailure (unittest)
http://bugs.python.org/issue10548  opened by michael.foord

#10550: Windows: leak in test_concurrent_futures
http://bugs.python.org/issue10550  opened by skrah

#10551: mimetypes read from the registry should not overwrite standard
http://bugs.python.org/issue10551  opened by kovid

#10552: Tools/unicode/gencodec.py error
http://bugs.python.org/issue10552  opened by belopolsky

#10553: Add optimize argument to builtin compile() and byte-compilatio
http://bugs.python.org/issue10553  opened by georg.brandl

#10556: test_zipimport_support mucks up with modules
http://bugs.python.org/issue10556  opened by pitrou

#10557: Malformed error message from float()
http://bugs.python.org/issue10557  opened by belopolsky

#10558: non-standard processing of several configure options ignores "
http://bugs.python.org/issue10558  opened by ned.deily

#10559: NameError in tutorial/interpreter
http://bugs.python.org/issue10559  opened by eric.araujo

#10560: Fixes for Windows sources
http://bugs.python.org/issue10560  opened by Carlo_Bramini

#10563: Spurious newline in time.ctime
http://bugs.python.org/issue10563  opened by Gerrit.Holl

#10566: gdb debugging support additions (Tools/gdb/libpython.py)
http://bugs.python.org/issue10566  opened by eggy

#10570: curses.tigetstr() returns bytes, but curses.tparm() expects a 
http://bugs.python.org/issue10570  opened by jwilk

#10571: "setup.py upload --sign" broken: TypeError: 'str' does not sup
http://bugs.python.org/issue10571  opened by jwilk

#10572: Move unittest test package to Lib/test
http://bugs.python.org/issue10572  opened by michael.foord

#10573: Consistency in unittest assert methods: order of actual, expec
http://bugs.python.org/issue10573  opened by michael.foord

#10574: email.header.decode_header fails if the string contains multip
http://bugs.python.org/issue10574  opened by starsareblueandfaraway

#10576: Add a progress callback to gcmodule
http://bugs.python.org/issue10576  opened by krisvale

#10577: (Fancy) URL opener stuck when trying to open redirected url
http://bugs.python.org/issue10577  opened by xhresko

#10580: Installer sentence in bold
http://bugs.python.org/issue10580  opened by Retro

#10581: Review and document string format accepted in numeric data typ
http://bugs.python.org/issue10581  opened by belopolsky

#10582: PyErr_PrintEx exits silently when passed SystemExit exception
http://bugs.python.org/issue10582  opened by Marc.Horowitz

#10583: Encoding issue with chm help in 2.7.1
http://bugs.python.org/issue10583  opened by flashk

#10587: Document the meaning of str methods
http://bugs.python.org/issue10587  opened by belopolsky

#10588: imp.find_module raises unexpected SyntaxError
http://bugs.python.org/issue10588  opened by emile.anclin

#10589: I/O ABC docs should specify which methods have implementations
http://bugs.python.org/issue10589  opened by stutzbach

#10590: Parameter type error for xml.sax.parseString(string, ...)
http://bugs.python.org/issue10590  opened by Thomas.Ryan

#10592: pprint module doesn't work well with OrderedDicts
http://bugs.python.org/issue10592  opened by mikez302

#10595: Adding a syslog.conf reader in syslog
http://bugs.python.org/issue10595  opened by tarek

#10596: modulo operator bug
http://bugs.python.org/issue10596  opened by Sergio.??lut??in

#10598: curses fails to import on Solaris
http://bugs.python.org/issue10598  opened by rtarnell

#10599: sgmllib.parse_endtag() is not respecting quoted text
http://bugs.python.org/issue10599  opened by Michael.Brooks

#10601: sys.displayhook: use backslashreplace error handler if repr(va
http://bugs.python.org/issue10601  opened by haypo

#10605: ElementTree documentation
http://bugs.python.org/issue10605  opened by adrian_nye

#10608: Add a section to Windows FAQ explaining os.symlink
http://bugs.python.org/issue10608  opened by brian.curtin

#10609: dbm documentation example doesn't work (iteritems())
http://bugs.python.org/issue10609  opened by sandro.tosi

#10610: Correct the float(), int() and complex() documentation
http://bugs.python.org/issue10610  opened by lemburg

#10611: sys.exit() in a test causes a test run to die
http://bugs.python.org/issue10611  opened by michael.foord

#10612: StopTestRun exception to halt test run
http://bugs.python.org/issue10612  opened by michael.foord

#10614: ZipFile and CP932 encoding
http://bugs.python.org/issue10614  opened by ocean-city

#10615: Trivial mingw compile fixes
http://bugs.python.org/issue10615  opened by jonny

#10616: Change PyObject_AsCharBuffer() error message
http://bugs.python.org/issue10616  opened by haypo



Most recent 15 issues with no replies (15)
==========================================

#10615: Trivial mingw compile fixes
http://bugs.python.org/issue10615

#10612: StopTestRun exception to halt test run
http://bugs.python.org/issue10612

#10611: sys.exit() in a test causes a test run to die
http://bugs.python.org/issue10611

#10608: Add a section to Windows FAQ explaining os.symlink
http://bugs.python.org/issue10608

#10605: ElementTree documentation
http://bugs.python.org/issue10605

#10595: Adding a syslog.conf reader in syslog
http://bugs.python.org/issue10595

#10590: Parameter type error for xml.sax.parseString(string, ...)
http://bugs.python.org/issue10590

#10588: imp.find_module raises unexpected SyntaxError
http://bugs.python.org/issue10588

#10583: Encoding issue with chm help in 2.7.1
http://bugs.python.org/issue10583

#10582: PyErr_PrintEx exits silently when passed SystemExit exception
http://bugs.python.org/issue10582

#10573: Consistency in unittest assert methods: order of actual, expec
http://bugs.python.org/issue10573

#10556: test_zipimport_support mucks up with modules
http://bugs.python.org/issue10556

#10546: UTF-16-LE and UTF-16-BE support non-BMP characters
http://bugs.python.org/issue10546

#10545: remove or rewrite "Using Backslash to Continue Statements" ant
http://bugs.python.org/issue10545

#10538: PyArg_ParseTuple("s*") does not always incref object
http://bugs.python.org/issue10538



Most recent 15 issues waiting for review (15)
=============================================

#10615: Trivial mingw compile fixes
http://bugs.python.org/issue10615

#10601: sys.displayhook: use backslashreplace error handler if repr(va
http://bugs.python.org/issue10601

#10598: curses fails to import on Solaris
http://bugs.python.org/issue10598

#10589: I/O ABC docs should specify which methods have implementations
http://bugs.python.org/issue10589

#10576: Add a progress callback to gcmodule
http://bugs.python.org/issue10576

#10571: "setup.py upload --sign" broken: TypeError: 'str' does not sup
http://bugs.python.org/issue10571

#10566: gdb debugging support additions (Tools/gdb/libpython.py)
http://bugs.python.org/issue10566

#10563: Spurious newline in time.ctime
http://bugs.python.org/issue10563

#10559: NameError in tutorial/interpreter
http://bugs.python.org/issue10559

#10558: non-standard processing of several configure options ignores "
http://bugs.python.org/issue10558

#10557: Malformed error message from float()
http://bugs.python.org/issue10557

#10553: Add optimize argument to builtin compile() and byte-compilatio
http://bugs.python.org/issue10553

#10552: Tools/unicode/gencodec.py error
http://bugs.python.org/issue10552

#10550: Windows: leak in test_concurrent_futures
http://bugs.python.org/issue10550

#10546: UTF-16-LE and UTF-16-BE support non-BMP characters
http://bugs.python.org/issue10546



Top 10 most discussed issues (10)
=================================

#3243: Support iterable bodies in httplib
http://bugs.python.org/issue3243  42 msgs

#10557: Malformed error message from float()
http://bugs.python.org/issue10557  34 msgs

#10542: Py_UNICODE_NEXT and other macros for surrogates
http://bugs.python.org/issue10542  23 msgs

#10552: Tools/unicode/gencodec.py error
http://bugs.python.org/issue10552  11 msgs

#10576: Add a progress callback to gcmodule
http://bugs.python.org/issue10576  11 msgs

#9264: trace.py documentation is incomplete
http://bugs.python.org/issue9264  10 msgs

#10262: Add --soabi option to `configure`
http://bugs.python.org/issue10262  10 msgs

#10572: Move unittest test package to Lib/test
http://bugs.python.org/issue10572  10 msgs

#10544: yield expression inside generator expression does nothing
http://bugs.python.org/issue10544   8 msgs

#10551: mimetypes read from the registry should not overwrite standard
http://bugs.python.org/issue10551   8 msgs



Issues closed (64)
==================

#2001: Pydoc interactive browsing enhancement
http://bugs.python.org/issue2001  closed by ncoghlan

#4113: Add custom __repr__ to functools.partial
http://bugs.python.org/issue4113  closed by belopolsky

#4335: inspect.getsourcelines ignores last line in module
http://bugs.python.org/issue4335  closed by belopolsky

#6594: json C serializer performance tied to structure depth on some 
http://bugs.python.org/issue6594  closed by pitrou

#6628: IDLE freezes after encountering a syntax error
http://bugs.python.org/issue6628  closed by ned.deily

#6886: cgi.py runs python, not python3
http://bugs.python.org/issue6886  closed by eric.araujo

#6887: executables in lib use /usr/bin/env python, not python3
http://bugs.python.org/issue6887  closed by eric.araujo

#7830: Flatten nested functools.partial
http://bugs.python.org/issue7830  closed by belopolsky

#7911: unittest.TestCase.longMessage should default to True in Python
http://bugs.python.org/issue7911  closed by michael.foord

#8685: set(range(100000)).difference(set()) is slow
http://bugs.python.org/issue8685  closed by pitrou

#8710: test_xpickle: compat tests: workaround for missing test_suppor
http://bugs.python.org/issue8710  closed by skrah

#8805: urllib should support SSL contexts
http://bugs.python.org/issue8805  closed by pitrou

#8989: email.utils.make_msgid: specify domain
http://bugs.python.org/issue8989  closed by r.david.murray

#9198: Should repr() print unicode characters outside the BMP?
http://bugs.python.org/issue9198  closed by haypo

#9299: os.makedirs(): Add a keyword argument to suppress "File exists
http://bugs.python.org/issue9299  closed by eric.araujo

#9528: Add pure Python implementation of time module to CPython
http://bugs.python.org/issue9528  closed by belopolsky

#9573: importing a module that executes fork() raises RuntimeError
http://bugs.python.org/issue9573  closed by ncoghlan

#9598: untabify.py fails on files that contain non-ascii characters
http://bugs.python.org/issue9598  closed by belopolsky

#9873: urllib.parse: Allow bytes in some APIs that use string literal
http://bugs.python.org/issue9873  closed by ncoghlan

#9915: speeding up sorting with a key
http://bugs.python.org/issue9915  closed by stutzbach

#10047: python-2.6.6 coredump running newspipe
http://bugs.python.org/issue10047  closed by wiz

#10273: Clean-up Unittest API
http://bugs.python.org/issue10273  closed by ezio.melotti

#10323: Final state of underlying sequence in islice
http://bugs.python.org/issue10323  closed by rhettinger

#10360: _weakrefset.WeakSet.__contains__ should not propagate TypeErro
http://bugs.python.org/issue10360  closed by georg.brandl

#10383: test_os leaks under Windows
http://bugs.python.org/issue10383  closed by skrah

#10464: netrc module not parsing passwords containing #s.
http://bugs.python.org/issue10464  closed by r.david.murray

#10499: Modular interpolation in configparser
http://bugs.python.org/issue10499  closed by lukasz.langa

#10500: Palevo.DZ worm msix86 installer 3.x installer
http://bugs.python.org/issue10500  closed by loewis

#10518: Bring back callable()
http://bugs.python.org/issue10518  closed by pitrou

#10532: A bug related to matching the empty string
http://bugs.python.org/issue10532  closed by ned.deily

#10539: Regular expression not checking 'range' element on 1st char in
http://bugs.python.org/issue10539  closed by georg.brandl

#10547: FreeBSD: wrong value for LDSHARED in sysconfig
http://bugs.python.org/issue10547  closed by skrah

#10549: help(cls1) breaks when cls1 has staticmethod(cls2) attribute
http://bugs.python.org/issue10549  closed by georg.brandl

#10554: Context management support for subprocess.Popen
http://bugs.python.org/issue10554  closed by eric.araujo

#10555: AIX 5.3 - GCC - Python 2.7 Shared Library Support - Fatal Pyth
http://bugs.python.org/issue10555  closed by Anurag.Chourasia

#10561: The pdb command 'clear bpnumber' may delete more than one brea
http://bugs.python.org/issue10561  closed by orsenthil

#10562: Change 'j' for imaginary unit into an 'i'
http://bugs.python.org/issue10562  closed by mark.dickinson

#10564: maths float error
http://bugs.python.org/issue10564  closed by mark.dickinson

#10565: isinstance(x, collections.Iterator) can return True, when x is
http://bugs.python.org/issue10565  closed by belopolsky

#10567: Unicode space character \u200b unrecognised a space
http://bugs.python.org/issue10567  closed by belopolsky

#10568: Python and the Unicode Character Database
http://bugs.python.org/issue10568  closed by belopolsky

#10569: abc: `issubclass([], my_abstract_type)` raises exception
http://bugs.python.org/issue10569  closed by amaury.forgeotdarc

#10575: makeunicodedata.py does not support Unihan digit data
http://bugs.python.org/issue10575  closed by loewis

#10578: Add mock PyPI server to test distutils
http://bugs.python.org/issue10578  closed by eric.araujo

#10579: Is ``o[key]`` equivalent to PyMapping_HasKeyString?
http://bugs.python.org/issue10579  closed by naoki

#10584: Bad links in doc of zipfile.open
http://bugs.python.org/issue10584  closed by georg.brandl

#10585: 2.6 maintenance branch erroneously uses unittest.TestCase.asse
http://bugs.python.org/issue10585  closed by barry

#10586: Enhanced cache access API for functools.lru_cache
http://bugs.python.org/issue10586  closed by ncoghlan

#10591: test_os failure in refleak runs
http://bugs.python.org/issue10591  closed by brian.curtin

#10593: LRU Cache with maxsize=None
http://bugs.python.org/issue10593  closed by rhettinger

#10594: Typo in PyList_New doc.
http://bugs.python.org/issue10594  closed by georg.brandl

#10597: Py_SetPythonHome document shows same url twice.
http://bugs.python.org/issue10597  closed by georg.brandl

#10600: surrogateescape'd paths not readable on Windows XP.
http://bugs.python.org/issue10600  closed by loewis

#10602: Wrong assert* method in test_csv.test_register_kwargs masks er
http://bugs.python.org/issue10602  closed by eric.araujo

#10603: __file__ not usable, with certain paths on windows XP.
http://bugs.python.org/issue10603  closed by r.david.murray

#10604: Allocating too much memory for pathes
http://bugs.python.org/issue10604  closed by georg.brandl

#10606: Misspelling of Jamie Zawinski's surname in urllib.parse docstr
http://bugs.python.org/issue10606  closed by dmalcolm

#10607: Document of PyOS_(v)snprintf is wrong.
http://bugs.python.org/issue10607  closed by naoki

#10613: gzip._PaddedFile calls getattr with arguments in reversed orde
http://bugs.python.org/issue10613  closed by eric.araujo

#940286: pydoc.Helper.help() ignores input/output init parameters
http://bugs.python.org/issue940286  closed by georg.brandl

#1745035: DoS smtpd vulnerability
http://bugs.python.org/issue1745035  closed by georg.brandl

#1486713: HTMLParser : A auto-tolerant parsing mode
http://bugs.python.org/issue1486713  closed by r.david.murray

#975556: HTMLParser lukewarm on bogus bare attribute chars
http://bugs.python.org/issue975556  closed by r.david.murray

#1046092: HTMLParser fix to accept malformed tag attributes
http://bugs.python.org/issue1046092  closed by r.david.murray

From g.brandl at gmx.net  Fri Dec  3 18:09:39 2010
From: g.brandl at gmx.net (Georg Brandl)
Date: Fri, 03 Dec 2010 18:09:39 +0100
Subject: [Python-Dev] "buffer interface" messages
In-Reply-To: <20101203175722.6124d0a6@pitrou.net>
References: <20101202180651.F2AB0EE9CD@mail.python.org>	<201012031016.05063.victor.stinner@haypocalc.com>	<20101203151129.74C5222A88F@kimball.webabinitio.net>	<20101203172814.6aa8ba0e@pitrou.net>	<AANLkTikn8KpffwYKOxHS_kCazah05hfztJ7CDBh9iJW=@mail.gmail.com>
	<20101203175722.6124d0a6@pitrou.net>
Message-ID: <idb8cn$vjo$1@dough.gmane.org>

Am 03.12.2010 17:57, schrieb Antoine Pitrou:
> On Sat, 4 Dec 2010 02:45:42 +1000
> Nick Coghlan <ncoghlan at gmail.com> wrote:
>> On Sat, Dec 4, 2010 at 2:28 AM, Antoine Pitrou <solipsis at pitrou.net> wrote:
>> > On Fri, 03 Dec 2010 10:11:29 -0500
>> > "R. David Murray" <rdmurray at bitdance.com> wrote:
>> >> >
>> >> > >>> 'abc'.transform('hex')
>> >> > TypeError: 'str' does not support the buffer interface
>> >> > >>> b'abc'.transform('rot13')
>> >> > TypeError: expected an object with the buffer interface
>> >>
>> >> I find these 'buffer interface' error messages to be the most confusing
>> >> error message I get out of Python3 no matter what context they show up
>> >> in.  I have no idea what they are telling me.  That issue is more
>> >> general than transform/untransform, but perhaps it could be fixed
>> >> for transform/untransform in particular.
>> >
>> > I agree. "buffer interface" is a technicality that the Python user
>> > doesn't do about (unless (s)he also writes C extensions). How about
>> > "expected a bytes-compatible object"?
>> 
>> Why not "binary data interface"? That's what they're actually looking for.
> 
> I don't think it's more understandable, since it's not a well-known
> Python concept.
> (and in this specific case, the transform() method is on str and bytes,
> not on arbitrary bytes-like objects)

"object that can be handled as bytes"?
"object that provides bytes"?

>> It seems odd for 'rot13' to be throwing that error though.
> 
> Agreed.

rot13 is a str<->str codec.

Georg


From solipsis at pitrou.net  Fri Dec  3 18:29:23 2010
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Fri, 3 Dec 2010 18:29:23 +0100
Subject: [Python-Dev] "buffer interface" messages
References: <20101202180651.F2AB0EE9CD@mail.python.org>
	<201012031016.05063.victor.stinner@haypocalc.com>
	<20101203151129.74C5222A88F@kimball.webabinitio.net>
	<20101203172814.6aa8ba0e@pitrou.net>
	<AANLkTikn8KpffwYKOxHS_kCazah05hfztJ7CDBh9iJW=@mail.gmail.com>
	<20101203175722.6124d0a6@pitrou.net> <idb8cn$vjo$1@dough.gmane.org>
Message-ID: <20101203182923.500be9da@pitrou.net>

On Fri, 03 Dec 2010 18:09:39 +0100
Georg Brandl <g.brandl at gmx.net> wrote:
> Am 03.12.2010 17:57, schrieb Antoine Pitrou:
> > On Sat, 4 Dec 2010 02:45:42 +1000
> > Nick Coghlan <ncoghlan at gmail.com> wrote:
> >> On Sat, Dec 4, 2010 at 2:28 AM, Antoine Pitrou <solipsis at pitrou.net> wrote:
> >> > On Fri, 03 Dec 2010 10:11:29 -0500
> >> > "R. David Murray" <rdmurray at bitdance.com> wrote:
> >> >> >
> >> >> > >>> 'abc'.transform('hex')
> >> >> > TypeError: 'str' does not support the buffer interface
> >> >> > >>> b'abc'.transform('rot13')
> >> >> > TypeError: expected an object with the buffer interface
> >> >>
> >> >> I find these 'buffer interface' error messages to be the most confusing
> >> >> error message I get out of Python3 no matter what context they show up
> >> >> in.  I have no idea what they are telling me.  That issue is more
> >> >> general than transform/untransform, but perhaps it could be fixed
> >> >> for transform/untransform in particular.
> >> >
> >> > I agree. "buffer interface" is a technicality that the Python user
> >> > doesn't do about (unless (s)he also writes C extensions). How about
> >> > "expected a bytes-compatible object"?
> >> 
> >> Why not "binary data interface"? That's what they're actually looking for.
> > 
> > I don't think it's more understandable, since it's not a well-known
> > Python concept.
> > (and in this specific case, the transform() method is on str and bytes,
> > not on arbitrary bytes-like objects)
> 
> "object that can be handled as bytes"?
> "object that provides bytes"?
> 
> >> It seems odd for 'rot13' to be throwing that error though.
> > 
> > Agreed.
> 
> rot13 is a str<->str codec.

Then why does it claim to expect an object with the buffer interface?
bytes has the buffer interface, while str doesn't ;)

I'll also mention that getting a TypeError when you call a method with
the correct type of argument is a bit surprising.

cheers

Antoine.



From g.brandl at gmx.net  Fri Dec  3 18:45:08 2010
From: g.brandl at gmx.net (Georg Brandl)
Date: Fri, 03 Dec 2010 18:45:08 +0100
Subject: [Python-Dev] "buffer interface" messages
In-Reply-To: <20101203182923.500be9da@pitrou.net>
References: <20101202180651.F2AB0EE9CD@mail.python.org>	<201012031016.05063.victor.stinner@haypocalc.com>	<20101203151129.74C5222A88F@kimball.webabinitio.net>	<20101203172814.6aa8ba0e@pitrou.net>	<AANLkTikn8KpffwYKOxHS_kCazah05hfztJ7CDBh9iJW=@mail.gmail.com>	<20101203175722.6124d0a6@pitrou.net>
	<idb8cn$vjo$1@dough.gmane.org> <20101203182923.500be9da@pitrou.net>
Message-ID: <idbaf8$al9$1@dough.gmane.org>

Am 03.12.2010 18:29, schrieb Antoine Pitrou:
> On Fri, 03 Dec 2010 18:09:39 +0100
> Georg Brandl <g.brandl at gmx.net> wrote:
>> Am 03.12.2010 17:57, schrieb Antoine Pitrou:
>> > On Sat, 4 Dec 2010 02:45:42 +1000
>> > Nick Coghlan <ncoghlan at gmail.com> wrote:
>> >> On Sat, Dec 4, 2010 at 2:28 AM, Antoine Pitrou <solipsis at pitrou.net> wrote:
>> >> > On Fri, 03 Dec 2010 10:11:29 -0500
>> >> > "R. David Murray" <rdmurray at bitdance.com> wrote:
>> >> >> >
>> >> >> > >>> 'abc'.transform('hex')
>> >> >> > TypeError: 'str' does not support the buffer interface
>> >> >> > >>> b'abc'.transform('rot13')
>> >> >> > TypeError: expected an object with the buffer interface
>> >> >>
>> >> >> I find these 'buffer interface' error messages to be the most confusing
>> >> >> error message I get out of Python3 no matter what context they show up
>> >> >> in.  I have no idea what they are telling me.  That issue is more
>> >> >> general than transform/untransform, but perhaps it could be fixed
>> >> >> for transform/untransform in particular.
>> >> >
>> >> > I agree. "buffer interface" is a technicality that the Python user
>> >> > doesn't do about (unless (s)he also writes C extensions). How about
>> >> > "expected a bytes-compatible object"?
>> >> 
>> >> Why not "binary data interface"? That's what they're actually looking for.
>> > 
>> > I don't think it's more understandable, since it's not a well-known
>> > Python concept.
>> > (and in this specific case, the transform() method is on str and bytes,
>> > not on arbitrary bytes-like objects)
>> 
>> "object that can be handled as bytes"?
>> "object that provides bytes"?
>> 
>> >> It seems odd for 'rot13' to be throwing that error though.
>> > 
>> > Agreed.
>> 
>> rot13 is a str<->str codec.
> 
> Then why does it claim to expect an object with the buffer interface?
> bytes has the buffer interface, while str doesn't ;)

I'll leave it to you to figure it out.  (Hint: look at the traceback.)

> I'll also mention that getting a TypeError when you call a method with
> the correct type of argument is a bit surprising.

Why? The codec has the wrong type :)  But seriously, MAL already proposed
a bit more information on codecs, so that they can let it be known which
types they convert between.

Georg


From rdmurray at bitdance.com  Fri Dec  3 18:58:32 2010
From: rdmurray at bitdance.com (R. David Murray)
Date: Fri, 03 Dec 2010 12:58:32 -0500
Subject: [Python-Dev] transform() and untransform() methods,
	and the codec registry
In-Reply-To: <AANLkTiktmxr_eH-S9ihcwFoqd6yJeN03Hes_urH7Qt3A@mail.gmail.com>
References: <20101202180651.F2AB0EE9CD@mail.python.org>
	<201012031016.05063.victor.stinner@haypocalc.com>
	<20101203151129.74C5222A88F@kimball.webabinitio.net>
	<AANLkTiktmxr_eH-S9ihcwFoqd6yJeN03Hes_urH7Qt3A@mail.gmail.com>
Message-ID: <20101203175832.88F00236D8B@kimball.webabinitio.net>

On Fri, 03 Dec 2010 11:14:56 -0500, Alexander Belopolsky <alexander.belopolsky at gmail.com> wrote:
> On Fri, Dec 3, 2010 at 10:11 AM, R. David Murray <rdmurray at bitdance.com> wrote:
> ..
> > Please also recall that transform/untransform was discussed before
> > the release of Python 3.0 and was approved at the time, but it just
> > did not get implemented before the 3.0 release.
> >
> 
> Can you provide a link?  My search for transform on python-dev came out with

It was linked from the issue, if I recall correctly.  I do remember
reading the thread from the python-3000 list, linked by someone
somewhere :)

> http://mail.python.org/pipermail/python-dev/2010-June/100564.html
> 
> where you seem to oppose these methods.  Also, new methods to builtins

It looks to me like I was agreeing that transform/untrasnform should
do only bytes->bytes or str->str regardless of what codec name you
passed them.

> fall under the language moratorium (but can be approved on a
> case-by-case basis):
> 
> http://www.python.org/dev/peps/pep-3003/#case-by-case-exemptions
> 
> Is there an effort to document these exceptions?  I expected such
> approvals to be added to PEP 3003, but apparently this was not the
> case.

I believe MAL's thought was that the addition of these methods had
been approved pre-moratorium, but I don't know if that is a
sufficient argument or not.

--
R. David Murray                                      www.bitdance.com

From guido at python.org  Fri Dec  3 19:05:32 2010
From: guido at python.org (Guido van Rossum)
Date: Fri, 3 Dec 2010 10:05:32 -0800
Subject: [Python-Dev] transform() and untransform() methods,
	and the codec registry
In-Reply-To: <20101203175832.88F00236D8B@kimball.webabinitio.net>
References: <20101202180651.F2AB0EE9CD@mail.python.org>
	<201012031016.05063.victor.stinner@haypocalc.com>
	<20101203151129.74C5222A88F@kimball.webabinitio.net>
	<AANLkTiktmxr_eH-S9ihcwFoqd6yJeN03Hes_urH7Qt3A@mail.gmail.com>
	<20101203175832.88F00236D8B@kimball.webabinitio.net>
Message-ID: <AANLkTikF2w-mmNVgYch=40Rmdc_OvRV3wOYzthvNBbpO@mail.gmail.com>

On Fri, Dec 3, 2010 at 9:58 AM, R. David Murray <rdmurray at bitdance.com> wrote:
> On Fri, 03 Dec 2010 11:14:56 -0500, Alexander Belopolsky <alexander.belopolsky at gmail.com> wrote:
>> On Fri, Dec 3, 2010 at 10:11 AM, R. David Murray <rdmurray at bitdance.com> wrote:
>> ..
>> > Please also recall that transform/untransform was discussed before
>> > the release of Python 3.0 and was approved at the time, but it just
>> > did not get implemented before the 3.0 release.
>> >
>>
>> Can you provide a link? ?My search for transform on python-dev came out with
>
> It was linked from the issue, if I recall correctly. ?I do remember
> reading the thread from the python-3000 list, linked by someone
> somewhere :)
>
>> http://mail.python.org/pipermail/python-dev/2010-June/100564.html
>>
>> where you seem to oppose these methods. ?Also, new methods to builtins
>
> It looks to me like I was agreeing that transform/untrasnform should
> do only bytes->bytes or str->str regardless of what codec name you
> passed them.
>
>> fall under the language moratorium (but can be approved on a
>> case-by-case basis):
>>
>> http://www.python.org/dev/peps/pep-3003/#case-by-case-exemptions
>>
>> Is there an effort to document these exceptions? ?I expected such
>> approvals to be added to PEP 3003, but apparently this was not the
>> case.
>
> I believe MAL's thought was that the addition of these methods had
> been approved pre-moratorium, but I don't know if that is a
> sufficient argument or not.

It is not.

The moratorium is intended to freeze the state of the language as
implemented, not whatever was discussed and approved but didn't get
implemented (that'd be a hole big enough to drive a truck through, as
the saying goes :-).

Regardless of what I or others may have said before, I am not
currently a fan of adding transform() to either str or bytes.

-- 
--Guido van Rossum (python.org/~guido)

From contactprashantat at gmail.com  Fri Dec  3 19:25:10 2010
From: contactprashantat at gmail.com (Prashant Kumar)
Date: Fri, 3 Dec 2010 23:55:10 +0530
Subject: [Python-Dev] Porting Ideas
In-Reply-To: <4CF85850.2070305@netwok.org>
References: <AANLkTing_iT7Y6FOZPWh=CUwtK8owvYtSdE7aXf0W181@mail.gmail.com>
	<4CF85850.2070305@netwok.org>
Message-ID: <AANLkTi=pKiZGWh5n_+yoVqbymgrY0Dhmc55ecWSihEjj@mail.gmail.com>

On 12/3/10, ?ric Araujo <merwok at netwok.org> wrote:
> Hi Prashant,
>
> Python 3 support in distutils2 is not entirely finished, it?s an
> interesting and challenging task.
>
> Another idea: convert the python.org internal scripts to use Python 3,
> for example starting with patches for http://code.python.org/hg/peps/ .
> This would not have any impact on the community, but it?s easy work
> that?d help the Python developers to eat their own dogfood.
>
> Regards
>
>
     Thanks for the suggestion.

From alexander.belopolsky at gmail.com  Fri Dec  3 19:28:08 2010
From: alexander.belopolsky at gmail.com (Alexander Belopolsky)
Date: Fri, 3 Dec 2010 13:28:08 -0500
Subject: [Python-Dev] transform() and untransform() methods,
	and the codec registry
In-Reply-To: <201012031016.05063.victor.stinner@haypocalc.com>
References: <20101202180651.F2AB0EE9CD@mail.python.org>
	<201012031016.05063.victor.stinner@haypocalc.com>
Message-ID: <AANLkTikoRZJ4m3oGBhzgJ1_8WoJWf6ik_jm_uPt600pc@mail.gmail.com>

On Fri, Dec 3, 2010 at 4:16 AM, Victor Stinner
<victor.stinner at haypocalc.com> wrote:
..
> I don't like transform() and untransform() because I think that we should not
> add too much operations to the base types (bytes and str), and they do
> implicit module import. I prefer explicit module import (eg. import binascii;
> binascii.hexlify(b'to hex')).

+1

Implicit imports are currently subtly broken with no solution in sight.  See

http://bugs.python.org/issue8098
http://bugs.python.org/issue7980

In fact, once the language moratorium is over, I will argue that
str.encode() and byte.decode() should deprecate encoding argument and
just do UTF-8 encoding/decoding.  Hopefully by that time most people
will forget that other encodings exist.  (I can dream, right?)  Python
can still provide the codec machinery but not tie it to str/bytes
builtins.  After all, neither encode/decode nor transform/unstransform
fully utilize the power of codec design.

From brett at python.org  Fri Dec  3 20:22:18 2010
From: brett at python.org (Brett Cannon)
Date: Fri, 3 Dec 2010 11:22:18 -0800
Subject: [Python-Dev] [Python-checkins] r86945 -
	python/branches/py3k/Lib/test/__main__.py
In-Reply-To: <20101203013401.A454BEE981@mail.python.org>
References: <20101203013401.A454BEE981@mail.python.org>
Message-ID: <AANLkTimqLv3qx_oSzTu8E3cjJ19_DwLR3pqGbx+SRaUW@mail.gmail.com>

I just want to say thanks for doing this, Michael. __main__.py is IMO
woefully underused and it's great to see Python dogfooding the feature
along with making it easier to explain how to run our unit tests.

On Thu, Dec 2, 2010 at 17:34, michael.foord <python-checkins at python.org> wrote:
> Author: michael.foord
> Date: Fri Dec ?3 02:34:01 2010
> New Revision: 86945
>
> Log:
> Initial implementation of Lib/test/__main__.py so we can run tests with 'python -m test'
>
> Added:
> ? python/branches/py3k/Lib/test/__main__.py ? (contents, props changed)
>
> Added: python/branches/py3k/Lib/test/__main__.py
> ==============================================================================
> --- (empty file)
> +++ python/branches/py3k/Lib/test/__main__.py ? Fri Dec ?3 02:34:01 2010
> @@ -0,0 +1,38 @@
> +import os
> +import sys
> +import sysconfig
> +
> +from test import support
> +from test.regrtest import main
> +
> +# findtestdir() gets the dirname out of __file__, so we have to make it
> +# absolute before changing the working directory.
> +# For example __file__ may be relative when running trace or profile.
> +# See issue #9323.
> +__file__ = os.path.abspath(__file__)
> +
> +# sanity check
> +assert __file__ == os.path.abspath(sys.argv[0])
> +
> +# When tests are run from the Python build directory, it is best practice
> +# to keep the test files in a subfolder. ?It eases the cleanup of leftover
> +# files using command "make distclean".
> +if sysconfig.is_python_build():
> + ? ?TEMPDIR = os.path.join(sysconfig.get_config_var('srcdir'), 'build')
> + ? ?TEMPDIR = os.path.abspath(TEMPDIR)
> + ? ?if not os.path.exists(TEMPDIR):
> + ? ? ? ?os.mkdir(TEMPDIR)
> +
> +# Define a writable temp dir that will be used as cwd while running
> +# the tests. The name of the dir includes the pid to allow parallel
> +# testing (see the -j option).
> +TESTCWD = 'test_python_{}'.format(os.getpid())
> +
> +TESTCWD = os.path.join(TEMPDIR, TESTCWD)
> +
> +# Run the tests in a context manager that temporary changes the CWD to a
> +# temporary and writable directory. If it's not possible to create or
> +# change the CWD, the original CWD will be used. The original CWD is
> +# available from support.SAVEDCWD.
> +with support.temp_cwd(TESTCWD, quiet=True):
> + ? ?main()
> _______________________________________________
> Python-checkins mailing list
> Python-checkins at python.org
> http://mail.python.org/mailman/listinfo/python-checkins
>

From martin at v.loewis.de  Fri Dec  3 23:39:19 2010
From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=)
Date: Fri, 03 Dec 2010 23:39:19 +0100
Subject: [Python-Dev] PEP 384 accepted
In-Reply-To: <4CF8E22B.7050005@netwok.org>
References: <AANLkTimQsiU+nyX6CdqQ8JBybJdnx=1EMxX9mefHbx9H@mail.gmail.com>	<4CF8008B.60402@v.loewis.de>	<AANLkTikbRJEFYE_TvN++GYhmvSUu3hdhf4znOti7oty1@mail.gmail.com>	<4CF80964.4070300@v.loewis.de>	<AANLkTikcT0WGo3FhFVq=N6ktToNenBDNNcC4eSpefcFF@mail.gmail.com>	<4CF80D6C.6050703@v.loewis.de>	<AANLkTi=s5NMxjPdRQRcupNBzdqjMhLO6TbFWtQzi0TEw@mail.gmail.com>	<4CF81208.8090705@v.loewis.de>	<AANLkTinspUo88oUnd3cX776mMRMU6bqyOaq9Og7JLwhU@mail.gmail.com>	<4CF81AE1.5050900@v.loewis.de>
	<4CF880F7.9000800@netwok.org> <4CF89CCF.20307@v.loewis.de>
	<4CF8E22B.7050005@netwok.org>
Message-ID: <4CF97197.2050105@v.loewis.de>

>> For example, I keep running into the issue that distutils doesn't
>> currently support parallel builds. I have been pondering supporting
>> "-j" for building extensions, using both unbounded "-j" and the GNU make
>> style -jN build server. However, I know that the patch will be rejected,
>> so I don't even start working on it.
> This would be a very useful feature for distutils2.

But I'm not interested at all in having it in distutils2. I want the
Python build itself to use it, and alas, I can't because of the freeze.

Regards,
Martin

From merwok at netwok.org  Fri Dec  3 23:48:48 2010
From: merwok at netwok.org (=?UTF-8?B?w4lyaWMgQXJhdWpv?=)
Date: Fri, 03 Dec 2010 23:48:48 +0100
Subject: [Python-Dev] PEP 384 accepted
In-Reply-To: <4CF97197.2050105@v.loewis.de>
References: <AANLkTimQsiU+nyX6CdqQ8JBybJdnx=1EMxX9mefHbx9H@mail.gmail.com>	<4CF8008B.60402@v.loewis.de>	<AANLkTikbRJEFYE_TvN++GYhmvSUu3hdhf4znOti7oty1@mail.gmail.com>	<4CF80964.4070300@v.loewis.de>	<AANLkTikcT0WGo3FhFVq=N6ktToNenBDNNcC4eSpefcFF@mail.gmail.com>	<4CF80D6C.6050703@v.loewis.de>	<AANLkTi=s5NMxjPdRQRcupNBzdqjMhLO6TbFWtQzi0TEw@mail.gmail.com>	<4CF81208.8090705@v.loewis.de>	<AANLkTinspUo88oUnd3cX776mMRMU6bqyOaq9Og7JLwhU@mail.gmail.com>	<4CF81AE1.5050900@v.loewis.de>
	<4CF880F7.9000800@netwok.org> <4CF89CCF.20307@v.loewis.de>
	<4CF8E22B.7050005@netwok.org> <4CF97197.2050105@v.loewis.de>
Message-ID: <4CF973D0.4060800@netwok.org>

> But I'm not interested at all in having it in distutils2. I want the
> Python build itself to use it, and alas, I can't because of the freeze.
You can?t in 3.2, true.  Neither can you in 3.1, or any previous
version.  If you implement it in distutils2, you have very good chances
to get it for 3.3.  Isn?t that a win?

(BTW: congratulations on having PEP 384 accepted and merged.)

Regards


From martin at v.loewis.de  Fri Dec  3 23:52:41 2010
From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=)
Date: Fri, 03 Dec 2010 23:52:41 +0100
Subject: [Python-Dev] PEP 384 accepted
In-Reply-To: <4CF973D0.4060800@netwok.org>
References: <AANLkTimQsiU+nyX6CdqQ8JBybJdnx=1EMxX9mefHbx9H@mail.gmail.com>	<4CF8008B.60402@v.loewis.de>	<AANLkTikbRJEFYE_TvN++GYhmvSUu3hdhf4znOti7oty1@mail.gmail.com>	<4CF80964.4070300@v.loewis.de>	<AANLkTikcT0WGo3FhFVq=N6ktToNenBDNNcC4eSpefcFF@mail.gmail.com>	<4CF80D6C.6050703@v.loewis.de>	<AANLkTi=s5NMxjPdRQRcupNBzdqjMhLO6TbFWtQzi0TEw@mail.gmail.com>	<4CF81208.8090705@v.loewis.de>	<AANLkTinspUo88oUnd3cX776mMRMU6bqyOaq9Og7JLwhU@mail.gmail.com>	<4CF81AE1.5050900@v.loewis.de>
	<4CF880F7.9000800@netwok.org> <4CF89CCF.20307@v.loewis.de>
	<4CF8E22B.7050005@netwok.org> <4CF97197.2050105@v.loewis.de>
	<4CF973D0.4060800@netwok.org>
Message-ID: <4CF974B9.6040509@v.loewis.de>

Am 03.12.2010 23:48, schrieb ?ric Araujo:
>> But I'm not interested at all in having it in distutils2. I want the
>> Python build itself to use it, and alas, I can't because of the freeze.
> You can?t in 3.2, true.  Neither can you in 3.1, or any previous
> version.  If you implement it in distutils2, you have very good chances
> to get it for 3.3.  Isn?t that a win?

It is, unfortunately, a very weak promise. Until distutils2 is
integrated in Python, I probably won't spend any time on it.

Regards,
Martin

From dimaqq at gmail.com  Fri Dec  3 23:55:41 2010
From: dimaqq at gmail.com (Dima Tisnek)
Date: Fri, 3 Dec 2010 15:55:41 -0700
Subject: [Python-Dev] gc ideas -- sparse memory
Message-ID: <AANLkTi=AchQA36yTgHrqiDovCaR1diYvRjXTg4j5P_Ni@mail.gmail.com>

How hard or reasonable would it be to free memory pages on OS level?

[pcmiiw] Gabage collection within a generation involves moving live
objects to compact the generation storage. This separates the memory
region into 2 parts "live" and "cleared", the pointer to the beginning
of the "cleared" part is where next allocation is going to happen.

When this is done, does Python gc move the objects preserving order or
does it try to populate garbaged slot with some live object
disregarding order? Earlier case is more applicable, but latter case
is a target for below too.

If we were to look at memory regions from the OS point of view, they
are allocated as pages (or possibly as hugetlb pages). So if we are to
compact something like this [LL__][_L__][____][L_LL][LFFF]  where []
is a page, L is live object and _ is garbage and F is free memory,
would it not make more sense to tell OS that [____] is not needed
anymore, and not move some of the consequtive [L_LL][LFFF] at all, or
at least not move those objects as far down the memory region?

This would of course have a certain overhead of tracking which pages
are given back to OS and mapping them back when needed, but at the
same time, be beneficial because fewer objets are moved and also
possibly improve cpu cache performance because objects won't be moved
so far out.

p.s. if someone has an athoritative link to modern python gc design,
please let me know.

From tjreedy at udel.edu  Sat Dec  4 00:04:58 2010
From: tjreedy at udel.edu (Terry Reedy)
Date: Fri, 03 Dec 2010 18:04:58 -0500
Subject: [Python-Dev] gc ideas -- sparse memory
In-Reply-To: <AANLkTi=AchQA36yTgHrqiDovCaR1diYvRjXTg4j5P_Ni@mail.gmail.com>
References: <AANLkTi=AchQA36yTgHrqiDovCaR1diYvRjXTg4j5P_Ni@mail.gmail.com>
Message-ID: <idbt2p$1jg$1@dough.gmane.org>

On 12/3/2010 5:55 PM, Dima Tisnek wrote:
> How hard or reasonable would it be to free memory pages on OS level?
>
> [pcmiiw] Gabage collection within a generation involves moving live
> objects to compact the generation storage. This separates the memory
> region into 2 parts "live" and "cleared", the pointer to the beginning
> of the "cleared" part is where next allocation is going to happen.
>
> When this is done, does Python gc move the objects preserving order or
> does it try to populate garbaged slot with some live object
> disregarding order? Earlier case is more applicable, but latter case
> is a target for below too.
>
> If we were to look at memory regions from the OS point of view, they
> are allocated as pages (or possibly as hugetlb pages). So if we are to
> compact something like this [LL__][_L__][____][L_LL][LFFF]  where []
> is a page, L is live object and _ is garbage and F is free memory,
> would it not make more sense to tell OS that [____] is not needed
> anymore, and not move some of the consequtive [L_LL][LFFF] at all, or
> at least not move those objects as far down the memory region?
>
> This would of course have a certain overhead of tracking which pages
> are given back to OS and mapping them back when needed, but at the
> same time, be beneficial because fewer objets are moved and also
> possibly improve cpu cache performance because objects won't be moved
> so far out.
>
> p.s. if someone has an athoritative link to modern python gc design,
> please let me know.

gc is implementation specific. CPython uses ref counting + cycle gc. A 
constraint on all implementations is that objects have a fixed, unique 
id during their lifetime. CPython uses the address as the id, so it 
cannot move objects. Other implementations do differently. Compacting gc 
requires an id to current address table or something.

-- 
Terry Jan Reedy


From lukasz at langa.pl  Fri Dec  3 23:45:33 2010
From: lukasz at langa.pl (=?utf-8?Q?=C5=81ukasz_Langa?=)
Date: Fri, 3 Dec 2010 23:45:33 +0100
Subject: [Python-Dev] [Python-checkins] r86976 - in
	python/branches/py3k: Doc/library/configparser.rst
	Doc/library/fileformats.rst Lib/configparser.py
	Lib/test/test_cfgparser.py Misc/NEWS
In-Reply-To: <4CF9386E.4000205@netwok.org>
References: <20101203162800.9A740EE9B9@mail.python.org>
	<4CF9386E.4000205@netwok.org>
Message-ID: <5CE836CF-4A20-4F50-97D5-91E35E194004@langa.pl>

Wiadomo?? napisana przez ?ric Araujo w dniu 2010-12-03, o godz. 19:35:

> Hello,
> 
>> Author: lukasz.langa
>> New Revision: 86976
>> Log: Issue 10499: Modular interpolation in configparser
>> 
>> Modified: python/branches/py3k/Doc/library/configparser.rst
> Is the module still backward compatible with the 3.1 version, modulo
> fixed bugs?  I haven?t been able to follow closely all the great
> improvements you?ve been making, and there has been a lot of movement in
> the code, so I?m not sure.

There have been minor incompatibilities, all have been thoroughly discussed with Fred and other developers. No changes should cause silent breakage, though.

> Thanks for taking up configparser.  Maybe it will become so useful and
> extensible that you?ll have to release it on PyPI for older Pythons :)

Sure. Why not? :)

>> Modified: python/branches/py3k/Doc/library/fileformats.rst
> This looks like unrelated changes that slipped in the commit. 

As we discussed this on IRC, this is an unfortunate slip caused by me trying to wrap up documentation updates in one go. Will remember to do that separately now. Thanks!

>> Modified: python/branches/py3k/Lib/configparser.py
>> 
>>         Raise DuplicateSectionError if a section by the specified name
>>         already exists. Raise ValueError if name is DEFAULT.
>>         """
>> -        if section == self._default_section:
>> +        if section == self.default_section:
>>             raise ValueError('Invalid section name: %s' % section)
> I think it?s the only error message using %s instead of %r.  The quotes
> added by %r typically help spot names with spaces (embedded or trailing)
> and the like.

Corrected in rev 86999, thanks.

>> +        options = list(d.keys())
>> +        if raw:
>> +            return [(option, d[option])
>> +                    for option in options]
>> +        else:
>> +            return [(option, self._interpolation.before_get(self, section,
>> +                                                            option, d[option],
>> +                                                            d))
>> +                    for option in options]
> The list call seems unneeded.  Minor style point: I avoid such dangling
> arguments that don?t read great, I prefer to go to a newline after the
> opening paren:
>               return [(option, self._interpolation.before_get(
>                   self, section, option, d[option], d)) for
>                   option in options]

You're right that my formatting looks ugly but yours is not that much better either ;) How about:

        value_getter = lambda option: self._interpolation.before_get(self,
            section, option, d[option], d)
        if raw:
            value_getter = lambda option: d[option]
        return [(option, value_getter(option))
                for option in d.keys()]

-- 
Best regards,
?ukasz Langa
tel. +48 791 080 144
WWW http://lukasz.langa.pl/

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20101203/668b63f3/attachment.html>

From foom at fuhm.net  Sat Dec  4 00:11:57 2010
From: foom at fuhm.net (James Y Knight)
Date: Fri, 3 Dec 2010 18:11:57 -0500
Subject: [Python-Dev] PEP 384 accepted
In-Reply-To: <4CF974B9.6040509@v.loewis.de>
References: <AANLkTimQsiU+nyX6CdqQ8JBybJdnx=1EMxX9mefHbx9H@mail.gmail.com>	<4CF8008B.60402@v.loewis.de>	<AANLkTikbRJEFYE_TvN++GYhmvSUu3hdhf4znOti7oty1@mail.gmail.com>	<4CF80964.4070300@v.loewis.de>	<AANLkTikcT0WGo3FhFVq=N6ktToNenBDNNcC4eSpefcFF@mail.gmail.com>	<4CF80D6C.6050703@v.loewis.de>	<AANLkTi=s5NMxjPdRQRcupNBzdqjMhLO6TbFWtQzi0TEw@mail.gmail.com>	<4CF81208.8090705@v.loewis.de>	<AANLkTinspUo88oUnd3cX776mMRMU6bqyOaq9Og7JLwhU@mail.gmail.com>	<4CF81AE1.5050900@v.loewis.de>
	<4CF880F7.9000800@netwok.org> <4CF89CCF.20307@v.loewis.de>
	<4CF8E22B.7050005@netwok.org> <4CF97197.2050105@v.loewis.de>
	<4CF973D0.4060800@netwok.org> <4CF974B9.6040509@v.loewis.de>
Message-ID: <294C7411-A987-45CE-ACE1-6DEFB17DA0C6@fuhm.net>

On Dec 3, 2010, at 5:52 PM, Martin v. L?wis wrote:

> Am 03.12.2010 23:48, schrieb ?ric Araujo:
>>> But I'm not interested at all in having it in distutils2. I want the
>>> Python build itself to use it, and alas, I can't because of the freeze.
>> You can?t in 3.2, true.  Neither can you in 3.1, or any previous
>> version.  If you implement it in distutils2, you have very good chances
>> to get it for 3.3.  Isn?t that a win?
> 
> It is, unfortunately, a very weak promise. Until distutils2 is
> integrated in Python, I probably won't spend any time on it.

It seems like it'd be a good idea to start integrating distutils2 into python trunk immediately after the 3.2 branch is cut, then.

James


From dimaqq at gmail.com  Sat Dec  4 00:13:34 2010
From: dimaqq at gmail.com (Dima Tisnek)
Date: Fri, 3 Dec 2010 16:13:34 -0700
Subject: [Python-Dev] gc ideas -- dynamic profiling
Message-ID: <AANLkTi=r-R1SfdnrJPEHw5Jhrmz2dZFFh+kiJuYXekkt@mail.gmail.com>

Python organizes objects into 3 generations, ephemeral, short- and long-lived.

When object is created it is place in ephemeral, if it lives long
enough, it is move to short-lived and so on.

q1 are generations placed in separate memory regions, or are all
generations in one memory regions and there is a pointer that
signifies the boundary between generations?

I propose to track hot spots in python, that is contexts where most of
allocations occur, and instrument these with counters that essentially
tell how often an object generated here ends up killed in ephemeral,
short-, long-lived garbage collector run or is in fac tstill alive. If
a particular allocation context creates objects that are likely to be
long-lived, allocator could skip frst 2 generations altogether
(generations are separate regions) or preload the object with high
survival count (if q1 is single region).

On the other hand, if we know where most allocations occur, we can
presume that most of these allocations are ephemeral, otherwise we run
out of memory anyway, if this is indeed so, it makes my point moot.

Implications are extra code to define context, extra pointer back to
context from every allocation (or alterntively a weakref from
allocation point to every object it generated) and real-time
accounting as to what happens to these objects.

It should be possible to approach this problem statistically, that is
instrument only every 100s object or so.

Context could be simple, e.g. bytecode operation 3 on line 45 in
module "junk", or more complex, e.g. call stack
str<-generator at 238382<-function "foo"<-function "boo"<-...; or even
patterns like ''' str<-any depth<-function "big" '''; clearly figuring
out what hotspots are is already non-trivial, the more coplex the
definition of context the more it borders downright impossible.

p.s. can anyone share modern cpython profiling results to shed some
light on how important gc optimization really is?

From foom at fuhm.net  Sat Dec  4 00:15:17 2010
From: foom at fuhm.net (James Y Knight)
Date: Fri, 3 Dec 2010 18:15:17 -0500
Subject: [Python-Dev] gc ideas -- sparse memory
In-Reply-To: <idbt2p$1jg$1@dough.gmane.org>
References: <AANLkTi=AchQA36yTgHrqiDovCaR1diYvRjXTg4j5P_Ni@mail.gmail.com>
	<idbt2p$1jg$1@dough.gmane.org>
Message-ID: <203DD6FC-D9F7-4E7D-B49C-CABC1DF7352B@fuhm.net>

On Dec 3, 2010, at 6:04 PM, Terry Reedy wrote:
> gc is implementation specific. CPython uses ref counting + cycle gc. A constraint on all implementations is that objects have a fixed, unique id during their lifetime. CPython uses the address as the id, so it cannot move objects. Other implementations do differently. Compacting gc requires an id to current address table or something.

It's somewhat unfortuante that python has this constraint, instead of the looser: "objects have a fixed id during their lifetime", which is much easier to implement, and practically as useful.

James


From martin at v.loewis.de  Sat Dec  4 00:15:48 2010
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sat, 04 Dec 2010 00:15:48 +0100
Subject: [Python-Dev] gc ideas -- sparse memory
In-Reply-To: <AANLkTi=AchQA36yTgHrqiDovCaR1diYvRjXTg4j5P_Ni@mail.gmail.com>
References: <AANLkTi=AchQA36yTgHrqiDovCaR1diYvRjXTg4j5P_Ni@mail.gmail.com>
Message-ID: <4CF97A24.5020106@v.loewis.de>

Am 03.12.2010 23:55, schrieb Dima Tisnek:
> How hard or reasonable would it be to free memory pages on OS level?

Very easy. Python already does that.

> [pcmiiw] Gabage collection within a generation involves moving live
> objects to compact the generation storage. This separates the memory
> region into 2 parts "live" and "cleared", the pointer to the beginning
> of the "cleared" part is where next allocation is going to happen.

I think you are talking about copying collectors here. This is not how
Python's garbage collection works.

> When this is done, does Python gc move the objects preserving order or
> does it try to populate garbaged slot with some live object
> disregarding order? Earlier case is more applicable, but latter case
> is a target for below too.

(C)Python's garbage collector is not moving objects at all.

> If we were to look at memory regions from the OS point of view, they
> are allocated as pages (or possibly as hugetlb pages). So if we are to
> compact something like this [LL__][_L__][____][L_LL][LFFF]  where []
> is a page, L is live object and _ is garbage and F is free memory,
> would it not make more sense to tell OS that [____] is not needed
> anymore, and not move some of the consequtive [L_LL][LFFF] at all, or
> at least not move those objects as far down the memory region?

See above. Python does no moving of objects whatsoever.

Regards,
Martin

From dimaqq at gmail.com  Sat Dec  4 00:19:44 2010
From: dimaqq at gmail.com (Dima Tisnek)
Date: Fri, 3 Dec 2010 16:19:44 -0700
Subject: [Python-Dev] gc ideas -- sparse memory
In-Reply-To: <idbt2p$1jg$1@dough.gmane.org>
References: <AANLkTi=AchQA36yTgHrqiDovCaR1diYvRjXTg4j5P_Ni@mail.gmail.com>
	<idbt2p$1jg$1@dough.gmane.org>
Message-ID: <AANLkTi=bGGh1_Z3-ey+4rbvNHHmi=SgOSUo_TQjvK_p-@mail.gmail.com>

Oh my bad, I must've confused python with some research paper.
Unique id is not so hard to make without an address.

While on this topic, what is the real need for unique ids?
Also I reckon not all objects need a unique id like this, e.g.
interned strings, simple data types and hashable and comparable
objects could perhaps survive without unique id?

On 3 December 2010 16:04, Terry Reedy <tjreedy at udel.edu> wrote:
> On 12/3/2010 5:55 PM, Dima Tisnek wrote:
>>
>> How hard or reasonable would it be to free memory pages on OS level?
>>
>> [pcmiiw] Gabage collection within a generation involves moving live
>> objects to compact the generation storage. This separates the memory
>> region into 2 parts "live" and "cleared", the pointer to the beginning
>> of the "cleared" part is where next allocation is going to happen.
>>
>> When this is done, does Python gc move the objects preserving order or
>> does it try to populate garbaged slot with some live object
>> disregarding order? Earlier case is more applicable, but latter case
>> is a target for below too.
>>
>> If we were to look at memory regions from the OS point of view, they
>> are allocated as pages (or possibly as hugetlb pages). So if we are to
>> compact something like this [LL__][_L__][____][L_LL][LFFF] ?where []
>> is a page, L is live object and _ is garbage and F is free memory,
>> would it not make more sense to tell OS that [____] is not needed
>> anymore, and not move some of the consequtive [L_LL][LFFF] at all, or
>> at least not move those objects as far down the memory region?
>>
>> This would of course have a certain overhead of tracking which pages
>> are given back to OS and mapping them back when needed, but at the
>> same time, be beneficial because fewer objets are moved and also
>> possibly improve cpu cache performance because objects won't be moved
>> so far out.
>>
>> p.s. if someone has an athoritative link to modern python gc design,
>> please let me know.
>
> gc is implementation specific. CPython uses ref counting + cycle gc. A
> constraint on all implementations is that objects have a fixed, unique id
> during their lifetime. CPython uses the address as the id, so it cannot move
> objects. Other implementations do differently. Compacting gc requires an id
> to current address table or something.
>
> --
> Terry Jan Reedy
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> http://mail.python.org/mailman/options/python-dev/dimaqq%40gmail.com
>

From martin at v.loewis.de  Sat Dec  4 00:21:52 2010
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sat, 04 Dec 2010 00:21:52 +0100
Subject: [Python-Dev] gc ideas -- dynamic profiling
In-Reply-To: <AANLkTi=r-R1SfdnrJPEHw5Jhrmz2dZFFh+kiJuYXekkt@mail.gmail.com>
References: <AANLkTi=r-R1SfdnrJPEHw5Jhrmz2dZFFh+kiJuYXekkt@mail.gmail.com>
Message-ID: <4CF97B90.8090802@v.loewis.de>

> q1 are generations placed in separate memory regions, or are all
> generations in one memory regions and there is a pointer that
> signifies the boundary between generations?

You should really start reading the source code. See Modules/gcmodule.c.

To answer your question: neither, nor. All objects are in one region,
and there is no pointer separating the generations. Instead, all objects
belonging to one generation form a double-linked list.

> I propose to track hot spots in python, that is contexts where most of
> allocations occur, and instrument these with counters that essentially
> tell how often an object generated here ends up killed in ephemeral,
> short-, long-lived garbage collector run or is in fac tstill alive. If
> a particular allocation context creates objects that are likely to be
> long-lived, allocator could skip frst 2 generations altogether
> (generations are separate regions) or preload the object with high
> survival count (if q1 is single region).

We would consider such a proposal only if you had *actual evidence*
that it improves things, rather than just having a reasoning that it
might.

Regards,
Martin

From tjreedy at udel.edu  Sat Dec  4 00:35:04 2010
From: tjreedy at udel.edu (Terry Reedy)
Date: Fri, 03 Dec 2010 18:35:04 -0500
Subject: [Python-Dev] PEP 384 accepted
In-Reply-To: <4CF974B9.6040509@v.loewis.de>
References: <AANLkTimQsiU+nyX6CdqQ8JBybJdnx=1EMxX9mefHbx9H@mail.gmail.com>	<4CF8008B.60402@v.loewis.de>	<AANLkTikbRJEFYE_TvN++GYhmvSUu3hdhf4znOti7oty1@mail.gmail.com>	<4CF80964.4070300@v.loewis.de>	<AANLkTikcT0WGo3FhFVq=N6ktToNenBDNNcC4eSpefcFF@mail.gmail.com>	<4CF80D6C.6050703@v.loewis.de>	<AANLkTi=s5NMxjPdRQRcupNBzdqjMhLO6TbFWtQzi0TEw@mail.gmail.com>	<4CF81208.8090705@v.loewis.de>	<AANLkTinspUo88oUnd3cX776mMRMU6bqyOaq9Og7JLwhU@mail.gmail.com>	<4CF81AE1.5050900@v.loewis.de>	<4CF880F7.9000800@netwok.org>
	<4CF89CCF.20307@v.loewis.de>	<4CF8E22B.7050005@netwok.org>
	<4CF97197.2050105@v.loewis.de>	<4CF973D0.4060800@netwok.org>
	<4CF974B9.6040509@v.loewis.de>
Message-ID: <idbur7$8hn$1@dough.gmane.org>

On 12/3/2010 5:52 PM, "Martin v. L?wis" wrote:
> Am 03.12.2010 23:48, schrieb ?ric Araujo:
>>> But I'm not interested at all in having it in distutils2. I want the
>>> Python build itself to use it, and alas, I can't because of the freeze.
>> You can?t in 3.2, true.  Neither can you in 3.1, or any previous
>> version.  If you implement it in distutils2, you have very good chances
>> to get it for 3.3.  Isn?t that a win?
>
> It is, unfortunately, a very weak promise. Until distutils2 is
> integrated in Python, I probably won't spend any time on it.

?ric, I have the impression from Tarek and you together that D2 is still 
in alpha only because it is not feature frozen and that it is as capable 
and stable as D1. I do not know what Martin means by 'integrate' (other 
than that he be able to use it to build Python), but if my first 
sentence is true, I cannot help but wonder whether a snapshot of D2 
could be included with 3.2, perhaps as '_distribute2' (note leading 
underscore) at least for Python's use. The doc, if any, could just say 
'Development snapshot of D2.a4 (or whatever) for building Python. Other 
uses should get the latest public release from PyPI."

-- 
Terry Jan Reedy



From martin at v.loewis.de  Sat Dec  4 00:45:27 2010
From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=)
Date: Sat, 04 Dec 2010 00:45:27 +0100
Subject: [Python-Dev] gc ideas -- sparse memory
In-Reply-To: <AANLkTi=bGGh1_Z3-ey+4rbvNHHmi=SgOSUo_TQjvK_p-@mail.gmail.com>
References: <AANLkTi=AchQA36yTgHrqiDovCaR1diYvRjXTg4j5P_Ni@mail.gmail.com>	<idbt2p$1jg$1@dough.gmane.org>
	<AANLkTi=bGGh1_Z3-ey+4rbvNHHmi=SgOSUo_TQjvK_p-@mail.gmail.com>
Message-ID: <4CF98117.5060301@v.loewis.de>

> Oh my bad, I must've confused python with some research paper.
> Unique id is not so hard to make without an address.
> 
> While on this topic, what is the real need for unique ids?

They are absolutely needed for mutable objects. For immutable ones,
it would be ok to claim that they are identical if they are equal
(assuming they support equality - which is tricky for things like NaN).

Of course, the C API has lots of assumptions that identity and address
are really the same thing.

Regards,
Martin

From martin at v.loewis.de  Sat Dec  4 00:46:49 2010
From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=)
Date: Sat, 04 Dec 2010 00:46:49 +0100
Subject: [Python-Dev] PEP 384 accepted
In-Reply-To: <idbur7$8hn$1@dough.gmane.org>
References: <AANLkTimQsiU+nyX6CdqQ8JBybJdnx=1EMxX9mefHbx9H@mail.gmail.com>	<4CF8008B.60402@v.loewis.de>	<AANLkTikbRJEFYE_TvN++GYhmvSUu3hdhf4znOti7oty1@mail.gmail.com>	<4CF80964.4070300@v.loewis.de>	<AANLkTikcT0WGo3FhFVq=N6ktToNenBDNNcC4eSpefcFF@mail.gmail.com>	<4CF80D6C.6050703@v.loewis.de>	<AANLkTi=s5NMxjPdRQRcupNBzdqjMhLO6TbFWtQzi0TEw@mail.gmail.com>	<4CF81208.8090705@v.loewis.de>	<AANLkTinspUo88oUnd3cX776mMRMU6bqyOaq9Og7JLwhU@mail.gmail.com>	<4CF81AE1.5050900@v.loewis.de>	<4CF880F7.9000800@netwok.org>	<4CF89CCF.20307@v.loewis.de>	<4CF8E22B.7050005@netwok.org>	<4CF97197.2050105@v.loewis.de>	<4CF973D0.4060800@netwok.org>	<4CF974B9.6040509@v.loewis.de>
	<idbur7$8hn$1@dough.gmane.org>
Message-ID: <4CF98169.6020106@v.loewis.de>

Am 04.12.2010 00:35, schrieb Terry Reedy:
> On 12/3/2010 5:52 PM, "Martin v. L?wis" wrote:
>> Am 03.12.2010 23:48, schrieb ?ric Araujo:
>>>> But I'm not interested at all in having it in distutils2. I want the
>>>> Python build itself to use it, and alas, I can't because of the freeze.
>>> You can?t in 3.2, true.  Neither can you in 3.1, or any previous
>>> version.  If you implement it in distutils2, you have very good chances
>>> to get it for 3.3.  Isn?t that a win?
>>
>> It is, unfortunately, a very weak promise. Until distutils2 is
>> integrated in Python, I probably won't spend any time on it.
> 
> ?ric, I have the impression from Tarek and you together that D2 is still
> in alpha only because it is not feature frozen and that it is as capable
> and stable as D1. I do not know what Martin means by 'integrate' (other
> than that he be able to use it to build Python)

That the master copy of the source code is in the Python source repository.

Regards,
Martin

From tjreedy at udel.edu  Sat Dec  4 01:00:15 2010
From: tjreedy at udel.edu (Terry Reedy)
Date: Fri, 03 Dec 2010 19:00:15 -0500
Subject: [Python-Dev] PEP 384 accepted
In-Reply-To: <4CF98169.6020106@v.loewis.de>
References: <AANLkTimQsiU+nyX6CdqQ8JBybJdnx=1EMxX9mefHbx9H@mail.gmail.com>	<4CF8008B.60402@v.loewis.de>	<AANLkTikbRJEFYE_TvN++GYhmvSUu3hdhf4znOti7oty1@mail.gmail.com>	<4CF80964.4070300@v.loewis.de>	<AANLkTikcT0WGo3FhFVq=N6ktToNenBDNNcC4eSpefcFF@mail.gmail.com>	<4CF80D6C.6050703@v.loewis.de>	<AANLkTi=s5NMxjPdRQRcupNBzdqjMhLO6TbFWtQzi0TEw@mail.gmail.com>	<4CF81208.8090705@v.loewis.de>	<AANLkTinspUo88oUnd3cX776mMRMU6bqyOaq9Og7JLwhU@mail.gmail.com>	<4CF81AE1.5050900@v.loewis.de>	<4CF880F7.9000800@netwok.org>	<4CF89CCF.20307@v.loewis.de>	<4CF8E22B.7050005@netwok.org>	<4CF97197.2050105@v.loewis.de>	<4CF973D0.4060800@netwok.org>	<4CF974B9.6040509@v.loewis.de>	<idbur7$8hn$1@dough.gmane.org>
	<4CF98169.6020106@v.loewis.de>
Message-ID: <idc0ad$cj6$1@dough.gmane.org>

On 12/3/2010 6:46 PM, "Martin v. L?wis" wrote:

>> and stable as D1. I do not know what Martin means by 'integrate' (other
>> than that he be able to use it to build Python)
>
> That the master copy of the source code is in the Python source repository.

Is a separate branch acceptible, as long as you can commit changes?
Does the meaning of 'repository' change any with the hg switch?

-- 
Terry Jan Reedy



From tjreedy at udel.edu  Sat Dec  4 01:05:19 2010
From: tjreedy at udel.edu (Terry Reedy)
Date: Fri, 03 Dec 2010 19:05:19 -0500
Subject: [Python-Dev] gc ideas -- sparse memory
In-Reply-To: <203DD6FC-D9F7-4E7D-B49C-CABC1DF7352B@fuhm.net>
References: <AANLkTi=AchQA36yTgHrqiDovCaR1diYvRjXTg4j5P_Ni@mail.gmail.com>	<idbt2p$1jg$1@dough.gmane.org>
	<203DD6FC-D9F7-4E7D-B49C-CABC1DF7352B@fuhm.net>
Message-ID: <idc0ju$fd7$1@dough.gmane.org>

On 12/3/2010 6:15 PM, James Y Knight wrote:
> On Dec 3, 2010, at 6:04 PM, Terry Reedy wrote:
>> gc is implementation specific. CPython uses ref counting + cycle
>> gc. A constraint on all implementations is that objects have a
>> fixed, unique id during their lifetime. CPython uses the address as
>> the id, so it cannot move objects. Other implementations do
>> differently. Compacting gc requires an id to current address table
>> or something.

I left out that the id must be an int.

> It's somewhat unfortuante that python has this constraint, instead of
> the looser: "objects have a fixed id during their lifetime", which is
> much easier to implement, and practically as useful.

Given that the only different between 'fixed and unique' and 'fixed' is 
the uniqueness part, I do not understand 'practically as useful'. 
Duplicate ids (in the extreme, that same for all objects) hardly seem 
useful at all.

-- 
Terry Jan Reedy


From martin at v.loewis.de  Sat Dec  4 01:41:32 2010
From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=)
Date: Sat, 04 Dec 2010 01:41:32 +0100
Subject: [Python-Dev] PEP 384 accepted
In-Reply-To: <idc0ad$cj6$1@dough.gmane.org>
References: <AANLkTimQsiU+nyX6CdqQ8JBybJdnx=1EMxX9mefHbx9H@mail.gmail.com>	<4CF8008B.60402@v.loewis.de>	<AANLkTikbRJEFYE_TvN++GYhmvSUu3hdhf4znOti7oty1@mail.gmail.com>	<4CF80964.4070300@v.loewis.de>	<AANLkTikcT0WGo3FhFVq=N6ktToNenBDNNcC4eSpefcFF@mail.gmail.com>	<4CF80D6C.6050703@v.loewis.de>	<AANLkTi=s5NMxjPdRQRcupNBzdqjMhLO6TbFWtQzi0TEw@mail.gmail.com>	<4CF81208.8090705@v.loewis.de>	<AANLkTinspUo88oUnd3cX776mMRMU6bqyOaq9Og7JLwhU@mail.gmail.com>	<4CF81AE1.5050900@v.loewis.de>	<4CF880F7.9000800@netwok.org>	<4CF89CCF.20307@v.loewis.de>	<4CF8E22B.7050005@netwok.org>	<4CF97197.2050105@v.loewis.de>	<4CF973D0.4060800@netwok.org>	<4CF974B9.6040509@v.loewis.de>	<idbur7$8hn$1@dough.gmane.org>	<4CF98169.6020106@v.loewis.de>
	<idc0ad$cj6$1@dough.gmane.org>
Message-ID: <4CF98E3C.3050803@v.loewis.de>

Am 04.12.2010 01:00, schrieb Terry Reedy:
> On 12/3/2010 6:46 PM, "Martin v. L?wis" wrote:
> 
>>> and stable as D1. I do not know what Martin means by 'integrate' (other
>>> than that he be able to use it to build Python)
>>
>> That the master copy of the source code is in the Python source
>> repository.
> 
> Is a separate branch acceptible, as long as you can commit changes?

No. I want the buildbots be able to use the code, in the Python trunk.

> Does the meaning of 'repository' change any with the hg switch?

No. The code would need to be pushed to the master repository, so that
the buildbots can fetch it from there.

In short, I'm not very interested in contributing to a tool that has no
users (and is the fifths of its kind).

Regards,
Martin


From foom at fuhm.net  Sat Dec  4 01:46:46 2010
From: foom at fuhm.net (James Y Knight)
Date: Fri, 3 Dec 2010 19:46:46 -0500
Subject: [Python-Dev] gc ideas -- sparse memory
In-Reply-To: <idc0ju$fd7$1@dough.gmane.org>
References: <AANLkTi=AchQA36yTgHrqiDovCaR1diYvRjXTg4j5P_Ni@mail.gmail.com>	<idbt2p$1jg$1@dough.gmane.org>
	<203DD6FC-D9F7-4E7D-B49C-CABC1DF7352B@fuhm.net>
	<idc0ju$fd7$1@dough.gmane.org>
Message-ID: <FCA0D391-678F-485A-99F2-1ADF8AC562B5@fuhm.net>

On Dec 3, 2010, at 7:05 PM, Terry Reedy wrote:
> I left out that the id must be an int.
> 
>> It's somewhat unfortuante that python has this constraint, instead of
>> the looser: "objects have a fixed id during their lifetime", which is
>> much easier to implement, and practically as useful.
> 
> Given that the only different between 'fixed and unique' and 'fixed' is the uniqueness part, I do not understand 'practically as useful'. Duplicate ids (in the extreme, that same for all objects) hardly seem useful at all.

Sure they are. This is what Java provides you, for example. If you have fixed, but potentially non-unique ids (in Java you get this using "identityHashCode()"), you can still make an identity hashtable. You simply need to *also* check using "is" that the two objects really are the same one after finding the hash bin using id.

It'd be a quality of implementation issue whether an implementation gives you the same value for every object. It should not, of course, if it wants programs to have reasonable performance. Same sort of thing as how __hash__ should not return 0 for everything.

Besides identity-hashtables, the main other thing id gets used for is to have some identifier string in a printer (e.g. "<class Foo at 0x1234567890>"). There, it's generally "good enough" to use an id which is not guaranteed to be, but often is, unique. It works for Java...

James

From tjreedy at udel.edu  Sat Dec  4 04:50:10 2010
From: tjreedy at udel.edu (Terry Reedy)
Date: Fri, 03 Dec 2010 22:50:10 -0500
Subject: [Python-Dev] gc ideas -- sparse memory
In-Reply-To: <FCA0D391-678F-485A-99F2-1ADF8AC562B5@fuhm.net>
References: <AANLkTi=AchQA36yTgHrqiDovCaR1diYvRjXTg4j5P_Ni@mail.gmail.com>	<idbt2p$1jg$1@dough.gmane.org>	<203DD6FC-D9F7-4E7D-B49C-CABC1DF7352B@fuhm.net>	<idc0ju$fd7$1@dough.gmane.org>
	<FCA0D391-678F-485A-99F2-1ADF8AC562B5@fuhm.net>
Message-ID: <idcdph$pv1$1@dough.gmane.org>

On 12/3/2010 7:46 PM, James Y Knight wrote:

> Sure they are. This is what Java provides you, for example. If you
> have fixed, but potentially non-unique ids (in Java you get this
> using "identityHashCode()"), you can still make an identity

I do not see the point of calling a (non-unique) hash value the identity

> hashtable. You simply need to *also* check using "is" that the two

In Python, that unique isness is the identify.

(a is b) == (id(a) == id(b)) by definition.

> objects really are the same one after finding the hash bin using id.

by using the hash value, which is how Python dict operate.

-- 
Terry Jan Reedy


From cs at zip.com.au  Sat Dec  4 05:06:45 2010
From: cs at zip.com.au (Cameron Simpson)
Date: Sat, 4 Dec 2010 15:06:45 +1100
Subject: [Python-Dev] gc ideas -- sparse memory
In-Reply-To: <203DD6FC-D9F7-4E7D-B49C-CABC1DF7352B@fuhm.net>
References: <203DD6FC-D9F7-4E7D-B49C-CABC1DF7352B@fuhm.net>
Message-ID: <20101204040645.GA32257@cskk.homeip.net>

On 03Dec2010 18:15, James Y Knight <foom at fuhm.net> wrote:
| On Dec 3, 2010, at 6:04 PM, Terry Reedy wrote:
| > gc is implementation specific. CPython uses ref counting + cycle
| > gc. A constraint on all implementations is that objects have a fixed,
| > unique id during their lifetime. CPython uses the address as the id, so
| > it cannot move objects. Other implementations do differently. Compacting
| > gc requires an id to current address table or something.
| 
| It's somewhat unfortuante that python has this constraint, instead of
| the looser: "objects have a fixed id during their lifetime", which is
| much easier to implement, and practically as useful.

Python doesn't have the constraint you imagine; it _does_ have "objects
have a fixed id during their lifetime".

_CPython_ has this constraint because it uses the address as the id,
which is free and needs no papping or extra storage. Of course, it
removes certain freedoms from the GC etc as a side effect.
-- 
Cameron Simpson <cs at zip.com.au> DoD#743
http://www.cskk.ezoshosting.com/cs/

The number of cylinders for this disk is set to 364737.
There is nothing wrong with that, but this is larger than 1024, [...]
- fdisk on our new RAID 07oct2007 :-)

From foom at fuhm.net  Sat Dec  4 05:34:43 2010
From: foom at fuhm.net (James Y Knight)
Date: Fri, 3 Dec 2010 23:34:43 -0500
Subject: [Python-Dev] gc ideas -- sparse memory
In-Reply-To: <idcdph$pv1$1@dough.gmane.org>
References: <AANLkTi=AchQA36yTgHrqiDovCaR1diYvRjXTg4j5P_Ni@mail.gmail.com>	<idbt2p$1jg$1@dough.gmane.org>	<203DD6FC-D9F7-4E7D-B49C-CABC1DF7352B@fuhm.net>	<idc0ju$fd7$1@dough.gmane.org>
	<FCA0D391-678F-485A-99F2-1ADF8AC562B5@fuhm.net>
	<idcdph$pv1$1@dough.gmane.org>
Message-ID: <DD9AE029-32D2-43CF-BB4A-CF788830C593@fuhm.net>

On Dec 3, 2010, at 10:50 PM, Terry Reedy wrote:
> On 12/3/2010 7:46 PM, James Y Knight wrote:
> 
>> Sure they are. This is what Java provides you, for example. If you
>> have fixed, but potentially non-unique ids (in Java you get this
>> using "identityHashCode()"), you can still make an identity
> 
> I do not see the point of calling a (non-unique) hash value the identity

My point was simply that a) it's an unfortunate constraint on potential GC implementations that objects need to have a fixed and unique id in Python, and b) that it's not actually necessary to have such a constraint (in the abstract sense of required; obviously it's a requirement upon Python *today*, due to existing code which depends upon that promise). 

Would you be happier if I had said "it's unfortunate that Python has an "id" function instead of an "identityHashValue" function? I suppose that's what I really meant. Python the language would not have been harmed had it had from the start an identityHashValue() function instead of an id() function. In the CPython implementation, it may even have had the exact same behavior, but would've allowed other implementations more flexibility.

James


From tjreedy at udel.edu  Sat Dec  4 06:39:23 2010
From: tjreedy at udel.edu (Terry Reedy)
Date: Sat, 04 Dec 2010 00:39:23 -0500
Subject: [Python-Dev] gc ideas -- sparse memory
In-Reply-To: <20101204040645.GA32257@cskk.homeip.net>
References: <203DD6FC-D9F7-4E7D-B49C-CABC1DF7352B@fuhm.net>
	<20101204040645.GA32257@cskk.homeip.net>
Message-ID: <idck69$ccj$1@dough.gmane.org>

On 12/3/2010 11:06 PM, Cameron Simpson wrote:
> On 03Dec2010 18:15, James Y Knight<foom at fuhm.net>  wrote:
> | On Dec 3, 2010, at 6:04 PM, Terry Reedy wrote:
> |>  gc is implementation specific. CPython uses ref counting + cycle
> |>  gc. A constraint on all implementations is that objects have a fixed,
> |>  unique id during their lifetime. CPython uses the address as the id, so
> |>  it cannot move objects. Other implementations do differently. Compacting
> |>  gc requires an id to current address table or something.
> |
> | It's somewhat unfortuante that python has this constraint, instead of
> | the looser: "objects have a fixed id during their lifetime", which is
> | much easier to implement, and practically as useful.
>
> Python doesn't have the constraint you imagine; it _does_ have "objects
> have a fixed id during their lifetime".

"id(object)
Return the ?identity? of an object. This is an integer which is 
guaranteed to be unique and constant for this object during its lifetime."

Of course, other implementations are free to change builtins, but code 
that depends on CPython's definitions will not run.

-- 
Terry Jan Reedy



From dimaqq at gmail.com  Sat Dec  4 06:47:53 2010
From: dimaqq at gmail.com (Dima Tisnek)
Date: Fri, 3 Dec 2010 22:47:53 -0700
Subject: [Python-Dev] gc ideas -- sparse memory
In-Reply-To: <4CF98117.5060301@v.loewis.de>
References: <AANLkTi=AchQA36yTgHrqiDovCaR1diYvRjXTg4j5P_Ni@mail.gmail.com>
	<idbt2p$1jg$1@dough.gmane.org>
	<AANLkTi=bGGh1_Z3-ey+4rbvNHHmi=SgOSUo_TQjvK_p-@mail.gmail.com>
	<4CF98117.5060301@v.loewis.de>
Message-ID: <AANLkTin8TZ7gZg1tkbefN5WjkP4_GeX4sE-wrvbyVSQT@mail.gmail.com>

On 3 December 2010 16:45, "Martin v. L?wis" <martin at v.loewis.de> wrote:
>> Oh my bad, I must've confused python with some research paper.
>> Unique id is not so hard to make without an address.
>>
>> While on this topic, what is the real need for unique ids?
>
> They are absolutely needed for mutable objects. For immutable ones,
> it would be ok to claim that they are identical if they are equal
> (assuming they support equality - which is tricky for things like NaN).

Indeed, but do ids really need to be unique and fixed at the same time?

a is b # (if atomic) needs unique ids, but doesn't really need fixed ids
a[b]   # needs fixed hash, but not strictly a globally unique id

I can imagine an implementaion of pickle for example that uses unique
and fixed as a given to detect cycles, etc; but that would be
implementation detail.

It seems to me unique and fixed id implies that it is stored somewhere
(with incref beforehands and decref afterwards), however a proper
reference to an object could be used just as well.

Am I still missing something?

>
> Of course, the C API has lots of assumptions that identity and address
> are really the same thing.
>
> Regards,
> Martin
>

From fijall at gmail.com  Sat Dec  4 07:21:30 2010
From: fijall at gmail.com (Maciej Fijalkowski)
Date: Sat, 4 Dec 2010 08:21:30 +0200
Subject: [Python-Dev] gc ideas -- sparse memory
In-Reply-To: <DD9AE029-32D2-43CF-BB4A-CF788830C593@fuhm.net>
References: <AANLkTi=AchQA36yTgHrqiDovCaR1diYvRjXTg4j5P_Ni@mail.gmail.com>
	<idbt2p$1jg$1@dough.gmane.org>
	<203DD6FC-D9F7-4E7D-B49C-CABC1DF7352B@fuhm.net>
	<idc0ju$fd7$1@dough.gmane.org>
	<FCA0D391-678F-485A-99F2-1ADF8AC562B5@fuhm.net>
	<idcdph$pv1$1@dough.gmane.org>
	<DD9AE029-32D2-43CF-BB4A-CF788830C593@fuhm.net>
Message-ID: <AANLkTimq-Ln5EAhY=djHmaJV_gzL_EjoOprf5WsvNDrQ@mail.gmail.com>

On Sat, Dec 4, 2010 at 6:34 AM, James Y Knight <foom at fuhm.net> wrote:
> On Dec 3, 2010, at 10:50 PM, Terry Reedy wrote:
>> On 12/3/2010 7:46 PM, James Y Knight wrote:
>>
>>> Sure they are. This is what Java provides you, for example. If you
>>> have fixed, but potentially non-unique ids (in Java you get this
>>> using "identityHashCode()"), you can still make an identity
>>
>> I do not see the point of calling a (non-unique) hash value the identity
>
> My point was simply that a) it's an unfortunate constraint on potential GC implementations that objects need to have a fixed and unique id in Python, and b) that it's not actually necessary to have such a constraint (in the abstract sense of required; obviously it's a requirement upon Python *today*, due to existing code which depends upon that promise).
>
> Would you be happier if I had said "it's unfortunate that Python has an "id" function instead of an "identityHashValue" function? I suppose that's what I really meant. Python the language would not have been harmed had it had from the start an identityHashValue() function instead of an id() function. In the CPython implementation, it may even have had the exact same behavior, but would've allowed other implementations more flexibility.
>
> James
>

I don't see how this related to moving vs non-moving GC. PyPy (and I
believe IronPython and Java) all have fixed unique ids that are not
necesarilly their addresses. The only problem is that id() computed
that way is more costly performance-wise, but works.

From steve at pearwood.info  Sat Dec  4 07:48:13 2010
From: steve at pearwood.info (Steven D'Aprano)
Date: Sat, 04 Dec 2010 17:48:13 +1100
Subject: [Python-Dev] gc ideas -- sparse memory
In-Reply-To: <DD9AE029-32D2-43CF-BB4A-CF788830C593@fuhm.net>
References: <AANLkTi=AchQA36yTgHrqiDovCaR1diYvRjXTg4j5P_Ni@mail.gmail.com>	<idbt2p$1jg$1@dough.gmane.org>	<203DD6FC-D9F7-4E7D-B49C-CABC1DF7352B@fuhm.net>	<idc0ju$fd7$1@dough.gmane.org>	<FCA0D391-678F-485A-99F2-1ADF8AC562B5@fuhm.net>	<idcdph$pv1$1@dough.gmane.org>
	<DD9AE029-32D2-43CF-BB4A-CF788830C593@fuhm.net>
Message-ID: <4CF9E42D.3010302@pearwood.info>

James Y Knight wrote:
> On Dec 3, 2010, at 10:50 PM, Terry Reedy wrote:
>> On 12/3/2010 7:46 PM, James Y Knight wrote:
>>
>>> Sure they are. This is what Java provides you, for example. If you
>>> have fixed, but potentially non-unique ids (in Java you get this
>>> using "identityHashCode()"), you can still make an identity
>> I do not see the point of calling a (non-unique) hash value the identity
> 
> My point was simply that a) it's an unfortunate constraint on potential GC implementations that objects need to have a fixed and unique id in Python, and b) that it's not actually necessary to have such a constraint (in the abstract sense of required; obviously it's a requirement upon Python *today*, due to existing code which depends upon that promise). 

I'm afraid I don't follow you. Unless you're suggesting some sort of 
esoteric object system whereby objects *don't* have identity (e.g. where 
objects are emergent properties of some sort of distributed, 
non-localised "information"), any object naturally has an identity -- 
itself.

It seems to me that an identify function must obey one constraint:

* Two objects which exist simultaneously have the same identity if
   and only if they are the same object i.e. id(x) == id(y) if and
   only if x is y.

Other than that, an implementation is free to make id() behave any way 
they like. CPython uses the memory location of the object, but Jython 
and IronPython use an incrementing counter which is never re-used for 
the lifetime of the Python process. CPython's implementation implies 
that objects may not be moved in memory, but that's not a language 
constraint, that's an implementation issue.

It seems counter-productive to me to bother with an identity function 
which doesn't meet that constraint. If id(x) == id(y) implies nothing 
about x and y (they may, or may not, be the same object) then what's the 
point? Why would you bother using that function when you could just use 
x == y instead?

> Would you be happier if I had said "it's unfortunate that Python has an "id" function instead of an "identityHashValue" function? I suppose that's what I really meant. Python the language would not have been harmed had it had from the start an identityHashValue() function instead of an id() function. In the CPython implementation, it may even have had the exact same behavior, but would've allowed other implementations more flexibility.

No, because I don't see what the point of identityHashValue or why you 
would ever bother to use it.


-- 
Steven

From stephen at xemacs.org  Sat Dec  4 09:13:45 2010
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Sat, 04 Dec 2010 17:13:45 +0900
Subject: [Python-Dev] Python and the Unicode Character Database
In-Reply-To: <1291380666.3624.0.camel@localhost.localdomain>
References: <AANLkTinVpySuRFkjf-Sc+7a91zrxhuQpqPQL5nM9u1w+@mail.gmail.com>
	<AANLkTinQLK7+5NfEp-2nMiJeDEKRxKN9Zfs4ihv-NMyH@mail.gmail.com>
	<87wrnv43v5.fsf@uwakimon.sk.tsukuba.ac.jp>
	<id3umc$ps8$1@dough.gmane.org> <4CF5841E.6020401@v.loewis.de>
	<4CF619C1.8030403@pearwood.info> <4CF6CCF9.6000100@v.loewis.de>
	<4CF6E5AF.7080308@pearwood.info>
	<AANLkTikuUki5LgisVHrSMGZtmWZxxy9A8TOwny7dVCMe@mail.gmail.com>
	<id72oo$cdp$2@dough.gmane.org>
	<AANLkTikd0zDjr2GQ3_4C7BLZZstePQAiHee9m2ZpgCEh@mail.gmail.com>
	<20101202143633.6eb17b8d@pitrou.net>
	<AANLkTikiziBN87-7gM3rweODnXjEuK3DfhEni1sTDYtJ@mail.gmail.com>
	<1291308960.3576.28.camel@localhost.localdomain>
	<AANLkTinBzCQ4gi42iNXkSqWAEkdPWZzs=+C5sr0f0vpB@mail.gmail.com>
	<1291316131.3576.43.camel@localhost.localdomain>
	<87fwufxxio.fsf@uwakimon.sk.tsukuba.ac.jp>
	<1291380666.3624.0.camel@localhost.localdomain>
Message-ID: <877hfqx8ee.fsf@uwakimon.sk.tsukuba.ac.jp>

Antoine Pitrou writes:
 > Le vendredi 03 d?cembre 2010 ? 13:58 +0900, Stephen J. Turnbull a
 > ?crit :
 > > Antoine Pitrou writes:
 > > 
 > >  > The legacy format argument looks like a red herring to me. When
 > >  > converting from a format to another it is the programmer's job to
 > >  > his/her job right.
 > > 
 > > Uhmmmmmm, the argument *for* this "feature" proposed by several people
 > > is that Python's numeric constructors do it (right) so that the
 > > programmer doesn't have to.
 > 
 > As far as I understand, Alexander was talking about a legacy pre-unicode
 > text format. We don't have to support this.

*I* didn't say we *should* support it.  I'm saying that *others'*
argument for not restricting the formats accepting by string to number
converters to something well-defined and AFAIK universally understood
by users (developers of Python programs *and* end-users) is that we
*already* support this.

Alexander, Martin, and I are basically just pointing out that no, the
"support" we have via the built-in numeric constructors is incomplete
and nonconforming.  We feel that is a bug to be fixed by (1)
implementing the definition as currently found in the documents, and
(2) moving the non-ASCII support to another module (or, as a
compromise, supporting non-ASCII digits via an argument to the
built-ins -- that was my proposal, I don't know if Alexander or Martin
would find it acceptable).

Given that some committers (MAL, you?) don't even consider that
accepting and converting a string containing digits from multiple
scripts as a single number is a bug, I'd really rather that this
bug/feature not be embedded in the interpreter.  I suppose that as a
built-in rather than syntax, technically it doesn't fall under the
moratorium, but it makes me nervous....

From stephen at xemacs.org  Sat Dec  4 09:31:04 2010
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Sat, 04 Dec 2010 17:31:04 +0900
Subject: [Python-Dev] transform() and untransform() methods,
	and the codec registry
In-Reply-To: <AANLkTikoRZJ4m3oGBhzgJ1_8WoJWf6ik_jm_uPt600pc@mail.gmail.com>
References: <20101202180651.F2AB0EE9CD@mail.python.org>
	<201012031016.05063.victor.stinner@haypocalc.com>
	<AANLkTikoRZJ4m3oGBhzgJ1_8WoJWf6ik_jm_uPt600pc@mail.gmail.com>
Message-ID: <8762vax7lj.fsf@uwakimon.sk.tsukuba.ac.jp>

Alexander Belopolsky writes:

 > In fact, once the language moratorium is over, I will argue that
 > str.encode() and byte.decode() should deprecate encoding argument and
 > just do UTF-8 encoding/decoding.  Hopefully by that time most people
 > will forget that other encodings exist.  (I can dream, right?)

It's just a dream.  There's a pile of archival material, often on R/O
media, out there that won't be transcoded any more quickly than the
inscriptions on Tutankhamun's tomb.

Remember, Python is a language used to implement such translations.
It's not an application.  I think it would be reasonable to make UTF-8
the *default* encoding on all platforms, except for internal OS
functions, where Windows will presumably continue to use UTF-16 and
*nix distros will probably continue to agree to disagree about whether
on-disk format is NFD or NFC (and the Python language as yet doesn't
know about NFC v. NFD, although the library does).

In the discussion of PEP 263, I proposed that the external encoding of
Python scripts themselves be fixed as UTF-8, and other encodings would
have to be pretranslated by an appropriate codec.  That was shouted
down by the European contingent, who wanted to continue using Latin-1
and Latin-2 without codecs or a wrapper to call them transparently.
However, this time around you might get a more sympathetic hearing.


From stefan_ml at behnel.de  Sat Dec  4 09:35:11 2010
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Sat, 04 Dec 2010 09:35:11 +0100
Subject: [Python-Dev] PEP 384 accepted
In-Reply-To: <4CF8008B.60402@v.loewis.de>
References: <AANLkTimQsiU+nyX6CdqQ8JBybJdnx=1EMxX9mefHbx9H@mail.gmail.com>
	<4CF8008B.60402@v.loewis.de>
Message-ID: <idcug0$afp$1@dough.gmane.org>

"Martin v. L?wis", 02.12.2010 21:24:
>> Since discussion has trailed off without any blocking objections, I'm
>> accepting PEP 384. Martin, you may mark the PEP accepted and proceed
>> with merging the implementation for the beta on Saturday.
>
> Thanks! will do (I'll also take into consideration the proposed changes).

Note that the PEP states under "Signature Changes":

"""
A number of functions currently expect a specific struct, even though 
callers typically have PyObject* available. These have been changed to 
expect PyObject* as the parameter; this will cause warnings in applications 
that currently explicitly cast to the parameter type.
"""

That's not true for C++ compilation where I now get an error - 3 failing 
tests in Cython's test suite tonight due to the new PyEval_EvalCode() 
signature.

I'm not really complaining (the API fixes are long overdue), just leaving a 
comment that what a compiler considers a warning or error pretty much 
depends on compiler, platform and configuration. So the above statement can 
be considered euphemistic.

Stefan


From martin at v.loewis.de  Sat Dec  4 09:43:55 2010
From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=)
Date: Sat, 04 Dec 2010 09:43:55 +0100
Subject: [Python-Dev] gc ideas -- sparse memory
In-Reply-To: <AANLkTin8TZ7gZg1tkbefN5WjkP4_GeX4sE-wrvbyVSQT@mail.gmail.com>
References: <AANLkTi=AchQA36yTgHrqiDovCaR1diYvRjXTg4j5P_Ni@mail.gmail.com>	<idbt2p$1jg$1@dough.gmane.org>	<AANLkTi=bGGh1_Z3-ey+4rbvNHHmi=SgOSUo_TQjvK_p-@mail.gmail.com>	<4CF98117.5060301@v.loewis.de>
	<AANLkTin8TZ7gZg1tkbefN5WjkP4_GeX4sE-wrvbyVSQT@mail.gmail.com>
Message-ID: <4CF9FF4B.1030004@v.loewis.de>

> Am I still missing something?

Apparently. The hole C API would break if objects would move in memory.
Since they have to stay at fixed addresses, it's easy enough to use the
address as ID. There is no problem to be solved here.

Regards,
Martin

From eliben at gmail.com  Sat Dec  4 09:48:38 2010
From: eliben at gmail.com (Eli Bendersky)
Date: Sat, 4 Dec 2010 10:48:38 +0200
Subject: [Python-Dev] python 2 for building python 3
Message-ID: <AANLkTin7ynm7Srnp4DLa2B+825cUL+jSaKqq+Gvz34OD@mail.gmail.com>

To build the trunk Python 3k, it seems that Python 2 has to be installed.
This is because Objects/typeslots.py is called in the Makefile, and this
file uses Python 2 "print statement" syntax, which fails if $(PYTHON) is
actually Python 3.

Can anyone reproduce this?

Eli
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20101204/a2834256/attachment.html>

From martin at v.loewis.de  Sat Dec  4 09:49:12 2010
From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=)
Date: Sat, 04 Dec 2010 09:49:12 +0100
Subject: [Python-Dev] PEP 384 accepted
In-Reply-To: <idcug0$afp$1@dough.gmane.org>
References: <AANLkTimQsiU+nyX6CdqQ8JBybJdnx=1EMxX9mefHbx9H@mail.gmail.com>	<4CF8008B.60402@v.loewis.de>
	<idcug0$afp$1@dough.gmane.org>
Message-ID: <4CFA0088.8020407@v.loewis.de>

> I'm not really complaining (the API fixes are long overdue), just
> leaving a comment that what a compiler considers a warning or error
> pretty much depends on compiler, platform and configuration.

No no no. It does *not* depend on compiler, platform, or configuration.
It *only* depends on the programming language. So for C++, it's an
API change; for C, it's not.

Still, I can point that out.

Regards,
Martin

From martin at v.loewis.de  Sat Dec  4 09:59:08 2010
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sat, 04 Dec 2010 09:59:08 +0100
Subject: [Python-Dev] gc ideas -- sparse memory
In-Reply-To: <4CF9E42D.3010302@pearwood.info>
References: <AANLkTi=AchQA36yTgHrqiDovCaR1diYvRjXTg4j5P_Ni@mail.gmail.com>	<idbt2p$1jg$1@dough.gmane.org>	<203DD6FC-D9F7-4E7D-B49C-CABC1DF7352B@fuhm.net>	<idc0ju$fd7$1@dough.gmane.org>	<FCA0D391-678F-485A-99F2-1ADF8AC562B5@fuhm.net>	<idcdph$pv1$1@dough.gmane.org>	<DD9AE029-32D2-43CF-BB4A-CF788830C593@fuhm.net>
	<4CF9E42D.3010302@pearwood.info>
Message-ID: <4CFA02DC.3060807@v.loewis.de>

> I'm afraid I don't follow you. Unless you're suggesting some sort of
> esoteric object system whereby objects *don't* have identity (e.g. where
> objects are emergent properties of some sort of distributed,
> non-localised "information"), any object naturally has an identity --
> itself.

Not in Java or C#. It is in these languages possible to determine
whether to references refer to the same object. However, objects don't
naturally have a distinct identification (be it an integer or something
else).

If you really want to associate unique numbers with objects in these
languages, the common approach is to put them into an identity
dictionary as keys.

> It seems counter-productive to me to bother with an identity function
> which doesn't meet that constraint. If id(x) == id(y) implies nothing
> about x and y (they may, or may not, be the same object) then what's the
> point?

See James' explanation: it would be possible to use this as the
foundation of an identity hash table.

> Why would you bother using that function when you could just use
> x == y instead?

Because in a hash table, you also need a hash value.

Regards,
Martn

From martin at v.loewis.de  Sat Dec  4 10:13:11 2010
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sat, 04 Dec 2010 10:13:11 +0100
Subject: [Python-Dev] python 2 for building python 3
In-Reply-To: <AANLkTin7ynm7Srnp4DLa2B+825cUL+jSaKqq+Gvz34OD@mail.gmail.com>
References: <AANLkTin7ynm7Srnp4DLa2B+825cUL+jSaKqq+Gvz34OD@mail.gmail.com>
Message-ID: <4CFA0627.7010202@v.loewis.de>

Am 04.12.2010 09:48, schrieb Eli Bendersky:
> To build the trunk Python 3k, it seems that Python 2 has to be installed.
> This is because Objects/typeslots.py is called in the Makefile, and this
> file uses Python 2 "print statement" syntax, which fails if $(PYTHON) is
> actually Python 3.

This shouldn't be necessary, as typeslots.inc is also checked into
subversion, and should have a newer time stamp than typeslots.inc
(perhaps not currently, but that is the plan, anyway).

In any case, I now made the script 2-vs-3-agnostic.

Regards,
Martin

From eliben at gmail.com  Sat Dec  4 10:20:57 2010
From: eliben at gmail.com (Eli Bendersky)
Date: Sat, 4 Dec 2010 11:20:57 +0200
Subject: [Python-Dev] python 2 for building python 3
In-Reply-To: <4CFA0627.7010202@v.loewis.de>
References: <AANLkTin7ynm7Srnp4DLa2B+825cUL+jSaKqq+Gvz34OD@mail.gmail.com>
	<4CFA0627.7010202@v.loewis.de>
Message-ID: <AB0ACAA1-94D2-4ED1-AD4C-29ABD274E3DE@gmail.com>





On Dec 4, 2010, at 11:13, "Martin v. L?wis" <martin at v.loewis.de> wrote:

> Am 04.12.2010 09:48, schrieb Eli Bendersky:
>> To build the trunk Python 3k, it seems that Python 2 has to be installed.
>> This is because Objects/typeslots.py is called in the Makefile, and this
>> file uses Python 2 "print statement" syntax, which fails if $(PYTHON) is
>> actually Python 3.
> 
> This shouldn't be necessary, as typeslots.inc is also checked into
> subversion, and should have a newer time stamp than typeslots.inc
> (perhaps not currently, but that is the plan, anyway).
> 
> In any case, I now made the script 2-vs-3-agnostic.
> 
> Regards,
> Martin

I had the error pop out in a real build, so the time stamp is probably not a 100% guarantee. Therefore I think it's good that you made it version agnostic. This should be probably be a requirement for all scripts that are part of the build.

Thanks,
Eli


From steve at pearwood.info  Sat Dec  4 11:54:31 2010
From: steve at pearwood.info (Steven D'Aprano)
Date: Sat, 04 Dec 2010 21:54:31 +1100
Subject: [Python-Dev] gc ideas -- sparse memory
In-Reply-To: <4CFA02DC.3060807@v.loewis.de>
References: <AANLkTi=AchQA36yTgHrqiDovCaR1diYvRjXTg4j5P_Ni@mail.gmail.com>	<idbt2p$1jg$1@dough.gmane.org>	<203DD6FC-D9F7-4E7D-B49C-CABC1DF7352B@fuhm.net>	<idc0ju$fd7$1@dough.gmane.org>	<FCA0D391-678F-485A-99F2-1ADF8AC562B5@fuhm.net>	<idcdph$pv1$1@dough.gmane.org>	<DD9AE029-32D2-43CF-BB4A-CF788830C593@fuhm.net>
	<4CF9E42D.3010302@pearwood.info> <4CFA02DC.3060807@v.loewis.de>
Message-ID: <4CFA1DE7.3010409@pearwood.info>

Martin v. L?wis wrote:
>> I'm afraid I don't follow you. Unless you're suggesting some sort of
>> esoteric object system whereby objects *don't* have identity (e.g. where
>> objects are emergent properties of some sort of distributed,
>> non-localised "information"), any object naturally has an identity --
>> itself.
> 
> Not in Java or C#. It is in these languages possible to determine
> whether to references refer to the same object. However, objects don't
> naturally have a distinct identification (be it an integer or something
> else).

Surely even in Java or C#, objects have an *identity* even if the 
language doesn't provide a way to query their distinct *identification*. 
An object stored at one memory location is distinct from any other 
object stored at different memory locations at that same moment in time, 
regardless of whether or not the language gives you a convenient label 
for that identity. Even if that memory location can change during the 
lifetime of the object, at any one moment, object X is a different 
object from every other object.

The fact that we can even talk about "this object" versus "that object" 
implies that objects have identity.

To put it in Python terms, if the id() function were removed, it would 
no longer be possible to get the unique identification associated with 
an object, but you could still compare the identity of two objects using 
`is`.

Of course, I'm only talking about objects. In Java there are values 
which aren't objects, such as ints and floats. That's irrelevant for our 
discussion, because Python has no such values.


> If you really want to associate unique numbers with objects in these
> languages, the common approach is to put them into an identity
> dictionary as keys.
> 
>> It seems counter-productive to me to bother with an identity function
>> which doesn't meet that constraint. If id(x) == id(y) implies nothing
>> about x and y (they may, or may not, be the same object) then what's the
>> point?
> 
> See James' explanation: it would be possible to use this as the
> foundation of an identity hash table.

I'm afraid James' explanation didn't shed any light on the question to 
me. It seems to me that Java's IdentityHashValue[sic -- I think the 
correct function name is actually IdentityHashCode] is equivalent to 
Python's hash(), not to Python's id(), and claiming it is related to 
identity is misleading and confusing.

I don't think I'm alone here -- it seems to me that even among Java 
programmers, the same criticisms have been raised:

http://bugs.sun.com/bugdatabase/view_bug.do?bug%5Fid=6321873
http://deepakjha.wordpress.com/2008/07/31/interesting-fact-about-identityhashcode-method-in-javalangsystem-class/

Like hash(), IdentityHashCode doesn't make any promises about identity 
at all. Two distinct objects could have the same hash code, and a 
further test is needed to distinguish them.


>> Why would you bother using that function when you could just use
>> x == y instead?
> 
> Because in a hash table, you also need a hash value.

Well, sure, in a hash table you need a hash value. But I was talking 
about an id() function.

So is that it? Is IdentityHashValue (or *Code, as the case may be) just 
a longer name for hash()?



-- 
Steven


From solipsis at pitrou.net  Sat Dec  4 12:17:18 2010
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Sat, 04 Dec 2010 12:17:18 +0100
Subject: [Python-Dev] Python and the Unicode Character Database
In-Reply-To: <877hfqx8ee.fsf@uwakimon.sk.tsukuba.ac.jp>
References: <AANLkTinVpySuRFkjf-Sc+7a91zrxhuQpqPQL5nM9u1w+@mail.gmail.com>
	<AANLkTinQLK7+5NfEp-2nMiJeDEKRxKN9Zfs4ihv-NMyH@mail.gmail.com>
	<87wrnv43v5.fsf@uwakimon.sk.tsukuba.ac.jp>
	<id3umc$ps8$1@dough.gmane.org>
	<4CF5841E.6020401@v.loewis.de> <4CF619C1.8030403@pearwood.info>
	<4CF6CCF9.6000100@v.loewis.de> <4CF6E5AF.7080308@pearwood.info>
	<AANLkTikuUki5LgisVHrSMGZtmWZxxy9A8TOwny7dVCMe@mail.gmail.com>
	<id72oo$cdp$2@dough.gmane.org>
	<AANLkTikd0zDjr2GQ3_4C7BLZZstePQAiHee9m2ZpgCEh@mail.gmail.com>
	<20101202143633.6eb17b8d@pitrou.net>
	<AANLkTikiziBN87-7gM3rweODnXjEuK3DfhEni1sTDYtJ@mail.gmail.com>
	<1291308960.3576.28.camel@localhost.localdomain>
	<AANLkTinBzCQ4gi42iNXkSqWAEkdPWZzs=+C5sr0f0vpB@mail.gmail.com>
	<1291316131.3576.43.camel@localhost.localdomain>
	<87fwufxxio.fsf@uwakimon.sk.tsukuba.ac.jp>
	<1291380666.3624.0.camel@localhost.localdomain>
	<877hfqx8ee.fsf@uwakimon.sk.tsukuba.ac.jp>
Message-ID: <1291461438.3534.2.camel@localhost.localdomain>

Le samedi 04 d?cembre 2010 ? 17:13 +0900, Stephen J. Turnbull a ?crit :
> Antoine Pitrou writes:
>  > Le vendredi 03 d?cembre 2010 ? 13:58 +0900, Stephen J. Turnbull a
>  > ?crit :
>  > > Antoine Pitrou writes:
>  > > 
>  > >  > The legacy format argument looks like a red herring to me. When
>  > >  > converting from a format to another it is the programmer's job to
>  > >  > his/her job right.
>  > > 
>  > > Uhmmmmmm, the argument *for* this "feature" proposed by several people
>  > > is that Python's numeric constructors do it (right) so that the
>  > > programmer doesn't have to.
>  > 
>  > As far as I understand, Alexander was talking about a legacy pre-unicode
>  > text format. We don't have to support this.
> 
> *I* didn't say we *should* support it.  I'm saying that *others'*
> argument for not restricting the formats accepting by string to number
> converters to something well-defined and AFAIK universally understood
> by users (developers of Python programs *and* end-users) is that we
> *already* support this.

As far as I can parse your sentence, I think you are mistaken.

Regards

Antoine.



From solipsis at pitrou.net  Sat Dec  4 12:27:10 2010
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Sat, 4 Dec 2010 12:27:10 +0100
Subject: [Python-Dev] PEP 384 accepted
References: <AANLkTimQsiU+nyX6CdqQ8JBybJdnx=1EMxX9mefHbx9H@mail.gmail.com>
	<4CF8008B.60402@v.loewis.de>
	<AANLkTikbRJEFYE_TvN++GYhmvSUu3hdhf4znOti7oty1@mail.gmail.com>
	<4CF80964.4070300@v.loewis.de>
	<AANLkTikcT0WGo3FhFVq=N6ktToNenBDNNcC4eSpefcFF@mail.gmail.com>
	<4CF80D6C.6050703@v.loewis.de>
	<AANLkTi=s5NMxjPdRQRcupNBzdqjMhLO6TbFWtQzi0TEw@mail.gmail.com>
	<4CF81208.8090705@v.loewis.de>
	<AANLkTinspUo88oUnd3cX776mMRMU6bqyOaq9Og7JLwhU@mail.gmail.com>
	<4CF81AE1.5050900@v.loewis.de> <4CF880F7.9000800@netwok.org>
	<4CF89CCF.20307@v.loewis.de> <4CF8E22B.7050005@netwok.org>
	<4CF97197.2050105@v.loewis.de> <4CF973D0.4060800@netwok.org>
	<4CF974B9.6040509@v.loewis.de>
	<294C7411-A987-45CE-ACE1-6DEFB17DA0C6@fuhm.net>
Message-ID: <20101204122710.794d5fe8@pitrou.net>

On Fri, 3 Dec 2010 18:11:57 -0500
James Y Knight <foom at fuhm.net> wrote:

> On Dec 3, 2010, at 5:52 PM, Martin v. L?wis wrote:
> 
> > Am 03.12.2010 23:48, schrieb ?ric Araujo:
> >>> But I'm not interested at all in having it in distutils2. I want the
> >>> Python build itself to use it, and alas, I can't because of the freeze.
> >> You can?t in 3.2, true.  Neither can you in 3.1, or any previous
> >> version.  If you implement it in distutils2, you have very good chances
> >> to get it for 3.3.  Isn?t that a win?
> > 
> > It is, unfortunately, a very weak promise. Until distutils2 is
> > integrated in Python, I probably won't spend any time on it.
> 
> It seems like it'd be a good idea to start integrating distutils2 into python trunk immediately after the 3.2 branch is cut, then.

+1 from me.

Antoine.



From solipsis at pitrou.net  Sat Dec  4 12:37:35 2010
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Sat, 4 Dec 2010 12:37:35 +0100
Subject: [Python-Dev] gc ideas -- sparse memory
References: <203DD6FC-D9F7-4E7D-B49C-CABC1DF7352B@fuhm.net>
	<20101204040645.GA32257@cskk.homeip.net>
Message-ID: <20101204123735.56609926@pitrou.net>

On Sat, 4 Dec 2010 15:06:45 +1100
Cameron Simpson <cs at zip.com.au> wrote:
> On 03Dec2010 18:15, James Y Knight <foom at fuhm.net> wrote:
> | On Dec 3, 2010, at 6:04 PM, Terry Reedy wrote:
> | > gc is implementation specific. CPython uses ref counting + cycle
> | > gc. A constraint on all implementations is that objects have a fixed,
> | > unique id during their lifetime. CPython uses the address as the id, so
> | > it cannot move objects. Other implementations do differently. Compacting
> | > gc requires an id to current address table or something.
> | 
> | It's somewhat unfortuante that python has this constraint, instead of
> | the looser: "objects have a fixed id during their lifetime", which is
> | much easier to implement, and practically as useful.
> 
> Python doesn't have the constraint you imagine; it _does_ have "objects
> have a fixed id during their lifetime".
> 
> _CPython_ has this constraint because it uses the address as the id,
> which is free and needs no papping or extra storage.

Well, the main reason CPython has this constraint is that the C API
mandates constant object pointers. That we can then reuse the pointer
for id() is only a consequence of this.

Regards

Antoine.



From solipsis at pitrou.net  Sat Dec  4 12:41:09 2010
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Sat, 4 Dec 2010 12:41:09 +0100
Subject: [Python-Dev] python 2 for building python 3
References: <AANLkTin7ynm7Srnp4DLa2B+825cUL+jSaKqq+Gvz34OD@mail.gmail.com>
	<4CFA0627.7010202@v.loewis.de>
	<AB0ACAA1-94D2-4ED1-AD4C-29ABD274E3DE@gmail.com>
Message-ID: <20101204124109.07745d71@pitrou.net>

On Sat, 4 Dec 2010 11:20:57 +0200
Eli Bendersky <eliben at gmail.com> wrote:
> 
> I had the error pop out in a real build, so the time stamp is probably not a 100% guarantee. Therefore I think it's good that you made it version agnostic. This should be probably be a requirement for all scripts that are part of the build.

Er, normally you don't need *any* Python installed to build Python (be
it 3.x or 2.x). It would be better if this didn't change.

Regards

Antoine.



From solipsis at pitrou.net  Sat Dec  4 12:45:02 2010
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Sat, 4 Dec 2010 12:45:02 +0100
Subject: [Python-Dev] r87010 - in python/branches/py3k:
 Doc/library/subprocess.rst Lib/subprocess.py Lib/test/test_subprocess.py
References: <20101204091044.C6CBEEE985@mail.python.org>
Message-ID: <20101204124502.37ec1594@pitrou.net>

On Sat,  4 Dec 2010 10:10:44 +0100 (CET)
gregory.p.smith <python-checkins at python.org> wrote:
> Author: gregory.p.smith
> Date: Sat Dec  4 10:10:44 2010
> New Revision: 87010
> 
> Log:
> issue7213 + issue2320: Cause a DeprecationWarning if the close_fds argument is
> not passed to subprocess.Popen as the default value will be changing in a
> future Python to the safer and more often desired value of True.

That doesn't seem to be a good idea under Windows, is it?

(?Note that on Windows, you cannot set *close_fds* to true and
also redirect the standard handles by setting *stdin*, *stdout* or
*stderr*.?)



From martin at v.loewis.de  Sat Dec  4 13:22:07 2010
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sat, 04 Dec 2010 13:22:07 +0100
Subject: [Python-Dev] gc ideas -- sparse memory
In-Reply-To: <4CFA1DE7.3010409@pearwood.info>
References: <AANLkTi=AchQA36yTgHrqiDovCaR1diYvRjXTg4j5P_Ni@mail.gmail.com>	<idbt2p$1jg$1@dough.gmane.org>	<203DD6FC-D9F7-4E7D-B49C-CABC1DF7352B@fuhm.net>	<idc0ju$fd7$1@dough.gmane.org>	<FCA0D391-678F-485A-99F2-1ADF8AC562B5@fuhm.net>	<idcdph$pv1$1@dough.gmane.org>	<DD9AE029-32D2-43CF-BB4A-CF788830C593@fuhm.net>	<4CF9E42D.3010302@pearwood.info>
	<4CFA02DC.3060807@v.loewis.de> <4CFA1DE7.3010409@pearwood.info>
Message-ID: <4CFA326F.7030202@v.loewis.de>

> Surely even in Java or C#, objects have an *identity* even if the
> language doesn't provide a way to query their distinct *identification*.

When people said "the id of an object should this or that", they always
meant identification, not identity (perhaps except for you). Nobody
would propose that the identity (in your wording) of two distinct
objects might be the same, or might change over time.

> I'm afraid James' explanation didn't shed any light on the question to
> me. It seems to me that Java's IdentityHashValue[sic -- I think the
> correct function name is actually IdentityHashCode] is equivalent to
> Python's hash()

No, it's not. java.lang.Object.hashCode() is equivalent to Python's
hash(). In particular, for both of these, the following requirement
holds: object that *compare* equal must hash equal.

This is not the case for the identity hash code, or Python's id
function: objects that compare equal may still have different ids,
or different java identity hash codes.

> not to Python's id(), and claiming it is related to
> identity is misleading and confusing.

No, it's right on spot. This is *not* about the regular value hash,
but about the identity hash.

> Like hash(), IdentityHashCode doesn't make any promises about identity
> at all.

It sure does: calling identityHashCode on the very same object twice
will always return the same value - even after a garbage collection.

So if two references refer to the same object, calling identityHashCode
on both of them gives the same value.

This is similar to the relationship between equals and hashCode.

> Two distinct objects could have the same hash code, and a
> further test is needed to distinguish them.

Correct. However, references with different identity hash codes will
never refer to the same object.

>>> Why would you bother using that function when you could just use
>>> x == y instead?
>>
>> Because in a hash table, you also need a hash value.
> 
> Well, sure, in a hash table you need a hash value. But I was talking
> about an id() function.
> 
> So is that it? Is IdentityHashValue (or *Code, as the case may be) just
> a longer name for hash()?

Of course not.

Regards,
Martin

From stephen at xemacs.org  Sat Dec  4 13:33:06 2010
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Sat, 04 Dec 2010 21:33:06 +0900
Subject: [Python-Dev] gc ideas -- sparse memory
In-Reply-To: <4CFA1DE7.3010409@pearwood.info>
References: <AANLkTi=AchQA36yTgHrqiDovCaR1diYvRjXTg4j5P_Ni@mail.gmail.com>
	<idbt2p$1jg$1@dough.gmane.org>
	<203DD6FC-D9F7-4E7D-B49C-CABC1DF7352B@fuhm.net>
	<idc0ju$fd7$1@dough.gmane.org>
	<FCA0D391-678F-485A-99F2-1ADF8AC562B5@fuhm.net>
	<idcdph$pv1$1@dough.gmane.org>
	<DD9AE029-32D2-43CF-BB4A-CF788830C593@fuhm.net>
	<4CF9E42D.3010302@pearwood.info> <4CFA02DC.3060807@v.loewis.de>
	<4CFA1DE7.3010409@pearwood.info>
Message-ID: <87y685wwe5.fsf@uwakimon.sk.tsukuba.ac.jp>

Steven D'Aprano writes:
 > Martin v. L?wis wrote:

 > >> It seems counter-productive to me to bother with an identity function
 > >> which doesn't meet that constraint. If id(x) == id(y) implies nothing
 > >> about x and y (they may, or may not, be the same object) then what's the
 > >> point?
 > > 
 > > See James' explanation: it would be possible to use this as the
 > > foundation of an identity hash table.
 > 
 > I'm afraid James' explanation didn't shed any light on the question to 
 > me. It seems to me that Java's IdentityHashValue[sic -- I think the 
 > correct function name is actually IdentityHashCode] is equivalent to 
 > Python's hash(), not to Python's id(), and claiming it is related to 
 > identity is misleading and confusing.

Not quite equivalent.  Python's hash() obeys a certain additional
constraint (that numeric values that compare equal have the same
hash), and IdentityHashValue presumably won't work at all on numbers
in Java (since they aren't objects).

Bikeshedding aside, I have to agree with you: there's no point in
calling such a function id() or IdentityAnything().  Everything you
need to know (including both the fact that it could be an efficient
way to search for a unique object, and the possible non-uniqueness of
the object assigned that code) is contained in the word "hash".



From fijall at gmail.com  Sat Dec  4 14:01:47 2010
From: fijall at gmail.com (Maciej Fijalkowski)
Date: Sat, 4 Dec 2010 15:01:47 +0200
Subject: [Python-Dev] gc ideas -- sparse memory
In-Reply-To: <20101204123735.56609926@pitrou.net>
References: <203DD6FC-D9F7-4E7D-B49C-CABC1DF7352B@fuhm.net>
	<20101204040645.GA32257@cskk.homeip.net>
	<20101204123735.56609926@pitrou.net>
Message-ID: <AANLkTi=Wq_tVnS+aDvjccr+OXJPhenTvFitVR4KNz+Lg@mail.gmail.com>

On Sat, Dec 4, 2010 at 1:37 PM, Antoine Pitrou <solipsis at pitrou.net> wrote:
> On Sat, 4 Dec 2010 15:06:45 +1100
> Cameron Simpson <cs at zip.com.au> wrote:
>> On 03Dec2010 18:15, James Y Knight <foom at fuhm.net> wrote:
>> | On Dec 3, 2010, at 6:04 PM, Terry Reedy wrote:
>> | > gc is implementation specific. CPython uses ref counting + cycle
>> | > gc. A constraint on all implementations is that objects have a fixed,
>> | > unique id during their lifetime. CPython uses the address as the id, so
>> | > it cannot move objects. Other implementations do differently. Compacting
>> | > gc requires an id to current address table or something.
>> |
>> | It's somewhat unfortuante that python has this constraint, instead of
>> | the looser: "objects have a fixed id during their lifetime", which is
>> | much easier to implement, and practically as useful.
>>
>> Python doesn't have the constraint you imagine; it _does_ have "objects
>> have a fixed id during their lifetime".
>>
>> _CPython_ has this constraint because it uses the address as the id,
>> which is free and needs no papping or extra storage.
>
> Well, the main reason CPython has this constraint is that the C API
> mandates constant object pointers. That we can then reuse the pointer
> for id() is only a consequence of this.
>
> Regards
>
> Antoine.
>

Yes.

I think further discussion of Java-vs-Python makes no sense (and it's
proven that C-API-less python can exist and be compatible)

From dickinsm at gmail.com  Sat Dec  4 14:23:19 2010
From: dickinsm at gmail.com (Mark Dickinson)
Date: Sat, 4 Dec 2010 13:23:19 +0000
Subject: [Python-Dev] python 2 for building python 3
In-Reply-To: <20101204124109.07745d71@pitrou.net>
References: <AANLkTin7ynm7Srnp4DLa2B+825cUL+jSaKqq+Gvz34OD@mail.gmail.com>
	<4CFA0627.7010202@v.loewis.de>
	<AB0ACAA1-94D2-4ED1-AD4C-29ABD274E3DE@gmail.com>
	<20101204124109.07745d71@pitrou.net>
Message-ID: <AANLkTimhZBCKKXiP4Cxb72z-6eSy68v2UQXdxcGAO8AP@mail.gmail.com>

On Sat, Dec 4, 2010 at 11:41 AM, Antoine Pitrou <solipsis at pitrou.net> wrote:
> Er, normally you don't need *any* Python installed to build Python (be
> it 3.x or 2.x).

Are you sure about this?  I remember needing an existing Python to
building Python 2.7 on a new python-less install of FreeBSD a couple
of months ago.  But maybe that was just an issue with timestamps on
files.  I'll see if I can reproduce.

Mark

From solipsis at pitrou.net  Sat Dec  4 14:36:06 2010
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Sat, 04 Dec 2010 14:36:06 +0100
Subject: [Python-Dev] python 2 for building python 3
In-Reply-To: <AANLkTimhZBCKKXiP4Cxb72z-6eSy68v2UQXdxcGAO8AP@mail.gmail.com>
References: <AANLkTin7ynm7Srnp4DLa2B+825cUL+jSaKqq+Gvz34OD@mail.gmail.com>
	<4CFA0627.7010202@v.loewis.de>
	<AB0ACAA1-94D2-4ED1-AD4C-29ABD274E3DE@gmail.com>
	<20101204124109.07745d71@pitrou.net>
	<AANLkTimhZBCKKXiP4Cxb72z-6eSy68v2UQXdxcGAO8AP@mail.gmail.com>
Message-ID: <1291469766.3534.6.camel@localhost.localdomain>

Le samedi 04 d?cembre 2010 ? 13:23 +0000, Mark Dickinson a ?crit :
> On Sat, Dec 4, 2010 at 11:41 AM, Antoine Pitrou <solipsis at pitrou.net> wrote:
> > Er, normally you don't need *any* Python installed to build Python (be
> > it 3.x or 2.x).
> 
> Are you sure about this?  I remember needing an existing Python to
> building Python 2.7 on a new python-less install of FreeBSD a couple
> of months ago.

I remember compiling a Python 2.6 (2.5?) on a machine with no Python
installed. But that was long ago.

(of course, this is difficult to check on a standard development
machine :-))

Regards

Antoine.



From dickinsm at gmail.com  Sat Dec  4 14:39:20 2010
From: dickinsm at gmail.com (Mark Dickinson)
Date: Sat, 4 Dec 2010 13:39:20 +0000
Subject: [Python-Dev] python 2 for building python 3
In-Reply-To: <AANLkTimhZBCKKXiP4Cxb72z-6eSy68v2UQXdxcGAO8AP@mail.gmail.com>
References: <AANLkTin7ynm7Srnp4DLa2B+825cUL+jSaKqq+Gvz34OD@mail.gmail.com>
	<4CFA0627.7010202@v.loewis.de>
	<AB0ACAA1-94D2-4ED1-AD4C-29ABD274E3DE@gmail.com>
	<20101204124109.07745d71@pitrou.net>
	<AANLkTimhZBCKKXiP4Cxb72z-6eSy68v2UQXdxcGAO8AP@mail.gmail.com>
Message-ID: <AANLkTingqrC1rd7v7qVhEphG_42j8N=5anTZCT0mgdrz@mail.gmail.com>

On Sat, Dec 4, 2010 at 1:23 PM, Mark Dickinson <dickinsm at gmail.com> wrote:
> On Sat, Dec 4, 2010 at 11:41 AM, Antoine Pitrou <solipsis at pitrou.net> wrote:
>> Er, normally you don't need *any* Python installed to build Python (be
>> it 3.x or 2.x).
>
> Are you sure about this? ?I remember needing an existing Python to
> building Python 2.7 on a new python-less install of FreeBSD a couple
> of months ago. ?But maybe that was just an issue with timestamps on
> files. ?I'll see if I can reproduce.

With a fresh checkout of the release27-maint branch on an Ubuntu
64-bit VM, with /usr/bin/python renamed to /usr/bin/python_not_here, a
'./configure && make' fails with:

[...]
checking for socklen_t... yes
checking for build directories... done
configure: creating ./config.status
config.status: creating Makefile.pre
config.status: creating Modules/Setup.config
config.status: creating Misc/python.pc
config.status: creating Modules/ld_so_aix
config.status: creating pyconfig.h
creating Modules/Setup
creating Modules/Setup.local
creating Makefile
./Parser/asdl_c.py -h ./Include ./Parser/Python.asdl
/usr/bin/env: python: No such file or directory
make: *** [Include/Python-ast.h] Error 127

I think this is the same problem that I saw on the FreeBSD VM.

Mark

From benjamin at python.org  Sat Dec  4 14:40:41 2010
From: benjamin at python.org (Benjamin Peterson)
Date: Sat, 4 Dec 2010 07:40:41 -0600
Subject: [Python-Dev] python 2 for building python 3
In-Reply-To: <AANLkTingqrC1rd7v7qVhEphG_42j8N=5anTZCT0mgdrz@mail.gmail.com>
References: <AANLkTin7ynm7Srnp4DLa2B+825cUL+jSaKqq+Gvz34OD@mail.gmail.com>
	<4CFA0627.7010202@v.loewis.de>
	<AB0ACAA1-94D2-4ED1-AD4C-29ABD274E3DE@gmail.com>
	<20101204124109.07745d71@pitrou.net>
	<AANLkTimhZBCKKXiP4Cxb72z-6eSy68v2UQXdxcGAO8AP@mail.gmail.com>
	<AANLkTingqrC1rd7v7qVhEphG_42j8N=5anTZCT0mgdrz@mail.gmail.com>
Message-ID: <AANLkTi=Hmrqgp_d6MP9RBBW7bfwk6byfOOwxvL3WhbS1@mail.gmail.com>

2010/12/4 Mark Dickinson <dickinsm at gmail.com>:
> On Sat, Dec 4, 2010 at 1:23 PM, Mark Dickinson <dickinsm at gmail.com> wrote:
>> On Sat, Dec 4, 2010 at 11:41 AM, Antoine Pitrou <solipsis at pitrou.net> wrote:
>>> Er, normally you don't need *any* Python installed to build Python (be
>>> it 3.x or 2.x).
>>
>> Are you sure about this? ?I remember needing an existing Python to
>> building Python 2.7 on a new python-less install of FreeBSD a couple
>> of months ago. ?But maybe that was just an issue with timestamps on
>> files. ?I'll see if I can reproduce.
>
> With a fresh checkout of the release27-maint branch on an Ubuntu
> 64-bit VM, with /usr/bin/python renamed to /usr/bin/python_not_here, a
> './configure && make' fails with:
>
> [...]
> checking for socklen_t... yes
> checking for build directories... done
> configure: creating ./config.status
> config.status: creating Makefile.pre
> config.status: creating Modules/Setup.config
> config.status: creating Misc/python.pc
> config.status: creating Modules/ld_so_aix
> config.status: creating pyconfig.h
> creating Modules/Setup
> creating Modules/Setup.local
> creating Makefile
> ./Parser/asdl_c.py -h ./Include ./Parser/Python.asdl
> /usr/bin/env: python: No such file or directory
> make: *** [Include/Python-ast.h] Error 127
>
> I think this is the same problem that I saw on the FreeBSD VM.

You have to touch Include/Python-ast.h and Python/Python-ast.c. We do
this for release tarballs.



-- 
Regards,
Benjamin

From solipsis at pitrou.net  Sat Dec  4 14:41:55 2010
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Sat, 04 Dec 2010 14:41:55 +0100
Subject: [Python-Dev] python 2 for building python 3
In-Reply-To: <AANLkTingqrC1rd7v7qVhEphG_42j8N=5anTZCT0mgdrz@mail.gmail.com>
References: <AANLkTin7ynm7Srnp4DLa2B+825cUL+jSaKqq+Gvz34OD@mail.gmail.com>
	<4CFA0627.7010202@v.loewis.de>
	<AB0ACAA1-94D2-4ED1-AD4C-29ABD274E3DE@gmail.com>
	<20101204124109.07745d71@pitrou.net>
	<AANLkTimhZBCKKXiP4Cxb72z-6eSy68v2UQXdxcGAO8AP@mail.gmail.com>
	<AANLkTingqrC1rd7v7qVhEphG_42j8N=5anTZCT0mgdrz@mail.gmail.com>
Message-ID: <1291470115.3534.8.camel@localhost.localdomain>

Le samedi 04 d?cembre 2010 ? 13:39 +0000, Mark Dickinson a ?crit :
> On Sat, Dec 4, 2010 at 1:23 PM, Mark Dickinson <dickinsm at gmail.com> wrote:
> > On Sat, Dec 4, 2010 at 11:41 AM, Antoine Pitrou <solipsis at pitrou.net> wrote:
> >> Er, normally you don't need *any* Python installed to build Python (be
> >> it 3.x or 2.x).
> >
> > Are you sure about this?  I remember needing an existing Python to
> > building Python 2.7 on a new python-less install of FreeBSD a couple
> > of months ago.  But maybe that was just an issue with timestamps on
> > files.  I'll see if I can reproduce.
> 
> With a fresh checkout of the release27-maint branch on an Ubuntu
> 64-bit VM, with /usr/bin/python renamed to /usr/bin/python_not_here, a
> './configure && make' fails with:

How about with the release tarball? Perhaps SVN doesn't get timestamps
right.




From dickinsm at gmail.com  Sat Dec  4 14:58:07 2010
From: dickinsm at gmail.com (Mark Dickinson)
Date: Sat, 4 Dec 2010 13:58:07 +0000
Subject: [Python-Dev] python 2 for building python 3
In-Reply-To: <AANLkTi=Hmrqgp_d6MP9RBBW7bfwk6byfOOwxvL3WhbS1@mail.gmail.com>
References: <AANLkTin7ynm7Srnp4DLa2B+825cUL+jSaKqq+Gvz34OD@mail.gmail.com>
	<4CFA0627.7010202@v.loewis.de>
	<AB0ACAA1-94D2-4ED1-AD4C-29ABD274E3DE@gmail.com>
	<20101204124109.07745d71@pitrou.net>
	<AANLkTimhZBCKKXiP4Cxb72z-6eSy68v2UQXdxcGAO8AP@mail.gmail.com>
	<AANLkTingqrC1rd7v7qVhEphG_42j8N=5anTZCT0mgdrz@mail.gmail.com>
	<AANLkTi=Hmrqgp_d6MP9RBBW7bfwk6byfOOwxvL3WhbS1@mail.gmail.com>
Message-ID: <AANLkTikDQVw4CEkdekJyu0P5ZWiQC+ySm0QmTseZcDTP@mail.gmail.com>

On Sat, Dec 4, 2010 at 1:40 PM, Benjamin Peterson <benjamin at python.org> wrote:
> You have to touch Include/Python-ast.h and Python/Python-ast.c. We do
> this for release tarballs.

Ah, that makes sense.  Thanks.

Mark

From eliben at gmail.com  Sat Dec  4 15:03:42 2010
From: eliben at gmail.com (Eli Bendersky)
Date: Sat, 4 Dec 2010 16:03:42 +0200
Subject: [Python-Dev] python 2 for building python 3
In-Reply-To: <1291470115.3534.8.camel@localhost.localdomain>
References: <AANLkTin7ynm7Srnp4DLa2B+825cUL+jSaKqq+Gvz34OD@mail.gmail.com>
	<4CFA0627.7010202@v.loewis.de>
	<AB0ACAA1-94D2-4ED1-AD4C-29ABD274E3DE@gmail.com>
	<20101204124109.07745d71@pitrou.net>
	<AANLkTimhZBCKKXiP4Cxb72z-6eSy68v2UQXdxcGAO8AP@mail.gmail.com>
	<AANLkTingqrC1rd7v7qVhEphG_42j8N=5anTZCT0mgdrz@mail.gmail.com>
	<1291470115.3534.8.camel@localhost.localdomain>
Message-ID: <AANLkTikBzjzUVW5yjThawrcg2pYo+0xpcnVxRms=EDYn@mail.gmail.com>

On Sat, Dec 4, 2010 at 15:41, Antoine Pitrou <solipsis at pitrou.net> wrote:

> Le samedi 04 d?cembre 2010 ? 13:39 +0000, Mark Dickinson a ?crit :
> > On Sat, Dec 4, 2010 at 1:23 PM, Mark Dickinson <dickinsm at gmail.com>
> wrote:
> > > On Sat, Dec 4, 2010 at 11:41 AM, Antoine Pitrou <solipsis at pitrou.net>
> wrote:
> > >> Er, normally you don't need *any* Python installed to build Python (be
> > >> it 3.x or 2.x).
> > >
> > > Are you sure about this?  I remember needing an existing Python to
> > > building Python 2.7 on a new python-less install of FreeBSD a couple
> > > of months ago.  But maybe that was just an issue with timestamps on
> > > files.  I'll see if I can reproduce.
> >
> > With a fresh checkout of the release27-maint branch on an Ubuntu
> > 64-bit VM, with /usr/bin/python renamed to /usr/bin/python_not_here, a
> > './configure && make' fails with:
>
> How about with the release tarball? Perhaps SVN doesn't get timestamps
> right.
>
>
My original problem was that I was re-running 'make' on a svn py3k branch
checkout, which already had a compiled ./python exe in its root (Python 3.2,
of course). Since the script Objects/typeslots.py (which Martin checked in
just yesterday) required Python 2, this failed, although "python" on my
machine actually refers to 2.6.5. The failure then happened since in the
root of the Python build, "python" referred to the local Python 3
executable. Relying on timestamps sounds a bit too brittle.

I think it would be nice if:

1. Parts of the Makefile that use Python checked if Python is installed and
generate a useful error if not.
2. All Python scripts that are part of the build should be 2-vs-3 version
agnostic for the time being (= until Python 2 is distant history, which
won't happen soon)

Eli
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20101204/de57e038/attachment.html>

From stephen at xemacs.org  Sat Dec  4 15:22:54 2010
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Sat, 04 Dec 2010 23:22:54 +0900
Subject: [Python-Dev] gc ideas -- sparse memory
In-Reply-To: <4CFA326F.7030202@v.loewis.de>
References: <AANLkTi=AchQA36yTgHrqiDovCaR1diYvRjXTg4j5P_Ni@mail.gmail.com>
	<idbt2p$1jg$1@dough.gmane.org>
	<203DD6FC-D9F7-4E7D-B49C-CABC1DF7352B@fuhm.net>
	<idc0ju$fd7$1@dough.gmane.org>
	<FCA0D391-678F-485A-99F2-1ADF8AC562B5@fuhm.net>
	<idcdph$pv1$1@dough.gmane.org>
	<DD9AE029-32D2-43CF-BB4A-CF788830C593@fuhm.net>
	<4CF9E42D.3010302@pearwood.info> <4CFA02DC.3060807@v.loewis.de>
	<4CFA1DE7.3010409@pearwood.info> <4CFA326F.7030202@v.loewis.de>
Message-ID: <87wrnpwrb5.fsf@uwakimon.sk.tsukuba.ac.jp>

Quotes are out of order.

"Martin v. L?wis" writes:

 > No, it's not. java.lang.Object.hashCode() is equivalent to Python's
 > hash(). In particular, for both of these, the following requirement
 > holds: object that *compare* equal must hash equal.

Aha.  I see, now.  That is indeed a different concept.  Mutable
objects retain their identities across mutations.

 > > Two distinct objects could have the same hash code, and a
 > > further test is needed to distinguish them.
 > 
 > Correct. However, references with different identity hash codes will
 > never refer to the same object.

Why is useful to expose an identity hash?  AFAICS it is *only* useful
in building an identity hash table.  If so, why not just provide id()
or the is operator or both and be done with it?

This is different from exposing the value hash AFAICS, where it may be
possible to optimize value hashing of composite objects by computing
their hashes from the hashes of subobjects, perhaps omitting some
subobjects that don't affect equality (eg, Emacs strings may carry
display properties that are ignored when they are compared as
strings).

With identity, I don't see much point in leaving that up to the
programmer.  In implementations with stable object addresses, the
address is a fine implementation of identity_hash() -- perfect, in
fact.  Otherwise, you either have to omit mutable members from the
computation or store the hashcode in the object.  In the former
situation, you will likely get very coarse hashing, and the set of
immutable members (if any ;-) can be computed by the compiler.  In the
latter, you may as well store an id directly and avoid the whole hash
concept for that type (except as a space optimization, but that again
implies coarse hashing).

 > [F]or the identity hash code, or Python's id function: objects that
 > compare equal may still have different ids, or different java
 > identity hash codes.

ISTM that's no problem, you have to do an extra comparison for
identity when the codes are equal either way.  Of course the identity
hash code is a more precise optimization, but that doesn't make hash()
unusable for this purpose (cf Sun's built-in implementation of
IdentityHashCode that always returns 2).

The problem presumably lies in the fact that there are "id()-ifiable"
objects that, because they are mutable, either are unhashable (Python
hash) or would have an unstable value hash (Lisp sxhash).

Ie, AFAICS

def identity_hash (thing):
    "Wrapper for hash() suitable for building an identity hash table."
    try:
        return hash(thing)
    except TypeError:
        return 0

would be a fine implementation for Python.<wink>

 > > Like hash(), IdentityHashCode doesn't make any promises about identity
 > > at all.
 > 
 > It sure does: calling identityHashCode on the very same object twice
 > will always return the same value - even after a garbage
 > collection.

Well, if you want to put it that way, so does hash().  I read Steven
as placing more emphasis on "like hash()" than on "at all".


From a.badger at gmail.com  Sat Dec  4 17:42:32 2010
From: a.badger at gmail.com (Toshio Kuratomi)
Date: Sat, 4 Dec 2010 08:42:32 -0800
Subject: [Python-Dev] PEP 384 accepted
In-Reply-To: <4CF974B9.6040509@v.loewis.de>
References: <AANLkTi=s5NMxjPdRQRcupNBzdqjMhLO6TbFWtQzi0TEw@mail.gmail.com>
	<4CF81208.8090705@v.loewis.de>
	<AANLkTinspUo88oUnd3cX776mMRMU6bqyOaq9Og7JLwhU@mail.gmail.com>
	<4CF81AE1.5050900@v.loewis.de> <4CF880F7.9000800@netwok.org>
	<4CF89CCF.20307@v.loewis.de> <4CF8E22B.7050005@netwok.org>
	<4CF97197.2050105@v.loewis.de> <4CF973D0.4060800@netwok.org>
	<4CF974B9.6040509@v.loewis.de>
Message-ID: <20101204164232.GH3015@unaka.lan>

On Fri, Dec 03, 2010 at 11:52:41PM +0100, "Martin v. L?wis" wrote:
> Am 03.12.2010 23:48, schrieb ?ric Araujo:
> >> But I'm not interested at all in having it in distutils2. I want the
> >> Python build itself to use it, and alas, I can't because of the freeze.
> > You can?t in 3.2, true.  Neither can you in 3.1, or any previous
> > version.  If you implement it in distutils2, you have very good chances
> > to get it for 3.3.  Isn?t that a win?
> 
> It is, unfortunately, a very weak promise. Until distutils2 is
> integrated in Python, I probably won't spend any time on it.
> 
At the language summit it was proposed and seemed generally accepted (maybe
I took silence as consent... it's been almost a year now) that bold new
modules (and bold rewrites of existing modules since it fell out of the
distutils/2 discussion) should get implemented in a module on pypi before
being merged into the python stdlib.  If you wouldn't want to work on any of
those modules until they were actually integrated into Python, it sounds
like you disagree with that as a general practice?

-Toshio
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 198 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/python-dev/attachments/20101204/0e01d259/attachment.pgp>

From ziade.tarek at gmail.com  Sat Dec  4 17:55:13 2010
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Sat, 4 Dec 2010 17:55:13 +0100
Subject: [Python-Dev] PEP 384 accepted
In-Reply-To: <20101204122710.794d5fe8@pitrou.net>
References: <AANLkTimQsiU+nyX6CdqQ8JBybJdnx=1EMxX9mefHbx9H@mail.gmail.com>
	<4CF8008B.60402@v.loewis.de>
	<AANLkTikbRJEFYE_TvN++GYhmvSUu3hdhf4znOti7oty1@mail.gmail.com>
	<4CF80964.4070300@v.loewis.de>
	<AANLkTikcT0WGo3FhFVq=N6ktToNenBDNNcC4eSpefcFF@mail.gmail.com>
	<4CF80D6C.6050703@v.loewis.de>
	<AANLkTi=s5NMxjPdRQRcupNBzdqjMhLO6TbFWtQzi0TEw@mail.gmail.com>
	<4CF81208.8090705@v.loewis.de>
	<AANLkTinspUo88oUnd3cX776mMRMU6bqyOaq9Og7JLwhU@mail.gmail.com>
	<4CF81AE1.5050900@v.loewis.de> <4CF880F7.9000800@netwok.org>
	<4CF89CCF.20307@v.loewis.de> <4CF8E22B.7050005@netwok.org>
	<4CF97197.2050105@v.loewis.de> <4CF973D0.4060800@netwok.org>
	<4CF974B9.6040509@v.loewis.de>
	<294C7411-A987-45CE-ACE1-6DEFB17DA0C6@fuhm.net>
	<20101204122710.794d5fe8@pitrou.net>
Message-ID: <AANLkTinCgx-42jAx97E8OD1k7Nzr1Yg8hE2Jo_MHfJr1@mail.gmail.com>

On Sat, Dec 4, 2010 at 12:27 PM, Antoine Pitrou <solipsis at pitrou.net> wrote:
> On Fri, 3 Dec 2010 18:11:57 -0500
> James Y Knight <foom at fuhm.net> wrote:
>
>> On Dec 3, 2010, at 5:52 PM, Martin v. L?wis wrote:
>>
>> > Am 03.12.2010 23:48, schrieb ?ric Araujo:
>> >>> But I'm not interested at all in having it in distutils2. I want the
>> >>> Python build itself to use it, and alas, I can't because of the freeze.
>> >> You can?t in 3.2, true. ?Neither can you in 3.1, or any previous
>> >> version. ?If you implement it in distutils2, you have very good chances
>> >> to get it for 3.3. ?Isn?t that a win?
>> >
>> > It is, unfortunately, a very weak promise. Until distutils2 is
>> > integrated in Python, I probably won't spend any time on it.
>>
>> It seems like it'd be a good idea to start integrating distutils2 into python trunk immediately after the 3.2 branch is cut, then.
>
> +1 from me.

+1 too.

Just to clarify my position in a few sentences:

- I was told not to progressively change distutils
- the PEPs for my changes were accepted under the condition that the
changes would be made in distutils2
- PEP 384 is yet another accepted PEP, why the rule would change -
because it's a small feature ?
- I have tons of small features I had to revert and push back in distutils2

If Martin changes make it into Distutils1, I am going to ask that a
collection of small features that I have should go there too, because
they "should not break things"

Regards
Tarek

-- 
Tarek Ziad? | http://ziade.org

From merwok at netwok.org  Sat Dec  4 18:49:04 2010
From: merwok at netwok.org (=?UTF-8?B?w4lyaWMgQXJhdWpv?=)
Date: Sat, 04 Dec 2010 18:49:04 +0100
Subject: [Python-Dev] PEP 384 accepted
In-Reply-To: <AANLkTinCgx-42jAx97E8OD1k7Nzr1Yg8hE2Jo_MHfJr1@mail.gmail.com>
References: <AANLkTimQsiU+nyX6CdqQ8JBybJdnx=1EMxX9mefHbx9H@mail.gmail.com>	<4CF8008B.60402@v.loewis.de>	<AANLkTikbRJEFYE_TvN++GYhmvSUu3hdhf4znOti7oty1@mail.gmail.com>	<4CF80964.4070300@v.loewis.de>	<AANLkTikcT0WGo3FhFVq=N6ktToNenBDNNcC4eSpefcFF@mail.gmail.com>	<4CF80D6C.6050703@v.loewis.de>	<AANLkTi=s5NMxjPdRQRcupNBzdqjMhLO6TbFWtQzi0TEw@mail.gmail.com>	<4CF81208.8090705@v.loewis.de>	<AANLkTinspUo88oUnd3cX776mMRMU6bqyOaq9Og7JLwhU@mail.gmail.com>	<4CF81AE1.5050900@v.loewis.de>
	<4CF880F7.9000800@netwok.org>	<4CF89CCF.20307@v.loewis.de>
	<4CF8E22B.7050005@netwok.org>	<4CF97197.2050105@v.loewis.de>
	<4CF973D0.4060800@netwok.org>	<4CF974B9.6040509@v.loewis.de>	<294C7411-A987-45CE-ACE1-6DEFB17DA0C6@fuhm.net>	<20101204122710.794d5fe8@pitrou.net>
	<AANLkTinCgx-42jAx97E8OD1k7Nzr1Yg8hE2Jo_MHfJr1@mail.gmail.com>
Message-ID: <4CFA7F10.3010308@netwok.org>

Le 04/12/2010 17:55, Tarek Ziad? a ?crit :
> On Sat, Dec 4, 2010 at 12:27 PM, Antoine Pitrou <solipsis at pitrou.net> wrote:
>>> It seems like it'd be a good idea to start integrating distutils2 into 
>>> python trunk immediately after the 3.2 branch is cut, then.
>>
>> +1 from me.
> 
> +1 too.

+1, and I take responsibility to propose a merge plan to python-dev
after 3.2 is released and carry it out.

Regards


From greg at krypto.org  Sat Dec  4 19:14:28 2010
From: greg at krypto.org (Gregory P. Smith)
Date: Sat, 4 Dec 2010 10:14:28 -0800
Subject: [Python-Dev] r87010 - in python/branches/py3k:
 Doc/library/subprocess.rst Lib/subprocess.py Lib/test/test_subprocess.py
In-Reply-To: <20101204124502.37ec1594@pitrou.net>
References: <20101204091044.C6CBEEE985@mail.python.org>
	<20101204124502.37ec1594@pitrou.net>
Message-ID: <AANLkTi=L3CrH0Vbb5Q-UmXqzDVMJaVfeKv7YJ6Eq0QYo@mail.gmail.com>

On Sat, Dec 4, 2010 at 3:45 AM, Antoine Pitrou <solipsis at pitrou.net> wrote:

> On Sat,  4 Dec 2010 10:10:44 +0100 (CET)
> gregory.p.smith <python-checkins at python.org> wrote:
> > Author: gregory.p.smith
> > Date: Sat Dec  4 10:10:44 2010
> > New Revision: 87010
> >
> > Log:
> > issue7213 + issue2320: Cause a DeprecationWarning if the close_fds
> argument is
> > not passed to subprocess.Popen as the default value will be changing in a
> > future Python to the safer and more often desired value of True.
>
> That doesn't seem to be a good idea under Windows, is it?
>
> (?Note that on Windows, you cannot set *close_fds* to true and
> also redirect the standard handles by setting *stdin*, *stdout* or
> *stderr*.?)
>

Regardless of what platform you are on I think the warning makes sense, it
was just worded too strongly.  It is asking people to be explicit with
close_fds.  Explicitly setting close_fds=False when that is desired is good.

I modified the docs and warning message to just say that the default
close_fds behavior will change in the future without specifying exactly what
it will be.  It could make sense for the default to be a soft-True on
windows that changes behavior if any of the std handles are set if that
matches what users expect and find easiest.  We may want to reconsider its
entire future in the face of the new pass_fds parameter, etc.  Those are 3.3
decisions.

-gps
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20101204/22a2a7f0/attachment.html>

From p.f.moore at gmail.com  Sat Dec  4 20:28:34 2010
From: p.f.moore at gmail.com (Paul Moore)
Date: Sat, 4 Dec 2010 19:28:34 +0000
Subject: [Python-Dev] r87010 - in python/branches/py3k:
 Doc/library/subprocess.rst Lib/subprocess.py Lib/test/test_subprocess.py
In-Reply-To: <AANLkTi=L3CrH0Vbb5Q-UmXqzDVMJaVfeKv7YJ6Eq0QYo@mail.gmail.com>
References: <20101204091044.C6CBEEE985@mail.python.org>
	<20101204124502.37ec1594@pitrou.net>
	<AANLkTi=L3CrH0Vbb5Q-UmXqzDVMJaVfeKv7YJ6Eq0QYo@mail.gmail.com>
Message-ID: <AANLkTi=wq5sfRGe6-EHFGtyLbWakWdx8HRZUZca0vS5u@mail.gmail.com>

On 4 December 2010 18:14, Gregory P. Smith <greg at krypto.org> wrote:
>
>
> On Sat, Dec 4, 2010 at 3:45 AM, Antoine Pitrou <solipsis at pitrou.net> wrote:
>>
>> On Sat, ?4 Dec 2010 10:10:44 +0100 (CET)
>> gregory.p.smith <python-checkins at python.org> wrote:
>> > Author: gregory.p.smith
>> > Date: Sat Dec ?4 10:10:44 2010
>> > New Revision: 87010
>> >
>> > Log:
>> > issue7213 + issue2320: Cause a DeprecationWarning if the close_fds
>> > argument is
>> > not passed to subprocess.Popen as the default value will be changing in
>> > a
>> > future Python to the safer and more often desired value of True.
>>
>> That doesn't seem to be a good idea under Windows, is it?
>>
>> (?Note that on Windows, you cannot set *close_fds* to true and
>> also redirect the standard handles by setting *stdin*, *stdout* or
>> *stderr*.?)
>
> Regardless of what platform you are on I think the warning makes sense, it
> was just worded too strongly. ?It is asking people to be explicit with
> close_fds. ?Explicitly setting close_fds=False when that is desired is good.
> I modified the docs and warning message to just say that the default
> close_fds behavior will change in the future without specifying exactly what
> it will be. ?It could make sense for the default to be a soft-True on
> windows that changes behavior if any of the std handles are set if that
> matches what users expect and find easiest. ?We may want to reconsider its
> entire future in the face of the new pass_fds parameter, etc. ?Those are 3.3
> decisions.

This sounds like omitting the close_fds parameter is now considered
ill-advised, if not outright wrong. That's silly - I certainly never
specify close_fds, expecting the module to do the correct thing if I
don't know/care enough to say. I use Popen as a convenience function,
so ignoring low-level details is the whole point in my opinion (and
close_fds *is* a low-level detail for what I do, on Windows).

At the very least, the whole of the section "Replacing Older Functions
with the subprocess Module" (with a couple of small exceptions) is in
need of updating, as it is recommending bad practice (not specifying
close_fds) at the moment...

OK, I'm exaggerating a touch here. But can someone point me at the
discussion of this change? Or if there hasn't been one, can we have
one (i.e., to start the ball rolling, can someone justify the change,
please).

Paul.

From martin at v.loewis.de  Sat Dec  4 20:37:14 2010
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sat, 04 Dec 2010 20:37:14 +0100
Subject: [Python-Dev] gc ideas -- sparse memory
In-Reply-To: <87wrnpwrb5.fsf@uwakimon.sk.tsukuba.ac.jp>
References: <AANLkTi=AchQA36yTgHrqiDovCaR1diYvRjXTg4j5P_Ni@mail.gmail.com>	<idbt2p$1jg$1@dough.gmane.org>	<203DD6FC-D9F7-4E7D-B49C-CABC1DF7352B@fuhm.net>	<idc0ju$fd7$1@dough.gmane.org>	<FCA0D391-678F-485A-99F2-1ADF8AC562B5@fuhm.net>	<idcdph$pv1$1@dough.gmane.org>	<DD9AE029-32D2-43CF-BB4A-CF788830C593@fuhm.net>	<4CF9E42D.3010302@pearwood.info>	<4CFA02DC.3060807@v.loewis.de>	<4CFA1DE7.3010409@pearwood.info>	<4CFA326F.7030202@v.loewis.de>
	<87wrnpwrb5.fsf@uwakimon.sk.tsukuba.ac.jp>
Message-ID: <4CFA986A.8050200@v.loewis.de>

> Why is useful to expose an identity hash?  AFAICS it is *only* useful
> in building an identity hash table.  If so, why not just provide id()
> or the is operator or both and be done with it?

That's precisely James' point: Java provides the identity hash
*instead* of the id() function (i.e. it does not have an equivalent
of id()). Doing so gives greater liberties in implementing Java.
For example, an implementation that would only use the type and
not the instance for identity hash would still be conforming
(as would one that always returns 0).

> With identity, I don't see much point in leaving that up to the
> programmer.  In implementations with stable object addresses, the
> address is a fine implementation of identity_hash() -- perfect, in
> fact.

Of course. James' complaint is that Python-the-language mandates
support for an id() function, though - a requirement that even
implementations that don't have stable object addresses now must
support. If Python mandated only identity hash properties of the
id() function, alternative implementations could be simplified.

> ISTM that's no problem, you have to do an extra comparison for
> identity when the codes are equal either way.  Of course the identity
> hash code is a more precise optimization, but that doesn't make hash()
> unusable for this purpose (cf Sun's built-in implementation of
> IdentityHashCode that always returns 2).

That's only the case if the hash() result is guaranteed not
to change. In some applications, it may be desirable to have that
as an absolute guarantee (rather than just being a convention).

No, you can't substitute identity hash with hash: the value hash of an
object may change over time, whereas the identity hash must not.

Regards,
Martin

From martin at v.loewis.de  Sat Dec  4 20:43:27 2010
From: martin at v.loewis.de (=?ISO-8859-15?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sat, 04 Dec 2010 20:43:27 +0100
Subject: [Python-Dev] PEP 384 accepted
In-Reply-To: <20101204164232.GH3015@unaka.lan>
References: <AANLkTi=s5NMxjPdRQRcupNBzdqjMhLO6TbFWtQzi0TEw@mail.gmail.com>
	<4CF81208.8090705@v.loewis.de>
	<AANLkTinspUo88oUnd3cX776mMRMU6bqyOaq9Og7JLwhU@mail.gmail.com>
	<4CF81AE1.5050900@v.loewis.de> <4CF880F7.9000800@netwok.org>
	<4CF89CCF.20307@v.loewis.de> <4CF8E22B.7050005@netwok.org>
	<4CF97197.2050105@v.loewis.de> <4CF973D0.4060800@netwok.org>
	<4CF974B9.6040509@v.loewis.de> <20101204164232.GH3015@unaka.lan>
Message-ID: <4CFA99DF.9040907@v.loewis.de>

> At the language summit it was proposed and seemed generally accepted (maybe
> I took silence as consent... it's been almost a year now) that bold new
> modules (and bold rewrites of existing modules since it fell out of the
> distutils/2 discussion) should get implemented in a module on pypi before
> being merged into the python stdlib.  If you wouldn't want to work on any of
> those modules until they were actually integrated into Python, it sounds
> like you disagree with that as a general practice?

No - I just don't feel responsible for projects outside of CPython
(unless I have an interest in the project as a user, i.e. to scratch my
own itches). I have no use for a distutils2 package whatsoever - to
install Python packages, I would either use the builtin distutils
(preferred), or setuptools, or distribute (if the package author
requires setuptools).

I agree that new functionality should stand on its own merits first.
And if it fails that test (i.e. it doesn't get much attraction from
users when stand-alone), it shouldn't be included into Python at all.

Now, distutils2 is different: it's *not* new functionality. So perhaps
yes: I disagree with the principle that bold rewrites should be
developed independently. Such work would be much better carried out in a
branch - it will never stand on its own.

Regards,
Martin

From martin at v.loewis.de  Sat Dec  4 20:59:01 2010
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sat, 04 Dec 2010 20:59:01 +0100
Subject: [Python-Dev] python 2 for building python 3
In-Reply-To: <AANLkTikBzjzUVW5yjThawrcg2pYo+0xpcnVxRms=EDYn@mail.gmail.com>
References: <AANLkTin7ynm7Srnp4DLa2B+825cUL+jSaKqq+Gvz34OD@mail.gmail.com>	<4CFA0627.7010202@v.loewis.de>	<AB0ACAA1-94D2-4ED1-AD4C-29ABD274E3DE@gmail.com>	<20101204124109.07745d71@pitrou.net>	<AANLkTimhZBCKKXiP4Cxb72z-6eSy68v2UQXdxcGAO8AP@mail.gmail.com>	<AANLkTingqrC1rd7v7qVhEphG_42j8N=5anTZCT0mgdrz@mail.gmail.com>	<1291470115.3534.8.camel@localhost.localdomain>
	<AANLkTikBzjzUVW5yjThawrcg2pYo+0xpcnVxRms=EDYn@mail.gmail.com>
Message-ID: <4CFA9D85.3030707@v.loewis.de>

> 1. Parts of the Makefile that use Python checked if Python is installed
> and generate a useful error if not.
> 2. All Python scripts that are part of the build should be 2-vs-3
> version agnostic for the time being (= until Python 2 is distant
> history, which won't happen soon)

I think this is asked too much, unless you want to also contribute
code for that (which I really consider unnecessary). Users who build
from the source repository just need to get their build environment
right.

It's unfortunate that subversion doesn't provide relative ordering of
files after an update, perhaps Mercurial is better with that. For a
release, the time stamps in the tar file will do fine.

Regards,
Martin

From greg at krypto.org  Sat Dec  4 21:13:15 2010
From: greg at krypto.org (Gregory P. Smith)
Date: Sat, 4 Dec 2010 12:13:15 -0800
Subject: [Python-Dev] r87010 - in python/branches/py3k:
 Doc/library/subprocess.rst Lib/subprocess.py Lib/test/test_subprocess.py
In-Reply-To: <AANLkTi=wq5sfRGe6-EHFGtyLbWakWdx8HRZUZca0vS5u@mail.gmail.com>
References: <20101204091044.C6CBEEE985@mail.python.org>
	<20101204124502.37ec1594@pitrou.net>
	<AANLkTi=L3CrH0Vbb5Q-UmXqzDVMJaVfeKv7YJ6Eq0QYo@mail.gmail.com>
	<AANLkTi=wq5sfRGe6-EHFGtyLbWakWdx8HRZUZca0vS5u@mail.gmail.com>
Message-ID: <AANLkTinKW7fdKHZ1HiS3D0DkOpenuDkptufQwKwXaFk3@mail.gmail.com>

On Sat, Dec 4, 2010 at 11:28 AM, Paul Moore <p.f.moore at gmail.com> wrote:

> On 4 December 2010 18:14, Gregory P. Smith <greg at krypto.org> wrote:
> >
> >
> > On Sat, Dec 4, 2010 at 3:45 AM, Antoine Pitrou <solipsis at pitrou.net>
> wrote:
> >>
> >> On Sat,  4 Dec 2010 10:10:44 +0100 (CET)
> >> gregory.p.smith <python-checkins at python.org> wrote:
> >> > Author: gregory.p.smith
> >> > Date: Sat Dec  4 10:10:44 2010
> >> > New Revision: 87010
> >> >
> >> > Log:
> >> > issue7213 + issue2320: Cause a DeprecationWarning if the close_fds
> >> > argument is
> >> > not passed to subprocess.Popen as the default value will be changing
> in
> >> > a
> >> > future Python to the safer and more often desired value of True.
> >>
> >> That doesn't seem to be a good idea under Windows, is it?
> >>
> >> (?Note that on Windows, you cannot set *close_fds* to true and
> >> also redirect the standard handles by setting *stdin*, *stdout* or
> >> *stderr*.?)
> >
> > Regardless of what platform you are on I think the warning makes sense,
> it
> > was just worded too strongly.  It is asking people to be explicit with
> > close_fds.  Explicitly setting close_fds=False when that is desired is
> good.
> > I modified the docs and warning message to just say that the default
> > close_fds behavior will change in the future without specifying exactly
> what
> > it will be.  It could make sense for the default to be a soft-True on
> > windows that changes behavior if any of the std handles are set if that
> > matches what users expect and find easiest.  We may want to reconsider
> its
> > entire future in the face of the new pass_fds parameter, etc.  Those are
> 3.3
> > decisions.
>
> This sounds like omitting the close_fds parameter is now considered
> ill-advised, if not outright wrong. That's silly - I certainly never
> specify close_fds, expecting the module to do the correct thing if I
> don't know/care enough to say. I use Popen as a convenience function,
> so ignoring low-level details is the whole point in my opinion (and
> close_fds *is* a low-level detail for what I do, on Windows).
>
> At the very least, the whole of the section "Replacing Older Functions
> with the subprocess Module" (with a couple of small exceptions) is in
> need of updating, as it is recommending bad practice (not specifying
> close_fds) at the moment...
>
> OK, I'm exaggerating a touch here. But can someone point me at the
> discussion of this change? Or if there hasn't been one, can we have
> one (i.e., to start the ball rolling, can someone justify the change,
> please).
>
> Paul.
>

Making the change was intended to force the discussion.  I'm glad that
worked. :)

I don't like the thought of requiring people to specify close_fds either but
the default is a bad one (see http://bugs.python.org/issue7213 and
http://bugs.python.org/issue2320) so we should change it.

The real question is how should we go about doing the change?  This warning
asking people to always specify close_fds=True is heavy handed.  Other
places in the stdlib and docs no doubt still need to be updated to reflect
it, etc.


Options that seem worthy of debate:

A) The DeprecationWarning that exists in py3k as of today.

B) Remove the DeprecationWarning, simply document that people should be
explicit about it if they care at all and that the default will change in
3.3.

C) Never change close_fds behavior.  we're stuck with it for life.

D) Deprecate close_fds so that it becomes a legacy no-op.  the new pass_fds
flag could evolve into this.  this will break existing code that depends on
close_fds one way or another.


I'm in 100% agreement that forcing people to pass close_fds in makes the API
less friendly (granted, people already find it unfriendly but why make it
worse?).

Option B seems the most friendly to me.

Option D is always on the table but I haven't planned out what a future
without it should look like.  I prefer requiring people who need open file
descriptors to pass them in; a semaphore for "all fds" could be created and
pass_fds=ALL becomes the new "close_fds=False" (I think one of the bugs
suggests this).

-gps
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20101204/711d4feb/attachment.html>

From alexander.belopolsky at gmail.com  Sat Dec  4 21:29:41 2010
From: alexander.belopolsky at gmail.com (Alexander Belopolsky)
Date: Sat, 4 Dec 2010 15:29:41 -0500
Subject: [Python-Dev] Python and the Unicode Character Database
In-Reply-To: <AANLkTinXv1QSGb=oRUJutwsrWjHBppYn8UcbEeczEGFn@mail.gmail.com>
References: <AANLkTinVpySuRFkjf-Sc+7a91zrxhuQpqPQL5nM9u1w+@mail.gmail.com>
	<20101128214311.092abd35@pitrou.net> <4CF2D4E9.3060607@v.loewis.de>
	<4CF2DAD7.2000408@egenix.com> <4CF2DFD1.10901@v.loewis.de>
	<4CF7FC01.80307@egenix.com> <4CF8004D.1080408@v.loewis.de>
	<AANLkTik6LKOWrwRTrQn+tFETU5eWfuinf+=T=_yeOA8u@mail.gmail.com>
	<AANLkTinXv1QSGb=oRUJutwsrWjHBppYn8UcbEeczEGFn@mail.gmail.com>
Message-ID: <AANLkTikT2SL4h+ZAkz0PZe5G=Ecg8O971_8rX3W4MN1m@mail.gmail.com>

On Fri, Dec 3, 2010 at 12:10 AM, Alexander Belopolsky
<alexander.belopolsky at gmail.com> wrote:
..
> I don't think decimal module should support non-European decimal
> digits. ?The only place where it can make some sense is in int()
> because here we have a fighting chance of producing a reasonable
> definition. ? The motivating use case is conversion of numerical data
> extracted from text using simple '\d+' ?regex matches.
>

It turns out, this use case does not quite work in Python either:

>>> re.compile(r'\s+(\d+)\s+').match(' \u2081\u2082\u2083   ').group(1)
'???'
>>> int(_)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
UnicodeEncodeError: 'decimal' codec can't encode character '\u2081' in
position 0: invalid decimal Unicode string

This may actually be a bug in Python regex implementation because
Unicode standard seems to recommend that '\d' be interpreted as gc =
Decimal_Number (Nd):

http://unicode.org/reports/tr18/#Compatibility_Properties

I actually wonder if Python's re module can claim to provide even
Basic Unicode Support.

http://unicode.org/reports/tr18/#Basic_Unicode_Support

> Here is how I would do it:
>
> 1. ?String x of non-European decimal digits is only accepted in
> int(x), but not by int(x, 0) or int(x, 10).
> 2. ?If x contains one or more non-European digits, then
>
> ? ?(a) ?all digits must be from the same block:
>
> ? ? ?def basepoint(c):
> ? ? ? ? ? ?return ord(c) - unicodedata.digit(c)
> ? ? ?all(basepoint(c) == basepoint(x[0]) for c in x) -> True
>
> ? ? (b) and '+' or '-' sign is not alowed.
>
> 3. A character c is a digit if it matches '\d' regex. ?I think this
> means unicodedata.category(c) -> 'Nd'.
>
> Condition 2(b) is important because there is no clear way to define
> what is acceptable as '+' or '-' using Unicode character properties
> and not all number systems even support local form of negation. ?(It
> is also YAGNI.)
>

From tjreedy at udel.edu  Sat Dec  4 23:51:44 2010
From: tjreedy at udel.edu (Terry Reedy)
Date: Sat, 04 Dec 2010 17:51:44 -0500
Subject: [Python-Dev] [Python-checkins] r87010 - in
	python/branches/py3k:	Doc/library/subprocess.rst
	Lib/subprocess.py	Lib/test/test_subprocess.py
In-Reply-To: <20101204091044.C6CBEEE985@mail.python.org>
References: <20101204091044.C6CBEEE985@mail.python.org>
Message-ID: <4CFAC600.5040905@udel.edu>


> DeprecationWarnings that show up in a lot of existing code are controversial
> and have caused pain in the past.  I'd like to leave this on for 3.2 beta1 and
> see how things go.  We can remove the warning if it is deemed too noisy during
> any betas.  (case study: the md5 and sha module DeprecationWarnings are loathed
> around the world as those modules were never going to be removed in 2.x and
> 2to3 has a fixer for code that uses them)

My understanding is that DeprecationWarnings are now off by default in 
part because of issues like the above. So they should only be seen by 
programmers who turn them of for the purpose of seeing them so they can 
future-proof their code before it fails. As a result, there have been 
several added recently in efforts to clean up the APis.

The real issue is changing the default. If allowed, should it be 
switched in one release or should there be an intermediate release with 
no default. Which are you planning?

My feeling is that we should be able to correct bad-enough mistakes. 
Code can easily be made portable across all 3.x versions by given an 
explicit value and not depending on defaults. I think there should be an 
intermediate version to cause overt failure of code that is not 
upgraded, because the programmer neglected either to turn 
DeprecationWarnings on for test runs or to read the News releases.

Terry J. Reedy

From martin at v.loewis.de  Sat Dec  4 23:58:56 2010
From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=)
Date: Sat, 04 Dec 2010 23:58:56 +0100
Subject: [Python-Dev] Python and the Unicode Character Database
In-Reply-To: <AANLkTikT2SL4h+ZAkz0PZe5G=Ecg8O971_8rX3W4MN1m@mail.gmail.com>
References: <AANLkTinVpySuRFkjf-Sc+7a91zrxhuQpqPQL5nM9u1w+@mail.gmail.com>	<20101128214311.092abd35@pitrou.net>	<4CF2D4E9.3060607@v.loewis.de>	<4CF2DAD7.2000408@egenix.com>	<4CF2DFD1.10901@v.loewis.de>	<4CF7FC01.80307@egenix.com>	<4CF8004D.1080408@v.loewis.de>	<AANLkTik6LKOWrwRTrQn+tFETU5eWfuinf+=T=_yeOA8u@mail.gmail.com>	<AANLkTinXv1QSGb=oRUJutwsrWjHBppYn8UcbEeczEGFn@mail.gmail.com>
	<AANLkTikT2SL4h+ZAkz0PZe5G=Ecg8O971_8rX3W4MN1m@mail.gmail.com>
Message-ID: <4CFAC7B0.3080307@v.loewis.de>

> I actually wonder if Python's re module can claim to provide even
> Basic Unicode Support.

Do you really wonder? Most definitely it does not.

Regards,
Martin

From p.f.moore at gmail.com  Sun Dec  5 00:07:04 2010
From: p.f.moore at gmail.com (Paul Moore)
Date: Sat, 4 Dec 2010 23:07:04 +0000
Subject: [Python-Dev] r87010 - in python/branches/py3k:
 Doc/library/subprocess.rst Lib/subprocess.py Lib/test/test_subprocess.py
In-Reply-To: <AANLkTinKW7fdKHZ1HiS3D0DkOpenuDkptufQwKwXaFk3@mail.gmail.com>
References: <20101204091044.C6CBEEE985@mail.python.org>
	<20101204124502.37ec1594@pitrou.net>
	<AANLkTi=L3CrH0Vbb5Q-UmXqzDVMJaVfeKv7YJ6Eq0QYo@mail.gmail.com>
	<AANLkTi=wq5sfRGe6-EHFGtyLbWakWdx8HRZUZca0vS5u@mail.gmail.com>
	<AANLkTinKW7fdKHZ1HiS3D0DkOpenuDkptufQwKwXaFk3@mail.gmail.com>
Message-ID: <AANLkTi=+5w1+BDNraM+Yo8B9w5L8O70u57P4SrR6EvJ2@mail.gmail.com>

On 4 December 2010 20:13, Gregory P. Smith <greg at krypto.org> wrote:
>> This sounds like omitting the close_fds parameter is now considered
>> ill-advised, if not outright wrong.
[...]
> Making the change was intended to force the discussion. ?I'm glad that
> worked. :)

:-)

> I don't like the thought of requiring people to specify close_fds either but
> the default is a bad one (see http://bugs.python.org/issue7213
> and?http://bugs.python.org/issue2320) so we should change it.

Both of these issues seem to be for Unix (based on the fact that they
use grep and cat as executables - I haven't dug very deeply into
this). I work on Windows only, so I'm happy to take the experts'
opinions on that OS.

Is there an issue on Windows? If not, and given how different FD
inheritance is on Windows, I'd argue that in the absence of bug
reports, there's no need to change behaviour on Windows.

> The real question is how should we go about doing the change? ?This warning
> asking people to always specify close_fds=True is heavy handed. ?Other
> places in the stdlib and docs no doubt still need to be updated to reflect
> it, etc.
>
> Options that seem worthy of debate:
> A) The DeprecationWarning that exists in py3k as of today.

Given that the option appears not to be getting deprecated (just the
default changed), this seems wrong.

I know DeprecationWarnings are off by default. If someone switches
them on, they'd expect to have to address them. But what should they
do in this case? Unless they have thought hard about FD inheritance,
and then explicitly decided to use the default because it matches what
they want (as opposed to specifying it explicitly, and documenting
their intent clearly), nothing. Or they use the default expecting it
to be good enough, and suddenly have to worry if they are relying on
something that's changing (I have no feel for how likely it is that
they'll be affected by the change, other than to say that personally,
none of my code would be). That seems to me like a bad use of a
DeprecationWarning, as a non-trivial amount of the time there's
nothing to do.

> B) Remove the DeprecationWarning, simply document that people should be
> explicit about it if they care at all and that the default will change in
> 3.3.

That seems reasonable to me. But I can see that the backward
compatibility rules might apply here. On the other hand, given the bug
reports, surely this is a bug fix?

> C) Never change close_fds behavior. ?we're stuck with it for life.

Sounds harsh. But look at it another way - if compatibility rules mean
that we can't change the default, the reported bugs need to be fixed
another way. Or behaviour needs to be documented more explicitly so
that the bugs can be downgraded to "user error" caused by not reading
the (new, improved) documentation.

Compatibility rules are to protect working programs. What proportion
of programs using the default are "working" and what proportion are
"wrong"? It seems to me that most are "working", so compatibility
applies.

> D) Deprecate close_fds so that it becomes a legacy no-op. ?the new pass_fds
> flag could evolve into this. ?this will break existing code that depends on
> close_fds one way or another.

This works for me, as I never specify close_fds.

> I'm in 100% agreement that forcing people to pass close_fds in makes the API
> less friendly (granted, people already find it unfriendly but why make it
> worse?).

I'd say making the parameter mandatory is highly unfriendly - to the
point of making the API feel like an "advanced" one, not suitable for
simple cases like replacing os.system.

> Option B seems the most friendly to me.
> Option D is always on the table but I haven't planned out what a future
> without it should look like. ?I prefer requiring people who need open file
> descriptors to pass them in; a semaphore for "all fds" could be created and
> pass_fds=ALL becomes the new "close_fds=False" (I think one of the bugs
> suggests this).

In my view:

1. Don't change Windows behaviour unless there's a demonstrable issue.
The Popen code is split into 2 distinct paths internally anyway, so
that shouldn't be a problem. If documenting the behaviour becomes
awkward should Unix behaviour change and Windows doesn't, that
indicates to me that there's a problem with the behaviour ("if it's
hard to explain...").

2. Users who don't specify close_fds because they don't care (as
opposed to because they explicitly want the current default, but
choose not to say so - I appreciate that this is a fine distinction,
difficult to police in practice) should see no change in behaviour,
apart from identified bugs being fixed. If you're not hitting a bug,
you should see no change at all. Keep the interface straightforward
for people who don't know (or don't want to worry about) the
subtleties of FD inheritance.

The original goal was for subprocess to replace os.system, os.popen,
os.spawn, etc. That's never quite happened because subprocess is just
a little bit too conceptually complex for those basic tasks. Let's not
make it worse!

Paul.

From fuzzyman at voidspace.org.uk  Sun Dec  5 00:10:01 2010
From: fuzzyman at voidspace.org.uk (Michael Foord)
Date: Sat, 04 Dec 2010 23:10:01 +0000
Subject: [Python-Dev] PEP 384 accepted
In-Reply-To: <20101204122710.794d5fe8@pitrou.net>
References: <AANLkTimQsiU+nyX6CdqQ8JBybJdnx=1EMxX9mefHbx9H@mail.gmail.com>	<4CF8008B.60402@v.loewis.de>	<AANLkTikbRJEFYE_TvN++GYhmvSUu3hdhf4znOti7oty1@mail.gmail.com>	<4CF80964.4070300@v.loewis.de>	<AANLkTikcT0WGo3FhFVq=N6ktToNenBDNNcC4eSpefcFF@mail.gmail.com>	<4CF80D6C.6050703@v.loewis.de>	<AANLkTi=s5NMxjPdRQRcupNBzdqjMhLO6TbFWtQzi0TEw@mail.gmail.com>	<4CF81208.8090705@v.loewis.de>	<AANLkTinspUo88oUnd3cX776mMRMU6bqyOaq9Og7JLwhU@mail.gmail.com>	<4CF81AE1.5050900@v.loewis.de>
	<4CF880F7.9000800@netwok.org>	<4CF89CCF.20307@v.loewis.de>
	<4CF8E22B.7050005@netwok.org>	<4CF97197.2050105@v.loewis.de>
	<4CF973D0.4060800@netwok.org>	<4CF974B9.6040509@v.loewis.de>	<294C7411-A987-45CE-ACE1-6DEFB17DA0C6@fuhm.net>
	<20101204122710.794d5fe8@pitrou.net>
Message-ID: <4CFACA49.5060102@voidspace.org.uk>

On 04/12/2010 11:27, Antoine Pitrou wrote:
> On Fri, 3 Dec 2010 18:11:57 -0500
> James Y Knight<foom at fuhm.net>  wrote:
>
>> On Dec 3, 2010, at 5:52 PM, Martin v. L?wis wrote:
>>
>>> Am 03.12.2010 23:48, schrieb ?ric Araujo:
>>>>> But I'm not interested at all in having it in distutils2. I want the
>>>>> Python build itself to use it, and alas, I can't because of the freeze.
>>>> You can?t in 3.2, true.  Neither can you in 3.1, or any previous
>>>> version.  If you implement it in distutils2, you have very good chances
>>>> to get it for 3.3.  Isn?t that a win?
>>> It is, unfortunately, a very weak promise. Until distutils2 is
>>> integrated in Python, I probably won't spend any time on it.
>> It seems like it'd be a good idea to start integrating distutils2 into python trunk immediately after the 3.2 branch is cut, then.
> +1 from me.
>

And from me.

Michael

> Antoine.
>
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk


-- 

http://www.voidspace.org.uk/

READ CAREFULLY. By accepting and reading this email you agree,
on behalf of your employer, to release me from all obligations
and waivers arising from any and all NON-NEGOTIATED agreements,
licenses, terms-of-service, shrinkwrap, clickwrap, browsewrap,
confidentiality, non-disclosure, non-compete and acceptable use
policies (?BOGUS AGREEMENTS?) that I have entered into with your
employer, its partners, licensors, agents and assigns, in
perpetuity, without prejudice to my ongoing rights and privileges.
You further represent that you have the authority to release me
from any BOGUS AGREEMENTS on behalf of your employer.


From p.f.moore at gmail.com  Sun Dec  5 00:15:19 2010
From: p.f.moore at gmail.com (Paul Moore)
Date: Sat, 4 Dec 2010 23:15:19 +0000
Subject: [Python-Dev] [Python-checkins] r87010 - in
 python/branches/py3k: Doc/library/subprocess.rst Lib/subprocess.py
 Lib/test/test_subprocess.py
In-Reply-To: <4CFAC600.5040905@udel.edu>
References: <20101204091044.C6CBEEE985@mail.python.org>
	<4CFAC600.5040905@udel.edu>
Message-ID: <AANLkTikiPz5OWjv40sQa3XgSTpBEZp9KRfbqBp-Y46nE@mail.gmail.com>

On 4 December 2010 22:51, Terry Reedy <tjreedy at udel.edu> wrote:
> My feeling is that we should be able to correct bad-enough mistakes.

That's fair. Is this a "bad-enough mistake"? From a brief reading of
the 2 bug reports, combined with my own trouble-free experience with
Popen (always leaving close_fds as default), it doesn't seem so, to
me. If someone provides evidence that this is a serious bug in the
API, then that's a different matter.

Paul.

From p.f.moore at gmail.com  Sun Dec  5 00:17:49 2010
From: p.f.moore at gmail.com (Paul Moore)
Date: Sat, 4 Dec 2010 23:17:49 +0000
Subject: [Python-Dev] r87010 - in python/branches/py3k:
 Doc/library/subprocess.rst Lib/subprocess.py Lib/test/test_subprocess.py
In-Reply-To: <AANLkTi=+5w1+BDNraM+Yo8B9w5L8O70u57P4SrR6EvJ2@mail.gmail.com>
References: <20101204091044.C6CBEEE985@mail.python.org>
	<20101204124502.37ec1594@pitrou.net>
	<AANLkTi=L3CrH0Vbb5Q-UmXqzDVMJaVfeKv7YJ6Eq0QYo@mail.gmail.com>
	<AANLkTi=wq5sfRGe6-EHFGtyLbWakWdx8HRZUZca0vS5u@mail.gmail.com>
	<AANLkTinKW7fdKHZ1HiS3D0DkOpenuDkptufQwKwXaFk3@mail.gmail.com>
	<AANLkTi=+5w1+BDNraM+Yo8B9w5L8O70u57P4SrR6EvJ2@mail.gmail.com>
Message-ID: <AANLkTim4atz10-F_H-nHmCmNqqtfrxaQQFCQkYrVJzNX@mail.gmail.com>

On 4 December 2010 23:07, Paul Moore <p.f.moore at gmail.com> wrote:
> Is there an issue on Windows? If not, and given how different FD
> inheritance is on Windows, I'd argue that in the absence of bug
> reports, there's no need to change behaviour on Windows.

Actually, from the error message I just got:

ValueError: close_fds is not supported on Windows platforms if you
redirect stdin/stdout/stderr

So on Windows, for the issues mentioned (both of which involve
redirected stdin/out/err), leaving the current default appears to be
not only advisable, but required.

I suspect an os-dependent default would be ugly to document...

Paul.

From v+python at g.nevcal.com  Sun Dec  5 01:26:47 2010
From: v+python at g.nevcal.com (Glenn Linderman)
Date: Sat, 04 Dec 2010 16:26:47 -0800
Subject: [Python-Dev] r87010 - in python/branches/py3k:
 Doc/library/subprocess.rst Lib/subprocess.py Lib/test/test_subprocess.py
In-Reply-To: <AANLkTi=+5w1+BDNraM+Yo8B9w5L8O70u57P4SrR6EvJ2@mail.gmail.com>
References: <20101204091044.C6CBEEE985@mail.python.org>	<20101204124502.37ec1594@pitrou.net>	<AANLkTi=L3CrH0Vbb5Q-UmXqzDVMJaVfeKv7YJ6Eq0QYo@mail.gmail.com>	<AANLkTi=wq5sfRGe6-EHFGtyLbWakWdx8HRZUZca0vS5u@mail.gmail.com>	<AANLkTinKW7fdKHZ1HiS3D0DkOpenuDkptufQwKwXaFk3@mail.gmail.com>
	<AANLkTi=+5w1+BDNraM+Yo8B9w5L8O70u57P4SrR6EvJ2@mail.gmail.com>
Message-ID: <4CFADC47.7020702@g.nevcal.com>

On 12/4/2010 3:07 PM, Paul Moore wrote:
> The original goal was for subprocess to replace os.system, os.popen,
> os.spawn, etc. That's never quite happened because subprocess is just
> a little bit too conceptually complex for those basic tasks.

Is that way?  I didn't find it particularly hard to learn, given the 
"cheat sheet" of techniques for doing the replacements.

However, it is a bit deficient in providing non-blocking handling 
primitives for actually communicating with interactive spawned 
programs.  subprocess.communicate provides one technique, which works 
for an extremely limited set of circumstances: I've offered some 
alternatives in http://bugs.python.org/issue10482 that greatly expand 
the types of communications that can be achieved without deadlock.  Of 
course, it is still a small fraction of the possible circumstances, and 
doesn't really handle the hard case of feeding a program and 
concurrently analyzing its output to determine its future feedings: that 
still requires a complex, explicitly threaded and synchronized program.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20101204/381f1e56/attachment.html>

From brian.curtin at gmail.com  Sun Dec  5 03:08:35 2010
From: brian.curtin at gmail.com (Brian Curtin)
Date: Sat, 4 Dec 2010 20:08:35 -0600
Subject: [Python-Dev] [Python-checkins] r87070 -
	python/branches/py3k/Lib/test/test_shutil.py
In-Reply-To: <20101205020417.16069EE9C9@mail.python.org>
References: <20101205020417.16069EE9C9@mail.python.org>
Message-ID: <AANLkTinZVmcXp2SMvWC24VDzapsX8cWSe-Cs5vJTG+-v@mail.gmail.com>

On Sat, Dec 4, 2010 at 20:04, hirokazu.yamamoto
<python-checkins at python.org>wrote:

> Author: hirokazu.yamamoto
> Date: Sun Dec  5 03:04:16 2010
> New Revision: 87070
>
> Log:
> Now can reproduce the error on AMD64 Windows Server 2008
> even where os.symlink is not supported.
>
>
> Modified:
>   python/branches/py3k/Lib/test/test_shutil.py
>
> Modified: python/branches/py3k/Lib/test/test_shutil.py
>
> ==============================================================================
> --- python/branches/py3k/Lib/test/test_shutil.py        (original)
> +++ python/branches/py3k/Lib/test/test_shutil.py        Sun Dec  5 03:04:16
> 2010
> @@ -271,25 +271,33 @@
>             shutil.rmtree(src_dir)
>             shutil.rmtree(os.path.dirname(dst_dir))
>
> -    @unittest.skipUnless(hasattr(os, "symlink"),
> -                         "Missing symlink implementation")
> +    @unittest.skipUnless(hasattr(os, 'link'), 'requires os.link')
>     def test_dont_copy_file_onto_link_to_itself(self):
>         # bug 851123.
>         os.mkdir(TESTFN)
>         src = os.path.join(TESTFN, 'cheese')
>         dst = os.path.join(TESTFN, 'shop')
>         try:
> -            f = open(src, 'w')
> -            f.write('cheddar')
> -            f.close()
> -
> -            if hasattr(os, "link"):
> -                os.link(src, dst)
> -                self.assertRaises(shutil.Error, shutil.copyfile, src, dst)
> -                with open(src, 'r') as f:
> -                    self.assertEqual(f.read(), 'cheddar')
> -                os.remove(dst)
> +            with open(src, 'w') as f:
> +                f.write('cheddar')
> +            os.link(src, dst)
> +            self.assertRaises(shutil.Error, shutil.copyfile, src, dst)
> +            with open(src, 'r') as f:
> +                self.assertEqual(f.read(), 'cheddar')
> +            os.remove(dst)
> +        finally:
> +            shutil.rmtree(TESTFN, ignore_errors=True)
>
> +    @unittest.skipUnless(hasattr(os, "symlink"),
> +                         "Missing symlink implementation")
> +    def test_dont_copy_file_onto_symlink_to_itself(self):
> +        # bug 851123.
> +        os.mkdir(TESTFN)
> +        src = os.path.join(TESTFN, 'cheese')
> +        dst = os.path.join(TESTFN, 'shop')
> +        try:
> +            with open(src, 'w') as f:
> +                f.write('cheddar')
>             # Using `src` here would mean we end up with a symlink pointing
>             # to TESTFN/TESTFN/cheese, while it should point at
>             # TESTFN/cheese.
> @@ -299,10 +307,7 @@
>                 self.assertEqual(f.read(), 'cheddar')
>             os.remove(dst)
>         finally:
> -            try:
> -                shutil.rmtree(TESTFN)
> -            except OSError:
> -                pass
> +            shutil.rmtree(TESTFN, ignore_errors=True)
>
>     @unittest.skipUnless(hasattr(os, "symlink"),
>                          "Missing symlink implementation")
> _______________________________________________
> Python-checkins mailing list
> Python-checkins at python.org
> http://mail.python.org/mailman/listinfo/python-checkins
>

I created #10540 for this issue, but the patch I have on there is just a bad
hack. I need to fix os.path.samefile for hard links, which might be easier
if we keep st_ino data in stat structures on Windows.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20101204/f2ab10c2/attachment.html>

From ocean-city at m2.ccsnet.ne.jp  Sun Dec  5 04:15:33 2010
From: ocean-city at m2.ccsnet.ne.jp (Hirokazu Yamamoto)
Date: Sun, 05 Dec 2010 12:15:33 +0900
Subject: [Python-Dev] [Python-checkins] r87070
	-	python/branches/py3k/Lib/test/test_shutil.py
In-Reply-To: <AANLkTinZVmcXp2SMvWC24VDzapsX8cWSe-Cs5vJTG+-v@mail.gmail.com>
References: <20101205020417.16069EE9C9@mail.python.org>
	<AANLkTinZVmcXp2SMvWC24VDzapsX8cWSe-Cs5vJTG+-v@mail.gmail.com>
Message-ID: <4CFB03D5.2060204@m2.ccsnet.ne.jp>

On 2010/12/05 11:08, Brian Curtin wrote:
> I created #10540 for this issue, but the patch I have on there is just a bad
> hack. I need to fix os.path.samefile for hard links, which might be easier
> if we keep st_ino data in stat structures on Windows.

MSDN says,
http://msdn.microsoft.com/en-us/library/aa363788%28v=VS.85%29.aspx
> In the NTFS file system, a file keeps the same file ID until it is deleted.

Hard link can be live in NTFS only, so maybe we can say st_ino data is
valid until hard links are established.


From ocean-city at m2.ccsnet.ne.jp  Sun Dec  5 05:09:37 2010
From: ocean-city at m2.ccsnet.ne.jp (Hirokazu Yamamoto)
Date: Sun, 05 Dec 2010 13:09:37 +0900
Subject: [Python-Dev] [Python-checkins] r87070
	-	python/branches/py3k/Lib/test/test_shutil.py
In-Reply-To: <AANLkTinZVmcXp2SMvWC24VDzapsX8cWSe-Cs5vJTG+-v@mail.gmail.com>
References: <20101205020417.16069EE9C9@mail.python.org>
	<AANLkTinZVmcXp2SMvWC24VDzapsX8cWSe-Cs5vJTG+-v@mail.gmail.com>
Message-ID: <4CFB1081.8030805@m2.ccsnet.ne.jp>

I missed it, st_dev is not set yet. When I set st_dev with
dwVolumeSerialNumber, it was sometimes negative. Is it OK?

From stephen at xemacs.org  Sun Dec  5 09:17:37 2010
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Sun, 05 Dec 2010 17:17:37 +0900
Subject: [Python-Dev] gc ideas -- sparse memory
In-Reply-To: <4CFA986A.8050200@v.loewis.de>
References: <AANLkTi=AchQA36yTgHrqiDovCaR1diYvRjXTg4j5P_Ni@mail.gmail.com>
	<idbt2p$1jg$1@dough.gmane.org>
	<203DD6FC-D9F7-4E7D-B49C-CABC1DF7352B@fuhm.net>
	<idc0ju$fd7$1@dough.gmane.org>
	<FCA0D391-678F-485A-99F2-1ADF8AC562B5@fuhm.net>
	<idcdph$pv1$1@dough.gmane.org>
	<DD9AE029-32D2-43CF-BB4A-CF788830C593@fuhm.net>
	<4CF9E42D.3010302@pearwood.info> <4CFA02DC.3060807@v.loewis.de>
	<4CFA1DE7.3010409@pearwood.info> <4CFA326F.7030202@v.loewis.de>
	<87wrnpwrb5.fsf@uwakimon.sk.tsukuba.ac.jp>
	<4CFA986A.8050200@v.loewis.de>
Message-ID: <87sjycws4e.fsf@uwakimon.sk.tsukuba.ac.jp>

"Martin v. L?wis" writes:
 > > Why is useful to expose an identity hash?  AFAICS it is *only* useful
 > > in building an identity hash table.  If so, why not just provide id()
 > > or the is operator or both and be done with it?
 > 
 > That's precisely James' point: Java provides the identity hash
 > *instead* of the id() function (i.e. it does not have an equivalent
 > of id()). Doing so gives greater liberties in implementing Java.

Yes, we understand that it makes the implementer's job easier.  *Why
bother having an identity hash at all?*  Having taken away id() and
provided maximum leisure to the implementer via

def identity_hash(object):
    return 42

is there *any* benefit left for the user/developer?  All I see is
costs: costs in implementation, costs in debugging.  And AFAICS this
is a problem that can be solved once and reused by everybody who needs
id(); why does every developer need to write his own id() function?


From ncoghlan at gmail.com  Sun Dec  5 10:39:47 2010
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sun, 5 Dec 2010 19:39:47 +1000
Subject: [Python-Dev] [Python-checkins] r87066 -
	python/branches/py3k/Doc/whatsnew/3.2.rst
In-Reply-To: <20101204225625.CDFD8EEA29@mail.python.org>
References: <20101204225625.CDFD8EEA29@mail.python.org>
Message-ID: <AANLkTik+_QXvokx4=-cr8NzmEPP35PTwj-zXJe=jZ5+1@mail.gmail.com>

On Sun, Dec 5, 2010 at 8:56 AM, raymond.hettinger
<python-checkins at python.org> wrote:
> +.. seealso::
> +
> + ? :pep:`384` - PYC Repository Directories
> + ? ? ?PEP written by Martin von Loewis.
> +

The PEP title here should be "Defining a Stable ABI" (I noticed a
later checkin fixing the PEP 3148 see-also title, but I didn't see
anything for this one).

Cheers,
Nick.

-- 
Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia

From vinay_sajip at yahoo.co.uk  Sun Dec  5 12:36:05 2010
From: vinay_sajip at yahoo.co.uk (Vinay Sajip)
Date: Sun, 5 Dec 2010 11:36:05 +0000 (UTC)
Subject: [Python-Dev] Repo frozen for 3.2
Message-ID: <loom.20101205T123251-490@post.gmane.org>

I've just been notified via being added to the nosy list of

http://bugs.python.org/issue10627

about the deprecation of ConfigParser for 3.2. I presume I was added to this
list because logging.config uses ConfigParser, but logging.config doesn't use
any interpolation features so I could easily change all usages to
SafeConfigParser; should I do it now (before the 3.2 beta is cut) or hold off
for now?

Regards,

Vinay Sajip


From tseaver at palladion.com  Sun Dec  5 13:45:02 2010
From: tseaver at palladion.com (Tres Seaver)
Date: Sun, 05 Dec 2010 07:45:02 -0500
Subject: [Python-Dev] r87010 - in python/branches/py3k:
 Doc/library/subprocess.rst Lib/subprocess.py Lib/test/test_subprocess.py
In-Reply-To: <AANLkTinKW7fdKHZ1HiS3D0DkOpenuDkptufQwKwXaFk3@mail.gmail.com>
References: <20101204091044.C6CBEEE985@mail.python.org>	<20101204124502.37ec1594@pitrou.net>	<AANLkTi=L3CrH0Vbb5Q-UmXqzDVMJaVfeKv7YJ6Eq0QYo@mail.gmail.com>	<AANLkTi=wq5sfRGe6-EHFGtyLbWakWdx8HRZUZca0vS5u@mail.gmail.com>
	<AANLkTinKW7fdKHZ1HiS3D0DkOpenuDkptufQwKwXaFk3@mail.gmail.com>
Message-ID: <idg1gd$9nu$1@dough.gmane.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 12/04/2010 03:13 PM, Gregory P. Smith wrote:
> On Sat, Dec 4, 2010 at 11:28 AM, Paul Moore <p.f.moore at gmail.com> wrote:
> 
>> On 4 December 2010 18:14, Gregory P. Smith <greg at krypto.org> wrote:
>>>
>>>
>>> On Sat, Dec 4, 2010 at 3:45 AM, Antoine Pitrou <solipsis at pitrou.net>
>> wrote:
>>>>
>>>> On Sat,  4 Dec 2010 10:10:44 +0100 (CET)
>>>> gregory.p.smith <python-checkins at python.org> wrote:
>>>>> Author: gregory.p.smith
>>>>> Date: Sat Dec  4 10:10:44 2010
>>>>> New Revision: 87010
>>>>>
>>>>> Log:
>>>>> issue7213 + issue2320: Cause a DeprecationWarning if the close_fds
>>>>> argument is
>>>>> not passed to subprocess.Popen as the default value will be changing
>> in
>>>>> a
>>>>> future Python to the safer and more often desired value of True.
>>>>
>>>> That doesn't seem to be a good idea under Windows, is it?
>>>>
>>>> (?Note that on Windows, you cannot set *close_fds* to true and
>>>> also redirect the standard handles by setting *stdin*, *stdout* or
>>>> *stderr*.?)
>>>
>>> Regardless of what platform you are on I think the warning makes sense,
>> it
>>> was just worded too strongly.  It is asking people to be explicit with
>>> close_fds.  Explicitly setting close_fds=False when that is desired is
>> good.
>>> I modified the docs and warning message to just say that the default
>>> close_fds behavior will change in the future without specifying exactly
>> what
>>> it will be.  It could make sense for the default to be a soft-True on
>>> windows that changes behavior if any of the std handles are set if that
>>> matches what users expect and find easiest.  We may want to reconsider
>> its
>>> entire future in the face of the new pass_fds parameter, etc.  Those are
>> 3.3
>>> decisions.
>>
>> This sounds like omitting the close_fds parameter is now considered
>> ill-advised, if not outright wrong. That's silly - I certainly never
>> specify close_fds, expecting the module to do the correct thing if I
>> don't know/care enough to say. I use Popen as a convenience function,
>> so ignoring low-level details is the whole point in my opinion (and
>> close_fds *is* a low-level detail for what I do, on Windows).
>>
>> At the very least, the whole of the section "Replacing Older Functions
>> with the subprocess Module" (with a couple of small exceptions) is in
>> need of updating, as it is recommending bad practice (not specifying
>> close_fds) at the moment...
>>
>> OK, I'm exaggerating a touch here. But can someone point me at the
>> discussion of this change? Or if there hasn't been one, can we have
>> one (i.e., to start the ball rolling, can someone justify the change,
>> please).
>>
>> Paul.
>>
> 
> Making the change was intended to force the discussion.  I'm glad that
> worked. :)
> 
> I don't like the thought of requiring people to specify close_fds either but
> the default is a bad one (see http://bugs.python.org/issue7213 and
> http://bugs.python.org/issue2320) so we should change it.
> 
> The real question is how should we go about doing the change?  This warning
> asking people to always specify close_fds=True is heavy handed.  Other
> places in the stdlib and docs no doubt still need to be updated to reflect
> it, etc.
> 
> 
> Options that seem worthy of debate:
> 
> A) The DeprecationWarning that exists in py3k as of today.
> 
> B) Remove the DeprecationWarning, simply document that people should be
> explicit about it if they care at all and that the default will change in
> 3.3.
> 
> C) Never change close_fds behavior.  we're stuck with it for life.
> 
> D) Deprecate close_fds so that it becomes a legacy no-op.  the new pass_fds
> flag could evolve into this.  this will break existing code that depends on
> close_fds one way or another.
> 
> 
> I'm in 100% agreement that forcing people to pass close_fds in makes the API
> less friendly (granted, people already find it unfriendly but why make it
> worse?).
> 
> Option B seems the most friendly to me.

+1 from me.  People who don't care about the 'close_fds' behavior one
way or the other shouldn't get a warning which only helps the tiny (I
assert) minority who a) *do* care but b) don't pass the flag explicitly.



Tres..
- -- 
===================================================================
Tres Seaver          +1 540-429-0999          tseaver at palladion.com
Palladion Software   "Excellence by Design"    http://palladion.com
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.10 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAkz7iU4ACgkQ+gerLs4ltQ4SJgCfePUImv5OSHzzZ4QJvzUz1oYJ
LhAAoKRut3AfGkS23hghQx9pd3D0WF3p
=y8hn
-----END PGP SIGNATURE-----


From lukasz at langa.pl  Sun Dec  5 14:29:16 2010
From: lukasz at langa.pl (=?utf-8?Q?=C5=81ukasz_Langa?=)
Date: Sun, 5 Dec 2010 14:29:16 +0100
Subject: [Python-Dev] Repo frozen for 3.2
In-Reply-To: <loom.20101205T123251-490@post.gmane.org>
References: <loom.20101205T123251-490@post.gmane.org>
Message-ID: <FDB859C6-CC3C-4C73-AF52-01BF30BA2933@langa.pl>

Reposted for reference what originally only went to Vinay.

Wiadomo?? napisana przez Vinay Sajip w dniu 2010-12-05, o godz. 12:36:

> I've just been notified via being added to the nosy list of
> 
> http://bugs.python.org/issue10627
> 
> about the deprecation of ConfigParser for 3.2. I presume I was added to this
> list because logging.config uses ConfigParser, but logging.config doesn't use
> any interpolation features so I could easily change all usages to
> SafeConfigParser; should I do it now (before the 3.2 beta is cut) or hold off
> for now?
> 

Hold off, just like distutils2 etc. :) I finished the deprecation patch too late and so #10627 was created literally hours (if not minutes) before the freeze. Sorry for that.

On a related note, if you're sure logging users don't use any interpolation, you can also use SafeConfigParser(interpolation=None) so then all values become raw by default (e.g. people can use Python string formatting directives, % signs etc.). We can discuss this later on  when the time comes for that.

-- 
Best regards,
?ukasz Langa
tel. +48 791 080 144
WWW http://lukasz.langa.pl/

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20101205/69c53efa/attachment.html>

From merwok at netwok.org  Sun Dec  5 15:19:16 2010
From: merwok at netwok.org (=?UTF-8?B?w4lyaWMgQXJhdWpv?=)
Date: Sun, 05 Dec 2010 15:19:16 +0100
Subject: [Python-Dev] Unanswered reactions to python-checkins
In-Reply-To: <4C8D59F3.9020101@netwok.org>
References: <20100912163158.7902EF11D@mail.python.org>
	<4C8D59F3.9020101@netwok.org>
Message-ID: <4CFB9F64.90504@netwok.org>

Hello,

Three messages sent in reaction to python-checkins email have not got
any reply so far, so I?m resending them.

Regards


Nick, in reaction to the reprlib.recursive_repr commit:
>> > +        # Can't use functools.wraps() here because of bootstrap issues
>> > +        wrapper.__module__ = getattr(user_function, '__module__')
>> > +        wrapper.__doc__ = getattr(user_function, '__doc__')
>> > +        wrapper.__name__ = getattr(user_function, '__name__')
>> > +        return wrapper
> Perhaps add __wrapped__ as well?
> 
> (I assume that, similar to _collections before it was made a builtin,
> the bootstrap issue is that _functools is an extension module rather
> than builtin, but reprlib is needed when building the extension
> modules?)


Me, about a zlib-related change in Misc/NEWS:
> +zlib
> +----
> +
> +The :mod:`zlib` extension is built using an included copy of the zlib
> +sources unless the zlib version found on the system is too old to be
> +used for the build::
> 
> Unless or if?  Building with an included copy *if* the system one is too
> old makes sense to me, not the contrary.  Am I not seeing something?


Me, about a change to winsound.PlaySound:
>  Extension Modules
>  -----------------
>  
> +- Issue #6317: Now winsound.PlaySound only accepts unicode.
> +
>  - Issue #6317: Now winsound.PlaySound can accept non ascii filename.
I think the new entry should have replaced the older: ?only accepts?
trumps ?can accept?.


From solipsis at pitrou.net  Sun Dec  5 15:20:41 2010
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Sun, 5 Dec 2010 15:20:41 +0100
Subject: [Python-Dev] r87010 - in python/branches/py3k:
 Doc/library/subprocess.rst Lib/subprocess.py Lib/test/test_subprocess.py
In-Reply-To: <AANLkTim4atz10-F_H-nHmCmNqqtfrxaQQFCQkYrVJzNX@mail.gmail.com>
References: <20101204091044.C6CBEEE985@mail.python.org>
	<20101204124502.37ec1594@pitrou.net>
	<AANLkTi=L3CrH0Vbb5Q-UmXqzDVMJaVfeKv7YJ6Eq0QYo@mail.gmail.com>
	<AANLkTi=wq5sfRGe6-EHFGtyLbWakWdx8HRZUZca0vS5u@mail.gmail.com>
	<AANLkTinKW7fdKHZ1HiS3D0DkOpenuDkptufQwKwXaFk3@mail.gmail.com>
	<AANLkTi=+5w1+BDNraM+Yo8B9w5L8O70u57P4SrR6EvJ2@mail.gmail.com>
	<AANLkTim4atz10-F_H-nHmCmNqqtfrxaQQFCQkYrVJzNX@mail.gmail.com>
Message-ID: <20101205152041.06be7433@pitrou.net>

On Sat, 4 Dec 2010 23:17:49 +0000
Paul Moore <p.f.moore at gmail.com> wrote:
> On 4 December 2010 23:07, Paul Moore <p.f.moore at gmail.com> wrote:
> > Is there an issue on Windows? If not, and given how different FD
> > inheritance is on Windows, I'd argue that in the absence of bug
> > reports, there's no need to change behaviour on Windows.
> 
> Actually, from the error message I just got:
> 
> ValueError: close_fds is not supported on Windows platforms if you
> redirect stdin/stdout/stderr
> 
> So on Windows, for the issues mentioned (both of which involve
> redirected stdin/out/err), leaving the current default appears to be
> not only advisable, but required.
> 
> I suspect an os-dependent default would be ugly to document...

How about a best-effort behaviour? Setting close_fds to True would only
close fds where possible (i.e., not under Windows when piping either of
stdin, stdout, stderr).

Regards

Antoine.

From g.brandl at gmx.net  Sun Dec  5 15:23:42 2010
From: g.brandl at gmx.net (Georg Brandl)
Date: Sun, 05 Dec 2010 15:23:42 +0100
Subject: [Python-Dev] r87010 - in python/branches/py3k:
 Doc/library/subprocess.rst Lib/subprocess.py Lib/test/test_subprocess.py
In-Reply-To: <20101205152041.06be7433@pitrou.net>
References: <20101204091044.C6CBEEE985@mail.python.org>	<20101204124502.37ec1594@pitrou.net>	<AANLkTi=L3CrH0Vbb5Q-UmXqzDVMJaVfeKv7YJ6Eq0QYo@mail.gmail.com>	<AANLkTi=wq5sfRGe6-EHFGtyLbWakWdx8HRZUZca0vS5u@mail.gmail.com>	<AANLkTinKW7fdKHZ1HiS3D0DkOpenuDkptufQwKwXaFk3@mail.gmail.com>	<AANLkTi=+5w1+BDNraM+Yo8B9w5L8O70u57P4SrR6EvJ2@mail.gmail.com>	<AANLkTim4atz10-F_H-nHmCmNqqtfrxaQQFCQkYrVJzNX@mail.gmail.com>
	<20101205152041.06be7433@pitrou.net>
Message-ID: <idg7e4$nc$1@dough.gmane.org>

Am 05.12.2010 15:20, schrieb Antoine Pitrou:
> On Sat, 4 Dec 2010 23:17:49 +0000
> Paul Moore <p.f.moore at gmail.com> wrote:
>> On 4 December 2010 23:07, Paul Moore <p.f.moore at gmail.com> wrote:
>> > Is there an issue on Windows? If not, and given how different FD
>> > inheritance is on Windows, I'd argue that in the absence of bug
>> > reports, there's no need to change behaviour on Windows.
>> 
>> Actually, from the error message I just got:
>> 
>> ValueError: close_fds is not supported on Windows platforms if you
>> redirect stdin/stdout/stderr
>> 
>> So on Windows, for the issues mentioned (both of which involve
>> redirected stdin/out/err), leaving the current default appears to be
>> not only advisable, but required.
>> 
>> I suspect an os-dependent default would be ugly to document...
> 
> How about a best-effort behaviour? Setting close_fds to True would only
> close fds where possible (i.e., not under Windows when piping either of
> stdin, stdout, stderr).

That sounds reasonable.

Georg


From ocean-city at m2.ccsnet.ne.jp  Sun Dec  5 17:12:15 2010
From: ocean-city at m2.ccsnet.ne.jp (Hirokazu Yamamoto)
Date: Mon, 06 Dec 2010 01:12:15 +0900
Subject: [Python-Dev] Unanswered reactions to python-checkins
In-Reply-To: <4CFB9F64.90504@netwok.org>
References: <20100912163158.7902EF11D@mail.python.org>	<4C8D59F3.9020101@netwok.org>
	<4CFB9F64.90504@netwok.org>
Message-ID: <4CFBB9DF.1010607@m2.ccsnet.ne.jp>

On 2010/12/05 23:19, ?ric Araujo wrote:
> Me, about a change to winsound.PlaySound:
>>   Extension Modules
>>   -----------------
>>
>> +- Issue #6317: Now winsound.PlaySound only accepts unicode.
>> +
>>   - Issue #6317: Now winsound.PlaySound can accept non ascii filename.
> I think the new entry should have replaced the older: ?only accepts?
> trumps ?can accept?.

This is two difference thing. Ordinary winsound.PlaySound accepted both
bytes and str, but cannot accept non ascii filename. Now it accepts
only str, but can accept non ascii filename.

I can read "can accept" as "can accept str, but also bytes", but this
is not true.

From fdrake at acm.org  Sun Dec  5 18:10:25 2010
From: fdrake at acm.org (Fred Drake)
Date: Sun, 5 Dec 2010 12:10:25 -0500
Subject: [Python-Dev] Repo frozen for 3.2
In-Reply-To: <FDB859C6-CC3C-4C73-AF52-01BF30BA2933@langa.pl>
References: <loom.20101205T123251-490@post.gmane.org>
	<FDB859C6-CC3C-4C73-AF52-01BF30BA2933@langa.pl>
Message-ID: <AANLkTikw+3wYShqpkKQcgbJkfUprYS0Bswn-iWVs8xd4@mail.gmail.com>

2010/12/5 ?ukasz Langa <lukasz at langa.pl>:
> On a related note, if you're sure logging users don't use any interpolation,
> you can also use SafeConfigParser(interpolation=None) so then all values
> become raw by default (e.g. people can use Python string formatting
> directives, % signs etc.). We can discuss this later on ?when the time comes
> for that.

This is the hard part, though.  So long as the users decide whether to
use the interpolation features, it has to be treated as an API
compatibility issue.

The interpolation syntax is a feature of the language being parsed
more than a code-level feature.  It's actually a good thing logging is
using the ancient ConfigParser, since the interpolation handling is so
broken there's unlikely to be any affected uses of '%' in working
configurations.


? -Fred

--
Fred L. Drake, Jr.? ? <fdrake at acm.org>
"A storm broke loose in my mind."? --Albert Einstein

From p.f.moore at gmail.com  Sun Dec  5 19:02:07 2010
From: p.f.moore at gmail.com (Paul Moore)
Date: Sun, 5 Dec 2010 18:02:07 +0000
Subject: [Python-Dev] r87010 - in python/branches/py3k:
 Doc/library/subprocess.rst Lib/subprocess.py Lib/test/test_subprocess.py
In-Reply-To: <20101205152041.06be7433@pitrou.net>
References: <20101204091044.C6CBEEE985@mail.python.org>
	<20101204124502.37ec1594@pitrou.net>
	<AANLkTi=L3CrH0Vbb5Q-UmXqzDVMJaVfeKv7YJ6Eq0QYo@mail.gmail.com>
	<AANLkTi=wq5sfRGe6-EHFGtyLbWakWdx8HRZUZca0vS5u@mail.gmail.com>
	<AANLkTinKW7fdKHZ1HiS3D0DkOpenuDkptufQwKwXaFk3@mail.gmail.com>
	<AANLkTi=+5w1+BDNraM+Yo8B9w5L8O70u57P4SrR6EvJ2@mail.gmail.com>
	<AANLkTim4atz10-F_H-nHmCmNqqtfrxaQQFCQkYrVJzNX@mail.gmail.com>
	<20101205152041.06be7433@pitrou.net>
Message-ID: <AANLkTinMDC_eLKfNvJEG+6av19UQ=YExfjb8+gF-acX6@mail.gmail.com>

On 5 December 2010 14:20, Antoine Pitrou <solipsis at pitrou.net> wrote:
> How about a best-effort behaviour? Setting close_fds to True would only
> close fds where possible (i.e., not under Windows when piping either of
> stdin, stdout, stderr).

Is that plausible? I thought that it's possible to close fds, but
doesn't necessarily do the right thing. If it's possible to do this,
I'd be OK with it, but if it could cause problems just as subtle as
the ones we already have, I don't see the benefit.

Paul.

From skip at pobox.com  Sun Dec  5 19:03:33 2010
From: skip at pobox.com (skip at pobox.com)
Date: Sun, 5 Dec 2010 12:03:33 -0600
Subject: [Python-Dev] r87010 - in python/branches/py3k:
 Doc/library/subprocess.rst Lib/subprocess.py Lib/test/test_subprocess.py
In-Reply-To: <4CFADC47.7020702@g.nevcal.com>
References: <20101204091044.C6CBEEE985@mail.python.org>
	<20101204124502.37ec1594@pitrou.net>
	<AANLkTi=L3CrH0Vbb5Q-UmXqzDVMJaVfeKv7YJ6Eq0QYo@mail.gmail.com>
	<AANLkTi=wq5sfRGe6-EHFGtyLbWakWdx8HRZUZca0vS5u@mail.gmail.com>
	<AANLkTinKW7fdKHZ1HiS3D0DkOpenuDkptufQwKwXaFk3@mail.gmail.com>
	<AANLkTi=+5w1+BDNraM+Yo8B9w5L8O70u57P4SrR6EvJ2@mail.gmail.com>
	<4CFADC47.7020702@g.nevcal.com>
Message-ID: <19707.54261.55334.594655@montanaro.dyndns.org>


    Glenn> On 12/4/2010 3:07 PM, Paul Moore wrote:
    >> The original goal was for subprocess to replace os.system, os.popen,
    >> os.spawn, etc. That's never quite happened because subprocess is just
    >> a little bit too conceptually complex for those basic tasks.

    Glenn> Is that way?  I didn't find it particularly hard to learn, given
    Glenn> the "cheat sheet" of techniques for doing the replacements.

For 99% of my usage (I suspect for most other peoples' as well, at least on
Unix-y systems), this is all I need:

    for line in os.popen("some pipeline"):
        do_stuff(line)

No cheat sheet necessary.  I don't see how subprocess could have made that
common idiom any simpler.  Maybe it's better at doing esoteric stuff,
however that falls into the 1% where a simple os.popen isn't adequate.

Skip

From skip at pobox.com  Sun Dec  5 19:06:32 2010
From: skip at pobox.com (skip at pobox.com)
Date: Sun, 5 Dec 2010 12:06:32 -0600
Subject: [Python-Dev] Repo frozen for 3.2
In-Reply-To: <loom.20101205T123251-490@post.gmane.org>
References: <loom.20101205T123251-490@post.gmane.org>
Message-ID: <19707.54440.612130.268676@montanaro.dyndns.org>


    Vinay> ... the deprecation of ConfigParser for 3.2....

What's the rush?  It's been deprecated, not removed.

Skip

From greg at krypto.org  Sun Dec  5 20:08:43 2010
From: greg at krypto.org (Gregory P. Smith)
Date: Sun, 5 Dec 2010 11:08:43 -0800
Subject: [Python-Dev] r87010 - in python/branches/py3k:
 Doc/library/subprocess.rst Lib/subprocess.py Lib/test/test_subprocess.py
In-Reply-To: <idg1gd$9nu$1@dough.gmane.org>
References: <20101204091044.C6CBEEE985@mail.python.org>
	<20101204124502.37ec1594@pitrou.net>
	<AANLkTi=L3CrH0Vbb5Q-UmXqzDVMJaVfeKv7YJ6Eq0QYo@mail.gmail.com>
	<AANLkTi=wq5sfRGe6-EHFGtyLbWakWdx8HRZUZca0vS5u@mail.gmail.com>
	<AANLkTinKW7fdKHZ1HiS3D0DkOpenuDkptufQwKwXaFk3@mail.gmail.com>
	<idg1gd$9nu$1@dough.gmane.org>
Message-ID: <AANLkTimvozn1RmfNF5P7RrTx4ZwLROZ2jRbL0gy8J=+-@mail.gmail.com>

On Sun, Dec 5, 2010 at 4:45 AM, Tres Seaver <tseaver at palladion.com> wrote:

> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> On 12/04/2010 03:13 PM, Gregory P. Smith wrote:
> >
> > Making the change was intended to force the discussion.  I'm glad that
> > worked. :)
> >
> > I don't like the thought of requiring people to specify close_fds either
> but
> > the default is a bad one (see http://bugs.python.org/issue7213 and
> > http://bugs.python.org/issue2320) so we should change it.
> >
> > The real question is how should we go about doing the change?  This
> warning
> > asking people to always specify close_fds=True is heavy handed.  Other
> > places in the stdlib and docs no doubt still need to be updated to
> reflect
> > it, etc.
> >
> >
> > Options that seem worthy of debate:
> >
> > A) The DeprecationWarning that exists in py3k as of today.
> >
> > B) Remove the DeprecationWarning, simply document that people should be
> > explicit about it if they care at all and that the default will change in
> > 3.3.
> >
> > C) Never change close_fds behavior.  we're stuck with it for life.
> >
> > D) Deprecate close_fds so that it becomes a legacy no-op.  the new
> pass_fds
> > flag could evolve into this.  this will break existing code that depends
> on
> > close_fds one way or another.
> >
> >
> > I'm in 100% agreement that forcing people to pass close_fds in makes the
> API
> > less friendly (granted, people already find it unfriendly but why make it
> > worse?).
> >
> > Option B seems the most friendly to me.
>
> +1 from me.  People who don't care about the 'close_fds' behavior one
> way or the other shouldn't get a warning which only helps the tiny (I
> assert) minority who a) *do* care but b) don't pass the flag explicitly.
>
>
Sleeping on the issue some more and pondering it...

Is there any _good_ reason not to just make the close_fds default change in
3.2 today?  No warning (since they're never seen by most people anyways).
 Document it in Misc/NEWS and whatsnew.rst.

The most common use case is likely the simpler one where close_fds is not
specified because the caller doesn't care, not because they are relying on
the default being False.

Another reason why I think changing it today is okay: This is Python 3.x.
 It has not yet seen wide adoption.  It isn't likely to see wide adoption
until 3.2.  People migrating code to 3.2 are more likely to be migrating
from 2.x than from 3.1.  2to3 can have a fixer added to specify
close_fds=False on subprocess calls.

Consider this option B-prime.

-gps
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20101205/a8204afa/attachment.html>

From tseaver at palladion.com  Sun Dec  5 20:11:18 2010
From: tseaver at palladion.com (Tres Seaver)
Date: Sun, 05 Dec 2010 14:11:18 -0500
Subject: [Python-Dev] Repo frozen for 3.2
In-Reply-To: <19707.54440.612130.268676@montanaro.dyndns.org>
References: <loom.20101205T123251-490@post.gmane.org>
	<19707.54440.612130.268676@montanaro.dyndns.org>
Message-ID: <idgo4l$9or$1@dough.gmane.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 12/05/2010 01:06 PM, skip at pobox.com wrote:
> 
>     Vinay> ... the deprecation of ConfigParser for 3.2....
> 
> What's the rush?  It's been deprecated, not removed.

Having modules in the stdlib ship trigger (not raise)
DeprecationWarnings is bad housekeeping / "broken window" stuff:  it
leads application developers to ignore them.


Tres.
- -- 
===================================================================
Tres Seaver          +1 540-429-0999          tseaver at palladion.com
Palladion Software   "Excellence by Design"    http://palladion.com
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.10 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAkz749YACgkQ+gerLs4ltQ6WyACcDrHkqRVYl11ctTJ5AswTK51Q
evMAni7X2BdCbkyXzX7vLQCNMzoMj1yg
=Ui6Y
-----END PGP SIGNATURE-----


From solipsis at pitrou.net  Sun Dec  5 20:12:51 2010
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Sun, 5 Dec 2010 20:12:51 +0100
Subject: [Python-Dev] r87010 - in python/branches/py3k:
 Doc/library/subprocess.rst Lib/subprocess.py Lib/test/test_subprocess.py
References: <20101204091044.C6CBEEE985@mail.python.org>
	<20101204124502.37ec1594@pitrou.net>
	<AANLkTi=L3CrH0Vbb5Q-UmXqzDVMJaVfeKv7YJ6Eq0QYo@mail.gmail.com>
	<AANLkTi=wq5sfRGe6-EHFGtyLbWakWdx8HRZUZca0vS5u@mail.gmail.com>
	<AANLkTinKW7fdKHZ1HiS3D0DkOpenuDkptufQwKwXaFk3@mail.gmail.com>
	<idg1gd$9nu$1@dough.gmane.org>
	<AANLkTimvozn1RmfNF5P7RrTx4ZwLROZ2jRbL0gy8J=+-@mail.gmail.com>
Message-ID: <20101205201251.2339b854@pitrou.net>

On Sun, 5 Dec 2010 11:08:43 -0800
"Gregory P. Smith" <greg at krypto.org> wrote:
> Sleeping on the issue some more and pondering it...
> 
> Is there any _good_ reason not to just make the close_fds default change in
> 3.2 today?  No warning (since they're never seen by most people anyways).
>  Document it in Misc/NEWS and whatsnew.rst.

Yes, it will break many scripts under Windows.
That's the core of the issue, really. I agree under Unix it makes
sense to change the default.

Regards

Antoine.



From greg at krypto.org  Sun Dec  5 20:21:32 2010
From: greg at krypto.org (Gregory P. Smith)
Date: Sun, 5 Dec 2010 11:21:32 -0800
Subject: [Python-Dev] r87010 - in python/branches/py3k:
 Doc/library/subprocess.rst Lib/subprocess.py Lib/test/test_subprocess.py
In-Reply-To: <20101205201251.2339b854@pitrou.net>
References: <20101204091044.C6CBEEE985@mail.python.org>
	<20101204124502.37ec1594@pitrou.net>
	<AANLkTi=L3CrH0Vbb5Q-UmXqzDVMJaVfeKv7YJ6Eq0QYo@mail.gmail.com>
	<AANLkTi=wq5sfRGe6-EHFGtyLbWakWdx8HRZUZca0vS5u@mail.gmail.com>
	<AANLkTinKW7fdKHZ1HiS3D0DkOpenuDkptufQwKwXaFk3@mail.gmail.com>
	<idg1gd$9nu$1@dough.gmane.org>
	<AANLkTimvozn1RmfNF5P7RrTx4ZwLROZ2jRbL0gy8J=+-@mail.gmail.com>
	<20101205201251.2339b854@pitrou.net>
Message-ID: <AANLkTimRJ67t_K7m9rzOFnUzqH4PNHemxxp4mMd=U-J0@mail.gmail.com>

On Sun, Dec 5, 2010 at 11:12 AM, Antoine Pitrou <solipsis at pitrou.net> wrote:

> On Sun, 5 Dec 2010 11:08:43 -0800
> "Gregory P. Smith" <greg at krypto.org> wrote:
> > Sleeping on the issue some more and pondering it...
> >
> > Is there any _good_ reason not to just make the close_fds default change
> in
> > 3.2 today?  No warning (since they're never seen by most people anyways).
> >  Document it in Misc/NEWS and whatsnew.rst.
>
> Yes, it will break many scripts under Windows.
> That's the core of the issue, really. I agree under Unix it makes
> sense to change the default.
>

Sorry, yes, I was ignoring windows in the above statement.  I only want the
change in default on unix, windows has different needs.


>
> Regards
>
> Antoine.
>
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> http://mail.python.org/mailman/options/python-dev/greg%40krypto.org
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20101205/ed9bae6d/attachment.html>

From martin at v.loewis.de  Sun Dec  5 22:48:49 2010
From: martin at v.loewis.de (=?ISO-8859-15?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sun, 05 Dec 2010 22:48:49 +0100
Subject: [Python-Dev] PEP 11: Dropping support for ten year old systems
Message-ID: <4CFC08C1.3080609@v.loewis.de>

I'd like to tighten PEP 11, and declare a policy that systems
older than ten years at the point of a feature release are not
supported anymore by default. Older systems where support is still
maintained need to be explicitly listed in the PEP, along with
the name of the responsible maintainer (I think this would currently
only cover OS2/EMX which is maintained by Andrew MacIntyre).

Support for such old platforms can then be removed from the codebase
immediately, no need to go through a PEP 11 deprecation cycle.

As a consequence, I would then like to remove support for Solaris
versions older than Solaris 8 (released in January 2000, last updated
by Sun in 2004) from the configure script for 3.2b2. A number of other
tests in configure.in could probably also be removed, although I
personally won't touch them before 3.2.

The other major system affected by this would be Windows 2000, for which
we already decided to not support it anymore.

Opinions?

Regards,
Martin

From solipsis at pitrou.net  Sun Dec  5 23:14:30 2010
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Sun, 5 Dec 2010 23:14:30 +0100
Subject: [Python-Dev] PEP 11: Dropping support for ten year old systems
References: <4CFC08C1.3080609@v.loewis.de>
Message-ID: <20101205231430.1e12ad59@pitrou.net>

On Sun, 05 Dec 2010 22:48:49 +0100
"Martin v. L?wis" <martin at v.loewis.de> wrote:
> I'd like to tighten PEP 11, and declare a policy that systems
> older than ten years at the point of a feature release are not
> supported anymore by default. Older systems where support is still
> maintained need to be explicitly listed in the PEP, along with
> the name of the responsible maintainer (I think this would currently
> only cover OS2/EMX which is maintained by Andrew MacIntyre).
> 
> Support for such old platforms can then be removed from the codebase
> immediately, no need to go through a PEP 11 deprecation cycle.

Seconded!

> The other major system affected by this would be Windows 2000, for which
> we already decided to not support it anymore.

Is there any 2000-specific code (as opposed to XP-compatible)?

Regards

Antoine.



From victor.stinner at haypocalc.com  Sun Dec  5 23:25:27 2010
From: victor.stinner at haypocalc.com (Victor Stinner)
Date: Sun, 5 Dec 2010 23:25:27 +0100
Subject: [Python-Dev] transform() and untransform() methods,
	and the codec registry
In-Reply-To: <8762vax7lj.fsf@uwakimon.sk.tsukuba.ac.jp>
References: <20101202180651.F2AB0EE9CD@mail.python.org>
	<AANLkTikoRZJ4m3oGBhzgJ1_8WoJWf6ik_jm_uPt600pc@mail.gmail.com>
	<8762vax7lj.fsf@uwakimon.sk.tsukuba.ac.jp>
Message-ID: <201012052325.27903.victor.stinner@haypocalc.com>

On Saturday 04 December 2010 09:31:04 you wrote:
> Alexander Belopolsky writes:
>  > In fact, once the language moratorium is over, I will argue that
>  > str.encode() and byte.decode() should deprecate encoding argument and
>  > just do UTF-8 encoding/decoding.  Hopefully by that time most people
>  > will forget that other encodings exist.  (I can dream, right?)
> 
> It's just a dream.  There's a pile of archival material, often on R/O
> media, out there that won't be transcoded any more quickly than the
> inscriptions on Tutankhamun's tomb.

Not only, many libraries expect use bytes arguments encoded to a specific 
encoding (eg. locale encoding). Said differenlty, only few libraries written in 
C accept wchar* strings.

The Linux kernel (or many, or all, UNIX/BSD kernels) only manipulate byte 
strings. The libc only accept wide characters for a few operations. I don't 
know how to open a file with an unicode path with the Linux libc: you have to 
encode it...

Alexander: you should first patch all UNIX/BSD kernels to use unicode 
everywhere, then patch all libc implementations, and then all libraries 
(written in C). After that, you can have a break.

Victor

From greg.ewing at canterbury.ac.nz  Sun Dec  5 23:29:27 2010
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Mon, 06 Dec 2010 11:29:27 +1300
Subject: [Python-Dev] gc ideas -- sparse memory
In-Reply-To: <4CF9FF4B.1030004@v.loewis.de>
References: <AANLkTi=AchQA36yTgHrqiDovCaR1diYvRjXTg4j5P_Ni@mail.gmail.com>
	<idbt2p$1jg$1@dough.gmane.org>
	<AANLkTi=bGGh1_Z3-ey+4rbvNHHmi=SgOSUo_TQjvK_p-@mail.gmail.com>
	<4CF98117.5060301@v.loewis.de>
	<AANLkTin8TZ7gZg1tkbefN5WjkP4_GeX4sE-wrvbyVSQT@mail.gmail.com>
	<4CF9FF4B.1030004@v.loewis.de>
Message-ID: <4CFC1247.4070703@canterbury.ac.nz>

Martin v. L?wis wrote:
> The hole C API would break if objects would move in memory.
> Since they have to stay at fixed addresses, it's easy enough to use the
> address as ID.

Yes. Some of the discussion here seems to be assuming that the
reason Python doesn't move objects is so that it can use the
address as a unique ID. But it's the other way around.

-- 
Greg


From brett at python.org  Mon Dec  6 00:26:05 2010
From: brett at python.org (Brett Cannon)
Date: Sun, 5 Dec 2010 15:26:05 -0800
Subject: [Python-Dev] PEP 11: Dropping support for ten year old systems
In-Reply-To: <20101205231430.1e12ad59@pitrou.net>
References: <4CFC08C1.3080609@v.loewis.de> <20101205231430.1e12ad59@pitrou.net>
Message-ID: <AANLkTikfHdsXt7rJK3gs+GQetoBwSFAW+BB142xRRFYF@mail.gmail.com>

On Sun, Dec 5, 2010 at 14:14, Antoine Pitrou <solipsis at pitrou.net> wrote:
> On Sun, 05 Dec 2010 22:48:49 +0100
> "Martin v. L?wis" <martin at v.loewis.de> wrote:
>> I'd like to tighten PEP 11, and declare a policy that systems
>> older than ten years at the point of a feature release are not
>> supported anymore by default. Older systems where support is still
>> maintained need to be explicitly listed in the PEP, along with
>> the name of the responsible maintainer (I think this would currently
>> only cover OS2/EMX which is maintained by Andrew MacIntyre).
>>
>> Support for such old platforms can then be removed from the codebase
>> immediately, no need to go through a PEP 11 deprecation cycle.
>
> Seconded!

Thirded!

-Brett

>
>> The other major system affected by this would be Windows 2000, for which
>> we already decided to not support it anymore.
>
> Is there any 2000-specific code (as opposed to XP-compatible)?
>
> Regards
>
> Antoine.
>
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/brett%40python.org
>

From martin at v.loewis.de  Mon Dec  6 00:35:47 2010
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Mon, 06 Dec 2010 00:35:47 +0100
Subject: [Python-Dev] PEP 11: Dropping support for ten year old systems
In-Reply-To: <20101205231430.1e12ad59@pitrou.net>
References: <4CFC08C1.3080609@v.loewis.de> <20101205231430.1e12ad59@pitrou.net>
Message-ID: <4CFC21D3.4060607@v.loewis.de>

>> The other major system affected by this would be Windows 2000, for which
>> we already decided to not support it anymore.
> 
> Is there any 2000-specific code (as opposed to XP-compatible)?

Yes: a number of APIs didn't exist in W2k, so we currently use
LoadLibrary/GetProcAddress to call them. These can be replaced with
regular calls. _WINNT (or whatever the macro is called) then needs to
be bumped accordingly.

Regards,
Martin

From tjreedy at udel.edu  Mon Dec  6 00:45:02 2010
From: tjreedy at udel.edu (Terry Reedy)
Date: Sun, 05 Dec 2010 18:45:02 -0500
Subject: [Python-Dev] PEP 11: Dropping support for ten year old systems
In-Reply-To: <4CFC08C1.3080609@v.loewis.de>
References: <4CFC08C1.3080609@v.loewis.de>
Message-ID: <idh85s$q98$1@dough.gmane.org>

On 12/5/2010 4:48 PM, "Martin v. L?wis" wrote:
> I'd like to tighten PEP 11, and declare a policy that systems
> older than ten years at the point of a feature release are not
> supported anymore by default. Older systems where support is still
> maintained need to be explicitly listed in the PEP, along with
> the name of the responsible maintainer (I think this would currently
> only cover OS2/EMX which is maintained by Andrew MacIntyre).
>
> Support for such old platforms can then be removed from the codebase
> immediately, no need to go through a PEP 11 deprecation cycle.
>
> As a consequence, I would then like to remove support for Solaris
> versions older than Solaris 8 (released in January 2000, last updated
> by Sun in 2004) from the configure script for 3.2b2. A number of other
> tests in configure.in could probably also be removed, although I
> personally won't touch them before 3.2.
>
> The other major system affected by this would be Windows 2000, for which
> we already decided to not support it anymore.

WinXP (released August 2001) should be supported a lot longer than 
another year ;-) . It is still supported and installed on new systems.

-- 
Terry Jan Reedy



From v+python at g.nevcal.com  Mon Dec  6 00:55:43 2010
From: v+python at g.nevcal.com (Glenn Linderman)
Date: Sun, 05 Dec 2010 15:55:43 -0800
Subject: [Python-Dev] r87010 - in python/branches/py3k:
 Doc/library/subprocess.rst Lib/subprocess.py Lib/test/test_subprocess.py
In-Reply-To: <19707.54261.55334.594655@montanaro.dyndns.org>
References: <20101204091044.C6CBEEE985@mail.python.org>
	<20101204124502.37ec1594@pitrou.net>
	<AANLkTi=L3CrH0Vbb5Q-UmXqzDVMJaVfeKv7YJ6Eq0QYo@mail.gmail.com>
	<AANLkTi=wq5sfRGe6-EHFGtyLbWakWdx8HRZUZca0vS5u@mail.gmail.com>
	<AANLkTinKW7fdKHZ1HiS3D0DkOpenuDkptufQwKwXaFk3@mail.gmail.com>
	<AANLkTi=+5w1+BDNraM+Yo8B9w5L8O70u57P4SrR6EvJ2@mail.gmail.com>
	<4CFADC47.7020702@g.nevcal.com>
	<19707.54261.55334.594655@montanaro.dyndns.org>
Message-ID: <4CFC267F.7000105@g.nevcal.com>

On 12/5/2010 10:03 AM, skip at pobox.com wrote:
>      Glenn>  On 12/4/2010 3:07 PM, Paul Moore wrote:
>      >>  The original goal was for subprocess to replace os.system, os.popen,
>      >>  os.spawn, etc. That's never quite happened because subprocess is just
>      >>  a little bit too conceptually complex for those basic tasks.
>
>      Glenn>  Is that way?  I didn't find it particularly hard to learn, given
>      Glenn>  the "cheat sheet" of techniques for doing the replacements.
>
> For 99% of my usage (I suspect for most other peoples' as well, at least on
> Unix-y systems), this is all I need:
>
>      for line in os.popen("some pipeline"):
>          do_stuff(line)
>
> No cheat sheet necessary.  I don't see how subprocess could have made that
> common idiom any simpler.  Maybe it's better at doing esoteric stuff,
> however that falls into the 1% where a simple os.popen isn't adequate.
>
> Skip

So it couldn't make it any simpler.  For your 99% usage, the question 
is, does it make it harder?  And the answer is, at least a little bit... 
you have to type more...

import subprocess
for line in subprocess.Popen("cmd", shell=True, stdout=PIPE).stdout:
     do_stuff(line)

Sad.  It does seem like some wrapper functions could have been provided 
to make the easy cases at least as easy as they were before...

My 99% case involves web servers and CGI-type interfaces.  And 
subprocess doesn't provide quite enough to handle the job.  It is a bit 
more complex than your pipeline case, and subprocess does allow the job 
to be done, and allows it to be done better than popen or popen2 could 
ever do it.  But the helper functions I suggest in the issue would make 
it lots easier.  And probably, it would be nice to include starting the 
threads within the helper functions, too.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20101205/2ac8b2c7/attachment-0001.html>

From eliben at gmail.com  Mon Dec  6 05:18:43 2010
From: eliben at gmail.com (Eli Bendersky)
Date: Mon, 6 Dec 2010 06:18:43 +0200
Subject: [Python-Dev] PEP 11: Dropping support for ten year old systems
In-Reply-To: <idh85s$q98$1@dough.gmane.org>
References: <4CFC08C1.3080609@v.loewis.de> <idh85s$q98$1@dough.gmane.org>
Message-ID: <AANLkTikHX68wEZAd9wMe5AJD-jvUdCLymhkbkdhx5=yS@mail.gmail.com>

> The other major system affected by this would be Windows 2000, for which
>> we already decided to not support it anymore.
>>
>
> WinXP (released August 2001) should be supported a lot longer than another
> year ;-) . It is still supported and installed on new systems.
>

Good catch. Windows XP, according to reports online has somewhere between
50-75% marketshare of all personal computers in the world. The Vista fiasco
has created a large void which is only now starting to get filled with
Windows 7, but it will yet take a long time to dethrone XP.

Eli
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20101206/4b89ee0f/attachment.html>

From ncoghlan at gmail.com  Mon Dec  6 05:25:30 2010
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Mon, 6 Dec 2010 14:25:30 +1000
Subject: [Python-Dev] transform() and untransform() methods,
	and the codec registry
In-Reply-To: <201012052325.27903.victor.stinner@haypocalc.com>
References: <20101202180651.F2AB0EE9CD@mail.python.org>
	<AANLkTikoRZJ4m3oGBhzgJ1_8WoJWf6ik_jm_uPt600pc@mail.gmail.com>
	<8762vax7lj.fsf@uwakimon.sk.tsukuba.ac.jp>
	<201012052325.27903.victor.stinner@haypocalc.com>
Message-ID: <AANLkTin1x-B5+7h3sN_kzrcQeke=vw-pjqZ3ZPTi6nXb@mail.gmail.com>

On Mon, Dec 6, 2010 at 8:25 AM, Victor Stinner
<victor.stinner at haypocalc.com> wrote:
> Not only, many libraries expect use bytes arguments encoded to a specific
> encoding (eg. locale encoding). Said differenlty, only few libraries written in
> C accept wchar* strings.
>
> The Linux kernel (or many, or all, UNIX/BSD kernels) only manipulate byte
> strings. The libc only accept wide characters for a few operations. I don't
> know how to open a file with an unicode path with the Linux libc: you have to
> encode it...
>
> Alexander: you should first patch all UNIX/BSD kernels to use unicode
> everywhere, then patch all libc implementations, and then all libraries
> (written in C). After that, you can have a break.

Slightly less ambitious is to get them all to agree to standardise on
UTF-8 as the encoding mechanism (which is actually in the process of
happening, it just has a long way to go).

However, as a glue language, it is part of Python's role to help
manage the transition from legacy encodings to UTF-8, so it will be a
very long time before the idea of removing support for the encoding
argument becomes even remotely feasible.

Cheers,
Nick.

-- 
Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia

From ncoghlan at gmail.com  Mon Dec  6 05:36:13 2010
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Mon, 6 Dec 2010 14:36:13 +1000
Subject: [Python-Dev] PEP 11: Dropping support for ten year old systems
In-Reply-To: <4CFC08C1.3080609@v.loewis.de>
References: <4CFC08C1.3080609@v.loewis.de>
Message-ID: <AANLkTinvJ9AUeHH58r_Kbfk9mTZGoSLHtG0UpnDqmsRU@mail.gmail.com>

On Mon, Dec 6, 2010 at 7:48 AM, "Martin v. L?wis" <martin at v.loewis.de> wrote:
> I'd like to tighten PEP 11, and declare a policy that systems
> older than ten years at the point of a feature release are not
> supported anymore by default. Older systems where support is still
> maintained need to be explicitly listed in the PEP, along with
> the name of the responsible maintainer (I think this would currently
> only cover OS2/EMX which is maintained by Andrew MacIntyre).
>
> Support for such old platforms can then be removed from the codebase
> immediately, no need to go through a PEP 11 deprecation cycle.
>
> As a consequence, I would then like to remove support for Solaris
> versions older than Solaris 8 (released in January 2000, last updated
> by Sun in 2004) from the configure script for 3.2b2. A number of other
> tests in configure.in could probably also be removed, although I
> personally won't touch them before 3.2.
>
> The other major system affected by this would be Windows 2000, for which
> we already decided to not support it anymore.
>
> Opinions?

I would prefer to be guided by vendor EOL dates rather than our own
arbitrary 10 year limit. The EOL guide I would suggest is "Is the
vendor still fixing bugs in that release?".

For Solaris 8, the answer to that question is no (Phase 1 EOL support
ended in October: http://www.sun.com/service/eosl/eosl_solaris.html)

For Windows 2000, the answer is no (Extended Support ended in July:
http://support.microsoft.com/lifecycle/?p1=7274)

For Windows XP, the answer is yes (Extended Support doesn't end until
April 2014: http://windows.microsoft.com/en-us/windows/products/lifecycle#section_2)

Since the "Is the vendor still patching it?" guideline gives the
"right" answer for the 3 systems mentioned in this thread, it will
likely do a better job of covering anomalies like XP than a flat year
limit would.

Cheers,
Nick.

-- 
Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia

From ocean-city at m2.ccsnet.ne.jp  Mon Dec  6 06:04:46 2010
From: ocean-city at m2.ccsnet.ne.jp (Hirokazu Yamamoto)
Date: Mon, 06 Dec 2010 14:04:46 +0900
Subject: [Python-Dev] PEP 11: Dropping support for ten year old systems
In-Reply-To: <4CFC08C1.3080609@v.loewis.de>
References: <4CFC08C1.3080609@v.loewis.de>
Message-ID: <4CFC6EEE.2080505@m2.ccsnet.ne.jp>

On 2010/12/06 6:48, "Martin v. L?wis" wrote:
> The other major system affected by this would be Windows 2000, for which
> we already decided to not support it anymore.
>
> Opinions?

I'm +1/2 for supporting Windows 2000...

From martin at v.loewis.de  Mon Dec  6 08:28:27 2010
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Mon, 06 Dec 2010 08:28:27 +0100
Subject: [Python-Dev] PEP 11: Dropping support for ten year old systems
In-Reply-To: <AANLkTinvJ9AUeHH58r_Kbfk9mTZGoSLHtG0UpnDqmsRU@mail.gmail.com>
References: <4CFC08C1.3080609@v.loewis.de>
	<AANLkTinvJ9AUeHH58r_Kbfk9mTZGoSLHtG0UpnDqmsRU@mail.gmail.com>
Message-ID: <4CFC909B.10208@v.loewis.de>

Am 06.12.2010 05:36, schrieb Nick Coghlan:
> On Mon, Dec 6, 2010 at 7:48 AM, "Martin v. L?wis" <martin at v.loewis.de> wrote:
>> I'd like to tighten PEP 11, and declare a policy that systems
>> older than ten years at the point of a feature release are not
>> supported anymore by default. Older systems where support is still
>> maintained need to be explicitly listed in the PEP, along with
>> the name of the responsible maintainer (I think this would currently
>> only cover OS2/EMX which is maintained by Andrew MacIntyre).
>>
>> Support for such old platforms can then be removed from the codebase
>> immediately, no need to go through a PEP 11 deprecation cycle.
>>
>> As a consequence, I would then like to remove support for Solaris
>> versions older than Solaris 8 (released in January 2000, last updated
>> by Sun in 2004) from the configure script for 3.2b2. A number of other
>> tests in configure.in could probably also be removed, although I
>> personally won't touch them before 3.2.
>>
>> The other major system affected by this would be Windows 2000, for which
>> we already decided to not support it anymore.
>>
>> Opinions?
> 
> I would prefer to be guided by vendor EOL dates rather than our own
> arbitrary 10 year limit. The EOL guide I would suggest is "Is the
> vendor still fixing bugs in that release?".

If available, perhaps. Not all system vendors have such a policy,
or it may not be easy to find. If that is part of the policy, I'd
like to add the specific systems where we use this approach along
with the URLs that list the policies in the PEP.

As a counter-example, I think the only way to phase out support
for old OpenBSD releases is that we set a date.

If this policy is applied to Debian also, we may get unhappy users,
since Debian releases get out of love fairly quickly. OTOH, supporting
old Linux releases really isn't that much of a problem.

Regards,
Martin

From sungsuha at gmail.com  Mon Dec  6 08:26:54 2010
From: sungsuha at gmail.com (Seung =?utf-8?b?U29vLA==?= Ha)
Date: Mon, 6 Dec 2010 07:26:54 +0000 (UTC)
Subject: [Python-Dev] PEP 11: Dropping support for ten year old systems
References: <4CFC08C1.3080609@v.loewis.de>
	<AANLkTinvJ9AUeHH58r_Kbfk9mTZGoSLHtG0UpnDqmsRU@mail.gmail.com>
Message-ID: <loom.20101206T080949-953@post.gmane.org>

Nick Coghlan <ncoghlan <at> gmail.com> writes:

> On Mon, Dec 6, 2010 at 7:48 AM, "Martin v. L?wis" <martin <at> v.loewis.de> 
wrote:
> > I'd like to tighten PEP 11
> > Opinions?
> 
> I would prefer to be guided by vendor EOL dates rather than our own
> arbitrary 10 year limit. The EOL guide I would suggest is "Is the
> vendor still fixing bugs in that release?".
> 
> Since the "Is the vendor still patching it?" guideline gives the
> "right" answer for the 3 systems mentioned in this thread, it will
> likely do a better job of covering anomalies like XP than a flat year
> limit would.
+1

I think Nick has a point.

If the vendor is willing to patch/support said version, then I think it would be 
worthwhile for the Python community to provide support 

EOL dates of prominent Linux distribution :

RHEL:
https://access.redhat.com/support/policy/updates/errata/
Ubuntu:
http://en.wikipedia.org/wiki/List_of_Ubuntu_releases#Version_timeline
(http://www.ubuntu.com/products/ubuntu/release-cycle seems to be down)
SuSe
http://support.novell.com/lifecycle/

I do foresee a possible side-effect regarding Fedora as "maintained for 
approximately 13 months", because "Release X is supported until one 
month after the release of Release X+2."

http://fedoraproject.org/wiki/LifeCycle

Considering the nature of the Fedora project, dropping unsupported fedora 
distributions may or may not be helpful for Pyhton and it's users.

Also, it is not clear what to do about distributions/OSs 
without any official EOL or life cycles.



From ncoghlan at gmail.com  Mon Dec  6 08:51:30 2010
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Mon, 6 Dec 2010 17:51:30 +1000
Subject: [Python-Dev] PEP 11: Dropping support for ten year old systems
In-Reply-To: <4CFC909B.10208@v.loewis.de>
References: <4CFC08C1.3080609@v.loewis.de>
	<AANLkTinvJ9AUeHH58r_Kbfk9mTZGoSLHtG0UpnDqmsRU@mail.gmail.com>
	<4CFC909B.10208@v.loewis.de>
Message-ID: <AANLkTi=f-9YKEws6DwXzHrciP4SitjsQG0mZaLBaWwRO@mail.gmail.com>

On Mon, Dec 6, 2010 at 5:28 PM, "Martin v. L?wis" <martin at v.loewis.de> wrote:
> Am 06.12.2010 05:36, schrieb Nick Coghlan:
>> On Mon, Dec 6, 2010 at 7:48 AM, "Martin v. L?wis" <martin at v.loewis.de> wrote:
>>> I'd like to tighten PEP 11, and declare a policy that systems
>>> older than ten years at the point of a feature release are not
>>> supported anymore by default. Older systems where support is still
>>> maintained need to be explicitly listed in the PEP, along with
>>> the name of the responsible maintainer (I think this would currently
>>> only cover OS2/EMX which is maintained by Andrew MacIntyre).
>>>
>>> Support for such old platforms can then be removed from the codebase
>>> immediately, no need to go through a PEP 11 deprecation cycle.
>>>
>>> As a consequence, I would then like to remove support for Solaris
>>> versions older than Solaris 8 (released in January 2000, last updated
>>> by Sun in 2004) from the configure script for 3.2b2. A number of other
>>> tests in configure.in could probably also be removed, although I
>>> personally won't touch them before 3.2.
>>>
>>> The other major system affected by this would be Windows 2000, for which
>>> we already decided to not support it anymore.
>>>
>>> Opinions?
>>
>> I would prefer to be guided by vendor EOL dates rather than our own
>> arbitrary 10 year limit. The EOL guide I would suggest is "Is the
>> vendor still fixing bugs in that release?".
>
> If available, perhaps. Not all system vendors have such a policy,
> or it may not be easy to find. If that is part of the policy, I'd
> like to add the specific systems where we use this approach along
> with the URLs that list the policies in the PEP.
>
> As a counter-example, I think the only way to phase out support
> for old OpenBSD releases is that we set a date.

I would be fine with an EOL based policy for single-vendor platforms
(specifically Solaris and Windows) and a date-based policy for
everything else.

Cheers,
Nick.

-- 
Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia

From asmodai at in-nomine.org  Mon Dec  6 09:36:09 2010
From: asmodai at in-nomine.org (Jeroen Ruigrok van der Werven)
Date: Mon, 6 Dec 2010 09:36:09 +0100
Subject: [Python-Dev] PEP 11: Dropping support for ten year old systems
In-Reply-To: <4CFC909B.10208@v.loewis.de>
References: <4CFC08C1.3080609@v.loewis.de>
	<AANLkTinvJ9AUeHH58r_Kbfk9mTZGoSLHtG0UpnDqmsRU@mail.gmail.com>
	<4CFC909B.10208@v.loewis.de>
Message-ID: <20101206083609.GE29719@nexus.in-nomine.org>

-On [20101206 08:30], "Martin v. L?wis" (martin at v.loewis.de) wrote:
>As a counter-example, I think the only way to phase out support
>for old OpenBSD releases is that we set a date.

If you want, I can provide you with specifics on the BSD platforms of what
is currently in use, EOL and the future.

Also, with regard to Windows 2000, XP and its APIs. Didn't XP already switch
a bunch of APIs around that 2000 doesn't use? (Need to find that migration
document.)

-- 
Jeroen Ruigrok van der Werven <asmodai(-at-)in-nomine.org> / asmodai
????? ?????? ??? ?? ??????
http://www.in-nomine.org/ | http://www.rangaku.org/ | GPG: 2EAC625B
Nothing is more honorable than enlightenment, nothing is more beautiful
than virtue...

From martin at v.loewis.de  Mon Dec  6 10:08:05 2010
From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=)
Date: Mon, 06 Dec 2010 10:08:05 +0100
Subject: [Python-Dev] PEP 11: Dropping support for ten year old systems
In-Reply-To: <20101206083609.GE29719@nexus.in-nomine.org>
References: <4CFC08C1.3080609@v.loewis.de>
	<AANLkTinvJ9AUeHH58r_Kbfk9mTZGoSLHtG0UpnDqmsRU@mail.gmail.com>
	<4CFC909B.10208@v.loewis.de>
	<20101206083609.GE29719@nexus.in-nomine.org>
Message-ID: <4CFCA7F5.8080709@v.loewis.de>

Am 06.12.2010 09:36, schrieb Jeroen Ruigrok van der Werven:
> -On [20101206 08:30], "Martin v. L?wis" (martin at v.loewis.de) wrote:
>> As a counter-example, I think the only way to phase out support
>> for old OpenBSD releases is that we set a date.
> 
> If you want, I can provide you with specifics on the BSD platforms of what
> is currently in use, EOL and the future.

If that's publicly documented from a single starting page, having the
URL of that page would be appreciated. Also, do users accept that
policy? (i.e. would they feel sad if Python releases don't support BSD
releases anymore that don't get patches?)

For Windows and Solaris, it seems that some users continue to use the
system after the vendor stops producing patches, and dislike the
prospect of not having Python releases for it anymore. However, they
are in clear minority, so by our current policy for minority platforms
(no need to support them), that's fine. This really triggered the "ten
years" proposal: for quite some time now (20 years) people stop losing
interest in operating systems after ten years.

> Also, with regard to Windows 2000, XP and its APIs. Didn't XP already switch
> a bunch of APIs around that 2000 doesn't use? (Need to find that migration
> document.)

I don't understand the question: what does it meant to switch an API
around? XP has added new API that wasn't available in 2000, if that's
what you are asking.

Regards,
Martin


From martin at v.loewis.de  Mon Dec  6 10:18:30 2010
From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=)
Date: Mon, 06 Dec 2010 10:18:30 +0100
Subject: [Python-Dev] PEP 11: Dropping support for ten year old systems
In-Reply-To: <loom.20101206T080949-953@post.gmane.org>
References: <4CFC08C1.3080609@v.loewis.de>	<AANLkTinvJ9AUeHH58r_Kbfk9mTZGoSLHtG0UpnDqmsRU@mail.gmail.com>
	<loom.20101206T080949-953@post.gmane.org>
Message-ID: <4CFCAA66.6060901@v.loewis.de>

> EOL dates of prominent Linux distribution :

I think I would need more information than that. Nick's proposal was
more specific: when does the vendor stop producing patches? This is
a clear criterion, and one that I support.

> RHEL:
> https://access.redhat.com/support/policy/updates/errata/

My interpretation: Python support until end of production phase 3 (7 years).

> Ubuntu:
> http://en.wikipedia.org/wiki/List_of_Ubuntu_releases#Version_timeline
> (http://www.ubuntu.com/products/ubuntu/release-cycle seems to be down)

I'd prefer something more official than Wikipedia, though.

> SuSe
> http://support.novell.com/lifecycle/

My interpretation: Python support until end of Extended support phase
(7 years).

So by this policy, RHEL and SuSE users would be off worse than with
my original proposal (10 years).

> Considering the nature of the Fedora project, dropping unsupported fedora 
> distributions may or may not be helpful for Pyhton and it's users.

Again, for Linux, I think the issue is somewhat less critical: in terms
of portability and ABI stability, it seems like they manage best (i.e.
we have least version-dependent code for Linux in Python, probably
because a "Linux version" doesn't exist in the first place, so
distributions must provide source and binary compatibility even
across vendors, making such support across versions more easy).

> Also, it is not clear what to do about distributions/OSs 
> without any official EOL or life cycles.

Here my proposal stands: 10 years, by default.

Regards,
Martin

From ziade.tarek at gmail.com  Mon Dec  6 10:42:50 2010
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Mon, 6 Dec 2010 10:42:50 +0100
Subject: [Python-Dev] PEP 384 accepted
In-Reply-To: <4CFA99DF.9040907@v.loewis.de>
References: <AANLkTi=s5NMxjPdRQRcupNBzdqjMhLO6TbFWtQzi0TEw@mail.gmail.com>
	<4CF81208.8090705@v.loewis.de>
	<AANLkTinspUo88oUnd3cX776mMRMU6bqyOaq9Og7JLwhU@mail.gmail.com>
	<4CF81AE1.5050900@v.loewis.de> <4CF880F7.9000800@netwok.org>
	<4CF89CCF.20307@v.loewis.de> <4CF8E22B.7050005@netwok.org>
	<4CF97197.2050105@v.loewis.de> <4CF973D0.4060800@netwok.org>
	<4CF974B9.6040509@v.loewis.de> <20101204164232.GH3015@unaka.lan>
	<4CFA99DF.9040907@v.loewis.de>
Message-ID: <AANLkTikpsdSOWZpSV3Y4pW68jarq7Ste0Z_t=NZv64gj@mail.gmail.com>

On Sat, Dec 4, 2010 at 8:43 PM, "Martin v. L?wis" <martin at v.loewis.de> wrote:
...
> Now, distutils2 is different: it's *not* new functionality. So perhaps
> yes: I disagree with the principle that bold rewrites should be
> developed independently. Such work would be much better carried out in a
> branch - it will never stand on its own.

So we had several +1 on James' idea to bring back distutils2 to the
trunk after 3.2 final, and it seems that you are not against this
idea.

So we will do this unless you object

Regards
Tarek

-- 
Tarek Ziad? | http://ziade.org

From sungsuha at gmail.com  Mon Dec  6 12:04:19 2010
From: sungsuha at gmail.com (Seung =?utf-8?b?U29vLA==?= Ha)
Date: Mon, 6 Dec 2010 11:04:19 +0000 (UTC)
Subject: [Python-Dev] PEP 11: Dropping support for ten year old systems
References: <4CFC08C1.3080609@v.loewis.de>
	<AANLkTinvJ9AUeHH58r_Kbfk9mTZGoSLHtG0UpnDqmsRU@mail.gmail.com>
	<4CFC909B.10208@v.loewis.de>
	<AANLkTi=f-9YKEws6DwXzHrciP4SitjsQG0mZaLBaWwRO@mail.gmail.com>
Message-ID: <loom.20101206T120325-754@post.gmane.org>

Nick Coghlan <ncoghlan <at> gmail.com> writes:

> I would be fine with an EOL based policy for single-vendor platforms
> (specifically Solaris and Windows) and a date-based policy for
> everything else.

+1
I also think this would be for the best.




From andrew at bemusement.org  Mon Dec  6 12:10:12 2010
From: andrew at bemusement.org (Andrew Bennetts)
Date: Mon, 6 Dec 2010 22:10:12 +1100
Subject: [Python-Dev] PEP 11: Dropping support for ten year old systems
In-Reply-To: <4CFCAA66.6060901@v.loewis.de>
References: <4CFC08C1.3080609@v.loewis.de>
	<AANLkTinvJ9AUeHH58r_Kbfk9mTZGoSLHtG0UpnDqmsRU@mail.gmail.com>
	<loom.20101206T080949-953@post.gmane.org>
	<4CFCAA66.6060901@v.loewis.de>
Message-ID: <20101206111012.GA22072@aihal.home.puzzling.org>

"Martin v. L?wis" wrote:
[...]
> > Ubuntu:
> > http://en.wikipedia.org/wiki/List_of_Ubuntu_releases#Version_timeline
> > (http://www.ubuntu.com/products/ubuntu/release-cycle seems to be down)
> 
> I'd prefer something more official than Wikipedia, though.

https://wiki.ubuntu.com/Releases

-Andrew.


From mal at egenix.com  Mon Dec  6 12:39:43 2010
From: mal at egenix.com (M.-A. Lemburg)
Date: Mon, 06 Dec 2010 12:39:43 +0100
Subject: [Python-Dev] transform() and untransform() methods,
 and the codec registry
In-Reply-To: <AANLkTikF2w-mmNVgYch=40Rmdc_OvRV3wOYzthvNBbpO@mail.gmail.com>
References: <20101202180651.F2AB0EE9CD@mail.python.org>	<201012031016.05063.victor.stinner@haypocalc.com>	<20101203151129.74C5222A88F@kimball.webabinitio.net>	<AANLkTiktmxr_eH-S9ihcwFoqd6yJeN03Hes_urH7Qt3A@mail.gmail.com>	<20101203175832.88F00236D8B@kimball.webabinitio.net>
	<AANLkTikF2w-mmNVgYch=40Rmdc_OvRV3wOYzthvNBbpO@mail.gmail.com>
Message-ID: <4CFCCB7F.9080605@egenix.com>

Guido van Rossum wrote:
> On Fri, Dec 3, 2010 at 9:58 AM, R. David Murray <rdmurray at bitdance.com> wrote:
>> On Fri, 03 Dec 2010 11:14:56 -0500, Alexander Belopolsky <alexander.belopolsky at gmail.com> wrote:
>>> On Fri, Dec 3, 2010 at 10:11 AM, R. David Murray <rdmurray at bitdance.com> wrote:
>>> ..
>>>> Please also recall that transform/untransform was discussed before
>>>> the release of Python 3.0 and was approved at the time, but it just
>>>> did not get implemented before the 3.0 release.
>>>>
>>>
>>> Can you provide a link?  My search for transform on python-dev came out with
>>
>> It was linked from the issue, if I recall correctly.  I do remember
>> reading the thread from the python-3000 list, linked by someone
>> somewhere :)
>>
>>> http://mail.python.org/pipermail/python-dev/2010-June/100564.html
>>>
>>> where you seem to oppose these methods.  Also, new methods to builtins
>>
>> It looks to me like I was agreeing that transform/untrasnform should
>> do only bytes->bytes or str->str regardless of what codec name you
>> passed them.
>>
>>> fall under the language moratorium (but can be approved on a
>>> case-by-case basis):
>>>
>>> http://www.python.org/dev/peps/pep-3003/#case-by-case-exemptions
>>>
>>> Is there an effort to document these exceptions?  I expected such
>>> approvals to be added to PEP 3003, but apparently this was not the
>>> case.
>>
>> I believe MAL's thought was that the addition of these methods had
>> been approved pre-moratorium, 

Indeed.

>> but I don't know if that is a
>> sufficient argument or not.
> 
> It is not.
> 
> The moratorium is intended to freeze the state of the language as
> implemented, not whatever was discussed and approved but didn't get
> implemented (that'd be a hole big enough to drive a truck through, as
> the saying goes :-).

Sure, but those two particular methods only provide interfaces
to the codecs sub-system without actually requiring any major
implementation changes.

Furthermore, they "help ease adoption of Python 3.x" (quoted from
PEP 3003), since the functionality they add back was removed from
Python 3.0 in a way that makes it difficult to port Python2
applications to Python3.

> Regardless of what I or others may have said before, I am not
> currently a fan of adding transform() to either str or bytes.

How should I read this ? Do want the methods to be removed again
and added back in 3.3 ?

Frankly, I'm a bit tired of constantly having to argue against
cutting down the Unicode and codec support in Python3.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Dec 06 2010)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try our new mxODBC.Connect Python Database Interface for free ! ::::


   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
           Registered at Amtsgericht Duesseldorf: HRB 46611
               http://www.egenix.com/company/contact/

From ulrich.eckhardt at dominolaser.com  Mon Dec  6 13:18:55 2010
From: ulrich.eckhardt at dominolaser.com (Ulrich Eckhardt)
Date: Mon, 6 Dec 2010 13:18:55 +0100
Subject: [Python-Dev] python 2 for building python 3
In-Reply-To: <1291470115.3534.8.camel@localhost.localdomain>
References: <AANLkTin7ynm7Srnp4DLa2B+825cUL+jSaKqq+Gvz34OD@mail.gmail.com> 
	<AANLkTingqrC1rd7v7qVhEphG_42j8N=5anTZCT0mgdrz@mail.gmail.com> 
	<1291470115.3534.8.camel@localhost.localdomain>
Message-ID: <201012061318.55922.ulrich.eckhardt@dominolaser.com>

On Saturday 04 December 2010, Antoine Pitrou wrote:
> Perhaps SVN doesn't get timestamps right.

SVN doesn't touch timestamps explicitly, it just writes the files as they come 
and the system gives them the timestamps. This also makes sense, because the 
build system depends on them - you don't want the build to skip compiling 
files after rewinding your working copy to an older version.

That said, you can tell SVN to do something with timestamps, I'd have to 
search a bit in order to find out what exactly that is.

Uli

**************************************************************************************
Domino Laser GmbH, Fangdieckstra?e 75a, 22547 Hamburg, Deutschland
Gesch?ftsf?hrer: Thorsten F?cking, Amtsgericht Hamburg HR B62 932
**************************************************************************************
Visit our website at <http://www.dominolaser.com/>
**************************************************************************************
Diese E-Mail einschlie?lich s?mtlicher Anh?nge ist nur f?r den Adressaten bestimmt und kann vertrauliche Informationen enthalten. Bitte benachrichtigen Sie den Absender umgehend, falls Sie nicht der beabsichtigte Empf?nger sein sollten. Die E-Mail ist in diesem Fall zu l?schen und darf weder gelesen, weitergeleitet, ver?ffentlicht oder anderweitig benutzt werden.
E-Mails k?nnen durch Dritte gelesen werden und Viren sowie nichtautorisierte ?nderungen enthalten. Domino Laser GmbH ist f?r diese Folgen nicht verantwortlich.
**************************************************************************************



From tseaver at palladion.com  Mon Dec  6 13:47:30 2010
From: tseaver at palladion.com (Tres Seaver)
Date: Mon, 06 Dec 2010 07:47:30 -0500
Subject: [Python-Dev] python 2 for building python 3
In-Reply-To: <201012061318.55922.ulrich.eckhardt@dominolaser.com>
References: <AANLkTin7ynm7Srnp4DLa2B+825cUL+jSaKqq+Gvz34OD@mail.gmail.com>
	<AANLkTingqrC1rd7v7qVhEphG_42j8N=5anTZCT0mgdrz@mail.gmail.com>
	<1291470115.3534.8.camel@localhost.localdomain>
	<201012061318.55922.ulrich.eckhardt@dominolaser.com>
Message-ID: <idim0v$gjs$1@dough.gmane.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 12/06/2010 07:18 AM, Ulrich Eckhardt wrote:
> On Saturday 04 December 2010, Antoine Pitrou wrote:
>> Perhaps SVN doesn't get timestamps right.
> 
> SVN doesn't touch timestamps explicitly, it just writes the files as they come 
> and the system gives them the timestamps. This also makes sense, because the 
> build system depends on them - you don't want the build to skip compiling 
> files after rewinding your working copy to an older version.
> 
> That said, you can tell SVN to do something with timestamps, I'd have to 
> search a bit in order to find out what exactly that is.

If you set the 'use-commit-times' config flag, subversion will keep
timestamps in sync with the repository commit times during 'checkout',
'update', 'switch', etc. operations (it does this by default during
'export', which is likekt why the tarball process works):

http://svnbook.red-bean.com/en/1.1/ch07.html


Tres.
- -- 
===================================================================
Tres Seaver          +1 540-429-0999          tseaver at palladion.com
Palladion Software   "Excellence by Design"    http://palladion.com
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.10 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAkz822EACgkQ+gerLs4ltQ5tdwCfV/iScJcoxymb/REIBm6VFcbK
TUIAoNl5LLaFgifdV8BjiuOIw5YmnaqR
=H7+7
-----END PGP SIGNATURE-----


From flub at devork.be  Mon Dec  6 14:40:08 2010
From: flub at devork.be (Floris Bruynooghe)
Date: Mon, 6 Dec 2010 13:40:08 +0000
Subject: [Python-Dev] PEP 11: Dropping support for ten year old systems
In-Reply-To: <4CFCAA66.6060901@v.loewis.de>
References: <4CFC08C1.3080609@v.loewis.de>
	<AANLkTinvJ9AUeHH58r_Kbfk9mTZGoSLHtG0UpnDqmsRU@mail.gmail.com>
	<loom.20101206T080949-953@post.gmane.org>
	<4CFCAA66.6060901@v.loewis.de>
Message-ID: <AANLkTi=7+ykS2kzrv3M9Fsv+V5zZpw1-5Tt=ckZ397XC@mail.gmail.com>

On 6 December 2010 09:18, "Martin v. L?wis" <martin at v.loewis.de> wrote:
>> Also, it is not clear what to do about distributions/OSs
>> without any official EOL or life cycles.
>
> Here my proposal stands: 10 years, by default.

How about max(EOL, 10years)?  That sounds like it could be a useful guideline.

(Personally I'd be sad to see Solaris 8 go in the next few years)

Regards
Floris


-- 
Debian GNU/Linux -- The Power of Freedom
www.debian.org | www.gnu.org | www.kernel.org

From martin at v.loewis.de  Mon Dec  6 20:03:26 2010
From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=)
Date: Mon, 06 Dec 2010 20:03:26 +0100
Subject: [Python-Dev] python 2 for building python 3
In-Reply-To: <201012061318.55922.ulrich.eckhardt@dominolaser.com>
References: <AANLkTin7ynm7Srnp4DLa2B+825cUL+jSaKqq+Gvz34OD@mail.gmail.com>
	<AANLkTingqrC1rd7v7qVhEphG_42j8N=5anTZCT0mgdrz@mail.gmail.com>
	<1291470115.3534.8.camel@localhost.localdomain>
	<201012061318.55922.ulrich.eckhardt@dominolaser.com>
Message-ID: <4CFD337E.9050300@v.loewis.de>

Am 06.12.2010 13:18, schrieb Ulrich Eckhardt:
> On Saturday 04 December 2010, Antoine Pitrou wrote:
>> Perhaps SVN doesn't get timestamps right.
> 
> SVN doesn't touch timestamps explicitly, it just writes the files as they come 
> and the system gives them the timestamps. This also makes sense, because the 
> build system depends on them - you don't want the build to skip compiling 
> files after rewinding your working copy to an older version.
> 
> That said, you can tell SVN to do something with timestamps, I'd have to 
> search a bit in order to find out what exactly that is.

What you can ask it to is to use commit times. As you say, this is risky
because it may cause build steps to be skipped.

What I'm asking for is that the version control gives the files new time
stamps, but still imposes a timestamp order, at least on modern
file systems (and, as an option, also on old ones).

Suppose you do an update spanning 20 revisions. The update should:
1. get the current time T at the beginning of the update
2. fetch all files, and update the contents
3. Touch all files, in the order of the revisions, using microsecond
   steps (i.e. the oldest file gets T, the next revision T+1?s, the
   next one T+2?s).
As the update probably takes longer than 20?s, all files will have
timestamps in the past when the update is done.

If the file system does not support subsecond timestamps, you can do
the same in second resolution, but that might give you files dated
in the future (which make complains about - but you'ld get over with
that after 20s). ms might offer some middle ground (but I think all
filesystems supporting ms resolution will also support ?s).

Regards,
Martin

From martin at v.loewis.de  Mon Dec  6 19:55:15 2010
From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=)
Date: Mon, 06 Dec 2010 19:55:15 +0100
Subject: [Python-Dev] PEP 11: Dropping support for ten year old systems
In-Reply-To: <AANLkTi=7+ykS2kzrv3M9Fsv+V5zZpw1-5Tt=ckZ397XC@mail.gmail.com>
References: <4CFC08C1.3080609@v.loewis.de>	<AANLkTinvJ9AUeHH58r_Kbfk9mTZGoSLHtG0UpnDqmsRU@mail.gmail.com>	<loom.20101206T080949-953@post.gmane.org>	<4CFCAA66.6060901@v.loewis.de>
	<AANLkTi=7+ykS2kzrv3M9Fsv+V5zZpw1-5Tt=ckZ397XC@mail.gmail.com>
Message-ID: <4CFD3193.2030906@v.loewis.de>

Am 06.12.2010 14:40, schrieb Floris Bruynooghe:
> On 6 December 2010 09:18, "Martin v. L?wis" <martin at v.loewis.de> wrote:
>>> Also, it is not clear what to do about distributions/OSs
>>> without any official EOL or life cycles.
>>
>> Here my proposal stands: 10 years, by default.
> 
> How about max(EOL, 10years)?  That sounds like it could be a useful guideline.
> 
> (Personally I'd be sad to see Solaris 8 go in the next few years)

I guess we'll be sorry, then: under that policy, max(EOL, 10years) comes
out as "not supported" (not sure whether you were aware of that).

Of course, with these old systems, I really wonder: why do they need
current Python releases? 2.7 will remain available and maintained for
some time, and 3.1 will at least see security fixes for some more time -
something that the base system itself doesn't receive anymore. So
if you needed a Python release for Solaris 8, you could just use Python
2.3, no? We are not going to take the sources of old releases offline.

Regards,
Martin

From raymond.hettinger at gmail.com  Mon Dec  6 20:06:56 2010
From: raymond.hettinger at gmail.com (Raymond Hettinger)
Date: Mon, 6 Dec 2010 11:06:56 -0800
Subject: [Python-Dev] transform() and untransform() methods,
	and the codec registry
In-Reply-To: <AANLkTikF2w-mmNVgYch=40Rmdc_OvRV3wOYzthvNBbpO@mail.gmail.com>
References: <20101202180651.F2AB0EE9CD@mail.python.org>
	<201012031016.05063.victor.stinner@haypocalc.com>
	<20101203151129.74C5222A88F@kimball.webabinitio.net>
	<AANLkTiktmxr_eH-S9ihcwFoqd6yJeN03Hes_urH7Qt3A@mail.gmail.com>
	<20101203175832.88F00236D8B@kimball.webabinitio.net>
	<AANLkTikF2w-mmNVgYch=40Rmdc_OvRV3wOYzthvNBbpO@mail.gmail.com>
Message-ID: <76C191DD-614F-4E45-A294-E6527A9935DF@gmail.com>


On Dec 3, 2010, at 10:05 AM, Guido van Rossum wrote:
> Regardless of what I or others may have said before, I am not
> currently a fan of adding transform() to either str or bytes.


Just to clarify, is your preference to go back to the Python 2.x way 
and use encode()/decode() for both unicode encodings and other
other transformations (uu, zlib, bz2, quopri, etc)?

FWIW, transform()/untransform() are already in the 3.2 beta.
Do you want them taken out?   If so, do you want the codecs
made accessible with encode/decode or have them continue
to be inaccessible in 3.x?


Raymond


From raymond.hettinger at gmail.com  Mon Dec  6 20:21:28 2010
From: raymond.hettinger at gmail.com (Raymond Hettinger)
Date: Mon, 6 Dec 2010 11:21:28 -0800
Subject: [Python-Dev] Repo frozen for 3.2
In-Reply-To: <loom.20101205T123251-490@post.gmane.org>
References: <loom.20101205T123251-490@post.gmane.org>
Message-ID: <5E183509-BBED-426B-BC70-DDA5640AD302@gmail.com>


On Dec 5, 2010, at 3:36 AM, Vinay Sajip wrote:

> I've just been notified via being added to the nosy list of
> 
> http://bugs.python.org/issue10627
> 
> about the deprecation of ConfigParser for 3.2. I presume I was added to this
> list because logging.config uses ConfigParser, but logging.config doesn't use
> any interpolation features so I could easily change all usages to
> SafeConfigParser; should I do it now (before the 3.2 beta is cut) or hold off
> for now?

+1 for doing the update now.   There's no advantage in continuing to use
the old ConfigParser.

With respect to beta feature freeze, that should really just mean not adding
prominent new major features.  We're still free to implement bug fixes,
make the language more consistent, propagate the existing changes
(such as SafeConfigParser), tweak any of the new APIs, and reconsider or
rework some of the new features (such as transform/untransform). 


Raymond


P.S.  We really ought to stop with the SafeFoo naming convention.
It is only descriptive to the person who wrote the class or function,
not to the user who will immediately wonder, "safe from what?"
Safe can mean "sanitized", "won't raise an error", "has a default value",
"doesn't use eval/exec", "is secure", "has a known encoding", etc.

From tjreedy at udel.edu  Mon Dec  6 20:25:56 2010
From: tjreedy at udel.edu (Terry Reedy)
Date: Mon, 06 Dec 2010 14:25:56 -0500
Subject: [Python-Dev] PEP 11: Dropping support for ten year old systems
In-Reply-To: <4CFCA7F5.8080709@v.loewis.de>
References: <4CFC08C1.3080609@v.loewis.de>	<AANLkTinvJ9AUeHH58r_Kbfk9mTZGoSLHtG0UpnDqmsRU@mail.gmail.com>	<4CFC909B.10208@v.loewis.de>	<20101206083609.GE29719@nexus.in-nomine.org>
	<4CFCA7F5.8080709@v.loewis.de>
Message-ID: <idjdc1$c86$1@dough.gmane.org>

On 12/6/2010 4:08 AM, "Martin v. L?wis" wrote:

> For Windows and Solaris, it seems that some users continue to use the
> system after the vendor stops producing patches, and dislike the
> prospect of not having Python releases for it anymore. However, they
> are in clear minority, so by our current policy for minority platforms

I quite suspect that XP will be in major use (more than say, current 
BSD) for some years after MS stops official support. Why rush to drop 
it? Is there much XP specific stuff in the current xp/vista/7 code?

It seems to me that the rule should be something like "around 10 years 
or end of support, as modified by popularity, the burden of continued 
support, the availability of test machines, and the availability of people".

-- 
Terry Jan Reedy



From martin at v.loewis.de  Mon Dec  6 20:26:12 2010
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Mon, 06 Dec 2010 20:26:12 +0100
Subject: [Python-Dev] transform() and untransform() methods,
 and the codec registry
In-Reply-To: <76C191DD-614F-4E45-A294-E6527A9935DF@gmail.com>
References: <20101202180651.F2AB0EE9CD@mail.python.org>	<201012031016.05063.victor.stinner@haypocalc.com>	<20101203151129.74C5222A88F@kimball.webabinitio.net>	<AANLkTiktmxr_eH-S9ihcwFoqd6yJeN03Hes_urH7Qt3A@mail.gmail.com>	<20101203175832.88F00236D8B@kimball.webabinitio.net>	<AANLkTikF2w-mmNVgYch=40Rmdc_OvRV3wOYzthvNBbpO@mail.gmail.com>
	<76C191DD-614F-4E45-A294-E6527A9935DF@gmail.com>
Message-ID: <4CFD38D4.10507@v.loewis.de>

> FWIW, transform()/untransform() are already in the 3.2 beta.
> Do you want them taken out?   If so, do you want the codecs
> made accessible with encode/decode or have them continue
> to be inaccessible in 3.x?

Notice that they wouldn't be inaccessible even if transform
is taken out again: codecs.lookup would still find them.

Regards,
Martin

From me+python at ixokai.io  Mon Dec  6 20:34:41 2010
From: me+python at ixokai.io (Stephen Hansen)
Date: Mon, 06 Dec 2010 11:34:41 -0800
Subject: [Python-Dev] PEP 11: Dropping support for ten year old systems
In-Reply-To: <4CFD3193.2030906@v.loewis.de>
References: <4CFC08C1.3080609@v.loewis.de>	<AANLkTinvJ9AUeHH58r_Kbfk9mTZGoSLHtG0UpnDqmsRU@mail.gmail.com>	<loom.20101206T080949-953@post.gmane.org>	<4CFCAA66.6060901@v.loewis.de>	<AANLkTi=7+ykS2kzrv3M9Fsv+V5zZpw1-5Tt=ckZ397XC@mail.gmail.com>
	<4CFD3193.2030906@v.loewis.de>
Message-ID: <4CFD3AD1.70801@ixokai.io>

On 12/6/10 10:55 AM, "Martin v. L?wis" wrote:
> Of course, with these old systems, I really wonder: why do they need
> current Python releases? 2.7 will remain available and maintained for
> some time, and 3.1 will at least see security fixes for some more time -
> something that the base system itself doesn't receive anymore. So
> if you needed a Python release for Solaris 8, you could just use Python
> 2.3, no? We are not going to take the sources of old releases offline.

For things like Solaris 8, I have no thoughts one way or the other-- but
considering Windows XP is hitting 10 years next year-- Personally? My
entirely theoretical timetable for when I think I'll be able to finally
upgrade to Python 3 (where I'll skip to whatever the latest Python 3
is), is actually shorter by at least a few years then my timetable of
when I think I'll be able to drop support of Windows XP. Unfortunately.

WinXP is old but *pervasive* still in my experience with small
businesses / customers. Many aren't even considering making a plan for
Win7 yet.

So if two years rolls around and Python 3.x (where 'x' is 'whatever is
current') isn't supported on Windows XP, I'll be very sad, and will have
to be stuck on Python 3.x-1 for .. awhile, where "awhile" is out of my
control and up to the Masses who are unable or can't be bothered with
fixing what works for them w/ WinXP.

-- 

   Stephen Hansen
   ... Also: Ixokai
   ... Mail: me+python (AT) ixokai (DOT) io
   ... Blog: http://meh.ixokai.io/

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 487 bytes
Desc: OpenPGP digital signature
URL: <http://mail.python.org/pipermail/python-dev/attachments/20101206/a918d7b8/attachment-0001.pgp>

From fdrake at acm.org  Mon Dec  6 20:40:25 2010
From: fdrake at acm.org (Fred Drake)
Date: Mon, 6 Dec 2010 14:40:25 -0500
Subject: [Python-Dev] Repo frozen for 3.2
In-Reply-To: <5E183509-BBED-426B-BC70-DDA5640AD302@gmail.com>
References: <loom.20101205T123251-490@post.gmane.org>
	<5E183509-BBED-426B-BC70-DDA5640AD302@gmail.com>
Message-ID: <AANLkTi=A+OJK+dkJHGhfX6q-FeAtcBpYH+Y4-VzQqZDc@mail.gmail.com>

On Mon, Dec 6, 2010 at 2:21 PM, Raymond Hettinger
<raymond.hettinger at gmail.com> wrote:
> We really ought to stop with the SafeFoo naming convention.
> It is only descriptive to the person who wrote the class or function,
> not to the user who will immediately wonder, "safe from what?"

Safe from bad vampire movies, of course!

I'd not recognize the current Safe* class names as a pattern; there
are currently two in the py3k trunk:

    configparser.SafeConfigParser
        -- very much a poor name

    xmlrpc.client.SafeTransport
        -- perhaps should have been SSLTransport or HTTPSTransport

I agree the "Safe" prefix isn't meaningful.  SafeConfigParser might
even be my fault.

Michael Foord has lobbied to end up with the "preferred" configparser
class to be named (eventually) ConfigParser, with good reason.  It's
not clear to me that we want to do that for backward compatibility
reasons (as I've argued elsewhere).  Were it not for that issue, I'd
be in favor of using different/better names.  (And there's still space
for better names, if we can create new names that avoid the b/w
compatibility issues.)


? -Fred

--
Fred L. Drake, Jr.? ? <fdrake at acm.org>
"A storm broke loose in my mind."? --Albert Einstein

From martin at v.loewis.de  Mon Dec  6 21:46:15 2010
From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=)
Date: Mon, 06 Dec 2010 21:46:15 +0100
Subject: [Python-Dev] PEP 11: Dropping support for ten year old systems
In-Reply-To: <idjdc1$c86$1@dough.gmane.org>
References: <4CFC08C1.3080609@v.loewis.de>	<AANLkTinvJ9AUeHH58r_Kbfk9mTZGoSLHtG0UpnDqmsRU@mail.gmail.com>	<4CFC909B.10208@v.loewis.de>	<20101206083609.GE29719@nexus.in-nomine.org>	<4CFCA7F5.8080709@v.loewis.de>
	<idjdc1$c86$1@dough.gmane.org>
Message-ID: <4CFD4B97.2090409@v.loewis.de>

Am 06.12.2010 20:25, schrieb Terry Reedy:
> On 12/6/2010 4:08 AM, "Martin v. L?wis" wrote:
> 
>> For Windows and Solaris, it seems that some users continue to use the
>> system after the vendor stops producing patches, and dislike the
>> prospect of not having Python releases for it anymore. However, they
>> are in clear minority, so by our current policy for minority platforms
> 
> I quite suspect that XP will be in major use (more than say, current
> BSD) for some years after MS stops official support. Why rush to drop
> it?

What rush to drop it, specifically? It will be supported as long as
Microsoft provides patches, as per Nick's amendment. For Windows XP,
the extended lifecycle support (end of security patches) will be on
April 8, 2014. I wouldn't call that "rushing", at this point in time.
By our current release pace, Python 3.5 might be the first release
to not support XP anymore.

> Is there much XP specific stuff in the current xp/vista/7 code?

I don't know; I haven't investigated unsupporting XP yet. I'm concerned
about Windows 2000, at the moment.

> It seems to me that the rule should be something like "around 10 years
> or end of support, as modified by popularity, the burden of continued
> support, the availability of test machines, and the availability of
> people".

In my original posting, I proposed a clause where support could be
extended as long as an individual steps forward to provide that support.
So if XP remains popular by the time Microsoft stops providing patches
for it, some volunteer would have to step forward.

Regards,
Martin

From raymond.hettinger at gmail.com  Mon Dec  6 22:02:41 2010
From: raymond.hettinger at gmail.com (Raymond Hettinger)
Date: Mon, 6 Dec 2010 13:02:41 -0800
Subject: [Python-Dev] Repo frozen for 3.2
In-Reply-To: <AANLkTi=A+OJK+dkJHGhfX6q-FeAtcBpYH+Y4-VzQqZDc@mail.gmail.com>
References: <loom.20101205T123251-490@post.gmane.org>
	<5E183509-BBED-426B-BC70-DDA5640AD302@gmail.com>
	<AANLkTi=A+OJK+dkJHGhfX6q-FeAtcBpYH+Y4-VzQqZDc@mail.gmail.com>
Message-ID: <19D1A664-2C13-4D9F-9A45-B4DEBB4E77E1@gmail.com>


On Dec 6, 2010, at 11:40 AM, Fred Drake wrote:

> On Mon, Dec 6, 2010 at 2:21 PM, Raymond Hettinger
> <raymond.hettinger at gmail.com> wrote:
>> We really ought to stop with the SafeFoo naming convention.
>> It is only descriptive to the person who wrote the class or function,
>> not to the user who will immediately wonder, "safe from what?"
> 
> Safe from bad vampire movies, of course!
> 
> I'd not recognize the current Safe* class names as a pattern; there
> are currently two in the py3k trunk:
> 
>    configparser.SafeConfigParser
>        -- very much a poor name
> 
>    xmlrpc.client.SafeTransport
>        -- perhaps should have been SSLTransport or HTTPSTransport
> 
> I agree the "Safe" prefix isn't meaningful. 

IIRC, pprint has a safe_repr() and string.Template has safe_substitute()
and pydoc has a safe import.

Never new there was so much danger in the standard library :-)


Raymond

From amauryfa at gmail.com  Mon Dec  6 22:02:48 2010
From: amauryfa at gmail.com (Amaury Forgeot d'Arc)
Date: Mon, 6 Dec 2010 22:02:48 +0100
Subject: [Python-Dev] PEP 11: Dropping support for ten year old systems
In-Reply-To: <4CFD4B97.2090409@v.loewis.de>
References: <4CFC08C1.3080609@v.loewis.de>
	<AANLkTinvJ9AUeHH58r_Kbfk9mTZGoSLHtG0UpnDqmsRU@mail.gmail.com>
	<4CFC909B.10208@v.loewis.de>
	<20101206083609.GE29719@nexus.in-nomine.org>
	<4CFCA7F5.8080709@v.loewis.de> <idjdc1$c86$1@dough.gmane.org>
	<4CFD4B97.2090409@v.loewis.de>
Message-ID: <AANLkTimm9JToiK8HjRWQoSzW7AU7cWCjX=AiE1imxaHH@mail.gmail.com>

Hi,

2010/12/6 "Martin v. L?wis" <martin at v.loewis.de>

> In my original posting, I proposed a clause where support could be
> extended as long as an individual steps forward to provide that support.
> So if XP remains popular by the time Microsoft stops providing patches
> for it, some volunteer would have to step forward.
>

+1. Such a clause is already used to keep supporting the old VC6.0 compiler
(a.k.a VC98!)

-- 
Amaury Forgeot d'Arc
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20101206/06ca79c3/attachment.html>

From fdrake at acm.org  Mon Dec  6 22:09:17 2010
From: fdrake at acm.org (Fred Drake)
Date: Mon, 6 Dec 2010 16:09:17 -0500
Subject: [Python-Dev] Repo frozen for 3.2
In-Reply-To: <19D1A664-2C13-4D9F-9A45-B4DEBB4E77E1@gmail.com>
References: <loom.20101205T123251-490@post.gmane.org>
	<5E183509-BBED-426B-BC70-DDA5640AD302@gmail.com>
	<AANLkTi=A+OJK+dkJHGhfX6q-FeAtcBpYH+Y4-VzQqZDc@mail.gmail.com>
	<19D1A664-2C13-4D9F-9A45-B4DEBB4E77E1@gmail.com>
Message-ID: <AANLkTin0Z44MGJBQiSbBRPNgAYoyTGLgwkU2ehVU9z5q@mail.gmail.com>

On Mon, Dec 6, 2010 at 4:02 PM, Raymond Hettinger
<raymond.hettinger at gmail.com> wrote:
> IIRC, pprint has a safe_repr() and string.Template has safe_substitute()
> and pydoc has a safe import.

pprint.saferepr
    Ok, this one's my fault as well.  Probably should just be named repr.

string.Template.safe_substitute
    Agree on this as well.

pydoc.safeimport
    Not documented, so I don't really care what it's called.

> Never new there was so much danger in the standard library :-)

You should have known this, Raymond!  It's software.


? -Fred

--
Fred L. Drake, Jr.? ? <fdrake at acm.org>
"A storm broke loose in my mind."? --Albert Einstein

From tjreedy at udel.edu  Mon Dec  6 23:15:19 2010
From: tjreedy at udel.edu (Terry Reedy)
Date: Mon, 06 Dec 2010 17:15:19 -0500
Subject: [Python-Dev] PEP 11: Dropping support for ten year old systems
In-Reply-To: <4CFD4B97.2090409@v.loewis.de>
References: <4CFC08C1.3080609@v.loewis.de>	<AANLkTinvJ9AUeHH58r_Kbfk9mTZGoSLHtG0UpnDqmsRU@mail.gmail.com>	<4CFC909B.10208@v.loewis.de>	<20101206083609.GE29719@nexus.in-nomine.org>	<4CFCA7F5.8080709@v.loewis.de>	<idjdc1$c86$1@dough.gmane.org>
	<4CFD4B97.2090409@v.loewis.de>
Message-ID: <idjn9j$tls$1@dough.gmane.org>

On 12/6/2010 3:46 PM, "Martin v. L?wis" wrote:
> Am 06.12.2010 20:25, schrieb Terry Reedy:

>> I quite suspect that XP will be in major use (more than say, current
>> BSD) for some years after MS stops official support. Why rush to drop
>> it?
>
> What rush to drop it,

On the day MS stops support. But it is a moot support until them.

-- 
Terry Jan Reedy



From georg at python.org  Mon Dec  6 22:46:48 2010
From: georg at python.org (Georg Brandl)
Date: Mon, 06 Dec 2010 22:46:48 +0100
Subject: [Python-Dev] [RELEASED] Python 3.2 beta 1
Message-ID: <4CFD59C8.4070006@python.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On behalf of the Python development team, I'm happy to announce the
first of two beta preview releases of Python 3.2.

Python 3.2 is a continuation of the efforts to improve and stabilize the
Python 3.x line.  Since the final release of Python 2.7, the 2.x line
will only receive bugfixes, and new features are developed for 3.x only.

Since PEP 3003, the Moratorium on Language Changes, is in effect, there
are no changes in Python's syntax and built-in types in Python 3.2.
Development efforts concentrated on the standard library and support for
porting code to Python 3.  Highlights are:

* numerous improvements to the unittest module
* PEP 3147, support for .pyc repository directories
* PEP 3149, support for version tagged dynamic libraries
* PEP 3148, a new futures library for concurrent programming
* PEP 384, a stable ABI for extension modules
* PEP 391, dictionary-based logging configuration
* an overhauled GIL implementation that reduces contention
* an extended email package that handles bytes messages
* countless fixes regarding bytes/string issues; among them full
  support for a bytes environment (filenames, environment variables)
* many consistency and behavior fixes for numeric operations
* a sysconfig module to access configuration information
* a pure-Python implementation of the datetime module
* additions to the shutil module, among them archive file support
* improvements to pdb, the Python debugger

For a more extensive list of changes in 3.2, see

    http://docs.python.org/3.2/whatsnew/3.2.html

To download Python 3.2 visit:

    http://www.python.org/download/releases/3.2/

Please consider trying Python 3.2 with your code and reporting any bugs
you may notice to:

    http://bugs.python.org/


Enjoy!

- --
Georg Brandl, Release Manager
georg at python.org
(on behalf of the entire python-dev team and 3.2's contributors)

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.16 (GNU/Linux)

iEYEARECAAYFAkz9WcgACgkQN9GcIYhpnLBRYwCeMmH1GMmKOx9fVk8a/F0/TOzj
Vp0AoIHYBNcxV/U0AXIwMGWFHi1bAB+a
=KBam
-----END PGP SIGNATURE-----

From solipsis at pitrou.net  Mon Dec  6 23:24:57 2010
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Mon, 06 Dec 2010 23:24:57 +0100
Subject: [Python-Dev] [python-committers] [RELEASED] Python 3.2 beta 1
In-Reply-To: <4CFD59C8.4070006@python.org>
References: <4CFD59C8.4070006@python.org>
Message-ID: <1291674297.3579.5.camel@localhost.localdomain>


> Development efforts concentrated on the standard library and support for
> porting code to Python 3.  Highlights are:
> 
> [snip]

* ...a much improved ssl module

Regards

Antoine.



From dmalcolm at redhat.com  Mon Dec  6 23:55:03 2010
From: dmalcolm at redhat.com (David Malcolm)
Date: Mon, 06 Dec 2010 17:55:03 -0500
Subject: [Python-Dev] PEP 11: Dropping support for ten year old systems
In-Reply-To: <4CFCAA66.6060901@v.loewis.de>
References: <4CFC08C1.3080609@v.loewis.de>
	<AANLkTinvJ9AUeHH58r_Kbfk9mTZGoSLHtG0UpnDqmsRU@mail.gmail.com>
	<loom.20101206T080949-953@post.gmane.org>
	<4CFCAA66.6060901@v.loewis.de>
Message-ID: <1291676103.15083.28.camel@radiator.bos.redhat.com>

On Mon, 2010-12-06 at 10:18 +0100, "Martin v. L?wis" wrote:
> > EOL dates of prominent Linux distribution :
> 
> I think I would need more information than that. Nick's proposal was
> more specific: when does the vendor stop producing patches? This is
> a clear criterion, and one that I support.
> 
> > RHEL:
> > https://access.redhat.com/support/policy/updates/errata/
> 
> My interpretation: Python support until end of production phase 3 (7 years).

(...)

> So by this policy, RHEL and SuSE users would be off worse than with
> my original proposal (10 years).

Red Hat continues to provide patches for RHEL within the "Extended Life
Cycle" (years 8, 9 and 10), but it's an optional add-on.

So another interpretation of the above with Nick's proposal could be 10
years on RHEL.  (though obviously I'm biased in favor of RHEL)

Approaching this from another angle: please do add me to the "nosy" on
any compatibility bugs with running latest python code on RHEL.  I'm
also looking into getting RHEL buildbot machines, FWIW.


> > Considering the nature of the Fedora project, dropping unsupported fedora 
> > distributions may or may not be helpful for Pyhton and it's users.
> 
> Again, for Linux, I think the issue is somewhat less critical: in terms
> of portability and ABI stability, it seems like they manage best (i.e.
> we have least version-dependent code for Linux in Python, probably
> because a "Linux version" doesn't exist in the first place, so
> distributions must provide source and binary compatibility even
> across vendors, making such support across versions more easy).

The other compat issues are in the toolchain: e.g. very recent versions
of gcc .  In downstream Fedora, we tend to be amongst the first to run
into new compilation warnings (and, occasionally, "exciting"
code-generation bugs...)

But this tends to be the opposite kind of problem: beginning of life,
rather than end-of-life, and these sorts of things will need fixing for
every Linux build eventually.

FWIW, I'm trying to keep Fedora's "system" python 2 and python 3 builds
as up-to-date as reasonable, so Fedora users will (I hope) be running
fairly recent code python as is.  We have 2.7 as /usr/bin/python as of
F14, for instance.


Hope this is helpful
Dave


From martin at v.loewis.de  Tue Dec  7 00:05:15 2010
From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=)
Date: Tue, 07 Dec 2010 00:05:15 +0100
Subject: [Python-Dev] PEP 11: Dropping support for ten year old systems
In-Reply-To: <1291676103.15083.28.camel@radiator.bos.redhat.com>
References: <4CFC08C1.3080609@v.loewis.de>	
	<AANLkTinvJ9AUeHH58r_Kbfk9mTZGoSLHtG0UpnDqmsRU@mail.gmail.com>	
	<loom.20101206T080949-953@post.gmane.org>
	<4CFCAA66.6060901@v.loewis.de>
	<1291676103.15083.28.camel@radiator.bos.redhat.com>
Message-ID: <4CFD6C2B.8030708@v.loewis.de>

>> So by this policy, RHEL and SuSE users would be off worse than with
>> my original proposal (10 years).
> 
> Red Hat continues to provide patches for RHEL within the "Extended Life
> Cycle" (years 8, 9 and 10), but it's an optional add-on.

My understanding is that you keep the patches available - but you
don't produce any new ones, right?

> So another interpretation of the above with Nick's proposal could be 10
> years on RHEL.  (though obviously I'm biased in favor of RHEL)

I wouldn't count mere availability of old patches on the server as
"support".

> Approaching this from another angle: please do add me to the "nosy" on
> any compatibility bugs with running latest python code on RHEL.

I'll try to remember, but really can't promise. But then, as I said
before: I think Linux support in Python is particularly easy. For
example, there isn't a single distribution-specific test in
configure.in.

> The other compat issues are in the toolchain: e.g. very recent versions
> of gcc .  In downstream Fedora, we tend to be amongst the first to run
> into new compilation warnings (and, occasionally, "exciting"
> code-generation bugs...)

Dropping support for old gcc versions (or other old compiler versions)
is probably an issue on its own. It will be difficult to figure out
what work-arounds are in place for what particular compiler glitch.

Regards,
Martin

From fuzzyman at voidspace.org.uk  Tue Dec  7 00:45:33 2010
From: fuzzyman at voidspace.org.uk (Michael Foord)
Date: Mon, 06 Dec 2010 23:45:33 +0000
Subject: [Python-Dev] Repo frozen for 3.2
In-Reply-To: <19D1A664-2C13-4D9F-9A45-B4DEBB4E77E1@gmail.com>
References: <loom.20101205T123251-490@post.gmane.org>	<5E183509-BBED-426B-BC70-DDA5640AD302@gmail.com>	<AANLkTi=A+OJK+dkJHGhfX6q-FeAtcBpYH+Y4-VzQqZDc@mail.gmail.com>
	<19D1A664-2C13-4D9F-9A45-B4DEBB4E77E1@gmail.com>
Message-ID: <4CFD759D.1090700@voidspace.org.uk>

On 06/12/2010 21:02, Raymond Hettinger wrote:
> On Dec 6, 2010, at 11:40 AM, Fred Drake wrote:
>
>> On Mon, Dec 6, 2010 at 2:21 PM, Raymond Hettinger
>> <raymond.hettinger at gmail.com>  wrote:
>>> We really ought to stop with the SafeFoo naming convention.
>>> It is only descriptive to the person who wrote the class or function,
>>> not to the user who will immediately wonder, "safe from what?"
>> Safe from bad vampire movies, of course!
>>
>> I'd not recognize the current Safe* class names as a pattern; there
>> are currently two in the py3k trunk:
>>
>>     configparser.SafeConfigParser
>>         -- very much a poor name
>>
>>     xmlrpc.client.SafeTransport
>>         -- perhaps should have been SSLTransport or HTTPSTransport
>>
>> I agree the "Safe" prefix isn't meaningful.
> IIRC, pprint has a safe_repr() and string.Template has safe_substitute()
> and pydoc has a safe import.
>
> Never new there was so much danger in the standard library :-)

What would you name those functions instead?

(SafeConfigParser is a naff name and only needed because ConfigParser is 
broken.)

Michael

>
> Raymond
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk


-- 

http://www.voidspace.org.uk/

READ CAREFULLY. By accepting and reading this email you agree,
on behalf of your employer, to release me from all obligations
and waivers arising from any and all NON-NEGOTIATED agreements,
licenses, terms-of-service, shrinkwrap, clickwrap, browsewrap,
confidentiality, non-disclosure, non-compete and acceptable use
policies (?BOGUS AGREEMENTS?) that I have entered into with your
employer, its partners, licensors, agents and assigns, in
perpetuity, without prejudice to my ongoing rights and privileges.
You further represent that you have the authority to release me
from any BOGUS AGREEMENTS on behalf of your employer.


From db3l.net at gmail.com  Tue Dec  7 01:24:39 2010
From: db3l.net at gmail.com (David Bolen)
Date: Mon, 06 Dec 2010 19:24:39 -0500
Subject: [Python-Dev] kill_python on windows buildbots (was Re: Stable
	buildbots)
References: <20101113133712.60e9be27@pitrou.net>
	<m2lj4wpqzt.fsf@valheru.db3l.homeip.net>
	<AANLkTimRZ_h9kpbQrXiD2rgP77EqUAHNPEWMrvAyoLSC@mail.gmail.com>
	<m2hbfkp9xm.fsf@valheru.db3l.homeip.net>
	<4CEB7E12.1070201@snakebite.org>
	<m2zkszochi.fsf@valheru.db3l.homeip.net>
Message-ID: <m2mxoil9a0.fsf_-_@valheru.db3l.homeip.net>

I previously wrote:

> I suspect the problem may be on the "identify which process to kill"
> rather than the "kill it" part, but it's definitely going to take time
> to figure that out for sure.  While the approach kill_python takes is
> much more appropriate, since we don't currently have multiple builds
> running simultaneously (and for me the machines are dedicated as build
> slaves, so I won't be having my own python_d), a more blanket kill
> operation is safe enough.

For anyone interested, I caught (well, Georg Brandl caught it first) a
case on Saturday with some hung processes on the Win7 buildbot that I
was able to verify kill_python failed to kill.  This was after having
a few instances where it did work fine.

I've created issue 10641 to track.  I also noticed another recent
issue (10136) that is also related to kill_python missing processes,
but given that it works in my case some of the time, and is always
called via the same path, not sure if that can be my problem.

I also realized that some fraction of the other cases I have seen
might not have truly been an issue, since from what I can see
kill_python is only run at the start of a build process, so hung
processes (even killable ones) from a prior build hang around until
the next build takes place.  They can however, interfere with the svn
checkout so things never get to the point of using kill_python.  So
maybe kill_python's use should be moved to the clean stage?

-- David


From rdmurray at bitdance.com  Tue Dec  7 03:30:08 2010
From: rdmurray at bitdance.com (R. David Murray)
Date: Mon, 06 Dec 2010 21:30:08 -0500
Subject: [Python-Dev] Repo frozen for 3.2
In-Reply-To: <19D1A664-2C13-4D9F-9A45-B4DEBB4E77E1@gmail.com>
References: <loom.20101205T123251-490@post.gmane.org>
	<5E183509-BBED-426B-BC70-DDA5640AD302@gmail.com>
	<AANLkTi=A+OJK+dkJHGhfX6q-FeAtcBpYH+Y4-VzQqZDc@mail.gmail.com>
	<19D1A664-2C13-4D9F-9A45-B4DEBB4E77E1@gmail.com>
Message-ID: <20101207023008.12D21207603@kimball.webabinitio.net>

On Mon, 06 Dec 2010 13:02:41 -0800, Raymond Hettinger <raymond.hettinger at gmail.com> wrote:
> On Dec 6, 2010, at 11:40 AM, Fred Drake wrote:
> > On Mon, Dec 6, 2010 at 2:21 PM, Raymond Hettinger
> > <raymond.hettinger at gmail.com> wrote:
> >> We really ought to stop with the SafeFoo naming convention.
> >> It is only descriptive to the person who wrote the class or function,
> >> not to the user who will immediately wonder, "safe from what?"
> > 
> > Safe from bad vampire movies, of course!
> > 
> > I'd not recognize the current Safe* class names as a pattern; there
> > are currently two in the py3k trunk:
> > 
> >    configparser.SafeConfigParser
> >        -- very much a poor name
> > 
> >    xmlrpc.client.SafeTransport
> >        -- perhaps should have been SSLTransport or HTTPSTransport
> > 
> > I agree the "Safe" prefix isn't meaningful. 
> 
> IIRC, pprint has a safe_repr() and string.Template has safe_substitute()
> and pydoc has a safe import.
> 
> Never new there was so much danger in the standard library :-)

unittest has an (internal) safe_repr, too.  I understand all four
of these to be "versions of xxx that won't raise".  I think that's
a reasonable use of the word 'safe', but perhaps there is something
better.

SafeConfigParser doesn't follow that pattern, of course, though it
is perhaps true that it would raise errors a bit less often... :)

--
R. David Murray                                      www.bitdance.com

From alexander.belopolsky at gmail.com  Tue Dec  7 04:03:16 2010
From: alexander.belopolsky at gmail.com (Alexander Belopolsky)
Date: Mon, 6 Dec 2010 22:03:16 -0500
Subject: [Python-Dev] Python and the Unicode Character Database
In-Reply-To: <4CFAC7B0.3080307@v.loewis.de>
References: <AANLkTinVpySuRFkjf-Sc+7a91zrxhuQpqPQL5nM9u1w+@mail.gmail.com>
	<20101128214311.092abd35@pitrou.net> <4CF2D4E9.3060607@v.loewis.de>
	<4CF2DAD7.2000408@egenix.com> <4CF2DFD1.10901@v.loewis.de>
	<4CF7FC01.80307@egenix.com> <4CF8004D.1080408@v.loewis.de>
	<AANLkTik6LKOWrwRTrQn+tFETU5eWfuinf+=T=_yeOA8u@mail.gmail.com>
	<AANLkTinXv1QSGb=oRUJutwsrWjHBppYn8UcbEeczEGFn@mail.gmail.com>
	<AANLkTikT2SL4h+ZAkz0PZe5G=Ecg8O971_8rX3W4MN1m@mail.gmail.com>
	<4CFAC7B0.3080307@v.loewis.de>
Message-ID: <AANLkTikBQ0KpiMCtFM4dTzG4r8Y9=eyE4EmWC57c-Fzf@mail.gmail.com>

On Sat, Dec 4, 2010 at 5:58 PM, "Martin v. L?wis" <martin at v.loewis.de> wrote:
>> I actually wonder if Python's re module can claim to provide even
>> Basic Unicode Support.
>
> Do you really wonder? Most definitely it does not.
>

Were you more optimistic four years ago?

http://bugs.python.org/issue1528154#msg54864

I was hoping that regex syntax would be useful in
explaining/documenting Python text processing routines (including
string to number conversions) without a heavy dose of Unicode
terminology.

From alexander.belopolsky at gmail.com  Tue Dec  7 05:46:54 2010
From: alexander.belopolsky at gmail.com (Alexander Belopolsky)
Date: Mon, 6 Dec 2010 23:46:54 -0500
Subject: [Python-Dev] transform() and untransform() methods,
	and the codec registry
In-Reply-To: <201012052325.27903.victor.stinner@haypocalc.com>
References: <20101202180651.F2AB0EE9CD@mail.python.org>
	<AANLkTikoRZJ4m3oGBhzgJ1_8WoJWf6ik_jm_uPt600pc@mail.gmail.com>
	<8762vax7lj.fsf@uwakimon.sk.tsukuba.ac.jp>
	<201012052325.27903.victor.stinner@haypocalc.com>
Message-ID: <AANLkTi=XdV7hr4k1OJXT9mj3C7-R6QOU5Y_1Tsu5H2nJ@mail.gmail.com>

On Sun, Dec 5, 2010 at 5:25 PM, Victor Stinner
<victor.stinner at haypocalc.com> wrote:
> On Saturday 04 December 2010 09:31:04 you wrote:
>> Alexander Belopolsky writes:
>> ?> In fact, once the language moratorium is over, I will argue that
>> ?> str.encode() and byte.decode() should deprecate encoding argument and
>> ?> just do UTF-8 encoding/decoding. ?Hopefully by that time most people
>> ?> will forget that other encodings exist. ?(I can dream, right?)
>>
>> It's just a dream. ?There's a pile of archival material, often on R/O
>> media, out there that won't be transcoded any more quickly than the
>> inscriptions on Tutankhamun's tomb.
>
> Not only, many libraries expect use bytes arguments encoded to a specific
> encoding (eg. locale encoding). Said differenlty, only few libraries written in
> C accept wchar* strings.
>

My proposal has nothing to do with C-API.  It only concerns Python API
of the builtin str type.

> The Linux kernel (or many, or all, UNIX/BSD kernels) only manipulate byte
> strings. The libc only accept wide characters for a few operations. I don't
> know how to open a file with an unicode path with the Linux libc: you have to
> encode it...
>

Yes, but hopefully the encoding used by the filesystem will be UTF-8.
For Python users, however, encoding details will hopefully be hidden
by the open() call.   Yes, I am aware of the many problems with
divining the filesystem encoding, but instructing application
developers to supply their own fsencoding in
open(filepath.encode(fsencoding)) calls is not very helpful.

> Alexander: you should first patch all UNIX/BSD kernels to use unicode
> everywhere, then patch all libc implementations, and then all libraries
> (written in C). After that, you can have a break.
>

As Martin explained later in this thread with respect to the
transform() method, removing codec argument from str.encode() method
does not imply removing the codecs themselves.    If I need a method
to encode strings to say koi8_r encoding, I can easily access it
directly:

>>> from encodings import koi8_r
>>> to_koi8_r = koi8_r.Codec().encode
>>> to_koi8_r('???')
(b'\xcb\xcf\xc4', 3)

More likely, however, I will only need en/decoding to read/write
legacy files and rather than encoding the strings explicitly before
writing into a file, I will just open that file with the correct
encoding.

Having all encodings accessible in a str method only promotes a
programming style where bytes objects can contain differently encoded
strings in different parts of the program.  Instead, well-written
programs should decode bytes on input, do all processing with str type
and decode on output.  When strings need to be passed to char* C APIs,
they should be encoded in UTF-8.  Many C APIs originally designed for
ASCII actually produce meaningful results when given  UTF-8 bytes.
(Supporting such usage was one of the design goals of UTF-8.)

From ncoghlan at gmail.com  Tue Dec  7 06:06:13 2010
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Tue, 7 Dec 2010 15:06:13 +1000
Subject: [Python-Dev] transform() and untransform() methods,
	and the codec registry
In-Reply-To: <AANLkTi=XdV7hr4k1OJXT9mj3C7-R6QOU5Y_1Tsu5H2nJ@mail.gmail.com>
References: <20101202180651.F2AB0EE9CD@mail.python.org>
	<AANLkTikoRZJ4m3oGBhzgJ1_8WoJWf6ik_jm_uPt600pc@mail.gmail.com>
	<8762vax7lj.fsf@uwakimon.sk.tsukuba.ac.jp>
	<201012052325.27903.victor.stinner@haypocalc.com>
	<AANLkTi=XdV7hr4k1OJXT9mj3C7-R6QOU5Y_1Tsu5H2nJ@mail.gmail.com>
Message-ID: <AANLkTinZT0mLYHuCntHi7Jy6firot3D65wGGes9Ks4vG@mail.gmail.com>

On Tue, Dec 7, 2010 at 2:46 PM, Alexander Belopolsky
<alexander.belopolsky at gmail.com> wrote:
> Having all encodings accessible in a str method only promotes a
> programming style where bytes objects can contain differently encoded
> strings in different parts of the program. ?Instead, well-written
> programs should decode bytes on input, do all processing with str type
> and decode on output. ?When strings need to be passed to char* C APIs,
> they should be encoded in UTF-8. ?Many C APIs originally designed for
> ASCII actually produce meaningful results when given ?UTF-8 bytes.
> (Supporting such usage was one of the design goals of UTF-8.)

This world sounds nice, but it isn't the one that exists right now.
Practicality beats purity and all that :)

Cheers,
Nick.

-- 
Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia

From alexander.belopolsky at gmail.com  Tue Dec  7 06:57:43 2010
From: alexander.belopolsky at gmail.com (Alexander Belopolsky)
Date: Tue, 7 Dec 2010 00:57:43 -0500
Subject: [Python-Dev] transform() and untransform() methods,
	and the codec registry
In-Reply-To: <AANLkTinZT0mLYHuCntHi7Jy6firot3D65wGGes9Ks4vG@mail.gmail.com>
References: <20101202180651.F2AB0EE9CD@mail.python.org>
	<AANLkTikoRZJ4m3oGBhzgJ1_8WoJWf6ik_jm_uPt600pc@mail.gmail.com>
	<8762vax7lj.fsf@uwakimon.sk.tsukuba.ac.jp>
	<201012052325.27903.victor.stinner@haypocalc.com>
	<AANLkTi=XdV7hr4k1OJXT9mj3C7-R6QOU5Y_1Tsu5H2nJ@mail.gmail.com>
	<AANLkTinZT0mLYHuCntHi7Jy6firot3D65wGGes9Ks4vG@mail.gmail.com>
Message-ID: <AANLkTinF5OJq5hk8k1DQrKHJkzS2q7_0G9KC8i4zEyV4@mail.gmail.com>

On Tue, Dec 7, 2010 at 12:06 AM, Nick Coghlan <ncoghlan at gmail.com> wrote:
> On Tue, Dec 7, 2010 at 2:46 PM, Alexander Belopolsky
> <alexander.belopolsky at gmail.com> wrote:
>> Having all encodings accessible in a str method only promotes a
>> programming style where bytes objects can contain differently encoded
>> strings in different parts of the program. ?Instead, well-written
>> programs should decode bytes on input, do all processing with str type
>> and decode on output. ?When strings need to be passed to char* C APIs,
>> they should be encoded in UTF-8. ?Many C APIs originally designed for
>> ASCII actually produce meaningful results when given ?UTF-8 bytes.
>> (Supporting such usage was one of the design goals of UTF-8.)
>
> This world sounds nice, but it isn't the one that exists right now.
> Practicality beats purity and all that :)

.. and default encoding being fixed as UTF-8 already goes 99% of the
way to that world.  As long as I can use encode/decode without an
argument, it does not bother me much that they can take one.  These
methods are also much easier to ignore than the transform/untransform
pair simply because it is only one method per class.
transform/untransform have much larger mental footprint not only
because there are two of them in both str and bytes, but also because
both str and bytes have a synonymously named translate method.  With
43 non-special methods, str interface is already huge.  The
transform() method with a suitable set of codecs could possibly
replace things like expandtabs() or swapcase(), but that would be like
writing x.transform('exp') and x.unstransform('exp') instead of
math.exp(x) and math.log(x).

From lukasz at langa.pl  Tue Dec  7 09:24:37 2010
From: lukasz at langa.pl (=?utf-8?Q?=C5=81ukasz_Langa?=)
Date: Tue, 7 Dec 2010 09:24:37 +0100
Subject: [Python-Dev] [RELEASED] Python 3.2 beta 1
In-Reply-To: <4CFD59C8.4070006@python.org>
References: <4CFD59C8.4070006@python.org>
Message-ID: <433FCEE2-CD6C-48BD-90DF-2990B27E4EED@langa.pl>

Wiadomo?? napisana przez Georg Brandl w dniu 2010-12-06, o godz. 22:46:

> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
> 
> On behalf of the Python development team, I'm happy to announce the
> first of two beta preview releases of Python 3.2.
> 
> Python 3.2 is a continuation of the efforts to improve and stabilize the
> Python 3.x line.  Since the final release of Python 2.7, the 2.x line
> will only receive bugfixes, and new features are developed for 3.x only.
> 
> Since PEP 3003, the Moratorium on Language Changes, is in effect, there
> are no changes in Python's syntax and built-in types in Python 3.2.
> Development efforts concentrated on the standard library and support for
> porting code to Python 3.  Highlights are:
> 
> [snip]

* configparser 1.1: new API using the mapping protocol access, support for pluggable interpolation handlers, additional interpolation handler (ExtendedInterpolation) which supports the zc.buildout syntax, support for alternative option/value delimiters, support for customization of accepted INI file structure (e.g. comment prefixes, name of the DEFAULT section, indentation, empty lines in multiline values, etc.), support for specifying encoding for read operations, ConfigParser class deprecated in favor of SafeConfigParser, lots of other small changes.

-- 
Best regards,
?ukasz Langa
tel. +48 791 080 144
WWW http://lukasz.langa.pl/

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20101207/1ccb5e59/attachment.html>

From flub at devork.be  Tue Dec  7 13:22:29 2010
From: flub at devork.be (Floris Bruynooghe)
Date: Tue, 7 Dec 2010 12:22:29 +0000
Subject: [Python-Dev] PEP 11: Dropping support for ten year old systems
In-Reply-To: <4CFD3193.2030906@v.loewis.de>
References: <4CFC08C1.3080609@v.loewis.de>
	<AANLkTinvJ9AUeHH58r_Kbfk9mTZGoSLHtG0UpnDqmsRU@mail.gmail.com>
	<loom.20101206T080949-953@post.gmane.org>
	<4CFCAA66.6060901@v.loewis.de>
	<AANLkTi=7+ykS2kzrv3M9Fsv+V5zZpw1-5Tt=ckZ397XC@mail.gmail.com>
	<4CFD3193.2030906@v.loewis.de>
Message-ID: <AANLkTik5_dcoEt=q5-7QU5AjKAAv6ooSDLYesUnL38mW@mail.gmail.com>

On 6 December 2010 18:55, "Martin v. L?wis" <martin at v.loewis.de> wrote:
> Am 06.12.2010 14:40, schrieb Floris Bruynooghe:
>> On 6 December 2010 09:18, "Martin v. L?wis" <martin at v.loewis.de> wrote:
>>>> Also, it is not clear what to do about distributions/OSs
>>>> without any official EOL or life cycles.
>>>
>>> Here my proposal stands: 10 years, by default.
>>
>> How about max(EOL, 10years)? ?That sounds like it could be a useful guideline.
>>
>> (Personally I'd be sad to see Solaris 8 go in the next few years)
>
> I guess we'll be sorry, then: under that policy, max(EOL, 10years) comes
> out as "not supported" (not sure whether you were aware of that).

I was :-)
Which is why I said "sad" and not "panic" or something.  I do realise
I'm probably in the minority and that it doesn't justify the burden on
everyone in the Python community.

> We are not going to take the sources of old releases offline.

exactly

Regards
Floris

-- 
Debian GNU/Linux -- The Power of Freedom
www.debian.org | www.gnu.org | www.kernel.org

From vlastimil.brom at gmail.com  Tue Dec  7 14:02:47 2010
From: vlastimil.brom at gmail.com (Vlastimil Brom)
Date: Tue, 7 Dec 2010 14:02:47 +0100
Subject: [Python-Dev] Python and the Unicode Character Database
In-Reply-To: <AANLkTikBQ0KpiMCtFM4dTzG4r8Y9=eyE4EmWC57c-Fzf@mail.gmail.com>
References: <AANLkTinVpySuRFkjf-Sc+7a91zrxhuQpqPQL5nM9u1w+@mail.gmail.com>
	<20101128214311.092abd35@pitrou.net> <4CF2D4E9.3060607@v.loewis.de>
	<4CF2DAD7.2000408@egenix.com> <4CF2DFD1.10901@v.loewis.de>
	<4CF7FC01.80307@egenix.com> <4CF8004D.1080408@v.loewis.de>
	<AANLkTik6LKOWrwRTrQn+tFETU5eWfuinf+=T=_yeOA8u@mail.gmail.com>
	<AANLkTinXv1QSGb=oRUJutwsrWjHBppYn8UcbEeczEGFn@mail.gmail.com>
	<AANLkTikT2SL4h+ZAkz0PZe5G=Ecg8O971_8rX3W4MN1m@mail.gmail.com>
	<4CFAC7B0.3080307@v.loewis.de>
	<AANLkTikBQ0KpiMCtFM4dTzG4r8Y9=eyE4EmWC57c-Fzf@mail.gmail.com>
Message-ID: <AANLkTinXctFV081nS-SS3zS_FRW4Z2AcTWGn-KQAS-4q@mail.gmail.com>

2010/12/7 Alexander Belopolsky <alexander.belopolsky at gmail.com>:
> On Sat, Dec 4, 2010 at 5:58 PM, "Martin v. L?wis" <martin at v.loewis.de> wrote:
>>> I actually wonder if Python's re module can claim to provide even
>>> Basic Unicode Support.
>>
>> Do you really wonder? Most definitely it does not.
>>
>
> Were you more optimistic four years ago?
>
> http://bugs.python.org/issue1528154#msg54864
>
> I was hoping that regex syntax would be useful in
> explaining/documenting Python text processing routines (including
> string to number conversions) without a heavy dose of Unicode
> terminology.
>

The new regex version
http://bugs.python.org/issue2636
supports much more features, including unicode properties, and the
mentioned possix classes etc. but definitely not all of the
requirements of that rather "generous" list.
http://www.unicode.org/reports/tr18/
It seems, e.g. in Perl, there are some omissions too
http://perldoc.perl.org/perlunicode.html#Unicode-Regular-Expression-Support-Level

Do you know of any re engine fully complying to to tr18, even at the
first level: "Basic Unicode Support"?

vbr

From tseaver at palladion.com  Tue Dec  7 15:38:30 2010
From: tseaver at palladion.com (Tres Seaver)
Date: Tue, 07 Dec 2010 09:38:30 -0500
Subject: [Python-Dev] kill_python on windows buildbots (was Re: Stable
	buildbots)
In-Reply-To: <m2mxoil9a0.fsf_-_@valheru.db3l.homeip.net>
References: <20101113133712.60e9be27@pitrou.net>	<m2lj4wpqzt.fsf@valheru.db3l.homeip.net>	<AANLkTimRZ_h9kpbQrXiD2rgP77EqUAHNPEWMrvAyoLSC@mail.gmail.com>	<m2hbfkp9xm.fsf@valheru.db3l.homeip.net>	<4CEB7E12.1070201@snakebite.org>	<m2zkszochi.fsf@valheru.db3l.homeip.net>
	<m2mxoil9a0.fsf_-_@valheru.db3l.homeip.net>
Message-ID: <idlgt6$8vq$2@dough.gmane.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 12/06/2010 07:24 PM, David Bolen wrote:
> I previously wrote:
> 
>> I suspect the problem may be on the "identify which process to kill"
>> rather than the "kill it" part, but it's definitely going to take time
>> to figure that out for sure.  While the approach kill_python takes is
>> much more appropriate, since we don't currently have multiple builds
>> running simultaneously (and for me the machines are dedicated as build
>> slaves, so I won't be having my own python_d), a more blanket kill
>> operation is safe enough.
> 
> For anyone interested, I caught (well, Georg Brandl caught it first) a
> case on Saturday with some hung processes on the Win7 buildbot that I
> was able to verify kill_python failed to kill.  This was after having
> a few instances where it did work fine.
> 
> I've created issue 10641 to track.  I also noticed another recent
> issue (10136) that is also related to kill_python missing processes,
> but given that it works in my case some of the time, and is always
> called via the same path, not sure if that can be my problem.
> 
> I also realized that some fraction of the other cases I have seen
> might not have truly been an issue, since from what I can see
> kill_python is only run at the start of a build process, so hung
> processes (even killable ones) from a prior build hang around until
> the next build takes place.  They can however, interfere with the svn
> checkout so things never get to the point of using kill_python.  So
> maybe kill_python's use should be moved to the clean stage?

Maybe belt-and-suspenders it in both places.


Tres.
- -- 
===================================================================
Tres Seaver          +1 540-429-0999          tseaver at palladion.com
Palladion Software   "Excellence by Design"    http://palladion.com
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.10 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAkz+RuYACgkQ+gerLs4ltQ58OgCgiMs1JAdFjkOGxrJ3X3nB1k18
iKcAoMCP+MYumDe0r/XkZr29e7loACxP
=wrWe
-----END PGP SIGNATURE-----


From alexander.belopolsky at gmail.com  Tue Dec  7 15:41:59 2010
From: alexander.belopolsky at gmail.com (Alexander Belopolsky)
Date: Tue, 7 Dec 2010 09:41:59 -0500
Subject: [Python-Dev] Python and the Unicode Character Database
In-Reply-To: <AANLkTinXctFV081nS-SS3zS_FRW4Z2AcTWGn-KQAS-4q@mail.gmail.com>
References: <AANLkTinVpySuRFkjf-Sc+7a91zrxhuQpqPQL5nM9u1w+@mail.gmail.com>
	<20101128214311.092abd35@pitrou.net> <4CF2D4E9.3060607@v.loewis.de>
	<4CF2DAD7.2000408@egenix.com> <4CF2DFD1.10901@v.loewis.de>
	<4CF7FC01.80307@egenix.com> <4CF8004D.1080408@v.loewis.de>
	<AANLkTik6LKOWrwRTrQn+tFETU5eWfuinf+=T=_yeOA8u@mail.gmail.com>
	<AANLkTinXv1QSGb=oRUJutwsrWjHBppYn8UcbEeczEGFn@mail.gmail.com>
	<AANLkTikT2SL4h+ZAkz0PZe5G=Ecg8O971_8rX3W4MN1m@mail.gmail.com>
	<4CFAC7B0.3080307@v.loewis.de>
	<AANLkTikBQ0KpiMCtFM4dTzG4r8Y9=eyE4EmWC57c-Fzf@mail.gmail.com>
	<AANLkTinXctFV081nS-SS3zS_FRW4Z2AcTWGn-KQAS-4q@mail.gmail.com>
Message-ID: <AANLkTintLL6opU+WH85KcPOiDTjh=G7skTC7cXjyoAEh@mail.gmail.com>

On Tue, Dec 7, 2010 at 8:02 AM, Vlastimil Brom <vlastimil.brom at gmail.com> wrote:
..
> It seems, e.g. in Perl, there are some omissions too
> http://perldoc.perl.org/perlunicode.html#Unicode-Regular-Expression-Support-Level
>
> Do you know of any re engine fully complying to to tr18, even at the
> first level: "Basic Unicode Support"?
>
I would say Perl comes very close.  At least it explicitly documents
the missing features and offers workarounds in its reference manual.
I am actually not as concerned about missing features as I am about
non-conformance in the widely used features such as digits' matching
with '\d'.

From db3l.net at gmail.com  Tue Dec  7 16:11:58 2010
From: db3l.net at gmail.com (David Bolen)
Date: Tue, 07 Dec 2010 10:11:58 -0500
Subject: [Python-Dev] kill_python on windows buildbots
References: <20101113133712.60e9be27@pitrou.net>
	<m2lj4wpqzt.fsf@valheru.db3l.homeip.net>
	<AANLkTimRZ_h9kpbQrXiD2rgP77EqUAHNPEWMrvAyoLSC@mail.gmail.com>
	<m2hbfkp9xm.fsf@valheru.db3l.homeip.net>
	<4CEB7E12.1070201@snakebite.org>
	<m2zkszochi.fsf@valheru.db3l.homeip.net>
	<m2mxoil9a0.fsf_-_@valheru.db3l.homeip.net>
	<idlgt6$8vq$2@dough.gmane.org>
Message-ID: <m2fwu9lirl.fsf@valheru.db3l.homeip.net>

Tres Seaver <tseaver at palladion.com> writes:

> Maybe belt-and-suspenders it in both places.

The clean batch file is also called from the build step, so relocating it
there should maintain the existing behavior as well.

Hirokazu (ocean-city) pointed out in my new issue an earlier issue he
created (#9973) that included a patch for such a change.

In thinking about it some more, I suppose there's still a small window
for a loss of communication during a test which results in clean not
being run (which could then block the next svn checkout without an
opportunity to kill the processes), so maybe the right place is
actually at the end of the test batch file which is the step during
which such hung processes might get generated?  I don't know the
history, if any, of it's current location in the flow.

Relocating the use of kill_python seems reasonable to me, after which
we could sort of wait and see if we run into any other hangs that
interfere with the builds.  I can't make such a change myself though.

-- David


From dmalcolm at redhat.com  Tue Dec  7 16:56:47 2010
From: dmalcolm at redhat.com (David Malcolm)
Date: Tue, 07 Dec 2010 10:56:47 -0500
Subject: [Python-Dev] PEP 11: Dropping support for ten year old systems
In-Reply-To: <4CFD6C2B.8030708@v.loewis.de>
References: <4CFC08C1.3080609@v.loewis.de>
	<AANLkTinvJ9AUeHH58r_Kbfk9mTZGoSLHtG0UpnDqmsRU@mail.gmail.com>
	<loom.20101206T080949-953@post.gmane.org>
	<4CFCAA66.6060901@v.loewis.de>
	<1291676103.15083.28.camel@radiator.bos.redhat.com>
	<4CFD6C2B.8030708@v.loewis.de>
Message-ID: <1291737407.20128.17.camel@radiator.bos.redhat.com>

On Tue, 2010-12-07 at 00:05 +0100, "Martin v. L?wis" wrote: 
> >> So by this policy, RHEL and SuSE users would be off worse than with
> >> my original proposal (10 years).
> > 
> > Red Hat continues to provide patches for RHEL within the "Extended Life
> > Cycle" (years 8, 9 and 10), but it's an optional add-on.
> 
> My understanding is that you keep the patches available - but you
> don't produce any new ones, right?

It typically involves backporting (and testing) security fixes to the
older versions of the various OS packages.  Whether or not the results
of that work count as "new patches" is debatable.

I don't know if CentOS and the other people who rebuild the RHEL sources
track those final 3 years.

Dave



From dmalcolm at redhat.com  Tue Dec  7 16:56:47 2010
From: dmalcolm at redhat.com (David Malcolm)
Date: Tue, 07 Dec 2010 10:56:47 -0500
Subject: [Python-Dev] PEP 11: Dropping support for ten year old systems
In-Reply-To: <4CFD6C2B.8030708@v.loewis.de>
References: <4CFC08C1.3080609@v.loewis.de>
	<AANLkTinvJ9AUeHH58r_Kbfk9mTZGoSLHtG0UpnDqmsRU@mail.gmail.com>
	<loom.20101206T080949-953@post.gmane.org>
	<4CFCAA66.6060901@v.loewis.de>
	<1291676103.15083.28.camel@radiator.bos.redhat.com>
	<4CFD6C2B.8030708@v.loewis.de>
Message-ID: <1291737407.20128.17.camel@radiator.bos.redhat.com>

On Tue, 2010-12-07 at 00:05 +0100, "Martin v. L?wis" wrote: 
> >> So by this policy, RHEL and SuSE users would be off worse than with
> >> my original proposal (10 years).
> > 
> > Red Hat continues to provide patches for RHEL within the "Extended Life
> > Cycle" (years 8, 9 and 10), but it's an optional add-on.
> 
> My understanding is that you keep the patches available - but you
> don't produce any new ones, right?

It typically involves backporting (and testing) security fixes to the
older versions of the various OS packages.  Whether or not the results
of that work count as "new patches" is debatable.

I don't know if CentOS and the other people who rebuild the RHEL sources
track those final 3 years.

Dave



From merwok at netwok.org  Tue Dec  7 18:24:33 2010
From: merwok at netwok.org (=?UTF-8?B?w4lyaWMgQXJhdWpv?=)
Date: Tue, 07 Dec 2010 18:24:33 +0100
Subject: [Python-Dev] [Python-checkins] r87118 - in
 python/branches/py3k: Mac/BuildScript/build-installer.py Makefile.pre.in
In-Reply-To: <20101207144105.C0E88EEB67@mail.python.org>
References: <20101207144105.C0E88EEB67@mail.python.org>
Message-ID: <4CFE6DD1.9030801@netwok.org>

Hello,

> Author: ronald.oussoren
> New Revision: 87118
> Log: Two small changes to adjust framework builds to the new stable ABI
> 
> Modified: python/branches/py3k/Mac/BuildScript/build-installer.py
> ==============================================================================
> +    LDVERSION=None
> +    VERSION=None
> +    ABIFLAGS=None
> +
> +    with open(os.path.join(buildDir, 'Makefile')) as fp:
> +        for ln in fp:
> +            if ln.startswith('VERSION='):
> +                VERSION=ln.split()[1]
> +            if ln.startswith('ABIFLAGS='):
> +                ABIFLAGS=ln.split()[1]
> +
> +            if ln.startswith('LDVERSION='):
> +                LDVERSION=ln.split()[1]
> +
> +    LDVERSION = LDVERSION.replace('$(VERSION)', VERSION)
> +    LDVERSION = LDVERSION.replace('$(ABIFLAGS)', ABIFLAGS)

Isn?t this a perfect use case for the new sysconfig module?

Regards


From martin at v.loewis.de  Tue Dec  7 21:20:21 2010
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Tue, 07 Dec 2010 21:20:21 +0100
Subject: [Python-Dev] Python and the Unicode Character Database
In-Reply-To: <AANLkTikBQ0KpiMCtFM4dTzG4r8Y9=eyE4EmWC57c-Fzf@mail.gmail.com>
References: <AANLkTinVpySuRFkjf-Sc+7a91zrxhuQpqPQL5nM9u1w+@mail.gmail.com>	<20101128214311.092abd35@pitrou.net>
	<4CF2D4E9.3060607@v.loewis.de>	<4CF2DAD7.2000408@egenix.com>
	<4CF2DFD1.10901@v.loewis.de>	<4CF7FC01.80307@egenix.com>
	<4CF8004D.1080408@v.loewis.de>	<AANLkTik6LKOWrwRTrQn+tFETU5eWfuinf+=T=_yeOA8u@mail.gmail.com>	<AANLkTinXv1QSGb=oRUJutwsrWjHBppYn8UcbEeczEGFn@mail.gmail.com>	<AANLkTikT2SL4h+ZAkz0PZe5G=Ecg8O971_8rX3W4MN1m@mail.gmail.com>	<4CFAC7B0.3080307@v.loewis.de>
	<AANLkTikBQ0KpiMCtFM4dTzG4r8Y9=eyE4EmWC57c-Fzf@mail.gmail.com>
Message-ID: <4CFE9705.4020800@v.loewis.de>

Am 07.12.2010 04:03, schrieb Alexander Belopolsky:
> On Sat, Dec 4, 2010 at 5:58 PM, "Martin v. L?wis" <martin at v.loewis.de> wrote:
>>> I actually wonder if Python's re module can claim to provide even
>>> Basic Unicode Support.
>>
>> Do you really wonder? Most definitely it does not.
>>
> 
> Were you more optimistic four years ago?
> 
> http://bugs.python.org/issue1528154#msg54864

Not at all. I thought back then, and think now, that Python should,
but doesn't, support TR#18. I don't view that lack as a severe problem,
though, and apparently none of the other contributors did so, either.

Regards,
Martin

From vinay_sajip at yahoo.co.uk  Tue Dec  7 21:26:06 2010
From: vinay_sajip at yahoo.co.uk (Vinay Sajip)
Date: Tue, 7 Dec 2010 20:26:06 +0000 (UTC)
Subject: [Python-Dev] Using logging in the stdlib and its unit tests
Message-ID: <loom.20101207T212306-892@post.gmane.org>

This issue was brought to my notice today:

http://bugs.python.org/issue10626

and reference was made in the comments to possible obstacles facing stdlib
maintainers who might wish to use logging in the stdlib and in its unit tests.

>From my perspective and as mentioned in the logging documentation, library code
which uses logging should add a NullHandler instance to any top-level logger,
which will avoid any "No handlers could be found for logger XXX" message if no
logging handlers have been set up. This applies to stdlib code, too, though it
would be good if a logger namespace could be agreed for stdlib usage. (The
logging package itself uses the logger "py.warnings" to redirect warnings to
logging when configured to do so. Perhaps we could standardize on "py.XXX" for
stdlib usage?)

I would suggest that when unit testing, rather than adding StreamHandlers to log
to stderr, that something like TestHandler and Matcher from this post:

http://plumberjack.blogspot.com/2010/09/unit-testing-and-logging.html

This will allow assertion checking of logged messages without resorting to
StringIO, getvalue() etc. If people think it's a good idea, I can add the
TestHandler/Matcher classes to the unit test infrastructure (they wouldn't
become part of the public API, at least until 3.3, but I presume they could be
used in the stdlib unit tests).

On the question of using logging in the stdlib and its unit tests, I would like
to throw this open to python-dev: is there anyone here who wants to use logging
but feels that they can't because of some shortcoming in logging? AFAIK there
should be no obstacles. The preferred approach in stdlib code is as I have
outlined above: add a NullHandler to top-level loggers to avoid
misconfiguration, document the logger names you use, and avoid adding any other
handlers by default. For unit testing, add a TestHandler (or equivalent) to your
top-level logger at the start of the test, make any assertions you need to
regarding what has been logged, remove the handler and close it at the end of
the test. I don't believe there is any inherent conflict or incompatibility
between logger usage in the stdlib, use of logging assertions in unit tests and
use of handlers by application code, but I am happy to have any mistake on my
part pointed out.

>From what I've seen, concurrent.futures adds a StreamHandler which is removed in
the unit test and replaced by StreamHandler pointing to a different stream.
This, I believe, should be changed in line with what I've said above.

Comments?

Regards,

Vinay Sajip


From nad at acm.org  Tue Dec  7 21:48:18 2010
From: nad at acm.org (Ned Deily)
Date: Tue, 07 Dec 2010 12:48:18 -0800
Subject: [Python-Dev] [Python-checkins] r87118 - in
	python/branches/py3k: Mac/BuildScript/build-installer.py
	Makefile.pre.in
References: <20101207144105.C0E88EEB67@mail.python.org>
	<4CFE6DD1.9030801@netwok.org>
Message-ID: <nad-9EC57B.12481807122010@dough.gmane.org>

In article <4CFE6DD1.9030801 at netwok.org>,
 ?ric Araujo <merwok at netwok.org> wrote:
> > Author: ronald.oussoren
> > New Revision: 87118
> > Log: Two small changes to adjust framework builds to the new stable ABI
> > 
> > Modified: python/branches/py3k/Mac/BuildScript/build-installer.py
> > ============================================================================
> > ==
> > +    LDVERSION=None
> > +    VERSION=None
> > +    ABIFLAGS=None
> > +
> > +    with open(os.path.join(buildDir, 'Makefile')) as fp:
> > +        for ln in fp:
> > +            if ln.startswith('VERSION='):
> > +                VERSION=ln.split()[1]
> > +            if ln.startswith('ABIFLAGS='):
> > +                ABIFLAGS=ln.split()[1]
> > +
> > +            if ln.startswith('LDVERSION='):
> > +                LDVERSION=ln.split()[1]
> > +
> > +    LDVERSION = LDVERSION.replace('$(VERSION)', VERSION)
> > +    LDVERSION = LDVERSION.replace('$(ABIFLAGS)', ABIFLAGS)
> 
> Isn?t this a perfect use case for the new sysconfig module?

There is no guarantee which version of Python the build-installer script 
is being run with and, hence, no guarantee which version of sysconfig is 
available.  Typically it is an older system-supplied Python which is 
fine as it is merely used to run build-installer (which includes running 
Sphinx to build the documentation) and otherwise has no impact on the 
Python being built.

-- 
 Ned Deily,
 nad at acm.org


From solipsis at pitrou.net  Tue Dec  7 21:50:53 2010
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Tue, 7 Dec 2010 21:50:53 +0100
Subject: [Python-Dev] Using logging in the stdlib and its unit tests
References: <loom.20101207T212306-892@post.gmane.org>
Message-ID: <20101207215053.025ecf52@pitrou.net>

On Tue, 7 Dec 2010 20:26:06 +0000 (UTC)
Vinay Sajip <vinay_sajip at yahoo.co.uk> wrote:
> 
> >From my perspective and as mentioned in the logging documentation, library code
> which uses logging should add a NullHandler instance to any top-level logger,
> which will avoid any "No handlers could be found for logger XXX" message if no
> logging handlers have been set up. This applies to stdlib code, too, though it
> would be good if a logger namespace could be agreed for stdlib usage. (The
> logging package itself uses the logger "py.warnings" to redirect warnings to
> logging when configured to do so. Perhaps we could standardize on "py.XXX" for
> stdlib usage?)

I thought "error" and "critical" messages were logged to stderr by
default? Isn't it the case?

If any library defining a logger must also add a NullHandler just in
case, ISTM that complicates a lot the use of logging (and could explain
its impopularity). Both for library writers and application writers,
apparently (since the former will have to add the NullHandler, and the
latter will have to override it so that error messages get printed out
rather than lost).

Regards

Antoine.



From lukasz at langa.pl  Tue Dec  7 22:04:36 2010
From: lukasz at langa.pl (=?utf-8?Q?=C5=81ukasz_Langa?=)
Date: Tue, 7 Dec 2010 22:04:36 +0100
Subject: [Python-Dev] Using logging in the stdlib and its unit tests
In-Reply-To: <20101207215053.025ecf52@pitrou.net>
References: <loom.20101207T212306-892@post.gmane.org>
	<20101207215053.025ecf52@pitrou.net>
Message-ID: <8E88AE41-04D5-4B5A-9ECB-1CC6A137B3C0@langa.pl>

Wiadomo?? napisana przez Antoine Pitrou w dniu 2010-12-07, o godz. 21:50:

> If any library defining a logger must also add a NullHandler just in
> case, ISTM that complicates a lot the use of logging (and could explain
> its impopularity). Both for library writers and application writers,
> apparently (since the former will have to add the NullHandler, and the
> latter will have to override it so that error messages get printed out
> rather than lost).

ISTM that the stdlib is a special case here. If you're writing an application then the "No handlers could be found" message is actually useful because there's hardly any reason no to include one.

If you're writing a library, well, that depends. Part of libraries might want to have their specific default handlers, part of them might want to leave that to the library user (e.g. "No handlers can be found" by default) and part of them might want to silence logs by default.

One way or the other, we should really default to the convenience of application developers. This is currently the case.

-- 
Best regards,
?ukasz Langa
tel. +48 791 080 144
WWW http://lukasz.langa.pl/

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20101207/e858ed48/attachment.html>

From solipsis at pitrou.net  Tue Dec  7 22:19:34 2010
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Tue, 7 Dec 2010 22:19:34 +0100
Subject: [Python-Dev] Using logging in the stdlib and its unit tests
In-Reply-To: <8E88AE41-04D5-4B5A-9ECB-1CC6A137B3C0@langa.pl>
References: <loom.20101207T212306-892@post.gmane.org>
	<20101207215053.025ecf52@pitrou.net>
	<8E88AE41-04D5-4B5A-9ECB-1CC6A137B3C0@langa.pl>
Message-ID: <20101207221934.725be79d@pitrou.net>

On Tue, 7 Dec 2010 22:04:36 +0100
?ukasz Langa <lukasz at langa.pl> wrote:
> Wiadomo?? napisana przez Antoine Pitrou w dniu 2010-12-07, o godz. 21:50:
> 
> > If any library defining a logger must also add a NullHandler just in
> > case, ISTM that complicates a lot the use of logging (and could explain
> > its impopularity). Both for library writers and application writers,
> > apparently (since the former will have to add the NullHandler, and the
> > latter will have to override it so that error messages get printed out
> > rather than lost).
> 
> ISTM that the stdlib is a special case here. If you're writing an application then the "No handlers could be found" message is actually useful because there's hardly any reason no to include one.

Why do you say that? Not having to add a handler is certainly useful
when you are doing some quick prototyping or simply writing a script
(situations in which you still want to get error messages displayed
properly by the libraries).

> One way or the other, we should really default to the convenience of application developers. This is currently the case.

Why wouldn't there be a default convenience of printing out errors?
It's already the case for the root handler, so why would other handler
be treated differently?

>>> import logging
>>> logging.debug("foo")
>>> logging.error("bar")
ERROR:root:bar

Thanks

Antoine.

From alexander.belopolsky at gmail.com  Tue Dec  7 22:21:46 2010
From: alexander.belopolsky at gmail.com (Alexander Belopolsky)
Date: Tue, 7 Dec 2010 16:21:46 -0500
Subject: [Python-Dev] Python and the Unicode Character Database
In-Reply-To: <AANLkTinXctFV081nS-SS3zS_FRW4Z2AcTWGn-KQAS-4q@mail.gmail.com>
References: <AANLkTinVpySuRFkjf-Sc+7a91zrxhuQpqPQL5nM9u1w+@mail.gmail.com>
	<20101128214311.092abd35@pitrou.net> <4CF2D4E9.3060607@v.loewis.de>
	<4CF2DAD7.2000408@egenix.com> <4CF2DFD1.10901@v.loewis.de>
	<4CF7FC01.80307@egenix.com> <4CF8004D.1080408@v.loewis.de>
	<AANLkTik6LKOWrwRTrQn+tFETU5eWfuinf+=T=_yeOA8u@mail.gmail.com>
	<AANLkTinXv1QSGb=oRUJutwsrWjHBppYn8UcbEeczEGFn@mail.gmail.com>
	<AANLkTikT2SL4h+ZAkz0PZe5G=Ecg8O971_8rX3W4MN1m@mail.gmail.com>
	<4CFAC7B0.3080307@v.loewis.de>
	<AANLkTikBQ0KpiMCtFM4dTzG4r8Y9=eyE4EmWC57c-Fzf@mail.gmail.com>
	<AANLkTinXctFV081nS-SS3zS_FRW4Z2AcTWGn-KQAS-4q@mail.gmail.com>
Message-ID: <AANLkTinm3wi31g+Y5SjneOp8HGSLX2wLxjbaW2-aWEUA@mail.gmail.com>

On Tue, Dec 7, 2010 at 8:02 AM, Vlastimil Brom <vlastimil.brom at gmail.com> wrote:
..
> Do you know of any re engine fully complying to to tr18, even at the
> first level: "Basic Unicode Support"?
>
"""
ICU Regular Expressions conform to Unicode Technical Standard #18 ,
Unicode Regular Expressions, level 1, and in addition include Default
Word boundaries and Name Properties from level 2.
""" http://userguide.icu-project.org/strings/regexp

From eric at trueblade.com  Tue Dec  7 22:57:20 2010
From: eric at trueblade.com (Eric Smith)
Date: Tue, 07 Dec 2010 16:57:20 -0500
Subject: [Python-Dev] Using logging in the stdlib and its unit tests
In-Reply-To: <loom.20101207T212306-892@post.gmane.org>
References: <loom.20101207T212306-892@post.gmane.org>
Message-ID: <4CFEADC0.50609@trueblade.com>

On 12/07/2010 03:26 PM, Vinay Sajip wrote:
> I would suggest that when unit testing, rather than adding StreamHandlers to log
> to stderr, that something like TestHandler and Matcher from this post:
>
> http://plumberjack.blogspot.com/2010/09/unit-testing-and-logging.html
>
> This will allow assertion checking of logged messages without resorting to
> StringIO, getvalue() etc. If people think it's a good idea, I can add the
> TestHandler/Matcher classes to the unit test infrastructure (they wouldn't
> become part of the public API, at least until 3.3, but I presume they could be
> used in the stdlib unit tests).

Wouldn't it make more sense to add these to test.support? I don't think 
we make any guarantees about its API being stable, although I have a 
faint recollection of that being debated in the past.

Eric.

From vlastimil.brom at gmail.com  Tue Dec  7 23:19:44 2010
From: vlastimil.brom at gmail.com (Vlastimil Brom)
Date: Tue, 7 Dec 2010 23:19:44 +0100
Subject: [Python-Dev] Python and the Unicode Character Database
In-Reply-To: <AANLkTinm3wi31g+Y5SjneOp8HGSLX2wLxjbaW2-aWEUA@mail.gmail.com>
References: <AANLkTinVpySuRFkjf-Sc+7a91zrxhuQpqPQL5nM9u1w+@mail.gmail.com>
	<20101128214311.092abd35@pitrou.net> <4CF2D4E9.3060607@v.loewis.de>
	<4CF2DAD7.2000408@egenix.com> <4CF2DFD1.10901@v.loewis.de>
	<4CF7FC01.80307@egenix.com> <4CF8004D.1080408@v.loewis.de>
	<AANLkTik6LKOWrwRTrQn+tFETU5eWfuinf+=T=_yeOA8u@mail.gmail.com>
	<AANLkTinXv1QSGb=oRUJutwsrWjHBppYn8UcbEeczEGFn@mail.gmail.com>
	<AANLkTikT2SL4h+ZAkz0PZe5G=Ecg8O971_8rX3W4MN1m@mail.gmail.com>
	<4CFAC7B0.3080307@v.loewis.de>
	<AANLkTikBQ0KpiMCtFM4dTzG4r8Y9=eyE4EmWC57c-Fzf@mail.gmail.com>
	<AANLkTinXctFV081nS-SS3zS_FRW4Z2AcTWGn-KQAS-4q@mail.gmail.com>
	<AANLkTinm3wi31g+Y5SjneOp8HGSLX2wLxjbaW2-aWEUA@mail.gmail.com>
Message-ID: <AANLkTiknOBeqZqhWavoN=xPD78LfXNpr-zoLw5AoYe8g@mail.gmail.com>

2010/12/7 Alexander Belopolsky <alexander.belopolsky at gmail.com>:
> On Tue, Dec 7, 2010 at 8:02 AM, Vlastimil Brom <vlastimil.brom at gmail.com> wrote:
> ..
>> Do you know of any re engine fully complying to to tr18, even at the
>> first level: "Basic Unicode Support"?
>>
> """
> ICU Regular Expressions conform to Unicode Technical Standard #18 ,
> Unicode Regular Expressions, level 1, and in addition include Default
> Word boundaries and Name Properties from level 2.
> """ http://userguide.icu-project.org/strings/regexp
>

Thanks for the pointer, I wasn't aware of that project.
Anyway I am quite happy with the mentioned regex library and can live
with trading this full compliance for some non-unicode goodies (like
unbounded lookbehinds etc.), but I see, it's beyond the point here.
Not that my opinion matters, but I can't think of, say, "union,
intersection and set-difference of Unicode sets" as a basic feature or
consider it a part of "a minimal level for useful Unicode support."

vbr

From lukasz at langa.pl  Tue Dec  7 23:33:24 2010
From: lukasz at langa.pl (=?utf-8?Q?=C5=81ukasz_Langa?=)
Date: Tue, 7 Dec 2010 23:33:24 +0100
Subject: [Python-Dev] Using logging in the stdlib and its unit tests
In-Reply-To: <20101207221934.725be79d@pitrou.net>
References: <loom.20101207T212306-892@post.gmane.org>
	<20101207215053.025ecf52@pitrou.net>
	<8E88AE41-04D5-4B5A-9ECB-1CC6A137B3C0@langa.pl>
	<20101207221934.725be79d@pitrou.net>
Message-ID: <9CF9C003-B62D-444E-B988-6ECEB6229F93@langa.pl>

Wiadomo?? napisana przez Antoine Pitrou w dniu 2010-12-07, o godz. 22:19:

>> If you're writing an application then the "No handlers could be found" message is actually useful because there's hardly any reason no to include one.
> 
> Why do you say that? Not having to add a handler is certainly useful
> when you are doing some quick prototyping or simply writing a script
> (situations in which you still want to get error messages displayed
> properly by the libraries).
> 
>> One way or the other, we should really default to the convenience of application developers. This is currently the case.
> 
> Why wouldn't there be a default convenience of printing out errors?
> It's already the case for the root handler, so why would other handler
> be treated differently?
> 
>>>> import logging
>>>> logging.debug("foo")
>>>> logging.error("bar")
> ERROR:root:bar

If you're arguing that instead of writing "No handler", our logging library could just as easily default to a simplistic stderr handler for errors, then I agree. Both the convenience and consistency arguments you provided are convincing. See, that was 3 times a con* in one sentence!

Then again, warning an application developer that some handler is not configured that probably should be, is still valuable IMHO. Unless explicitly silenced.

-- 
Best regards,
?ukasz Langa
tel. +48 791 080 144
WWW http://lukasz.langa.pl/

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20101207/11c7e172/attachment.html>

From robert.kern at gmail.com  Tue Dec  7 23:59:36 2010
From: robert.kern at gmail.com (Robert Kern)
Date: Tue, 07 Dec 2010 16:59:36 -0600
Subject: [Python-Dev] Using logging in the stdlib and its unit tests
In-Reply-To: <loom.20101207T212306-892@post.gmane.org>
References: <loom.20101207T212306-892@post.gmane.org>
Message-ID: <idme8q$9bm$1@dough.gmane.org>

On 12/7/10 2:26 PM, Vinay Sajip wrote:
> This issue was brought to my notice today:
>
> http://bugs.python.org/issue10626
>
> and reference was made in the comments to possible obstacles facing stdlib
> maintainers who might wish to use logging in the stdlib and in its unit tests.
>
>> From my perspective and as mentioned in the logging documentation, library code
> which uses logging should add a NullHandler instance to any top-level logger,
> which will avoid any "No handlers could be found for logger XXX" message if no
> logging handlers have been set up.

I've done that before in my own library code, then quickly realized that it was 
a bad idea. Adding a NullHandler silently prevents logging.basicConfig() from 
working. Embedding that kind of restriction into library code (usually by a mere 
import!) made users unhappy and added an additional burden on the library 
developers who had to make sure to do that everywhere they used logging.

If I had my druthers, I would simply remove the "No handlers could be found for 
logger XXX" message. If we always wrote entire applications from the ground up, 
it makes a lot of sense. The person that writes the code that issues logs is the 
same person that writes the code to configure logging for reading. If you add 
logging in one place, you almost certainly want to use it somewhere else and not 
setting up logging is probably an error. But when you want to write reusable 
libraries, those roles become separated.

As a library author, I would dearly love to just add logging liberally without 
placing any additional burden to the users of my library. If my users wants to 
read those logs, he will configure logging. If he doesn't, he won't. With the 
current behavior, I can't do that. If I add logging, he has to add code just to 
silence a message that is meaningless to him (after I get the support emails 
asking if things are broken and explain how to silence it). If I add a 
NullHandler, I remove the ability for him to use logging.basicConfig(), the 
easiest and most straightforward way for him to add logging to his application.

This is only my personal anecdotal experience, but the current behavior has 
always wasted my time and never saved any of my time.

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
  that is made terrible by our own mad attempt to interpret it as though it had
  an underlying truth."
   -- Umberto Eco


From brett at python.org  Wed Dec  8 00:03:47 2010
From: brett at python.org (Brett Cannon)
Date: Tue, 7 Dec 2010 15:03:47 -0800
Subject: [Python-Dev] Using logging in the stdlib and its unit tests
In-Reply-To: <4CFEADC0.50609@trueblade.com>
References: <loom.20101207T212306-892@post.gmane.org>
	<4CFEADC0.50609@trueblade.com>
Message-ID: <AANLkTinjhdvvBBs=Sv+tqQZp6DSos45Ya-Vwui-VZa2n@mail.gmail.com>

On Tue, Dec 7, 2010 at 13:57, Eric Smith <eric at trueblade.com> wrote:
> On 12/07/2010 03:26 PM, Vinay Sajip wrote:
>>
>> I would suggest that when unit testing, rather than adding StreamHandlers
>> to log
>> to stderr, that something like TestHandler and Matcher from this post:
>>
>> http://plumberjack.blogspot.com/2010/09/unit-testing-and-logging.html
>>
>> This will allow assertion checking of logged messages without resorting to
>> StringIO, getvalue() etc. If people think it's a good idea, I can add the
>> TestHandler/Matcher classes to the unit test infrastructure (they wouldn't
>> become part of the public API, at least until 3.3, but I presume they
>> could be
>> used in the stdlib unit tests).
>
> Wouldn't it make more sense to add these to test.support? I don't think we
> make any guarantees about its API being stable, although I have a faint
> recollection of that being debated in the past.

We make no guarantees. The debate was whether we accidentally painted
ourselves into a corner by documenting the module a tad for core
developer's benefit. The warning has been made much more explicit in
the docs that test.support is not meant for outside usage.

From robert.kern at gmail.com  Wed Dec  8 00:21:44 2010
From: robert.kern at gmail.com (Robert Kern)
Date: Tue, 07 Dec 2010 17:21:44 -0600
Subject: [Python-Dev] Using logging in the stdlib and its unit tests
In-Reply-To: <idme8q$9bm$1@dough.gmane.org>
References: <loom.20101207T212306-892@post.gmane.org>
	<idme8q$9bm$1@dough.gmane.org>
Message-ID: <idmfi8$dts$1@dough.gmane.org>

On 12/7/10 4:59 PM, Robert Kern wrote:
> On 12/7/10 2:26 PM, Vinay Sajip wrote:
>> This issue was brought to my notice today:
>>
>> http://bugs.python.org/issue10626
>>
>> and reference was made in the comments to possible obstacles facing stdlib
>> maintainers who might wish to use logging in the stdlib and in its unit tests.
>>
>>> From my perspective and as mentioned in the logging documentation, library code
>> which uses logging should add a NullHandler instance to any top-level logger,
>> which will avoid any "No handlers could be found for logger XXX" message if no
>> logging handlers have been set up.
>
> I've done that before in my own library code, then quickly realized that it was
> a bad idea. Adding a NullHandler silently prevents logging.basicConfig() from
> working.

Only on the root handler. Forget this bit of my argument (and the statements 
that directly follow from it). The rest of my argument should stand on its own, 
though.

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
  that is made terrible by our own mad attempt to interpret it as though it had
  an underlying truth."
   -- Umberto Eco


From barry at python.org  Wed Dec  8 00:38:55 2010
From: barry at python.org (Barry Warsaw)
Date: Tue, 7 Dec 2010 18:38:55 -0500
Subject: [Python-Dev] Using logging in the stdlib and its unit tests
In-Reply-To: <idme8q$9bm$1@dough.gmane.org>
References: <loom.20101207T212306-892@post.gmane.org>
	<idme8q$9bm$1@dough.gmane.org>
Message-ID: <20101207183855.52ad8926@mission>

On Dec 07, 2010, at 04:59 PM, Robert Kern wrote:

>As a library author, I would dearly love to just add logging liberally
>without placing any additional burden to the users of my library. If my users
>wants to read those logs, he will configure logging. If he doesn't, he
>won't. With the current behavior, I can't do that. If I add logging, he has
>to add code just to silence a message that is meaningless to him (after I get
>the support emails asking if things are broken and explain how to silence
>it). If I add a NullHandler, I remove the ability for him to use
>logging.basicConfig(), the easiest and most straightforward way for him to
>add logging to his application.

+1

-Barry
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 836 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/python-dev/attachments/20101207/82167d30/attachment-0001.pgp>

From vinay_sajip at yahoo.co.uk  Wed Dec  8 00:45:39 2010
From: vinay_sajip at yahoo.co.uk (Vinay Sajip)
Date: Tue, 7 Dec 2010 23:45:39 +0000 (UTC)
Subject: [Python-Dev] Using logging in the stdlib and its unit tests
References: <loom.20101207T212306-892@post.gmane.org>
	<20101207215053.025ecf52@pitrou.net>
Message-ID: <loom.20101208T002100-91@post.gmane.org>

Antoine Pitrou <solipsis <at> pitrou.net> writes:

> 
> I thought "error" and "critical" messages were logged to stderr by
> default? Isn't it the case?
> 

Only if you call basicConfig() or use the logging.debug(), logging.info(), etc.
module-level convenience functions (which call basicConfig under the hood).

> If any library defining a logger must also add a NullHandler just in
> case, ISTM that complicates a lot the use of logging (and could explain
> its impopularity). Both for library writers and application writers,
> apparently (since the former will have to add the NullHandler, and the
> latter will have to override it so that error messages get printed out
> rather than lost).
> 

No - the application developer doesn't have to do anything special. Adding the
NullHandler is only needed for the top-level logger of a library package, so
it's hardly more onerous than doing an import. It doesn't complicate anything,
as far as I can see. It's to be done once per library, and trivial in comparison
with all the work needed to develop, test, and package a library. For the
stdlib, for example, if we were to standardize and say that all stdlib loggers
were to be located below the "py" namespace, then I could put the relevant
NullHandler-adding code in the logging package and no other stdlib maintainer
would have to do anything other than name their stdlib loggers beginning with
"py." (e.g. "py.concurrent.futures") for things to work as expected. If you
don't like "py", then pick "std", "stdlib" or whatever.

When is the NullHandler needed? Only for cases where an application developer
uses a library which does logging under the covers (for those users who might be
interested in logging its operations), but where that application developer
doesn't use logging themselves for that application. In such a case, a library
call which generated a logging event would cause logging to look for handlers,
and if one is not found, print that one-off "no handlers could be found for
logger XXX" message. This message is there because errors should not pass
silently; but in the case of an application developer oblivious of logging who
uses a library which uses logging under the hood, you can't argue that that
developer has made any error. So the library developer, by adding a NullHandler,
can improve things for their users by not forcing them to configure logging
handlers (in order to avoid the message).

#mylib.py
import logging

logger = logging.getLogger(__name__)
logger.addHandler(logging.NullHandler())

def func():
    logger.warning('Something happened!')

#myapp.py
import mylib

mylib.func()

If you run myapp.py, no message is printed. If you comment out the "addHandler"
line and re-run, then you see

No handlers could be found for logger "mylib"

which is the misconfiguration message. With the line commented back in, you as
an application developer get expected behaviour (no spurious messages about
missing handlers). When you decide to use logging in myapp.py, simply insert 

import logging

logging.basicConfig()

before calling mylib.func(), and you get

WARNING:mylib:Something happened!

which is what you would expect.

Regards,

Vinay Sajip



From vinay_sajip at yahoo.co.uk  Wed Dec  8 00:58:54 2010
From: vinay_sajip at yahoo.co.uk (Vinay Sajip)
Date: Tue, 7 Dec 2010 23:58:54 +0000 (UTC)
Subject: [Python-Dev] Using logging in the stdlib and its unit tests
References: <loom.20101207T212306-892@post.gmane.org>
	<idme8q$9bm$1@dough.gmane.org>
Message-ID: <loom.20101208T004711-264@post.gmane.org>

Robert Kern <robert.kern <at> gmail.com> writes:


> If I had my druthers, I would simply remove the "No handlers could be
> found for logger XXX" message. If we always wrote entire applications
> from the ground up, it makes a lot of sense. The person that writes the
> code that issues logs is the same person that writes the code to configure
> logging for reading. If you add logging in one place, you almost certainly
> want to use it somewhere else and not setting up logging is probably an
> error. But when you want to write reusable  libraries, those roles become
> separated.

Exactly - we do use third-party libraries. When logging was added there was some
debate about whether this one-off message should be printed, and in balance it
was thought better to print this, not least because people would be unfamiliar
with logging (as it was then new) and so be not unlikely to misconfigure it. No
indication of this at all would be double-plus-ungood.

> As a library author, I would dearly love to just add logging liberally
> without placing any additional burden to the users of my library.
> If my users wants to read those logs, he will configure logging. If he
> doesn't, he won't. With the current behavior, I can't do that. If I add
> logging, he has to add code just to silence a message that is meaningless
> to him (after I get the support emails asking if things are broken and
> explain how to silence it). If I add a NullHandler, I remove the ability
> for him to use logging.basicConfig(), the easiest and most straightforward
> way for him to add logging to his application.

You don't remove the ability for him to use basicConfig() - that's another
mistake in your post (you already corrected the other mistake in your
self-response). See my example with mylib.py/myapp.py in another post on this
thread, in response to Antoine.

> This is only my personal anecdotal experience, but the current behavior has 
> always wasted my time and never saved any of my time.
> 

That's because I think you're doing it wrong, or else I've misunderstood your
use case. NullHandler is the way to go, and doesn't preclude the use of
basicConfig() UNLESS you choose to add the NullHandler to the root logger (not
what you're supposed to do - you're supposed to add it to the top-level logger
*of your library*, not the top-level logger of the logging hierarchy.)

See the mylib/myapp example and tell me which it is - your mistake or mine?

Regards,

Vinay Sajip



From vinay_sajip at yahoo.co.uk  Wed Dec  8 01:01:25 2010
From: vinay_sajip at yahoo.co.uk (Vinay Sajip)
Date: Wed, 8 Dec 2010 00:01:25 +0000 (UTC)
Subject: [Python-Dev] Using logging in the stdlib and its unit tests
References: <loom.20101207T212306-892@post.gmane.org>
	<idme8q$9bm$1@dough.gmane.org> <20101207183855.52ad8926@mission>
Message-ID: <loom.20101208T005917-98@post.gmane.org>

Barry Warsaw <barry <at> python.org> writes:

> 
> On Dec 07, 2010, at 04:59 PM, Robert Kern wrote:
> 
> >As a library author, I would dearly love to just add logging liberally
> >without placing any additional burden to the users of my library. If my users
> >wants to read those logs, he will configure logging. If he doesn't, he
> >won't. With the current behavior, I can't do that. If I add logging, he has
> >to add code just to silence a message that is meaningless to him (after I get
> >the support emails asking if things are broken and explain how to silence
> >it). If I add a NullHandler, I remove the ability for him to use
> >logging.basicConfig(), the easiest and most straightforward way for him to
> >add logging to his application.
> 
> +1
> 

Barry, if you mean +1 as in "I agree this is how it should work", then we're all
agreed. But I think Robert is wrong that NullHandler precludes use of
basicConfig - when NullHandler is added to a library logger (rather than the
root logger), basicConfig() works as expected. See the mylib/myapp example in my
other post on this thread.

Regards,

Vinay Sajip




From python at mrabarnett.plus.com  Wed Dec  8 01:07:56 2010
From: python at mrabarnett.plus.com (MRAB)
Date: Wed, 08 Dec 2010 00:07:56 +0000
Subject: [Python-Dev] Is PyPI down?
Message-ID: <4CFECC5C.8040001@mrabarnett.plus.com>

I can't get http://pypi.python.org and I've double-checked using
http://downforeveryoneorjustme.com/.

From solipsis at pitrou.net  Wed Dec  8 01:09:13 2010
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Wed, 8 Dec 2010 01:09:13 +0100
Subject: [Python-Dev] Using logging in the stdlib and its unit tests
References: <loom.20101207T212306-892@post.gmane.org>
	<20101207215053.025ecf52@pitrou.net>
	<loom.20101208T002100-91@post.gmane.org>
Message-ID: <20101208010913.363c6105@pitrou.net>

On Tue, 7 Dec 2010 23:45:39 +0000 (UTC)
Vinay Sajip <vinay_sajip at yahoo.co.uk> wrote:
> Antoine Pitrou <solipsis <at> pitrou.net> writes:
> 
> > 
> > I thought "error" and "critical" messages were logged to stderr by
> > default? Isn't it the case?
> > 
> 
> Only if you call basicConfig() or use the logging.debug(), logging.info(), etc.
> module-level convenience functions (which call basicConfig under the hood).

Why wouldn't it be the default for all logging calls ? Such special
cases don't really make things easy to remember.

> When is the NullHandler needed? Only for cases where an application developer
> uses a library which does logging under the covers (for those users who might be
> interested in logging its operations), but where that application developer
> doesn't use logging themselves for that application.

You seem pretty tied up to the "application developer" situation. There
are cases (scripts, prototyping, etc.) where you certainly want to see
error messages (errors should not pass silently) but don't want to
configure logging for each of the libraries you use.

Having convenient and understandable defaults would go a long way
towards making logging more usable, IMO.

Regards

Antoine.



From vinay_sajip at yahoo.co.uk  Wed Dec  8 01:09:28 2010
From: vinay_sajip at yahoo.co.uk (Vinay Sajip)
Date: Wed, 8 Dec 2010 00:09:28 +0000 (UTC)
Subject: [Python-Dev] Using logging in the stdlib and its unit tests
References: <loom.20101207T212306-892@post.gmane.org>
	<4CFEADC0.50609@trueblade.com>
Message-ID: <loom.20101208T010818-58@post.gmane.org>

Eric Smith <eric <at> trueblade.com> writes:

 
> Wouldn't it make more sense to add these to test.support? I don't think 
> we make any guarantees about its API being stable, although I have a 
> faint recollection of that being debated in the past.
> 

That's what I mean when I said "unit test infrastructure" - adding to
test.support. Sorry, I should have been more explicit.

Regards,

Vinay Sajip



From richard at python.org  Wed Dec  8 01:50:17 2010
From: richard at python.org (Richard Jones)
Date: Wed, 8 Dec 2010 11:50:17 +1100
Subject: [Python-Dev] Is PyPI down?
In-Reply-To: <4CFECC5C.8040001@mrabarnett.plus.com>
References: <4CFECC5C.8040001@mrabarnett.plus.com>
Message-ID: <AANLkTi=B10EuLP3_Vbzq1dBN=-DNeT5kTMkDYA_QfQNH@mail.gmail.com>

The monitor said it went away at 23:50 UTC and came back 22 minutes
later at 00:12 UTC.


     Richard

On Wed, Dec 8, 2010 at 11:07 AM, MRAB <python at mrabarnett.plus.com> wrote:
> I can't get http://pypi.python.org and I've double-checked using
> http://downforeveryoneorjustme.com/.
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> http://mail.python.org/mailman/options/python-dev/richard%40python.org
>

From eric at trueblade.com  Wed Dec  8 01:57:58 2010
From: eric at trueblade.com (Eric Smith)
Date: Tue, 07 Dec 2010 19:57:58 -0500
Subject: [Python-Dev] Using logging in the stdlib and its unit tests
In-Reply-To: <loom.20101208T010818-58@post.gmane.org>
References: <loom.20101207T212306-892@post.gmane.org>	<4CFEADC0.50609@trueblade.com>
	<loom.20101208T010818-58@post.gmane.org>
Message-ID: <4CFED816.1000204@trueblade.com>

On 12/07/2010 07:09 PM, Vinay Sajip wrote:
> Eric Smith<eric<at>  trueblade.com>  writes:
>
>
>> Wouldn't it make more sense to add these to test.support? I don't think
>> we make any guarantees about its API being stable, although I have a
>> faint recollection of that being debated in the past.
>>
>
> That's what I mean when I said "unit test infrastructure" - adding to
> test.support. Sorry, I should have been more explicit.

My bad. I read it as "unittest infrastructure".

Eric.


From ncoghlan at gmail.com  Wed Dec  8 02:00:50 2010
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Wed, 8 Dec 2010 11:00:50 +1000
Subject: [Python-Dev] Using logging in the stdlib and its unit tests
In-Reply-To: <20101208010913.363c6105@pitrou.net>
References: <loom.20101207T212306-892@post.gmane.org>
	<20101207215053.025ecf52@pitrou.net>
	<loom.20101208T002100-91@post.gmane.org>
	<20101208010913.363c6105@pitrou.net>
Message-ID: <AANLkTim4ZPoch18+-qkunuYm-zcyh-vbhMOv9YVVEqo9@mail.gmail.com>

On Wed, Dec 8, 2010 at 10:09 AM, Antoine Pitrou <solipsis at pitrou.net> wrote:
> On Tue, 7 Dec 2010 23:45:39 +0000 (UTC)
> Vinay Sajip <vinay_sajip at yahoo.co.uk> wrote:
>> Antoine Pitrou <solipsis <at> pitrou.net> writes:
>>
>> >
>> > I thought "error" and "critical" messages were logged to stderr by
>> > default? Isn't it the case?
>> >
>>
>> Only if you call basicConfig() or use the logging.debug(), logging.info(), etc.
>> module-level convenience functions (which call basicConfig under the hood).
>
> Why wouldn't it be the default for all logging calls ? Such special
> cases don't really make things easy to remember.

Indeed - I was very surprised to find just now that calling
"logging.warn('Whatever')" is not the same as doing "log =
logging.getLogger(); log.warn('Whatever')".

Adding a NullHandler isn't the right thing to do - the behaviour I
would want for any standard library logging that hasn't been
explicitly configured otherwise is to do what the root logger does
under basicConfig(): debug() and info() get suppressed, warn(),
error(), critical() and exception() go to stderr. It seems to me like
that would be more useful default behaviour in general than emitting a
warning about not finding a handler.

So my suggestion would be:
1. In the absence of a "config" call, make the "no handler" path in
loggers emit messages *as if* basicConfig() has been called (without
actually calling it)
2. Remove the implicit calls to basicConfig() from the module level
convenience function

How *feasible* that idea is to implement, I don't know.

Cheers,
Nick.
-- 
Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia

From vinay_sajip at yahoo.co.uk  Wed Dec  8 02:19:45 2010
From: vinay_sajip at yahoo.co.uk (Vinay Sajip)
Date: Wed, 8 Dec 2010 01:19:45 +0000 (UTC)
Subject: [Python-Dev] Using logging in the stdlib and its unit tests
References: <loom.20101207T212306-892@post.gmane.org>
	<20101207215053.025ecf52@pitrou.net>
	<loom.20101208T002100-91@post.gmane.org>
	<20101208010913.363c6105@pitrou.net>
Message-ID: <loom.20101208T020511-443@post.gmane.org>

Antoine Pitrou <solipsis <at> pitrou.net> writes:

> 
> Why wouldn't it be the default for all logging calls ? Such special
> cases don't really make things easy to remember.
>

One size doesn't fit all. Everything's documented reasonably well. If you use it
often, you remember it. If you use it seldom, you might need to look things up.
 

> You seem pretty tied up to the "application developer" situation. There
> are cases (scripts, prototyping, etc.) where you certainly want to see
> error messages (errors should not pass silently) but don't want to
> configure logging for each of the libraries you use.

You don't have to configure each of the libraries you use. The "application
developer" = "The developer who is not writing a library" = "The person writing
a script, prototyping, etc." I don't mean Application with a capital A. I'm just
using it as a convenient label to mean a library user who is not writing a
library themselves.

 
> Having convenient and understandable defaults would go a long way
> towards making logging more usable, IMO.
> 

The defaults are I believe reasonably convenient, or at least not inconvenient:

1. If writing a library, add a NullHandler to the top-level logger of your
library, don't add any other handlers by default (i.e. on import), document the
names of the loggers you use. Otherwise, see the following.

2. If writing a simple script, prototyping etc. i.e. for casual use, use
logging.debug(), logging.info() etc. That's easy enough and convenient enough
for casual users to remember, I hope. All WARNING, ERROR and CRITICAL events in
your code and in that of any libraries you use will be printed to sys.stderr.

3. If slightly less than very casual use (e.g. you want to log to file, or want
a specific message format), call basicConfig(). Just the one extra line to
remember to do, hopefully not too inconvenient. All WARNING, ERROR and CRITICAL
events in your code and in that of any libraries you use will be printed to
sys.stderr, unless you specify a file to log to and/or a different logging
level, in which case what you specified will be used as destination/threshold
for logging messages.

4. For more advanced use, use programmatic configuration, fileConfig or
dictConfig to specify a configuration. Convenience is inversely proportional to
how involved the configuration is.

If you can specify what you think is inconvenient in the above, that would be
helpful.

Regards,

Vinay Sajip


From barry at python.org  Wed Dec  8 02:20:21 2010
From: barry at python.org (Barry Warsaw)
Date: Tue, 7 Dec 2010 20:20:21 -0500
Subject: [Python-Dev] Using logging in the stdlib and its unit tests
In-Reply-To: <loom.20101208T005917-98@post.gmane.org>
References: <loom.20101207T212306-892@post.gmane.org>
	<idme8q$9bm$1@dough.gmane.org> <20101207183855.52ad8926@mission>
	<loom.20101208T005917-98@post.gmane.org>
Message-ID: <20101207202021.794d25bd@snowdog>

On Dec 08, 2010, at 12:01 AM, Vinay Sajip wrote:

>Barry, if you mean +1 as in "I agree this is how it should work", then
>we're all agreed.

Yep, that's what I meant!

-Barry
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 836 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/python-dev/attachments/20101207/5f8bac48/attachment.pgp>

From vinay_sajip at yahoo.co.uk  Wed Dec  8 02:51:44 2010
From: vinay_sajip at yahoo.co.uk (Vinay Sajip)
Date: Wed, 8 Dec 2010 01:51:44 +0000 (UTC)
Subject: [Python-Dev] Using logging in the stdlib and its unit tests
References: <loom.20101207T212306-892@post.gmane.org>
	<20101207215053.025ecf52@pitrou.net>
	<loom.20101208T002100-91@post.gmane.org>
	<20101208010913.363c6105@pitrou.net>
	<AANLkTim4ZPoch18+-qkunuYm-zcyh-vbhMOv9YVVEqo9@mail.gmail.com>
Message-ID: <loom.20101208T022102-46@post.gmane.org>

Nick Coghlan <ncoghlan <at> gmail.com> writes:

> Indeed - I was very surprised to find just now that calling
> "logging.warn('Whatever')" is not the same as doing "log =
> logging.getLogger(); log.warn('Whatever')".

Don't know why you'd be surprised: it's been that way since logging was added to
Python, and the logging.debug() etc. are documented as convenience methods for
casual use in throwaway scripts, interactive sessions etc. The convenience is in
that you don't need to specify a logger (the root logger is used) and that
basicConfig() is called for you.
 
> Adding a NullHandler isn't the right thing to do - the behaviour I
> would want for any standard library logging that hasn't been
> explicitly configured otherwise is to do what the root logger does
> under basicConfig(): debug() and info() get suppressed, warn(),
> error(), critical() and exception() go to stderr. It seems to me like
> that would be more useful default behaviour in general than emitting a
> warning about not finding a handler.
>
> So my suggestion would be:
> 1. In the absence of a "config" call, make the "no handler" path in
> loggers emit messages *as if* basicConfig() has been called (without
> actually calling it)
> 2. Remove the implicit calls to basicConfig() from the module level
> convenience function
> 
> How *feasible* that idea is to implement, I don't know.
> 

What about "Special cases aren't special enough to break the rules."? Why should
logging from stdlib modules be treated differently from the case of logging from
library modules in general? It could be argued that the behaviour you're
proposing is confusing/inconsistent to some people.

Regards,

Vinay Sajip


From robert.kern at gmail.com  Wed Dec  8 03:33:33 2010
From: robert.kern at gmail.com (Robert Kern)
Date: Tue, 07 Dec 2010 20:33:33 -0600
Subject: [Python-Dev] Using logging in the stdlib and its unit tests
In-Reply-To: <loom.20101208T004711-264@post.gmane.org>
References: <loom.20101207T212306-892@post.gmane.org>	<idme8q$9bm$1@dough.gmane.org>
	<loom.20101208T004711-264@post.gmane.org>
Message-ID: <idmqpt$rbq$1@dough.gmane.org>

On 2010-12-07 17:58 , Vinay Sajip wrote:
> Robert Kern<robert.kern<at>  gmail.com>  writes:
>
>
>> If I had my druthers, I would simply remove the "No handlers could be
>> found for logger XXX" message. If we always wrote entire applications
>> from the ground up, it makes a lot of sense. The person that writes the
>> code that issues logs is the same person that writes the code to configure
>> logging for reading. If you add logging in one place, you almost certainly
>> want to use it somewhere else and not setting up logging is probably an
>> error. But when you want to write reusable  libraries, those roles become
>> separated.
>
> Exactly - we do use third-party libraries. When logging was added there was some
> debate about whether this one-off message should be printed, and in balance it
> was thought better to print this, not least because people would be unfamiliar
> with logging (as it was then new) and so be not unlikely to misconfigure it. No
> indication of this at all would be double-plus-ungood.

I really don't understand how this view can be consistent with the practice of 
adding NullHandler to loggers. If this message is so important to prevent 
misconfiguration, then why should a library author decide to silence it for his 
users?

I think that the chances of a misconfiguration that the warning would catch. are 
small in practice. I don't have any solid survey data on how people configure 
their loggers, but I strongly suspect that almost all configurations include a 
catch-all root logger and that most of those *only* consist of that root logger.

>> As a library author, I would dearly love to just add logging liberally
>> without placing any additional burden to the users of my library.
>> If my users wants to read those logs, he will configure logging. If he
>> doesn't, he won't. With the current behavior, I can't do that. If I add
>> logging, he has to add code just to silence a message that is meaningless
>> to him (after I get the support emails asking if things are broken and
>> explain how to silence it). If I add a NullHandler, I remove the ability
>> for him to use logging.basicConfig(), the easiest and most straightforward
>> way for him to add logging to his application.
>
> You don't remove the ability for him to use basicConfig() - that's another
> mistake in your post (you already corrected the other mistake in your
> self-response). See my example with mylib.py/myapp.py in another post on this
> thread, in response to Antoine.

Same mistake. I intended the correction to apply to all such statements in my post.

>> This is only my personal anecdotal experience, but the current behavior has
>> always wasted my time and never saved any of my time.
>
> That's because I think you're doing it wrong, or else I've misunderstood your
> use case.

I will try to state it more plainly: the message has always been a false 
positive in my experience. It has never helped me fix a real problem and has 
otherwise wasted my time.

> NullHandler is the way to go, and doesn't preclude the use of
> basicConfig() UNLESS you choose to add the NullHandler to the root logger (not
> what you're supposed to do - you're supposed to add it to the top-level logger
> *of your library*, not the top-level logger of the logging hierarchy.)
>
> See the mylib/myapp example and tell me which it is - your mistake or mine?

I think that boilerplate should be minimized. If using getLogger() should almost 
always be followed by adding a NullHandler, then it should be the default 
behavior. The easiest way to achieve this effect is to simply not issue the 
warning message.

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
  that is made terrible by our own mad attempt to interpret it as though it had
  an underlying truth."
   -- Umberto Eco


From ocean-city at m2.ccsnet.ne.jp  Wed Dec  8 07:12:32 2010
From: ocean-city at m2.ccsnet.ne.jp (Hirokazu Yamamoto)
Date: Wed, 08 Dec 2010 15:12:32 +0900
Subject: [Python-Dev] kill_python on windows buildbots
In-Reply-To: <m2fwu9lirl.fsf@valheru.db3l.homeip.net>
References: <20101113133712.60e9be27@pitrou.net>	<m2lj4wpqzt.fsf@valheru.db3l.homeip.net>	<AANLkTimRZ_h9kpbQrXiD2rgP77EqUAHNPEWMrvAyoLSC@mail.gmail.com>	<m2hbfkp9xm.fsf@valheru.db3l.homeip.net>	<4CEB7E12.1070201@snakebite.org>	<m2zkszochi.fsf@valheru.db3l.homeip.net>	<m2mxoil9a0.fsf_-_@valheru.db3l.homeip.net>	<idlgt6$8vq$2@dough.gmane.org>
	<m2fwu9lirl.fsf@valheru.db3l.homeip.net>
Message-ID: <4CFF21D0.7050705@m2.ccsnet.ne.jp>

On 2010/12/08 0:11, David Bolen wrote:
> In thinking about it some more, I suppose there's still a small window
> for a loss of communication during a test which results in clean not
> being run (which could then block the next svn checkout without an
> opportunity to kill the processes), so maybe the right place is
> actually at the end of the test batch file which is the step during
> which such hung processes might get generated?  I don't know the
> history, if any, of it's current location in the flow.

Yes, but test can freeze. In that case, I'm worried that
   (snip)
   rt.bat .... # freeze here (will be halt by buildbot)
   vcbuild .... & kill_python_d # Will this be called?
in test.bat.

From ncoghlan at gmail.com  Wed Dec  8 08:28:34 2010
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Wed, 8 Dec 2010 17:28:34 +1000
Subject: [Python-Dev] Using logging in the stdlib and its unit tests
In-Reply-To: <loom.20101208T022102-46@post.gmane.org>
References: <loom.20101207T212306-892@post.gmane.org>
	<20101207215053.025ecf52@pitrou.net>
	<loom.20101208T002100-91@post.gmane.org>
	<20101208010913.363c6105@pitrou.net>
	<AANLkTim4ZPoch18+-qkunuYm-zcyh-vbhMOv9YVVEqo9@mail.gmail.com>
	<loom.20101208T022102-46@post.gmane.org>
Message-ID: <AANLkTi=fQPKUs9-oAuGM3Gzn8uwi3tnMBMTy1WUFwX_p@mail.gmail.com>

On Wed, Dec 8, 2010 at 11:51 AM, Vinay Sajip <vinay_sajip at yahoo.co.uk> wrote:
> Nick Coghlan <ncoghlan <at> gmail.com> writes:
>
>> Indeed - I was very surprised to find just now that calling
>> "logging.warn('Whatever')" is not the same as doing "log =
>> logging.getLogger(); log.warn('Whatever')".
>
> Don't know why you'd be surprised: it's been that way since logging was added to
> Python, and the logging.debug() etc. are documented as convenience methods for
> casual use in throwaway scripts, interactive sessions etc. The convenience is in
> that you don't need to specify a logger (the root logger is used) and that
> basicConfig() is called for you.

I've previously only used logging as an application author, hence I
hadn't noticed some of the subtleties that come up when using logging
in library code where you don't control which handlers are going to be
installed at runtime.

The surprise came from not realising there was a basicConfig() call
hidden inside the convenience APIs, a fact which is *not* mentioned in
the docstrings. It may be mentioned in the main documentation, but I
didn't look at that at the time - there was nothing to suggest the
docstrings were lacking pertinent information:

>>> print(logging.warn.__doc__)

    Log a message with severity 'WARNING' on the root logger.

>>> print(logging.getLogger().warn.__doc__)

        Log 'msg % args' with severity 'WARNING'.

        To pass exception information, use the keyword argument exc_info with
        a true value, e.g.

        logger.warning("Houston, we have a %s", "bit of a problem", exc_info=1)

>> So my suggestion would be:
>> 1. In the absence of a "config" call, make the "no handler" path in
>> loggers emit messages *as if* basicConfig() has been called (without
>> actually calling it)
>> 2. Remove the implicit calls to basicConfig() from the module level
>> convenience function
>>
>> How *feasible* that idea is to implement, I don't know.
>>
>
> What about "Special cases aren't special enough to break the rules."? Why should
> logging from stdlib modules be treated differently from the case of logging from
> library modules in general? It could be argued that the behaviour you're
> proposing is confusing/inconsistent to some people.

I'm not proposing that the standard library be special-cased, I'm
proposing that the default behaviour of an unconfigured logging module
in general be changed to something more useful (i.e. warnings and
errors printed to stderr, everything else suppressed), rather than
unhelpfully suppressing *all* output except for an "Oh, I'm throwing
output away" message the first time it happens.

The specific use case that triggered my interest in this default
behaviour is one where concurrent.futures *can't* raise an exception
because it knows nothing is going to be in a position to handle that
exception (due to the way callbacks are handled, c.f knows it provides
the outermost exception handler in the affected call stack). Instead
it has to catch the error and display it *somewhere* (i.e. it's very
similar to the use cases for PyErr_WriteUnraisable at the C level).

The options for handling this are:

1. Write the error detail directly to stderr. (Unless the default
behaviour of logging changes, that is what I am going to suggest Brian
do, as it exactly mimics the behaviour of the PyErr_WriteUnraisable
API).
2. Write it to the root logger with the convenience APIs (Possible
option, but I don't like the global state impact of implicitly calling
basicConfig() from library code)
3. Add a StdErr handler for the logger (this is what is currently
implemented, and again, I don't like it because of the global state
impact on something that should be completely under an application's
control)

Basically, the current behaviour of logging is such that libraries
*cannot* use it for unraisable warnings and error messages, as the
messages will be suppressed unless the application takes steps to see
them. That is OK for debug and info messages, but unacceptable for
warnings and errors. A throwaway script using concurrent.futures needs
to know if callbacks are failing, and that needs to happen without any
logging related boilerplate in the script itself.

If, however, an application completely lacking even a call to
logging.basicConfig() would still see warnings and errors, then
libraries could safely use the module without needing to worry about
applications needing an particular boilerplate in order to see the
unraisable errors and warnings that are emitted.

Regards,
Nick.

-- 
Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia

From g.brandl at gmx.net  Wed Dec  8 09:09:45 2010
From: g.brandl at gmx.net (Georg Brandl)
Date: Wed, 08 Dec 2010 09:09:45 +0100
Subject: [Python-Dev] Using logging in the stdlib and its unit tests
In-Reply-To: <20101208010913.363c6105@pitrou.net>
References: <loom.20101207T212306-892@post.gmane.org>	<20101207215053.025ecf52@pitrou.net>	<loom.20101208T002100-91@post.gmane.org>
	<20101208010913.363c6105@pitrou.net>
Message-ID: <idneln$t7i$1@dough.gmane.org>

Am 08.12.2010 01:09, schrieb Antoine Pitrou:
> On Tue, 7 Dec 2010 23:45:39 +0000 (UTC)
> Vinay Sajip <vinay_sajip at yahoo.co.uk> wrote:
>> Antoine Pitrou <solipsis <at> pitrou.net> writes:
>> 
>> > 
>> > I thought "error" and "critical" messages were logged to stderr by
>> > default? Isn't it the case?
>> > 
>> 
>> Only if you call basicConfig() or use the logging.debug(), logging.info(), etc.
>> module-level convenience functions (which call basicConfig under the hood).
> 
> Why wouldn't it be the default for all logging calls ? Such special
> cases don't really make things easy to remember.
> 
>> When is the NullHandler needed? Only for cases where an application developer
>> uses a library which does logging under the covers (for those users who might be
>> interested in logging its operations), but where that application developer
>> doesn't use logging themselves for that application.
> 
> You seem pretty tied up to the "application developer" situation. There
> are cases (scripts, prototyping, etc.) where you certainly want to see
> error messages (errors should not pass silently) but don't want to
> configure logging for each of the libraries you use.

But errors don't pass silently, do they?  The usual way to present errors
is still by raising exceptions.

Georg


From g.brandl at gmx.net  Wed Dec  8 09:13:49 2010
From: g.brandl at gmx.net (Georg Brandl)
Date: Wed, 08 Dec 2010 09:13:49 +0100
Subject: [Python-Dev] [RELEASED] Python 3.2 beta 1
In-Reply-To: <433FCEE2-CD6C-48BD-90DF-2990B27E4EED@langa.pl>
References: <4CFD59C8.4070006@python.org>
	<433FCEE2-CD6C-48BD-90DF-2990B27E4EED@langa.pl>
Message-ID: <idnetb$vaj$1@dough.gmane.org>

Am 07.12.2010 09:24, schrieb ?ukasz Langa:
> Wiadomo?? napisana przez Georg Brandl w dniu 2010-12-06, o godz. 22:46:
> 
>> -----BEGIN PGP SIGNED MESSAGE-----
>> Hash: SHA1
>>
>> On behalf of the Python development team, I'm happy to announce the
>> first of two beta preview releases of Python 3.2.
>>
>> Python 3.2 is a continuation of the efforts to improve and stabilize the
>> Python 3.x line.  Since the final release of Python 2.7, the 2.x line
>> will only receive bugfixes, and new features are developed for 3.x only.
>>
>> Since PEP 3003, the Moratorium on Language Changes, is in effect, there
>> are no changes in Python's syntax and built-in types in Python 3.2.
>> Development efforts concentrated on the standard library and support for
>> porting code to Python 3.  Highlights are:
>>
>> [snip]
> 
> * configparser 1.1: new API using the mapping protocol access, support for
> pluggable interpolation handlers, additional interpolation handler
> (ExtendedInterpolation) which supports the zc.buildout syntax, support for
> alternative option/value delimiters, support for customization of accepted INI
> file structure (e.g. comment prefixes, name of the DEFAULT section, indentation,
> empty lines in multiline values, etc.), support for specifying encoding for read
> operations, ConfigParser class deprecated in favor of SafeConfigParser, lots of
> other small changes.

Thanks, I've added this and the ssl module for the next announcement.

Georg


From vinay_sajip at yahoo.co.uk  Wed Dec  8 09:32:58 2010
From: vinay_sajip at yahoo.co.uk (Vinay Sajip)
Date: Wed, 8 Dec 2010 08:32:58 +0000 (UTC)
Subject: [Python-Dev] Using logging in the stdlib and its unit tests
References: <loom.20101207T212306-892@post.gmane.org>
	<20101207215053.025ecf52@pitrou.net>
	<loom.20101208T002100-91@post.gmane.org>
	<20101208010913.363c6105@pitrou.net>
	<AANLkTim4ZPoch18+-qkunuYm-zcyh-vbhMOv9YVVEqo9@mail.gmail.com>
	<loom.20101208T022102-46@post.gmane.org>
	<AANLkTi=fQPKUs9-oAuGM3Gzn8uwi3tnMBMTy1WUFwX_p@mail.gmail.com>
Message-ID: <loom.20101208T085644-791@post.gmane.org>

Nick Coghlan <ncoghlan <at> gmail.com> writes:

> The surprise came from not realising there was a basicConfig() call
> hidden inside the convenience APIs, a fact which is *not* mentioned in
> the docstrings. It may be mentioned in the main documentation, but I
> didn't look at that at the time - there was nothing to suggest the
> docstrings were lacking pertinent information:
> 

You're right that this is missing from the docstrings; I'll rectify that.

> I'm not proposing that the standard library be special-cased, I'm
> proposing that the default behaviour of an unconfigured logging module
> in general be changed to something more useful (i.e. warnings and
> errors printed to stderr, everything else suppressed), rather than
> unhelpfully suppressing *all* output except for an "Oh, I'm throwing
> output away" message the first time it happens.

I don't have a philosophical problem with this, but there might be differing
opinions about this not just on python-dev but also in the wider community. I'd
definitely want logging to do the right thing and minimize inconvenience as much
as possible. There's also possibly a backwards compatibility dimension to this,
but I'm not sure to what extent a change like this would really affect people in
practice (as nothing will change if logging is configured).

> The specific use case that triggered my interest in this default
> behaviour is one where concurrent.futures *can't* raise an exception
> because it knows nothing is going to be in a position to handle that
> exception (due to the way callbacks are handled, c.f knows it provides
> the outermost exception handler in the affected call stack). Instead
> it has to catch the error and display it *somewhere* (i.e. it's very
> similar to the use cases for PyErr_WriteUnraisable at the C level).
> 
> The options for handling this are:
> 
> 1. Write the error detail directly to stderr. (Unless the default
> behaviour of logging changes, that is what I am going to suggest Brian
> do, as it exactly mimics the behaviour of the PyErr_WriteUnraisable
> API).
> 2. Write it to the root logger with the convenience APIs (Possible
> option, but I don't like the global state impact of implicitly calling
> basicConfig() from library code)
> 3. Add a StdErr handler for the logger (this is what is currently
> implemented, and again, I don't like it because of the global state
> impact on something that should be completely under an application's
> control)
> 
> Basically, the current behaviour of logging is such that libraries
> *cannot* use it for unraisable warnings and error messages, as the
> messages will be suppressed unless the application takes steps to see
> them. That is OK for debug and info messages, but unacceptable for
> warnings and errors. A throwaway script using concurrent.futures needs
> to know if callbacks are failing, and that needs to happen without any
> logging related boilerplate in the script itself.
> 
> If, however, an application completely lacking even a call to
> logging.basicConfig() would still see warnings and errors, then
> libraries could safely use the module without needing to worry about
> applications needing an particular boilerplate in order to see the
> unraisable errors and warnings that are emitted.
> 

Thanks for the detailed explanation. I agree that unraisable warnings and errors
need to be handled somehow. There is a way in which this can be done without
affecting a logging configuration, viz. logging can define a "handler of last
resort" (not attached to any logger) which is invoked when there are no
user-specified handlers. This would by default be a StreamHandler writing to
sys.stderr with a threshold of WARNING (or perhaps ERROR). Thus sounds like a
better option than a direct write to sys.stderr, since you can't change the
latter behaviour easily if you want to do something else instead.

This is of course a backwards-incompatible change to logging semantics: instead
of saying that logging will be silent unless explicitly asked to produce output,
we're saying that logging will always produce output for warnings and errors (or
perhaps just errors), unless explicitly silenced. This is of course in line with
the Zen of Python; the present behaviour, which is not so aligned, is based on
the idea that logging should not affect program behaviour if it's not wanted by
the program developer (as opposed to library developer).

It would also mean changing the documentation about NullHandler to say: "If you
have messages which must get out when you can't raise an exception, then don't
add a NullHandler to your top-level loggers."

Regards,

Vinay Sajip


From vinay_sajip at yahoo.co.uk  Wed Dec  8 09:51:44 2010
From: vinay_sajip at yahoo.co.uk (Vinay Sajip)
Date: Wed, 8 Dec 2010 08:51:44 +0000 (UTC)
Subject: [Python-Dev] Using logging in the stdlib and its unit tests
References: <loom.20101207T212306-892@post.gmane.org>	<idme8q$9bm$1@dough.gmane.org>
	<loom.20101208T004711-264@post.gmane.org>
	<idmqpt$rbq$1@dough.gmane.org>
Message-ID: <loom.20101208T093346-122@post.gmane.org>

Robert Kern <robert.kern <at> gmail.com> writes:

> 
> I really don't understand how this view can be consistent with the
> practice of adding NullHandler to loggers. If this message is so important
> to prevent misconfiguration, then why should a library author decide to
> silence it for his users?

Because the application developer knows more about the end-user audience for
their application, they are better placed to know how logging should work for
their application. It's not an error for a particular application developer to
decide that nothing should be produced by logging for a particular application;
they (particularly when casual users) would be confused by the misconfiguration
message due to logging by a library they're using.

The library author's users are the application developers who use the library,
not the end users who use their applications. Sometimes they're the same people,
I know, but I just think of them as wearing different hats :-)

> [...] I strongly suspect that almost all configurations include a 
> catch-all root logger and that most of those *only* consist of that
> root logger.

That doesn't seem right: your comment might be conflating loggers with handlers.
The common pattern would be (or should be) to name loggers according to __name__
in the modules which use logging, but only configure *handlers* for the root
logger. That way, logging messages indicate their origin (because of the
__name__ convention) but you only need to add handlers at the root logger to
capture all the logging information.

> Same mistake. I intended the correction to apply to all such statements in
> my post.

Oh, right. I thought this was a different case.

> I think that boilerplate should be minimized. If using getLogger() should
> almost always be followed by adding a NullHandler, then it should be the
> default behavior. The easiest way to achieve this effect is to simply not
> issue the warning message.

getLogger() should NOT "almost always be followed by adding a NullHandler". For
example, in Django, only the logger named "django" would have that handler
added; no other Django logger (e.g. "django.db.models") would need to have that
handler added.

Regards,

Vinay Sajip


From solipsis at pitrou.net  Wed Dec  8 10:41:45 2010
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Wed, 8 Dec 2010 10:41:45 +0100
Subject: [Python-Dev] Using logging in the stdlib and its unit tests
References: <loom.20101207T212306-892@post.gmane.org>
	<20101207215053.025ecf52@pitrou.net>
	<loom.20101208T002100-91@post.gmane.org>
	<20101208010913.363c6105@pitrou.net>
	<AANLkTim4ZPoch18+-qkunuYm-zcyh-vbhMOv9YVVEqo9@mail.gmail.com>
	<loom.20101208T022102-46@post.gmane.org>
Message-ID: <20101208104145.41ed925b@pitrou.net>

On Wed, 8 Dec 2010 01:51:44 +0000 (UTC)
Vinay Sajip <vinay_sajip at yahoo.co.uk> wrote:
>  
> > Adding a NullHandler isn't the right thing to do - the behaviour I
> > would want for any standard library logging that hasn't been
> > explicitly configured otherwise is to do what the root logger does
> > under basicConfig(): debug() and info() get suppressed, warn(),
> > error(), critical() and exception() go to stderr. It seems to me like
> > that would be more useful default behaviour in general than emitting a
> > warning about not finding a handler.
> >
> > So my suggestion would be:
> > 1. In the absence of a "config" call, make the "no handler" path in
> > loggers emit messages *as if* basicConfig() has been called (without
> > actually calling it)
> > 2. Remove the implicit calls to basicConfig() from the module level
> > convenience function
> > 
> > How *feasible* that idea is to implement, I don't know.
> > 
> 
> What about "Special cases aren't special enough to break the rules."?

Well, it seems we are asking to remove the special cases, since we are
asking for the special case to become the norm.

Antoine.



From solipsis at pitrou.net  Wed Dec  8 10:42:59 2010
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Wed, 8 Dec 2010 10:42:59 +0100
Subject: [Python-Dev] Using logging in the stdlib and its unit tests
References: <loom.20101207T212306-892@post.gmane.org>
	<20101207215053.025ecf52@pitrou.net>
	<loom.20101208T002100-91@post.gmane.org>
	<20101208010913.363c6105@pitrou.net> <idneln$t7i$1@dough.gmane.org>
Message-ID: <20101208104259.70f5699a@pitrou.net>

On Wed, 08 Dec 2010 09:09:45 +0100
Georg Brandl <g.brandl at gmx.net> wrote:

> Am 08.12.2010 01:09, schrieb Antoine Pitrou:
> > On Tue, 7 Dec 2010 23:45:39 +0000 (UTC)
> > Vinay Sajip <vinay_sajip at yahoo.co.uk> wrote:
> >> Antoine Pitrou <solipsis <at> pitrou.net> writes:
> >> 
> >> > 
> >> > I thought "error" and "critical" messages were logged to stderr by
> >> > default? Isn't it the case?
> >> > 
> >> 
> >> Only if you call basicConfig() or use the logging.debug(), logging.info(), etc.
> >> module-level convenience functions (which call basicConfig under the hood).
> > 
> > Why wouldn't it be the default for all logging calls ? Such special
> > cases don't really make things easy to remember.
> > 
> >> When is the NullHandler needed? Only for cases where an application developer
> >> uses a library which does logging under the covers (for those users who might be
> >> interested in logging its operations), but where that application developer
> >> doesn't use logging themselves for that application.
> > 
> > You seem pretty tied up to the "application developer" situation. There
> > are cases (scripts, prototyping, etc.) where you certainly want to see
> > error messages (errors should not pass silently) but don't want to
> > configure logging for each of the libraries you use.
> 
> But errors don't pass silently, do they?  The usual way to present errors
> is still by raising exceptions.

Or logging them.
http://docs.python.org/dev/library/logging.html#logging.Logger.exception



From solipsis at pitrou.net  Wed Dec  8 11:03:09 2010
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Wed, 8 Dec 2010 11:03:09 +0100
Subject: [Python-Dev] Using logging in the stdlib and its unit tests
References: <loom.20101207T212306-892@post.gmane.org>
	<20101207215053.025ecf52@pitrou.net>
	<loom.20101208T002100-91@post.gmane.org>
	<20101208010913.363c6105@pitrou.net>
	<loom.20101208T020511-443@post.gmane.org>
Message-ID: <20101208110309.56cf2a22@pitrou.net>

On Wed, 8 Dec 2010 01:19:45 +0000 (UTC)
Vinay Sajip <vinay_sajip at yahoo.co.uk> wrote:
> Antoine Pitrou <solipsis <at> pitrou.net> writes:
> 
> > 
> > Why wouldn't it be the default for all logging calls ? Such special
> > cases don't really make things easy to remember.
> >
> 
> One size doesn't fit all. Everything's documented reasonably well. If you use it
> often, you remember it. If you use it seldom, you might need to look things up.

My point is that the default behaviour should be helpful.

I would point out that logging is really the kind of thing people don't
want to spend time with. I know it's not very gratifying for the module
author (you!), but nobody expects (*) to have to first configure a
logging library when they want to use a third-party library. They
expect it to work *and* give useful error report by default.

(*) (not in the Python land anyway; Java might be different)

> The defaults are I believe reasonably convenient, or at least not inconvenient:
> 
> 1. If writing a library, add a NullHandler to the top-level logger of your
> library, don't add any other handlers by default (i.e. on import), document the
> names of the loggers you use. Otherwise, see the following.

First, you cannot call it a "default", since the library writer has to
make it explicit.
Second, I don't find that convenient at all. When I use a third-party
lib I want the errors to be displayed, not silenced. I'm willing to bet
that most people have the same expectation. *Especially* when
prototyping, since you're not sure you've got everything right.

So, that's two problems:
- the library author has to explicit activate what you argue is a
  "default" behaviour (otherwise the library user gets an unhelpful
  message that handlers are not configured)
- the library user still has to override that "default" since it's not
  satisfying for most uses

There's a third problem with it:
- all this is not easily discoverable. You cannot expect each user of a
  third-party lib to *also* read the logging docs in addition to the
  third-party lib docs. On my Web browser,
  http://docs.python.org/dev/library/logging.html seems to be at least
  50 pages long.

For a comparison, the warnings module logs all warnings by default on
stderr without requiring any a priori configuration. And that's what
stderr is for (reporting errors).

> 2. If writing a simple script, prototyping etc. i.e. for casual use, use
> logging.debug(), logging.info() etc. That's easy enough and convenient enough
> for casual users to remember, I hope. All WARNING, ERROR and CRITICAL events in
> your code and in that of any libraries you use will be printed to sys.stderr.

Indeed, this is convenient. And that's what I'm arguing should be the
default for *all* loggers, not just the top-level one. There's no
reason that what is convenient for the root logger isn't for other
loggers.

> 3. If slightly less than very casual use (e.g. you want to log to file, or want
> a specific message format), call basicConfig(). Just the one extra line to
> remember to do, hopefully not too inconvenient.

I know that one line sounds "not too inconvenient" since you are the
library author and you are so used to it (and certainly like using it
and spend a lot of time improving its features, which is good).
I think most people, however, have different expectations (see
beginning of this message).

> 4. For more advanced use, use programmatic configuration, fileConfig or
> dictConfig to specify a configuration. Convenience is inversely proportional to
> how involved the configuration is.

That I have already noticed :)

Regards

Antoine.



From ncoghlan at gmail.com  Wed Dec  8 11:29:57 2010
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Wed, 8 Dec 2010 20:29:57 +1000
Subject: [Python-Dev] Using logging in the stdlib and its unit tests
In-Reply-To: <loom.20101208T085644-791@post.gmane.org>
References: <loom.20101207T212306-892@post.gmane.org>
	<20101207215053.025ecf52@pitrou.net>
	<loom.20101208T002100-91@post.gmane.org>
	<20101208010913.363c6105@pitrou.net>
	<AANLkTim4ZPoch18+-qkunuYm-zcyh-vbhMOv9YVVEqo9@mail.gmail.com>
	<loom.20101208T022102-46@post.gmane.org>
	<AANLkTi=fQPKUs9-oAuGM3Gzn8uwi3tnMBMTy1WUFwX_p@mail.gmail.com>
	<loom.20101208T085644-791@post.gmane.org>
Message-ID: <AANLkTingiYtWPUQgH1ZeXsqd82tm7vQ=3S2BrYPWh+p5@mail.gmail.com>

On Wed, Dec 8, 2010 at 6:32 PM, Vinay Sajip <vinay_sajip at yahoo.co.uk> wrote:
> Thanks for the detailed explanation. I agree that unraisable warnings and errors
> need to be handled somehow. There is a way in which this can be done without
> affecting a logging configuration, viz. logging can define a "handler of last
> resort" (not attached to any logger) which is invoked when there are no
> user-specified handlers. This would by default be a StreamHandler writing to
> sys.stderr with a threshold of WARNING (or perhaps ERROR).

A "handler of last resort" is exactly the kind of concept I had in
mind. Just not expressed quite so succinctly :)

> Thus sounds like a
> better option than a direct write to sys.stderr, since you can't change the
> latter behaviour easily if you want to do something else instead.

Yeah, this is why I can understand Brian's desire to use the logging
module rather than a direct write to stderr in the futures
implementation.

> This is of course a backwards-incompatible change to logging semantics: instead
> of saying that logging will be silent unless explicitly asked to produce output,
> we're saying that logging will always produce output for warnings and errors (or
> perhaps just errors), unless explicitly silenced. This is of course in line with
> the Zen of Python; the present behaviour, which is not so aligned, is based on
> the idea that logging should not affect program behaviour if it's not wanted by
> the program developer (as opposed to library developer).

Libraries that write directly to stderr affect program behaviour as
well - at least with logging, the application developer will have an
easier time of silencing (or redirecting) them.

Also, in situations where this default handler will be used, logging
currently produces output at least once (the warning that it couldn't
find a handler to use).

So while it is a behavioural change, it is one that should be
acceptable from a compatibility point of view.

> It would also mean changing the documentation about NullHandler to say: "If you
> have messages which must get out when you can't raise an exception, then don't
> add a NullHandler to your top-level loggers."

Agreed. We could also make it easy to replace the handler of last
resort so people can easily recover the old behaviour (by inserting
NullHandler in place of the default last resort handler).

Cheers,
Nick.

-- 
Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia

From g.brandl at gmx.net  Wed Dec  8 11:48:16 2010
From: g.brandl at gmx.net (Georg Brandl)
Date: Wed, 08 Dec 2010 11:48:16 +0100
Subject: [Python-Dev] Using logging in the stdlib and its unit tests
In-Reply-To: <20101208104259.70f5699a@pitrou.net>
References: <loom.20101207T212306-892@post.gmane.org>	<20101207215053.025ecf52@pitrou.net>	<loom.20101208T002100-91@post.gmane.org>	<20101208010913.363c6105@pitrou.net>
	<idneln$t7i$1@dough.gmane.org> <20101208104259.70f5699a@pitrou.net>
Message-ID: <idnnuv$9g8$1@dough.gmane.org>

Am 08.12.2010 10:42, schrieb Antoine Pitrou:

>> But errors don't pass silently, do they?  The usual way to present errors
>> is still by raising exceptions.
> 
> Or logging them.
> http://docs.python.org/dev/library/logging.html#logging.Logger.exception

Yes, thank you I'm aware of the exception() method.  But hopefully standard
library modules don't use it to report exceptions to code that uses them?

Georg


From solipsis at pitrou.net  Wed Dec  8 12:12:20 2010
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Wed, 8 Dec 2010 12:12:20 +0100
Subject: [Python-Dev] Using logging in the stdlib and its unit tests
References: <loom.20101207T212306-892@post.gmane.org>
	<20101207215053.025ecf52@pitrou.net>
	<loom.20101208T002100-91@post.gmane.org>
	<20101208010913.363c6105@pitrou.net> <idneln$t7i$1@dough.gmane.org>
	<20101208104259.70f5699a@pitrou.net> <idnnuv$9g8$1@dough.gmane.org>
Message-ID: <20101208121220.1ac1ed97@pitrou.net>

On Wed, 08 Dec 2010 11:48:16 +0100
Georg Brandl <g.brandl at gmx.net> wrote:
> Am 08.12.2010 10:42, schrieb Antoine Pitrou:
> 
> >> But errors don't pass silently, do they?  The usual way to present errors
> >> is still by raising exceptions.
> > 
> > Or logging them.
> > http://docs.python.org/dev/library/logging.html#logging.Logger.exception
> 
> Yes, thank you I'm aware of the exception() method.  But hopefully standard
> library modules don't use it to report exceptions to code that uses them?

I'm not aware of that, but there are certainly third-party libs using
it (think an HTTP server that wants to log an error in one of its
request handlers without the error taking the whole server down).

Antoine.



From vinay_sajip at yahoo.co.uk  Wed Dec  8 13:15:53 2010
From: vinay_sajip at yahoo.co.uk (Vinay Sajip)
Date: Wed, 8 Dec 2010 12:15:53 +0000 (UTC)
Subject: [Python-Dev] Using logging in the stdlib and its unit tests
References: <loom.20101207T212306-892@post.gmane.org>
	<20101207215053.025ecf52@pitrou.net>
	<loom.20101208T002100-91@post.gmane.org>
	<20101208010913.363c6105@pitrou.net>
	<loom.20101208T020511-443@post.gmane.org>
	<20101208110309.56cf2a22@pitrou.net>
Message-ID: <loom.20101208T123754-508@post.gmane.org>

Antoine Pitrou <solipsis <at> pitrou.net> writes:

> 
> My point is that the default behaviour should be helpful.
>

I can't disagree with that. Now if only we could agree what we mean by "default"
and "helpful" ;-)
 
> I would point out that logging is really the kind of thing people don't
> want to spend time with. I know it's not very gratifying for the module
> author (you!), but nobody expects (*) to have to first configure a

Logging's just a tool - I completely understand that it's just an adjunct to the
interesting stuff. Doesn't bother me a bit :-)

> logging library when they want to use a third-party library. They
> expect it to work *and* give useful error report by default.
> 
> (*) (not in the Python land anyway; Java might be different)

The Java thing is a red herring, I believe. It's more akin to the Unix idea of
minimum verbosity as a default. In general, you don't need to "configure a
library" except when you want specific behaviour from that library's loggers.

> First, you cannot call it a "default", since the library writer has to
> make it explicit.
> Second, I don't find that convenient at all. When I use a third-party
> lib I want the errors to be displayed, not silenced. I'm willing to bet
> that most people have the same expectation. *Especially* when
> prototyping, since you're not sure you've got everything right.
> 
> So, that's two problems:
> - the library author has to explicit activate what you argue is a
>   "default" behaviour (otherwise the library user gets an unhelpful
>   message that handlers are not configured)
> - the library user still has to override that "default" since it's not
>   satisfying for most uses
> 

See my comments to Nick Coghlan's post about getting messages out when you can't
raise an exception. I think the case is not as common as you suggest (because in
many instances, you would raise an exception to signal an error).

I'm not opposed to changing the way things work, but I think we need more of a
consensus because some backwards incompatibility will result, albeit not likely
to affect too many users of logging.

> There's a third problem with it:
> - all this is not easily discoverable. You cannot expect each user of a
>   third-party lib to *also* read the logging docs in addition to the
>   third-party lib docs. On my Web browser,
>   http://docs.python.org/dev/library/logging.html seems to be at least
>   50 pages long.

You're complaining about too much documentation?! Don't measure it by weight!
Seriously, I don't see what's wrong with a library developer reading the
documentation for any module they're using, logging included. "Configuring
logging for a library" is easily visible in the sidebar on that page. So I
wouldn't agree that "it's not easily discoverable". It's not exactly buried. And
it's the library developer who has to read it, not the user of the library. The
user of the library only needs to know what logger names are used by that
library, and that too only if they want to configure those loggers.

> For a comparison, the warnings module logs all warnings by default on
> stderr without requiring any a priori configuration. And that's what
> stderr is for (reporting errors).
> 

But that behaviour is *not* always wanted. That's why I was asked to add
warnings integration to logging - to redirect warning messages to logging.
Things are seldom as black and white as you're making them out to be here.

> Indeed, this is convenient. And that's what I'm arguing should be the
> default for *all* loggers, not just the top-level one. There's no
> reason that what is convenient for the root logger isn't for other
> loggers.

Loggers work in a hierarchy, and the package is specifically designed so that
you can attach handlers just to the root logger, and get messages from all code
in your application + its libraries to destinations that you configure there.
This is not special case behaviour for root logger versus other loggers - it's
just that the root logger is a convenient, logical place to attach handlers.

> I know that one line sounds "not too inconvenient" since you are the
> library author and you are so used to it (and certainly like using it
> and spend a lot of time improving its features, which is good).
> I think most people, however, have different expectations (see
> beginning of this message).

Hey, that's not why I think "not too inconvenient"! It's because

(a) It's no less inconvenient than "import foo", and it's hard to argue that one
line is really inconvenient for this case - remember, for the simplest uses you
can omit the basicConfig() call, since you can use logging.info() etc.

(b) For this case, since there's no telepathy in this world, you need some way
for the programmer to communicate their intent about the logging level, or
format string, or log file that they want to use. I stand by the statement that
one line (or more precisely, one function call) is not too much to ask here, so
yes - "not too inconvenient". That's emphatically not because I'm the module's
author and "used to it".

Regards,

Vinay Sajip


From vinay_sajip at yahoo.co.uk  Wed Dec  8 13:22:07 2010
From: vinay_sajip at yahoo.co.uk (Vinay Sajip)
Date: Wed, 8 Dec 2010 12:22:07 +0000 (UTC)
Subject: [Python-Dev] Using logging in the stdlib and its unit tests
References: <loom.20101207T212306-892@post.gmane.org>
	<20101207215053.025ecf52@pitrou.net>
	<loom.20101208T002100-91@post.gmane.org>
	<20101208010913.363c6105@pitrou.net> <idneln$t7i$1@dough.gmane.org>
	<20101208104259.70f5699a@pitrou.net> <idnnuv$9g8$1@dough.gmane.org>
	<20101208121220.1ac1ed97@pitrou.net>
Message-ID: <loom.20101208T131616-11@post.gmane.org>

Antoine Pitrou <solipsis <at> pitrou.net> writes:

> 
> On Wed, 08 Dec 2010 11:48:16 +0100
> Georg Brandl <g.brandl <at> gmx.net> wrote:
  But hopefully standard
> > library modules don't use it to report exceptions to code that uses them?
> 
> I'm not aware of that, but there are certainly third-party libs using
> it (think an HTTP server that wants to log an error in one of its
> request handlers without the error taking the whole server down).
> 

That's not the same thing as Georg is talking about, IIUC. The exception()
method is used in exception *handler* code to record that the exception
occurred, but the correct thing for any code to do when an error condition is
detected (in most situations at least) is to raise an exception. It would be
quite wrong for code to e.g. call logger.error(...) instead of raise ...

Regards,

Vinay Sajip


From p.f.moore at gmail.com  Wed Dec  8 14:02:00 2010
From: p.f.moore at gmail.com (Paul Moore)
Date: Wed, 8 Dec 2010 13:02:00 +0000
Subject: [Python-Dev] Using logging in the stdlib and its unit tests
In-Reply-To: <loom.20101208T085644-791@post.gmane.org>
References: <loom.20101207T212306-892@post.gmane.org>
	<20101207215053.025ecf52@pitrou.net>
	<loom.20101208T002100-91@post.gmane.org>
	<20101208010913.363c6105@pitrou.net>
	<AANLkTim4ZPoch18+-qkunuYm-zcyh-vbhMOv9YVVEqo9@mail.gmail.com>
	<loom.20101208T022102-46@post.gmane.org>
	<AANLkTi=fQPKUs9-oAuGM3Gzn8uwi3tnMBMTy1WUFwX_p@mail.gmail.com>
	<loom.20101208T085644-791@post.gmane.org>
Message-ID: <AANLkTim8zy5dUn3DOC+sHMcmkVDcGp+Z3jqZe0RjtYAB@mail.gmail.com>

On 8 December 2010 08:32, Vinay Sajip <vinay_sajip at yahoo.co.uk> wrote:
> Nick Coghlan <ncoghlan <at> gmail.com> writes:
>> I'm not proposing that the standard library be special-cased, I'm
>> proposing that the default behaviour of an unconfigured logging module
>> in general be changed to something more useful (i.e. warnings and
>> errors printed to stderr, everything else suppressed), rather than
>> unhelpfully suppressing *all* output except for an "Oh, I'm throwing
>> output away" message the first time it happens.
>
> I don't have a philosophical problem with this, but there might be differing
> opinions about this not just on python-dev but also in the wider community. I'd
> definitely want logging to do the right thing and minimize inconvenience as much
> as possible. There's also possibly a backwards compatibility dimension to this,
> but I'm not sure to what extent a change like this would really affect people in
> practice (as nothing will change if logging is configured).

To provide an application developer point of view, I'm +1 on this change.

I want library developers to be able to report errors/warnings and
have them appear by default, without me doing anything (or even
needing to be aware that the logging module is involved). The benefit
of the library developer using the logging module to me is that if I
deem it appropriate, I can direct the warnings/errors elsewhere simply
by configuring logging.

So, I'd like the library developer to feel that they can use logging
in precisely the same ways as they would write to sys.stderr, and with
the same expectations (output will be seen by default), but with less
reluctance because they know that I can easily adapt their output
according to my needs.

>> The options for handling this are:
>>
>> 1. Write the error detail directly to stderr. (Unless the default
>> behaviour of logging changes, that is what I am going to suggest Brian
>> do, as it exactly mimics the behaviour of the PyErr_WriteUnraisable
>> API).

I find library code that does this annoying. It's the right effect,
but leaves me with little or no control should I want to do something
like log to a file.

>> 2. Write it to the root logger with the convenience APIs (Possible
>> option, but I don't like the global state impact of implicitly calling
>> basicConfig() from library code)

Library code shouldn't use the root logger, and shouldn't call
basicConfig - I agree with Nick again.

>> 3. Add a StdErr handler for the logger (this is what is currently
>> implemented, and again, I don't like it because of the global state
>> impact on something that should be completely under an application's
>> control)

That seems like a best of a bad job answer. As an application user,
I'd expect to find it awkward to modify this default, purely because I
have to undo the configuration that the library had to do to work
around the logging library's defaults.

> Thanks for the detailed explanation. I agree that unraisable warnings and errors
> need to be handled somehow. There is a way in which this can be done without
> affecting a logging configuration, viz. logging can define a "handler of last
> resort" (not attached to any logger) which is invoked when there are no
> user-specified handlers. This would by default be a StreamHandler writing to
> sys.stderr with a threshold of WARNING (or perhaps ERROR). Thus sounds like a
> better option than a direct write to sys.stderr, since you can't change the
> latter behaviour easily if you want to do something else instead.

>From my POV as an application writer, I don't care how you implement
it (but I agree that your suggestion sounds more flexible). As long as
errors and warnings are printed by default, and I can set my own
configuration without having to undo the existing configuration, then
I'm happy.

> This is of course a backwards-incompatible change to logging semantics: instead
> of saying that logging will be silent unless explicitly asked to produce output,
> we're saying that logging will always produce output for warnings and errors (or
> perhaps just errors), unless explicitly silenced. This is of course in line with
> the Zen of Python; the present behaviour, which is not so aligned, is based on
> the idea that logging should not affect program behaviour if it's not wanted by
> the program developer (as opposed to library developer).

I can see that it's an incompatible change. But I'd still be in favour
of it for 3.2. In my view, it needs to be done, and it'll only get
harder to do as time goes on.

Paul.

From p.f.moore at gmail.com  Wed Dec  8 14:19:26 2010
From: p.f.moore at gmail.com (Paul Moore)
Date: Wed, 8 Dec 2010 13:19:26 +0000
Subject: [Python-Dev] Using logging in the stdlib and its unit tests
In-Reply-To: <loom.20101208T123754-508@post.gmane.org>
References: <loom.20101207T212306-892@post.gmane.org>
	<20101207215053.025ecf52@pitrou.net>
	<loom.20101208T002100-91@post.gmane.org>
	<20101208010913.363c6105@pitrou.net>
	<loom.20101208T020511-443@post.gmane.org>
	<20101208110309.56cf2a22@pitrou.net>
	<loom.20101208T123754-508@post.gmane.org>
Message-ID: <AANLkTikvH=r-EMUF9vXBUNV9n2B-7rc_Xhb5nFB3LODp@mail.gmail.com>

On 8 December 2010 12:15, Vinay Sajip <vinay_sajip at yahoo.co.uk> wrote:
> The Java thing is a red herring, I believe. It's more akin to the Unix idea of
> minimum verbosity as a default.

And yet Unix invented the concept of stderr, precisely to ensure that
there's a route for things the program wants to inform the user about
to get out.

The NullHandler approach could be seen as the equivalent of adding
2>/dev/null to every command by default.

> In general, you don't need to "configure a
> library" except when you want specific behaviour from that library's loggers.

The "specific behaviour" I want is for errors and warnings to go to
stderr. Every time. (Unless I'm writing a service that wants to
redirect all output to a log file, or something like that - and that's
where I'm really glad to have the logging module! And the fact that
the defaults tend to encourage library authors not to use it really
frustrate me as an applicatioo developer.)

> I'm not opposed to changing the way things work, but I think we need more of a
> consensus because some backwards incompatibility will result, albeit not likely
> to affect too many users of logging.

Hmm. As far as I can tell from reading the thread, everyone but you is
in favour of errors and warnings going to stderr by default. I may
have missed some dissenting views, but it certainly feels like a
consensus to me!

If you feel that the proposal needs to be presented to a wider group
than just python-dev then that's fair. But personally, I think that
python-dev will be a reasonably good indicator in this case.

>> For a comparison, the warnings module logs all warnings by default on
>> stderr without requiring any a priori configuration. And that's what
>> stderr is for (reporting errors).
>>
>
> But that behaviour is *not* always wanted. That's why I was asked to add
> warnings integration to logging - to redirect warning messages to logging.
> Things are seldom as black and white as you're making them out to be here.

Exactly!!!! Logging should be a BETTER option than direct writes to
stderr. At the moment, though, it's a CHOICE - writes to stderr, which
are visible by default but (very) inflexible, or logging which is
flexible but suppressed by default (OK, "if you follow the recommended
practice to avoid a user-hostile warning"). The problem is that some
people are voting with their feet that visibility matters more than
flexibility.

>> Indeed, this is convenient. And that's what I'm arguing should be the
>> default for *all* loggers, not just the top-level one. There's no
>> reason that what is convenient for the root logger isn't for other
>> loggers.
>
> Loggers work in a hierarchy, and the package is specifically designed so that
> you can attach handlers just to the root logger, and get messages from all code
> in your application + its libraries to destinations that you configure there.
> This is not special case behaviour for root logger versus other loggers - it's
> just that the root logger is a convenient, logical place to attach handlers.

But you don't because the library developer added a NullHandler which
you have to switch off!!!

> Hey, that's not why I think "not too inconvenient"! It's because
>
> (a) It's no less inconvenient than "import foo", and it's hard to argue that one
> line is really inconvenient for this case - remember, for the simplest uses you
> can omit the basicConfig() call, since you can use logging.info() etc.
>
> (b) For this case, since there's no telepathy in this world, you need some way
> for the programmer to communicate their intent about the logging level, or
> format string, or log file that they want to use. I stand by the statement that
> one line (or more precisely, one function call) is not too much to ask here, so
> yes - "not too inconvenient". That's emphatically not because I'm the module's
> author and "used to it".

FWIW, basicConfig has *never* been what I want. Except in the sense
that I have to use it to switch on logging to stderr (if only it
worked, but see the comment above about NullHandler) because that's
not the default. In all other cases, I've wanted a little bit more
(which may not be hard to achieve, but that's not the point here).

Saying that's convenient is like saying that adding
sys.stderr=open("/dev/tty", "w") is a convenient way of getting errors
displayed when you don't like the default destination of /dev/null.

Paul.

From g.brandl at gmx.net  Wed Dec  8 14:33:04 2010
From: g.brandl at gmx.net (Georg Brandl)
Date: Wed, 08 Dec 2010 14:33:04 +0100
Subject: [Python-Dev] Using logging in the stdlib and its unit tests
In-Reply-To: <loom.20101208T131616-11@post.gmane.org>
References: <loom.20101207T212306-892@post.gmane.org>	<20101207215053.025ecf52@pitrou.net>	<loom.20101208T002100-91@post.gmane.org>	<20101208010913.363c6105@pitrou.net>
	<idneln$t7i$1@dough.gmane.org>	<20101208104259.70f5699a@pitrou.net>
	<idnnuv$9g8$1@dough.gmane.org>	<20101208121220.1ac1ed97@pitrou.net>
	<loom.20101208T131616-11@post.gmane.org>
Message-ID: <ido1eh$lgb$1@dough.gmane.org>

Am 08.12.2010 13:22, schrieb Vinay Sajip:
> Antoine Pitrou <solipsis <at> pitrou.net> writes:
> 
>> 
>> On Wed, 08 Dec 2010 11:48:16 +0100
>> Georg Brandl <g.brandl <at> gmx.net> wrote:
>   But hopefully standard
>> > library modules don't use it to report exceptions to code that uses them?
>> 
>> I'm not aware of that, but there are certainly third-party libs using
>> it (think an HTTP server that wants to log an error in one of its
>> request handlers without the error taking the whole server down).
>> 
> 
> That's not the same thing as Georg is talking about, IIUC. The exception()
> method is used in exception *handler* code to record that the exception
> occurred, but the correct thing for any code to do when an error condition is
> detected (in most situations at least) is to raise an exception. It would be
> quite wrong for code to e.g. call logger.error(...) instead of raise ...

Exactly.  The HTTP server is of course a good example of an application, and
one of the most obvious candidates in the stdilb to use the logging module
instead of sys.stderr.write().

Georg


From vinay_sajip at yahoo.co.uk  Wed Dec  8 14:57:40 2010
From: vinay_sajip at yahoo.co.uk (Vinay Sajip)
Date: Wed, 8 Dec 2010 13:57:40 +0000 (UTC)
Subject: [Python-Dev] Using logging in the stdlib and its unit tests
References: <loom.20101207T212306-892@post.gmane.org>
	<20101207215053.025ecf52@pitrou.net>
	<loom.20101208T002100-91@post.gmane.org>
	<20101208010913.363c6105@pitrou.net>
	<loom.20101208T020511-443@post.gmane.org>
	<20101208110309.56cf2a22@pitrou.net>
	<loom.20101208T123754-508@post.gmane.org>
	<AANLkTikvH=r-EMUF9vXBUNV9n2B-7rc_Xhb5nFB3LODp@mail.gmail.com>
Message-ID: <loom.20101208T142253-180@post.gmane.org>

Paul Moore <p.f.moore <at> gmail.com> writes:

> And yet Unix invented the concept of stderr, precisely to ensure that
> there's a route for things the program wants to inform the user about
> to get out.
> 
> The NullHandler approach could be seen as the equivalent of adding
> 2>/dev/null to every command by default.
>

Not quite true. The program can (as of right now) ensure that all warnings and
messages get directed to stderr, simply by calling basicConfig(). That'll work
even if every library has added a NullHandler. Although you can argue that the
call to basicConfig() shouldn't be necessary, you tone comes across as a bit of
an overreaction.

> The "specific behaviour" I want is for errors and warnings to go to
> stderr. Every time. (Unless I'm writing a service that wants to
> redirect all output to a log file, or something like that - and that's
> where I'm really glad to have the logging module! And the fact that
> the defaults tend to encourage library authors not to use it really
> frustrate me as an applicatioo developer.)
>

I don't see where your assertion that "the defaults tend to encourage library
developers not to use it" comes from. If it's Antoine's post in this thread,
then I believe he was referring to the stdlib rather than libraries in general,
and I would also say that I don't believe it's a major factor; I would say that
inertia was a more pertinent factor. Of course I understand I could be wrong
about this, but I don't recall when a stdlib maintainer has said to me, "I want
to start using logging in stdlib module XXX, but can't justify it because ..." I
doubt whether making this change will result in a flurry of stdlib modules which
suddenly adopt logging, but stdlib maintainers can certainly prove me wrong
easily enough by stating otherwise in response to this. It's not as if I'm not
approachable - you will see that many improvements to the logging package exist
because of user feedback, both from python-dev and elsewhere.

> Hmm. As far as I can tell from reading the thread, everyone but you is
> in favour of errors and warnings going to stderr by default. I may
> have missed some dissenting views, but it certainly feels like a
> consensus to me!

"Everyone but you" - I think you mean "everyone on this thread but you", and
"everyone else on this thread" is not remotely close to "everyone else, period".
 
> If you feel that the proposal needs to be presented to a wider group
> than just python-dev then that's fair. But personally, I think that
> python-dev will be a reasonably good indicator in this case.

I don't dispute that, but there's not really been enough time for more than a
few people on python-dev to respond.
 
> > But that behaviour is *not* always wanted. That's why I was asked to add
> > warnings integration to logging - to redirect warning messages to logging.
> > Things are seldom as black and white as you're making them out to be here.
> 
> Exactly!!!! Logging should be a BETTER option than direct writes to
> stderr. At the moment, though, it's a CHOICE - writes to stderr, which
> are visible by default but (very) inflexible, or logging which is
> flexible but suppressed by default (OK, "if you follow the recommended
> practice to avoid a user-hostile warning"). The problem is that some
> people are voting with their feet that visibility matters more than
> flexibility.
>

I don't think it's unreasonable to ask which people are voting with their feet,
that you know of. This discussion was initiated by me after I got added to the
nosy list of an issue. It's not as if there has been a host of voters who have
asked for change and are walking away because of some intransigence on my part.
As you can see in my response to Nick's post, I am making concrete suggestions
on how the situation could be improved. It's not helping to talk about people
voting with their feet, especially as no serious attempt to engage with me on
this issue has been made in the past. Because of backward-compatibility
considerations, I would consider it premature to make this kind of change just
based on a request by one or two python-dev members. I have the highest level of
respect for people who have responded on this thread, but how can I be sure that
they're representing everyone else on this list? It's a great community, with a
lot of cohesion, but people do have divergent opinions. (I'm not talking about
me here. If there's a consensus about this, I've no problem implementing a
"handler of last resort".)

> But you don't because the library developer added a NullHandler which
> you have to switch off!!!
>

Where are you getting this "switch off" from? There's no switching off that
happens. The NullHandler remains in place. What you are doing is "switching on"
output to some specific place you want - console, file etc. - with a level of
verbosity and format that you choose.

> FWIW, basicConfig has *never* been what I want. Except in the sense
> that I have to use it to switch on logging to stderr (if only it
> worked, but see the comment above about NullHandler) because that's
> not the default. In all other cases, I've wanted a little bit more
> (which may not be hard to achieve, but that's not the point here).

Well, "may not be hard to achieve" is in line with "not too inconvenient", if
you ask me. And as for "if only it worked", that makes it sound like there's
some effect you can't achieve at very modest cost in terms of any code you have
to write.

> Saying that's convenient is like saying that adding
> sys.stderr=open("/dev/tty", "w") is a convenient way of getting errors
> displayed when you don't like the default destination of /dev/null.

I don't think I'm saying that at all. I see where you're coming from, but apart
from stating your point of view, IMO you're not adding to the discussion in the
same way that Nick, say, has.

Regards,


Vinay Sajip



From fdrake at acm.org  Wed Dec  8 15:09:59 2010
From: fdrake at acm.org (Fred Drake)
Date: Wed, 8 Dec 2010 09:09:59 -0500
Subject: [Python-Dev] Using logging in the stdlib and its unit tests
In-Reply-To: <AANLkTikvH=r-EMUF9vXBUNV9n2B-7rc_Xhb5nFB3LODp@mail.gmail.com>
References: <loom.20101207T212306-892@post.gmane.org>
	<20101207215053.025ecf52@pitrou.net>
	<loom.20101208T002100-91@post.gmane.org>
	<20101208010913.363c6105@pitrou.net>
	<loom.20101208T020511-443@post.gmane.org>
	<20101208110309.56cf2a22@pitrou.net>
	<loom.20101208T123754-508@post.gmane.org>
	<AANLkTikvH=r-EMUF9vXBUNV9n2B-7rc_Xhb5nFB3LODp@mail.gmail.com>
Message-ID: <AANLkTinknKGBd3WWrUdaxBR-hsAHRo8x6vWgjxmV=kd3@mail.gmail.com>

On Wed, Dec 8, 2010 at 8:19 AM, Paul Moore <p.f.moore at gmail.com> wrote:
> But you don't because the library developer added a NullHandler which
> you have to switch off!!!

I'm suspecting there's misunderstanding on this point.

If I have a logger "myns.mypackage" for my library, and register a
NullHandler for that, that doesn't need to be turned off for
applications.  What it accomplishes is that for all messages sent to
the "myns.mypackage" logger (and descendants) are handled by at least
one handler.  Log records are still propagated toward the root for
outer loggers to handle.  An application can also still choose to
install additional handlers for "myns.mypackage" if desired, and do
whatever output handling is appropriate.

The library-installed NullHandler on a library-scoped logger simply
ensures that the library doesn't trigger the message about un-handled
log records.

We seem to have consensus on not wanting that message triggered for
library logging is desirable, and this NullHandler trick provides a
way to handle that independent from other possible changes.


? -Fred

--
Fred L. Drake, Jr.? ? <fdrake at acm.org>
"A storm broke loose in my mind."? --Albert Einstein

From solipsis at pitrou.net  Wed Dec  8 15:27:59 2010
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Wed, 8 Dec 2010 15:27:59 +0100
Subject: [Python-Dev] Using logging in the stdlib and its unit tests
References: <loom.20101207T212306-892@post.gmane.org>
	<20101207215053.025ecf52@pitrou.net>
	<loom.20101208T002100-91@post.gmane.org>
	<20101208010913.363c6105@pitrou.net>
	<loom.20101208T020511-443@post.gmane.org>
	<20101208110309.56cf2a22@pitrou.net>
	<loom.20101208T123754-508@post.gmane.org>
Message-ID: <20101208152759.32b53cb2@pitrou.net>

On Wed, 8 Dec 2010 12:15:53 +0000 (UTC)
Vinay Sajip <vinay_sajip at yahoo.co.uk> wrote:
> > First, you cannot call it a "default", since the library writer has to
> > make it explicit.
> > Second, I don't find that convenient at all. When I use a third-party
> > lib I want the errors to be displayed, not silenced. I'm willing to bet
> > that most people have the same expectation. *Especially* when
> > prototyping, since you're not sure you've got everything right.
> > 
> > So, that's two problems:
> > - the library author has to explicit activate what you argue is a
> >   "default" behaviour (otherwise the library user gets an unhelpful
> >   message that handlers are not configured)
> > - the library user still has to override that "default" since it's not
> >   satisfying for most uses
> > 
> 
> See my comments to Nick Coghlan's post about getting messages out when you can't
> raise an exception. I think the case is not as common as you suggest (because in
> many instances, you would raise an exception to signal an error).

I'm not talking specifically about exceptions, but about errors in
general. If the case wasn't common, I'm not sure why the error() and
critical() methods would exist at all.

(of course I'm assuming error() is meant to output error messages. If
that's a wrong interpretation then I'm a bit puzzled :-))

> > There's a third problem with it:
> > - all this is not easily discoverable. You cannot expect each user of a
> >   third-party lib to *also* read the logging docs in addition to the
> >   third-party lib docs. On my Web browser,
> >   http://docs.python.org/dev/library/logging.html seems to be at least
> >   50 pages long.
> 
> You're complaining about too much documentation?! Don't measure it by weight!
> Seriously, I don't see what's wrong with a library developer reading the
> documentation for any module they're using, logging included.

I was talking about the user of a library, not its developer (see the
snippet quoted above).

> The
> user of the library only needs to know what logger names are used by that
> library, and that too only if they want to configure those loggers.

The thing is, they don't *want* to configure them, but you force them
to do some configuration if they don't want error messages to be
silenced.

> > For a comparison, the warnings module logs all warnings by default on
> > stderr without requiring any a priori configuration. And that's what
> > stderr is for (reporting errors).
> > 
> 
> But that behaviour is *not* always wanted.

Sure. What I'm arguing is that the warnings module's default behaviour
is much more useful than the logging module's default behaviour.

> That's why I was asked to add
> warnings integration to logging - to redirect warning messages to logging.

That's a separate topic :)

> (b) For this case, since there's no telepathy in this world, you need some way
> for the programmer to communicate their intent about the logging level, or
> format string, or log file that they want to use.

Well, the programmer doesn't have to communicate their intent about
what and how warnings get displayed by default, and I'm not sure the
warnings module has a telepathy routine inside (although you never
know, since IIRC Brett did some philosophy studies).

Regards

Antoine.



From fdrake at acm.org  Wed Dec  8 15:43:09 2010
From: fdrake at acm.org (Fred Drake)
Date: Wed, 8 Dec 2010 09:43:09 -0500
Subject: [Python-Dev] Using logging in the stdlib and its unit tests
In-Reply-To: <20101208152759.32b53cb2@pitrou.net>
References: <loom.20101207T212306-892@post.gmane.org>
	<20101207215053.025ecf52@pitrou.net>
	<loom.20101208T002100-91@post.gmane.org>
	<20101208010913.363c6105@pitrou.net>
	<loom.20101208T020511-443@post.gmane.org>
	<20101208110309.56cf2a22@pitrou.net>
	<loom.20101208T123754-508@post.gmane.org>
	<20101208152759.32b53cb2@pitrou.net>
Message-ID: <AANLkTi=iF0QJKJJGwjUJYNYuhHPU_3p9rJ=982h9gLKb@mail.gmail.com>

On Wed, Dec 8, 2010 at 9:27 AM, Antoine Pitrou <solipsis at pitrou.net> wrote:
> The thing is, they don't *want* to configure them, but you force them
> to do some configuration if they don't want error messages to be
> silenced.

As I tried to explain earlier, a NullHandler doesn't silence anything
except the message about logging not being configured. Propagation is
not controlled by the handlers, but by the loggers.


? -Fred

--
Fred L. Drake, Jr.? ? <fdrake at acm.org>
"A storm broke loose in my mind."? --Albert Einstein

From vinay_sajip at yahoo.co.uk  Wed Dec  8 15:54:09 2010
From: vinay_sajip at yahoo.co.uk (Vinay Sajip)
Date: Wed, 8 Dec 2010 14:54:09 +0000 (UTC)
Subject: [Python-Dev] Using logging in the stdlib and its unit tests
References: <loom.20101207T212306-892@post.gmane.org>
	<20101207215053.025ecf52@pitrou.net>
	<loom.20101208T002100-91@post.gmane.org>
	<20101208010913.363c6105@pitrou.net>
	<loom.20101208T020511-443@post.gmane.org>
	<20101208110309.56cf2a22@pitrou.net>
	<loom.20101208T123754-508@post.gmane.org>
	<20101208152759.32b53cb2@pitrou.net>
Message-ID: <loom.20101208T154218-14@post.gmane.org>

Antoine Pitrou <solipsis <at> pitrou.net> writes:

> I'm not talking specifically about exceptions, but about errors in
> general. If the case wasn't common, I'm not sure why the error() and
> critical() methods would exist at all.
> 
> (of course I'm assuming error() is meant to output error messages. If
> that's a wrong interpretation then I'm a bit puzzled )
> 

Then again, it's not always helpful when *library* code prints out messages to
stderr. A user of the library might prefer that an error code be returned or an
exception raised, so they can deal with the condition most appropriately.

> I was talking about the user of a library, not its developer (see the
> snippet quoted above).

Yes, but see my response to that. A user of a library doesn't need to know the
specifics of what loggers that library uses, they just need to make a positive
statement of where they want warning and error messages go, and how they want
them formatted.

BTW we haven't discussed this yet - but these enabled-by-default messages, what
format should they have? (e.g. just message, logger name + message, logger name
+ severity + message etc.) It might sound like a trivial point, but it also
seems like whatever you pick, some bikeshed discussions would ensue. At present,
the developer is encouraged to choose (as they have to call basicConfig anyway),
avoiding that bikeshedding altogether.

> The thing is, they don't *want* to configure them, but you force them
> to do some configuration if they don't want error messages to be
> silenced.
>

True, but it need be no more than a call to basicConfig(), so "not too
inconvenient" ;-)

> Sure. What I'm arguing is that the warnings module's default behaviour
> is much more useful than the logging module's default behaviour.
>

Sure, that's what we're discussing, all right.

>> That's why I was asked to add
>> warnings integration to logging - to redirect warning messages to logging.
> 
> That's a separate topic :)
>

If you say so ;-)
 
>> (b) For this case, since there's no telepathy in this world, you need some
>> way for the programmer to communicate their intent about the logging level,
>> or format string, or log file that they want to use.
> 
> Well, the programmer doesn't have to communicate their intent about
> what and how warnings get displayed by default, and I'm not sure the
> warnings module has a telepathy routine inside (although you never
> know, since IIRC Brett did some philosophy studies).

See my earlier comment about formatting and the potential for bikeshedding ;-)

Regards,

Vinay Sajip


From ncoghlan at gmail.com  Wed Dec  8 15:56:14 2010
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Thu, 9 Dec 2010 00:56:14 +1000
Subject: [Python-Dev] Using logging in the stdlib and its unit tests
In-Reply-To: <AANLkTi=iF0QJKJJGwjUJYNYuhHPU_3p9rJ=982h9gLKb@mail.gmail.com>
References: <loom.20101207T212306-892@post.gmane.org>
	<20101207215053.025ecf52@pitrou.net>
	<loom.20101208T002100-91@post.gmane.org>
	<20101208010913.363c6105@pitrou.net>
	<loom.20101208T020511-443@post.gmane.org>
	<20101208110309.56cf2a22@pitrou.net>
	<loom.20101208T123754-508@post.gmane.org>
	<20101208152759.32b53cb2@pitrou.net>
	<AANLkTi=iF0QJKJJGwjUJYNYuhHPU_3p9rJ=982h9gLKb@mail.gmail.com>
Message-ID: <AANLkTinHr4t3bgsa7g3Y8oSH-e0xNbJCEYP5yvBR3sTT@mail.gmail.com>

On Thu, Dec 9, 2010 at 12:43 AM, Fred Drake <fdrake at acm.org> wrote:
> On Wed, Dec 8, 2010 at 9:27 AM, Antoine Pitrou <solipsis at pitrou.net> wrote:
>> The thing is, they don't *want* to configure them, but you force them
>> to do some configuration if they don't want error messages to be
>> silenced.
>
> As I tried to explain earlier, a NullHandler doesn't silence anything
> except the message about logging not being configured. Propagation is
> not controlled by the handlers, but by the loggers.

If the only handler is NullHandler though, (or if there are no
handlers configured at all) then output does get silenced by default,
even errors and warning messages. The application has to (directly or
indirectly) configure logging with additional handlers in order for
the output to appear somewhere. That's the behaviour that is
inappropriate for use cases like the concurrent.futures handling of
unraisable errors that started this discussion over on the bug
tracker.

Cheers,
Nick.

-- 
Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia

From ncoghlan at gmail.com  Wed Dec  8 15:52:52 2010
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Thu, 9 Dec 2010 00:52:52 +1000
Subject: [Python-Dev] Using logging in the stdlib and its unit tests
In-Reply-To: <20101208152759.32b53cb2@pitrou.net>
References: <loom.20101207T212306-892@post.gmane.org>
	<20101207215053.025ecf52@pitrou.net>
	<loom.20101208T002100-91@post.gmane.org>
	<20101208010913.363c6105@pitrou.net>
	<loom.20101208T020511-443@post.gmane.org>
	<20101208110309.56cf2a22@pitrou.net>
	<loom.20101208T123754-508@post.gmane.org>
	<20101208152759.32b53cb2@pitrou.net>
Message-ID: <AANLkTi=Lpr8uzRj1Mm7oO_4y2wdUBoR_+0XD+fN868zV@mail.gmail.com>

On Thu, Dec 9, 2010 at 12:27 AM, Antoine Pitrou <solipsis at pitrou.net> wrote:
> On Wed, 8 Dec 2010 12:15:53 +0000 (UTC)
> Vinay Sajip <vinay_sajip at yahoo.co.uk> wrote:
>> See my comments to Nick Coghlan's post about getting messages out when you can't
>> raise an exception. I think the case is not as common as you suggest (because in
>> many instances, you would raise an exception to signal an error).
>
> I'm not talking specifically about exceptions, but about errors in
> general. If the case wasn't common, I'm not sure why the error() and
> critical() methods would exist at all.
>
> (of course I'm assuming error() is meant to output error messages. If
> that's a wrong interpretation then I'm a bit puzzled :-))

As I see it, there aren't many cases at the *library* level where
logging errors is more appropriate than raising exceptions:
- operations where there is no outer exception handler, and the
library knows it (e.g. callback processing)
- methods that are guaranteed not to raise exceptions (e.g. top level
event handlers)

Some batch processing operations may also work that way, but those are
a little more questionable (it may be better to report a result
summary and leave it up to the application to decide what to do with
any errors encountered along the way)

However, it *is* the case that logging is currently ill-suited to
tasks that are otherwise handled by writing directly to sys.stderr. I
suspect that has influenced some people's views of its suitability for
use in published libraries, even though it doesn't actually affect
most libraries (I suspect it is a significant chunk of what underlay
my reservations about using logging in library code). Since the
standard library *does* include modules that currently write to
stderr, then adjusting logging to make it better suited to this task
will help us as well as anyone else with similar problems.

That said, while I think Vinay's suggested "handler of last resort"
solution is a good one and something we should be doing for 3.2, I'm
also happy to let the idea bake for at least a few weeks.

Cheers,
Nick.

P.S. On a completely unrelated note, has anyone thought about creating
a write-only TextIO stream that outputs received writes via the
logging module?

-- 
Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia

From alexander.belopolsky at gmail.com  Wed Dec  8 17:28:30 2010
From: alexander.belopolsky at gmail.com (Alexander Belopolsky)
Date: Wed, 8 Dec 2010 11:28:30 -0500
Subject: [Python-Dev] Using logging in the stdlib and its unit tests
In-Reply-To: <AANLkTi=Lpr8uzRj1Mm7oO_4y2wdUBoR_+0XD+fN868zV@mail.gmail.com>
References: <loom.20101207T212306-892@post.gmane.org>
	<20101207215053.025ecf52@pitrou.net>
	<loom.20101208T002100-91@post.gmane.org>
	<20101208010913.363c6105@pitrou.net>
	<loom.20101208T020511-443@post.gmane.org>
	<20101208110309.56cf2a22@pitrou.net>
	<loom.20101208T123754-508@post.gmane.org>
	<20101208152759.32b53cb2@pitrou.net>
	<AANLkTi=Lpr8uzRj1Mm7oO_4y2wdUBoR_+0XD+fN868zV@mail.gmail.com>
Message-ID: <AANLkTimkAvhNf2OpR_1EQdnb-HmB0x9WzTnDSza9WS0Z@mail.gmail.com>

On Wed, Dec 8, 2010 at 9:52 AM, Nick Coghlan <ncoghlan at gmail.com> wrote:
..
> P.S. On a completely unrelated note, has anyone thought about creating
> a write-only TextIO stream that outputs received writes via the
> logging module?

I've done something similar for C++ iostreams many moons ago.  The
idea was to prepend all lines written to std::log with timestamp and
other information.   It worked reasonably well, but involved a lot of
C++ hackery at the time.   I think this is a great idea and would
allow existing libraries that use sys.stderr for messages start using
say the root logger instead when user redefines sys.stderr.

From robert.kern at gmail.com  Wed Dec  8 17:35:59 2010
From: robert.kern at gmail.com (Robert Kern)
Date: Wed, 08 Dec 2010 10:35:59 -0600
Subject: [Python-Dev] Using logging in the stdlib and its unit tests
In-Reply-To: <loom.20101208T093346-122@post.gmane.org>
References: <loom.20101207T212306-892@post.gmane.org>	<idme8q$9bm$1@dough.gmane.org>	<loom.20101208T004711-264@post.gmane.org>	<idmqpt$rbq$1@dough.gmane.org>
	<loom.20101208T093346-122@post.gmane.org>
Message-ID: <idoc5h$e5o$1@dough.gmane.org>

On 12/8/10 2:51 AM, Vinay Sajip wrote:
> Robert Kern<robert.kern<at>  gmail.com>  writes:
>
>>
>> I really don't understand how this view can be consistent with the
>> practice of adding NullHandler to loggers. If this message is so important
>> to prevent misconfiguration, then why should a library author decide to
>> silence it for his users?
>
> Because the application developer knows more about the end-user audience for
> their application, they are better placed to know how logging should work for
> their application. It's not an error for a particular application developer to
> decide that nothing should be produced by logging for a particular application;
> they (particularly when casual users) would be confused by the misconfiguration
> message due to logging by a library they're using.
>
> The library author's users are the application developers who use the library,
> not the end users who use their applications. Sometimes they're the same people,
> I know, but I just think of them as wearing different hats :-)

I'm sorry, but it's not at all clear that you have understood my point. There is 
no way for me to parse your words as a sensible reply to what I said.

Let's say I write a library called Foo. I want to add logging to my functions. 
You want to write an application called Bar that uses Foo and you want to 
configure logging for your application (at the very least to provide a default 
if not production). The warning is supposed to help you not make mistakes when 
configuring logging in your application. If I, library author, attach 
NullHandlers to all of Foo's loggers, then you will not get that warning if you 
forget to add handlers the Foo loggers. My adding the NullHandler to Foo 
prevented that warning that you consider to be so important.

I don't think the warning helps much, if at all.

>> [...] I strongly suspect that almost all configurations include a
>> catch-all root logger and that most of those *only* consist of that
>> root logger.
>
> That doesn't seem right: your comment might be conflating loggers with handlers.
> The common pattern would be (or should be) to name loggers according to __name__
> in the modules which use logging, but only configure *handlers* for the root
> logger. That way, logging messages indicate their origin (because of the
> __name__ convention) but you only need to add handlers at the root logger to
> capture all the logging information.

Yes. That's what I meant.

>> I think that boilerplate should be minimized. If using getLogger() should
>> almost always be followed by adding a NullHandler, then it should be the
>> default behavior. The easiest way to achieve this effect is to simply not
>> issue the warning message.
>
> getLogger() should NOT "almost always be followed by adding a NullHandler". For
> example, in Django, only the logger named "django" would have that handler
> added; no other Django logger (e.g. "django.db.models") would need to have that
> handler added.

In a large package (particularly a namespace package), I can't guarantee that 
any particular module will get imported. I will want to be able to import just 
foo.bar.baz without needing to worry about whether foo.setup_logging got 
imported and ran the logging configuration as a side-effect. I want to be able 
to loosen the coupling between modules across my package, not add more coupling.

But in any case, while adding a NullHandler to just the package's root logger 
helps you to avoid needing a NullHandler on every logger, the effect is the 
same. Almost all loggers effectively terminate in a NullHandler either directly 
or through a chain of parent loggers. So why not just make it the default?

Personally, I would back the proposals being made elsewhere in this thread, that 
in the absence of configuration, warnings and errors should be printed to stderr 
no matter where they come from. This gives useful behavior out-of-the-box 
without configuration but remains completely configurable. Library errors don't 
pass silently, but logging allows people to silence them explicitly. It 
separates the concerns of library authors (who should never touch logging 
configuration and shouldn't be required to think about it) from those of the 
application authors and application users.

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
  that is made terrible by our own mad attempt to interpret it as though it had
  an underlying truth."
   -- Umberto Eco


From vinay_sajip at yahoo.co.uk  Wed Dec  8 17:46:59 2010
From: vinay_sajip at yahoo.co.uk (Vinay Sajip)
Date: Wed, 8 Dec 2010 16:46:59 +0000 (UTC)
Subject: [Python-Dev] Using logging in the stdlib and its unit tests
References: <loom.20101207T212306-892@post.gmane.org>
	<20101207215053.025ecf52@pitrou.net>
	<loom.20101208T002100-91@post.gmane.org>
	<20101208010913.363c6105@pitrou.net>
	<loom.20101208T020511-443@post.gmane.org>
	<20101208110309.56cf2a22@pitrou.net>
	<loom.20101208T123754-508@post.gmane.org>
	<20101208152759.32b53cb2@pitrou.net>
	<AANLkTi=Lpr8uzRj1Mm7oO_4y2wdUBoR_+0XD+fN868zV@mail.gmail.com>
Message-ID: <loom.20101208T173828-875@post.gmane.org>

Nick Coghlan <ncoghlan <at> gmail.com> writes:

> That said, while I think Vinay's suggested "handler of last resort"
> solution is a good one and something we should be doing for 3.2, I'm
> also happy to let the idea bake for at least a few weeks.

I agree on the baking part, since it will allow time for any drawbacks to be
spotted or better solutions found. There are also the questions of what level
and format to use for the handler of last resort, before it can actually be
implemented.

> 
> P.S. On a completely unrelated note, has anyone thought about creating
> a write-only TextIO stream that outputs received writes via the
> logging module?
> 

Is this for use at the C level? At the Python level, there's a post I wrote a
while back which shows how to use a logger like an output stream:

http://plumberjack.blogspot.com/2009/09/how-to-treat-logger-like-output-stream.html

Regards,


Vinay Sajip


From ncoghlan at gmail.com  Wed Dec  8 18:28:36 2010
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Thu, 9 Dec 2010 03:28:36 +1000
Subject: [Python-Dev] Redirecting stdout/stderr to the logging module (was
 Re: Using logging in the stdlib and its unit tests)
Message-ID: <AANLkTimpOaCC75JPLadT=xghrKfKR4nA7rA=EUfByPOJ@mail.gmail.com>

On Thu, Dec 9, 2010 at 2:46 AM, Vinay Sajip <vinay_sajip at yahoo.co.uk> wrote:
> Nick Coghlan <ncoghlan <at> gmail.com> writes:
>> P.S. On a completely unrelated note, has anyone thought about creating
>> a write-only TextIO stream that outputs received writes via the
>> logging module?
>>
>
> Is this for use at the C level? At the Python level, there's a post I wrote a
> while back which shows how to use a logger like an output stream:
>
> http://plumberjack.blogspot.com/2009/09/how-to-treat-logger-like-output-stream.html

Similar in concept, but more full-featured (i.e. supporting the
relevant IO ABCs), allowing it to be used as a replacement for
stdout/stderr.

Cheers,
Nick.

-- 
Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia

From ncoghlan at gmail.com  Wed Dec  8 18:32:27 2010
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Thu, 9 Dec 2010 03:32:27 +1000
Subject: [Python-Dev] Using logging in the stdlib and its unit tests
In-Reply-To: <loom.20101208T173828-875@post.gmane.org>
References: <loom.20101207T212306-892@post.gmane.org>
	<20101207215053.025ecf52@pitrou.net>
	<loom.20101208T002100-91@post.gmane.org>
	<20101208010913.363c6105@pitrou.net>
	<loom.20101208T020511-443@post.gmane.org>
	<20101208110309.56cf2a22@pitrou.net>
	<loom.20101208T123754-508@post.gmane.org>
	<20101208152759.32b53cb2@pitrou.net>
	<AANLkTi=Lpr8uzRj1Mm7oO_4y2wdUBoR_+0XD+fN868zV@mail.gmail.com>
	<loom.20101208T173828-875@post.gmane.org>
Message-ID: <AANLkTikuujKCVjCWfjuFcr1-z05J=nTsRfyQw55MZheu@mail.gmail.com>

On Thu, Dec 9, 2010 at 2:46 AM, Vinay Sajip <vinay_sajip at yahoo.co.uk> wrote:
> Nick Coghlan <ncoghlan <at> gmail.com> writes:
>
>> That said, while I think Vinay's suggested "handler of last resort"
>> solution is a good one and something we should be doing for 3.2, I'm
>> also happy to let the idea bake for at least a few weeks.
>
> I agree on the baking part, since it will allow time for any drawbacks to be
> spotted or better solutions found. There are also the questions of what level
> and format to use for the handler of last resort, before it can actually be
> implemented.

As a starting point, I'd say warnings and above, no formatting (i.e.
just the message). To minimise bikeshedding, I'd like to be guided by
the idea that this is a more configurable alternative to printing
directly to stderr, but in the absence of application level
configuration, you wouldn't be able to tell which approach the library
was using just by looking at the program output.

Cheers,
Nick.

-- 
Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia

From vinay_sajip at yahoo.co.uk  Wed Dec  8 18:43:06 2010
From: vinay_sajip at yahoo.co.uk (Vinay Sajip)
Date: Wed, 8 Dec 2010 17:43:06 +0000 (UTC)
Subject: [Python-Dev] Using logging in the stdlib and its unit tests
References: <loom.20101207T212306-892@post.gmane.org>	<idme8q$9bm$1@dough.gmane.org>	<loom.20101208T004711-264@post.gmane.org>	<idmqpt$rbq$1@dough.gmane.org>
	<loom.20101208T093346-122@post.gmane.org>
	<idoc5h$e5o$1@dough.gmane.org>
Message-ID: <loom.20101208T174838-504@post.gmane.org>

Robert Kern <robert.kern <at> gmail.com> writes:

> I'm sorry, but it's not at all clear that you have understood my point.
> There is no way for me to parse your words as a sensible reply to what I
> said.
> 
> Let's say I write a library called Foo. I want to add logging to my
> functions. You want to write an application called Bar that uses Foo and
> you want to configure logging for your application (at the very least to
> provide a default if not production). The warning is supposed to help you
> not make mistakes when configuring logging in your application. If I,
> library author, attach NullHandlers to all of Foo's loggers, then you will
> not get that warning if you forget to add handlers the Foo loggers.
> My adding the NullHandler to Foo prevented that warning that you consider
> to be so important.
 
I understand, and what you say above is correct.

> I don't think the warning helps much, if at all.

Perhaps not. I'm not so hung up on the importance of the message now, but it
certainly *was* important when logging was first introduced, and users couldn't
expect to get the same level of help on comp.lang.python that they now can.
Today there are quite a few people who can help people with finger-trouble
logging issues.

Consider the scenarios when not having the current behaviour would bite you:

1. You're new to Python/new to logging. You write some code, perhaps across
several modules, which code makes logging calls, but you omit to configure any
handlers, whether through forgetting or not understanding what needs to be done.
Without the message, you've no idea why no logging messages appear, no matter
how much you fiddle with levels.
2. You write some code and decide you don't want to use logging, but use some
third party libraries which do. You don't care about that, so it's annoying to
have "no handlers could be found for logger XXX" messages printed to console.
You berate the library developer for their lack of consideration.

Perhaps you don't find yourself in these situations, but surely you sympathize
with people who do? How would you propose to address both those scenarios?

> In a large package (particularly a namespace package), I can't guarantee
> that any particular module will get imported. I will want to be able to
> import just foo.bar.baz without needing to worry about whether
> foo.setup_logging got imported and ran the logging configuration as a
> side-effect. I want to be able to loosen the coupling between modules
> across my package, not add more coupling.
> 

I'm not sure what coupling you're talking about - perhaps you can illustrate
with an example. If I develop a package "foo.bar" which is part of namespace
package "foo", and use loggers named __name__ in my code, and add a NullHandler
to logger "foo.bar", that's all I have to do. Likewise, if another person
develops "foo.baz" and they add a NullHandler to "foo.baz", then where's the
coupling between the two packages? They needn't even know about each other.

> But in any case, while adding a NullHandler to just the package's root logger
> helps you to avoid needing a NullHandler on every logger, the effect is the 
> same. Almost all loggers effectively terminate in a NullHandler either
> directly or through a chain of parent loggers. So why not just make it the
> default?
 

There's no "termination" when a NullHandler is encountered. Django has dozens of
modules, many of which could use logging, but only one NullHandler needs to be
added for the whole of Django. The effect you say is the same is not: when
adding new modules to Django, for example, no additional NullHandler adding
needs to be done.

I don't want to appear rude, but this seems to be another mistake (or perhaps a
variant of the same mistake as before) you are making about how logging works.
You obviously feel strongly about it, and if you have a specific use case which
is causing/has caused you pain, please spell it out for me (on comp.lang.python,
say) and I will try to help sort out the problem.

<offtopic>

As am off-topic example, Armin Ronacher kept on saying in various posts and
presentations that you couldn't use stdlib logging for web applications, that
there were fundamental problems with it. But when he actually sent me his
specific problem statement, I gave him a solution without spending too much time
on it (see
http://plumberjack.blogspot.com/2010/09/configuring-logging-for-web.html if you
care). I'm not trying to be obstructive, honestly.

</offtopic>

> Personally, I would back the proposals being made elsewhere in this thread,
> that in the absence of configuration, warnings and errors should be printed
> to stderr no matter where they come from. This gives useful behavior
> out-of-the-box

As you might have seen in my response to Nick's post, I've made specific
proposals about this myself. I agree with Nick's viee that some more time should
be given for discussion, suggestions, others to express views, and just for the
ideas to bake for a bit.

> without configuration but remains completely configurable. Library errors 
> don't pass silently, but logging allows people to silence them explicitly.
> It separates the concerns of library authors (who should never touch logging
> configuration and shouldn't be required to think about it) from those of the
> application authors and application users.

You surprise me with "library authors (who should never touch logging
configuration and shouldn't be required to think about it)". If you use logging,
you should think about it at least a little, just as with any other software you
use. For example, about default levels, propagation, etc. etc. The Django team,
for example, have expended a reasonable amount of thought on how best to add
logging support to their library; I don't believe their time has been wasted.

Regards,


Vinay Sajip


From v+python at g.nevcal.com  Wed Dec  8 18:51:44 2010
From: v+python at g.nevcal.com (Glenn Linderman)
Date: Wed, 08 Dec 2010 09:51:44 -0800
Subject: [Python-Dev] Using logging in the stdlib and its unit tests
In-Reply-To: <loom.20101208T123754-508@post.gmane.org>
References: <loom.20101207T212306-892@post.gmane.org>	<20101207215053.025ecf52@pitrou.net>	<loom.20101208T002100-91@post.gmane.org>	<20101208010913.363c6105@pitrou.net>	<loom.20101208T020511-443@post.gmane.org>	<20101208110309.56cf2a22@pitrou.net>
	<loom.20101208T123754-508@post.gmane.org>
Message-ID: <4CFFC5B0.3040006@g.nevcal.com>

On 12/8/2010 4:15 AM, Vinay Sajip wrote:
> You're complaining about too much documentation?! Don't measure it by weight!
On 12/8/2010 5:57 AM, Vinay Sajip wrote:
> Of course I understand I could be wrong
> about this, but I don't recall when a stdlib maintainer has said to me, "I want
> to start using logging in stdlib module XXX, but can't justify it because ..."

So I'm a fairly new Python user, but 30 years programming experience.

When I first looked at the Python logging facility (and again today, 
when I looked again), I found a simple example of debugging logging.  
Then a bunch of stuff about how to configure rotating logs.  Then 
something about logging levels. And then a dissertation on the theory of 
loggers "The logging library takes a modular approach and offers the 
several categories of components: loggers, handlers, filters, and 
formatters."  And I hadn't gotten through 10% of the documentation yet, 
per the scrollbar.

My reaction the first time was "Sounds sophisticated and complex.  I 
think I'll do something simpler for now, and maybe someday, when I have 
a spare week, I'll read the documentation and see if the benefits are 
worth the effort."

OK, so now you've discovered that too much documentation can be a turn 
off... at least, if it is presented from the top to describe the 
sophistication of the facility, rather than how easy it is to use (if it 
is, I still haven't gotten to 10%, and I still don't know that).

 From this thread, it sounds like it might be that logging could be easy 
to use.  Although part of my turn off from too much documentation and 
sophistication is a concern about the overhead of reading in and 
compiling a module that is that sophisticated for simple programs.

Now if I was convinced that

(1) it always gets read in anyway as part of Python startup (8% of the 
documentation didn't say that), so that no matter how simple my 
alternate facility, I pay the cost of loading the logger anyway, and

(2) that it would take less than a week to comprehend the basics of what 
I need to learn to use it, (in other words, if the 8% of the 
documentation I've read, actually gave sufficient simple examples to use 
applications and libraries, and left the sohistication and complexity 
and theory to later sections)

then I might have used the logger, instead of writing, in my personal 
library of "useful functions":

_logfh = sys.stderr
def log( text=None, textpre=None, filename=None, mode=None
          ):  # version 2010/11/11
     global _logfh
     if filename is not None:
         if mode is None:
             mode = "a"
         _logfh = open( filename, mode, **_wenc )
     if text is not None:
         if textpre is None:
             _logfh.write( str( text ) + "\n")
         else:
             _logfh.write( str( textpre ) + ": " + str( text ) + "\n")
     _logfh.flush()
     return _logfh


You see, I can create that faster than I read 8% of the documentation 
for logger, which didn't convince me that using the logger was better 
than the above, and it has been working fine... and I can tweak it when 
and if I need more sophistication.  I'm well aware that my function 
provides much less than what logger provides.  But in the first 8% of 
the documentation, I hadn't learned how to be able to use logging from a 
multiple module program in an easy and consistent manner...

The example shows a 4 line cost... it is not clear from the first 8% of 
the documentation if it requires 4 lines in every module; it is not 
clear if I can or should or if by default, the file I configure for one 
module is available to other modules, such that only two lines are 
necessary in those modules (import, logging.debug (or other levels of 
message)), etc.  And I'd have to read more than 8% of the documentation 
to find that out.

Now I'm not saying that I only read 8% of the documentation for modules 
that I want to use, in general, before I give up in disgust.  But I 
start reading, and estimate the cost of learning vs the cost of 
rewriting as I go.  logger documentation is large, so reading it all 
before starting to use it is unlikely; since the first part didn't get 
me started soon enough, I "put it on a shelf", where it still is.

If the first part had shown an example of how to use logger, in simple 
mode, in a trivial but multi-module application (because most trivial 
application do involve multiple modules, even if only one is 
user-written), then as I went along, I'd have likely learned more about 
the features as needed.  There are lots of interesting sounding features 
in the Table of Contents.  "Someday" I might decide to spend the week to 
read about them.

Hope this perspective helps, somehow.

Glenn

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20101208/ac576592/attachment.html>

From solipsis at pitrou.net  Wed Dec  8 19:15:37 2010
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Wed, 8 Dec 2010 19:15:37 +0100
Subject: [Python-Dev] Using logging in the stdlib and its unit tests
References: <loom.20101207T212306-892@post.gmane.org>
	<20101207215053.025ecf52@pitrou.net>
	<loom.20101208T002100-91@post.gmane.org>
	<20101208010913.363c6105@pitrou.net>
	<loom.20101208T020511-443@post.gmane.org>
	<20101208110309.56cf2a22@pitrou.net>
	<loom.20101208T123754-508@post.gmane.org>
	<20101208152759.32b53cb2@pitrou.net>
	<loom.20101208T154218-14@post.gmane.org>
Message-ID: <20101208191537.44fc3ad8@pitrou.net>

On Wed, 8 Dec 2010 14:54:09 +0000 (UTC)
Vinay Sajip <vinay_sajip at yahoo.co.uk> wrote:
> Antoine Pitrou <solipsis <at> pitrou.net> writes:
> 
> > I'm not talking specifically about exceptions, but about errors in
> > general. If the case wasn't common, I'm not sure why the error() and
> > critical() methods would exist at all.
> > 
> > (of course I'm assuming error() is meant to output error messages. If
> > that's a wrong interpretation then I'm a bit puzzled )
> > 
> 
> Then again, it's not always helpful when *library* code prints out messages to
> stderr. A user of the library might prefer that an error code be returned or an
> exception raised, so they can deal with the condition most appropriately.

Sorry, what are you arguing about and what kind of answer are you
expecting? Obviously error() is preferrable in some cases and other
means of communicating the error are preferrable in other cases. That's
kind of stating the obvious to me.

But since you are the one you wrote the library and added error() in
the first place, why are you trying to convince me that error() is
not useful? Perhaps you should explain how error() is supposed to be
used for if it's not supposed to log errors.

> > I was talking about the user of a library, not its developer (see the
> > snippet quoted above).
> 
> Yes, but see my response to that. A user of a library doesn't need to know the
> specifics of what loggers that library uses, they just need to make a positive
> statement of where they want warning and error messages go, and how they want
> them formatted.

And my point, again, is that they want it to work by default *without*
making such a positive statement. I'm not sure if I'm supposed to
explain this using different words since you're not appearing to get it.

> BTW we haven't discussed this yet - but these enabled-by-default messages, what
> format should they have? (e.g. just message, logger name + message, logger name
> + severity + message etc.) It might sound like a trivial point, but it also
> seems like whatever you pick, some bikeshed discussions would ensue.

I'm sorry but your way of argumenting is really puzzling to me. You are
trying to find counter-arguments against something which you *already*
implemented yourself in logging.error().

If you're interested in having a theoretical argument about what a
beautiful design should be (and if you want to argue about your own
design decisions), I'll leave the discussion here. I'm only interested
in the *practical* matter of logging having an useful behaviour by
default, which apparently you don't really care about. Fair enough,
but this whole thread is quite telling, IMO, about the general
impedance mismatch between the logging design and the expectations of
casual users (which is 99% of them, certainly).

Regards

Antoine.



From solipsis at pitrou.net  Wed Dec  8 19:18:07 2010
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Wed, 8 Dec 2010 19:18:07 +0100
Subject: [Python-Dev] Redirecting stdout/stderr to the logging module
 (was Re: Using logging in the stdlib and its unit tests)
References: <AANLkTimpOaCC75JPLadT=xghrKfKR4nA7rA=EUfByPOJ@mail.gmail.com>
Message-ID: <20101208191807.5979d53d@pitrou.net>

On Thu, 9 Dec 2010 03:28:36 +1000
Nick Coghlan <ncoghlan at gmail.com> wrote:

> On Thu, Dec 9, 2010 at 2:46 AM, Vinay Sajip <vinay_sajip at yahoo.co.uk> wrote:
> > Nick Coghlan <ncoghlan <at> gmail.com> writes:
> >> P.S. On a completely unrelated note, has anyone thought about creating
> >> a write-only TextIO stream that outputs received writes via the
> >> logging module?
> >>
> >
> > Is this for use at the C level? At the Python level, there's a post I wrote a
> > while back which shows how to use a logger like an output stream:
> >
> > http://plumberjack.blogspot.com/2009/09/how-to-treat-logger-like-output-stream.html
> 
> Similar in concept, but more full-featured (i.e. supporting the
> relevant IO ABCs), allowing it to be used as a replacement for
> stdout/stderr.

That would be an interesting idea. It would fit in the logging module
rather than the io module IMO.
However, there are some limitations due to the fact that logging is
line-based while people may output arbitrary text on stderr (perhaps
ASCII-formatted tables, who knows).

Regards

Antoine.



From vinay_sajip at yahoo.co.uk  Wed Dec  8 19:44:46 2010
From: vinay_sajip at yahoo.co.uk (Vinay Sajip)
Date: Wed, 8 Dec 2010 18:44:46 +0000 (UTC)
Subject: [Python-Dev] Redirecting stdout/stderr to the logging module
	(was Re: Using logging in the stdlib and its unit tests)
References: <AANLkTimpOaCC75JPLadT=xghrKfKR4nA7rA=EUfByPOJ@mail.gmail.com>
	<20101208191807.5979d53d@pitrou.net>
Message-ID: <loom.20101208T193553-525@post.gmane.org>

Antoine Pitrou <solipsis <at> pitrou.net> writes:

> However, there are some limitations due to the fact that logging is
> line-based while people may output arbitrary text on stderr (perhaps
> ASCII-formatted tables, who knows).

True, though the wrapper could easily buffer partial output internally to
support line-based output formats. Actually the line-based nature is a function
of the handler and/or formatter and just reflects what people tend to want most
of the time.

Regards,

Vinay Sajip



From p.f.moore at gmail.com  Wed Dec  8 19:49:52 2010
From: p.f.moore at gmail.com (Paul Moore)
Date: Wed, 8 Dec 2010 18:49:52 +0000
Subject: [Python-Dev] Using logging in the stdlib and its unit tests
In-Reply-To: <AANLkTi=Lpr8uzRj1Mm7oO_4y2wdUBoR_+0XD+fN868zV@mail.gmail.com>
References: <loom.20101207T212306-892@post.gmane.org>
	<20101207215053.025ecf52@pitrou.net>
	<loom.20101208T002100-91@post.gmane.org>
	<20101208010913.363c6105@pitrou.net>
	<loom.20101208T020511-443@post.gmane.org>
	<20101208110309.56cf2a22@pitrou.net>
	<loom.20101208T123754-508@post.gmane.org>
	<20101208152759.32b53cb2@pitrou.net>
	<AANLkTi=Lpr8uzRj1Mm7oO_4y2wdUBoR_+0XD+fN868zV@mail.gmail.com>
Message-ID: <AANLkTimkhukhxYGEuXJ5FPasHsHhWGzvfmgD9t27ffvN@mail.gmail.com>

On 8 December 2010 14:52, Nick Coghlan <ncoghlan at gmail.com> wrote:
> As I see it, there aren't many cases at the *library* level where
> logging errors is more appropriate than raising exceptions:

On a slightly tangential note, what do you think of the idea of
library code including info or debug level logging? In effect, tracing
and diagnostic code built in and available simply by changing the
logging level?

Paul.

From vinay_sajip at yahoo.co.uk  Wed Dec  8 20:00:31 2010
From: vinay_sajip at yahoo.co.uk (Vinay Sajip)
Date: Wed, 8 Dec 2010 19:00:31 +0000 (UTC)
Subject: [Python-Dev] Using logging in the stdlib and its unit tests
References: <loom.20101207T212306-892@post.gmane.org>
	<20101207215053.025ecf52@pitrou.net>
	<loom.20101208T002100-91@post.gmane.org>
	<20101208010913.363c6105@pitrou.net>
	<loom.20101208T020511-443@post.gmane.org>
	<20101208110309.56cf2a22@pitrou.net>
	<loom.20101208T123754-508@post.gmane.org>
	<20101208152759.32b53cb2@pitrou.net>
	<loom.20101208T154218-14@post.gmane.org>
	<20101208191537.44fc3ad8@pitrou.net>
Message-ID: <loom.20101208T194544-118@post.gmane.org>

Antoine Pitrou <solipsis <at> pitrou.net> writes:

> But since you are the one you wrote the library and added error() in
> the first place, why are you trying to convince me that error() is
> not useful? Perhaps you should explain how error() is supposed to be
> used for if it's not supposed to log errors.

I didn't say error() [or actually exception(), which you mentioned] was useless,
I merely pointed out what the normal pattern of usage was. Georg, at least,
seemed to be agreeing with me.

> And my point, again, is that they want it to work by default *without*
> making such a positive statement. I'm not sure if I'm supposed to
> explain this using different words since you're not appearing to get it.

No, I'm getting it, I think. See my responses to Nick's posts.
 
> I'm sorry but your way of argumenting is really puzzling to me. You are
> trying to find counter-arguments against something which you *already*
> implemented yourself in logging.error().

I don't think I am, so one of us is misunderstanding the other, or perhaps both
of us are misunderstanding each other.

> If you're interested in having a theoretical argument about what a
> beautiful design should be (and if you want to argue about your own
> design decisions), I'll leave the discussion here. I'm only interested

No, I'm not interested in that. Like you I'm interested in practical matters,
but unlike you I have to make some implementation choices, going forward - so it
shouldn't be surprising that I want to talk around the issue to make sure that
no wrong decision gets made. Sorry if that offends you because it sometimes
seems to be stating the (to you) obvious.

> in the *practical* matter of logging having an useful behaviour by
> default, which apparently you don't really care about.

Actually, I don't think my response to Nick's post (about concurrent.futures)
could be characterized as "I don't care", as I even made a specific proposal
about how a change could be implemented.

Regards,

Vinay Sajip



From vinay_sajip at yahoo.co.uk  Wed Dec  8 20:04:36 2010
From: vinay_sajip at yahoo.co.uk (Vinay Sajip)
Date: Wed, 8 Dec 2010 19:04:36 +0000 (UTC)
Subject: [Python-Dev] Using logging in the stdlib and its unit tests
References: <loom.20101207T212306-892@post.gmane.org>
	<20101207215053.025ecf52@pitrou.net>
	<loom.20101208T002100-91@post.gmane.org>
	<20101208010913.363c6105@pitrou.net>
	<loom.20101208T020511-443@post.gmane.org>
	<20101208110309.56cf2a22@pitrou.net>
	<loom.20101208T123754-508@post.gmane.org>
	<20101208152759.32b53cb2@pitrou.net>
	<AANLkTi=Lpr8uzRj1Mm7oO_4y2wdUBoR_+0XD+fN868zV@mail.gmail.com>
	<AANLkTimkhukhxYGEuXJ5FPasHsHhWGzvfmgD9t27ffvN@mail.gmail.com>
Message-ID: <loom.20101208T200123-470@post.gmane.org>

Paul Moore <p.f.moore <at> gmail.com> writes:

> 
> On 8 December 2010 14:52, Nick Coghlan <ncoghlan <at> gmail.com> wrote:
> > As I see it, there aren't many cases at the *library* level where
> > logging errors is more appropriate than raising exceptions:
> 
> On a slightly tangential note, what do you think of the idea of
> library code including info or debug level logging? In effect, tracing
> and diagnostic code built in and available simply by changing the
> logging level?

That's how it works right now. You get info() and debug() messages sent via
calls in library code, just by changing the level of (say) the root logger.

Regards,

Vinay Sajip


From vinay_sajip at yahoo.co.uk  Wed Dec  8 20:11:48 2010
From: vinay_sajip at yahoo.co.uk (Vinay Sajip)
Date: Wed, 8 Dec 2010 19:11:48 +0000 (UTC)
Subject: [Python-Dev] Using logging in the stdlib and its unit tests
References: <loom.20101207T212306-892@post.gmane.org>
	<20101207215053.025ecf52@pitrou.net>
	<loom.20101208T002100-91@post.gmane.org>
	<20101208010913.363c6105@pitrou.net>
	<loom.20101208T020511-443@post.gmane.org>
	<20101208110309.56cf2a22@pitrou.net>
	<loom.20101208T123754-508@post.gmane.org>
	<20101208152759.32b53cb2@pitrou.net>
	<AANLkTi=Lpr8uzRj1Mm7oO_4y2wdUBoR_+0XD+fN868zV@mail.gmail.com>
	<loom.20101208T173828-875@post.gmane.org>
	<AANLkTikuujKCVjCWfjuFcr1-z05J=nTsRfyQw55MZheu@mail.gmail.com>
Message-ID: <loom.20101208T200741-46@post.gmane.org>

Nick Coghlan <ncoghlan <at> gmail.com> writes:

> As a starting point, I'd say warnings and above, no formatting (i.e.
> just the message). To minimise bikeshedding, I'd like to be guided by
> the idea that this is a more configurable alternative to printing
> directly to stderr, but in the absence of application level
> configuration, you wouldn't be able to tell which approach the library
> was using just by looking at the program output.

Makes sense. I know it's only a small change at the implementation level but the
impact may be larger (due to it being a backwards-incompatible behaviour
change), and the little details need to be agreed, so does it make sense to
create a PEP about this? What do people think - is this bureaucratic overkill?

Regards,

Vinay Sajip



From skip at pobox.com  Wed Dec  8 20:14:25 2010
From: skip at pobox.com (skip at pobox.com)
Date: Wed, 8 Dec 2010 13:14:25 -0600
Subject: [Python-Dev] Using logging in the stdlib and its unit tests
In-Reply-To: <loom.20101208T200123-470@post.gmane.org>
References: <loom.20101207T212306-892@post.gmane.org>
	<20101207215053.025ecf52@pitrou.net>
	<loom.20101208T002100-91@post.gmane.org>
	<20101208010913.363c6105@pitrou.net>
	<loom.20101208T020511-443@post.gmane.org>
	<20101208110309.56cf2a22@pitrou.net>
	<loom.20101208T123754-508@post.gmane.org>
	<20101208152759.32b53cb2@pitrou.net>
	<AANLkTi=Lpr8uzRj1Mm7oO_4y2wdUBoR_+0XD+fN868zV@mail.gmail.com>
	<AANLkTimkhukhxYGEuXJ5FPasHsHhWGzvfmgD9t27ffvN@mail.gmail.com>
	<loom.20101208T200123-470@post.gmane.org>
Message-ID: <19711.55569.982114.312501@montanaro.dyndns.org>


    >> On a slightly tangential note, what do you think of the idea of
    >> library code including info or debug level logging? In effect,
    >> tracing and diagnostic code built in and available simply by changing
    >> the logging level?

    Vinay> That's how it works right now. You get info() and debug()
    Vinay> messages sent via calls in library code, just by changing the
    Vinay> level of (say) the root logger.

There can be performance implications if you log heavily.  I don't know how
the code is organized, but functionally these two calls are equivalent:

    >>> logging.error("error 1 2 3 %s" % "yup")
    ERROR:root:error 1 2 3 yup
    >>> logging.error("error 1 2 3 %s", "yup")
    ERROR:root:error 1 2 3 yup

The second form should be preferred in library code as long as the format
string expansion is deferred until after the test is made to emit the
message.

Skip

From solipsis at pitrou.net  Wed Dec  8 20:16:42 2010
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Wed, 8 Dec 2010 20:16:42 +0100
Subject: [Python-Dev] Using logging in the stdlib and its unit tests
References: <loom.20101207T212306-892@post.gmane.org>
	<20101207215053.025ecf52@pitrou.net>
	<loom.20101208T002100-91@post.gmane.org>
	<20101208010913.363c6105@pitrou.net>
	<loom.20101208T020511-443@post.gmane.org>
	<20101208110309.56cf2a22@pitrou.net>
	<loom.20101208T123754-508@post.gmane.org>
	<20101208152759.32b53cb2@pitrou.net>
	<loom.20101208T154218-14@post.gmane.org>
	<20101208191537.44fc3ad8@pitrou.net>
	<loom.20101208T194544-118@post.gmane.org>
Message-ID: <20101208201642.429851ff@pitrou.net>

On Wed, 8 Dec 2010 19:00:31 +0000 (UTC)
Vinay Sajip <vinay_sajip at yahoo.co.uk> wrote:
> 
> > If you're interested in having a theoretical argument about what a
> > beautiful design should be (and if you want to argue about your own
> > design decisions), I'll leave the discussion here. I'm only interested
> 
> No, I'm not interested in that. Like you I'm interested in practical matters,
> but unlike you I have to make some implementation choices, going forward - so it
> shouldn't be surprising that I want to talk around the issue to make sure that
> no wrong decision gets made. Sorry if that offends you because it sometimes
> seems to be stating the (to you) obvious.

Ok, I'm sorry for the harsh words.
I really hope this discussions leads to somewhere.

Regards

Antoine.



From vinay_sajip at yahoo.co.uk  Wed Dec  8 20:40:06 2010
From: vinay_sajip at yahoo.co.uk (Vinay Sajip)
Date: Wed, 8 Dec 2010 19:40:06 +0000 (UTC)
Subject: [Python-Dev] Using logging in the stdlib and its unit tests
References: <loom.20101207T212306-892@post.gmane.org>
	<20101207215053.025ecf52@pitrou.net>
	<loom.20101208T002100-91@post.gmane.org>
	<20101208010913.363c6105@pitrou.net>
	<loom.20101208T020511-443@post.gmane.org>
	<20101208110309.56cf2a22@pitrou.net>
	<loom.20101208T123754-508@post.gmane.org>
	<20101208152759.32b53cb2@pitrou.net>
	<loom.20101208T154218-14@post.gmane.org>
	<20101208191537.44fc3ad8@pitrou.net>
	<loom.20101208T194544-118@post.gmane.org>
	<20101208201642.429851ff@pitrou.net>
Message-ID: <loom.20101208T203913-21@post.gmane.org>

Antoine Pitrou <solipsis <at> pitrou.net> writes:


> Ok, I'm sorry for the harsh words.
> I really hope this discussions leads to somewhere.

No offence taken, and do do I :-)

Regards,

Vinay Sajip


From vinay_sajip at yahoo.co.uk  Wed Dec  8 20:42:00 2010
From: vinay_sajip at yahoo.co.uk (Vinay Sajip)
Date: Wed, 8 Dec 2010 19:42:00 +0000 (UTC)
Subject: [Python-Dev] Using logging in the stdlib and its unit tests
References: <loom.20101207T212306-892@post.gmane.org>
	<20101207215053.025ecf52@pitrou.net>
	<loom.20101208T002100-91@post.gmane.org>
	<20101208010913.363c6105@pitrou.net>
	<loom.20101208T020511-443@post.gmane.org>
	<20101208110309.56cf2a22@pitrou.net>
	<loom.20101208T123754-508@post.gmane.org>
	<20101208152759.32b53cb2@pitrou.net>
	<AANLkTi=Lpr8uzRj1Mm7oO_4y2wdUBoR_+0XD+fN868zV@mail.gmail.com>
	<AANLkTimkhukhxYGEuXJ5FPasHsHhWGzvfmgD9t27ffvN@mail.gmail.com>
	<loom.20101208T200123-470@post.gmane.org>
	<19711.55569.982114.312501@montanaro.dyndns.org>
Message-ID: <loom.20101208T204057-238@post.gmane.org>

 <skip <at> pobox.com> writes:

>     >>> logging.error("error 1 2 3 %s" % "yup")
>     ERROR:root:error 1 2 3 yup
>     >>> logging.error("error 1 2 3 %s", "yup")
>     ERROR:root:error 1 2 3 yup
> 
> The second form should be preferred in library code as long as the format
> string expansion is deferred until after the test is made to emit the
> message.

Yes, and the string expansion normally is deferred to happen as late as possible.

Regards,

Vinay



From vinay_sajip at yahoo.co.uk  Wed Dec  8 20:44:52 2010
From: vinay_sajip at yahoo.co.uk (Vinay Sajip)
Date: Wed, 8 Dec 2010 19:44:52 +0000 (UTC)
Subject: [Python-Dev] Using logging in the stdlib and its unit tests
References: <loom.20101207T212306-892@post.gmane.org>
	<20101207215053.025ecf52@pitrou.net>
	<loom.20101208T002100-91@post.gmane.org>
	<20101208010913.363c6105@pitrou.net>
	<loom.20101208T020511-443@post.gmane.org>
	<20101208110309.56cf2a22@pitrou.net>
	<loom.20101208T123754-508@post.gmane.org>
	<20101208152759.32b53cb2@pitrou.net>
	<loom.20101208T154218-14@post.gmane.org>
	<20101208191537.44fc3ad8@pitrou.net>
	<loom.20101208T194544-118@post.gmane.org>
	<20101208201642.429851ff@pitrou.net>
	<loom.20101208T203913-21@post.gmane.org>
Message-ID: <loom.20101208T204336-578@post.gmane.org>

Vinay Sajip <vinay_sajip <at> yahoo.co.uk> writes:


> No offence taken, and do do I 

s/do do/so do/

Perhaps it was a Freudian slip admitting that I *am* a dodo!

Regards,

Vinay



From kirklin.mcdonald at gmail.com  Wed Dec  8 21:09:22 2010
From: kirklin.mcdonald at gmail.com (Kirk McDonald)
Date: Wed, 8 Dec 2010 12:09:22 -0800
Subject: [Python-Dev] Status of PEP 3143?
Message-ID: <AANLkTin3_LSU2CsRNH_ZYvL9LB7AMjBQR=1ji1QFWgww@mail.gmail.com>

What is the status of PEP 3143? It appears to be aimed at 3.2, but there
does not appear to be a resolution on it.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20101208/0e6799e6/attachment.html>

From benjamin at python.org  Wed Dec  8 21:12:31 2010
From: benjamin at python.org (Benjamin Peterson)
Date: Wed, 8 Dec 2010 14:12:31 -0600
Subject: [Python-Dev] Status of PEP 3143?
In-Reply-To: <AANLkTin3_LSU2CsRNH_ZYvL9LB7AMjBQR=1ji1QFWgww@mail.gmail.com>
References: <AANLkTin3_LSU2CsRNH_ZYvL9LB7AMjBQR=1ji1QFWgww@mail.gmail.com>
Message-ID: <AANLkTinmOM1+RkWCk1MrjtdPJPbsrWgZH5pAB=QtOPYH@mail.gmail.com>

2010/12/8 Kirk McDonald <kirklin.mcdonald at gmail.com>:
> What is the status of PEP 3143? It appears to be aimed at 3.2, but there
> does not appear to be a resolution on it.

It's after beta freeze now, so definitely not 3.2. AFAIK, though, it's
never been discussed here.



-- 
Regards,
Benjamin

From robert.kern at gmail.com  Wed Dec  8 21:20:45 2010
From: robert.kern at gmail.com (Robert Kern)
Date: Wed, 08 Dec 2010 14:20:45 -0600
Subject: [Python-Dev] Using logging in the stdlib and its unit tests
In-Reply-To: <loom.20101208T174838-504@post.gmane.org>
References: <loom.20101207T212306-892@post.gmane.org>	<idme8q$9bm$1@dough.gmane.org>	<loom.20101208T004711-264@post.gmane.org>	<idmqpt$rbq$1@dough.gmane.org>	<loom.20101208T093346-122@post.gmane.org>	<idoc5h$e5o$1@dough.gmane.org>
	<loom.20101208T174838-504@post.gmane.org>
Message-ID: <idopav$jqg$1@dough.gmane.org>

If we're all converging on adding a "handler of last resort" and dropping the 
warning message, we can just let this branch of the thread drop. But if you want 
to continue, I already had most of the following already written. I hope it 
clears some things up more than it muddies them further. :-)

On 12/8/10 11:43 AM, Vinay Sajip wrote:
> Robert Kern<robert.kern<at>  gmail.com>  writes:

>> I don't think the warning helps much, if at all.
>
> Perhaps not. I'm not so hung up on the importance of the message now, but it
> certainly *was* important when logging was first introduced, and users couldn't
> expect to get the same level of help on comp.lang.python that they now can.
> Today there are quite a few people who can help people with finger-trouble
> logging issues.

I am not commenting on the reasonableness of the behavior when it was 
introduced, just what I think is the best behavior now.

> Consider the scenarios when not having the current behaviour would bite you:
>
> 1. You're new to Python/new to logging. You write some code, perhaps across
> several modules, which code makes logging calls, but you omit to configure any
> handlers, whether through forgetting or not understanding what needs to be done.
> Without the message, you've no idea why no logging messages appear, no matter
> how much you fiddle with levels.
> 2. You write some code and decide you don't want to use logging, but use some
> third party libraries which do. You don't care about that, so it's annoying to
> have "no handlers could be found for logger XXX" messages printed to console.
> You berate the library developer for their lack of consideration.
>
> Perhaps you don't find yourself in these situations, but surely you sympathize
> with people who do? How would you propose to address both those scenarios?

I am quite familiar with the latter in the third-party library author role. 
However, the messages I get aren't about lack of consideration but "What is 
wrong with my code? Are things broken?" They think the warning comes from my code.

As for the former, I'm not sure how that would cause much confusion. If I'm 
interested in getting the logged information, wouldn't I first find out how to 
configure logging? It's just about the very first thing you see reading the 
docs. It's one of the fundamental operations using the logging package. I can't 
imagine that this is a common failure mode. I think that the documentation 
ameliorates it better than the warning message.

>> In a large package (particularly a namespace package), I can't guarantee
>> that any particular module will get imported. I will want to be able to
>> import just foo.bar.baz without needing to worry about whether
>> foo.setup_logging got imported and ran the logging configuration as a
>> side-effect. I want to be able to loosen the coupling between modules
>> across my package, not add more coupling.
>
> I'm not sure what coupling you're talking about - perhaps you can illustrate
> with an example. If I develop a package "foo.bar" which is part of namespace
> package "foo", and use loggers named __name__ in my code, and add a NullHandler
> to logger "foo.bar", that's all I have to do. Likewise, if another person
> develops "foo.baz" and they add a NullHandler to "foo.baz", then where's the
> coupling between the two packages? They needn't even know about each other.

You were saying (in the part of the conversation that you snipped) that one 
would just add a NullHandler to the "django" logger to take care of messages 
sent to "django.db.models" and all of the other child loggers. *That* introduces 
a tighter coupling than I am comfortable with. All modules that use logging must 
make sure that the code that adds the NullHandler to the "django" logger gets 
executed. It's certainly reasonable to make that effort sometimes, but there are 
good reasons to avoid it, too. Just using logging should not force my hand.

If I do keep things decoupled, then I am adding a NullHandler to nearly every 
logger anyways, and I return to my original point that this is repetitive 
boilerplate and should be eliminated by appropriate defaults.

>> But in any case, while adding a NullHandler to just the package's root logger
>> helps you to avoid needing a NullHandler on every logger, the effect is the
>> same. Almost all loggers effectively terminate in a NullHandler either
>> directly or through a chain of parent loggers. So why not just make it the
>> default?
>
> There's no "termination" when a NullHandler is encountered.

All I meant is that it eventually reaches a NullHandler so no warning message is 
issued. Whether it actually stops there or not is irrelevant to my point. My 
point is that it is rarely the case that you deliberately want to have a logger 
which will cause the ?No handlers could be found" message. Whether you add the 
NullHandler to each logger or to one parent logger, you almost always want that 
behavior. That's why I think it should be the default behavior.

> Django has dozens of
> modules, many of which could use logging, but only one NullHandler needs to be
> added for the whole of Django. The effect you say is the same is not: when
> adding new modules to Django, for example, no additional NullHandler adding
> needs to be done.

The "effect" that I am talking about is that the log messages eventually get 
handled by a NullHandler and no warning message about missing handlers is sent. 
I'm not talking about the effect on developer effort.

The only thing adding a NullHandler accomplishes is to silence the warning. I 
think that accomplishing that effect in this manner is bad design. It's not 
obvious, particularly pre-2.7 where you had to roll your own NullHandler. It 
introduces a special case ("library authors should never add Handlers except 
they *should* add a NullHandler").

> I don't want to appear rude, but this seems to be another mistake (or perhaps a
> variant of the same mistake as before) you are making about how logging works.

It seems that I am being particularly unclear in stating my points. You also 
seem to be addressing small errors or misunderstandings in my statements at 
length and ignoring the substantive points. I understand that it is not 
intentional, but it is frustrating.

> You obviously feel strongly about it, and if you have a specific use case which
> is causing/has caused you pain, please spell it out for me (on comp.lang.python,
> say) and I will try to help sort out the problem.

I want to write library code that liberally uses logging. I want to avoid doing 
any configuration of logging in my library code. I do not want users of my 
library code to need to configure logging until they actually want to change the 
(sensible) defaults. Is that clear enough?

>> without configuration but remains completely configurable. Library errors
>> don't pass silently, but logging allows people to silence them explicitly.
>> It separates the concerns of library authors (who should never touch logging
>> configuration and shouldn't be required to think about it) from those of the
>> application authors and application users.
>
> You surprise me with "library authors (who should never touch logging
> configuration and shouldn't be required to think about it)". If you use logging,
> you should think about it at least a little, just as with any other software you
> use. For example, about default levels, propagation, etc. etc. The Django team,
> for example, have expended a reasonable amount of thought on how best to add
> logging support to their library; I don't believe their time has been wasted.

Fair enough. I don't disagree. Please drop "and shouldn't be required to think 
about it" from my statements.

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
  that is made terrible by our own mad attempt to interpret it as though it had
  an underlying truth."
   -- Umberto Eco


From g.brandl at gmx.net  Wed Dec  8 21:45:03 2010
From: g.brandl at gmx.net (Georg Brandl)
Date: Wed, 08 Dec 2010 21:45:03 +0100
Subject: [Python-Dev] Status of PEP 3143?
In-Reply-To: <AANLkTinmOM1+RkWCk1MrjtdPJPbsrWgZH5pAB=QtOPYH@mail.gmail.com>
References: <AANLkTin3_LSU2CsRNH_ZYvL9LB7AMjBQR=1ji1QFWgww@mail.gmail.com>
	<AANLkTinmOM1+RkWCk1MrjtdPJPbsrWgZH5pAB=QtOPYH@mail.gmail.com>
Message-ID: <idoqoj$q52$1@dough.gmane.org>

Am 08.12.2010 21:12, schrieb Benjamin Peterson:
> 2010/12/8 Kirk McDonald <kirklin.mcdonald at gmail.com>:
>> What is the status of PEP 3143? It appears to be aimed at 3.2, but there
>> does not appear to be a resolution on it.
> 
> It's after beta freeze now, so definitely not 3.2. AFAIK, though, it's
> never been discussed here.

I might add that now would be a good time to put any candidate library into
PyPI, in order to experience enough real-world usage in time for inclusion
in Python 3.3.

Georg


From tjreedy at udel.edu  Wed Dec  8 22:09:45 2010
From: tjreedy at udel.edu (Terry Reedy)
Date: Wed, 08 Dec 2010 16:09:45 -0500
Subject: [Python-Dev] Using logging in the stdlib and its unit tests
In-Reply-To: <loom.20101208T194544-118@post.gmane.org>
References: <loom.20101207T212306-892@post.gmane.org>	<20101207215053.025ecf52@pitrou.net>	<loom.20101208T002100-91@post.gmane.org>	<20101208010913.363c6105@pitrou.net>	<loom.20101208T020511-443@post.gmane.org>	<20101208110309.56cf2a22@pitrou.net>	<loom.20101208T123754-508@post.gmane.org>	<20101208152759.32b53cb2@pitrou.net>	<loom.20101208T154218-14@post.gmane.org>	<20101208191537.44fc3ad8@pitrou.net>
	<loom.20101208T194544-118@post.gmane.org>
Message-ID: <idos6q$27i$1@dough.gmane.org>

On 12/8/2010 2:00 PM, Vinay Sajip wrote:

> Actually, I don't think my response to Nick's post (about concurrent.futures)
> could be characterized as "I don't care", as I even made a specific proposal
> about how a change could be implemented.

Your proposal struck me as probably the best way forward. Can you code 
it up and put a patch on the tracker that people can test before the 
next beta?

A couple of other notes:
1. The number of participants in this thread is probably larger than 
average. So I think the responses so far can be taken as representative 
of those here who care enough to say something.
2. To me, Django is more an app framework than a mere 'library', and one 
with extra problems of error reporting in that it needs to send messages 
to both client and server and possibly elsewhere. So I as a user would 
expect it to embody extra thought about message disposition relative to 
a typical library.

-- 
Terry Jan Reedy


From ben+python at benfinney.id.au  Wed Dec  8 22:16:54 2010
From: ben+python at benfinney.id.au (Ben Finney)
Date: Thu, 09 Dec 2010 08:16:54 +1100
Subject: [Python-Dev] Status of PEP 3143?
References: <AANLkTin3_LSU2CsRNH_ZYvL9LB7AMjBQR=1ji1QFWgww@mail.gmail.com>
	<AANLkTinmOM1+RkWCk1MrjtdPJPbsrWgZH5pAB=QtOPYH@mail.gmail.com>
Message-ID: <87oc8wyng9.fsf@benfinney.id.au>

Benjamin Peterson <benjamin at python.org> writes:

> 2010/12/8 Kirk McDonald <kirklin.mcdonald at gmail.com>:
> > What is the status of PEP 3143? It appears to be aimed at 3.2, but
> > there does not appear to be a resolution on it.

I am the PEP 3143 author and champion. It hasn't been presented for a
resolution yet.

Development of the reference implementation is complete but not yet
suitable for standard library inclusion; progress in that direction has
stalled due to intermittent communication between the developers.

Further questions specifically about the PEP content or its
implementation to me directly, if you like. Or in ?comp.lang.python?, if
you prefer.

> It's after beta freeze now, so definitely not 3.2.

So it's not abandoned, but I don't know which version should be the
current target. What change should I make to the PEP in such a case?

-- 
 \          ?I used to be an airline pilot. I got fired because I kept |
  `\       locking the keys in the plane. They caught me on an 80 foot |
_o__)                    stepladder with a coathanger.? ?Steven Wright |
Ben Finney


From benjamin at python.org  Wed Dec  8 22:34:14 2010
From: benjamin at python.org (Benjamin Peterson)
Date: Wed, 8 Dec 2010 15:34:14 -0600
Subject: [Python-Dev] Status of PEP 3143?
In-Reply-To: <87oc8wyng9.fsf@benfinney.id.au>
References: <AANLkTin3_LSU2CsRNH_ZYvL9LB7AMjBQR=1ji1QFWgww@mail.gmail.com>
	<AANLkTinmOM1+RkWCk1MrjtdPJPbsrWgZH5pAB=QtOPYH@mail.gmail.com>
	<87oc8wyng9.fsf@benfinney.id.au>
Message-ID: <AANLkTi=TuSuStGHMGnGK5T9K0Qef5kZjdNZcTHONowzs@mail.gmail.com>

2010/12/8 Ben Finney <ben+python at benfinney.id.au>:
> Benjamin Peterson <benjamin at python.org> writes:
>
>> 2010/12/8 Kirk McDonald <kirklin.mcdonald at gmail.com>:
>> > What is the status of PEP 3143? It appears to be aimed at 3.2, but
>> > there does not appear to be a resolution on it.
>
> I am the PEP 3143 author and champion. It hasn't been presented for a
> resolution yet.
>
> Development of the reference implementation is complete but not yet
> suitable for standard library inclusion; progress in that direction has
> stalled due to intermittent communication between the developers.
>
> Further questions specifically about the PEP content or its
> implementation to me directly, if you like. Or in ?comp.lang.python?, if
> you prefer.
>
>> It's after beta freeze now, so definitely not 3.2.
>
> So it's not abandoned, but I don't know which version should be the
> current target. What change should I make to the PEP in such a case?

Put 3.3 or 3.x if you're thinking really long term. :)


-- 
Regards,
Benjamin

From p.f.moore at gmail.com  Thu Dec  9 00:06:46 2010
From: p.f.moore at gmail.com (Paul Moore)
Date: Wed, 8 Dec 2010 23:06:46 +0000
Subject: [Python-Dev] Using logging in the stdlib and its unit tests
In-Reply-To: <loom.20101208T200123-470@post.gmane.org>
References: <loom.20101207T212306-892@post.gmane.org>
	<20101207215053.025ecf52@pitrou.net>
	<loom.20101208T002100-91@post.gmane.org>
	<20101208010913.363c6105@pitrou.net>
	<loom.20101208T020511-443@post.gmane.org>
	<20101208110309.56cf2a22@pitrou.net>
	<loom.20101208T123754-508@post.gmane.org>
	<20101208152759.32b53cb2@pitrou.net>
	<AANLkTi=Lpr8uzRj1Mm7oO_4y2wdUBoR_+0XD+fN868zV@mail.gmail.com>
	<AANLkTimkhukhxYGEuXJ5FPasHsHhWGzvfmgD9t27ffvN@mail.gmail.com>
	<loom.20101208T200123-470@post.gmane.org>
Message-ID: <AANLkTik7ErHNHjjOKxKZtGbyRcxEUdTNjs+vte1V1OkN@mail.gmail.com>

On 8 December 2010 19:04, Vinay Sajip <vinay_sajip at yahoo.co.uk> wrote:
> Paul Moore <p.f.moore <at> gmail.com> writes:
>
>>
>> On 8 December 2010 14:52, Nick Coghlan <ncoghlan <at> gmail.com> wrote:
>> > As I see it, there aren't many cases at the *library* level where
>> > logging errors is more appropriate than raising exceptions:
>>
>> On a slightly tangential note, what do you think of the idea of
>> library code including info or debug level logging? In effect, tracing
>> and diagnostic code built in and available simply by changing the
>> logging level?
>
> That's how it works right now. You get info() and debug() messages sent via
> calls in library code, just by changing the level of (say) the root logger.

You misunderstand me. I know that's how those levels work. What I'm
not sure about (and I think would be interesting and potentially
useful information) is whether the individuals participating in this
thread feel that liberal use of info and debug level logging in
library code is useful. All the discussion thus far has been about
warning and error levels (mainly because people seem less comfortable
with the "suppress by default" approach for those levels, whereas it's
natural for debug and info levels).

I guess I'm wondering what best practices for logging might be.
(Actually, that might make the subject of an interesting blog posting,
if you're looking for suggestions :-)).

Paul.

From vinay_sajip at yahoo.co.uk  Thu Dec  9 00:35:34 2010
From: vinay_sajip at yahoo.co.uk (Vinay Sajip)
Date: Wed, 8 Dec 2010 23:35:34 +0000 (UTC)
Subject: [Python-Dev] Using logging in the stdlib and its unit tests
References: <loom.20101207T212306-892@post.gmane.org>
	<20101207215053.025ecf52@pitrou.net>
	<loom.20101208T002100-91@post.gmane.org>
	<20101208010913.363c6105@pitrou.net>
	<loom.20101208T020511-443@post.gmane.org>
	<20101208110309.56cf2a22@pitrou.net>
	<loom.20101208T123754-508@post.gmane.org>
	<20101208152759.32b53cb2@pitrou.net>
	<AANLkTi=Lpr8uzRj1Mm7oO_4y2wdUBoR_+0XD+fN868zV@mail.gmail.com>
	<AANLkTimkhukhxYGEuXJ5FPasHsHhWGzvfmgD9t27ffvN@mail.gmail.com>
	<loom.20101208T200123-470@post.gmane.org>
	<AANLkTik7ErHNHjjOKxKZtGbyRcxEUdTNjs+vte1V1OkN@mail.gmail.com>
Message-ID: <loom.20101209T003122-184@post.gmane.org>

Paul Moore <p.f.moore <at> gmail.com> writes:

> You misunderstand me. I know that's how those levels work. What I'm
> not sure about (and I think would be interesting and potentially
> useful information) is whether the individuals participating in this
> thread feel that liberal use of info and debug level logging in
> library code is useful. All the discussion thus far has been about
> warning and error levels (mainly because people seem less comfortable
> with the "suppress by default" approach for those levels, whereas it's
> natural for debug and info levels).


OK I see, sorry - I just didn't get that from your original choice of words. I'm
probably not firing on all cylinders right now :-)
 
> I guess I'm wondering what best practices for logging might be.
> (Actually, that might make the subject of an interesting blog posting,
> if you're looking for suggestions ).

Thanks for the suggestion, I'll think about posting on this topic after
ruminating on it for a bit.

Regards,

Vinay Sajip


From v+python at g.nevcal.com  Thu Dec  9 01:50:48 2010
From: v+python at g.nevcal.com (Glenn Linderman)
Date: Wed, 08 Dec 2010 16:50:48 -0800
Subject: [Python-Dev] Using logging in the stdlib and its unit tests
In-Reply-To: <loom.20101208T174838-504@post.gmane.org>
References: <loom.20101207T212306-892@post.gmane.org>	<idme8q$9bm$1@dough.gmane.org>	<loom.20101208T004711-264@post.gmane.org>	<idmqpt$rbq$1@dough.gmane.org>	<loom.20101208T093346-122@post.gmane.org>	<idoc5h$e5o$1@dough.gmane.org>
	<loom.20101208T174838-504@post.gmane.org>
Message-ID: <4D0027E8.5020808@g.nevcal.com>

On 12/8/2010 9:43 AM, Vinay Sajip wrote:
> <offtopic>
>
> As am off-topic example, Armin Ronacher kept on saying in various posts and
> presentations that you couldn't use stdlib logging for web applications, that
> there were fundamental problems with it. But when he actually sent me his
> specific problem statement, I gave him a solution without spending too much time
> on it (see
> http://plumberjack.blogspot.com/2010/09/configuring-logging-for-web.html  if you
> care). I'm not trying to be obstructive, honestly.
>
> </offtopic>

Since my application is also web server related, I was curious enough to 
look at this blog posting to see if it would be a good starter example 
that is missing from the manual.  I don't think it is, as it is somewhat 
complex, although it is multi-module, and even multi-thread, so it might 
make a good example for other parts of the documentation (but maybe they 
already have good examples; I've only looked at the first 8%).

In trying to understand it, I couldn't figure out why the WebApp class 
needs to keep track of the threads that are running the that web app.  
Why isn't it good enough for the thread to know the name of the app?  
The filter copies the appName from the thread to the record; the 
InjectingFilter.filter not replace

         record.appName  =  tlocal.appName
         tname  =  threading.currentThread().getName()
         return  tname  in  self.app.threads

by

	record.appName = tlocal.appName
	return appName == self.app.name

and get the same effect, without needing to mainting the self.threads "database"?

Or what am I missing?

Glenn


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20101208/bebf354b/attachment.html>

From python at mrabarnett.plus.com  Thu Dec  9 03:56:40 2010
From: python at mrabarnett.plus.com (MRAB)
Date: Thu, 09 Dec 2010 02:56:40 +0000
Subject: [Python-Dev] Can't compile regex module with Python 3.2
Message-ID: <4D004568.4000905@mrabarnett.plus.com>

The regex module calls _PyUnicode_IsWhitespace, which is mapped by
unicodeobject.h to either _PyUnicodeUCS2_IsWhitespace or
_PyUnicodeUCS4_IsWhitespace.

 From Python 2.5 to Python 3.1 the library pythonXX.lib contains either
_PyUnicodeUCS2_IsWhitespace or _PyUnicodeUCS4_IsWhitespace.

However, in Python 3.2b1 the library python32.lib contains only
_PyUnicode_IsWhitespace, therefore breaking the build.

Is this change intentional? If so, why does unicodeobject.h still do
the mapping?

Windows XP Pro, 32-bit.

From merwok at netwok.org  Thu Dec  9 04:45:37 2010
From: merwok at netwok.org (=?UTF-8?B?w4lyaWMgQXJhdWpv?=)
Date: Thu, 09 Dec 2010 04:45:37 +0100
Subject: [Python-Dev] [Python-checkins] r86855 - in
 python/branches/py3k: Doc/library/unittest.rst Lib/unittest/case.py
In-Reply-To: <20101129013825.EF8DAEEB1E@mail.python.org>
References: <20101129013825.EF8DAEEB1E@mail.python.org>
Message-ID: <4D0050E1.3000707@netwok.org>

Hello,

> Author: raymond.hettinger
> Date: Mon Nov 29 02:38:25 2010
> New Revision: 86855
> Log: Do not add an obsolete unittest name to Py3.2.

> Modified: python/branches/py3k/Lib/unittest/case.py
> -    # Old name for assertCountEqual()
> -    assertItemsEqual = assertCountEqual

When we merge distutils2 back into the stdlib, our tests will have to
work with stdlib unittest in 3.3 or unittest2 for older Pythons (we?ll
still make standalone releases for them).  unittest2 doesn?t have
assertCountEqual, unittest in 3.2+ doesn?t have assertItemsEqual, what?s
the plan for compatibility?

Regards


From martin at v.loewis.de  Thu Dec  9 06:47:52 2010
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Thu, 09 Dec 2010 06:47:52 +0100
Subject: [Python-Dev] Can't compile regex module with Python 3.2
In-Reply-To: <4D004568.4000905@mrabarnett.plus.com>
References: <4D004568.4000905@mrabarnett.plus.com>
Message-ID: <4D006D88.10300@v.loewis.de>

Am 09.12.2010 03:56, schrieb MRAB:
> The regex module calls _PyUnicode_IsWhitespace, which is mapped by
> unicodeobject.h to either _PyUnicodeUCS2_IsWhitespace or
> _PyUnicodeUCS4_IsWhitespace.
> 
> From Python 2.5 to Python 3.1 the library pythonXX.lib contains either
> _PyUnicodeUCS2_IsWhitespace or _PyUnicodeUCS4_IsWhitespace.
> 
> However, in Python 3.2b1 the library python32.lib contains only
> _PyUnicode_IsWhitespace, therefore breaking the build.
> 
> Is this change intentional? If so, why does unicodeobject.h still do
> the mapping?

Are you sure about this? It's not intentional (except in the limited ABI).

Regards,
Martin

From alexander.belopolsky at gmail.com  Thu Dec  9 06:57:04 2010
From: alexander.belopolsky at gmail.com (Alexander Belopolsky)
Date: Thu, 9 Dec 2010 00:57:04 -0500
Subject: [Python-Dev] Can't compile regex module with Python 3.2
In-Reply-To: <4D006D88.10300@v.loewis.de>
References: <4D004568.4000905@mrabarnett.plus.com> <4D006D88.10300@v.loewis.de>
Message-ID: <AANLkTikoFuJzZt6BCm=HQnzpwKc3BRnTgu8KcubOTVow@mail.gmail.com>

On Thu, Dec 9, 2010 at 12:47 AM, "Martin v. L?wis" <martin at v.loewis.de> wrote:
..
>> However, in Python 3.2b1 the library python32.lib contains only
>> _PyUnicode_IsWhitespace, therefore breaking the build.
>>
>> Is this change intentional? If so, why does unicodeobject.h still do
>> the mapping?
>
> Are you sure about this? It's not intentional (except in the limited ABI).

this does seem to be intentional:


$ svn log -r 84177
------------------------------------------------------------------------
r84177 | amaury.forgeotdarc | 2010-08-18 16:44:58 -0400 (Wed, 18 Aug
2010) | 9 lines

#5127: Even on narrow unicode builds, the C functions that access the Unicode
Database (Py_UNICODE_TOLOWER, Py_UNICODE_ISDECIMAL, and others) now accept
and return characters from the full Unicode range (Py_UCS4).

The differences from Python code are few:
- unicodedata.numeric(), unicodedata.decimal() and unicodedata.digit()
  now return the correct value for large code points
- repr() may consider more characters as printable.

------------------------------------------------------------------------
http://svn.python.org/view?view=rev&revision=84177

From ncoghlan at gmail.com  Thu Dec  9 06:57:16 2010
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Thu, 9 Dec 2010 15:57:16 +1000
Subject: [Python-Dev] Using logging in the stdlib and its unit tests
In-Reply-To: <AANLkTimkhukhxYGEuXJ5FPasHsHhWGzvfmgD9t27ffvN@mail.gmail.com>
References: <loom.20101207T212306-892@post.gmane.org>
	<20101207215053.025ecf52@pitrou.net>
	<loom.20101208T002100-91@post.gmane.org>
	<20101208010913.363c6105@pitrou.net>
	<loom.20101208T020511-443@post.gmane.org>
	<20101208110309.56cf2a22@pitrou.net>
	<loom.20101208T123754-508@post.gmane.org>
	<20101208152759.32b53cb2@pitrou.net>
	<AANLkTi=Lpr8uzRj1Mm7oO_4y2wdUBoR_+0XD+fN868zV@mail.gmail.com>
	<AANLkTimkhukhxYGEuXJ5FPasHsHhWGzvfmgD9t27ffvN@mail.gmail.com>
Message-ID: <AANLkTinsCE3gJG5LM2RvwL9sVTF6p8O=6wCVwbwYa_1Z@mail.gmail.com>

On Thu, Dec 9, 2010 at 4:49 AM, Paul Moore <p.f.moore at gmail.com> wrote:
> On 8 December 2010 14:52, Nick Coghlan <ncoghlan at gmail.com> wrote:
>> As I see it, there aren't many cases at the *library* level where
>> logging errors is more appropriate than raising exceptions:
>
> On a slightly tangential note, what do you think of the idea of
> library code including info or debug level logging? In effect, tracing
> and diagnostic code built in and available simply by changing the
> logging level?

It's a trade-off between performance and the ability to diagnose
faults and what you want to do will vary by application and problem
domain. My background is in working on distributed projects where we
need enough diagnostic information to debug one-off failures of the
live system, so I'm in favour of the idea in general, but doing it
*can* carry a fairly hefty speed cost (you can ameliorate that to some
degree by handing off the actual I/O to a separate thread, but the
system would still be faster if the log messages weren't there).

As Skip says, you definitely want to take advantage of the lazy string
formatting when doing this with the Python logging module.

Cheers,
Nick.

-- 
Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia

From vinay_sajip at yahoo.co.uk  Thu Dec  9 09:26:35 2010
From: vinay_sajip at yahoo.co.uk (Vinay Sajip)
Date: Thu, 9 Dec 2010 08:26:35 +0000 (UTC)
Subject: [Python-Dev] Using logging in the stdlib and its unit tests
References: <loom.20101207T212306-892@post.gmane.org>	<idme8q$9bm$1@dough.gmane.org>	<loom.20101208T004711-264@post.gmane.org>	<idmqpt$rbq$1@dough.gmane.org>	<loom.20101208T093346-122@post.gmane.org>	<idoc5h$e5o$1@dough.gmane.org>
	<loom.20101208T174838-504@post.gmane.org>
	<4D0027E8.5020808@g.nevcal.com>
Message-ID: <loom.20101209T091738-637@post.gmane.org>

Glenn Linderman <v+python <at> g.nevcal.com> writes:

> Or what am I missing?

That threads are not necessarily dedicated to apps, in a real world setting.
Depending on the server implementation, a single thread could be asked to handle
requests for different apps over its lifetime. So the only way of knowing which
threads are currently servicing a particular app is to maintain a set of them.

Regards,

Vinay Sajip



From vinay_sajip at yahoo.co.uk  Thu Dec  9 09:39:13 2010
From: vinay_sajip at yahoo.co.uk (Vinay Sajip)
Date: Thu, 9 Dec 2010 08:39:13 +0000 (UTC)
Subject: [Python-Dev] Using logging in the stdlib and its unit tests
References: <loom.20101207T212306-892@post.gmane.org>	<idme8q$9bm$1@dough.gmane.org>	<loom.20101208T004711-264@post.gmane.org>	<idmqpt$rbq$1@dough.gmane.org>	<loom.20101208T093346-122@post.gmane.org>	<idoc5h$e5o$1@dough.gmane.org>
	<loom.20101208T174838-504@post.gmane.org>
	<4D0027E8.5020808@g.nevcal.com>
	<loom.20101209T091738-637@post.gmane.org>
Message-ID: <loom.20101209T093511-634@post.gmane.org>

Vinay Sajip <vinay_sajip <at> yahoo.co.uk> writes:

> 
> Glenn Linderman <v+python <at> g.nevcal.com> writes:
> 
> > Or what am I missing?
> 

And one more thing: the filters for *both* apps are called for a given request.
One will return True, the other will return False.

Bear in mind that the intention of the post is to be didactic, so it's possible
there are some improvements that could be made when implementing for production.

Regards,

Vinay Sajip


From v+python at g.nevcal.com  Thu Dec  9 10:29:51 2010
From: v+python at g.nevcal.com (Glenn Linderman)
Date: Thu, 09 Dec 2010 01:29:51 -0800
Subject: [Python-Dev] Using logging in the stdlib and its unit tests
In-Reply-To: <loom.20101209T091738-637@post.gmane.org>
References: <loom.20101207T212306-892@post.gmane.org>	<idme8q$9bm$1@dough.gmane.org>	<loom.20101208T004711-264@post.gmane.org>	<idmqpt$rbq$1@dough.gmane.org>	<loom.20101208T093346-122@post.gmane.org>	<idoc5h$e5o$1@dough.gmane.org>	<loom.20101208T174838-504@post.gmane.org>	<4D0027E8.5020808@g.nevcal.com>
	<loom.20101209T091738-637@post.gmane.org>
Message-ID: <4D00A18F.60208@g.nevcal.com>

On 12/9/2010 12:26 AM, Vinay Sajip wrote:
> Glenn Linderman<v+python<at>  g.nevcal.com>  writes:
>> >  Or what am I missing?
> That threads are not necessarily dedicated to apps, in a real world setting.
> Depending on the server implementation, a single thread could be asked to handle
> requests for different apps over its lifetime. So the only way of knowing which
> threads are currently servicing a particular app is to maintain a set of them.

Agreed, they are not necessarily dedicated to apps.  But while they are 
running the app, they have the appname in their thread local storage, 
no?    So if a thread has the appname in its thread local storage, is it 
not servicing that app, at that moment?  And if it is, why is that 
insufficient?  That is my question, and you've sidestepped it.


> And one more thing: the filters for*both*  apps are called for a given request.
> One will return True, the other will return False.
>
> Bear in mind that the intention of the post is to be didactic, so it's possible
> there are some improvements that could be made when implementing for production.

OK, I just learned what "didactic" meant, so you've taught me something.

I understood that both filters would be called.  I understood, that in 
production, it would probably not be necessary to log messages to both 
the app log and the global log, that the global log would be there just 
for things that are not app-specific.

But I still don't see a need to maintain a set of threads running the 
app in the app object, if the app keeps track of what app it is running, 
in a spot that is accessible to the filter (which it seems to be).  I 
don't see how it is beneficial to keep track of two separate data 
structures that could serve the same purpose.

I realize you designed this in 10 minutes, and probably took twice that 
long to write it up, so didn't necessarily analyze it for efficiency.  
But I'm asking for that analysis now, if there is any real need for the 
app to track the set of threads, _for the purpose of the problem being 
solved_?  I understand there might be other reasons for which that might 
be useful, but for the logging it simply seems to be inefficient 
redundancy... and if it isn't, then I don't understand the example, yet, 
and I'm trying to.

So since you hand-waved, instead of giving a straight answer, and then 
maybe your second message was an attempt to back-pedal a bit, not sure, 
I went ahead and downloaded your code, made changes to remove the set of 
threads as outlined (see modified functions below), and it seems to run 
just as correctly. I thought it was an obvious question, while trying to 
understand the code, and maybe learn about the logger, and I guess I 
have, a little.  And maybe some other things too.  Please further 
explain if I am still missing something.  Under what conditions of the 
problem you were solving does the code below fail?

class InjectingFilter(logging.Filter):
     def __init__(self, app):
         self.app = app

     def filter(self, record):
         record.method = tlocal.request.method
         record.ip = tlocal.request.ip
         record.appName = tlocal.appName
         return record.appName == self.app.name
         # tname = threading.currentThread().getName()
         # return tname in self.app.threads

class WebApp:
     def __init__(self, name):
         self.name = name
         self.threads = set()
         handler = logging.FileHandler(name + '.log', 'w')
         f = InjectingFilter(self)
         handler.setFormatter(formatter)
         handler.addFilter(f)
         root.addHandler(handler)
         self.num_requests = 0

     def process_request(self, request):
         tlocal.request = request
         tlocal.appName = self.name
         # tname = threading.currentThread().getName()
         # self.threads.add(tname)
         self.num_requests += 1
         try:
             logger.debug('Request processing started')
             webapplib.useful()
             logger.debug('Request processing finished')
         finally:
             pass
             # self.threads.remove(tname)

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20101209/0097cd7e/attachment-0001.html>

From vinay_sajip at yahoo.co.uk  Thu Dec  9 10:52:13 2010
From: vinay_sajip at yahoo.co.uk (Vinay Sajip)
Date: Thu, 9 Dec 2010 09:52:13 +0000 (UTC)
Subject: [Python-Dev] Using logging in the stdlib and its unit tests
References: <loom.20101207T212306-892@post.gmane.org>	<idme8q$9bm$1@dough.gmane.org>	<loom.20101208T004711-264@post.gmane.org>	<idmqpt$rbq$1@dough.gmane.org>	<loom.20101208T093346-122@post.gmane.org>	<idoc5h$e5o$1@dough.gmane.org>	<loom.20101208T174838-504@post.gmane.org>	<4D0027E8.5020808@g.nevcal.com>
	<loom.20101209T091738-637@post.gmane.org>
	<4D00A18F.60208@g.nevcal.com>
Message-ID: <loom.20101209T104731-336@post.gmane.org>

Glenn Linderman <v+python <at> g.nevcal.com> writes:


>     Agreed, they are not necessarily dedicated to apps.? But while they
>     are running the app, they have the appname in their thread local
>     storage, no???? So if a thread has the appname in its thread local
>     storage, is it not servicing that app, at that moment?? And if it
>     is, why is that insufficient?? That is my question, and you've
>     sidestepped it.

Sorry, I didn't mean to side-step: it's just that when reading your earlier post
(on the Web using Gmane), all the lines of Python in your post ran together, so
I didn't quite see what you were getting at.

Having looked at it more carefully, you're right: the threads set is not needed.
You can just "return tlocal.appName == self.app.name". I missed that, so thanks
for teaching me something, too :-)

Regards,

Vinay Sajip


From fuzzyman at voidspace.org.uk  Thu Dec  9 11:57:27 2010
From: fuzzyman at voidspace.org.uk (Michael Foord)
Date: Thu, 09 Dec 2010 10:57:27 +0000
Subject: [Python-Dev] [Python-checkins] r86855 - in
 python/branches/py3k: Doc/library/unittest.rst Lib/unittest/case.py
In-Reply-To: <4D0050E1.3000707@netwok.org>
References: <20101129013825.EF8DAEEB1E@mail.python.org>
	<4D0050E1.3000707@netwok.org>
Message-ID: <4D00B617.5090602@voidspace.org.uk>

On 09/12/2010 03:45, ?ric Araujo wrote:
> Hello,
>
>> Author: raymond.hettinger
>> Date: Mon Nov 29 02:38:25 2010
>> New Revision: 86855
>> Log: Do not add an obsolete unittest name to Py3.2.
>> Modified: python/branches/py3k/Lib/unittest/case.py
>> -    # Old name for assertCountEqual()
>> -    assertItemsEqual = assertCountEqual
> When we merge distutils2 back into the stdlib, our tests will have to
> work with stdlib unittest in 3.3 or unittest2 for older Pythons (we?ll
> still make standalone releases for them).  unittest2 doesn?t have
> assertCountEqual, unittest in 3.2+ doesn?t have assertItemsEqual, what?s
> the plan for compatibility?

unittest2 will continue to track changes in unittest. A 0.6.0 release is 
planned, with feature parity with the version of unittest in Python 3.2. 
Of course if you want to help that would be much appreciated. :-)

All the best,

Michael Foord

> Regards


-- 

http://www.voidspace.org.uk/

READ CAREFULLY. By accepting and reading this email you agree,
on behalf of your employer, to release me from all obligations
and waivers arising from any and all NON-NEGOTIATED agreements,
licenses, terms-of-service, shrinkwrap, clickwrap, browsewrap,
confidentiality, non-disclosure, non-compete and acceptable use
policies (?BOGUS AGREEMENTS?) that I have entered into with your
employer, its partners, licensors, agents and assigns, in
perpetuity, without prejudice to my ongoing rights and privileges.
You further represent that you have the authority to release me
from any BOGUS AGREEMENTS on behalf of your employer.


From tnagyemail-mail at yahoo.fr  Thu Dec  9 13:26:02 2010
From: tnagyemail-mail at yahoo.fr (Thomas Nagy)
Date: Thu, 9 Dec 2010 12:26:02 +0000 (GMT)
Subject: [Python-Dev] futures API
Message-ID: <330887.27553.qm@web27506.mail.ukl.yahoo.com>

Hello,

I am looking forward to replacing a piece of code (http://code.google.com/p/waf/source/browse/trunk/waflib/Runner.py#86) by the futures module which was announced in python 3.2 beta. I am a bit stuck with it, so I have a few questions about the futures:

1. Is the futures API frozen?
2. How hard would it be to return the tasks processed in an output queue to process/consume the results while they are returned? The code does not seem to be very open for monkey patching.
3. How hard would it be to add new tasks dynamically (after a task is executed) and have the futures object never complete?
4. Is there a performance evaluation of the futures code (execution overhead) ?

Thanks,
Thomas



      

From ocean-city at m2.ccsnet.ne.jp  Thu Dec  9 13:49:49 2010
From: ocean-city at m2.ccsnet.ne.jp (Hirokazu Yamamoto)
Date: Thu, 09 Dec 2010 21:49:49 +0900
Subject: [Python-Dev] OpenSSL Vulnerability (openssl-1.0.0a)
In-Reply-To: <20101124162312.2109.1025683352.divmod.xquotient.215@localhost.localdomain>
References: <4CEB3F72.7000006@m2.ccsnet.ne.jp>	<AANLkTi=P2_4U2X+G6RQ6=T9yvxNZ1keCgXt+=+-34ajX@mail.gmail.com>	<20101123150219.29e20374@pitrou.net>	<720EFE43-119F-4F2F-BCB1-939275B5FA6E@twistedmatrix.com>	<1290585733.3642.2.camel@localhost.localdomain>	<20101124150106.2109.660794265.divmod.xquotient.197@localhost.localdomain>	<20101124161120.5ddd106c@pitrou.net>
	<20101124162312.2109.1025683352.divmod.xquotient.215@localhost.localdomain>
Message-ID: <4D00D06D.5060201@m2.ccsnet.ne.jp>

On 2010/11/25 1:23, exarkun at twistedmatrix.com wrote:
> Ah. Okay, then Python 3.2 would be vulnerable. Good thing it isn't
> released yet. ;)

It seems OpenSSL 1.0.0c out.

http://openssl.org/news/secadv_20101202.txt

 > 02-Dec-2010: 	   Security Advisory: ciphersuite downgrade fix
 > 02-Dec-2010: 	   OpenSSL 1.0.0c is now available, including important 
 > bug and security fixes
 > 02-Dec-2010: 	   OpenSSL 0.9.8q is now available, including important 
 > bug and security fixes

From alexander.belopolsky at gmail.com  Thu Dec  9 15:51:36 2010
From: alexander.belopolsky at gmail.com (Alexander Belopolsky)
Date: Thu, 9 Dec 2010 09:51:36 -0500
Subject: [Python-Dev] The fate of transform() and untransform() methods
Message-ID: <AANLkTimCi7Ovz3Yxc7__3ohWKjGMrKZxuPmuSkQ7POjy@mail.gmail.com>

On Fri, Dec 3, 2010 at 1:05 PM, Guido van Rossum <guido at python.org> wrote:
> On Fri, Dec 3, 2010 at 9:58 AM, R. David Murray <rdmurray at bitdance.com> wrote:
..
>> I believe MAL's thought was that the addition of these methods had
>> been approved pre-moratorium, but I don't know if that is a
>> sufficient argument or not.
>
> It is not.
>
> The moratorium is intended to freeze the state of the language as
> implemented, not whatever was discussed and approved but didn't get
> implemented (that'd be a hole big enough to drive a truck through, as
> the saying goes :-).
>
> Regardless of what I or others may have said before, I am not
> currently a fan of adding transform() to either str or bytes.
>

I would like to restart the discussion under a separate subject
because the original thread [1] went off the specific topic of the six
new methods (2 methods x 3 types) added to builtins shortly before 3.2
beta was released. [2]  The ticket that introduced the change is
currently closed [3] even though the last message suggests that at
least part of the change needs to be reverted.

Note that reverting  part of the patch is not entirely trivial because
new codecs' documentation refers to bytes.[un]transform() both in the
docstrings and the library reference.

I think it will be the best to revert r86934 and resume the discussion
of adding this functionality to 3.3 when we won't be constrained by
the language moratorium.  I will write a separate message with my
thoughts about adding bytes codecs in 3.3.  Let's keep this thread
focused on what has to be done for 3.2.


[1] http://mail.python.org/pipermail/python-dev/2010-December/106353.html
[2] http://svn.python.org/view?view=rev&revision=86934
[3] http://bugs.python.org/issue7475

From mal at egenix.com  Thu Dec  9 16:03:20 2010
From: mal at egenix.com (M.-A. Lemburg)
Date: Thu, 09 Dec 2010 16:03:20 +0100
Subject: [Python-Dev] The fate of transform() and untransform() methods
In-Reply-To: <AANLkTimCi7Ovz3Yxc7__3ohWKjGMrKZxuPmuSkQ7POjy@mail.gmail.com>
References: <AANLkTimCi7Ovz3Yxc7__3ohWKjGMrKZxuPmuSkQ7POjy@mail.gmail.com>
Message-ID: <4D00EFB8.1090204@egenix.com>


Alexander Belopolsky wrote:
> On Fri, Dec 3, 2010 at 1:05 PM, Guido van Rossum <guido at python.org> wrote:
>> On Fri, Dec 3, 2010 at 9:58 AM, R. David Murray <rdmurray at bitdance.com> wrote:
> ..
>>> I believe MAL's thought was that the addition of these methods had
>>> been approved pre-moratorium, but I don't know if that is a
>>> sufficient argument or not.
>>
>> It is not.
>>
>> The moratorium is intended to freeze the state of the language as
>> implemented, not whatever was discussed and approved but didn't get
>> implemented (that'd be a hole big enough to drive a truck through, as
>> the saying goes :-).
>>
>> Regardless of what I or others may have said before, I am not
>> currently a fan of adding transform() to either str or bytes.
>>
> 
> I would like to restart the discussion under a separate subject
> because the original thread [1] went off the specific topic of the six
> new methods (2 methods x 3 types) added to builtins shortly before 3.2
> beta was released. [2]  The ticket that introduced the change is
> currently closed [3] even though the last message suggests that at
> least part of the change needs to be reverted.

That's for Guido to decide.

The moratorium, if at all, would only cover the new methods,
not the other changes (readding the codecs and fixing the
codecs.py module to work with bytes as well as Unicode), all
of which were already discussed at length in several previous
discussion, on tickets and on python-dev.

I don't see much point in going through the same discussions
over and over again. Fortunately, I'm on vacation next week,
so don't have to go through all this again ;-)

Cheers,
-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Dec 09 2010)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try our new mxODBC.Connect Python Database Interface for free ! ::::


   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
           Registered at Amtsgericht Duesseldorf: HRB 46611
               http://www.egenix.com/company/contact/

From fuzzyman at voidspace.org.uk  Thu Dec  9 16:13:36 2010
From: fuzzyman at voidspace.org.uk (Michael Foord)
Date: Thu, 09 Dec 2010 15:13:36 +0000
Subject: [Python-Dev] The fate of transform() and untransform() methods
In-Reply-To: <4D00EFB8.1090204@egenix.com>
References: <AANLkTimCi7Ovz3Yxc7__3ohWKjGMrKZxuPmuSkQ7POjy@mail.gmail.com>
	<4D00EFB8.1090204@egenix.com>
Message-ID: <4D00F220.4070305@voidspace.org.uk>

On 09/12/2010 15:03, M.-A. Lemburg wrote:
> Alexander Belopolsky wrote:
>> On Fri, Dec 3, 2010 at 1:05 PM, Guido van Rossum<guido at python.org>  wrote:
>>> On Fri, Dec 3, 2010 at 9:58 AM, R. David Murray<rdmurray at bitdance.com>  wrote:
>> ..
>>>> I believe MAL's thought was that the addition of these methods had
>>>> been approved pre-moratorium, but I don't know if that is a
>>>> sufficient argument or not.
>>> It is not.
>>>
>>> The moratorium is intended to freeze the state of the language as
>>> implemented, not whatever was discussed and approved but didn't get
>>> implemented (that'd be a hole big enough to drive a truck through, as
>>> the saying goes :-).
>>>
>>> Regardless of what I or others may have said before, I am not
>>> currently a fan of adding transform() to either str or bytes.
>>>
>> I would like to restart the discussion under a separate subject
>> because the original thread [1] went off the specific topic of the six
>> new methods (2 methods x 3 types) added to builtins shortly before 3.2
>> beta was released. [2]  The ticket that introduced the change is
>> currently closed [3] even though the last message suggests that at
>> least part of the change needs to be reverted.
> That's for Guido to decide.
>

Well, Guido *already* said no to transform / untransform in the previous 
thread.

Michael

> The moratorium, if at all, would only cover the new methods,
> not the other changes (readding the codecs and fixing the
> codecs.py module to work with bytes as well as Unicode), all
> of which were already discussed at length in several previous
> discussion, on tickets and on python-dev.
>
> I don't see much point in going through the same discussions
> over and over again. Fortunately, I'm on vacation next week,
> so don't have to go through all this again ;-)
>
> Cheers,


-- 

http://www.voidspace.org.uk/

READ CAREFULLY. By accepting and reading this email you agree,
on behalf of your employer, to release me from all obligations
and waivers arising from any and all NON-NEGOTIATED agreements,
licenses, terms-of-service, shrinkwrap, clickwrap, browsewrap,
confidentiality, non-disclosure, non-compete and acceptable use
policies (?BOGUS AGREEMENTS?) that I have entered into with your
employer, its partners, licensors, agents and assigns, in
perpetuity, without prejudice to my ongoing rights and privileges.
You further represent that you have the authority to release me
from any BOGUS AGREEMENTS on behalf of your employer.


From brett at python.org  Thu Dec  9 17:36:56 2010
From: brett at python.org (Brett Cannon)
Date: Thu, 9 Dec 2010 08:36:56 -0800
Subject: [Python-Dev] futures API
In-Reply-To: <330887.27553.qm@web27506.mail.ukl.yahoo.com>
References: <330887.27553.qm@web27506.mail.ukl.yahoo.com>
Message-ID: <AANLkTikujgbUwVxqGXUbcuxrBu5E7wbObKjeddcqRo_d@mail.gmail.com>

On Thu, Dec 9, 2010 at 04:26, Thomas Nagy <tnagyemail-mail at yahoo.fr> wrote:
> Hello,
>
> I am looking forward to replacing a piece of code (http://code.google.com/p/waf/source/browse/trunk/waflib/Runner.py#86) by the futures module which was announced in python 3.2 beta. I am a bit stuck with it, so I have a few questions about the futures:
>
> 1. Is the futures API frozen?

It will be once Python 3.2 final is released.

-Brett

> 2. How hard would it be to return the tasks processed in an output queue to process/consume the results while they are returned? The code does not seem to be very open for monkey patching.
> 3. How hard would it be to add new tasks dynamically (after a task is executed) and have the futures object never complete?
> 4. Is there a performance evaluation of the futures code (execution overhead) ?
>
> Thanks,
> Thomas
>
>
>
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/brett%40python.org
>

From fuzzyman at voidspace.org.uk  Thu Dec  9 17:48:25 2010
From: fuzzyman at voidspace.org.uk (Michael Foord)
Date: Thu, 09 Dec 2010 16:48:25 +0000
Subject: [Python-Dev] futures API
In-Reply-To: <AANLkTikujgbUwVxqGXUbcuxrBu5E7wbObKjeddcqRo_d@mail.gmail.com>
References: <330887.27553.qm@web27506.mail.ukl.yahoo.com>
	<AANLkTikujgbUwVxqGXUbcuxrBu5E7wbObKjeddcqRo_d@mail.gmail.com>
Message-ID: <4D010859.1020807@voidspace.org.uk>

On 09/12/2010 16:36, Brett Cannon wrote:
> On Thu, Dec 9, 2010 at 04:26, Thomas Nagy<tnagyemail-mail at yahoo.fr>  wrote:
>> Hello,
>>
>> I am looking forward to replacing a piece of code (http://code.google.com/p/waf/source/browse/trunk/waflib/Runner.py#86) by the futures module which was announced in python 3.2 beta. I am a bit stuck with it, so I have a few questions about the futures:
>>
>> 1. Is the futures API frozen?
> It will be once Python 3.2 final is released.

Now that 3.2 beta 1 is out the api is effectively frozen though, the 
only changes that should be made are bugfixes unless you can convince 
the release manager that an api change is really *essential*.

Michael

> -Brett
>
>> 2. How hard would it be to return the tasks processed in an output queue to process/consume the results while they are returned? The code does not seem to be very open for monkey patching.
>> 3. How hard would it be to add new tasks dynamically (after a task is executed) and have the futures object never complete?
>> 4. Is there a performance evaluation of the futures code (execution overhead) ?
>>
>> Thanks,
>> Thomas
>>
>>
>>
>>
>> _______________________________________________
>> Python-Dev mailing list
>> Python-Dev at python.org
>> http://mail.python.org/mailman/listinfo/python-dev
>> Unsubscribe: http://mail.python.org/mailman/options/python-dev/brett%40python.org
>>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk


-- 

http://www.voidspace.org.uk/

READ CAREFULLY. By accepting and reading this email you agree,
on behalf of your employer, to release me from all obligations
and waivers arising from any and all NON-NEGOTIATED agreements,
licenses, terms-of-service, shrinkwrap, clickwrap, browsewrap,
confidentiality, non-disclosure, non-compete and acceptable use
policies (?BOGUS AGREEMENTS?) that I have entered into with your
employer, its partners, licensors, agents and assigns, in
perpetuity, without prejudice to my ongoing rights and privileges.
You further represent that you have the authority to release me
from any BOGUS AGREEMENTS on behalf of your employer.


From mal at egenix.com  Thu Dec  9 17:56:07 2010
From: mal at egenix.com (M.-A. Lemburg)
Date: Thu, 09 Dec 2010 17:56:07 +0100
Subject: [Python-Dev] The fate of transform() and untransform() methods
In-Reply-To: <4D00F220.4070305@voidspace.org.uk>
References: <AANLkTimCi7Ovz3Yxc7__3ohWKjGMrKZxuPmuSkQ7POjy@mail.gmail.com>	<4D00EFB8.1090204@egenix.com>
	<4D00F220.4070305@voidspace.org.uk>
Message-ID: <4D010A27.1070902@egenix.com>



Michael Foord wrote:
> On 09/12/2010 15:03, M.-A. Lemburg wrote:
>> Alexander Belopolsky wrote:
>>> On Fri, Dec 3, 2010 at 1:05 PM, Guido van Rossum<guido at python.org> 
>>> wrote:
>>>> On Fri, Dec 3, 2010 at 9:58 AM, R. David
>>>> Murray<rdmurray at bitdance.com>  wrote:
>>> ..
>>>>> I believe MAL's thought was that the addition of these methods had
>>>>> been approved pre-moratorium, but I don't know if that is a
>>>>> sufficient argument or not.
>>>> It is not.
>>>>
>>>> The moratorium is intended to freeze the state of the language as
>>>> implemented, not whatever was discussed and approved but didn't get
>>>> implemented (that'd be a hole big enough to drive a truck through, as
>>>> the saying goes :-).
>>>>
>>>> Regardless of what I or others may have said before, I am not
>>>> currently a fan of adding transform() to either str or bytes.
>>>>
>>> I would like to restart the discussion under a separate subject
>>> because the original thread [1] went off the specific topic of the six
>>> new methods (2 methods x 3 types) added to builtins shortly before 3.2
>>> beta was released. [2]  The ticket that introduced the change is
>>> currently closed [3] even though the last message suggests that at
>>> least part of the change needs to be reverted.
>> That's for Guido to decide.
>>
> 
> Well, Guido *already* said no to transform / untransform in the previous
> thread.

I'm not sure he did and asked for clarification (see attached email).

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Dec 09 2010)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try our new mxODBC.Connect Python Database Interface for free ! ::::


   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
           Registered at Amtsgericht Duesseldorf: HRB 46611
               http://www.egenix.com/company/contact/
-------------- next part --------------
An embedded message was scrubbed...
From: "M.-A. Lemburg" <mal at egenix.com>
Subject: Re: [Python-Dev] transform() and untransform() methods,
	and the codec registry
Date: Mon, 06 Dec 2010 12:39:43 +0100
Size: 7468
URL: <http://mail.python.org/pipermail/python-dev/attachments/20101209/0a50768d/attachment.mht>

From alexander.belopolsky at gmail.com  Thu Dec  9 18:06:46 2010
From: alexander.belopolsky at gmail.com (Alexander Belopolsky)
Date: Thu, 9 Dec 2010 12:06:46 -0500
Subject: [Python-Dev] The fate of transform() and untransform() methods
In-Reply-To: <4D00EFB8.1090204@egenix.com>
References: <AANLkTimCi7Ovz3Yxc7__3ohWKjGMrKZxuPmuSkQ7POjy@mail.gmail.com>
	<4D00EFB8.1090204@egenix.com>
Message-ID: <AANLkTimGt9Q98LfbQuyQFDaxh0xpA+Y2D5TVvchOQGYU@mail.gmail.com>

On Thu, Dec 9, 2010 at 10:03 AM, M.-A. Lemburg <mal at egenix.com> wrote:
> Alexander Belopolsky wrote:
..
>>?The ticket that introduced the change is
>> currently closed [3] even though the last message suggests that at
>> least part of the change needs to be reverted.
>
> That's for Guido to decide.
>
The decision will probably rest with the release manager, but Guido
has clearly voiced his opinion.  Note that I deliberately removed the
codecs part form the subject, so that we can focus on the fate of
{str,bytes,bytearray}.{transform,untransform} methods.  I don't see
any dispute about the fact that adding these methods breaks the
moratorium or any suggestion that a case-by-case exception has been
approved for this particular case.  I don't think there is an option
to keep these methods.  The two available options are:

1. Revert  r86934.
2. Revert C code changes, but keep the codecs and remove references to
transform/untransform from documentation.

From brian at sweetapp.com  Thu Dec  9 18:02:41 2010
From: brian at sweetapp.com (Brian Quinlan)
Date: Thu, 9 Dec 2010 09:02:41 -0800
Subject: [Python-Dev] futures API
In-Reply-To: <330887.27553.qm@web27506.mail.ukl.yahoo.com>
References: <330887.27553.qm@web27506.mail.ukl.yahoo.com>
Message-ID: <85AA5A0F-9EC4-4BBF-9A30-1547F4DA59E8@sweetapp.com>


On Dec 9, 2010, at 4:26 AM, Thomas Nagy wrote:

> Hello,
>
> I am looking forward to replacing a piece of code (http://code.google.com/p/waf/source/browse/trunk/waflib/Runner.py#86 
> ) by the futures module which was announced in python 3.2 beta. I am  
> a bit stuck with it, so I have a few questions about the futures:
>
> 1. Is the futures API frozen?

Yes.

> 2. How hard would it be to return the tasks processed in an output  
> queue to process/consume the results while they are returned? The  
> code does not seem to be very open for monkey patching.

You can associate a callback with a submitted future. That callback  
could add the future to your queue.

> 3. How hard would it be to add new tasks dynamically (after a task  
> is executed) and have the futures object never complete?

I'm not sure that I understand your question. You can submit new work  
to an Executor at until time until it is shutdown and a work item can  
take as long to complete as you want. If you are contemplating tasks  
that don't complete then maybe you could be better just scheduling a  
thread.

> 4. Is there a performance evaluation of the futures code (execution  
> overhead) ?

No. Scott Dial did make some performance improvements so he might have  
a handle on its overhead.

Cheers,
Brian

> Thanks,
> Thomas
>
>
>
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/brian%40sweetapp.com


From mal at egenix.com  Thu Dec  9 18:32:29 2010
From: mal at egenix.com (M.-A. Lemburg)
Date: Thu, 09 Dec 2010 18:32:29 +0100
Subject: [Python-Dev] The fate of transform() and untransform() methods
In-Reply-To: <AANLkTimGt9Q98LfbQuyQFDaxh0xpA+Y2D5TVvchOQGYU@mail.gmail.com>
References: <AANLkTimCi7Ovz3Yxc7__3ohWKjGMrKZxuPmuSkQ7POjy@mail.gmail.com>	<4D00EFB8.1090204@egenix.com>
	<AANLkTimGt9Q98LfbQuyQFDaxh0xpA+Y2D5TVvchOQGYU@mail.gmail.com>
Message-ID: <4D0112AD.6030103@egenix.com>

Alexander Belopolsky wrote:
> On Thu, Dec 9, 2010 at 10:03 AM, M.-A. Lemburg <mal at egenix.com> wrote:
>> Alexander Belopolsky wrote:
> ..
>>>  The ticket that introduced the change is
>>> currently closed [3] even though the last message suggests that at
>>> least part of the change needs to be reverted.
>>
>> That's for Guido to decide.
>>
> The decision will probably rest with the release manager, but Guido
> has clearly voiced his opinion. 

FYI: Georg explicitly asked me whether I would have the patch ready
for 3.2 and since I didn't have time to work on it, he volunteered
to implement it, which I'd like to thank him for !

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Dec 09 2010)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try our new mxODBC.Connect Python Database Interface for free ! ::::


   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
           Registered at Amtsgericht Duesseldorf: HRB 46611
               http://www.egenix.com/company/contact/

From guido at python.org  Thu Dec  9 19:42:27 2010
From: guido at python.org (Guido van Rossum)
Date: Thu, 9 Dec 2010 10:42:27 -0800
Subject: [Python-Dev] transform() and untransform() methods,
	and the codec registry
In-Reply-To: <4CFCCB7F.9080605@egenix.com>
References: <20101202180651.F2AB0EE9CD@mail.python.org>
	<201012031016.05063.victor.stinner@haypocalc.com>
	<20101203151129.74C5222A88F@kimball.webabinitio.net>
	<AANLkTiktmxr_eH-S9ihcwFoqd6yJeN03Hes_urH7Qt3A@mail.gmail.com>
	<20101203175832.88F00236D8B@kimball.webabinitio.net>
	<AANLkTikF2w-mmNVgYch=40Rmdc_OvRV3wOYzthvNBbpO@mail.gmail.com>
	<4CFCCB7F.9080605@egenix.com>
Message-ID: <AANLkTi=+9B8ez2sXkg-7uQp_SQ=zaYA_0KiydpybGuzs@mail.gmail.com>

On Mon, Dec 6, 2010 at 3:39 AM, M.-A. Lemburg <mal at egenix.com> wrote:
> Guido van Rossum wrote:
>> The moratorium is intended to freeze the state of the language as
>> implemented, not whatever was discussed and approved but didn't get
>> implemented (that'd be a hole big enough to drive a truck through, as
>> the saying goes :-).
>
> Sure, but those two particular methods only provide interfaces
> to the codecs sub-system without actually requiring any major
> implementation changes.
>
> Furthermore, they "help ease adoption of Python 3.x" (quoted from
> PEP 3003), since the functionality they add back was removed from
> Python 3.0 in a way that makes it difficult to port Python2
> applications to Python3.
>
>> Regardless of what I or others may have said before, I am not
>> currently a fan of adding transform() to either str or bytes.
>
> How should I read this ? Do want the methods to be removed again
> and added back in 3.3 ?

Given that it's in 3.2b1 I'm okay with keeping it. That's at best a
+0. I'd be -0 if it wasn't already in. But anyway this should suffice
to keep it in unless there are others strongly opposed.

> Frankly, I'm a bit tired of constantly having to argue against
> cutting down the Unicode and codec support in Python3.

But transform() isn't really about Unicode or codec support -- it is
about string-to-string and bytes-to-bytes transformations. At least
the transform() API is clear about the distinction between codecs
(which translate between bytes and string) and transforms (which keep
the type unchanged) -- though I still don't like that the registries
for transforms and codecs use the same namespace. Also bytes-bytes and
string-string transforms use the same namespace even though the
typical transform only supports one or the other. E.g. IMO all of the
following should raise LookupError:

>>> b'abc'.transform('rot13')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/guido/p3/Lib/encodings/rot_13.py", line 16, in encode
    return (input.translate(rot13_map), len(input))
TypeError: expected an object with the buffer interface

>>> b'abc'.decode('rot13')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/guido/p3/Lib/encodings/rot_13.py", line 19, in decode
    return (input.translate(rot13_map), len(input))
AttributeError: 'memoryview' object has no attribute 'translate'

>>> 'abc'.encode('rot13')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: encoder did not return a bytes object (type=str)

>>> b''.decode('rot13')
''

The latter may be a separate bug; b''.decode('anything') seems to not
even attempt to look up the codec.

-- 
--Guido van Rossum (python.org/~guido)

From alexander.belopolsky at gmail.com  Thu Dec  9 19:55:08 2010
From: alexander.belopolsky at gmail.com (Alexander Belopolsky)
Date: Thu, 9 Dec 2010 13:55:08 -0500
Subject: [Python-Dev] transform() and untransform() methods,
	and the codec registry
In-Reply-To: <AANLkTi=+9B8ez2sXkg-7uQp_SQ=zaYA_0KiydpybGuzs@mail.gmail.com>
References: <20101202180651.F2AB0EE9CD@mail.python.org>
	<201012031016.05063.victor.stinner@haypocalc.com>
	<20101203151129.74C5222A88F@kimball.webabinitio.net>
	<AANLkTiktmxr_eH-S9ihcwFoqd6yJeN03Hes_urH7Qt3A@mail.gmail.com>
	<20101203175832.88F00236D8B@kimball.webabinitio.net>
	<AANLkTikF2w-mmNVgYch=40Rmdc_OvRV3wOYzthvNBbpO@mail.gmail.com>
	<4CFCCB7F.9080605@egenix.com>
	<AANLkTi=+9B8ez2sXkg-7uQp_SQ=zaYA_0KiydpybGuzs@mail.gmail.com>
Message-ID: <AANLkTinew=h2RFmTE4p6wLJ=hHuOH7YxAn6UhdjOXimy@mail.gmail.com>

On Thu, Dec 9, 2010 at 1:42 PM, Guido van Rossum <guido at python.org> wrote:
..
> string-string transforms use the same namespace even though the
> typical transform only supports one or the other. E.g. IMO all of the
> following should raise LookupError:
>
>>>> b'abc'.transform('rot13')
> Traceback (most recent call last):
> ..
> ? ?return (input.translate(rot13_map), len(input))
> TypeError: expected an object with the buffer interface

This is actually *very* misleading because

>>> 'abc'.transform('rot13')
'nop'

works even though 'abc' is not "an object with the buffer interface".


>>> memoryview('abc')
Traceback (most recent call last):
  ..
TypeError: cannot make memory view because object does not have the
buffer interface

From solipsis at pitrou.net  Thu Dec  9 20:17:49 2010
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Thu, 9 Dec 2010 20:17:49 +0100
Subject: [Python-Dev] transform() and untransform() methods,
	and the codec registry
References: <20101202180651.F2AB0EE9CD@mail.python.org>
	<201012031016.05063.victor.stinner@haypocalc.com>
	<20101203151129.74C5222A88F@kimball.webabinitio.net>
	<AANLkTiktmxr_eH-S9ihcwFoqd6yJeN03Hes_urH7Qt3A@mail.gmail.com>
	<20101203175832.88F00236D8B@kimball.webabinitio.net>
	<AANLkTikF2w-mmNVgYch=40Rmdc_OvRV3wOYzthvNBbpO@mail.gmail.com>
	<4CFCCB7F.9080605@egenix.com>
	<AANLkTi=+9B8ez2sXkg-7uQp_SQ=zaYA_0KiydpybGuzs@mail.gmail.com>
	<AANLkTinew=h2RFmTE4p6wLJ=hHuOH7YxAn6UhdjOXimy@mail.gmail.com>
Message-ID: <20101209201749.79edd5ce@pitrou.net>

On Thu, 9 Dec 2010 13:55:08 -0500
Alexander Belopolsky <alexander.belopolsky at gmail.com> wrote:

> On Thu, Dec 9, 2010 at 1:42 PM, Guido van Rossum <guido at python.org> wrote:
> ..
> > string-string transforms use the same namespace even though the
> > typical transform only supports one or the other. E.g. IMO all of the
> > following should raise LookupError:
> >
> >>>> b'abc'.transform('rot13')
> > Traceback (most recent call last):
> > ..
> > ? ?return (input.translate(rot13_map), len(input))
> > TypeError: expected an object with the buffer interface
> 
> This is actually *very* misleading because
> 
> >>> 'abc'.transform('rot13')
> 'nop'
> 
> works even though 'abc' is not "an object with the buffer interface".

Agreed. It was already pointed out in the parent thread.
I would say my opinion on keeping transform()/untransform() is +0 if
these error messages (and preferably the exception type as well) get
improved. Otherwise we'd better revert them and add a more polished
version in 3.3.

Regards

Antoine.



From merwok at netwok.org  Thu Dec  9 20:19:06 2010
From: merwok at netwok.org (=?UTF-8?B?w4lyaWMgQXJhdWpv?=)
Date: Thu, 09 Dec 2010 20:19:06 +0100
Subject: [Python-Dev] transform() and untransform() methods,
 and the codec registry
In-Reply-To: <AANLkTi=+9B8ez2sXkg-7uQp_SQ=zaYA_0KiydpybGuzs@mail.gmail.com>
References: <20101202180651.F2AB0EE9CD@mail.python.org>	<201012031016.05063.victor.stinner@haypocalc.com>	<20101203151129.74C5222A88F@kimball.webabinitio.net>	<AANLkTiktmxr_eH-S9ihcwFoqd6yJeN03Hes_urH7Qt3A@mail.gmail.com>	<20101203175832.88F00236D8B@kimball.webabinitio.net>	<AANLkTikF2w-mmNVgYch=40Rmdc_OvRV3wOYzthvNBbpO@mail.gmail.com>	<4CFCCB7F.9080605@egenix.com>
	<AANLkTi=+9B8ez2sXkg-7uQp_SQ=zaYA_0KiydpybGuzs@mail.gmail.com>
Message-ID: <4D012BAA.80401@netwok.org>

Le 09/12/2010 19:42, Guido van Rossum a ?crit :
> Given that it's in 3.2b1 I'm okay with keeping it. That's at best a
> +0. [...]

> though I still don't like that the registries for transforms and
> codecs use the same namespace. Also bytes-bytes and
> string-string transforms use the same namespace even though the
> typical transform only supports one or the other. E.g. IMO all of the
> following should raise LookupError: [...]

Although I?d regret not having transform/untransform in 3.2, I think
there needs to be a discussion about this namespace issue before the new
methods ship.

Regards


From python at mrabarnett.plus.com  Thu Dec  9 21:21:32 2010
From: python at mrabarnett.plus.com (MRAB)
Date: Thu, 09 Dec 2010 20:21:32 +0000
Subject: [Python-Dev] Can't compile regex module with Python 3.2
In-Reply-To: <AANLkTikoFuJzZt6BCm=HQnzpwKc3BRnTgu8KcubOTVow@mail.gmail.com>
References: <4D004568.4000905@mrabarnett.plus.com>	<4D006D88.10300@v.loewis.de>
	<AANLkTikoFuJzZt6BCm=HQnzpwKc3BRnTgu8KcubOTVow@mail.gmail.com>
Message-ID: <4D013A4C.4090703@mrabarnett.plus.com>

On 09/12/2010 05:57, Alexander Belopolsky wrote:
> On Thu, Dec 9, 2010 at 12:47 AM, "Martin v. L?wis"<martin at v.loewis.de>  wrote:
> ..
>>> However, in Python 3.2b1 the library python32.lib contains only
>>> _PyUnicode_IsWhitespace, therefore breaking the build.
>>>
>>> Is this change intentional? If so, why does unicodeobject.h still do
>>> the mapping?
>>
>> Are you sure about this? It's not intentional (except in the limited ABI).
>
> this does seem to be intentional:
>
>
> $ svn log -r 84177
> ------------------------------------------------------------------------
> r84177 | amaury.forgeotdarc | 2010-08-18 16:44:58 -0400 (Wed, 18 Aug
> 2010) | 9 lines
>
> #5127: Even on narrow unicode builds, the C functions that access the Unicode
> Database (Py_UNICODE_TOLOWER, Py_UNICODE_ISDECIMAL, and others) now accept
> and return characters from the full Unicode range (Py_UCS4).
>
> The differences from Python code are few:
> - unicodedata.numeric(), unicodedata.decimal() and unicodedata.digit()
>    now return the correct value for large code points
> - repr() may consider more characters as printable.
>
> ------------------------------------------------------------------------
> http://svn.python.org/view?view=rev&revision=84177
>
I have it compiling now.

From alexander.belopolsky at gmail.com  Thu Dec  9 21:29:35 2010
From: alexander.belopolsky at gmail.com (Alexander Belopolsky)
Date: Thu, 9 Dec 2010 15:29:35 -0500
Subject: [Python-Dev] transform() and untransform() methods,
	and the codec registry
In-Reply-To: <20101209201749.79edd5ce@pitrou.net>
References: <20101202180651.F2AB0EE9CD@mail.python.org>
	<201012031016.05063.victor.stinner@haypocalc.com>
	<20101203151129.74C5222A88F@kimball.webabinitio.net>
	<AANLkTiktmxr_eH-S9ihcwFoqd6yJeN03Hes_urH7Qt3A@mail.gmail.com>
	<20101203175832.88F00236D8B@kimball.webabinitio.net>
	<AANLkTikF2w-mmNVgYch=40Rmdc_OvRV3wOYzthvNBbpO@mail.gmail.com>
	<4CFCCB7F.9080605@egenix.com>
	<AANLkTi=+9B8ez2sXkg-7uQp_SQ=zaYA_0KiydpybGuzs@mail.gmail.com>
	<AANLkTinew=h2RFmTE4p6wLJ=hHuOH7YxAn6UhdjOXimy@mail.gmail.com>
	<20101209201749.79edd5ce@pitrou.net>
Message-ID: <AANLkTimY8L=TZpnazBHsTQWz_oqH6yXUjVNnWr=5=xk3@mail.gmail.com>

On Thu, Dec 9, 2010 at 2:17 PM, Antoine Pitrou <solipsis at pitrou.net> wrote:
> On Thu, 9 Dec 2010 13:55:08 -0500
> Alexander Belopolsky <alexander.belopolsky at gmail.com> wrote:
..
>> This is actually *very* misleading because
>>
>> >>> 'abc'.transform('rot13')
>> 'nop'
>>
>> works even though 'abc' is not "an object with the buffer interface".
>
> Agreed. It was already pointed out in the parent thread.
> I would say my opinion on keeping transform()/untransform() is +0 if
> these error messages (and preferably the exception type as well) get
> improved. Otherwise we'd better revert them and add a more polished
> version in 3.3.


Error messages is only one of the problems.  User confusion over which
codec supports which types is another.  Why, for example rot13 works
on str and not on bytes?  It only affects ASCII range, so isn't it
natural to expect  b'abc'.transform('rot13') to work?  Well,
presumably this is so because Caesar did not know about bytes and his
"cypher" was about character shuffling.  In this case, should't it
also shuffle other code points assigned to Latin letters?  Given how
"useful" rot13 is in practice, I feel that it was only added to
justify adding str.transform().

There are other problems raised on the issue and not addressed in the
tracker discussion.  For example, both Victor and I expressed concern
about having builitn methods that do import behind the scenes.
Granted, this issue already exists with encode/decode methods, but
these are usable without providing an explicit encoding and in this
form do not have side-effects.

Another problem is that with str.transform(), users are encouraged to
write programs in which data stored in strings is not always
interpreted as Unicode.  For example, when I see a 'n' in a string
variable, it may actually mean 'a' because it has been scrambled by
rot13.   Again, rot13 is not a realistic example, but as users are
encouraged to create their own string to string codecs, we may soon
find ourselves in the same mess as we have with 2.x programs trying to
support multiple locales.

As far as realistic examples go, Unicode transformations such as case
folding, normalization or decimal to ASCII translation have not been
considered in str.transform() design.  The
str.transform/str.untransform pair may or may not be a good solution
for these cases.  One obvious issue being that these transformations
are often not invertible.

I admit I have more questions than answers at this point, but a design
that adds the same two methods to three builtin types with very
different usage patterns (str, bytes and bytearray) does not seem to
be well thought out.

The str type already has 40+ methods many of which are not well-suited
to handle the complexities inherent in Unicode.   Rather than rushing
in two more methods that will prove to be about as useful as
str.swapcase(), lets consider actual use cases and come up with a
design that will properly address them.

From db3l.net at gmail.com  Thu Dec  9 22:34:25 2010
From: db3l.net at gmail.com (David Bolen)
Date: Thu, 09 Dec 2010 16:34:25 -0500
Subject: [Python-Dev] kill_python on windows buildbots
References: <20101113133712.60e9be27@pitrou.net>
	<m2lj4wpqzt.fsf@valheru.db3l.homeip.net>
	<AANLkTimRZ_h9kpbQrXiD2rgP77EqUAHNPEWMrvAyoLSC@mail.gmail.com>
	<m2hbfkp9xm.fsf@valheru.db3l.homeip.net>
	<4CEB7E12.1070201@snakebite.org>
	<m2zkszochi.fsf@valheru.db3l.homeip.net>
	<m2mxoil9a0.fsf_-_@valheru.db3l.homeip.net>
	<idlgt6$8vq$2@dough.gmane.org>
	<m2fwu9lirl.fsf@valheru.db3l.homeip.net>
	<4CFF21D0.7050705@m2.ccsnet.ne.jp>
Message-ID: <m2bp4uljfi.fsf@valheru.db3l.homeip.net>

Hirokazu Yamamoto <ocean-city at m2.ccsnet.ne.jp> writes:

> Yes, but test can freeze. In that case, I'm worried that
>   (snip)
>   rt.bat .... # freeze here (will be halt by buildbot)
>   vcbuild .... & kill_python_d # Will this be called?
> in test.bat.

Yeah, you're right.  It may be impossible to completely eliminate the
risk of a process getting stuck without doing something external to
the build process, so long as the first thing a new build tries to do
is an svn checkout that will fail if the process is still running.

Having the kill operation in clean.bat covers the vast majority of the
common cases with a minimum change, so seems the simplest.

-- David


From ncoghlan at gmail.com  Thu Dec  9 23:18:00 2010
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Fri, 10 Dec 2010 08:18:00 +1000
Subject: [Python-Dev] [Python-checkins] r87145 -
	python/branches/py3k/Doc/whatsnew/3.2.rst
In-Reply-To: <20101209164154.5A45DEEAA3@mail.python.org>
References: <20101209164154.5A45DEEAA3@mail.python.org>
Message-ID: <AANLkTi=_npoxQC+tfzkonT8jyH_d1BNOdgsn8c_esGaw@mail.gmail.com>

On Fri, Dec 10, 2010 at 2:41 AM, raymond.hettinger
<python-checkins at python.org> wrote:
> @@ -588,7 +593,12 @@
> ? pointing to the original callable function. ?This allows wrapped functions to
> ? be introspected. ?It also copies :attr:`__annotations__` if defined. ?And now
> ? it also gracefully skips over missing attributes such as :attr:`__doc__` which
> - ?might not be defined for the wrapped callable.
> + ?might not be defined for the wrapped callable:
> +
> + ?>>> callable(max)
> + ?True
> + ?>>> callable(20)
> + ?False

Placeholder example?

Cheers,
Nick.

--
Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia

From raymond.hettinger at gmail.com  Thu Dec  9 23:39:38 2010
From: raymond.hettinger at gmail.com (Raymond Hettinger)
Date: Thu, 9 Dec 2010 14:39:38 -0800
Subject: [Python-Dev] futures API
In-Reply-To: <85AA5A0F-9EC4-4BBF-9A30-1547F4DA59E8@sweetapp.com>
References: <330887.27553.qm@web27506.mail.ukl.yahoo.com>
	<85AA5A0F-9EC4-4BBF-9A30-1547F4DA59E8@sweetapp.com>
Message-ID: <1A7546B0-2999-481B-904A-D1711DB65FEB@gmail.com>


On Dec 9, 2010, at 9:02 AM, Brian Quinlan wrote:

> 
> On Dec 9, 2010, at 4:26 AM, Thomas Nagy wrote:
> 
>> Hello,
>> 
>> I am looking forward to replacing a piece of code (http://code.google.com/p/waf/source/browse/trunk/waflib/Runner.py#86) by the futures module which was announced in python 3.2 beta. I am a bit stuck with it, so I have a few questions about the futures:
>> 
>> 1. Is the futures API frozen?
> 
> Yes.

Yes, unless the current API is defective in some way.  A beta1 release is a chance for everyone to exercise the new API and discover whether it is problematic in any real world applications.

> 
>> 2. How hard would it be to return the tasks processed in an output queue to process/consume the results while they are returned? The code does not seem to be very open for monkey patching.
> 
> You can associate a callback with a submitted future. That callback could add the future to your queue.

The would be a good example for the docs.

> 
>> 3. How hard would it be to add new tasks dynamically (after a task is executed) and have the futures object never complete?
> 
> I'm not sure that I understand your question. You can submit new work to an Executor at until time until it is shutdown and a work item can take as long to complete as you want. If you are contemplating tasks that don't complete then maybe you could be better just scheduling a thread.
> 
>> 4. Is there a performance evaluation of the futures code (execution overhead) ?
> 
> No. Scott Dial did make some performance improvements so he might have a handle on its overhead.

FWIW, the source code is short and readable.  From my quick read, it looks to be a relatively thin wrapper/adapter around existing tools.  Most of the work still gets done by the threads or processes themselves.  Think of this as a cleaner, more centralized API around the current toolset -- there is no deep new technology under the hood.

Raymond

From raymond.hettinger at gmail.com  Thu Dec  9 23:45:57 2010
From: raymond.hettinger at gmail.com (Raymond Hettinger)
Date: Thu, 9 Dec 2010 14:45:57 -0800
Subject: [Python-Dev] [Python-checkins] r87145 -
	python/branches/py3k/Doc/whatsnew/3.2.rst
In-Reply-To: <AANLkTi=_npoxQC+tfzkonT8jyH_d1BNOdgsn8c_esGaw@mail.gmail.com>
References: <20101209164154.5A45DEEAA3@mail.python.org>
	<AANLkTi=_npoxQC+tfzkonT8jyH_d1BNOdgsn8c_esGaw@mail.gmail.com>
Message-ID: <09990EAC-E8CA-4ADD-807B-93E6650215D7@gmail.com>


On Dec 9, 2010, at 2:18 PM, Nick Coghlan wrote:

> On Fri, Dec 10, 2010 at 2:41 AM, raymond.hettinger
> <python-checkins at python.org> wrote:
>> @@ -588,7 +593,12 @@
>>   pointing to the original callable function.  This allows wrapped functions to
>>   be introspected.  It also copies :attr:`__annotations__` if defined.  And now
>>   it also gracefully skips over missing attributes such as :attr:`__doc__` which
>> -  might not be defined for the wrapped callable.
>> +  might not be defined for the wrapped callable:
>> +
>> +  >>> callable(max)
>> +  True
>> +  >>> callable(20)
>> +  False
> 
> Placeholder example?
> 

Trivial function begets a trivial example :-)

If you think it warrants more, I'm open to suggestions.


Raymond


From raymond.hettinger at gmail.com  Thu Dec  9 23:54:24 2010
From: raymond.hettinger at gmail.com (Raymond Hettinger)
Date: Thu, 9 Dec 2010 14:54:24 -0800
Subject: [Python-Dev] sWAPcASE  Was:  transform() and untransform() methods,
	and the codec registry
In-Reply-To: <AANLkTimY8L=TZpnazBHsTQWz_oqH6yXUjVNnWr=5=xk3@mail.gmail.com>
References: <20101202180651.F2AB0EE9CD@mail.python.org>
	<201012031016.05063.victor.stinner@haypocalc.com>
	<20101203151129.74C5222A88F@kimball.webabinitio.net>
	<AANLkTiktmxr_eH-S9ihcwFoqd6yJeN03Hes_urH7Qt3A@mail.gmail.com>
	<20101203175832.88F00236D8B@kimball.webabinitio.net>
	<AANLkTikF2w-mmNVgYch=40Rmdc_OvRV3wOYzthvNBbpO@mail.gmail.com>
	<4CFCCB7F.9080605@egenix.com>
	<AANLkTi=+9B8ez2sXkg-7uQp_SQ=zaYA_0KiydpybGuzs@mail.gmail.com>
	<AANLkTinew=h2RFmTE4p6wLJ=hHuOH7YxAn6UhdjOXimy@mail.gmail.com>
	<20101209201749.79edd5ce@pitrou.net>
	<AANLkTimY8L=TZpnazBHsTQWz_oqH6yXUjVNnWr=5=xk3@mail.gmail.com>
Message-ID: <22FDCC84-2D68-41DE-A421-80C823CB3020@gmail.com>


On Dec 9, 2010, at 12:29 PM, Alexander Belopolsky wrote:
> 
> The str type already has 40+ methods many of which are not well-suited
> to handle the complexities inherent in Unicode.   Rather than rushing
> in two more methods that will prove to be about as useful as
> str.swapcase(), lets consider actual use cases and come up with a
> design that will properly address them.

It would make me happy if we could agree to kill or at least mortally wound
str.swapcase().    I did some research on what it is go for and found
that it is a vestige of an old word processor command to handle
the case where a user accidentally left the caps lock key turned-on.
AFAICT using Google's code search, it has nearly zero value for
Python scripts.   It does have a cost however, the code search turned-up
many cases where people were writing string like objects and included
swapcase() just so they could match the built-in API.

It's time for swapcase() to go the way of the dinosaurs.


Raymond

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20101209/57c1a585/attachment.html>

From eric at trueblade.com  Thu Dec  9 23:54:12 2010
From: eric at trueblade.com (Eric Smith)
Date: Thu, 09 Dec 2010 17:54:12 -0500
Subject: [Python-Dev] [Python-checkins] r87145
	-	python/branches/py3k/Doc/whatsnew/3.2.rst
In-Reply-To: <09990EAC-E8CA-4ADD-807B-93E6650215D7@gmail.com>
References: <20101209164154.5A45DEEAA3@mail.python.org>	<AANLkTi=_npoxQC+tfzkonT8jyH_d1BNOdgsn8c_esGaw@mail.gmail.com>
	<09990EAC-E8CA-4ADD-807B-93E6650215D7@gmail.com>
Message-ID: <4D015E14.3040807@trueblade.com>

On 12/9/2010 5:45 PM, Raymond Hettinger wrote:
>
> On Dec 9, 2010, at 2:18 PM, Nick Coghlan wrote:
>
>> On Fri, Dec 10, 2010 at 2:41 AM, raymond.hettinger
>> <python-checkins at python.org>  wrote:
>>> @@ -588,7 +593,12 @@
>>>    pointing to the original callable function.  This allows wrapped functions to
>>>    be introspected.  It also copies :attr:`__annotations__` if defined.  And now
>>>    it also gracefully skips over missing attributes such as :attr:`__doc__` which
>>> -  might not be defined for the wrapped callable.
>>> +  might not be defined for the wrapped callable:
>>> +
>>> +>>>  callable(max)
>>> +  True
>>> +>>>  callable(20)
>>> +  False
>>
>> Placeholder example?
>>
>
> Trivial function begets a trivial example :-)
>
> If you think it warrants more, I'm open to suggestions.

I think the issue is that the section is talking about functools.wraps, 
the example is for callable().

From merwok at netwok.org  Fri Dec 10 00:03:39 2010
From: merwok at netwok.org (=?UTF-8?B?w4lyaWMgQXJhdWpv?=)
Date: Fri, 10 Dec 2010 00:03:39 +0100
Subject: [Python-Dev] [Python-checkins] r86855 - in
 python/branches/py3k: Doc/library/unittest.rst Lib/unittest/case.py
In-Reply-To: <4D00B617.5090602@voidspace.org.uk>
References: <20101129013825.EF8DAEEB1E@mail.python.org>
	<4D0050E1.3000707@netwok.org> <4D00B617.5090602@voidspace.org.uk>
Message-ID: <4D01604B.4070400@netwok.org>

Le 09/12/2010 11:57, Michael Foord a ?crit :
> unittest2 will continue to track changes in unittest. A 0.6.0 release is 
> planned, with feature parity with the version of unittest in Python 3.2.
All right.  We?ll just run a sed over our tests and bump our required
unittest2 version.  Thanks for the answer.

> Of course if you want to help that would be much appreciated. :-)
I would if I could, but I have my own synchronization to take care of :)

Best regards


From martin at v.loewis.de  Fri Dec 10 00:17:28 2010
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Fri, 10 Dec 2010 00:17:28 +0100
Subject: [Python-Dev] OpenSSL Vulnerability (openssl-1.0.0a)
In-Reply-To: <4D00D06D.5060201@m2.ccsnet.ne.jp>
References: <4CEB3F72.7000006@m2.ccsnet.ne.jp>	<AANLkTi=P2_4U2X+G6RQ6=T9yvxNZ1keCgXt+=+-34ajX@mail.gmail.com>	<20101123150219.29e20374@pitrou.net>	<720EFE43-119F-4F2F-BCB1-939275B5FA6E@twistedmatrix.com>	<1290585733.3642.2.camel@localhost.localdomain>	<20101124150106.2109.660794265.divmod.xquotient.197@localhost.localdomain>	<20101124161120.5ddd106c@pitrou.net>	<20101124162312.2109.1025683352.divmod.xquotient.215@localhost.localdomain>
	<4D00D06D.5060201@m2.ccsnet.ne.jp>
Message-ID: <4D016388.1060906@v.loewis.de>

Am 09.12.2010 13:49, schrieb Hirokazu Yamamoto:
> On 2010/11/25 1:23, exarkun at twistedmatrix.com wrote:
>> Ah. Okay, then Python 3.2 would be vulnerable. Good thing it isn't
>> released yet. ;)
> 
> It seems OpenSSL 1.0.0c out.
> 
> http://openssl.org/news/secadv_20101202.txt
> 
>> 02-Dec-2010:        Security Advisory: ciphersuite downgrade fix
>> 02-Dec-2010:        OpenSSL 1.0.0c is now available, including
> important > bug and security fixes
>> 02-Dec-2010:        OpenSSL 0.9.8q is now available, including
> important > bug and security fixes

I don't plan upgrading the Windows build before 3.2; I have already
patched the OpenSSL copy that we use.

Regards,
Martin

From eric at trueblade.com  Fri Dec 10 00:10:38 2010
From: eric at trueblade.com (Eric Smith)
Date: Thu, 09 Dec 2010 18:10:38 -0500
Subject: [Python-Dev] sWAPcASE Was: transform() and untransform()
 methods, and the codec registry
In-Reply-To: <22FDCC84-2D68-41DE-A421-80C823CB3020@gmail.com>
References: <20101202180651.F2AB0EE9CD@mail.python.org>	<201012031016.05063.victor.stinner@haypocalc.com>	<20101203151129.74C5222A88F@kimball.webabinitio.net>	<AANLkTiktmxr_eH-S9ihcwFoqd6yJeN03Hes_urH7Qt3A@mail.gmail.com>	<20101203175832.88F00236D8B@kimball.webabinitio.net>	<AANLkTikF2w-mmNVgYch=40Rmdc_OvRV3wOYzthvNBbpO@mail.gmail.com>	<4CFCCB7F.9080605@egenix.com>	<AANLkTi=+9B8ez2sXkg-7uQp_SQ=zaYA_0KiydpybGuzs@mail.gmail.com>	<AANLkTinew=h2RFmTE4p6wLJ=hHuOH7YxAn6UhdjOXimy@mail.gmail.com>	<20101209201749.79edd5ce@pitrou.net>	<AANLkTimY8L=TZpnazBHsTQWz_oqH6yXUjVNnWr=5=xk3@mail.gmail.com>
	<22FDCC84-2D68-41DE-A421-80C823CB3020@gmail.com>
Message-ID: <4D0161EE.2080506@trueblade.com>

On 12/9/2010 5:54 PM, Raymond Hettinger wrote:
> It would make me happy if we could agree to kill or at least mortally wound
> str.swapcase(). I did some research on what it is go for and found
> that it is a vestige of an old word processor command to handle
> the case where a user accidentally left the caps lock key turned-on.
> AFAICT using Google's code search, it has nearly zero value for
> Python scripts. It does have a cost however, the code search turned-up
> many cases where people were writing string like objects and included
> swapcase() just so they could match the built-in API.
>
> It's time for swapcase() to go the way of the dinosaurs.

+1, assuming the normal deprecation process.

If we're looking to reduce the number of methods on str, I wouldn't mind 
seeing center() and zfill() also go away, since they're trivially 
replaced by format().

 >>> '3'.zfill(10)
'0000000003'
 >>> format('3', '>010')
'0000000003'

 >>> '3'.center(10)
'    3     '
 >>> format('3', '^10')
'    3     '

 >>> '3'.center(10, '-')
'----3-----'
 >>> format('3', '-^10')
'----3-----'

Although I'll grant this the case for the demise of center() and zfill() 
isn't as strong as for swapcase(). It's truly useless.

Eric.

From martin at v.loewis.de  Fri Dec 10 00:23:32 2010
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Fri, 10 Dec 2010 00:23:32 +0100
Subject: [Python-Dev] Can't compile regex module with Python 3.2
In-Reply-To: <AANLkTikoFuJzZt6BCm=HQnzpwKc3BRnTgu8KcubOTVow@mail.gmail.com>
References: <4D004568.4000905@mrabarnett.plus.com>	<4D006D88.10300@v.loewis.de>
	<AANLkTikoFuJzZt6BCm=HQnzpwKc3BRnTgu8KcubOTVow@mail.gmail.com>
Message-ID: <4D0164F4.9040500@v.loewis.de>

Am 09.12.2010 06:57, schrieb Alexander Belopolsky:
> On Thu, Dec 9, 2010 at 12:47 AM, "Martin v. L?wis" <martin at v.loewis.de> wrote:
> ..
>>> However, in Python 3.2b1 the library python32.lib contains only
>>> _PyUnicode_IsWhitespace, therefore breaking the build.
>>>
>>> Is this change intentional? If so, why does unicodeobject.h still do
>>> the mapping?
>>
>> Are you sure about this? It's not intentional (except in the limited ABI).
> 
> this does seem to be intentional:

Oops, right; I was confusing this issue 8654.

Then going back to the OP: Why exactly does that break the build? There
should be no problem - you shouldn't see _PyUnicodeUCS2_IsWhitespace
anywhere in 3.2.

Regards,
Martin

From ncoghlan at gmail.com  Fri Dec 10 00:28:55 2010
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Fri, 10 Dec 2010 09:28:55 +1000
Subject: [Python-Dev] [Python-checkins] r87145 -
	python/branches/py3k/Doc/whatsnew/3.2.rst
In-Reply-To: <4D015E14.3040807@trueblade.com>
References: <20101209164154.5A45DEEAA3@mail.python.org>
	<AANLkTi=_npoxQC+tfzkonT8jyH_d1BNOdgsn8c_esGaw@mail.gmail.com>
	<09990EAC-E8CA-4ADD-807B-93E6650215D7@gmail.com>
	<4D015E14.3040807@trueblade.com>
Message-ID: <AANLkTins49x12wwcpidqCLF0OeS6Udp22h_1kU0G0Xi4@mail.gmail.com>

On Fri, Dec 10, 2010 at 8:54 AM, Eric Smith <eric at trueblade.com> wrote:
> On 12/9/2010 5:45 PM, Raymond Hettinger wrote:
>> Trivial function begets a trivial example :-)
>>
>> If you think it warrants more, I'm open to suggestions.
>
> I think the issue is that the section is talking about functools.wraps, the
> example is for callable().

Yeah, sorry about that - I should have been clearer as to what I was
asking about.

It looks like the callable example had been copied to get the
formatting started for the updated functools.wraps example, but the
actual example for the latter didn't get filled in.

Cheers,
Nick.

-- 
Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia

From solipsis at pitrou.net  Fri Dec 10 00:29:30 2010
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Fri, 10 Dec 2010 00:29:30 +0100
Subject: [Python-Dev] sWAPcASE Was: transform() and untransform()
 methods, and the codec registry
References: <20101202180651.F2AB0EE9CD@mail.python.org>
	<201012031016.05063.victor.stinner@haypocalc.com>
	<20101203151129.74C5222A88F@kimball.webabinitio.net>
	<AANLkTiktmxr_eH-S9ihcwFoqd6yJeN03Hes_urH7Qt3A@mail.gmail.com>
	<20101203175832.88F00236D8B@kimball.webabinitio.net>
	<AANLkTikF2w-mmNVgYch=40Rmdc_OvRV3wOYzthvNBbpO@mail.gmail.com>
	<4CFCCB7F.9080605@egenix.com>
	<AANLkTi=+9B8ez2sXkg-7uQp_SQ=zaYA_0KiydpybGuzs@mail.gmail.com>
	<AANLkTinew=h2RFmTE4p6wLJ=hHuOH7YxAn6UhdjOXimy@mail.gmail.com>
	<20101209201749.79edd5ce@pitrou.net>
	<AANLkTimY8L=TZpnazBHsTQWz_oqH6yXUjVNnWr=5=xk3@mail.gmail.com>
	<22FDCC84-2D68-41DE-A421-80C823CB3020@gmail.com>
	<4D0161EE.2080506@trueblade.com>
Message-ID: <20101210002930.451efe65@pitrou.net>

On Thu, 09 Dec 2010 18:10:38 -0500
Eric Smith <eric at trueblade.com> wrote:
> On 12/9/2010 5:54 PM, Raymond Hettinger wrote:
> > It would make me happy if we could agree to kill or at least mortally wound
> > str.swapcase(). I did some research on what it is go for and found
> > that it is a vestige of an old word processor command to handle
> > the case where a user accidentally left the caps lock key turned-on.
> > AFAICT using Google's code search, it has nearly zero value for
> > Python scripts. It does have a cost however, the code search turned-up
> > many cases where people were writing string like objects and included
> > swapcase() just so they could match the built-in API.
> >
> > It's time for swapcase() to go the way of the dinosaurs.
> 
> +1, assuming the normal deprecation process.
> 
> If we're looking to reduce the number of methods on str, I wouldn't mind 
> seeing center() and zfill() also go away, since they're trivially 
> replaced by format().

Well, it's only trivial when you know format()'s wicked mini-language by
heart :/  center() is easy and obvious. zfill() is arguably a bit too
specialized.

Regards

Antoine.



From stutzbach at google.com  Fri Dec 10 00:35:18 2010
From: stutzbach at google.com (Daniel Stutzbach)
Date: Thu, 9 Dec 2010 15:35:18 -0800
Subject: [Python-Dev] Can't compile regex module with Python 3.2
In-Reply-To: <4D004568.4000905@mrabarnett.plus.com>
References: <4D004568.4000905@mrabarnett.plus.com>
Message-ID: <AANLkTinRY=Vf6RJ3-i=rsARwbmiHYYx6kRGjWmXP_SE4@mail.gmail.com>

On Wed, Dec 8, 2010 at 6:56 PM, MRAB <python at mrabarnett.plus.com> wrote:

> Is this change intentional? If so, why does unicodeobject.h still do
> the mapping?
>

In 3.2b1, unicodeobject.h doesn't map _PyUnicode_IsWhitespace:

http://svn.python.org/view/python/tags/r32b1/Include/unicodeobject.h?view=markup

Do you have an old unicodeobject.h somehow?

-- 
Daniel Stutzbach
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20101209/7e9687f2/attachment.html>

From python at mrabarnett.plus.com  Fri Dec 10 00:59:50 2010
From: python at mrabarnett.plus.com (MRAB)
Date: Thu, 09 Dec 2010 23:59:50 +0000
Subject: [Python-Dev] Can't compile regex module with Python 3.2
In-Reply-To: <AANLkTinRY=Vf6RJ3-i=rsARwbmiHYYx6kRGjWmXP_SE4@mail.gmail.com>
References: <4D004568.4000905@mrabarnett.plus.com>
	<AANLkTinRY=Vf6RJ3-i=rsARwbmiHYYx6kRGjWmXP_SE4@mail.gmail.com>
Message-ID: <4D016D76.2060806@mrabarnett.plus.com>

On 09/12/2010 23:35, Daniel Stutzbach wrote:
> On Wed, Dec 8, 2010 at 6:56 PM, MRAB <python at mrabarnett.plus.com
> <mailto:python at mrabarnett.plus.com>> wrote:
>
>     Is this change intentional? If so, why does unicodeobject.h still do
>     the mapping?
>
>
> In 3.2b1, unicodeobject.h doesn't map _PyUnicode_IsWhitespace:
>
> http://svn.python.org/view/python/tags/r32b1/Include/unicodeobject.h?view=markup
>
> Do you have an old unicodeobject.h somehow?
>
Basically, yes.

I'm already having to deal with 2.5, 2.6, 2.7 and 3.1, and there was a
mismatch between older 3.2 files from the trunk downloaded a while back
and the 3.2b1 .lib file.

It all seems to be working now.

From raymond.hettinger at gmail.com  Fri Dec 10 01:18:14 2010
From: raymond.hettinger at gmail.com (Raymond Hettinger)
Date: Thu, 9 Dec 2010 16:18:14 -0800
Subject: [Python-Dev] API for the new sysconfig module
Message-ID: <B710BA0A-7097-4070-9DA4-E33149620833@gmail.com>

Does anyone know why this needed a separate module and so many accessor functions?
ISTM it mostly could have been reduced to single call returning a nested dictionary.

Raymond



from sysconfig import *
import json

def sysconf():
    return dict(paths = get_paths(),
                config_vars = get_config_vars(),
                platform = get_platform(),
                version = get_python_version(),
                scheme_names = get_scheme_names(),
                )

print(json.dumps(sysconf(), indent=2))

{
  "config_vars": {
    "EXE": ".exe", 
    "VERSION": "32", 
    "py_version_nodot": "32", 
    "exec_prefix": "C:\\Python32", 
    "platbase": "C:\\Python32", 
    "userbase": "C:\\Documents and Settings\\Raymond\\Application Data\\Python", 
    "LIBDEST": "C:\\Python32\\Lib", 
    "py_version_short": "3.2", 
    "prefix": "C:\\Python32", 
    "base": "C:\\Python32", 
    "SO": ".pyd", 
    "projectbase": "C:\\Python32", 
    "BINLIBDEST": "C:\\Python32\\Lib", 
    "srcdir": "C:\\Python32", 
    "py_version": "3.2b1", 
    "abiflags": "", 
    "INCLUDEPY": "C:\\Python32\\Include", 
    "BINDIR": "C:\\Python32"
  }, 
  "platform": "win32", 
  "version": "3.2", 
  "scheme_names": [
    "nt", 
    "nt_user", 
    "os2", 
    "os2_home", 
    "osx_framework_user", 
    "posix_home", 
    "posix_prefix", 
    "posix_user"
  ], 
  "paths": {
    "platinclude": "C:\\Python32\\Include", 
    "platstdlib": "C:\\Python32\\Lib", 
    "platlib": "C:\\Python32\\Lib\\site-packages", 
    "purelib": "C:\\Python32\\Lib\\site-packages", 
    "stdlib": "C:\\Python32\\Lib", 
    "scripts": "C:\\Python32\\Scripts", 
    "include": "C:\\Python32\\Include", 
    "data": "C:\\Python32"
  }
}


From brian at sweetapp.com  Fri Dec 10 01:22:03 2010
From: brian at sweetapp.com (Brian Quinlan)
Date: Thu, 9 Dec 2010 16:22:03 -0800
Subject: [Python-Dev] futures API
In-Reply-To: <1A7546B0-2999-481B-904A-D1711DB65FEB@gmail.com>
References: <330887.27553.qm@web27506.mail.ukl.yahoo.com>
	<85AA5A0F-9EC4-4BBF-9A30-1547F4DA59E8@sweetapp.com>
	<1A7546B0-2999-481B-904A-D1711DB65FEB@gmail.com>
Message-ID: <E01EB956-3636-4293-BE7A-6A13DCC4E39F@sweetapp.com>


On Dec 9, 2010, at 2:39 PM, Raymond Hettinger wrote:

>
> On Dec 9, 2010, at 9:02 AM, Brian Quinlan wrote:
>
>>
>> On Dec 9, 2010, at 4:26 AM, Thomas Nagy wrote:
>>
>>> Hello,
>>>
>>> I am looking forward to replacing a piece of code (http://code.google.com/p/waf/source/browse/trunk/waflib/Runner.py#86 
>>> ) by the futures module which was announced in python 3.2 beta. I  
>>> am a bit stuck with it, so I have a few questions about the futures:
>>>
>>> 1. Is the futures API frozen?
>>
>> Yes.
>
> Yes, unless the current API is defective in some way.  A beta1  
> release is a chance for everyone to exercise the new API and  
> discover whether it is problematic in any real world applications.
>
>>
>>> 2. How hard would it be to return the tasks processed in an output  
>>> queue to process/consume the results while they are returned? The  
>>> code does not seem to be very open for monkey patching.
>>
>> You can associate a callback with a submitted future. That callback  
>> could add the future to your queue.
>
> The would be a good example for the docs.

I don't know what Thomas' use case is but I expect that taking the  
results of a future and asynchronously sticking it in another queue is  
not typical.

Cheers,
Brian

>>
>>> 3. How hard would it be to add new tasks dynamically (after a task  
>>> is executed) and have the futures object never complete?
>>
>> I'm not sure that I understand your question. You can submit new  
>> work to an Executor at until time until it is shutdown and a work  
>> item can take as long to complete as you want. If you are  
>> contemplating tasks that don't complete then maybe you could be  
>> better just scheduling a thread.
>>
>>> 4. Is there a performance evaluation of the futures code  
>>> (execution overhead) ?
>>
>> No. Scott Dial did make some performance improvements so he might  
>> have a handle on its overhead.
>
> FWIW, the source code is short and readable.  From my quick read, it  
> looks to be a relatively thin wrapper/adapter around existing  
> tools.  Most of the work still gets done by the threads or processes  
> themselves.  Think of this as a cleaner, more centralized API around  
> the current toolset -- there is no deep new technology under the hood.
>
> Raymond
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/brian%40sweetapp.com


From ncoghlan at gmail.com  Fri Dec 10 01:26:09 2010
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Fri, 10 Dec 2010 10:26:09 +1000
Subject: [Python-Dev] Format factories (was Re: sWAPcASE Was: transform()
 and untransform() methods, and the codec registry)
Message-ID: <AANLkTi=rLuuLkFzrpt7REsm91-7xfBmFcMD__B5rQxo6@mail.gmail.com>

On Fri, Dec 10, 2010 at 9:29 AM, Antoine Pitrou <solipsis at pitrou.net> wrote:
> On Thu, 09 Dec 2010 18:10:38 -0500
> Eric Smith <eric at trueblade.com> wrote:
>> If we're looking to reduce the number of methods on str, I wouldn't mind
>> seeing center() and zfill() also go away, since they're trivially
>> replaced by format().
>
> Well, it's only trivial when you know format()'s wicked mini-language by
> heart :/ ?center() is easy and obvious. zfill() is arguably a bit too
> specialized.

I've occasionally wondered if string formatting [1] and the struct
module [2] would benefit from format building functions that made them
easier to use without necessarily learning the cryptic mini-languages
off by heart.

For example, a "string.make_format_spec" function might have a signature like:

def make_format_spec(fill=None, align=None, sign=None, width=None,
precision=None, display_type='s', alternate=False, commas=False,
numeric_case=None)

"align" would accept not only the actual format symbols ('<', '>',
'=', '^'), but also the corresponding names ('left', 'right',
'numeric', 'center').

"numeric_case" would accept None, 'upper' or 'lower' (affecting the
formatting of hex and floating point values). If the stated numeric
case differs from that specified by the display type, raise a
ValueError. For an unspecified numeric case, the affected display
types would default to 'lower'.

Similarly, "display_type" would accept long names in addition to the
character codes:

's': 'str'
'b': 'binary'
'c': 'chr'
'd': 'int'
'o': 'octal'
'x', 'X': 'hex' (numeric case controls display of digits A-F)
'n': 'locale'
'e', 'E': 'exponent' (numeric case controls display of exponent as
well as infinite and NaN values)
'f', 'F': 'float' (numeric case controls display of exponent as well
as infinite and NaN values)
'g', 'G': 'general' (numeric case controls display of exponent as well
as infinite and NaN values)
'%': 'percent' (numeric case controls display of exponent as well as
infinite and NaN values)

There could also be a corresponding parse_format_spec that produced a
named tuple with the appropriate details.

Cheers,
Nick.

[1] http://docs.python.org/dev/library/string#format-specification-mini-language
[2] http://docs.python.org/dev/library/struct#format-strings

-- 
Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia

From ncoghlan at gmail.com  Fri Dec 10 01:31:43 2010
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Fri, 10 Dec 2010 10:31:43 +1000
Subject: [Python-Dev] API for the new sysconfig module
In-Reply-To: <B710BA0A-7097-4070-9DA4-E33149620833@gmail.com>
References: <B710BA0A-7097-4070-9DA4-E33149620833@gmail.com>
Message-ID: <AANLkTimw5zcOUmDaw+6mWUw8ODbKPRzpTV=1nxkvY+P9@mail.gmail.com>

On Fri, Dec 10, 2010 at 10:18 AM, Raymond Hettinger
<raymond.hettinger at gmail.com> wrote:
> Does anyone know why this needed a separate module and so many accessor functions?
> ISTM it mostly could have been reduced to single call returning a nested dictionary.

Tarek will likely answer for himself, but I believe it is a matter of
providing like-for-like replacements of existing distutils APIs,
specifically:
http://docs.python.org/dev/distutils/apiref.html#module-distutils.sysconfig

The old way required source code available ability at runtime, the new
sysconfig module captures everything it needs at build time.

Cheers,
Nick.

-- 
Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia

From guido at python.org  Fri Dec 10 01:59:11 2010
From: guido at python.org (Guido van Rossum)
Date: Thu, 9 Dec 2010 16:59:11 -0800
Subject: [Python-Dev] sWAPcASE Was: transform() and untransform()
 methods, and the codec registry
In-Reply-To: <22FDCC84-2D68-41DE-A421-80C823CB3020@gmail.com>
References: <20101202180651.F2AB0EE9CD@mail.python.org>
	<201012031016.05063.victor.stinner@haypocalc.com>
	<20101203151129.74C5222A88F@kimball.webabinitio.net>
	<AANLkTiktmxr_eH-S9ihcwFoqd6yJeN03Hes_urH7Qt3A@mail.gmail.com>
	<20101203175832.88F00236D8B@kimball.webabinitio.net>
	<AANLkTikF2w-mmNVgYch=40Rmdc_OvRV3wOYzthvNBbpO@mail.gmail.com>
	<4CFCCB7F.9080605@egenix.com>
	<AANLkTi=+9B8ez2sXkg-7uQp_SQ=zaYA_0KiydpybGuzs@mail.gmail.com>
	<AANLkTinew=h2RFmTE4p6wLJ=hHuOH7YxAn6UhdjOXimy@mail.gmail.com>
	<20101209201749.79edd5ce@pitrou.net>
	<AANLkTimY8L=TZpnazBHsTQWz_oqH6yXUjVNnWr=5=xk3@mail.gmail.com>
	<22FDCC84-2D68-41DE-A421-80C823CB3020@gmail.com>
Message-ID: <AANLkTi=j0zp5DHbsn0joWJVRFHR9vNLhH98r_9RhS5BR@mail.gmail.com>

On Thu, Dec 9, 2010 at 2:54 PM, Raymond Hettinger
<raymond.hettinger at gmail.com> wrote:
>
> On Dec 9, 2010, at 12:29 PM, Alexander Belopolsky wrote:
>
> The str type already has 40+ methods many of which are not well-suited
> to handle the complexities inherent in Unicode. ??Rather than rushing
> in two more methods that will prove to be about as useful as
> str.swapcase(), lets consider actual use cases and come up with a
> design that will properly address them.
>
> It would make me happy if we could agree to kill or at least mortally wound
> str.swapcase(). ? ?I did some research on what it is go for and found
> that it is a vestige of an old word processor command to handle
> the case where a user accidentally left the caps lock key turned-on.
> AFAICT using Google's code search, it has nearly zero value for
> Python scripts. ? It does have a cost however, the code search turned-up
> many cases where people were writing string like objects and included
> swapcase() just so they could match the built-in API.
> It's time for swapcase() to go the way of the dinosaurs.

I fully admit the mistakes I made by introducing swapcase() and a few
other weird string methods. It happened many eons ago; before they
were string methods they were functions in the "string" module.

At the same time I cannot bring up any enthusiasm for trying to kill
these off. The cost of the deprecation process is probably more than
the cost of keeping them around, if you integrate over the entire
community for t -> infinity. I don't see them as harmful. We had our
chance to kill them in Py3k, and we didn't.

-- 
--Guido van Rossum (python.org/~guido)

From greg at krypto.org  Fri Dec 10 05:24:33 2010
From: greg at krypto.org (Gregory P. Smith)
Date: Thu, 9 Dec 2010 20:24:33 -0800
Subject: [Python-Dev] Using logging in the stdlib and its unit tests
In-Reply-To: <loom.20101208T022102-46@post.gmane.org>
References: <loom.20101207T212306-892@post.gmane.org>
	<20101207215053.025ecf52@pitrou.net>
	<loom.20101208T002100-91@post.gmane.org>
	<20101208010913.363c6105@pitrou.net>
	<AANLkTim4ZPoch18+-qkunuYm-zcyh-vbhMOv9YVVEqo9@mail.gmail.com>
	<loom.20101208T022102-46@post.gmane.org>
Message-ID: <AANLkTikFNUPA5Cvbz8cOAfznSOEe906O8zOfOC=2Cj+H@mail.gmail.com>

On Tue, Dec 7, 2010 at 5:51 PM, Vinay Sajip <vinay_sajip at yahoo.co.uk> wrote:

> Nick Coghlan <ncoghlan <at> gmail.com> writes:
>
> > Indeed - I was very surprised to find just now that calling
> > "logging.warn('Whatever')" is not the same as doing "log =
> > logging.getLogger(); log.warn('Whatever')".
>
> Don't know why you'd be surprised: it's been that way since logging was
> added to
> Python, and the logging.debug() etc. are documented as convenience methods
> for
> casual use in throwaway scripts, interactive sessions etc. The convenience
> is in
> that you don't need to specify a logger (the root logger is used) and that
> basicConfig() is called for you.
>

Hahaha. :)  Well, I won't be suggesting to anyone at work that we throw away
our entire bazillion line codebase just because all of it happily relies on
logging.{debug,info,warn,error,exception} functions and all log messages go
through a single root logger.

I'd argue that anyone using a multi-logger hierarchy has already implemented
overkill and that the default for anyone wanting to log something should be
to simply call the above functions directly from the logging module.

This simplistic easy usage somewhat echo's Glenn's comment on this thread
about logging seeming way to daunting as presented today.  It needn't be.

-gps
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20101209/4b9fa3d1/attachment.html>

From greg at krypto.org  Fri Dec 10 05:29:19 2010
From: greg at krypto.org (Gregory P. Smith)
Date: Thu, 9 Dec 2010 20:29:19 -0800
Subject: [Python-Dev] Using logging in the stdlib and its unit tests
In-Reply-To: <4CFFC5B0.3040006@g.nevcal.com>
References: <loom.20101207T212306-892@post.gmane.org>
	<20101207215053.025ecf52@pitrou.net>
	<loom.20101208T002100-91@post.gmane.org>
	<20101208010913.363c6105@pitrou.net>
	<loom.20101208T020511-443@post.gmane.org>
	<20101208110309.56cf2a22@pitrou.net>
	<loom.20101208T123754-508@post.gmane.org>
	<4CFFC5B0.3040006@g.nevcal.com>
Message-ID: <AANLkTik54r_SwZP_Lvx_C0vYmqjmf364MZ0s3CNC=y=h@mail.gmail.com>

On Wed, Dec 8, 2010 at 9:51 AM, Glenn Linderman
<v+python at g.nevcal.com<v%2Bpython at g.nevcal.com>
> wrote:

>  On 12/8/2010 4:15 AM, Vinay Sajip wrote:
>
> You're complaining about too much documentation?! Don't measure it by weight!
>
>  On 12/8/2010 5:57 AM, Vinay Sajip wrote:
>
> Of course I understand I could be wrong
> about this, but I don't recall when a stdlib maintainer has said to me, "I want
> to start using logging in stdlib module XXX, but can't justify it because ..."
>
>
> So I'm a fairly new Python user, but 30 years programming experience.
>
> When I first looked at the Python logging facility (and again today, when I
> looked again), I found a simple example of debugging logging.  Then a bunch
> of stuff about how to configure rotating logs.  Then something about logging
> levels. And then a dissertation on the theory of loggers "The logging
> library takes a modular approach and offers the several categories of
> components: loggers, handlers, filters, and formatters."  And I hadn't
> gotten through 10% of the documentation yet, per the scrollbar.
>
> My reaction the first time was "Sounds sophisticated and complex.  I think
> I'll do something simpler for now, and maybe someday, when I have a spare
> week, I'll read the documentation and see if the benefits are worth the
> effort."
>
> OK, so now you've discovered that too much documentation can be a turn
> off... at least, if it is presented from the top to describe the
> sophistication of the facility, rather than how easy it is to use (if it is,
> I still haven't gotten to 10%, and I still don't know that).
>

Exactly.  All I ever recommend people do is:

import logging
...
    logging.warn('doing something a bit odd.')
...
    for x in thing:
        logging.debug('working on %r', x)
...


And be done with it.  If they are controlling their __main__ they'll
probably want to call a common function to setup the log message formatting
and where it gets written to from there.  Otherwise ignore that and leave
that up to the person using your library (directly, indirectly or otherwise)
to do that in their __main__ code.

For logging to be useful it needs to be dirt simple.  The above is.

-gps
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20101209/06191ded/attachment.html>

From vinay_sajip at yahoo.co.uk  Fri Dec 10 09:06:49 2010
From: vinay_sajip at yahoo.co.uk (Vinay Sajip)
Date: Fri, 10 Dec 2010 08:06:49 +0000 (UTC)
Subject: [Python-Dev] Using logging in the stdlib and its unit tests
References: <loom.20101207T212306-892@post.gmane.org>
	<20101207215053.025ecf52@pitrou.net>
	<loom.20101208T002100-91@post.gmane.org>
	<20101208010913.363c6105@pitrou.net>
	<AANLkTim4ZPoch18+-qkunuYm-zcyh-vbhMOv9YVVEqo9@mail.gmail.com>
	<loom.20101208T022102-46@post.gmane.org>
	<AANLkTikFNUPA5Cvbz8cOAfznSOEe906O8zOfOC=2Cj+H@mail.gmail.com>
Message-ID: <loom.20101210T085332-877@post.gmane.org>

Gregory P. Smith <greg <at> krypto.org> writes:

> Hahaha. :) ?Well, I won't be suggesting to anyone at work that we throw away
> our entire bazillion line codebase just because all of it happily relies on
> logging.{debug,info,warn,error,exception} functions and all log messages go
> through a single root logger.

That's a perfectly valid way of using logging: if the convenience functions suit
your needs, there's no need to use anything else.

> I'd argue that anyone using a multi-logger hierarchy has already implemented
> overkill and that the default for anyone wanting to log something should be to
> simply call the above functions directly from the logging module.

Some people need the logger hierarchy and it's there for them, but no-one is
forced to use it.

> 
> This simplistic easy usage somewhat echo's Glenn's comment on this thread
about logging seeming way to daunting as presented today. ?It needn't be.
> 

Indeed, and the very first code sample in the logging documentation shows
exactly the simplistic easy usage you're talking about. I can't see why anyone
would be scared off by that example.

Regards,

Vinay Sajip


From v+python at g.nevcal.com  Fri Dec 10 09:31:23 2010
From: v+python at g.nevcal.com (Glenn Linderman)
Date: Fri, 10 Dec 2010 00:31:23 -0800
Subject: [Python-Dev] Using logging in the stdlib and its unit tests
In-Reply-To: <loom.20101210T085332-877@post.gmane.org>
References: <loom.20101207T212306-892@post.gmane.org>	<20101207215053.025ecf52@pitrou.net>	<loom.20101208T002100-91@post.gmane.org>	<20101208010913.363c6105@pitrou.net>	<AANLkTim4ZPoch18+-qkunuYm-zcyh-vbhMOv9YVVEqo9@mail.gmail.com>	<loom.20101208T022102-46@post.gmane.org>	<AANLkTikFNUPA5Cvbz8cOAfznSOEe906O8zOfOC=2Cj+H@mail.gmail.com>
	<loom.20101210T085332-877@post.gmane.org>
Message-ID: <4D01E55B.2050507@g.nevcal.com>

On 12/10/2010 12:06 AM, Vinay Sajip wrote:
>> >  This simplistic easy usage somewhat echo's Glenn's comment on this thread
> about logging seeming way to daunting as presented today.  It needn't be.
>> >  
> Indeed, and the very first code sample in the logging documentation shows
> exactly the simplistic easy usage you're talking about. I can't see why anyone
> would be scared off by that example.

It didn't scare me off.  But it wasn't _quite_ comprehensive enough to 
keep me from being put off by the next bit of manual, (about 8%, as 
mentioned before), which had lots of discussion, but no more examples, 
so what is lacking up front is a few more examples of using the 
convenience methods as Greg apparently does, without configuration, but 
showing use in multiple modules.  After it is shown to be simple to get 
started in a multi-module code base, then all the refinements can be 
explained and added, but by then people are already using the logger, 
and learning becomes incremental.

Otherwise, people are scared off or put off by the need to learn lots of 
new terms (logger, filter, handler, formatter, and configuration) before 
they get any value out of it.

The next thing after the simple example is a rotating log 
configuration.  A handy feature, of course, but hardly the second thing 
that one needs to learn about logging.  Maybe the 4th or 5th or later.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20101210/1b4891f2/attachment.html>

From v+python at g.nevcal.com  Fri Dec 10 09:36:57 2010
From: v+python at g.nevcal.com (Glenn Linderman)
Date: Fri, 10 Dec 2010 00:36:57 -0800
Subject: [Python-Dev] Using logging in the stdlib and its unit tests
In-Reply-To: <AANLkTik54r_SwZP_Lvx_C0vYmqjmf364MZ0s3CNC=y=h@mail.gmail.com>
References: <loom.20101207T212306-892@post.gmane.org>
	<20101207215053.025ecf52@pitrou.net>
	<loom.20101208T002100-91@post.gmane.org>
	<20101208010913.363c6105@pitrou.net>
	<loom.20101208T020511-443@post.gmane.org>
	<20101208110309.56cf2a22@pitrou.net>
	<loom.20101208T123754-508@post.gmane.org>
	<4CFFC5B0.3040006@g.nevcal.com>
	<AANLkTik54r_SwZP_Lvx_C0vYmqjmf364MZ0s3CNC=y=h@mail.gmail.com>
Message-ID: <4D01E6A9.80209@g.nevcal.com>

On 12/9/2010 8:29 PM, Gregory P. Smith wrote:
> Exactly.  All I ever recommend people do is:
>
> import logging
> ...
>     logging.warn('doing something a bit odd.')
> ...
>     for x in thing:
>         logging.debug('working on %r', x)
> ...
>
>
> And be done with it.  If they are controlling their __main__ they'll 
> probably want to call a common function to setup the log message 
> formatting and where it gets written to from there.


Greg -- can you flesh out that last line, to remove the need for my 
parenthetical uncertainties below)?

So with your warn and debug calls, and the "common function to setup 
..." (whatever this is, and however often and from where it needs to be 
called in a multi-module environment) at the top of the logging manual, 
it might be more approachable.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20101210/b7461b44/attachment.html>

From stephen at xemacs.org  Fri Dec 10 10:47:50 2010
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Fri, 10 Dec 2010 18:47:50 +0900
Subject: [Python-Dev] Using logging in the stdlib and its unit tests
In-Reply-To: <loom.20101210T085332-877@post.gmane.org>
References: <loom.20101207T212306-892@post.gmane.org>
	<20101207215053.025ecf52@pitrou.net>
	<loom.20101208T002100-91@post.gmane.org>
	<20101208010913.363c6105@pitrou.net>
	<AANLkTim4ZPoch18+-qkunuYm-zcyh-vbhMOv9YVVEqo9@mail.gmail.com>
	<loom.20101208T022102-46@post.gmane.org>
	<AANLkTikFNUPA5Cvbz8cOAfznSOEe906O8zOfOC=2Cj+H@mail.gmail.com>
	<loom.20101210T085332-877@post.gmane.org>
Message-ID: <87tyimrmbd.fsf@uwakimon.sk.tsukuba.ac.jp>

Vinay Sajip writes:

 > Indeed, and the very first code sample in the logging documentation
 > shows exactly the simplistic easy usage you're talking about. I
 > can't see why anyone would be scared off by that example.

They're not scared by that example.  What you need is a paragraph
below it that says

    """
    Do you think the above is all you should need?  If so, you're
    right.  You can stop reading now.  If you think you need more,
    we've got that, too.  Read on (you may need more coffee).
    """ 

No?

From hrvoje.niksic at avl.com  Fri Dec 10 11:37:19 2010
From: hrvoje.niksic at avl.com (Hrvoje Niksic)
Date: Fri, 10 Dec 2010 11:37:19 +0100
Subject: [Python-Dev] Using logging in the stdlib and its unit tests
In-Reply-To: <87tyimrmbd.fsf@uwakimon.sk.tsukuba.ac.jp>
References: <loom.20101207T212306-892@post.gmane.org>	<20101207215053.025ecf52@pitrou.net>	<loom.20101208T002100-91@post.gmane.org>	<20101208010913.363c6105@pitrou.net>	<AANLkTim4ZPoch18+-qkunuYm-zcyh-vbhMOv9YVVEqo9@mail.gmail.com>	<loom.20101208T022102-46@post.gmane.org>	<AANLkTikFNUPA5Cvbz8cOAfznSOEe906O8zOfOC=2Cj+H@mail.gmail.com>	<loom.20101210T085332-877@post.gmane.org>
	<87tyimrmbd.fsf@uwakimon.sk.tsukuba.ac.jp>
Message-ID: <4D0202DF.3040707@avl.com>



On 12/10/2010 10:47 AM, Stephen J. Turnbull wrote:
> Vinay Sajip writes:
>
>   >  Indeed, and the very first code sample in the logging documentation
>   >  shows exactly the simplistic easy usage you're talking about. I
>   >  can't see why anyone would be scared off by that example.
>
> They're not scared by that example.  What you need is a paragraph
> below it that says
>
>      """
>      Do you think the above is all you should need?  If so, you're
>      right.  You can stop reading now.  If you think you need more,
>      we've got that, too.  Read on (you may need more coffee).
>      """

+1

From phd at phd.pp.ru  Fri Dec 10 12:33:27 2010
From: phd at phd.pp.ru (Oleg Broytman)
Date: Fri, 10 Dec 2010 14:33:27 +0300
Subject: [Python-Dev] Using logging in the stdlib and its unit tests
In-Reply-To: <87tyimrmbd.fsf@uwakimon.sk.tsukuba.ac.jp>
References: <loom.20101207T212306-892@post.gmane.org>
	<20101207215053.025ecf52@pitrou.net>
	<loom.20101208T002100-91@post.gmane.org>
	<20101208010913.363c6105@pitrou.net>
	<AANLkTim4ZPoch18+-qkunuYm-zcyh-vbhMOv9YVVEqo9@mail.gmail.com>
	<loom.20101208T022102-46@post.gmane.org>
	<AANLkTikFNUPA5Cvbz8cOAfznSOEe906O8zOfOC=2Cj+H@mail.gmail.com>
	<loom.20101210T085332-877@post.gmane.org>
	<87tyimrmbd.fsf@uwakimon.sk.tsukuba.ac.jp>
Message-ID: <20101210113327.GA16785@phd.pp.ru>

On Fri, Dec 10, 2010 at 06:47:50PM +0900, Stephen J. Turnbull wrote:
> Vinay Sajip writes:
>  > Indeed, and the very first code sample in the logging documentation
>  > shows exactly the simplistic easy usage you're talking about. I
>  > can't see why anyone would be scared off by that example.
> 
> They're not scared by that example.  What you need is a paragraph
> below it that says
> 
>     """
>     Do you think the above is all you should need?  If so, you're
>     right.  You can stop reading now.  If you think you need more,
>     we've got that, too.  Read on (you may need more coffee).
>     """ 

   Better yet (IMHO) would be to split the huge page into "Logging: Simple
start" and "Logging: Advanced usage (for the brave of of heart)".

Oleg.
-- 
     Oleg Broytman            http://phd.pp.ru/            phd at phd.pp.ru
           Programmers don't die, they just GOSUB without RETURN.

From vinay_sajip at yahoo.co.uk  Fri Dec 10 12:46:18 2010
From: vinay_sajip at yahoo.co.uk (Vinay Sajip)
Date: Fri, 10 Dec 2010 11:46:18 +0000 (UTC)
Subject: [Python-Dev] Using logging in the stdlib and its unit tests
References: <loom.20101207T212306-892@post.gmane.org>	<20101207215053.025ecf52@pitrou.net>	<loom.20101208T002100-91@post.gmane.org>	<20101208010913.363c6105@pitrou.net>	<loom.20101208T020511-443@post.gmane.org>	<20101208110309.56cf2a22@pitrou.net>	<loom.20101208T123754-508@post.gmane.org>	<20101208152759.32b53cb2@pitrou.net>	<loom.20101208T154218-14@post.gmane.org>	<20101208191537.44fc3ad8@pitrou.net>
	<loom.20101208T194544-118@post.gmane.org>
	<idos6q$27i$1@dough.gmane.org>
Message-ID: <loom.20101210T124320-67@post.gmane.org>

Terry Reedy <tjreedy <at> udel.edu> writes:

> Your proposal struck me as probably the best way forward. Can you code 
> it up and put a patch on the tracker that people can test before the 
> next beta?

Coded up (including unit test) and checked into py3k branch, r87157. Regressions
pass OK. Old behaviour can be obtained by setting logging.lastResort = None.

Regards,

Vinay Sajip


From solipsis at pitrou.net  Fri Dec 10 12:58:50 2010
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Fri, 10 Dec 2010 12:58:50 +0100
Subject: [Python-Dev] Using logging in the stdlib and its unit tests
References: <loom.20101207T212306-892@post.gmane.org>
	<20101207215053.025ecf52@pitrou.net>
	<loom.20101208T002100-91@post.gmane.org>
	<20101208010913.363c6105@pitrou.net>
	<loom.20101208T020511-443@post.gmane.org>
	<20101208110309.56cf2a22@pitrou.net>
	<loom.20101208T123754-508@post.gmane.org>
	<20101208152759.32b53cb2@pitrou.net>
	<loom.20101208T154218-14@post.gmane.org>
	<20101208191537.44fc3ad8@pitrou.net>
	<loom.20101208T194544-118@post.gmane.org>
	<idos6q$27i$1@dough.gmane.org>
	<loom.20101210T124320-67@post.gmane.org>
Message-ID: <20101210125850.76b6e65b@pitrou.net>


Hey Vinay,

On Fri, 10 Dec 2010 11:46:18 +0000 (UTC)
Vinay Sajip <vinay_sajip at yahoo.co.uk> wrote:
> Terry Reedy <tjreedy <at> udel.edu> writes:
> 
> > Your proposal struck me as probably the best way forward. Can you code 
> > it up and put a patch on the tracker that people can test before the 
> > next beta?
> 
> Coded up (including unit test) and checked into py3k branch, r87157. Regressions
> pass OK. Old behaviour can be obtained by setting logging.lastResort = None.

When you make significant changes outside of logging, it would be nice
if you first opened an issue and posted the patch for review. There's
nothing obviously wrong with what you committed, but I think it's
principally better to ask first for advice.

(then there's nothing wrong with committing if you didn't get an
opinion past a week or two)

Regards

Antoine.



From ncoghlan at gmail.com  Fri Dec 10 13:22:10 2010
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Fri, 10 Dec 2010 22:22:10 +1000
Subject: [Python-Dev] [Python-checkins] r87157 - in
 python/branches/py3k: Lib/logging/__init__.py Lib/test/test_logging.py
 Misc/NEWS
In-Reply-To: <20101210114257.EB346ED47@mail.python.org>
References: <20101210114257.EB346ED47@mail.python.org>
Message-ID: <AANLkTinmPx-gGKKrxHd+=X9KmgR1P=iOP_GQ7gztiEJ0@mail.gmail.com>

On Fri, Dec 10, 2010 at 9:42 PM, vinay.sajip <python-checkins at python.org> wrote:
> Author: vinay.sajip
> Date: Fri Dec 10 12:42:57 2010
> New Revision: 87157
>
> Log:
> logging: added handler of last resort.

Nice, that ended up fitting in quite neatly.

Cheers,
Nick.

-- 
Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia

From vinay_sajip at yahoo.co.uk  Fri Dec 10 13:34:48 2010
From: vinay_sajip at yahoo.co.uk (Vinay Sajip)
Date: Fri, 10 Dec 2010 12:34:48 +0000 (UTC)
Subject: [Python-Dev] Using logging in the stdlib and its unit tests
References: <loom.20101207T212306-892@post.gmane.org>
	<20101207215053.025ecf52@pitrou.net>
	<loom.20101208T002100-91@post.gmane.org>
	<20101208010913.363c6105@pitrou.net>
	<loom.20101208T020511-443@post.gmane.org>
	<20101208110309.56cf2a22@pitrou.net>
	<loom.20101208T123754-508@post.gmane.org>
	<20101208152759.32b53cb2@pitrou.net>
	<loom.20101208T154218-14@post.gmane.org>
	<20101208191537.44fc3ad8@pitrou.net>
	<loom.20101208T194544-118@post.gmane.org>
	<idos6q$27i$1@dough.gmane.org>
	<loom.20101210T124320-67@post.gmane.org>
	<20101210125850.76b6e65b@pitrou.net>
Message-ID: <loom.20101210T131723-716@post.gmane.org>

Antoine Pitrou <solipsis <at> pitrou.net> writes:

Hi Antoine,

> When you make significant changes outside of logging, it would be nice
> if you first opened an issue and posted the patch for review. There's
> nothing obviously wrong with what you committed, but I think it's
> principally better to ask first for advice.

Sorry, which significant changes outside of logging do you mean? If you're
referring to my addition of TestHandler and Matcher to test.support, I did
mention this earlier in the thread, with a link to the code (though it wasn't on
a patch added to the tracker, I agree). I can revert this change, if you want.

The changes in r87157 are to Lib/logging/__init__.py, Lib/test/test_logging.py
and Misc/NEWS - so IMO not outside of logging. Perhaps wrongly, I expected that
these straightforward-seeming changes wouldn't adversely affect anyone else's
workflow, except perhaps making it easier for Brian Quinlan to make changes to
test_concurrent_futures to resolve #10626. I did run regressions and everything
passed OK.

> (then there's nothing wrong with committing if you didn't get an
> opinion past a week or two)
 
Sorry if my eagerness has stepped on any toes - it's not intentional. The diff
of the changes is here:

https://gist.github.com/736120

Comments are of course welcome.

Regards,

Vinay Sajip



From solipsis at pitrou.net  Fri Dec 10 14:11:07 2010
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Fri, 10 Dec 2010 14:11:07 +0100
Subject: [Python-Dev] Using logging in the stdlib and its unit tests
References: <loom.20101207T212306-892@post.gmane.org>
	<20101207215053.025ecf52@pitrou.net>
	<loom.20101208T002100-91@post.gmane.org>
	<20101208010913.363c6105@pitrou.net>
	<loom.20101208T020511-443@post.gmane.org>
	<20101208110309.56cf2a22@pitrou.net>
	<loom.20101208T123754-508@post.gmane.org>
	<20101208152759.32b53cb2@pitrou.net>
	<loom.20101208T154218-14@post.gmane.org>
	<20101208191537.44fc3ad8@pitrou.net>
	<loom.20101208T194544-118@post.gmane.org>
	<idos6q$27i$1@dough.gmane.org>
	<loom.20101210T124320-67@post.gmane.org>
	<20101210125850.76b6e65b@pitrou.net>
	<loom.20101210T131723-716@post.gmane.org>
Message-ID: <20101210141107.2c21f9f4@pitrou.net>

On Fri, 10 Dec 2010 12:34:48 +0000 (UTC)
Vinay Sajip <vinay_sajip at yahoo.co.uk> wrote:
> > When you make significant changes outside of logging, it would be nice
> > if you first opened an issue and posted the patch for review. There's
> > nothing obviously wrong with what you committed, but I think it's
> > principally better to ask first for advice.
> 
> Sorry, which significant changes outside of logging do you mean? If you're
> referring to my addition of TestHandler and Matcher to test.support, I did
> mention this earlier in the thread, with a link to the code (though it wasn't on
> a patch added to the tracker, I agree). I can revert this change, if you want.

Yes, I am talking about this one. I am not asking that you revert it; I
am just saying that it is generally appreciated if substantial patches
get proposed on the tracker before being committed.

In this case since it's in test.support we can probably improve it on
the fly, of course.

Regards

Antoine.



From tnagyemail-mail at yahoo.fr  Fri Dec 10 14:36:17 2010
From: tnagyemail-mail at yahoo.fr (Thomas Nagy)
Date: Fri, 10 Dec 2010 13:36:17 +0000 (GMT)
Subject: [Python-Dev] futures API
Message-ID: <774742.79683.qm@web27506.mail.ukl.yahoo.com>

--- El jue, 9/12/10, Brian Quinlan escribi?:
> On Dec 9, 2010, at 4:26 AM, Thomas Nagy wrote:
> 
> > I am looking forward to replacing a piece of code (http://code.google.com/p/waf/source/browse/trunk/waflib/Runner.py#86)
> by the futures module which was announced in python 3.2
> beta. I am a bit stuck with it, so I have a few questions
> about the futures:
> > 
> > 1. Is the futures API frozen?
> 
> Yes.
> 
> > 2. How hard would it be to return the tasks processed
> in an output queue to process/consume the results while they
> are returned? The code does not seem to be very open for
> monkey patching.
> 
> You can associate a callback with a submitted future. That
> callback could add the future to your queue.

Ok, it works. I was thinking the object was cleaned up immediately after it was used.

> > 3. How hard would it be to add new tasks dynamically
> (after a task is executed) and have the futures object never
> complete?
> 
> I'm not sure that I understand your question. You can
> submit new work to an Executor at until time until it is
> shutdown and a work item can take as long to complete as you
> want. If you are contemplating tasks that don't complete
> then maybe you could be better just scheduling a thread.
> 
> > 4. Is there a performance evaluation of the futures
> code (execution overhead) ?
> 
> No. Scott Dial did make some performance improvements so he
> might have a handle on its overhead.

Ok.

I have a process running for a long time, and which may use futures of different max_workers count. I think it is not too far-fetched to create a new futures object each time. Yet, the execution becomes slower after each call, for example with http://freehackers.org/~tnagy/futures_test.py:

"""
import concurrent.futures
from queue import Queue
import datetime

class counter(object):
? ? def __init__(self, fut):
? ? ? ? self.fut = fut

? ? def run(self):
? ? ? ? def look_busy(num, obj):
? ? ? ? ? ? tot = 0
? ? ? ? ? ? for x in range(num):
? ? ? ? ? ? ? ? tot += x
? ? ? ? ? ? obj.out_q.put(tot)

? ? ? ? start = datetime.datetime.utcnow()
? ? ? ? self.count = 0
? ? ? ? self.out_q = Queue(0)
? ? ? ? for x in range(1000):
? ? ? ? ? ? self.count += 1
? ? ? ? ? ? self.fut.submit(look_busy, self.count, self)

? ? ? ? while self.count:
? ? ? ? ? ? self.count -= 1
? ? ? ? ? ? self.out_q.get()

? ? ? ? delta = datetime.datetime.utcnow() - start
? ? ? ? print(delta.total_seconds())

fut = concurrent.futures.ThreadPoolExecutor(max_workers=20)
for x in range(100):
? ? # comment the following line
? ? fut = concurrent.futures.ThreadPoolExecutor(max_workers=20)
? ? c = counter(fut)
? ? c.run()
"""

The runtime grows after each step:
0.216451
0.225186
0.223725
0.222274
0.230964
0.240531
0.24137
0.252393
0.249948
0.257153
...

Is there a mistake in this piece of code?

Thanks,
Thomas



      

From vinay_sajip at yahoo.co.uk  Fri Dec 10 14:37:22 2010
From: vinay_sajip at yahoo.co.uk (Vinay Sajip)
Date: Fri, 10 Dec 2010 13:37:22 +0000 (UTC)
Subject: [Python-Dev] Using logging in the stdlib and its unit tests
References: <loom.20101207T212306-892@post.gmane.org>
	<20101207215053.025ecf52@pitrou.net>
	<loom.20101208T002100-91@post.gmane.org>
	<20101208010913.363c6105@pitrou.net>
	<loom.20101208T020511-443@post.gmane.org>
	<20101208110309.56cf2a22@pitrou.net>
	<loom.20101208T123754-508@post.gmane.org>
	<20101208152759.32b53cb2@pitrou.net>
	<loom.20101208T154218-14@post.gmane.org>
	<20101208191537.44fc3ad8@pitrou.net>
	<loom.20101208T194544-118@post.gmane.org>
	<idos6q$27i$1@dough.gmane.org>
	<loom.20101210T124320-67@post.gmane.org>
	<20101210125850.76b6e65b@pitrou.net>
	<loom.20101210T131723-716@post.gmane.org>
	<20101210141107.2c21f9f4@pitrou.net>
Message-ID: <loom.20101210T143457-74@post.gmane.org>

Antoine Pitrou <solipsis <at> pitrou.net> writes:


> Yes, I am talking about this one. I am not asking that you revert it; I
> am just saying that it is generally appreciated if substantial patches
> get proposed on the tracker before being committed.
> 

OK, sorry - point taken.

> In this case since it's in test.support we can probably improve it on
> the fly, of course.

For sure.

Regards,

Vinay


From merwok at netwok.org  Fri Dec 10 15:20:17 2010
From: merwok at netwok.org (=?UTF-8?B?w4lyaWMgQXJhdWpv?=)
Date: Fri, 10 Dec 2010 15:20:17 +0100
Subject: [Python-Dev] API for the new sysconfig module
In-Reply-To: <AANLkTimw5zcOUmDaw+6mWUw8ODbKPRzpTV=1nxkvY+P9@mail.gmail.com>
References: <B710BA0A-7097-4070-9DA4-E33149620833@gmail.com>
	<AANLkTimw5zcOUmDaw+6mWUw8ODbKPRzpTV=1nxkvY+P9@mail.gmail.com>
Message-ID: <4D023721.1070904@netwok.org>

Hi,

Original discussion for the sysconfig module was short:
http://mail.python.org/pipermail/python-dev/2009-May/089520.html

Tarek will reply better, but I think the issue to solve was to move
sysconfig out of distutils, improving its API a bit in the process but
not overhauling it completely.  A single function returning a nested
dictionary could be added if deemed useful, but to me the existing API
separates concerns (paths and variables for example) without being
confusing nor having too many names.

Exceptions: get_python_version could be removed; get_scheme_names and
get_path_names could be replaced by constants.  Maybe Tarek chose
accessors to keep freedom for later, or just for consistency.

sysconfig being new in 2.7 and 3.2, I don?t know how compatibility
concerns apply.  I think it would be bad to have highlighted the new
module in 2.7 and then remove it in 3.2, since 2.6 and 2.7 have the goal
of helping people moving to 3.x.  Also, I don?t see where those 800
lines of code would live if the sysconfig module is removed; maybe in
platform, which is already long? (it could be made smaller with some
code modernization, removal of the popen function, but not much smaller).

Final note: with 3.2 being in beta, I don?t know how much can be changed
now.

> The old way required source code available ability at runtime, the new
> sysconfig module captures everything it needs at build time.
Not yet :)  See http://bugs.python.org/issue9878

Regards


From stephen at xemacs.org  Fri Dec 10 17:01:10 2010
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Sat, 11 Dec 2010 01:01:10 +0900
Subject: [Python-Dev] Using logging in the stdlib and its unit tests
In-Reply-To: <20101210113327.GA16785@phd.pp.ru>
References: <loom.20101207T212306-892@post.gmane.org>
	<20101207215053.025ecf52@pitrou.net>
	<loom.20101208T002100-91@post.gmane.org>
	<20101208010913.363c6105@pitrou.net>
	<AANLkTim4ZPoch18+-qkunuYm-zcyh-vbhMOv9YVVEqo9@mail.gmail.com>
	<loom.20101208T022102-46@post.gmane.org>
	<AANLkTikFNUPA5Cvbz8cOAfznSOEe906O8zOfOC=2Cj+H@mail.gmail.com>
	<loom.20101210T085332-877@post.gmane.org>
	<87tyimrmbd.fsf@uwakimon.sk.tsukuba.ac.jp>
	<20101210113327.GA16785@phd.pp.ru>
Message-ID: <87oc8tsjll.fsf@uwakimon.sk.tsukuba.ac.jp>

Oleg Broytman writes:

 >    Better yet (IMHO) would be to split the huge page into "Logging: Simple
 > start" and "Logging: Advanced usage (for the brave of of heart)".

Splitting is OK, but I disagree about the gloss "for the brave of
heart".

In my experience, if it is a YAGNI, the complexity is nearly
impenetrable.  If you *do* need it, it's not at all difficult to
understand what the complexity is for, and it doesn't even look all
that complex because it matches up with the problem you need to solve.

If the documentation is still a deterrent, that's a problem with the
documentation and it should be improved.  AFAICT, making it clear that
exporting all the internal flexibility is for those who need it, while
most users will rarely need it, should be enough.  But I'm not a good
test case, since I already am familiar with XEmacs's similar system.

From status at bugs.python.org  Fri Dec 10 18:07:04 2010
From: status at bugs.python.org (Python tracker)
Date: Fri, 10 Dec 2010 18:07:04 +0100 (CET)
Subject: [Python-Dev] Summary of Python tracker Issues
Message-ID: <20101210170704.5A4541DBA6@psf.upfronthosting.co.za>


ACTIVITY SUMMARY (2010-12-03 - 2010-12-10)
Python tracker at http://bugs.python.org/

To view or respond to any of the issues listed below, click on the issue.
Do NOT respond to this message.

Issues counts and deltas:
  open    2557 (+20)
  closed 19899 (+38)
  total  22456 (+58)

Open issues with patches: 1083 


Issues opened (40)
==================

#9101: reference json format in file formats chapter
http://bugs.python.org/issue9101  reopened by eric.araujo

#10553: Add optimize argument to builtin compile() and byte-compilatio
http://bugs.python.org/issue10553  reopened by eric.araujo

#10617: Collections ABCs can???t be linked to
http://bugs.python.org/issue10617  opened by eric.araujo

#10618: regression in subprocess.call() command quoting
http://bugs.python.org/issue10618  opened by djc

#10619: Failed module loading in test discovery loses traceback
http://bugs.python.org/issue10619  opened by michael.foord

#10620: `python -m uniittest` should work with file paths as well as t
http://bugs.python.org/issue10620  opened by michael.foord

#10622: WebKit browsers show superfluous scrollbars in html docs
http://bugs.python.org/issue10622  opened by davide.rizzo

#10626: Bad interaction between test_logging and test_concurrent_futur
http://bugs.python.org/issue10626  opened by pitrou

#10627: Remove usage of deprecated configparser.ConfigParser class in 
http://bugs.python.org/issue10627  opened by lukasz.langa

#10631: ZipFile and current directory change
http://bugs.python.org/issue10631  opened by ocean-city

#10632: multiprocessing generates a fatal error
http://bugs.python.org/issue10632  opened by bquinlan

#10634: Windows timezone changes not reflected by time.localtime
http://bugs.python.org/issue10634  opened by eric.pruitt

#10635: Calling subprocess.Popen with preexec_fn=signal.pause blocks f
http://bugs.python.org/issue10635  opened by joseph.h.garvin

#10636: subprocess module has race	condition with SIGCHLD	handlers
http://bugs.python.org/issue10636  opened by joseph.h.garvin

#10639: reindent.py converts newlines to platform default
http://bugs.python.org/issue10639  opened by jaraco

#10640: SystemError on pickling bytes >= 4GB
http://bugs.python.org/issue10640  opened by hagen

#10641: kill_python sometimes fails to kill processes on buildbots
http://bugs.python.org/issue10641  opened by db3l

#10642: site.py crashes on python startup due to defective .pth file
http://bugs.python.org/issue10642  opened by dwr2

#10644: socket loses data, calling send()/sendall() on invalid socket 
http://bugs.python.org/issue10644  opened by diekmann

#10645: egg-info file in lib-dynload
http://bugs.python.org/issue10645  opened by ronaldoussoren

#10646: ntpath.samefile doesn't work for hard links
http://bugs.python.org/issue10646  opened by brian.curtin

#10647: scrollbar crash in non-US locale format settings
http://bugs.python.org/issue10647  opened by hfischer

#10648: Extend peepholer to reverse loads or stores instead of build/u
http://bugs.python.org/issue10648  opened by serprex

#10650: decimal.py: quantize(): excess digits with watchexp=0
http://bugs.python.org/issue10650  opened by skrah

#10652: test___all_ + test_tcl fails (Windows installed binary)
http://bugs.python.org/issue10652  opened by ocean-city

#10653: test_time test_strptime fails on windows
http://bugs.python.org/issue10653  opened by ocean-city

#10654: test_datetime fails on Python3.2 windows binary
http://bugs.python.org/issue10654  opened by ocean-city

#10655: Wrong powerpc define in Python/ceval.c
http://bugs.python.org/issue10655  opened by adrian

#10656: "Out of tree" build fails on AIX 5.3
http://bugs.python.org/issue10656  opened by pedz

#10657: os.lstat/os.stat/os.fstat don't set st_dev (st_rdev) on Window
http://bugs.python.org/issue10657  opened by ocean-city

#10663: configure shouldn't set a default OPT
http://bugs.python.org/issue10663  opened by pitrou

#10664: xml.sax.expatreader should support namespace prefixes
http://bugs.python.org/issue10664  opened by fdrake

#10665: Update and expand unicodedata module documentation
http://bugs.python.org/issue10665  opened by belopolsky

#10666: OS X installer variants have confusing readline differences
http://bugs.python.org/issue10666  opened by ned.deily

#10667: collections.Counter object in C
http://bugs.python.org/issue10667  opened by jpeel

#10669: Remove Deprecation Warnings
http://bugs.python.org/issue10669  opened by RusiMody

#10670: Provide search scope limits
http://bugs.python.org/issue10670  opened by techtonik

#10671: urllib2 redirect to another host doesn't work
http://bugs.python.org/issue10671  opened by kirax

#10673: multiprocess.Process join method - timeout indistinguishable f
http://bugs.python.org/issue10673  opened by Brian.Cain

#10674: Unused dictmaker symbol in 2.* grammar
http://bugs.python.org/issue10674  opened by salty-horse



Most recent 15 issues with no replies (15)
==========================================

#10674: Unused dictmaker symbol in 2.* grammar
http://bugs.python.org/issue10674

#10670: Provide search scope limits
http://bugs.python.org/issue10670

#10666: OS X installer variants have confusing readline differences
http://bugs.python.org/issue10666

#10664: xml.sax.expatreader should support namespace prefixes
http://bugs.python.org/issue10664

#10663: configure shouldn't set a default OPT
http://bugs.python.org/issue10663

#10657: os.lstat/os.stat/os.fstat don't set st_dev (st_rdev) on Window
http://bugs.python.org/issue10657

#10656: "Out of tree" build fails on AIX 5.3
http://bugs.python.org/issue10656

#10646: ntpath.samefile doesn't work for hard links
http://bugs.python.org/issue10646

#10632: multiprocessing generates a fatal error
http://bugs.python.org/issue10632

#10619: Failed module loading in test discovery loses traceback
http://bugs.python.org/issue10619

#10617: Collections ABCs can???t be linked to
http://bugs.python.org/issue10617

#10612: StopTestRun exception to halt test run
http://bugs.python.org/issue10612

#10608: Add a section to Windows FAQ explaining os.symlink
http://bugs.python.org/issue10608

#10605: ElementTree documentation
http://bugs.python.org/issue10605

#10595: Adding a syslog.conf reader in syslog
http://bugs.python.org/issue10595



Most recent 15 issues waiting for review (15)
=============================================

#10673: multiprocess.Process join method - timeout indistinguishable f
http://bugs.python.org/issue10673

#10671: urllib2 redirect to another host doesn't work
http://bugs.python.org/issue10671

#10667: collections.Counter object in C
http://bugs.python.org/issue10667

#10665: Update and expand unicodedata module documentation
http://bugs.python.org/issue10665

#10657: os.lstat/os.stat/os.fstat don't set st_dev (st_rdev) on Window
http://bugs.python.org/issue10657

#10653: test_time test_strptime fails on windows
http://bugs.python.org/issue10653

#10652: test___all_ + test_tcl fails (Windows installed binary)
http://bugs.python.org/issue10652

#10648: Extend peepholer to reverse loads or stores instead of build/u
http://bugs.python.org/issue10648

#10639: reindent.py converts newlines to platform default
http://bugs.python.org/issue10639

#10627: Remove usage of deprecated configparser.ConfigParser class in 
http://bugs.python.org/issue10627

#10626: Bad interaction between test_logging and test_concurrent_futur
http://bugs.python.org/issue10626

#10622: WebKit browsers show superfluous scrollbars in html docs
http://bugs.python.org/issue10622

#10615: Trivial mingw compile fixes
http://bugs.python.org/issue10615

#10611: sys.exit() in a test causes a test run to die
http://bugs.python.org/issue10611

#10598: curses fails to import on Solaris
http://bugs.python.org/issue10598



Top 10 most discussed issues (10)
=================================

#10516: Add list.clear() and list.copy()
http://bugs.python.org/issue10516  22 msgs

#10615: Trivial mingw compile fixes
http://bugs.python.org/issue10615  19 msgs

#10626: Bad interaction between test_logging and test_concurrent_futur
http://bugs.python.org/issue10626  15 msgs

#6697: Check that _PyUnicode_AsString() result is not NULL
http://bugs.python.org/issue6697  13 msgs

#10367: "python setup.py sdist upload --show-response" can fail with "
http://bugs.python.org/issue10367  10 msgs

#8194: Incompatible API change in xmlrpclib.Transport.parse_response(
http://bugs.python.org/issue8194   9 msgs

#10453: Add -h/--help option to compileall
http://bugs.python.org/issue10453   9 msgs

#10653: test_time test_strptime fails on windows
http://bugs.python.org/issue10653   9 msgs

#10654: test_datetime fails on Python3.2 windows binary
http://bugs.python.org/issue10654   9 msgs

#9517: Make test.script_helper more comprehensive, and use it in the 
http://bugs.python.org/issue9517   7 msgs



Issues closed (37)
==================

#7245: better Ctrl-C support in pdb (program can be resumed) (issue21
http://bugs.python.org/issue7245  closed by georg.brandl

#7905: Shelf 'keyencoding' keyword argument is undocumented and does 
http://bugs.python.org/issue7905  closed by georg.brandl

#9227: can't import Tkinter / use IDLE after installing Python 2.7 on
http://bugs.python.org/issue9227  closed by ronaldoussoren

#9518: PyModuleDef_HEAD_INIT does not explicitly initialize all field
http://bugs.python.org/issue9518  closed by dmalcolm

#10107: Quitting IDLE on Mac doesn't save unsaved code
http://bugs.python.org/issue10107  closed by ronaldoussoren

#10272: SSL timeouts should raise socket.timeout, not a generic SSLErr
http://bugs.python.org/issue10272  closed by pitrou

#10292: tarinfo should use relative symlinks
http://bugs.python.org/issue10292  closed by lars.gustaebel

#10406: IDLE 2.7 on OS X does not enable Rstrip extension by default
http://bugs.python.org/issue10406  closed by ronaldoussoren

#10478: Ctrl-C locks up the interpreter
http://bugs.python.org/issue10478  closed by pitrou

#10545: remove or rewrite "Using Backslash to Continue Statements" ant
http://bugs.python.org/issue10545  closed by terry.reedy

#10546: UTF-16-LE and UTF-16-BE support non-BMP characters
http://bugs.python.org/issue10546  closed by haypo

#10596: modulo operator bug
http://bugs.python.org/issue10596  closed by mark.dickinson

#10601: sys.displayhook: use backslashreplace error handler if repr(va
http://bugs.python.org/issue10601  closed by haypo

#10621: >>> 1 + 2j --> (1 + 2j) and not (1+2j)
http://bugs.python.org/issue10621  closed by mark.dickinson

#10623: What???s New In Python 3.2 document refers to PEP 382: Definin
http://bugs.python.org/issue10623  closed by loewis

#10624: Move requires_IEEE_754 decorator from test_complex into test.s
http://bugs.python.org/issue10624  closed by eric.smith

#10625: There is no test for repr(complex(-0.,1.)) special handling
http://bugs.python.org/issue10625  closed by eric.smith

#10628: Typos in 3.2 what???s new
http://bugs.python.org/issue10628  closed by georg.brandl

#10629: Arbitrary precision
http://bugs.python.org/issue10629  closed by georg.brandl

#10630: dict_proxy.keys() / values() / items() are lists
http://bugs.python.org/issue10630  closed by benjamin.peterson

#10633: string.format() Unexpected output with numeric '#' prefix and 
http://bugs.python.org/issue10633  closed by r.david.murray

#10637: Calling CloseHandle twice	(Modules/posixmodule.c)
http://bugs.python.org/issue10637  closed by ocean-city

#10638: PyArg_ParseTuple: refcount in nested tuples
http://bugs.python.org/issue10638  closed by arigo

#10643: OSX proxy exceptions parse bug in urllib (crash)
http://bugs.python.org/issue10643  closed by ned.deily

#10649: Attempting to override special methods of a function object do
http://bugs.python.org/issue10649  closed by r.david.murray

#10651: chr(13) is added before chr(10) when issuing ".write" under wi
http://bugs.python.org/issue10651  closed by amaury.forgeotdarc

#10658: Link to source code is broken
http://bugs.python.org/issue10658  closed by georg.brandl

#10659: Hook scripts for immediate doc build system
http://bugs.python.org/issue10659  closed by georg.brandl

#10660: format() to lower and uppercase
http://bugs.python.org/issue10660  closed by r.david.murray

#10661: ElementTree QName has a very uninformative repr()
http://bugs.python.org/issue10661  closed by georg.brandl

#10662: Typo in concurrent.futures, "seperate"
http://bugs.python.org/issue10662  closed by georg.brandl

#10668: Array tests have nonsense in them
http://bugs.python.org/issue10668  closed by georg.brandl

#10672: ["with"] new in version 2.6 instead of 2.5
http://bugs.python.org/issue10672  closed by georg.brandl

#1772833: -q (quiet) option for python interpreter
http://bugs.python.org/issue1772833  closed by georg.brandl

#1513299: Clean up usage of map() in the stdlib
http://bugs.python.org/issue1513299  closed by georg.brandl

#1569291: Speed-up in array_repeat()
http://bugs.python.org/issue1569291  closed by georg.brandl

#1571170: Some numeric characters are still not recognized
http://bugs.python.org/issue1571170  closed by andersch

From ncoghlan at gmail.com  Fri Dec 10 18:31:08 2010
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sat, 11 Dec 2010 03:31:08 +1000
Subject: [Python-Dev] futures API
In-Reply-To: <774742.79683.qm@web27506.mail.ukl.yahoo.com>
References: <774742.79683.qm@web27506.mail.ukl.yahoo.com>
Message-ID: <AANLkTi=k64fat5O0fY3udCXoxoT+TY=ieV0gQC-4A3gX@mail.gmail.com>

On Fri, Dec 10, 2010 at 11:36 PM, Thomas Nagy <tnagyemail-mail at yahoo.fr> wrote:
> fut = concurrent.futures.ThreadPoolExecutor(max_workers=20)
> for x in range(100):
> ? ? # comment the following line
> ? ? fut = concurrent.futures.ThreadPoolExecutor(max_workers=20)
> ? ? c = counter(fut)
> ? ? c.run()
> """
> The runtime grows after each step:
> Is there a mistake in this piece of code?

This isn't an "overhead" question, it's a "how prompt is the resource
release" question.

Given that you've created circular references between the futures and
your counter class, the answer is probably "not prompt at all".

Cheers,
Nick.

-- 
Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia

From brian at sweetapp.com  Fri Dec 10 18:41:52 2010
From: brian at sweetapp.com (Brian Quinlan)
Date: Fri, 10 Dec 2010 09:41:52 -0800
Subject: [Python-Dev] futures API
In-Reply-To: <774742.79683.qm@web27506.mail.ukl.yahoo.com>
References: <774742.79683.qm@web27506.mail.ukl.yahoo.com>
Message-ID: <961F4C62-DD20-4C11-88B0-A45427B27616@sweetapp.com>

Oops. I accidentally replied off-list:


On Dec 10, 2010, at 5:36 AM, Thomas Nagy wrote:

> --- El jue, 9/12/10, Brian Quinlan escribi?:
>> On Dec 9, 2010, at 4:26 AM, Thomas Nagy wrote:
>>
>>> I am looking forward to replacing a piece of code (http://code.google.com/p/waf/source/browse/trunk/waflib/Runner.py#86 
>>> )
>> by the futures module which was announced in python 3.2
>> beta. I am a bit stuck with it, so I have a few questions
>> about the futures:
>>>
>>> 1. Is the futures API frozen?
>>
>> Yes.
>>
>>> 2. How hard would it be to return the tasks processed
>> in an output queue to process/consume the results while they
>> are returned? The code does not seem to be very open for
>> monkey patching.
>>
>> You can associate a callback with a submitted future. That
>> callback could add the future to your queue.
>
> Ok, it works. I was thinking the object was cleaned up immediately  
> after it was used.
>
>>> 3. How hard would it be to add new tasks dynamically
>> (after a task is executed) and have the futures object never
>> complete?
>>
>> I'm not sure that I understand your question. You can
>> submit new work to an Executor at until time until it is
>> shutdown and a work item can take as long to complete as you
>> want. If you are contemplating tasks that don't complete
>> then maybe you could be better just scheduling a thread.
>>
>>> 4. Is there a performance evaluation of the futures
>> code (execution overhead) ?
>>
>> No. Scott Dial did make some performance improvements so he
>> might have a handle on its overhead.
>
> Ok.
>
> I have a process running for a long time, and which may use futures  
> of different max_workers count. I think it is not too far-fetched to  
> create a new futures object each time. Yet, the execution becomes  
> slower after each call, for example with http://freehackers.org/~tnagy/futures_test.py 
> :
>
> """
> import concurrent.futures
> from queue import Queue
> import datetime
>
> class counter(object):
>    def __init__(self, fut):
>        self.fut = fut
>
>    def run(self):
>        def look_busy(num, obj):
>            tot = 0
>            for x in range(num):
>                tot += x
>            obj.out_q.put(tot)
>
>        start = datetime.datetime.utcnow()
>        self.count = 0
>        self.out_q = Queue(0)
>        for x in range(1000):
>            self.count += 1
>            self.fut.submit(look_busy, self.count, self)
>
>        while self.count:
>            self.count -= 1
>            self.out_q.get()
>
>        delta = datetime.datetime.utcnow() - start
>        print(delta.total_seconds())
>
> fut = concurrent.futures.ThreadPoolExecutor(max_workers=20)
> for x in range(100):
>    # comment the following line
>    fut = concurrent.futures.ThreadPoolExecutor(max_workers=20)
>    c = counter(fut)
>    c.run()
> """
>
> The runtime grows after each step:
> 0.216451
> 0.225186
> 0.223725
> 0.222274
> 0.230964
> 0.240531
> 0.24137
> 0.252393
> 0.249948
> 0.257153
> ...
>
> Is there a mistake in this piece of code?

There is no mistake that I can see but I suspect that the circular  
references that you are building are causing the ThreadPoolExecutor to  
take a long time to be collected. Try adding:

	c = counter(fut)
	c.run()
+	fut.shutdown()

Even if that fixes your problem, I still don't fully understand these  
numbers because I would expect the runtime to fall after a while as  
ThreadPoolExecutors are collected.

Cheers,
Brian


> Thanks,
> Thomas
>
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20101210/1b45d817/attachment.html>

From guido at python.org  Fri Dec 10 18:54:56 2010
From: guido at python.org (Guido van Rossum)
Date: Fri, 10 Dec 2010 09:54:56 -0800
Subject: [Python-Dev] transform() and untransform() methods,
	and the codec registry
In-Reply-To: <20101209201749.79edd5ce@pitrou.net>
References: <20101202180651.F2AB0EE9CD@mail.python.org>
	<201012031016.05063.victor.stinner@haypocalc.com>
	<20101203151129.74C5222A88F@kimball.webabinitio.net>
	<AANLkTiktmxr_eH-S9ihcwFoqd6yJeN03Hes_urH7Qt3A@mail.gmail.com>
	<20101203175832.88F00236D8B@kimball.webabinitio.net>
	<AANLkTikF2w-mmNVgYch=40Rmdc_OvRV3wOYzthvNBbpO@mail.gmail.com>
	<4CFCCB7F.9080605@egenix.com>
	<AANLkTi=+9B8ez2sXkg-7uQp_SQ=zaYA_0KiydpybGuzs@mail.gmail.com>
	<AANLkTinew=h2RFmTE4p6wLJ=hHuOH7YxAn6UhdjOXimy@mail.gmail.com>
	<20101209201749.79edd5ce@pitrou.net>
Message-ID: <AANLkTi=qrAPNTo9_mBCwEdsUZCJdSZBntC5cUf+KV2-A@mail.gmail.com>

On Thu, Dec 9, 2010 at 11:17 AM, Antoine Pitrou <solipsis at pitrou.net> wrote:
> On Thu, 9 Dec 2010 13:55:08 -0500
> Alexander Belopolsky <alexander.belopolsky at gmail.com> wrote:
>
>> On Thu, Dec 9, 2010 at 1:42 PM, Guido van Rossum <guido at python.org> wrote:
>> ..
>> > string-string transforms use the same namespace even though the
>> > typical transform only supports one or the other. E.g. IMO all of the
>> > following should raise LookupError:
>> >
>> >>>> b'abc'.transform('rot13')
>> > Traceback (most recent call last):
>> > ..
>> > ? ?return (input.translate(rot13_map), len(input))
>> > TypeError: expected an object with the buffer interface
>>
>> This is actually *very* misleading because
>>
>> >>> 'abc'.transform('rot13')
>> 'nop'
>>
>> works even though 'abc' is not "an object with the buffer interface".
>
> Agreed. It was already pointed out in the parent thread.
> I would say my opinion on keeping transform()/untransform() is +0 if
> these error messages (and preferably the exception type as well) get
> improved. Otherwise we'd better revert them and add a more polished
> version in 3.3.

Let me change my opinion then and say that I'm -1 on keeping the
confusing situation. It can be fixed either by postponing transform()
until 3.3 or by fixing the error checking and namespace issue.

Sorry about the reversal of pronouncements, but my original approval
was very weak indeed and mostly driven by "stick to status quo unless
there's a serious problem". The followup made it clear that the errors
and namespace issue are more seriously than I estimated at first.

-- 
--Guido van Rossum (python.org/~guido)

From janssen at parc.com  Fri Dec 10 19:16:18 2010
From: janssen at parc.com (Bill Janssen)
Date: Fri, 10 Dec 2010 10:16:18 PST
Subject: [Python-Dev] Using logging in the stdlib and its unit tests
In-Reply-To: <4D01E55B.2050507@g.nevcal.com>
References: <loom.20101207T212306-892@post.gmane.org>
	<20101207215053.025ecf52@pitrou.net>
	<loom.20101208T002100-91@post.gmane.org>
	<20101208010913.363c6105@pitrou.net>
	<AANLkTim4ZPoch18+-qkunuYm-zcyh-vbhMOv9YVVEqo9@mail.gmail.com>
	<loom.20101208T022102-46@post.gmane.org>
	<AANLkTikFNUPA5Cvbz8cOAfznSOEe906O8zOfOC=2Cj+H@mail.gmail.com>
	<loom.20101210T085332-877@post.gmane.org>
	<4D01E55B.2050507@g.nevcal.com>
Message-ID: <3231.1292004978@parc.com>

Glenn Linderman <v+python at g.nevcal.com> wrote:

> On 12/10/2010 12:06 AM, Vinay Sajip wrote:
> >> >  This simplistic easy usage somewhat echo's Glenn's comment on this thread
> > about logging seeming way to daunting as presented today.  It needn't be.
> >> >  
> > Indeed, and the very first code sample in the logging documentation shows
> > exactly the simplistic easy usage you're talking about. I can't see why anyone
> > would be scared off by that example.
> 
> It didn't scare me off.  But it wasn't _quite_ comprehensive enough to
> keep me from being put off by the next bit of manual, (about 8%, as
> mentioned before), which had lots of discussion, but no more examples,
> so what is lacking up front is a few more examples of using the
> convenience methods as Greg apparently does, without configuration,
> but showing use in multiple modules.  After it is shown to be simple
> to get started in a multi-module code base, then all the refinements
> can be explained and added, but by then people are already using the
> logger, and learning becomes incremental.

Just a note: after spacing out over this thread for a bit, I thought I'd
try out logging, which I'd never tried before, in some code I was
writing.  I added a couple of lines like

    logging.info("foo is %s", foo)

where I'd usually write

    sys.stderr.write("foo is %s\n" % foo)

Liked it just fine -- easier to write.  But nothing came out!  Because
it's got this odd idea (from Java?) about "severities" of messages, and
by default passes nothing less "threatening" than warnings, because
logger.root is initialized by default to WARNING.

So, still a bit disconcerting for newbies.

Bill

From tnagyemail-mail at yahoo.fr  Fri Dec 10 19:51:11 2010
From: tnagyemail-mail at yahoo.fr (Thomas Nagy)
Date: Fri, 10 Dec 2010 18:51:11 +0000 (GMT)
Subject: [Python-Dev] futures API
Message-ID: <441626.55995.qm@web27505.mail.ukl.yahoo.com>

--- El vie, 10/12/10, Brian Quinlan escribi?:
> On Dec 10, 2010, at 5:36 AM, Thomas Nagy wrote: 
> > I have a process running for a long time, and which
> may use futures of different max_workers count. I think it
> is not too far-fetched to create a new futures object each
> time. Yet, the execution becomes slower after each call, for
> example with http://freehackers.org/~tnagy/futures_test.py:
> > 
> > """
> > import concurrent.futures
> > from queue import Queue
> > import datetime
> > 
> > class counter(object):
> >? ???def __init__(self, fut):
> >? ? ? ???self.fut = fut
> > 
> >? ???def run(self):
> >? ? ? ???def
> look_busy(num, obj):
> >? ? ? ? ?
> ???tot = 0
> >? ? ? ? ?
> ???for x in range(num):
> >? ? ? ? ? ? ?
> ???tot += x
> >? ? ? ? ?
> ???obj.out_q.put(tot)
> > 
> >? ? ? ???start =
> datetime.datetime.utcnow()
> >? ? ? ???self.count = 0
> >? ? ? ???self.out_q =
> Queue(0)
> >? ? ? ???for x in
> range(1000):
> >? ? ? ? ?
> ???self.count += 1
> >? ? ? ? ?
> ???self.fut.submit(look_busy, self.count,
> self)
> > 
> >? ? ? ???while
> self.count:
> >? ? ? ? ?
> ???self.count -= 1
> >? ? ? ? ?
> ???self.out_q.get()
> > 
> >? ? ? ???delta =
> datetime.datetime.utcnow() - start
> >? ? ?
> ???print(delta.total_seconds())
> > 
> > fut =
> concurrent.futures.ThreadPoolExecutor(max_workers=20)
> > for x in range(100):
> >? ???# comment the following line
> >? ???fut =
> concurrent.futures.ThreadPoolExecutor(max_workers=20)
> >? ???c = counter(fut)
> >? ???c.run()
> > """
> > 
> > The runtime grows after each step:
> > 0.216451
> > 0.225186
> > 0.223725
> > 0.222274
> > 0.230964
> > 0.240531
> > 0.24137
> > 0.252393
> > 0.249948
> > 0.257153
> > ...
> > 
> > Is there a mistake in this piece of code?
> 
> There is no mistake that I can see but I suspect that the
> circular references that you are building are causing the
> ThreadPoolExecutor to take a long time to be collected. Try
> adding:
> 
> ??? c = counter(fut)
> ??? c.run()
> +??? fut.shutdown()
> 
> Even if that fixes your problem, I still don't fully
> understand this because I would expect the runtime to fall
> after a while as ThreadPoolExecutors are collected.

The shutdown call is indeed a good fix :-) Here is the time response of the calls to counter() when shutdown is not called:
http://www.freehackers.org/~tnagy/runtime_futures.png

After trying to stop the program by using CTRL+C, the following error may appear, after which the process cannot be interrupted:

"""
19:18:12 /tmp/build> python3.2 futures_test.py
0.389657
0.417173
0.416513
0.421424
0.449666
0.482273
^CTraceback (most recent call last):
? File "futures_test.py", line 36, in <module>
? ? c.run()
? File "futures_test.py", line 22, in run
? ? self.fut.submit(look_busy, self.count, self)
? File "/usr/local/lib/python3.2/concurrent/futures/thread.py", line 114, in submit
? ? self._work_queue.put(w)
? File "/usr/local/lib/python3.2/queue.py", line 135, in put
? ? self.not_full.acquire()
KeyboardInterrupt
"""

It is not expected, is it?

Thomas



      

From brian at sweetapp.com  Fri Dec 10 20:24:42 2010
From: brian at sweetapp.com (Brian Quinlan)
Date: Fri, 10 Dec 2010 11:24:42 -0800
Subject: [Python-Dev] futures API
In-Reply-To: <441626.55995.qm@web27505.mail.ukl.yahoo.com>
References: <441626.55995.qm@web27505.mail.ukl.yahoo.com>
Message-ID: <C13B1791-1281-4DFB-B144-350AE7A12FF8@sweetapp.com>


On Dec 10, 2010, at 10:51 AM, Thomas Nagy wrote:

> --- El vie, 10/12/10, Brian Quinlan escribi?:
>> On Dec 10, 2010, at 5:36 AM, Thomas Nagy wrote:
>>> I have a process running for a long time, and which
>> may use futures of different max_workers count. I think it
>> is not too far-fetched to create a new futures object each
>> time. Yet, the execution becomes slower after each call, for
>> example with http://freehackers.org/~tnagy/futures_test.py:
>>>
>>> """
>>> import concurrent.futures
>>> from queue import Queue
>>> import datetime
>>>
>>> class counter(object):
>>>      def __init__(self, fut):
>>>          self.fut = fut
>>>
>>>      def run(self):
>>>          def
>> look_busy(num, obj):
>>>
>>    tot = 0
>>>
>>    for x in range(num):
>>>
>>    tot += x
>>>
>>    obj.out_q.put(tot)
>>>
>>>          start =
>> datetime.datetime.utcnow()
>>>          self.count = 0
>>>          self.out_q =
>> Queue(0)
>>>          for x in
>> range(1000):
>>>
>>    self.count += 1
>>>
>>    self.fut.submit(look_busy, self.count,
>> self)
>>>
>>>          while
>> self.count:
>>>
>>    self.count -= 1
>>>
>>    self.out_q.get()
>>>
>>>          delta =
>> datetime.datetime.utcnow() - start
>>>
>>    print(delta.total_seconds())
>>>
>>> fut =
>> concurrent.futures.ThreadPoolExecutor(max_workers=20)
>>> for x in range(100):
>>>      # comment the following line
>>>      fut =
>> concurrent.futures.ThreadPoolExecutor(max_workers=20)
>>>      c = counter(fut)
>>>      c.run()
>>> """
>>>
>>> The runtime grows after each step:
>>> 0.216451
>>> 0.225186
>>> 0.223725
>>> 0.222274
>>> 0.230964
>>> 0.240531
>>> 0.24137
>>> 0.252393
>>> 0.249948
>>> 0.257153
>>> ...
>>>
>>> Is there a mistake in this piece of code?
>>
>> There is no mistake that I can see but I suspect that the
>> circular references that you are building are causing the
>> ThreadPoolExecutor to take a long time to be collected. Try
>> adding:
>>
>>     c = counter(fut)
>>     c.run()
>> +    fut.shutdown()
>>
>> Even if that fixes your problem, I still don't fully
>> understand this because I would expect the runtime to fall
>> after a while as ThreadPoolExecutors are collected.
>
> The shutdown call is indeed a good fix :-) Here is the time response  
> of the calls to counter() when shutdown is not called:
> http://www.freehackers.org/~tnagy/runtime_futures.png

FWIW, I think that you are confusion the term "future" with  
"executor". A future represents a single work item. An executor  
creates futures and schedules their underlying work.

Hmmm....that is very suspicious - it looks like the  
ThreadPoolExecutors are not being collected. If you are feeling bored  
you could figure out why not :-)

> After trying to stop the program by using CTRL+C, the following  
> error may appear, after which the process cannot be interrupted:
>
> """
> 19:18:12 /tmp/build> python3.2 futures_test.py
> 0.389657
> 0.417173
> 0.416513
> 0.421424
> 0.449666
> 0.482273
> ^CTraceback (most recent call last):
>   File "futures_test.py", line 36, in <module>
>     c.run()
>   File "futures_test.py", line 22, in run
>     self.fut.submit(look_busy, self.count, self)
>   File "/usr/local/lib/python3.2/concurrent/futures/thread.py", line  
> 114, in submit
>     self._work_queue.put(w)
>   File "/usr/local/lib/python3.2/queue.py", line 135, in put
>     self.not_full.acquire()
> KeyboardInterrupt
> """
>
> It is not expected, is it?

It isn't surprising. Python lock acquisitions are not interruptible  
and anytime you interrupt a program that manipulates locks you may  
kill the code that was going to cause the lock to be released.

Cheers,
Brian

> Thomas
>
>
>
>


From vinay_sajip at yahoo.co.uk  Fri Dec 10 20:39:08 2010
From: vinay_sajip at yahoo.co.uk (Vinay Sajip)
Date: Fri, 10 Dec 2010 19:39:08 +0000 (UTC)
Subject: [Python-Dev] Using logging in the stdlib and its unit tests
References: <loom.20101207T212306-892@post.gmane.org>
	<20101207215053.025ecf52@pitrou.net>
	<loom.20101208T002100-91@post.gmane.org>
	<20101208010913.363c6105@pitrou.net>
	<AANLkTim4ZPoch18+-qkunuYm-zcyh-vbhMOv9YVVEqo9@mail.gmail.com>
	<loom.20101208T022102-46@post.gmane.org>
	<AANLkTikFNUPA5Cvbz8cOAfznSOEe906O8zOfOC=2Cj+H@mail.gmail.com>
	<loom.20101210T085332-877@post.gmane.org>
	<4D01E55B.2050507@g.nevcal.com> <3231.1292004978@parc.com>
Message-ID: <loom.20101210T202041-859@post.gmane.org>

Bill Janssen <janssen <at> parc.com> writes:

> Liked it just fine -- easier to write.  But nothing came out!  Because
> it's got this odd idea (from Java?) about "severities" of messages, and
> by default passes nothing less "threatening" than warnings, because
> logger.root is initialized by default to WARNING.

The idea about severity of messages is based on the premise that there might be
different audiences for logged messages, and that these can change over time. A
fairly common developer perspective on logging is that a logger equates to a
destination (e.g. console or file) and the only audience for logging is the
developer themselves. So, "handlers" and "loggers" seem complicated, perhaps
over-complicated.

In more complex systems, there are different stakeholders: end users, sys
admins, support teams, development teams. You have to consider "what happened"
(logging event), "where it happened" (logger name), "how important it is"
(severity or level) and "who wants to know" (handler), because not everyone
wants to know about everything all the time. That's pretty independent of which
language you're developing in; it's more a function of the complexity of the
system you're developing. Since Python is used in some fairly large system
scenarios - NASA, Google, etc. - it doesn't make sense not to have the whole
enchilada of logging functionality, at the risk of making it seem complicated to
users with less complex needs. And importance maps reasonably naturally onto an
idea of integer levels.

The default setting of WARNING is not from Java either - it's the Unix idea that
verbosity should be kept to a minimum unless specifically requested. So,
warnings and errors should be shown, but info and debug not. Imagine if the
default level was DEBUG - lots of detailed stuff would be displayed, which would
be off-putting for most users.

Regards,

Vinay Sajip



From tnagyemail-mail at yahoo.fr  Fri Dec 10 20:39:18 2010
From: tnagyemail-mail at yahoo.fr (Thomas Nagy)
Date: Fri, 10 Dec 2010 19:39:18 +0000 (GMT)
Subject: [Python-Dev] futures API
In-Reply-To: <441626.55995.qm@web27505.mail.ukl.yahoo.com>
Message-ID: <953842.41772.qm@web27502.mail.ukl.yahoo.com>

--- El vie, 10/12/10, Thomas Nagy escribi?:
> --- El vie, 10/12/10, Brian Quinlan
> escribi?:
> > On Dec 10, 2010, at 5:36 AM, Thomas Nagy wrote: 
> > > I have a process running for a long time, and
> which
> > may use futures of different max_workers count. I
> think it
> > is not too far-fetched to create a new futures object
> each
> > time. Yet, the execution becomes slower after each
> call, for
> > example with http://freehackers.org/~tnagy/futures_test.py:
> > > 
> > > """
> > > import concurrent.futures
> > > from queue import Queue
> > > import datetime
> > > 
> > > class counter(object):
> > >? ???def __init__(self, fut):
> > >? ? ? ???self.fut = fut
> > > 
> > >? ???def run(self):
> > >? ? ? ???def
> > look_busy(num, obj):
> > >? ? ? ? ?
> > ???tot = 0
> > >? ? ? ? ?
> > ???for x in range(num):
> > >? ? ? ? ? ? ?
> > ???tot += x
> > >? ? ? ? ?
> > ???obj.out_q.put(tot)
> > > 
> > >? ? ? ???start =
> > datetime.datetime.utcnow()
> > >? ? ? ???self.count = 0
> > >? ? ? ???self.out_q =
> > Queue(0)
> > >? ? ? ???for x in
> > range(1000):
> > >? ? ? ? ?
> > ???self.count += 1
> > >? ? ? ? ?
> > ???self.fut.submit(look_busy, self.count,
> > self)
> > > 
> > >? ? ? ???while
> > self.count:
> > >? ? ? ? ?
> > ???self.count -= 1
> > >? ? ? ? ?
> > ???self.out_q.get()
> > > 
> > >? ? ? ???delta =
> > datetime.datetime.utcnow() - start
> > >? ? ?
> > ???print(delta.total_seconds())
> > > 
> > > fut =
> > concurrent.futures.ThreadPoolExecutor(max_workers=20)
> > > for x in range(100):
> > >? ???# comment the following line
> > >? ???fut =
> > concurrent.futures.ThreadPoolExecutor(max_workers=20)
> > >? ???c = counter(fut)
> > >? ???c.run()
> > > """
> > > 
> > > The runtime grows after each step:
> > > 0.216451
> > > 0.225186
> > > 0.223725
> > > 0.222274
> > > 0.230964
> > > 0.240531
> > > 0.24137
> > > 0.252393
> > > 0.249948
> > > 0.257153
> > > ...
> > > 
> > > Is there a mistake in this piece of code?
> > 
> > There is no mistake that I can see but I suspect that
> the
> > circular references that you are building are causing
> the
> > ThreadPoolExecutor to take a long time to be
> collected. Try
> > adding:
> > 
> > ??? c = counter(fut)
> > ??? c.run()
> > +??? fut.shutdown()
> > 
> > Even if that fixes your problem, I still don't fully
> > understand this because I would expect the runtime to
> fall
> > after a while as ThreadPoolExecutors are collected.
> 
> The shutdown call is indeed a good fix :-) Here is the time
> response of the calls to counter() when shutdown is not
> called:
> http://www.freehackers.org/~tnagy/runtime_futures.png
> 
> After trying to stop the program by using CTRL+C, the
> following error may appear, after which the process cannot
> be interrupted:
> 
> """
> 19:18:12 /tmp/build> python3.2 futures_test.py
> 0.389657
> 0.417173
> 0.416513
> 0.421424
> 0.449666
> 0.482273
> ^CTraceback (most recent call last):
> ? File "futures_test.py", line 36, in <module>
> ? ? c.run()
> ? File "futures_test.py", line 22, in run
> ? ? self.fut.submit(look_busy, self.count, self)
> ? File
> "/usr/local/lib/python3.2/concurrent/futures/thread.py",
> line 114, in submit
> ? ? self._work_queue.put(w)
> ? File "/usr/local/lib/python3.2/queue.py", line 135, in
> put
> ? ? self.not_full.acquire()
> KeyboardInterrupt
> """
> 
> It is not expected, is it?

The problem also occurs when using a callback:
http://www.freehackers.org/~tnagy/futures_test2.py

If it is necessary to catch KeyboardInterrupt exceptions to cancel the futures execution, then how about adding this detail to the docs?

Thomas



      

From brian at sweetapp.com  Fri Dec 10 21:07:48 2010
From: brian at sweetapp.com (Brian Quinlan)
Date: Fri, 10 Dec 2010 12:07:48 -0800
Subject: [Python-Dev] futures API
In-Reply-To: <953842.41772.qm@web27502.mail.ukl.yahoo.com>
References: <953842.41772.qm@web27502.mail.ukl.yahoo.com>
Message-ID: <8674D52A-07EF-4D4A-898A-983DE0AC6BC3@sweetapp.com>


On Dec 10, 2010, at 11:39 AM, Thomas Nagy wrote:

> --- El vie, 10/12/10, Thomas Nagy escribi?:
>> --- El vie, 10/12/10, Brian Quinlan
>> escribi?:
>>> On Dec 10, 2010, at 5:36 AM, Thomas Nagy wrote:
>>>> I have a process running for a long time, and
>> which
>>> may use futures of different max_workers count. I
>> think it
>>> is not too far-fetched to create a new futures object
>> each
>>> time. Yet, the execution becomes slower after each
>> call, for
>>> example with http://freehackers.org/~tnagy/futures_test.py:
>>>>
>>>> """
>>>> import concurrent.futures
>>>> from queue import Queue
>>>> import datetime
>>>>
>>>> class counter(object):
>>>>      def __init__(self, fut):
>>>>          self.fut = fut
>>>>
>>>>      def run(self):
>>>>          def
>>> look_busy(num, obj):
>>>>
>>>    tot = 0
>>>>
>>>    for x in range(num):
>>>>
>>>    tot += x
>>>>
>>>    obj.out_q.put(tot)
>>>>
>>>>          start =
>>> datetime.datetime.utcnow()
>>>>          self.count = 0
>>>>          self.out_q =
>>> Queue(0)
>>>>          for x in
>>> range(1000):
>>>>
>>>    self.count += 1
>>>>
>>>    self.fut.submit(look_busy, self.count,
>>> self)
>>>>
>>>>          while
>>> self.count:
>>>>
>>>    self.count -= 1
>>>>
>>>    self.out_q.get()
>>>>
>>>>          delta =
>>> datetime.datetime.utcnow() - start
>>>>
>>>    print(delta.total_seconds())
>>>>
>>>> fut =
>>> concurrent.futures.ThreadPoolExecutor(max_workers=20)
>>>> for x in range(100):
>>>>      # comment the following line
>>>>      fut =
>>> concurrent.futures.ThreadPoolExecutor(max_workers=20)
>>>>      c = counter(fut)
>>>>      c.run()
>>>> """
>>>>
>>>> The runtime grows after each step:
>>>> 0.216451
>>>> 0.225186
>>>> 0.223725
>>>> 0.222274
>>>> 0.230964
>>>> 0.240531
>>>> 0.24137
>>>> 0.252393
>>>> 0.249948
>>>> 0.257153
>>>> ...
>>>>
>>>> Is there a mistake in this piece of code?
>>>
>>> There is no mistake that I can see but I suspect that
>> the
>>> circular references that you are building are causing
>> the
>>> ThreadPoolExecutor to take a long time to be
>> collected. Try
>>> adding:
>>>
>>>     c = counter(fut)
>>>     c.run()
>>> +    fut.shutdown()
>>>
>>> Even if that fixes your problem, I still don't fully
>>> understand this because I would expect the runtime to
>> fall
>>> after a while as ThreadPoolExecutors are collected.
>>
>> The shutdown call is indeed a good fix :-) Here is the time
>> response of the calls to counter() when shutdown is not
>> called:
>> http://www.freehackers.org/~tnagy/runtime_futures.png
>>
>> After trying to stop the program by using CTRL+C, the
>> following error may appear, after which the process cannot
>> be interrupted:
>>
>> """
>> 19:18:12 /tmp/build> python3.2 futures_test.py
>> 0.389657
>> 0.417173
>> 0.416513
>> 0.421424
>> 0.449666
>> 0.482273
>> ^CTraceback (most recent call last):
>>   File "futures_test.py", line 36, in <module>
>>     c.run()
>>   File "futures_test.py", line 22, in run
>>     self.fut.submit(look_busy, self.count, self)
>>   File
>> "/usr/local/lib/python3.2/concurrent/futures/thread.py",
>> line 114, in submit
>>     self._work_queue.put(w)
>>   File "/usr/local/lib/python3.2/queue.py", line 135, in
>> put
>>     self.not_full.acquire()
>> KeyboardInterrupt
>> """
>>
>> It is not expected, is it?
>
> The problem also occurs when using a callback:
> http://www.freehackers.org/~tnagy/futures_test2.py
>
> If it is necessary to catch KeyboardInterrupt exceptions to cancel  
> the futures execution, then how about adding this detail to the docs?

AFAIK, catching KeyboardInterrupt exceptions is not sufficient.

Cheers,
Brian

> Thomas
>
>
>
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/brian%40sweetapp.com


From raymond.hettinger at gmail.com  Fri Dec 10 21:27:26 2010
From: raymond.hettinger at gmail.com (Raymond Hettinger)
Date: Fri, 10 Dec 2010 12:27:26 -0800
Subject: [Python-Dev] API for the new sysconfig module
In-Reply-To: <4D023721.1070904@netwok.org>
References: <B710BA0A-7097-4070-9DA4-E33149620833@gmail.com>
	<AANLkTimw5zcOUmDaw+6mWUw8ODbKPRzpTV=1nxkvY+P9@mail.gmail.com>
	<4D023721.1070904@netwok.org>
Message-ID: <DE77D448-1001-4250-8AC8-36EFC6CBBF15@gmail.com>

On Dec 10, 2010, at 6:20 AM, ?ric Araujo wrote:

> Final note: with 3.2 being in beta, I don?t know how much can be changed
> now.

Part of the purpose of a beta, and in our case, two betas is to give
people a chance to exercise new APIs and fix them before they
become set in stone two months later.

IMO, sysconfig did not warrant a whole module.  The pile of
awkward accessor functions is harder to learn / remember
than the simple data structure shown in the last email.

Rather than using two levels of dictionary, it's also possible
to use a named tuple if you think that is more clean looking:

   >>> c = sys.sysconfig()             
   >>> c.config_vars.get('SO)
   '.pyd'
   >>> c.platform
   'win32'
    >>> c.paths.get('stdlib')
    'C:\\Python32\\Lib'

    >>> # the named tuple fields:
    >>> c.fields
    ['config_vars', 'platform', 'version', 'scheme_names', 'paths']

This would be a straight-forward API that uses existing, well-known
tools (attribute access and dict.get) instead of an unnecessary 
nest of ad-hoc accessor functions living in a new module.

Tastes may vary on writing getter functions for every category
of interest but we did not need a whole new module for this.
The referenced email didn't indicate much if any thought about
the API design, so I think should get that thought now.

If those functions were already public in distutils, it is trivial
to write an adapter for them there.  


Raymond

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20101210/350bf6c0/attachment.html>

From guido at python.org  Fri Dec 10 21:31:19 2010
From: guido at python.org (Guido van Rossum)
Date: Fri, 10 Dec 2010 12:31:19 -0800
Subject: [Python-Dev] Using logging in the stdlib and its unit tests
In-Reply-To: <loom.20101210T202041-859@post.gmane.org>
References: <loom.20101207T212306-892@post.gmane.org>
	<20101207215053.025ecf52@pitrou.net>
	<loom.20101208T002100-91@post.gmane.org>
	<20101208010913.363c6105@pitrou.net>
	<AANLkTim4ZPoch18+-qkunuYm-zcyh-vbhMOv9YVVEqo9@mail.gmail.com>
	<loom.20101208T022102-46@post.gmane.org>
	<AANLkTikFNUPA5Cvbz8cOAfznSOEe906O8zOfOC=2Cj+H@mail.gmail.com>
	<loom.20101210T085332-877@post.gmane.org>
	<4D01E55B.2050507@g.nevcal.com>
	<3231.1292004978@parc.com> <loom.20101210T202041-859@post.gmane.org>
Message-ID: <AANLkTi=PnWE5MbpyWKZHJU4Abk-OeVfKEJYpGX1E6g3=@mail.gmail.com>

On Fri, Dec 10, 2010 at 11:39 AM, Vinay Sajip <vinay_sajip at yahoo.co.uk> wrote:
> Bill Janssen <janssen <at> parc.com> writes:
>
>> Liked it just fine -- easier to write. ?But nothing came out! ?Because
>> it's got this odd idea (from Java?) about "severities" of messages, and
>> by default passes nothing less "threatening" than warnings, because
>> logger.root is initialized by default to WARNING.
>
> The idea about severity of messages is based on the premise that there might be
> different audiences for logged messages, and that these can change over time. A
> fairly common developer perspective on logging is that a logger equates to a
> destination (e.g. console or file) and the only audience for logging is the
> developer themselves. So, "handlers" and "loggers" seem complicated, perhaps
> over-complicated.
>
> In more complex systems, there are different stakeholders: end users, sys
> admins, support teams, development teams. You have to consider "what happened"
> (logging event), "where it happened" (logger name), "how important it is"
> (severity or level) and "who wants to know" (handler), because not everyone
> wants to know about everything all the time. That's pretty independent of which
> language you're developing in; it's more a function of the complexity of the
> system you're developing. Since Python is used in some fairly large system
> scenarios - NASA, Google, etc. - it doesn't make sense not to have the whole
> enchilada of logging functionality, at the risk of making it seem complicated to
> users with less complex needs. And importance maps reasonably naturally onto an
> idea of integer levels.
>
> The default setting of WARNING is not from Java either - it's the Unix idea that
> verbosity should be kept to a minimum unless specifically requested. So,
> warnings and errors should be shown, but info and debug not. Imagine if the
> default level was DEBUG - lots of detailed stuff would be displayed, which would
> be off-putting for most users.

And yet, I have helped many people who were baffled by exactly what
Bill observed: logging.info() didn't do anything. Maybe the default
should be INFO?

-- 
--Guido van Rossum (python.org/~guido)

From solipsis at pitrou.net  Fri Dec 10 21:49:19 2010
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Fri, 10 Dec 2010 21:49:19 +0100
Subject: [Python-Dev] Using logging in the stdlib and its unit tests
References: <loom.20101207T212306-892@post.gmane.org>
	<20101207215053.025ecf52@pitrou.net>
	<loom.20101208T002100-91@post.gmane.org>
	<20101208010913.363c6105@pitrou.net>
	<AANLkTim4ZPoch18+-qkunuYm-zcyh-vbhMOv9YVVEqo9@mail.gmail.com>
	<loom.20101208T022102-46@post.gmane.org>
	<AANLkTikFNUPA5Cvbz8cOAfznSOEe906O8zOfOC=2Cj+H@mail.gmail.com>
	<loom.20101210T085332-877@post.gmane.org>
	<4D01E55B.2050507@g.nevcal.com> <3231.1292004978@parc.com>
	<loom.20101210T202041-859@post.gmane.org>
	<AANLkTi=PnWE5MbpyWKZHJU4Abk-OeVfKEJYpGX1E6g3=@mail.gmail.com>
Message-ID: <20101210214919.4ac63d27@pitrou.net>

On Fri, 10 Dec 2010 12:31:19 -0800
Guido van Rossum <guido at python.org> wrote:
> > The default setting of WARNING is not from Java either - it's the Unix idea that
> > verbosity should be kept to a minimum unless specifically requested. So,
> > warnings and errors should be shown, but info and debug not. Imagine if the
> > default level was DEBUG - lots of detailed stuff would be displayed, which would
> > be off-putting for most users.
> 
> And yet, I have helped many people who were baffled by exactly what
> Bill observed: logging.info() didn't do anything. Maybe the default
> should be INFO?

Funny, because displaying only errors and silencing other messages is
exactly what I expected logging to do by default.

Regards

Antoine.



From solipsis at pitrou.net  Fri Dec 10 21:56:00 2010
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Fri, 10 Dec 2010 21:56:00 +0100
Subject: [Python-Dev] API for the new sysconfig module
References: <B710BA0A-7097-4070-9DA4-E33149620833@gmail.com>
	<AANLkTimw5zcOUmDaw+6mWUw8ODbKPRzpTV=1nxkvY+P9@mail.gmail.com>
	<4D023721.1070904@netwok.org>
	<DE77D448-1001-4250-8AC8-36EFC6CBBF15@gmail.com>
Message-ID: <20101210215600.56f4eaf2@pitrou.net>

On Fri, 10 Dec 2010 12:27:26 -0800
Raymond Hettinger <raymond.hettinger at gmail.com> wrote:
> 
> IMO, sysconfig did not warrant a whole module.

Where would you put it?

> Rather than using two levels of dictionary, it's also possible
> to use a named tuple if you think that is more clean looking:
> 
>    >>> c = sys.sysconfig()             
>    >>> c.config_vars.get('SO)
>    '.pyd'

Some of these things are computed at runtime by parsing makefiles and
other stuff. You don't want to do it as soon as the module is imported.
There may also be some bootstrap issues when importing some of the
necessary modules from the top level.

The current API is fine IMO. Perhaps it could be refined a bit, but
there's nothing horrendous. It's not for everyone to use; creating
dict-like classes over these APIs would be a terrible waste of time,
and add a useless maintenance burden.

> If those functions were already public in distutils, it is trivial
> to write an adapter for them there.

The whole point was to separate them from distutils.

Regards

Antoine.



From v+python at g.nevcal.com  Fri Dec 10 21:57:29 2010
From: v+python at g.nevcal.com (Glenn Linderman)
Date: Fri, 10 Dec 2010 12:57:29 -0800
Subject: [Python-Dev] Using logging in the stdlib and its unit tests
In-Reply-To: <20101210214919.4ac63d27@pitrou.net>
References: <loom.20101207T212306-892@post.gmane.org>	<20101207215053.025ecf52@pitrou.net>	<loom.20101208T002100-91@post.gmane.org>	<20101208010913.363c6105@pitrou.net>	<AANLkTim4ZPoch18+-qkunuYm-zcyh-vbhMOv9YVVEqo9@mail.gmail.com>	<loom.20101208T022102-46@post.gmane.org>	<AANLkTikFNUPA5Cvbz8cOAfznSOEe906O8zOfOC=2Cj+H@mail.gmail.com>	<loom.20101210T085332-877@post.gmane.org>	<4D01E55B.2050507@g.nevcal.com>
	<3231.1292004978@parc.com>	<loom.20101210T202041-859@post.gmane.org>	<AANLkTi=PnWE5MbpyWKZHJU4Abk-OeVfKEJYpGX1E6g3=@mail.gmail.com>
	<20101210214919.4ac63d27@pitrou.net>
Message-ID: <4D029439.7040708@g.nevcal.com>

On 12/10/2010 12:49 PM, Antoine Pitrou wrote:
>> And yet, I have helped many people who were baffled by exactly what
>> >  Bill observed: logging.info() didn't do anything. Maybe the default
>> >  should be INFO?
> Funny, because displaying only errors and silencing other messages is
> exactly what I expected logging to do by default.

So we are slowly learning the things that should be on the first couple 
pages of the logging docs...

1) simple example for one file programs, include an example of 
specifying output severity threshold.  I'm with Antoine here on my 
expectations.

2) example for multi-module, showing how a single logging destination 
causes logging to happen in all modules, at the same level (if that is 
the case, which I hope it is).

3) Maybe a small discussion of log formatting should be next?  So the 
user realizes he shouldn't do the message formatting himself?

4) Then rotating logs for long-running programs.

And then the rest of the more feature-filled stuff.  I agree logging 
needs to be full-featured, but many of the features are only necessary 
in complex environments, and simple stuff should be simple, if it can 
be, and those cases explained first.  Hopefully a useful subset for 
simple cases can be described in 1-3 pages, and then the complex 
features after that.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20101210/f84839b7/attachment.html>

From janssen at parc.com  Fri Dec 10 22:26:02 2010
From: janssen at parc.com (Bill Janssen)
Date: Fri, 10 Dec 2010 13:26:02 PST
Subject: [Python-Dev] Using logging in the stdlib and its unit tests
In-Reply-To: <AANLkTi=PnWE5MbpyWKZHJU4Abk-OeVfKEJYpGX1E6g3=@mail.gmail.com>
References: <loom.20101207T212306-892@post.gmane.org>
	<20101207215053.025ecf52@pitrou.net>
	<loom.20101208T002100-91@post.gmane.org>
	<20101208010913.363c6105@pitrou.net>
	<AANLkTim4ZPoch18+-qkunuYm-zcyh-vbhMOv9YVVEqo9@mail.gmail.com>
	<loom.20101208T022102-46@post.gmane.org>
	<AANLkTikFNUPA5Cvbz8cOAfznSOEe906O8zOfOC=2Cj+H@mail.gmail.com>
	<loom.20101210T085332-877@post.gmane.org>
	<4D01E55B.2050507@g.nevcal.com> <3231.1292004978@parc.com>
	<loom.20101210T202041-859@post.gmane.org>
	<AANLkTi=PnWE5MbpyWKZHJU4Abk-OeVfKEJYpGX1E6g3=@mail.gmail.com>
Message-ID: <9235.1292016362@parc.com>

Guido van Rossum <guido at python.org> wrote:

> And yet, I have helped many people who were baffled by exactly what
> Bill observed: logging.info() didn't do anything. Maybe the default
> should be INFO?

Yeah, I was curious enough to read the code and find out why.  But many
won't.

By the way, I tried reading the 2.7 docs first, and didn't figure it out
from them.  So I looked at the code.

Bill

From janssen at parc.com  Fri Dec 10 22:30:11 2010
From: janssen at parc.com (Bill Janssen)
Date: Fri, 10 Dec 2010 13:30:11 PST
Subject: [Python-Dev] Using logging in the stdlib and its unit tests
In-Reply-To: <4D029439.7040708@g.nevcal.com>
References: <loom.20101207T212306-892@post.gmane.org>
	<20101207215053.025ecf52@pitrou.net>
	<loom.20101208T002100-91@post.gmane.org>
	<20101208010913.363c6105@pitrou.net>
	<AANLkTim4ZPoch18+-qkunuYm-zcyh-vbhMOv9YVVEqo9@mail.gmail.com>
	<loom.20101208T022102-46@post.gmane.org>
	<AANLkTikFNUPA5Cvbz8cOAfznSOEe906O8zOfOC=2Cj+H@mail.gmail.com>
	<loom.20101210T085332-877@post.gmane.org>
	<4D01E55B.2050507@g.nevcal.com> <3231.1292004978@parc.com>
	<loom.20101210T202041-859@post.gmane.org>
	<AANLkTi=PnWE5MbpyWKZHJU4Abk-OeVfKEJYpGX1E6g3=@mail.gmail.com>
	<20101210214919.4ac63d27@pitrou.net>
	<4D029439.7040708@g.nevcal.com>
Message-ID: <9308.1292016611@parc.com>

Glenn Linderman <v+python at g.nevcal.com> wrote:

> 1) simple example for one file programs, include an example of
> specifying output severity threshold.  I'm with Antoine here on my
> expectations.

Yes, once I put

    logging.basicConfig(stream=sys.stdout, level=logging.DEBUG)

in my main(), I got what I thought I'd get in the first place.  All
logging messages.  But I can see Antoine's point: if we use this in the
stdlib, you don't want that default.  I think that logging events
(context) have to come into this; you can't do it with just severity
alone.  I'd expect to have different settings, by default, for __main__
and for http.client, for instance.

Bill

From raymond.hettinger at gmail.com  Fri Dec 10 22:35:03 2010
From: raymond.hettinger at gmail.com (Raymond Hettinger)
Date: Fri, 10 Dec 2010 13:35:03 -0800
Subject: [Python-Dev] API for the new sysconfig module
In-Reply-To: <20101210215600.56f4eaf2@pitrou.net>
References: <B710BA0A-7097-4070-9DA4-E33149620833@gmail.com>
	<AANLkTimw5zcOUmDaw+6mWUw8ODbKPRzpTV=1nxkvY+P9@mail.gmail.com>
	<4D023721.1070904@netwok.org>
	<DE77D448-1001-4250-8AC8-36EFC6CBBF15@gmail.com>
	<20101210215600.56f4eaf2@pitrou.net>
Message-ID: <948611D4-66C3-4924-8737-DAC2F8A5DE35@gmail.com>


On Dec 10, 2010, at 12:56 PM, Antoine Pitrou wrote:

> On Fri, 10 Dec 2010 12:27:26 -0800
> Raymond Hettinger <raymond.hettinger at gmail.com> wrote:
>> 
>> IMO, sysconfig did not warrant a whole module.
> 
> Where would you put it?

A single function in the sys module.

> 
>> Rather than using two levels of dictionary, it's also possible
>> to use a named tuple if you think that is more clean looking:
>> 
>>>>> c = sys.sysconfig()             
>>>>> c.config_vars.get('SO)
>>   '.pyd'
> 
> Some of these things are computed at runtime by parsing makefiles and
> other stuff. You don't want to do it as soon as the module is imported.

The proposal is for a function that does the computation when invoked,
not when imported.  The function returns a named tuple so that we can
use Python's builtin accessors like attributes and dict.get().


Raymond

From rdmurray at bitdance.com  Fri Dec 10 22:59:02 2010
From: rdmurray at bitdance.com (R. David Murray)
Date: Fri, 10 Dec 2010 16:59:02 -0500
Subject: [Python-Dev] API for the new sysconfig module
In-Reply-To: <B710BA0A-7097-4070-9DA4-E33149620833@gmail.com>
References: <B710BA0A-7097-4070-9DA4-E33149620833@gmail.com>
Message-ID: <20101210215902.CDB5B1AB60E@kimball.webabinitio.net>

On Thu, 09 Dec 2010 16:18:14 -0800, Raymond Hettinger <raymond.hettinger at gmail.com> wrote:
> Does anyone know why this needed a separate module and so many accessor functions?

Originally sysconfig was moved *out* of distutils, and distutils was
changed to use it.  But that proved to be as fragile as many other
distutils changes.  When distutils was reverted, sysconfig was kept
because of the long term desire to move it out of distutils.  At that
point I think it would have been very easy to propose API changes, but I
suppose it didn't occur to anyone to think about that possibility.

> ISTM it mostly could have been reduced to single call returning a nested dictionary.

If what was returned was, as you suggested on IRC, a set of named tuples,
it seems to me that would be a very convenient interface.  And since
all values (as I understand it) are intended to be static (even if some
aren't at the moment), it seems to make sense from an implementation
standpoint as well.

Like ??ric, I'm not sure what the implications of the existing module
having been released in 2.7 and 3.2 beta are in terms of making such an
API change.

--
R. David Murray                                      www.bitdance.com

From vinay_sajip at yahoo.co.uk  Fri Dec 10 23:16:39 2010
From: vinay_sajip at yahoo.co.uk (Vinay Sajip)
Date: Fri, 10 Dec 2010 22:16:39 +0000 (UTC)
Subject: [Python-Dev] Using logging in the stdlib and its unit tests
References: <loom.20101207T212306-892@post.gmane.org>
	<20101207215053.025ecf52@pitrou.net>
	<loom.20101208T002100-91@post.gmane.org>
	<20101208010913.363c6105@pitrou.net>
	<AANLkTim4ZPoch18+-qkunuYm-zcyh-vbhMOv9YVVEqo9@mail.gmail.com>
	<loom.20101208T022102-46@post.gmane.org>
	<AANLkTikFNUPA5Cvbz8cOAfznSOEe906O8zOfOC=2Cj+H@mail.gmail.com>
	<loom.20101210T085332-877@post.gmane.org>
	<4D01E55B.2050507@g.nevcal.com> <3231.1292004978@parc.com>
	<loom.20101210T202041-859@post.gmane.org>
	<AANLkTi=PnWE5MbpyWKZHJU4Abk-OeVfKEJYpGX1E6g3=@mail.gmail.com>
	<20101210214919.4ac63d27@pitrou.net>
Message-ID: <loom.20101210T224910-511@post.gmane.org>

Antoine Pitrou <solipsis <at> pitrou.net> writes:

> Guido van Rossum <guido <at> python.org> wrote:
> > And yet, I have helped many people who were baffled by exactly what
> > Bill observed: logging.info() didn't do anything. Maybe the default
> > should be INFO?
> 
> Funny, because displaying only errors and silencing other messages is
> exactly what I expected logging to do by default.

The one thing I'm sure of is that it's hard to find more than a couple of
people with the same view about what the default should be. Anyone for
bikesheds? :-)

IMO as long as it's just a small amount of work to get the specific effect
that you want, it doesn't really matter too much what the default is - though
of course I'd like it to be "right", whatever that means ;-) And whatever it
is, it's going to upset *some* group of people because they have to add one
line (or so) to get it to do what they want, and they feel really strongly
that they shouldn't have to add that one line. In a way, I suppose I'm not
unhappy that this is all we're arguing about and not anything more
substantive. Some useful feedback about the documentation has emerged, too. I
would (naturally) like to see broader adoption of logging in the standard
library, and the view from Python core developers who have expressed opinions
on this thread seems to be that the present behaviour is an obstacle to that
adoption. I have my views but I'm not hidebound by them, hence the proposed
change and the posting about it on the logging blog and c.l.py, just to let
people know what's coming.

It'll be interesting to see, once this change is implemented, whether the
suggested obstacle really has been an obstacle, rather than (as I suspect)
the low level of logging adoption in the stdlib being largely being a matter
of plain inertia. But even if it helps make possible a better resolution of
#10626 (as an archetype of unraisable-exception scenarios), that would be a
good outcome.

Regards,

Vinay Sajip


From vinay_sajip at yahoo.co.uk  Fri Dec 10 23:21:11 2010
From: vinay_sajip at yahoo.co.uk (Vinay Sajip)
Date: Fri, 10 Dec 2010 22:21:11 +0000 (UTC)
Subject: [Python-Dev] Using logging in the stdlib and its unit tests
References: <loom.20101207T212306-892@post.gmane.org>
	<20101207215053.025ecf52@pitrou.net>
	<loom.20101208T002100-91@post.gmane.org>
	<20101208010913.363c6105@pitrou.net>
	<AANLkTim4ZPoch18+-qkunuYm-zcyh-vbhMOv9YVVEqo9@mail.gmail.com>
	<loom.20101208T022102-46@post.gmane.org>
	<AANLkTikFNUPA5Cvbz8cOAfznSOEe906O8zOfOC=2Cj+H@mail.gmail.com>
	<loom.20101210T085332-877@post.gmane.org>
	<4D01E55B.2050507@g.nevcal.com> <3231.1292004978@parc.com>
	<loom.20101210T202041-859@post.gmane.org>
	<AANLkTi=PnWE5MbpyWKZHJU4Abk-OeVfKEJYpGX1E6g3=@mail.gmail.com>
	<20101210214919.4ac63d27@pitrou.net>
	<4D029439.7040708@g.nevcal.com> <9308.1292016611@parc.com>
Message-ID: <loom.20101210T231728-370@post.gmane.org>

Bill Janssen <janssen <at> parc.com> writes:

> stdlib, you don't want that default.  I think that logging events
> (context) have to come into this; you can't do it with just severity
> alone.  I'd expect to have different settings, by default, for __main__
> and for http.client, for instance.
> 

Although e.g. http.client may set a default severity, the developer integrating
it into an application is free to override that severity for the purposes of
their application. Typically, there are different severities set for different
parts of an application at different times, depending on where the focus of
diagnostic scrutiny is at those times.

Severity is not the only filtering mechanism, merely the simplest and most
common one, that people can readily grasp.

Regards,

Vinay Sajip



From ncoghlan at gmail.com  Sat Dec 11 03:07:18 2010
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sat, 11 Dec 2010 12:07:18 +1000
Subject: [Python-Dev] futures API
In-Reply-To: <8674D52A-07EF-4D4A-898A-983DE0AC6BC3@sweetapp.com>
References: <953842.41772.qm@web27502.mail.ukl.yahoo.com>
	<8674D52A-07EF-4D4A-898A-983DE0AC6BC3@sweetapp.com>
Message-ID: <AANLkTin6K983RPUuaeG4+6wR__V5giA_JhVROM0vKz5M@mail.gmail.com>

On Sat, Dec 11, 2010 at 6:07 AM, Brian Quinlan <brian at sweetapp.com> wrote:
>> The problem also occurs when using a callback:
>> http://www.freehackers.org/~tnagy/futures_test2.py
>>
>> If it is necessary to catch KeyboardInterrupt exceptions to cancel the
>> futures execution, then how about adding this detail to the docs?
>
> AFAIK, catching KeyboardInterrupt exceptions is not sufficient.

finally blocks and with statements can get you a fairly long way,
though. futures does everything right on this front, as far as I can
see.

In this case, the problem is in the design of the test program. It
*must* get exactly as many items in the queue as were submitted to the
executor. If one is ever missing (e.g. due to a poorly timed
KeyboardInterrupt in the callback, as clearly happened in the
presented trace), it will hang in the final call to self.out_q.get().
There's a reason Queue.get offers both nonblocking and
block-with-timeout functionality. (Alternately, the management of
out_q and count could be made smarter, such that an exception in
adding a result to out_q triggered an appropriate adjustment in the
count of expected values)

I also agree with Brian that the variable naming for the sample code
and comments like "may use futures of different max_workers count. I
think it is not too far-fetched to create a new futures object each
time" suggests a fundamental confusion of the terms "future" and
"executor". Executors are not futures - the return value from an
executor's submit method is a future.

Cheers,
Nick.

-- 
Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia

From ncoghlan at gmail.com  Sat Dec 11 03:13:37 2010
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sat, 11 Dec 2010 12:13:37 +1000
Subject: [Python-Dev] (Non-) use of weakref.WeakSet in concurrent.futures
 (was Re: futures API)
Message-ID: <AANLkTik0RdwqPBiwOJEHB3WmB8EUkSBN4+dNhVwJaE6R@mail.gmail.com>

On Sat, Dec 11, 2010 at 6:07 AM, Brian Quinlan <brian at sweetapp.com> wrote:
> AFAIK, catching KeyboardInterrupt exceptions is not sufficient.

Getting away from the flaws in the test code in this thread, is there
any particular reason futures is rolling its own variant of
weakref.WeakSet for the _thread_references cleanup sets? It seems to
me that using WeakSet would mean you could get rid of
_remove_dead_thread_references altogether (the implicit callbacks
would handle that part), and then a set() call in _python_exit would
give you concrete references to work with for cleanup purposes.

Cheers,
Nick.

-- 
Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia

From stephen at xemacs.org  Sat Dec 11 03:35:50 2010
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Sat, 11 Dec 2010 11:35:50 +0900
Subject: [Python-Dev] Using logging in the stdlib and its unit tests
In-Reply-To: <4D029439.7040708@g.nevcal.com>
References: <loom.20101207T212306-892@post.gmane.org>
	<20101207215053.025ecf52@pitrou.net>
	<loom.20101208T002100-91@post.gmane.org>
	<20101208010913.363c6105@pitrou.net>
	<AANLkTim4ZPoch18+-qkunuYm-zcyh-vbhMOv9YVVEqo9@mail.gmail.com>
	<loom.20101208T022102-46@post.gmane.org>
	<AANLkTikFNUPA5Cvbz8cOAfznSOEe906O8zOfOC=2Cj+H@mail.gmail.com>
	<loom.20101210T085332-877@post.gmane.org>
	<4D01E55B.2050507@g.nevcal.com> <3231.1292004978@parc.com>
	<loom.20101210T202041-859@post.gmane.org>
	<AANLkTi=PnWE5MbpyWKZHJU4Abk-OeVfKEJYpGX1E6g3=@mail.gmail.com>
	<20101210214919.4ac63d27@pitrou.net>
	<4D029439.7040708@g.nevcal.com>
Message-ID: <87k4jhrq7t.fsf@uwakimon.sk.tsukuba.ac.jp>

Glenn Linderman writes:

 > 1) simple example for one file programs, include an example of 
 > specifying output severity threshold.  I'm with Antoine here on my 
 > expectations.
 > 
 > 2) example for multi-module, showing how a single logging destination 
 > causes logging to happen in all modules, at the same level (if that is 
 > the case, which I hope it is).
 > 
 > 3) Maybe a small discussion of log formatting should be next?  So the 
 > user realizes he shouldn't do the message formatting himself?

As advice, this shouldn't need to be more than a couple of lines, with
a two line example, right?  But it does deserve a prominent header.

 > 4) Then rotating logs for long-running programs.
 > 
 > And then the rest of the more feature-filled stuff.

+1 for this agenda.

From tjreedy at udel.edu  Sat Dec 11 03:42:19 2010
From: tjreedy at udel.edu (Terry Reedy)
Date: Fri, 10 Dec 2010 21:42:19 -0500
Subject: [Python-Dev] Using logging in the stdlib and its unit tests
In-Reply-To: <loom.20101210T224910-511@post.gmane.org>
References: <loom.20101207T212306-892@post.gmane.org>	<20101207215053.025ecf52@pitrou.net>	<loom.20101208T002100-91@post.gmane.org>	<20101208010913.363c6105@pitrou.net>	<AANLkTim4ZPoch18+-qkunuYm-zcyh-vbhMOv9YVVEqo9@mail.gmail.com>	<loom.20101208T022102-46@post.gmane.org>	<AANLkTikFNUPA5Cvbz8cOAfznSOEe906O8zOfOC=2Cj+H@mail.gmail.com>	<loom.20101210T085332-877@post.gmane.org>	<4D01E55B.2050507@g.nevcal.com>
	<3231.1292004978@parc.com>	<loom.20101210T202041-859@post.gmane.org>	<AANLkTi=PnWE5MbpyWKZHJU4Abk-OeVfKEJYpGX1E6g3=@mail.gmail.com>	<20101210214919.4ac63d27@pitrou.net>
	<loom.20101210T224910-511@post.gmane.org>
Message-ID: <iduoeb$q36$1@dough.gmane.org>

On 12/10/2010 5:16 PM, Vinay Sajip wrote:

> IMO as long as it's just a small amount of work to get the specific effect
> that you want, it doesn't really matter too much what the default is - though
> of course I'd like it to be "right", whatever that means ;-)

I think the default should accomodate both internal uses (stdlib and 
elsewhere) and beginner uses. I suspect the latter would mean outputting 
most everything, to the convenience functions work (as a beginner might 
expect).


-- 
Terry Jan Reedy


From tjreedy at udel.edu  Sat Dec 11 03:44:12 2010
From: tjreedy at udel.edu (Terry Reedy)
Date: Fri, 10 Dec 2010 21:44:12 -0500
Subject: [Python-Dev] API for the new sysconfig module
In-Reply-To: <20101210215902.CDB5B1AB60E@kimball.webabinitio.net>
References: <B710BA0A-7097-4070-9DA4-E33149620833@gmail.com>
	<20101210215902.CDB5B1AB60E@kimball.webabinitio.net>
Message-ID: <iduohr$q36$2@dough.gmane.org>

On 12/10/2010 4:59 PM, R. David Murray wrote:

> Like ?ric, I'm not sure what the implications of the existing module
> having been released in 2.7 and 3.2 beta are in terms of making such an
> API change.

I am with Raymond on this: the purpose of betas is so we can test *and* 
make changes. No one should base production software on a beta release.

-- 
Terry Jan Reedy



From ncoghlan at gmail.com  Sat Dec 11 03:51:17 2010
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sat, 11 Dec 2010 12:51:17 +1000
Subject: [Python-Dev] API for the new sysconfig module
In-Reply-To: <20101210215902.CDB5B1AB60E@kimball.webabinitio.net>
References: <B710BA0A-7097-4070-9DA4-E33149620833@gmail.com>
	<20101210215902.CDB5B1AB60E@kimball.webabinitio.net>
Message-ID: <AANLkTikXg0E5DQuY8HxxrqkoZj0NzTpNcTyetLuBi_xc@mail.gmail.com>

On Sat, Dec 11, 2010 at 7:59 AM, R. David Murray <rdmurray at bitdance.com> wrote:
> On Thu, 09 Dec 2010 16:18:14 -0800, Raymond Hettinger <raymond.hettinger at gmail.com> wrote:
>> ISTM it mostly could have been reduced to single call returning a nested dictionary.
>
> If what was returned was, as you suggested on IRC, a set of named tuples,
> it seems to me that would be a very convenient interface. ?And since
> all values (as I understand it) are intended to be static (even if some
> aren't at the moment), it seems to make sense from an implementation
> standpoint as well.
>
> Like ?ric, I'm not sure what the implications of the existing module
> having been released in 2.7 and 3.2 beta are in terms of making such an
> API change.

We've reverted from beta to pseudo-alpha after discovering
sufficiently egregious mistakes in the past. (we didn't *actually*
revert the build naming because release IDs that don't increase
monotonically can confuse matters). However, I don't think this is
sufficiently egregious to qualify.

My own reasons for advocating retention of the separate module:

1. Something equivalent to the current sysconfig module is going to be
needed to actually work out the contents of the data structure.
Rewriting all that in C would be silly, so the most that could happen
is to rename "sysconfig" to "_sysconfig".

2. Invoking Python code from sys module functions is possible but
untidy (generally involving implicit module imports). Since we need a
module anyway for implementation purposes, why not make it public?

3. The sysconfig module docs provide a nicer space to talk about the
purpose of the provided information rather than squirreling it away in
a corner of the sys module docs

4. "python -m sysconfig" provides a useful dump of the system status

5. The API design did not get much *new* thought, as the idea was to
allow users to do anything that the old distutils.sysconfig could do,
but without requiring them to depend on distutils to get that
functionality. The easiest way to ensure that is to just copy the
existing API rather than trying to get creative. Don't think of it as
adding a "whole new module", thinking of it as decoupling a relatively
small existing module (i.e. distutils.sysconfig) from a larger package
that isn't installed by default on all systems (i.e. distutils).

A concrete problem with Raymond's idea in particular is that there is
actually a full set of paths defined for *each scheme*. get_path() and
get_paths() let you ignore this most of the time, since there is an
appropriate default scheme defined for each platform, so they just use
that if you don't specify one explicitly. A named tuple could
obviously expose the paths for all of the schemes, with the paths for
the default scheme duplicated at the top level, but the data structure
starts looking very messy at that point. The named tuple idea also
conflates meta-data about the sysconfig data (the list of scheme names
and path names) with the actual data for the current platform
(platform, version, paths, config_vars).

So I think reusing the tried and tested API is easily justified in
this case. What may be desirable in 3.3 (particularly with the
distutils2 reintegration taking place), is to revisit the sysconfig
API to see if there are better ways to expose some of the data (such
as a function returning a named tuple containing the raw data
underlying the output of "python -m sysconfig").

Cheers,
Nick.

-- 
Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia

From ncoghlan at gmail.com  Sat Dec 11 03:55:25 2010
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sat, 11 Dec 2010 12:55:25 +1000
Subject: [Python-Dev] API for the new sysconfig module
In-Reply-To: <iduohr$q36$2@dough.gmane.org>
References: <B710BA0A-7097-4070-9DA4-E33149620833@gmail.com>
	<20101210215902.CDB5B1AB60E@kimball.webabinitio.net>
	<iduohr$q36$2@dough.gmane.org>
Message-ID: <AANLkTikWzL1R5cpTpOFE5dzrZ=rRy0CEWJHYJXhYc6ZV@mail.gmail.com>

On Sat, Dec 11, 2010 at 12:44 PM, Terry Reedy <tjreedy at udel.edu> wrote:
> On 12/10/2010 4:59 PM, R. David Murray wrote:
>
>> Like ?ric, I'm not sure what the implications of the existing module
>> having been released in 2.7 and 3.2 beta are in terms of making such an
>> API change.
>
> I am with Raymond on this: the purpose of betas is so we can test *and* make
> changes. No one should base production software on a beta release.

It's also the key difference between betas and release candidates.
Completely new features are still out after the first beta, but
tweaking the APIs of new-in-this-release features is still possible
with sufficient justification.

Cheers,
Nick.

-- 
Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia

From ethan at stoneleaf.us  Sat Dec 11 05:19:23 2010
From: ethan at stoneleaf.us (Ethan Furman)
Date: Fri, 10 Dec 2010 20:19:23 -0800
Subject: [Python-Dev] Using logging in the stdlib and its unit tests
In-Reply-To: <87oc8tsjll.fsf@uwakimon.sk.tsukuba.ac.jp>
References: <loom.20101207T212306-892@post.gmane.org>	<20101207215053.025ecf52@pitrou.net>	<loom.20101208T002100-91@post.gmane.org>	<20101208010913.363c6105@pitrou.net>	<AANLkTim4ZPoch18+-qkunuYm-zcyh-vbhMOv9YVVEqo9@mail.gmail.com>	<loom.20101208T022102-46@post.gmane.org>	<AANLkTikFNUPA5Cvbz8cOAfznSOEe906O8zOfOC=2Cj+H@mail.gmail.com>	<loom.20101210T085332-877@post.gmane.org>	<87tyimrmbd.fsf@uwakimon.sk.tsukuba.ac.jp>	<20101210113327.GA16785@phd.pp.ru>
	<87oc8tsjll.fsf@uwakimon.sk.tsukuba.ac.jp>
Message-ID: <4D02FBCB.50600@stoneleaf.us>

Stephen J. Turnbull wrote:
> They're not scared by that example.  What you need is a paragraph
> below it that says
> 
>     """
>     Do you think the above is all you should need?  If so, you're
>     right.  You can stop reading now.  If you think you need more,
>     we've got that, too.  Read on (you may need more coffee).
>     """ 

+1

Oleg Broytman writes:
> 
>  >    Better yet (IMHO) would be to split the huge page into "Logging: Simple
>  > start" and "Logging: Advanced usage (for the brave of of heart)".
> 
> Splitting is OK, but I disagree about the gloss "for the brave of
> heart".
> 
> In my experience, if it is a YAGNI, the complexity is nearly
> impenetrable.  If you *do* need it, it's not at all difficult to
> understand what the complexity is for, and it doesn't even look all
> that complex because it matches up with the problem you need to solve.
> 
> If the documentation is still a deterrent, that's a problem with the
> documentation and it should be improved.  AFAICT, making it clear that
> exporting all the internal flexibility is for those who need it, while
> most users will rarely need it, should be enough.  But I'm not a good
> test case, since I already am familiar with XEmacs's similar system.


I think I'm a pretty good test case -- knew nothing about logging, still 
don't know much, found documentation comprehensive but unweildy, and 
would *still* benefit from a more extensive (though still short) 
beginner's section, with the prominent paragraph stating I now know 
enough for simple cases.  :)

Oh, and awesome module, by the way.  Thank you.

~Ethan~

From ncoghlan at gmail.com  Sat Dec 11 06:06:20 2010
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sat, 11 Dec 2010 15:06:20 +1000
Subject: [Python-Dev] Using logging in the stdlib and its unit tests
In-Reply-To: <iduoeb$q36$1@dough.gmane.org>
References: <loom.20101207T212306-892@post.gmane.org>
	<20101207215053.025ecf52@pitrou.net>
	<loom.20101208T002100-91@post.gmane.org>
	<20101208010913.363c6105@pitrou.net>
	<AANLkTim4ZPoch18+-qkunuYm-zcyh-vbhMOv9YVVEqo9@mail.gmail.com>
	<loom.20101208T022102-46@post.gmane.org>
	<AANLkTikFNUPA5Cvbz8cOAfznSOEe906O8zOfOC=2Cj+H@mail.gmail.com>
	<loom.20101210T085332-877@post.gmane.org>
	<4D01E55B.2050507@g.nevcal.com> <3231.1292004978@parc.com>
	<loom.20101210T202041-859@post.gmane.org>
	<AANLkTi=PnWE5MbpyWKZHJU4Abk-OeVfKEJYpGX1E6g3=@mail.gmail.com>
	<20101210214919.4ac63d27@pitrou.net>
	<loom.20101210T224910-511@post.gmane.org>
	<iduoeb$q36$1@dough.gmane.org>
Message-ID: <AANLkTi=Q8mZb0oDe5xz7Evb0y8ohgkWh=X7E1OgCrUka@mail.gmail.com>

On Sat, Dec 11, 2010 at 12:42 PM, Terry Reedy <tjreedy at udel.edu> wrote:
> On 12/10/2010 5:16 PM, Vinay Sajip wrote:
>
>> IMO as long as it's just a small amount of work to get the specific effect
>> that you want, it doesn't really matter too much what the default is -
>> though
>> of course I'd like it to be "right", whatever that means ;-)
>
> I think the default should accomodate both internal uses (stdlib and
> elsewhere) and beginner uses. I suspect the latter would mean outputting
> most everything, to the convenience functions work (as a beginner might
> expect).

I think the main point of confusion is going to be what happens to
.info() messages by default.

With Vinay's latest changes, the 3.2 default is going to be:

debug() -> /dev/null
info() -> /dev/null
warning() -> stderr
error() -> stderr
critical() -> stderr
exception() -> stderr

This will apply both to the convenience functions (where this is
already the case due to the implicit basicConfig() call and the
default level setting on the root logger) and to logger methods (where
the new "handler of last resort" will apply instead of emitting the
comparatively unhelpful "no handler found" error message).

This is the correct behaviour for library code and even many
applications - silent when all goes well, but if anything goes wrong
that can't be flagged with an exception, the errors are displayed by
default. While Vinay is right that inertia plays a large part in not
using the logging module in the standard library, the new behaviour
for direct calls to loggers *is* a direct result of a problem
encountered in the concurrent.futures tests due to the fact that Brian
previously couldn't rely on an unconfigured logging module doing the
right thing.

However, the confusion that this setup will engender is that
encountered by Bill: by default, info() messages are silenced rather
than displayed on stdout.

Notice that even the recommended "basicConfig" approach to resolving
this is subtly flawed, since your choices are to either display info
messages on stderr (by specifying just a level argument) or to
redirect warning() et al to stdout instead of stderr (by also
specifying a stream argument). The "correct" answer (info and below to
stdout, warning and above to stderr) is actually quite tricky to set
up, since handlers have no innate concept of a "maximum level", so you
have to provide a filter function (more on that below [1]).

Basically, as of 3.2, the correct "default" use is likely to be:

print() for information you want to appear on stdout by default
(especially in scripts)
logging.debug() and .info() for additional debugging information that
should be silenced by default

warnings.warn() for actual programmatic warnings that should obey
warning filter rules (logging can hook the warnings display mechanism)
logging.warn() for warnings that should never raise an exception and
should be emitted every time they are encountered

raise an exception for normal error reporting
logging.error(), .critical() and .exception() for error reporting that
cannot raise an exception for some reason (e.g. long running processes
that are simply recording that an exception occurred)

Libraries can now safely use their own getLogger(__name__) loggers for
the latter two uses without worrying about the output being lost by
the default logging configuration, so they become a much better option
than printing directly to sys.stderr. Most importantly, someone
debugging the app will be able to insert additional context
information (e.g. stack traces, local variables values) without
needing to alter the code emitting the error message.

Regards,
Nick.

[1] As an exercise, I decided to actually configure recommended
console logging output style via the logging module. It proved to be
quite informative as to where some of the flaws in the current
documentation lie.

First and foremost, it was genuinely hard for me to figure out that
one of my key problems was actually that the root logger is created
with a level of WARNING by default so my handlers weren't even being
invoked. The only place I found this very important detail was in the
documentation of Logger.setLevel()! There really should be a section
early on in the logging documentation describing the root logger, how
to use it for basic logging, and its default settings (with and
without a call to "basicConfig"). Such a section would then include a
link to the latter section on the logging heirarchy.

In addition to that, the documentation of the LogRecord class should
really include a table of attribute names that are available for use
in formatting and filtering. Initially I thought the attribute names
corresponded to the parameter names, and subsequently resorted to a
dir() call in my filter to figure out what the actual names were after
"lvl" caused an AttributeError. Armed with the fact that the attribute
I wanted was called "levelno", I was able to discover the list of
LogRecord attributes was actually in the *Formatter* documentation. If
a proper list of attributes was added to the LogRecord documentation,
then the current table in the formatting section could be removed and
replaced with a reference to the table of attributes and some notes on
formatting (when the documentation for the new styles feature goes in
would probably be an appropriate time to fix this). Similarly, the
section on filters could point directly to that table when describing
possible filtering criteria.

Anyway, the shortest way I could find of setting this up (debug
silenced, info written to stdout, warning and above written to
stderr):

  import sys, logging
  root = logging.getLogger()
  # Turns out the level of the root logger is set to WARNING by default
  # That should be much easier to find out from the logging docs!
  root.setLevel(logging.DEBUG)
  # Send WARNING and above to stderr
  # basicConfig() is no help, since it sets the specified level on the
root logger
  # but NOT on the default handler it installs. To emulate console output
  # we want to skip the formatting anyway.
  err = logging.StreamHandler()
  err.setLevel(logging.WARNING)
  root.addHandler(err)
  # Send INFO to stdout
  def is_info(record):
      return record.levelno == logging.INFO
  out = logging.StreamHandler(stream=sys.stdout)
  out.addFilter(is_info)
  root.addHandler(out)

Cheers,
Nick.

-- 
Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia

From ncoghlan at gmail.com  Sat Dec 11 06:24:39 2010
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sat, 11 Dec 2010 15:24:39 +1000
Subject: [Python-Dev] Using logging in the stdlib and its unit tests
In-Reply-To: <AANLkTi=Q8mZb0oDe5xz7Evb0y8ohgkWh=X7E1OgCrUka@mail.gmail.com>
References: <loom.20101207T212306-892@post.gmane.org>
	<20101207215053.025ecf52@pitrou.net>
	<loom.20101208T002100-91@post.gmane.org>
	<20101208010913.363c6105@pitrou.net>
	<AANLkTim4ZPoch18+-qkunuYm-zcyh-vbhMOv9YVVEqo9@mail.gmail.com>
	<loom.20101208T022102-46@post.gmane.org>
	<AANLkTikFNUPA5Cvbz8cOAfznSOEe906O8zOfOC=2Cj+H@mail.gmail.com>
	<loom.20101210T085332-877@post.gmane.org>
	<4D01E55B.2050507@g.nevcal.com> <3231.1292004978@parc.com>
	<loom.20101210T202041-859@post.gmane.org>
	<AANLkTi=PnWE5MbpyWKZHJU4Abk-OeVfKEJYpGX1E6g3=@mail.gmail.com>
	<20101210214919.4ac63d27@pitrou.net>
	<loom.20101210T224910-511@post.gmane.org>
	<iduoeb$q36$1@dough.gmane.org>
	<AANLkTi=Q8mZb0oDe5xz7Evb0y8ohgkWh=X7E1OgCrUka@mail.gmail.com>
Message-ID: <AANLkTikN_378ZH2kCW-WJ2wWDVZrYb2zrQ9N1QMRvkg0@mail.gmail.com>

On Sat, Dec 11, 2010 at 3:06 PM, Nick Coghlan <ncoghlan at gmail.com> wrote:
> Basically, as of 3.2, the correct "default" use is likely to be:
>
> print() for information you want to appear on stdout by default
> (especially in scripts)
> logging.debug() and .info() for additional debugging information that
> should be silenced by default
>
> warnings.warn() for actual programmatic warnings that should obey
> warning filter rules (logging can hook the warnings display mechanism)
> logging.warn() for warnings that should never raise an exception and
> should be emitted every time they are encountered
>
> raise an exception for normal error reporting
> logging.error(), .critical() and .exception() for error reporting that
> cannot raise an exception for some reason (e.g. long running processes
> that are simply recording that an exception occurred)

This could actually make a reasonably good basic for a "task oriented"
subsection of the logging documentation. Something like:

===============
Task: Display console output for ordinary usage of a command line
script or program
Tool: Use print()

Task: Report events that occur during normal operation of a program
(e.g. for status monitoring or fault investigation)
Tool: Use logging.info() (or logging.debug() for especially voluminous
or expensive to calculate output)

Task: Issue a warning regarding a particular runtime event
Tool: Use warnings.warn() in library code if the issue is avoidable
and the client application should be modified to eliminate the
warning.
Use logging.warn() if there is nothing the client application can do
about the situation, but the event should still be noted

Task: Report an error regarding a particular runtime event
Tool: Raise an exception

Task: Report suppression of an error without raising an exception
(e.g. error handler in a long-running server process)
Tool: Use logging.error(), logging.exception(), or logging.critical()
as appropriate to the specific error and application domain

Note: all references to the root level logging functions in the above
recommendations also cover direct invocation of the corresponding
methods on specific logger instances.
===============

Cheers,
Nick.

-- 
Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia

From ncoghlan at gmail.com  Sat Dec 11 06:38:52 2010
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sat, 11 Dec 2010 15:38:52 +1000
Subject: [Python-Dev] [Python-checkins] r87162 -
	python/branches/release27-maint/Doc/library/stdtypes.rst
In-Reply-To: <20101211004102.E773DEE98A@mail.python.org>
References: <20101211004102.E773DEE98A@mail.python.org>
Message-ID: <AANLkTinwVb7=L9CLJ2RKuF3bMvv5S8=TYFo3xROkF=4T@mail.gmail.com>

On Sat, Dec 11, 2010 at 10:41 AM, raymond.hettinger
<python-checkins at python.org> wrote:
> Author: raymond.hettinger
> Date: Sat Dec 11 01:41:02 2010
> New Revision: 87162
>
> Log:
> Issue 2690: ?Doc fixup. ?xrange() objects are slicable.
>
>
> Modified:
> ? python/branches/release27-maint/Doc/library/stdtypes.rst

Are you sure about that one?

Python 2.7.0+ (trunk:86033, Nov  1 2010, 00:42:57)
[GCC 4.4.3] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> xrange(10)[1:2]
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: sequence index must be integer, not 'slice'

Cheers,
Nick.

-- 
Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia

From vinay_sajip at yahoo.co.uk  Sat Dec 11 07:18:57 2010
From: vinay_sajip at yahoo.co.uk (Vinay Sajip)
Date: Sat, 11 Dec 2010 06:18:57 +0000 (UTC)
Subject: [Python-Dev] Using logging in the stdlib and its unit tests
References: <loom.20101207T212306-892@post.gmane.org>
	<20101207215053.025ecf52@pitrou.net>
	<loom.20101208T002100-91@post.gmane.org>
	<20101208010913.363c6105@pitrou.net>
	<AANLkTim4ZPoch18+-qkunuYm-zcyh-vbhMOv9YVVEqo9@mail.gmail.com>
	<loom.20101208T022102-46@post.gmane.org>
	<AANLkTikFNUPA5Cvbz8cOAfznSOEe906O8zOfOC=2Cj+H@mail.gmail.com>
	<loom.20101210T085332-877@post.gmane.org>
	<4D01E55B.2050507@g.nevcal.com> <3231.1292004978@parc.com>
	<loom.20101210T202041-859@post.gmane.org>
	<AANLkTi=PnWE5MbpyWKZHJU4Abk-OeVfKEJYpGX1E6g3=@mail.gmail.com>
	<20101210214919.4ac63d27@pitrou.net>
	<loom.20101210T224910-511@post.gmane.org>
	<iduoeb$q36$1@dough.gmane.org>
	<AANLkTi=Q8mZb0oDe5xz7Evb0y8ohgkWh=X7E1OgCrUka@mail.gmail.com>
Message-ID: <loom.20101211T064001-431@post.gmane.org>

Nick,

Thanks for the detailed exposition.

> Notice that even the recommended "basicConfig" approach to resolving
> this is subtly flawed, since your choices are to either display info

Right. basicConfig() is pretty basic.

> Basically, as of 3.2, the correct "default" use is likely to be:

The suggestions make sense.

> [1] As an exercise, I decided to actually configure recommended
> console logging output style via the logging module. It proved to be
> quite informative as to where some of the flaws in the current
> documentation lie.
[snip]
> documentation of Logger.setLevel()! There really should be a section
> early on in the logging documentation describing the root logger, how

Agreed.
 
> In addition to that, the documentation of the LogRecord class should
> really include a table of attribute names that are available for use
> in formatting and filtering. Initially I thought the attribute names

Yes.

> formatting (when the documentation for the new styles feature goes in
> would probably be an appropriate time to fix this). Similarly, the

Sorry, what do you mean by "new styles feature"?

> Anyway, the shortest way I could find of setting this up (debug

Yes, except that you could have set the root logger level to INFO rather than
DEBUG.

This raises some more questions. Obviously, we don't want users to have to go
through these steps themselves. Once these steps have been finalised, the last
resort handler could be modified to do all this. Should this handler be part of
the public API, so that it can be inherited from etc?

I realise we're in beta and hence feature freeze ... just sayin'. We're all
consenting adults here, after all :-)

Regards,

Vinay Sajip


From vinay_sajip at yahoo.co.uk  Sat Dec 11 07:21:14 2010
From: vinay_sajip at yahoo.co.uk (Vinay Sajip)
Date: Sat, 11 Dec 2010 06:21:14 +0000 (UTC)
Subject: [Python-Dev] Using logging in the stdlib and its unit tests
References: <loom.20101207T212306-892@post.gmane.org>
	<20101207215053.025ecf52@pitrou.net>
	<loom.20101208T002100-91@post.gmane.org>
	<20101208010913.363c6105@pitrou.net>
	<AANLkTim4ZPoch18+-qkunuYm-zcyh-vbhMOv9YVVEqo9@mail.gmail.com>
	<loom.20101208T022102-46@post.gmane.org>
	<AANLkTikFNUPA5Cvbz8cOAfznSOEe906O8zOfOC=2Cj+H@mail.gmail.com>
	<loom.20101210T085332-877@post.gmane.org>
	<4D01E55B.2050507@g.nevcal.com> <3231.1292004978@parc.com>
	<loom.20101210T202041-859@post.gmane.org>
	<AANLkTi=PnWE5MbpyWKZHJU4Abk-OeVfKEJYpGX1E6g3=@mail.gmail.com>
	<20101210214919.4ac63d27@pitrou.net>
	<loom.20101210T224910-511@post.gmane.org>
	<iduoeb$q36$1@dough.gmane.org>
	<AANLkTi=Q8mZb0oDe5xz7Evb0y8ohgkWh=X7E1OgCrUka@mail.gmail.com>
	<AANLkTikN_378ZH2kCW-WJ2wWDVZrYb2zrQ9N1QMRvkg0@mail.gmail.com>
Message-ID: <loom.20101211T072018-194@post.gmane.org>

Nick Coghlan <ncoghlan <at> gmail.com> writes:


> This could actually make a reasonably good basic for a "task oriented"
> subsection of the logging documentation. Something like:
> 

Good suggestion, I'll see what I can do.

Thanks,


Vinay Sajip


From v+python at g.nevcal.com  Sat Dec 11 07:25:54 2010
From: v+python at g.nevcal.com (Glenn Linderman)
Date: Fri, 10 Dec 2010 22:25:54 -0800
Subject: [Python-Dev] Using logging in the stdlib and its unit tests
In-Reply-To: <AANLkTi=Q8mZb0oDe5xz7Evb0y8ohgkWh=X7E1OgCrUka@mail.gmail.com>
References: <loom.20101207T212306-892@post.gmane.org>	<20101207215053.025ecf52@pitrou.net>	<loom.20101208T002100-91@post.gmane.org>	<20101208010913.363c6105@pitrou.net>	<AANLkTim4ZPoch18+-qkunuYm-zcyh-vbhMOv9YVVEqo9@mail.gmail.com>	<loom.20101208T022102-46@post.gmane.org>	<AANLkTikFNUPA5Cvbz8cOAfznSOEe906O8zOfOC=2Cj+H@mail.gmail.com>	<loom.20101210T085332-877@post.gmane.org>	<4D01E55B.2050507@g.nevcal.com>
	<3231.1292004978@parc.com>	<loom.20101210T202041-859@post.gmane.org>	<AANLkTi=PnWE5MbpyWKZHJU4Abk-OeVfKEJYpGX1E6g3=@mail.gmail.com>	<20101210214919.4ac63d27@pitrou.net>	<loom.20101210T224910-511@post.gmane.org>	<iduoeb$q36$1@dough.gmane.org>
	<AANLkTi=Q8mZb0oDe5xz7Evb0y8ohgkWh=X7E1OgCrUka@mail.gmail.com>
Message-ID: <4D031972.5090105@g.nevcal.com>

On 12/10/2010 9:06 PM, Nick Coghlan wrote:
> Anyway, the shortest way I could find of setting this up (debug
> silenced, info written to stdout, warning and above written to
> stderr):
>
>    import sys, logging
>    root = logging.getLogger()
>    # Turns out the level of the root logger is set to WARNING by default
>    # That should be much easier to find out from the logging docs!
>    root.setLevel(logging.DEBUG)
>    # Send WARNING and above to stderr
>    # basicConfig() is no help, since it sets the specified level on the
> root logger
>    # but NOT on the default handler it installs. To emulate console output
>    # we want to skip the formatting anyway.
>    err = logging.StreamHandler()
>    err.setLevel(logging.WARNING)
>    root.addHandler(err)
>    # Send INFO to stdout
>    def is_info(record):
>        return record.levelno == logging.INFO
>    out = logging.StreamHandler(stream=sys.stdout)
>    out.addFilter(is_info)
>    root.addHandler(out)

Wow, that's way longer than I expected... almost as long as my "cheap 
logger".  Greg didn't flesh out the "setup" stuff, but you have, so this 
is educational.  Thanks.

Not sure I want my INFO to go to stdout, by the way.  That corresponds 
to "debugging print statement" behavior, but even my "cheap logger" 
actually puts stuff in a file instead, and gets it out of the way of the 
normal output.

I'd suggest that this be "simplified" to put all the messages in the 
same (disk) file, but then I'm sure that there are other preferences 
than mine... you clearly thought about what you wanted/thought would be 
useful, in coming up with the above.

On 12/10/2010 9:24 PM, Nick Coghlan wrote:
> This could actually make a reasonably good basic for a "task oriented"
> subsection of the logging documentation. Something like:

Yep, agree.  But sadly, for each point, there may be multiple options 
(your StreamHandler, but I'd want a FileHandler; your separation of 
messages by level, my wanting them combined; etc.)

Your comment about basicConfig setting the level on the root logger, but 
not on the default handler making it useless is opaque to me, but is 
there perhaps room for another basic setup API that could get the setup 
code down to a line or two in simple cases?  Maybe 3 parameters:

1.  minimum level to be generated, which would be passed through to the 
root logger and anything else defined by this basic setup API

2. whether debug and info should go to the same or different stream/file 
as warn+ messages.  Maybe this is a stream or filename or None, the last 
implying to use the warn+ output.

3. Where to send the warn+ output.

maybe a 4. Maximum (size, count) of the round-robin log files, assuming 
that either or both of 2 & 3 specify files.

Would that be a useful set of functionality to bundle?  And could it be 
extended, when the user wants more power, or would it have to be 
replaced, because it gets in the way of the user that wants more power?




-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20101210/38ef8dae/attachment.html>

From ncoghlan at gmail.com  Sat Dec 11 09:00:39 2010
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sat, 11 Dec 2010 18:00:39 +1000
Subject: [Python-Dev] Using logging in the stdlib and its unit tests
In-Reply-To: <4D031972.5090105@g.nevcal.com>
References: <loom.20101207T212306-892@post.gmane.org>
	<20101207215053.025ecf52@pitrou.net>
	<loom.20101208T002100-91@post.gmane.org>
	<20101208010913.363c6105@pitrou.net>
	<AANLkTim4ZPoch18+-qkunuYm-zcyh-vbhMOv9YVVEqo9@mail.gmail.com>
	<loom.20101208T022102-46@post.gmane.org>
	<AANLkTikFNUPA5Cvbz8cOAfznSOEe906O8zOfOC=2Cj+H@mail.gmail.com>
	<loom.20101210T085332-877@post.gmane.org>
	<4D01E55B.2050507@g.nevcal.com> <3231.1292004978@parc.com>
	<loom.20101210T202041-859@post.gmane.org>
	<AANLkTi=PnWE5MbpyWKZHJU4Abk-OeVfKEJYpGX1E6g3=@mail.gmail.com>
	<20101210214919.4ac63d27@pitrou.net>
	<loom.20101210T224910-511@post.gmane.org>
	<iduoeb$q36$1@dough.gmane.org>
	<AANLkTi=Q8mZb0oDe5xz7Evb0y8ohgkWh=X7E1OgCrUka@mail.gmail.com>
	<4D031972.5090105@g.nevcal.com>
Message-ID: <AANLkTimpq=hpV+LCcQLK7C4O7spbBadjBAFr3Jkh98bC@mail.gmail.com>

On Sat, Dec 11, 2010 at 4:25 PM, Glenn Linderman <v+python at g.nevcal.com> wrote:
> On 12/10/2010 9:24 PM, Nick Coghlan wrote:
> This could actually make a reasonably good basic for a "task oriented"
> subsection of the logging documentation. Something like:
>
> Yep, agree.? But sadly, for each point, there may be multiple options (your
> StreamHandler, but I'd want a FileHandler; your separation of messages by
> level, my wanting them combined; etc.)

No, no, no, that's the *whole point* of using logging. The library
writer doesn't really *care* about where the messages end up - that is
entirely in the hands of the application developer when they choose
which handlers to install. The only situation that the library writer
cares about is the one that tripped up concurrent.futures and that has
already been changed for 3.2b2: that warnings and errors written to a
logger were silenced by default if the application never even called
basicConfig(). That has now been fixed so they will be emitted on
sys.stderr instead (including obeying any redirections of stderr over
the lifetime of the program).

Notice that my task list is *entirely* from the point of view of the
person emitting the messages. How those messages are later displayed
is then up to the application writer (or the logging module default
settings, if the application writer doesn't do anything specific).

> Your comment about basicConfig setting the level on the root logger, but not
> on the default handler making it useless is opaque to me,

An early version of my code used basicConfig to create the stderr
StreamHandler, but that turned out to be pointless since I wanted
different levels on the logger and the handler.

> but is there
> perhaps room for another basic setup API that could get the setup code down
> to a line or two in simple cases?

For 3.2? No.

For 3.3? Maybe.

> Would that be a useful set of functionality to bundle?? And could it be
> extended, when the user wants more power, or would it have to be replaced,
> because it gets in the way of the user that wants more power?

Logging already has powerful configuration mechanisms, especially
following the addition of dictConfig
(http://docs.python.org/dev/library/logging#logging-config-dictschema),
so it really doesn't need anything else along those lines.

At the simpler end, basicConfig already covers sending all messages to
a single stream with a single format - it's only split-stream handling
(such as stderr/stdout potentially being different endpoints) that it
can't cope with.

What may make more sense than yet another global config mechanism, is
a module level "addHandler" helper function along the following lines:

from logging import Formatter, FileHandler, StreamHandler, getLogger
def addHandler(*, handler=None, stream=None, filename=None, filemode='a',
                          format=None, datefmt=None, style='{',
                          level=None, max_level=None, filters=(),
                          logger=None):
    """stream, filename, level, format, datefmt, style: as per
logging.basicConfig

        handler: use a precreated handler instead of creating a new one
        logger: logger to add the handler to (uses root logger if none
specified)
        filters: an iterable of filters to add to the handler
        max_level: can optionally limit the handler to messages of a
certain level and below
    """
    # Create the handler if one hasn't been passed in
    if handler is None:
        if filename is not None:
            handler = FileHandler(filename, filemode)
        else:
            handler = StreamHandler(stream)
    # Set up the formatting of the log messages
    # New API, so it can default to str.format instead of %-formatting
    formatter = Formatter(format, datefmt, style)
    handler.setFormatter(formatter)
    # Set up filtering of which messages to handle
    if level is not None:
        handler.setLevel(level)
    if max_level is not None:
        def level_ok(record):
            return record.levelno <= max_level
        handler.addFilter(level_ok)
    for filter in filters:
        handler.addFilter(filter)
    # Add the fully configured handler to the specified logger
    if logger is None:
        logger = getLogger()
    logger.addHandler(handler)
    return handler

# Previous set up example is now only three lines long
import sys, logging
# Let root logger handlers see all messages
logging.getLogger().setLevel(logging.NOTSET)
# Send WARNING and above to stderr
addHandler(stream=sys.stderr, level=logging.WARNING)
# Send INFO to stdout
addHandler(stream=sys.stdout, level=logging.INFO, max_level=logging.INFO)

logging.info("Hello world!")
logging.warn("Hello world!")

Cheers,
Nick.

-- 
Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia

From ncoghlan at gmail.com  Sat Dec 11 09:14:27 2010
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sat, 11 Dec 2010 18:14:27 +1000
Subject: [Python-Dev] Using logging in the stdlib and its unit tests
In-Reply-To: <AANLkTimpq=hpV+LCcQLK7C4O7spbBadjBAFr3Jkh98bC@mail.gmail.com>
References: <loom.20101207T212306-892@post.gmane.org>
	<20101207215053.025ecf52@pitrou.net>
	<loom.20101208T002100-91@post.gmane.org>
	<20101208010913.363c6105@pitrou.net>
	<AANLkTim4ZPoch18+-qkunuYm-zcyh-vbhMOv9YVVEqo9@mail.gmail.com>
	<loom.20101208T022102-46@post.gmane.org>
	<AANLkTikFNUPA5Cvbz8cOAfznSOEe906O8zOfOC=2Cj+H@mail.gmail.com>
	<loom.20101210T085332-877@post.gmane.org>
	<4D01E55B.2050507@g.nevcal.com> <3231.1292004978@parc.com>
	<loom.20101210T202041-859@post.gmane.org>
	<AANLkTi=PnWE5MbpyWKZHJU4Abk-OeVfKEJYpGX1E6g3=@mail.gmail.com>
	<20101210214919.4ac63d27@pitrou.net>
	<loom.20101210T224910-511@post.gmane.org>
	<iduoeb$q36$1@dough.gmane.org>
	<AANLkTi=Q8mZb0oDe5xz7Evb0y8ohgkWh=X7E1OgCrUka@mail.gmail.com>
	<4D031972.5090105@g.nevcal.com>
	<AANLkTimpq=hpV+LCcQLK7C4O7spbBadjBAFr3Jkh98bC@mail.gmail.com>
Message-ID: <AANLkTi=03r13HjPyupjLwime4N90vdkCjJO_6QVE=Fso@mail.gmail.com>

On Sat, Dec 11, 2010 at 6:00 PM, Nick Coghlan <ncoghlan at gmail.com> wrote:
> What may make more sense than yet another global config mechanism, is
> a module level "addHandler" helper function along the following lines:

For a more readable version of that example, take a look at the copy I
put up over on the Python Cookbook site:
http://code.activestate.com/recipes/577496-simple-creation-configuration-and-installation-of-/

Cheers,
Nick.

-- 
Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia

From ziade.tarek at gmail.com  Sat Dec 11 09:26:35 2010
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Sat, 11 Dec 2010 09:26:35 +0100
Subject: [Python-Dev] API for the new sysconfig module
In-Reply-To: <AANLkTikXg0E5DQuY8HxxrqkoZj0NzTpNcTyetLuBi_xc@mail.gmail.com>
References: <B710BA0A-7097-4070-9DA4-E33149620833@gmail.com>
	<20101210215902.CDB5B1AB60E@kimball.webabinitio.net>
	<AANLkTikXg0E5DQuY8HxxrqkoZj0NzTpNcTyetLuBi_xc@mail.gmail.com>
Message-ID: <AANLkTikNB3f7HDJ+QP-ta9meORr-651tayBwYSOgAL0W@mail.gmail.com>

Sorry to get late in the discussion I am travelling.

Nick resumes well the motivations behind sysconfig.

I'll emphase that this module could hold more functions in the future that
could be useful to other python implementations to abstract what is a python
installation. E.g. more than paths and variables.

On Dec 11, 2010 3:53 AM, "Nick Coghlan" <ncoghlan at gmail.com> wrote:

On Sat, Dec 11, 2010 at 7:59 AM, R. David Murray <rdmurray at bitdance.com>
wrote:
> On Thu, 09 Dec 201...

>> ISTM it mostly could have been reduced to single call returning a nested
dictionary.
>
> If what ...
We've reverted from beta to pseudo-alpha after discovering
sufficiently egregious mistakes in the past. (we didn't *actually*
revert the build naming because release IDs that don't increase
monotonically can confuse matters). However, I don't think this is
sufficiently egregious to qualify.

My own reasons for advocating retention of the separate module:

1. Something equivalent to the current sysconfig module is going to be
needed to actually work out the contents of the data structure.
Rewriting all that in C would be silly, so the most that could happen
is to rename "sysconfig" to "_sysconfig".

2. Invoking Python code from sys module functions is possible but
untidy (generally involving implicit module imports). Since we need a
module anyway for implementation purposes, why not make it public?

3. The sysconfig module docs provide a nicer space to talk about the
purpose of the provided information rather than squirreling it away in
a corner of the sys module docs

4. "python -m sysconfig" provides a useful dump of the system status

5. The API design did not get much *new* thought, as the idea was to
allow users to do anything that the old distutils.sysconfig could do,
but without requiring them to depend on distutils to get that
functionality. The easiest way to ensure that is to just copy the
existing API rather than trying to get creative. Don't think of it as
adding a "whole new module", thinking of it as decoupling a relatively
small existing module (i.e. distutils.sysconfig) from a larger package
that isn't installed by default on all systems (i.e. distutils).

A concrete problem with Raymond's idea in particular is that there is
actually a full set of paths defined for *each scheme*. get_path() and
get_paths() let you ignore this most of the time, since there is an
appropriate default scheme defined for each platform, so they just use
that if you don't specify one explicitly. A named tuple could
obviously expose the paths for all of the schemes, with the paths for
the default scheme duplicated at the top level, but the data structure
starts looking very messy at that point. The named tuple idea also
conflates meta-data about the sysconfig data (the list of scheme names
and path names) with the actual data for the current platform
(platform, version, paths, config_vars).

So I think reusing the tried and tested API is easily justified in
this case. What may be desirable in 3.3 (particularly with the
distutils2 reintegration taking place), is to revisit the sysconfig
API to see if there are better ways to expose some of the data (such
as a function returning a named tuple containing the raw data
underlying the output of "python -m sysconfig").


Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
________________...

Python-Dev mailing list
Python-Dev at python.org
http://mail.python.org/mailman/listinfo/python-dev
Uns...
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20101211/39dc0c69/attachment.html>

From ncoghlan at gmail.com  Sat Dec 11 09:35:29 2010
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sat, 11 Dec 2010 18:35:29 +1000
Subject: [Python-Dev] Using logging in the stdlib and its unit tests
In-Reply-To: <loom.20101211T064001-431@post.gmane.org>
References: <loom.20101207T212306-892@post.gmane.org>
	<20101207215053.025ecf52@pitrou.net>
	<loom.20101208T002100-91@post.gmane.org>
	<20101208010913.363c6105@pitrou.net>
	<AANLkTim4ZPoch18+-qkunuYm-zcyh-vbhMOv9YVVEqo9@mail.gmail.com>
	<loom.20101208T022102-46@post.gmane.org>
	<AANLkTikFNUPA5Cvbz8cOAfznSOEe906O8zOfOC=2Cj+H@mail.gmail.com>
	<loom.20101210T085332-877@post.gmane.org>
	<4D01E55B.2050507@g.nevcal.com> <3231.1292004978@parc.com>
	<loom.20101210T202041-859@post.gmane.org>
	<AANLkTi=PnWE5MbpyWKZHJU4Abk-OeVfKEJYpGX1E6g3=@mail.gmail.com>
	<20101210214919.4ac63d27@pitrou.net>
	<loom.20101210T224910-511@post.gmane.org>
	<iduoeb$q36$1@dough.gmane.org>
	<AANLkTi=Q8mZb0oDe5xz7Evb0y8ohgkWh=X7E1OgCrUka@mail.gmail.com>
	<loom.20101211T064001-431@post.gmane.org>
Message-ID: <AANLkTinvjTwBwzoNKK+0u8Z74+2cH_a5Qt2vRvupJF1M@mail.gmail.com>

On Sat, Dec 11, 2010 at 4:18 PM, Vinay Sajip <vinay_sajip at yahoo.co.uk> wrote:
>> formatting (when the documentation for the new styles feature goes in
>> would probably be an appropriate time to fix this). Similarly, the
>
> Sorry, what do you mean by "new styles feature"?

The switching between percent, str.format and string.Template
formatting. It turns out that has been documented, but the table in
question is still written from a percent-style formatting point of
view.

>> Anyway, the shortest way I could find of setting this up (debug
>
> Yes, except that you could have set the root logger level to INFO rather than
> DEBUG.
>
> This raises some more questions. Obviously, we don't want users to have to go
> through these steps themselves. Once these steps have been finalised, the last
> resort handler could be modified to do all this. Should this handler be part of
> the public API, so that it can be inherited from etc?

The "lazy" stream handler might be useful to make public in some way.
For example, rather than hardcoding sys.stderr, it could take a
callable that it uses to retrieve the stream. That kind of change can
wait until 3.3 though - what is there now should be fine to let us get
concurrent.futures and the logging tests to play nicely together.

As far as whether anything else in the logging defaults needs to
change, I don't think so. I'm now happy that if an application needs
to emit text on stdout as part of its normal operation, then that
isn't what the logging module is for - logging messages are intended
to record "this happened, then that happened, then this other thing
happened". They are *not* properly part of an application's console UI
(which is reflected in the whole "silent by default" and "console
output defaults to sys.stderr" philosophy). The change to the default
behaviour was just to make logging.warning() etc a suitable
replacement for writing directly to sys.stderr when exceptions can't
be raised, which is in keeping with the existing spirit of the logging
module.

> I realise we're in beta and hence feature freeze ... just sayin'. We're all
> consenting adults here, after all :-)

I don't think we should be adding more *code* at this stage. But
documentation fixes and adjustments may be a good idea. I've learned
quite a bit myself about the logging module in the course of this
discussion, so it would be nice if some of that could be captured and
used to improve the documentation.

That said, something that might be interesting is to see what my
addHandler() recipe [1] would do for the examples in the logging
documentation. If it makes a big enough difference, then it may be
worth talking to Georg about adding it, even for 3.2. Alternatively,
if you like the idea, but we don't want to break the feature freeze
for it, then it may make a nice "see-also" link from the docs. I find
it useful from a "here are all the things I can configure on a handler
in one easy bundle" point of view, even aside from any benefits in
reduced typing.

Cheers,
Nick.

[1] http://code.activestate.com/recipes/577496-simple-creation-configuration-and-installation-of-/

-- 
Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia

From vinay_sajip at yahoo.co.uk  Sat Dec 11 09:52:45 2010
From: vinay_sajip at yahoo.co.uk (Vinay Sajip)
Date: Sat, 11 Dec 2010 08:52:45 +0000 (UTC)
Subject: [Python-Dev] Using logging in the stdlib and its unit tests
References: <loom.20101207T212306-892@post.gmane.org>	<20101207215053.025ecf52@pitrou.net>	<loom.20101208T002100-91@post.gmane.org>	<20101208010913.363c6105@pitrou.net>	<AANLkTim4ZPoch18+-qkunuYm-zcyh-vbhMOv9YVVEqo9@mail.gmail.com>	<loom.20101208T022102-46@post.gmane.org>	<AANLkTikFNUPA5Cvbz8cOAfznSOEe906O8zOfOC=2Cj+H@mail.gmail.com>	<loom.20101210T085332-877@post.gmane.org>	<4D01E55B.2050507@g.nevcal.com>
	<3231.1292004978@parc.com>	<loom.20101210T202041-859@post.gmane.org>	<AANLkTi=PnWE5MbpyWKZHJU4Abk-OeVfKEJYpGX1E6g3=@mail.gmail.com>	<20101210214919.4ac63d27@pitrou.net>	<loom.20101210T224910-511@post.gmane.org>	<iduoeb$q36$1@dough.gmane.org>
	<AANLkTi=Q8mZb0oDe5xz7Evb0y8ohgkWh=X7E1OgCrUka@mail.gmail.com>
	<4D031972.5090105@g.nevcal.com>
Message-ID: <loom.20101211T094138-775@post.gmane.org>

Glenn Linderman <v+python <at> g.nevcal.com> writes:

>     Yep, agree.? But sadly, for each point, there may be multiple
>     options (your StreamHandler, but I'd want a FileHandler; your
>     separation of messages by level, my wanting them combined; etc.)

That's partly why logging hasn't made much effort in the "default" department:
everyone wants something different.

>     Your comment about basicConfig setting the level on the root logger,
>     but not on the default handler making it useless is opaque to me,

I think he just means that he can't use basicConfig because it just sets the
root logger's level to whatever's passed in, but what he actually wants to set
are two different levels: INFO on the logger, WARNING on the stderr handler.
On top of that, he wants to have a stdout handler which only outputs INFO
events, and to get that effect he has to add the filter to the stdout handler
(otherwise it would output INFO + anything at a higher level, too).

>     but is there perhaps room for another basic setup API that could get
>     the setup code down to a line or two in simple cases?? Maybe 3
>     parameters:
[snip]
>     Would that be a useful set of functionality to bundle?? And could it
>     be extended, when the user wants more power, or would it have to be
>     replaced, because it gets in the way of the user that wants more
>     power?
> 

It's possible that perhaps there could be some 'canned configs', as I've
mentioned in another post on this thread. Some more thinking needs to be done
around this area.

Regards,

Vinay Sajip



From ncoghlan at gmail.com  Sat Dec 11 10:07:55 2010
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sat, 11 Dec 2010 19:07:55 +1000
Subject: [Python-Dev] Using logging in the stdlib and its unit tests
In-Reply-To: <AANLkTi=Q8mZb0oDe5xz7Evb0y8ohgkWh=X7E1OgCrUka@mail.gmail.com>
References: <loom.20101207T212306-892@post.gmane.org>
	<20101207215053.025ecf52@pitrou.net>
	<loom.20101208T002100-91@post.gmane.org>
	<20101208010913.363c6105@pitrou.net>
	<AANLkTim4ZPoch18+-qkunuYm-zcyh-vbhMOv9YVVEqo9@mail.gmail.com>
	<loom.20101208T022102-46@post.gmane.org>
	<AANLkTikFNUPA5Cvbz8cOAfznSOEe906O8zOfOC=2Cj+H@mail.gmail.com>
	<loom.20101210T085332-877@post.gmane.org>
	<4D01E55B.2050507@g.nevcal.com> <3231.1292004978@parc.com>
	<loom.20101210T202041-859@post.gmane.org>
	<AANLkTi=PnWE5MbpyWKZHJU4Abk-OeVfKEJYpGX1E6g3=@mail.gmail.com>
	<20101210214919.4ac63d27@pitrou.net>
	<loom.20101210T224910-511@post.gmane.org>
	<iduoeb$q36$1@dough.gmane.org>
	<AANLkTi=Q8mZb0oDe5xz7Evb0y8ohgkWh=X7E1OgCrUka@mail.gmail.com>
Message-ID: <AANLkTi=DDZHt0Ai4Pisi-mx=U5FUwxt6XL7S9f1r1xxX@mail.gmail.com>

On Sat, Dec 11, 2010 at 3:06 PM, Nick Coghlan <ncoghlan at gmail.com> wrote:
> However, the confusion that this setup will engender is that
> encountered by Bill: by default, info() messages are silenced rather
> than displayed on stdout.
>
> Notice that even the recommended "basicConfig" approach to resolving
> this is subtly flawed, since your choices are to either display info
> messages on stderr (by specifying just a level argument) or to
> redirect warning() et al to stdout instead of stderr (by also
> specifying a stream argument). The "correct" answer (info and below to
> stdout, warning and above to stderr) is actually quite tricky to set
> up, since handlers have no innate concept of a "maximum level", so you
> have to provide a filter function (more on that below [1]).

I need to correct this comment, since it's wrong and
basicConfig(level=logging.INFO) actually does the right thing.

As Glenn mentioned later in the thread, the output of logging.info and
logging.debug messages is *distinct* from an application's normal
operational output that is emitted on stdout. So making it easy to
emit such messages on stderr is the right thing to do - it's OK that
it requires a bit of additional set up to get them onto stdout instead
(since they really don't belong there most of the time).

I know my own rule of thumb is going to be updated along the lines of
"Am I writing to sys.stderr, or guarding output with a verbosity flag?
Then perhaps I should be using the logging module rather than print
statements".

Cheers,
Nick.

-- 
Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia

From vinay_sajip at yahoo.co.uk  Sat Dec 11 10:28:14 2010
From: vinay_sajip at yahoo.co.uk (Vinay Sajip)
Date: Sat, 11 Dec 2010 09:28:14 +0000 (UTC)
Subject: [Python-Dev] Using logging in the stdlib and its unit tests
References: <loom.20101207T212306-892@post.gmane.org>
	<20101207215053.025ecf52@pitrou.net>
	<loom.20101208T002100-91@post.gmane.org>
	<20101208010913.363c6105@pitrou.net>
	<AANLkTim4ZPoch18+-qkunuYm-zcyh-vbhMOv9YVVEqo9@mail.gmail.com>
	<loom.20101208T022102-46@post.gmane.org>
	<AANLkTikFNUPA5Cvbz8cOAfznSOEe906O8zOfOC=2Cj+H@mail.gmail.com>
	<loom.20101210T085332-877@post.gmane.org>
	<4D01E55B.2050507@g.nevcal.com> <3231.1292004978@parc.com>
	<loom.20101210T202041-859@post.gmane.org>
	<AANLkTi=PnWE5MbpyWKZHJU4Abk-OeVfKEJYpGX1E6g3=@mail.gmail.com>
	<20101210214919.4ac63d27@pitrou.net>
	<loom.20101210T224910-511@post.gmane.org>
	<iduoeb$q36$1@dough.gmane.org>
	<AANLkTi=Q8mZb0oDe5xz7Evb0y8ohgkWh=X7E1OgCrUka@mail.gmail.com>
	<loom.20101211T064001-431@post.gmane.org>
	<AANLkTinvjTwBwzoNKK+0u8Z74+2cH_a5Qt2vRvupJF1M@mail.gmail.com>
Message-ID: <loom.20101211T095653-346@post.gmane.org>

Nick Coghlan <ncoghlan <at> gmail.com> writes:

> The switching between percent, str.format and string.Template
> formatting. It turns out that has been documented, but the table in
> question is still written from a percent-style formatting point of
> view.

Oh, right - yes. I presume you mean the table in the formatter documentation
which shows %(message)s etc.
 

> The "lazy" stream handler might be useful to make public in some way.
> For example, rather than hardcoding sys.stderr, it could take a
> callable that it uses to retrieve the stream. That kind of change can
> wait until 3.3 though - what is there now should be fine to let us get
> concurrent.futures and the logging tests to play nicely together.

Actually if we're to change things to print INFO to stdout and WARNING+ to
stderr, I'd change the last resort handler class to be more than just defining
a lazy stream property: perhaps something like [untested]

class _DefaultHandler(StreamHandler):
    def __init__(self, level=NOTSET):
        """
        Initialize the handler.
        """
        Handler.__init__(self, level)

    def emit(self, record):
        if record.levelno == INFO:
            self.stream = sys.stdout
        else:
            self.stream = sys.stderr
        super(_DefaultHandler, self).emit(record)

> As far as whether anything else in the logging defaults needs to
> change, I don't think so. I'm now happy that if an application needs

I agree it can be refined later, but if we agreed that INFO goes to stdout,
then something like the above could be put in place for 3.2 (perhaps 3.2b2).
 
> output defaults to sys.stderr" philosophy). The change to the default
> behaviour was just to make logging.warning() etc a suitable
> replacement for writing directly to sys.stderr when exceptions can't
> be raised, which is in keeping with the existing spirit of the logging
> module.

Agreed.
 
> I don't think we should be adding more *code* at this stage. But
> documentation fixes and adjustments may be a good idea. I've learned
> quite a bit myself about the logging module in the course of this
> discussion, so it would be nice if some of that could be captured and
> used to improve the documentation.

Yes, you (and Glenn, and others) have given me quite a bit to work on, on that
score!
 
> That said, something that might be interesting is to see what my
> addHandler() recipe [1] would do for the examples in the logging
> documentation. If it makes a big enough difference, then it may be
> worth talking to Georg about adding it, even for 3.2. Alternatively,
> if you like the idea, but we don't want to break the feature freeze
> for it, then it may make a nice "see-also" link from the docs. I find
> it useful from a "here are all the things I can configure on a handler
> in one easy bundle" point of view, even aside from any benefits in
> reduced typing.

Actually I've been thinking about a handlers= optional argument for basicConfig
for some time (which, if provided, would override file and stream arguments).
But it seemed a little unwieldy and came across plainly as the afterthought it
admittedly was :-)

Regards,

Vinay Sajip



From v+python at g.nevcal.com  Sat Dec 11 10:30:58 2010
From: v+python at g.nevcal.com (Glenn Linderman)
Date: Sat, 11 Dec 2010 01:30:58 -0800
Subject: [Python-Dev] Using logging in the stdlib and its unit tests
In-Reply-To: <AANLkTimpq=hpV+LCcQLK7C4O7spbBadjBAFr3Jkh98bC@mail.gmail.com>
References: <loom.20101207T212306-892@post.gmane.org>	<20101207215053.025ecf52@pitrou.net>	<loom.20101208T002100-91@post.gmane.org>	<20101208010913.363c6105@pitrou.net>	<AANLkTim4ZPoch18+-qkunuYm-zcyh-vbhMOv9YVVEqo9@mail.gmail.com>	<loom.20101208T022102-46@post.gmane.org>	<AANLkTikFNUPA5Cvbz8cOAfznSOEe906O8zOfOC=2Cj+H@mail.gmail.com>	<loom.20101210T085332-877@post.gmane.org>	<4D01E55B.2050507@g.nevcal.com>	<3231.1292004978@parc.com>	<loom.20101210T202041-859@post.gmane.org>	<AANLkTi=PnWE5MbpyWKZHJU4Abk-OeVfKEJYpGX1E6g3=@mail.gmail.com>	<20101210214919.4ac63d27@pitrou.net>	<loom.20101210T224910-511@post.gmane.org>	<iduoeb$q36$1@dough.gmane.org>	<AANLkTi=Q8mZb0oDe5xz7Evb0y8ohgkWh=X7E1OgCrUka@mail.gmail.com>	<4D031972.5090105@g.nevcal.com>
	<AANLkTimpq=hpV+LCcQLK7C4O7spbBadjBAFr3Jkh98bC@mail.gmail.com>
Message-ID: <4D0344D2.3070006@g.nevcal.com>

On 12/11/2010 12:00 AM, Nick Coghlan wrote:
> On Sat, Dec 11, 2010 at 4:25 PM, Glenn Linderman<v+python at g.nevcal.com>  wrote:
>> On 12/10/2010 9:24 PM, Nick Coghlan wrote:
>> This could actually make a reasonably good basic for a "task oriented"
>> subsection of the logging documentation. Something like:
>>
>> Yep, agree.  But sadly, for each point, there may be multiple options (your
>> StreamHandler, but I'd want a FileHandler; your separation of messages by
>> level, my wanting them combined; etc.)
> No, no, no, that's the *whole point* of using logging. The library
> writer doesn't really *care* about where the messages end up - that is
> entirely in the hands of the application developer when they choose
> which handlers to install. The only situation that the library writer
> cares about is the one that tripped up concurrent.futures and that has
> already been changed for 3.2b2: that warnings and errors written to a
> logger were silenced by default if the application never even called
> basicConfig(). That has now been fixed so they will be emitted on
> sys.stderr instead (including obeying any redirections of stderr over
> the lifetime of the program).
>
> Notice that my task list is *entirely* from the point of view of the
> person emitting the messages. How those messages are later displayed
> is then up to the application writer (or the logging module default
> settings, if the application writer doesn't do anything specific).

Yes, yes, yes... that's what your list was!  I see that now!  But it is 
interesting that in missing that point, and reading your logging setup 
code, I saw that the same list of tasks is also exactly the sorts of 
things that you would want to control, for simple uses of the logger.  
Does basicConfig give that control?  I don't know, the first 8% of the 
logger documentation don't tell me that, they only give me a canned 
example (actually 3) without explanation of its full function.  Someday 
I'll read more.  Until then, basicConfig is opaque: I know only three 
incantations for using it, and not even any surety that they can be 
mixed, or that it can be called more than once in the same program.


>> Your comment about basicConfig setting the level on the root logger, but not
>> on the default handler making it useless is opaque to me,
> An early version of my code used basicConfig to create the stderr
> StreamHandler, but that turned out to be pointless since I wanted
> different levels on the logger and the handler.
>
>> but is there
>> perhaps room for another basic setup API that could get the setup code down
>> to a line or two in simple cases?
> For 3.2? No.
>
> For 3.3? Maybe.
>
>> Would that be a useful set of functionality to bundle?  And could it be
>> extended, when the user wants more power, or would it have to be replaced,
>> because it gets in the way of the user that wants more power?
> Logging already has powerful configuration mechanisms, especially
> following the addition of dictConfig
> (http://docs.python.org/dev/library/logging#logging-config-dictschema),
> so it really doesn't need anything else along those lines.

Sure; that is the replacement technique.  And maybe that is OK, if there 
are no useful intermediates between basicConfig (whatever it might be 
capable of) and dictConfig (or the other various config techniques that 
preceded it and still exist, if I understand correctly from has been 
said here on python-dev).

> At the simpler end, basicConfig already covers sending all messages to
> a single stream with a single format - it's only split-stream handling
> (such as stderr/stdout potentially being different endpoints) that it
> can't cope with.

OK, so I'd probably be fine with basicConfig, even though you weren't, 
once I understand it a little better.

> What may make more sense than yet another global config mechanism, is
> a module level "addHandler" helper function along the following lines:

And once I learn enough to understand the addHandler code, I will have 
read a lot of the logger documentation than I have to date, or it will 
have been restructured enough that re-reading the first 8%, or maybe 12% 
if the easy-to-understand part grows!  But I like what you said here:

> # Previous set up example is now only three lines long
> import sys, logging
> # Let root logger handlers see all messages
> logging.getLogger().setLevel(logging.NOTSET)
> # Send WARNING and above to stderr
> addHandler(stream=sys.stderr, level=logging.WARNING)
> # Send INFO to stdout
> addHandler(stream=sys.stdout, level=logging.INFO, max_level=logging.INFO)
>
> logging.info("Hello world!")
> logging.warn("Hello world!")

And this subset usage of addHandler is reasonably understandable for 
beginners, although clearly the complexity of all the parameters must 
add power that is not needed in the simple case, that you feel could be 
useful in the complex case.  And I would change the third line from 
level=logging.INFO to level=logging.DEBUG to see lots more on stdout, 
correct?

Looking back at the logger's simple examples section, I see that 
getLogger is used before being defined, is not explained, nor is its 
parameter explained.  So it is not clear how to go from the first simple 
example to the second simple example.  I wonder if that is where I 
started getting a bit of uncertainty about the ability to quickly learn 
what was going on... the rotating log example uses several APIs that 
haven't been used before, without explaining them at all.  Can that 
example actually be added to the previous example?  What is the magic 
'MyLogger'?  Where has basicConfig() gone in the second example?

I'm not expecting answers to the above questions here, but I'd love to 
see the documentation improved to the point that I don't have them any 
more :)

Glenn
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20101211/05c86a1b/attachment.html>

From v+python at g.nevcal.com  Sat Dec 11 10:40:29 2010
From: v+python at g.nevcal.com (Glenn Linderman)
Date: Sat, 11 Dec 2010 01:40:29 -0800
Subject: [Python-Dev] Using logging in the stdlib and its unit tests
In-Reply-To: <loom.20101211T095653-346@post.gmane.org>
References: <loom.20101207T212306-892@post.gmane.org>	<20101207215053.025ecf52@pitrou.net>	<loom.20101208T002100-91@post.gmane.org>	<20101208010913.363c6105@pitrou.net>	<AANLkTim4ZPoch18+-qkunuYm-zcyh-vbhMOv9YVVEqo9@mail.gmail.com>	<loom.20101208T022102-46@post.gmane.org>	<AANLkTikFNUPA5Cvbz8cOAfznSOEe906O8zOfOC=2Cj+H@mail.gmail.com>	<loom.20101210T085332-877@post.gmane.org>	<4D01E55B.2050507@g.nevcal.com>
	<3231.1292004978@parc.com>	<loom.20101210T202041-859@post.gmane.org>	<AANLkTi=PnWE5MbpyWKZHJU4Abk-OeVfKEJYpGX1E6g3=@mail.gmail.com>	<20101210214919.4ac63d27@pitrou.net>	<loom.20101210T224910-511@post.gmane.org>	<iduoeb$q36$1@dough.gmane.org>	<AANLkTi=Q8mZb0oDe5xz7Evb0y8ohgkWh=X7E1OgCrUka@mail.gmail.com>	<loom.20101211T064001-431@post.gmane.org>	<AANLkTinvjTwBwzoNKK+0u8Z74+2cH_a5Qt2vRvupJF1M@mail.gmail.com>
	<loom.20101211T095653-346@post.gmane.org>
Message-ID: <4D03470D.4090407@g.nevcal.com>

On 12/11/2010 1:28 AM, Vinay Sajip wrote:
> Nick Coghlan<ncoghlan<at>  gmail.com>  writes:
>> The "lazy" stream handler might be useful to make public in some way.
>> For example, rather than hardcoding sys.stderr, it could take a
>> callable that it uses to retrieve the stream. That kind of change can
>> wait until 3.3 though - what is there now should be fine to let us get
>> concurrent.futures and the logging tests to play nicely together.
> Actually if we're to change things to print INFO to stdout and WARNING+ to
> stderr, I'd change the last resort handler class to be more than just defining
> a lazy stream property: perhaps something like [untested]
>
> class _DefaultHandler(StreamHandler):
>      def __init__(self, level=NOTSET):
>          """
>          Initialize the handler.
>          """
>          Handler.__init__(self, level)
>
>      def emit(self, record):
>          if record.levelno == INFO:
>              self.stream = sys.stdout
>          else:
>              self.stream = sys.stderr
>          super(_DefaultHandler, self).emit(record)

So I think Nick's configuration stuff is something that is supposed to 
be done by the application, not by the library, or am I missing that 
too?  And so if it is done by the application, then Nick's, or any other 
application writer's, preferences for where to send which messages can 
likely be accommodated.  But I think that by default anything that is 
not part of the application's defined output, should go to stderr... 
even INFO messages.  I have no problem with Nick wanting them on 
stdout,  but I would have a problem with libraries (or the logger of 
last resort) sending them there by default.


>> As far as whether anything else in the logging defaults needs to
>> change, I don't think so. I'm now happy that if an application needs
> I agree it can be refined later, but if we agreed that INFO goes to stdout,
> then something like the above could be put in place for 3.2 (perhaps 3.2b2).

I'd find that hard to agree to, but I probably don't count.  I like 
Nick's statement.  Libraries spewing messages to stderr is fine, spewing 
them to stdout is not, by default.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20101211/4b5c6786/attachment.html>

From v+python at g.nevcal.com  Sat Dec 11 10:44:33 2010
From: v+python at g.nevcal.com (Glenn Linderman)
Date: Sat, 11 Dec 2010 01:44:33 -0800
Subject: [Python-Dev] Using logging in the stdlib and its unit tests
In-Reply-To: <AANLkTi=DDZHt0Ai4Pisi-mx=U5FUwxt6XL7S9f1r1xxX@mail.gmail.com>
References: <loom.20101207T212306-892@post.gmane.org>	<20101207215053.025ecf52@pitrou.net>	<loom.20101208T002100-91@post.gmane.org>	<20101208010913.363c6105@pitrou.net>	<AANLkTim4ZPoch18+-qkunuYm-zcyh-vbhMOv9YVVEqo9@mail.gmail.com>	<loom.20101208T022102-46@post.gmane.org>	<AANLkTikFNUPA5Cvbz8cOAfznSOEe906O8zOfOC=2Cj+H@mail.gmail.com>	<loom.20101210T085332-877@post.gmane.org>	<4D01E55B.2050507@g.nevcal.com>
	<3231.1292004978@parc.com>	<loom.20101210T202041-859@post.gmane.org>	<AANLkTi=PnWE5MbpyWKZHJU4Abk-OeVfKEJYpGX1E6g3=@mail.gmail.com>	<20101210214919.4ac63d27@pitrou.net>	<loom.20101210T224910-511@post.gmane.org>	<iduoeb$q36$1@dough.gmane.org>	<AANLkTi=Q8mZb0oDe5xz7Evb0y8ohgkWh=X7E1OgCrUka@mail.gmail.com>
	<AANLkTi=DDZHt0Ai4Pisi-mx=U5FUwxt6XL7S9f1r1xxX@mail.gmail.com>
Message-ID: <4D034801.5020308@g.nevcal.com>

On 12/11/2010 1:07 AM, Nick Coghlan wrote:
> As Glenn mentioned later in the thread, the output of logging.info and
> logging.debug messages is*distinct*  from an application's normal
> operational output that is emitted on stdout. So making it easy to
> emit such messages on stderr is the right thing to do - it's OK that
> it requires a bit of additional set up to get them onto stdout instead
> (since they really don't belong there most of the time).
>
> I know my own rule of thumb is going to be updated along the lines of
> "Am I writing to sys.stderr, or guarding output with a verbosity flag?
> Then perhaps I should be using the logging module rather than print
> statements".

I'm in agreement with all this.  Thanks for the correction.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20101211/5e133ae7/attachment.html>

From ncoghlan at gmail.com  Sat Dec 11 11:51:53 2010
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sat, 11 Dec 2010 20:51:53 +1000
Subject: [Python-Dev] Using logging in the stdlib and its unit tests
In-Reply-To: <loom.20101211T095653-346@post.gmane.org>
References: <loom.20101207T212306-892@post.gmane.org>
	<20101207215053.025ecf52@pitrou.net>
	<loom.20101208T002100-91@post.gmane.org>
	<20101208010913.363c6105@pitrou.net>
	<AANLkTim4ZPoch18+-qkunuYm-zcyh-vbhMOv9YVVEqo9@mail.gmail.com>
	<loom.20101208T022102-46@post.gmane.org>
	<AANLkTikFNUPA5Cvbz8cOAfznSOEe906O8zOfOC=2Cj+H@mail.gmail.com>
	<loom.20101210T085332-877@post.gmane.org>
	<4D01E55B.2050507@g.nevcal.com> <3231.1292004978@parc.com>
	<loom.20101210T202041-859@post.gmane.org>
	<AANLkTi=PnWE5MbpyWKZHJU4Abk-OeVfKEJYpGX1E6g3=@mail.gmail.com>
	<20101210214919.4ac63d27@pitrou.net>
	<loom.20101210T224910-511@post.gmane.org>
	<iduoeb$q36$1@dough.gmane.org>
	<AANLkTi=Q8mZb0oDe5xz7Evb0y8ohgkWh=X7E1OgCrUka@mail.gmail.com>
	<loom.20101211T064001-431@post.gmane.org>
	<AANLkTinvjTwBwzoNKK+0u8Z74+2cH_a5Qt2vRvupJF1M@mail.gmail.com>
	<loom.20101211T095653-346@post.gmane.org>
Message-ID: <AANLkTikOfM93gscpb3tCC9VhZDuyjPRb4b7RAOnheiXj@mail.gmail.com>

On Sat, Dec 11, 2010 at 7:28 PM, Vinay Sajip <vinay_sajip at yahoo.co.uk> wrote:
> Nick Coghlan <ncoghlan <at> gmail.com> writes:
> Actually if we're to change things to print INFO to stdout and WARNING+ to
> stderr,

I suspect our messages crossed paths in mid-stream, but I actually
realised I was wrong on that front - the behaviour already provided by
logging.basicConfig is the better way to go. The missing piece in the
story of "how do I create X kind of output" is plain old vanilla print
statements and similar direct writes to sys.stdout. The logging module
is a tool to track what is happening in a program, not a tool for
providing a console based UI.

Cheers,
Nick.

-- 
Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia

From contactprashantat at gmail.com  Sat Dec 11 11:36:46 2010
From: contactprashantat at gmail.com (Prashant Kumar)
Date: Sat, 11 Dec 2010 16:06:46 +0530
Subject: [Python-Dev] porting python.org
Message-ID: <AANLkTikz6ZwDiv1qeN2+Bvi6NiZ6P-ph2EeYjtPkqK78@mail.gmail.com>

I was wondering if we could contribute in porting of python.org website over
to python3k. If we can, what are the steps to be taken and other necessary
requirements?

regards,
prashant
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20101211/8321e7fc/attachment.html>

From vinay_sajip at yahoo.co.uk  Sat Dec 11 12:52:27 2010
From: vinay_sajip at yahoo.co.uk (Vinay Sajip)
Date: Sat, 11 Dec 2010 11:52:27 +0000 (UTC)
Subject: [Python-Dev] Using logging in the stdlib and its unit tests
References: <loom.20101207T212306-892@post.gmane.org>	<20101207215053.025ecf52@pitrou.net>	<loom.20101208T002100-91@post.gmane.org>	<20101208010913.363c6105@pitrou.net>	<AANLkTim4ZPoch18+-qkunuYm-zcyh-vbhMOv9YVVEqo9@mail.gmail.com>	<loom.20101208T022102-46@post.gmane.org>	<AANLkTikFNUPA5Cvbz8cOAfznSOEe906O8zOfOC=2Cj+H@mail.gmail.com>	<loom.20101210T085332-877@post.gmane.org>	<4D01E55B.2050507@g.nevcal.com>	<3231.1292004978@parc.com>	<loom.20101210T202041-859@post.gmane.org>	<AANLkTi=PnWE5MbpyWKZHJU4Abk-OeVfKEJYpGX1E6g3=@mail.gmail.com>	<20101210214919.4ac63d27@pitrou.net>	<loom.20101210T224910-511@post.gmane.org>	<iduoeb$q36$1@dough.gmane.org>	<AANLkTi=Q8mZb0oDe5xz7Evb0y8ohgkWh=X7E1OgCrUka@mail.gmail.com>	<4D031972.5090105@g.nevcal.com>
	<AANLkTimpq=hpV+LCcQLK7C4O7spbBadjBAFr3Jkh98bC@mail.gmail.com>
	<4D0344D2.3070006@g.nevcal.com>
Message-ID: <loom.20101211T123500-62@post.gmane.org>

Glenn Linderman <v+python <at> g.nevcal.com> writes:


>     the logger.? Does basicConfig give that control?? I don't know, the
>     first 8% of the logger documentation don't tell me that, they only
>     give me a canned example (actually 3) without explanation of its
>     full function.? Someday I'll read more.? Until then, basicConfig is
>     opaque: I know only three incantations for using it, and not even
>     any surety that they can be mixed, or that it can be called more
>     than once in the same program.

You've made some helpful observations about improving the logging documentation.
I'm grateful to you for those and would like to incorporate your feedback. But I
feel you're making somewhat heavy weather about basicConfig() and the "8%". It's
just 20 lines of code and not especially complicated code at that. It would
probably take you less time to look at that code than to post again about how
little information there is in the top 8% of the logging documentation, I think
you've got your point across. Here are a couple of hopefully useful links about
basicConfig():

>From the 2.7.1 docs - documentation gleaned from the docstring:

http://docs.python.org/library/logging.html#logging.basicConfig

>From the source code for 2.7.1, docstring followed by the code:

http://svn.python.org/view/python/branches/release27-maint/Lib/logging/__init__.py?revision=87157&view=markup#basicConfig

I will try to incorporate more basic examples at the top of the documentation,
but surely you don't want me to add more verbiage about basicConfig when your
overall feeling is that there's too much documentation?

Regards,

Vinay Sajip



From vinay_sajip at yahoo.co.uk  Sat Dec 11 13:06:40 2010
From: vinay_sajip at yahoo.co.uk (Vinay Sajip)
Date: Sat, 11 Dec 2010 12:06:40 +0000 (UTC)
Subject: [Python-Dev] Using logging in the stdlib and its unit tests
References: <loom.20101207T212306-892@post.gmane.org>
	<20101207215053.025ecf52@pitrou.net>
	<loom.20101208T002100-91@post.gmane.org>
	<20101208010913.363c6105@pitrou.net>
	<AANLkTim4ZPoch18+-qkunuYm-zcyh-vbhMOv9YVVEqo9@mail.gmail.com>
	<loom.20101208T022102-46@post.gmane.org>
	<AANLkTikFNUPA5Cvbz8cOAfznSOEe906O8zOfOC=2Cj+H@mail.gmail.com>
	<loom.20101210T085332-877@post.gmane.org>
	<4D01E55B.2050507@g.nevcal.com> <3231.1292004978@parc.com>
	<loom.20101210T202041-859@post.gmane.org>
	<AANLkTi=PnWE5MbpyWKZHJU4Abk-OeVfKEJYpGX1E6g3=@mail.gmail.com>
	<20101210214919.4ac63d27@pitrou.net>
	<loom.20101210T224910-511@post.gmane.org>
	<iduoeb$q36$1@dough.gmane.org>
	<AANLkTi=Q8mZb0oDe5xz7Evb0y8ohgkWh=X7E1OgCrUka@mail.gmail.com>
	<loom.20101211T064001-431@post.gmane.org>
	<AANLkTinvjTwBwzoNKK+0u8Z74+2cH_a5Qt2vRvupJF1M@mail.gmail.com>
	<loom.20101211T095653-346@post.gmane.org>
	<AANLkTikOfM93gscpb3tCC9VhZDuyjPRb4b7RAOnheiXj@mail.gmail.com>
Message-ID: <loom.20101211T125341-431@post.gmane.org>

Nick Coghlan <ncoghlan <at> gmail.com> writes:

> the logging module is a tool to track what is happening in a program, not a
> tool for providing a console based UI.

That was certainly the thinking behind how it worked before my recent changes,
but I completely accept that it wasn't helpful in the concurrent.futures
scenario, and agree with the comments you made around this in your earlier
posts. However, what Antoine has said on this thread (and others have concurred)
leads me to believe that he wants to use it not purely as a tracking tool (as
I've termed it, "an adjunct to doing real work") but also as a tool to provide
program output (e.g. doing some of the program's real work, say by calling
error() to print an error message).

Regards,

Vinay Sajip


From solipsis at pitrou.net  Sat Dec 11 15:17:03 2010
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Sat, 11 Dec 2010 15:17:03 +0100
Subject: [Python-Dev] API for the new sysconfig module
References: <B710BA0A-7097-4070-9DA4-E33149620833@gmail.com>
	<20101210215902.CDB5B1AB60E@kimball.webabinitio.net>
	<iduohr$q36$2@dough.gmane.org>
	<AANLkTikWzL1R5cpTpOFE5dzrZ=rRy0CEWJHYJXhYc6ZV@mail.gmail.com>
Message-ID: <20101211151703.09c8675e@pitrou.net>

On Sat, 11 Dec 2010 12:55:25 +1000
Nick Coghlan <ncoghlan at gmail.com> wrote:

> On Sat, Dec 11, 2010 at 12:44 PM, Terry Reedy <tjreedy at udel.edu> wrote:
> > On 12/10/2010 4:59 PM, R. David Murray wrote:
> >
> >> Like ?ric, I'm not sure what the implications of the existing module
> >> having been released in 2.7 and 3.2 beta are in terms of making such an
> >> API change.
> >
> > I am with Raymond on this: the purpose of betas is so we can test *and* make
> > changes. No one should base production software on a beta release.
> 
> It's also the key difference between betas and release candidates.

Seems that both you and Terry have missed the part where the sysconfig
API is already part of 2.7, though.

Regards

Antoine.



From tnagyemail-mail at yahoo.fr  Sat Dec 11 15:44:35 2010
From: tnagyemail-mail at yahoo.fr (Thomas Nagy)
Date: Sat, 11 Dec 2010 14:44:35 +0000 (GMT)
Subject: [Python-Dev] futures API
In-Reply-To: <C13B1791-1281-4DFB-B144-350AE7A12FF8@sweetapp.com>
Message-ID: <608275.24121.qm@web27507.mail.ukl.yahoo.com>

--- El vie, 10/12/10, Brian Quinlan escribi?:
> On Dec 10, 2010, at 10:51 AM, Thomas Nagy wrote:
> > --- El vie, 10/12/10, Brian Quinlan escribi?:
> >> On Dec 10, 2010, at 5:36 AM, Thomas Nagy wrote:
> >>> I have a process running for a long time, and
> which
> >> may use futures of different max_workers count. I
> think it
> >> is not too far-fetched to create a new futures
> object each
> >> time. Yet, the execution becomes slower after each
> call, for
> >> example with http://freehackers.org/~tnagy/futures_test.py:
> >>>
> >>> """
> >>> import concurrent.futures
> >>> from queue import Queue
> >>> import datetime
> >>>
> >>> class counter(object):
> >>>? ? ? def __init__(self, fut):
> >>>? ? ? ? ? self.fut =
> fut
> >>>
> >>>? ? ? def run(self):
> >>>? ? ? ? ? def
> >> look_busy(num, obj):
> >>>
> >>? ? tot = 0
> >>>
> >>? ? for x in range(num):
> >>>
> >>? ? tot += x
> >>>
> >>? ? obj.out_q.put(tot)
> >>>
> >>>? ? ? ? ? start =
> >> datetime.datetime.utcnow()
> >>>? ? ? ? ? self.count =
> 0
> >>>? ? ? ? ? self.out_q
> =
> >> Queue(0)
> >>>? ? ? ? ? for x in
> >> range(1000):
> >>>
> >>? ? self.count += 1
> >>>
> >>? ? self.fut.submit(look_busy,
> self.count,
> >> self)
> >>>
> >>>? ? ? ? ? while
> >> self.count:
> >>>
> >>? ? self.count -= 1
> >>>
> >>? ? self.out_q.get()
> >>>
> >>>? ? ? ? ? delta =
> >> datetime.datetime.utcnow() - start
> >>>
> >>? ? print(delta.total_seconds())
> >>>
> >>> fut =
> >>
> concurrent.futures.ThreadPoolExecutor(max_workers=20)
> >>> for x in range(100):
> >>>? ? ? # comment the following
> line
> >>>? ? ? fut =
> >>
> concurrent.futures.ThreadPoolExecutor(max_workers=20)
> >>>? ? ? c = counter(fut)
> >>>? ? ? c.run()
> >>> """
> >>>
> >>> The runtime grows after each step:
> >>> 0.216451
> >>> 0.225186
> >>> 0.223725
> >>> 0.222274
> >>> 0.230964
> >>> 0.240531
> >>> 0.24137
> >>> 0.252393
> >>> 0.249948
> >>> 0.257153
> >>> ...
> >>>
> >>> Is there a mistake in this piece of code?
> >>
> >> There is no mistake that I can see but I suspect
> that the
> >> circular references that you are building are
> causing the
> >> ThreadPoolExecutor to take a long time to be
> collected. Try
> >> adding:
> >>
> >>? ???c = counter(fut)
> >>? ???c.run()
> >> +? ? fut.shutdown()
> >>
> >> Even if that fixes your problem, I still don't
> fully
> >> understand this because I would expect the runtime
> to fall
> >> after a while as ThreadPoolExecutors are
> collected.
> >
> > The shutdown call is indeed a good fix :-) Here is the
> time response? 
> > of the calls to counter() when shutdown is not
> called:
> > http://www.freehackers.org/~tnagy/runtime_futures.png
> 
> FWIW, I think that you are confusion the term "future"
> with? 
> "executor". A future represents a single work item. An
> executor? 
> creates futures and schedules their underlying work.

Ah yes, sorry. I have also realized that the executor is not the killer feature I was expecting, it can only replace a little part of the code I have: controlling the exceptions and the workflow is the most complicated part.

I have also observed a minor performance degradation with the executor replacement (3 seconds for 5000 work items). The amount of work items processed by unit of time does not seem to be a straight line: http://www.freehackers.org/~tnagy/runtime_futures_2.png . Out of curiosity, what is the "_thread_references" for?

The source file for the example is in:
http://www.freehackers.org/~tnagy/futures_test3.py

The diagram was created by:
http://www.freehackers.org/~tnagy/futures_test3.plot

Thomas



      

From scott+python-dev at scottdial.com  Sat Dec 11 17:17:30 2010
From: scott+python-dev at scottdial.com (Scott Dial)
Date: Sat, 11 Dec 2010 11:17:30 -0500
Subject: [Python-Dev] futures API
In-Reply-To: <608275.24121.qm@web27507.mail.ukl.yahoo.com>
References: <608275.24121.qm@web27507.mail.ukl.yahoo.com>
Message-ID: <4D03A41A.5090302@scottdial.com>

On 12/11/2010 9:44 AM, Thomas Nagy wrote:
> The amount of work items processed by unit of time does not seem to be a straight line: http://www.freehackers.org/~tnagy/runtime_futures_2.png . Out of curiosity, what is the "_thread_references" for?
> 
> The source file for the example is in:
> http://www.freehackers.org/~tnagy/futures_test3.py
> 
> The diagram was created by:
> http://www.freehackers.org/~tnagy/futures_test3.plot

You're test code does 50,000,000 of list appends. I suspect your
benchmark is telling you more about the behavior of large lists than the
overhead of the futures module. You should retry that experiment with
the list pre-allocated. Beyond that, the curve in that line is not
exactly a large amount of variance from a straight line.

-- 
Scott Dial
scott at scottdial.com
scodial at cs.indiana.edu

From ncoghlan at gmail.com  Sat Dec 11 18:18:43 2010
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sun, 12 Dec 2010 03:18:43 +1000
Subject: [Python-Dev] Using logging in the stdlib and its unit tests
In-Reply-To: <loom.20101211T125341-431@post.gmane.org>
References: <loom.20101207T212306-892@post.gmane.org>
	<20101207215053.025ecf52@pitrou.net>
	<loom.20101208T002100-91@post.gmane.org>
	<20101208010913.363c6105@pitrou.net>
	<AANLkTim4ZPoch18+-qkunuYm-zcyh-vbhMOv9YVVEqo9@mail.gmail.com>
	<loom.20101208T022102-46@post.gmane.org>
	<AANLkTikFNUPA5Cvbz8cOAfznSOEe906O8zOfOC=2Cj+H@mail.gmail.com>
	<loom.20101210T085332-877@post.gmane.org>
	<4D01E55B.2050507@g.nevcal.com> <3231.1292004978@parc.com>
	<loom.20101210T202041-859@post.gmane.org>
	<AANLkTi=PnWE5MbpyWKZHJU4Abk-OeVfKEJYpGX1E6g3=@mail.gmail.com>
	<20101210214919.4ac63d27@pitrou.net>
	<loom.20101210T224910-511@post.gmane.org>
	<iduoeb$q36$1@dough.gmane.org>
	<AANLkTi=Q8mZb0oDe5xz7Evb0y8ohgkWh=X7E1OgCrUka@mail.gmail.com>
	<loom.20101211T064001-431@post.gmane.org>
	<AANLkTinvjTwBwzoNKK+0u8Z74+2cH_a5Qt2vRvupJF1M@mail.gmail.com>
	<loom.20101211T095653-346@post.gmane.org>
	<AANLkTikOfM93gscpb3tCC9VhZDuyjPRb4b7RAOnheiXj@mail.gmail.com>
	<loom.20101211T125341-431@post.gmane.org>
Message-ID: <AANLkTimK--Vb2+ztEXsOtDMQsV3r7G_m=0KqFd4zFYGQ@mail.gmail.com>

On Sat, Dec 11, 2010 at 10:06 PM, Vinay Sajip <vinay_sajip at yahoo.co.uk> wrote:
> Nick Coghlan <ncoghlan <at> gmail.com> writes:
>
>> the logging module is a tool to track what is happening in a program, not a
>> tool for providing a console based UI.
>
> That was certainly the thinking behind how it worked before my recent changes,
> but I completely accept that it wasn't helpful in the concurrent.futures
> scenario, and agree with the comments you made around this in your earlier
> posts. However, what Antoine has said on this thread (and others have concurred)
> leads me to believe that he wants to use it not purely as a tracking tool (as
> I've termed it, "an adjunct to doing real work") but also as a tool to provide
> program output (e.g. doing some of the program's real work, say by calling
> error() to print an error message).

I think that's actually just a case where the lines get a little
blurry. From logging's point of view, it is still being used to say
"Hey, this particular event happened during program execution". We're
just adjusting our display policy to say that there are certain kinds
of events that should be made visible to the user by default, but
piping them into the logging machinery is preferable to writing them
directly to sys,stderr.

By contrast, most information that is written to sys.stdout isn't
really reporting events that happened in the program - it is actual
meaningful data that is related to the task the program accomplishes.

Cheers,
Nick.

-- 
Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia

From g.brandl at gmx.net  Sat Dec 11 18:19:59 2010
From: g.brandl at gmx.net (Georg Brandl)
Date: Sat, 11 Dec 2010 18:19:59 +0100
Subject: [Python-Dev] porting python.org
In-Reply-To: <AANLkTikz6ZwDiv1qeN2+Bvi6NiZ6P-ph2EeYjtPkqK78@mail.gmail.com>
References: <AANLkTikz6ZwDiv1qeN2+Bvi6NiZ6P-ph2EeYjtPkqK78@mail.gmail.com>
Message-ID: <ie0bup$a70$1@dough.gmane.org>

Am 11.12.2010 11:36, schrieb Prashant Kumar:
> I was wondering if we could contribute in porting of python.org
> <http://python.org> website over to python3k. If we can, what are the steps to
> be taken and other necessary requirements?

The SVN repository for python.org is currently not publicly available, but you
can get access asking on the pydotorg-www at python.org mailing list.

An excerpt from the README:

External packages required:

  Mako (http://www.makotemplates.org/)
  [Note that Mako requires setuptools to install out-of-the-box.  You can
   also just copy the lib/mako/ tree into your Python's site-packages
   directory if you prefer not to install setuptools.]

  PyYAML (http://pyyaml.org/wiki/PyYAML)

  Docutils (http://docutils.sf.net)

So the first step would be to determine whether Mako and PyYAML are ported
yet, and if not, help porting them.  Docutils is ported already.

cheers,
Georg


From ncoghlan at gmail.com  Sat Dec 11 18:21:43 2010
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sun, 12 Dec 2010 03:21:43 +1000
Subject: [Python-Dev] API for the new sysconfig module
In-Reply-To: <20101211151703.09c8675e@pitrou.net>
References: <B710BA0A-7097-4070-9DA4-E33149620833@gmail.com>
	<20101210215902.CDB5B1AB60E@kimball.webabinitio.net>
	<iduohr$q36$2@dough.gmane.org>
	<AANLkTikWzL1R5cpTpOFE5dzrZ=rRy0CEWJHYJXhYc6ZV@mail.gmail.com>
	<20101211151703.09c8675e@pitrou.net>
Message-ID: <AANLkTikNKxtpTj-36nS1r+h9c0YKDaU6skW+n0S=kGa-@mail.gmail.com>

On Sun, Dec 12, 2010 at 12:17 AM, Antoine Pitrou <solipsis at pitrou.net> wrote:
> On Sat, 11 Dec 2010 12:55:25 +1000
> Nick Coghlan <ncoghlan at gmail.com> wrote:
>
>> On Sat, Dec 11, 2010 at 12:44 PM, Terry Reedy <tjreedy at udel.edu> wrote:
>> > On 12/10/2010 4:59 PM, R. David Murray wrote:
>> >
>> >> Like ?ric, I'm not sure what the implications of the existing module
>> >> having been released in 2.7 and 3.2 beta are in terms of making such an
>> >> API change.
>> >
>> > I am with Raymond on this: the purpose of betas is so we can test *and* make
>> > changes. No one should base production software on a beta release.
>>
>> It's also the key difference between betas and release candidates.
>
> Seems that both you and Terry have missed the part where the sysconfig
> API is already part of 2.7, though.

I had missed that, yes*. I suspect Raymond may have missed it as well.

Cheers,
Nick.

*(I sometimes lose track of which changes were made in both branches
pre-2.7, which ones were mode post-2.7 release, and which ones went in
pre-2.7, but were 3.x only regardless)

-- 
Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia

From tseaver at palladion.com  Sat Dec 11 18:24:22 2010
From: tseaver at palladion.com (Tres Seaver)
Date: Sat, 11 Dec 2010 12:24:22 -0500
Subject: [Python-Dev] Using logging in the stdlib and its unit tests
In-Reply-To: <loom.20101211T095653-346@post.gmane.org>
References: <loom.20101207T212306-892@post.gmane.org>	<20101207215053.025ecf52@pitrou.net>	<loom.20101208T002100-91@post.gmane.org>	<20101208010913.363c6105@pitrou.net>	<AANLkTim4ZPoch18+-qkunuYm-zcyh-vbhMOv9YVVEqo9@mail.gmail.com>	<loom.20101208T022102-46@post.gmane.org>	<AANLkTikFNUPA5Cvbz8cOAfznSOEe906O8zOfOC=2Cj+H@mail.gmail.com>	<loom.20101210T085332-877@post.gmane.org>	<4D01E55B.2050507@g.nevcal.com>
	<3231.1292004978@parc.com>	<loom.20101210T202041-859@post.gmane.org>	<AANLkTi=PnWE5MbpyWKZHJU4Abk-OeVfKEJYpGX1E6g3=@mail.gmail.com>	<20101210214919.4ac63d27@pitrou.net>	<loom.20101210T224910-511@post.gmane.org>	<iduoeb$q36$1@dough.gmane.org>	<AANLkTi=Q8mZb0oDe5xz7Evb0y8ohgkWh=X7E1OgCrUka@mail.gmail.com>	<loom.20101211T064001-431@post.gmane.org>	<AANLkTinvjTwBwzoNKK+0u8Z74+2cH_a5Qt2vRvupJF1M@mail.gmail.com>
	<loom.20101211T095653-346@post.gmane.org>
Message-ID: <ie0cc6$d2f$1@dough.gmane.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 12/11/2010 04:28 AM, Vinay Sajip wrote:

> Actually if we're to change things to print INFO to stdout and WARNING+ to
> stderr,

ACK!  That would be an awful default -- stdout belongs to the
application, not to "meta" stuff like logging, which should only go to
stderr.  Much better to document the "how-to" for setting up something
like that than to make it a default.



Tres.
- -- 
===================================================================
Tres Seaver          +1 540-429-0999          tseaver at palladion.com
Palladion Software   "Excellence by Design"    http://palladion.com
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.10 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAk0Ds8YACgkQ+gerLs4ltQ6BBACfTWWcfl6000DX9SWCYvd5Tf7n
qIcAniTD6EHRhrXA+eVpgyrlqvv12apq
=oR03
-----END PGP SIGNATURE-----


From sandro.tosi at gmail.com  Sat Dec 11 18:37:14 2010
From: sandro.tosi at gmail.com (Sandro Tosi)
Date: Sat, 11 Dec 2010 18:37:14 +0100
Subject: [Python-Dev] porting python.org
In-Reply-To: <ie0bup$a70$1@dough.gmane.org>
References: <AANLkTikz6ZwDiv1qeN2+Bvi6NiZ6P-ph2EeYjtPkqK78@mail.gmail.com>
	<ie0bup$a70$1@dough.gmane.org>
Message-ID: <AANLkTin7=qBw8mq5MZX7SeKf_0xzcTYXk5mbOU6cTSfi@mail.gmail.com>

Hi,

On Sat, Dec 11, 2010 at 18:19, Georg Brandl <g.brandl at gmx.net> wrote:
> ?Mako (http://www.makotemplates.org/)
> ?[Note that Mako requires setuptools to install out-of-the-box. ?You can
> ? also just copy the lib/mako/ tree into your Python's site-packages
> ? directory if you prefer not to install setuptools.]

Mako is compatible with py3k (via 2to3) since version 0.3 (but later
versions has a better support). [1] explains how to have a py3k
installation of mako.

[1] http://www.makotemplates.org/trac/browser/README.py3k

> ?PyYAML (http://pyyaml.org/wiki/PyYAML)

"PyYAML also supports Python 3" from the homepage, the version
introducing py3k support is 3.08 .

Just my 2c,
-- 
Sandro Tosi (aka morph, morpheus, matrixhasu)
My website: http://matrixhasu.altervista.org/
Me at Debian: http://wiki.debian.org/SandroTosi

From ncoghlan at gmail.com  Sat Dec 11 18:51:06 2010
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sun, 12 Dec 2010 03:51:06 +1000
Subject: [Python-Dev] Using logging in the stdlib and its unit tests
In-Reply-To: <ie0cc6$d2f$1@dough.gmane.org>
References: <loom.20101207T212306-892@post.gmane.org>
	<20101207215053.025ecf52@pitrou.net>
	<loom.20101208T002100-91@post.gmane.org>
	<20101208010913.363c6105@pitrou.net>
	<AANLkTim4ZPoch18+-qkunuYm-zcyh-vbhMOv9YVVEqo9@mail.gmail.com>
	<loom.20101208T022102-46@post.gmane.org>
	<AANLkTikFNUPA5Cvbz8cOAfznSOEe906O8zOfOC=2Cj+H@mail.gmail.com>
	<loom.20101210T085332-877@post.gmane.org>
	<4D01E55B.2050507@g.nevcal.com> <3231.1292004978@parc.com>
	<loom.20101210T202041-859@post.gmane.org>
	<AANLkTi=PnWE5MbpyWKZHJU4Abk-OeVfKEJYpGX1E6g3=@mail.gmail.com>
	<20101210214919.4ac63d27@pitrou.net>
	<loom.20101210T224910-511@post.gmane.org>
	<iduoeb$q36$1@dough.gmane.org>
	<AANLkTi=Q8mZb0oDe5xz7Evb0y8ohgkWh=X7E1OgCrUka@mail.gmail.com>
	<loom.20101211T064001-431@post.gmane.org>
	<AANLkTinvjTwBwzoNKK+0u8Z74+2cH_a5Qt2vRvupJF1M@mail.gmail.com>
	<loom.20101211T095653-346@post.gmane.org>
	<ie0cc6$d2f$1@dough.gmane.org>
Message-ID: <AANLkTikhXYdM04UaSSeYBf5kSYkYrEex3doy813fxWHv@mail.gmail.com>

On Sun, Dec 12, 2010 at 3:24 AM, Tres Seaver <tseaver at palladion.com> wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> On 12/11/2010 04:28 AM, Vinay Sajip wrote:
>
>> Actually if we're to change things to print INFO to stdout and WARNING+ to
>> stderr,
>
> ACK! ?That would be an awful default -- stdout belongs to the
> application, not to "meta" stuff like logging, which should only go to
> stderr. ?Much better to document the "how-to" for setting up something
> like that than to make it a default.

Don't worry, that isn't going to happen. Vinay was just responding to
something he thought I was suggesting. I wasn't really, but I can
understand someone getting that impression (as I was using that
behaviour as my running example of "so just what *is* involved if an
applications wants to configure logging like that?").

Cheers,
Nick.

-- 
Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia

From v+python at g.nevcal.com  Sat Dec 11 20:17:05 2010
From: v+python at g.nevcal.com (Glenn Linderman)
Date: Sat, 11 Dec 2010 11:17:05 -0800
Subject: [Python-Dev] Using logging in the stdlib and its unit tests
In-Reply-To: <loom.20101211T123500-62@post.gmane.org>
References: <loom.20101207T212306-892@post.gmane.org>	<20101207215053.025ecf52@pitrou.net>	<loom.20101208T002100-91@post.gmane.org>	<20101208010913.363c6105@pitrou.net>	<AANLkTim4ZPoch18+-qkunuYm-zcyh-vbhMOv9YVVEqo9@mail.gmail.com>	<loom.20101208T022102-46@post.gmane.org>	<AANLkTikFNUPA5Cvbz8cOAfznSOEe906O8zOfOC=2Cj+H@mail.gmail.com>	<loom.20101210T085332-877@post.gmane.org>	<4D01E55B.2050507@g.nevcal.com>	<3231.1292004978@parc.com>	<loom.20101210T202041-859@post.gmane.org>	<AANLkTi=PnWE5MbpyWKZHJU4Abk-OeVfKEJYpGX1E6g3=@mail.gmail.com>	<20101210214919.4ac63d27@pitrou.net>	<loom.20101210T224910-511@post.gmane.org>	<iduoeb$q36$1@dough.gmane.org>	<AANLkTi=Q8mZb0oDe5xz7Evb0y8ohgkWh=X7E1OgCrUka@mail.gmail.com>	<4D031972.5090105@g.nevcal.com>	<AANLkTimpq=hpV+LCcQLK7C4O7spbBadjBAFr3Jkh98bC@mail.gmail.com>	<4D0344D2.3070006@g.nevcal.com>
	<loom.20101211T123500-62@post.gmane.org>
Message-ID: <4D03CE31.3040700@g.nevcal.com>

On 12/11/2010 3:52 AM, Vinay Sajip wrote:
> I will try to incorporate more basic examples at the top of the documentation,
> but surely you don't want me to add more verbiage about basicConfig when your
> overall feeling is that there's too much documentation?

I try not to post unless I feel there is a new detail; yes the 8% is 
probably repetitive, but at present, going further, at least 
sequentially, does feel like a large mountain to climb... suddenly 
becoming significantly steeper there.

I really can't say much about the 92% of the documentation,  because of 
the mountain.

Whether the existing documentation can be reordered to avoid the steep 
part, but rather provide a gentle upward path, I cannot say.  If it is 
properly organized reference material, then probably adding more about 
something at the beginning is appropriate, to give the foundation, 
justification, and assistance for climbing the slope.

My overall feeling is not necessarily that there is too much 
documentation (there is a lot, but maybe that is a good thing, I should 
read it some week) but that it suffers in not providing useful 
information for the beginner at the front.  I've listed what I've 
gleaned here and tried to point out the things that I still don't know, 
and tried to point out where things became less than obvious along the 
way.  I do think it was primarily because the intro material is too 
brief and too disconnected, that I quit reading at the 8% mark (or so).  
If the rest of the large document were to be similarly disconnected, it 
wouldn't be worth my time to read it... and if I had to read it all, 
just to understand enough to get started with simple logging, then it 
would be faster to write my own, which is the path I took.  I suffer 
with my own being primitive, but I can tweak it when I need to.  I still 
don't fully understand the nuances between handlers and filters, and 
likely won't until I read more of the documentation.

Jumping directly to the documentation (at the 28% mark, I'd have had to 
read a long way to find it) for basicConfig per your links helps, but 
sadly,  because the first paragraph is full of the terms "logger", 
"handler", and "formatter", which I have only rudimentary ideas about 
what they do, being a beginner, I almost gave up again.  But the 
description of the valid keywords for **kwargs was actually 
understandable, at the end of that documentation.  I also learned that 
calling basicConfig more than once in a program is probably useless, and 
it should be nearly the first logger API called, if it is used.  The 
first paragraph should be split, perhaps: "Does basic configuration for 
the logging system." is almost redundant with the name, but is OK if 
first.  The rest of that paragraph, and the next three, should probably 
be moved after the list of kwargs keys and descriptions.  Yes, the word 
handler is used a bit, but it is associated there with concepts like 
file names and stream handles, and that eases the uncertainty; together 
with the simple example at the front, and the verbage there, I can see 
that I specify a file name, and it makes the logger log to that file :)  
And the level parameter is fine.  I'm less sure about format and datefmt 
and why I would want to specify them or how, and am surprised to find 
that format is to be used by a "handler" not a "formatter", but the 
simple example shows me I can leave those out for starters.

Personally, I suffer in writing documentation; when I attempt to do so, 
and people read it, they usually tell me that after about the third 
read, they finally understand it, but sadly it takes them three reads.  
It is interesting to be looking at the logger from the other side... 
maybe I'll be able to improve the documentation I write, after analyzing 
why the logger documentation was hard for me to approach.


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20101211/84082181/attachment.html>

From brett at python.org  Sat Dec 11 20:21:41 2010
From: brett at python.org (Brett Cannon)
Date: Sat, 11 Dec 2010 11:21:41 -0800
Subject: [Python-Dev] Format factories (was Re: sWAPcASE Was:
 transform() and untransform() methods, and the codec registry)
In-Reply-To: <AANLkTi=rLuuLkFzrpt7REsm91-7xfBmFcMD__B5rQxo6@mail.gmail.com>
References: <AANLkTi=rLuuLkFzrpt7REsm91-7xfBmFcMD__B5rQxo6@mail.gmail.com>
Message-ID: <AANLkTi=4VNE+nkFzSRfdyDmuRQ9JaK7JLmQJ2ptymt=f@mail.gmail.com>

On Thu, Dec 9, 2010 at 16:26, Nick Coghlan <ncoghlan at gmail.com> wrote:
> On Fri, Dec 10, 2010 at 9:29 AM, Antoine Pitrou <solipsis at pitrou.net> wrote:
>> On Thu, 09 Dec 2010 18:10:38 -0500
>> Eric Smith <eric at trueblade.com> wrote:
>>> If we're looking to reduce the number of methods on str, I wouldn't mind
>>> seeing center() and zfill() also go away, since they're trivially
>>> replaced by format().
>>
>> Well, it's only trivial when you know format()'s wicked mini-language by
>> heart :/ ?center() is easy and obvious. zfill() is arguably a bit too
>> specialized.
>
> I've occasionally wondered if string formatting [1] and the struct
> module [2] would benefit from format building functions that made them
> easier to use without necessarily learning the cryptic mini-languages
> off by heart.
>
> For example, a "string.make_format_spec" function might have a signature like:
>
> def make_format_spec(fill=None, align=None, sign=None, width=None,
> precision=None, display_type='s', alternate=False, commas=False,
> numeric_case=None)
>
> "align" would accept not only the actual format symbols ('<', '>',
> '=', '^'), but also the corresponding names ('left', 'right',
> 'numeric', 'center').
>
> "numeric_case" would accept None, 'upper' or 'lower' (affecting the
> formatting of hex and floating point values). If the stated numeric
> case differs from that specified by the display type, raise a
> ValueError. For an unspecified numeric case, the affected display
> types would default to 'lower'.
>
> Similarly, "display_type" would accept long names in addition to the
> character codes:
>
> 's': 'str'
> 'b': 'binary'
> 'c': 'chr'
> 'd': 'int'
> 'o': 'octal'
> 'x', 'X': 'hex' (numeric case controls display of digits A-F)
> 'n': 'locale'
> 'e', 'E': 'exponent' (numeric case controls display of exponent as
> well as infinite and NaN values)
> 'f', 'F': 'float' (numeric case controls display of exponent as well
> as infinite and NaN values)
> 'g', 'G': 'general' (numeric case controls display of exponent as well
> as infinite and NaN values)
> '%': 'percent' (numeric case controls display of exponent as well as
> infinite and NaN values)
>
> There could also be a corresponding parse_format_spec that produced a
> named tuple with the appropriate details.

But is this worth it since once you write it you won't be changing it
again, suggesting that taking the time to look up the formatting rules
isn't that much harder and wouldn't burden us w/ writing such
functions and trying to come up with a good API. I suspect what would
be more helpful is either have a rather detailed docstring for
str.format or to at least put the URL to the docs which explains the
mini-language to make it easier to find.

-Brett

>
> Cheers,
> Nick.
>
> [1] http://docs.python.org/dev/library/string#format-specification-mini-language
> [2] http://docs.python.org/dev/library/struct#format-strings
>
> --
> Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/brett%40python.org
>

From brett at python.org  Sat Dec 11 20:32:55 2010
From: brett at python.org (Brett Cannon)
Date: Sat, 11 Dec 2010 11:32:55 -0800
Subject: [Python-Dev] Using logging in the stdlib and its unit tests
In-Reply-To: <loom.20101211T072018-194@post.gmane.org>
References: <loom.20101207T212306-892@post.gmane.org>
	<20101207215053.025ecf52@pitrou.net>
	<loom.20101208T002100-91@post.gmane.org>
	<20101208010913.363c6105@pitrou.net>
	<AANLkTim4ZPoch18+-qkunuYm-zcyh-vbhMOv9YVVEqo9@mail.gmail.com>
	<loom.20101208T022102-46@post.gmane.org>
	<AANLkTikFNUPA5Cvbz8cOAfznSOEe906O8zOfOC=2Cj+H@mail.gmail.com>
	<loom.20101210T085332-877@post.gmane.org>
	<4D01E55B.2050507@g.nevcal.com>
	<3231.1292004978@parc.com> <loom.20101210T202041-859@post.gmane.org>
	<AANLkTi=PnWE5MbpyWKZHJU4Abk-OeVfKEJYpGX1E6g3=@mail.gmail.com>
	<20101210214919.4ac63d27@pitrou.net>
	<loom.20101210T224910-511@post.gmane.org>
	<iduoeb$q36$1@dough.gmane.org>
	<AANLkTi=Q8mZb0oDe5xz7Evb0y8ohgkWh=X7E1OgCrUka@mail.gmail.com>
	<AANLkTikN_378ZH2kCW-WJ2wWDVZrYb2zrQ9N1QMRvkg0@mail.gmail.com>
	<loom.20101211T072018-194@post.gmane.org>
Message-ID: <AANLkTikbjCwVkjAdU6CqDsso7AhHeU4i0uK3v-VnTFVE@mail.gmail.com>

On Fri, Dec 10, 2010 at 22:21, Vinay Sajip <vinay_sajip at yahoo.co.uk> wrote:
> Nick Coghlan <ncoghlan <at> gmail.com> writes:
>
>
>> This could actually make a reasonably good basic for a "task oriented"
>> subsection of the logging documentation. Something like:
>>
>
> Good suggestion, I'll see what I can do.

Just wanted to +1 on some task-oriented (or at least simple intro)
docs going into the logging module.

The length of the docs is rather daunting I always have a hard time
getting started (the first example is for file logging and the second
one does log rotation when all I want is printing to stderr!). All I
want is an opener saying, "here are the functions to call to send
stuff to the log; the defaults for each go here and here (e.g.,
stderr, /dev/null), and if you want to change those settings, here is
how to e.g., get info() to go to stderr. Read on for more if you need
something fancier (e.g., writing to a file)". That and making it not
an error when I call logging.error() w/o setting an explicit logger
would go a long way to making the logging module much more accessible
IMO.

From jyasskin at gmail.com  Sat Dec 11 21:40:16 2010
From: jyasskin at gmail.com (Jeffrey Yasskin)
Date: Sat, 11 Dec 2010 12:40:16 -0800
Subject: [Python-Dev] Format factories (was Re: sWAPcASE Was:
 transform() and untransform() methods, and the codec registry)
In-Reply-To: <AANLkTi=4VNE+nkFzSRfdyDmuRQ9JaK7JLmQJ2ptymt=f@mail.gmail.com>
References: <AANLkTi=rLuuLkFzrpt7REsm91-7xfBmFcMD__B5rQxo6@mail.gmail.com>
	<AANLkTi=4VNE+nkFzSRfdyDmuRQ9JaK7JLmQJ2ptymt=f@mail.gmail.com>
Message-ID: <AANLkTinpZgQW8GyUNz47RvOpOhmfqP+kE1hm8qEeLb7X@mail.gmail.com>

On Sat, Dec 11, 2010 at 11:21 AM, Brett Cannon <brett at python.org> wrote:
> On Thu, Dec 9, 2010 at 16:26, Nick Coghlan <ncoghlan at gmail.com> wrote:
>> On Fri, Dec 10, 2010 at 9:29 AM, Antoine Pitrou <solipsis at pitrou.net> wrote:
>>> On Thu, 09 Dec 2010 18:10:38 -0500
>>> Eric Smith <eric at trueblade.com> wrote:
>>>> If we're looking to reduce the number of methods on str, I wouldn't mind
>>>> seeing center() and zfill() also go away, since they're trivially
>>>> replaced by format().
>>>
>>> Well, it's only trivial when you know format()'s wicked mini-language by
>>> heart :/ ?center() is easy and obvious. zfill() is arguably a bit too
>>> specialized.
>>
>> I've occasionally wondered if string formatting [1] and the struct
>> module [2] would benefit from format building functions that made them
>> easier to use without necessarily learning the cryptic mini-languages
>> off by heart.
>>
>> For example, a "string.make_format_spec" function might have a signature like:
>>
>> def make_format_spec(fill=None, align=None, sign=None, width=None,
>> precision=None, display_type='s', alternate=False, commas=False,
>> numeric_case=None)
>>
>> "align" would accept not only the actual format symbols ('<', '>',
>> '=', '^'), but also the corresponding names ('left', 'right',
>> 'numeric', 'center').
>>
>> "numeric_case" would accept None, 'upper' or 'lower' (affecting the
>> formatting of hex and floating point values). If the stated numeric
>> case differs from that specified by the display type, raise a
>> ValueError. For an unspecified numeric case, the affected display
>> types would default to 'lower'.
>>
>> Similarly, "display_type" would accept long names in addition to the
>> character codes:
>>
>> 's': 'str'
>> 'b': 'binary'
>> 'c': 'chr'
>> 'd': 'int'
>> 'o': 'octal'
>> 'x', 'X': 'hex' (numeric case controls display of digits A-F)
>> 'n': 'locale'
>> 'e', 'E': 'exponent' (numeric case controls display of exponent as
>> well as infinite and NaN values)
>> 'f', 'F': 'float' (numeric case controls display of exponent as well
>> as infinite and NaN values)
>> 'g', 'G': 'general' (numeric case controls display of exponent as well
>> as infinite and NaN values)
>> '%': 'percent' (numeric case controls display of exponent as well as
>> infinite and NaN values)
>>
>> There could also be a corresponding parse_format_spec that produced a
>> named tuple with the appropriate details.
>
> But is this worth it since once you write it you won't be changing it
> again,

Having a make_format_spec() would also help people trying to read the code.

> suggesting that taking the time to look up the formatting rules
> isn't that much harder and wouldn't burden us w/ writing such
> functions and trying to come up with a good API. I suspect what would
> be more helpful is either have a rather detailed docstring for
> str.format or to at least put the URL to the docs which explains the
> mini-language to make it easier to find.
>>
>> Cheers,
>> Nick.
>>
>> [1] http://docs.python.org/dev/library/string#format-specification-mini-language
>> [2] http://docs.python.org/dev/library/struct#format-strings
>>
>> --
>> Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia
>> _______________________________________________
>> Python-Dev mailing list
>> Python-Dev at python.org
>> http://mail.python.org/mailman/listinfo/python-dev
>> Unsubscribe: http://mail.python.org/mailman/options/python-dev/brett%40python.org
>>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/jyasskin%40gmail.com
>

From brian at sweetapp.com  Sat Dec 11 21:53:15 2010
From: brian at sweetapp.com (Brian Quinlan)
Date: Sat, 11 Dec 2010 12:53:15 -0800
Subject: [Python-Dev] futures API
In-Reply-To: <608275.24121.qm@web27507.mail.ukl.yahoo.com>
References: <608275.24121.qm@web27507.mail.ukl.yahoo.com>
Message-ID: <B2438CD8-1EC2-4123-8E2A-B36829241D6C@sweetapp.com>


On Dec 11, 2010, at 6:44 AM, Thomas Nagy wrote:

> --- El vie, 10/12/10, Brian Quinlan escribi?:
>> On Dec 10, 2010, at 10:51 AM, Thomas Nagy wrote:
>>> --- El vie, 10/12/10, Brian Quinlan escribi?:
>>>> On Dec 10, 2010, at 5:36 AM, Thomas Nagy wrote:
>>>>> I have a process running for a long time, and
>> which
>>>> may use futures of different max_workers count. I
>> think it
>>>> is not too far-fetched to create a new futures
>> object each
>>>> time. Yet, the execution becomes slower after each
>> call, for
>>>> example with http://freehackers.org/~tnagy/futures_test.py:
>>>>>
>>>>> """
>>>>> import concurrent.futures
>>>>> from queue import Queue
>>>>> import datetime
>>>>>
>>>>> class counter(object):
>>>>>       def __init__(self, fut):
>>>>>           self.fut =
>> fut
>>>>>
>>>>>       def run(self):
>>>>>           def
>>>> look_busy(num, obj):
>>>>>
>>>>     tot = 0
>>>>>
>>>>     for x in range(num):
>>>>>
>>>>     tot += x
>>>>>
>>>>     obj.out_q.put(tot)
>>>>>
>>>>>           start =
>>>> datetime.datetime.utcnow()
>>>>>           self.count =
>> 0
>>>>>           self.out_q
>> =
>>>> Queue(0)
>>>>>           for x in
>>>> range(1000):
>>>>>
>>>>     self.count += 1
>>>>>
>>>>     self.fut.submit(look_busy,
>> self.count,
>>>> self)
>>>>>
>>>>>           while
>>>> self.count:
>>>>>
>>>>     self.count -= 1
>>>>>
>>>>     self.out_q.get()
>>>>>
>>>>>           delta =
>>>> datetime.datetime.utcnow() - start
>>>>>
>>>>     print(delta.total_seconds())
>>>>>
>>>>> fut =
>>>>
>> concurrent.futures.ThreadPoolExecutor(max_workers=20)
>>>>> for x in range(100):
>>>>>       # comment the following
>> line
>>>>>       fut =
>>>>
>> concurrent.futures.ThreadPoolExecutor(max_workers=20)
>>>>>       c = counter(fut)
>>>>>       c.run()
>>>>> """
>>>>>
>>>>> The runtime grows after each step:
>>>>> 0.216451
>>>>> 0.225186
>>>>> 0.223725
>>>>> 0.222274
>>>>> 0.230964
>>>>> 0.240531
>>>>> 0.24137
>>>>> 0.252393
>>>>> 0.249948
>>>>> 0.257153
>>>>> ...
>>>>>
>>>>> Is there a mistake in this piece of code?
>>>>
>>>> There is no mistake that I can see but I suspect
>> that the
>>>> circular references that you are building are
>> causing the
>>>> ThreadPoolExecutor to take a long time to be
>> collected. Try
>>>> adding:
>>>>
>>>>      c = counter(fut)
>>>>      c.run()
>>>> +    fut.shutdown()
>>>>
>>>> Even if that fixes your problem, I still don't
>> fully
>>>> understand this because I would expect the runtime
>> to fall
>>>> after a while as ThreadPoolExecutors are
>> collected.
>>>
>>> The shutdown call is indeed a good fix :-) Here is the
>> time response
>>> of the calls to counter() when shutdown is not
>> called:
>>> http://www.freehackers.org/~tnagy/runtime_futures.png
>>
>> FWIW, I think that you are confusion the term "future"
>> with
>> "executor". A future represents a single work item. An
>> executor
>> creates futures and schedules their underlying work.
>
> Ah yes, sorry. I have also realized that the executor is not the  
> killer feature I was expecting, it can only replace a little part of  
> the code I have: controlling the exceptions and the workflow is the  
> most complicated part.
>
> I have also observed a minor performance degradation with the  
> executor replacement (3 seconds for 5000 work items). The amount of  
> work items processed by unit of time does not seem to be a straight  
> line: http://www.freehackers.org/~tnagy/runtime_futures_2.png .

That looks pretty linear to me.

> Out of curiosity, what is the "_thread_references" for?

There is a big comment above it in the code:

# Workers are created as daemon threads. This is done to allow the  
interpreter
# to exit when there are still idle threads in a ThreadPoolExecutor's  
thread
# pool (i.e. shutdown() was not called). However, allowing workers to  
die with
# the interpreter has two undesirable properties:
#   - The workers would still be running during interpretor shutdown,
#     meaning that they would fail in unpredictable ways.
#   - The workers could be killed while evaluating a work item, which  
could
#     be bad if the callable being evaluated has external side-effects  
e.g.
#     writing to a file.
#
# To work around this problem, an exit handler is installed which  
tells the
# workers to exit when their work queues are empty and then waits  
until the
# threads finish.

_thread_references = set()
_shutdown = False

def _python_exit():
     global _shutdown
     _shutdown = True
     for thread_reference in _thread_references:
         thread = thread_reference()
         if thread is not None:
             thread.join()

Is it still unclear why it is there? Maybe you could propose some  
additional documentation.

Cheers,
Brian

> The source file for the example is in:
> http://www.freehackers.org/~tnagy/futures_test3.py
>
> The diagram was created by:
> http://www.freehackers.org/~tnagy/futures_test3.plot
>
> Thomas
>
>
>
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/brian%40sweetapp.com


From ben+python at benfinney.id.au  Sat Dec 11 22:15:11 2010
From: ben+python at benfinney.id.au (Ben Finney)
Date: Sun, 12 Dec 2010 08:15:11 +1100
Subject: [Python-Dev] Status of PEP 3143?
References: <AANLkTin3_LSU2CsRNH_ZYvL9LB7AMjBQR=1ji1QFWgww@mail.gmail.com>
	<AANLkTinmOM1+RkWCk1MrjtdPJPbsrWgZH5pAB=QtOPYH@mail.gmail.com>
	<87oc8wyng9.fsf@benfinney.id.au>
	<AANLkTi=TuSuStGHMGnGK5T9K0Qef5kZjdNZcTHONowzs@mail.gmail.com>
Message-ID: <877hfgypsw.fsf@benfinney.id.au>

Benjamin Peterson <benjamin at python.org> writes:

> 2010/12/8 Ben Finney <ben+python at benfinney.id.au>:
> > So it's not abandoned, but I don't know which version should be the
> > current target. What change should I make to the PEP in such a case?
>
> Put 3.3 or 3.x if you're thinking really long term. :)

Done now (<URL:http://svn.python.org/projects/peps/trunk/>, revision
87170).

-- 
 \       ?We must respect the other fellow's religion, but only in the |
  `\       sense and to the extent that we respect his theory that his |
_o__)     wife is beautiful and his children smart.? ?Henry L. Mencken |
Ben Finney


From raymond.hettinger at gmail.com  Sat Dec 11 22:18:49 2010
From: raymond.hettinger at gmail.com (Raymond Hettinger)
Date: Sat, 11 Dec 2010 13:18:49 -0800
Subject: [Python-Dev] API for the new sysconfig module
In-Reply-To: <AANLkTikNKxtpTj-36nS1r+h9c0YKDaU6skW+n0S=kGa-@mail.gmail.com>
References: <B710BA0A-7097-4070-9DA4-E33149620833@gmail.com>
	<20101210215902.CDB5B1AB60E@kimball.webabinitio.net>
	<iduohr$q36$2@dough.gmane.org>
	<AANLkTikWzL1R5cpTpOFE5dzrZ=rRy0CEWJHYJXhYc6ZV@mail.gmail.com>
	<20101211151703.09c8675e@pitrou.net>
	<AANLkTikNKxtpTj-36nS1r+h9c0YKDaU6skW+n0S=kGa-@mail.gmail.com>
Message-ID: <037003C3-70F1-4430-8CA4-55C122F40C86@gmail.com>


On Dec 11, 2010, at 9:21 AM, Nick Coghlan wrote:

> On Sun, Dec 12, 2010 at 12:17 AM, Antoine Pitrou <solipsis at pitrou.net> wrote:
>> On Sat, 11 Dec 2010 12:55:25 +1000
>> Nick Coghlan <ncoghlan at gmail.com> wrote:
>> 
>>> On Sat, Dec 11, 2010 at 12:44 PM, Terry Reedy <tjreedy at udel.edu> wrote:
>>>> On 12/10/2010 4:59 PM, R. David Murray wrote:
>>>> 
>>>>> Like ?ric, I'm not sure what the implications of the existing module
>>>>> having been released in 2.7 and 3.2 beta are in terms of making such an
>>>>> API change.
>>>> 
>>>> I am with Raymond on this: the purpose of betas is so we can test *and* make
>>>> changes. No one should base production software on a beta release.
>>> 
>>> It's also the key difference between betas and release candidates.
>> 
>> Seems that both you and Terry have missed the part where the sysconfig
>> API is already part of 2.7, though.
> 
> I had missed that, yes*. I suspect Raymond may have missed it as well.
> 
> Cheers,
> Nick.
> 
> *(I sometimes lose track of which changes were made in both branches
> pre-2.7, which ones were mode post-2.7 release, and which ones went in
> pre-2.7, but were 3.x only regardless)

Right.  I missed that it was already in 2.7.
So, now we're stuck with it, forever.


Raymond

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20101211/bb4e8250/attachment-0001.html>

From tjreedy at udel.edu  Sat Dec 11 22:28:25 2010
From: tjreedy at udel.edu (Terry Reedy)
Date: Sat, 11 Dec 2010 16:28:25 -0500
Subject: [Python-Dev] porting python.org
In-Reply-To: <ie0bup$a70$1@dough.gmane.org>
References: <AANLkTikz6ZwDiv1qeN2+Bvi6NiZ6P-ph2EeYjtPkqK78@mail.gmail.com>
	<ie0bup$a70$1@dough.gmane.org>
Message-ID: <ie0qdo$5b0$1@dough.gmane.org>

On 12/11/2010 12:19 PM, Georg Brandl wrote:
> Am 11.12.2010 11:36, schrieb Prashant Kumar:
>> I was wondering if we could contribute in porting of python.org
>> <http://python.org>  website over to python3k.

I think this is an excellent idea. It will test Python3 and the modules 
and external packages used and once successful, advertise that all such 
are production-ready Py3 software. If and when there is a test version I 
can access (py3.python.org?), I will happily help test by browsing 
around and downloading stuff.

-- 
Terry Jan Reedy


From ben+python at benfinney.id.au  Sat Dec 11 22:47:51 2010
From: ben+python at benfinney.id.au (Ben Finney)
Date: Sun, 12 Dec 2010 08:47:51 +1100
Subject: [Python-Dev] Status of PEP 3143?
References: <AANLkTin3_LSU2CsRNH_ZYvL9LB7AMjBQR=1ji1QFWgww@mail.gmail.com>
	<AANLkTinmOM1+RkWCk1MrjtdPJPbsrWgZH5pAB=QtOPYH@mail.gmail.com>
	<87oc8wyng9.fsf@benfinney.id.au>
	<AANLkTi=TuSuStGHMGnGK5T9K0Qef5kZjdNZcTHONowzs@mail.gmail.com>
	<877hfgypsw.fsf@benfinney.id.au>
Message-ID: <871v5oyoag.fsf@benfinney.id.au>

Ben Finney <ben+python at benfinney.id.au> writes:

> Done now (<URL:http://svn.python.org/projects/peps/trunk/>, revision
> 87170).

Thanks to Georg Brandl for applying the change on my behalf.

-- 
 \       ?As the most participatory form of mass speech yet developed, |
  `\    the Internet deserves the highest protection from governmental |
_o__)                   intrusion.? ?U.S. District Court Judge Dalzell |
Ben Finney


From g.brandl at gmx.net  Sat Dec 11 23:15:58 2010
From: g.brandl at gmx.net (Georg Brandl)
Date: Sat, 11 Dec 2010 23:15:58 +0100
Subject: [Python-Dev] porting python.org
In-Reply-To: <ie0qdo$5b0$1@dough.gmane.org>
References: <AANLkTikz6ZwDiv1qeN2+Bvi6NiZ6P-ph2EeYjtPkqK78@mail.gmail.com>	<ie0bup$a70$1@dough.gmane.org>
	<ie0qdo$5b0$1@dough.gmane.org>
Message-ID: <ie0t9n$g3g$1@dough.gmane.org>

Am 11.12.2010 22:28, schrieb Terry Reedy:
> On 12/11/2010 12:19 PM, Georg Brandl wrote:
>> Am 11.12.2010 11:36, schrieb Prashant Kumar:
>>> I was wondering if we could contribute in porting of python.org
>>> <http://python.org>  website over to python3k.
> 
> I think this is an excellent idea. It will test Python3 and the modules 
> and external packages used and once successful, advertise that all such 
> are production-ready Py3 software. If and when there is a test version I 
> can access (py3.python.org?), I will happily help test by browsing 
> around and downloading stuff.

I don't think a separate test site is useful.  Either the build process
works with Python 3 or it doesn't -- in the latter case you'll just get
an exception while running "make".

Georg


From steve at pearwood.info  Sun Dec 12 02:22:43 2010
From: steve at pearwood.info (Steven D'Aprano)
Date: Sun, 12 Dec 2010 12:22:43 +1100
Subject: [Python-Dev] futures API
In-Reply-To: <B2438CD8-1EC2-4123-8E2A-B36829241D6C@sweetapp.com>
References: <608275.24121.qm@web27507.mail.ukl.yahoo.com>
	<B2438CD8-1EC2-4123-8E2A-B36829241D6C@sweetapp.com>
Message-ID: <4D0423E3.4020805@pearwood.info>

Brian Quinlan wrote:
> 
> On Dec 11, 2010, at 6:44 AM, Thomas Nagy wrote:

>> I have also observed a minor performance degradation with the executor 
>> replacement (3 seconds for 5000 work items). The amount of work items 
>> processed by unit of time does not seem to be a straight line: 
>> http://www.freehackers.org/~tnagy/runtime_futures_2.png .
> 
> That looks pretty linear to me.

Close to, but not quite. The graph seems to be slightly curved, with the 
amount of work done per second decreasing for large amounts of work. 
Assuming that this performance degradation is real, and not an artifact 
of the measurement technique, it seems to be quite small. I'd be happy 
to describe it as "linear" in the same way we describe dictionary 
lookups as constant-time, even though technically that's not strictly 
true. (They're linear in the number of items with a matching hash, and 
there are probably other complications as well.)

As drawn, the curve seems to fall away like a log graph, which might 
suggest to the casual viewer that this is a good thing. It may be better 
to reverse the axes, that is to have the independent variable, number of 
tasks, on the horizontal axis, and the dependent variable, cost (time 
taken), vertically. This will make it clear that the incremental cost of 
doing one extra task increases (slightly) as the number of tasks goes up.



-- 
Steven


From ncoghlan at gmail.com  Sun Dec 12 03:30:37 2010
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sun, 12 Dec 2010 12:30:37 +1000
Subject: [Python-Dev] Format factories (was Re: sWAPcASE Was:
 transform() and untransform() methods, and the codec registry)
In-Reply-To: <AANLkTi=4VNE+nkFzSRfdyDmuRQ9JaK7JLmQJ2ptymt=f@mail.gmail.com>
References: <AANLkTi=rLuuLkFzrpt7REsm91-7xfBmFcMD__B5rQxo6@mail.gmail.com>
	<AANLkTi=4VNE+nkFzSRfdyDmuRQ9JaK7JLmQJ2ptymt=f@mail.gmail.com>
Message-ID: <AANLkTikTZs8FdiyM3cXagrmnrS5YivnfHx-gtPcPFx4n@mail.gmail.com>

On Sun, Dec 12, 2010 at 5:21 AM, Brett Cannon <brett at python.org> wrote:
> But is this worth it since once you write it you won't be changing it
> again, suggesting that taking the time to look up the formatting rules
> isn't that much harder and wouldn't burden us w/ writing such
> functions and trying to come up with a good API. I suspect what would
> be more helpful is either have a rather detailed docstring for
> str.format or to at least put the URL to the docs which explains the
> mini-language to make it easier to find.

Yes, it may be worth it, since it greatly simplifies things like
generating a format spec programmatically rather than hardcoding it.
You can also run it once at the interactive prompt to figure out the
format string you want to hard code, then include the API call as a
comment for concise documentation of what your format string does. I
helped *define* the new format spec and I still need to reference the
docs to create trickier ones. A factory function would be far more
convenient than burdening str.format's docstring with a full
definition of the format spec syntax.

Cheers,
Nick.

-- 
Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia

From ncoghlan at gmail.com  Sun Dec 12 03:33:34 2010
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sun, 12 Dec 2010 12:33:34 +1000
Subject: [Python-Dev] futures API
In-Reply-To: <B2438CD8-1EC2-4123-8E2A-B36829241D6C@sweetapp.com>
References: <608275.24121.qm@web27507.mail.ukl.yahoo.com>
	<B2438CD8-1EC2-4123-8E2A-B36829241D6C@sweetapp.com>
Message-ID: <AANLkTimfLNZYkOGkPY9RjLV-gBY6C1EiawgUAofFwi5r@mail.gmail.com>

On Sun, Dec 12, 2010 at 6:53 AM, Brian Quinlan <brian at sweetapp.com> wrote:
> Is it still unclear why it is there? Maybe you could propose some additional
> documentation.

Did you get my question the other day as to whether a
weakref.WeakKeySet might be a better choice? I believe you would be
able to get rid of the periodic sweep for dead references if you did
that, and I didn't spot any obvious downsides.

Cheers,
Nick.

-- 
Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia

From tnagyemail-mail at yahoo.fr  Sun Dec 12 03:32:18 2010
From: tnagyemail-mail at yahoo.fr (Thomas Nagy)
Date: Sun, 12 Dec 2010 02:32:18 +0000 (GMT)
Subject: [Python-Dev] futures API
Message-ID: <229168.97019.qm@web27508.mail.ukl.yahoo.com>

--- El s?b, 11/12/10, Brian Quinlan escribi?:
> 
> On Dec 11, 2010, at 6:44 AM, Thomas Nagy wrote:
> 
> > --- El vie, 10/12/10, Brian Quinlan escribi?:
> >> On Dec 10, 2010, at 10:51 AM, Thomas Nagy wrote:
> >>> --- El vie, 10/12/10, Brian Quinlan
> escribi?:
> >>>> On Dec 10, 2010, at 5:36 AM, Thomas Nagy
> wrote:
> >>>>> I have a process running for a long
> time, and
> >> which
> >>>> may use futures of different max_workers
> count. I
> >> think it
> >>>> is not too far-fetched to create a new
> futures
> >> object each
> >>>> time. Yet, the execution becomes slower
> after each
> >> call, for
> >>>> example with http://freehackers.org/~tnagy/futures_test.py:
> >>>>> 
> >>>>> """
> >>>>> import concurrent.futures
> >>>>> from queue import Queue
> >>>>> import datetime
> >>>>> 
> >>>>> class counter(object):
> >>>>>? ? ???def
> __init__(self, fut):
> >>>>>? ? ? ?
> ???self.fut =
> >> fut
> >>>>> 
> >>>>>? ? ???def
> run(self):
> >>>>>? ? ? ?
> ???def
> >>>> look_busy(num, obj):
> >>>>> 
> >>>>? ???tot = 0
> >>>>> 
> >>>>? ???for x in
> range(num):
> >>>>> 
> >>>>? ???tot += x
> >>>>> 
> >>>>?
> ???obj.out_q.put(tot)
> >>>>> 
> >>>>>? ? ? ?
> ???start =
> >>>> datetime.datetime.utcnow()
> >>>>>? ? ? ?
> ???self.count =
> >> 0
> >>>>>? ? ? ?
> ???self.out_q
> >> =
> >>>> Queue(0)
> >>>>>? ? ? ?
> ???for x in
> >>>> range(1000):
> >>>>> 
> >>>>? ???self.count += 1
> >>>>> 
> >>>>?
> ???self.fut.submit(look_busy,
> >> self.count,
> >>>> self)
> >>>>> 
> >>>>>? ? ? ?
> ???while
> >>>> self.count:
> >>>>> 
> >>>>? ???self.count -= 1
> >>>>> 
> >>>>? ???self.out_q.get()
> >>>>> 
> >>>>>? ? ? ?
> ???delta =
> >>>> datetime.datetime.utcnow() - start
> >>>>> 
> >>>>?
> ???print(delta.total_seconds())
> >>>>> 
> >>>>> fut =
> >>>> 
> >>
> concurrent.futures.ThreadPoolExecutor(max_workers=20)
> >>>>> for x in range(100):
> >>>>>? ? ???#
> comment the following
> >> line
> >>>>>? ? ???fut =
> >>>> 
> >>
> concurrent.futures.ThreadPoolExecutor(max_workers=20)
> >>>>>? ? ???c =
> counter(fut)
> >>>>>? ?
> ???c.run()
> >>>>> """
> >>>>> 
> >>>>> The runtime grows after each step:
> >>>>> 0.216451
> >>>>> 0.225186
> >>>>> 0.223725
> >>>>> 0.222274
> >>>>> 0.230964
> >>>>> 0.240531
> >>>>> 0.24137
> >>>>> 0.252393
> >>>>> 0.249948
> >>>>> 0.257153
> >>>>> ...
> >>>>> 
> >>>>> Is there a mistake in this piece of
> code?
> >>>> 
> >>>> There is no mistake that I can see but I
> suspect
> >> that the
> >>>> circular references that you are building
> are
> >> causing the
> >>>> ThreadPoolExecutor to take a long time to
> be
> >> collected. Try
> >>>> adding:
> >>>> 
> >>>>? ? ? c = counter(fut)
> >>>>? ? ? c.run()
> >>>> +? ? fut.shutdown()
> >>>> 
> >>>> Even if that fixes your problem, I still
> don't
> >> fully
> >>>> understand this because I would expect the
> runtime
> >> to fall
> >>>> after a while as ThreadPoolExecutors are
> >> collected.
> >>> 
> >>> The shutdown call is indeed a good fix :-)
> Here is the
> >> time response
> >>> of the calls to counter() when shutdown is
> not
> >> called:
> >>> http://www.freehackers.org/~tnagy/runtime_futures.png
> >> 
> >> FWIW, I think that you are confusion the term
> "future"
> >> with
> >> "executor". A future represents a single work
> item. An
> >> executor
> >> creates futures and schedules their underlying
> work.
> > 
> > Ah yes, sorry. I have also realized that the executor
> is not the killer feature I was expecting, it can only
> replace a little part of the code I have: controlling the
> exceptions and the workflow is the most complicated part.
> > 
> > I have also observed a minor performance degradation
> with the executor replacement (3 seconds for 5000 work
> items). The amount of work items processed by unit of time
> does not seem to be a straight line: http://www.freehackers.org/~tnagy/runtime_futures_2.png
> .
> 
> That looks pretty linear to me.

Ok.
 
> > Out of curiosity, what is the "_thread_references"
> for?
> 
> There is a big comment above it in the code:
> 
> # Workers are created as daemon threads. This is done to
> allow the interpreter
> # to exit when there are still idle threads in a
> ThreadPoolExecutor's thread
> # pool (i.e. shutdown() was not called). However, allowing
> workers to die with
> # the interpreter has two undesirable properties:
> #???- The workers would still be running
> during interpretor shutdown,
> #? ???meaning that they would fail in
> unpredictable ways.
> #???- The workers could be killed while
> evaluating a work item, which could
> #? ???be bad if the callable being
> evaluated has external side-effects e.g.
> #? ???writing to a file.
> #
> # To work around this problem, an exit handler is installed
> which tells the
> # workers to exit when their work queues are empty and then
> waits until the
> # threads finish.
> 
> _thread_references = set()
> _shutdown = False
> 
> def _python_exit():
> ? ? global _shutdown
> ? ? _shutdown = True
> ? ? for thread_reference in _thread_references:
> ? ? ? ? thread = thread_reference()
> ? ? ? ? if thread is not None:
> ? ? ? ? ? ? thread.join()
> 
> Is it still unclear why it is there? Maybe you could
> propose some additional documentation.

I was thinking that if exceptions have to be caught - and it is likely to be the case in general - then this scheme is redundant. Now I see that the threads are getting their work items from a queue, so it is clear now.

Thanks for all the information,

Thomas



      

From brian at sweetapp.com  Sun Dec 12 03:36:47 2010
From: brian at sweetapp.com (Brian Quinlan)
Date: Sat, 11 Dec 2010 18:36:47 -0800
Subject: [Python-Dev] futures API
In-Reply-To: <AANLkTimfLNZYkOGkPY9RjLV-gBY6C1EiawgUAofFwi5r@mail.gmail.com>
References: <608275.24121.qm@web27507.mail.ukl.yahoo.com>
	<B2438CD8-1EC2-4123-8E2A-B36829241D6C@sweetapp.com>
	<AANLkTimfLNZYkOGkPY9RjLV-gBY6C1EiawgUAofFwi5r@mail.gmail.com>
Message-ID: <2FA356B1-88E6-43C7-8118-70CAD2BBB315@sweetapp.com>


On Dec 11, 2010, at 6:33 PM, Nick Coghlan wrote:

> On Sun, Dec 12, 2010 at 6:53 AM, Brian Quinlan <brian at sweetapp.com>  
> wrote:
>> Is it still unclear why it is there? Maybe you could propose some  
>> additional
>> documentation.
>
> Did you get my question the other day as to whether a
> weakref.WeakKeySet might be a better choice? I believe you would be
> able to get rid of the periodic sweep for dead references if you did
> that, and I didn't spot any obvious downsides.

No I didn't, sorry! Could you resent it if it has more details then  
the paragraph above?

Cheers,
Brian

> Cheers,
> Nick.
>
> -- 
> Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia


From ncoghlan at gmail.com  Sun Dec 12 07:05:38 2010
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sun, 12 Dec 2010 16:05:38 +1000
Subject: [Python-Dev] futures API
In-Reply-To: <2FA356B1-88E6-43C7-8118-70CAD2BBB315@sweetapp.com>
References: <608275.24121.qm@web27507.mail.ukl.yahoo.com>
	<B2438CD8-1EC2-4123-8E2A-B36829241D6C@sweetapp.com>
	<AANLkTimfLNZYkOGkPY9RjLV-gBY6C1EiawgUAofFwi5r@mail.gmail.com>
	<2FA356B1-88E6-43C7-8118-70CAD2BBB315@sweetapp.com>
Message-ID: <AANLkTinhc-Pzr+bakj3s_CWEif9E7EvC+U6s46-8Bfku@mail.gmail.com>

On Sun, Dec 12, 2010 at 12:36 PM, Brian Quinlan <brian at sweetapp.com> wrote:
>
> On Dec 11, 2010, at 6:33 PM, Nick Coghlan wrote:
>
>> On Sun, Dec 12, 2010 at 6:53 AM, Brian Quinlan <brian at sweetapp.com> wrote:
>>>
>>> Is it still unclear why it is there? Maybe you could propose some
>>> additional
>>> documentation.
>>
>> Did you get my question the other day as to whether a
>> weakref.WeakKeySet might be a better choice? I believe you would be
>> able to get rid of the periodic sweep for dead references if you did
>> that, and I didn't spot any obvious downsides.
>
> No I didn't, sorry! Could you resent it if it has more details then the
> paragraph above?

There wasn't much more detail, as there actually isn't a great deal to
the idea. This was the main comment in the previous email:

"It seems to me that using WeakSet would mean you could get rid of
_remove_dead_thread_references altogether (the implicit callbacks
would handle that part), and then a set() call in _python_exit would
give you concrete references to work with for cleanup purposes."

Cheers,
Nick.

-- 
Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia

From lukasz at langa.pl  Sun Dec 12 13:01:42 2010
From: lukasz at langa.pl (=?utf-8?Q?=C5=81ukasz_Langa?=)
Date: Sun, 12 Dec 2010 13:01:42 +0100
Subject: [Python-Dev] API for the new sysconfig module
In-Reply-To: <037003C3-70F1-4430-8CA4-55C122F40C86@gmail.com>
References: <B710BA0A-7097-4070-9DA4-E33149620833@gmail.com>
	<20101210215902.CDB5B1AB60E@kimball.webabinitio.net>
	<iduohr$q36$2@dough.gmane.org>
	<AANLkTikWzL1R5cpTpOFE5dzrZ=rRy0CEWJHYJXhYc6ZV@mail.gmail.com>
	<20101211151703.09c8675e@pitrou.net>
	<AANLkTikNKxtpTj-36nS1r+h9c0YKDaU6skW+n0S=kGa-@mail.gmail.com>
	<037003C3-70F1-4430-8CA4-55C122F40C86@gmail.com>
Message-ID: <27F9BCAE-74C1-4863-85A4-1E411E1AC06A@langa.pl>

Wiadomo?? napisana przez Raymond Hettinger w dniu 2010-12-11, o godz. 22:18:

>> *(I sometimes lose track of which changes were made in both branches
>> pre-2.7, which ones were mode post-2.7 release, and which ones went in
>> pre-2.7, but were 3.x only regardless)
> 
> Right.  I missed that it was already in 2.7.
> So, now we're stuck with it, forever.

Why not deprecate it for 3.2 (easy since it's probably not yet used anywhere anyway, even in 2.7) and introduce sys.sysconfig. I really like that much better than Java-like accessor functions.

-- 
Best regards,
?ukasz Langa
tel. +48 791 080 144
WWW http://lukasz.langa.pl/

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20101212/95146c50/attachment.html>

From solipsis at pitrou.net  Sun Dec 12 14:42:31 2010
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Sun, 12 Dec 2010 14:42:31 +0100
Subject: [Python-Dev] API for the new sysconfig module
References: <B710BA0A-7097-4070-9DA4-E33149620833@gmail.com>
	<20101210215902.CDB5B1AB60E@kimball.webabinitio.net>
	<iduohr$q36$2@dough.gmane.org>
	<AANLkTikWzL1R5cpTpOFE5dzrZ=rRy0CEWJHYJXhYc6ZV@mail.gmail.com>
	<20101211151703.09c8675e@pitrou.net>
	<AANLkTikNKxtpTj-36nS1r+h9c0YKDaU6skW+n0S=kGa-@mail.gmail.com>
	<037003C3-70F1-4430-8CA4-55C122F40C86@gmail.com>
	<27F9BCAE-74C1-4863-85A4-1E411E1AC06A@langa.pl>
Message-ID: <20101212144231.772efa58@pitrou.net>

On Sun, 12 Dec 2010 13:01:42 +0100
?ukasz Langa <lukasz at langa.pl> wrote:

> Wiadomo?? napisana przez Raymond Hettinger w dniu 2010-12-11, o godz. 22:18:
> 
> >> *(I sometimes lose track of which changes were made in both branches
> >> pre-2.7, which ones were mode post-2.7 release, and which ones went in
> >> pre-2.7, but were 3.x only regardless)
> > 
> > Right.  I missed that it was already in 2.7.
> > So, now we're stuck with it, forever.
> 
> Why not deprecate it for 3.2 (easy since it's probably not yet used anywhere anyway, even in 2.7) and introduce sys.sysconfig.

We already had a lot of churn around these APIs (distutils & friends). I
don't think it's a good idea to introduce even more churn.
Oh and by the way it's too late to add or remove features from 3.2.

> I really like that much better than Java-like accessor functions.

Do you actually use sysconfig yourself? It's quite a specialized
module, and I don't think API elegance arguments have a great weight
here.

Regards

Antoine.



From barry at python.org  Sun Dec 12 15:05:06 2010
From: barry at python.org (Barry Warsaw)
Date: Sun, 12 Dec 2010 09:05:06 -0500
Subject: [Python-Dev] API for the new sysconfig module
In-Reply-To: <20101212144231.772efa58@pitrou.net>
References: <B710BA0A-7097-4070-9DA4-E33149620833@gmail.com>
	<20101210215902.CDB5B1AB60E@kimball.webabinitio.net>
	<iduohr$q36$2@dough.gmane.org>
	<AANLkTikWzL1R5cpTpOFE5dzrZ=rRy0CEWJHYJXhYc6ZV@mail.gmail.com>
	<20101211151703.09c8675e@pitrou.net>
	<AANLkTikNKxtpTj-36nS1r+h9c0YKDaU6skW+n0S=kGa-@mail.gmail.com>
	<037003C3-70F1-4430-8CA4-55C122F40C86@gmail.com>
	<27F9BCAE-74C1-4863-85A4-1E411E1AC06A@langa.pl>
	<20101212144231.772efa58@pitrou.net>
Message-ID: <20101212090506.7c22638e@mission>

On Dec 12, 2010, at 02:42 PM, Antoine Pitrou wrote:

>On Sun, 12 Dec 2010 13:01:42 +0100
>?ukasz Langa <lukasz at langa.pl> wrote:
>
>> Wiadomo?? napisana przez Raymond Hettinger w dniu 2010-12-11, o godz. 22:18:
>> 
>> >> *(I sometimes lose track of which changes were made in both branches
>> >> pre-2.7, which ones were mode post-2.7 release, and which ones went in
>> >> pre-2.7, but were 3.x only regardless)
>> > 
>> > Right.  I missed that it was already in 2.7.
>> > So, now we're stuck with it, forever.
>> 
>> Why not deprecate it for 3.2 (easy since it's probably not yet used anywhere anyway, even in 2.7) and introduce sys.sysconfig.
>
>We already had a lot of churn around these APIs (distutils & friends). I
>don't think it's a good idea to introduce even more churn.
>Oh and by the way it's too late to add or remove features from 3.2.
>
>> I really like that much better than Java-like accessor functions.
>
>Do you actually use sysconfig yourself? It's quite a specialized
>module, and I don't think API elegance arguments have a great weight
>here.

I have used them and I do think they're fairly ugly and unwieldy.  However, I
agree that we should not rush into a different design.  Since sysconfig was
essentially refactored out of distutils (and now we have two ways to do it!),
and we're getting distutils2 for 3.3, I think it would be better to work out a
more natural design for sysconfig for 3.3.  Then the sysconfig module can go
through the natural deprecation cycle.

-Barry

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 836 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/python-dev/attachments/20101212/38b7dddc/attachment.pgp>

From ncoghlan at gmail.com  Sun Dec 12 15:05:44 2010
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Mon, 13 Dec 2010 00:05:44 +1000
Subject: [Python-Dev] API for the new sysconfig module
In-Reply-To: <20101212144231.772efa58@pitrou.net>
References: <B710BA0A-7097-4070-9DA4-E33149620833@gmail.com>
	<20101210215902.CDB5B1AB60E@kimball.webabinitio.net>
	<iduohr$q36$2@dough.gmane.org>
	<AANLkTikWzL1R5cpTpOFE5dzrZ=rRy0CEWJHYJXhYc6ZV@mail.gmail.com>
	<20101211151703.09c8675e@pitrou.net>
	<AANLkTikNKxtpTj-36nS1r+h9c0YKDaU6skW+n0S=kGa-@mail.gmail.com>
	<037003C3-70F1-4430-8CA4-55C122F40C86@gmail.com>
	<27F9BCAE-74C1-4863-85A4-1E411E1AC06A@langa.pl>
	<20101212144231.772efa58@pitrou.net>
Message-ID: <AANLkTikqc6fSsB_aNnOThJNtLPg9NTdwpt3oMyLxtiRu@mail.gmail.com>

On Sun, Dec 12, 2010 at 11:42 PM, Antoine Pitrou <solipsis at pitrou.net> wrote:
>> I really like that much better than Java-like accessor functions.
>
> Do you actually use sysconfig yourself? It's quite a specialized
> module, and I don't think API elegance arguments have a great weight
> here.

I would also like those advocating replacement of a tried and tested
API with "oh, you can just use a single function for it" to rewrite
distutils2 and the other tools that currently used distutils.sysconfig
based on their "single function" approach before *anything* gets
touched.

This is not an API that was invented on a whim. It is
well-established, with existing use cases that are essential to the
wider Python ecosystem, and are more easily managed on some Linux
distributions if they don't involve a dependency on distutils.
Proposing to throw it away in favour of an ill-defined single function
that indiscriminately mixes system data with metadata about that data
because some people that don't even use the module find it
aesthetically displeasing seems... unwise.

Regards,
Nick.

-- 
Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia

From ncoghlan at gmail.com  Sun Dec 12 15:44:22 2010
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Mon, 13 Dec 2010 00:44:22 +1000
Subject: [Python-Dev] Using logging in the stdlib and its unit tests
In-Reply-To: <AANLkTikbjCwVkjAdU6CqDsso7AhHeU4i0uK3v-VnTFVE@mail.gmail.com>
References: <loom.20101207T212306-892@post.gmane.org>
	<20101207215053.025ecf52@pitrou.net>
	<loom.20101208T002100-91@post.gmane.org>
	<20101208010913.363c6105@pitrou.net>
	<AANLkTim4ZPoch18+-qkunuYm-zcyh-vbhMOv9YVVEqo9@mail.gmail.com>
	<loom.20101208T022102-46@post.gmane.org>
	<AANLkTikFNUPA5Cvbz8cOAfznSOEe906O8zOfOC=2Cj+H@mail.gmail.com>
	<loom.20101210T085332-877@post.gmane.org>
	<4D01E55B.2050507@g.nevcal.com> <3231.1292004978@parc.com>
	<loom.20101210T202041-859@post.gmane.org>
	<AANLkTi=PnWE5MbpyWKZHJU4Abk-OeVfKEJYpGX1E6g3=@mail.gmail.com>
	<20101210214919.4ac63d27@pitrou.net>
	<loom.20101210T224910-511@post.gmane.org>
	<iduoeb$q36$1@dough.gmane.org>
	<AANLkTi=Q8mZb0oDe5xz7Evb0y8ohgkWh=X7E1OgCrUka@mail.gmail.com>
	<AANLkTikN_378ZH2kCW-WJ2wWDVZrYb2zrQ9N1QMRvkg0@mail.gmail.com>
	<loom.20101211T072018-194@post.gmane.org>
	<AANLkTikbjCwVkjAdU6CqDsso7AhHeU4i0uK3v-VnTFVE@mail.gmail.com>
Message-ID: <AANLkTi=BDNkDdiPJpH3AvtVcnOuc+EVtG9B8MLFyVac-@mail.gmail.com>

On Sun, Dec 12, 2010 at 5:32 AM, Brett Cannon <brett at python.org> wrote:
> On Fri, Dec 10, 2010 at 22:21, Vinay Sajip <vinay_sajip at yahoo.co.uk> wrote:
>> Nick Coghlan <ncoghlan <at> gmail.com> writes:
>>
>>
>>> This could actually make a reasonably good basic for a "task oriented"
>>> subsection of the logging documentation. Something like:
>>>
>>
>> Good suggestion, I'll see what I can do.
>
> Just wanted to +1 on some task-oriented (or at least simple intro)
> docs going into the logging module.

I think Vinay has made some great improvements to the logging module
docs in the last day or two. The latest version out of SVN is
available on the site at the usual location:
http://docs.python.org/dev/library/logging

I am putting some minor notes here for Vinay's benefit (I can put them
on the tracker instead, if he would prefer):

General
  It may be worth talking to Georg about how best to split the logging
docs up into multiple files. The sidebar menu is getting kinda
overwhelmed.

14.7.1.1
  Parenthetical comment in first row of second table should start with
"(e.g. for" not "(e.g. or"

14.7.1.8
  Probably best to say "that's it for the basic tutorial" and then
point people towards the advanced tutorial in 14.7.2 before setting
them loose on the rest of the docs. The advanced tutorial defines the
terminology and gives the necessary structure to help keep the
detailed docs in perspective without being overwhelmed by the detail.

14.7.2.1
  Something appears to have gone wrong with the first bulleted list.
It is missing the "These are the configuration methods:" intro text,
as well as a bullet for add/removeHandler
  The "does not address filters" part should cross-reference the
detailed section on filter objects
  This section should state explicitly whether or not the level
setting on a child logger affects which messages it passes to its
parent logger

14.7.2.5
  This section is out of date, and needs to be caveated to make it
clear that it applies only to version prior to Python 3.2 (for 3.2, it
can describe the new handler of last resort behaviour)

And that's the end of the two tutorials... very nice update :)

Cheers,
Nick.

-- 
Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia

From stefan-usenet at bytereef.org  Sun Dec 12 15:37:03 2010
From: stefan-usenet at bytereef.org (Stefan Krah)
Date: Sun, 12 Dec 2010 15:37:03 +0100
Subject: [Python-Dev] API for the new sysconfig module
In-Reply-To: <27F9BCAE-74C1-4863-85A4-1E411E1AC06A@langa.pl>
References: <B710BA0A-7097-4070-9DA4-E33149620833@gmail.com>
	<20101210215902.CDB5B1AB60E@kimball.webabinitio.net>
	<iduohr$q36$2@dough.gmane.org>
	<AANLkTikWzL1R5cpTpOFE5dzrZ=rRy0CEWJHYJXhYc6ZV@mail.gmail.com>
	<20101211151703.09c8675e@pitrou.net>
	<AANLkTikNKxtpTj-36nS1r+h9c0YKDaU6skW+n0S=kGa-@mail.gmail.com>
	<037003C3-70F1-4430-8CA4-55C122F40C86@gmail.com>
	<27F9BCAE-74C1-4863-85A4-1E411E1AC06A@langa.pl>
Message-ID: <20101212143703.GA15235@yoda.bytereef.org>

Lukasz Langa <lukasz at langa.pl> wrote:
> Wiadomo?? napisana przez Raymond Hettinger w dniu 2010-12-11, o godz. 22:18:
> 
>     Right.  I missed that it was already in 2.7.
>     So, now we're stuck with it, forever.
> 
> Why not deprecate it for 3.2 (easy since it's probably not yet used anywhere
> anyway, even in 2.7) and introduce sys.sysconfig. I really like that much
> better than Java-like accessor functions.


When I use sysconfig, I just want to get things done as quickly and painlessly
as possible. The API suits me just fine (in fact, I find it one of the APIs
that are easy to remember).


Given that sysconfig will always contain a certain amount of hackery and
will always change to accommodate new systems, I'd prefer that it remains
a standalone module.


Stefan Krah



From ziade.tarek at gmail.com  Sun Dec 12 16:53:20 2010
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Sun, 12 Dec 2010 16:53:20 +0100
Subject: [Python-Dev] API for the new sysconfig module
In-Reply-To: <20101212090506.7c22638e@mission>
References: <B710BA0A-7097-4070-9DA4-E33149620833@gmail.com>
	<20101210215902.CDB5B1AB60E@kimball.webabinitio.net>
	<iduohr$q36$2@dough.gmane.org>
	<AANLkTikWzL1R5cpTpOFE5dzrZ=rRy0CEWJHYJXhYc6ZV@mail.gmail.com>
	<20101211151703.09c8675e@pitrou.net>
	<AANLkTikNKxtpTj-36nS1r+h9c0YKDaU6skW+n0S=kGa-@mail.gmail.com>
	<037003C3-70F1-4430-8CA4-55C122F40C86@gmail.com>
	<27F9BCAE-74C1-4863-85A4-1E411E1AC06A@langa.pl>
	<20101212144231.772efa58@pitrou.net>
	<20101212090506.7c22638e@mission>
Message-ID: <AANLkTim4RN5W6xbJut0M8YU_=81Jr7TbEYd29iQPkQgA@mail.gmail.com>

On Sun, Dec 12, 2010 at 3:05 PM, Barry Warsaw <barry at python.org> wrote:
> On Dec 12, 2010, at 02:42 PM, Antoine Pitrou wrote:
>
>>On Sun, 12 Dec 2010 13:01:42 +0100
>>?ukasz Langa <lukasz at langa.pl> wrote:
>>
>>> Wiadomo?? napisana przez Raymond Hettinger w dniu 2010-12-11, o godz. 22:18:
>>>
>>> >> *(I sometimes lose track of which changes were made in both branches
>>> >> pre-2.7, which ones were mode post-2.7 release, and which ones went in
>>> >> pre-2.7, but were 3.x only regardless)
>>> >
>>> > Right. ?I missed that it was already in 2.7.
>>> > So, now we're stuck with it, forever.
>>>
>>> Why not deprecate it for 3.2 (easy since it's probably not yet used anywhere anyway, even in 2.7) and introduce sys.sysconfig.
>>
>>We already had a lot of churn around these APIs (distutils & friends). I
>>don't think it's a good idea to introduce even more churn.
>>Oh and by the way it's too late to add or remove features from 3.2.
>>
>>> I really like that much better than Java-like accessor functions.
>>
>>Do you actually use sysconfig yourself? It's quite a specialized
>>module, and I don't think API elegance arguments have a great weight
>>here.
>
> I have used them and I do think they're fairly ugly and unwieldy. ?However, I
> agree that we should not rush into a different design. ?Since sysconfig was
> essentially refactored out of distutils (and now we have two ways to do it!),
> and we're getting distutils2 for 3.3, I think it would be better to work out a
> more natural design for sysconfig for 3.3. ?Then the sysconfig module can go
> through the natural deprecation cycle.

I don't think any API refactoring worth the pain here. I don't see the
proposed changes make the caller's code that much better.
The existing one is good enough in my opinion.


Tarek
-- 
Tarek Ziad? | http://ziade.org

From vinay_sajip at yahoo.co.uk  Sun Dec 12 18:41:46 2010
From: vinay_sajip at yahoo.co.uk (Vinay Sajip)
Date: Sun, 12 Dec 2010 17:41:46 +0000 (UTC)
Subject: [Python-Dev] Using logging in the stdlib and its unit tests
References: <loom.20101207T212306-892@post.gmane.org>
	<20101207215053.025ecf52@pitrou.net>
	<loom.20101208T002100-91@post.gmane.org>
	<20101208010913.363c6105@pitrou.net>
	<AANLkTim4ZPoch18+-qkunuYm-zcyh-vbhMOv9YVVEqo9@mail.gmail.com>
	<loom.20101208T022102-46@post.gmane.org>
	<AANLkTikFNUPA5Cvbz8cOAfznSOEe906O8zOfOC=2Cj+H@mail.gmail.com>
	<loom.20101210T085332-877@post.gmane.org>
	<4D01E55B.2050507@g.nevcal.com> <3231.1292004978@parc.com>
	<loom.20101210T202041-859@post.gmane.org>
	<AANLkTi=PnWE5MbpyWKZHJU4Abk-OeVfKEJYpGX1E6g3=@mail.gmail.com>
	<20101210214919.4ac63d27@pitrou.net>
	<loom.20101210T224910-511@post.gmane.org>
	<iduoeb$q36$1@dough.gmane.org>
	<AANLkTi=Q8mZb0oDe5xz7Evb0y8ohgkWh=X7E1OgCrUka@mail.gmail.com>
	<AANLkTikN_378ZH2kCW-WJ2wWDVZrYb2zrQ9N1QMRvkg0@mail.gmail.com>
	<loom.20101211T072018-194@post.gmane.org>
	<AANLkTikbjCwVkjAdU6CqDsso7AhHeU4i0uK3v-VnTFVE@mail.gmail.com>
	<AANLkTi=BDNkDdiPJpH3AvtVcnOuc+EVtG9B8MLFyVac-@mail.gmail.com>
Message-ID: <loom.20101212T183926-679@post.gmane.org>

Nick Coghlan <ncoghlan <at> gmail.com> writes:

Gosh, Nick, that was fast! I'm still making changes, but thanks for spotting and
highlighting the typos and omissions. I've just checked in a further update;
hopefully it'll get built soon so we can all see the latest changes.

Regards,

Vinay Sajip


From zeljko.grk at gmail.com  Sun Dec 12 20:04:49 2010
From: zeljko.grk at gmail.com (Zeljko)
Date: Sun, 12 Dec 2010 19:04:49 +0000 (UTC)
Subject: [Python-Dev] use case for bytes.format
Message-ID: <loom.20101212T185313-188@post.gmane.org>

I'm considering to write some pure python pdf lib from from scratch, but found 
there is no built-in formating for bytes. 
It's very frustrating to use some inefficient surogate funtion, which is gong to 
be called thousands times.




From p.f.moore at gmail.com  Sun Dec 12 23:26:10 2010
From: p.f.moore at gmail.com (Paul Moore)
Date: Sun, 12 Dec 2010 22:26:10 +0000
Subject: [Python-Dev] Using logging in the stdlib and its unit tests
In-Reply-To: <4D029439.7040708@g.nevcal.com>
References: <loom.20101207T212306-892@post.gmane.org>
	<20101207215053.025ecf52@pitrou.net>
	<loom.20101208T002100-91@post.gmane.org>
	<20101208010913.363c6105@pitrou.net>
	<AANLkTim4ZPoch18+-qkunuYm-zcyh-vbhMOv9YVVEqo9@mail.gmail.com>
	<loom.20101208T022102-46@post.gmane.org>
	<AANLkTikFNUPA5Cvbz8cOAfznSOEe906O8zOfOC=2Cj+H@mail.gmail.com>
	<loom.20101210T085332-877@post.gmane.org>
	<4D01E55B.2050507@g.nevcal.com> <3231.1292004978@parc.com>
	<loom.20101210T202041-859@post.gmane.org>
	<AANLkTi=PnWE5MbpyWKZHJU4Abk-OeVfKEJYpGX1E6g3=@mail.gmail.com>
	<20101210214919.4ac63d27@pitrou.net>
	<4D029439.7040708@g.nevcal.com>
Message-ID: <AANLkTim3i0oD9DkUwsQnE3y=zNPNhEM+GOMKL2-4cz+q@mail.gmail.com>

On 10 December 2010 20:57, Glenn Linderman <v+python at g.nevcal.com> wrote:
> On 12/10/2010 12:49 PM, Antoine Pitrou wrote:
>
> And yet, I have helped many people who were baffled by exactly what
>> Bill observed: logging.info() didn't do anything. Maybe the default
>> should be INFO?
>
> Funny, because displaying only errors and silencing other messages is
> exactly what I expected logging to do by default.
>
> So we are slowly learning the things that should be on the first couple
> pages of the logging docs...
>
> 1) simple example for one file programs, include an example of specifying
> output severity threshold.? I'm with Antoine here on my expectations.
>
> 2) example for multi-module, showing how a single logging destination causes
> logging to happen in all modules, at the same level (if that is the case,
> which I hope it is).
>
> 3) Maybe a small discussion of log formatting should be next?? So the user
> realizes he shouldn't do the message formatting himself?
>
> 4) Then rotating logs for long-running programs.

The thing *I* hit very early was wanting to add a command lime option
to my script to set the logging level. I'd have liked to be able to
add --log=INFO/DEBUG/... but to do that I seem to need to write my own
mapping between level names and numbers. A simple example of how to
tie command line options to logging config would be a great addition
to the documentation.

Paul.

From tjreedy at udel.edu  Sun Dec 12 23:33:19 2010
From: tjreedy at udel.edu (Terry Reedy)
Date: Sun, 12 Dec 2010 17:33:19 -0500
Subject: [Python-Dev] use case for bytes.format
In-Reply-To: <loom.20101212T185313-188@post.gmane.org>
References: <loom.20101212T185313-188@post.gmane.org>
Message-ID: <ie3ijd$3fo$1@dough.gmane.org>

On 12/12/2010 2:04 PM, Zeljko wrote:

This post should have gone to python-list, mirrored as 
gmane.comp.python.general. Please limit any further response to either 
of those (or c.l.p) and delete pydev.

> I'm considering to write some pure python pdf lib from from scratch, but found
> there is no built-in formating for bytes.

Bytes do, but you should use text str for general text manipulation.

> It's very frustrating to use some inefficient surogate funtion, which is gong to
> be called thousands times.

This sentence has 3 spelling mistakes, 2 of which would be caught with a 
mail client like Thunderbird (free) that has spelling correction. Please 
consider switching.

-- 
Terry Jan Reedy


From merwok at netwok.org  Sun Dec 12 23:33:22 2010
From: merwok at netwok.org (=?UTF-8?B?w4lyaWMgQXJhdWpv?=)
Date: Sun, 12 Dec 2010 23:33:22 +0100
Subject: [Python-Dev] Using logging in the stdlib and its unit tests
In-Reply-To: <loom.20101212T183926-679@post.gmane.org>
References: <loom.20101207T212306-892@post.gmane.org>	<20101207215053.025ecf52@pitrou.net>	<loom.20101208T002100-91@post.gmane.org>	<20101208010913.363c6105@pitrou.net>	<AANLkTim4ZPoch18+-qkunuYm-zcyh-vbhMOv9YVVEqo9@mail.gmail.com>	<loom.20101208T022102-46@post.gmane.org>	<AANLkTikFNUPA5Cvbz8cOAfznSOEe906O8zOfOC=2Cj+H@mail.gmail.com>	<loom.20101210T085332-877@post.gmane.org>	<4D01E55B.2050507@g.nevcal.com>
	<3231.1292004978@parc.com>	<loom.20101210T202041-859@post.gmane.org>	<AANLkTi=PnWE5MbpyWKZHJU4Abk-OeVfKEJYpGX1E6g3=@mail.gmail.com>	<20101210214919.4ac63d27@pitrou.net>	<loom.20101210T224910-511@post.gmane.org>	<iduoeb$q36$1@dough.gmane.org>	<AANLkTi=Q8mZb0oDe5xz7Evb0y8ohgkWh=X7E1OgCrUka@mail.gmail.com>	<AANLkTikN_378ZH2kCW-WJ2wWDVZrYb2zrQ9N1QMRvkg0@mail.gmail.com>	<loom.20101211T072018-194@post.gmane.org>	<AANLkTikbjCwVkjAdU6CqDsso7AhHeU4i0uK3v-VnTFVE@mail.gmail.com>	<AANLkTi=BDNkDdiPJpH3AvtVcnOuc+EVtG9B8MLFyVac-@mail.gmail.com>
	<loom.20101212T183926-679@post.gmane.org>
Message-ID: <4D054DB2.8020204@netwok.org>

Hi,

I suggest to replace ?error? with ?event? in the module doc synopsis.

Regards


From v+python at g.nevcal.com  Mon Dec 13 01:42:49 2010
From: v+python at g.nevcal.com (Glenn Linderman)
Date: Sun, 12 Dec 2010 16:42:49 -0800
Subject: [Python-Dev] Using logging in the stdlib and its unit tests
In-Reply-To: <AANLkTim3i0oD9DkUwsQnE3y=zNPNhEM+GOMKL2-4cz+q@mail.gmail.com>
References: <loom.20101207T212306-892@post.gmane.org>	<20101207215053.025ecf52@pitrou.net>	<loom.20101208T002100-91@post.gmane.org>	<20101208010913.363c6105@pitrou.net>	<AANLkTim4ZPoch18+-qkunuYm-zcyh-vbhMOv9YVVEqo9@mail.gmail.com>	<loom.20101208T022102-46@post.gmane.org>	<AANLkTikFNUPA5Cvbz8cOAfznSOEe906O8zOfOC=2Cj+H@mail.gmail.com>	<loom.20101210T085332-877@post.gmane.org>	<4D01E55B.2050507@g.nevcal.com>	<3231.1292004978@parc.com>	<loom.20101210T202041-859@post.gmane.org>	<AANLkTi=PnWE5MbpyWKZHJU4Abk-OeVfKEJYpGX1E6g3=@mail.gmail.com>	<20101210214919.4ac63d27@pitrou.net>	<4D029439.7040708@g.nevcal.com>
	<AANLkTim3i0oD9DkUwsQnE3y=zNPNhEM+GOMKL2-4cz+q@mail.gmail.com>
Message-ID: <4D056C09.3090104@g.nevcal.com>

On 12/12/2010 2:26 PM, Paul Moore wrote:
> The thing*I*  hit very early was wanting to add a command lime option
> to my script to set the logging level. I'd have liked to be able to
> add --log=INFO/DEBUG/... but to do that I seem to need to write my own
> mapping between level names and numbers. A simple example of how to
> tie command line options to logging config would be a great addition
> to the documentation.

Oh?

import * from logger # change some details to avoid this
basicConfig( level= eval( opt.loglevel )
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20101212/d632f9e0/attachment.html>

From v+python at g.nevcal.com  Mon Dec 13 01:53:05 2010
From: v+python at g.nevcal.com (Glenn Linderman)
Date: Sun, 12 Dec 2010 16:53:05 -0800
Subject: [Python-Dev] Using logging in the stdlib and its unit tests
In-Reply-To: <loom.20101212T183926-679@post.gmane.org>
References: <loom.20101207T212306-892@post.gmane.org>	<20101207215053.025ecf52@pitrou.net>	<loom.20101208T002100-91@post.gmane.org>	<20101208010913.363c6105@pitrou.net>	<AANLkTim4ZPoch18+-qkunuYm-zcyh-vbhMOv9YVVEqo9@mail.gmail.com>	<loom.20101208T022102-46@post.gmane.org>	<AANLkTikFNUPA5Cvbz8cOAfznSOEe906O8zOfOC=2Cj+H@mail.gmail.com>	<loom.20101210T085332-877@post.gmane.org>	<4D01E55B.2050507@g.nevcal.com>
	<3231.1292004978@parc.com>	<loom.20101210T202041-859@post.gmane.org>	<AANLkTi=PnWE5MbpyWKZHJU4Abk-OeVfKEJYpGX1E6g3=@mail.gmail.com>	<20101210214919.4ac63d27@pitrou.net>	<loom.20101210T224910-511@post.gmane.org>	<iduoeb$q36$1@dough.gmane.org>	<AANLkTi=Q8mZb0oDe5xz7Evb0y8ohgkWh=X7E1OgCrUka@mail.gmail.com>	<AANLkTikN_378ZH2kCW-WJ2wWDVZrYb2zrQ9N1QMRvkg0@mail.gmail.com>	<loom.20101211T072018-194@post.gmane.org>	<AANLkTikbjCwVkjAdU6CqDsso7AhHeU4i0uK3v-VnTFVE@mail.gmail.com>	<AANLkTi=BDNkDdiPJpH3AvtVcnOuc+EVtG9B8MLFyVac-@mail.gmail.com>
	<loom.20101212T183926-679@post.gmane.org>
Message-ID: <4D056E71.3020704@g.nevcal.com>

On 12/12/2010 9:41 AM, Vinay Sajip wrote:
> Gosh, Nick, that was fast! I'm still making changes, but thanks for spotting and
> highlighting the typos and omissions. I've just checked in a further update;
> hopefully it'll get built soon so we can all see the latest changes.

I'm not as fast as Nick, but let me add that these changes to the 
documentation are surely helpful to me.

I've read 12% now, of a bigger base, but it was very approachable, and 
I've come away with being ready to scrap my little logger, I know what I 
need to do to make my multi-module logging work with the logging module 
instead, to greater benefit than my little logger, and the only 
"advanced technique" that I think I need to learn at the moment is 
formatters, so next chance I get I'll read about those.

The mountain doesn't look as steep, now!  Thanks for the fast reaction time.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20101212/93ab7bd4/attachment.html>

From robert.kern at gmail.com  Mon Dec 13 02:22:47 2010
From: robert.kern at gmail.com (Robert Kern)
Date: Sun, 12 Dec 2010 19:22:47 -0600
Subject: [Python-Dev] Using logging in the stdlib and its unit tests
In-Reply-To: <4D056C09.3090104@g.nevcal.com>
References: <loom.20101207T212306-892@post.gmane.org>	<20101207215053.025ecf52@pitrou.net>	<loom.20101208T002100-91@post.gmane.org>	<20101208010913.363c6105@pitrou.net>	<AANLkTim4ZPoch18+-qkunuYm-zcyh-vbhMOv9YVVEqo9@mail.gmail.com>	<loom.20101208T022102-46@post.gmane.org>	<AANLkTikFNUPA5Cvbz8cOAfznSOEe906O8zOfOC=2Cj+H@mail.gmail.com>	<loom.20101210T085332-877@post.gmane.org>	<4D01E55B.2050507@g.nevcal.com>	<3231.1292004978@parc.com>	<loom.20101210T202041-859@post.gmane.org>	<AANLkTi=PnWE5MbpyWKZHJU4Abk-OeVfKEJYpGX1E6g3=@mail.gmail.com>	<20101210214919.4ac63d27@pitrou.net>	<4D029439.7040708@g.nevcal.com>	<AANLkTim3i0oD9DkUwsQnE3y=zNPNhEM+GOMKL2-4cz+q@mail.gmail.com>
	<4D056C09.3090104@g.nevcal.com>
Message-ID: <ie3sh7$5r9$1@dough.gmane.org>

On 2010-12-12 18:42 , Glenn Linderman wrote:
> On 12/12/2010 2:26 PM, Paul Moore wrote:
>> The thing*I*  hit very early was wanting to add a command lime option
>> to my script to set the logging level. I'd have liked to be able to
>> add --log=INFO/DEBUG/... but to do that I seem to need to write my own
>> mapping between level names and numbers. A simple example of how to
>> tie command line options to logging config would be a great addition
>> to the documentation.
>
> Oh?
>
> import * from logger # change some details to avoid this
> basicConfig( level= eval( opt.loglevel )

level = getattr(logging, opt.logLevel)

or

level = logging._levelNames[opt.logLevel]

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
  that is made terrible by our own mad attempt to interpret it as though it had
  an underlying truth."
   -- Umberto Eco


From ncoghlan at gmail.com  Mon Dec 13 04:30:24 2010
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Mon, 13 Dec 2010 13:30:24 +1000
Subject: [Python-Dev] Using logging in the stdlib and its unit tests
In-Reply-To: <ie3sh7$5r9$1@dough.gmane.org>
References: <loom.20101207T212306-892@post.gmane.org>
	<20101207215053.025ecf52@pitrou.net>
	<loom.20101208T002100-91@post.gmane.org>
	<20101208010913.363c6105@pitrou.net>
	<AANLkTim4ZPoch18+-qkunuYm-zcyh-vbhMOv9YVVEqo9@mail.gmail.com>
	<loom.20101208T022102-46@post.gmane.org>
	<AANLkTikFNUPA5Cvbz8cOAfznSOEe906O8zOfOC=2Cj+H@mail.gmail.com>
	<loom.20101210T085332-877@post.gmane.org>
	<4D01E55B.2050507@g.nevcal.com> <3231.1292004978@parc.com>
	<loom.20101210T202041-859@post.gmane.org>
	<AANLkTi=PnWE5MbpyWKZHJU4Abk-OeVfKEJYpGX1E6g3=@mail.gmail.com>
	<20101210214919.4ac63d27@pitrou.net>
	<4D029439.7040708@g.nevcal.com>
	<AANLkTim3i0oD9DkUwsQnE3y=zNPNhEM+GOMKL2-4cz+q@mail.gmail.com>
	<4D056C09.3090104@g.nevcal.com> <ie3sh7$5r9$1@dough.gmane.org>
Message-ID: <AANLkTi=sg2V5+3M4HstShC3BpbEAzSKzDdSNStnVYFuy@mail.gmail.com>

On Mon, Dec 13, 2010 at 11:22 AM, Robert Kern <robert.kern at gmail.com> wrote:
> level = getattr(logging, opt.logLevel)

While this is the approach I would recommend, it does have a couple of
downsides:

1. An upper() call is also needed to allow strings like "info" instead
of "INFO":
2. If an integer is available, it would be nice to return it
unmodified (or, if we ever get named values in the standard library,
convert it to that)
3. The asymmetry with "logging.getLevelName" grates a bit

So it would be far more obvious if there was a "logging.getLevel"
counterpart to "logging.getLevelName" that looked something like:

def getLevel(level):
  try:
    return operator.index(level) # Integers and equivalents
  except TypeError:
    pass
  try:
    key = level.upper()
  except Exception as ex:
    raise TypeError("Log level must be an integer or string") from ex
  return globals()[key]


> level = logging._levelNames[opt.logLevel]

That doesn't work (_levelNames maps from integers to strings, we want
the mapping from strings to integers and it is only the module globals
that provides that).

Regards,
Nick.

-- 
Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia

From ncoghlan at gmail.com  Mon Dec 13 04:45:50 2010
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Mon, 13 Dec 2010 13:45:50 +1000
Subject: [Python-Dev] [Python-checkins] r87202 -
	python/branches/py3k/Doc/library/logging.rst
In-Reply-To: <20101212224535.C828DEE995@mail.python.org>
References: <20101212224535.C828DEE995@mail.python.org>
Message-ID: <AANLkTi=Cavk5a+Nwn+2mLMU2nsPLHq_OFQX-y2tSbvpp@mail.gmail.com>

On Mon, Dec 13, 2010 at 8:45 AM, vinay.sajip <python-checkins at python.org> wrote:
> +to get the value which you'll pass to :func:`basicConfig` via the *level*
> +argument. You may want to error check any user input value, perhaps as in the
> +following example::
> +
> + ? # assuming loglevel is bound to the string value obtained from the
> + ? # command line argument. Convert to upper case to allow the user to
> + ? # specify --log=DEBUG or --log=debug
> + ? numeric_level = getattr(logging, loglevel.upper(), None)
> + ? assert numeric_level is not None, 'Invalid log level: %s' % loglevel
> + ? logging.basicConfig(level=numeric_level, ...)

Minor nit - using asserts to check user input is generally a bad idea.
A more explicit check might be a better example:

if not isinstance(numeric_level, int):
   raise ValueError('Invalid log level: %s' % loglevel)

This also covers the case where someone does something weird like pass
in "basic_format" as a logging level.

Cheers,
Nick.

-- 
Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia

From robert.kern at gmail.com  Mon Dec 13 05:13:12 2010
From: robert.kern at gmail.com (Robert Kern)
Date: Sun, 12 Dec 2010 22:13:12 -0600
Subject: [Python-Dev] Using logging in the stdlib and its unit tests
In-Reply-To: <AANLkTi=sg2V5+3M4HstShC3BpbEAzSKzDdSNStnVYFuy@mail.gmail.com>
References: <loom.20101207T212306-892@post.gmane.org>	<20101207215053.025ecf52@pitrou.net>	<loom.20101208T002100-91@post.gmane.org>	<20101208010913.363c6105@pitrou.net>	<AANLkTim4ZPoch18+-qkunuYm-zcyh-vbhMOv9YVVEqo9@mail.gmail.com>	<loom.20101208T022102-46@post.gmane.org>	<AANLkTikFNUPA5Cvbz8cOAfznSOEe906O8zOfOC=2Cj+H@mail.gmail.com>	<loom.20101210T085332-877@post.gmane.org>	<4D01E55B.2050507@g.nevcal.com>
	<3231.1292004978@parc.com>	<loom.20101210T202041-859@post.gmane.org>	<AANLkTi=PnWE5MbpyWKZHJU4Abk-OeVfKEJYpGX1E6g3=@mail.gmail.com>	<20101210214919.4ac63d27@pitrou.net>	<4D029439.7040708@g.nevcal.com>	<AANLkTim3i0oD9DkUwsQnE3y=zNPNhEM+GOMKL2-4cz+q@mail.gmail.com>	<4D056C09.3090104@g.nevcal.com>
	<ie3sh7$5r9$1@dough.gmane.org>
	<AANLkTi=sg2V5+3M4HstShC3BpbEAzSKzDdSNStnVYFuy@mail.gmail.com>
Message-ID: <ie46gp$5ns$1@dough.gmane.org>

On 2010-12-12 21:30 , Nick Coghlan wrote:
> On Mon, Dec 13, 2010 at 11:22 AM, Robert Kern<robert.kern at gmail.com>  wrote:
>> level = getattr(logging, opt.logLevel)
>
> While this is the approach I would recommend, it does have a couple of
> downsides:
>
> 1. An upper() call is also needed to allow strings like "info" instead
> of "INFO":
> 2. If an integer is available, it would be nice to return it
> unmodified (or, if we ever get named values in the standard library,
> convert it to that)
> 3. The asymmetry with "logging.getLevelName" grates a bit
>
> So it would be far more obvious if there was a "logging.getLevel"
> counterpart to "logging.getLevelName" that looked something like:
>
> def getLevel(level):
>    try:
>      return operator.index(level) # Integers and equivalents
>    except TypeError:
>      pass
>    try:
>      key = level.upper()
>    except Exception as ex:
>      raise TypeError("Log level must be an integer or string") from ex
>    return globals()[key]

I don't think that the implementation should use globals(). I wouldn't want 
logging.getLevel('basic_format') to work. Instead, it should look it up in the 
known set of levels.

>> level = logging._levelNames[opt.logLevel]
>
> That doesn't work (_levelNames maps from integers to strings, we want
> the mapping from strings to integers and it is only the module globals
> that provides that).

At least in Python 2.6, it maps both ways. But then again, it is an 
_implementation _detail that should not be relied upon in your programs. I would 
suggest that there should be two dictionaries as part of the documented API, one 
mapping numbers to names and one mapping names to numbers. Or functions/methods 
returning said dictionaries. Having the entire mappings at hand is more useful 
than having functions that do the translation. They would allow you to 
auto-generate UIs (e.g. the help text for a --log-level argument or a radio box 
widget in a GUI). Having separate mappings makes them easier to work with than 
the 2.6-style _levelNames mapping.

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
  that is made terrible by our own mad attempt to interpret it as though it had
  an underlying truth."
   -- Umberto Eco


From ncoghlan at gmail.com  Mon Dec 13 05:27:21 2010
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Mon, 13 Dec 2010 14:27:21 +1000
Subject: [Python-Dev] Using logging in the stdlib and its unit tests
In-Reply-To: <ie46gp$5ns$1@dough.gmane.org>
References: <loom.20101207T212306-892@post.gmane.org>
	<20101207215053.025ecf52@pitrou.net>
	<loom.20101208T002100-91@post.gmane.org>
	<20101208010913.363c6105@pitrou.net>
	<AANLkTim4ZPoch18+-qkunuYm-zcyh-vbhMOv9YVVEqo9@mail.gmail.com>
	<loom.20101208T022102-46@post.gmane.org>
	<AANLkTikFNUPA5Cvbz8cOAfznSOEe906O8zOfOC=2Cj+H@mail.gmail.com>
	<loom.20101210T085332-877@post.gmane.org>
	<4D01E55B.2050507@g.nevcal.com> <3231.1292004978@parc.com>
	<loom.20101210T202041-859@post.gmane.org>
	<AANLkTi=PnWE5MbpyWKZHJU4Abk-OeVfKEJYpGX1E6g3=@mail.gmail.com>
	<20101210214919.4ac63d27@pitrou.net>
	<4D029439.7040708@g.nevcal.com>
	<AANLkTim3i0oD9DkUwsQnE3y=zNPNhEM+GOMKL2-4cz+q@mail.gmail.com>
	<4D056C09.3090104@g.nevcal.com> <ie3sh7$5r9$1@dough.gmane.org>
	<AANLkTi=sg2V5+3M4HstShC3BpbEAzSKzDdSNStnVYFuy@mail.gmail.com>
	<ie46gp$5ns$1@dough.gmane.org>
Message-ID: <AANLkTi=0oZs-qOJ8JGfthX7zQ51U06zAZ0rawiGxH08n@mail.gmail.com>

On Mon, Dec 13, 2010 at 2:13 PM, Robert Kern <robert.kern at gmail.com> wrote:
>>> level = logging._levelNames[opt.logLevel]
>>
>> That doesn't work (_levelNames maps from integers to strings, we want
>> the mapping from strings to integers and it is only the module globals
>> that provides that).
>
> At least in Python 2.6, it maps both ways. But then again, it is an
> _implementation _detail that should not be relied upon in your programs.

Ah, you're quite right - I didn't notice that when looking at the
contents (the first entries happened to map levels to names)

> I
> would suggest that there should be two dictionaries as part of the
> documented API, one mapping numbers to names and one mapping names to
> numbers. Or functions/methods returning said dictionaries. Having the entire
> mappings at hand is more useful than having functions that do the
> translation. They would allow you to auto-generate UIs (e.g. the help text
> for a --log-level argument or a radio box widget in a GUI). Having separate
> mappings makes them easier to work with than the 2.6-style _levelNames
> mapping.

Definitely something worth considering for 3.3.

Cheers,
Nick.

-- 
Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia

From rich at noir.com  Fri Dec 10 21:20:13 2010
From: rich at noir.com (K. Richard Pixley)
Date: Fri, 10 Dec 2010 12:20:13 -0800
Subject: [Python-Dev] logging doc broken
Message-ID: <4D028B7D.9030308@noir.com>

I'm not sure where to report this but the online doc appears to be 
mismatched to python-2.6.5 for the logging module.

Specifically, for a dir of an instance of a LogRecord, I'm seeing:

['__doc__', '__init__', '__module__', '__str__', 'args', 'created', 
'exc_info', 'exc_text', 'filename', 'funcName', 'getMessage', 
'levelname', 'levelno', 'lineno', 'module', 'msecs', 'msg', 'name', 
'pathname', 'process', 'processName', 'relativeCreated', 'thread', 
'threadName']

while the documentation lists a different set of attributes including "lvl".

--rich

From ncoghlan at gmail.com  Mon Dec 13 11:21:18 2010
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Mon, 13 Dec 2010 20:21:18 +1000
Subject: [Python-Dev] logging doc broken
In-Reply-To: <4D028B7D.9030308@noir.com>
References: <4D028B7D.9030308@noir.com>
Message-ID: <AANLkTi=H=QyLOJUYq0_6vGwvAVA2H4-dGYf2-DHFmdz4@mail.gmail.com>

On Sat, Dec 11, 2010 at 6:20 AM, K. Richard Pixley <rich at noir.com> wrote:
> I'm not sure where to report this but the online doc appears to be
> mismatched to python-2.6.5 for the logging module.
>
> Specifically, for a dir of an instance of a LogRecord, I'm seeing:
>
> ['__doc__', '__init__', '__module__', '__str__', 'args', 'created',
> 'exc_info', 'exc_text', 'filename', 'funcName', 'getMessage', 'levelname',
> 'levelno', 'lineno', 'module', 'msecs', 'msg', 'name', 'pathname',
> 'process', 'processName', 'relativeCreated', 'thread', 'threadName']
>
> while the documentation lists a different set of attributes including "lvl".

As discussed on this list recently, the logging documentation can
unfortunately give the impression that the attributes of the log
record are the same as the arguments to the log record constructor.
This is not the case - the actual list of attributes can be found in
the table showing the useful LogRecord attributes that are available
to formatters (and filters) when processing records:
http://docs.python.org/library/logging#formatter-objects

Vinay is currently working on updates to the logging documentation,
and this is one of the things that is likely to change (with the table
of attributes moved to the LogRecord section and referenced from the
sections on Formatter and Filter objects).

Regards,
Nick.

-- 
Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia

From vinay_sajip at yahoo.co.uk  Mon Dec 13 11:31:35 2010
From: vinay_sajip at yahoo.co.uk (Vinay Sajip)
Date: Mon, 13 Dec 2010 10:31:35 +0000 (UTC)
Subject: [Python-Dev] logging doc broken
References: <4D028B7D.9030308@noir.com>
Message-ID: <loom.20101213T112956-97@post.gmane.org>

K. Richard Pixley <rich <at> noir.com> writes:

> 
> I'm not sure where to report this but the online doc appears to be 
> mismatched to python-2.6.5 for the logging module.
> 
> Specifically, for a dir of an instance of a LogRecord, I'm seeing:
> 
> ['__doc__', '__init__', '__module__', '__str__', 'args', 'created', 
> 'exc_info', 'exc_text', 'filename', 'funcName', 'getMessage', 
> 'levelname', 'levelno', 'lineno', 'module', 'msecs', 'msg', 'name', 
> 'pathname', 'process', 'processName', 'relativeCreated', 'thread', 
> 'threadName']
> 
> while the documentation lists a different set of attributes including "lvl".
> 

Please report this on bugs.python.org. Check that you were referring to the 2.6
documentation - I didn't see any references to attributes there (following a
quick scan). Ideally, link to the wrong doc section on docs.python.org in your
bug report.

Thanks,

Vinay Sajip


From mail at timgolden.me.uk  Mon Dec 13 11:35:08 2010
From: mail at timgolden.me.uk (Tim Golden)
Date: Mon, 13 Dec 2010 10:35:08 +0000
Subject: [Python-Dev] logging doc broken
In-Reply-To: <loom.20101213T112956-97@post.gmane.org>
References: <4D028B7D.9030308@noir.com>
	<loom.20101213T112956-97@post.gmane.org>
Message-ID: <4D05F6DC.5040900@timgolden.me.uk>

On 13/12/2010 10:31, Vinay Sajip wrote:
[...]
> Ideally, link to the wrong doc section on docs.python.org in your bug report.


Now that's not a piece of advice you see very often :)

TJG

From ncoghlan at gmail.com  Mon Dec 13 11:38:35 2010
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Mon, 13 Dec 2010 20:38:35 +1000
Subject: [Python-Dev] logging doc broken
In-Reply-To: <loom.20101213T112956-97@post.gmane.org>
References: <4D028B7D.9030308@noir.com>
	<loom.20101213T112956-97@post.gmane.org>
Message-ID: <AANLkTinyeHJqy6x82Zx7HhhK8yaGfde+n20odrQHw1pq@mail.gmail.com>

On Mon, Dec 13, 2010 at 8:31 PM, Vinay Sajip <vinay_sajip at yahoo.co.uk> wrote:
> Please report this on bugs.python.org. Check that you were referring to the 2.6
> documentation - I didn't see any references to attributes there (following a
> quick scan). Ideally, link to the wrong doc section on docs.python.org in your
> bug report.

The reference to "lvl" makes me think this is the same mistake I made
the other day (i.e. thinking the constructor parameters are also the
LogRecord attributes)

Cheers,
Nick.

-- 
Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia

From vinay_sajip at yahoo.co.uk  Mon Dec 13 11:58:32 2010
From: vinay_sajip at yahoo.co.uk (Vinay Sajip)
Date: Mon, 13 Dec 2010 10:58:32 +0000 (UTC)
Subject: [Python-Dev] logging doc broken
References: <4D028B7D.9030308@noir.com>
	<AANLkTi=H=QyLOJUYq0_6vGwvAVA2H4-dGYf2-DHFmdz4@mail.gmail.com>
Message-ID: <loom.20101213T114011-992@post.gmane.org>

Nick Coghlan <ncoghlan <at> gmail.com> writes:

> Vinay is currently working on updates to the logging documentation,
> and this is one of the things that is likely to change (with the table
> of attributes moved to the LogRecord section and referenced from the
> sections on Formatter and Filter objects).

Yes, and to compound matters, the LogRecord constructor documentation lists the
attribute values in the place you'd expect the parameters to appear, using the
doc markup for attributes. This I think is what causes the confusion, although
the markup is different to that used for parameters it's not clear at first
glance.

However, these changes are after 2.7 and 3.1: In 2.6 documentation the
parameters are named as in the source, but there's no reference in the
LogRecord reference documentation about any attributes of the LogRecord.

Perhaps it would be best to do the following:

- Add constructor params using normal parameter markup (:param:)
- Create a separate section "LogRecord Attributes" with a table containing attr
name, description, how to use with % formatting and how to use with
{}-formatting. There is some overlap here with the parameter documentation for
the LogRecord constructor, but I can put in a rider which explains that the
param names and attr names are slightly different. (I could kick myself now for
not being more scrupulous originally, but I'm not sure I can go back and change
the param names in the source code now because in theory, someone might be
constructing a LogRecord using LogRecord(**kwargs)).

- Link to the latter section from the Formatter, Filter sections.

Anyone see problems with this, or have a better suggestion?

Regards,

Vinay Sajip



From vinay_sajip at yahoo.co.uk  Mon Dec 13 12:04:26 2010
From: vinay_sajip at yahoo.co.uk (Vinay Sajip)
Date: Mon, 13 Dec 2010 11:04:26 +0000 (UTC)
Subject: [Python-Dev] logging doc broken
References: <4D028B7D.9030308@noir.com>
	<loom.20101213T112956-97@post.gmane.org>
	<4D05F6DC.5040900@timgolden.me.uk>
Message-ID: <loom.20101213T115957-410@post.gmane.org>

Tim Golden <mail <at> timgolden.me.uk> writes:

> 
> On 13/12/2010 10:31, Vinay Sajip wrote:
> [...]
> > Ideally, link to the wrong doc section on docs.python.org in your bug report.
> 
> Now that's not a piece of advice you see very often :)
> 

True, but this area changed after 2.6 was released (after even 2.7, IIRC), so I
want to be sure that if I'm going to change the doc sources on release26-maint,
I'm doing the right thing. The 2.6 docs seem consistent with the 2.6 code, and
while confusing (because the distinction between constructor param names and
attr names is not spelled out), I don't see 2.6 docs as broken (or at least, I
don't see where the breakage is).

Regards,

Vinay Sajip




From ncoghlan at gmail.com  Mon Dec 13 12:17:26 2010
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Mon, 13 Dec 2010 21:17:26 +1000
Subject: [Python-Dev] logging doc broken
In-Reply-To: <loom.20101213T114011-992@post.gmane.org>
References: <4D028B7D.9030308@noir.com>
	<AANLkTi=H=QyLOJUYq0_6vGwvAVA2H4-dGYf2-DHFmdz4@mail.gmail.com>
	<loom.20101213T114011-992@post.gmane.org>
Message-ID: <AANLkTikxfER39nmJcOusXOkV368PT3W+Rbj-ZdGUk9mF@mail.gmail.com>

On Mon, Dec 13, 2010 at 8:58 PM, Vinay Sajip <vinay_sajip at yahoo.co.uk> wrote:
> Nick Coghlan <ncoghlan <at> gmail.com> writes:
>
>> Vinay is currently working on updates to the logging documentation,
>> and this is one of the things that is likely to change (with the table
>> of attributes moved to the LogRecord section and referenced from the
>> sections on Formatter and Filter objects).
>
> Yes, and to compound matters, the LogRecord constructor documentation lists the
> attribute values in the place you'd expect the parameters to appear, using the
> doc markup for attributes. This I think is what causes the confusion, although
> the markup is different to that used for parameters it's not clear at first
> glance.
>
> However, these changes are after 2.7 and 3.1: In 2.6 documentation the
> parameters are named as in the source, but there's no reference in the
> LogRecord reference documentation about any attributes of the LogRecord.

That's the problem though - if you don't read the intro paragraph
closely (which I didn't do), it's easy to jump to the conclusion that
those are also the attributes (since there is nothing else in the
section).

> Perhaps it would be best to do the following:
>
> - Add constructor params using normal parameter markup (:param:)
> - Create a separate section "LogRecord Attributes" with a table containing attr
> name, description, how to use with % formatting and how to use with
> {}-formatting. There is some overlap here with the parameter documentation for
> the LogRecord constructor, but I can put in a rider which explains that the
> param names and attr names are slightly different. (I could kick myself now for
> not being more scrupulous originally, but I'm not sure I can go back and change
> the param names in the source code now because in theory, someone might be
> constructing a LogRecord using LogRecord(**kwargs)).
>
> - Link to the latter section from the Formatter, Filter sections.
>
> Anyone see problems with this, or have a better suggestion?

Yep, that's what I had assumed you were going to do (this did come up
in the big thread about the logging docs, but you may have missed it).

You're also right about the backwards compatibility problem with
changing the parameter names. It's the one downside of keyword
arguments - it makes the parameter names part of a function's public
API, so they are much harder to fix when you make a mistake.

That said, you can't get a one-to-one correspondence in this case
anyway, since some of the parameters relate to multiple attributes
(e.g. "lvl" becomes both "levelNo" and "levelName").

It is rare that anyone will be constructing a LogRecord manually
though, so I don't think the parameter names matter all that much in
practice. It is far more common that people will be creating them
implicitly through the Logger event notification methods or by
deserialising them.

Cheers,
Nick.

-- 
Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia

From ncoghlan at gmail.com  Mon Dec 13 12:21:36 2010
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Mon, 13 Dec 2010 21:21:36 +1000
Subject: [Python-Dev] logging doc broken
In-Reply-To: <loom.20101213T115957-410@post.gmane.org>
References: <4D028B7D.9030308@noir.com>
	<loom.20101213T112956-97@post.gmane.org>
	<4D05F6DC.5040900@timgolden.me.uk>
	<loom.20101213T115957-410@post.gmane.org>
Message-ID: <AANLkTi=rhFJADqZpy9GTvojyPjm=98sdwoeRn3iu8Sx-@mail.gmail.com>

On Mon, Dec 13, 2010 at 9:04 PM, Vinay Sajip <vinay_sajip at yahoo.co.uk> wrote:
> Tim Golden <mail <at> timgolden.me.uk> writes:
>
>>
>> On 13/12/2010 10:31, Vinay Sajip wrote:
>> [...]
>> > Ideally, link to the wrong doc section on docs.python.org in your bug report.
>>
>> Now that's not a piece of advice you see very often :)
>>
>
> True, but this area changed after 2.6 was released (after even 2.7, IIRC), so I
> want to be sure that if I'm going to change the doc sources on release26-maint,
> I'm doing the right thing.

It was more a comment on the fact that, at first glance, that sentence
looks like an instuction to provide an incorrect link, rather than to
provide a link to the section of the docs that is incorrect. Just a
quirk of English grammar :)

Cheers,
Nick.

-- 
Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia

From zuo at chopin.edu.pl  Mon Dec 13 12:44:49 2010
From: zuo at chopin.edu.pl (Jan Kaliszewski)
Date: Mon, 13 Dec 2010 12:44:49 +0100
Subject: [Python-Dev] A grammatical oddity: trailing commas in argument
 lists -- continuation
Message-ID: <20101213114449.GA2348@chopin.edu.pl>

Dear Python Developers,

It is s my first post to python-dev, so let me introduce myself briefly:
Jan Kaliszewski, programmer and composer, sometimes also NGO activist.

Coming to the matter... The discussion started with remark by Mark
Dickinson about such a syntax oddity:

> def f(a, b,): ...     is fine, but
> def f(*, a, b,): ...  is a SyntaxError

Then some other similar oddities were pointed at (*args/**kwargs-related
ones as well as calls like f(*, a=3,) causing SyntaxError too).

References:
* http://mail.python.org/pipermail/python-dev/2010-July/101636.html
* http://bugs.python.org/issue9232
* http://bugs.python.org/issue10682

But yesterday both mentioned issues has been closed as rejected -- with
suggestion that it would probably require a PEP to modify Python in this
aspect (as there is no clear consensus). So I'd opt for re-opening the
discussion -- I suppose that more people could be interested in solving
the issue (at least after the end of PEP 3003 moratorium period).

I think that seeing that:

    def f(a, b): ...
    def f(a, b,): ...
    def f(a, *, b): ...
    def f(a, *args, b): ...
    x(1, 2, 3, 4, z=5)
    x(1, 2, 3, 4, z=5,)
    x(1, *(2,3,4), z=5)

...are ok, then --

    def f(a, *, b,): ...
    def f(a, *args, b,): ...
    x(1, *(2,3,4), z=5,): ...

...should be ok as well, and consequently --

    def f(a, *args,): ...
    def f(a, **kwargs,): ...
    x(1, *(2,3,4),)
    x(1, **dict(z=6),)

...should also be ok.

Please also note that Py3k's function annotations make one-def-argument-
-per-line formattig style the most suitable in some cases, e.g.:

    def my_func(
        spam:"Very tasty and nutritious piece of food",
        ham:"For experts only",
        *more_spam:"Not less tasty and not less nutritious!",
        spammish_inquisition:"Nobody expects this!",
    ) -> "Spam, spam, spam, spam, spam, spam, spam, spam, spam, spam":
        ...

Regards,

Jan Kaliszewski


From ncoghlan at gmail.com  Mon Dec 13 14:25:58 2010
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Mon, 13 Dec 2010 23:25:58 +1000
Subject: [Python-Dev] A grammatical oddity: trailing commas in argument
 lists -- continuation
In-Reply-To: <20101213114449.GA2348@chopin.edu.pl>
References: <20101213114449.GA2348@chopin.edu.pl>
Message-ID: <AANLkTi=OM921Xt7ZFv=E6-Gz3sLeLPYC4-BUGOv-f+-L@mail.gmail.com>

On Mon, Dec 13, 2010 at 9:44 PM, Jan Kaliszewski <zuo at chopin.edu.pl> wrote:
> I think that seeing that:
>
> ? ?def f(a, b): ...
> ? ?def f(a, *, b): ...
> ? ?def f(a, *args, b): ...
> ? ?x(1, 2, 3, 4, z=5)
> ? ?x(1, *(2,3,4), z=5)

As per the closure of the affected tickets, the likely outcome of such
a discussion would be the deprecation and subsequent removal of
support for the following two options:

    def f(a, b,): ...
    x(1, 2, 3, 4, z=5,): ...

Function arguments are not lists. Even when separated onto multiple
lines, the closing "):" should remain on the final line with other
content.

That would be a lot of hassle to get rid of something that people
probably aren't doing in the first place, though.

Regards,
Nick.

-- 
Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia

From mail at timgolden.me.uk  Mon Dec 13 14:29:12 2010
From: mail at timgolden.me.uk (Tim Golden)
Date: Mon, 13 Dec 2010 13:29:12 +0000
Subject: [Python-Dev] logging doc broken
In-Reply-To: <AANLkTi=rhFJADqZpy9GTvojyPjm=98sdwoeRn3iu8Sx-@mail.gmail.com>
References: <4D028B7D.9030308@noir.com>	<loom.20101213T112956-97@post.gmane.org>	<4D05F6DC.5040900@timgolden.me.uk>	<loom.20101213T115957-410@post.gmane.org>
	<AANLkTi=rhFJADqZpy9GTvojyPjm=98sdwoeRn3iu8Sx-@mail.gmail.com>
Message-ID: <4D061FA8.2000100@timgolden.me.uk>

On 13/12/2010 11:21, Nick Coghlan wrote:
> On Mon, Dec 13, 2010 at 9:04 PM, Vinay Sajip<vinay_sajip at yahoo.co.uk>  wrote:
>> Tim Golden<mail<at>  timgolden.me.uk>  writes:
>>
>>>
>>> On 13/12/2010 10:31, Vinay Sajip wrote:
>>> [...]
>>>> Ideally, link to the wrong doc section on docs.python.org in your bug report.
>>>
>>> Now that's not a piece of advice you see very often :)
>>>
>>
>> True, but this area changed after 2.6 was released (after even 2.7, IIRC), so I
>> want to be sure that if I'm going to change the doc sources on release26-maint,
>> I'm doing the right thing.
>
> It was more a comment on the fact that, at first glance, that sentence
> looks like an instuction to provide an incorrect link, rather than to
> provide a link to the section of the docs that is incorrect. Just a
> quirk of English grammar :)

Thanks, Nick. That is what I meant. I wanted to indicate that
it was tongue-in-cheek, but I never can remember the sequence
of characters which means that...

TJG

From fuzzyman at voidspace.org.uk  Mon Dec 13 14:39:44 2010
From: fuzzyman at voidspace.org.uk (Michael Foord)
Date: Mon, 13 Dec 2010 13:39:44 +0000
Subject: [Python-Dev] A grammatical oddity: trailing commas in argument
 lists -- continuation
In-Reply-To: <AANLkTi=OM921Xt7ZFv=E6-Gz3sLeLPYC4-BUGOv-f+-L@mail.gmail.com>
References: <20101213114449.GA2348@chopin.edu.pl>
	<AANLkTi=OM921Xt7ZFv=E6-Gz3sLeLPYC4-BUGOv-f+-L@mail.gmail.com>
Message-ID: <4D062220.8040501@voidspace.org.uk>

On 13/12/2010 13:25, Nick Coghlan wrote:
> On Mon, Dec 13, 2010 at 9:44 PM, Jan Kaliszewski<zuo at chopin.edu.pl>  wrote:
>> I think that seeing that:
>>
>>     def f(a, b): ...
>>     def f(a, *, b): ...
>>     def f(a, *args, b): ...
>>     x(1, 2, 3, 4, z=5)
>>     x(1, *(2,3,4), z=5)
> As per the closure of the affected tickets, the likely outcome of such
> a discussion would be the deprecation and subsequent removal of
> support for the following two options:
>
>      def f(a, b,): ...
>      x(1, 2, 3, 4, z=5,): ...
>
> Function arguments are not lists. Even when separated onto multiple
> lines, the closing "):" should remain on the final line with other
> content.

Why?

For very long signatures I still mildly prefer this:

def f(self, first, second, third, fourth,
foo=None, bar=None, baz=None,
spam=None, eggs=None, ham=None
):

Over putting the closing paren: on the last line of the def.

Of course not having such long signatures is even more preferable, but 
*sometimes* they are needed.

All the best,

Michael Foord
> That would be a lot of hassle to get rid of something that people
> probably aren't doing in the first place, though.
>
> Regards,
> Nick.
>


-- 

http://www.voidspace.org.uk/

READ CAREFULLY. By accepting and reading this email you agree,
on behalf of your employer, to release me from all obligations
and waivers arising from any and all NON-NEGOTIATED agreements,
licenses, terms-of-service, shrinkwrap, clickwrap, browsewrap,
confidentiality, non-disclosure, non-compete and acceptable use
policies (?BOGUS AGREEMENTS?) that I have entered into with your
employer, its partners, licensors, agents and assigns, in
perpetuity, without prejudice to my ongoing rights and privileges.
You further represent that you have the authority to release me
from any BOGUS AGREEMENTS on behalf of your employer.


From tseaver at palladion.com  Mon Dec 13 14:42:34 2010
From: tseaver at palladion.com (Tres Seaver)
Date: Mon, 13 Dec 2010 08:42:34 -0500
Subject: [Python-Dev] A grammatical oddity: trailing commas in argument
	lists -- continuation
In-Reply-To: <AANLkTi=OM921Xt7ZFv=E6-Gz3sLeLPYC4-BUGOv-f+-L@mail.gmail.com>
References: <20101213114449.GA2348@chopin.edu.pl>
	<AANLkTi=OM921Xt7ZFv=E6-Gz3sLeLPYC4-BUGOv-f+-L@mail.gmail.com>
Message-ID: <ie57s5$r0u$1@dough.gmane.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 12/13/2010 08:25 AM, Nick Coghlan wrote:
> On Mon, Dec 13, 2010 at 9:44 PM, Jan Kaliszewski <zuo at chopin.edu.pl> wrote:
>> I think that seeing that:
>>
>>    def f(a, b): ...
>>    def f(a, *, b): ...
>>    def f(a, *args, b): ...
>>    x(1, 2, 3, 4, z=5)
>>    x(1, *(2,3,4), z=5)
> 
> As per the closure of the affected tickets, the likely outcome of such
> a discussion would be the deprecation and subsequent removal of
> support for the following two options:
> 
>     def f(a, b,): ...
>     x(1, 2, 3, 4, z=5,): ...
> 
> Function arguments are not lists. Even when separated onto multiple
> lines, the closing "):" should remain on the final line with other
> content.
> 
> That would be a lot of hassle to get rid of something that people
> probably aren't doing in the first place, though.

I actually make use of the feature when dealing with APIs which both a)
take lots of arguments (more than fit comfortably on two lines at
whatever indentation they are called), and b) have optional trailing
arguments:  I always leave the trailing comma in place in such cases,
with the closing paren on the following line, so that adding or removing
an argument at the end of the list stays consistent (the diffs are
better, too, when I use this pattern).



Tres.
- -- 
===================================================================
Tres Seaver          +1 540-429-0999          tseaver at palladion.com
Palladion Software   "Excellence by Design"    http://palladion.com
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.10 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAk0GIsoACgkQ+gerLs4ltQ5HLwCfYFSyVrFtt04h6a39hyK6BD2c
t8oAoJdXNS7wIsjF34ZiOQCwQGq9Qs2v
=ZWqW
-----END PGP SIGNATURE-----


From ncoghlan at gmail.com  Mon Dec 13 14:54:28 2010
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Mon, 13 Dec 2010 23:54:28 +1000
Subject: [Python-Dev] A grammatical oddity: trailing commas in argument
 lists -- continuation
In-Reply-To: <ie57s5$r0u$1@dough.gmane.org>
References: <20101213114449.GA2348@chopin.edu.pl>
	<AANLkTi=OM921Xt7ZFv=E6-Gz3sLeLPYC4-BUGOv-f+-L@mail.gmail.com>
	<ie57s5$r0u$1@dough.gmane.org>
Message-ID: <AANLkTi=A12uchxgY8jihv2CgrEanRR_jtJmM8WCRfbXq@mail.gmail.com>

On Mon, Dec 13, 2010 at 11:42 PM, Tres Seaver <tseaver at palladion.com> wrote:
> I actually make use of the feature when dealing with APIs which both a)
> take lots of arguments (more than fit comfortably on two lines at
> whatever indentation they are called), and b) have optional trailing
> arguments: ?I always leave the trailing comma in place in such cases,
> with the closing paren on the following line, so that adding or removing
> an argument at the end of the list stays consistent (the diffs are
> better, too, when I use this pattern).

My personal preferences aren't strong either way, but the issues were
closed because committers voiced opinions against making this
consistent in the other direction (i.e. always allowing the trailing
comma).

I don't know that a full PEP is really needed, but the status quo is
that some committers said no and others don't really care about the
issue all that much, so the current behaviour is going to remain in
place unless those in the first group change their mind (or Guido
weighs in and says "change it"). Creating a PEP is one way to carry
out such persuasion (probably overkill though).

Cheers,
Nick.

-- 
Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia

From zuo at chopin.edu.pl  Mon Dec 13 15:18:34 2010
From: zuo at chopin.edu.pl (Jan Kaliszewski)
Date: Mon, 13 Dec 2010 15:18:34 +0100
Subject: [Python-Dev] A grammatical oddity: trailing commas in argument
 lists -- continuation
In-Reply-To: <AANLkTi=OM921Xt7ZFv=E6-Gz3sLeLPYC4-BUGOv-f+-L@mail.gmail.com>
References: <20101213114449.GA2348@chopin.edu.pl>
	<AANLkTi=OM921Xt7ZFv=E6-Gz3sLeLPYC4-BUGOv-f+-L@mail.gmail.com>
Message-ID: <20101213141834.GA5056@chopin.edu.pl>

Nick Coghlan dixit (2010-12-13, 23:25):

> Function arguments are not lists. Even when separated onto multiple
> lines, the closing "):" should remain on the final line with other
> content.

Not necessarily, IMHO.

1.
What about my example with '-> xxx' return-value annotation? (especially
when that annotation is a long expression)

2.
There are two argument-list-formatting idioms I apply -- depending on
which is more suitable in a particular case:

a) 
when argument specs/expressions are not very long and rather if their
number is not very big:

    def function(argument_spec1, argument_spec2, argument_spec3,
                 argument_spec4, argument_spec5, argument_spec6):

    function_call(expression1, expression2, expression3,
                  expression4, expression5, expression6)

b)
for long argument lists and/or argument specs/expressions (e.g. when
default values or argument annotations are defined as long expressions):

    def function(
        long_argument_spec1,
        long_argument_spec2,
        long_argument_spec3,
        long_argument_spec4,
        long_argument_spec5,
        long_argument_spec6,
    ):

    function_call(
        long_expression1,
        long_expression2,
        long_expression3,
        long_expression4,
        long_expression5,
        long_expression6,
    )

Note that option 'b' is more convenient for refactorization, diffs etc.

Regards,
*j


From vinay_sajip at yahoo.co.uk  Mon Dec 13 16:01:09 2010
From: vinay_sajip at yahoo.co.uk (Vinay Sajip)
Date: Mon, 13 Dec 2010 15:01:09 +0000 (UTC)
Subject: [Python-Dev] logging doc broken
References: <4D028B7D.9030308@noir.com>
	<AANLkTi=H=QyLOJUYq0_6vGwvAVA2H4-dGYf2-DHFmdz4@mail.gmail.com>
	<loom.20101213T114011-992@post.gmane.org>
	<AANLkTikxfER39nmJcOusXOkV368PT3W+Rbj-ZdGUk9mF@mail.gmail.com>
Message-ID: <loom.20101213T155701-528@post.gmane.org>

Nick Coghlan <ncoghlan <at> gmail.com> writes:

> Yep, that's what I had assumed you were going to do (this did come up
> in the big thread about the logging docs, but you may have missed it).

I hadn't missed it - I'm just spelling out in more detail what I'm going to do.
 

> That said, you can't get a one-to-one correspondence in this case
> anyway, since some of the parameters relate to multiple attributes
> (e.g. "lvl" becomes both "levelNo" and "levelName").
> 
> It is rare that anyone will be constructing a LogRecord manually
> though, so I don't think the parameter names matter all that much in
> practice. It is far more common that people will be creating them
> implicitly through the Logger event notification methods or by
> deserialising them.

I agree, it's a pretty unlikely scenario. Of course, I do make use of the
parameter names in the dictConfig approach, so I can't really complain. I should
hopefully remember this when I'm writing out a method or function signature for
a public API :-)

Regards,

Vinay



From vinay_sajip at yahoo.co.uk  Mon Dec 13 16:04:15 2010
From: vinay_sajip at yahoo.co.uk (Vinay Sajip)
Date: Mon, 13 Dec 2010 15:04:15 +0000 (UTC)
Subject: [Python-Dev] logging doc broken
References: <4D028B7D.9030308@noir.com>	<loom.20101213T112956-97@post.gmane.org>	<4D05F6DC.5040900@timgolden.me.uk>	<loom.20101213T115957-410@post.gmane.org>
	<AANLkTi=rhFJADqZpy9GTvojyPjm=98sdwoeRn3iu8Sx-@mail.gmail.com>
	<4D061FA8.2000100@timgolden.me.uk>
Message-ID: <loom.20101213T160127-938@post.gmane.org>

Tim Golden <mail <at> timgolden.me.uk> writes:


> > It was more a comment on the fact that, at first glance, that sentence
> > looks like an instuction to provide an incorrect link, rather than to
> > provide a link to the section of the docs that is incorrect. Just a
> > quirk of English grammar :)
> 
> Thanks, Nick. That is what I meant. I wanted to indicate that
> it was tongue-in-cheek, but I never can remember the sequence
> of characters which means that...
> 

Aaargh! You're right of course, I was too hasty in typing "link to the wrong doc
section" rather than the only slightly longer "link to the doc section which is
wrong".

Tim - when you find those emoticons, be sure to let me know the one for
"facepalm" ;-)

Regards,

Vinay



From rdmurray at bitdance.com  Mon Dec 13 16:51:19 2010
From: rdmurray at bitdance.com (R. David Murray)
Date: Mon, 13 Dec 2010 10:51:19 -0500
Subject: [Python-Dev] A grammatical oddity: trailing commas in argument
	lists -- continuation
In-Reply-To: <AANLkTi=OM921Xt7ZFv=E6-Gz3sLeLPYC4-BUGOv-f+-L@mail.gmail.com>
References: <20101213114449.GA2348@chopin.edu.pl>
	<AANLkTi=OM921Xt7ZFv=E6-Gz3sLeLPYC4-BUGOv-f+-L@mail.gmail.com>
Message-ID: <20101213155119.A52A621CD91@kimball.webabinitio.net>

On Mon, 13 Dec 2010 23:25:58 +1000, Nick Coghlan <ncoghlan at gmail.com> wrote:
> On Mon, Dec 13, 2010 at 9:44 PM, Jan Kaliszewski <zuo at chopin.edu.pl> wrote:
> > I think that seeing that:
> >
> > =A0 =A0def f(a, b): ...
> > =A0 =A0def f(a, *, b): ...
> > =A0 =A0def f(a, *args, b): ...
> > =A0 =A0x(1, 2, 3, 4, z=3D5)
> > =A0 =A0x(1, *(2,3,4), z=3D5)
> 
> As per the closure of the affected tickets, the likely outcome of such
> a discussion would be the deprecation and subsequent removal of
> support for the following two options:
> 
>     def f(a, b,): ...
>     x(1, 2, 3, 4, z=3D5,): ...
> 
> Function arguments are not lists. Even when separated onto multiple
> lines, the closing "):" should remain on the final line with other
> content.
> 
> That would be a lot of hassle to get rid of something that people
> probably aren't doing in the first place, though.

Counter examples from google code search:
    
    http://www.google.com/codesearch/p?hl=en#copo3dCwf5E/django/utils/simplejson/encoder.py&q=^\s*\):&sa=N&cd=5&ct=rc
        (also appears in json in the stdlib)
    http://www.google.com/codesearch/p?hl=en#algXCqBNNP0/vendor/python-clientform/ClientForm.py&q=^\ *\):&sa=N&cd=3&ct=rc
        (class def)
    http://www.google.com/codesearch/p?hl=en#KT-ZlRkUunU/trunk/code/output/ExprParser.py&q=def\(.*,\s\):&sa=N&cd=2&ct=rc
    http://www.google.com/codesearch/p?hl=en#XnG7n8Mjf2s/GoogleSearch.py&q=def\(.*,\s\):&sa=N&cd=3&ct=rc
    http://www.google.com/codesearch/p?hl=en#MokQ50OeeyU/src/python/ndogen/parser/matlab/parser.py&q=def\(.*,\s\):&sa=N&cd=5&ct=rc

Not many, granted, but not zero, either, and I'm sure there are lots
more out there[*].  I do especially like the fact that there is one in the
stdlib :)

It seems like the status quo is fine.  I wouldn't object to it being
made more consistent.  I would object to removing the existing cases.

--
R. David Murray                                      www.bitdance.com

[*] code search's response to various regexes was somewhat surprising;
expressions I thought should have been supersets resulted in fewer
hits.  Nor could I think of a way to search for function invocations
ending with a comma.  Then again, I usually make lots of mistakes
with regexes.

From guido at python.org  Mon Dec 13 17:54:16 2010
From: guido at python.org (Guido van Rossum)
Date: Mon, 13 Dec 2010 08:54:16 -0800
Subject: [Python-Dev] A grammatical oddity: trailing commas in argument
 lists -- continuation
In-Reply-To: <ie57s5$r0u$1@dough.gmane.org>
References: <20101213114449.GA2348@chopin.edu.pl>
	<AANLkTi=OM921Xt7ZFv=E6-Gz3sLeLPYC4-BUGOv-f+-L@mail.gmail.com>
	<ie57s5$r0u$1@dough.gmane.org>
Message-ID: <AANLkTi=NGOVL8kVKHef7Ox_xTkKO8K0vNG495Bzk3gYV@mail.gmail.com>

On Mon, Dec 13, 2010 at 5:42 AM, Tres Seaver <tseaver at palladion.com> wrote:
> I actually make use of the feature when dealing with APIs which both a)
> take lots of arguments (more than fit comfortably on two lines at
> whatever indentation they are called), and b) have optional trailing
> arguments: ?I always leave the trailing comma in place in such cases,
> with the closing paren on the following line, so that adding or removing
> an argument at the end of the list stays consistent (the diffs are
> better, too, when I use this pattern).

Same here, and it's a soft style rule at Google that trailing commas
are good -- they can help produce shorter diffs. I'm at least +0 on
allowing trailing commas in the situation the OP mentioned.

-- 
--Guido van Rossum (python.org/~guido)

From vinay_sajip at yahoo.co.uk  Mon Dec 13 19:55:12 2010
From: vinay_sajip at yahoo.co.uk (Vinay Sajip)
Date: Mon, 13 Dec 2010 18:55:12 +0000 (UTC)
Subject: [Python-Dev] logging doc broken
References: <4D028B7D.9030308@noir.com>
	<AANLkTi=H=QyLOJUYq0_6vGwvAVA2H4-dGYf2-DHFmdz4@mail.gmail.com>
	<loom.20101213T114011-992@post.gmane.org>
	<AANLkTikxfER39nmJcOusXOkV368PT3W+Rbj-ZdGUk9mF@mail.gmail.com>
Message-ID: <loom.20101213T195056-209@post.gmane.org>

Nick Coghlan <ncoghlan <at> gmail.com> writes:

> Yep, that's what I had assumed you were going to do (this did come up
> in the big thread about the logging docs, but you may have missed it).
> 

Ok, I've now checked in this change, and would be grateful for any feedback.
Time for a break :-)

Thanks a lot for all your patience and valuable feedback, Nick. It's been very
helpful, and I hope it will make the logging docs more accessible to those
who've been put off in the past. I've emailed Georg about how best to reorganise
into:

intro + basic tutorial (at current location)
advanced tutorial
reference
cookbook

and I'm waiting for his suggestions on how best to implement (in terms of
breaking up into different files etc).

Thanks and regards,

Vinay


From alexander.belopolsky at gmail.com  Mon Dec 13 20:09:02 2010
From: alexander.belopolsky at gmail.com (Alexander Belopolsky)
Date: Mon, 13 Dec 2010 14:09:02 -0500
Subject: [Python-Dev] A grammatical oddity: trailing commas in argument
 lists -- continuation
In-Reply-To: <AANLkTi=NGOVL8kVKHef7Ox_xTkKO8K0vNG495Bzk3gYV@mail.gmail.com>
References: <20101213114449.GA2348@chopin.edu.pl>
	<AANLkTi=OM921Xt7ZFv=E6-Gz3sLeLPYC4-BUGOv-f+-L@mail.gmail.com>
	<ie57s5$r0u$1@dough.gmane.org>
	<AANLkTi=NGOVL8kVKHef7Ox_xTkKO8K0vNG495Bzk3gYV@mail.gmail.com>
Message-ID: <AANLkTikacA1YPHPQ-chjibtH50P=bKkf-CrAoO-VZE=+@mail.gmail.com>

On Mon, Dec 13, 2010 at 11:54 AM, Guido van Rossum <guido at python.org> wrote:
> I'm at least +0 on
> allowing trailing commas in the situation the OP mentioned.
>

FWIW, I am also about +0.5 on allowing trailing comma.  Note that in a
similar situation,  the C standardization committee has erred on the
side of consistency:

"""
A new feature of C99: a common extension in many implementations
allows a trailing comma after the list of enumeration constants. The
Committee decided to adopt this feature as an innocuous extension that
mirrors the trailing commas allowed in initializers.
""" http://www.open-std.org/jtc1/sc22/wg14/www/C99RationaleV5.10.pdf

Similarly, I find allowing trailing comma in keyword only arguments
lists to be an innocuous extension that mirrors the trailing commas
allowed in the positional arguments lists.   A possible benefit that I
have not seen mentioned is that if developer decides to convert some
of the trailing arguments in her function to keyword only, she does
not have to worry about removing the trailing comma.

From solipsis at pitrou.net  Mon Dec 13 20:17:59 2010
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Mon, 13 Dec 2010 20:17:59 +0100
Subject: [Python-Dev] A grammatical oddity: trailing commas in argument
 lists -- continuation
References: <20101213114449.GA2348@chopin.edu.pl>
	<AANLkTi=OM921Xt7ZFv=E6-Gz3sLeLPYC4-BUGOv-f+-L@mail.gmail.com>
	<ie57s5$r0u$1@dough.gmane.org>
	<AANLkTi=NGOVL8kVKHef7Ox_xTkKO8K0vNG495Bzk3gYV@mail.gmail.com>
	<AANLkTikacA1YPHPQ-chjibtH50P=bKkf-CrAoO-VZE=+@mail.gmail.com>
Message-ID: <20101213201759.422b15de@pitrou.net>

On Mon, 13 Dec 2010 14:09:02 -0500
Alexander Belopolsky <alexander.belopolsky at gmail.com> wrote:

> On Mon, Dec 13, 2010 at 11:54 AM, Guido van Rossum <guido at python.org> wrote:
> > I'm at least +0 on
> > allowing trailing commas in the situation the OP mentioned.
> >
> 
> FWIW, I am also about +0.5 on allowing trailing comma.  Note that in a
> similar situation,  the C standardization committee has erred on the
> side of consistency:
> 
> """
> A new feature of C99: a common extension in many implementations
> allows a trailing comma after the list of enumeration constants. The
> Committee decided to adopt this feature as an innocuous extension that
> mirrors the trailing commas allowed in initializers.
> """ http://www.open-std.org/jtc1/sc22/wg14/www/C99RationaleV5.10.pdf
> 
> Similarly, I find allowing trailing comma in keyword only arguments
> lists to be an innocuous extension that mirrors the trailing commas
> allowed in the positional arguments lists.

+1 from me as well. Special cases are hard to remember.

Regards

Antoine.



From v+python at g.nevcal.com  Mon Dec 13 20:26:44 2010
From: v+python at g.nevcal.com (Glenn Linderman)
Date: Mon, 13 Dec 2010 11:26:44 -0800
Subject: [Python-Dev] A grammatical oddity: trailing commas in argument
 lists -- continuation
In-Reply-To: <20101213201759.422b15de@pitrou.net>
References: <20101213114449.GA2348@chopin.edu.pl>	<AANLkTi=OM921Xt7ZFv=E6-Gz3sLeLPYC4-BUGOv-f+-L@mail.gmail.com>	<ie57s5$r0u$1@dough.gmane.org>	<AANLkTi=NGOVL8kVKHef7Ox_xTkKO8K0vNG495Bzk3gYV@mail.gmail.com>	<AANLkTikacA1YPHPQ-chjibtH50P=bKkf-CrAoO-VZE=+@mail.gmail.com>
	<20101213201759.422b15de@pitrou.net>
Message-ID: <4D067374.8050200@g.nevcal.com>

On 12/13/2010 11:17 AM, Antoine Pitrou wrote:
> On Mon, 13 Dec 2010 14:09:02 -0500
> Alexander Belopolsky<alexander.belopolsky at gmail.com>  wrote:
>
>> On Mon, Dec 13, 2010 at 11:54 AM, Guido van Rossum<guido at python.org>  wrote:
>>> I'm at least +0 on
>>> allowing trailing commas in the situation the OP mentioned.
>>>
>> FWIW, I am also about +0.5 on allowing trailing comma.  Note that in a
>> similar situation,  the C standardization committee has erred on the
>> side of consistency:
>>
>> """
>> A new feature of C99: a common extension in many implementations
>> allows a trailing comma after the list of enumeration constants. The
>> Committee decided to adopt this feature as an innocuous extension that
>> mirrors the trailing commas allowed in initializers.
>> """ http://www.open-std.org/jtc1/sc22/wg14/www/C99RationaleV5.10.pdf
>>
>> Similarly, I find allowing trailing comma in keyword only arguments
>> lists to be an innocuous extension that mirrors the trailing commas
>> allowed in the positional arguments lists.
> +1 from me as well. Special cases are hard to remember.

+1.  I tend to put them in, and then take out the ones that Python won't 
accept.
Then, when I add another item to the list, Python tells me to go back 
and put the one I took out back in again, and take out the one I put in 
at the new end of the list.  Annoying.  (for vertically arranged lists, 
one per line, primarily, with ) on the last line by itself.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20101213/80c37037/attachment.html>

From tjreedy at udel.edu  Mon Dec 13 20:34:14 2010
From: tjreedy at udel.edu (Terry Reedy)
Date: Mon, 13 Dec 2010 14:34:14 -0500
Subject: [Python-Dev] logging doc broken
In-Reply-To: <loom.20101213T115957-410@post.gmane.org>
References: <4D028B7D.9030308@noir.com>	<loom.20101213T112956-97@post.gmane.org>	<4D05F6DC.5040900@timgolden.me.uk>
	<loom.20101213T115957-410@post.gmane.org>
Message-ID: <ie5sfk$9ro$1@dough.gmane.org>

On 12/13/2010 6:04 AM, Vinay Sajip wrote:

> True, but this area changed after 2.6 was released (after even 2.7, IIRC), so I
> want to be sure that if I'm going to change the doc sources on release26-maint,

2.6 is in security-fix only mode. I am not sure if that precludes doc 
fixes, on the chance that there will ever be a security fix, as it does 
non-security bug fixes, but is it hardly worth the bother compared to 
improving active releases.

-- 
Terry Jan Reedy


From dickinsm at gmail.com  Mon Dec 13 20:39:32 2010
From: dickinsm at gmail.com (Mark Dickinson)
Date: Mon, 13 Dec 2010 19:39:32 +0000
Subject: [Python-Dev] A grammatical oddity: trailing commas in argument
 lists -- continuation
In-Reply-To: <20101213155119.A52A621CD91@kimball.webabinitio.net>
References: <20101213114449.GA2348@chopin.edu.pl>
	<AANLkTi=OM921Xt7ZFv=E6-Gz3sLeLPYC4-BUGOv-f+-L@mail.gmail.com>
	<20101213155119.A52A621CD91@kimball.webabinitio.net>
Message-ID: <AANLkTimCQYd8Q6BJQPXtr3mo_Rk=MZi4iZBcUX5z4uTB@mail.gmail.com>

On Mon, Dec 13, 2010 at 3:51 PM, R. David Murray <rdmurray at bitdance.com> wrote:
> It seems like the status quo is fine. ?I wouldn't object to it being
> made more consistent. ?I would object to removing the existing cases.

Same here, on all three counts.  In one of the projects I'm currently
working on, we've settled on a style that does quite a lot of:

my_thing = Thing(
    foo = Foo(arg1, arg2, ...),
    bar = Bar(arg3, arg4, ...),
    ...
)

and I've found the trailing comma very convenient during refactoring
and API experimentation.  (There's still good fun to be had arguing
about the indentation of that closing parenthesis, though.)

Mar

From v+python at g.nevcal.com  Mon Dec 13 21:08:31 2010
From: v+python at g.nevcal.com (Glenn Linderman)
Date: Mon, 13 Dec 2010 12:08:31 -0800
Subject: [Python-Dev] A grammatical oddity: trailing commas in argument
 lists -- continuation
In-Reply-To: <AANLkTimCQYd8Q6BJQPXtr3mo_Rk=MZi4iZBcUX5z4uTB@mail.gmail.com>
References: <20101213114449.GA2348@chopin.edu.pl>	<AANLkTi=OM921Xt7ZFv=E6-Gz3sLeLPYC4-BUGOv-f+-L@mail.gmail.com>	<20101213155119.A52A621CD91@kimball.webabinitio.net>
	<AANLkTimCQYd8Q6BJQPXtr3mo_Rk=MZi4iZBcUX5z4uTB@mail.gmail.com>
Message-ID: <4D067D3F.8030206@g.nevcal.com>

On 12/13/2010 11:39 AM, Mark Dickinson wrote:
> my_thing = Thing(
>      foo = Foo(arg1, arg2, ...),
>      bar = Bar(arg3, arg4, ...),
>      ...
> )
>
> and I've found the trailing comma very convenient during refactoring
> and API experimentation.  (There's still good fun to be had arguing
> about the indentation of that closing parenthesis, though.)


Clearly it needs to be indented one level, because it is a continuation 
of the prior line, just like the foo and bar and ... lines are 
continuations and therefore indented.

I'd have argued differently for languages that use {} to delimit blocks.

Enjoy!

From cs at zip.com.au  Mon Dec 13 22:06:50 2010
From: cs at zip.com.au (Cameron Simpson)
Date: Tue, 14 Dec 2010 08:06:50 +1100
Subject: [Python-Dev] A grammatical oddity: trailing commas in argument
	lists --	continuation
In-Reply-To: <20101213201759.422b15de@pitrou.net>
References: <20101213201759.422b15de@pitrou.net>
Message-ID: <20101213210650.GA31778@cskk.homeip.net>

On 13Dec2010 20:17, Antoine Pitrou <solipsis at pitrou.net> wrote:
| On Mon, 13 Dec 2010 14:09:02 -0500
| Alexander Belopolsky <alexander.belopolsky at gmail.com> wrote:
| 
| > On Mon, Dec 13, 2010 at 11:54 AM, Guido van Rossum <guido at python.org> wrote:
| > > I'm at least +0 on
| > > allowing trailing commas in the situation the OP mentioned.
| > >
| > 
| > FWIW, I am also about +0.5 on allowing trailing comma.  Note that in a
| > similar situation,  the C standardization committee has erred on the
| > side of consistency:
| > 
| > """
| > A new feature of C99: a common extension in many implementations
| > allows a trailing comma after the list of enumeration constants. The
| > Committee decided to adopt this feature as an innocuous extension that
| > mirrors the trailing commas allowed in initializers.
| > """ http://www.open-std.org/jtc1/sc22/wg14/www/C99RationaleV5.10.pdf
| > 
| > Similarly, I find allowing trailing comma in keyword only arguments
| > lists to be an innocuous extension that mirrors the trailing commas
| > allowed in the positional arguments lists.
| 
| +1 from me as well. Special cases are hard to remember.

+1 again. Both the special cases and probably an example of diff
friendliness:

  x = f(a,
        b=3,
        ##c=4,  hacking at dev time
        d=5)

Cheers,
-- 
Cameron Simpson <cs at zip.com.au> DoD#743
http://www.cskk.ezoshosting.com/cs/

A Master is someone who started before you did. - Gary Zukav

From tjreedy at udel.edu  Mon Dec 13 22:21:20 2010
From: tjreedy at udel.edu (Terry Reedy)
Date: Mon, 13 Dec 2010 16:21:20 -0500
Subject: [Python-Dev] A grammatical oddity: trailing commas in argument
	lists -- continuation
In-Reply-To: <20101213201759.422b15de@pitrou.net>
References: <20101213114449.GA2348@chopin.edu.pl>	<AANLkTi=OM921Xt7ZFv=E6-Gz3sLeLPYC4-BUGOv-f+-L@mail.gmail.com>	<ie57s5$r0u$1@dough.gmane.org>	<AANLkTi=NGOVL8kVKHef7Ox_xTkKO8K0vNG495Bzk3gYV@mail.gmail.com>	<AANLkTikacA1YPHPQ-chjibtH50P=bKkf-CrAoO-VZE=+@mail.gmail.com>
	<20101213201759.422b15de@pitrou.net>
Message-ID: <ie62oc$9r9$1@dough.gmane.org>

On 12/13/2010 2:17 PM, Antoine Pitrou wrote:
> On Mon, 13 Dec 2010 14:09:02 -0500
> Alexander Belopolsky<alexander.belopolsky at gmail.com>  wrote:
>
>> On Mon, Dec 13, 2010 at 11:54 AM, Guido van Rossum<guido at python.org>  wrote:
>>> I'm at least +0 on
>>> allowing trailing commas in the situation the OP mentioned.
>>>
>>
>> FWIW, I am also about +0.5 on allowing trailing comma.  Note that in a
>> similar situation,  the C standardization committee has erred on the
>> side of consistency:
>>
>> """
>> A new feature of C99: a common extension in many implementations
>> allows a trailing comma after the list of enumeration constants. The
>> Committee decided to adopt this feature as an innocuous extension that
>> mirrors the trailing commas allowed in initializers.
>> """ http://www.open-std.org/jtc1/sc22/wg14/www/C99RationaleV5.10.pdf
>>
>> Similarly, I find allowing trailing comma in keyword only arguments
>> lists to be an innocuous extension that mirrors the trailing commas
>> allowed in the positional arguments lists.
>
> +1 from me as well. Special cases are hard to remember.

Same here. A strong +1 for a consistent rule (always or never allowed) 
with a +1 for always given others use case of one param/arg per line.

So I think the issues should be reopened.

-- 
Terry Jan Reedy


From raymond.hettinger at gmail.com  Mon Dec 13 22:55:17 2010
From: raymond.hettinger at gmail.com (Raymond Hettinger)
Date: Mon, 13 Dec 2010 13:55:17 -0800
Subject: [Python-Dev] A grammatical oddity: trailing commas in argument
	lists -- continuation
In-Reply-To: <ie62oc$9r9$1@dough.gmane.org>
References: <20101213114449.GA2348@chopin.edu.pl>	<AANLkTi=OM921Xt7ZFv=E6-Gz3sLeLPYC4-BUGOv-f+-L@mail.gmail.com>	<ie57s5$r0u$1@dough.gmane.org>	<AANLkTi=NGOVL8kVKHef7Ox_xTkKO8K0vNG495Bzk3gYV@mail.gmail.com>	<AANLkTikacA1YPHPQ-chjibtH50P=bKkf-CrAoO-VZE=+@mail.gmail.com>
	<20101213201759.422b15de@pitrou.net> <ie62oc$9r9$1@dough.gmane.org>
Message-ID: <0DE0FA27-5792-4017-A0B0-4B1AF109FA76@gmail.com>


On Dec 13, 2010, at 1:21 PM, Terry Reedy wrote:

> Same here. A strong +1 for a consistent rule (always or never allowed) with a +1 for always given others use case of one param/arg per line.



It seems to me that a trailing comma in an argument list is more likely to be a user error than a deliberate comma-for-the-future. 


Raymond



From guido at python.org  Mon Dec 13 23:16:58 2010
From: guido at python.org (Guido van Rossum)
Date: Mon, 13 Dec 2010 14:16:58 -0800
Subject: [Python-Dev] A grammatical oddity: trailing commas in argument
 lists -- continuation
In-Reply-To: <0DE0FA27-5792-4017-A0B0-4B1AF109FA76@gmail.com>
References: <20101213114449.GA2348@chopin.edu.pl>
	<AANLkTi=OM921Xt7ZFv=E6-Gz3sLeLPYC4-BUGOv-f+-L@mail.gmail.com>
	<ie57s5$r0u$1@dough.gmane.org>
	<AANLkTi=NGOVL8kVKHef7Ox_xTkKO8K0vNG495Bzk3gYV@mail.gmail.com>
	<AANLkTikacA1YPHPQ-chjibtH50P=bKkf-CrAoO-VZE=+@mail.gmail.com>
	<20101213201759.422b15de@pitrou.net> <ie62oc$9r9$1@dough.gmane.org>
	<0DE0FA27-5792-4017-A0B0-4B1AF109FA76@gmail.com>
Message-ID: <AANLkTi=kT=wCAvuzjxXCqqo6cyL==2u-YmNcZEdqNO0+@mail.gmail.com>

On Mon, Dec 13, 2010 at 1:55 PM, Raymond Hettinger
<raymond.hettinger at gmail.com> wrote:
>
> On Dec 13, 2010, at 1:21 PM, Terry Reedy wrote:
>
>> Same here. A strong +1 for a consistent rule (always or never allowed) with a +1 for always given others use case of one param/arg per line.
>
>
>
> It seems to me that a trailing comma in an argument list is more likely to be a user error than a deliberate comma-for-the-future.

Really? Have you observed this? Even if it was inserted by mistake, it
is harmless. Python has a long tradition of allowing redundant
trailing commas in comma-separated lists, and it is habit-forming.
That's the issue the OP had: he expected it to work in the one context
where it doesn't.

-- 
--Guido van Rossum (python.org/~guido)

From lukasz at langa.pl  Mon Dec 13 23:22:29 2010
From: lukasz at langa.pl (=?utf-8?Q?=C5=81ukasz_Langa?=)
Date: Mon, 13 Dec 2010 23:22:29 +0100
Subject: [Python-Dev] configparser 1.1 - one last bug fix needed
Message-ID: <4A6BBCF7-6491-48C3-A02A-84EEA299B1DB@langa.pl>

Hi there.
 
There's one last thing that needs to be done with configparser for 3.2.
Raymond, Fred, Michael and Georg already expressed their approval on that so
unless anybody finds a flaw in the idea expressed below, I'm going to make
the change for 3.2b2:

- the ConfigParser class will be removed

- the SafeConfigParser class will be renamed to ConfigParser

- 2to3 will rename SafeConfigParser classes to ConfigParser

- 2to3 will warn on the subtle behaviour change when ConfigParser classes
  are found

What's the difference?
----------------------

Both ConfigParser and SafeConfigParser implement interpolation, e.g.  option
values can contain special tokens similar to those implemented by Python's
string formatting: %(example)s. These tokens are replaced during get()
operations by values from respective keys (either from the current section
or from the DEFAULT section).

SafeConfigParser was originally introduced to fix a couple of ConfigParser
problems:

- when a token didn't match the %(name)s syntax, it was simply treated as
  a raw value. This caused configuration errors like %var or %(no_closing_s)
  to be missed.

- if someone actually wanted to store arbitrary strings in values, including
  Python formatting strings, there was no way to escape %(name)s in the
  configuration. The programmer had to know in advance that some value may
  hold %(name)s and only get() values from that option using 
  
  get('section', 'option', raw=True)

  Then however, that option could not use interpolation anymore.

- set() originally allowed to store non-string values in the parser. This
  was not meant to be a feature and caused trouble when the user tried to
  save the configuration to a file or get the stored values back using typed
  get() methods.

SafeConfigParser solves these problems by validating interpolation syntax
(only %(name)s or %% are allowed, the latter being an escaped percent sign)
and raising exceptions on syntax errors, and validating type on set()
operations so that no non-string values can be passed to the parser using
the API.

Why change that?
----------------

When ConfigParser was left alone, it remained the default choice for most
end users, including our own distutils and logging libs. This was a very
weak choice, and most current ConfigParser users are not aware of the
interpolation quirks. I had to close a couple of issues related to people
trying to store non-string values internally in the parser.

The current situation needlessly complicates the documentation. Explaining
all the above quirks to each new user who only wants to parse an INI file is
weak at best. Moreover, users trust Python to do the right thing by default
and according to their intuition. In this case, going for the default
configparser.ConfigParser class without consulting the documentation is
clearly a suboptimal choice.

One last argument is that SafeConfigParser is an awkward name. It implicates
the other parsers are somehow unsafe, or that this specific parser protects
users from something. This is generally considered a naming antipattern.

When?
-----

You might ask whether this can be done for 3.2 (e.g. is that a feature or
a bugfix). In Raymond's words, the beta process should be used to flesh out
the APIs, test whether they work as expected and fix suboptimal decisions
before we hit the release candidate stage. He consideres this essentially
a bugfix and I agree.

You might ask why do that now and not for 3.3. We believe that 3.2 is the
last possible moment of introducing a change like that. The adoption rate is
currently still low and application authors porting projects from 2.x expect
incompatibilities. When they are non-issues, handled by 2to3, there's
nothing to be afraid of.

But isn't that... INCOMPATIBLE?!
--------------------------------

Yes, it is. Thanks to the low py3k adoption rate now's the only moment where
there's marginal risk of introducing silent incompatibility (there are
hardly any py3k projects out there). Projects ported from Python 2.x will be
informed by 2to3 of the change. We believe that this will fix more bugs than
it introduces.

Support for bare % signs would be the single case where ConfigParser might
have appeared a more natural solution. In those cases we expect that users
will rather choose to turn off interpolation whatsoever.

Summary
-------

If you have any strong, justified arguments against this bugfix, speak up.
Otherwise the change will be made on Thursday.

-- 
Interpolating regards,
?ukasz Langa
tel. +48 791 080 144
WWW http://lukasz.langa.pl/



From chris at simplistix.co.uk  Tue Dec 14 00:41:28 2010
From: chris at simplistix.co.uk (Chris Withers)
Date: Mon, 13 Dec 2010 23:41:28 +0000
Subject: [Python-Dev] Using logging in the stdlib and its unit tests
In-Reply-To: <loom.20101207T212306-892@post.gmane.org>
References: <loom.20101207T212306-892@post.gmane.org>
Message-ID: <4D06AF28.3040601@simplistix.co.uk>

On 07/12/2010 20:26, Vinay Sajip wrote:
> I would suggest that when unit testing, rather than adding StreamHandlers to log
> to stderr, that something like TestHandler and Matcher from this post:
>
> http://plumberjack.blogspot.com/2010/09/unit-testing-and-logging.html

For Python 2, my testfixtures package has had some helpful bits in this 
area for a while now:

http://packages.python.org/testfixtures/logging.html

I find it important to be able to check my code is logging what I think 
it should be logging!

cheers,

Chris

-- 
Simplistix - Content Management, Batch Processing & Python Consulting
            - http://www.simplistix.co.uk

From v+python at g.nevcal.com  Tue Dec 14 00:46:46 2010
From: v+python at g.nevcal.com (Glenn Linderman)
Date: Mon, 13 Dec 2010 15:46:46 -0800
Subject: [Python-Dev] A grammatical oddity: trailing commas in argument
 lists -- continuation
In-Reply-To: <0DE0FA27-5792-4017-A0B0-4B1AF109FA76@gmail.com>
References: <20101213114449.GA2348@chopin.edu.pl>	<AANLkTi=OM921Xt7ZFv=E6-Gz3sLeLPYC4-BUGOv-f+-L@mail.gmail.com>	<ie57s5$r0u$1@dough.gmane.org>	<AANLkTi=NGOVL8kVKHef7Ox_xTkKO8K0vNG495Bzk3gYV@mail.gmail.com>	<AANLkTikacA1YPHPQ-chjibtH50P=bKkf-CrAoO-VZE=+@mail.gmail.com>	<20101213201759.422b15de@pitrou.net>
	<ie62oc$9r9$1@dough.gmane.org>
	<0DE0FA27-5792-4017-A0B0-4B1AF109FA76@gmail.com>
Message-ID: <4D06B066.5090603@g.nevcal.com>

On 12/13/2010 1:55 PM, Raymond Hettinger wrote:
> It seems to me that a trailing comma in an argument list is more likely to be a user error than a deliberate comma-for-the-future.

It seems to me that a trailing comma, especially in the case of one 
parameter per line, is a deliberate comma-for-the-future.  It's a good 
reminder that you are dealing with a list of some sort, rather than a 
statement, when you look at just one parameter on the line.

Especially if the ) is on the next line, which I prefer.

Yes, a parameter list is not a python list, nor a python tuple, but it 
is still a generic, comma-separated list, and all such are more 
conveniently dealt with, in the multi-line case, if trailing commas are 
permitted.  And, of course, the one-entry tuple needs the comma to 
differentiate it from some other expression, forcing the trailing comma 
into the syntax... so there can be no consistent rule for all commas 
that doesn't permit trailing commas.

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20101213/a49de9cc/attachment.html>

From ncoghlan at gmail.com  Tue Dec 14 01:08:32 2010
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Tue, 14 Dec 2010 10:08:32 +1000
Subject: [Python-Dev] logging doc broken
In-Reply-To: <ie5sfk$9ro$1@dough.gmane.org>
References: <4D028B7D.9030308@noir.com>
	<loom.20101213T112956-97@post.gmane.org>
	<4D05F6DC.5040900@timgolden.me.uk>
	<loom.20101213T115957-410@post.gmane.org>
	<ie5sfk$9ro$1@dough.gmane.org>
Message-ID: <AANLkTiknCunMNPk4BQ6s823SJnmVuMq=eSYWSgRHy0Hc@mail.gmail.com>

On Tue, Dec 14, 2010 at 5:34 AM, Terry Reedy <tjreedy at udel.edu> wrote:

> On 12/13/2010 6:04 AM, Vinay Sajip wrote:
>
>  True, but this area changed after 2.6 was released (after even 2.7, IIRC),
>> so I
>> want to be sure that if I'm going to change the doc sources on
>> release26-maint,
>>
>
> 2.6 is in security-fix only mode. I am not sure if that precludes doc
> fixes, on the chance that there will ever be a security fix, as it does
> non-security bug fixes, but is it hardly worth the bother compared to
> improving active releases.
>

Fixing it in the 2.7 maintenance branch will be sufficient to modify
docs.python.org, which is likely the most important place to update.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20101214/83c63707/attachment.html>

From ncoghlan at gmail.com  Tue Dec 14 01:16:07 2010
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Tue, 14 Dec 2010 10:16:07 +1000
Subject: [Python-Dev] logging doc broken
In-Reply-To: <loom.20101213T195056-209@post.gmane.org>
References: <4D028B7D.9030308@noir.com>
	<AANLkTi=H=QyLOJUYq0_6vGwvAVA2H4-dGYf2-DHFmdz4@mail.gmail.com>
	<loom.20101213T114011-992@post.gmane.org>
	<AANLkTikxfER39nmJcOusXOkV368PT3W+Rbj-ZdGUk9mF@mail.gmail.com>
	<loom.20101213T195056-209@post.gmane.org>
Message-ID: <AANLkTi=tnEjwgfJSwzjQ3VnjLXSzCgxj8g46XypYmfd6@mail.gmail.com>

On Tue, Dec 14, 2010 at 4:55 AM, Vinay Sajip <vinay_sajip at yahoo.co.uk>wrote:

> Nick Coghlan <ncoghlan <at> gmail.com> writes:
>
> > Yep, that's what I had assumed you were going to do (this did come up
> > in the big thread about the logging docs, but you may have missed it).
> >
>
> Ok, I've now checked in this change, and would be grateful for any
> feedback.
> Time for a break :-)
>

Hmm, that may not have built correctly, since it isn't showing up in the web
version of the dev docs yet. I skimmed the diff on python-checkins though,
and it looked good to me.

I think the revised docs should make it *much* easier for people to get a
handle on how the logging system works (I know my own understanding of it is
significantly better than it was a couple of weeks ago).

Cheers,
Nick.

P.S. I'm off visiting family for a couple of weeks, so my email access is
going to be sketchy for a while.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20101214/d57a6630/attachment.html>

From ncoghlan at gmail.com  Tue Dec 14 01:20:04 2010
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Tue, 14 Dec 2010 10:20:04 +1000
Subject: [Python-Dev] configparser 1.1 - one last bug fix needed
In-Reply-To: <4A6BBCF7-6491-48C3-A02A-84EEA299B1DB@langa.pl>
References: <4A6BBCF7-6491-48C3-A02A-84EEA299B1DB@langa.pl>
Message-ID: <AANLkTin_ao0+2CANVY4ywQPkfjs+S1=D+KaZ5xJyquoS@mail.gmail.com>

+1 from me.

If anyone complains too much, perhaps we can offer to retain the old
ConfigParser as "_BuggyConfigParser"? (that idea is only partially
tongue-in-cheek...)

Regards,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20101214/a6db8361/attachment.html>

From lukasz at langa.pl  Tue Dec 14 01:26:44 2010
From: lukasz at langa.pl (=?utf-8?Q?=C5=81ukasz_Langa?=)
Date: Tue, 14 Dec 2010 01:26:44 +0100
Subject: [Python-Dev] configparser 1.1 - one last bug fix needed
In-Reply-To: <AANLkTin_ao0+2CANVY4ywQPkfjs+S1=D+KaZ5xJyquoS@mail.gmail.com>
References: <4A6BBCF7-6491-48C3-A02A-84EEA299B1DB@langa.pl>
	<AANLkTin_ao0+2CANVY4ywQPkfjs+S1=D+KaZ5xJyquoS@mail.gmail.com>
Message-ID: <16704819-23F0-4C2D-8A42-3C3D06FE99EF@langa.pl>

Wiadomo?? napisana przez Nick Coghlan w dniu 2010-12-14, o godz. 01:20:

> +1 from me.
> 
> If anyone complains too much, perhaps we can offer to retain the old ConfigParser as "_BuggyConfigParser"? (that idea is only partially tongue-in-cheek...)
> 

We may leave the LegacyInterpolation class if someone *really* needs to support that. That way RawConfigParser(interpolation=LegacyInterpolation()) is effectively the old ConfigParser.

-- 
Nondeprecatorily yours,
?ukasz Langa
tel. +48 791 080 144
WWW http://lukasz.langa.pl/

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20101214/80a63eb4/attachment.html>

From ncoghlan at gmail.com  Tue Dec 14 01:31:01 2010
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Tue, 14 Dec 2010 10:31:01 +1000
Subject: [Python-Dev] A grammatical oddity: trailing commas in argument
 lists -- continuation
In-Reply-To: <AANLkTimCQYd8Q6BJQPXtr3mo_Rk=MZi4iZBcUX5z4uTB@mail.gmail.com>
References: <20101213114449.GA2348@chopin.edu.pl>
	<AANLkTi=OM921Xt7ZFv=E6-Gz3sLeLPYC4-BUGOv-f+-L@mail.gmail.com>
	<20101213155119.A52A621CD91@kimball.webabinitio.net>
	<AANLkTimCQYd8Q6BJQPXtr3mo_Rk=MZi4iZBcUX5z4uTB@mail.gmail.com>
Message-ID: <AANLkTin-6=yQaVf4tgqkY3O_hVNFhmvH9=NKDQPEU9Y8@mail.gmail.com>

On Tue, Dec 14, 2010 at 5:39 AM, Mark Dickinson <dickinsm at gmail.com> wrote:

> On Mon, Dec 13, 2010 at 3:51 PM, R. David Murray <rdmurray at bitdance.com>
> wrote:
> > It seems like the status quo is fine.  I wouldn't object to it being
> > made more consistent.  I would object to removing the existing cases.
>
> Same here, on all three counts.  In one of the projects I'm currently
> working on, we've settled on a style that does quite a lot of:
>
> my_thing = Thing(
>    foo = Foo(arg1, arg2, ...),
>    bar = Bar(arg3, arg4, ...),
>    ...
> )
>
> and I've found the trailing comma very convenient during refactoring
> and API experimentation.  (There's still good fun to be had arguing
> about the indentation of that closing parenthesis, though.)
>

Another valid use case that occurred to me is building up a string-keyed
dictionary:

mapping = dict(
  x=1,
  y=2,
  z=3,
)

So, on reflection, removing the existing cases where it is supported is
certainly unreasonable, which makes the consistency argument that much
stronger.

For the record, I reopened issue #9232 (noting the lack of consensus), and
(as someone suggested on the tracker) changed the resolution on the other
one to be as a duplicate of #9232.

Cheers,
Nick.

P.S. As I noted in the logging discussion, my email access is going to be a
bit sketchy for the next couple of weeks.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20101214/7100b7b1/attachment.html>

From solipsis at pitrou.net  Tue Dec 14 01:34:40 2010
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Tue, 14 Dec 2010 01:34:40 +0100
Subject: [Python-Dev] configparser 1.1 - one last bug fix needed
References: <4A6BBCF7-6491-48C3-A02A-84EEA299B1DB@langa.pl>
	<AANLkTin_ao0+2CANVY4ywQPkfjs+S1=D+KaZ5xJyquoS@mail.gmail.com>
Message-ID: <20101214013440.290f2a37@pitrou.net>

On Tue, 14 Dec 2010 10:20:04 +1000
Nick Coghlan <ncoghlan at gmail.com> wrote:

> +1 from me.
> 
> If anyone complains too much, perhaps we can offer to retain the old
> ConfigParser as "_BuggyConfigParser"? (that idea is only partially
> tongue-in-cheek...)

Or we can put it in the "buggy" module which receives all buggy code.

Regards

Antoine.



From vinay_sajip at yahoo.co.uk  Tue Dec 14 02:22:43 2010
From: vinay_sajip at yahoo.co.uk (Vinay Sajip)
Date: Tue, 14 Dec 2010 01:22:43 +0000 (UTC)
Subject: [Python-Dev] logging doc broken
References: <4D028B7D.9030308@noir.com>
	<AANLkTi=H=QyLOJUYq0_6vGwvAVA2H4-dGYf2-DHFmdz4@mail.gmail.com>
	<loom.20101213T114011-992@post.gmane.org>
	<AANLkTikxfER39nmJcOusXOkV368PT3W+Rbj-ZdGUk9mF@mail.gmail.com>
	<loom.20101213T195056-209@post.gmane.org>
	<AANLkTi=tnEjwgfJSwzjQ3VnjLXSzCgxj8g46XypYmfd6@mail.gmail.com>
Message-ID: <loom.20101214T021940-846@post.gmane.org>

Nick Coghlan <ncoghlan <at> gmail.com> writes:


> Hmm, that may not have built correctly, since it isn't showing up in the web
> version of the dev docs yet. I skimmed the diff on python-checkins though, and
> it looked good to me.

I'll keep an eye on the build (built OK on my machine but I'm not sure what the
build trigger is for docs.python.org).

> I think the revised docs should make it *much* easier for people to get a
> handle on how the logging system works (I know my own understanding of it is
> significantly better than it was a couple of weeks ago).

Glad to hear that.

> P.S. I'm off visiting family for a couple of weeks, so my email access is
> going to be sketchy for a while.

Have a nice time, and thanks for all the help.

Regards,

Vinay




From raymond.hettinger at gmail.com  Tue Dec 14 07:21:33 2010
From: raymond.hettinger at gmail.com (Raymond Hettinger)
Date: Mon, 13 Dec 2010 22:21:33 -0800
Subject: [Python-Dev] A grammatical oddity: trailing commas in argument
	lists -- continuation
In-Reply-To: <AANLkTi=kT=wCAvuzjxXCqqo6cyL==2u-YmNcZEdqNO0+@mail.gmail.com>
References: <20101213114449.GA2348@chopin.edu.pl>
	<AANLkTi=OM921Xt7ZFv=E6-Gz3sLeLPYC4-BUGOv-f+-L@mail.gmail.com>
	<ie57s5$r0u$1@dough.gmane.org>
	<AANLkTi=NGOVL8kVKHef7Ox_xTkKO8K0vNG495Bzk3gYV@mail.gmail.com>
	<AANLkTikacA1YPHPQ-chjibtH50P=bKkf-CrAoO-VZE=+@mail.gmail.com>
	<20101213201759.422b15de@pitrou.net> <ie62oc$9r9$1@dough.gmane.org>
	<0DE0FA27-5792-4017-A0B0-4B1AF109FA76@gmail.com>
	<AANLkTi=kT=wCAvuzjxXCqqo6cyL==2u-YmNcZEdqNO0+@mail.gmail.com>
Message-ID: <6406C2A8-3042-4D88-992D-7389C4C413FC@gmail.com>


On Dec 13, 2010, at 2:16 PM, Guido van Rossum wrote:

> On Mon, Dec 13, 2010 at 1:55 PM, Raymond Hettinger
> <raymond.hettinger at gmail.com> wrote:
>> 
>> It seems to me that a trailing comma in an argument list is more likely to be a user error than a deliberate comma-for-the-future.
> 
> Really? Have you observed this? Even if it was inserted by mistake, it
> is harmless.

I only have one data point, my own mistakes.  The SyntaxError
has occasionally been helpful to me when working out a function
signature or to detect a copy and paste error.  In both cases,
it meant that there was supposed to be another argument
and it had been either forgotten or mispasted.

Also, if I were reviewing someone else's code and saw
a trailing comma in a function definition, it would
seem weird and I would wonder if the author intended
a different signature.

FWIW, this isn't important to me at all.  Was just noting
my own experience.  Don't put assign much weight to it,
I don't have much of a preference either way.

> Python has a long tradition of allowing redundant
> trailing commas in comma-separated lists, and it is habit-forming.

Right.  I see that in the wild quite often and use it myself.


Raymond


From steve at pearwood.info  Tue Dec 14 12:24:07 2010
From: steve at pearwood.info (Steven D'Aprano)
Date: Tue, 14 Dec 2010 22:24:07 +1100
Subject: [Python-Dev] configparser 1.1 - one last bug fix needed
In-Reply-To: <20101214013440.290f2a37@pitrou.net>
References: <4A6BBCF7-6491-48C3-A02A-84EEA299B1DB@langa.pl>	<AANLkTin_ao0+2CANVY4ywQPkfjs+S1=D+KaZ5xJyquoS@mail.gmail.com>
	<20101214013440.290f2a37@pitrou.net>
Message-ID: <4D0753D7.9060102@pearwood.info>

Antoine Pitrou wrote:
> On Tue, 14 Dec 2010 10:20:04 +1000
> Nick Coghlan <ncoghlan at gmail.com> wrote:
> 
>> +1 from me.
>>
>> If anyone complains too much, perhaps we can offer to retain the old
>> ConfigParser as "_BuggyConfigParser"? (that idea is only partially
>> tongue-in-cheek...)
> 
> Or we can put it in the "buggy" module which receives all buggy code.

The good thing about that idea is that maintenance of buggy.py will be 
so simple!



-- 
Steven


From fdrake at acm.org  Tue Dec 14 12:38:51 2010
From: fdrake at acm.org (Fred Drake)
Date: Tue, 14 Dec 2010 06:38:51 -0500
Subject: [Python-Dev] configparser 1.1 - one last bug fix needed
In-Reply-To: <4D0753D7.9060102@pearwood.info>
References: <4A6BBCF7-6491-48C3-A02A-84EEA299B1DB@langa.pl>
	<AANLkTin_ao0+2CANVY4ywQPkfjs+S1=D+KaZ5xJyquoS@mail.gmail.com>
	<20101214013440.290f2a37@pitrou.net> <4D0753D7.9060102@pearwood.info>
Message-ID: <AANLkTimvXp-GYMHpuZ0gbGkVOmMaWziKqMwKB=S_ChTc@mail.gmail.com>

On Tue, Dec 14, 2010 at 6:24 AM, Steven D'Aprano <steve at pearwood.info> wrote:
> The good thing about that idea is that maintenance of buggy.py will be so
> simple!

It's self-describing, and needs no further documentation.  :-)


? -Fred

--
Fred L. Drake, Jr.? ? <fdrake at acm.org>
"A storm broke loose in my mind."? --Albert Einstein

From raymond.hettinger at gmail.com  Tue Dec 14 18:01:28 2010
From: raymond.hettinger at gmail.com (Raymond Hettinger)
Date: Tue, 14 Dec 2010 09:01:28 -0800
Subject: [Python-Dev] configparser 1.1 - one last bug fix needed
In-Reply-To: <AANLkTimvXp-GYMHpuZ0gbGkVOmMaWziKqMwKB=S_ChTc@mail.gmail.com>
References: <4A6BBCF7-6491-48C3-A02A-84EEA299B1DB@langa.pl>
	<AANLkTin_ao0+2CANVY4ywQPkfjs+S1=D+KaZ5xJyquoS@mail.gmail.com>
	<20101214013440.290f2a37@pitrou.net>
	<4D0753D7.9060102@pearwood.info>
	<AANLkTimvXp-GYMHpuZ0gbGkVOmMaWziKqMwKB=S_ChTc@mail.gmail.com>
Message-ID: <7549C51E-0E6F-456F-9469-BBBECB5E8867@gmail.com>


On Dec 14, 2010, at 3:38 AM, Fred Drake wrote:

> On Tue, Dec 14, 2010 at 6:24 AM, Steven D'Aprano <steve at pearwood.info> wrote:
>> The good thing about that idea is that maintenance of buggy.py will be so
>> simple!
> 
> It's self-describing, and needs no further documentation.  :-)
> 

And psychologically more effective than deprecation.
Anyone writing "import buggy" probably feels like they
have got some explaining to do :-)


Raymond
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20101214/aad9acf5/attachment.html>

From mail at timgolden.me.uk  Tue Dec 14 18:06:20 2010
From: mail at timgolden.me.uk (Tim Golden)
Date: Tue, 14 Dec 2010 17:06:20 +0000
Subject: [Python-Dev] configparser 1.1 - one last bug fix needed
In-Reply-To: <7549C51E-0E6F-456F-9469-BBBECB5E8867@gmail.com>
References: <4A6BBCF7-6491-48C3-A02A-84EEA299B1DB@langa.pl>	<AANLkTin_ao0+2CANVY4ywQPkfjs+S1=D+KaZ5xJyquoS@mail.gmail.com>	<20101214013440.290f2a37@pitrou.net>	<4D0753D7.9060102@pearwood.info>	<AANLkTimvXp-GYMHpuZ0gbGkVOmMaWziKqMwKB=S_ChTc@mail.gmail.com>
	<7549C51E-0E6F-456F-9469-BBBECB5E8867@gmail.com>
Message-ID: <4D07A40C.90507@timgolden.me.uk>

On 14/12/2010 17:01, Raymond Hettinger wrote:
>
> On Dec 14, 2010, at 3:38 AM, Fred Drake wrote:
>
>> On Tue, Dec 14, 2010 at 6:24 AM, Steven D'Aprano<steve at pearwood.info>  wrote:
>>> The good thing about that idea is that maintenance of buggy.py will be so
>>> simple!
>>
>> It's self-describing, and needs no further documentation.  :-)
>>
>
> And psychologically more effective than deprecation.
> Anyone writing "import buggy" probably feels like they
> have got some explaining to do :-)

They might be intro retro-robotics:

   http://www.anf.nildram.co.uk/beebcontrol/buggies/bbc_buggy/index.html

(For those not in the UK, this was a very popular schools
project kit on the back of the famous BBC micro initiative)

TJG

From ethan at stoneleaf.us  Tue Dec 14 18:30:51 2010
From: ethan at stoneleaf.us (Ethan Furman)
Date: Tue, 14 Dec 2010 09:30:51 -0800
Subject: [Python-Dev] A grammatical oddity: trailing commas in argument
 lists -- continuation
In-Reply-To: <ie62oc$9r9$1@dough.gmane.org>
References: <20101213114449.GA2348@chopin.edu.pl>	<AANLkTi=OM921Xt7ZFv=E6-Gz3sLeLPYC4-BUGOv-f+-L@mail.gmail.com>	<ie57s5$r0u$1@dough.gmane.org>	<AANLkTi=NGOVL8kVKHef7Ox_xTkKO8K0vNG495Bzk3gYV@mail.gmail.com>	<AANLkTikacA1YPHPQ-chjibtH50P=bKkf-CrAoO-VZE=+@mail.gmail.com>	<20101213201759.422b15de@pitrou.net>
	<ie62oc$9r9$1@dough.gmane.org>
Message-ID: <4D07A9CB.4060409@stoneleaf.us>

Terry Reedy wrote:
> On 12/13/2010 2:17 PM, Antoine Pitrou wrote:
>> On Mon, 13 Dec 2010 14:09:02 -0500
>> Alexander Belopolsky<alexander.belopolsky at gmail.com>  wrote:
>>
>>> On Mon, Dec 13, 2010 at 11:54 AM, Guido van Rossum<guido at python.org>  
>>> wrote:
>>>> I'm at least +0 on
>>>> allowing trailing commas in the situation the OP mentioned.
>>>>
>>>
>>> FWIW, I am also about +0.5 on allowing trailing comma.  Note that in a
>>> similar situation,  the C standardization committee has erred on the
>>> side of consistency:
>>>
>>> """
>>> A new feature of C99: a common extension in many implementations
>>> allows a trailing comma after the list of enumeration constants. The
>>> Committee decided to adopt this feature as an innocuous extension that
>>> mirrors the trailing commas allowed in initializers.
>>> """ http://www.open-std.org/jtc1/sc22/wg14/www/C99RationaleV5.10.pdf
>>>
>>> Similarly, I find allowing trailing comma in keyword only arguments
>>> lists to be an innocuous extension that mirrors the trailing commas
>>> allowed in the positional arguments lists.
>>
>> +1 from me as well. Special cases are hard to remember.
> 
> Same here. A strong +1 for a consistent rule (always or never allowed) 
> with a +1 for always given others use case of one param/arg per line.

+1 on consistency.

+1 on allowing the trailing comma.

~Ethan~

From orsenthil at gmail.com  Wed Dec 15 09:51:32 2010
From: orsenthil at gmail.com (Senthil Kumaran)
Date: Wed, 15 Dec 2010 16:51:32 +0800
Subject: [Python-Dev] Two urllib issues
Message-ID: <AANLkTikqcT2SebEMeOVN22aNRN6CWeQLJBxf-iwvVuqB@mail.gmail.com>

I know it is late to add features in beta release, but still I thought I
would ask for a little leeway for these issues, especially as they don't
change any API signatures.

http://bugs.python.org/issue3243
Has patch, tests and docs

http://bugs.python.org/issue1508475

I have patch ready and shall add the tests and docs too.

Nothing is dependent on those changes, just that that it would be good to
have.

Any suggestions on the above? Georg, is it okay if I push this in before
17th?

-- 
Senthil
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20101215/18d8b541/attachment.html>

From solipsis at pitrou.net  Wed Dec 15 13:00:24 2010
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Wed, 15 Dec 2010 13:00:24 +0100
Subject: [Python-Dev] r87253 -
	tracker/instances/python-dev/detectors/autonosy.py
References: <20101215005337.D91D4EE9EF@mail.python.org>
Message-ID: <20101215130024.31612572@pitrou.net>

On Wed, 15 Dec 2010 01:53:37 +0100 (CET)
ezio.melotti <python-checkins at python.org> wrote:

> Author: ezio.melotti
> Date: Wed Dec 15 01:53:37 2010
> New Revision: 87253
> 
> Log:
> #363: add automatically release managers to the nosy list for release blockers. Initial patch by Georg Brandl.


You should probably add deferred blockers too.

Antoine.



From solipsis at pitrou.net  Wed Dec 15 19:39:40 2010
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Wed, 15 Dec 2010 19:39:40 +0100
Subject: [Python-Dev] Remove HTTP 0.9 support
Message-ID: <20101215193940.31262534@pitrou.net>


Hello,

I would like to remove HTTP 0.9 support from http.client and
http.server. I've opened an issue at http://bugs.python.org/issue10711
for that. Would anyone think it's a bad idea?

(HTTP 1.0 was devised in 1996)

Regards

Antoine.



From fdrake at acm.org  Wed Dec 15 21:02:24 2010
From: fdrake at acm.org (Fred Drake)
Date: Wed, 15 Dec 2010 15:02:24 -0500
Subject: [Python-Dev] Remove HTTP 0.9 support
In-Reply-To: <20101215193940.31262534@pitrou.net>
References: <20101215193940.31262534@pitrou.net>
Message-ID: <AANLkTimM-EbLgTpfmqSE-oi3snHwLfHig1ES_y9sed_d@mail.gmail.com>

On Wed, Dec 15, 2010 at 1:39 PM, Antoine Pitrou <solipsis at pitrou.net> wrote:
> I would like to remove HTTP 0.9 support from http.client and
> http.server.

+1


? -Fred

--
Fred L. Drake, Jr.? ? <fdrake at acm.org>
"A storm broke loose in my mind."? --Albert Einstein

From v+python at g.nevcal.com  Wed Dec 15 21:58:51 2010
From: v+python at g.nevcal.com (Glenn Linderman)
Date: Wed, 15 Dec 2010 12:58:51 -0800
Subject: [Python-Dev] Remove HTTP 0.9 support
In-Reply-To: <20101215193940.31262534@pitrou.net>
References: <20101215193940.31262534@pitrou.net>
Message-ID: <4D092C0B.8090300@g.nevcal.com>

On 12/15/2010 10:39 AM, Antoine Pitrou wrote:
> Hello,
>
> I would like to remove HTTP 0.9 support from http.client and
> http.server. I've opened an issue at http://bugs.python.org/issue10711
> for that. Would anyone think it's a bad idea?
>
> (HTTP 1.0 was devised in 1996)

Please address the following comment from the server.py source:

     # The default request version.  This only affects responses up until
     # the point where the request line is parsed, so it mainly decides what
     # the client gets back when sending a malformed request line.
     # Most web servers default to HTTP 0.9, i.e. don't send a status line.
     default_request_version = "HTTP/0.9"

I realize this is a somewhat obscure point, and in general, if your 
interest in http.client and http.server implies that some of the many 
outstanding bug reports for that code will get resolved, I have no 
concern for dropping support for HTTP 0.9 protocol, other than the above.

Glenn
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20101215/ab860286/attachment.html>

From solipsis at pitrou.net  Wed Dec 15 22:25:10 2010
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Wed, 15 Dec 2010 22:25:10 +0100
Subject: [Python-Dev] Remove HTTP 0.9 support
References: <20101215193940.31262534@pitrou.net>
	<4D092C0B.8090300@g.nevcal.com>
Message-ID: <20101215222510.0b8aa7e7@pitrou.net>

On Wed, 15 Dec 2010 12:58:51 -0800
Glenn Linderman <v+python at g.nevcal.com> wrote:
> On 12/15/2010 10:39 AM, Antoine Pitrou wrote:
> > Hello,
> >
> > I would like to remove HTTP 0.9 support from http.client and
> > http.server. I've opened an issue at http://bugs.python.org/issue10711
> > for that. Would anyone think it's a bad idea?
> >
> > (HTTP 1.0 was devised in 1996)
> 
> Please address the following comment from the server.py source:
> 
>      # The default request version.  This only affects responses up until
>      # the point where the request line is parsed, so it mainly decides what
>      # the client gets back when sending a malformed request line.
>      # Most web servers default to HTTP 0.9, i.e. don't send a status line.
>      default_request_version = "HTTP/0.9"

What do you mean by "address"? The patch changes this to 1.0.
And, as the comment says, this only affects what happens when the
client sends a syntactically invalid request line, so whether the server
does a 0.9-style or 1.0-style response is unimportant.

Regards

Antoine.



From v+python at g.nevcal.com  Wed Dec 15 23:20:54 2010
From: v+python at g.nevcal.com (Glenn Linderman)
Date: Wed, 15 Dec 2010 14:20:54 -0800
Subject: [Python-Dev] Remove HTTP 0.9 support
In-Reply-To: <20101215222510.0b8aa7e7@pitrou.net>
References: <20101215193940.31262534@pitrou.net>	<4D092C0B.8090300@g.nevcal.com>
	<20101215222510.0b8aa7e7@pitrou.net>
Message-ID: <4D093F46.6050106@g.nevcal.com>

On 12/15/2010 1:25 PM, Antoine Pitrou wrote:
> On Wed, 15 Dec 2010 12:58:51 -0800
> Glenn Linderman<v+python at g.nevcal.com>  wrote:
>> On 12/15/2010 10:39 AM, Antoine Pitrou wrote:
>>> Hello,
>>>
>>> I would like to remove HTTP 0.9 support from http.client and
>>> http.server. I've opened an issue at http://bugs.python.org/issue10711
>>> for that. Would anyone think it's a bad idea?
>>>
>>> (HTTP 1.0 was devised in 1996)
>> Please address the following comment from the server.py source:
>>
>>       # The default request version.  This only affects responses up until
>>       # the point where the request line is parsed, so it mainly decides what
>>       # the client gets back when sending a malformed request line.
>>       # Most web servers default to HTTP 0.9, i.e. don't send a status line.
>>       default_request_version = "HTTP/0.9"
> What do you mean by "address"? The patch changes this to 1.0.
> And, as the comment says, this only affects what happens when the
> client sends a syntactically invalid request line, so whether the server
> does a 0.9-style or 1.0-style response is unimportant.

Just what you did... justify the unimportance of not changing it :)  
Since now it is different than "most web servers".
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20101215/8a9eb178/attachment.html>

From solipsis at pitrou.net  Wed Dec 15 23:29:27 2010
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Wed, 15 Dec 2010 23:29:27 +0100
Subject: [Python-Dev] Remove HTTP 0.9 support
References: <20101215193940.31262534@pitrou.net>
	<4D092C0B.8090300@g.nevcal.com>
	<20101215222510.0b8aa7e7@pitrou.net>
	<4D093F46.6050106@g.nevcal.com>
Message-ID: <20101215232927.475f925c@pitrou.net>

On Wed, 15 Dec 2010 14:20:54 -0800
Glenn Linderman <v+python at g.nevcal.com> wrote:

> On 12/15/2010 1:25 PM, Antoine Pitrou wrote:
> > On Wed, 15 Dec 2010 12:58:51 -0800
> > Glenn Linderman<v+python at g.nevcal.com>  wrote:
> >> On 12/15/2010 10:39 AM, Antoine Pitrou wrote:
> >>> Hello,
> >>>
> >>> I would like to remove HTTP 0.9 support from http.client and
> >>> http.server. I've opened an issue at http://bugs.python.org/issue10711
> >>> for that. Would anyone think it's a bad idea?
> >>>
> >>> (HTTP 1.0 was devised in 1996)
> >> Please address the following comment from the server.py source:
> >>
> >>       # The default request version.  This only affects responses up until
> >>       # the point where the request line is parsed, so it mainly decides what
> >>       # the client gets back when sending a malformed request line.
> >>       # Most web servers default to HTTP 0.9, i.e. don't send a status line.
> >>       default_request_version = "HTTP/0.9"
> > What do you mean by "address"? The patch changes this to 1.0.
> > And, as the comment says, this only affects what happens when the
> > client sends a syntactically invalid request line, so whether the server
> > does a 0.9-style or 1.0-style response is unimportant.
> 
> Just what you did... justify the unimportance of not changing it :)  
> Since now it is different than "most web servers".

Well, I think the "most web servers" comment itself is outdated.
Try e.g. www.mozilla.org or www.google.com or www.msn.com.
(but www.python.org or www.apache.org still have the legacy behaviour)

Regards

Antoine.



From fuzzyman at voidspace.org.uk  Thu Dec 16 01:35:59 2010
From: fuzzyman at voidspace.org.uk (Michael Foord)
Date: Thu, 16 Dec 2010 00:35:59 +0000
Subject: [Python-Dev] [Python-checkins] r87296 -
	python/branches/py3k/Doc/library/test.rst
In-Reply-To: <20101216002330.E0DBAEE993@mail.python.org>
References: <20101216002330.E0DBAEE993@mail.python.org>
Message-ID: <AANLkTimWC1vpL76-jSeV5dPrbR12ZOa2_wyeU2N2YuT9@mail.gmail.com>

On 16 December 2010 00:23, eric.araujo <python-checkins at python.org> wrote:

> Author: eric.araujo
> Date: Thu Dec 16 01:23:30 2010
> New Revision: 87296
>
> Log:
> Advertise ?python -m? instead of direct filename.
>
>
> Modified:
>   python/branches/py3k/Doc/library/test.rst
>
> Modified: python/branches/py3k/Doc/library/test.rst
>
> ==============================================================================
> --- python/branches/py3k/Doc/library/test.rst   (original)
> +++ python/branches/py3k/Doc/library/test.rst   Thu Dec 16 01:23:30 2010
> @@ -168,14 +168,14 @@
>
>  Running :mod:`test.regrtest` directly allows what resources are available
> for
>  tests to use to be set. You do this by using the ``-u`` command-line
> -option. Run :program:`python regrtest.py -uall` to turn on all
> +option. Run :program:`python -m regrtest -uall` to turn on all
>

Shouldn't this be `python -m test.regrtest`, or even just `python -m test`?

Michael


>  resources; specifying ``all`` as an option for ``-u`` enables all
>  possible resources. If all but one resource is desired (a more common
> case), a
>  comma-separated list of resources that are not desired may be listed after
> -``all``. The command :program:`python regrtest.py -uall,-audio,-largefile`
> +``all``. The command :program:`python -m regrtest -uall,-audio,-largefile`
>  will run :mod:`test.regrtest` with all resources except the ``audio`` and
>  ``largefile`` resources. For a list of all resources and more command-line
> -options, run :program:`python regrtest.py -h`.
> +options, run :program:`python -m regrtest -h`.
>
>  Some other ways to execute the regression tests depend on what platform
> the
>  tests are being executed on. On Unix, you can run :program:`make test` at
> the
> _______________________________________________
> Python-checkins mailing list
> Python-checkins at python.org
> http://mail.python.org/mailman/listinfo/python-checkins
>



-- 
http://www.voidspace.org.uk
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20101216/75ee45ba/attachment.html>

From merwok at netwok.org  Thu Dec 16 02:07:43 2010
From: merwok at netwok.org (=?UTF-8?B?w4lyaWMgQXJhdWpv?=)
Date: Thu, 16 Dec 2010 02:07:43 +0100
Subject: [Python-Dev] [Python-checkins] r87296
	-	python/branches/py3k/Doc/library/test.rst
In-Reply-To: <AANLkTimWC1vpL76-jSeV5dPrbR12ZOa2_wyeU2N2YuT9@mail.gmail.com>
References: <20101216002330.E0DBAEE993@mail.python.org>
	<AANLkTimWC1vpL76-jSeV5dPrbR12ZOa2_wyeU2N2YuT9@mail.gmail.com>
Message-ID: <4D09665F.603@netwok.org>

>> -option. Run :program:`python regrtest.py -uall` to turn on all
>> +option. Run :program:`python -m regrtest -uall` to turn on all
> 
> Shouldn't this be `python -m test.regrtest`, or even just `python -m test`?

It should :)  I was about to fix just that but then noticed I could
remove more references to regrtest (after all just an implementation
detail), so I rewrote the section title and first few lines of the section.

My stupid email client forces rewrapping, so I?m attaching the diff as a
separate file. I can repost the patch to the tracker if it?s
inconvenient to review in email, but I?d like to commit soon since I
have a big doc merge to do after this :)

Best regards
-------------- next part --------------
A non-text attachment was scrubbed...
Name: fix-test.rst.diff
Type: text/x-diff
Size: 2611 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/python-dev/attachments/20101216/6fe81973/attachment.diff>

From fuzzyman at voidspace.org.uk  Thu Dec 16 02:13:42 2010
From: fuzzyman at voidspace.org.uk (Michael Foord)
Date: Thu, 16 Dec 2010 01:13:42 +0000
Subject: [Python-Dev] [Python-checkins] r87296
	-	python/branches/py3k/Doc/library/test.rst
In-Reply-To: <4D09665F.603@netwok.org>
References: <20101216002330.E0DBAEE993@mail.python.org>
	<AANLkTimWC1vpL76-jSeV5dPrbR12ZOa2_wyeU2N2YuT9@mail.gmail.com>
	<4D09665F.603@netwok.org>
Message-ID: <4D0967C6.4040408@voidspace.org.uk>

On 16/12/2010 01:07, ?ric Araujo wrote:
>>> -option. Run :program:`python regrtest.py -uall` to turn on all
>>> +option. Run :program:`python -m regrtest -uall` to turn on all
>> Shouldn't this be `python -m test.regrtest`, or even just `python -m test`?
> It should :)  I was about to fix just that but then noticed I could
> remove more references to regrtest (after all just an implementation
> detail), so I rewrote the section title and first few lines of the section.
>
> My stupid email client forces rewrapping, so I?m attaching the diff as a
> separate file. I can repost the patch to the tracker if it?s
> inconvenient to review in email, but I?d like to commit soon since I
> have a big doc merge to do after this :)

The diff looks good to me.

All the best,

Michael


> Best regards


-- 

http://www.voidspace.org.uk/

May you do good and not evil
May you find forgiveness for yourself and forgive others
May you share freely, never taking more than you give.
-- the sqlite blessing http://www.sqlite.org/different.html


From merwok at netwok.org  Thu Dec 16 02:36:12 2010
From: merwok at netwok.org (=?UTF-8?B?w4lyaWMgQXJhdWpv?=)
Date: Thu, 16 Dec 2010 02:36:12 +0100
Subject: [Python-Dev] [Python-checkins] r87299 - in
 python/branches/py3k: Doc/library/configparser.rst Lib/configparser.py
 Lib/test/test_cfgparser.py
In-Reply-To: <20101216011622.871CDEE9D2@mail.python.org>
References: <20101216011622.871CDEE9D2@mail.python.org>
Message-ID: <4D096D0C.8020506@netwok.org>

> Author: lukasz.langa
> New Revision: 87299
> 
> Log:
> Broken ConfigParser removed, SafeConfigParser renamed to ConfigParser.
> Life is beatiful once again.
IIIUC, this change makes bugs requesting use of SafeConfigParser in
distutils and logging obsolete.

> @@ -1139,6 +1122,6 @@
>  
>  if __name__ == "__main__":
>      if "-c" in sys.argv:
> -        test_coverage('/tmp/cmd.cover')
> +        test_coverage('/tmp/configparser.cover')
>      else:
>          test_main()

Consider using the tempfile module.  You need to print the filename on
stderr, I think.  Alternatively, remove this custom functionality
entirely and move it to regrtest or unittest.

Cheers


From merwok at netwok.org  Thu Dec 16 02:41:26 2010
From: merwok at netwok.org (=?UTF-8?B?w4lyaWMgQXJhdWpv?=)
Date: Thu, 16 Dec 2010 02:41:26 +0100
Subject: [Python-Dev] [Python-checkins] r87296
	-	python/branches/py3k/Doc/library/test.rst
In-Reply-To: <4D0967C6.4040408@voidspace.org.uk>
References: <20101216002330.E0DBAEE993@mail.python.org>
	<AANLkTimWC1vpL76-jSeV5dPrbR12ZOa2_wyeU2N2YuT9@mail.gmail.com>
	<4D09665F.603@netwok.org> <4D0967C6.4040408@voidspace.org.uk>
Message-ID: <4D096E46.20801@netwok.org>

> The diff looks good to me.

Committed as r87300, thank you sir!


From merwok at netwok.org  Thu Dec 16 03:41:48 2010
From: merwok at netwok.org (=?UTF-8?B?w4lyaWMgQXJhdWpv?=)
Date: Thu, 16 Dec 2010 03:41:48 +0100
Subject: [Python-Dev] PEPs and versionadded
Message-ID: <4D097C6C.1010109@netwok.org>

Hi,

I noticed that changes related to PEP 3147 and PEP 3149 in Doc haven?t
been accompanied by versionadded/versionchanged directives.

Is that on purpose, meaning that everyone should be aware of these PEPs
when reading 3.2 docs, or just an oversight?

Regards


From barry at python.org  Thu Dec 16 02:20:56 2010
From: barry at python.org (Barry Warsaw)
Date: Wed, 15 Dec 2010 20:20:56 -0500
Subject: [Python-Dev] PEPs and versionadded
In-Reply-To: <4D097C6C.1010109@netwok.org>
References: <4D097C6C.1010109@netwok.org>
Message-ID: <20101215202056.3665b462@mission>

On Dec 16, 2010, at 03:41 AM, ?ric Araujo wrote:

>I noticed that changes related to PEP 3147 and PEP 3149 in Doc haven?t
>been accompanied by versionadded/versionchanged directives.
>
>Is that on purpose, meaning that everyone should be aware of these PEPs
>when reading 3.2 docs, or just an oversight?

It's an oversight.  The changes should be accompanied by version* directives.
I see you already committed a change for compileall.rst - thanks for that!
Please feel free to add anything you find missing or submit a bug report and
assign it to me.

-Barry
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 836 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/python-dev/attachments/20101215/697d7464/attachment.pgp>

From merwok at netwok.org  Thu Dec 16 07:34:37 2010
From: merwok at netwok.org (=?UTF-8?B?w4lyaWMgQXJhdWpv?=)
Date: Thu, 16 Dec 2010 07:34:37 +0100
Subject: [Python-Dev] PEPs and versionadded
In-Reply-To: <20101215202056.3665b462@mission>
References: <4D097C6C.1010109@netwok.org> <20101215202056.3665b462@mission>
Message-ID: <4D09B2FD.7080806@netwok.org>

> It's an oversight.  The changes should be accompanied by version* directives.
> I see you already committed a change for compileall.rst - thanks for that!
> Please feel free to add anything you find missing or submit a bug report and
> assign it to me.

The PEP 3149 commits did not touch Doc, apart from sys.rst which got its
versionadded added (duh) a short time after the code changes.  I don?t
know if the introduction of ABI flags ought to be documented more
prominently; Doc/using seems still accurate, and Doc/distutils is rather
vague about extensions, so I guess we?re good.

PEP 3147 modified a handful of files in Doc; I?ve added missing
directives in r87312.

__cached__ is only described in library/runpy.rst and c-api/import.rst.
 If there is one canonical place to define what a module is (I couldn?t
tell from http://docs.python.org/dev/genindex-M), I think it should be
mentioned there.  Copy-pasting the paragraph about __cached__ from the
PEP would suffice.

Both PEPs are well covered in whatsnew.  Three cheers for Raymond!

Regards


From orsenthil at gmail.com  Thu Dec 16 09:14:04 2010
From: orsenthil at gmail.com (Senthil Kumaran)
Date: Thu, 16 Dec 2010 16:14:04 +0800
Subject: [Python-Dev] Remove HTTP 0.9 support
In-Reply-To: <4D093F46.6050106@g.nevcal.com>
References: <20101215193940.31262534@pitrou.net>
	<4D092C0B.8090300@g.nevcal.com>
	<20101215222510.0b8aa7e7@pitrou.net>
	<4D093F46.6050106@g.nevcal.com>
Message-ID: <20101216081404.GA4603@rubuntu>

On Wed, Dec 15, 2010 at 02:20:54PM -0800, Glenn Linderman wrote:
>         On 12/15/2010 10:39 AM, Antoine Pitrou wrote:
>             I would like to remove HTTP 0.9 support from http.client and
>             http.server. I've opened an issue at http://bugs.python.org/issue10711
>             for that. Would anyone think it's a bad idea?
> 
>             (HTTP 1.0 was devised in 1996)
> 
>         Please address the following comment from the server.py source:
> 
>              # The default request version.  This only affects responses up until
>              # the point where the request line is parsed, so it mainly decides what
>              # the client gets back when sending a malformed request line.
>              # Most web servers default to HTTP 0.9, i.e. don't send a status line.
>              default_request_version = "HTTP/0.9"
> 
>     What do you mean by "address"? The patch changes this to 1.0.
>     And, as the comment says, this only affects what happens when the
>     client sends a syntactically invalid request line, so whether the server
>     does a 0.9-style or 1.0-style response is unimportant.
> 
> 
> Just what you did... justify the unimportance of not changing it :)  Since now
> it is different than "most web servers".

+1 to removing HTTP 0.9 related code in http.client and http.server.

Until today, I hadn't cared to read any details of HTTP 0.9 except
that I knew that some code was present in http library to support it.
Reading a bit about the HTTP 0.9 'Internet Draft' written 1991 is
enough to convince that, if we have to fall back on any old behavior
falling back to HTTP 1.0 is perfectly fine.

Regarding this comment

# Most web servers default to HTTP 0.9, i.e. don't send a status line.

Even HTTP 0.9 says that response SHOULD start with status line, but
gives a suggestion that clients can "tolerate" bad server server
behaviors when they don't send the status line and in that the case
response is the body.

http://www.w3.org/Protocols/HTTP/Response.html

So, It cannot be associated with the behavior "most webservers", back
then and even more so now.

-- 
Senthil


From orsenthil at gmail.com  Thu Dec 16 09:14:56 2010
From: orsenthil at gmail.com (Senthil Kumaran)
Date: Thu, 16 Dec 2010 16:14:56 +0800
Subject: [Python-Dev] Remove HTTP 0.9 support
In-Reply-To: <20101215232927.475f925c@pitrou.net>
References: <20101215193940.31262534@pitrou.net>
	<4D092C0B.8090300@g.nevcal.com>
	<20101215222510.0b8aa7e7@pitrou.net>
	<4D093F46.6050106@g.nevcal.com>
	<20101215232927.475f925c@pitrou.net>
Message-ID: <20101216081456.GB4603@rubuntu>

On Wed, Dec 15, 2010 at 11:29:27PM +0100, Antoine Pitrou wrote:
> Well, I think the "most web servers" comment itself is outdated.
> Try e.g. www.mozilla.org or www.google.com or www.msn.com.
> (but www.python.org or www.apache.org still have the legacy behaviour)

What legacy behavior did you observe in these?

-- 
Senthil

From nd at perlig.de  Thu Dec 16 07:42:08 2010
From: nd at perlig.de (=?iso-8859-1?q?Andr=E9_Malo?=)
Date: Thu, 16 Dec 2010 07:42:08 +0100
Subject: [Python-Dev] Remove HTTP 0.9 support
In-Reply-To: <20101215193940.31262534@pitrou.net>
References: <20101215193940.31262534@pitrou.net>
Message-ID: <201012160742.08474@news.perlig.de>

* Antoine Pitrou wrote:

> Hello,
>
> I would like to remove HTTP 0.9 support from http.client and
> http.server. I've opened an issue at http://bugs.python.org/issue10711
> for that. Would anyone think it's a bad idea?
>
> (HTTP 1.0 was devised in 1996)

HTTP/0.9 support is still recommended (RFC 2616 is from 1999, but still 
current).

I'm wondering, why you would consider touching that at all. Is it broken? 
Does it stand in the way of anything? If not, why throw away a feature?

nd
-- 
Already I've seen people (really!) write web URLs in the form:
http:\\some.site.somewhere
[...] How soon until greengrocers start writing "apples $1\pound"
or something?                           -- Joona I Palaste in clc

From dpritsos at extremepro.gr  Thu Dec 16 12:09:41 2010
From: dpritsos at extremepro.gr (Dimitrios Pritsos)
Date: Thu, 16 Dec 2010 13:09:41 +0200
Subject: [Python-Dev] Multiprocessing module - Synergeticprocessing (my
 custom module) - Pickling unPickling issues
Message-ID: <4D09F375.40208@extremepro.gr>


Hello Core Developers,

My name is Dimitrios and I am newbie in python. I am working on a 
Project (part of my PhD) that is called Synergeticprocessing module. 
Initially is imitating the multiprocessing built in module but the 
processes are distributed on a LAN and not Locally. The main issue I 
have is with Pickle module. And I think I found some kind of BUG in the 
built in multiprocessing module.

(Synergeticprocessing module is located at GitHub: 
https://github.com/dpritsos/synergeticprocessing)

Starting with the "BUG". In case someone uses the multiprocessing.Pool 
of processes he/she has to face the problem of types.MehtodType 
Impossible pickling. That is you cannot dispatch an class instance 
method to the to the Process Pool. However, digging in to the Source 
Code of the module there are few lines that resolve this issue however 
are not activated or they are faultily activated so they do not work. 
This is the 'BUG'

_@ ....../multiprocessing/forking.py_


.
.
.

#
# Try making some callable types picklable
#

from pickle import Pickler
class ForkingPickler(Pickler):
     dispatch = Pickler.dispatch.copy()

     @classmethod
     def register(cls, type, reduce):
         def dispatcher(self, obj):
             rv = reduce(obj)
             self.save_reduce(obj=obj, *rv)
         cls.dispatch[type] = dispatcher

def _reduce_method(m):
     if m.im_self is None:
         return getattr, (m.im_class, m.im_func.func_name)
     else:
         return getattr, (m.im_self, m.im_func.func_name)
ForkingPickler.register(type(ForkingPickler.save), _reduce_method)

def _reduce_method_descriptor(m):
     return getattr, (m.__objclass__, m.__name__)
ForkingPickler.register(type(list.append), _reduce_method_descriptor)
ForkingPickler.register(type(int.__add__), _reduce_method_descriptor)

#def _reduce_builtin_function_or_method(m):
#    return getattr, (m.__self__, m.__name__)
#ForkingPickler.register(type(list().append), 
_reduce_builtin_function_or_method)
#ForkingPickler.register(type(int().__add__), 
_reduce_builtin_function_or_method)
.
.
.

The RED lines are not doing the job, for some reason they are not 
managing to register the GREEN function as a global reduce/pickling 
function even if you call the registration function into you __main__ 
script.

The solution I found is just to do this
*
import copy_reg
import types*

def _reduce_method(m):
     if m.im_self is None:
         return getattr, (m.im_class, m.im_func.func_name)
     else:
         return getattr, (m.im_self, m.im_func.func_name)

*copy_reg.pickle(types.MethodType, _reduce_method)*
.
.
.

Doing that everything works FINE. But ONLY for local methods i.e. the 
ones that their class is defined on the __main__ script or other import-ed.

In case you want to send something remotely (in an other machine) or to 
an other __main__ script running separately then you get a message like 
this:

'module' object has no attribute '<my_class>'

The only way to resolve this is firstly to import a script that has 
<my_class> defined there and everything works fine.

SO the problems it seems to be that the *m.im_class*  (look code above) 
has some attribute __module__ defined as __module__ = '__main__' or 
something like that. And this is the reason why remote script cannot 
execute the function. I mean that the _reduce_method() above DOES is 
pickling the whole CLASS object so there is no reason not to be executed 
at the remote script. Besides it does as mentioned above in you just 
import this the user defined class form an other script.


I have already spent about 12 weeks working on building my 
synergeticPool and resolve the issue of Pickling and only 2 days needed 
for the code of the Pool and the rest of the time was spent for the 
Pickling issues, and study all the Class related mechanics of python. 
That was the reason I ve started digging the multipocessessing module 
and found this say 'BUG', and finally sent this email.

Best Regards,


Dimitrios








-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20101216/73c0c1e7/attachment.html>

From fuzzyman at voidspace.org.uk  Thu Dec 16 13:29:59 2010
From: fuzzyman at voidspace.org.uk (Michael Foord)
Date: Thu, 16 Dec 2010 12:29:59 +0000
Subject: [Python-Dev] Multiprocessing module - Synergeticprocessing (my
 custom module) - Pickling unPickling issues
In-Reply-To: <4D09F375.40208@extremepro.gr>
References: <4D09F375.40208@extremepro.gr>
Message-ID: <4D0A0647.2050900@voidspace.org.uk>

On 16/12/2010 11:09, Dimitrios Pritsos wrote:
>
> Hello Core Developers,
>
> My name is Dimitrios and I am newbie in python. I am working on a 
> Project (part of my PhD) that is called Synergeticprocessing module. 
> Initially is imitating the multiprocessing built in module but the 
> processes are distributed on a LAN and not Locally. The main issue I 
> have is with Pickle module. And I think I found some kind of BUG in 
> the built in multiprocessing module.
>

Hello Dimitrios,

Please post your bug report to the Python bug tracker. As you think you 
have a fix for the issue it is much more likely to be applied quickly if 
you can package it in the form of a test that demonstrates the issue and 
a patch that fixes it.

http://bugs.python.org/

All the best,

Michael Foord



> (Synergeticprocessing module is located at GitHub: 
> https://github.com/dpritsos/synergeticprocessing)
>
> Starting with the "BUG". In case someone uses the multiprocessing.Pool 
> of processes he/she has to face the problem of types.MehtodType 
> Impossible pickling. That is you cannot dispatch an class instance 
> method to the to the Process Pool. However, digging in to the Source 
> Code of the module there are few lines that resolve this issue however 
> are not activated or they are faultily activated so they do not work. 
> This is the 'BUG'
>
> _@ ....../multiprocessing/forking.py_
>
>
> .
> .
> .
>
> #
> # Try making some callable types picklable
> #
>
> from pickle import Pickler
> class ForkingPickler(Pickler):
>     dispatch = Pickler.dispatch.copy()
>
>     @classmethod
>     def register(cls, type, reduce):
>         def dispatcher(self, obj):
>             rv = reduce(obj)
>             self.save_reduce(obj=obj, *rv)
>         cls.dispatch[type] = dispatcher
>
> def _reduce_method(m):
>     if m.im_self is None:
>         return getattr, (m.im_class, m.im_func.func_name)
>     else:
>         return getattr, (m.im_self, m.im_func.func_name)
> ForkingPickler.register(type(ForkingPickler.save), _reduce_method)
>
> def _reduce_method_descriptor(m):
>     return getattr, (m.__objclass__, m.__name__)
> ForkingPickler.register(type(list.append), _reduce_method_descriptor)
> ForkingPickler.register(type(int.__add__), _reduce_method_descriptor)
>
> #def _reduce_builtin_function_or_method(m):
> #    return getattr, (m.__self__, m.__name__)
> #ForkingPickler.register(type(list().append), 
> _reduce_builtin_function_or_method)
> #ForkingPickler.register(type(int().__add__), 
> _reduce_builtin_function_or_method)
> .
> .
> .
>
> The RED lines are not doing the job, for some reason they are not 
> managing to register the GREEN function as a global reduce/pickling 
> function even if you call the registration function into you __main__ 
> script.
>
> The solution I found is just to do this
> *
> import copy_reg
> import types*
>
> def _reduce_method(m):
>     if m.im_self is None:
>         return getattr, (m.im_class, m.im_func.func_name)
>     else:
>         return getattr, (m.im_self, m.im_func.func_name)
>
> *copy_reg.pickle(types.MethodType, _reduce_method)*
> .
> .
> .
>
> Doing that everything works FINE. But ONLY for local methods i.e. the 
> ones that their class is defined on the __main__ script or other 
> import-ed.
>
> In case you want to send something remotely (in an other machine) or 
> to an other __main__ script running separately then you get a message 
> like this:
>
> 'module' object has no attribute '<my_class>'
>
> The only way to resolve this is firstly to import a script that has 
> <my_class> defined there and everything works fine.
>
> SO the problems it seems to be that the *m.im_class*  (look code 
> above) has some attribute __module__ defined as __module__ = 
> '__main__' or something like that. And this is the reason why remote 
> script cannot execute the function. I mean that the _reduce_method() 
> above DOES is pickling the whole CLASS object so there is no reason 
> not to be executed at the remote script. Besides it does as mentioned 
> above in you just import this the user defined class form an other 
> script.
>
>
> I have already spent about 12 weeks working on building my 
> synergeticPool and resolve the issue of Pickling and only 2 days 
> needed for the code of the Pool and the rest of the time was spent for 
> the Pickling issues, and study all the Class related mechanics of 
> python. That was the reason I ve started digging the multipocessessing 
> module and found this say 'BUG', and finally sent this email.
>
> Best Regards,
>
>
> Dimitrios
>
>
>
>
>
>
>
>
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk


-- 

http://www.voidspace.org.uk/

May you do good and not evil
May you find forgiveness for yourself and forgive others
May you share freely, never taking more than you give.
-- the sqlite blessing http://www.sqlite.org/different.html

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20101216/7d6691f3/attachment.html>

From solipsis at pitrou.net  Thu Dec 16 14:20:37 2010
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Thu, 16 Dec 2010 14:20:37 +0100
Subject: [Python-Dev] Remove HTTP 0.9 support
In-Reply-To: <20101216081456.GB4603@rubuntu>
References: <20101215193940.31262534@pitrou.net>
	<4D092C0B.8090300@g.nevcal.com> <20101215222510.0b8aa7e7@pitrou.net>
	<4D093F46.6050106@g.nevcal.com> <20101215232927.475f925c@pitrou.net>
	<20101216081456.GB4603@rubuntu>
Message-ID: <1292505637.3689.0.camel@localhost.localdomain>

Le jeudi 16 d?cembre 2010 ? 16:14 +0800, Senthil Kumaran a ?crit :
> On Wed, Dec 15, 2010 at 11:29:27PM +0100, Antoine Pitrou wrote:
> > Well, I think the "most web servers" comment itself is outdated.
> > Try e.g. www.mozilla.org or www.google.com or www.msn.com.
> > (but www.python.org or www.apache.org still have the legacy behaviour)
> 
> What legacy behavior did you observe in these?

-> Request:
xyzzy

-> Response:
<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
<html><head>
<title>405 Method Not Allowed</title>
</head><body>
<h1>Method Not Allowed</h1>
<p>The requested method xyzzy is not allowed for the URL /.</p>
<hr>
<address>Apache/2.3.8 (Unix) mod_ssl/2.3.8 OpenSSL/1.0.0a Server at
www.apache.org Port 80</address>
</body></html>

(notice how the response has no headers)



From dpritsos at extremepro.gr  Thu Dec 16 14:52:19 2010
From: dpritsos at extremepro.gr (Dimitrios Pritsos)
Date: Thu, 16 Dec 2010 15:52:19 +0200
Subject: [Python-Dev] Multiprocessing module - Synergeticprocessing (my
 custom module) - Pickling unPickling issues
In-Reply-To: <4D0A0647.2050900@voidspace.org.uk>
References: <4D09F375.40208@extremepro.gr> <4D0A0647.2050900@voidspace.org.uk>
Message-ID: <4D0A1993.2030309@extremepro.gr>

On 12/16/2010 02:29 PM, Michael Foord wrote:
> On 16/12/2010 11:09, Dimitrios Pritsos wrote:
>>
>> Hello Core Developers,
>>
>> My name is Dimitrios and I am newbie in python. I am working on a 
>> Project (part of my PhD) that is called Synergeticprocessing module. 
>> Initially is imitating the multiprocessing built in module but the 
>> processes are distributed on a LAN and not Locally. The main issue I 
>> have is with Pickle module. And I think I found some kind of BUG in 
>> the built in multiprocessing module.
>>
>
> Hello Dimitrios,
>
> Please post your bug report to the Python bug tracker. As you think 
> you have a fix for the issue it is much more likely to be applied 
> quickly if you can package it in the form of a test that demonstrates 
> the issue and a patch that fixes it.
>
> http://bugs.python.org/
>
> All the best,
>
> Michael Foord
>



Hello Michael,

OK I will do sent it to the bug tracker. But what about the last issue 
i.e. that even if the class is transfered-and-pickled-unpickled it 
raises an exception if the class is defined into the __main__ script and 
not imported from an other one. (see below) could you please guide me 
where should I post this for someone to help me fix it!

Best Regards,

Dimitrios




>
>
>> (Synergeticprocessing module is located at GitHub: 
>> https://github.com/dpritsos/synergeticprocessing)
>>
>> Starting with the "BUG". In case someone uses the 
>> multiprocessing.Pool of processes he/she has to face the problem of 
>> types.MehtodType Impossible pickling. That is you cannot dispatch an 
>> class instance method to the to the Process Pool. However, digging in 
>> to the Source Code of the module there are few lines that resolve 
>> this issue however are not activated or they are faultily activated 
>> so they do not work. This is the 'BUG'
>>
>> _@ ....../multiprocessing/forking.py_
>>
>>
>> .
>> .
>> .
>>
>> #
>> # Try making some callable types picklable
>> #
>>
>> from pickle import Pickler
>> class ForkingPickler(Pickler):
>>     dispatch = Pickler.dispatch.copy()
>>
>>     @classmethod
>>     def register(cls, type, reduce):
>>         def dispatcher(self, obj):
>>             rv = reduce(obj)
>>             self.save_reduce(obj=obj, *rv)
>>         cls.dispatch[type] = dispatcher
>>
>> def _reduce_method(m):
>>     if m.im_self is None:
>>         return getattr, (m.im_class, m.im_func.func_name)
>>     else:
>>         return getattr, (m.im_self, m.im_func.func_name)
>> ForkingPickler.register(type(ForkingPickler.save), _reduce_method)
>>
>> def _reduce_method_descriptor(m):
>>     return getattr, (m.__objclass__, m.__name__)
>> ForkingPickler.register(type(list.append), _reduce_method_descriptor)
>> ForkingPickler.register(type(int.__add__), _reduce_method_descriptor)
>>
>> #def _reduce_builtin_function_or_method(m):
>> #    return getattr, (m.__self__, m.__name__)
>> #ForkingPickler.register(type(list().append), 
>> _reduce_builtin_function_or_method)
>> #ForkingPickler.register(type(int().__add__), 
>> _reduce_builtin_function_or_method)
>> .
>> .
>> .
>>
>> The RED lines are not doing the job, for some reason they are not 
>> managing to register the GREEN function as a global reduce/pickling 
>> function even if you call the registration function into you __main__ 
>> script.
>>
>> The solution I found is just to do this
>> *
>> import copy_reg
>> import types*
>>
>> def _reduce_method(m):
>>     if m.im_self is None:
>>         return getattr, (m.im_class, m.im_func.func_name)
>>     else:
>>         return getattr, (m.im_self, m.im_func.func_name)
>>
>> *copy_reg.pickle(types.MethodType, _reduce_method)*
>> .
>> .
>> .
>>
>> Doing that everything works FINE. But ONLY for local methods i.e. the 
>> ones that their class is defined on the __main__ script or other 
>> import-ed.
>>
>> In case you want to send something remotely (in an other machine) or 
>> to an other __main__ script running separately then you get a message 
>> like this:
>>
>> 'module' object has no attribute '<my_class>'
>>
>> The only way to resolve this is firstly to import a script that has 
>> <my_class> defined there and everything works fine.
>>
>> SO the problems it seems to be that the *m.im_class*  (look code 
>> above) has some attribute __module__ defined as __module__ = 
>> '__main__' or something like that. And this is the reason why remote 
>> script cannot execute the function. I mean that the _reduce_method() 
>> above DOES is pickling the whole CLASS object so there is no reason 
>> not to be executed at the remote script. Besides it does as mentioned 
>> above in you just import this the user defined class form an other 
>> script.
>>
>>
>> I have already spent about 12 weeks working on building my 
>> synergeticPool and resolve the issue of Pickling and only 2 days 
>> needed for the code of the Pool and the rest of the time was spent 
>> for the Pickling issues, and study all the Class related mechanics of 
>> python. That was the reason I ve started digging the 
>> multipocessessing module and found this say 'BUG', and finally sent 
>> this email.
>>
>> Best Regards,
>>
>>
>> Dimitrios
>>
>>
>>
>>
>>
>>
>>
>>
>>
>> _______________________________________________
>> Python-Dev mailing list
>> Python-Dev at python.org
>> http://mail.python.org/mailman/listinfo/python-dev
>> Unsubscribe:http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk
>
>
> -- 
>
> http://www.voidspace.org.uk/
>
> May you do good and not evil
> May you find forgiveness for yourself and forgive others
> May you share freely, never taking more than you give.
> -- the sqlite blessinghttp://www.sqlite.org/different.html

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20101216/b5fc7d38/attachment.html>

From solipsis at pitrou.net  Thu Dec 16 15:00:34 2010
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Thu, 16 Dec 2010 15:00:34 +0100
Subject: [Python-Dev] Multiprocessing module - Synergeticprocessing (my
 custom module) - Pickling unPickling issues
References: <4D09F375.40208@extremepro.gr> <4D0A0647.2050900@voidspace.org.uk>
	<4D0A1993.2030309@extremepro.gr>
Message-ID: <20101216150034.04711183@pitrou.net>

On Thu, 16 Dec 2010 15:52:19 +0200
Dimitrios Pritsos <dpritsos at extremepro.gr> wrote:
> 
> Hello Michael,
> 
> OK I will do sent it to the bug tracker. But what about the last issue 
> i.e. that even if the class is transfered-and-pickled-unpickled it 
> raises an exception if the class is defined into the __main__ script and 
> not imported from an other one. (see below) could you please guide me 
> where should I post this for someone to help me fix it!

The likely explanation is that the __main__ module in the parent
process isn't the same as in the child process, so fetching the class
from that module doesn't work.
If that's really important for you you could open a separate issue.

Regards

Antoine.



From solipsis at pitrou.net  Thu Dec 16 15:23:05 2010
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Thu, 16 Dec 2010 15:23:05 +0100
Subject: [Python-Dev] Remove HTTP 0.9 support
References: <20101215193940.31262534@pitrou.net>
	<201012160742.08474@news.perlig.de>
Message-ID: <20101216152305.13b1fd3a@pitrou.net>

On Thu, 16 Dec 2010 07:42:08 +0100
Andr? Malo <nd at perlig.de> wrote:
> * Antoine Pitrou wrote:
> 
> > Hello,
> >
> > I would like to remove HTTP 0.9 support from http.client and
> > http.server. I've opened an issue at http://bugs.python.org/issue10711
> > for that. Would anyone think it's a bad idea?
> >
> > (HTTP 1.0 was devised in 1996)
> 
> HTTP/0.9 support is still recommended (RFC 2616 is from 1999, but still 
> current).
> 
> I'm wondering, why you would consider touching that at all. Is it broken? 
> Does it stand in the way of anything? If not, why throw away a feature?

Well, it complicates maintenance and makes fixing issues such as
http://bugs.python.org/issue6791 less likely.

Note that the patch still accepts servers and clients which advertise
themselves as 0.9 (using "HTTP/0.9" as a version string). It just
removes support for the style of "simple response" without headers that
0.9 allowed.

Regards

Antoine.



From exarkun at twistedmatrix.com  Thu Dec 16 16:25:56 2010
From: exarkun at twistedmatrix.com (exarkun at twistedmatrix.com)
Date: Thu, 16 Dec 2010 15:25:56 -0000
Subject: [Python-Dev] Multiprocessing module - Synergeticprocessing
	(my	custom module) - Pickling unPickling issues
In-Reply-To: <20101216150034.04711183@pitrou.net>
References: <4D09F375.40208@extremepro.gr> <4D0A0647.2050900@voidspace.org.uk>
	<4D0A1993.2030309@extremepro.gr>
	<20101216150034.04711183@pitrou.net>
Message-ID: <20101216152556.2058.1297404892.divmod.xquotient.262@localhost.localdomain>

On 02:00 pm, solipsis at pitrou.net wrote:
>On Thu, 16 Dec 2010 15:52:19 +0200
>Dimitrios Pritsos <dpritsos at extremepro.gr> wrote:
>>
>>Hello Michael,
>>
>>OK I will do sent it to the bug tracker. But what about the last issue
>>i.e. that even if the class is transfered-and-pickled-unpickled it
>>raises an exception if the class is defined into the __main__ script 
>>and
>>not imported from an other one. (see below) could you please guide me
>>where should I post this for someone to help me fix it!
>
>The likely explanation is that the __main__ module in the parent
>process isn't the same as in the child process, so fetching the class
>from that module doesn't work.
>If that's really important for you you could open a separate issue.

And another option is to use the simple, pleasant fix of not defining 
any functions or classes in __main__.  Define them in a real module and 
import them for use in __main__.

Jean-Paul

From nd at perlig.de  Thu Dec 16 16:52:43 2010
From: nd at perlig.de (=?iso-8859-1?q?Andr=E9_Malo?=)
Date: Thu, 16 Dec 2010 16:52:43 +0100
Subject: [Python-Dev] Remove HTTP 0.9 support
In-Reply-To: <20101216152305.13b1fd3a@pitrou.net>
References: <20101215193940.31262534@pitrou.net>
	<201012160742.08474@news.perlig.de>
	<20101216152305.13b1fd3a@pitrou.net>
Message-ID: <201012161652.43949.nd@perlig.de>

On Thursday 16 December 2010 15:23:05 Antoine Pitrou wrote:
> On Thu, 16 Dec 2010 07:42:08 +0100
>
> Andr? Malo <nd at perlig.de> wrote:
> > * Antoine Pitrou wrote:
> > > Hello,
> > >
> > > I would like to remove HTTP 0.9 support from http.client and
> > > http.server. I've opened an issue at http://bugs.python.org/issue10711
> > > for that. Would anyone think it's a bad idea?
> > >
> > > (HTTP 1.0 was devised in 1996)
> >
> > HTTP/0.9 support is still recommended (RFC 2616 is from 1999, but still
> > current).
> >
> > I'm wondering, why you would consider touching that at all. Is it broken?
> > Does it stand in the way of anything? If not, why throw away a feature?
>
> Well, it complicates maintenance and makes fixing issues such as
> http://bugs.python.org/issue6791 less likely.

I'd vote for removing it from the client code and keeping it in the server.

> Note that the patch still accepts servers and clients which advertise
> themselves as 0.9 (using "HTTP/0.9" as a version string).

HTTP/0.9 doesn't *have* a version string.

GET /foo

is a HTTP/0.9 request.

GET /foo HTTP/0.9

isn't actually (it's a paradoxon, alright ;). It simply isn't a valid HTTP 
request, which would demand a 505 response.

nd

From rdmurray at bitdance.com  Thu Dec 16 17:04:46 2010
From: rdmurray at bitdance.com (R. David Murray)
Date: Thu, 16 Dec 2010 11:04:46 -0500
Subject: [Python-Dev] [Python-checkins] r87310 - in
	python/branches/release27-maint: Doc/library/compileall.rst
	Lib/compileall.py
In-Reply-To: <20101216061502.43686EE992@mail.python.org>
References: <20101216061502.43686EE992@mail.python.org>
Message-ID: <20101216160446.40E9D1FFBDE@kimball.webabinitio.net>

On Thu, 16 Dec 2010 07:15:02 +0100, eric.araujo <python-checkins at python.org> wrote:
> Modified: python/branches/release27-maint/Doc/library/compileall.rst
> ==============================================================================
> --- python/branches/release27-maint/Doc/library/compileall.rst	(original)
> +++ python/branches/release27-maint/Doc/library/compileall.rst	Thu Dec 16 07:15:02 2010
> @@ -1,4 +1,3 @@
> -
>  :mod:`compileall` --- Byte-compile Python libraries
>  ===================================================
>  
> @@ -50,14 +49,14 @@
>  
>     Expand list with its content (file and directory names).

I realize you didn't write this line, but note that '-' is accepted as
an argument and means "read the list from stdin".

> -.. versionadded:: 2.7
> -   The ``-i``  option.
> +.. versionchanged:: 2.7
> +   Added the ``-i``  option.
>  
>  
>  Public functions
>  ----------------
>  
> -.. function:: compile_dir(dir[, maxlevels[, ddir[, force[,  rx[, quiet]]]]])
> +.. function:: compile_dir(dir[, maxlevels[, ddir[, force[, rx[, quiet]]]]])
>  
>     Recursively descend the directory tree named by *dir*, compiling all :file:`.py`
>     files along the way.  The *maxlevels* parameter is used to limit the depth of
> @@ -72,6 +71,23 @@
>     If *quiet* is true, nothing is printed to the standard output in normal
>     operation.
>  
> +
> +.. function:: compile_file(fullname[, ddir[, force[, rx[, quiet]]]])
> +
> +   Compile the file with path *fullname*.  If *ddir* is given, it is used as the
> +   base path from  which the filename used in error messages will be generated.
> +   If *force* is true, modules are re-compiled even if the timestamp is up to
> +   date.

Although this is copied from the other descriptions of *ddir*, it and the
other instances (and the description of the -d option) should all really
be fixed.  As I discovered when writing the tests for the -d option,
what ddir is is the path that is "baked in" to the .pyc file.  In very
old versions of Python that meant it was the path that would show up in
tracebacks as the path to the source file.  In modern Pythons the ddir
path shows up if and only if the .py file does not exist and the .pyc
file is being run directly.  In 3.2, this means it will never show up
normally, since you can't even run the .pyc file without moving it out of
__pycache__.  Which means 'ddir' is henceforth useful only to those people
who want to package sourceless distributions of the python code.  (If you
want to see this in action check out the -d tests in test_compileall.)

So, 'in error messages' really means 'in tracebacks if the .py file
doesn't exist'.

--
R. David Murray                                      www.bitdance.com

From fdrake at acm.org  Thu Dec 16 17:20:04 2010
From: fdrake at acm.org (Fred Drake)
Date: Thu, 16 Dec 2010 11:20:04 -0500
Subject: [Python-Dev] Remove HTTP 0.9 support
In-Reply-To: <201012161652.43949.nd@perlig.de>
References: <20101215193940.31262534@pitrou.net>
	<201012160742.08474@news.perlig.de>
	<20101216152305.13b1fd3a@pitrou.net> <201012161652.43949.nd@perlig.de>
Message-ID: <AANLkTi=kRj7WJHAMFLwDd8Axfr0dGoDZ9HThDiOhvXLJ@mail.gmail.com>

On Thu, Dec 16, 2010 at 10:52 AM, Andr? Malo <nd at perlig.de> wrote:
> I'd vote for removing it from the client code and keeping it in the server.

If it must be maintained anywhere, it should be in the client,
according to the basic principle of "accept what you can, generate
carefully."

Python.org's HTTP/0.9 responses appear to be in response to HTTP/0.9
requests only.  A request claiming to be HTTP 1.0, but without a Host:
header, gets a redirect to the same page.

I'm still in favor of removing HTTP 0.9 support entirely.


? -Fred

--
Fred L. Drake, Jr.? ? <fdrake at acm.org>
"A storm broke loose in my mind."? --Albert Einstein

From merwok at netwok.org  Thu Dec 16 17:41:22 2010
From: merwok at netwok.org (=?UTF-8?B?w4lyaWMgQXJhdWpv?=)
Date: Thu, 16 Dec 2010 17:41:22 +0100
Subject: [Python-Dev] [Python-checkins] r87310 - in
 python/branches/release27-maint: Doc/library/compileall.rst
 Lib/compileall.py
In-Reply-To: <20101216160446.40E9D1FFBDE@kimball.webabinitio.net>
References: <20101216061502.43686EE992@mail.python.org>
	<20101216160446.40E9D1FFBDE@kimball.webabinitio.net>
Message-ID: <4D0A4132.5020407@netwok.org>

Hi,

Thanks for double-checking.

When I first looked into compileall, I opened
http://bugs.python.org/issue10454 where I state that I find the
description of those options unclear or even not understandable, so your
diagnosis that I just copied text is right.

A rewrite to fully cover the module functionality in clear English is
needed.  Your email was very helpful; can you turn it into a patch and
post it to the bug?  Thanks in advance.

Regards


From orsenthil at gmail.com  Thu Dec 16 17:52:14 2010
From: orsenthil at gmail.com (Senthil Kumaran)
Date: Fri, 17 Dec 2010 00:52:14 +0800
Subject: [Python-Dev] Remove HTTP 0.9 support
In-Reply-To: <1292505637.3689.0.camel@localhost.localdomain>
References: <20101215193940.31262534@pitrou.net>
	<4D092C0B.8090300@g.nevcal.com>
	<20101215222510.0b8aa7e7@pitrou.net>
	<4D093F46.6050106@g.nevcal.com>
	<20101215232927.475f925c@pitrou.net>
	<20101216081456.GB4603@rubuntu>
	<1292505637.3689.0.camel@localhost.localdomain>
Message-ID: <20101216165214.GA7964@rubuntu>

On Thu, Dec 16, 2010 at 02:20:37PM +0100, Antoine Pitrou wrote:
> > > Try e.g. www.mozilla.org or www.google.com or www.msn.com.
> > > (but www.python.org or www.apache.org still have the legacy behaviour)
> > 
> > What legacy behavior did you observe in these?
> 
> -> Request:
> xyzzy
> 
> -> Response:
> <!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
> (notice how the response has no headers)
> 

Well, Error response without headers was observed in www.mozilla.org
and www.google.com for Invalid requests.

But, I observed something surprising at www.apache.org
If you do GET / HTTP/1.0
You do get the valid Response with headers.

But if you do GET /
You get a valid response Without headers.

I was afraid if this behavior was to support HTTP 0.9 style where the
the reponse is sent without the headers.

Actually, it is turning out to be true:

http://ftp.ics.uci.edu/pub/ietf/http/rfc1945.html#Response

According to HTTP 1.0, When a request is Simple-Request, it means a
VERB URL (without a version) and it generally corresponds to HTTP 0.9
And when a server receives such a Simple-Request, it sends a
Simple-Response where it does not send the headers back.

I think, the same is exhibited by other Servers as well
www.google.com, www.mozilla.org where for Invalid Request without
version, you are sending a Simple-Request (HTTP 0.9) style and getting
the corresponding response.

Given these, any assumption that servers no longer support HTTP/0.9
becomes false. So nuking it will require some thought.

-- 
Senthil



-- 

From foom at fuhm.net  Thu Dec 16 17:21:37 2010
From: foom at fuhm.net (James Y Knight)
Date: Thu, 16 Dec 2010 11:21:37 -0500
Subject: [Python-Dev] Remove HTTP 0.9 support
In-Reply-To: <20101216081404.GA4603@rubuntu>
References: <20101215193940.31262534@pitrou.net>
	<4D092C0B.8090300@g.nevcal.com>
	<20101215222510.0b8aa7e7@pitrou.net>
	<4D093F46.6050106@g.nevcal.com> <20101216081404.GA4603@rubuntu>
Message-ID: <892B9BB4-C2F4-43FF-9CA4-98A2A8A51EDA@fuhm.net>


On Dec 16, 2010, at 3:14 AM, Senthil Kumaran wrote:

> Even HTTP 0.9 says that response SHOULD start with status line, but
> gives a suggestion that clients can "tolerate" bad server server
> behaviors when they don't send the status line and in that the case
> response is the body.
> 
> http://www.w3.org/Protocols/HTTP/Response.html
> 
> So, It cannot be associated with the behavior "most webservers", back
> then and even more so now.

Actually no. That document is describing almost-HTTP 1.0. Here is the actual document you were looking for:
http://www.w3.org/Protocols/HTTP/AsImplemented.html

HTTP 0.9 had no headers, no status line, nothing but "GET $url <crlf>" and a stream of data in response.

James

From orsenthil at gmail.com  Thu Dec 16 17:56:13 2010
From: orsenthil at gmail.com (Senthil Kumaran)
Date: Fri, 17 Dec 2010 00:56:13 +0800
Subject: [Python-Dev] Remove HTTP 0.9 support
In-Reply-To: <892B9BB4-C2F4-43FF-9CA4-98A2A8A51EDA@fuhm.net>
References: <20101215193940.31262534@pitrou.net>
	<4D092C0B.8090300@g.nevcal.com>
	<20101215222510.0b8aa7e7@pitrou.net>
	<4D093F46.6050106@g.nevcal.com> <20101216081404.GA4603@rubuntu>
	<892B9BB4-C2F4-43FF-9CA4-98A2A8A51EDA@fuhm.net>
Message-ID: <20101216165613.GB7964@rubuntu>

On Thu, Dec 16, 2010 at 11:21:37AM -0500, James Y Knight wrote:
> > Even HTTP 0.9 says that response SHOULD start with status line, but
> > gives a suggestion that clients can "tolerate" bad server server
> > behaviors when they don't send the status line and in that the case
> > response is the body.
> > 
> > http://www.w3.org/Protocols/HTTP/Response.html
> > 
> > So, It cannot be associated with the behavior "most webservers", back
> > then and even more so now.
> 
> Actually no. That document is describing almost-HTTP 1.0. 

Yeah. I know it was almost-HTTP 1.0, but the same docs say that if
protocol version was not specified, it is assumed to be 0.9. So, I
thought it was a good reference point to understand the behavior.

> Here is the actual document you were looking for:
> http://www.w3.org/Protocols/HTTP/AsImplemented.html
> 
> HTTP 0.9 had no headers, no status line, nothing but "GET $url <crlf>" and a stream of data in response.

Actually, you are right. I seems be be actual defined behavior of HTTP
0.9. As explained in that above doc and also in RFC 1945 Request
section.

-- 
Senthil

From orsenthil at gmail.com  Thu Dec 16 17:58:00 2010
From: orsenthil at gmail.com (Senthil Kumaran)
Date: Fri, 17 Dec 2010 00:58:00 +0800
Subject: [Python-Dev] Remove HTTP 0.9 support
In-Reply-To: <201012161652.43949.nd@perlig.de>
References: <20101215193940.31262534@pitrou.net>
	<201012160742.08474@news.perlig.de>
	<20101216152305.13b1fd3a@pitrou.net>
	<201012161652.43949.nd@perlig.de>
Message-ID: <20101216165800.GC7964@rubuntu>

On Thu, Dec 16, 2010 at 04:52:43PM +0100, Andr? Malo wrote:
> HTTP/0.9 doesn't *have* a version string.
> 
> GET /foo
> 
> is a HTTP/0.9 request.
> 
> GET /foo HTTP/0.9
> 
> isn't actually (it's a paradoxon, alright ;). It simply isn't a valid HTTP 
> request, which would demand a 505 response.

Yes, this is an important point. Many webservers seem to exhibit the
HTTP 0.9 response behaviors when you don't specify the version.


-- 
Senthil

From solipsis at pitrou.net  Thu Dec 16 18:02:45 2010
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Thu, 16 Dec 2010 18:02:45 +0100
Subject: [Python-Dev] Remove HTTP 0.9 support
In-Reply-To: <20101216165214.GA7964@rubuntu>
References: <20101215193940.31262534@pitrou.net>
	<4D092C0B.8090300@g.nevcal.com>
	<20101215222510.0b8aa7e7@pitrou.net>
	<4D093F46.6050106@g.nevcal.com>
	<20101215232927.475f925c@pitrou.net>
	<20101216081456.GB4603@rubuntu>
	<1292505637.3689.0.camel@localhost.localdomain>
	<20101216165214.GA7964@rubuntu>
Message-ID: <20101216180245.19251acf@pitrou.net>

On Fri, 17 Dec 2010 00:52:14 +0800
Senthil Kumaran <orsenthil at gmail.com> wrote:
> Actually, it is turning out to be true:
> 
> http://ftp.ics.uci.edu/pub/ietf/http/rfc1945.html#Response
> 
> According to HTTP 1.0, When a request is Simple-Request, it means a
> VERB URL (without a version) and it generally corresponds to HTTP 0.9
> And when a server receives such a Simple-Request, it sends a
> Simple-Response where it does not send the headers back.
> 
> I think, the same is exhibited by other Servers as well
> www.google.com, www.mozilla.org where for Invalid Request without
> version, you are sending a Simple-Request (HTTP 0.9) style and getting
> the corresponding response.

Yes, but only error or redirect responses:

$ nc www.google.fr 80
GET /
HTTP/1.0 302 Found
Location: http://www.google.fr/
[etc.]

$ nc www.mozilla.org 80
GET /
<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
<html><head>
<title>403 Forbidden</title>
</head><body>
<h1>Forbidden</h1>
<p>You don't have permission to access /error/noindex.html
on this server.</p>
</body></html>


That's quite understandable, since most HTTP servers will expect a
"host" header to know which site is actually desired.
So a HTTP 0.9 client sending Simple-Requests has very little chance of
being useful these days.

Regards

Antoine.

From guido at python.org  Thu Dec 16 18:34:09 2010
From: guido at python.org (Guido van Rossum)
Date: Thu, 16 Dec 2010 09:34:09 -0800
Subject: [Python-Dev] Remove HTTP 0.9 support
In-Reply-To: <20101216180245.19251acf@pitrou.net>
References: <20101215193940.31262534@pitrou.net>
	<4D092C0B.8090300@g.nevcal.com>
	<20101215222510.0b8aa7e7@pitrou.net> <4D093F46.6050106@g.nevcal.com>
	<20101215232927.475f925c@pitrou.net> <20101216081456.GB4603@rubuntu>
	<1292505637.3689.0.camel@localhost.localdomain>
	<20101216165214.GA7964@rubuntu>
	<20101216180245.19251acf@pitrou.net>
Message-ID: <AANLkTikWYZ7Y6DD6BPDVmcuaJwgkdJuCC7+_xYCh6mQd@mail.gmail.com>

All this talk of modern servers that also still support HTTP/0.9 is
irrelevant. Unless anybody knows of a server that *only* supports HTTP
0.9 (and that's relevant to users of httplib) let's please kill
support in the client.

-- 
--Guido van Rossum (python.org/~guido)

From exarkun at twistedmatrix.com  Thu Dec 16 19:30:18 2010
From: exarkun at twistedmatrix.com (exarkun at twistedmatrix.com)
Date: Thu, 16 Dec 2010 18:30:18 -0000
Subject: [Python-Dev] Remove HTTP 0.9 support
In-Reply-To: <20101216180245.19251acf@pitrou.net>
References: <20101215193940.31262534@pitrou.net>
	<4D092C0B.8090300@g.nevcal.com>
	<20101215222510.0b8aa7e7@pitrou.net>
	<4D093F46.6050106@g.nevcal.com>
	<20101215232927.475f925c@pitrou.net>
	<20101216081456.GB4603@rubuntu>
	<1292505637.3689.0.camel@localhost.localdomain>
	<20101216165214.GA7964@rubuntu>
	<20101216180245.19251acf@pitrou.net>
Message-ID: <20101216183018.2058.1108731637.divmod.xquotient.281@localhost.localdomain>

On 05:02 pm, solipsis at pitrou.net wrote:
>On Fri, 17 Dec 2010 00:52:14 +0800
>Senthil Kumaran <orsenthil at gmail.com> wrote:
>>Actually, it is turning out to be true:
>>
>>http://ftp.ics.uci.edu/pub/ietf/http/rfc1945.html#Response
>>
>>According to HTTP 1.0, When a request is Simple-Request, it means a
>>VERB URL (without a version) and it generally corresponds to HTTP 0.9
>>And when a server receives such a Simple-Request, it sends a
>>Simple-Response where it does not send the headers back.
>>
>>I think, the same is exhibited by other Servers as well
>>www.google.com, www.mozilla.org where for Invalid Request without
>>version, you are sending a Simple-Request (HTTP 0.9) style and getting
>>the corresponding response.
>
>Yes, but only error or redirect responses:
>
>$ nc www.google.fr 80
>GET /
>HTTP/1.0 302 Found
>Location: http://www.google.fr/
>[etc.]

Note that by using `nc` to test this, you're already generating an 
illegal request (unless you're doing something very special with your 
keyboard ;).  When you hit return, nc will send \n.  However, lines are 
delimited by \r\n in HTTP.

I doubt this makes a difference to the point being discussed, but it 
_could_.  I suggest performing your tests with telnet, instead.

Jean-Paul

From nd at perlig.de  Thu Dec 16 19:48:11 2010
From: nd at perlig.de (=?iso-8859-1?q?Andr=E9_Malo?=)
Date: Thu, 16 Dec 2010 19:48:11 +0100
Subject: [Python-Dev] Remove HTTP 0.9 support
In-Reply-To: <AANLkTi=kRj7WJHAMFLwDd8Axfr0dGoDZ9HThDiOhvXLJ@mail.gmail.com>
References: <20101215193940.31262534@pitrou.net>
	<201012161652.43949.nd@perlig.de>
	<AANLkTi=kRj7WJHAMFLwDd8Axfr0dGoDZ9HThDiOhvXLJ@mail.gmail.com>
Message-ID: <201012161948.11644@news.perlig.de>

* Fred Drake wrote:

> On Thu, Dec 16, 2010 at 10:52 AM, Andr? Malo <nd at perlig.de> wrote:
> > I'd vote for removing it from the client code and keeping it in the
> > server.
>
> If it must be maintained anywhere, it should be in the client,
> according to the basic principle of "accept what you can, generate
> carefully."

*scratching head* exactly why I would keep support in the server.

nd
-- 
package Hacker::Perl::Another::Just;print
qq~@{[reverse split/::/ =>__PACKAGE__]}~;

#  Andr? Malo  #  http://www.perlig.de  #

From fdrake at acm.org  Thu Dec 16 19:47:49 2010
From: fdrake at acm.org (Fred Drake)
Date: Thu, 16 Dec 2010 13:47:49 -0500
Subject: [Python-Dev] Remove HTTP 0.9 support
In-Reply-To: <20101216183018.2058.1108731637.divmod.xquotient.281@localhost.localdomain>
References: <20101215193940.31262534@pitrou.net>
	<4D092C0B.8090300@g.nevcal.com>
	<20101215222510.0b8aa7e7@pitrou.net> <4D093F46.6050106@g.nevcal.com>
	<20101215232927.475f925c@pitrou.net> <20101216081456.GB4603@rubuntu>
	<1292505637.3689.0.camel@localhost.localdomain>
	<20101216165214.GA7964@rubuntu>
	<20101216180245.19251acf@pitrou.net>
	<20101216183018.2058.1108731637.divmod.xquotient.281@localhost.localdomain>
Message-ID: <AANLkTi=O_a9VzAG+-iSBPKScfDRUXTftVQ+E645eWKxb@mail.gmail.com>

On Thu, Dec 16, 2010 at 1:30 PM,  <exarkun at twistedmatrix.com> wrote:
> I doubt this makes a difference to the point being discussed, but it
> _could_. ?I suggest performing your tests with telnet, instead.

I received similar results using telnet earlier today.


? -Fred

--
Fred L. Drake, Jr.? ? <fdrake at acm.org>
"A storm broke loose in my mind."? --Albert Einstein

From greg.ewing at canterbury.ac.nz  Thu Dec 16 23:15:24 2010
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Fri, 17 Dec 2010 11:15:24 +1300
Subject: [Python-Dev] Remove HTTP 0.9 support
In-Reply-To: <20101216165214.GA7964@rubuntu>
References: <20101215193940.31262534@pitrou.net>
	<4D092C0B.8090300@g.nevcal.com> <20101215222510.0b8aa7e7@pitrou.net>
	<4D093F46.6050106@g.nevcal.com> <20101215232927.475f925c@pitrou.net>
	<20101216081456.GB4603@rubuntu>
	<1292505637.3689.0.camel@localhost.localdomain>
	<20101216165214.GA7964@rubuntu>
Message-ID: <4D0A8F7C.4080007@canterbury.ac.nz>

Senthil Kumaran wrote:

> Given these, any assumption that servers no longer support HTTP/0.9
> becomes false.

But as long as httplib only sends requests with a version
number >= 1.0, it should be able to expect headers in the
response, shouldn't it?

-- 
Greg

From stefan_ml at behnel.de  Fri Dec 17 08:19:07 2010
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Fri, 17 Dec 2010 08:19:07 +0100
Subject: [Python-Dev] nonlocal x = value
Message-ID: <ief2tb$i9n$1@dough.gmane.org>

Hi,

it seems that Py3 doesn't support setting a "nonlocal" value as part of the 
"nonlocal" command

     Python 3.2a4+ (py3k:86480, Nov 16 2010, 16:43:22)
     [GCC 4.4.3] on linux2
     Type "help", "copyright", "credits" or "license" for more information.
     >>> def x():
     ...   y = 5
     ...   def f():
     ...     nonlocal y = 6
       File "<stdin>", line 4
         nonlocal y = 6
                    ^
     SyntaxError: invalid syntax

even though the PEP clearly describes this feature.

http://www.python.org/dev/peps/pep-3104/#proposed-solution

Either the PEP or the implementation should be updated. Personally, I think 
the PEP should be changed as I think that the syntax complicates the 
grammar more than it's worth. Also, the moratorium applies here, given that 
Py3.1 does not implement this.

Comments?

Stefan


From benjamin at python.org  Fri Dec 17 17:08:06 2010
From: benjamin at python.org (Benjamin Peterson)
Date: Fri, 17 Dec 2010 10:08:06 -0600
Subject: [Python-Dev] nonlocal x = value
In-Reply-To: <ief2tb$i9n$1@dough.gmane.org>
References: <ief2tb$i9n$1@dough.gmane.org>
Message-ID: <AANLkTi=okg+12bS2AVOrAeKDu2j0e4DnNi_5F9fjB_bc@mail.gmail.com>

2010/12/17 Stefan Behnel <stefan_ml at behnel.de>:
> Hi,
>
> it seems that Py3 doesn't support setting a "nonlocal" value as part of the
> "nonlocal" command
>
> ? ?Python 3.2a4+ (py3k:86480, Nov 16 2010, 16:43:22)
> ? ?[GCC 4.4.3] on linux2
> ? ?Type "help", "copyright", "credits" or "license" for more information.
> ? ?>>> def x():
> ? ?... ? y = 5
> ? ?... ? def f():
> ? ?... ? ? nonlocal y = 6
> ? ? ?File "<stdin>", line 4
> ? ? ? ?nonlocal y = 6
> ? ? ? ? ? ? ? ? ? ^
> ? ?SyntaxError: invalid syntax
>
> even though the PEP clearly describes this feature.
>
> http://www.python.org/dev/peps/pep-3104/#proposed-solution
>
> Either the PEP or the implementation should be updated. Personally, I think
> the PEP should be changed as I think that the syntax complicates the grammar
> more than it's worth. Also, the moratorium applies here, given that Py3.1
> does not implement this.
>
> Comments?

There's a issue and a patch for this somewhere. I personally don't
care; it's not too painful to write two lines.



-- 
Regards,
Benjamin

From lvh at laurensvh.be  Fri Dec 17 17:52:17 2010
From: lvh at laurensvh.be (Laurens Van Houtven)
Date: Fri, 17 Dec 2010 17:52:17 +0100
Subject: [Python-Dev] nonlocal x = value
In-Reply-To: <ief2tb$i9n$1@dough.gmane.org>
References: <ief2tb$i9n$1@dough.gmane.org>
Message-ID: <AANLkTikZi3DEqx7-QgR4wZ337E-OC_Xk7E9NZaqw1jaf@mail.gmail.com>

+1 for throwing it out of the PEP. Assignment is a thing,
nonlocal/global is a thing, don't mix them up :) (That in addition to
the grammar cleanliness argument Stephan already made)

cheers
lvh.

From status at bugs.python.org  Fri Dec 17 18:07:04 2010
From: status at bugs.python.org (Python tracker)
Date: Fri, 17 Dec 2010 18:07:04 +0100 (CET)
Subject: [Python-Dev] Summary of Python tracker Issues
Message-ID: <20101217170704.4F0151CCAD@psf.upfronthosting.co.za>


ACTIVITY SUMMARY (2010-12-10 - 2010-12-17)
Python tracker at http://bugs.python.org/

To view or respond to any of the issues listed below, click on the issue.
Do NOT respond to this message.

Issues counts and deltas:
  open    2537 (-20)
  closed 19971 (+72)
  total  22508 (+52)

Open issues with patches: 1070 


Issues opened (25)
==================

#9232: Allow trailing comma in any function argument list.
http://bugs.python.org/issue9232  reopened by ncoghlan

#10642: Improve the error message of addpackage() (site.py) for defect
http://bugs.python.org/issue10642  reopened by r.david.murray

#10679: "make altinstall" may clobber OS provided scripts
http://bugs.python.org/issue10679  opened by ncoghlan

#10680: argparse: titles and add_mutually_exclusive_group don't mix (e
http://bugs.python.org/issue10680  opened by Mads.Michelsen

#10684: Shutil.move deletes file/folder in windows while renaming
http://bugs.python.org/issue10684  opened by harish

#10685: trace does nto ignore --ignore-module
http://bugs.python.org/issue10685  opened by RusiMody

#10686: email.Generator should use unknown-8bit encoded words for head
http://bugs.python.org/issue10686  opened by r.david.murray

#10690: IDLE Crash when running/saving Module
http://bugs.python.org/issue10690  opened by David_Anon

#10694: zipfile.py end of central directory detection not robust
http://bugs.python.org/issue10694  opened by KevinH

#10697: host and port attributes not documented well in function urlli
http://bugs.python.org/issue10697  opened by JTMoon79

#10701: Error pickling a dict
http://bugs.python.org/issue10701  opened by belopolsky

#10702: bytes and bytearray methods are not documented
http://bugs.python.org/issue10702  opened by belopolsky

#10708: Misc/porting should be folded in to the development FAQ
http://bugs.python.org/issue10708  opened by pitrou

#10709: Misc/AIX-NOTES needs updating
http://bugs.python.org/issue10709  opened by pitrou

#10711: Rip out HTTP 0.9 client support
http://bugs.python.org/issue10711  opened by pitrou

#10712: 2to3 fixer for deprecated unittest method names
http://bugs.python.org/issue10712  opened by ezio.melotti

#10713: re module doesn't describe string boundaries for \b
http://bugs.python.org/issue10713  opened by ralph.corderoy

#10715: uninformative error message
http://bugs.python.org/issue10715  opened by Phillip.M.Feldman

#10716: Modernize pydoc to use CSS
http://bugs.python.org/issue10716  opened by rhettinger

#10717: Multiprocessing module Pickling unPickling issues
http://bugs.python.org/issue10717  opened by dimitrios

#10720: test_threadsignals hang on FreeBSD 6.4
http://bugs.python.org/issue10720  opened by pitrou

#10721: Remove HTTP 0.9 server support
http://bugs.python.org/issue10721  opened by pitrou

#10722: IDLE's subprocess didnit make connection ..... Python 2.7
http://bugs.python.org/issue10722  opened by plovet

#10723: Undocumented built-in exceptions
http://bugs.python.org/issue10723  opened by flashk

#10725: Better cache instrumentation
http://bugs.python.org/issue10725  opened by rhettinger



Most recent 15 issues with no replies (15)
==========================================

#10720: test_threadsignals hang on FreeBSD 6.4
http://bugs.python.org/issue10720

#10717: Multiprocessing module Pickling unPickling issues
http://bugs.python.org/issue10717

#10713: re module doesn't describe string boundaries for \b
http://bugs.python.org/issue10713

#10709: Misc/AIX-NOTES needs updating
http://bugs.python.org/issue10709

#10708: Misc/porting should be folded in to the development FAQ
http://bugs.python.org/issue10708

#10701: Error pickling a dict
http://bugs.python.org/issue10701

#10686: email.Generator should use unknown-8bit encoded words for head
http://bugs.python.org/issue10686

#10685: trace does nto ignore --ignore-module
http://bugs.python.org/issue10685

#10684: Shutil.move deletes file/folder in windows while renaming
http://bugs.python.org/issue10684

#10666: OS X installer variants have confusing readline differences
http://bugs.python.org/issue10666

#10664: xml.sax.expatreader should support namespace prefixes
http://bugs.python.org/issue10664

#10663: configure shouldn't set a default OPT
http://bugs.python.org/issue10663

#10657: os.lstat/os.stat/os.fstat don't set st_dev (st_rdev) on Window
http://bugs.python.org/issue10657

#10656: "Out of tree" build fails on AIX 5.3
http://bugs.python.org/issue10656

#10646: ntpath.samefile doesn't work for hard links
http://bugs.python.org/issue10646



Most recent 15 issues waiting for review (15)
=============================================

#10725: Better cache instrumentation
http://bugs.python.org/issue10725

#10723: Undocumented built-in exceptions
http://bugs.python.org/issue10723

#10721: Remove HTTP 0.9 server support
http://bugs.python.org/issue10721

#10712: 2to3 fixer for deprecated unittest method names
http://bugs.python.org/issue10712

#10711: Rip out HTTP 0.9 client support
http://bugs.python.org/issue10711

#10694: zipfile.py end of central directory detection not robust
http://bugs.python.org/issue10694

#10673: multiprocess.Process join method - timeout indistinguishable f
http://bugs.python.org/issue10673

#10671: urllib2 redirect to another host doesn't work
http://bugs.python.org/issue10671

#10665: Expand unicodedata module documentation
http://bugs.python.org/issue10665

#10657: os.lstat/os.stat/os.fstat don't set st_dev (st_rdev) on Window
http://bugs.python.org/issue10657

#10653: test_time test_strptime fails on windows
http://bugs.python.org/issue10653

#10652: test___all_ + test_tcl fails (Windows installed binary)
http://bugs.python.org/issue10652

#10648: Extend peepholer to reverse loads or stores instead of build/u
http://bugs.python.org/issue10648

#10639: reindent.py converts newlines to platform default
http://bugs.python.org/issue10639

#10626: Bad interaction between test_logging and test_concurrent_futur
http://bugs.python.org/issue10626



Top 10 most discussed issues (10)
=================================

#10716: Modernize pydoc to use CSS
http://bugs.python.org/issue10716  12 msgs

#6791: httplib read status memory usage
http://bugs.python.org/issue6791  11 msgs

#9234: argparse: aliases for positional arguments (subparsers)
http://bugs.python.org/issue9234  11 msgs

#10254: unicodedata.normalize('NFC', s) regression
http://bugs.python.org/issue10254  11 msgs

#10711: Rip out HTTP 0.9 client support
http://bugs.python.org/issue10711  11 msgs

#9232: Allow trailing comma in any function argument list.
http://bugs.python.org/issue9232   8 msgs

#9264: trace.py documentation is incomplete
http://bugs.python.org/issue9264   8 msgs

#9286: email.utils.parseaddr returns garbage for invalid input
http://bugs.python.org/issue9286   5 msgs

#10515: csv sniffer does not recognize quotes at the end of line
http://bugs.python.org/issue10515   5 msgs

#10642: Improve the error message of addpackage() (site.py) for defect
http://bugs.python.org/issue10642   5 msgs



Issues closed (60)
==================

#2576: httplib read() very slow due to lack of socket buffer
http://bugs.python.org/issue2576  closed by pitrou

#4236: Crash when importing builtin module during interpreter shutdow
http://bugs.python.org/issue4236  closed by r.david.murray

#4402: os.getenv('PATH')   return different result between 2.5 and 3.
http://bugs.python.org/issue4402  closed by r.david.murray

#4661: email.parser: impossible to read messages encoded in a differe
http://bugs.python.org/issue4661  closed by r.david.murray

#4766: email documentation needs to be precise about strings/bytes
http://bugs.python.org/issue4766  closed by r.david.murray

#6454: Add "example" keyword argument to optparse constructor
http://bugs.python.org/issue6454  closed by rhettinger

#6559: add pass_fds paramter to subprocess.Popen()
http://bugs.python.org/issue6559  closed by gregory.p.smith

#7183: did 2.6.3 regress for some uses of the __doc__ property?
http://bugs.python.org/issue7183  closed by r.david.murray

#7213: subprocess leaks open file descriptors between Popen instances
http://bugs.python.org/issue7213  closed by gregory.p.smith

#8127: Add link to PortingPythonToPy3k to What's New	documentation
http://bugs.python.org/issue8127  closed by r.david.murray

#8753: Py_ReprEnter and Py_ReprLeave are undocumented
http://bugs.python.org/issue8753  closed by stutzbach

#8844: Condition.wait() doesn't raise KeyboardInterrupt
http://bugs.python.org/issue8844  closed by pitrou

#9048: no OS X buildbots in the stable list
http://bugs.python.org/issue9048  closed by r.david.murray

#9162: License for multiprocessing files
http://bugs.python.org/issue9162  closed by r.david.murray

#9721: urlparse.urljoin() cuts off last base character with semicolon
http://bugs.python.org/issue9721  closed by orsenthil

#10188: tempfile.TemporaryDirectory may throw errors at shutdown
http://bugs.python.org/issue10188  closed by ncoghlan

#10454: Clarify compileall command-line options
http://bugs.python.org/issue10454  closed by r.david.murray

#10509: PyTokenizer_FindEncoding can lead to a segfault if bad charact
http://bugs.python.org/issue10509  closed by Trundle

#10534: difflib.SequenceMatcher: expose junk sets, deprecate undocumen
http://bugs.python.org/issue10534  closed by terry.reedy

#10559: NameError in tutorial/interpreter
http://bugs.python.org/issue10559  closed by r.david.murray

#10618: regression in subprocess.call() command quoting
http://bugs.python.org/issue10618  closed by r.david.murray

#10627: Remove usage of deprecated configparser.ConfigParser class in 
http://bugs.python.org/issue10627  closed by lukasz.langa

#10634: Windows timezone changes not reflected by time.localtime
http://bugs.python.org/issue10634  closed by eric.pruitt

#10667: collections.Counter object in C
http://bugs.python.org/issue10667  closed by rhettinger

#10670: Provide search scope limits
http://bugs.python.org/issue10670  closed by r.david.murray

#10674: Unused dictmaker symbol in 2.* grammar
http://bugs.python.org/issue10674  closed by benjamin.peterson

#10675: unittest should have an assertChanges context manager
http://bugs.python.org/issue10675  closed by r.david.murray

#10676: Confusing note in Numeric Types
http://bugs.python.org/issue10676  closed by georg.brandl

#10677: "make altinstall" includes ABI codes in versioned executable n
http://bugs.python.org/issue10677  closed by barry

#10678: email.utils.mktime_tz Giving wrong result , by ignoring Timezo
http://bugs.python.org/issue10678  closed by r.david.murray

#10681: PySlice_GetIndices() signature changed
http://bugs.python.org/issue10681  closed by loewis

#10682: With '*args' or even bare '*' in def/call argument list, trail
http://bugs.python.org/issue10682  closed by loewis

#10683: PreLinkEvent error under VC2010
http://bugs.python.org/issue10683  closed by krisvale

#10687: Python fails to install with empty ABI flags
http://bugs.python.org/issue10687  closed by barry

#10688: pydoc removes lib directory
http://bugs.python.org/issue10688  closed by r.david.murray

#10689: _scproxy extension is NOT build
http://bugs.python.org/issue10689  closed by ned.deily

#10691: make testall no longer working on Darwin
http://bugs.python.org/issue10691  closed by r.david.murray

#10692: imap lib server compabilities
http://bugs.python.org/issue10692  closed by silversky

#10693: error in documentation of distutils.archive_util.make_zipfile
http://bugs.python.org/issue10693  closed by eric.araujo

#10695: telnetlib.Telnet port number int/str inconsistency
http://bugs.python.org/issue10695  closed by r.david.murray

#10696: port not split in function urllib.parse.urlsplit
http://bugs.python.org/issue10696  closed by r.david.murray

#10698: doctest load_tests() typo
http://bugs.python.org/issue10698  closed by r.david.murray

#10699: fix incorrect help doc with time.tzset
http://bugs.python.org/issue10699  closed by r.david.murray

#10700: python pickle.dumps AssertionError
http://bugs.python.org/issue10700  closed by belopolsky

#10703: Regex 0.1.20101210
http://bugs.python.org/issue10703  closed by belopolsky

#10704: Regex 0.1.20101210 Python 3.1 install problem Mac OS X 10.6.5
http://bugs.python.org/issue10704  closed by belopolsky

#10705: HTTPConnection.set_debuglevel has no information about level r
http://bugs.python.org/issue10705  closed by r.david.murray

#10706: kill runtests.sh
http://bugs.python.org/issue10706  closed by pitrou

#10707: compileall is broken
http://bugs.python.org/issue10707  closed by r.david.murray

#10710: Is Misc/setuid-prog.c still needed?
http://bugs.python.org/issue10710  closed by pitrou

#10714: httpserver request length
http://bugs.python.org/issue10714  closed by pitrou

#10718: Idle 2.7.1 from 64-bit installer crashes on OS X when source i
http://bugs.python.org/issue10718  closed by ned.deily

#10719: compileall no longer warns when cli arguments name non-existen
http://bugs.python.org/issue10719  closed by r.david.murray

#10724: socket.close close telnet with RST
http://bugs.python.org/issue10724  closed by r.david.murray

#10726: pydoc: don???t display raw reST in keyword help
http://bugs.python.org/issue10726  closed by georg.brandl

#1459867: Message.as_string should  use "mangle_from_=unixfrom"?
http://bugs.python.org/issue1459867  closed by r.david.murray

#1078919: email.Header (via add_header) encodes non-ASCII content incorr
http://bugs.python.org/issue1078919  closed by r.david.murray

#1628205: socket.readline() interface doesn't handle EINTR properly
http://bugs.python.org/issue1628205  closed by pitrou

#1243654: Faster output if message already has a boundary
http://bugs.python.org/issue1243654  closed by r.david.murray

#775964: fix test_grp failing when NIS entries present
http://bugs.python.org/issue775964  closed by r.david.murray

From alexander.belopolsky at gmail.com  Fri Dec 17 18:13:47 2010
From: alexander.belopolsky at gmail.com (Alexander Belopolsky)
Date: Fri, 17 Dec 2010 12:13:47 -0500
Subject: [Python-Dev] nonlocal x = value
In-Reply-To: <AANLkTikZi3DEqx7-QgR4wZ337E-OC_Xk7E9NZaqw1jaf@mail.gmail.com>
References: <ief2tb$i9n$1@dough.gmane.org>
	<AANLkTikZi3DEqx7-QgR4wZ337E-OC_Xk7E9NZaqw1jaf@mail.gmail.com>
Message-ID: <AANLkTi=d9eEKev2d-UmW-tGYskeL4DSpWs=93MZHrHKc@mail.gmail.com>

On Fri, Dec 17, 2010 at 11:52 AM, Laurens Van Houtven <lvh at laurensvh.be> wrote:
> +1 for throwing it out of the PEP. Assignment is a thing,
> nonlocal/global is a thing, don't mix them up :) (That in addition to
> the grammar cleanliness argument Stephan already made)

Another +1 for the same reasons.  Also, since global does not allow
assignment, neither should nonlocal.

From benjamin at python.org  Fri Dec 17 18:33:50 2010
From: benjamin at python.org (Benjamin Peterson)
Date: Fri, 17 Dec 2010 11:33:50 -0600
Subject: [Python-Dev] nonlocal x = value
In-Reply-To: <AANLkTi=d9eEKev2d-UmW-tGYskeL4DSpWs=93MZHrHKc@mail.gmail.com>
References: <ief2tb$i9n$1@dough.gmane.org>
	<AANLkTikZi3DEqx7-QgR4wZ337E-OC_Xk7E9NZaqw1jaf@mail.gmail.com>
	<AANLkTi=d9eEKev2d-UmW-tGYskeL4DSpWs=93MZHrHKc@mail.gmail.com>
Message-ID: <AANLkTimehqNzUiUQ9gcLsAwxYThxbMGgbDk=6Fe1ji1F@mail.gmail.com>

2010/12/17 Alexander Belopolsky <alexander.belopolsky at gmail.com>:
> On Fri, Dec 17, 2010 at 11:52 AM, Laurens Van Houtven <lvh at laurensvh.be> wrote:
>> +1 for throwing it out of the PEP. Assignment is a thing,
>> nonlocal/global is a thing, don't mix them up :) (That in addition to
>> the grammar cleanliness argument Stephan already made)
>
> Another +1 for the same reasons. ?Also, since global does not allow
> assignment, neither should nonlocal.

Note that the PEP stated that global would also be extended.



-- 
Regards,
Benjamin

From alexander.belopolsky at gmail.com  Fri Dec 17 19:00:53 2010
From: alexander.belopolsky at gmail.com (Alexander Belopolsky)
Date: Fri, 17 Dec 2010 13:00:53 -0500
Subject: [Python-Dev] nonlocal x = value
In-Reply-To: <AANLkTimehqNzUiUQ9gcLsAwxYThxbMGgbDk=6Fe1ji1F@mail.gmail.com>
References: <ief2tb$i9n$1@dough.gmane.org>
	<AANLkTikZi3DEqx7-QgR4wZ337E-OC_Xk7E9NZaqw1jaf@mail.gmail.com>
	<AANLkTi=d9eEKev2d-UmW-tGYskeL4DSpWs=93MZHrHKc@mail.gmail.com>
	<AANLkTimehqNzUiUQ9gcLsAwxYThxbMGgbDk=6Fe1ji1F@mail.gmail.com>
Message-ID: <AANLkTiktmvQYyR-jaVkOL9dVCvTq9AiPEtkfT0XhgMej@mail.gmail.com>

On Fri, Dec 17, 2010 at 12:33 PM, Benjamin Peterson <benjamin at python.org> wrote:
..
>> Another +1 for the same reasons. ?Also, since global does not allow
>> assignment, neither should nonlocal.
>
> Note that the PEP stated that global would also be extended.

I missed that, so for future reference, the PEP says:

"""
A shorthand form is also permitted, in which nonlocal is prepended to
an assignment or augmented assignment:

nonlocal x = 3
The above has exactly the same meaning as nonlocal x; x = 3. (Guido
supports a similar form of the global statement.)
"""

and refers to Guido's post at

http://mail.python.org/pipermail/python-3000/2006-November/004166.html

In any case, the relevant issue is

http://bugs.python.org/issue4199

and it should probably be marked as "after moratorium".  Meanwhile an
implementation status note can be added to the PEP to avoid this issue
being brought up again.

From python at mrabarnett.plus.com  Sat Dec 18 01:08:47 2010
From: python at mrabarnett.plus.com (MRAB)
Date: Sat, 18 Dec 2010 00:08:47 +0000
Subject: [Python-Dev] Locale-specific formatting
Message-ID: <4D0BFB8F.2040301@mrabarnett.plus.com>

I had a thought about locale-specific formatting.

Currently, when we want to do locale-specific formatting we use the
locale module like this:

 >>> locale.format("%d", 12345, grouping=False)
'12345'
 >>> locale.format("%d", 12345, grouping=True)
'12,345'

This makes it harder to use more than one locale at a time, or one
which is different from the default.

My thought was that we could specify a locale in the format
specification mini-language and the parameter list of str.format,
something like this:

 >>> loc = locale.getlocale()
 >>> "{0:@1}".format(12345, loc)
'12345'
 >>> "{0:, at 1}".format(12345, loc)
'12,345'
...
 >>> "UK says {value:,.1f at uk} and France says 
{value:,.1f at france}".format(value=12345, uk=uk_loc, france=france_loc)
'UK says 1,234.5 and France says 1 234,5'

Comments?

From victor.stinner at haypocalc.com  Sat Dec 18 01:55:36 2010
From: victor.stinner at haypocalc.com (Victor Stinner)
Date: Sat, 18 Dec 2010 01:55:36 +0100
Subject: [Python-Dev] Issue #8863 adds a new PYTHONNOFAULTHANDLER
	environment variable
Message-ID: <4D0C0688.1080108@haypocalc.com>

Hi,

In 2008, I proposed a patch to raise a Python exception on SIGSEVG 
signal. In some cases, it's possible to catch the exception, log it, and 
continue the execution. But because in some cases, the Python internal 
state is corrupted, the idea was rejected (see the issue #3999).

Someone asked me to display the Python backtrace on SIGSEGV, instead of 
raising an exception. I implemented this idea in issue #8863. After 9 
versions, I think that the patch is ready for inclusion. It catchs 
SIGSEGV, SIGFPE, SIGBUS and SIGILL signals, and also display the Python 
backtrace on fatal errors. Because some operating systems have their own 
fault handler (eg. Ubuntu with apport), Dave Malcolm asked me to add an 
option disable the Python handler. That's why I added the 
PYTHONNOFAULTHANDLER environment variable (we all love long variable 
names!). For an OS vendor, it looks like an environment variable is more 
practical than a command line option. Eg. Mandriva sets 
PYTHONDONTWRITEBYTECODE for the whole system.

Raymond Hettinger asked me on IRC to write an email about the new 
environment variable, so here is the question: do you agree to add the 
new variable?

Victor

From rdmurray at bitdance.com  Sat Dec 18 05:00:28 2010
From: rdmurray at bitdance.com (R. David Murray)
Date: Fri, 17 Dec 2010 23:00:28 -0500
Subject: [Python-Dev] Locale-specific formatting
In-Reply-To: <4D0BFB8F.2040301@mrabarnett.plus.com>
References: <4D0BFB8F.2040301@mrabarnett.plus.com>
Message-ID: <20101218040028.08CBD237968@kimball.webabinitio.net>

On Sat, 18 Dec 2010 00:08:47 +0000, MRAB <python at mrabarnett.plus.com> wrote:
> I had a thought about locale-specific formatting.
> 
> Currently, when we want to do locale-specific formatting we use the
> locale module like this:
> 
>  >>> locale.format("%d", 12345, grouping=False)
> '12345'
>  >>> locale.format("%d", 12345, grouping=True)
> '12,345'
> 
> This makes it harder to use more than one locale at a time, or one
> which is different from the default.
> 
> My thought was that we could specify a locale in the format
> specification mini-language and the parameter list of str.format,
> something like this:
> 
>  >>> loc = locale.getlocale()
>  >>> "{0:@1}".format(12345, loc)
> '12345'
>  >>> "{0:, at 1}".format(12345, loc)
> '12,345'
> ...
>  >>> "UK says {value:,.1f at uk} and France says 
> {value:,.1f at france}".format(value=12345, uk=uk_loc, france=france_loc)
> 'UK says 1,234.5 and France says 1 234,5'
> 
> Comments?

There was at least one long thread on this on python-ideas.  Probably
worth finding and reading before proceeding with a new discussion... :)

I think it was part of the discussion that ultimately led to PEP 378.

--
R. David Murray                                      www.bitdance.com

From martin at v.loewis.de  Sat Dec 18 10:26:34 2010
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sat, 18 Dec 2010 10:26:34 +0100
Subject: [Python-Dev] Locale-specific formatting
In-Reply-To: <4D0BFB8F.2040301@mrabarnett.plus.com>
References: <4D0BFB8F.2040301@mrabarnett.plus.com>
Message-ID: <4D0C7E4A.3010302@v.loewis.de>

> Comments?

How do you implement that? In particular, how do you retrieve
information for different locales in a single program?

Regards,
Martin

From g.brandl at gmx.net  Sat Dec 18 12:37:09 2010
From: g.brandl at gmx.net (Georg Brandl)
Date: Sat, 18 Dec 2010 12:37:09 +0100
Subject: [Python-Dev] r87188 - python/branches/py3k/Doc/library/test.rst
In-Reply-To: <20101212182525.C5C6DEE9F3@mail.python.org>
References: <20101212182525.C5C6DEE9F3@mail.python.org>
Message-ID: <iei6ft$hq8$1@dough.gmane.org>

I'd rather keep this a note.  We don't want to use warnings except for cases
where there is a possibility of security implications or crashes.

Georg

Am 12.12.2010 19:25, schrieb antoine.pitrou:
> Author: antoine.pitrou
> Date: Sun Dec 12 19:25:25 2010
> New Revision: 87188
> 
> Log:
> Make this a warning and fix indentation
> 
> 
> 
> Modified:
>    python/branches/py3k/Doc/library/test.rst
> 
> Modified: python/branches/py3k/Doc/library/test.rst
> ==============================================================================
> --- python/branches/py3k/Doc/library/test.rst	(original)
> +++ python/branches/py3k/Doc/library/test.rst	Sun Dec 12 19:25:25 2010
> @@ -5,12 +5,12 @@
>     :synopsis: Regression tests package containing the testing suite for Python.
>  .. sectionauthor:: Brett Cannon <brett at python.org>
>  
> -.. note::
> -    The :mod:`test` package is meant for internal use by Python only. It is
> -    documented for the benefit of the core developers of Python. Any use of
> -    this package outside of Python's standard library is discouraged as code
> -    mentioned here can change or be removed without notice between releases of
> -    Python.
> +.. warning::
> +   The :mod:`test` package is meant for internal use by Python only. It is
> +   documented for the benefit of the core developers of Python. Any use of
> +   this package outside of Python's standard library is discouraged as code
> +   mentioned here can change or be removed without notice between releases of
> +   Python.
>  
>  
>  The :mod:`test` package contains all regression tests for Python as well as the



From solipsis at pitrou.net  Sat Dec 18 13:07:34 2010
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Sat, 18 Dec 2010 13:07:34 +0100
Subject: [Python-Dev] r87188 - python/branches/py3k/Doc/library/test.rst
References: <20101212182525.C5C6DEE9F3@mail.python.org>
	<iei6ft$hq8$1@dough.gmane.org>
Message-ID: <20101218130734.1f2f471a@pitrou.net>

On Sat, 18 Dec 2010 12:37:09 +0100
Georg Brandl <g.brandl at gmx.net> wrote:
> I'd rather keep this a note.  We don't want to use warnings except for cases
> where there is a possibility of security implications or crashes.

Well, there'll be a crash as soon as someone relies on an API we decide
to change or remove :)




From g.brandl at gmx.net  Sat Dec 18 13:11:09 2010
From: g.brandl at gmx.net (Georg Brandl)
Date: Sat, 18 Dec 2010 13:11:09 +0100
Subject: [Python-Dev] r87188 - python/branches/py3k/Doc/library/test.rst
In-Reply-To: <20101218130734.1f2f471a@pitrou.net>
References: <20101212182525.C5C6DEE9F3@mail.python.org>	<iei6ft$hq8$1@dough.gmane.org>
	<20101218130734.1f2f471a@pitrou.net>
Message-ID: <iei8fl$qb5$1@dough.gmane.org>

Am 18.12.2010 13:07, schrieb Antoine Pitrou:
> On Sat, 18 Dec 2010 12:37:09 +0100
> Georg Brandl <g.brandl at gmx.net> wrote:
>> I'd rather keep this a note.  We don't want to use warnings except for cases
>> where there is a possibility of security implications or crashes.
> 
> Well, there'll be a crash as soon as someone relies on an API we decide
> to change or remove :)

Ah, but I meant a real crash, not an exception.  Think marshal or ctypes.

Georg


From g.brandl at gmx.net  Sat Dec 18 13:12:33 2010
From: g.brandl at gmx.net (Georg Brandl)
Date: Sat, 18 Dec 2010 13:12:33 +0100
Subject: [Python-Dev] nonlocal x = value
In-Reply-To: <AANLkTikZi3DEqx7-QgR4wZ337E-OC_Xk7E9NZaqw1jaf@mail.gmail.com>
References: <ief2tb$i9n$1@dough.gmane.org>
	<AANLkTikZi3DEqx7-QgR4wZ337E-OC_Xk7E9NZaqw1jaf@mail.gmail.com>
Message-ID: <iei8i9$qb5$2@dough.gmane.org>

Am 17.12.2010 17:52, schrieb Laurens Van Houtven:
> +1 for throwing it out of the PEP. Assignment is a thing,
> nonlocal/global is a thing, don't mix them up :) (That in addition to
> the grammar cleanliness argument Stephan already made)

The trouble is what to make of

nonlocal x = 3, y

Is it two nonlocal declarations or one with a tuple assignment?

Georg


From g.brandl at gmx.net  Sat Dec 18 13:21:45 2010
From: g.brandl at gmx.net (Georg Brandl)
Date: Sat, 18 Dec 2010 13:21:45 +0100
Subject: [Python-Dev] Issue #8863 adds a new PYTHONNOFAULTHANDLER
	environment variable
In-Reply-To: <4D0C0688.1080108@haypocalc.com>
References: <4D0C0688.1080108@haypocalc.com>
Message-ID: <iei93h$tla$1@dough.gmane.org>

Am 18.12.2010 01:55, schrieb Victor Stinner:
> Hi,
> 
> In 2008, I proposed a patch to raise a Python exception on SIGSEVG 
> signal. In some cases, it's possible to catch the exception, log it, and 
> continue the execution. But because in some cases, the Python internal 
> state is corrupted, the idea was rejected (see the issue #3999).
> 
> Someone asked me to display the Python backtrace on SIGSEGV, instead of 
> raising an exception. I implemented this idea in issue #8863. After 9 
> versions, I think that the patch is ready for inclusion. It catchs 
> SIGSEGV, SIGFPE, SIGBUS and SIGILL signals, and also display the Python 
> backtrace on fatal errors. Because some operating systems have their own 
> fault handler (eg. Ubuntu with apport), Dave Malcolm asked me to add an 
> option disable the Python handler. That's why I added the 
> PYTHONNOFAULTHANDLER environment variable (we all love long variable 
> names!). For an OS vendor, it looks like an environment variable is more 
> practical than a command line option. Eg. Mandriva sets 
> PYTHONDONTWRITEBYTECODE for the whole system.
> 
> Raymond Hettinger asked me on IRC to write an email about the new 
> environment variable, so here is the question: do you agree to add the 
> new variable?

I very much like having a traceback on (some) segmentation faults, but
it's clear there needs to be a way to turn it off.  An environment variable
seems to be the obvious choice (for the reasons you stated for
PYTHONDONTWRITEBYTECODE).

Georg


From solipsis at pitrou.net  Sat Dec 18 13:25:37 2010
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Sat, 18 Dec 2010 13:25:37 +0100
Subject: [Python-Dev] r87188 - python/branches/py3k/Doc/library/test.rst
References: <20101212182525.C5C6DEE9F3@mail.python.org>
	<iei6ft$hq8$1@dough.gmane.org> <20101218130734.1f2f471a@pitrou.net>
	<iei8fl$qb5$1@dough.gmane.org>
Message-ID: <20101218132537.649de53b@pitrou.net>

On Sat, 18 Dec 2010 13:11:09 +0100
Georg Brandl <g.brandl at gmx.net> wrote:

> Am 18.12.2010 13:07, schrieb Antoine Pitrou:
> > On Sat, 18 Dec 2010 12:37:09 +0100
> > Georg Brandl <g.brandl at gmx.net> wrote:
> >> I'd rather keep this a note.  We don't want to use warnings except for cases
> >> where there is a possibility of security implications or crashes.
> > 
> > Well, there'll be a crash as soon as someone relies on an API we decide
> > to change or remove :)
> 
> Ah, but I meant a real crash, not an exception.  Think marshal or ctypes.

We can add one if necessary ;)

Antoine.



From foom at fuhm.net  Sat Dec 18 14:50:49 2010
From: foom at fuhm.net (James Y Knight)
Date: Sat, 18 Dec 2010 08:50:49 -0500
Subject: [Python-Dev] Issue #8863 adds a new PYTHONNOFAULTHANDLER
	environment variable
In-Reply-To: <4D0C0688.1080108@haypocalc.com>
References: <4D0C0688.1080108@haypocalc.com>
Message-ID: <B880623A-1EA5-41CA-A58C-C46C93BED8DC@fuhm.net>


On Dec 17, 2010, at 7:55 PM, Victor Stinner wrote:

> Hi,
> 
> In 2008, I proposed a patch to raise a Python exception on SIGSEVG signal. In some cases, it's possible to catch the exception, log it, and continue the execution. But because in some cases, the Python internal state is corrupted, the idea was rejected (see the issue #3999).
> 
> Someone asked me to display the Python backtrace on SIGSEGV, instead of raising an exception. I implemented this idea in issue #8863. After 9 versions, I think that the patch is ready for inclusion. It catchs SIGSEGV, SIGFPE, SIGBUS and SIGILL signals, and also display the Python backtrace on fatal errors. Because some operating systems have their own fault handler (eg. Ubuntu with apport), Dave Malcolm asked me to add an option disable the Python handler. 

I think instead of calling abort() to kill the process, you should:
- install the signal handler with SA_NODEFER|SA_RESETHAND (or if sigaction is not present, explicitly reset the action to SIG_DFL and unblock first thing upon entering the handler), and then,
- at the end of the handler, kill(getpid(), orig_signal) in order to abort the process.

This has two advantages: 1) the process's exit code will actually show the correct signal, 2) it might let the OS fault handlers work properly as well -- I'm not sure. If it does, you may want to experiment with whether having or omitting SA_NODEFER gives a better backtrace (from the OS mechanism) in that case.

If #2 actually works, you may not even need the env var, which would be nice. 

James

From victor.stinner at haypocalc.com  Sat Dec 18 14:57:12 2010
From: victor.stinner at haypocalc.com (Victor Stinner)
Date: Sat, 18 Dec 2010 14:57:12 +0100
Subject: [Python-Dev] Issue #8863 adds a new
 PYTHONNOFAULTHANDLER	environment variable
In-Reply-To: <iei93h$tla$1@dough.gmane.org>
References: <4D0C0688.1080108@haypocalc.com> <iei93h$tla$1@dough.gmane.org>
Message-ID: <4D0CBDB8.6010608@haypocalc.com>

Le 18/12/2010 13:21, Georg Brandl a ?crit :
> I very much like having a traceback on (some) segmentation faults,
Why do you say "some" segmentation faults?
> but it's clear there needs to be a way to turn it off.  An environment variable
> seems to be the obvious choice (for the reasons you stated for
> PYTHONDONTWRITEBYTECODE).
Amaury proposed a new function sys.setsegfaultenabled(). Such function 
can be used in a customized site module. I think that I will implement 
it in my next patch, before commiting the new feature. I suppose that we 
also need a function to get the status of the fault handler.

I just don't like the name. I would prefer sys.setfaulthandlerenabled(), 
I maybe just sys.setfaulthandler(). As this feature is specific to 
CPython, should it be protected function? So:
  * sys._setfaulthandler(bool): enable/disable the fault handler
  * sys._getfaulthandler()->bool: get the status (enabled/disabled) of 
the fault handler

Victor

From phd at phd.pp.ru  Sat Dec 18 16:33:31 2010
From: phd at phd.pp.ru (Oleg Broytman)
Date: Sat, 18 Dec 2010 18:33:31 +0300
Subject: [Python-Dev] Locale-specific formatting
In-Reply-To: <4D0BFB8F.2040301@mrabarnett.plus.com>
References: <4D0BFB8F.2040301@mrabarnett.plus.com>
Message-ID: <20101218153331.GA22017@phd.pp.ru>

On Sat, Dec 18, 2010 at 12:08:47AM +0000, MRAB wrote:
> This makes it harder to use more than one locale at a time

   This is quite a known problem, not specific to Python. Locale
settings are global for a process, and this is one of the thousands
reasons why locale is considered so horrible.
   ICU is perhaps the only way around the problem.

Oleg.
-- 
     Oleg Broytman            http://phd.pp.ru/            phd at phd.pp.ru
           Programmers don't die, they just GOSUB without RETURN.

From g.brandl at gmx.net  Sat Dec 18 17:23:12 2010
From: g.brandl at gmx.net (Georg Brandl)
Date: Sat, 18 Dec 2010 17:23:12 +0100
Subject: [Python-Dev] Issue #8863 adds a new PYTHONNOFAULTHANDLER
	environment variable
In-Reply-To: <4D0CBDB8.6010608@haypocalc.com>
References: <4D0C0688.1080108@haypocalc.com> <iei93h$tla$1@dough.gmane.org>
	<4D0CBDB8.6010608@haypocalc.com>
Message-ID: <iein87$mud$1@dough.gmane.org>

Am 18.12.2010 14:57, schrieb Victor Stinner:
> Le 18/12/2010 13:21, Georg Brandl a ?crit :
>> I very much like having a traceback on (some) segmentation faults,
> Why do you say "some" segmentation faults?

Well, without a closer I assume that for some crashes it's just not
possible anymore for the Python interpreter to even print out the
traceback?

Georg


From g.brandl at gmx.net  Sat Dec 18 17:50:38 2010
From: g.brandl at gmx.net (Georg Brandl)
Date: Sat, 18 Dec 2010 17:50:38 +0100
Subject: [Python-Dev] Issue #8863 adds a new PYTHONNOFAULTHANDLER
	environment variable
In-Reply-To: <4D0CBDB8.6010608@haypocalc.com>
References: <4D0C0688.1080108@haypocalc.com> <iei93h$tla$1@dough.gmane.org>
	<4D0CBDB8.6010608@haypocalc.com>
Message-ID: <ieiorl$tlu$1@dough.gmane.org>

Am 18.12.2010 14:57, schrieb Victor Stinner:
> Le 18/12/2010 13:21, Georg Brandl a ?crit :
>> I very much like having a traceback on (some) segmentation faults,
> Why do you say "some" segmentation faults?
>> but it's clear there needs to be a way to turn it off.  An environment variable
>> seems to be the obvious choice (for the reasons you stated for
>> PYTHONDONTWRITEBYTECODE).
> Amaury proposed a new function sys.setsegfaultenabled(). Such function 
> can be used in a customized site module. I think that I will implement 
> it in my next patch, before commiting the new feature. I suppose that we 
> also need a function to get the status of the fault handler.
> 
> I just don't like the name. I would prefer sys.setfaulthandlerenabled(), 
> I maybe just sys.setfaulthandler(). As this feature is specific to 
> CPython, should it be protected function? So:
>   * sys._setfaulthandler(bool): enable/disable the fault handler
>   * sys._getfaulthandler()->bool: get the status (enabled/disabled) of 
> the fault handler

In any case, this is coming pretty late; beta 2 is scheduled for this
weekend, and even if this is something that only kicks in when all hope
is lost anyway, it is a new feature.  I should like to hear approval
from a few more devs before I will let this go into 3.2.

Georg



From solipsis at pitrou.net  Sat Dec 18 18:50:03 2010
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Sat, 18 Dec 2010 18:50:03 +0100
Subject: [Python-Dev] Issue #8863 adds a new PYTHONNOFAULTHANDLER
 environment variable
References: <4D0C0688.1080108@haypocalc.com> <iei93h$tla$1@dough.gmane.org>
	<4D0CBDB8.6010608@haypocalc.com> <ieiorl$tlu$1@dough.gmane.org>
Message-ID: <20101218185003.3298ba57@pitrou.net>

On Sat, 18 Dec 2010 17:50:38 +0100
Georg Brandl <g.brandl at gmx.net> wrote:
> Am 18.12.2010 14:57, schrieb Victor Stinner:
> > Le 18/12/2010 13:21, Georg Brandl a ?crit :
> >> I very much like having a traceback on (some) segmentation faults,
> > Why do you say "some" segmentation faults?
> >> but it's clear there needs to be a way to turn it off.  An environment variable
> >> seems to be the obvious choice (for the reasons you stated for
> >> PYTHONDONTWRITEBYTECODE).
> > Amaury proposed a new function sys.setsegfaultenabled(). Such function 
> > can be used in a customized site module. I think that I will implement 
> > it in my next patch, before commiting the new feature. I suppose that we 
> > also need a function to get the status of the fault handler.
> > 
> > I just don't like the name. I would prefer sys.setfaulthandlerenabled(), 
> > I maybe just sys.setfaulthandler(). As this feature is specific to 
> > CPython, should it be protected function? So:
> >   * sys._setfaulthandler(bool): enable/disable the fault handler
> >   * sys._getfaulthandler()->bool: get the status (enabled/disabled) of 
> > the fault handler
> 
> In any case, this is coming pretty late; beta 2 is scheduled for this
> weekend, and even if this is something that only kicks in when all hope
> is lost anyway, it is a new feature.  I should like to hear approval
> from a few more devs before I will let this go into 3.2.

I'm +1 on Victor's patch, although I don't think the sys function is
useful at all. The env var is enough IMO.
(I would even be for backporting it to the bugfix branches if there's
no downside)

Regards

Antoine.



From python at mrabarnett.plus.com  Sat Dec 18 19:26:43 2010
From: python at mrabarnett.plus.com (MRAB)
Date: Sat, 18 Dec 2010 18:26:43 +0000
Subject: [Python-Dev] Locale-specific formatting
In-Reply-To: <4D0C7E4A.3010302@v.loewis.de>
References: <4D0BFB8F.2040301@mrabarnett.plus.com>
	<4D0C7E4A.3010302@v.loewis.de>
Message-ID: <4D0CFCE3.2050906@mrabarnett.plus.com>

On 18/12/2010 09:26, "Martin v. L?wis" wrote:
>> Comments?
>
> How do you implement that? In particular, how do you retrieve
> information for different locales in a single program?
>
The locale module would be able to return a named locale dict:

 >>> loc = locale.getnamedlocale('en_UK')

or:

 >>> loc = locale.namedlocales['en_UK']

The dict would be like what locale.localeconv() returns.

The str.format method would then be able to use that dict.

From alexander.belopolsky at gmail.com  Sat Dec 18 19:55:00 2010
From: alexander.belopolsky at gmail.com (Alexander Belopolsky)
Date: Sat, 18 Dec 2010 13:55:00 -0500
Subject: [Python-Dev] Issue #8863 adds a new PYTHONNOFAULTHANDLER
 environment variable
In-Reply-To: <ieiorl$tlu$1@dough.gmane.org>
References: <4D0C0688.1080108@haypocalc.com> <iei93h$tla$1@dough.gmane.org>
	<4D0CBDB8.6010608@haypocalc.com> <ieiorl$tlu$1@dough.gmane.org>
Message-ID: <AANLkTimq+Qpt80u50Nj+RTgM7CgfLPsYWR2RHgDkZkP6@mail.gmail.com>

On Sat, Dec 18, 2010 at 11:50 AM, Georg Brandl <g.brandl at gmx.net> wrote:
..
> In any case, this is coming pretty late; beta 2 is scheduled for this
> weekend, and even if this is something that only kicks in when all hope
> is lost anyway, it is a new feature. ?I should like to hear approval
> from a few more devs before I will let this go into 3.2.
>

I am -1 on the feature as written.  I would be -0 if it did not
install signal handlers by default and even better was implemented in
a separate module, not in core.

This feature is similar to the one that is implemented by R and with
R, it was always a problem when R was embedded as a library.  It has
always been a welcome feature of Python that its core did not mess
with the application global state.

Before this is approved, I would like to see a discussion of the
consequences for embedded Python and particularly for the case when
there are multiple interpreters in the same process.

I think it would be best to start with this feature as an library
module or even as a 3rd party add-on and see how useful it will be
found in the wild.

From martin at v.loewis.de  Sat Dec 18 20:36:50 2010
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sat, 18 Dec 2010 20:36:50 +0100
Subject: [Python-Dev] Issue #8863 adds a new PYTHONNOFAULTHANDLER
 environment variable
In-Reply-To: <AANLkTimq+Qpt80u50Nj+RTgM7CgfLPsYWR2RHgDkZkP6@mail.gmail.com>
References: <4D0C0688.1080108@haypocalc.com>
	<iei93h$tla$1@dough.gmane.org>	<4D0CBDB8.6010608@haypocalc.com>
	<ieiorl$tlu$1@dough.gmane.org>
	<AANLkTimq+Qpt80u50Nj+RTgM7CgfLPsYWR2RHgDkZkP6@mail.gmail.com>
Message-ID: <4D0D0D52.5020204@v.loewis.de>

Am 18.12.2010 19:55, schrieb Alexander Belopolsky:
> On Sat, Dec 18, 2010 at 11:50 AM, Georg Brandl <g.brandl at gmx.net> wrote:
> ..
>> In any case, this is coming pretty late; beta 2 is scheduled for this
>> weekend, and even if this is something that only kicks in when all hope
>> is lost anyway, it is a new feature.  I should like to hear approval
>> from a few more devs before I will let this go into 3.2.
>>
> 
> I am -1 on the feature as written.  I would be -0 if it did not
> install signal handlers by default and even better was implemented in
> a separate module, not in core.

This is also what I think. Installing a signal handler is a fairly
drastic action, and I don't think the code has been sufficiently
reviewed yet.

So I'd rather see this deferred until after 3.2, to allow for additional
reviews (e.g. if some of the Twisted guys would approve it - that would
be convincing).

I also share Alexander's concern that Python just shouldn't mess with
signal handlers at all, ideally. So some trade-off has to be found to
address that concern (e.g. by making the signal handlers only active
for the executable interpreter, but not in the embedded case).

Regards,
Martin

From solipsis at pitrou.net  Sat Dec 18 21:18:36 2010
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Sat, 18 Dec 2010 21:18:36 +0100
Subject: [Python-Dev] Issue #8863 adds a new PYTHONNOFAULTHANDLER
 environment variable
References: <4D0C0688.1080108@haypocalc.com> <iei93h$tla$1@dough.gmane.org>
	<4D0CBDB8.6010608@haypocalc.com> <ieiorl$tlu$1@dough.gmane.org>
	<AANLkTimq+Qpt80u50Nj+RTgM7CgfLPsYWR2RHgDkZkP6@mail.gmail.com>
	<4D0D0D52.5020204@v.loewis.de>
Message-ID: <20101218211836.0c2b0792@pitrou.net>

On Sat, 18 Dec 2010 20:36:50 +0100
"Martin v. L?wis" <martin at v.loewis.de> wrote:
> Am 18.12.2010 19:55, schrieb Alexander Belopolsky:
> > On Sat, Dec 18, 2010 at 11:50 AM, Georg Brandl <g.brandl at gmx.net> wrote:
> > ..
> >> In any case, this is coming pretty late; beta 2 is scheduled for this
> >> weekend, and even if this is something that only kicks in when all hope
> >> is lost anyway, it is a new feature.  I should like to hear approval
> >> from a few more devs before I will let this go into 3.2.
> >>
> > 
> > I am -1 on the feature as written.  I would be -0 if it did not
> > install signal handlers by default and even better was implemented in
> > a separate module, not in core.
> 
> This is also what I think. Installing a signal handler is a fairly
> drastic action, and I don't think the code has been sufficiently
> reviewed yet.

How much more review should it receive?

> I also share Alexander's concern that Python just shouldn't mess with
> signal handlers at all, ideally. So some trade-off has to be found to
> address that concern (e.g. by making the signal handlers only active
> for the executable interpreter, but not in the embedded case).

Well, Python already does (and also apparently messes with other things
such as FPU state), so that's a separate issue altogether.

Regards

Antoine.



From solipsis at pitrou.net  Sat Dec 18 21:48:30 2010
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Sat, 18 Dec 2010 21:48:30 +0100
Subject: [Python-Dev] r87389 - in python/branches/py3k:
 Doc/library/unittest.rst Lib/unittest/case.py Misc/NEWS
References: <20101218200004.99110EE985@mail.python.org>
Message-ID: <20101218214830.1aa8be32@pitrou.net>

On Sat, 18 Dec 2010 21:00:04 +0100 (CET)
ezio.melotti <python-checkins at python.org> wrote:
> Author: ezio.melotti
> Date: Sat Dec 18 21:00:04 2010
> New Revision: 87389
> 
> Log:
> #10573: use actual/expected consistently in unittest methods.

IMHO, this should be reverted. The API currently doesn't treat these
arguments differently, so they should really be labeled "first" and
"second". Otherwise, the user will wrongly assume that the signature is
asymmetric and that they should be careful about which order they pass
the arguments in.

Regards

Antoine.



From g.brandl at gmx.net  Sat Dec 18 22:04:02 2010
From: g.brandl at gmx.net (Georg Brandl)
Date: Sat, 18 Dec 2010 22:04:02 +0100
Subject: [Python-Dev] A grammatical oddity: trailing commas in argument
	lists -- continuation
In-Reply-To: <4D067D3F.8030206@g.nevcal.com>
References: <20101213114449.GA2348@chopin.edu.pl>	<AANLkTi=OM921Xt7ZFv=E6-Gz3sLeLPYC4-BUGOv-f+-L@mail.gmail.com>	<20101213155119.A52A621CD91@kimball.webabinitio.net>	<AANLkTimCQYd8Q6BJQPXtr3mo_Rk=MZi4iZBcUX5z4uTB@mail.gmail.com>
	<4D067D3F.8030206@g.nevcal.com>
Message-ID: <iej7mo$pt7$1@dough.gmane.org>

Am 13.12.2010 21:08, schrieb Glenn Linderman:
> On 12/13/2010 11:39 AM, Mark Dickinson wrote:
>> my_thing = Thing(
>>      foo = Foo(arg1, arg2, ...),
>>      bar = Bar(arg3, arg4, ...),
>>      ...
>> )
>>
>> and I've found the trailing comma very convenient during refactoring
>> and API experimentation.  (There's still good fun to be had arguing
>> about the indentation of that closing parenthesis, though.)
> 
> 
> Clearly it needs to be indented one level, because it is a continuation 
> of the prior line, just like the foo and bar and ... lines are 
> continuations and therefore indented.

Clearly Emacs is superior to Vim because (insert some random fact here).

Clearly the only thing that is clear about coding style details (even if
we all more or less agree on PEP 8) is that it is a matter of personal
taste.

Georg


From v+python at g.nevcal.com  Sat Dec 18 23:41:09 2010
From: v+python at g.nevcal.com (Glenn Linderman)
Date: Sat, 18 Dec 2010 14:41:09 -0800
Subject: [Python-Dev] A grammatical oddity: trailing commas in argument
 lists -- continuation
In-Reply-To: <iej7mo$pt7$1@dough.gmane.org>
References: <20101213114449.GA2348@chopin.edu.pl>	<AANLkTi=OM921Xt7ZFv=E6-Gz3sLeLPYC4-BUGOv-f+-L@mail.gmail.com>	<20101213155119.A52A621CD91@kimball.webabinitio.net>	<AANLkTimCQYd8Q6BJQPXtr3mo_Rk=MZi4iZBcUX5z4uTB@mail.gmail.com>	<4D067D3F.8030206@g.nevcal.com>
	<iej7mo$pt7$1@dough.gmane.org>
Message-ID: <4D0D3885.80903@g.nevcal.com>

On 12/18/2010 1:04 PM, Georg Brandl wrote:
> Am 13.12.2010 21:08, schrieb Glenn Linderman:
>> On 12/13/2010 11:39 AM, Mark Dickinson wrote:
>>> my_thing = Thing(
>>>       foo = Foo(arg1, arg2, ...),
>>>       bar = Bar(arg3, arg4, ...),
>>>       ...
>>> )
>>>
>>> and I've found the trailing comma very convenient during refactoring
>>> and API experimentation.  (There's still good fun to be had arguing
>>> about the indentation of that closing parenthesis, though.)
>>
>> Clearly it needs to be indented one level, because it is a continuation
>> of the prior line, just like the foo and bar and ... lines are
>> continuations and therefore indented.
> Clearly Emacs is superior to Vim because (insert some random fact here).

Sure you have that right.

> Clearly the only thing that is clear about coding style details (even if
> we all more or less agree on PEP 8) is that it is a matter of personal
> taste.
And this too.

But apparently you missed the fact that Mark wanted some good fun 
arguing about the indentation of the closing parenthesis... and didn't 
quote my "Enjoy!" that implied that that was all I was giving him.

But then, you are release manager, which would make it very difficult, 
but hopefully you can still have a Merry Christmas! (or whatever 
end-of-year holiday suits your fancy)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20101218/e8abfd5a/attachment.html>

From tjreedy at udel.edu  Sat Dec 18 23:46:23 2010
From: tjreedy at udel.edu (Terry Reedy)
Date: Sat, 18 Dec 2010 17:46:23 -0500
Subject: [Python-Dev] r87389 - in python/branches/py3k:
 Doc/library/unittest.rst Lib/unittest/case.py Misc/NEWS
In-Reply-To: <20101218214830.1aa8be32@pitrou.net>
References: <20101218200004.99110EE985@mail.python.org>
	<20101218214830.1aa8be32@pitrou.net>
Message-ID: <iejdjt$ik0$1@dough.gmane.org>

On 12/18/2010 3:48 PM, Antoine Pitrou wrote:
> On Sat, 18 Dec 2010 21:00:04 +0100 (CET)
> ezio.melotti<python-checkins at python.org>  wrote:
>> Author: ezio.melotti
>> Date: Sat Dec 18 21:00:04 2010
>> New Revision: 87389
>>
>> Log:
>> #10573: use actual/expected consistently in unittest methods.

Change was requested by M. Foord and R. Hettinger (and G.Brandl for b2).

> IMHO, this should be reverted. The API currently doesn't treat these
> arguments differently, so they should really be labeled "first" and
> "second". Otherwise, the user will wrongly assume that the signature is
> asymmetric and that they should be careful about which order they pass
> the arguments in.

The error report on assert failure *is* often asymmetrical ;=).
 From Michael's post:
"This is particularly relevant for the methods that produce 'diffed' 
output on failure - as the order determines whether mismatched items are 
missing from the expected or additional to the expected."

This change struck me as a nice bit of polishing.

-- 
Terry Jan Reedy


From tjreedy at udel.edu  Sun Dec 19 00:21:24 2010
From: tjreedy at udel.edu (Terry Reedy)
Date: Sat, 18 Dec 2010 18:21:24 -0500
Subject: [Python-Dev] Locale-specific formatting
In-Reply-To: <20101218153331.GA22017@phd.pp.ru>
References: <4D0BFB8F.2040301@mrabarnett.plus.com>
	<20101218153331.GA22017@phd.pp.ru>
Message-ID: <iejfli$bc$1@dough.gmane.org>

On 12/18/2010 10:33 AM, Oleg Broytman wrote:

>     This is quite a known problem, not specific to Python. Locale
> settings are global for a process, and this is one of the thousands
> reasons why locale is considered so horrible.
>     ICU is perhaps the only way around the problem.

This is about the third mention of 'ICU' in the last few weeks, so I 
looked it up: International Components for Unicode
http://site.icu-project.org/
Several libraries (C/C++,Java), including prebuilt binaries for Windows 
(and some others).
There is already a Python .cpp wrapper (but no Windows binaries, which 
limits usefulness)
http://pyicu.osafoundation.org/
http://pypi.python.org/pypi/PyICU/1.0.1

-- 
Terry Jan Reedy


From grygoriy.fuchedzhy at gmail.com  Sat Dec 18 22:37:40 2010
From: grygoriy.fuchedzhy at gmail.com (Grygoriy Fuchedzhy)
Date: Sat, 18 Dec 2010 23:37:40 +0200
Subject: [Python-Dev] [feature request] add .svgz to mimetypes.suffix_map
Message-ID: <AANLkTinFYbOASZMbRruk5hGAUpF6VLSWxWRCfW1COqM-@mail.gmail.com>

Hi.
I've created bug on bugtracker, and then I was told there that I should post
this on this mailing list for discussion.
Here is link to bug:  http://bugs.python.org/issue10730

Please add '.svgz': '.svg.gz' map to mimetypes.suffix_map

-- 
Grygoriy Fuchedzhy
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20101218/42cdd76a/attachment.html>

From martin at v.loewis.de  Sun Dec 19 01:31:37 2010
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sun, 19 Dec 2010 01:31:37 +0100
Subject: [Python-Dev] Locale-specific formatting
In-Reply-To: <4D0CFCE3.2050906@mrabarnett.plus.com>
References: <4D0BFB8F.2040301@mrabarnett.plus.com>	<4D0C7E4A.3010302@v.loewis.de>
	<4D0CFCE3.2050906@mrabarnett.plus.com>
Message-ID: <4D0D5269.1060304@v.loewis.de>

Am 18.12.2010 19:26, schrieb MRAB:
> On 18/12/2010 09:26, "Martin v. L?wis" wrote:
>>> Comments?
>>
>> How do you implement that? In particular, how do you retrieve
>> information for different locales in a single program?
>>
> The locale module would be able to return a named locale dict:
> 
>>>> loc = locale.getnamedlocale('en_UK')

Ok, so, in turn: how do you implement *that* (what C API
specifically do you use to implement that getnamedlocale function)?

Regards,
Martin

From python at mrabarnett.plus.com  Sun Dec 19 02:11:36 2010
From: python at mrabarnett.plus.com (MRAB)
Date: Sun, 19 Dec 2010 01:11:36 +0000
Subject: [Python-Dev] Locale-specific formatting
In-Reply-To: <4D0D5269.1060304@v.loewis.de>
References: <4D0BFB8F.2040301@mrabarnett.plus.com>	<4D0C7E4A.3010302@v.loewis.de>
	<4D0CFCE3.2050906@mrabarnett.plus.com>
	<4D0D5269.1060304@v.loewis.de>
Message-ID: <4D0D5BC8.7080804@mrabarnett.plus.com>

On 19/12/2010 00:31, "Martin v. L?wis" wrote:
> Am 18.12.2010 19:26, schrieb MRAB:
>> On 18/12/2010 09:26, "Martin v. L?wis" wrote:
>>>> Comments?
>>>
>>> How do you implement that? In particular, how do you retrieve
>>> information for different locales in a single program?
>>>
>> The locale module would be able to return a named locale dict:
>>
>>>>> loc = locale.getnamedlocale('en_UK')
>
> Ok, so, in turn: how do you implement *that* (what C API
> specifically do you use to implement that getnamedlocale function)?
>
I suppose there could be some sort of locale database. A downloadable,
up-to-date copy of the database could be maintained on the Python
website.

From tjreedy at udel.edu  Sun Dec 19 03:24:32 2010
From: tjreedy at udel.edu (Terry Reedy)
Date: Sat, 18 Dec 2010 21:24:32 -0500
Subject: [Python-Dev] [feature request] add .svgz to mimetypes.suffix_map
In-Reply-To: <AANLkTinFYbOASZMbRruk5hGAUpF6VLSWxWRCfW1COqM-@mail.gmail.com>
References: <AANLkTinFYbOASZMbRruk5hGAUpF6VLSWxWRCfW1COqM-@mail.gmail.com>
Message-ID: <iejqcu$3g3$1@dough.gmane.org>

On 12/18/2010 4:37 PM, Grygoriy Fuchedzhy wrote:
> Hi.
> I've created bug on bugtracker, and then I was told there that I should
> post this on this mailing list for discussion.
> Here is link to bug: http://bugs.python.org/issue10730
>
> Please add'.svgz':'.svg.gz'  map to mimetypes.suffix_map

This issue is actually a request to add .svg to the types map and the 
the abbreviation .svgz for .svg.gx to the suffix_map.

I believe Scalable Vector Graphics are well on the way to becoming *the* 
standard vector graphics format for the web, especially with upcoming 
IE9 support. https://secure.wikimedia.org/wikipedia/en/wiki/Svg
It (they) are already supported by the other major browsers. In 
addition, "Google announced on 31 August 2010 that it had begun to index 
SVG content on the web, whether it is in standalone files or embedded in 
HTML."

So it might be sensible to add these now rather than two years from now. 
But if it is too late for 3.2, then I expect addition for 3.3 will be 
obvious.

-- 
Terry Jan Reedy


From guido at python.org  Sun Dec 19 05:23:49 2010
From: guido at python.org (Guido van Rossum)
Date: Sat, 18 Dec 2010 20:23:49 -0800
Subject: [Python-Dev] r87389 - in python/branches/py3k:
 Doc/library/unittest.rst Lib/unittest/case.py Misc/NEWS
In-Reply-To: <iejdjt$ik0$1@dough.gmane.org>
References: <20101218200004.99110EE985@mail.python.org>
	<20101218214830.1aa8be32@pitrou.net> <iejdjt$ik0$1@dough.gmane.org>
Message-ID: <AANLkTi=ADkUMc5tQGDv2yniteyfPCPXU=XaCr+GscR+a@mail.gmail.com>

I may be unique, but I fear there is no great answer. On the one hand
I almost always code it as e.g. assertEqual(actual, expected), which
matches my preference for e.g. "if x == 5:" rather than "if 5 == x:".
On the other hand in those assert* functions that show a nice diff of
two lists, when reading such a diff my expectation is that "old, new"
corresponds to "expected, actual". Which then freaks me out until I
realize that I coded it as "actual, expected"... And yet "expected,
actual" still looks weird to me. :-(

On Sat, Dec 18, 2010 at 2:46 PM, Terry Reedy <tjreedy at udel.edu> wrote:
> On 12/18/2010 3:48 PM, Antoine Pitrou wrote:
>>
>> On Sat, 18 Dec 2010 21:00:04 +0100 (CET)
>> ezio.melotti<python-checkins at python.org> ?wrote:
>>>
>>> Author: ezio.melotti
>>> Date: Sat Dec 18 21:00:04 2010
>>> New Revision: 87389
>>>
>>> Log:
>>> #10573: use actual/expected consistently in unittest methods.
>
> Change was requested by M. Foord and R. Hettinger (and G.Brandl for b2).
>
>> IMHO, this should be reverted. The API currently doesn't treat these
>> arguments differently, so they should really be labeled "first" and
>> "second". Otherwise, the user will wrongly assume that the signature is
>> asymmetric and that they should be careful about which order they pass
>> the arguments in.
>
> The error report on assert failure *is* often asymmetrical ;=).
> From Michael's post:
> "This is particularly relevant for the methods that produce 'diffed' output
> on failure - as the order determines whether mismatched items are missing
> from the expected or additional to the expected."
>
> This change struck me as a nice bit of polishing.
>
> --
> Terry Jan Reedy
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> http://mail.python.org/mailman/options/python-dev/guido%40python.org
>



-- 
--Guido van Rossum (python.org/~guido)

From rdmurray at bitdance.com  Sun Dec 19 07:52:37 2010
From: rdmurray at bitdance.com (R. David Murray)
Date: Sun, 19 Dec 2010 01:52:37 -0500
Subject: [Python-Dev] r87389 - in python/branches/py3k:
	Doc/library/unittest.rst Lib/unittest/case.py Misc/NEWS
In-Reply-To: <AANLkTi=ADkUMc5tQGDv2yniteyfPCPXU=XaCr+GscR+a@mail.gmail.com>
References: <20101218200004.99110EE985@mail.python.org>
	<20101218214830.1aa8be32@pitrou.net> <iejdjt$ik0$1@dough.gmane.org>
	<AANLkTi=ADkUMc5tQGDv2yniteyfPCPXU=XaCr+GscR+a@mail.gmail.com>
Message-ID: <20101219065237.9FF242379ED@kimball.webabinitio.net>

On Sat, 18 Dec 2010 20:23:49 -0800, Guido van Rossum <guido at python.org> wrote:
> I may be unique, but I fear there is no great answer. On the one hand
> I almost always code it as e.g. assertEqual(actual, expected), which
> matches my preference for e.g. "if x =3D=3D 5:" rather than "if 5 =3D=3D x:=
> ".
> On the other hand in those assert* functions that show a nice diff of
> two lists, when reading such a diff my expectation is that "old, new"
> corresponds to "expected, actual". Which then freaks me out until I
> realize that I coded it as "actual, expected"... And yet "expected,
> actual" still looks weird to me. :-(

You aren't unique, I feel the same way.  But it seems to me that the most
important thing is to be consistent, so that I don't freak out for long.

--
R. David Murray                                      www.bitdance.com

From phd at phd.pp.ru  Sun Dec 19 11:44:00 2010
From: phd at phd.pp.ru (Oleg Broytman)
Date: Sun, 19 Dec 2010 13:44:00 +0300
Subject: [Python-Dev] Locale-specific formatting
In-Reply-To: <iejfli$bc$1@dough.gmane.org>
References: <4D0BFB8F.2040301@mrabarnett.plus.com>
	<20101218153331.GA22017@phd.pp.ru> <iejfli$bc$1@dough.gmane.org>
Message-ID: <20101219104400.GA21740@phd.pp.ru>

On Sat, Dec 18, 2010 at 06:21:24PM -0500, Terry Reedy wrote:
> On 12/18/2010 10:33 AM, Oleg Broytman wrote:
>
>>     This is quite a known problem, not specific to Python. Locale
>> settings are global for a process, and this is one of the thousands
>> reasons why locale is considered so horrible.
>>     ICU is perhaps the only way around the problem.
>
> This is about the third mention of 'ICU' in the last few weeks, so I  
> looked it up: International Components for Unicode
> http://site.icu-project.org/
> Several libraries (C/C++,Java), including prebuilt binaries for Windows  
> (and some others).
> There is already a Python .cpp wrapper (but no Windows binaries, which  
> limits usefulness)
> http://pyicu.osafoundation.org/
> http://pypi.python.org/pypi/PyICU/1.0.1

   A month ago there was a long thread that mentioned ICU many times:
http://mail.python.org/pipermail/python-dev/2010-November/106068.html

Oleg.
-- 
     Oleg Broytman            http://phd.pp.ru/            phd at phd.pp.ru
           Programmers don't die, they just GOSUB without RETURN.

From solipsis at pitrou.net  Sun Dec 19 14:13:55 2010
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Sun, 19 Dec 2010 14:13:55 +0100
Subject: [Python-Dev] r87389 - in python/branches/py3k:
 Doc/library/unittest.rst Lib/unittest/case.py Misc/NEWS
References: <20101218200004.99110EE985@mail.python.org>
	<20101218214830.1aa8be32@pitrou.net> <iejdjt$ik0$1@dough.gmane.org>
	<AANLkTi=ADkUMc5tQGDv2yniteyfPCPXU=XaCr+GscR+a@mail.gmail.com>
Message-ID: <20101219141355.5ac7cd2f@pitrou.net>

On Sat, 18 Dec 2010 20:23:49 -0800
Guido van Rossum <guido at python.org> wrote:
> I may be unique, but I fear there is no great answer. On the one hand
> I almost always code it as e.g. assertEqual(actual, expected), which
> matches my preference for e.g. "if x == 5:" rather than "if 5 == x:".
> On the other hand in those assert* functions that show a nice diff of
> two lists, when reading such a diff my expectation is that "old, new"
> corresponds to "expected, actual". Which then freaks me out until I
> realize that I coded it as "actual, expected"... And yet "expected,
> actual" still looks weird to me. :-(

This could be nicely resolved by renaming the arguments "a" and "b",
and having the diff display "a, b". It's quite natural (both the diff
ordering and the arguments ordering), and they are consistent with each
other.

Regards

Antoine.



From solipsis at pitrou.net  Sun Dec 19 14:23:02 2010
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Sun, 19 Dec 2010 14:23:02 +0100
Subject: [Python-Dev] r87399 - in python/branches/py3k:
 Doc/library/http.client.rst Doc/library/urllib.request.rst
 Lib/http/client.py Lib/test/test_httplib.py Lib/test/test_urllib2.py
 Lib/urllib/request.py Misc/NEWS
References: <20101219104952.EABC8EE9A9@mail.python.org>
Message-ID: <20101219142302.399ef3b4@pitrou.net>


Hello Senthil,

> +                raise TypeError("data should be byte-like object\

Should be "bytes-like".

> +                    request.add_unredirected_header(
> +                            'Content-length', '%d' % len(mv) * mv.itemsize)

There is an operator precedence problem here:

>>> "%d" % 4 * 5
'44444'


> +        """
> +        file_obj = io.StringIO()
> +        file_obj.write("Something\nSomething\nSomething\n")
> +

Why is this whole thing commented out? If it wasn't, you would have seen
the test failing.

Thanks

Antoine.



From guido at python.org  Sun Dec 19 19:41:34 2010
From: guido at python.org (Guido van Rossum)
Date: Sun, 19 Dec 2010 10:41:34 -0800
Subject: [Python-Dev] r87389 - in python/branches/py3k:
 Doc/library/unittest.rst Lib/unittest/case.py Misc/NEWS
In-Reply-To: <20101219141355.5ac7cd2f@pitrou.net>
References: <20101218200004.99110EE985@mail.python.org>
	<20101218214830.1aa8be32@pitrou.net> <iejdjt$ik0$1@dough.gmane.org>
	<AANLkTi=ADkUMc5tQGDv2yniteyfPCPXU=XaCr+GscR+a@mail.gmail.com>
	<20101219141355.5ac7cd2f@pitrou.net>
Message-ID: <AANLkTinyvb6+Xf=owdV22j6YoAEVjbTXV2tetQzdD8r4@mail.gmail.com>

On Sun, Dec 19, 2010 at 5:13 AM, Antoine Pitrou <solipsis at pitrou.net> wrote:
> On Sat, 18 Dec 2010 20:23:49 -0800
> Guido van Rossum <guido at python.org> wrote:
>> I may be unique, but I fear there is no great answer. On the one hand
>> I almost always code it as e.g. assertEqual(actual, expected), which
>> matches my preference for e.g. "if x == 5:" rather than "if 5 == x:".
>> On the other hand in those assert* functions that show a nice diff of
>> two lists, when reading such a diff my expectation is that "old, new"
>> corresponds to "expected, actual". Which then freaks me out until I
>> realize that I coded it as "actual, expected"... And yet "expected,
>> actual" still looks weird to me. :-(
>
> This could be nicely resolved by renaming the arguments "a" and "b",
> and having the diff display "a, b". It's quite natural (both the diff
> ordering and the arguments ordering), and they are consistent with each
> other.

So 'a' stands for 'after' and 'b' for 'before', right? :-)

-- 
--Guido van Rossum (python.org/~guido)

From solipsis at pitrou.net  Sun Dec 19 19:49:49 2010
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Sun, 19 Dec 2010 19:49:49 +0100
Subject: [Python-Dev] r87389 - in python/branches/py3k:
 Doc/library/unittest.rst Lib/unittest/case.py Misc/NEWS
In-Reply-To: <AANLkTinyvb6+Xf=owdV22j6YoAEVjbTXV2tetQzdD8r4@mail.gmail.com>
References: <20101218200004.99110EE985@mail.python.org>
	<20101218214830.1aa8be32@pitrou.net> <iejdjt$ik0$1@dough.gmane.org>
	<AANLkTi=ADkUMc5tQGDv2yniteyfPCPXU=XaCr+GscR+a@mail.gmail.com>
	<20101219141355.5ac7cd2f@pitrou.net>
	<AANLkTinyvb6+Xf=owdV22j6YoAEVjbTXV2tetQzdD8r4@mail.gmail.com>
Message-ID: <1292784589.3679.1.camel@localhost.localdomain>

Le dimanche 19 d?cembre 2010 ? 10:41 -0800, Guido van Rossum a ?crit :
> On Sun, Dec 19, 2010 at 5:13 AM, Antoine Pitrou <solipsis at pitrou.net> wrote:
> > On Sat, 18 Dec 2010 20:23:49 -0800
> > Guido van Rossum <guido at python.org> wrote:
> >> I may be unique, but I fear there is no great answer. On the one hand
> >> I almost always code it as e.g. assertEqual(actual, expected), which
> >> matches my preference for e.g. "if x == 5:" rather than "if 5 == x:".
> >> On the other hand in those assert* functions that show a nice diff of
> >> two lists, when reading such a diff my expectation is that "old, new"
> >> corresponds to "expected, actual". Which then freaks me out until I
> >> realize that I coded it as "actual, expected"... And yet "expected,
> >> actual" still looks weird to me. :-(
> >
> > This could be nicely resolved by renaming the arguments "a" and "b",
> > and having the diff display "a, b". It's quite natural (both the diff
> > ordering and the arguments ordering), and they are consistent with each
> > other.
> 
> So 'a' stands for 'after' and 'b' for 'before', right? :-)

Ouch. I guess I don't natively think in English.




From martin at v.loewis.de  Sun Dec 19 19:49:30 2010
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sun, 19 Dec 2010 19:49:30 +0100
Subject: [Python-Dev] Locale-specific formatting
In-Reply-To: <4D0D5BC8.7080804@mrabarnett.plus.com>
References: <4D0BFB8F.2040301@mrabarnett.plus.com>	<4D0C7E4A.3010302@v.loewis.de>	<4D0CFCE3.2050906@mrabarnett.plus.com>	<4D0D5269.1060304@v.loewis.de>
	<4D0D5BC8.7080804@mrabarnett.plus.com>
Message-ID: <4D0E53BA.40101@v.loewis.de>

> I suppose there could be some sort of locale database. A downloadable,
> up-to-date copy of the database could be maintained on the Python
> website.

I think you are quite underestimating the implementation effort.
So -0 on your original proposal until such a thing actually exists.

Regards,
Martin

From martin at v.loewis.de  Sun Dec 19 19:53:43 2010
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sun, 19 Dec 2010 19:53:43 +0100
Subject: [Python-Dev] Issue #8863 adds a new PYTHONNOFAULTHANDLER
 environment variable
In-Reply-To: <20101218211836.0c2b0792@pitrou.net>
References: <4D0C0688.1080108@haypocalc.com>
	<iei93h$tla$1@dough.gmane.org>	<4D0CBDB8.6010608@haypocalc.com>
	<ieiorl$tlu$1@dough.gmane.org>	<AANLkTimq+Qpt80u50Nj+RTgM7CgfLPsYWR2RHgDkZkP6@mail.gmail.com>	<4D0D0D52.5020204@v.loewis.de>
	<20101218211836.0c2b0792@pitrou.net>
Message-ID: <4D0E54B7.2070604@v.loewis.de>

>> This is also what I think. Installing a signal handler is a fairly
>> drastic action, and I don't think the code has been sufficiently
>> reviewed yet.
> 
> How much more review should it receive?

There should be at least one reviewer with an established track record
of being interested/knowledgable in the POSIX APIs and cross-platform
aspects of it. As I said, any of the Twisted guys would qualify.

> Well, Python already does (and also apparently messes with other things
> such as FPU state), so that's a separate issue altogether.

Not at all. Some signal handlers are more likely to interfere with the
the rest of the application than others.

Regards,
Martin

From solipsis at pitrou.net  Sun Dec 19 20:20:22 2010
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Sun, 19 Dec 2010 20:20:22 +0100
Subject: [Python-Dev] Issue #8863 adds a new PYTHONNOFAULTHANDLER
 environment variable
In-Reply-To: <4D0E54B7.2070604@v.loewis.de>
References: <4D0C0688.1080108@haypocalc.com> <iei93h$tla$1@dough.gmane.org>
	<4D0CBDB8.6010608@haypocalc.com> <ieiorl$tlu$1@dough.gmane.org>
	<AANLkTimq+Qpt80u50Nj+RTgM7CgfLPsYWR2RHgDkZkP6@mail.gmail.com>
	<4D0D0D52.5020204@v.loewis.de> <20101218211836.0c2b0792@pitrou.net>
	<4D0E54B7.2070604@v.loewis.de>
Message-ID: <1292786422.3679.24.camel@localhost.localdomain>


Le dimanche 19 d?cembre 2010 ? 19:53 +0100, "Martin v. L?wis" a ?crit :
> >> This is also what I think. Installing a signal handler is a fairly
> >> drastic action, and I don't think the code has been sufficiently
> >> reviewed yet.
> > 
> > How much more review should it receive?
> 
> There should be at least one reviewer with an established track record
> of being interested/knowledgable in the POSIX APIs and cross-platform
> aspects of it.
> As I said, any of the Twisted guys would qualify.

James Knight already posted in this thread. Is this sufficient to you?

I don't think it's reasonable to expect "the Twisted guys" (whoever they
are...) to come and review our code, while they struggle with their own
bug queue. It would certainly be beneficial for the stdlib to get as
much review attention as you are asking for this patch, but it is also
totally impractical to expect such a level of attention from a community
of volunteers.

If we aren't knowledgeable enough to maintain signal handling code, then
we should simply remove the signal module from the stdlib (with an
appropriate deprecation period, that is) and let interested people
maintain it as a third-party module. That would also close some
lingering issues in the tracker, and free up some of our time.

> > Well, Python already does (and also apparently messes with other things
> > such as FPU state), so that's a separate issue altogether.
> 
> Not at all. Some signal handlers are more likely to interfere with the
> the rest of the application than others.

Can you clarify why you think those signal handlers fall into that
description? After all, SIGINT, SIGILL and friends only get triggered in
catastrophic conditions.

Thanks

Antoine.



From raymond.hettinger at gmail.com  Sun Dec 19 20:55:55 2010
From: raymond.hettinger at gmail.com (Raymond Hettinger)
Date: Sun, 19 Dec 2010 11:55:55 -0800
Subject: [Python-Dev] r87389 - in python/branches/py3k:
	Doc/library/unittest.rst Lib/unittest/case.py Misc/NEWS
In-Reply-To: <AANLkTinyvb6+Xf=owdV22j6YoAEVjbTXV2tetQzdD8r4@mail.gmail.com>
References: <20101218200004.99110EE985@mail.python.org>
	<20101218214830.1aa8be32@pitrou.net> <iejdjt$ik0$1@dough.gmane.org>
	<AANLkTi=ADkUMc5tQGDv2yniteyfPCPXU=XaCr+GscR+a@mail.gmail.com>
	<20101219141355.5ac7cd2f@pitrou.net>
	<AANLkTinyvb6+Xf=owdV22j6YoAEVjbTXV2tetQzdD8r4@mail.gmail.com>
Message-ID: <31185718-F276-46CE-82E2-6C6B6EC920E8@gmail.com>


On Dec 19, 2010, at 10:41 AM, Guido van Rossum wrote:

> On Sun, Dec 19, 2010 at 5:13 AM, Antoine Pitrou <solipsis at pitrou.net> wrote:
>> On Sat, 18 Dec 2010 20:23:49 -0800
>> Guido van Rossum <guido at python.org> wrote:
>>> I may be unique, but I fear there is no great answer. On the one hand
>>> I almost always code it as e.g. assertEqual(actual, expected), which
>>> matches my preference for e.g. "if x == 5:" rather than "if 5 == x:".
>>> On the other hand in those assert* functions that show a nice diff of
>>> two lists, when reading such a diff my expectation is that "old, new"
>>> corresponds to "expected, actual". Which then freaks me out until I
>>> realize that I coded it as "actual, expected"... And yet "expected,
>>> actual" still looks weird to me. :-(
>> 
>> This could be nicely resolved by renaming the arguments "a" and "b",
>> and having the diff display "a, b". It's quite natural (both the diff
>> ordering and the arguments ordering), and they are consistent with each
>> other.
> 
> So 'a' stands for 'after' and 'b' for 'before', right? :-)

If you go down the a / b path instead of actual/expected,
the diffs are straight-forward but some of the other
output styles needed to be changed also (replace the
messages for "unexpected" and "missing" elements to
"things in a but not in b" and "things in b but not in a".


Raymond


From alexander.belopolsky at gmail.com  Sun Dec 19 21:38:41 2010
From: alexander.belopolsky at gmail.com (Alexander Belopolsky)
Date: Sun, 19 Dec 2010 15:38:41 -0500
Subject: [Python-Dev] Issue #8863 adds a new PYTHONNOFAULTHANDLER
 environment variable
In-Reply-To: <1292786422.3679.24.camel@localhost.localdomain>
References: <4D0C0688.1080108@haypocalc.com> <iei93h$tla$1@dough.gmane.org>
	<4D0CBDB8.6010608@haypocalc.com> <ieiorl$tlu$1@dough.gmane.org>
	<AANLkTimq+Qpt80u50Nj+RTgM7CgfLPsYWR2RHgDkZkP6@mail.gmail.com>
	<4D0D0D52.5020204@v.loewis.de> <20101218211836.0c2b0792@pitrou.net>
	<4D0E54B7.2070604@v.loewis.de>
	<1292786422.3679.24.camel@localhost.localdomain>
Message-ID: <AANLkTi=QZofCaQRR-M_gG=HX4vLKRCxB5gsm8iGEZJwR@mail.gmail.com>

On Sun, Dec 19, 2010 at 2:20 PM, Antoine Pitrou <solipsis at pitrou.net> wrote:
..
>> There should be at least one reviewer with an established track record
>> of being interested/knowledgable in the POSIX APIs and cross-platform
>> aspects of it.
>> As I said, any of the Twisted guys would qualify.
>
> James Knight already posted in this thread. Is this sufficient to you?
>

I cannot speak for Martin of course, but I think we should first see
if James Knight's recommendations get incorporated into the patch and
whether that will be sufficient for  James Knight to speak in favor of
applying the patch.

From victor.stinner at haypocalc.com  Sun Dec 19 22:59:01 2010
From: victor.stinner at haypocalc.com (Victor Stinner)
Date: Sun, 19 Dec 2010 22:59:01 +0100
Subject: [Python-Dev] Issue #8863 adds a new PYTHONNOFAULTHANDLER
 environment variable
In-Reply-To: <4D0E54B7.2070604@v.loewis.de>
References: <4D0C0688.1080108@haypocalc.com> <iei93h$tla$1@dough.gmane.org>
	<4D0CBDB8.6010608@haypocalc.com> <ieiorl$tlu$1@dough.gmane.org>
	<AANLkTimq+Qpt80u50Nj+RTgM7CgfLPsYWR2RHgDkZkP6@mail.gmail.com>
	<4D0D0D52.5020204@v.loewis.de> <20101218211836.0c2b0792@pitrou.net>
	<4D0E54B7.2070604@v.loewis.de>
Message-ID: <1292795941.5914.23.camel@marge>

Le dimanche 19 d?cembre 2010 ? 19:53 +0100, "Martin v. L?wis" a ?crit :
> There should be at least one reviewer with an established track record
> of being interested/knowledgable in the POSIX APIs and cross-platform
> aspects of it.

Functions used by the fault handler:
 - write()
 - PyUnicode_Check()
 - PyFrame_GetLineNumber()
 - DebugBreak() (Windows, in debug mode, only)
 - abort()
 - (macro) PyUnicode_GET_SIZE() and PyUnicode_AS_UNICODE()
 - PyUnicode_Check(), PyFrame_Check()
 - PyFrame_GetLineNumber()
 - _Py_atomic_load_relaxed()
 - _PyThreadState_GetFrame()

I suppose that all *Py*() functions are POSIX compliants. DebugBreak()
is specific to Windows. Only write() and abort() have to be POSIX
compliant. According to Linux manual pages:

 - abort() conforms to SVr4, POSIX.1-2001, 4.3BSD, C89, C99.
 - write() conforms to SVr4, 4.3BSD, POSIX.1-2001

--

Functions used by the other parts of the patch:
 - signal() or sigaction()+sigemptyset()
 - (optional) sigaltstack()
 - PyMem_Malloc(), PyMem_Free()
 - Py_GETENV()
 - (optional) getpid(), kill() (used by the tests)

signal() conforms to C89, C99, POSIX.1-2001.

So I don't see any function incompatible with POSIX.

(Don't read old versions of my patch, read the latest version)

--

About the cross-platform question: it looks like SIGFPE fault is not
catchable on Windows. I only tried Linux and Windows, but there are a
unit test for each signal and we have 60+ buildbots with various
platforms. Can't we wait for the buildbot results?

> > Well, Python already does (and also apparently messes with other things
> > such as FPU state), so that's a separate issue altogether.
> 
> Not at all. Some signal handlers are more likely to interfere with the
> the rest of the application than others.

Python installs its own SIGINT handler. I think that a SIGINT handler is
more intrusive than a SIGSEGV or SIGILL handler.

Victor


From victor.stinner at haypocalc.com  Sun Dec 19 23:10:43 2010
From: victor.stinner at haypocalc.com (Victor Stinner)
Date: Sun, 19 Dec 2010 23:10:43 +0100
Subject: [Python-Dev] Issue #8863 adds a new PYTHONNOFAULTHANDLER
 environment variable
In-Reply-To: <iein87$mud$1@dough.gmane.org>
References: <4D0C0688.1080108@haypocalc.com> <iei93h$tla$1@dough.gmane.org>
	<4D0CBDB8.6010608@haypocalc.com>  <iein87$mud$1@dough.gmane.org>
Message-ID: <1292796643.5914.34.camel@marge>

Le samedi 18 d?cembre 2010 ? 17:23 +0100, Georg Brandl a ?crit :
> Well, without a closer I assume that for some crashes it's just not
> possible anymore for the Python interpreter to even print out the
> traceback?

The worst case occurs if the frame objects are corrupted, eg. if the
filename of a frame is an invalid object (eg. pointer to NULL or to a
corrupted unicode object).

If the file descriptor 2 is not stderr (eg. if there is no file
descriptor 2), the backtrace cannot be printed.

Except of these two issues, I think that the fault handler is always
able to display the backtrace, even on stack overflow or invalid memory
write.

--

Older versions of my patch were less reliable:

 - allocate memory on the heap => only use the stack (only few bytes)
 - call "complex" Python functions (like UTF-8 codec with
backslashreplace error handler) => use a very simple ASCII
+backslashreplace implementation and avoid complex functions
 - use not signal-safe functions like printf => use write() and simple
functions to format numbers
 - may go into an unlimited loop if there is a loop in the frame linked
list => limit the loop to MAX_DEPTH (100) iterations

Victor


From victor.stinner at haypocalc.com  Sun Dec 19 23:32:03 2010
From: victor.stinner at haypocalc.com (Victor Stinner)
Date: Sun, 19 Dec 2010 23:32:03 +0100
Subject: [Python-Dev] Issue #8863 adds a new PYTHONNOFAULTHANDLER
 environment variable
In-Reply-To: <AANLkTimq+Qpt80u50Nj+RTgM7CgfLPsYWR2RHgDkZkP6@mail.gmail.com>
References: <4D0C0688.1080108@haypocalc.com> <iei93h$tla$1@dough.gmane.org>
	<4D0CBDB8.6010608@haypocalc.com> <ieiorl$tlu$1@dough.gmane.org>
	<AANLkTimq+Qpt80u50Nj+RTgM7CgfLPsYWR2RHgDkZkP6@mail.gmail.com>
Message-ID: <1292797923.5914.55.camel@marge>

Le samedi 18 d?cembre 2010 ? 13:55 -0500, Alexander Belopolsky a ?crit :
> I am -1 on the feature as written.  I would be -0 if it did not
> install signal handlers by default and even better was implemented in
> a separate module, not in core.

I think that the feature is useless if it is disabled by default. How do
you know that you application will crash? The idea is to give
informations to the user when an application crashs: the user can use
the backtrace or send it to the developer. Segmentation faults are
usually not (easilly) reproductible :-( So even if you enable the fault
handler, you may not be able to replay the crash. Or even worse, the
fault may not occurs at all when you enable the fault handler...
(Heisenbugs!)

> This feature is similar to the one that is implemented by R and with
> R, it was always a problem when R was embedded as a library.

Which kind of problem? A conflict with an existing fault handler?

> It has always been a welcome feature of Python that its core did 
> not mess with the application global state.

What do mess the application global state?

> Before this is approved, I would like to see a discussion of the
> consequences for embedded Python

I suppose that if a fault occurs outside Python, the fault handler
displays an irrevelent or empty Python backtrace.

The fault handler is installed by Py_InitializeEx(), and uninstalled by
Py_Finalize().

If the program has its own fault handler or doesn't want Python fault
handler, it can set PYTHONNOFAULTHANDLER environment variable. Should we
add a C function to the Python API to configure (disable) the fault
handler?

I only know *two* applications catching faults: Mplayer and aMule. I
don't think that they embed Python. If they do, I think that these *two*
applications can do something special. Since there are only two
applications having their own fault handler, I don't think that we
should disable the fault handler by default.

(System wide fault handler, like Apport on Ubuntu, can be configured
system wide by setting the PYTHONNOFAULTHANDLER environment variable)

> and particularly for the case when there are multiple interpreters 
> in the same process.

What? Is it a myth or does Python really support multiple interpreters
in the same process? How is it possible? Who uses this?

_Py_DumpBacktrace() function (of my patch) uses
_Py_atomic_load_relaxed(&_PyThreadState_Current) to get the current
thread state. I picked up this instruction from PyThreadState_Get(). If
it doesn't work, PyThreadState_Get() should be fixed too.

Victor


From scott+python-dev at scottdial.com  Sun Dec 19 23:08:51 2010
From: scott+python-dev at scottdial.com (Scott Dial)
Date: Sun, 19 Dec 2010 17:08:51 -0500
Subject: [Python-Dev] Issue #8863 adds a new
 PYTHONNOFAULTHANDLER	environment variable
In-Reply-To: <B880623A-1EA5-41CA-A58C-C46C93BED8DC@fuhm.net>
References: <4D0C0688.1080108@haypocalc.com>
	<B880623A-1EA5-41CA-A58C-C46C93BED8DC@fuhm.net>
Message-ID: <4D0E8273.1040606@scottdial.com>

On 12/18/2010 8:50 AM, James Y Knight wrote:
> I think instead of calling abort() to kill the process, you should:
> - install the signal handler with SA_NODEFER|SA_RESETHAND (or if
> sigaction is not present, explicitly reset the action to SIG_DFL and
> unblock first thing upon entering the handler), and then,
> - at the end of the handler, kill(getpid(), orig_signal) in order to
> abort the process.

I concur with this being the correct way to right such a handler.

> This has two advantages: 1) the process's exit code will actually 
> show the correct signal,

It's more than an advantage: it's the only correct way to handle a
termination signal.

> 2) it might let the OS fault handlers work properly
> as well -- I'm not sure. If it does, you may want to experiment with
> whether having or omitting SA_NODEFER gives a better backtrace (from the
> OS mechanism) in that case.

Even if that doesn't work, things like the grsecurity patches to linux
use these signals to detect exploits and log them and do throttling.
Calling abort() effectively converts all of these faults into SIGABRT
terminations that are considered (more?) innocent terminations.

-- 
Scott Dial
scott at scottdial.com
scodial at cs.indiana.edu

-- 
Scott Dial
scott at scottdial.com
scodial at cs.indiana.edu

From victor.stinner at haypocalc.com  Sun Dec 19 23:37:58 2010
From: victor.stinner at haypocalc.com (Victor Stinner)
Date: Sun, 19 Dec 2010 23:37:58 +0100
Subject: [Python-Dev] Issue #8863 adds a new PYTHONNOFAULTHANDLER
 environment variable
In-Reply-To: <1292786422.3679.24.camel@localhost.localdomain>
References: <4D0C0688.1080108@haypocalc.com> <iei93h$tla$1@dough.gmane.org>
	<4D0CBDB8.6010608@haypocalc.com> <ieiorl$tlu$1@dough.gmane.org>
	<AANLkTimq+Qpt80u50Nj+RTgM7CgfLPsYWR2RHgDkZkP6@mail.gmail.com>
	<4D0D0D52.5020204@v.loewis.de> <20101218211836.0c2b0792@pitrou.net>
	<4D0E54B7.2070604@v.loewis.de>
	<1292786422.3679.24.camel@localhost.localdomain>
Message-ID: <1292798278.5914.56.camel@marge>

Le dimanche 19 d?cembre 2010 ? 20:20 +0100, Antoine Pitrou a ?crit :
> Can you clarify why you think those signal handlers fall into that
> description? After all, SIGINT, SIGILL and friends only get triggered in
> catastrophic conditions.

SIGSEGV, not SIGINT

Victor


From solipsis at pitrou.net  Sun Dec 19 23:51:03 2010
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Sun, 19 Dec 2010 23:51:03 +0100
Subject: [Python-Dev] Issue #8863 adds a new PYTHONNOFAULTHANDLER
 environment variable
References: <4D0C0688.1080108@haypocalc.com> <iei93h$tla$1@dough.gmane.org>
	<4D0CBDB8.6010608@haypocalc.com> <ieiorl$tlu$1@dough.gmane.org>
	<AANLkTimq+Qpt80u50Nj+RTgM7CgfLPsYWR2RHgDkZkP6@mail.gmail.com>
	<4D0D0D52.5020204@v.loewis.de> <20101218211836.0c2b0792@pitrou.net>
	<4D0E54B7.2070604@v.loewis.de>
	<1292786422.3679.24.camel@localhost.localdomain>
	<1292798278.5914.56.camel@marge>
Message-ID: <20101219235103.227aaad8@pitrou.net>

On Sun, 19 Dec 2010 23:37:58 +0100
Victor Stinner <victor.stinner at haypocalc.com> wrote:
> Le dimanche 19 d?cembre 2010 ? 20:20 +0100, Antoine Pitrou a ?crit :
> > Can you clarify why you think those signal handlers fall into that
> > description? After all, SIGINT, SIGILL and friends only get triggered in
> > catastrophic conditions.
> 
> SIGSEGV, not SIGINT

Oops, sorry for the mistake. I hope this didn't stir any confusion
beyond my own ;)

cheers

Antoine.



From victor.stinner at haypocalc.com  Mon Dec 20 00:02:25 2010
From: victor.stinner at haypocalc.com (Victor Stinner)
Date: Mon, 20 Dec 2010 00:02:25 +0100
Subject: [Python-Dev] Issue #8863 adds a new PYTHONNOFAULTHANDLER
 environment variable
In-Reply-To: <B880623A-1EA5-41CA-A58C-C46C93BED8DC@fuhm.net>
References: <4D0C0688.1080108@haypocalc.com>
	<B880623A-1EA5-41CA-A58C-C46C93BED8DC@fuhm.net>
Message-ID: <1292799745.5914.76.camel@marge>

Le samedi 18 d?cembre 2010 ? 08:50 -0500, James Y Knight a ?crit :
> I think instead of calling abort() to kill the process, you should:
> - install the signal handler with SA_NODEFER|SA_RESETHAND (or if sigaction is not present, explicitly reset the action to SIG_DFL and unblock first thing upon entering the handler), and then,
> - at the end of the handler, kill(getpid(), orig_signal) in order to abort the process.

I don't think that kill() is available on Windows.

We may call directly the previous handler in the new handler, but it is
more difficult. It looks like the prototype of a signal handler depends
on its options. Or is it possible to always pass (signum, siginfo,
ucontext) arguments? I don't know if siginfo and ucontect are portable
or not...

Another possible approach is to restore the previous handler in the
fault handler, and don't call abort(). The fault should occur again, and
so the previous signal handler will be called.

I choosed to not call the previous handler to keep the patch simple. If
you consider that we must call the previous handler, I can work on a new
patch doing that.

> This has two advantages: 1) the process's exit code will actually show the correct signal

Oh, I didn't noticed that the exit code is different.

> 2) it might let the OS fault handlers work properly as well -- I'm not sure. If it does, you may want to experiment with whether having or omitting SA_NODEFER gives a better backtrace (from the OS mechanism) in that case.

For the OS fault handler: we should try, I am not sure that calling
another signal handler (my fault handler) keeps the process state
unchanged (especially if sigaltstk() is not available).

I don't understand the SA_NODEFER option. What does it change for this
fault handler?

> If #2 actually works, you may not even need the env var, which would be nice.

I prefer to keep it optional because we don't know yet how it interacts
exactly with existing debug tools, OS tools, embedded Python
interpreters, etc.

Victor


From tjreedy at udel.edu  Mon Dec 20 00:54:55 2010
From: tjreedy at udel.edu (Terry Reedy)
Date: Sun, 19 Dec 2010 18:54:55 -0500
Subject: [Python-Dev] r87389 - in python/branches/py3k:
 Doc/library/unittest.rst Lib/unittest/case.py Misc/NEWS
In-Reply-To: <AANLkTinyvb6+Xf=owdV22j6YoAEVjbTXV2tetQzdD8r4@mail.gmail.com>
References: <20101218200004.99110EE985@mail.python.org>	<20101218214830.1aa8be32@pitrou.net>
	<iejdjt$ik0$1@dough.gmane.org>	<AANLkTi=ADkUMc5tQGDv2yniteyfPCPXU=XaCr+GscR+a@mail.gmail.com>	<20101219141355.5ac7cd2f@pitrou.net>
	<AANLkTinyvb6+Xf=owdV22j6YoAEVjbTXV2tetQzdD8r4@mail.gmail.com>
Message-ID: <iem60d$svg$1@dough.gmane.org>

On 12/19/2010 1:41 PM, Guido van Rossum wrote:
> On Sun, Dec 19, 2010 at 5:13 AM, Antoine Pitrou<solipsis at pitrou.net>  wrote:

>> This could be nicely resolved by renaming the arguments "a" and "b",
>> and having the diff display "a, b". It's quite natural (both the diff
>> ordering and the arguments ordering), and they are consistent with each
>> other.
>
> So 'a' stands for 'after' and 'b' for 'before', right? :-)

difflib uses 'a' and 'b' for before and after (orig,new in svn terms, 
with edits/diffs from a to b) respectively. Not really great. The docs 
then have to explain what 'a' and 'b' are and the implications for 
interpreting the output.

-- 
Terry Jan Reedy


From merwok at netwok.org  Mon Dec 20 01:09:19 2010
From: merwok at netwok.org (=?UTF-8?B?w4lyaWMgQXJhdWpv?=)
Date: Mon, 20 Dec 2010 01:09:19 +0100
Subject: [Python-Dev] Locale-specific formatting
In-Reply-To: <20101218153331.GA22017@phd.pp.ru>
References: <4D0BFB8F.2040301@mrabarnett.plus.com>
	<20101218153331.GA22017@phd.pp.ru>
Message-ID: <4D0E9EAF.6010000@netwok.org>

Le 18/12/2010 16:33, Oleg Broytman a ?crit :
>    This is quite a known problem, not specific to Python. Locale
> settings are global for a process, and this is one of the thousands
> reasons why locale is considered so horrible.
>    ICU is perhaps the only way around the problem.

Babel rocks: http://babel.edgewall.org/

Cheers


From victor.stinner at haypocalc.com  Mon Dec 20 01:19:35 2010
From: victor.stinner at haypocalc.com (Victor Stinner)
Date: Mon, 20 Dec 2010 01:19:35 +0100
Subject: [Python-Dev] Issue #8863 adds a new PYTHONNOFAULTHANDLER
 environment variable
In-Reply-To: <1292799745.5914.76.camel@marge>
References: <4D0C0688.1080108@haypocalc.com>
	<B880623A-1EA5-41CA-A58C-C46C93BED8DC@fuhm.net>
	<1292799745.5914.76.camel@marge>
Message-ID: <1292804375.5914.84.camel@marge>

Le lundi 20 d?cembre 2010 ? 00:02 +0100, Victor Stinner a ?crit :
> Another possible approach is to restore the previous handler in the
> fault handler, and don't call abort(). The fault should occur again, and
> so the previous signal handler will be called.

I implemented this simple approach in the version 10 of my patch: the
previous signal handler is called.
http://bugs.python.org/issue8863#msg124373

On my Debian Sid, the libc fatal error message is written as expected
(eg. "Segmentation fault") and the exit code is different for each fault
(eg. 132 for SIGILL). So we keep the existing behaviour, the new Python
fault handler just write more information if a fault occurs.

I suppose that it is compatible with the grsecurity thing and Apport,
but I didn't tried. Can someone test it?

I prefer to restore the previous signal handler than restoring the
*default* signal handler.

> I choosed to not call the previous handler to keep the patch simple.

It's cool: restoring the signal approach keeps the code simple, and it
looks like it works :-)

Victor


From solipsis at pitrou.net  Mon Dec 20 01:48:40 2010
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Mon, 20 Dec 2010 01:48:40 +0100
Subject: [Python-Dev] r87389 - in python/branches/py3k:
 Doc/library/unittest.rst Lib/unittest/case.py Misc/NEWS
References: <20101218200004.99110EE985@mail.python.org>
	<20101218214830.1aa8be32@pitrou.net> <iejdjt$ik0$1@dough.gmane.org>
	<AANLkTi=ADkUMc5tQGDv2yniteyfPCPXU=XaCr+GscR+a@mail.gmail.com>
	<20101219141355.5ac7cd2f@pitrou.net>
	<AANLkTinyvb6+Xf=owdV22j6YoAEVjbTXV2tetQzdD8r4@mail.gmail.com>
	<iem60d$svg$1@dough.gmane.org>
Message-ID: <20101220014840.6b734274@pitrou.net>

On Sun, 19 Dec 2010 18:54:55 -0500
Terry Reedy <tjreedy at udel.edu> wrote:
> On 12/19/2010 1:41 PM, Guido van Rossum wrote:
> > On Sun, Dec 19, 2010 at 5:13 AM, Antoine Pitrou<solipsis at pitrou.net>  wrote:
> 
> >> This could be nicely resolved by renaming the arguments "a" and "b",
> >> and having the diff display "a, b". It's quite natural (both the diff
> >> ordering and the arguments ordering), and they are consistent with each
> >> other.
> >
> > So 'a' stands for 'after' and 'b' for 'before', right? :-)
> 
> difflib uses 'a' and 'b' for before and after (orig,new in svn terms, 
> with edits/diffs from a to b) respectively. Not really great.

For a non-native English speaker, 'a' and 'b' don't evoke 'after' and
'before' but simply the first two letters of the latin alphabet, and
their ordering is therefore obvious with respect to function arguments.

By the way, hg uses a/b as well, and so does git apparently, so
Python's difflib is not exotic in that regard:

$ hg diff
diff -r 56867877575b README
--- a/README    Fri Dec 17 21:43:27 2010 +0100
+++ b/README    Mon Dec 20 01:42:57 2010 +0100
@@ -1,3 +1,4 @@
+some change
 This is Python version 3.2 beta 1
 =================================
 

Regards

Antoine.



From martin at v.loewis.de  Mon Dec 20 02:05:30 2010
From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=)
Date: Mon, 20 Dec 2010 02:05:30 +0100
Subject: [Python-Dev] Issue #8863 adds a new PYTHONNOFAULTHANDLER
 environment variable
In-Reply-To: <1292795941.5914.23.camel@marge>
References: <4D0C0688.1080108@haypocalc.com>
	<iei93h$tla$1@dough.gmane.org>	<4D0CBDB8.6010608@haypocalc.com>
	<ieiorl$tlu$1@dough.gmane.org>	<AANLkTimq+Qpt80u50Nj+RTgM7CgfLPsYWR2RHgDkZkP6@mail.gmail.com>	<4D0D0D52.5020204@v.loewis.de>
	<20101218211836.0c2b0792@pitrou.net>	<4D0E54B7.2070604@v.loewis.de>
	<1292795941.5914.23.camel@marge>
Message-ID: <4D0EABDA.3060702@v.loewis.de>

> Functions used by the fault handler:
>  - write()
>  - PyUnicode_Check()
>  - PyFrame_GetLineNumber()
>  - DebugBreak() (Windows, in debug mode, only)
>  - abort()
>  - (macro) PyUnicode_GET_SIZE() and PyUnicode_AS_UNICODE()
>  - PyUnicode_Check(), PyFrame_Check()
>  - PyFrame_GetLineNumber()
>  - _Py_atomic_load_relaxed()
>  - _PyThreadState_GetFrame()
> 
> I suppose that all *Py*() functions are POSIX compliants.

The problem is that merely being POSIX compliant would not be enough
to allow calling a function in a signal handler. Instead, the function
*also* needs to be async-signal safe. However,

>  - abort() conforms to SVr4, POSIX.1-2001, 4.3BSD, C89, C99.
>  - write() conforms to SVr4, 4.3BSD, POSIX.1-2001

These two functions are indeed async-signal safe. So assuming that
none of the functions above indirectly calls a non-async-signal-safe
function, this part is fine.

Looking at your function list, my other concern is that you are calling
Python API without holding the GIL, IIUC. In particular, you are
accessing _PyThreadState_Current, which may not point to the current
thread if the current thread has released the GIL.

Regards,
Martin

From martin at v.loewis.de  Mon Dec 20 02:11:34 2010
From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=)
Date: Mon, 20 Dec 2010 02:11:34 +0100
Subject: [Python-Dev] Issue #8863 adds a new PYTHONNOFAULTHANDLER
 environment variable
In-Reply-To: <1292797923.5914.55.camel@marge>
References: <4D0C0688.1080108@haypocalc.com>
	<iei93h$tla$1@dough.gmane.org>	<4D0CBDB8.6010608@haypocalc.com>
	<ieiorl$tlu$1@dough.gmane.org>	<AANLkTimq+Qpt80u50Nj+RTgM7CgfLPsYWR2RHgDkZkP6@mail.gmail.com>
	<1292797923.5914.55.camel@marge>
Message-ID: <4D0EAD46.209@v.loewis.de>

> What? Is it a myth or does Python really support multiple interpreters
> in the same process? How is it possible? Who uses this?

[Not sure if you are joking]

There is certainly some support for multiple interpreters in Python; the
most prominent user of this feature is mod_python.

Regards,
Martin

From fuzzyman at voidspace.org.uk  Mon Dec 20 01:57:04 2010
From: fuzzyman at voidspace.org.uk (Michael Foord)
Date: Mon, 20 Dec 2010 00:57:04 +0000
Subject: [Python-Dev] Issue #8863 adds a new PYTHONNOFAULTHANDLER
 environment variable
In-Reply-To: <1292799745.5914.76.camel@marge>
References: <4D0C0688.1080108@haypocalc.com>	<B880623A-1EA5-41CA-A58C-C46C93BED8DC@fuhm.net>
	<1292799745.5914.76.camel@marge>
Message-ID: <4D0EA9E0.4080707@voidspace.org.uk>

On 19/12/2010 23:02, Victor Stinner wrote:
> Le samedi 18 d?cembre 2010 ? 08:50 -0500, James Y Knight a ?crit :
>> I think instead of calling abort() to kill the process, you should:
>> - install the signal handler with SA_NODEFER|SA_RESETHAND (or if sigaction is not present, explicitly reset the action to SIG_DFL and unblock first thing upon entering the handler), and then,
>> - at the end of the handler, kill(getpid(), orig_signal) in order to abort the process.
> I don't think that kill() is available on Windows.

Thanks to Brian Curtin there is now an os.kill() on Windows, but it 
doesn't handle the full range of signals.

Michael

-- 

http://www.voidspace.org.uk/

May you do good and not evil
May you find forgiveness for yourself and forgive others
May you share freely, never taking more than you give.
-- the sqlite blessing http://www.sqlite.org/different.html


From victor.stinner at haypocalc.com  Mon Dec 20 05:53:49 2010
From: victor.stinner at haypocalc.com (Victor Stinner)
Date: Mon, 20 Dec 2010 05:53:49 +0100
Subject: [Python-Dev] Issue #8863 adds a new PYTHONNOFAULTHANDLER
 environment variable
In-Reply-To: <4D0EABDA.3060702@v.loewis.de>
References: <4D0C0688.1080108@haypocalc.com> <iei93h$tla$1@dough.gmane.org>
	<4D0CBDB8.6010608@haypocalc.com> <ieiorl$tlu$1@dough.gmane.org>
	<AANLkTimq+Qpt80u50Nj+RTgM7CgfLPsYWR2RHgDkZkP6@mail.gmail.com>
	<4D0D0D52.5020204@v.loewis.de> <20101218211836.0c2b0792@pitrou.net>
	<4D0E54B7.2070604@v.loewis.de> <1292795941.5914.23.camel@marge>
	<4D0EABDA.3060702@v.loewis.de>
Message-ID: <1292820829.4763.16.camel@marge>

Le lundi 20 d?cembre 2010 ? 02:05 +0100, "Martin v. L?wis" a ?crit :
> The problem is that merely being POSIX compliant would not be enough
> to allow calling a function in a signal handler. Instead, the function
> *also* needs to be async-signal safe.

Yes, this issue was fixed in an older version of my patch. Starting at
version 9, the signal handler only call signal-safe functions.

> Looking at your function list, my other concern is that you are calling
> Python API without holding the GIL, IIUC. In particular, you are
> accessing _PyThreadState_Current, which may not point to the current
> thread if the current thread has released the GIL.

Ah? Where does _PyThreadState_Current point to if the GIL is not hold
when the fault handler is called? My patch changes also Py_FatalError()
to display the Python backtrace.

It looks that _PyThreadState_Current can be NULL if the GIL is released.
In this case, _Py_DumpBacktrace() just do nothing. There is also a
gil_last_holder variable: can it be used to get the thread state
(especially if the thread state was deleted)?

I don't think that it will possible the acquire the GIL in
Py_FatalError() or in the fault handler.

Victor


From martin at v.loewis.de  Mon Dec 20 08:22:40 2010
From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=)
Date: Mon, 20 Dec 2010 08:22:40 +0100
Subject: [Python-Dev] Issue #8863 adds a new PYTHONNOFAULTHANDLER
 environment variable
In-Reply-To: <1292820829.4763.16.camel@marge>
References: <4D0C0688.1080108@haypocalc.com>
	<iei93h$tla$1@dough.gmane.org>	<4D0CBDB8.6010608@haypocalc.com>
	<ieiorl$tlu$1@dough.gmane.org>	<AANLkTimq+Qpt80u50Nj+RTgM7CgfLPsYWR2RHgDkZkP6@mail.gmail.com>	<4D0D0D52.5020204@v.loewis.de>
	<20101218211836.0c2b0792@pitrou.net>	<4D0E54B7.2070604@v.loewis.de>
	<1292795941.5914.23.camel@marge>	<4D0EABDA.3060702@v.loewis.de>
	<1292820829.4763.16.camel@marge>
Message-ID: <4D0F0440.5070806@v.loewis.de>

>> Looking at your function list, my other concern is that you are calling
>> Python API without holding the GIL, IIUC. In particular, you are
>> accessing _PyThreadState_Current, which may not point to the current
>> thread if the current thread has released the GIL.
> 
> Ah? Where does _PyThreadState_Current point to if the GIL is not hold
> when the fault handler is called?

The GIL is likely held by a different thread, then.
_PyThreadState_Current will point to the state of this other thread.

> It looks that _PyThreadState_Current can be NULL if the GIL is released.
> In this case, _Py_DumpBacktrace() just do nothing. There is also a
> gil_last_holder variable: can it be used to get the thread state
> (especially if the thread state was deleted)?

Of this thread? I don't think so. gil_last_holder might also refer to
a different thread.

> I don't think that it will possible the acquire the GIL in
> Py_FatalError() or in the fault handler.

I agree.

Regards,
Martin

From phd at phd.pp.ru  Mon Dec 20 10:23:02 2010
From: phd at phd.pp.ru (Oleg Broytman)
Date: Mon, 20 Dec 2010 12:23:02 +0300
Subject: [Python-Dev] Locale-specific formatting
In-Reply-To: <4D0E9EAF.6010000@netwok.org>
References: <4D0BFB8F.2040301@mrabarnett.plus.com>
	<20101218153331.GA22017@phd.pp.ru> <4D0E9EAF.6010000@netwok.org>
Message-ID: <20101220092302.GA5973@phd.pp.ru>

On Mon, Dec 20, 2010 at 01:09:19AM +0100, ??ric Araujo wrote:
> Le 18/12/2010 16:33, Oleg Broytman a ??crit :
> >    This is quite a known problem, not specific to Python. Locale
> > settings are global for a process, and this is one of the thousands
> > reasons why locale is considered so horrible.
> >    ICU is perhaps the only way around the problem.
> 
> Babel rocks: http://babel.edgewall.org/

   Unicode CLDR! Never heard of it before, thank you for pointing this
out.

Oleg.
-- 
     Oleg Broytman            http://phd.pp.ru/            phd at phd.pp.ru
           Programmers don't die, they just GOSUB without RETURN.

From steve at pearwood.info  Mon Dec 20 10:55:10 2010
From: steve at pearwood.info (Steven D'Aprano)
Date: Mon, 20 Dec 2010 20:55:10 +1100
Subject: [Python-Dev] r87389 - in python/branches/py3k:
 Doc/library/unittest.rst Lib/unittest/case.py Misc/NEWS
In-Reply-To: <20101220014840.6b734274@pitrou.net>
References: <20101218200004.99110EE985@mail.python.org>	<20101218214830.1aa8be32@pitrou.net>
	<iejdjt$ik0$1@dough.gmane.org>	<AANLkTi=ADkUMc5tQGDv2yniteyfPCPXU=XaCr+GscR+a@mail.gmail.com>	<20101219141355.5ac7cd2f@pitrou.net>	<AANLkTinyvb6+Xf=owdV22j6YoAEVjbTXV2tetQzdD8r4@mail.gmail.com>	<iem60d$svg$1@dough.gmane.org>
	<20101220014840.6b734274@pitrou.net>
Message-ID: <4D0F27FE.4020706@pearwood.info>

Antoine Pitrou wrote:

> For a non-native English speaker, 'a' and 'b' don't evoke 'after' and
> 'before' but simply the first two letters of the latin alphabet, and
> their ordering is therefore obvious with respect to function arguments.

It's not just non-native English speakers either. I too think of a, b as 
being first, second rather than after, before.


-- 
Steven


From stefan at bytereef.org  Mon Dec 20 12:08:35 2010
From: stefan at bytereef.org (Stefan Krah)
Date: Mon, 20 Dec 2010 12:08:35 +0100
Subject: [Python-Dev] Issue #8863 adds a new
	PYTHONNOFAULTHANDLER	environment variable
In-Reply-To: <AANLkTimq+Qpt80u50Nj+RTgM7CgfLPsYWR2RHgDkZkP6@mail.gmail.com>
References: <4D0C0688.1080108@haypocalc.com> <iei93h$tla$1@dough.gmane.org>
	<4D0CBDB8.6010608@haypocalc.com> <ieiorl$tlu$1@dough.gmane.org>
	<AANLkTimq+Qpt80u50Nj+RTgM7CgfLPsYWR2RHgDkZkP6@mail.gmail.com>
Message-ID: <20101220110835.GA13825@yoda.bytereef.org>

Alexander Belopolsky <alexander.belopolsky at gmail.com> wrote:
> On Sat, Dec 18, 2010 at 11:50 AM, Georg Brandl <g.brandl at gmx.net> wrote:
> ..
> > In any case, this is coming pretty late; beta 2 is scheduled for this
> > weekend, and even if this is something that only kicks in when all hope
> > is lost anyway, it is a new feature. ?I should like to hear approval
> > from a few more devs before I will let this go into 3.2.
> >
> 
> I am -1 on the feature as written.  I would be -0 if it did not
> install signal handlers by default and even better was implemented in
> a separate module, not in core.


I would not want this to be the default either. I think the output
is not particularly informative:

$ ./python Lib/test/crashers/gc_inspection.py 
(<object object at 0x7f01827ad870>, <NULL>, <NULL>, <NULL>, <NULL>, <NULL>, <NULL>, <NULL>, <NULL>, <NULL>)
Fatal Python error: Segmentation fault

Traceback (most recent call first):
  File "Lib/test/crashers/gc_inspection.py", line 29 in g
  File "Lib/test/crashers/gc_inspection.py", line 32 in <module>
Segmentation fault
Segmentation fault


Another test is hanging indefinitely (Ubuntu 64-bit):

$ ./python Lib/test/crashers/nasty_eq_vs_dict.py
[hanging with no output]



It's hard to get signal handlers right across multiple platforms; even
SIGINT catching does not work properly on OpenBSD:

http://bugs.python.org/issue8714


In short, I agree that having more signal handlers by default is not a
good idea.



Stefan Krah




From stefan at bytereef.org  Mon Dec 20 13:11:52 2010
From: stefan at bytereef.org (Stefan Krah)
Date: Mon, 20 Dec 2010 13:11:52 +0100
Subject: [Python-Dev] Issue #8863 adds a new
	PYTHONNOFAULTHANDLER	environment variable
In-Reply-To: <20101220110835.GA13825@yoda.bytereef.org>
References: <4D0C0688.1080108@haypocalc.com> <iei93h$tla$1@dough.gmane.org>
	<4D0CBDB8.6010608@haypocalc.com> <ieiorl$tlu$1@dough.gmane.org>
	<AANLkTimq+Qpt80u50Nj+RTgM7CgfLPsYWR2RHgDkZkP6@mail.gmail.com>
	<20101220110835.GA13825@yoda.bytereef.org>
Message-ID: <20101220121152.GA15386@yoda.bytereef.org>

Stefan Krah <stefan at bytereef.org> wrote:
> Another test is hanging indefinitely (Ubuntu 64-bit):
> 
> $ ./python Lib/test/crashers/nasty_eq_vs_dict.py
> [hanging with no output]


And this test does not report anything at all:

$ ./python Lib/test/crashers/compiler_recursion.py 
[no output at all]
$ echo $?
0


Without the patch:

$ ./python Lib/test/crashers/compiler_recursion.py 
Segmentation fault
$ echo $?
139



Stefan Krah



From fuzzyman at voidspace.org.uk  Mon Dec 20 14:00:26 2010
From: fuzzyman at voidspace.org.uk (Michael Foord)
Date: Mon, 20 Dec 2010 13:00:26 +0000
Subject: [Python-Dev] r87389 - in
 python/branches/py3k:	Doc/library/unittest.rst Lib/unittest/case.py
 Misc/NEWS
In-Reply-To: <31185718-F276-46CE-82E2-6C6B6EC920E8@gmail.com>
References: <20101218200004.99110EE985@mail.python.org>	<20101218214830.1aa8be32@pitrou.net>
	<iejdjt$ik0$1@dough.gmane.org>	<AANLkTi=ADkUMc5tQGDv2yniteyfPCPXU=XaCr+GscR+a@mail.gmail.com>	<20101219141355.5ac7cd2f@pitrou.net>	<AANLkTinyvb6+Xf=owdV22j6YoAEVjbTXV2tetQzdD8r4@mail.gmail.com>
	<31185718-F276-46CE-82E2-6C6B6EC920E8@gmail.com>
Message-ID: <4D0F536A.8050808@voidspace.org.uk>

On 19/12/2010 19:55, Raymond Hettinger wrote:
> On Dec 19, 2010, at 10:41 AM, Guido van Rossum wrote:
>
>> On Sun, Dec 19, 2010 at 5:13 AM, Antoine Pitrou<solipsis at pitrou.net>  wrote:
>>> On Sat, 18 Dec 2010 20:23:49 -0800
>>> Guido van Rossum<guido at python.org>  wrote:
>>>> I may be unique, but I fear there is no great answer. On the one hand
>>>> I almost always code it as e.g. assertEqual(actual, expected), which
>>>> matches my preference for e.g. "if x == 5:" rather than "if 5 == x:".
>>>> On the other hand in those assert* functions that show a nice diff of
>>>> two lists, when reading such a diff my expectation is that "old, new"
>>>> corresponds to "expected, actual". Which then freaks me out until I
>>>> realize that I coded it as "actual, expected"... And yet "expected,
>>>> actual" still looks weird to me. :-(
>>> This could be nicely resolved by renaming the arguments "a" and "b",
>>> and having the diff display "a, b". It's quite natural (both the diff
>>> ordering and the arguments ordering), and they are consistent with each
>>> other.
>> So 'a' stands for 'after' and 'b' for 'before', right? :-)
> If you go down the a / b path instead of actual/expected,
> the diffs are straight-forward but some of the other
> output styles needed to be changed also (replace the
> messages for "unexpected" and "missing" elements to
> "things in a but not in b" and "things in b but not in a".
>
>

Ah man, we've *nearly* finished bikeshedding about the names of unittest 
assert methods so its time to move onto the names and order of the 
arguments. Really?

I wouldn't use a/b but first/second [1] as they have a more obvious 
meaning.

However, I'm reluctant to move away from the actual/expected 
terminology. It's standard terminology for testing (used by all the 
other unit testing frameworks I looked at phpunit, JUnit and NUnit), but 
more importantly it is a useful way to think about testing - and one 
used by most devs I've worked with.

You fetch an 'actual' result by calling your code and compare it against 
a pre-computed 'expected' result. Hopefully the two are the same. 
Talking about your actual value and your expected value is a natural way 
to talk in testing, so it's a useful concept.

Once you use the 'actual' and 'expected' terminology you have a natural 
order for displaying failure message results: if an element is present 
in your actual but not in your expected then it is extra. If an element 
is in your expected but not in your actual then it is missing. 
Straightforward. (Of course it maybe that your actual is correct and it 
is your expected result needs correcting, that doesn't affect how 
failure messages should be presented though.)

The only thing left to decide is then the order - (actual, expected) or 
(expected, actual). Raymond, myself, Guido and Ezio have all expressed a 
preference for (actual, expected). I like this comment from Raymond on 
the relevant issue [2]:

     I also tend to use actual/expected and find the reverse to be a 
form Yoda-speak, "assert 5 == x", "perhaps misread the prophecy was", etc.

As the current ordering used within unittest is (actual, expected), to 
reverse it would be dumb (why should everyone using the current ordering 
reformat all their tests for the new order?).

So, -1 on dropping actual and expected. They're standard and useful 
terminology / concepts for testing.

If we do move to a more "agnostic" wording in the failure messages 
(whilst keeping actual / expected as argument names and in the 
documentation perhaps?) then I prefer first / second to a / b.

All the best,

Michael Foord

[1] Interestingly unittest did use (first, second) for assert argument 
names back in 2.1 when it was added: 
http://svn.python.org/view/python/branches/release21-maint/Lib/unittest.py?revision=24536&view=markup 

[2] http://bugs.python.org/issue10573
> Raymond
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk


-- 

http://www.voidspace.org.uk/

May you do good and not evil
May you find forgiveness for yourself and forgive others
May you share freely, never taking more than you give.
-- the sqlite blessinghttp://www.sqlite.org/different.html


From solipsis at pitrou.net  Mon Dec 20 14:47:21 2010
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Mon, 20 Dec 2010 14:47:21 +0100
Subject: [Python-Dev] r87389 - in python/branches/py3k:
 Doc/library/unittest.rst Lib/unittest/case.py Misc/NEWS
In-Reply-To: <4D0F536A.8050808@voidspace.org.uk>
References: <20101218200004.99110EE985@mail.python.org>
	<20101218214830.1aa8be32@pitrou.net> <iejdjt$ik0$1@dough.gmane.org>
	<AANLkTi=ADkUMc5tQGDv2yniteyfPCPXU=XaCr+GscR+a@mail.gmail.com>
	<20101219141355.5ac7cd2f@pitrou.net>
	<AANLkTinyvb6+Xf=owdV22j6YoAEVjbTXV2tetQzdD8r4@mail.gmail.com>
	<31185718-F276-46CE-82E2-6C6B6EC920E8@gmail.com>
	<4D0F536A.8050808@voidspace.org.uk>
Message-ID: <1292852841.3737.25.camel@localhost.localdomain>


Le lundi 20 d?cembre 2010 ? 13:00 +0000, Michael Foord a ?crit :
> Ah man, we've *nearly* finished bikeshedding about the names of unittest 
> assert methods so its time to move onto the names and order of the 
> arguments. Really?

Apparently someone decided this bikeshedding was important enough to
make an SVN commit out of it. If you think it isn't worth discussing
then perhaps it wasn't worth changing in the first place :)

> The only thing left to decide is then the order - (actual, expected) or 
> (expected, actual). Raymond, myself, Guido and Ezio have all expressed a 
> preference for (actual, expected). I like this comment from Raymond on 
> the relevant issue [2]:
> 
>      I also tend to use actual/expected and find the reverse to be a 
> form Yoda-speak, "assert 5 == x", "perhaps misread the prophecy was", etc.

Isn't it some kind of ethnocentric comment? "Natural" order is not the
same in all languages, and I don't see why "actual" should come before
"expected".

And the problem here is that (actual, expected) is in reverse order of
the diff displayed on error.

Now if you look at various TypeErrors raised in Python, the error
message is most often "expected <some type>, got <other type>". So there
expected always comes before actual, and apparently it was natural to
the authors of that code. Perhaps they are all Yoda-speakers.

And moreover, some methods such as assertRaises() already have their
args in the (expected, actual) order.

> If we do move to a more "agnostic" wording in the failure messages 
> (whilst keeping actual / expected as argument names and in the 
> documentation perhaps?) then I prefer first / second to a / b.

Well, no. The whole point of displaying results as a/b (or first/second)
is that they map intuitively to the first and second args. If, however,
you name those args "actual" and "expected", it becomes confusing.

Also, it wouldn't fix that your diff would still be in a weird order
compared to the method args (unless you think it's fine to display an
"expected - actual" diff).

Regards

Antoine.



From fuzzyman at voidspace.org.uk  Mon Dec 20 15:03:26 2010
From: fuzzyman at voidspace.org.uk (Michael Foord)
Date: Mon, 20 Dec 2010 14:03:26 +0000
Subject: [Python-Dev] r87389 - in python/branches/py3k:
 Doc/library/unittest.rst Lib/unittest/case.py Misc/NEWS
In-Reply-To: <1292852841.3737.25.camel@localhost.localdomain>
References: <20101218200004.99110EE985@mail.python.org>	<20101218214830.1aa8be32@pitrou.net>
	<iejdjt$ik0$1@dough.gmane.org>	<AANLkTi=ADkUMc5tQGDv2yniteyfPCPXU=XaCr+GscR+a@mail.gmail.com>	<20101219141355.5ac7cd2f@pitrou.net>	<AANLkTinyvb6+Xf=owdV22j6YoAEVjbTXV2tetQzdD8r4@mail.gmail.com>	<31185718-F276-46CE-82E2-6C6B6EC920E8@gmail.com>	<4D0F536A.8050808@voidspace.org.uk>
	<1292852841.3737.25.camel@localhost.localdomain>
Message-ID: <4D0F622E.7050902@voidspace.org.uk>

On 20/12/2010 13:47, Antoine Pitrou wrote:
> Le lundi 20 d?cembre 2010 ? 13:00 +0000, Michael Foord a ?crit :
>> Ah man, we've *nearly* finished bikeshedding about the names of unittest
>> assert methods so its time to move onto the names and order of the
>> arguments. Really?
> Apparently someone decided this bikeshedding was important enough to
> make an SVN commit out of it. If you think it isn't worth discussing
> then perhaps it wasn't worth changing in the first place :)
>

The only change was to use them consistently and the only code change 
was to re-order the arguments in a method that is new and not in any 
previous version of Python. You're arguing for a much bigger change.

>> The only thing left to decide is then the order - (actual, expected) or
>> (expected, actual). Raymond, myself, Guido and Ezio have all expressed a
>> preference for (actual, expected). I like this comment from Raymond on
>> the relevant issue [2]:
>>
>>       I also tend to use actual/expected and find the reverse to be a
>> form Yoda-speak, "assert 5 == x", "perhaps misread the prophecy was", etc.
> Isn't it some kind of ethnocentric comment? "Natural" order is not the
> same in all languages, and I don't see why "actual" should come before
> "expected".

Agreement that actual, expected was preferred came from an American, a 
Dutchman, an Englishman and an Italian. :-)

As it is what unittest currently uses anyway you'll need more than "I 
don't see why" to reverse it.

> And the problem here is that (actual, expected) is in reverse order of
> the diff displayed on error.
>

Diffing is completely an implementation detail of how the failure 
messages are generated. The important thing is that failure messages 
make sense with respect to actual result and expected result.

> Now if you look at various TypeErrors raised in Python, the error
> message is most often "expected<some type>, got<other type>". So there
> expected always comes before actual, and apparently it was natural to
> the authors of that code. Perhaps they are all Yoda-speakers.
>
> And moreover, some methods such as assertRaises() already have their
> args in the (expected, actual) order.
>
>> If we do move to a more "agnostic" wording in the failure messages
>> (whilst keeping actual / expected as argument names and in the
>> documentation perhaps?) then I prefer first / second to a / b.
> Well, no. The whole point of displaying results as a/b (or first/second)
> is that they map intuitively to the first and second args. If, however,
> you name those args "actual" and "expected", it becomes confusing.
>
> Also, it wouldn't fix that your diff would still be in a weird order
> compared to the method args (unless you think it's fine to display an
> "expected - actual" diff).
>

I told you how I think the diff should be generated and gave the reasons 
for it. If we use first / second in failure messages (but not 
necessarily in documentation or argument names - your objections 
notwithstanding) then the messages for the container asserts don't need 
to imply a direction (as Raymond suggested, wording like "in first but 
not in second" and "in second but not in first"). The string diffs are 
more problematic as they actually use difflib to generate the failure 
output.

I'm also against dropping the use of actual / expected concepts and 
terminology within unittest as I think they are useful. We don't 
necessarily need to use them in the failure outputs but it seems like 
you want them to be dropped altogether.

All the best,

Michael

> Regards
>
> Antoine.
>
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk


-- 

http://www.voidspace.org.uk/

May you do good and not evil
May you find forgiveness for yourself and forgive others
May you share freely, never taking more than you give.
-- the sqlite blessing http://www.sqlite.org/different.html


From vstinner at edenwall.com  Mon Dec 20 14:39:29 2010
From: vstinner at edenwall.com (Victor Stinner)
Date: Mon, 20 Dec 2010 14:39:29 +0100
Subject: [Python-Dev]
 =?iso-8859-1?q?Issue_=238863_adds_a_new_PYTHONNOFAUL?=
 =?iso-8859-1?q?THANDLER=09environment_variable?=
In-Reply-To: <20101220110835.GA13825@yoda.bytereef.org>
References: <4D0C0688.1080108@haypocalc.com>
	<AANLkTimq+Qpt80u50Nj+RTgM7CgfLPsYWR2RHgDkZkP6@mail.gmail.com>
	<20101220110835.GA13825@yoda.bytereef.org>
Message-ID: <201012201439.29282.vstinner@edenwall.com>

Le lundi 20 d?cembre 2010 12:08:35, Stefan Krah a ?crit :
> I think the output is not particularly informative:
> 
> $ ./python Lib/test/crashers/gc_inspection.py
> (<object object at 0x7f01827ad870>, <NULL>, <NULL>, <NULL>, <NULL>, <NULL>,
> <NULL>, <NULL>, <NULL>, <NULL>) Fatal Python error: Segmentation fault
> 
> Traceback (most recent call first):
>   File "Lib/test/crashers/gc_inspection.py", line 29 in g
>   File "Lib/test/crashers/gc_inspection.py", line 32 in <module>
> Segmentation fault
> Segmentation fault

The backtrace is valid. Don't you think that this backtrace is more useful 
than just "Segmentation fault"?

> Another test is hanging indefinitely (Ubuntu 64-bit):
> 
> $ ./python Lib/test/crashers/nasty_eq_vs_dict.py
> [hanging with no output]

I didn't tried with patch version 9, but with the patch version 10 I get:
-----------------
$ ./python Lib/test/crashers/nasty_eq_vs_dict.py 
Fatal Python error: Segmentation fault

Traceback (most recent call first):
  File "Lib/test/crashers/nasty_eq_vs_dict.py", line 16 in __hash__
  File "Lib/test/crashers/nasty_eq_vs_dict.py", line 40 in __fill_dict
  File "Lib/test/crashers/nasty_eq_vs_dict.py", line 30 in __eq__
  File "Lib/test/crashers/nasty_eq_vs_dict.py", line 47 in <module>
Segmentation fault
Erreur de segmentation
$ echo $?
139
-----------------

> And this test does not report anything at all:
> 
> $ ./python Lib/test/crashers/compiler_recursion.py
> [no output at all]
> $ echo $?
> 0

Patch version 9 doesn't call the previous handler. Please retry with patch 
version 10 (which call the previous handler).

With the patch version 10 I get:
-----------------
$ ./python Lib/test/crashers/compiler_recursion.py 
Fatal Python error: Segmentation fault

Traceback (most recent call first):
  File "Lib/test/crashers/compiler_recursion.py", line 5 in <module>
Segmentation fault
Erreur de segmentation
$ echo $?
139
-----------------

Victor

From victor.stinner at haypocalc.com  Mon Dec 20 15:24:22 2010
From: victor.stinner at haypocalc.com (Victor Stinner)
Date: Mon, 20 Dec 2010 15:24:22 +0100
Subject: [Python-Dev] Issue #8863 adds a new PYTHONNOFAULTHANDLER
	environment variable
In-Reply-To: <4D0F0440.5070806@v.loewis.de>
References: <4D0C0688.1080108@haypocalc.com> <1292820829.4763.16.camel@marge>
	<4D0F0440.5070806@v.loewis.de>
Message-ID: <201012201524.23023.victor.stinner@haypocalc.com>

Le lundi 20 d?cembre 2010 08:22:40, vous avez ?crit :
> >> Looking at your function list, my other concern is that you are calling
> >> Python API without holding the GIL, IIUC. In particular, you are
> >> accessing _PyThreadState_Current, which may not point to the current
> >> thread if the current thread has released the GIL.
> > 
> > Ah? Where does _PyThreadState_Current point to if the GIL is not hold
> > when the fault handler is called?
> 
> The GIL is likely held by a different thread, then.
> _PyThreadState_Current will point to the state of this other thread.

I tested when the GIL released: the fault handler is unable to retrieve "the" 
thread state and so the backtrace is not printed. Which thread state should be 
retrieve? I don't know yet: maybe the last active thread state?

It doesn't look trivial to fix this issue because if there is no thread (only 
the main thread), release the GIL clears the thread state pointer (to it to 
NULL) without storing it somewhere else (whereas with multiple threads, the 
last active thread is stored in "gil_last_holder". I should ask Antoine (or 
someone one) how the new GIL works.

> Of this thread? I don't think so. gil_last_holder might also refer to
> a different thread.

Another variable might be created to store the (pointer to the) last active 
thread state.

> > I don't think that it will possible the acquire the GIL in
> > Py_FatalError() or in the fault handler.
> 
> I agree.

We all love deadlocks :-)

Victor

From solipsis at pitrou.net  Mon Dec 20 15:31:37 2010
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Mon, 20 Dec 2010 15:31:37 +0100
Subject: [Python-Dev] r87389 - in python/branches/py3k:
 Doc/library/unittest.rst Lib/unittest/case.py Misc/NEWS
In-Reply-To: <4D0F622E.7050902@voidspace.org.uk>
References: <20101218200004.99110EE985@mail.python.org>
	<20101218214830.1aa8be32@pitrou.net> <iejdjt$ik0$1@dough.gmane.org>
	<AANLkTi=ADkUMc5tQGDv2yniteyfPCPXU=XaCr+GscR+a@mail.gmail.com>
	<20101219141355.5ac7cd2f@pitrou.net>
	<AANLkTinyvb6+Xf=owdV22j6YoAEVjbTXV2tetQzdD8r4@mail.gmail.com>
	<31185718-F276-46CE-82E2-6C6B6EC920E8@gmail.com>
	<4D0F536A.8050808@voidspace.org.uk>
	<1292852841.3737.25.camel@localhost.localdomain>
	<4D0F622E.7050902@voidspace.org.uk>
Message-ID: <1292855497.3737.55.camel@localhost.localdomain>

Le lundi 20 d?cembre 2010 ? 14:03 +0000, Michael Foord a ?crit :
> On 20/12/2010 13:47, Antoine Pitrou wrote:
> > Le lundi 20 d?cembre 2010 ? 13:00 +0000, Michael Foord a ?crit :
> >> Ah man, we've *nearly* finished bikeshedding about the names of unittest
> >> assert methods so its time to move onto the names and order of the
> >> arguments. Really?
> > Apparently someone decided this bikeshedding was important enough to
> > make an SVN commit out of it. If you think it isn't worth discussing
> > then perhaps it wasn't worth changing in the first place :)
> 
> The only change was to use them consistently and the only code change 
> was to re-order the arguments in a method that is new and not in any 
> previous version of Python. You're arguing for a much bigger change.

No, I'm first of all arguing for a "first/second" or "a/b" argument
naming. Which was exactly the case before the change that triggered this
thread.

> >> The only thing left to decide is then the order - (actual, expected) or
> >> (expected, actual). Raymond, myself, Guido and Ezio have all expressed a
> >> preference for (actual, expected). I like this comment from Raymond on
> >> the relevant issue [2]:
> >>
> >>       I also tend to use actual/expected and find the reverse to be a
> >> form Yoda-speak, "assert 5 == x", "perhaps misread the prophecy was", etc.
> > Isn't it some kind of ethnocentric comment? "Natural" order is not the
> > same in all languages, and I don't see why "actual" should come before
> > "expected".
> 
> Agreement that actual, expected was preferred came from an American, a 
> Dutchman, an Englishman and an Italian. :-)

I'm not sure what that's supposed to prove, unless you have problems
with the idea that what is natural for a couple of people isn't natural
for everyone. You also apparently missed that part:

> > Now if you look at various TypeErrors raised in Python, the error
> > message is most often "expected<some type>, got<other type>". So there
> > expected always comes before actual, and apparently it was natural to
> > the authors of that code. Perhaps they are all Yoda-speakers.


> As it is what unittest currently uses anyway you'll need more than "I 
> don't see why" to reverse it.

unittest doesn't "use it anyway", since it used first/second before that
change. Actually, as I pointed out, (expected, actual) is used in
assertRaises in friends.

> > And the problem here is that (actual, expected) is in reverse order of
> > the diff displayed on error.
> >
> 
> Diffing is completely an implementation detail of how the failure 
> messages are generated. The important thing is that failure messages 
> make sense with respect to actual result and expected result.

Which, again, they don't. Let's see:

    self.assertEqual(actual, expected)
AssertionError: 'a\nb\nc\ne\n' != 'a\nb\nc\nd\n'
  a
  b
  c
- e
+ d

The diff shows "expected - actual", but it would be logical (in your own
logic) to display "actual - expected". The whole issue disappears if you
drop this idea of naming the arguments "actual" and "expected".

> I'm also against dropping the use of actual / expected concepts and 
> terminology within unittest as I think they are useful. We don't 
> necessarily need to use them in the failure outputs but it seems like 
> you want them to be dropped altogether.

I'm saying that they cause confusion wrt. to the actual error display
(as Guido also admitted). Feel free to come up with a solution that
doesn't get rid of actual/expected, if that's what you want ;)

Regards

Antoine.



From stefan at bytereef.org  Mon Dec 20 15:55:57 2010
From: stefan at bytereef.org (Stefan Krah)
Date: Mon, 20 Dec 2010 15:55:57 +0100
Subject: [Python-Dev] Issue #8863 adds a
	new	PYTHONNOFAULTHANDLER?environment variable
In-Reply-To: <201012201439.29282.vstinner@edenwall.com>
References: <4D0C0688.1080108@haypocalc.com>
	<AANLkTimq+Qpt80u50Nj+RTgM7CgfLPsYWR2RHgDkZkP6@mail.gmail.com>
	<20101220110835.GA13825@yoda.bytereef.org>
	<201012201439.29282.vstinner@edenwall.com>
Message-ID: <20101220145557.GA17397@yoda.bytereef.org>

Victor Stinner <vstinner at edenwall.com> wrote:
> Le lundi 20 d?cembre 2010 12:08:35, Stefan Krah a ?crit :
> > I think the output is not particularly informative:
> > 
> > $ ./python Lib/test/crashers/gc_inspection.py
> > (<object object at 0x7f01827ad870>, <NULL>, <NULL>, <NULL>, <NULL>, <NULL>,
> > <NULL>, <NULL>, <NULL>, <NULL>) Fatal Python error: Segmentation fault
> > 
> > Traceback (most recent call first):
> >   File "Lib/test/crashers/gc_inspection.py", line 29 in g
> >   File "Lib/test/crashers/gc_inspection.py", line 32 in <module>
> > Segmentation fault
> > Segmentation fault
> 
> The backtrace is valid. Don't you think that this backtrace is more useful 
> than just "Segmentation fault"?

Perhaps I misunderstood, but I thought the purpose of the patch was to
let developers act more quickly on bug reports. I wonder if this output
really speeds up the process.

Do you have an example bug where this patch helps in finding the precise
location of a segfault?


> Patch version 9 doesn't call the previous handler. Please retry with patch 
> version 10 (which call the previous handler).

I did use version 10. I've verified the same behavior with a fresh py3k
checkout and this patch:

$ md5sum segfault_handler-10.patch
e1b5a9439d2b51d0a63f701dd694796d  segfault_handler-10.patch


My machine currently has a load average of 2. Perhaps you'll be able to
reproduce it if you crank up the load average.



Stefan Krah



From vstinner at edenwall.com  Mon Dec 20 16:33:12 2010
From: vstinner at edenwall.com (Victor Stinner)
Date: Mon, 20 Dec 2010 16:33:12 +0100
Subject: [Python-Dev]
 =?iso-8859-1?q?Issue_=238863_adds_a_new=09PYTHONNOFA?=
 =?iso-8859-1?q?ULTHANDLER=3Fenvironment_variable?=
In-Reply-To: <20101220145557.GA17397@yoda.bytereef.org>
References: <4D0C0688.1080108@haypocalc.com>
	<201012201439.29282.vstinner@edenwall.com>
	<20101220145557.GA17397@yoda.bytereef.org>
Message-ID: <201012201633.12988.vstinner@edenwall.com>

Le lundi 20 d?cembre 2010 15:55:57, Stefan Krah a ?crit :
> > The backtrace is valid. Don't you think that this backtrace is more
> > useful than just "Segmentation fault"?
> 
> Perhaps I misunderstood, but I thought the purpose of the patch was to
> let developers act more quickly on bug reports. I wonder if this output
> really speeds up the process.

The fault handler helps developers because they don't have to have a Python 
compiled in debug mode and to run the application in a debugger (like gdb).

If the developer is unable to reproduce an error, because it's an Heisenbug, 
or because the developer doesn't have the same OS, libraries and/or 
applications, the fault handler helps the developer to isolate the problem.

Many bugs only occur once and you cannot report a bug report because you don't 
have any useful information.

But the fault handler doesn't help graphical applications not started from a 
terminal...

> Do you have an example bug where this patch helps in finding the precise
> location of a segfault?

Any bug report which only contain "Segmentation fault" information. But I 
cannot give you a revelant example because users are too shy to report an 
issue if they don't have more information than just "the program crashed". 
With a fault handler displaying the Python backtrace, I hope that more *users* 
will report bugs.

Example of an issue with only the "Segmentation fault" information:
http://bugs.python.org/issue6011#msg121551
I am unable to reproduce this crash, and I'm very frustrated because I want to 
fix it! :-)

Another example which might benefit from the fault handler:
http://bugs.python.org/issue7424
---
Compiling /usr/pkg/lib/python2.6/test/test_builtin.py ...
[1]   Segmentation fault (core dumped) PYTHONPATH=/usr/...
*** Error code 139
---

If the reporter already wrote a minimal script, it doesn't help. The fault 
handler helps to write the minimal script (to isolate the bug).

> I did use version 10. I've verified the same behavior with a fresh py3k
> checkout and this patch: (...)
> 
> My machine currently has a load average of 2. Perhaps you'll be able to
> reproduce it if you crank up the load average.

What is your OS (name, version)? I only tested crashers on Linux (Debian Sid).

Victor

From rrr at ronadam.com  Mon Dec 20 17:47:33 2010
From: rrr at ronadam.com (Ron Adam)
Date: Mon, 20 Dec 2010 10:47:33 -0600
Subject: [Python-Dev] r87389 - in python/branches/py3k:
 Doc/library/unittest.rst Lib/unittest/case.py Misc/NEWS
In-Reply-To: <iejdjt$ik0$1@dough.gmane.org>
References: <20101218200004.99110EE985@mail.python.org>	<20101218214830.1aa8be32@pitrou.net>
	<iejdjt$ik0$1@dough.gmane.org>
Message-ID: <4D0F88A5.9070306@ronadam.com>



On 12/18/2010 04:46 PM, Terry Reedy wrote:
> On 12/18/2010 3:48 PM, Antoine Pitrou wrote:
>> On Sat, 18 Dec 2010 21:00:04 +0100 (CET)
>> ezio.melotti<python-checkins at python.org> wrote:
>>> Author: ezio.melotti
>>> Date: Sat Dec 18 21:00:04 2010
>>> New Revision: 87389
>>>
>>> Log:
>>> #10573: use actual/expected consistently in unittest methods.
>
> Change was requested by M. Foord and R. Hettinger (and G.Brandl for b2).
>
>> IMHO, this should be reverted. The API currently doesn't treat these
>> arguments differently, so they should really be labeled "first" and
>> "second". Otherwise, the user will wrongly assume that the signature is
>> asymmetric and that they should be careful about which order they pass
>> the arguments in.

I've always presumed it would make a difference in error displayed anyway.


> The error report on assert failure *is* often asymmetrical ;=).
>  From Michael's post:
> "This is particularly relevant for the methods that produce 'diffed' output
> on failure - as the order determines whether mismatched items are missing
> from the expected or additional to the expected."
>
> This change struck me as a nice bit of polishing.

I like ("actual", "expected") in the asserts.  It matches my expected 
ordering of "input"/"output" and how I use comparisons in 'if' statements.

I feel it is more important that the diffs are consistent with other diffs 
in python.

So (for me), changing the asymmetrical output to be symmetrical would be in 
the category of foolish consistency because changing that, introduces other 
inconsistencies I'd rather not have.

It doesn't bother me that the functions arguments aren't the same order of 
the diffs as long as the labels and wording are obvious enough in the 
messages.  So maybe the diff output can be improved a bit instead of 
changing the terms and ordering.

Ron


From scott.urban at isilon.com  Mon Dec 20 17:48:25 2010
From: scott.urban at isilon.com (Scott Urban)
Date: Mon, 20 Dec 2010 08:48:25 -0800
Subject: [Python-Dev] sqlite, DDL, and transactions
Message-ID: <20101220164825.GZ21083@surban.west.isilon.com>

Hi

The python sqlite module automatically commits open transactions
when it encounters a DDL statement.  This is unnecessary; DDL is
transaction in my testing (see attached).

Attached patch addresses the issue. Patch is against 2.6.1, but
looking at Trunk in svn, it seems like the patch is needed and
would apply. One issue I could foresee is that this behavior might
depend on the sqlite version in use (I'm on 3.6.10).

Thanks
Scott
-------------- next part --------------
A non-text attachment was scrubbed...
Name: pysql-transactions.2.diff
Type: text/x-diff
Size: 2545 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/python-dev/attachments/20101220/3c640c23/attachment.diff>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: test_sqlite_ddl.py
Type: text/x-python
Size: 3433 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/python-dev/attachments/20101220/3c640c23/attachment.py>

From stefan at bytereef.org  Mon Dec 20 17:56:32 2010
From: stefan at bytereef.org (Stefan Krah)
Date: Mon, 20 Dec 2010 17:56:32 +0100
Subject: [Python-Dev] Issue #8863 adds
	a	new?PYTHONNOFAULTHANDLER?environment variable
In-Reply-To: <201012201633.12988.vstinner@edenwall.com>
References: <4D0C0688.1080108@haypocalc.com>
	<201012201439.29282.vstinner@edenwall.com>
	<20101220145557.GA17397@yoda.bytereef.org>
	<201012201633.12988.vstinner@edenwall.com>
Message-ID: <20101220165632.GA19009@yoda.bytereef.org>

Victor Stinner <vstinner at edenwall.com> wrote:
> Le lundi 20 d?cembre 2010 15:55:57, Stefan Krah a ?crit :
> > > The backtrace is valid. Don't you think that this backtrace is more
> > > useful than just "Segmentation fault"?
> > 
> > Perhaps I misunderstood, but I thought the purpose of the patch was to
> > let developers act more quickly on bug reports. I wonder if this output
> > really speeds up the process.
> 
> The fault handler helps developers because they don't have to have a Python 
> compiled in debug mode and to run the application in a debugger (like gdb).
> 
> If the developer is unable to reproduce an error, because it's an Heisenbug, 
> or because the developer doesn't have the same OS, libraries and/or 
> applications, the fault handler helps the developer to isolate the problem.

This is what I'm sceptical about. I think the tracebacks are don't carry
enough information for that.


> Any bug report which only contain "Segmentation fault" information. But I 
> cannot give you a revelant example because users are too shy to report an 
> issue if they don't have more information than just "the program crashed". 
> With a fault handler displaying the Python backtrace, I hope that more *users* 
> will report bugs.

I wonder. It would be fine to make the tracebacks optional and tell the
user to rerun the program with the proper environment variable set.


> > I did use version 10. I've verified the same behavior with a fresh py3k
> > checkout and this patch: (...)
> > 
> > My machine currently has a load average of 2. Perhaps you'll be able to
> > reproduce it if you crank up the load average.
> 
> What is your OS (name, version)? I only tested crashers on Linux (Debian Sid).

Ubuntu 8.10, 64-bit, real hardware.


I also tested on FreeBSD-8.0-qemu:

$ ./python Lib/test/crashers/compiler_recursion.py 
Illegal instruction: 4 (core dumped)


$ ./python Lib/test/crashers/nasty_eq_vs_dict.py 
[hanging with no output]



Stefan Krah



From brian.curtin at gmail.com  Mon Dec 20 18:03:23 2010
From: brian.curtin at gmail.com (Brian Curtin)
Date: Mon, 20 Dec 2010 11:03:23 -0600
Subject: [Python-Dev] sqlite, DDL, and transactions
In-Reply-To: <20101220164825.GZ21083@surban.west.isilon.com>
References: <20101220164825.GZ21083@surban.west.isilon.com>
Message-ID: <AANLkTimxnchTPOB4SEbOoPGgqyrk2tm0is5JGGzY86m_@mail.gmail.com>

On Mon, Dec 20, 2010 at 10:48, Scott Urban <scott.urban at isilon.com> wrote:

> Hi
>
> The python sqlite module automatically commits open transactions
> when it encounters a DDL statement.  This is unnecessary; DDL is
> transaction in my testing (see attached).
>
> Attached patch addresses the issue. Patch is against 2.6.1, but
> looking at Trunk in svn, it seems like the patch is needed and
> would apply. One issue I could foresee is that this behavior might
> depend on the sqlite version in use (I'm on 3.6.10).
>
> Thanks
> Scott


Can you submit this to bugs.python.org? Bug reports and their patches on the
mailing list tend to get lost.

Additionally, 2.7 would be the 2.x version to fix this against
(branches/release27-maint when looking in svn).
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20101220/2b81e2d8/attachment.html>

From fuzzyman at voidspace.org.uk  Mon Dec 20 18:03:51 2010
From: fuzzyman at voidspace.org.uk (Michael Foord)
Date: Mon, 20 Dec 2010 17:03:51 +0000
Subject: [Python-Dev] sqlite, DDL, and transactions
In-Reply-To: <20101220164825.GZ21083@surban.west.isilon.com>
References: <20101220164825.GZ21083@surban.west.isilon.com>
Message-ID: <4D0F8C77.6020704@voidspace.org.uk>

On 20/12/2010 16:48, Scott Urban wrote:
> Hi
>
> The python sqlite module automatically commits open transactions
> when it encounters a DDL statement.  This is unnecessary; DDL is
> transaction in my testing (see attached).
>

Hello Scott,

Please post your patch to the Python bug tracker.

http://bugs.python.org/

All the best,

Michael Foord

> Attached patch addresses the issue. Patch is against 2.6.1, but
> looking at Trunk in svn, it seems like the patch is needed and
> would apply. One issue I could foresee is that this behavior might
> depend on the sqlite version in use (I'm on 3.6.10).
>
> Thanks
> Scott
>
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk


-- 

http://www.voidspace.org.uk/

May you do good and not evil
May you find forgiveness for yourself and forgive others
May you share freely, never taking more than you give.
-- the sqlite blessinghttp://www.sqlite.org/different.html

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20101220/b73e3c3d/attachment.html>

From alexander.belopolsky at gmail.com  Mon Dec 20 18:22:48 2010
From: alexander.belopolsky at gmail.com (Alexander Belopolsky)
Date: Mon, 20 Dec 2010 12:22:48 -0500
Subject: [Python-Dev] Issue #8863 adds a
 new?PYTHONNOFAULTHANDLER?environment variable
In-Reply-To: <20101220165632.GA19009@yoda.bytereef.org>
References: <4D0C0688.1080108@haypocalc.com>
	<201012201439.29282.vstinner@edenwall.com>
	<20101220145557.GA17397@yoda.bytereef.org>
	<201012201633.12988.vstinner@edenwall.com>
	<20101220165632.GA19009@yoda.bytereef.org>
Message-ID: <AANLkTimecoqHTiUPexqDeeFjA2QWpf+Hnz+9brR2BDfO@mail.gmail.com>

On Mon, Dec 20, 2010 at 11:56 AM, Stefan Krah <stefan at bytereef.org> wrote:
> Victor Stinner <vstinner at edenwall.com> wrote:
..
>> The fault handler helps developers because they don't have to have a Python
>> compiled in debug mode and to run the application in a debugger (like gdb).
>>
>> If the developer is unable to reproduce an error, because it's an Heisenbug,
>> or because the developer doesn't have the same OS, libraries and/or
>> applications, the fault handler helps the developer to isolate the problem.
>
> This is what I'm sceptical about. I think the tracebacks are don't carry
> enough information for that.
>

My concern is that the traceback generated by a signal handler may
actually contain *less* information that what would otherwise be
available from the core dump or other system diagnostic facilities.
Whenever you continue to execute code after  a fault occurs (even in
an alternative stack), you change register values and run the risk
that the program will not be able to produce a core dump at all or
will produce a core dump with misleading information in it.

I think Stefan's approach is the correct one: before we agree on this
feature, the proponents should present actual crashers for which
reports generated by the patch are actually helpful.

From rrr at ronadam.com  Mon Dec 20 17:47:33 2010
From: rrr at ronadam.com (Ron Adam)
Date: Mon, 20 Dec 2010 10:47:33 -0600
Subject: [Python-Dev] r87389 - in python/branches/py3k:
 Doc/library/unittest.rst Lib/unittest/case.py Misc/NEWS
In-Reply-To: <iejdjt$ik0$1@dough.gmane.org>
References: <20101218200004.99110EE985@mail.python.org>	<20101218214830.1aa8be32@pitrou.net>
	<iejdjt$ik0$1@dough.gmane.org>
Message-ID: <4D0F88A5.9070306@ronadam.com>



On 12/18/2010 04:46 PM, Terry Reedy wrote:
> On 12/18/2010 3:48 PM, Antoine Pitrou wrote:
>> On Sat, 18 Dec 2010 21:00:04 +0100 (CET)
>> ezio.melotti<python-checkins at python.org> wrote:
>>> Author: ezio.melotti
>>> Date: Sat Dec 18 21:00:04 2010
>>> New Revision: 87389
>>>
>>> Log:
>>> #10573: use actual/expected consistently in unittest methods.
>
> Change was requested by M. Foord and R. Hettinger (and G.Brandl for b2).
>
>> IMHO, this should be reverted. The API currently doesn't treat these
>> arguments differently, so they should really be labeled "first" and
>> "second". Otherwise, the user will wrongly assume that the signature is
>> asymmetric and that they should be careful about which order they pass
>> the arguments in.

I've always presumed it would make a difference in error displayed anyway.


> The error report on assert failure *is* often asymmetrical ;=).
>  From Michael's post:
> "This is particularly relevant for the methods that produce 'diffed' output
> on failure - as the order determines whether mismatched items are missing
> from the expected or additional to the expected."
>
> This change struck me as a nice bit of polishing.

I like ("actual", "expected") in the asserts.  It matches my expected 
ordering of "input"/"output" and how I use comparisons in 'if' statements.

I feel it is more important that the diffs are consistent with other diffs 
in python.

So (for me), changing the asymmetrical output to be symmetrical would be in 
the category of foolish consistency because changing that, introduces other 
inconsistencies I'd rather not have.

It doesn't bother me that the functions arguments aren't the same order of 
the diffs as long as the labels and wording are obvious enough in the 
messages.  So maybe the diff output can be improved a bit instead of 
changing the terms and ordering.

Ron

From gzlist at googlemail.com  Mon Dec 20 18:31:24 2010
From: gzlist at googlemail.com (Martin (gzlist))
Date: Mon, 20 Dec 2010 17:31:24 +0000
Subject: [Python-Dev] Issue #8863 adds a new PYTHONNOFAULTHANDLER
 environment variable
In-Reply-To: <4D0EA9E0.4080707@voidspace.org.uk>
References: <4D0C0688.1080108@haypocalc.com>
	<B880623A-1EA5-41CA-A58C-C46C93BED8DC@fuhm.net>
	<1292799745.5914.76.camel@marge>
	<4D0EA9E0.4080707@voidspace.org.uk>
Message-ID: <AANLkTi=5ejTcR9p8GKGsJP_DUPNVpO_a94ZgP=_ChVjM@mail.gmail.com>

On 20/12/2010, Michael Foord <fuzzyman at voidspace.org.uk> wrote:
> On 19/12/2010 23:02, Victor Stinner wrote:
>> I don't think that kill() is available on Windows.

Can just use raise instead.

But, is any of this change valid on Windows?

<http://msdn.microsoft.com/library/xdkz3x12>
"The SIGILL, SIGSEGV, and SIGTERM signals are not generated under
Windows NT. They are included for ANSI compatibility."

And the last signal this concerns is explicitly skipped in the tests
on the latest patch under windows with the note "SIGFPE cannot be
catched on Windows". Signals are a posix compatibility hack on win32,
they're unlikely to do what you want.

> Thanks to Brian Curtin there is now an os.kill() on Windows, but it
> doesn't handle the full range of signals.

Which uses GenerateConsoleCtrlEvent and is completely the wrong thing.

Martin

From vstinner at edenwall.com  Mon Dec 20 18:34:19 2010
From: vstinner at edenwall.com (Victor Stinner)
Date: Mon, 20 Dec 2010 18:34:19 +0100
Subject: [Python-Dev]
 =?iso-8859-1?q?Issue_=238863_adds_a_new_PYTHONNOFAUL?=
 =?iso-8859-1?q?THANDLER=09environment_variable?=
In-Reply-To: <20101220110835.GA13825@yoda.bytereef.org>
References: <4D0C0688.1080108@haypocalc.com>
	<AANLkTimq+Qpt80u50Nj+RTgM7CgfLPsYWR2RHgDkZkP6@mail.gmail.com>
	<20101220110835.GA13825@yoda.bytereef.org>
Message-ID: <201012201834.19264.vstinner@edenwall.com>

Le lundi 20 d?cembre 2010 12:08:35, Stefan Krah a ?crit :
> Another test is hanging indefinitely (Ubuntu 64-bit):
> 
> $ ./python Lib/test/crashers/nasty_eq_vs_dict.py
> [hanging with no output]

Oh, I understood. I always test with Python compiled using --with-pydebug. 
With pydebug, a segfault occurs. Without pydebug, there is an unlimited loop.

The patch is not related to the lack of segfault :-) (Retry without the patch)

Victor

From vstinner at edenwall.com  Mon Dec 20 18:51:09 2010
From: vstinner at edenwall.com (Victor Stinner)
Date: Mon, 20 Dec 2010 18:51:09 +0100
Subject: [Python-Dev] Issue #8863 adds a
	new?PYTHONNOFAULTHANDLER?environment variable
In-Reply-To: <AANLkTimecoqHTiUPexqDeeFjA2QWpf+Hnz+9brR2BDfO@mail.gmail.com>
References: <4D0C0688.1080108@haypocalc.com>
	<20101220165632.GA19009@yoda.bytereef.org>
	<AANLkTimecoqHTiUPexqDeeFjA2QWpf+Hnz+9brR2BDfO@mail.gmail.com>
Message-ID: <201012201851.09810.vstinner@edenwall.com>

Le lundi 20 d?cembre 2010 18:22:48, Alexander Belopolsky a ?crit :
> On Mon, Dec 20, 2010 at 11:56 AM, Stefan Krah <stefan at bytereef.org> wrote:
> > Victor Stinner <vstinner at edenwall.com> wrote:
> ..
> 
> >> The fault handler helps developers because they don't have to have a
> >> Python compiled in debug mode and to run the application in a debugger
> >> (like gdb).
> >> 
> >> If the developer is unable to reproduce an error, because it's an
> >> Heisenbug, or because the developer doesn't have the same OS, libraries
> >> and/or applications, the fault handler helps the developer to isolate
> >> the problem.
> > 
> > This is what I'm sceptical about. I think the tracebacks are don't carry
> > enough information for that.

Yes, in some cases the backtrace doesn't help. But why don't you want to 
display the backtrace if it can help in some other cases?

> My concern is that the traceback generated by a signal handler may
> actually contain *less* information that what would otherwise be
> available from the core dump or other system diagnostic facilities.

I changed my patch to call the previous signal handler starting at version 10. 
So the core dump function or OS fault handler will be called two. Said 
differently, it only gives *more* information, not less.

If you have a better OS fault handler than Python fault handler, you should 
disable Python fault handler by setting the environment variable.

> Whenever you continue to execute code after  a fault occurs (even in
> an alternative stack), you change register values and run the risk
> that the program will not be able to produce a core dump at all or
> will produce a core dump with misleading information in it.

Are you sure that a signal handler changes the registers? At least Linux saves 
integer and float registers before calling the signal handler, and then 
restores them.

The fault handler only changes some registers and write some bytes on the 
stack. The stack pointer is restored at exit, so I don't think that it causes 
troubles to use a core dump.

Dump a core file doesn't rely on process registers or mappings. Why do you 
think that it will break the core dump function?

Victor

From scott.urban at isilon.com  Mon Dec 20 19:05:17 2010
From: scott.urban at isilon.com (Scott Urban)
Date: Mon, 20 Dec 2010 10:05:17 -0800
Subject: [Python-Dev] sqlite, DDL, and transactions
In-Reply-To: <4D0F8C77.6020704@voidspace.org.uk>
References: <20101220164825.GZ21083@surban.west.isilon.com>
	<4D0F8C77.6020704@voidspace.org.uk>
Message-ID: <20101220180517.GB21083@surban.west.isilon.com>

On Mon, Dec 20, 2010 at 17:03:51, Michael Foord sent:
> On 20/12/2010 16:48, Scott Urban wrote:
> >Hi
> >
> >The python sqlite module automatically commits open transactions
> >when it encounters a DDL statement.  This is unnecessary; DDL is
> >transaction in my testing (see attached).
> >
> 
> Hello Scott,
> 
> Please post your patch to the Python bug tracker.
> 
> http://bugs.python.org/

http://bugs.python.org/issue10740

Thanks
Scott


From alexander.belopolsky at gmail.com  Mon Dec 20 19:09:32 2010
From: alexander.belopolsky at gmail.com (Alexander Belopolsky)
Date: Mon, 20 Dec 2010 13:09:32 -0500
Subject: [Python-Dev] Issue #8863 adds a
 new?PYTHONNOFAULTHANDLER?environment variable
In-Reply-To: <201012201851.09810.vstinner@edenwall.com>
References: <4D0C0688.1080108@haypocalc.com>
	<20101220165632.GA19009@yoda.bytereef.org>
	<AANLkTimecoqHTiUPexqDeeFjA2QWpf+Hnz+9brR2BDfO@mail.gmail.com>
	<201012201851.09810.vstinner@edenwall.com>
Message-ID: <AANLkTik+tH6UndVMwQGiwj0+2sFpHT379j4ZBpH_BEDR@mail.gmail.com>

On Mon, Dec 20, 2010 at 12:51 PM, Victor Stinner <vstinner at edenwall.com> wrote:
..
> Are you sure that a signal handler changes the registers? At least Linux saves
> integer and float registers before calling the signal handler, and then
> restores them.
>
What if the program crashes (again) in the signal handler before
having a chance to restore registers?  Also, can you point to
documentation that describes register restoring behavior?  I am not
sure all registers are (or even can be) restored.

> The fault handler only changes some registers and write some bytes on the
> stack. The stack pointer is restored at exit, so I don't think that it causes
> troubles to use a core dump.
>
> Dump a core file doesn't rely on process registers or mappings. Why do you
> think that it will break the core dump function?

Because I had actual experience with that using R.  Your code may be
better, but R brings up a dialog on segfault asking whether to produce
a core dump.  Occasionally, that core dump would be completely
unusable because it would point to an unrelated point in the code.  I
don't have a specific case handy because long ago I disabled segfault
handlers in the local R installation.  (I actually use RPy which loads
R into  Python as a shared library, so the two handlers will conflict
anyways.)

From rdmurray at bitdance.com  Mon Dec 20 19:46:01 2010
From: rdmurray at bitdance.com (R. David Murray)
Date: Mon, 20 Dec 2010 13:46:01 -0500
Subject: [Python-Dev] Issue #8863 adds a new
	PYTHONNOFAULTHANDLER?environment variable
In-Reply-To: <20101220145557.GA17397@yoda.bytereef.org>
References: <4D0C0688.1080108@haypocalc.com>
	<AANLkTimq+Qpt80u50Nj+RTgM7CgfLPsYWR2RHgDkZkP6@mail.gmail.com>
	<20101220110835.GA13825@yoda.bytereef.org>
	<201012201439.29282.vstinner@edenwall.com>
	<20101220145557.GA17397@yoda.bytereef.org>
Message-ID: <20101220184601.955F62002A9@kimball.webabinitio.net>

On Mon, 20 Dec 2010 15:55:57 +0100, Stefan Krah <stefan at bytereef.org> wrote:
> Victor Stinner <vstinner at edenwall.com> wrote:
> > Stefan Krah wrote:
> > > I think the output is not particularly informative:
> > > 
> > > $ ./python Lib/test/crashers/gc_inspection.py
> > > (<object object at 0x7f01827ad870>, <NULL>, <NULL>, <NULL>, <NULL>, <NULL>,
> > > <NULL>, <NULL>, <NULL>, <NULL>) Fatal Python error: Segmentation fault
> > >  
> > > Traceback (most recent call first):
> > >   File "Lib/test/crashers/gc_inspection.py", line 29 in g
> > >   File "Lib/test/crashers/gc_inspection.py", line 32 in <module>
> > > Segmentation fault
> > > Segmentation fault
> 
> > The backtrace is valid. Don't you think that this backtrace is more useful
> > than just "Segmentation fault"?
> 
> Perhaps I misunderstood, but I thought the purpose of the patch was to
> let developers act more quickly on bug reports. I wonder if this output
> really speeds up the process.
> 
> Do you have an example bug where this patch helps in finding the precise
> location of a segfault?

How is 'line 29 in g' not more precise than 'Segmentation fault'?

Now imagine that this error is being produced from a 20,000 line
application program bundle...  

--
R. David Murray                                      www.bitdance.com

From stefan at bytereef.org  Mon Dec 20 20:01:33 2010
From: stefan at bytereef.org (Stefan Krah)
Date: Mon, 20 Dec 2010 20:01:33 +0100
Subject: [Python-Dev] Issue #8863 adds a
	new	PYTHONNOFAULTHANDLER?environment variable
In-Reply-To: <201012201834.19264.vstinner@edenwall.com>
References: <4D0C0688.1080108@haypocalc.com>
	<AANLkTimq+Qpt80u50Nj+RTgM7CgfLPsYWR2RHgDkZkP6@mail.gmail.com>
	<20101220110835.GA13825@yoda.bytereef.org>
	<201012201834.19264.vstinner@edenwall.com>
Message-ID: <20101220190133.GA20442@yoda.bytereef.org>

Victor Stinner <vstinner at edenwall.com> wrote:
> Le lundi 20 d?cembre 2010 12:08:35, Stefan Krah a ?crit :
> > Another test is hanging indefinitely (Ubuntu 64-bit):
> > 
> > $ ./python Lib/test/crashers/nasty_eq_vs_dict.py
> > [hanging with no output]
> 
> Oh, I understood. I always test with Python compiled using --with-pydebug. 
> With pydebug, a segfault occurs. Without pydebug, there is an unlimited loop.

I'll open a feature request for more reliable segfaults. ;)


Still, on FreeBSD-8.0 the patch does not produce a traceback and also
appears to change the signal:


[without the patch]
$ ./python Lib/test/crashers/compiler_recursion.py 
Segmentation fault: 11 (core dumped)
$ echo $?
139


[with the patch]
$ ./python Lib/test/crashers/compiler_recursion.py 
Illegal instruction: 4 (core dumped)
$ echo $?
132


Stefan Krah



From v+python at g.nevcal.com  Mon Dec 20 21:41:04 2010
From: v+python at g.nevcal.com (Glenn Linderman)
Date: Mon, 20 Dec 2010 12:41:04 -0800
Subject: [Python-Dev] r87389 - in python/branches/py3k:
 Doc/library/unittest.rst Lib/unittest/case.py Misc/NEWS
In-Reply-To: <1292855497.3737.55.camel@localhost.localdomain>
References: <20101218200004.99110EE985@mail.python.org>	<20101218214830.1aa8be32@pitrou.net>
	<iejdjt$ik0$1@dough.gmane.org>	<AANLkTi=ADkUMc5tQGDv2yniteyfPCPXU=XaCr+GscR+a@mail.gmail.com>	<20101219141355.5ac7cd2f@pitrou.net>	<AANLkTinyvb6+Xf=owdV22j6YoAEVjbTXV2tetQzdD8r4@mail.gmail.com>	<31185718-F276-46CE-82E2-6C6B6EC920E8@gmail.com>	<4D0F536A.8050808@voidspace.org.uk>	<1292852841.3737.25.camel@localhost.localdomain>	<4D0F622E.7050902@voidspace.org.uk>
	<1292855497.3737.55.camel@localhost.localdomain>
Message-ID: <4D0FBF60.1040300@g.nevcal.com>

On 12/20/2010 6:31 AM, Antoine Pitrou wrote:
>> >  Diffing is completely an implementation detail of how the failure
>> >  messages are generated. The important thing is that failure messages
>> >  make sense with respect to actual result and expected result.
> Which, again, they don't. Let's see:
>
>      self.assertEqual(actual, expected)
> AssertionError: 'a\nb\nc\ne\n' != 'a\nb\nc\nd\n'
>    a
>    b
>    c
> - e
> + d
>
> The diff shows "expected - actual", but it would be logical (in your own
> logic) to display "actual - expected". The whole issue disappears if you
> drop this idea of naming the arguments "actual" and "expected".

I'm not a unittest user, although I probably will become one, in time, 
when I learn enough to contribute to Python, instead of just find bugs 
in it from use.

I don't much care what the parameters names are, although the terms 
actual and expected seem good for testing scenarios if properly used, 
but the above does not match what I would expect the behavior to be from 
a testing scenario: run the test, and tell me what changed from the 
expected results.

If the expected result (not parameter) is d and the actual result (not 
parameter) is e, the diff should show

   a
   b
   c
- d
+ e

Thinking-that-sometimes-a-novice's-expectations-are-relevant-to-such-discussions'ly 
yours,
Glenn



-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20101220/daeed91d/attachment.html>

From guido at python.org  Mon Dec 20 23:29:50 2010
From: guido at python.org (Guido van Rossum)
Date: Mon, 20 Dec 2010 14:29:50 -0800
Subject: [Python-Dev] r87389 - in python/branches/py3k:
 Doc/library/unittest.rst Lib/unittest/case.py Misc/NEWS
In-Reply-To: <4D0F27FE.4020706@pearwood.info>
References: <20101218200004.99110EE985@mail.python.org>
	<20101218214830.1aa8be32@pitrou.net> <iejdjt$ik0$1@dough.gmane.org>
	<AANLkTi=ADkUMc5tQGDv2yniteyfPCPXU=XaCr+GscR+a@mail.gmail.com>
	<20101219141355.5ac7cd2f@pitrou.net>
	<AANLkTinyvb6+Xf=owdV22j6YoAEVjbTXV2tetQzdD8r4@mail.gmail.com>
	<iem60d$svg$1@dough.gmane.org> <20101220014840.6b734274@pitrou.net>
	<4D0F27FE.4020706@pearwood.info>
Message-ID: <AANLkTin=+7JG6gyfVB46zzCG--SqUSnUZ_33ZyirDQfy@mail.gmail.com>

On Mon, Dec 20, 2010 at 1:55 AM, Steven D'Aprano <steve at pearwood.info> wrote:
> Antoine Pitrou wrote:
>
>> For a non-native English speaker, 'a' and 'b' don't evoke 'after' and
>> 'before' but simply the first two letters of the latin alphabet, and
>> their ordering is therefore obvious with respect to function arguments.
>
> It's not just non-native English speakers either. I too think of a, b as
> being first, second rather than after, before.

I was mostly being facetious (my main point being there's no perfect
solution here), though I *have* seen serious code using the
b=before/a=after convention.

-- 
--Guido van Rossum (python.org/~guido)

From martin at v.loewis.de  Tue Dec 21 00:00:39 2010
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Tue, 21 Dec 2010 00:00:39 +0100
Subject: [Python-Dev] Issue #8863 adds a
 new	PYTHONNOFAULTHANDLER?environment variable
In-Reply-To: <20101220184601.955F62002A9@kimball.webabinitio.net>
References: <4D0C0688.1080108@haypocalc.com>	<AANLkTimq+Qpt80u50Nj+RTgM7CgfLPsYWR2RHgDkZkP6@mail.gmail.com>	<20101220110835.GA13825@yoda.bytereef.org>	<201012201439.29282.vstinner@edenwall.com>	<20101220145557.GA17397@yoda.bytereef.org>
	<20101220184601.955F62002A9@kimball.webabinitio.net>
Message-ID: <4D0FE017.50509@v.loewis.de>

>> Do you have an example bug where this patch helps in finding the precise
>> location of a segfault?
> 
> How is 'line 29 in g' not more precise than 'Segmentation fault'?

I think Stefan is treating "precise location" as an absolute property,
not a relative one. The precise location would be the very instruction
causing the fault; this is something that a core dump provides, unless
a signal handler interferes with that.

In many crashes I fixed, the Python stack trace was irrelevant to
understand the nature of the crash. Of course, there were also counter-
examples where you'ld need the Python stack (or, rather, the entire
interpreter trace) to understand how the crashing case can occur in
the first place.

Regards,
Martin

From martin at v.loewis.de  Tue Dec 21 00:05:31 2010
From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=)
Date: Tue, 21 Dec 2010 00:05:31 +0100
Subject: [Python-Dev] Issue #8863 adds a new
 PYTHONNOFAULTHANDLER	environment variable
In-Reply-To: <201012201524.23023.victor.stinner@haypocalc.com>
References: <4D0C0688.1080108@haypocalc.com>
	<1292820829.4763.16.camel@marge>	<4D0F0440.5070806@v.loewis.de>
	<201012201524.23023.victor.stinner@haypocalc.com>
Message-ID: <4D0FE13B.2020003@v.loewis.de>

>> The GIL is likely held by a different thread, then.
>> _PyThreadState_Current will point to the state of this other thread.
> 
> I tested when the GIL released: the fault handler is unable to retrieve "the" 
> thread state and so the backtrace is not printed. Which thread state should be 
> retrieve? I don't know yet: maybe the last active thread state?
> 
> It doesn't look trivial to fix this issue because if there is no thread (only 
> the main thread), release the GIL clears the thread state pointer (to it to 
> NULL) without storing it somewhere else (whereas with multiple threads, the 
> last active thread is stored in "gil_last_holder". I should ask Antoine (or 
> someone one) how the new GIL works.

I don't think this is possible to fix. Just put a sleep() call before
the crash, and have a parallel thread perform a while-loop to see what
I mean: it should give you a stack trace of the wrong thread, possibly
touching transient objects which go away while you inspect them.

> Another variable might be created to store the (pointer to the) last active 
> thread state.

Please understand: the last active thread may not be the one that got
the signal. Instead, it may be another thread that keeps running while
your thread crashes.

Regards,
Martin

From ncoghlan at gmail.com  Tue Dec 21 02:38:21 2010
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Tue, 21 Dec 2010 12:38:21 +1100
Subject: [Python-Dev] Search-friendly shortcuts for Windows?
Message-ID: <AANLkTimF0jpWon-03bnR7qV31rtURBvxbGY5HpWRrwt8@mail.gmail.com>

Something I noticed after installing the 3.2 beta on my Windows 7
laptop is that the start menu shortcuts aren't particularly search
friendly. Searching loses the heirarchical information, so attempting
to directly locate "pyt" provides two separate "Python Command Line"
shortcuts, with no indicator as to which one will start which version.

Given the changing dynamics of the desktop launch menus to better
support direct access as an alternative to hierarchical navigation,
would it be reasonable to consider including the major version number
in the start menu shortcut names?

(Question is mainly for Martin, obviously, but I'm also curious if
anyone else has encountered the same thing)

Regards,
Nick.

-- 
Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia

From ncoghlan at gmail.com  Tue Dec 21 02:57:50 2010
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Tue, 21 Dec 2010 12:57:50 +1100
Subject: [Python-Dev] r87389 - in python/branches/py3k:
 Doc/library/unittest.rst Lib/unittest/case.py Misc/NEWS
In-Reply-To: <1292855497.3737.55.camel@localhost.localdomain>
References: <20101218200004.99110EE985@mail.python.org>
	<20101218214830.1aa8be32@pitrou.net> <iejdjt$ik0$1@dough.gmane.org>
	<AANLkTi=ADkUMc5tQGDv2yniteyfPCPXU=XaCr+GscR+a@mail.gmail.com>
	<20101219141355.5ac7cd2f@pitrou.net>
	<AANLkTinyvb6+Xf=owdV22j6YoAEVjbTXV2tetQzdD8r4@mail.gmail.com>
	<31185718-F276-46CE-82E2-6C6B6EC920E8@gmail.com>
	<4D0F536A.8050808@voidspace.org.uk>
	<1292852841.3737.25.camel@localhost.localdomain>
	<4D0F622E.7050902@voidspace.org.uk>
	<1292855497.3737.55.camel@localhost.localdomain>
Message-ID: <AANLkTi=pr227YH_8vdLBFASHYCRMN6RAxrdoF8f6kqkx@mail.gmail.com>

On Tue, Dec 21, 2010 at 1:31 AM, Antoine Pitrou <solipsis at pitrou.net> wrote:
>> Diffing is completely an implementation detail of how the failure
>> messages are generated. The important thing is that failure messages
>> make sense with respect to actual result and expected result.
>
> Which, again, they don't. Let's see:
>
> ? ?self.assertEqual(actual, expected)
> AssertionError: 'a\nb\nc\ne\n' != 'a\nb\nc\nd\n'
> ?a
> ?b
> ?c
> - e
> + d
>
> The diff shows "expected - actual", but it would be logical (in your own
> logic) to display "actual - expected". The whole issue disappears if you
> drop this idea of naming the arguments "actual" and "expected".

To make this a bit clearer...

>>> class Ex(ut.TestCase):
...   def demo(self):
...     self.assertEqual("actual", "expected")
...
>>> Ex("demo").demo()
Traceback (most recent call last):
  <snip TB details>
AssertionError: 'actual' != 'expected'
- actual
+ expected

For the actual/expected terminology the diff is the wrong way around
(as of 3.2b1, anyway).

My own +1 goes to keeping the actual/expected terminology (and
ordering) and adjusting the diffs accordingly (with a header noting
that the diff is old=expected, new=actual).

assertRaises() *is* an exception to the general actual/expected
pattern, but that asymmetry is forced by the ability to pass arbitrary
positional arguments to the function being tested (which later proved
useful for the context manager form as well).

Cheers,
Nick.

-- 
Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia

From tjreedy at udel.edu  Tue Dec 21 06:45:07 2010
From: tjreedy at udel.edu (Terry Reedy)
Date: Tue, 21 Dec 2010 00:45:07 -0500
Subject: [Python-Dev] Search-friendly shortcuts for Windows?
In-Reply-To: <AANLkTimF0jpWon-03bnR7qV31rtURBvxbGY5HpWRrwt8@mail.gmail.com>
References: <AANLkTimF0jpWon-03bnR7qV31rtURBvxbGY5HpWRrwt8@mail.gmail.com>
Message-ID: <iepet0$tpi$1@dough.gmane.org>

On 12/20/2010 8:38 PM, Nick Coghlan wrote:

> Given the changing dynamics of the desktop launch menus to better
> support direct access as an alternative to hierarchical navigation,
> would it be reasonable to consider including the major version number
> in the start menu shortcut names?

I would very much like that. I once changed some the entries but bugfix 
or later alpha/beta/gamma/final installation reset everything.

> (Question is mainly for Martin, obviously, but I'm also curious if
> anyone else has encountered the same thing)

Now that I have multiple versions of Python installed (for development 
testing), I have *three* identical 'IDLE (Python GUI)' entries in the 
left-hand frequently-used programs column of the Start panel. This is 
really useless and therefore a waste of space.

-- 
Terry Jan Reedy


From scott+python-dev at scottdial.com  Tue Dec 21 06:47:09 2010
From: scott+python-dev at scottdial.com (Scott Dial)
Date: Tue, 21 Dec 2010 00:47:09 -0500
Subject: [Python-Dev] Search-friendly shortcuts for Windows?
In-Reply-To: <AANLkTimF0jpWon-03bnR7qV31rtURBvxbGY5HpWRrwt8@mail.gmail.com>
References: <AANLkTimF0jpWon-03bnR7qV31rtURBvxbGY5HpWRrwt8@mail.gmail.com>
Message-ID: <4D103F5D.7060300@scottdial.com>

On 12/20/2010 8:38 PM, Nick Coghlan wrote:
> would it be reasonable to consider including the major version number
> in the start menu shortcut names?

+1 First thing I did was add an x.y prefix to the Python shortcuts.

There are a lot of application shortcuts on my Win7 system that have
version numbers that have no naming conflicts, so even if a single
version of Python was installed, it would not look out of place at all.

-- 
Scott Dial
scott at scottdial.com
scodial at cs.indiana.edu

From martin at v.loewis.de  Tue Dec 21 08:36:43 2010
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Tue, 21 Dec 2010 08:36:43 +0100
Subject: [Python-Dev] Search-friendly shortcuts for Windows?
In-Reply-To: <AANLkTimF0jpWon-03bnR7qV31rtURBvxbGY5HpWRrwt8@mail.gmail.com>
References: <AANLkTimF0jpWon-03bnR7qV31rtURBvxbGY5HpWRrwt8@mail.gmail.com>
Message-ID: <4D10590B.9040208@v.loewis.de>

> Given the changing dynamics of the desktop launch menus to better
> support direct access as an alternative to hierarchical navigation,
> would it be reasonable to consider including the major version number
> in the start menu shortcut names?
> 
> (Question is mainly for Martin, obviously, but I'm also curious if
> anyone else has encountered the same thing)

I can't see anything wrong with that, but I'm terrible with wording.
So somebody would have to spell out the exact wording that each of
the start menu items should have. In doing so, also take into
consideration that some people install 32-bit and 64-bit versions
simultaneously.

Regards,
Martin

From ncoghlan at gmail.com  Tue Dec 21 13:01:00 2010
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Tue, 21 Dec 2010 23:01:00 +1100
Subject: [Python-Dev] Search-friendly shortcuts for Windows?
In-Reply-To: <4D10590B.9040208@v.loewis.de>
References: <AANLkTimF0jpWon-03bnR7qV31rtURBvxbGY5HpWRrwt8@mail.gmail.com>
	<4D10590B.9040208@v.loewis.de>
Message-ID: <AANLkTi=0m2KXhYP0A=emrCg0zw3ALBYb9K2b5QDuB8T3@mail.gmail.com>

On Tue, Dec 21, 2010 at 6:36 PM, "Martin v. L?wis" <martin at v.loewis.de> wrote:
>> Given the changing dynamics of the desktop launch menus to better
>> support direct access as an alternative to hierarchical navigation,
>> would it be reasonable to consider including the major version number
>> in the start menu shortcut names?
>>
>> (Question is mainly for Martin, obviously, but I'm also curious if
>> anyone else has encountered the same thing)
>
> I can't see anything wrong with that, but I'm terrible with wording.
> So somebody would have to spell out the exact wording that each of
> the start menu items should have. In doing so, also take into
> consideration that some people install 32-bit and 64-bit versions
> simultaneously.

I put my initial proposal (reproduced below) at
http://bugs.python.org/issue10747. I'll let this thread run a bit and
see if there are any substantial improvements suggested, or valid
objections raised (I tried to pre-empt the length issue by pushing the
typically least important information to the end). I also considered
using the x86 and x64 abbreviations, but being explicit seemed
clearer.

Shortcuts currently installed:
- Python (command line)
- Python Manuals
- Module Docs
- IDLE (Python GUI)
- Uninstall Python

Initial proposal for 32 bit builds:
- Python 3.2 (command line - 32 bit)
- Python 3.2 Manuals
- Python 3.2 Docs Server (pydoc - 32 bit)
- IDLE (Python 3.2 GUI - 32 bit)
- Uninstall Python 3.2 (32 bit)

Initial proposal for 64 bit builds:
- Python 3.2 (command line - 64 bit)
- Python 3.2 Manuals
- Python 3.2 Docs Server (pydoc - 64 bit)
- IDLE (Python 3.2 GUI - 64 bit)
- Uninstall Python 3.2 (64 bit)

Cheers,
Nick.

-- 
Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia

From fuzzyman at voidspace.org.uk  Tue Dec 21 13:17:46 2010
From: fuzzyman at voidspace.org.uk (Michael Foord)
Date: Tue, 21 Dec 2010 12:17:46 +0000
Subject: [Python-Dev] r87389 - in python/branches/py3k:
 Doc/library/unittest.rst Lib/unittest/case.py Misc/NEWS
In-Reply-To: <AANLkTi=pr227YH_8vdLBFASHYCRMN6RAxrdoF8f6kqkx@mail.gmail.com>
References: <20101218200004.99110EE985@mail.python.org>	<20101218214830.1aa8be32@pitrou.net>
	<iejdjt$ik0$1@dough.gmane.org>	<AANLkTi=ADkUMc5tQGDv2yniteyfPCPXU=XaCr+GscR+a@mail.gmail.com>	<20101219141355.5ac7cd2f@pitrou.net>	<AANLkTinyvb6+Xf=owdV22j6YoAEVjbTXV2tetQzdD8r4@mail.gmail.com>	<31185718-F276-46CE-82E2-6C6B6EC920E8@gmail.com>	<4D0F536A.8050808@voidspace.org.uk>	<1292852841.3737.25.camel@localhost.localdomain>	<4D0F622E.7050902@voidspace.org.uk>	<1292855497.3737.55.camel@localhost.localdomain>
	<AANLkTi=pr227YH_8vdLBFASHYCRMN6RAxrdoF8f6kqkx@mail.gmail.com>
Message-ID: <4D109AEA.7010908@voidspace.org.uk>

On 21/12/2010 01:57, Nick Coghlan wrote:
> On Tue, Dec 21, 2010 at 1:31 AM, Antoine Pitrou<solipsis at pitrou.net>  wrote:
>>> Diffing is completely an implementation detail of how the failure
>>> messages are generated. The important thing is that failure messages
>>> make sense with respect to actual result and expected result.
>> Which, again, they don't. Let's see:
>>
>>     self.assertEqual(actual, expected)
>> AssertionError: 'a\nb\nc\ne\n' != 'a\nb\nc\nd\n'
>>   a
>>   b
>>   c
>> - e
>> + d
>>
>> The diff shows "expected - actual", but it would be logical (in your own
>> logic) to display "actual - expected". The whole issue disappears if you
>> drop this idea of naming the arguments "actual" and "expected".
> To make this a bit clearer...
>
>>>> class Ex(ut.TestCase):
> ...   def demo(self):
> ...     self.assertEqual("actual", "expected")
> ...
>>>> Ex("demo").demo()
> Traceback (most recent call last):
>    <snip TB details>
> AssertionError: 'actual' != 'expected'
> - actual
> + expected
>
> For the actual/expected terminology the diff is the wrong way around
> (as of 3.2b1, anyway).
>

The recent commit that sparked the controversy was supposed to ensure 
that all the asserts were documented consistently *and* worked as per 
the documentation. The error above is from assertMultiLineEqual.

assertListEqual has the same issue:

 >>> t.assertListEqual([1], [2])
Traceback (most recent call last):
   ...
AssertionError: Lists differ: [1] != [2]

First differing element 0:
1
2

- [1]
+ [2]

Interestingly assertSetEqual already uses the first/second symmetric 
wording:

 >>> t.assertSetEqual({1}, {2})
   ...
AssertionError: Items in the first set but not the second:
1
Items in the second set but not the first:
2


> My own +1 goes to keeping the actual/expected terminology (and
> ordering) and adjusting the diffs accordingly (with a header noting
> that the diff is old=expected, new=actual).
>

Well we don't have consensus. Whatever we do we need to be consistent, 
and in the absence of an agreement about a change we should at least 
make all the behaviour and documentation consistent.

 From this discussion and the discussion on the issue tracker:

Myself, Nick Coghlan and Ezio Melotti prefer (actual, expected)
Raymond like (actual, expected) but would be happy with symmetrical diffs
Guido prefers the (actual, expected) ordering but prefers diffs to show 
the other way round
R David Murray agreed with Guido
Terry Reedy liked the change
Glenn Linderman wants (actual, expected) and diffing to follow that
Ron Adam ditto

Symmetrical diffs (element in first not in second, element in second not 
in first) solves the problem without imposing an order on the arguments. 
Actually unittest *has* used (first, second) to refer to the arguments 
to asserts pretty much since its inception. Losing the (actual, 
expected) terminology is a cost of this but unittest hasn't emphasised 
this terminology in the past (as I thought it had).

This won't work for diffing strings (assertMultiLineEqual) which use 
difflib and needs a direction for the diff. As above it is currently the 
wrong way round for (actual, expected).

The other alternative is to make them consistent and follow Nick's 
suggestion adding the header note to the diffs that old=expected, 
new=actual.

> assertRaises() *is* an exception to the general actual/expected
> pattern, but that asymmetry is forced by the ability to pass arbitrary
> positional arguments to the function being tested (which later proved
> useful for the context manager form as well).
The (actual, expected) pattern matches the way almost everyone I've ever 
seen write if statements and asserts:

     if x == 5: rather than if 5 == x:
     assert x == 5 rather than assert 5 == x

It also matches functions like isinstance and issubclass.

On the other hand it doesn't match the way we report TypeErrors where we 
report "expected <some type>, got <other type>".

All the best,

Michael Foord
> Cheers,
> Nick.
>


-- 

http://www.voidspace.org.uk/

May you do good and not evil
May you find forgiveness for yourself and forgive others
May you share freely, never taking more than you give.
-- the sqlite blessing http://www.sqlite.org/different.html


From georg at python.org  Tue Dec 21 20:18:56 2010
From: georg at python.org (Georg Brandl)
Date: Tue, 21 Dec 2010 20:18:56 +0100
Subject: [Python-Dev] [RELEASED] Python 3.2 beta 2
Message-ID: <4D10FDA0.7090808@python.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On behalf of the Python development team, I'm happy to announce the
second beta preview release of Python 3.2.

Python 3.2 is a continuation of the efforts to improve and stabilize the
Python 3.x line.  Since the final release of Python 2.7, the 2.x line
will only receive bugfixes, and new features are developed for 3.x only.

Since PEP 3003, the Moratorium on Language Changes, is in effect, there
are no changes in Python's syntax and built-in types in Python 3.2.
Development efforts concentrated on the standard library and support for
porting code to Python 3.  Highlights are:

* numerous improvements to the unittest module
* PEP 3147, support for .pyc repository directories
* PEP 3149, support for version tagged dynamic libraries
* PEP 3148, a new futures library for concurrent programming
* PEP 384, a stable ABI for extension modules
* PEP 391, dictionary-based logging configuration
* an overhauled GIL implementation that reduces contention
* an extended email package that handles bytes messages
* countless fixes regarding bytes/string issues; among them full
  support for a bytes environment (filenames, environment variables)
* many consistency and behavior fixes for numeric operations
* a sysconfig module to access configuration information
* a pure-Python implementation of the datetime module
* additions to the shutil module, among them archive file support
* improvements to pdb, the Python debugger

For a more extensive list of changes in 3.2, see

    http://docs.python.org/3.2/whatsnew/3.2.html

To download Python 3.2 visit:

    http://www.python.org/download/releases/3.2/

Please consider trying Python 3.2 with your code and reporting any bugs
you may notice to:

    http://bugs.python.org/


Enjoy!

- -- 
Georg Brandl, Release Manager
georg at python.org
(on behalf of the entire python-dev team and 3.2's contributors)

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.11 (GNU/Linux)

iEYEARECAAYFAk0Q/aAACgkQN9GcIYhpnLDf8gCgkLGAsE+T3R505jZc1RxXDYsa
NSsAnRGaFjeTm9o2Z5O8FuIzTUG8t1PT
=hHzz
-----END PGP SIGNATURE-----

From v+python at g.nevcal.com  Tue Dec 21 21:27:50 2010
From: v+python at g.nevcal.com (Glenn Linderman)
Date: Tue, 21 Dec 2010 12:27:50 -0800
Subject: [Python-Dev] r87389 - in python/branches/py3k:
 Doc/library/unittest.rst Lib/unittest/case.py Misc/NEWS
In-Reply-To: <4D109AEA.7010908@voidspace.org.uk>
References: <20101218200004.99110EE985@mail.python.org>	<20101218214830.1aa8be32@pitrou.net>	<iejdjt$ik0$1@dough.gmane.org>	<AANLkTi=ADkUMc5tQGDv2yniteyfPCPXU=XaCr+GscR+a@mail.gmail.com>	<20101219141355.5ac7cd2f@pitrou.net>	<AANLkTinyvb6+Xf=owdV22j6YoAEVjbTXV2tetQzdD8r4@mail.gmail.com>	<31185718-F276-46CE-82E2-6C6B6EC920E8@gmail.com>	<4D0F536A.8050808@voidspace.org.uk>	<1292852841.3737.25.camel@localhost.localdomain>	<4D0F622E.7050902@voidspace.org.uk>	<1292855497.3737.55.camel@localhost.localdomain>	<AANLkTi=pr227YH_8vdLBFASHYCRMN6RAxrdoF8f6kqkx@mail.gmail.com>
	<4D109AEA.7010908@voidspace.org.uk>
Message-ID: <4D110DC6.6060905@g.nevcal.com>

On 12/21/2010 4:17 AM, Michael Foord wrote:
> Glenn Linderman wants (actual, expected) and diffing to follow that

If you say that is what I said, fine.  I might not have understood the 
example well enough to say the right thing.  I liked Nick's explanation, 
using the actual and expected words in his example, but

-expected
+actual
-old
+new

is what I would expect in the diff.  I didn't say anything about the 
parameters, I don't care, except that the documentation leads me to use 
it correctly, so that I get the above diff results.

Sadly, if the diff results are not as above, I would probably misuse the 
parameters to achieve it, unless doing something for which the standard 
is to do it "backwards".

From guido at python.org  Tue Dec 21 21:40:16 2010
From: guido at python.org (Guido van Rossum)
Date: Tue, 21 Dec 2010 12:40:16 -0800
Subject: [Python-Dev] r87389 - in python/branches/py3k:
 Doc/library/unittest.rst Lib/unittest/case.py Misc/NEWS
In-Reply-To: <4D109AEA.7010908@voidspace.org.uk>
References: <20101218200004.99110EE985@mail.python.org>
	<20101218214830.1aa8be32@pitrou.net> <iejdjt$ik0$1@dough.gmane.org>
	<AANLkTi=ADkUMc5tQGDv2yniteyfPCPXU=XaCr+GscR+a@mail.gmail.com>
	<20101219141355.5ac7cd2f@pitrou.net>
	<AANLkTinyvb6+Xf=owdV22j6YoAEVjbTXV2tetQzdD8r4@mail.gmail.com>
	<31185718-F276-46CE-82E2-6C6B6EC920E8@gmail.com>
	<4D0F536A.8050808@voidspace.org.uk>
	<1292852841.3737.25.camel@localhost.localdomain>
	<4D0F622E.7050902@voidspace.org.uk>
	<1292855497.3737.55.camel@localhost.localdomain>
	<AANLkTi=pr227YH_8vdLBFASHYCRMN6RAxrdoF8f6kqkx@mail.gmail.com>
	<4D109AEA.7010908@voidspace.org.uk>
Message-ID: <AANLkTikCskX5QqfRoXNKEe-jb5vimD7G9UDCKWMs_19N@mail.gmail.com>

On Tue, Dec 21, 2010 at 4:17 AM, Michael Foord
<fuzzyman at voidspace.org.uk> wrote:
> On 21/12/2010 01:57, Nick Coghlan wrote:
>>
>> On Tue, Dec 21, 2010 at 1:31 AM, Antoine Pitrou<solipsis at pitrou.net>
>> ?wrote:
>>>>
>>>> Diffing is completely an implementation detail of how the failure
>>>> messages are generated. The important thing is that failure messages
>>>> make sense with respect to actual result and expected result.
>>>
>>> Which, again, they don't. Let's see:
>>>
>>> ? ?self.assertEqual(actual, expected)
>>> AssertionError: 'a\nb\nc\ne\n' != 'a\nb\nc\nd\n'
>>> ?a
>>> ?b
>>> ?c
>>> - e
>>> + d
>>>
>>> The diff shows "expected - actual", but it would be logical (in your own
>>> logic) to display "actual - expected". The whole issue disappears if you
>>> drop this idea of naming the arguments "actual" and "expected".
>>
>> To make this a bit clearer...
>>
>>>>> class Ex(ut.TestCase):
>>
>> ... ? def demo(self):
>> ... ? ? self.assertEqual("actual", "expected")
>> ...
>>>>>
>>>>> Ex("demo").demo()
>>
>> Traceback (most recent call last):
>> ? <snip TB details>
>> AssertionError: 'actual' != 'expected'
>> - actual
>> + expected
>>
>> For the actual/expected terminology the diff is the wrong way around
>> (as of 3.2b1, anyway).
>>
>
> The recent commit that sparked the controversy was supposed to ensure that
> all the asserts were documented consistently *and* worked as per the
> documentation. The error above is from assertMultiLineEqual.
>
> assertListEqual has the same issue:
>
>>>> t.assertListEqual([1], [2])
> Traceback (most recent call last):
> ?...
> AssertionError: Lists differ: [1] != [2]
>
> First differing element 0:
> 1
> 2
>
> - [1]
> + [2]
>
> Interestingly assertSetEqual already uses the first/second symmetric
> wording:
>
>>>> t.assertSetEqual({1}, {2})
> ?...
> AssertionError: Items in the first set but not the second:
> 1
> Items in the second set but not the first:
> 2
>
>
>> My own +1 goes to keeping the actual/expected terminology (and
>> ordering) and adjusting the diffs accordingly (with a header noting
>> that the diff is old=expected, new=actual).
>>
>
> Well we don't have consensus. Whatever we do we need to be consistent, and
> in the absence of an agreement about a change we should at least make all
> the behaviour and documentation consistent.
>
> From this discussion and the discussion on the issue tracker:
>
> Myself, Nick Coghlan and Ezio Melotti prefer (actual, expected)
> Raymond like (actual, expected) but would be happy with symmetrical diffs
> Guido prefers the (actual, expected) ordering but prefers diffs to show the
> other way round

Actually I said there was no right answer.

I certainly do not want the diff output to treat the second arg as
"old" and the first one as "new" -- that would be just as confusing.

All in all I'd like to get rid of any vestiges of actual and expected;
I think the first/second wording is the best we can come up with it.

> R David Murray agreed with Guido

That's hard to believe since I don't agree with myself. :-)

> Terry Reedy liked the change
> Glenn Linderman wants (actual, expected) and diffing to follow that
> Ron Adam ditto
>
> Symmetrical diffs (element in first not in second, element in second not in
> first) solves the problem without imposing an order on the arguments.
> Actually unittest *has* used (first, second) to refer to the arguments to
> asserts pretty much since its inception. Losing the (actual, expected)
> terminology is a cost of this but unittest hasn't emphasised this
> terminology in the past (as I thought it had).
>
> This won't work for diffing strings (assertMultiLineEqual) which use difflib
> and needs a direction for the diff. As above it is currently the wrong way
> round for (actual, expected).
>
> The other alternative is to make them consistent and follow Nick's
> suggestion adding the header note to the diffs that old=expected,
> new=actual.
>
>> assertRaises() *is* an exception to the general actual/expected
>> pattern, but that asymmetry is forced by the ability to pass arbitrary
>> positional arguments to the function being tested (which later proved
>> useful for the context manager form as well).
>
> The (actual, expected) pattern matches the way almost everyone I've ever
> seen write if statements and asserts:
>
> ? ?if x == 5: rather than if 5 == x:
> ? ?assert x == 5 rather than assert 5 == x
>
> It also matches functions like isinstance and issubclass.
>
> On the other hand it doesn't match the way we report TypeErrors where we
> report "expected <some type>, got <other type>".
>
> All the best,
>
> Michael Foord
>>
>> Cheers,
>> Nick.
>>
>
>
> --
>
> http://www.voidspace.org.uk/
>
> May you do good and not evil
> May you find forgiveness for yourself and forgive others
> May you share freely, never taking more than you give.
> -- the sqlite blessing http://www.sqlite.org/different.html
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> http://mail.python.org/mailman/options/python-dev/guido%40python.org
>



-- 
--Guido van Rossum (python.org/~guido)

From ncoghlan at gmail.com  Wed Dec 22 02:12:29 2010
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Wed, 22 Dec 2010 12:12:29 +1100
Subject: [Python-Dev] r87389 - in python/branches/py3k:
 Doc/library/unittest.rst Lib/unittest/case.py Misc/NEWS
In-Reply-To: <4D109AEA.7010908@voidspace.org.uk>
References: <20101218200004.99110EE985@mail.python.org>
	<20101218214830.1aa8be32@pitrou.net> <iejdjt$ik0$1@dough.gmane.org>
	<AANLkTi=ADkUMc5tQGDv2yniteyfPCPXU=XaCr+GscR+a@mail.gmail.com>
	<20101219141355.5ac7cd2f@pitrou.net>
	<AANLkTinyvb6+Xf=owdV22j6YoAEVjbTXV2tetQzdD8r4@mail.gmail.com>
	<31185718-F276-46CE-82E2-6C6B6EC920E8@gmail.com>
	<4D0F536A.8050808@voidspace.org.uk>
	<1292852841.3737.25.camel@localhost.localdomain>
	<4D0F622E.7050902@voidspace.org.uk>
	<1292855497.3737.55.camel@localhost.localdomain>
	<AANLkTi=pr227YH_8vdLBFASHYCRMN6RAxrdoF8f6kqkx@mail.gmail.com>
	<4D109AEA.7010908@voidspace.org.uk>
Message-ID: <AANLkTimnGh2p+ucr88odGVXwmQ46DuKb5-y2WR2xHPGJ@mail.gmail.com>

On Tue, Dec 21, 2010 at 11:17 PM, Michael Foord
<fuzzyman at voidspace.org.uk> wrote:
> On 21/12/2010 01:57, Nick Coghlan wrote:
>> My own +1 goes to keeping the actual/expected terminology (and
>> ordering) and adjusting the diffs accordingly (with a header noting
>> that the diff is old=expected, new=actual).
>>
>
> Well we don't have consensus. Whatever we do we need to be consistent, and
> in the absence of an agreement about a change we should at least make all
> the behaviour and documentation consistent.
>
> From this discussion and the discussion on the issue tracker:
>
> Myself, Nick Coghlan and Ezio Melotti prefer (actual, expected)
> Raymond like (actual, expected) but would be happy with symmetrical diffs
> Guido prefers the (actual, expected) ordering but prefers diffs to show the
> other way round
> R David Murray agreed with Guido
> Terry Reedy liked the change
> Glenn Linderman wants (actual, expected) and diffing to follow that
> Ron Adam ditto
>
> Symmetrical diffs (element in first not in second, element in second not in
> first) solves the problem without imposing an order on the arguments.
> Actually unittest *has* used (first, second) to refer to the arguments to
> asserts pretty much since its inception. Losing the (actual, expected)
> terminology is a cost of this but unittest hasn't emphasised this
> terminology in the past (as I thought it had).

I actually agree with Guido that anything we do is going to be
suboptimal in some way. Encouraging the actual/expected ordering and
updating the diff output so "expected=old" strikes me as least bad,
but using the neutral first/second terminology and doing the diffs as
"first=old" wouldn't be terrible (although I'm personally -0 on it
because it encourages putting the expected value first in order to get
the diffs the right way around when an error occurs).

Cheers,
Nick.

-- 
Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia

From ncoghlan at gmail.com  Wed Dec 22 02:15:34 2010
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Wed, 22 Dec 2010 12:15:34 +1100
Subject: [Python-Dev] [python-committers] [RELEASED] Python 3.2 beta 2
In-Reply-To: <4D10FDA0.7090808@python.org>
References: <4D10FDA0.7090808@python.org>
Message-ID: <AANLkTin-oLhRK7+mbnk7pdyJh9zsZSAm7-V+JjvyBTAd@mail.gmail.com>

On Wed, Dec 22, 2010 at 6:18 AM, Georg Brandl <georg at python.org> wrote:
> Since PEP 3003, the Moratorium on Language Changes, is in effect, there
> are no changes in Python's syntax and built-in types in Python 3.2.

Minor nit - we actually did tweak a few of the builtin types a bit
(mostly the stuff to improve Sequence ABC conformance and to make
range objects more list-like)

Cheers,
Nick.

-- 
Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia

From guido at python.org  Wed Dec 22 02:37:05 2010
From: guido at python.org (Guido van Rossum)
Date: Tue, 21 Dec 2010 17:37:05 -0800
Subject: [Python-Dev] r87389 - in python/branches/py3k:
 Doc/library/unittest.rst Lib/unittest/case.py Misc/NEWS
In-Reply-To: <AANLkTimnGh2p+ucr88odGVXwmQ46DuKb5-y2WR2xHPGJ@mail.gmail.com>
References: <20101218200004.99110EE985@mail.python.org>
	<20101218214830.1aa8be32@pitrou.net> <iejdjt$ik0$1@dough.gmane.org>
	<AANLkTi=ADkUMc5tQGDv2yniteyfPCPXU=XaCr+GscR+a@mail.gmail.com>
	<20101219141355.5ac7cd2f@pitrou.net>
	<AANLkTinyvb6+Xf=owdV22j6YoAEVjbTXV2tetQzdD8r4@mail.gmail.com>
	<31185718-F276-46CE-82E2-6C6B6EC920E8@gmail.com>
	<4D0F536A.8050808@voidspace.org.uk>
	<1292852841.3737.25.camel@localhost.localdomain>
	<4D0F622E.7050902@voidspace.org.uk>
	<1292855497.3737.55.camel@localhost.localdomain>
	<AANLkTi=pr227YH_8vdLBFASHYCRMN6RAxrdoF8f6kqkx@mail.gmail.com>
	<4D109AEA.7010908@voidspace.org.uk>
	<AANLkTimnGh2p+ucr88odGVXwmQ46DuKb5-y2WR2xHPGJ@mail.gmail.com>
Message-ID: <AANLkTimF=YA6m-tqZGvDJu5M1U33NpKqo1MbjEqmd+of@mail.gmail.com>

On Tue, Dec 21, 2010 at 5:12 PM, Nick Coghlan <ncoghlan at gmail.com> wrote:
> I actually agree with Guido that anything we do is going to be
> suboptimal in some way. Encouraging the actual/expected ordering and
> updating the diff output so "expected=old" strikes me as least bad,
> but using the neutral first/second terminology and doing the diffs as
> "first=old" wouldn't be terrible (although I'm personally -0 on it
> because it encourages putting the expected value first in order to get
> the diffs the right way around when an error occurs).

There are several problems with the actual/expected terminology. First
of all, the arguments are primarily thought of as (and look)
positional, not as having names (and there's nothing you can do about
this in the docs -- people copy code without reading docs).
Furthermore, Java's jUnit puts expected first (and makes this part of
the culture/religion), so people coming from there will use that order
and be freaked out if you were to swap them. And last, the order of
diff arguments (old new) is also ingrained in the culture (which
actually matches the expected/actual order in my mind).

I think the least bad thing would be to drop any remnants of
expected/actual terminology, keep the diffs in the first-second order,
and let developers choose whether they put the expected value first or
second. Then of course there will still be the examples in the doc
(which some people *do* read and copy) -- I suppose we could alternate
here to emphasize that we don't have a preferred order.

-- 
--Guido van Rossum (python.org/~guido)

From tjreedy at udel.edu  Wed Dec 22 03:26:54 2010
From: tjreedy at udel.edu (Terry Reedy)
Date: Tue, 21 Dec 2010 21:26:54 -0500
Subject: [Python-Dev] r87389 - in python/branches/py3k:
 Doc/library/unittest.rst Lib/unittest/case.py Misc/NEWS
In-Reply-To: <4D109AEA.7010908@voidspace.org.uk>
References: <20101218200004.99110EE985@mail.python.org>	<20101218214830.1aa8be32@pitrou.net>	<iejdjt$ik0$1@dough.gmane.org>	<AANLkTi=ADkUMc5tQGDv2yniteyfPCPXU=XaCr+GscR+a@mail.gmail.com>	<20101219141355.5ac7cd2f@pitrou.net>	<AANLkTinyvb6+Xf=owdV22j6YoAEVjbTXV2tetQzdD8r4@mail.gmail.com>	<31185718-F276-46CE-82E2-6C6B6EC920E8@gmail.com>	<4D0F536A.8050808@voidspace.org.uk>	<1292852841.3737.25.camel@localhost.localdomain>	<4D0F622E.7050902@voidspace.org.uk>	<1292855497.3737.55.camel@localhost.localdomain>	<AANLkTi=pr227YH_8vdLBFASHYCRMN6RAxrdoF8f6kqkx@mail.gmail.com>
	<4D109AEA.7010908@voidspace.org.uk>
Message-ID: <iernlg$cib$1@dough.gmane.org>

On 12/21/2010 7:17 AM, Michael Foord wrote:

My first priority is that doc and code match.
Close second is consistency (hence, ease of learning and use) between 
various AssertXs.

> Symmetrical diffs (element in first not in second, element in second not
> in first) solves the problem without imposing an order on the arguments.

Where applicable, I prefer this as unambiguous output headings.

-- 
Terry Jan Reedy


From tjreedy at udel.edu  Wed Dec 22 03:39:05 2010
From: tjreedy at udel.edu (Terry Reedy)
Date: Tue, 21 Dec 2010 21:39:05 -0500
Subject: [Python-Dev] Search-friendly shortcuts for Windows?
In-Reply-To: <AANLkTi=0m2KXhYP0A=emrCg0zw3ALBYb9K2b5QDuB8T3@mail.gmail.com>
References: <AANLkTimF0jpWon-03bnR7qV31rtURBvxbGY5HpWRrwt8@mail.gmail.com>	<4D10590B.9040208@v.loewis.de>
	<AANLkTi=0m2KXhYP0A=emrCg0zw3ALBYb9K2b5QDuB8T3@mail.gmail.com>
Message-ID: <ieroca$eq5$1@dough.gmane.org>

On 12/21/2010 7:01 AM, Nick Coghlan wrote:
> On Tue, Dec 21, 2010 at 6:36 PM, "Martin v. L?wis"<martin at v.loewis.de>  wrote:
>>> Given the changing dynamics of the desktop launch menus to better
>>> support direct access as an alternative to hierarchical navigation,
>>> would it be reasonable to consider including the major version number
>>> in the start menu shortcut names?
>>>
>>> (Question is mainly for Martin, obviously, but I'm also curious if
>>> anyone else has encountered the same thing)
>>
>> I can't see anything wrong with that, but I'm terrible with wording.
>> So somebody would have to spell out the exact wording that each of
>> the start menu items should have. In doing so, also take into
>> consideration that some people install 32-bit and 64-bit versions
>> simultaneously.
>
> I put my initial proposal (reproduced below) at
> http://bugs.python.org/issue10747. I'll let this thread run a bit and
> see if there are any substantial improvements suggested, or valid
> objections raised (I tried to pre-empt the length issue by pushing the
> typically least important information to the end). I also considered
> using the x86 and x64 abbreviations, but being explicit seemed
> clearer.
>
> Shortcuts currently installed:
> - Python (command line)
> - Python Manuals
> - Module Docs
> - IDLE (Python GUI)
> - Uninstall Python
>
> Initial proposal for 32 bit builds:
> - Python 3.2 (command line - 32 bit)
> - Python 3.2 Manuals
> - Python 3.2 Docs Server (pydoc - 32 bit)
     Python 3.2 PyDoc Server (32 bit)
is shorter. But since pydoc.py is has no 'bitness',

     PyDoc Server (Python 3.2 - 32 bit)
is more in line with below

> - IDLE (Python 3.2 GUI - 32 bit)

I think of IDLE as an IDE, not a GUI, but IDLE IDE does not work too 
well. In any case, it also has no 'bitness'.

     IDLE GUI (Python 3.2 - 32 bit)

-- 
Terry Jan Reedy



From tjreedy at udel.edu  Wed Dec 22 03:16:28 2010
From: tjreedy at udel.edu (Terry Reedy)
Date: Tue, 21 Dec 2010 21:16:28 -0500
Subject: [Python-Dev] [Python-checkins] r87433
	-	python/branches/py3k/Doc/reference/lexical_analysis.rst
In-Reply-To: <20101222013736.8A9F7EE9A2@mail.python.org>
References: <20101222013736.8A9F7EE9A2@mail.python.org>
Message-ID: <4D115F7C.1020002@udel.edu>



On 12/21/2010 8:37 PM, alexander.belopolsky wrote:
> Author: alexander.belopolsky
> Date: Wed Dec 22 02:37:36 2010
> New Revision: 87433
>
> Log:
> Both PEP 3131 and the current implementation use NFKC normalization
> for identifiers.  Fixed the documentation to agree.
>
>
> Modified:
>     python/branches/py3k/Doc/reference/lexical_analysis.rst
>
> Modified: python/branches/py3k/Doc/reference/lexical_analysis.rst
> ==============================================================================
> --- python/branches/py3k/Doc/reference/lexical_analysis.rst	(original)
> +++ python/branches/py3k/Doc/reference/lexical_analysis.rst	Wed Dec 22 02:37:36 2010
> @@ -309,8 +309,8 @@
>   * *Nd* - decimal numbers
>   * *Pc* - connector punctuations
>
> -All identifiers are converted into the normal form NFC while parsing; comparison
> -of identifiers is based on NFC.
> +All identifiers are converted into the normal form NFKC while parsing; comparison
> +of identifiers is based on NFKC.
>
>   A non-normative HTML file listing all valid identifier characters for Unicode
>   4.1 can be found at

Has that file been updated for Unicode 6.0?

From alexander.belopolsky at gmail.com  Wed Dec 22 04:31:02 2010
From: alexander.belopolsky at gmail.com (Alexander Belopolsky)
Date: Tue, 21 Dec 2010 22:31:02 -0500
Subject: [Python-Dev] [Python-checkins] r87433 -
	python/branches/py3k/Doc/reference/lexical_analysis.rst
In-Reply-To: <4D115F7C.1020002@udel.edu>
References: <20101222013736.8A9F7EE9A2@mail.python.org>
	<4D115F7C.1020002@udel.edu>
Message-ID: <AANLkTikgdfPXS1dR=-0uHSy=9UbBKbyeh8DU46mvyaUw@mail.gmail.com>

On Tue, Dec 21, 2010 at 9:16 PM, Terry Reedy <tjreedy at udel.edu> wrote:
..
>> ?A non-normative HTML file listing all valid identifier characters for
>> Unicode
>> ?4.1 can be found at http://www.dcl.hpi.uni-potsdam.de/home/loewis/table-3131.html
>
> Has that file been updated for Unicode 6.0?

Apparently not. In Python 3.2:

>>> '\N{KANNADA SIGN JIHVAMULIYA}'.isidentifier()
True
$ curl -s  http://www.dcl.hpi.uni-potsdam.de/home/loewis/table-3131.html|
grep JIHVAMULIYA
$

From ncoghlan at gmail.com  Wed Dec 22 07:21:05 2010
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Wed, 22 Dec 2010 17:21:05 +1100
Subject: [Python-Dev] Search-friendly shortcuts for Windows?
In-Reply-To: <ieroca$eq5$1@dough.gmane.org>
References: <AANLkTimF0jpWon-03bnR7qV31rtURBvxbGY5HpWRrwt8@mail.gmail.com>
	<4D10590B.9040208@v.loewis.de>
	<AANLkTi=0m2KXhYP0A=emrCg0zw3ALBYb9K2b5QDuB8T3@mail.gmail.com>
	<ieroca$eq5$1@dough.gmane.org>
Message-ID: <AANLkTimWm8AsuumixBaBc7OFXqYk4kjkacayTuTuCMwz@mail.gmail.com>

On Wed, Dec 22, 2010 at 1:39 PM, Terry Reedy <tjreedy at udel.edu> wrote:
> ? ?Python 3.2 PyDoc Server (32 bit)
> is shorter. But since pydoc.py is has no 'bitness',
>
> ? ?PyDoc Server (Python 3.2 - 32 bit)
> is more in line with below
>
>> - IDLE (Python 3.2 GUI - 32 bit)
>
> I think of IDLE as an IDE, not a GUI, but IDLE IDE does not work too well.
> In any case, it also has no 'bitness'.
>
> ? ?IDLE GUI (Python 3.2 - 32 bit)

The IDLE and Pydoc links implicitly launch an interpreter though -
that interpreter definitely has a defined pointer size.

Cheers,
Nick.

-- 
Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia

From adal.chiriliuc at gmail.com  Wed Dec 22 07:23:39 2010
From: adal.chiriliuc at gmail.com (Adal Chiriliuc)
Date: Wed, 22 Dec 2010 08:23:39 +0200
Subject: [Python-Dev] Search-friendly shortcuts for Windows?
In-Reply-To: <AANLkTi=0m2KXhYP0A=emrCg0zw3ALBYb9K2b5QDuB8T3@mail.gmail.com>
References: <AANLkTimF0jpWon-03bnR7qV31rtURBvxbGY5HpWRrwt8@mail.gmail.com>
	<4D10590B.9040208@v.loewis.de>
	<AANLkTi=0m2KXhYP0A=emrCg0zw3ALBYb9K2b5QDuB8T3@mail.gmail.com>
Message-ID: <AANLkTinuVGAjTOdg6CNQEZ=xAm5y_1aH5o=FGHK+_80d@mail.gmail.com>

Microsoft recommendations:

Avoid putting a version number in a program name unless that is how
users normally refer to your program.

Put only program shortcuts on the Start menu. Don't put shortcuts to
the following items on the Start menu:
- Program uninstallers. Users access uninstallers through the Programs
control panel item.
- Help files. Users access Help topics directly from your program.

http://msdn.microsoft.com/en-us/library/aa511447.aspx

I think it's fine for Python to use a version number and to put links
to the manual in the Start Menu (since Python doesn't have a Help
menu). But the uninstall shortcut should go.

I also don't think that "command line" should be used in the title.

And why is the .CHM file called "Manuals"? Why the plural? When seeing
this title I thought that maybe it's a link to the Doc directory. Most
of the .CHM files that I have in my Start Menu are called "abc
Documentation", and a couple "abc Users Manual". The help file it's
also titled "Python v2.7 documentation" when opened, so maybe it
should be renamed to "Python 3.2 Documentation".

> Initial proposal for 32 bit builds:
> - Python 3.2 (command line - 32 bit)
> - Python 3.2 Manuals
> - Python 3.2 Docs Server (pydoc - 32 bit)
> - IDLE (Python 3.2 GUI - 32 bit)
> - Uninstall Python 3.2 (32 bit)
>
> Initial proposal for 64 bit builds:
> - Python 3.2 (command line - 64 bit)
> - Python 3.2 Manuals
> - Python 3.2 Docs Server (pydoc - 64 bit)
> - IDLE (Python 3.2 GUI - 64 bit)
> - Uninstall Python 3.2 (64 bit)

From eliben at gmail.com  Wed Dec 22 07:40:36 2010
From: eliben at gmail.com (Eli Bendersky)
Date: Wed, 22 Dec 2010 08:40:36 +0200
Subject: [Python-Dev] Search-friendly shortcuts for Windows?
In-Reply-To: <AANLkTi=0m2KXhYP0A=emrCg0zw3ALBYb9K2b5QDuB8T3@mail.gmail.com>
References: <AANLkTimF0jpWon-03bnR7qV31rtURBvxbGY5HpWRrwt8@mail.gmail.com>
	<4D10590B.9040208@v.loewis.de>
	<AANLkTi=0m2KXhYP0A=emrCg0zw3ALBYb9K2b5QDuB8T3@mail.gmail.com>
Message-ID: <AANLkTi=WFCwTE7u1a1pUvgQwFR-X6eeQ5MgJhX=os_Q4@mail.gmail.com>

> I put my initial proposal (reproduced below) at
> http://bugs.python.org/issue10747. I'll let this thread run a bit and
> see if there are any substantial improvements suggested, or valid
> objections raised (I tried to pre-empt the length issue by pushing the
> typically least important information to the end). I also considered
> using the x86 and x64 abbreviations, but being explicit seemed
> clearer.
>
>
In general, definite +1 on the idea of having version numbers in there. A
small comment: "command line" doesn't sound too helpful. Wouldn't
"interactive shell" (as they do in ActiveState's Python) be better?

Eli
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20101222/9e3a18a3/attachment.html>

From tjreedy at udel.edu  Wed Dec 22 07:49:14 2010
From: tjreedy at udel.edu (Terry Reedy)
Date: Wed, 22 Dec 2010 01:49:14 -0500
Subject: [Python-Dev] Search-friendly shortcuts for Windows?
In-Reply-To: <AANLkTinuVGAjTOdg6CNQEZ=xAm5y_1aH5o=FGHK+_80d@mail.gmail.com>
References: <AANLkTimF0jpWon-03bnR7qV31rtURBvxbGY5HpWRrwt8@mail.gmail.com>	<4D10590B.9040208@v.loewis.de>	<AANLkTi=0m2KXhYP0A=emrCg0zw3ALBYb9K2b5QDuB8T3@mail.gmail.com>
	<AANLkTinuVGAjTOdg6CNQEZ=xAm5y_1aH5o=FGHK+_80d@mail.gmail.com>
Message-ID: <ies71c$uof$1@dough.gmane.org>

On 12/22/2010 1:23 AM, Adal Chiriliuc wrote:
> Microsoft recommendations:

Irrelevant.
>
> Avoid putting a version number in a program name unless that is how
> users normally refer to your program.

Version numbers are the point of this issue, because people *do* have 
multiple version installed.

> Put only program shortcuts on the Start menu. Don't put shortcuts to
> the following items on the Start menu:
> - Program uninstallers. Users access uninstallers through the Programs
> control panel item.

Many programs put them there where they are EASY to access and invoke. 
Control panel and add/remove programs are absolute slugs when it comes 
to starting up.

> - Help files. Users access Help topics directly from your program.

Have you ever run the other items? IDLE has its own help. Control panel 
has no menu, and what help it does have is for its configuration dialog.

\
> http://msdn.microsoft.com/en-us/library/aa511447.aspx
>
> I think it's fine for Python to use a version number and to put links
> to the manual in the Start Menu (since Python doesn't have a Help
> menu). But the uninstall shortcut should go.
>
> I also don't think that "command line" should be used in the title.
>
> And why is the .CHM file called "Manuals"? Why the plural?

Because there are several.

  When seeing
> this title I thought that maybe it's a link to the Doc directory. Most
> of the .CHM files that I have in my Start Menu are called "abc
> Documentation", and a couple "abc Users Manual". The help file it's
> also titled "Python v2.7 documentation" when opened, so maybe it

That is only true of the 2.7 doc.

> should be renamed to "Python 3.2 Documentation".

The 3.2 doc already is, at least on my system.

>> Initial proposal for 32 bit builds:
>> - Python 3.2 (command line - 32 bit)
>> - Python 3.2 Manuals
>> - Python 3.2 Docs Server (pydoc - 32 bit)
>> - IDLE (Python 3.2 GUI - 32 bit)
>> - Uninstall Python 3.2 (32 bit)
>>
>> Initial proposal for 64 bit builds:
>> - Python 3.2 (command line - 64 bit)
>> - Python 3.2 Manuals
>> - Python 3.2 Docs Server (pydoc - 64 bit)
>> - IDLE (Python 3.2 GUI - 64 bit)
>> - Uninstall Python 3.2 (64 bit)


-- 
Terry Jan Reedy


From tjreedy at udel.edu  Wed Dec 22 07:50:43 2010
From: tjreedy at udel.edu (Terry Reedy)
Date: Wed, 22 Dec 2010 01:50:43 -0500
Subject: [Python-Dev] Search-friendly shortcuts for Windows?
In-Reply-To: <AANLkTimWm8AsuumixBaBc7OFXqYk4kjkacayTuTuCMwz@mail.gmail.com>
References: <AANLkTimF0jpWon-03bnR7qV31rtURBvxbGY5HpWRrwt8@mail.gmail.com>	<4D10590B.9040208@v.loewis.de>	<AANLkTi=0m2KXhYP0A=emrCg0zw3ALBYb9K2b5QDuB8T3@mail.gmail.com>	<ieroca$eq5$1@dough.gmane.org>
	<AANLkTimWm8AsuumixBaBc7OFXqYk4kjkacayTuTuCMwz@mail.gmail.com>
Message-ID: <ies743$uof$2@dough.gmane.org>

On 12/22/2010 1:21 AM, Nick Coghlan wrote:
> On Wed, Dec 22, 2010 at 1:39 PM, Terry Reedy<tjreedy at udel.edu>  wrote:
>>     Python 3.2 PyDoc Server (32 bit)
>> is shorter. But since pydoc.py is has no 'bitness',
>>
>>     PyDoc Server (Python 3.2 - 32 bit)
>> is more in line with below
>>
>>> - IDLE (Python 3.2 GUI - 32 bit)
>>
>> I think of IDLE as an IDE, not a GUI, but IDLE IDE does not work too well.
>> In any case, it also has no 'bitness'.
>>
>>     IDLE GUI (Python 3.2 - 32 bit)
>
> The IDLE and Pydoc links implicitly launch an interpreter though -
> that interpreter definitely has a defined pointer size.

That is exactly why I left the identity of the interpreter in parens. It 
is Python3.2 that is 32 or 64 bit, not IDLE or pydocs, so the modifier 
should be attached to Python 3.2 and not IDLE or pydocs.

-- 
Terry Jan Reedy


From phd at phd.pp.ru  Wed Dec 22 09:07:41 2010
From: phd at phd.pp.ru (Oleg Broytman)
Date: Wed, 22 Dec 2010 11:07:41 +0300
Subject: [Python-Dev] phd.pp.ru => phdru.name
Message-ID: <20101222080741.GA4130@phd.pp.ru>

Hello! I need to inform people I'm changing my online identity. Domain
phd.pp.ru will die Dec 27 (I'll try to reregister it to extend its life
for a few months). My new personal domain will be phdru.name, primary
email will be "phd" (in case one makes a mistake and write "phdru" two
times - the address is an alias for "phd").

The new domain is already delegated, email and site works. I'm working
on changing all my email subscriptions.

Oleg.
-- 
     Oleg Broytman            http://phd.pp.ru/            phd at phd.pp.ru
           Programmers don't die, they just GOSUB without RETURN.

From kristjan at ccpgames.com  Wed Dec 22 14:58:17 2010
From: kristjan at ccpgames.com (=?iso-8859-1?Q?Kristj=E1n_Valur_J=F3nsson?=)
Date: Wed, 22 Dec 2010 21:58:17 +0800
Subject: [Python-Dev] issue 10296
Message-ID: <2E034B571A5CE44E949B9FCC3B6D24EE5E5AE047@exchcn.ccp.ad.local>

I have this defect here http://bugs.python.org/issue10296
ctypes catches BreakPoint error on windows 32
that I posted a while back and had no response to.  This is ctypes.  Would anyone care to take a look?
K
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20101222/387e24bc/attachment.html>

From eric at trueblade.com  Wed Dec 22 14:09:46 2010
From: eric at trueblade.com (Eric Smith)
Date: Wed, 22 Dec 2010 08:09:46 -0500
Subject: [Python-Dev] [python-committers] [RELEASED] Python 3.2 beta 2
In-Reply-To: <iess0a$q23$1@dough.gmane.org>
References: <4D10FDA0.7090808@python.org>	<AANLkTin-oLhRK7+mbnk7pdyJh9zsZSAm7-V+JjvyBTAd@mail.gmail.com>
	<iess0a$q23$1@dough.gmane.org>
Message-ID: <4D11F89A.3080103@trueblade.com>

On 12/22/2010 8:46 AM, Georg Brandl wrote:
> Am 22.12.2010 02:15, schrieb Nick Coghlan:
>> On Wed, Dec 22, 2010 at 6:18 AM, Georg Brandl<georg at python.org>  wrote:
>>> Since PEP 3003, the Moratorium on Language Changes, is in effect, there
>>> are no changes in Python's syntax and built-in types in Python 3.2.
>>
>> Minor nit - we actually did tweak a few of the builtin types a bit
>> (mostly the stuff to improve Sequence ABC conformance and to make
>> range objects more list-like)
>
> Indeed, I'll fix this wording for the next announcement.  (And I will
> mention SSL, sorry Antoine).

If you're only going to mention some vague "some builtins had minor 
changes", then I'm fine with that. If you're going to enumerate all such 
changes, that will be a bigger job. There were 2 such changes I'm aware 
of: str.format_map (#6081) and the addition of alternate ("#") 
formatting to float, complex and decimal (#7094) __format__ methods.

For this announcement I don't think it's necessary to list them all.


From lvh at laurensvh.be  Wed Dec 22 23:11:57 2010
From: lvh at laurensvh.be (Laurens Van Houtven)
Date: Wed, 22 Dec 2010 23:11:57 +0100
Subject: [Python-Dev] nonlocal x = value
In-Reply-To: <iei8i9$qb5$2@dough.gmane.org>
References: <ief2tb$i9n$1@dough.gmane.org>
	<AANLkTikZi3DEqx7-QgR4wZ337E-OC_Xk7E9NZaqw1jaf@mail.gmail.com>
	<iei8i9$qb5$2@dough.gmane.org>
Message-ID: <AANLkTinmYt6or0aOz4bP1UE05T6EXuWyz7PeHcGUr0fT@mail.gmail.com>

On Sat, Dec 18, 2010 at 1:12 PM, Georg Brandl <g.brandl at gmx.net> wrote:
> Am 17.12.2010 17:52, schrieb Laurens Van Houtven:
>> +1 for throwing it out of the PEP. Assignment is a thing,
>> nonlocal/global is a thing, don't mix them up :) (That in addition to
>> the grammar cleanliness argument Stephan already made)
>
> The trouble is what to make of
>
> nonlocal x = 3, y
>
> Is it two nonlocal declarations or one with a tuple assignment?
>
> Georg

I'm not sure I understand. Isn't that another reason to throw it out?
If you don't allow such assignments, there can't be any ambiguity,
right? (Or am I missing something?)

lvh

From tjreedy at udel.edu  Thu Dec 23 00:41:08 2010
From: tjreedy at udel.edu (Terry Reedy)
Date: Wed, 22 Dec 2010 18:41:08 -0500
Subject: [Python-Dev] nonlocal x = value
In-Reply-To: <AANLkTinmYt6or0aOz4bP1UE05T6EXuWyz7PeHcGUr0fT@mail.gmail.com>
References: <ief2tb$i9n$1@dough.gmane.org>	<AANLkTikZi3DEqx7-QgR4wZ337E-OC_Xk7E9NZaqw1jaf@mail.gmail.com>	<iei8i9$qb5$2@dough.gmane.org>
	<AANLkTinmYt6or0aOz4bP1UE05T6EXuWyz7PeHcGUr0fT@mail.gmail.com>
Message-ID: <ieu2aj$jf0$1@dough.gmane.org>

On 12/22/2010 5:11 PM, Laurens Van Houtven wrote:
> On Sat, Dec 18, 2010 at 1:12 PM, Georg Brandl<g.brandl at gmx.net>  wrote:
>> Am 17.12.2010 17:52, schrieb Laurens Van Houtven:
>>> +1 for throwing it out of the PEP. Assignment is a thing,
>>> nonlocal/global is a thing, don't mix them up :) (That in addition to
>>> the grammar cleanliness argument Stephan already made)
>>
>> The trouble is what to make of
>>
>> nonlocal x = 3, y
>>
>> Is it two nonlocal declarations or one with a tuple assignment?
>>
>> Georg
>
> I'm not sure I understand. Isn't that another reason to throw it out?

I am sure he meant it to be, and I agree. We lived without
global x = 3, y
all these years.

> If you don't allow such assignments, there can't be any ambiguity,
> right? (Or am I missing something?)


-- 
Terry Jan Reedy


From victor.stinner at haypocalc.com  Thu Dec 23 03:08:13 2010
From: victor.stinner at haypocalc.com (Victor Stinner)
Date: Thu, 23 Dec 2010 03:08:13 +0100
Subject: [Python-Dev] Fault handler updated, now disabled by default
Message-ID: <1293070093.12866.14.camel@marge>

Hi,

Thanks to all your useful remarks, I improved my patch twice (version 10
and 11). I think that the last version (11) addresses all reported
issues. The most notable change is that the fault handler is now
disabled by default. It fixes also the function getting the thread
state, it now works in all cases (especially if the thread causing the
fault doesn't hold the GIL).

Summary of the patch (#8863):

 - Add an optional fault handler for SIGSEGV, SIGFPE, SIGILL and SIGBUS
signals displaying the Python backtrace
 - Setting PYTHONFAULTHANDLER environment variable, or using
   "-X faulthandler" command line option, enables it
 - If it's enabled, display also the Python backtrace on a fatal error

Use case: when a program crashs, the user reruns its application with
the fault handler enabled and tries to reproduce the crash. He/She can
send the Python backtrace to the developer, or use it directly (if
he/she understands it).

I think that this patch helps to isolate crashs in Python or libraries,
especially if the user doesn't have all tools needed to get a Python
backtrace (on a crash) and the developer is unable to reproduce the bug
(because it doesn't have the same OS or libraries). For example, gdb 7
and python-gdb.py are rare on Windows :-) (even on Linux!)

So, do you agree with the fault handler? Does someone want to give a
last review because I commit it?

http://bugs.python.org/issue8863

Victor


From vstinner at edenwall.com  Thu Dec 23 03:26:35 2010
From: vstinner at edenwall.com (Victor Stinner)
Date: Thu, 23 Dec 2010 03:26:35 +0100
Subject: [Python-Dev] Issue #8863 adds a
 new?PYTHONNOFAULTHANDLER?environment variable
In-Reply-To: <AANLkTik+tH6UndVMwQGiwj0+2sFpHT379j4ZBpH_BEDR@mail.gmail.com>
References: <4D0C0688.1080108@haypocalc.com>
	<20101220165632.GA19009@yoda.bytereef.org>
	<AANLkTimecoqHTiUPexqDeeFjA2QWpf+Hnz+9brR2BDfO@mail.gmail.com>
	<201012201851.09810.vstinner@edenwall.com>
	<AANLkTik+tH6UndVMwQGiwj0+2sFpHT379j4ZBpH_BEDR@mail.gmail.com>
Message-ID: <1293071195.12866.32.camel@marge>

Le lundi 20 d?cembre 2010 ? 13:09 -0500, Alexander Belopolsky a ?crit :
> What if the program crashes (again) in the signal handler before
> having a chance to restore registers?

If the fault handler crashs, the registers are not restored.

> Also, can you point to documentation that describes register restoring behavior?

I found an article telling that the kernel saves integer and float
registers when calling a signal handler, but I am unable to find it
again, sorry. And I cannot find an article telling the opposite.

Anyway, it would just be horrible if a signal handler was responsible to
save/restore integer registers. If the kernel doesn't do that for us, I
suppose that the compiler or something else does it for us.

Eg. signal_default_int_handler(), the Python handler of SIGINT signal,
doesn't save/restore registers, even if it erases many of them.

> Because I had actual experience with that using R.  Your code may be
> better, but R brings up a dialog on segfault asking ...

You cannot compare a simple handler using simple signal-safe functions
like write(), and a complex handler calling graphical functions. It is
not safe to call graphical functions in a signal handler.

> (I actually use RPy which loads R into  Python as a shared library, 
> so the two handlers will conflict anyways.)

It depends if R handler calls the previous handler. My fault handler is
"chained" with the previous handler (it calls indirectly the previous
handler). If R fault handler does the same, all handlers will be
executed correctly (I don't think that the execution order matter).

Victor


From martin at v.loewis.de  Thu Dec 23 03:37:21 2010
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Thu, 23 Dec 2010 03:37:21 +0100
Subject: [Python-Dev] Fault handler updated, now disabled by default
In-Reply-To: <1293070093.12866.14.camel@marge>
References: <1293070093.12866.14.camel@marge>
Message-ID: <4D12B5E1.7060608@v.loewis.de>

> So, do you agree with the fault handler? Does someone want to give a
> last review because I commit it?

It's a new feature, so regardless of whether it's correct or not
(which I haven't reviewed yet), I don't think it should go in before
3.2 is released.

Regards,
Martin

From simon at brunningonline.net  Thu Dec 23 10:40:12 2010
From: simon at brunningonline.net (Simon Brunning)
Date: Thu, 23 Dec 2010 09:40:12 +0000
Subject: [Python-Dev] r87389 - in python/branches/py3k:
 Doc/library/unittest.rst Lib/unittest/case.py Misc/NEWS
In-Reply-To: <AANLkTimF=YA6m-tqZGvDJu5M1U33NpKqo1MbjEqmd+of@mail.gmail.com>
References: <20101218200004.99110EE985@mail.python.org>
	<20101218214830.1aa8be32@pitrou.net> <iejdjt$ik0$1@dough.gmane.org>
	<AANLkTi=ADkUMc5tQGDv2yniteyfPCPXU=XaCr+GscR+a@mail.gmail.com>
	<20101219141355.5ac7cd2f@pitrou.net>
	<AANLkTinyvb6+Xf=owdV22j6YoAEVjbTXV2tetQzdD8r4@mail.gmail.com>
	<31185718-F276-46CE-82E2-6C6B6EC920E8@gmail.com>
	<4D0F536A.8050808@voidspace.org.uk>
	<1292852841.3737.25.camel@localhost.localdomain>
	<4D0F622E.7050902@voidspace.org.uk>
	<1292855497.3737.55.camel@localhost.localdomain>
	<AANLkTi=pr227YH_8vdLBFASHYCRMN6RAxrdoF8f6kqkx@mail.gmail.com>
	<4D109AEA.7010908@voidspace.org.uk>
	<AANLkTimnGh2p+ucr88odGVXwmQ46DuKb5-y2WR2xHPGJ@mail.gmail.com>
	<AANLkTimF=YA6m-tqZGvDJu5M1U33NpKqo1MbjEqmd+of@mail.gmail.com>
Message-ID: <AANLkTikm+V-H2EmHXYbQxxAhnAd6egHtWA+dCGx-V_tY@mail.gmail.com>

On 22 December 2010 01:37, Guido van Rossum <guido at python.org> wrote:
> Furthermore, Java's jUnit puts expected first (and makes this part of
> the culture/religion), so people coming from there will use that order
> and be freaked out if you were to swap them. And last, the order of
> diff arguments (old new) is also ingrained in the culture (which
> actually matches the expected/actual order in my mind).

For what it's worth, none of the (numerous) Java projects that I've
worked on over the last couple of years have used bare JUnit.
Assertions are often done using the Hamcrest library -
<http://code.google.com/p/hamcrest/>. The actual value comes first,
and this combined with the use of matchers makes the failure messages
very descriptive.

I'd hate to go back to one of those projects where the tests tell you
that two values don't match, but doesn?t tell me which is the expected
value. There is a Python port -
<http://code.google.com/p/hamcrest/wiki/TutorialPython>. I must get
around to trying that.

-- 
Cheers,
Simon B.

From orsenthil at gmail.com  Thu Dec 23 11:04:43 2010
From: orsenthil at gmail.com (Senthil Kumaran)
Date: Thu, 23 Dec 2010 18:04:43 +0800
Subject: [Python-Dev] [Python-checkins] r87270 -
 python/branches/py3k/Doc/library/urllib.request.rst
In-Reply-To: <20101215190726.2FFE2EE98A@mail.python.org>
References: <20101215190726.2FFE2EE98A@mail.python.org>
Message-ID: <20101223100443.GC1690@rubuntu>

On Wed, Dec 15, 2010 at 08:07:26PM +0100, antoine.pitrou wrote:
> Log:
> Move the urllib-inherited API to a distinguished section

Distinguished: Legacy. :)

-- 
Senthil

From victor.stinner at haypocalc.com  Thu Dec 23 18:20:42 2010
From: victor.stinner at haypocalc.com (Victor Stinner)
Date: Thu, 23 Dec 2010 18:20:42 +0100
Subject: [Python-Dev] Fault handler updated, now disabled by default
In-Reply-To: <4D12B5E1.7060608@v.loewis.de>
References: <1293070093.12866.14.camel@marge> <4D12B5E1.7060608@v.loewis.de>
Message-ID: <1293124842.26635.3.camel@marge>

Le jeudi 23 d?cembre 2010 ? 03:37 +0100, "Martin v. L?wis" a ?crit :
> > So, do you agree with the fault handler? Does someone want to give a
> > last review because I commit it?
> 
> It's a new feature, so regardless of whether it's correct or not
> (which I haven't reviewed yet), I don't think it should go in before
> 3.2 is released.

I tested my patch on Windows (XP), Ubuntu (10.4) and FreeBSD (8) and it
works correctly: all tests pass and the system fault handler (Windows
popup, Apport and core dump) is also called.

The fault handler is disabled by default and it is clearly separated
(eg. it doesn't touch the API of a module). Can't you make an exception
for this new feature?

The first answer to my previous email thread was Georg, answer starting
with "I very much like having a traceback on (some) segmentation
faults" :-)

Victor


From martin at v.loewis.de  Thu Dec 23 19:23:17 2010
From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=)
Date: Thu, 23 Dec 2010 19:23:17 +0100
Subject: [Python-Dev] Fault handler updated, now disabled by default
In-Reply-To: <1293124842.26635.3.camel@marge>
References: <1293070093.12866.14.camel@marge> <4D12B5E1.7060608@v.loewis.de>
	<1293124842.26635.3.camel@marge>
Message-ID: <4D139395.3000709@v.loewis.de>

> The fault handler is disabled by default and it is clearly separated
> (eg. it doesn't touch the API of a module). Can't you make an exception
> for this new feature?

Ultimately, it's for the release manager to decide, so I don't need to
make an exception. However, I think that special cases aren't special
enough to break the rules. I still wish that the beta releases had been
deferred until after the Mercurial switchover, but alas, the release
manager has decided otherwise - explicitly pointing out that the
rationale for releasing beta 1 was to stop accepting new features.

The motivation for freezing features before feature releases is not
so much that existing applications may be affected by API changes
(3.2 will change APIs, anyway, and users fully expect to need some
porting), but that the features that are about to be released need
some time for testing. This specific feature has seen very little
testing. Giving it a full release cycle (i.e. until 3.3) would
somewhat reduce the need for a more careful code review.

Regards,
Martin

From bobbyi at gmail.com  Thu Dec 23 19:29:17 2010
From: bobbyi at gmail.com (Bobby Impollonia)
Date: Thu, 23 Dec 2010 13:29:17 -0500
Subject: [Python-Dev] Issue #8863 adds a
 new?PYTHONNOFAULTHANDLER?environment variable
In-Reply-To: <1293071195.12866.32.camel@marge>
References: <4D0C0688.1080108@haypocalc.com>
	<20101220165632.GA19009@yoda.bytereef.org>
	<AANLkTimecoqHTiUPexqDeeFjA2QWpf+Hnz+9brR2BDfO@mail.gmail.com>
	<201012201851.09810.vstinner@edenwall.com>
	<AANLkTik+tH6UndVMwQGiwj0+2sFpHT379j4ZBpH_BEDR@mail.gmail.com>
	<1293071195.12866.32.camel@marge>
Message-ID: <AANLkTi=zc2yi5vxt2ixxUFP-=-A8tAsxAkuf4BHxPvJX@mail.gmail.com>

On Wed, Dec 22, 2010 at 9:26 PM, Victor Stinner <vstinner at edenwall.com> wrote:
> If the kernel doesn't do that for us, I
> suppose that the compiler or something else does it for us.

GCC does this for you if you declare your function with
__attribute__(signal). In general, the compiler doesn't know that a
function will be used as a signal and no portable way to indicate
this.

> Anyway, it would just be horrible if a signal handler was responsible to
> save/restore integer registers.

Horrible or not, the existence of __attribute__(signal) seems to
indicate that this is the case on some platform, or at least was
historically. I don't know whether it's the case now on any platform
that Python supports.

From martin at v.loewis.de  Thu Dec 23 19:57:21 2010
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Thu, 23 Dec 2010 19:57:21 +0100
Subject: [Python-Dev] Issue #8863 adds a
 new?PYTHONNOFAULTHANDLER?environment variable
In-Reply-To: <AANLkTi=zc2yi5vxt2ixxUFP-=-A8tAsxAkuf4BHxPvJX@mail.gmail.com>
References: <4D0C0688.1080108@haypocalc.com>	<20101220165632.GA19009@yoda.bytereef.org>	<AANLkTimecoqHTiUPexqDeeFjA2QWpf+Hnz+9brR2BDfO@mail.gmail.com>	<201012201851.09810.vstinner@edenwall.com>	<AANLkTik+tH6UndVMwQGiwj0+2sFpHT379j4ZBpH_BEDR@mail.gmail.com>	<1293071195.12866.32.camel@marge>
	<AANLkTi=zc2yi5vxt2ixxUFP-=-A8tAsxAkuf4BHxPvJX@mail.gmail.com>
Message-ID: <4D139B91.3000501@v.loewis.de>

> Horrible or not, the existence of __attribute__(signal) seems to
> indicate that this is the case on some platform, or at least was
> historically.

The signal attribute has an effect only on ATMEL AVR processors,
according to the documentation (and according to my source grep).
On other processors, gcc typically implements
__attribute__((interrupt)), meaning that the function is an interrupt
handler - something that you cannot express in regular C. This may
involve re-enabling interrupts on entry, but may also involve other
code generation aspects (such as using iret instead of ret on x86).

For AVR, there was apparently some compatibility issue, introducing
the need for two different ways to declare an ISR:
__attribute__((interrupt)) causes interrupts to be re-enabled on
entry (using the "sei" instruction); __attribute__((signal)) keeps
them disabled.

So this has nothing to do with Unix signals.

Regards,
Martin

From solipsis at pitrou.net  Thu Dec 23 20:55:28 2010
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Thu, 23 Dec 2010 20:55:28 +0100
Subject: [Python-Dev] r87445 - python/branches/py3k/Lib/numbers.py
References: <20101223184133.9DED1EE995@mail.python.org>
Message-ID: <20101223205528.1c365abd@pitrou.net>

On Thu, 23 Dec 2010 19:41:33 +0100 (CET)
eric.araujo <python-checkins at python.org> wrote:
>  
>      def __index__(self):
> -        """index(self)"""
> +        """someobject[self]"""

This is misleading as to what the method actually does, as you can read
in the implementation:

>          return int(self)




From g.brandl at gmx.net  Thu Dec 23 21:51:54 2010
From: g.brandl at gmx.net (Georg Brandl)
Date: Thu, 23 Dec 2010 21:51:54 +0100
Subject: [Python-Dev] nonlocal x = value
In-Reply-To: <AANLkTinmYt6or0aOz4bP1UE05T6EXuWyz7PeHcGUr0fT@mail.gmail.com>
References: <ief2tb$i9n$1@dough.gmane.org>	<AANLkTikZi3DEqx7-QgR4wZ337E-OC_Xk7E9NZaqw1jaf@mail.gmail.com>	<iei8i9$qb5$2@dough.gmane.org>
	<AANLkTinmYt6or0aOz4bP1UE05T6EXuWyz7PeHcGUr0fT@mail.gmail.com>
Message-ID: <if0cru$2e3$1@dough.gmane.org>

Am 22.12.2010 23:11, schrieb Laurens Van Houtven:
> On Sat, Dec 18, 2010 at 1:12 PM, Georg Brandl <g.brandl at gmx.net> wrote:
>> Am 17.12.2010 17:52, schrieb Laurens Van Houtven:
>>> +1 for throwing it out of the PEP. Assignment is a thing,
>>> nonlocal/global is a thing, don't mix them up :) (That in addition to
>>> the grammar cleanliness argument Stephan already made)
>>
>> The trouble is what to make of
>>
>> nonlocal x = 3, y
>>
>> Is it two nonlocal declarations or one with a tuple assignment?
>>
>> Georg
> 
> I'm not sure I understand. Isn't that another reason to throw it out?
> If you don't allow such assignments, there can't be any ambiguity,
> right? (Or am I missing something?)

Yes and no -- there may not be an ambiguity to the parser, but still to
the human.  Except if you disallow the syntax in any case, requiring
people to write

nonlocal x = (3, y)

which is then again inconsistent with ordinary assignment statements.

Georg


From g.brandl at gmx.net  Thu Dec 23 21:59:40 2010
From: g.brandl at gmx.net (Georg Brandl)
Date: Thu, 23 Dec 2010 21:59:40 +0100
Subject: [Python-Dev] Fault handler updated, now disabled by default
In-Reply-To: <4D139395.3000709@v.loewis.de>
References: <1293070093.12866.14.camel@marge>
	<4D12B5E1.7060608@v.loewis.de>	<1293124842.26635.3.camel@marge>
	<4D139395.3000709@v.loewis.de>
Message-ID: <if0dag$5a0$1@dough.gmane.org>

Am 23.12.2010 19:23, schrieb "Martin v. L?wis":
>> The fault handler is disabled by default and it is clearly separated
>> (eg. it doesn't touch the API of a module). Can't you make an exception
>> for this new feature?
> 
> Ultimately, it's for the release manager to decide, so I don't need to
> make an exception. However, I think that special cases aren't special
> enough to break the rules. I still wish that the beta releases had been
> deferred until after the Mercurial switchover, but alas, the release
> manager has decided otherwise - explicitly pointing out that the
> rationale for releasing beta 1 was to stop accepting new features.

The main rationale was that the Mercurial switchover is not a fixed target
and depends very much on volunteer effort, so in effect this would have
been akin to deferring 3.2 for an indefinite period.

> The motivation for freezing features before feature releases is not
> so much that existing applications may be affected by API changes
> (3.2 will change APIs, anyway, and users fully expect to need some
> porting), but that the features that are about to be released need
> some time for testing. This specific feature has seen very little
> testing. Giving it a full release cycle (i.e. until 3.3) would
> somewhat reduce the need for a more careful code review.

Exactly.  I did say I like the feature, but that was a) before beta 2
was released, now the next release is a release candidate, and b) this
thread showed that it is not at all obvious how the feature should look
like.  The fact that it isn't enabled by default also makes it seem less
attractive to me, but I understand the reasons why it shouldn't be on by
default.  Therefore, I'm not willing to make an exception here.

Georg


From lvh at laurensvh.be  Thu Dec 23 22:03:49 2010
From: lvh at laurensvh.be (Laurens Van Houtven)
Date: Thu, 23 Dec 2010 22:03:49 +0100
Subject: [Python-Dev] nonlocal x = value
In-Reply-To: <if0cru$2e3$1@dough.gmane.org>
References: <ief2tb$i9n$1@dough.gmane.org>
	<AANLkTikZi3DEqx7-QgR4wZ337E-OC_Xk7E9NZaqw1jaf@mail.gmail.com>
	<iei8i9$qb5$2@dough.gmane.org>
	<AANLkTinmYt6or0aOz4bP1UE05T6EXuWyz7PeHcGUr0fT@mail.gmail.com>
	<if0cru$2e3$1@dough.gmane.org>
Message-ID: <AANLkTik_D7oZkXTdZNB2=w=wN2gZeO17VTtQRZuNwsKM@mail.gmail.com>

On Thu, Dec 23, 2010 at 9:51 PM, Georg Brandl <g.brandl at gmx.net> wrote:
> Yes and no -- there may not be an ambiguity to the parser, but still to
> the human. ?Except if you disallow the syntax in any case, requiring
> people to write
>
> nonlocal x = (3, y)
>
> which is then again inconsistent with ordinary assignment statements.
>
> Georg

Right -- but (and hence the confusion) I was arguing for not mixing
global/nonlocal with assignment at all, and instead having nonlocal
and global only take one or more names. That would (obviously) remove
any such ambiguity ;-)

cheers
lvh

From merwok at netwok.org  Thu Dec 23 22:09:15 2010
From: merwok at netwok.org (=?UTF-8?B?w4lyaWMgQXJhdWpv?=)
Date: Thu, 23 Dec 2010 22:09:15 +0100
Subject: [Python-Dev] r87445 - python/branches/py3k/Lib/numbers.py
In-Reply-To: <20101223205528.1c365abd@pitrou.net>
References: <20101223184133.9DED1EE995@mail.python.org>
	<20101223205528.1c365abd@pitrou.net>
Message-ID: <4D13BA7B.7010402@netwok.org>

Le 23/12/2010 20:55, Antoine Pitrou a ?crit :
>>      def __index__(self):
>> -        """index(self)"""
>> +        """someobject[self]"""
> 
> This is misleading as to what the method actually does,
Really?  Unless I misunderstood the docs, __index__ is used when the
object is used as an index (or with bin or oct, but I didn?t want to
complicate the docstring, just fix it).

On IRC, R. David Murray said that I could just have deleted the
docstring.  I agree with that: I think magic methods never need a
docstring, since they?re documented once and for all in the language
reference.  (__init__ is not an exception: its parameters can be
documented in the class docstring.)

> as you can read in the implementation:
>>          return int(self)
The fact that __index__ is implemented thanks to int/__int__ here is a
detail IMO.

Regards


From g.brandl at gmx.net  Thu Dec 23 22:16:22 2010
From: g.brandl at gmx.net (Georg Brandl)
Date: Thu, 23 Dec 2010 22:16:22 +0100
Subject: [Python-Dev] nonlocal x = value
In-Reply-To: <AANLkTik_D7oZkXTdZNB2=w=wN2gZeO17VTtQRZuNwsKM@mail.gmail.com>
References: <ief2tb$i9n$1@dough.gmane.org>	<AANLkTikZi3DEqx7-QgR4wZ337E-OC_Xk7E9NZaqw1jaf@mail.gmail.com>	<iei8i9$qb5$2@dough.gmane.org>	<AANLkTinmYt6or0aOz4bP1UE05T6EXuWyz7PeHcGUr0fT@mail.gmail.com>	<if0cru$2e3$1@dough.gmane.org>
	<AANLkTik_D7oZkXTdZNB2=w=wN2gZeO17VTtQRZuNwsKM@mail.gmail.com>
Message-ID: <if0e9q$9ae$1@dough.gmane.org>

Am 23.12.2010 22:03, schrieb Laurens Van Houtven:
> On Thu, Dec 23, 2010 at 9:51 PM, Georg Brandl <g.brandl at gmx.net> wrote:
>> Yes and no -- there may not be an ambiguity to the parser, but still to
>> the human.  Except if you disallow the syntax in any case, requiring
>> people to write
>>
>> nonlocal x = (3, y)
>>
>> which is then again inconsistent with ordinary assignment statements.
>>
>> Georg
> 
> Right -- but (and hence the confusion) I was arguing for not mixing
> global/nonlocal with assignment at all, and instead having nonlocal
> and global only take one or more names. That would (obviously) remove
> any such ambiguity ;-)

Oh yes, I see -- not sure why I worded it like I did.  I just wanted to
reiterate the most obvious problematic point to people who hadn't followed
the earlier discussions about it.  Sorry.

Georg


From solipsis at pitrou.net  Thu Dec 23 22:28:44 2010
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Thu, 23 Dec 2010 22:28:44 +0100
Subject: [Python-Dev] r87445 - python/branches/py3k/Lib/numbers.py
In-Reply-To: <4D13BA7B.7010402@netwok.org>
References: <20101223184133.9DED1EE995@mail.python.org>
	<20101223205528.1c365abd@pitrou.net> <4D13BA7B.7010402@netwok.org>
Message-ID: <20101223222844.654d9dfe@pitrou.net>

On Thu, 23 Dec 2010 22:09:15 +0100
?ric Araujo <merwok at netwok.org> wrote:
> Le 23/12/2010 20:55, Antoine Pitrou a ?crit :
> >>      def __index__(self):
> >> -        """index(self)"""
> >> +        """someobject[self]"""
> > 
> > This is misleading as to what the method actually does,
> Really?  Unless I misunderstood the docs, __index__ is used when the
> object is used as an index (or with bin or oct, but I didn?t want to
> complicate the docstring, just fix it).

Well, it's only used for converting to an integer (also, it's used in
more cases than just indexing), so "someobject[self]" is not the actual
operation.

Regards

Antoine.

From martin at v.loewis.de  Thu Dec 23 23:09:28 2010
From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=)
Date: Thu, 23 Dec 2010 23:09:28 +0100
Subject: [Python-Dev] r87445 - python/branches/py3k/Lib/numbers.py
In-Reply-To: <4D13BA7B.7010402@netwok.org>
References: <20101223184133.9DED1EE995@mail.python.org>	<20101223205528.1c365abd@pitrou.net>
	<4D13BA7B.7010402@netwok.org>
Message-ID: <4D13C898.6070509@v.loewis.de>

Am 23.12.2010 22:09, schrieb ?ric Araujo:
> Le 23/12/2010 20:55, Antoine Pitrou a ?crit :
>>>      def __index__(self):
>>> -        """index(self)"""
>>> +        """someobject[self]"""
>>
>> This is misleading as to what the method actually does,
> Really?  Unless I misunderstood the docs, __index__ is used when the
> object is used as an index (or with bin or oct, but I didn?t want to
> complicate the docstring, just fix it).

In case Antoine's objection isn't clear yet: __index__ does *not*
perform the actual indexing, as opposed to what the docstring suggests.

Regards,
Martin

From gzlist at googlemail.com  Thu Dec 23 23:58:23 2010
From: gzlist at googlemail.com (Martin (gzlist))
Date: Thu, 23 Dec 2010 22:58:23 +0000
Subject: [Python-Dev] Fault handler updated, now disabled by default
In-Reply-To: <1293124842.26635.3.camel@marge>
References: <1293070093.12866.14.camel@marge> <4D12B5E1.7060608@v.loewis.de>
	<1293124842.26635.3.camel@marge>
Message-ID: <AANLkTimLvs7-mVheygqYfmF_u9NnLbtOzczPep92w8=x@mail.gmail.com>

On 23/12/2010, Victor Stinner <victor.stinner at haypocalc.com> wrote:
>
> I tested my patch on Windows (XP), Ubuntu (10.4) and FreeBSD (8) and it
> works correctly: all tests pass and the system fault handler (Windows
> popup, Apport and core dump) is also called.

Doesn't build for me without #ifdef HAVE_UNISTD_H in Python/fault.c
and you missed my comment about raise vs. kill in the other thread
which lets the SIGILL test run as well. With those changes, I get:

test_disabled (test.test_fault.FaultHandlerTests) ... ok
test_fatal_error (test.test_fault.FaultHandlerTests) ... FAIL
test_gil_released (test.test_fault.FaultHandlerTests) ... ok
test_sigbus (test.test_fault.FaultHandlerTests) ... skipped 'need
_testcapi.sigbus()'
test_sigfpe (test.test_fault.FaultHandlerTests) ... skipped 'SIGFPE
cannot be caught on Windows'
test_sigill (test.test_fault.FaultHandlerTests) ... ok
test_sigsegv (test.test_fault.FaultHandlerTests) ... ok
test_xoption (test.test_fault.FaultHandlerTests) ... ok

So, this does basically work as there is a SIGSEGV compatibility hack
in VS8 and later (I'd neglected to compare the language across the
different MSDN pages). The failure is due to the test not expecting an
extra note the assert gives at the end.

See attached interdiff for suggested changes.

Martin
-------------- next part --------------
A non-text attachment was scrubbed...
Name: issue8863_incr.diff
Type: application/octet-stream
Size: 2346 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/python-dev/attachments/20101223/ddd06a2d/attachment-0001.obj>

From martin at v.loewis.de  Fri Dec 24 00:39:02 2010
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Fri, 24 Dec 2010 00:39:02 +0100
Subject: [Python-Dev] [Python-checkins] r87458 -
	python/branches/py3k/Lib/gettext.py
In-Reply-To: <20101223224938.9A243EE997@mail.python.org>
References: <20101223224938.9A243EE997@mail.python.org>
Message-ID: <4D13DD96.2070400@v.loewis.de>

>      # "a?b:c" to "test(a,b,c)".
>      expr = re.compile(r'(.*?)\?(.*?):(.*)')
>      def repl(x):
> -        return "test(%s, %s, %s)" % (x.group(1), x.group(2),
> -                                     expr.sub(repl, x.group(3)))
> +        return "(%s if %s else %s)" % (x.group(2), x.group(1),
> +                                       expr.sub(repl, x.group(3)))

Please update the comment as well.

Regards,
Martin

From benjamin at python.org  Fri Dec 24 00:45:41 2010
From: benjamin at python.org (Benjamin Peterson)
Date: Thu, 23 Dec 2010 17:45:41 -0600
Subject: [Python-Dev] [Python-checkins] r87458 -
	python/branches/py3k/Lib/gettext.py
In-Reply-To: <4D13DD96.2070400@v.loewis.de>
References: <20101223224938.9A243EE997@mail.python.org>
	<4D13DD96.2070400@v.loewis.de>
Message-ID: <AANLkTiko-b-xDnmE63jW+PnejgOjAVEtt1A87tU6WNtQ@mail.gmail.com>

2010/12/23 "Martin v. L?wis" <martin at v.loewis.de>:
>> ? ? ?# "a?b:c" to "test(a,b,c)".
>> ? ? ?expr = re.compile(r'(.*?)\?(.*?):(.*)')
>> ? ? ?def repl(x):
>> - ? ? ? ?return "test(%s, %s, %s)" % (x.group(1), x.group(2),
>> - ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? expr.sub(repl, x.group(3)))
>> + ? ? ? ?return "(%s if %s else %s)" % (x.group(2), x.group(1),
>> + ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? expr.sub(repl, x.group(3)))
>
> Please update the comment as well.

Done.



-- 
Regards,
Benjamin

From ncoghlan at gmail.com  Fri Dec 24 02:08:08 2010
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Fri, 24 Dec 2010 11:08:08 +1000
Subject: [Python-Dev] [Python-checkins] r87445 -
	python/branches/py3k/Lib/numbers.py
In-Reply-To: <20101223184133.9DED1EE995@mail.python.org>
References: <20101223184133.9DED1EE995@mail.python.org>
Message-ID: <AANLkTinakrLr3sZSbUfGpVwR+iQouXZ-OgRYQZfBQ2GB@mail.gmail.com>

On Fri, Dec 24, 2010 at 4:41 AM, eric.araujo <python-checkins at python.org> wrote:
> Author: eric.araujo
> Date: Thu Dec 23 19:41:33 2010
> New Revision: 87445
>
> Log:
> Fix small inaccuracy: there is no index function

Yes, there is, it just isn't a builtin - it lives in the operator module.

> ? ? def __index__(self):
> - ? ? ? ?"""index(self)"""
> + ? ? ? ?"""someobject[self]"""
> ? ? ? ? return int(self)

Changing the docstring to say "operator.index(self)" would be the
clearest solution here. (Choosing to accept arbitrary index objects as
integer equivalents is up to the object being indexed, just like
interpreting slices is - a dict, for example, will never invoke
__index__ methods).

Cheers,
Nick.

-- 
Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia

From victor.stinner at haypocalc.com  Fri Dec 24 03:21:19 2010
From: victor.stinner at haypocalc.com (Victor Stinner)
Date: Fri, 24 Dec 2010 03:21:19 +0100
Subject: [Python-Dev] Fault handler updated, now disabled by default
In-Reply-To: <if0dag$5a0$1@dough.gmane.org>
References: <1293070093.12866.14.camel@marge> <4D12B5E1.7060608@v.loewis.de>
	<1293124842.26635.3.camel@marge> <4D139395.3000709@v.loewis.de>
	<if0dag$5a0$1@dough.gmane.org>
Message-ID: <1293157279.16757.3.camel@marge>

Le jeudi 23 d?cembre 2010 ? 21:59 +0100, Georg Brandl a ?crit :
> this thread showed that it is not at all obvious how the feature should look like

Ok, I understand. I closed #8863 (rejected) and I created a third party
module on the Python cheese shop:
http://pypi.python.org/pypi/faulthandler/

The module works on Linux, FreeBSD and Windows with Python 2.6
through 3.2.

A third party module can evolve faster outside Python.

Victor


From victor.stinner at haypocalc.com  Fri Dec 24 03:25:02 2010
From: victor.stinner at haypocalc.com (Victor Stinner)
Date: Fri, 24 Dec 2010 03:25:02 +0100
Subject: [Python-Dev] Fault handler updated, now disabled by default
In-Reply-To: <AANLkTimLvs7-mVheygqYfmF_u9NnLbtOzczPep92w8=x@mail.gmail.com>
References: <1293070093.12866.14.camel@marge> <4D12B5E1.7060608@v.loewis.de>
	<1293124842.26635.3.camel@marge>
	<AANLkTimLvs7-mVheygqYfmF_u9NnLbtOzczPep92w8=x@mail.gmail.com>
Message-ID: <1293157502.16757.7.camel@marge>

Le jeudi 23 d?cembre 2010 ? 22:58 +0000, Martin (gzlist) a ?crit :
> On 23/12/2010, Victor Stinner <victor.stinner at haypocalc.com> wrote:
> >
> > I tested my patch on Windows (XP), Ubuntu (10.4) and FreeBSD (8) and it
> > works correctly: all tests pass and the system fault handler (Windows
> > popup, Apport and core dump) is also called.
> 
> Doesn't build for me without #ifdef HAVE_UNISTD_H in Python/fault.c

Yes, I wrote a comment about that in the issue. But then I realized that
this header is not needed at all.

> you missed my comment about raise vs. kill in the other thread
> which lets the SIGILL test run as well

Oh, I didn't noticed that this change enables sigill() on Windows.
Nice :-)

I created a third party module from my patch:
http://pypi.python.org/pypi/faulthandler/

I commited your patch, thanks. I added you to the authors as a
contributor.

Victor


From g.brandl at gmx.net  Fri Dec 24 07:51:07 2010
From: g.brandl at gmx.net (Georg Brandl)
Date: Fri, 24 Dec 2010 07:51:07 +0100
Subject: [Python-Dev] Fault handler updated, now disabled by default
In-Reply-To: <1293157279.16757.3.camel@marge>
References: <1293070093.12866.14.camel@marge>
	<4D12B5E1.7060608@v.loewis.de>	<1293124842.26635.3.camel@marge>
	<4D139395.3000709@v.loewis.de>	<if0dag$5a0$1@dough.gmane.org>
	<1293157279.16757.3.camel@marge>
Message-ID: <if1fvd$pom$1@dough.gmane.org>

Am 24.12.2010 03:21, schrieb Victor Stinner:
> Le jeudi 23 d?cembre 2010 ? 21:59 +0100, Georg Brandl a ?crit :
>> this thread showed that it is not at all obvious how the feature should look like
> 
> Ok, I understand. I closed #8863 (rejected) and I created a third party
> module on the Python cheese shop:
> http://pypi.python.org/pypi/faulthandler/
> 
> The module works on Linux, FreeBSD and Windows with Python 2.6
> through 3.2.
> 
> A third party module can evolve faster outside Python.

Very good.  I take it you'll make a new issue then for inclusion in 3.3?

Georg


From hrvoje.niksic at avl.com  Fri Dec 24 09:45:56 2010
From: hrvoje.niksic at avl.com (Hrvoje Niksic)
Date: Fri, 24 Dec 2010 09:45:56 +0100
Subject: [Python-Dev] nonlocal x = value
In-Reply-To: <AANLkTik_D7oZkXTdZNB2=w=wN2gZeO17VTtQRZuNwsKM@mail.gmail.com>
References: <ief2tb$i9n$1@dough.gmane.org>	<AANLkTikZi3DEqx7-QgR4wZ337E-OC_Xk7E9NZaqw1jaf@mail.gmail.com>	<iei8i9$qb5$2@dough.gmane.org>	<AANLkTinmYt6or0aOz4bP1UE05T6EXuWyz7PeHcGUr0fT@mail.gmail.com>	<if0cru$2e3$1@dough.gmane.org>
	<AANLkTik_D7oZkXTdZNB2=w=wN2gZeO17VTtQRZuNwsKM@mail.gmail.com>
Message-ID: <4D145DC4.9000107@avl.com>

On 12/23/2010 10:03 PM, Laurens Van Houtven wrote:
> On Thu, Dec 23, 2010 at 9:51 PM, Georg Brandl<g.brandl at gmx.net>  wrote:
>>  Yes and no -- there may not be an ambiguity to the parser, but still to
>>  the human.  Except if you disallow the syntax in any case, requiring
>>  people to write
>>
>>  nonlocal x = (3, y)
>>
>>  which is then again inconsistent with ordinary assignment statements.
>
> Right -- but (and hence the confusion) I was arguing for not mixing
> global/nonlocal with assignment at all, and instead having nonlocal
> and global only take one or more names. That would (obviously) remove
> any such ambiguity ;-)

I would like to offer the opposing viewpoint: nonlocal x = value is a 
useful shortcut because nonlocal is used in closure callbacks where 
brevity matters.  The reason nonlocal is introduced is to change the 
variable, so it makes sense that the two can be done in the same line of 
code.

As for global x = value being disallowed, I have been annoyed at times 
with that, so that sounds like a good argument to change both.

Requiring the parentheses for tuple creation sounds like a good 
compromise for resolving the ambiguity, consistent with similar 
limitations of the generator expression syntax.

From fuzzyman at voidspace.org.uk  Fri Dec 24 13:59:41 2010
From: fuzzyman at voidspace.org.uk (Michael Foord)
Date: Fri, 24 Dec 2010 12:59:41 +0000
Subject: [Python-Dev] Fault handler updated, now disabled by default
In-Reply-To: <1293157279.16757.3.camel@marge>
References: <1293070093.12866.14.camel@marge>
	<4D12B5E1.7060608@v.loewis.de>	<1293124842.26635.3.camel@marge>
	<4D139395.3000709@v.loewis.de>	<if0dag$5a0$1@dough.gmane.org>
	<1293157279.16757.3.camel@marge>
Message-ID: <4D14993D.2040803@voidspace.org.uk>

On 24/12/2010 02:21, Victor Stinner wrote:
> Le jeudi 23 d?cembre 2010 ? 21:59 +0100, Georg Brandl a ?crit :
>> this thread showed that it is not at all obvious how the feature should look like
> Ok, I understand. I closed #8863 (rejected) and I created a third party
> module on the Python cheese shop:
> http://pypi.python.org/pypi/faulthandler/
>
> The module works on Linux, FreeBSD and Windows with Python 2.6
> through 3.2.
>
> A third party module can evolve faster outside Python.

I hope you will include it in 3.3 though; it is great functionality. I 
would really like to see it enabled by default as well.

It seemed from the discussion that the biggest barrier to enabling it by 
default was possible difficulties when embedding Python (multiple 
interpreters, potential conflicts with application signal handling). A 
public C-API to disable the functionality per interpreter would be one 
option for this. Another possibility would be providing a C-API to 
enable it and have the Python interpreter application call this, so that 
the functionality remains off by default for embedded interpreters but 
on for normal uses.

All the best,

Michael

> Victor
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk


-- 
http://www.voidspace.org.uk/

May you do good and not evil
May you find forgiveness for yourself and forgive others
May you share freely, never taking more than you give.
-- the sqlite blessing http://www.sqlite.org/different.html


From fuzzyman at voidspace.org.uk  Fri Dec 24 17:09:42 2010
From: fuzzyman at voidspace.org.uk (Michael Foord)
Date: Fri, 24 Dec 2010 16:09:42 +0000
Subject: [Python-Dev] r87389 - in python/branches/py3k:
 Doc/library/unittest.rst Lib/unittest/case.py Misc/NEWS
In-Reply-To: <iernlg$cib$1@dough.gmane.org>
References: <20101218200004.99110EE985@mail.python.org>	<20101218214830.1aa8be32@pitrou.net>	<iejdjt$ik0$1@dough.gmane.org>	<AANLkTi=ADkUMc5tQGDv2yniteyfPCPXU=XaCr+GscR+a@mail.gmail.com>	<20101219141355.5ac7cd2f@pitrou.net>	<AANLkTinyvb6+Xf=owdV22j6YoAEVjbTXV2tetQzdD8r4@mail.gmail.com>	<31185718-F276-46CE-82E2-6C6B6EC920E8@gmail.com>	<4D0F536A.8050808@voidspace.org.uk>	<1292852841.3737.25.camel@localhost.localdomain>	<4D0F622E.7050902@voidspace.org.uk>	<1292855497.3737.55.camel@localhost.localdomain>	<AANLkTi=pr227YH_8vdLBFASHYCRMN6RAxrdoF8f6kqkx@mail.gmail.com>	<4D109AEA.7010908@voidspace.org.uk>
	<iernlg$cib$1@dough.gmane.org>
Message-ID: <4D14C5C6.9070703@voidspace.org.uk>

On 22/12/2010 02:26, Terry Reedy wrote:
> On 12/21/2010 7:17 AM, Michael Foord wrote:
>
> My first priority is that doc and code match.
> Close second is consistency (hence, ease of learning and use) between 
> various AssertXs.
>
>> Symmetrical diffs (element in first not in second, element in second not
>> in first) solves the problem without imposing an order on the arguments.
>
> Where applicable, I prefer this as unambiguous output headings.
>
Could you explain what you mean?

All the best,

Michael Foord

-- 
http://www.voidspace.org.uk/

May you do good and not evil
May you find forgiveness for yourself and forgive others
May you share freely, never taking more than you give.
-- the sqlite blessing http://www.sqlite.org/different.html


From status at bugs.python.org  Fri Dec 24 18:07:05 2010
From: status at bugs.python.org (Python tracker)
Date: Fri, 24 Dec 2010 18:07:05 +0100 (CET)
Subject: [Python-Dev] Summary of Python tracker Issues
Message-ID: <20101224170705.3F7651CCBB@psf.upfronthosting.co.za>


ACTIVITY SUMMARY (2010-12-17 - 2010-12-24)
Python tracker at http://bugs.python.org/

To view or respond to any of the issues listed below, click on the issue.
Do NOT respond to this message.

Issues counts and deltas:
  open    2542 ( +5)
  closed 20009 (+38)
  total  22551 (+43)

Open issues with patches: 1069 


Issues opened (35)
==================

#3243: Support iterable bodies in httplib
http://bugs.python.org/issue3243  reopened by georg.brandl

#9090: Error code 10035 calling socket.recv() on a socket with a time
http://bugs.python.org/issue9090  reopened by ehohenstein

#10731: UnicodeDecodeError in OS X tkinter when binding to <MouseWheel
http://bugs.python.org/issue10731  opened by culler

#10733: plistlib rejects strings containing control characters
http://bugs.python.org/issue10733  opened by MLModel

#10734: test_ttk failure under Windows
http://bugs.python.org/issue10734  opened by pitrou

#10735: platform.architecture() gives misleading results for OS X mult
http://bugs.python.org/issue10735  opened by ned.deily

#10736: test_ttk_guionly fails on OS X using ActiveState Tcl 8.5.9 (Co
http://bugs.python.org/issue10736  opened by ned.deily

#10737: test_concurrent_futures failure on Windows
http://bugs.python.org/issue10737  opened by pitrou

#10738: webbrowser.py bug with Opera on Linux
http://bugs.python.org/issue10738  opened by NE1

#10739: Subprocess behavior on Windows
http://bugs.python.org/issue10739  opened by rosslagerwall

#10740: sqlite3 module should allow DDL statements in transactions
http://bugs.python.org/issue10740  opened by scott.urban

#10741: PyGILState_GetThisThreadState() lacks a doc entry
http://bugs.python.org/issue10741  opened by pitrou

#10742: memoryview.readonly attribute is not documented
http://bugs.python.org/issue10742  opened by flashk

#10744: ctypes arrays have incorrect buffer information (PEP-3118)
http://bugs.python.org/issue10744  opened by pv

#10745: setup.py install --user option undocumented
http://bugs.python.org/issue10745  opened by gotgenes

#10746: ctypes c_long & c_bool have incorrect PEP-3118 type codes
http://bugs.python.org/issue10746  opened by pv

#10747: Include version info in Windows shortcuts
http://bugs.python.org/issue10747  opened by ncoghlan

#10751: WSGIREF - REMOTE_USER and REMOTE-USER collision
http://bugs.python.org/issue10751  opened by Alex.Raitz

#10752: build_ssl.py is relying on unreliable behaviour of os.popen
http://bugs.python.org/issue10752  opened by srid

#10753: request_uri method of wsgiref module does not support RFC1808 
http://bugs.python.org/issue10753  opened by Timothy.Gates

#10755: Add posix.fdlistdir
http://bugs.python.org/issue10755  opened by rosslagerwall

#10756: Error in atexit._run_exitfuncs [...]  Exception expected for v
http://bugs.python.org/issue10756  opened by kaizhu

#10757: zipfile.write, arcname should be bytestring
http://bugs.python.org/issue10757  opened by connexion2000

#10758: posix_access swallows all errors
http://bugs.python.org/issue10758  opened by georg.brandl

#10759: HTMLParser.unescape() fails on HTML entities with incorrect sy
http://bugs.python.org/issue10759  opened by Martin.Potthast

#10760: tarfile doesn't handle sysfs well
http://bugs.python.org/issue10760  opened by Yoni.Tsafir

#10761: tarfile.extractall fails to overwrite symlinks
http://bugs.python.org/issue10761  opened by srid

#10762: strftime('%f') segfault
http://bugs.python.org/issue10762  opened by dleonard0

#10763: subprocess.communicate() doesn't close pipes on Windows
http://bugs.python.org/issue10763  opened by haypo

#10764: sysconfig and alternative implementations
http://bugs.python.org/issue10764  opened by michael.foord

#10765: Build regression from automation changes on windows
http://bugs.python.org/issue10765  opened by gz

#10766: optparse uses %s in gettext calls
http://bugs.python.org/issue10766  opened by eric.araujo

#10767: Lib/test/crashers/README is out of date
http://bugs.python.org/issue10767  opened by belopolsky

#10768: Bug in scrolledtext
http://bugs.python.org/issue10768  opened by quentel

#10769: ast: provide more useful range information
http://bugs.python.org/issue10769  opened by scummos



Most recent 15 issues with no replies (15)
==========================================

#10769: ast: provide more useful range information
http://bugs.python.org/issue10769

#10767: Lib/test/crashers/README is out of date
http://bugs.python.org/issue10767

#10766: optparse uses %s in gettext calls
http://bugs.python.org/issue10766

#10764: sysconfig and alternative implementations
http://bugs.python.org/issue10764

#10763: subprocess.communicate() doesn't close pipes on Windows
http://bugs.python.org/issue10763

#10761: tarfile.extractall fails to overwrite symlinks
http://bugs.python.org/issue10761

#10760: tarfile doesn't handle sysfs well
http://bugs.python.org/issue10760

#10756: Error in atexit._run_exitfuncs [...]  Exception expected for v
http://bugs.python.org/issue10756

#10752: build_ssl.py is relying on unreliable behaviour of os.popen
http://bugs.python.org/issue10752

#10751: WSGIREF - REMOTE_USER and REMOTE-USER collision
http://bugs.python.org/issue10751

#10747: Include version info in Windows shortcuts
http://bugs.python.org/issue10747

#10746: ctypes c_long & c_bool have incorrect PEP-3118 type codes
http://bugs.python.org/issue10746

#10745: setup.py install --user option undocumented
http://bugs.python.org/issue10745

#10744: ctypes arrays have incorrect buffer information (PEP-3118)
http://bugs.python.org/issue10744

#10742: memoryview.readonly attribute is not documented
http://bugs.python.org/issue10742



Most recent 15 issues waiting for review (15)
=============================================

#10766: optparse uses %s in gettext calls
http://bugs.python.org/issue10766

#10765: Build regression from automation changes on windows
http://bugs.python.org/issue10765

#10763: subprocess.communicate() doesn't close pipes on Windows
http://bugs.python.org/issue10763

#10759: HTMLParser.unescape() fails on HTML entities with incorrect sy
http://bugs.python.org/issue10759

#10755: Add posix.fdlistdir
http://bugs.python.org/issue10755

#10746: ctypes c_long & c_bool have incorrect PEP-3118 type codes
http://bugs.python.org/issue10746

#10744: ctypes arrays have incorrect buffer information (PEP-3118)
http://bugs.python.org/issue10744

#10742: memoryview.readonly attribute is not documented
http://bugs.python.org/issue10742

#10740: sqlite3 module should allow DDL statements in transactions
http://bugs.python.org/issue10740

#10739: Subprocess behavior on Windows
http://bugs.python.org/issue10739

#10735: platform.architecture() gives misleading results for OS X mult
http://bugs.python.org/issue10735

#10721: Remove HTTP 0.9 server support
http://bugs.python.org/issue10721

#10712: 2to3 fixer for deprecated unittest method names
http://bugs.python.org/issue10712

#10694: zipfile.py end of central directory detection not robust
http://bugs.python.org/issue10694

#10680: argparse: titles and add_mutually_exclusive_group don't mix (e
http://bugs.python.org/issue10680



Top 10 most discussed issues (10)
=================================

#10694: zipfile.py end of central directory detection not robust
http://bugs.python.org/issue10694  13 msgs

#2736: datetime needs an "epoch" method
http://bugs.python.org/issue2736  10 msgs

#3243: Support iterable bodies in httplib
http://bugs.python.org/issue3243   9 msgs

#4761: create Python wrappers for openat() and others
http://bugs.python.org/issue4761   9 msgs

#10254: unicodedata.normalize('NFC', s) regression
http://bugs.python.org/issue10254   7 msgs

#10735: platform.architecture() gives misleading results for OS X mult
http://bugs.python.org/issue10735   7 msgs

#10759: HTMLParser.unescape() fails on HTML entities with incorrect sy
http://bugs.python.org/issue10759   6 msgs

#7511: msvc9compiler.py: ValueError: [u'path']
http://bugs.python.org/issue7511   5 msgs

#9584: Allow curly brace expansion
http://bugs.python.org/issue9584   5 msgs

#10573: Consistency in unittest assert methods: order of actual, expec
http://bugs.python.org/issue10573   5 msgs



Issues closed (35)
==================

#2494: Can't round-trip datetimes<->timestamps prior to 1970 on Windo
http://bugs.python.org/issue2494  closed by r.david.murray

#3500: unbound methods of different classes compare equal
http://bugs.python.org/issue3500  closed by r.david.murray

#4078: asyncore fixes are not backwards compatible
http://bugs.python.org/issue4078  closed by r.david.murray

#4322: function with modified __name__ uses original name when there'
http://bugs.python.org/issue4322  closed by r.david.murray

#4369: Error building to a nonstandard prefix (with patch)
http://bugs.python.org/issue4369  closed by r.david.murray

#4496: misleading comment in urllib2
http://bugs.python.org/issue4496  closed by r.david.murray

#4871: zipfile can't decrypt
http://bugs.python.org/issue4871  closed by r.david.murray

#5587: vars() no longer has a useful __repr__
http://bugs.python.org/issue5587  closed by ezio.melotti

#8073: Test fail for sha512 -
http://bugs.python.org/issue8073  closed by gregory.p.smith

#8863: Display Python backtrace on SIGSEGV, SIGFPE and fatal error
http://bugs.python.org/issue8863  closed by haypo

#9063: TZ examples in datetime.rst are incorrect
http://bugs.python.org/issue9063  closed by belopolsky

#9234: argparse: aliases for positional arguments (subparsers)
http://bugs.python.org/issue9234  closed by bethard

#9286: email.utils.parseaddr returns garbage for invalid input
http://bugs.python.org/issue9286  closed by r.david.murray

#9857: SkipTest in tearDown is reported an as an error
http://bugs.python.org/issue9857  closed by michael.foord

#9907: interactive mode TAB does not insert on OS X built with editli
http://bugs.python.org/issue9907  closed by r.david.murray

#10404: IDLE on OS X popup menus do not work: cannot set/clear breakpo
http://bugs.python.org/issue10404  closed by r.david.murray

#10587: Document the meaning of str methods
http://bugs.python.org/issue10587  closed by belopolsky

#10611: sys.exit() in a test causes a test run to die
http://bugs.python.org/issue10611  closed by michael.foord

#10622: WebKit browsers show superfluous scrollbars in html docs
http://bugs.python.org/issue10622  closed by r.david.murray

#10671: urllib2 redirect to another host doesn't work
http://bugs.python.org/issue10671  closed by orsenthil

#10711: Rip out HTTP 0.9 client support
http://bugs.python.org/issue10711  closed by pitrou

#10715: Command name missing from exception in subprocess.Popen
http://bugs.python.org/issue10715  closed by eric.smith

#10723: Undocumented built-in exceptions
http://bugs.python.org/issue10723  closed by georg.brandl

#10725: Better cache instrumentation
http://bugs.python.org/issue10725  closed by rhettinger

#10727: intermittent test_cmd_line failure
http://bugs.python.org/issue10727  closed by ned.deily

#10728: argparse.ArgumentParser.print_help uses sys.stdout
http://bugs.python.org/issue10728  closed by r.david.murray

#10729: fwbackups python lib error
http://bugs.python.org/issue10729  closed by eric.smith

#10730: add .svgz to mimetypes.suffix_map and .svg to types_map
http://bugs.python.org/issue10730  closed by terry.reedy

#10732: Wrong destructor function type in Python/getargs.c
http://bugs.python.org/issue10732  closed by pitrou

#10743: 3.2's sysconfig doesn't work with virtualenv
http://bugs.python.org/issue10743  closed by srid

#10749: lots of warnings when generating logging docs
http://bugs.python.org/issue10749  closed by georg.brandl

#10750: "raw" attribute of buffered IO objects is assignable
http://bugs.python.org/issue10750  closed by pitrou

#10754: os.path.isfile doesn't work with some greek characters
http://bugs.python.org/issue10754  closed by r.david.murray

#10748: zipfile does not write empty ZIP structure if close() called a
http://bugs.python.org/issue10748  closed by georg.brandl

#1155362: Bugs in parsedate_tz
http://bugs.python.org/issue1155362  closed by r.david.murray

From tjreedy at udel.edu  Fri Dec 24 19:56:06 2010
From: tjreedy at udel.edu (Terry Reedy)
Date: Fri, 24 Dec 2010 13:56:06 -0500
Subject: [Python-Dev] r87389 - in python/branches/py3k:
 Doc/library/unittest.rst Lib/unittest/case.py Misc/NEWS
In-Reply-To: <4D14C5C6.9070703@voidspace.org.uk>
References: <20101218200004.99110EE985@mail.python.org>	<20101218214830.1aa8be32@pitrou.net>	<iejdjt$ik0$1@dough.gmane.org>	<AANLkTi=ADkUMc5tQGDv2yniteyfPCPXU=XaCr+GscR+a@mail.gmail.com>	<20101219141355.5ac7cd2f@pitrou.net>	<AANLkTinyvb6+Xf=owdV22j6YoAEVjbTXV2tetQzdD8r4@mail.gmail.com>	<31185718-F276-46CE-82E2-6C6B6EC920E8@gmail.com>	<4D0F536A.8050808@voidspace.org.uk>	<1292852841.3737.25.camel@localhost.localdomain>	<4D0F622E.7050902@voidspace.org.uk>	<1292855497.3737.55.camel@localhost.localdomain>	<AANLkTi=pr227YH_8vdLBFASHYCRMN6RAxrdoF8f6kqkx@mail.gmail.com>	<4D109AEA.7010908@voidspace.org.uk>	<iernlg$cib$1@dough.gmane.org>
	<4D14C5C6.9070703@voidspace.org.uk>
Message-ID: <if2qc5$qjf$1@dough.gmane.org>

On 12/24/2010 11:09 AM, Michael Foord wrote:
> On 22/12/2010 02:26, Terry Reedy wrote:
>> On 12/21/2010 7:17 AM, Michael Foord wrote:
>>
>> My first priority is that doc and code match.
>> Close second is consistency (hence, ease of learning and use) between
>> various AssertXs.
>>
>>> Symmetrical diffs (element in first not in second, element in second not
>>> in first) solves the problem without imposing an order on the arguments.
>>
>> Where applicable, I prefer this as unambiguous output headings.
>>
> Could you explain what you mean?

I was referring back to an output example symmetric diff that was 
clipped somewhere along the way:

In x not in y:
...
In y not in x:
...

rather than just using -,+ prefixes which are not necessarily 
self-explanatory. 'Not applicable' would refer to output from difflib 
which necessarily is ordered.


-- 
Terry Jan Reedy


From raymond.hettinger at gmail.com  Fri Dec 24 21:03:38 2010
From: raymond.hettinger at gmail.com (Raymond Hettinger)
Date: Fri, 24 Dec 2010 12:03:38 -0800
Subject: [Python-Dev] r87389 - in python/branches/py3k:
	Doc/library/unittest.rst Lib/unittest/case.py Misc/NEWS
In-Reply-To: <if2qc5$qjf$1@dough.gmane.org>
References: <20101218200004.99110EE985@mail.python.org>	<20101218214830.1aa8be32@pitrou.net>	<iejdjt$ik0$1@dough.gmane.org>	<AANLkTi=ADkUMc5tQGDv2yniteyfPCPXU=XaCr+GscR+a@mail.gmail.com>	<20101219141355.5ac7cd2f@pitrou.net>	<AANLkTinyvb6+Xf=owdV22j6YoAEVjbTXV2tetQzdD8r4@mail.gmail.com>	<31185718-F276-46CE-82E2-6C6B6EC920E8@gmail.com>	<4D0F536A.8050808@voidspace.org.uk>	<1292852841.3737.25.camel@localhost.localdomain>	<4D0F622E.7050902@voidspace.org.uk>	<1292855497.3737.55.camel@localhost.localdomain>	<AANLkTi=pr227YH_8vdLBFASHYCRMN6RAxrdoF8f6kqkx@mail.gmail.com>	<4D109AEA.7010908@voidspace.org.uk>	<iernlg$cib$1@dough.gmane.org>
	<4D14C5C6.9070703@voidspace.org.uk> <if2qc5$qjf$1@dough.gmane.org>
Message-ID: <763DCF04-033D-455B-A7BF-9D5813BBDFD4@gmail.com>


On Dec 24, 2010, at 10:56 AM, Terry Reedy wrote:

> On 12/24/2010 11:09 AM, Michael Foord wrote:
>> On 22/12/2010 02:26, Terry Reedy wrote:
>>> On 12/21/2010 7:17 AM, Michael Foord wrote:
>>> 
>>> My first priority is that doc and code match.
>>> Close second is consistency (hence, ease of learning and use) between
>>> various AssertXs.
>>> 
>>>> Symmetrical diffs (element in first not in second, element in second not
>>>> in first) solves the problem without imposing an order on the arguments.
>>> 
>>> Where applicable, I prefer this as unambiguous output headings.
>>> 
>> Could you explain what you mean?
> 
> I was referring back to an output example symmetric diff that was clipped somewhere along the way:
> 
> In x not in y:
> ...
> In y not in x:
> ...
> 
> rather than just using -,+ prefixes which are not necessarily self-explanatory. 'Not applicable' would refer to output from difflib which necessarily is ordered.

FWIW, I think + and - prefixes are much better for diffs that some made-up verbiage.  People are used to seeing diffs with + and -.   Anything else will be so contrived that it's net effect will be to make the output confusing and hard to interpret.

If you want, add two lines of explanation before the diff:
    + means "in x, not in y"
    -  means "in y, not it x"

The notion  of "making symmetric" can easily get carried too far, which a corresponding loss of useability.  You get 95% of the benefit from two small changes:

* Change the parameter names from "actual" and "expected" to "first" and "second"
* Change the words "unexpected" and "missing" to "in first, not in second" and "in second, not in first".

We have a strong history in using +/- and shouldn't throw away its brevity and clarity.


Raymond


From stephen at xemacs.org  Sat Dec 25 03:29:51 2010
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Sat, 25 Dec 2010 11:29:51 +0900
Subject: [Python-Dev] Fault handler updated, now disabled by default
In-Reply-To: <4D14993D.2040803@voidspace.org.uk>
References: <1293070093.12866.14.camel@marge> <4D12B5E1.7060608@v.loewis.de>
	<1293124842.26635.3.camel@marge> <4D139395.3000709@v.loewis.de>
	<if0dag$5a0$1@dough.gmane.org> <1293157279.16757.3.camel@marge>
	<4D14993D.2040803@voidspace.org.uk>
Message-ID: <87wrmymvog.fsf@uwakimon.sk.tsukuba.ac.jp>

Michael Foord writes:

 > It seemed from the discussion that the biggest barrier to enabling it by 
 > default was possible difficulties when embedding Python (multiple 
 > interpreters, potential conflicts with application signal handling). A 
 > public C-API to disable the functionality per interpreter would be one 
 > option for this.

That's not really good enough.  The point of installing a handler like
this is to "catch them squirmers".  All you have to do is override
some incautious developer's own squirmer-trap handler once, and Python
has made an Enemy-For-Life.  (This happened to XEmacs with esound.  I
immediately removed esound and anything that depends on it from my
workstation. ;-)  YMMV and you may think that that is not so important;
my point is that the proposal to provide a way to disable does not at
all address the objection.

 > Another possibility would be providing a C-API to enable it and
 > have the Python interpreter application call this, so that the
 > functionality remains off by default for embedded interpreters but
 > on for normal uses.

I think this is heading in the right direction.

Note: My own experience with such handlers has been positive, but it
does not involve embedding interpreters in either direction, so not
really helpful in addressing this objection.  Precisely *because* my
own experience has been positive, I worry about interfering with some
third party's handler.


From jcea at jcea.es  Sat Dec 25 05:40:19 2010
From: jcea at jcea.es (Jesus Cea)
Date: Sat, 25 Dec 2010 05:40:19 +0100
Subject: [Python-Dev] Compiling CPython to javascript
Message-ID: <4D1575B3.3050809@jcea.es>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Interesting development.

http://syntensity.com/static/python.html

http://syntensity.blogspot.com/

- -- 
Jesus Cea Avion                         _/_/      _/_/_/        _/_/_/
jcea at jcea.es - http://www.jcea.es/     _/_/    _/_/  _/_/    _/_/  _/_/
jabber / xmpp:jcea at jabber.org         _/_/    _/_/          _/_/_/_/_/
.                              _/_/  _/_/    _/_/          _/_/  _/_/
"Things are not so easy"      _/_/  _/_/    _/_/  _/_/    _/_/  _/_/
"My name is Dump, Core Dump"   _/_/_/        _/_/_/      _/_/  _/_/
"El amor es poner tu felicidad en la felicidad de otro" - Leibniz
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.10 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iQCVAwUBTRV1s5lgi5GaxT1NAQLO1gQAjwjfyVHmQOgNg9cBD/lDZFoHJhPSbfaY
KGnXVOVbXoAo6+o7Ne7mXXWamD38VK9et7dWckyWbqubHk1IGLuP9eCncKKjuX4j
UTFMG26i4p24oQ6AMyUs2wuC7ockZ4PK8r7vcPs3YxSRR5rx81ojaH6SFQ2qJJGc
kRTtNe+iExc=
=UPG8
-----END PGP SIGNATURE-----

From stefan_ml at behnel.de  Sat Dec 25 11:59:36 2010
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Sat, 25 Dec 2010 11:59:36 +0100
Subject: [Python-Dev] nonlocal x = value
In-Reply-To: <4D145DC4.9000107@avl.com>
References: <ief2tb$i9n$1@dough.gmane.org>	<AANLkTikZi3DEqx7-QgR4wZ337E-OC_Xk7E9NZaqw1jaf@mail.gmail.com>	<iei8i9$qb5$2@dough.gmane.org>	<AANLkTinmYt6or0aOz4bP1UE05T6EXuWyz7PeHcGUr0fT@mail.gmail.com>	<if0cru$2e3$1@dough.gmane.org>	<AANLkTik_D7oZkXTdZNB2=w=wN2gZeO17VTtQRZuNwsKM@mail.gmail.com>
	<4D145DC4.9000107@avl.com>
Message-ID: <if4iqo$5b4$1@dough.gmane.org>

Hrvoje Niksic, 24.12.2010 09:45:
> On 12/23/2010 10:03 PM, Laurens Van Houtven wrote:
>> On Thu, Dec 23, 2010 at 9:51 PM, Georg Brandl wrote:
>>> Yes and no -- there may not be an ambiguity to the parser, but still to
>>> the human. Except if you disallow the syntax in any case, requiring
>>> people to write
>>>
>>> nonlocal x = (3, y)
>>>
>>> which is then again inconsistent with ordinary assignment statements.
>>
>> Right -- but (and hence the confusion) I was arguing for not mixing
>> global/nonlocal with assignment at all, and instead having nonlocal
>> and global only take one or more names. That would (obviously) remove
>> any such ambiguity ;-)
>
> I would like to offer the opposing viewpoint: nonlocal x = value is a
> useful shortcut because nonlocal is used in closure callbacks where
> brevity matters.

I doubt that it really matters so much that one line more kills 
readability. It's still a relatively rare use case after all.


> The reason nonlocal is introduced is to change the
> variable, so it makes sense that the two can be done in the same line of
> code.

But still, this is just a special case. If the variable is changed more 
than once, you'd end up with one assignment with nonlocal and one without. 
This just adds to the growing list of code inconsistencies presented in 
this thread.


> As for global x = value being disallowed, I have been annoyed at times with
> that, so that sounds like a good argument to change both.
>
> Requiring the parentheses for tuple creation sounds like a good compromise
> for resolving the ambiguity, consistent with similar limitations of the
> generator expression syntax.

Why introduce such a pitfall for coders here? "Require" doesn't mean the 
parser can enforce it. If it's not there, it just means something else, so 
it's up to the coder to get it right. The exact same kind of situation that 
was fixed for the except syntax in Python 3.

Stefan


From g.brandl at gmx.net  Sat Dec 25 20:35:58 2010
From: g.brandl at gmx.net (Georg Brandl)
Date: Sat, 25 Dec 2010 20:35:58 +0100
Subject: [Python-Dev] r87478 - python/branches/py3k/Misc/maintainers.rst
In-Reply-To: <20101224215903.3E813EE9BA@mail.python.org>
References: <20101224215903.3E813EE9BA@mail.python.org>
Message-ID: <if5h5o$bq0$1@dough.gmane.org>

Eh, no.  That is a different user.  Alan didn't have Developer tracker
privileges yet though, so I fixed that.  Please revert.

Georg

Am 24.12.2010 22:59, schrieb terry.reedy:
> Author: terry.reedy
> Date: Fri Dec 24 22:59:03 2010
> New Revision: 87478
> 
> Log:
> Match current tracker name, though I do not know if still active.
> 
> 
> Modified:
>    python/branches/py3k/Misc/maintainers.rst
> 
> Modified: python/branches/py3k/Misc/maintainers.rst
> ==============================================================================
> --- python/branches/py3k/Misc/maintainers.rst	(original)
> +++ python/branches/py3k/Misc/maintainers.rst	Fri Dec 24 22:59:03 2010
> @@ -247,7 +247,7 @@
>  xml.sax.saxutils
>  xml.sax.xmlreader
>  xmlrpc              loewis
> -zipfile             alanmcintyre
> +zipfile             aimacintyre
>  zipimport
>  zlib
>  ==================  ===========



From victor.stinner at haypocalc.com  Sat Dec 25 22:58:00 2010
From: victor.stinner at haypocalc.com (Victor Stinner)
Date: Sat, 25 Dec 2010 22:58:00 +0100
Subject: [Python-Dev] Fault handler updated, now disabled by default
In-Reply-To: <4D14993D.2040803@voidspace.org.uk>
References: <1293070093.12866.14.camel@marge> <4D12B5E1.7060608@v.loewis.de>
	<1293124842.26635.3.camel@marge> <4D139395.3000709@v.loewis.de>
	<if0dag$5a0$1@dough.gmane.org> <1293157279.16757.3.camel@marge>
	<4D14993D.2040803@voidspace.org.uk>
Message-ID: <1293314280.21272.183.camel@marge>

Le vendredi 24 d?cembre 2010 ? 12:59 +0000, Michael Foord a ?crit :
> I hope you will include it in 3.3 though; it is great functionality.

I don't know, Python 3.3 is not planned yet. Anyway, faulthandler is
already available and you can use it on Python 2.5 through 3.2.

> I would really like to see it enabled by default as well.

For different reasons, it is not a good idea to enable it by default
(see the recent discussion on this list).

> It seemed from the discussion that the biggest barrier to enabling it by 
> default was possible difficulties when ...

Yes, that's why it was disabled by default in the latest version of my
patch.

> (multiple interpreters, 

I think that my fault handler works with multiple interpreters.

> potential conflicts with application signal handling)

Very few programs have an handler for SIGSEGV, SIGFPE, SIGILL or SIGBUS
signals (but yes, there are such programs/libraries like RPy or XEmacs).

> A public C-API to disable the functionality per interpreter would be one 
> option for this. Another possibility would be providing a C-API to 
> enable it and have the Python interpreter application call this, so that 
> the functionality remains off by default for embedded interpreters but 
> on for normal uses.

I think that it's better to disable it by default, and only enabled by
the user or by the developer of a specific application.

--

Anyway, you don't have to wait Python 3.3 (or Python 3.2): you can
already use faulthandler ;-) I created Windows installer for Python 2.6,
2.7 and 3.1 (I will do it for 2.5 too in the next faulthandler version,
faulthandler 1.0  tests don't work with Python 2.5).

faulthandler is a module: enable the handler is simple as "import
faulthandler". Said differently, it's also disabled by default, and only
enabled by the application developer. It's harder to enable it for the
user, the site module should maybe be patched to load the module if an
environment variable exist or a command line argument.

Victor


From merwok at netwok.org  Sat Dec 25 23:31:18 2010
From: merwok at netwok.org (=?UTF-8?B?w4lyaWMgQXJhdWpv?=)
Date: Sat, 25 Dec 2010 23:31:18 +0100
Subject: [Python-Dev] Fault handler updated, now disabled by default
In-Reply-To: <1293314280.21272.183.camel@marge>
References: <1293070093.12866.14.camel@marge>
	<4D12B5E1.7060608@v.loewis.de>	<1293124842.26635.3.camel@marge>
	<4D139395.3000709@v.loewis.de>	<if0dag$5a0$1@dough.gmane.org>
	<1293157279.16757.3.camel@marge>	<4D14993D.2040803@voidspace.org.uk>
	<1293314280.21272.183.camel@marge>
Message-ID: <4D1670B6.6000801@netwok.org>

> faulthandler is a module: enable the handler is simple as "import
> faulthandler".

That sounds like a source of unwanted behavior (aka problems) if the
handler is enabled by ?pydoc faulthandler? or by a pkgutil walk.  You
may want to consider using a function to enable the functionality (and
add one to disable it).

Regards


From exarkun at twistedmatrix.com  Sun Dec 26 15:10:49 2010
From: exarkun at twistedmatrix.com (exarkun at twistedmatrix.com)
Date: Sun, 26 Dec 2010 14:10:49 -0000
Subject: [Python-Dev] Fault handler updated, now disabled by default
In-Reply-To: <4D1670B6.6000801@netwok.org>
References: <1293070093.12866.14.camel@marge> <4D12B5E1.7060608@v.loewis.de>
	<1293124842.26635.3.camel@marge> <4D139395.3000709@v.loewis.de>
	<if0dag$5a0$1@dough.gmane.org> <1293157279.16757.3.camel@marge>
	<4D14993D.2040803@voidspace.org.uk>
	<1293314280.21272.183.camel@marge> <4D1670B6.6000801@netwok.org>
Message-ID: <20101226141049.5919.1054241426.divmod.xquotient.369@localhost.localdomain>

On 25 Dec, 10:31 pm, merwok at netwok.org wrote:
>>faulthandler is a module: enable the handler is simple as "import
>>faulthandler".
>
>That sounds like a source of unwanted behavior (aka problems) if the
>handler is enabled by  1Cpydoc faulthandler 1D or by a pkgutil walk.  You
>may want to consider using a function to enable the functionality (and
>add one to disable it).

Enormous +1.

Jean-Paul

From merwok at netwok.org  Sun Dec 26 18:13:53 2010
From: merwok at netwok.org (=?UTF-8?B?w4lyaWMgQXJhdWpv?=)
Date: Sun, 26 Dec 2010 18:13:53 +0100
Subject: [Python-Dev] [Python-checkins] r87445 -
	python/branches/py3k/Lib/numbers.py
In-Reply-To: <AANLkTinakrLr3sZSbUfGpVwR+iQouXZ-OgRYQZfBQ2GB@mail.gmail.com>
References: <20101223184133.9DED1EE995@mail.python.org>
	<AANLkTinakrLr3sZSbUfGpVwR+iQouXZ-OgRYQZfBQ2GB@mail.gmail.com>
Message-ID: <4D1777D1.3060104@netwok.org>

Le 24/12/2010 02:08, Nick Coghlan a ?crit :
> On Fri, Dec 24, 2010 at 4:41 AM, eric.araujo <python-checkins at python.org> wrote:
>> Fix small inaccuracy: there is no index function
> 
> Yes, there is, it just isn't a builtin - it lives in the operator module.
Defining object.__index__ with operator.index seems pretty circular to
me :)  http://docs.python.org/dev/reference/datamodel#object.__index__
does it, but it should be fixed IMO.  The difference between __int__ and
__index__ is also not clear.

>>     def __index__(self):
>> -        """index(self)"""
>> +        """someobject[self]"""
>>         return int(self)
> 
> Changing the docstring to say "operator.index(self)" would be the
> clearest solution here.
I disagree.  __add__ is documented as implementing +, not operator.add.

> (Choosing to accept arbitrary index objects as
> integer equivalents is up to the object being indexed, just like
> interpreting slices is - a dict, for example, will never invoke
> __index__ methods).
I honestly don?t know what the best fix is.  We could copy the doc from
datamodel (?called whenever Python needs an integer object (such as in
slicing, or in the built-in bin(), hex() and oct() functions)?).  I?ve
been told on IRC to let Mark Dickison decide how to fix the docstrings
in the numbers module (deleting them being of course an option: magic
methods are documented in the language reference, they don?t need
docstrings).

Regards


From rrr at ronadam.com  Sun Dec 26 19:09:34 2010
From: rrr at ronadam.com (Ron Adam)
Date: Sun, 26 Dec 2010 12:09:34 -0600
Subject: [Python-Dev] r87389 - in python/branches/py3k:
 Doc/library/unittest.rst Lib/unittest/case.py Misc/NEWS
In-Reply-To: <763DCF04-033D-455B-A7BF-9D5813BBDFD4@gmail.com>
References: <20101218200004.99110EE985@mail.python.org>	<20101218214830.1aa8be32@pitrou.net>	<iejdjt$ik0$1@dough.gmane.org>	<AANLkTi=ADkUMc5tQGDv2yniteyfPCPXU=XaCr+GscR+a@mail.gmail.com>	<20101219141355.5ac7cd2f@pitrou.net>	<AANLkTinyvb6+Xf=owdV22j6YoAEVjbTXV2tetQzdD8r4@mail.gmail.com>	<31185718-F276-46CE-82E2-6C6B6EC920E8@gmail.com>	<4D0F536A.8050808@voidspace.org.uk>	<1292852841.3737.25.camel@localhost.localdomain>	<4D0F622E.7050902@voidspace.org.uk>	<1292855497.3737.55.camel@localhost.localdomain>	<AANLkTi=pr227YH_8vdLBFASHYCRMN6RAxrdoF8f6kqkx@mail.gmail.com>	<4D109AEA.7010908@voidspace.org.uk>	<iernlg$cib$1@dough.gmane.org>	<4D14C5C6.9070703@voidspace.org.uk>
	<if2qc5$qjf$1@dough.gmane.org>
	<763DCF04-033D-455B-A7BF-9D5813BBDFD4@gmail.com>
Message-ID: <4D1784DE.6020809@ronadam.com>



On 12/24/2010 02:03 PM, Raymond Hettinger wrote:
>
> On Dec 24, 2010, at 10:56 AM, Terry Reedy wrote:
>
>> On 12/24/2010 11:09 AM, Michael Foord wrote:
>>> On 22/12/2010 02:26, Terry Reedy wrote:
>>>> On 12/21/2010 7:17 AM, Michael Foord wrote:
>>>>
>>>> My first priority is that doc and code match.
>>>> Close second is consistency (hence, ease of learning and use) between
>>>> various AssertXs.
>>>>
>>>>> Symmetrical diffs (element in first not in second, element in second not
>>>>> in first) solves the problem without imposing an order on the arguments.
>>>>
>>>> Where applicable, I prefer this as unambiguous output headings.
>>>>
>>> Could you explain what you mean?
>>

>> I was referring back to an output example symmetric diff that was
>> clipped somewhere along the way:
>>
>> In x not in y: ... In y not in x: ...
>>
>> rather than just using -,+ prefixes which are not necessarily
>> self-explanatory. 'Not applicable' would refer to output from difflib
>> which necessarily is ordered.

> FWIW, I think + and - prefixes are much better for diffs that some
> made-up verbiage.  People are used to seeing diffs with + and -.
> Anything else will be so contrived that it's net effect will be to make
> the output confusing and hard to interpret.

Agree.


> If you want, add two lines of explanation before the diff: + means "in
> x, not in y" -  means "in y, not it x"
>
> The notion  of "making symmetric" can easily get carried too far, which
> a corresponding loss of useability.

I agree with this also.

I don't understand the effort to make the tests be symmetric when many of 
the tests are non-symmetric.  (see list below)

I think the terms expected and actual are fine and help more than they 
hurt.  I think of these as "actual result" and "expected result". A clearer 
terminology might be "expr" and "expected_result".

Where a tests can be used *as if* they are symmetric, but the diff context 
is reversed, I think that that is ok.  It just needs a entry in the docs 
that says that will happen if you do it. That won't break tests already 
written.

Also notice (in the list below) that the use of 'a' and 'b' do not indicate 
a test is symmetric, but instead are used where they are *not-symmetric*. 
First and second could be used for those, but I think 'a' and 'b' have less 
mental luggage when it comes to visually seeing the meaning of the method 
signature in those cases.

Tests where the order is not important usually use numbered but like 
arguments, such as "expr1" and "expr2" or "list1" and "list2".  This makes 
sense to me.  "obj1" and "obj2" are just two objects.

The terms "x in y" and "x not in y" look like what you should get from 
containment or regex asserts.

I guess what I'm try to say is think of the whole picture when trying to 
make improvements like these, an idea that works for one or two things may 
not scale well.

Cheers,
    Ron


Non-symmetric assert methods.

assertDictContainsSubset(self, expected, actual, msg=None)
assertFalse(self, expr, msg=None)
assertGreater(self, a, b, msg=None)
assertGreaterEqual(self, a, b, msg=None)
assertIn(self, member, container, msg=None)
assertIsInstance(self, obj, cls, msg=None)
assertIsNone(self, obj, msg=None)
assertIsNotNone(self, obj, msg=None)
assertLess(self, a, b, msg=None)
assertLessEqual(self, a, b, msg=None)
assertNotIn(self, member, container, msg=None)
assertIsInstance(self, obj, cls, msg=None)
assertIsNone(self, obj, msg=None)
assertIsNotNone(self, obj, msg=None)
assertNotIn(self, member, container, msg=None)
assertNotIsInstance(self, obj, cls, msg=None)
assertRegex(self, text, expected_regex, msg=None)
assertNotRegexMatches(self, text, unexpected_regex, msg=None)
assertRaises(self, excClass, callableObj=None, *args, **kwargs)
assertRaisesRegex(self, expected_exception, expected_regex,
                    callable_obj=None, *args, **kwargs)
assertRegex(self, text, expected_regex, msg=None)
assertTrue(self, expr, msg=None)
assertWarns(self, expected_warning, callable_obj=None, *args, **kwargs)
assertWarnsRegex(self, expected_warning, expected_regex,
                   callable_obj=None, *args, **kwargs)


From rrr at ronadam.com  Sun Dec 26 19:09:34 2010
From: rrr at ronadam.com (Ron Adam)
Date: Sun, 26 Dec 2010 12:09:34 -0600
Subject: [Python-Dev] r87389 - in python/branches/py3k:
 Doc/library/unittest.rst Lib/unittest/case.py Misc/NEWS
In-Reply-To: <763DCF04-033D-455B-A7BF-9D5813BBDFD4@gmail.com>
References: <20101218200004.99110EE985@mail.python.org>	<20101218214830.1aa8be32@pitrou.net>	<iejdjt$ik0$1@dough.gmane.org>	<AANLkTi=ADkUMc5tQGDv2yniteyfPCPXU=XaCr+GscR+a@mail.gmail.com>	<20101219141355.5ac7cd2f@pitrou.net>	<AANLkTinyvb6+Xf=owdV22j6YoAEVjbTXV2tetQzdD8r4@mail.gmail.com>	<31185718-F276-46CE-82E2-6C6B6EC920E8@gmail.com>	<4D0F536A.8050808@voidspace.org.uk>	<1292852841.3737.25.camel@localhost.localdomain>	<4D0F622E.7050902@voidspace.org.uk>	<1292855497.3737.55.camel@localhost.localdomain>	<AANLkTi=pr227YH_8vdLBFASHYCRMN6RAxrdoF8f6kqkx@mail.gmail.com>	<4D109AEA.7010908@voidspace.org.uk>	<iernlg$cib$1@dough.gmane.org>	<4D14C5C6.9070703@voidspace.org.uk>
	<if2qc5$qjf$1@dough.gmane.org>
	<763DCF04-033D-455B-A7BF-9D5813BBDFD4@gmail.com>
Message-ID: <4D1784DE.6020809@ronadam.com>



On 12/24/2010 02:03 PM, Raymond Hettinger wrote:
>
> On Dec 24, 2010, at 10:56 AM, Terry Reedy wrote:
>
>> On 12/24/2010 11:09 AM, Michael Foord wrote:
>>> On 22/12/2010 02:26, Terry Reedy wrote:
>>>> On 12/21/2010 7:17 AM, Michael Foord wrote:
>>>>
>>>> My first priority is that doc and code match.
>>>> Close second is consistency (hence, ease of learning and use) between
>>>> various AssertXs.
>>>>
>>>>> Symmetrical diffs (element in first not in second, element in second not
>>>>> in first) solves the problem without imposing an order on the arguments.
>>>>
>>>> Where applicable, I prefer this as unambiguous output headings.
>>>>
>>> Could you explain what you mean?
>>

>> I was referring back to an output example symmetric diff that was
>> clipped somewhere along the way:
>>
>> In x not in y: ... In y not in x: ...
>>
>> rather than just using -,+ prefixes which are not necessarily
>> self-explanatory. 'Not applicable' would refer to output from difflib
>> which necessarily is ordered.

> FWIW, I think + and - prefixes are much better for diffs that some
> made-up verbiage.  People are used to seeing diffs with + and -.
> Anything else will be so contrived that it's net effect will be to make
> the output confusing and hard to interpret.

Agree.


> If you want, add two lines of explanation before the diff: + means "in
> x, not in y" -  means "in y, not it x"
>
> The notion  of "making symmetric" can easily get carried too far, which
> a corresponding loss of useability.

I agree with this also.

I don't understand the effort to make the tests be symmetric when many of 
the tests are non-symmetric.  (see list below)

I think the terms expected and actual are fine and help more than they 
hurt.  I think of these as "actual result" and "expected result". A clearer 
terminology might be "expr" and "expected_result".

Where a tests can be used *as if* they are symmetric, but the diff context 
is reversed, I think that that is ok.  It just needs a entry in the docs 
that says that will happen if you do it. That won't break tests already 
written.

Also notice (in the list below) that the use of 'a' and 'b' do not indicate 
a test is symmetric, but instead are used where they are *not-symmetric*. 
First and second could be used for those, but I think 'a' and 'b' have less 
mental luggage when it comes to visually seeing the meaning of the method 
signature in those cases.

Tests where the order is not important usually use numbered but like 
arguments, such as "expr1" and "expr2" or "list1" and "list2".  This makes 
sense to me.  "obj1" and "obj2" are just two objects.

The terms "x in y" and "x not in y" look like what you should get from 
containment or regex asserts.

I guess what I'm try to say is think of the whole picture when trying to 
make improvements like these, an idea that works for one or two things may 
not scale well.

Cheers,
    Ron


Non-symmetric assert methods.

assertDictContainsSubset(self, expected, actual, msg=None)
assertFalse(self, expr, msg=None)
assertGreater(self, a, b, msg=None)
assertGreaterEqual(self, a, b, msg=None)
assertIn(self, member, container, msg=None)
assertIsInstance(self, obj, cls, msg=None)
assertIsNone(self, obj, msg=None)
assertIsNotNone(self, obj, msg=None)
assertLess(self, a, b, msg=None)
assertLessEqual(self, a, b, msg=None)
assertNotIn(self, member, container, msg=None)
assertIsInstance(self, obj, cls, msg=None)
assertIsNone(self, obj, msg=None)
assertIsNotNone(self, obj, msg=None)
assertNotIn(self, member, container, msg=None)
assertNotIsInstance(self, obj, cls, msg=None)
assertRegex(self, text, expected_regex, msg=None)
assertNotRegexMatches(self, text, unexpected_regex, msg=None)
assertRaises(self, excClass, callableObj=None, *args, **kwargs)
assertRaisesRegex(self, expected_exception, expected_regex,
                    callable_obj=None, *args, **kwargs)
assertRegex(self, text, expected_regex, msg=None)
assertTrue(self, expr, msg=None)
assertWarns(self, expected_warning, callable_obj=None, *args, **kwargs)
assertWarnsRegex(self, expected_warning, expected_regex,
                   callable_obj=None, *args, **kwargs)



From svenbrauch at googlemail.com  Sun Dec 26 22:41:41 2010
From: svenbrauch at googlemail.com (Sven Brauch)
Date: Sun, 26 Dec 2010 22:41:41 +0100
Subject: [Python-Dev] Column offsets for attribute nodes
Message-ID: <AANLkTikQgHfL4SC5jguHHkps48-M2vHWWkwOHFnVDqSD@mail.gmail.com>

Hi there,

I recently filed a feature request in the tracker to change the
behaviour of the parser in terms of setting ranges on attribute AST
nodes, because I'm working on an application which needs more
information than is currently provided. I suggested to change the
behaviour from
foo.bar.baz # <- foo is said to start at column 0, bar at 0 and baz at
0 (current)
to
foo.bar.baz # <- foo starts at 0, bar at 3 and baz at 7 (suggestion)

In that discussion, there's been different opinions about which
behaviour is better; main arguments were "consistency" for the current
and "usefulness" for the suggested behaviour. It has been proposed to
ask the question on this list, that's why I'm doing that now. :)
The thread can be found here: http://bugs.python.org/issue10769

So, which version do you think to be better: the current one or the
suggested one?

Best regards,
Sven

From victor.stinner at haypocalc.com  Sun Dec 26 23:57:10 2010
From: victor.stinner at haypocalc.com (Victor Stinner)
Date: Sun, 26 Dec 2010 23:57:10 +0100
Subject: [Python-Dev] Fault handler updated, now disabled by default
In-Reply-To: <20101226141049.5919.1054241426.divmod.xquotient.369@localhost.localdomain>
References: <1293070093.12866.14.camel@marge> <4D12B5E1.7060608@v.loewis.de>
	<1293124842.26635.3.camel@marge> <4D139395.3000709@v.loewis.de>
	<if0dag$5a0$1@dough.gmane.org> <1293157279.16757.3.camel@marge>
	<4D14993D.2040803@voidspace.org.uk> <1293314280.21272.183.camel@marge>
	<4D1670B6.6000801@netwok.org>
	<20101226141049.5919.1054241426.divmod.xquotient.369@localhost.localdomain>
Message-ID: <1293404230.6036.0.camel@marge>

Le dimanche 26 d?cembre 2010 ? 14:10 +0000, exarkun at twistedmatrix.com a
?crit :
> On 25 Dec, 10:31 pm, merwok at netwok.org wrote:
> >>faulthandler is a module: enable the handler is simple as "import
> >>faulthandler".
> >
> >That sounds like a source of unwanted behavior (aka problems) if the
> >handler is enabled by  1Cpydoc faulthandler 1D or by a pkgutil walk.  You
> >may want to consider using a function to enable the functionality (and
> >add one to disable it).
> 
> Enormous +1.

I don't know pkgutil. How does it work? In which case would it load the
faulthandler module?

faulthandler is currently only written in C.

Victor


From benjamin at python.org  Mon Dec 27 00:34:55 2010
From: benjamin at python.org (Benjamin Peterson)
Date: Sun, 26 Dec 2010 17:34:55 -0600
Subject: [Python-Dev] Column offsets for attribute nodes
In-Reply-To: <AANLkTikQgHfL4SC5jguHHkps48-M2vHWWkwOHFnVDqSD@mail.gmail.com>
References: <AANLkTikQgHfL4SC5jguHHkps48-M2vHWWkwOHFnVDqSD@mail.gmail.com>
Message-ID: <AANLkTi=z+ZKioCqcN3MB9K-Xbw=WAO+ujtmid-K8jepD@mail.gmail.com>

2010/12/26 Sven Brauch <svenbrauch at googlemail.com>:
> Hi there,
>
> I recently filed a feature request in the tracker to change the
> behaviour of the parser in terms of setting ranges on attribute AST
> nodes, because I'm working on an application which needs more
> information than is currently provided. I suggested to change the
> behaviour from
> foo.bar.baz # <- foo is said to start at column 0, bar at 0 and baz at
> 0 (current)
> to
> foo.bar.baz # <- foo starts at 0, bar at 3 and baz at 7 (suggestion)
>
> In that discussion, there's been different opinions about which
> behaviour is better; main arguments were "consistency" for the current
> and "usefulness" for the suggested behaviour. It has been proposed to
> ask the question on this list, that's why I'm doing that now. :)

My argument against this change is that an attribute includes the
expression from which the attribute is being taken. This is consistent
with subscripts and calls, which both have the lineno and col_offset
of their source expressions.



-- 
Regards,
Benjamin

From svenbrauch at googlemail.com  Mon Dec 27 00:52:10 2010
From: svenbrauch at googlemail.com (Sven Brauch)
Date: Mon, 27 Dec 2010 00:52:10 +0100
Subject: [Python-Dev] Column offsets for attribute nodes
In-Reply-To: <AANLkTi=z+ZKioCqcN3MB9K-Xbw=WAO+ujtmid-K8jepD@mail.gmail.com>
References: <AANLkTikQgHfL4SC5jguHHkps48-M2vHWWkwOHFnVDqSD@mail.gmail.com>
	<AANLkTi=z+ZKioCqcN3MB9K-Xbw=WAO+ujtmid-K8jepD@mail.gmail.com>
Message-ID: <AANLkTimiF1YL+S46k=96exZ0An7y7x6HpwHt1-pM3riu@mail.gmail.com>

It should maybe be noted that the proposed patch would change that,
too, so it would be the same behaviour for all three types
(subscripts, calls, and attributes) again. Just more intuitive. :)

2010/12/27 Benjamin Peterson <benjamin at python.org>:
> 2010/12/26 Sven Brauch <svenbrauch at googlemail.com>:
>> Hi there,
>>
>> I recently filed a feature request in the tracker to change the
>> behaviour of the parser in terms of setting ranges on attribute AST
>> nodes, because I'm working on an application which needs more
>> information than is currently provided. I suggested to change the
>> behaviour from
>> foo.bar.baz # <- foo is said to start at column 0, bar at 0 and baz at
>> 0 (current)
>> to
>> foo.bar.baz # <- foo starts at 0, bar at 3 and baz at 7 (suggestion)
>>
>> In that discussion, there's been different opinions about which
>> behaviour is better; main arguments were "consistency" for the current
>> and "usefulness" for the suggested behaviour. It has been proposed to
>> ask the question on this list, that's why I'm doing that now. :)
>
> My argument against this change is that an attribute includes the
> expression from which the attribute is being taken. This is consistent
> with subscripts and calls, which both have the lineno and col_offset
> of their source expressions.
>
>
>
> --
> Regards,
> Benjamin
>

From ncoghlan at gmail.com  Mon Dec 27 01:01:21 2010
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Mon, 27 Dec 2010 10:01:21 +1000
Subject: [Python-Dev] [Python-checkins] r87445 -
	python/branches/py3k/Lib/numbers.py
In-Reply-To: <4D1777D1.3060104@netwok.org>
References: <20101223184133.9DED1EE995@mail.python.org>
	<AANLkTinakrLr3sZSbUfGpVwR+iQouXZ-OgRYQZfBQ2GB@mail.gmail.com>
	<4D1777D1.3060104@netwok.org>
Message-ID: <AANLkTiniiRe=sfN+ZOtFwSmf=FbtVwoybj0HCS8SgRiw@mail.gmail.com>

On Mon, Dec 27, 2010 at 3:13 AM, ?ric Araujo <merwok at netwok.org> wrote:
> Le 24/12/2010 02:08, Nick Coghlan a ?crit :
>> On Fri, Dec 24, 2010 at 4:41 AM, eric.araujo <python-checkins at python.org> wrote:
>>> Fix small inaccuracy: there is no index function
>>
>> Yes, there is, it just isn't a builtin - it lives in the operator module.
> Defining object.__index__ with operator.index seems pretty circular to
> me :) ?http://docs.python.org/dev/reference/datamodel#object.__index__
> does it, but it should be fixed IMO. ?The difference between __int__ and
> __index__ is also not clear.

Yes, the definition in the language reference could definitely be
improved to mention the semantics first, and then reference
operator.index second.

Possible wording "Indicates to the Python interpreter that the object
is semantically equivalent to the returned integer, rather than merely
supporting a possibly lossy coercion to an integer (i.e. as the
__int__ method allows for types like float and decimal.Decimal). This
allows non-builtin objects to be used as sequence indices, elements of
a slice definition, multiplies in sequence repetition, etc. Can be
invoked explicitly from Python code via operator.index()"

Removing the circularity from the definitions of __index__ and
operator.index doesn't have a great deal to do with the docstrings in
numbers.py, though.

>>> ? ? def __index__(self):
>>> - ? ? ? ?"""index(self)"""
>>> + ? ? ? ?"""someobject[self]"""
>>> ? ? ? ? return int(self)
>>
>> Changing the docstring to say "operator.index(self)" would be the
>> clearest solution here.
> I disagree. ?__add__ is documented as implementing +, not operator.add.

That's because "+" is the idiomatic spelling. operator.index *is* the
Python level spelling of obj.__index__() - there is no other way to
spell it (other than calling the method explicitly, which is subtly
different).

>> (Choosing to accept arbitrary index objects as
>> integer equivalents is up to the object being indexed, just like
>> interpreting slices is - a dict, for example, will never invoke
>> __index__ methods).
> I honestly don?t know what the best fix is. ?We could copy the doc from
> datamodel (?called whenever Python needs an integer object (such as in
> slicing, or in the built-in bin(), hex() and oct() functions)?). ?I?ve
> been told on IRC to let Mark Dickison decide how to fix the docstrings
> in the numbers module (deleting them being of course an option: magic
> methods are documented in the language reference, they don?t need
> docstrings).

Indeed. However, as a reference module for the numeric tower, it makes
a certain amount of sense to keep the docstrings in this particular
case.

Cheers,
Nick.

-- 
Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia

From v+python at g.nevcal.com  Mon Dec 27 01:03:06 2010
From: v+python at g.nevcal.com (Glenn Linderman)
Date: Sun, 26 Dec 2010 16:03:06 -0800
Subject: [Python-Dev] Column offsets for attribute nodes
In-Reply-To: <AANLkTikQgHfL4SC5jguHHkps48-M2vHWWkwOHFnVDqSD@mail.gmail.com>
References: <AANLkTikQgHfL4SC5jguHHkps48-M2vHWWkwOHFnVDqSD@mail.gmail.com>
Message-ID: <4D17D7BA.1020207@g.nevcal.com>

On 12/26/2010 1:41 PM, Sven Brauch wrote:
> Hi there,
>
> I recently filed a feature request in the tracker to change the
> behaviour of the parser in terms of setting ranges on attribute AST
> nodes, because I'm working on an application which needs more
> information than is currently provided. I suggested to change the
> behaviour from
> foo.bar.baz #<- foo is said to start at column 0, bar at 0 and baz at
> 0 (current)
> to
> foo.bar.baz #<- foo starts at 0, bar at 3 and baz at 7 (suggestion)
>
> In that discussion, there's been different opinions about which
> behaviour is better; main arguments were "consistency" for the current
> and "usefulness" for the suggested behaviour. It has been proposed to
> ask the question on this list, that's why I'm doing that now. :)
> The thread can be found here: http://bugs.python.org/issue10769
>
> So, which version do you think to be better: the current one or the
> suggested one?

The current one is better, but maybe bar at 4 and baz at 8 would be even 
better.

In other words, I don't think pointing at the "." is useful?  Was that 
your intention?
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20101226/d4cbe02d/attachment-0001.html>

From ncoghlan at gmail.com  Mon Dec 27 01:07:50 2010
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Mon, 27 Dec 2010 10:07:50 +1000
Subject: [Python-Dev] Fault handler updated, now disabled by default
In-Reply-To: <1293404230.6036.0.camel@marge>
References: <1293070093.12866.14.camel@marge> <4D12B5E1.7060608@v.loewis.de>
	<1293124842.26635.3.camel@marge> <4D139395.3000709@v.loewis.de>
	<if0dag$5a0$1@dough.gmane.org> <1293157279.16757.3.camel@marge>
	<4D14993D.2040803@voidspace.org.uk>
	<1293314280.21272.183.camel@marge> <4D1670B6.6000801@netwok.org>
	<20101226141049.5919.1054241426.divmod.xquotient.369@localhost.localdomain>
	<1293404230.6036.0.camel@marge>
Message-ID: <AANLkTi=bfT31LXx6OG2QV-NNAv2Jyebg0y2=MONhvw4D@mail.gmail.com>

On Mon, Dec 27, 2010 at 8:57 AM, Victor Stinner
<victor.stinner at haypocalc.com> wrote:
> Le dimanche 26 d?cembre 2010 ? 14:10 +0000, exarkun at twistedmatrix.com a
> ?crit :
>> On 25 Dec, 10:31 pm, merwok at netwok.org wrote:
>> >>faulthandler is a module: enable the handler is simple as "import
>> >>faulthandler".
>> >
>> >That sounds like a source of unwanted behavior (aka problems) if the
>> >handler is enabled by ?1Cpydoc faulthandler 1D or by a pkgutil walk. ?You
>> >may want to consider using a function to enable the functionality (and
>> >add one to disable it).
>>
>> Enormous +1.
>
> I don't know pkgutil. How does it work? In which case would it load the
> faulthandler module?
>
> faulthandler is currently only written in C.

pkgutil includes a function that lets you walk the entire module
heirarchy, implicitly importing everything, including all the builtin
modules. It's one of the reasons doing things as side-effects of
import is considered highly undesirable.

The pydoc tests do this when they bring the (docstring-based)
documentation server up to check its handling of HTTP requests. (we
recently picked up an implicit addition of a logging handler by
concurrent.futures due to this effect).

Cheers,
Nick.

-- 
Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia

From svenbrauch at googlemail.com  Mon Dec 27 01:14:53 2010
From: svenbrauch at googlemail.com (Sven Brauch)
Date: Mon, 27 Dec 2010 01:14:53 +0100
Subject: [Python-Dev] Column offsets for attribute nodes
In-Reply-To: <4D17D7BA.1020207@g.nevcal.com>
References: <AANLkTikQgHfL4SC5jguHHkps48-M2vHWWkwOHFnVDqSD@mail.gmail.com>
	<4D17D7BA.1020207@g.nevcal.com>
Message-ID: <AANLkTinoc6h7E5UA7hCJiAZSNk=SDJim1N1JE2huG0m1@mail.gmail.com>

Hey,

yeah, you're right, that would be even better. Currently it would also
give a 3 for bar in "foo.    bar.baz". I'd be fine with both
solutions, but yours seems more logical.

Bye,
Sven

2010/12/27 Glenn Linderman <v+python at g.nevcal.com>:
> On 12/26/2010 1:41 PM, Sven Brauch wrote:
>
> Hi there,
>
> I recently filed a feature request in the tracker to change the
> behaviour of the parser in terms of setting ranges on attribute AST
> nodes, because I'm working on an application which needs more
> information than is currently provided. I suggested to change the
> behaviour from
> foo.bar.baz # <- foo is said to start at column 0, bar at 0 and baz at
> 0 (current)
> to
> foo.bar.baz # <- foo starts at 0, bar at 3 and baz at 7 (suggestion)
>
> In that discussion, there's been different opinions about which
> behaviour is better; main arguments were "consistency" for the current
> and "usefulness" for the suggested behaviour. It has been proposed to
> ask the question on this list, that's why I'm doing that now. :)
> The thread can be found here: http://bugs.python.org/issue10769
>
> So, which version do you think to be better: the current one or the
> suggested one?
>
> The current one is better, but maybe bar at 4 and baz at 8 would be even
> better.
>
> In other words, I don't think pointing at the "." is useful?? Was that your
> intention?
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> http://mail.python.org/mailman/options/python-dev/svenbrauch%40googlemail.com
>
>

From ncoghlan at gmail.com  Mon Dec 27 01:15:45 2010
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Mon, 27 Dec 2010 10:15:45 +1000
Subject: [Python-Dev] Range __contains__ and objects with __index__ methods
Message-ID: <AANLkTinvY0PzvC0w4uxb8xNFmp2H4s1e=mZSOZMyazZi@mail.gmail.com>

Starting in Python 3.2, range() supports fast containment checking for
integers (i.e. based on an O(1) arithmetic calculation rather than an
O(N) iteration through the entire sequence).

Currently, this fast path ignores objects that implement __index__ -
they are relegated to the slow iterative search.

This seems wrong to me - the semantics of __index__ are such that it
is meant to be used for objects that are alternative representations
of the corresponding Python integers (e.g. numpy scalars, or integers
that use a particular number of bits in memory). Under that
interpretation, if an object provides __index__, we should use the
fast path instead of calling __eq__ multiple times.

Thoughts?

Regards,
Nick.

-- 
Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia

From ncoghlan at gmail.com  Mon Dec 27 01:32:58 2010
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Mon, 27 Dec 2010 10:32:58 +1000
Subject: [Python-Dev] Column offsets for attribute nodes
In-Reply-To: <AANLkTi=z+ZKioCqcN3MB9K-Xbw=WAO+ujtmid-K8jepD@mail.gmail.com>
References: <AANLkTikQgHfL4SC5jguHHkps48-M2vHWWkwOHFnVDqSD@mail.gmail.com>
	<AANLkTi=z+ZKioCqcN3MB9K-Xbw=WAO+ujtmid-K8jepD@mail.gmail.com>
Message-ID: <AANLkTikb14jFXFnFALAEWCFjjK5fAuHHRCCs7VcQbvqy@mail.gmail.com>

On Mon, Dec 27, 2010 at 9:34 AM, Benjamin Peterson <benjamin at python.org> wrote:
> 2010/12/26 Sven Brauch <svenbrauch at googlemail.com>:
>> In that discussion, there's been different opinions about which
>> behaviour is better; main arguments were "consistency" for the current
>> and "usefulness" for the suggested behaviour. It has been proposed to
>> ask the question on this list, that's why I'm doing that now. :)
>
> My argument against this change is that an attribute includes the
> expression from which the attribute is being taken. This is consistent
> with subscripts and calls, which both have the lineno and col_offset
> of their source expressions.

I'd like to see the impact on existing uses of this information
(primarily SyntaxErrors) before forming a firm opinion, but my initial
inclination is that retaining the column information for the
subattribute names is a better option.

Cheers,
Nick.

-- 
Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia

From tjreedy at udel.edu  Mon Dec 27 02:52:23 2010
From: tjreedy at udel.edu (Terry Reedy)
Date: Sun, 26 Dec 2010 20:52:23 -0500
Subject: [Python-Dev] Range __contains__ and objects with __index__
	methods
In-Reply-To: <AANLkTinvY0PzvC0w4uxb8xNFmp2H4s1e=mZSOZMyazZi@mail.gmail.com>
References: <AANLkTinvY0PzvC0w4uxb8xNFmp2H4s1e=mZSOZMyazZi@mail.gmail.com>
Message-ID: <if8rgl$289$1@dough.gmane.org>

On 12/26/2010 7:15 PM, Nick Coghlan wrote:
> Starting in Python 3.2, range() supports fast containment checking for
> integers (i.e. based on an O(1) arithmetic calculation rather than an
> O(N) iteration through the entire sequence).
>
> Currently, this fast path ignores objects that implement __index__ -
> they are relegated to the slow iterative search.
>
> This seems wrong to me - the semantics of __index__ are such that it
> is meant to be used for objects that are alternative representations
> of the corresponding Python integers (e.g. numpy scalars, or integers
> that use a particular number of bits in memory). Under that
> interpretation, if an object provides __index__, we should use the
> fast path instead of calling __eq__ multiple times.

If I understand, you are proposing 'replacing' o with o.__index__() 
(when possible) and proceeding on the fast path rather than iterating 
the range and comparing o for equality each value in the range (the slow 
path).

I suppose this would change semantics if o != o.__index__().
Equality is not specified in the manual:
"object.__index__(self)
Called to implement operator.index(). Also called whenever Python needs 
an integer object (such as in slicing, or in the built-in bin(), hex() 
and oct() functions). Must return an integer."
However
"operator.__index__(a)
Return a converted to an integer. Equivalent to a.__index__()."
comes close to implying equality (if possible).

What are the actual used of .__index__?

-- 
Terry Jan Reedy


From tjreedy at udel.edu  Mon Dec 27 02:58:34 2010
From: tjreedy at udel.edu (Terry Reedy)
Date: Sun, 26 Dec 2010 20:58:34 -0500
Subject: [Python-Dev] [Python-checkins] r87445 -
	python/branches/py3k/Lib/numbers.py
In-Reply-To: <AANLkTiniiRe=sfN+ZOtFwSmf=FbtVwoybj0HCS8SgRiw@mail.gmail.com>
References: <20101223184133.9DED1EE995@mail.python.org>	<AANLkTinakrLr3sZSbUfGpVwR+iQouXZ-OgRYQZfBQ2GB@mail.gmail.com>	<4D1777D1.3060104@netwok.org>
	<AANLkTiniiRe=sfN+ZOtFwSmf=FbtVwoybj0HCS8SgRiw@mail.gmail.com>
Message-ID: <if8rs8$367$1@dough.gmane.org>

On 12/26/2010 7:01 PM, Nick Coghlan wrote:

> Yes, the definition in the language reference could definitely be
> improved to mention the semantics first, and then reference
> operator.index second.
>
> Possible wording "Indicates to the Python interpreter that the object
> is semantically equivalent to the returned integer, rather than merely
> supporting a possibly lossy coercion to an integer

If that is the intent of __index__, the doc should say so more clearly. 
That clarification would change my answer to your question about range.

 > (i.e. as the
> __int__ method allows for types like float and decimal.Decimal). This
> allows non-builtin objects to be used as sequence indices, elements of
> a slice definition, multiplies in sequence repetition, etc. Can be
> invoked explicitly from Python code via operator.index()"
>
> Removing the circularity from the definitions of __index__ and
> operator.index doesn't have a great deal to do with the docstrings in
> numbers.py, though.

It is both related and needed though. IE, it is hard to answer questions 
about what to to with .index if the intended meaning of .index is not 
very clear ;-).

-- 
Terry Jan Reedy


From martin at v.loewis.de  Mon Dec 27 03:16:11 2010
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Mon, 27 Dec 2010 03:16:11 +0100
Subject: [Python-Dev] Range __contains__ and objects with
	__index__	methods
In-Reply-To: <if8rgl$289$1@dough.gmane.org>
References: <AANLkTinvY0PzvC0w4uxb8xNFmp2H4s1e=mZSOZMyazZi@mail.gmail.com>
	<if8rgl$289$1@dough.gmane.org>
Message-ID: <4D17F6EB.1080907@v.loewis.de>

> What are the actual used of .__index__?

Can you please rephrase this question?

Regards,
Martin

From ncoghlan at gmail.com  Mon Dec 27 07:13:16 2010
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Mon, 27 Dec 2010 16:13:16 +1000
Subject: [Python-Dev] Range __contains__ and objects with __index__
	methods
In-Reply-To: <if8rgl$289$1@dough.gmane.org>
References: <AANLkTinvY0PzvC0w4uxb8xNFmp2H4s1e=mZSOZMyazZi@mail.gmail.com>
	<if8rgl$289$1@dough.gmane.org>
Message-ID: <AANLkTikpVudasSZs8zOBw5dG3BNB0XHmGPuO8NRty5tb@mail.gmail.com>

On Mon, Dec 27, 2010 at 11:52 AM, Terry Reedy <tjreedy at udel.edu> wrote:
> Return a converted to an integer. Equivalent to a.__index__()."
> comes close to implying equality (if possible).
>
> What are the actual used of .__index__?

PEP 357 gives the original rationale - it was to allow integer-like
objects (such as numpy scalars) to be used as sequence indices, as
well as slice parameters.

I would have to grep the source to get a full list of uses, but I
believe it is already used in at least those two cases, as well as for
sequence repetition (via '*') and to identify permitted inputs to
bin/oct/hex.

Cheers,
Nick.

-- 
Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia

From raymond.hettinger at gmail.com  Mon Dec 27 10:43:33 2010
From: raymond.hettinger at gmail.com (Raymond Hettinger)
Date: Mon, 27 Dec 2010 01:43:33 -0800
Subject: [Python-Dev] nonlocal x = value
In-Reply-To: <if4iqo$5b4$1@dough.gmane.org>
References: <ief2tb$i9n$1@dough.gmane.org>	<AANLkTikZi3DEqx7-QgR4wZ337E-OC_Xk7E9NZaqw1jaf@mail.gmail.com>	<iei8i9$qb5$2@dough.gmane.org>	<AANLkTinmYt6or0aOz4bP1UE05T6EXuWyz7PeHcGUr0fT@mail.gmail.com>	<if0cru$2e3$1@dough.gmane.org>	<AANLkTik_D7oZkXTdZNB2=w=wN2gZeO17VTtQRZuNwsKM@mail.gmail.com>
	<4D145DC4.9000107@avl.com> <if4iqo$5b4$1@dough.gmane.org>
Message-ID: <9D8DAD45-36DF-4694-89CA-65C42FA846C3@gmail.com>


On Dec 25, 2010, at 2:59 AM, Stefan Behnel wrote:

> Hrvoje Niksic, 24.12.2010 09:45:
>> On 12/23/2010 10:03 PM, Laurens Van Houtven wrote:
>>> On Thu, Dec 23, 2010 at 9:51 PM, Georg Brandl wrote:
>>>> Yes and no -- there may not be an ambiguity to the parser, but still to
>>>> the human. Except if you disallow the syntax in any case, requiring
>>>> people to write
>>>> 
>>>> nonlocal x = (3, y)
>>>> 
>>>> which is then again inconsistent with ordinary assignment statements.
>>> 
>>> Right -- but (and hence the confusion) I was arguing for not mixing
>>> global/nonlocal with assignment at all, and instead having nonlocal
>>> and global only take one or more names. That would (obviously) remove
>>> any such ambiguity ;-)
>> 
>> I would like to offer the opposing viewpoint: nonlocal x = value is a
>> useful shortcut because nonlocal is used in closure callbacks where
>> brevity matters.
> 
> I doubt that it really matters so much that one line more kills readability. It's still a relatively rare use case after all.
> 
> 
>> The reason nonlocal is introduced is to change the
>> variable, so it makes sense that the two can be done in the same line of
>> code.

FWIW, I'm entirely opposed to doing an assignment in a nonlocal definition.

* It is easily mis-parsed by human (as shown by Georg's examples).
* It looks very much like an initialization of a local variable in many languages,
  but it is not -- the variable has already been initialized in another scope.
* It is not clear how to extend it to multiple variables (which we already allow).
* It is entirely unnecessary.  Just add a real assignment on the following line:
         local x
         x = 3, y
* We've had global declarations for a very long time and never needed
   (or wanted) an assignment for it:
        global x = 3, y
* The purported use case is rare (at best).  Special cases aren't worth breaking the rules.
   And the purported goal (saving one line) isn't much of a payoff.
* The language moratorium is ending but the aversion to non-essential 
   micro-syntax changes persists.
* And, Georg doesn't like it :-)


Raymond

From dickinsm at gmail.com  Mon Dec 27 11:19:20 2010
From: dickinsm at gmail.com (Mark Dickinson)
Date: Mon, 27 Dec 2010 10:19:20 +0000
Subject: [Python-Dev] [Python-checkins] r87445 -
	python/branches/py3k/Lib/numbers.py
In-Reply-To: <AANLkTinakrLr3sZSbUfGpVwR+iQouXZ-OgRYQZfBQ2GB@mail.gmail.com>
References: <20101223184133.9DED1EE995@mail.python.org>
	<AANLkTinakrLr3sZSbUfGpVwR+iQouXZ-OgRYQZfBQ2GB@mail.gmail.com>
Message-ID: <AANLkTinn8-mma-Q9Rrr1SYPAbRiEpuAdLkhqwHSAFAbn@mail.gmail.com>

On Fri, Dec 24, 2010 at 1:08 AM, Nick Coghlan <ncoghlan at gmail.com> wrote:
>> ? ? def __index__(self):
>> - ? ? ? ?"""index(self)"""
>> + ? ? ? ?"""someobject[self]"""
>> ? ? ? ? return int(self)
>
> Changing the docstring to say "operator.index(self)" would be the
> clearest solution here.

Agreed.  Certainly "someobject[self]" isn't right.

(There's also a question about whether __index__ should really be
defaulting to int, but that's another issue...)

Mark

From dickinsm at gmail.com  Mon Dec 27 11:31:54 2010
From: dickinsm at gmail.com (Mark Dickinson)
Date: Mon, 27 Dec 2010 10:31:54 +0000
Subject: [Python-Dev] nonlocal x = value
In-Reply-To: <9D8DAD45-36DF-4694-89CA-65C42FA846C3@gmail.com>
References: <ief2tb$i9n$1@dough.gmane.org>
	<AANLkTikZi3DEqx7-QgR4wZ337E-OC_Xk7E9NZaqw1jaf@mail.gmail.com>
	<iei8i9$qb5$2@dough.gmane.org>
	<AANLkTinmYt6or0aOz4bP1UE05T6EXuWyz7PeHcGUr0fT@mail.gmail.com>
	<if0cru$2e3$1@dough.gmane.org>
	<AANLkTik_D7oZkXTdZNB2=w=wN2gZeO17VTtQRZuNwsKM@mail.gmail.com>
	<4D145DC4.9000107@avl.com> <if4iqo$5b4$1@dough.gmane.org>
	<9D8DAD45-36DF-4694-89CA-65C42FA846C3@gmail.com>
Message-ID: <AANLkTi=FT7NbdLKCBo8Ft-O7-K=d9U8oi-L0AK-kKKOi@mail.gmail.com>

On Mon, Dec 27, 2010 at 9:43 AM, Raymond Hettinger
<raymond.hettinger at gmail.com> wrote:
> FWIW, I'm entirely opposed to doing an assignment in a nonlocal definition.
> [...]

-1 for assignment in nonlocal and global statements from me, too.

Mark

From ncoghlan at gmail.com  Mon Dec 27 13:25:03 2010
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Mon, 27 Dec 2010 22:25:03 +1000
Subject: [Python-Dev] nonlocal x = value
In-Reply-To: <AANLkTi=FT7NbdLKCBo8Ft-O7-K=d9U8oi-L0AK-kKKOi@mail.gmail.com>
References: <ief2tb$i9n$1@dough.gmane.org>
	<AANLkTikZi3DEqx7-QgR4wZ337E-OC_Xk7E9NZaqw1jaf@mail.gmail.com>
	<iei8i9$qb5$2@dough.gmane.org>
	<AANLkTinmYt6or0aOz4bP1UE05T6EXuWyz7PeHcGUr0fT@mail.gmail.com>
	<if0cru$2e3$1@dough.gmane.org>
	<AANLkTik_D7oZkXTdZNB2=w=wN2gZeO17VTtQRZuNwsKM@mail.gmail.com>
	<4D145DC4.9000107@avl.com> <if4iqo$5b4$1@dough.gmane.org>
	<9D8DAD45-36DF-4694-89CA-65C42FA846C3@gmail.com>
	<AANLkTi=FT7NbdLKCBo8Ft-O7-K=d9U8oi-L0AK-kKKOi@mail.gmail.com>
Message-ID: <AANLkTikj37h5oA0=sq63KDOr4nGRE4XwzXQyqWH9+ubT@mail.gmail.com>

On Mon, Dec 27, 2010 at 8:31 PM, Mark Dickinson <dickinsm at gmail.com> wrote:
> On Mon, Dec 27, 2010 at 9:43 AM, Raymond Hettinger
> <raymond.hettinger at gmail.com> wrote:
>> FWIW, I'm entirely opposed to doing an assignment in a nonlocal definition.
>> [...]
>
> -1 for assignment in nonlocal and global statements from me, too.

Indeed. The PEP should be updated to be clear that that part was never
implemented (referencing Raymond's post for the reasons why).

Cheers,
Nick.

-- 
Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia

From dickinsm at gmail.com  Mon Dec 27 13:44:26 2010
From: dickinsm at gmail.com (Mark Dickinson)
Date: Mon, 27 Dec 2010 12:44:26 +0000
Subject: [Python-Dev] Range __contains__ and objects with __index__
	methods
In-Reply-To: <AANLkTin+xeLQduUNrTGOXt_43oZ-f1bVMptd99xYDFXm@mail.gmail.com>
References: <AANLkTinvY0PzvC0w4uxb8xNFmp2H4s1e=mZSOZMyazZi@mail.gmail.com>
	<AANLkTi=qV5mp1U2Z=Z7cmbP5a4j+AMj=OB61NYJYb2Bq@mail.gmail.com>
	<AANLkTimON=FkV6qO9zj2gBUzEtk2C+aDiL4DJ=s7bTJw@mail.gmail.com>
	<AANLkTin+xeLQduUNrTGOXt_43oZ-f1bVMptd99xYDFXm@mail.gmail.com>
Message-ID: <AANLkTik8wRvCkw13DwJ7S+Bx1nF1CMJZwee6XqJD1fh=@mail.gmail.com>

Bah.  I meant to send this to the list.  (I suspect that Nick also
meant to send his reply to the list.)

On Mon, Dec 27, 2010 at 12:43 PM, Mark Dickinson <dickinsm at gmail.com> wrote:
> On Mon, Dec 27, 2010 at 12:23 PM, Nick Coghlan <ncoghlan at gmail.com> wrote:
>> The symmetry only breaks for a class that breaks the invariant:
>>
>> ? x == operator.index(x)
>>
>> Failing to guarantee that invariant seems to violate the whole spirit
>> of operator.index().
>
> Agreed. (Though one also needs transitivity, symmetry, etc., of
> equality, since what we really need is that x == y is interchangeable
> with operator.index(x) == y.)
>
>> Perhaps this is what we should be documenting as
>> the true distinguishing feature between the intended semantics of
>> __index__ and the semantics of __int__?
>
> Hmm. ?Perhaps. ?For me, this doesn't fully capture the intent of
> __index__, though.
>
> So the semantics (ignoring performance issues for now) would become:
>
> def range_contains(self, x)
> ? ?try:
> ? ? ? ?i = operator.index(x)
> ? ?except TypeError:
> ? ? ? ?i = x
> ? ?return any(i == y for y in self)
>
> ?
>
> This change sounds fine to me; ?it would be good to have a
> documentation note somewhere indicating that the range implementation
> assumes x == index(x), though. ?This might belong in the range
> documentation, or perhaps the __index__ method documentation could
> indicate that some builtins might have unpredictable behaviour if the
> identity is violated.
>
> Mark
>

From dickinsm at gmail.com  Mon Dec 27 13:46:21 2010
From: dickinsm at gmail.com (Mark Dickinson)
Date: Mon, 27 Dec 2010 12:46:21 +0000
Subject: [Python-Dev] Fwd: Range __contains__ and objects with __index__
	methods
In-Reply-To: <AANLkTi=qV5mp1U2Z=Z7cmbP5a4j+AMj=OB61NYJYb2Bq@mail.gmail.com>
References: <AANLkTinvY0PzvC0w4uxb8xNFmp2H4s1e=mZSOZMyazZi@mail.gmail.com>
	<AANLkTi=qV5mp1U2Z=Z7cmbP5a4j+AMj=OB61NYJYb2Bq@mail.gmail.com>
Message-ID: <AANLkTinPPk4sS-w-1=9mKNhjEzQKeEkyrKHfH0KGPa2Z@mail.gmail.com>

.. and here's my original reply to Nick, which was also intended to go
to the list.  Sorry, folks.

Mark



---------- Forwarded message ----------
From: Mark Dickinson <dickinsm at gmail.com>
Date: Mon, Dec 27, 2010 at 10:27 AM
Subject: Re: [Python-Dev] Range __contains__ and objects with __index__ methods
To: Nick Coghlan <ncoghlan at gmail.com>


On Mon, Dec 27, 2010 at 12:15 AM, Nick Coghlan <ncoghlan at gmail.com> wrote:
> Starting in Python 3.2, range() supports fast containment checking for
> integers (i.e. based on an O(1) arithmetic calculation rather than an
> O(N) iteration through the entire sequence).
>
> Currently, this fast path ignores objects that implement __index__ -
> they are relegated to the slow iterative search.
>
> This seems wrong to me [...]

Is seems to me that there are two separate issues here. Namely, (1)
Determine the semantics of 'x in range(...)' for an object x that
implements __index__, and (2) implement the containment check
efficiently.

At the moment, it looks as though the __index__ method is ignored
entirely for the containment check, so you're proposing a change in
semantics. ?If/when that change in semantics is made, fixing up the
code to do the containment check efficiently seems like it should be a
straightforward task.

With the proposed change in semantics, the simple definition (x in
range(...) iff x == y for some element y of range(...)) no longer
holds. ?I'm not convinced (but not unconvinced either) that it's worth
breaking that simplicity.

Mark

From g.brandl at gmx.net  Mon Dec 27 15:01:16 2010
From: g.brandl at gmx.net (Georg Brandl)
Date: Mon, 27 Dec 2010 15:01:16 +0100
Subject: [Python-Dev] nonlocal x = value
In-Reply-To: <9D8DAD45-36DF-4694-89CA-65C42FA846C3@gmail.com>
References: <ief2tb$i9n$1@dough.gmane.org>	<AANLkTikZi3DEqx7-QgR4wZ337E-OC_Xk7E9NZaqw1jaf@mail.gmail.com>	<iei8i9$qb5$2@dough.gmane.org>	<AANLkTinmYt6or0aOz4bP1UE05T6EXuWyz7PeHcGUr0fT@mail.gmail.com>	<if0cru$2e3$1@dough.gmane.org>	<AANLkTik_D7oZkXTdZNB2=w=wN2gZeO17VTtQRZuNwsKM@mail.gmail.com>	<4D145DC4.9000107@avl.com>
	<if4iqo$5b4$1@dough.gmane.org>
	<9D8DAD45-36DF-4694-89CA-65C42FA846C3@gmail.com>
Message-ID: <ifa69d$h9g$1@dough.gmane.org>

Am 27.12.2010 10:43, schrieb Raymond Hettinger:

> FWIW, I'm entirely opposed to doing an assignment in a nonlocal definition.
> 
> * It is easily mis-parsed by human (as shown by Georg's examples).
> * It looks very much like an initialization of a local variable in many languages,
>   but it is not -- the variable has already been initialized in another scope.

Hah, and we'd probably get requests for a mandatory declaration of locals, ala

local a = 5

for reasons of "consistency" or "explicit is better than implicit".

Georg



From vinay_sajip at yahoo.co.uk  Mon Dec 27 16:29:22 2010
From: vinay_sajip at yahoo.co.uk (Vinay Sajip)
Date: Mon, 27 Dec 2010 15:29:22 +0000 (UTC)
Subject: [Python-Dev] Loggers in the stdlib and logging configuration APIs
Message-ID: <loom.20101227T162842-639@post.gmane.org>

The logging configuration calls fileConfig and dictConfig disable all existing
loggers, and enable only loggers explicitly named in the configuration (and
their children). Although there is a disable_existing_loggers option for each
configuration API, which can be used to prevent disabling of existing loggers,
the default value of this option is True (for backward compatibility) and so
existing ("old") loggers are disabled by default.

This can lead to unexpected behaviour when loggers are used by the standard
library. A recent issue, 

http://bugs.python.org/issue10626

relates to test_concurrent_futures failing, but only when exercised twice with a
test_logging call in between, as in

regrtest.py test_concurrent_futures test_logging test_concurrent_futures

I've fixed test_logging so that it remembers the disabled state of existing
loggers and restores it, and now the above test works fine. However, since
recent changes in logging, we are expecting to use it to e.g. display messages
on stderr when exceptions can't be raised in stdlib code. Thus, I think I may
need to change the way disabling of loggers works, so I wanted to get some
feedback from python-dev both about possible approaches and when to apply them
(With 3.2 so close to release I'm not proposing to do anything precipitate, just
raising the issue for discussion).

Essentially, some or all of the loggers in the stdlib itself should perhaps be
immune to the "disable existing loggers" logic. Otherwise, a fileConfig() or
dictConfig() call with default semantics will prevent a stdlib message (logged
when exceptions can't be raised) from being displayed, unless a user explicitly
names those stdlib loggers in their configuration. This is not practical because
users would need to update their configurations whenever a new logger appeared
in the stdlib. Some possible approaches are:

1. Change the default configuration behaviour so that by default, existing
loggers are NOT disabled. This is of course strongly backwards-incompatible, but
would it make sense to consider it for 3.3? Perhaps we should add a warning?
2. Change the disabling logic so that it never disables stdlib loggers. This
raises the question of how to identify them, and there are several loggers in
the stdlib in 3.2:

     "py.warnings" (used by the logging package when warnings are redirected to
logging),
     "concurrent.futures",
     "http.cookiejar",
     "multiprocessing"

     However, it would be good to have some consistency of naming stdlib loggers
- perhaps using __name__ as is recommended practice for library and application
developers, but with a prefix such as "py." to indicate that it's a part of the
stdlib. This would make the disabling logic change easier as it could just check
the "py." prefix rather than have an internal list of logger names, which would
need more maintenance and be more error-prone.

There are quite possibly other approaches I haven't thought of. I'd be grateful
for your suggestions, as I'd like to minimise the impact on users, while making
it easy to add more logging to the stdlib.

Regards,

Vinay Sajip


From ethan at stoneleaf.us  Mon Dec 27 20:37:38 2010
From: ethan at stoneleaf.us (Ethan Furman)
Date: Mon, 27 Dec 2010 11:37:38 -0800
Subject: [Python-Dev] Issue #6210: Exception Chaining missing method for
	suppressing context
Message-ID: <4D18EB02.9040409@stoneleaf.us>

I see the last comment was added on the 4th.

Where should continued discussion take place -- bug-tracker, python-dev, 
... ?

~Ethan~

From brian.curtin at gmail.com  Mon Dec 27 20:38:10 2010
From: brian.curtin at gmail.com (Brian Curtin)
Date: Mon, 27 Dec 2010 13:38:10 -0600
Subject: [Python-Dev] Issue #6210: Exception Chaining missing method for
 suppressing context
In-Reply-To: <4D18EB02.9040409@stoneleaf.us>
References: <4D18EB02.9040409@stoneleaf.us>
Message-ID: <AANLkTimf0DC8kbdEvPxbHK6oGu6PK1oGiPRASfw-H=iV@mail.gmail.com>

On Mon, Dec 27, 2010 at 13:37, Ethan Furman <ethan at stoneleaf.us> wrote:

> I see the last comment was added on the 4th.
>
> Where should continued discussion take place -- bug-tracker, python-dev,
> ... ?
>
> ~Ethan~


Preferably on the bug tracker as to keep any discussion attached to the
issue. If the discussion becomes stalled for reasons other than time, a
discussion here might be warranted.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20101227/524a1971/attachment.html>

From v+python at g.nevcal.com  Mon Dec 27 20:41:39 2010
From: v+python at g.nevcal.com (Glenn Linderman)
Date: Mon, 27 Dec 2010 11:41:39 -0800
Subject: [Python-Dev] Loggers in the stdlib and logging configuration
 APIs
In-Reply-To: <loom.20101227T162842-639@post.gmane.org>
References: <loom.20101227T162842-639@post.gmane.org>
Message-ID: <4D18EBF3.9050202@g.nevcal.com>

On 12/27/2010 7:29 AM, Vinay Sajip wrote:
> The logging configuration calls fileConfig and dictConfig disable all existing
> loggers, and enable only loggers explicitly named in the configuration (and
> their children). Although there is a disable_existing_loggers option for each
> configuration API, which can be used to prevent disabling of existing loggers,
> the default value of this option is True (for backward compatibility) and so
> existing ("old") loggers are disabled by default.

Good that you recognized the bigger issue than 10626, and are wanting to 
address it.

It would be interesting to hear the use case for calling fileConfig 
and/or dictConfig and/or basicConfig more than once anyway (although I 
guess basicConfig doesn't disable anything).  That might shed some light 
on the approach that should be taken.

My "intuitive" thought about the matter is that loggers, once enabled, 
shouldn't be disabled except explicitly by name, not implicitly by a 
flag parameter.  So I'd expect a parameter that takes a list of loggers 
to disable, rather than a flag whose default value disables all of 
them.  Of course this doesn't address the how and why of backward 
compatibility needs.

Part of the problem, perhaps, is that stdlib loggers might be enabled 
before the application gets to its logging configuration call, and those 
would then be disabled accidentally.

As I prepare to add logging to my web application this week, I've only 
found the need to initialize logging in one place and at one time... I 
doubt that any of the loggers you mentioned as being in the 3.2 stdlib 
will have been enabled by then, so I'll not have a problem, but if they 
proliferate, I could eventually in future versions.  Granted, my 
application will likely be one of the simpler applications of logging, 
at least initially.

While I agree it would be nice to have a stdlib logger naming convention 
of some sort, I don't think the naming convention should be used to 
implicitly not disable particular loggers.  The only difference between 
stdlib and 3rd party libraries is whether they've become stable enough 
and useful enough for the whole Python community; many 3rd party 
libraries are quite stable enough and useful enough for particular 
applications, and could have similar issues as the stdlib regarding 
logger configuration.

Having an API to disable loggers using an explicit list would require a 
corresponding API to obtain from logging the list of currently enabled 
loggers, and perhaps currently disabled ones as well?  That way, if an 
application truly wanted to disable all currently enabled loggers, they 
would have an easy way to know the list... and perhaps the list of 
loggers they disable during configuration should be the first thing that 
is logged as a result of the xxxConfig API?  Then, at least, they would 
not be ignorant of the accidental disabling.   I guess even if the 
parameter were left as a default flag to turn off all loggers, if the 
list were logged, that would be alerting.  But some applications may not 
expect such a log entry.... but maybe that is the least instrusive 
backwards-compatible action that can be taken, because if an application 
does configure logging, they likely do expect to have log messages to 
view/process.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20101227/8726d6ee/attachment.html>

From svenbrauch at googlemail.com  Mon Dec 27 20:42:59 2010
From: svenbrauch at googlemail.com (Sven Brauch)
Date: Mon, 27 Dec 2010 20:42:59 +0100
Subject: [Python-Dev] Column offsets for attribute nodes
In-Reply-To: <AANLkTikb14jFXFnFALAEWCFjjK5fAuHHRCCs7VcQbvqy@mail.gmail.com>
References: <AANLkTikQgHfL4SC5jguHHkps48-M2vHWWkwOHFnVDqSD@mail.gmail.com>
	<AANLkTi=z+ZKioCqcN3MB9K-Xbw=WAO+ujtmid-K8jepD@mail.gmail.com>
	<AANLkTikb14jFXFnFALAEWCFjjK5fAuHHRCCs7VcQbvqy@mail.gmail.com>
Message-ID: <AANLkTin8ZbmmX3NbJZVBxnrzRHC1GHuti340tpLhG-78@mail.gmail.com>

Glenn wrote:
> [...] but maybe bar at 4 and baz at 8 would be even better.
> In other words, I don't think pointing at the "." is useful? [...]

Hi,

here's a patch which results in what Glenn Linderman suggested:

--- python-orig/Python/ast.c 2010-10-19 03:22:07.000000000 +0200
+++ python-new/Python/ast.c 2010-12-27 20:33:55.000000000 +0100
@@ -1615,7 +1615,7 @@
         if (!attr_id)
             return NULL;
         return Attribute(left_expr, attr_id, Load,
-                         LINENO(n), n->n_col_offset, c->c_arena);
+                         LINENO(CHILD(n, 1)), CHILD(n,
1)->n_col_offset, c->c_arena);
     }
     else {
         REQ(CHILD(n, 0), LSQB);
@@ -1742,8 +1742,6 @@
         tmp = ast_for_trailer(c, ch, e);
         if (!tmp)
             return NULL;
-        tmp->lineno = e->lineno;
-        tmp->col_offset = e->col_offset;
         e = tmp;
     }
     if (TYPE(CHILD(n, NCH(n) - 1)) == factor) {

This gives 0, 4, 8 as column offsets for foo.bar.baz and 0, 6, 14 for
foo.  bar.    baz, i.e. points to the beginning of the attribute
identifier.
What do you think?

Bye,
Sven

2010/12/27 Nick Coghlan <ncoghlan at gmail.com>:
> On Mon, Dec 27, 2010 at 9:34 AM, Benjamin Peterson <benjamin at python.org> wrote:
>> 2010/12/26 Sven Brauch <svenbrauch at googlemail.com>:
>>> In that discussion, there's been different opinions about which
>>> behaviour is better; main arguments were "consistency" for the current
>>> and "usefulness" for the suggested behaviour. It has been proposed to
>>> ask the question on this list, that's why I'm doing that now. :)
>>
>> My argument against this change is that an attribute includes the
>> expression from which the attribute is being taken. This is consistent
>> with subscripts and calls, which both have the lineno and col_offset
>> of their source expressions.
>
> I'd like to see the impact on existing uses of this information
> (primarily SyntaxErrors) before forming a firm opinion, but my initial
> inclination is that retaining the column information for the
> subattribute names is a better option.
>
> Cheers,
> Nick.
>
> --
> Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia
>

From g.brandl at gmx.net  Mon Dec 27 22:22:05 2010
From: g.brandl at gmx.net (Georg Brandl)
Date: Mon, 27 Dec 2010 22:22:05 +0100
Subject: [Python-Dev] r87518 - python/branches/py3k/Parser/tokenizer.c
In-Reply-To: <20101227201213.52E12EE989@mail.python.org>
References: <20101227201213.52E12EE989@mail.python.org>
Message-ID: <ifb03t$147$1@dough.gmane.org>

Am 27.12.2010 21:12, schrieb victor.stinner:
> Author: victor.stinner
> Date: Mon Dec 27 21:12:13 2010
> New Revision: 87518
> 
> Log:
> Issue #10778: decoding_fgets() decodes the filename from the filesystem
> encoding instead of UTF-8.
> 
> 
> Modified:
>    python/branches/py3k/Parser/tokenizer.c
> 
> Modified: python/branches/py3k/Parser/tokenizer.c
> ==============================================================================
> --- python/branches/py3k/Parser/tokenizer.c	(original)
> +++ python/branches/py3k/Parser/tokenizer.c	Mon Dec 27 21:12:13 2010
> @@ -545,6 +545,7 @@
>  {
>      char *line = NULL;
>      int badchar = 0;
> +    PyObject *filename;
>      for (;;) {
>          if (tok->decoding_state == STATE_NORMAL) {
>              /* We already have a codec associated with
> @@ -585,12 +586,16 @@
>      if (badchar) {
>          /* Need to add 1 to the line number, since this line
>             has not been counted, yet.  */
> -        PyErr_Format(PyExc_SyntaxError,
> -            "Non-UTF-8 code starting with '\\x%.2x' "
> -            "in file %.200s on line %i, "
> -            "but no encoding declared; "
> -            "see http://python.org/dev/peps/pep-0263/ for details",
> -            badchar, tok->filename, tok->lineno + 1);
> +        filename = PyUnicode_DecodeFSDefault(tok->filename);
> +        if (filename != NULL) {
> +            PyErr_Format(PyExc_SyntaxError,
> +                    "Non-UTF-8 code starting with '\\x%.2x' "
> +                    "in file %.200U on line %i, "
> +                    "but no encoding declared; "
> +                    "see http://python.org/dev/peps/pep-0263/ for details",
> +                    badchar, filename, tok->lineno + 1);
> +            Py_DECREF(filename);
> +        }

Hmm, and in case decoding fails, we return a Unicode error (without context)
instead of a syntax error?  Doesn't seem like a good trade-off when the file
name is just displayed in a message.

Georg


From victor.stinner at haypocalc.com  Tue Dec 28 01:06:06 2010
From: victor.stinner at haypocalc.com (Victor Stinner)
Date: Tue, 28 Dec 2010 01:06:06 +0100
Subject: [Python-Dev] r87518 - python/branches/py3k/Parser/tokenizer.c
In-Reply-To: <ifb03t$147$1@dough.gmane.org>
References: <20101227201213.52E12EE989@mail.python.org>
	<ifb03t$147$1@dough.gmane.org>
Message-ID: <1293494766.31514.12.camel@marge>

Le lundi 27 d?cembre 2010 ? 22:22 +0100, Georg Brandl a ?crit :
> Am 27.12.2010 21:12, schrieb victor.stinner:
> > Author: victor.stinner
> > Date: Mon Dec 27 21:12:13 2010
> > New Revision: 87518
> > 
> > Log:
> > Issue #10778: decoding_fgets() decodes the filename from the filesystem
> > encoding instead of UTF-8.
> > 

> Hmm, and in case decoding fails, we return a Unicode error (without context)
> instead of a syntax error?

Yes, but it is very unlikely. I don't see in which case the decoder can
fail. But a memory error can occur.

> Doesn't seem like a good trade-off when the file name is just displayed in a message.

What do you suggest?

--



From victor.stinner at haypocalc.com  Tue Dec 28 01:07:44 2010
From: victor.stinner at haypocalc.com (Victor Stinner)
Date: Tue, 28 Dec 2010 01:07:44 +0100
Subject: [Python-Dev] r87518 - python/branches/py3k/Parser/tokenizer.c
In-Reply-To: <ifb03t$147$1@dough.gmane.org>
References: <20101227201213.52E12EE989@mail.python.org>
	<ifb03t$147$1@dough.gmane.org>
Message-ID: <1293494864.31514.14.camel@marge>

(oops, I posted an incomplete message, stupid mailer)

Le lundi 27 d?cembre 2010 ? 22:22 +0100, Georg Brandl a ?crit :
> Am 27.12.2010 21:12, schrieb victor.stinner:
> > Author: victor.stinner
> > Date: Mon Dec 27 21:12:13 2010
> > New Revision: 87518
> > 
> > Log:
> > Issue #10778: decoding_fgets() decodes the filename from the filesystem
> > encoding instead of UTF-8.
> > 

> Hmm, and in case decoding fails, we return a Unicode error (without context)
> instead of a syntax error?

Yes, but it is very unlikely. I don't see in which case the decoder can
fail. But a memory error can occur.

> Doesn't seem like a good trade-off when the file name is just displayed in a message.

What do you suggest?

--

Prepare the decoded filename in PyParser_ParseStringFlagsFilenameEx()
and PyParser_ParseFileFlagsEx() avoids this issue.

Victor


From g.rodola at gmail.com  Tue Dec 28 03:04:31 2010
From: g.rodola at gmail.com (=?ISO-8859-1?Q?Giampaolo_Rodol=E0?=)
Date: Tue, 28 Dec 2010 03:04:31 +0100
Subject: [Python-Dev] os.getpriority() and os.setpriority()
Message-ID: <AANLkTikRdHNxooR1KLyvqdMRyKSCbC3qV1XKMff9O9BX@mail.gmail.com>

Hi all,
after having implemented something very similar in psutil:
http://code.google.com/p/psutil/issues/detail?id=142
...I decided to contribute a patch for exposing getpriority() and
setpriority() system calls in the os module:
http://bugs.python.org/issue10784

This was also raised some time ago:
http://mail.python.org/pipermail/python-ideas/2009-April/004028.html
...and it seems feedbacks were positive.

I think it would be nice to have Windows equivalents (GetPriorityClass
/ SetPriorityClass) as well but I'm not sure how since their notation
is different.


Regards,

--- Giampaolo
http://code.google.com/p/pyftpdlib
http://code.google.com/p/psutil

From rdmurray at bitdance.com  Tue Dec 28 05:07:40 2010
From: rdmurray at bitdance.com (R. David Murray)
Date: Mon, 27 Dec 2010 23:07:40 -0500
Subject: [Python-Dev] [Python-checkins] r87504 - in
	python/branches/py3k: Doc/c-api/exceptions.rst
	Include/pyerrors.h Include/warnings.h
In-Reply-To: <20101227014927.51BF5EE9A9@mail.python.org>
References: <20101227014927.51BF5EE9A9@mail.python.org>
Message-ID: <20101228040740.E1AE0228FC8@kimball.webabinitio.net>

On Mon, 27 Dec 2010 02:49:27 +0100, victor.stinner <python-checkins at python.org> wrote:
> Author: victor.stinner
> Date: Mon Dec 27 02:49:26 2010
> New Revision: 87504
> 
> Log:
> Issue #9738: Document encodings of error and warning functions
> 
> Modified:
>    python/branches/py3k/Doc/c-api/exceptions.rst
>    python/branches/py3k/Include/pyerrors.h
>    python/branches/py3k/Include/warnings.h
> 
> Modified: python/branches/py3k/Doc/c-api/exceptions.rst
> ==============================================================================
> --- python/branches/py3k/Doc/c-api/exceptions.rst	(original)
> +++ python/branches/py3k/Doc/c-api/exceptions.rst	Mon Dec 27 02:49:26 2010
> @@ -148,7 +148,8 @@
>     This function sets the error indicator and returns *NULL*.  *exception*
>     should be a Python exception class.  The *format* and subsequent
>     parameters help format the error message; they have the same meaning and
> -   values as in :c:func:`PyUnicode_FromFormat`.
> +   values as in :c:func:`PyUnicode_FromFormat`. *format* is an ASCII-encoding
> +   string.

ASCII-encoded string, or ASCII encoding-string?

>  .. c:function:: int PyErr_WarnFormat(PyObject *category, Py_ssize_t stack_level, const char *format, ...)
>  
>     Function similar to :c:func:`PyErr_WarnEx`, but use
> -   :c:func:`PyUnicode_FromFormat` to format the warning message.
> +   :c:func:`PyUnicode_FromFormat` to format the warning message.  *format* is
> +   an ASCII-encoded string.

So, the former, I'd guess :)

--
R. David Murray                                      www.bitdance.com

From rdmurray at bitdance.com  Tue Dec 28 05:13:40 2010
From: rdmurray at bitdance.com (R. David Murray)
Date: Mon, 27 Dec 2010 23:13:40 -0500
Subject: [Python-Dev] [Python-checkins] r87505 - in
	python/branches/py3k: Doc/c-api/unicode.rst Include/unicodeobject.h
In-Reply-To: <20101227014929.D926EEEA85@mail.python.org>
References: <20101227014929.D926EEEA85@mail.python.org>
Message-ID: <20101228041340.CDF93200C26@kimball.webabinitio.net>

On Mon, 27 Dec 2010 02:49:29 +0100, victor.stinner <python-checkins at python.org> wrote:
> Author: victor.stinner
> Date: Mon Dec 27 02:49:29 2010
> New Revision: 87505
> 
> Log:
> Issue #9738: document encodings of unicode functions
> 
> Modified:
>    python/branches/py3k/Doc/c-api/unicode.rst
>    python/branches/py3k/Include/unicodeobject.h
> 
> Modified: python/branches/py3k/Doc/c-api/unicode.rst
> ==============================================================================
> --- python/branches/py3k/Doc/c-api/unicode.rst	(original)
> +++ python/branches/py3k/Doc/c-api/unicode.rst	Mon Dec 27 02:49:29 2010
> @@ -1063,7 +1063,8 @@
>  .. c:function:: int PyUnicode_CompareWithASCIIString(PyObject *uni, char *string)
>  
>     Compare a unicode object, *uni*, with *string* and return -1, 0, 1 for less
> -   than, equal, and greater than, respectively.
> +   than, equal, and greater than, respectively. *string* is an ASCII-encoded
> +   string (it is interpreted as ISO-8859-1).

Does it mean anything to say that an ASCII string is interpreted as
ISO-8859-1?  If it is ASCII-encoded it shouldn't have any bytes with
the 8th bit set, leaving no room for interpretation.  So presumably
you mean it is (treated as) an ISO-8859-1 encoded string, despite the
function name?

--
R. David Murray                                      www.bitdance.com

From ncoghlan at gmail.com  Tue Dec 28 08:53:20 2010
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Tue, 28 Dec 2010 17:53:20 +1000
Subject: [Python-Dev] Loggers in the stdlib and logging configuration
	APIs
In-Reply-To: <loom.20101227T162842-639@post.gmane.org>
References: <loom.20101227T162842-639@post.gmane.org>
Message-ID: <AANLkTinD=k=CqLpjfiPzCqBn=w+J_5CuUA+GJzRwegB_@mail.gmail.com>

On Tue, Dec 28, 2010 at 1:29 AM, Vinay Sajip <vinay_sajip at yahoo.co.uk> wrote:
> ? ? However, it would be good to have some consistency of naming stdlib loggers
> - perhaps using __name__ as is recommended practice for library and application
> developers, but with a prefix such as "py." to indicate that it's a part of the
> stdlib. This would make the disabling logic change easier as it could just check
> the "py." prefix rather than have an internal list of logger names, which would
> need more maintenance and be more error-prone.

Unfortunately, the "py" package already claimed that namespace, so it
isn't really free for us to use in the standard library (even the
current "py.warnings" for redirected warnings may be misleading, as it
may give users the impression that package is involved somewhere along
the line). It is probably best just to go with the "__name__"
convention and not worry about being able to draw a clean distinction
between "standard library" and "third party" (that distinction doesn't
exist in the module heirarchy, so it isn't really reasonable to expect
it to exist in the logging heirarchy).

However, rather than a manually maintained list of low level loggers,
it may be feasible to just have a flag we can set on loggers that
makes them immune to the default implicit disabling. Then the config
calls can support three levels of logger disabling:
- leave all existing loggers enabled (existing option)
- leave only flagged loggers enabled (new default behaviour)
- disable all loggers not mentioned explicitly (new option)

Cheers,
Nick.

-- 
Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia

From vinay_sajip at yahoo.co.uk  Tue Dec 28 09:19:45 2010
From: vinay_sajip at yahoo.co.uk (Vinay Sajip)
Date: Tue, 28 Dec 2010 08:19:45 +0000 (UTC)
Subject: [Python-Dev]
	=?utf-8?q?Loggers_in_the_stdlib_and_logging_configur?=
	=?utf-8?q?ation=09APIs?=
References: <loom.20101227T162842-639@post.gmane.org>
	<AANLkTinD=k=CqLpjfiPzCqBn=w+J_5CuUA+GJzRwegB_@mail.gmail.com>
Message-ID: <loom.20101228T090905-998@post.gmane.org>

Nick Coghlan <ncoghlan <at> gmail.com> writes:

> Unfortunately, the "py" package already claimed that namespace, so it
> isn't really free for us to use in the standard library (even the
> current "py.warnings" for redirected warnings may be misleading, as it
> may give users the impression that package is involved somewhere along

Gosh, I had no idea the py package even existed.

> the line). It is probably best just to go with the "__name__"
> convention and not worry about being able to draw a clean distinction
> between "standard library" and "third party" (that distinction doesn't
> exist in the module heirarchy, so it isn't really reasonable to expect
> it to exist in the logging heirarchy).

True.

> However, rather than a manually maintained list of low level loggers,
> it may be feasible to just have a flag we can set on loggers that
> makes them immune to the default implicit disabling. Then the config

Yes, I thought about this option after I posted, using "leave_enabled" as the
attribute name, as well as possibly an API to register loggers for being skipped
by the disable logic. A logger flag is definitely easier all round.

> calls can support three levels of logger disabling:
> - leave all existing loggers enabled (existing option)

I think you mean disabled - that's the current behaviour.

> - leave only flagged loggers enabled (new default behaviour)
> - disable all loggers not mentioned explicitly (new option)

So far, this seems the best approach.

Thanks,

Vinay Sajip


From v+python at g.nevcal.com  Tue Dec 28 09:25:36 2010
From: v+python at g.nevcal.com (Glenn Linderman)
Date: Tue, 28 Dec 2010 00:25:36 -0800
Subject: [Python-Dev] Loggers in the stdlib and logging configuration
 APIs
In-Reply-To: <AANLkTinD=k=CqLpjfiPzCqBn=w+J_5CuUA+GJzRwegB_@mail.gmail.com>
References: <loom.20101227T162842-639@post.gmane.org>
	<AANLkTinD=k=CqLpjfiPzCqBn=w+J_5CuUA+GJzRwegB_@mail.gmail.com>
Message-ID: <4D199F00.4070503@g.nevcal.com>

On 12/27/2010 11:53 PM, Nick Coghlan wrote:
> However, rather than a manually maintained list of low level loggers,

Yes, a manually maintained list would be bad.

> it may be feasible to just have a flag we can set on loggers that
> makes them immune to the default implicit disabling. Then the config
> calls can support three levels of logger disabling:
> - leave all existing loggers enabled (existing option)
> - leave only flagged loggers enabled (new default behaviour)
> - disable all loggers not mentioned explicitly (new option)

I thought of the idea of a flag to make loggers immune, but it seemed it 
could lead to an arms race, where the knee-jerk reaction of discovering 
that a library's logger got disabled would be to set the flag, and, of 
course, the knee-jerk reaction of discovering that a library's logger 
produced an unwanted message would cause the application to use the new 
option, to "turn them all off again".  Then a new library logger option, 
to become immune to the new disable them all option, would be 
requested.  Etc.

That's why I suggested a list, although perhaps your scheme would be 
"more" backward compatible, as it defines a meaning for the default 
behavior that sounds somewhat similar to the current default 
behavior.... but isn't, really.

Interesting that you choose to disable all loggers not mentioned 
explicitly (a logger white list), whereas I suggested listing the ones 
to disable (a logger black list).

I suppose you suggested the white list, figuring that the application 
could choose which messages it wishes to see.

I chose a black list, figuring that new messages being introduced might 
be useful to the application, but if they never see them (because they 
are not in the white list), they wouldn't know about them.  Of course, 
either scheme can be programmed around, given an obtainable list of 
defined loggers.... but, thinking about it more, maybe that list cannot 
be obtainable, at xxxConfig time, because not all the modules that have 
loggers have been loaded yet... so maybe the question is not just what 
to do with pre-existing loggers at xxxConfig time, but also what to do 
with loggers created in the future that are not configured explicitly?

I can see where a white list would produce a more predictable, 
consistent set of known messages, and a black list would allow 
introduction of new messages (if the lists are both manually maintained, 
vs. programmatically adjusted to a specific white or black list), which 
could be disruptive to some applications... but perhaps shouldn't be.

But a new version of a module on the logger white list could introduce 
new messages, so that is not much different than a new version of a 
module not on the logger black list introducing new messages.  
Applications that use logging, really should be prepared for new 
messages to be displayed.  Testing won't discover them all, disabling 
them could result in useful messages being suppressed.  Should a 
module's documentation be expected to include lists of log messages that 
it may produce at any level of logging?

It really seems that applications that use logging should be prepared to 
discover messages in the log that they don't produce, and that an 
explicit configuration of the appropriate level should be done in the 
xxxConfig call for each logger.... and if they are not mentioned in the 
xxxConfig call, how much logging should they be allowed to do?  Should 
their existence, and lack of configuration be logged?
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20101228/99f180ae/attachment.html>

From v+python at g.nevcal.com  Tue Dec 28 09:28:18 2010
From: v+python at g.nevcal.com (Glenn Linderman)
Date: Tue, 28 Dec 2010 00:28:18 -0800
Subject: [Python-Dev] Loggers in the stdlib and logging configuration
 APIs
In-Reply-To: <loom.20101228T090905-998@post.gmane.org>
References: <loom.20101227T162842-639@post.gmane.org>	<AANLkTinD=k=CqLpjfiPzCqBn=w+J_5CuUA+GJzRwegB_@mail.gmail.com>
	<loom.20101228T090905-998@post.gmane.org>
Message-ID: <4D199FA2.2020504@g.nevcal.com>

On 12/28/2010 12:19 AM, Vinay Sajip wrote:
>> >  calls can support three levels of logger disabling:
>> >  - leave all existing loggers enabled (existing option)
> I think you mean disabled - that's the current behaviour.
>
>> >  - leave only flagged loggers enabled (new default behaviour)
>> >  - disable all loggers not mentioned explicitly (new option)

My words in his mouth, but I think he meant that the existing _default_ 
behavior should be changed, and the existing option (listed first), 
which is not default behaviour should be kept, and a third option to 
sort of achieve the current default behavior, but make it harder.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20101228/49017d07/attachment.html>

From ncoghlan at gmail.com  Tue Dec 28 09:53:12 2010
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Tue, 28 Dec 2010 18:53:12 +1000
Subject: [Python-Dev] Loggers in the stdlib and logging configuration
	APIs
In-Reply-To: <4D199FA2.2020504@g.nevcal.com>
References: <loom.20101227T162842-639@post.gmane.org>
	<AANLkTinD=k=CqLpjfiPzCqBn=w+J_5CuUA+GJzRwegB_@mail.gmail.com>
	<loom.20101228T090905-998@post.gmane.org>
	<4D199FA2.2020504@g.nevcal.com>
Message-ID: <AANLkTiksZX_r=5g2n2iAM8mEfXPd7TRp9DQ_Zm7yJ7VX@mail.gmail.com>

On Tue, Dec 28, 2010 at 6:28 PM, Glenn Linderman <v+python at g.nevcal.com> wrote:
> On 12/28/2010 12:19 AM, Vinay Sajip wrote:
>
>> calls can support three levels of logger disabling:
>> - leave all existing loggers enabled (existing option)
>
> I think you mean disabled - that's the current behaviour.
>
>> - leave only flagged loggers enabled (new default behaviour)
>> - disable all loggers not mentioned explicitly (new option)
>
> My words in his mouth, but I think he meant that the existing _default_
> behavior should be changed, and the existing option (listed first), which is
> not default behaviour should be kept, and a third option to sort of achieve
> the current default behavior, but make it harder.

It isn't really about making it harder - it is about changing the
default behaviour in a backwards compatible way.

As I see it, library level loggers (including those in the standard
library) should be left enabled unless explicitly turned off, while
application level loggers should be left disabled unless explicitly
turned on.

However, there is currently no way to indicate the default state for a
given logger that isn't mentioned explicitly in a detailed config -
they're all implicitly disabled (or enabled, if that option is passed
in as part of the config). Changing to a 3-tier system lets libraries
use logging for error messages without them being lost by the default
configuration process.

Cheers,
Nick.

-- 
Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia

From ncoghlan at gmail.com  Tue Dec 28 09:55:42 2010
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Tue, 28 Dec 2010 18:55:42 +1000
Subject: [Python-Dev] Loggers in the stdlib and logging configuration
	APIs
In-Reply-To: <AANLkTiksZX_r=5g2n2iAM8mEfXPd7TRp9DQ_Zm7yJ7VX@mail.gmail.com>
References: <loom.20101227T162842-639@post.gmane.org>
	<AANLkTinD=k=CqLpjfiPzCqBn=w+J_5CuUA+GJzRwegB_@mail.gmail.com>
	<loom.20101228T090905-998@post.gmane.org>
	<4D199FA2.2020504@g.nevcal.com>
	<AANLkTiksZX_r=5g2n2iAM8mEfXPd7TRp9DQ_Zm7yJ7VX@mail.gmail.com>
Message-ID: <AANLkTi=Qp2-bJKUS_07g1KcWEpNefrhJOexxKVeHp1t2@mail.gmail.com>

On Tue, Dec 28, 2010 at 6:53 PM, Nick Coghlan <ncoghlan at gmail.com> wrote:
> On Tue, Dec 28, 2010 at 6:28 PM, Glenn Linderman <v+python at g.nevcal.com> wrote:
>> On 12/28/2010 12:19 AM, Vinay Sajip wrote:
>>
>>> calls can support three levels of logger disabling:
>>> - leave all existing loggers enabled (existing option)
>>
>> I think you mean disabled - that's the current behaviour.
>>
>>> - leave only flagged loggers enabled (new default behaviour)
>>> - disable all loggers not mentioned explicitly (new option)
>>
>> My words in his mouth, but I think he meant that the existing _default_
>> behavior should be changed, and the existing option (listed first), which is
>> not default behaviour should be kept, and a third option to sort of achieve
>> the current default behavior, but make it harder.
>
> It isn't really about making it harder - it is about changing the
> default behaviour in a backwards compatible way.
>
> As I see it, library level loggers (including those in the standard
> library) should be left enabled unless explicitly turned off, while
> application level loggers should be left disabled unless explicitly
> turned on.

To put a bit more nuance on that view: if the library uses the logger
to report errors, it should *definitely* flag it to be implicitly
enabled. If it is only used for info messages, then preferences will
vary according to the library developer. The beauty of the logging
system is that application authors can get at the logging information
regardless of which default the library author chooses.

Cheers,
Nick.

-- 
Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia

From g.brandl at gmx.net  Tue Dec 28 10:12:46 2010
From: g.brandl at gmx.net (Georg Brandl)
Date: Tue, 28 Dec 2010 10:12:46 +0100
Subject: [Python-Dev] r87518 - python/branches/py3k/Parser/tokenizer.c
In-Reply-To: <1293494864.31514.14.camel@marge>
References: <20101227201213.52E12EE989@mail.python.org>	<ifb03t$147$1@dough.gmane.org>
	<1293494864.31514.14.camel@marge>
Message-ID: <ifc9ob$c14$1@dough.gmane.org>

Am 28.12.2010 01:07, schrieb Victor Stinner:
> (oops, I posted an incomplete message, stupid mailer)
> 
> Le lundi 27 d?cembre 2010 ? 22:22 +0100, Georg Brandl a ?crit :
>> Am 27.12.2010 21:12, schrieb victor.stinner:
>> > Author: victor.stinner
>> > Date: Mon Dec 27 21:12:13 2010
>> > New Revision: 87518
>> > 
>> > Log:
>> > Issue #10778: decoding_fgets() decodes the filename from the filesystem
>> > encoding instead of UTF-8.
>> > 
> 
>> Hmm, and in case decoding fails, we return a Unicode error (without context)
>> instead of a syntax error?
> 
> Yes, but it is very unlikely. I don't see in which case the decoder can
> fail. But a memory error can occur.
> 
>> Doesn't seem like a good trade-off when the file name is just displayed in a message.
> 
> What do you suggest?

If the probability is so low, it's probably not worth changing.  I'm just
somewhat sensitive to changes that enforce correctness by taking away
useful information from the user.

Georg


From victor.stinner at haypocalc.com  Tue Dec 28 10:28:51 2010
From: victor.stinner at haypocalc.com (Victor Stinner)
Date: Tue, 28 Dec 2010 10:28:51 +0100
Subject: [Python-Dev] [Python-checkins] r87505 - in
 python/branches/py3k: Doc/c-api/unicode.rst Include/unicodeobject.h
In-Reply-To: <20101228041340.CDF93200C26@kimball.webabinitio.net>
References: <20101227014929.D926EEEA85@mail.python.org>
	<20101228041340.CDF93200C26@kimball.webabinitio.net>
Message-ID: <1293528531.24502.10.camel@marge>

Le lundi 27 d?cembre 2010 ? 23:13 -0500, R. David Murray a ?crit :
> > Modified: python/branches/py3k/Doc/c-api/unicode.rst
> > ==============================================================================
> > --- python/branches/py3k/Doc/c-api/unicode.rst	(original)
> > +++ python/branches/py3k/Doc/c-api/unicode.rst	Mon Dec 27 02:49:29 2010
> > @@ -1063,7 +1063,8 @@
> >  .. c:function:: int PyUnicode_CompareWithASCIIString(PyObject *uni, char *string)
> >  
> >     Compare a unicode object, *uni*, with *string* and return -1, 0, 1 for less
> > -   than, equal, and greater than, respectively.
> > +   than, equal, and greater than, respectively. *string* is an ASCII-encoded
> > +   string (it is interpreted as ISO-8859-1).
> 
> Does it mean anything to say that an ASCII string is interpreted as
> ISO-8859-1?  If it is ASCII-encoded it shouldn't have any bytes with
> the 8th bit set, leaving no room for interpretation.  So presumably
> you mean it is (treated as) an ISO-8859-1 encoded string, despite the
> function name?

Oh. Someone noticed :-) I would like to say that it is better to pass
only ASCII-encoded string, but the function supports ISO-8859-1.

Would it be more clear to say that the function expects ISO-8859-1
encoded string?

But I don't want to patch the function.

Victor


From victor.stinner at haypocalc.com  Tue Dec 28 11:28:19 2010
From: victor.stinner at haypocalc.com (Victor Stinner)
Date: Tue, 28 Dec 2010 11:28:19 +0100
Subject: [Python-Dev] [Python-checkins] r87504 - in
 python/branches/py3k: Doc/c-api/exceptions.rst Include/pyerrors.h
 Include/warnings.h
In-Reply-To: <20101228040740.E1AE0228FC8@kimball.webabinitio.net>
References: <20101227014927.51BF5EE9A9@mail.python.org>
	<20101228040740.E1AE0228FC8@kimball.webabinitio.net>
Message-ID: <1293532099.24502.70.camel@marge>

Le lundi 27 d?cembre 2010 ? 23:07 -0500, R. David Murray a ?crit :
> On Mon, 27 Dec 2010 02:49:27 +0100, victor.stinner <python-checkins at python.org> wrote:
> > Author: victor.stinner
> > Date: Mon Dec 27 02:49:26 2010
> > New Revision: 87504
> > 
> > Log:
> > Issue #9738: Document encodings of error and warning functions

> ASCII-encoded string, or ASCII encoding-string?

Oh, I don't really care. Which one do you prefer? I used "ASCII-encoded
string" and "UTF-8 encoded string", because "UTF-8-encoded" has too many
"-". I should maybe be uniformized.

Victor


From martin at v.loewis.de  Tue Dec 28 11:40:01 2010
From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=)
Date: Tue, 28 Dec 2010 11:40:01 +0100
Subject: [Python-Dev] [Python-checkins] r87504 - in
 python/branches/py3k: Doc/c-api/exceptions.rst Include/pyerrors.h
 Include/warnings.h
In-Reply-To: <1293532099.24502.70.camel@marge>
References: <20101227014927.51BF5EE9A9@mail.python.org>	<20101228040740.E1AE0228FC8@kimball.webabinitio.net>
	<1293532099.24502.70.camel@marge>
Message-ID: <4D19BE81.3070806@v.loewis.de>

Am 28.12.2010 11:28, schrieb Victor Stinner:
> Le lundi 27 d?cembre 2010 ? 23:07 -0500, R. David Murray a ?crit :
>> On Mon, 27 Dec 2010 02:49:27 +0100, victor.stinner <python-checkins at python.org> wrote:
>>> Author: victor.stinner
>>> Date: Mon Dec 27 02:49:26 2010
>>> New Revision: 87504
>>>
>>> Log:
>>> Issue #9738: Document encodings of error and warning functions
> 
>> ASCII-encoded string, or ASCII encoding-string?
> 
> Oh, I don't really care. Which one do you prefer? I used "ASCII-encoded
> string" and "UTF-8 encoded string", because "UTF-8-encoded" has too many
> "-". I should maybe be uniformized.

David's other concern was whether it should be "encod*ed*" or
"encod*ing*". It seems he would prefer "encoded". Not sure what
he would prefer wrt. hyphens (but apparently, you would put
the hyphen after if it's "encoding", and before if it's "encoded";
not sure why that is).

Regards,
Martin

From victor.stinner at haypocalc.com  Tue Dec 28 12:01:43 2010
From: victor.stinner at haypocalc.com (Victor Stinner)
Date: Tue, 28 Dec 2010 12:01:43 +0100
Subject: [Python-Dev] r87518 - python/branches/py3k/Parser/tokenizer.c
In-Reply-To: <ifc9ob$c14$1@dough.gmane.org>
References: <20101227201213.52E12EE989@mail.python.org>
	<ifb03t$147$1@dough.gmane.org> <1293494864.31514.14.camel@marge>
	<ifc9ob$c14$1@dough.gmane.org>
Message-ID: <1293534103.24502.88.camel@marge>

Le mardi 28 d?cembre 2010 ? 10:12 +0100, Georg Brandl a ?crit :
> >> > Author: victor.stinner
> >> > Date: Mon Dec 27 21:12:13 2010
> >> > New Revision: 87518
> >> > 
> >> > Log:
> >> > Issue #10778: decoding_fgets() decodes the filename from the filesystem
> >> > encoding instead of UTF-8.
> > 
> >> Hmm, and in case decoding fails, we return a Unicode error (without context)
> >> instead of a syntax error?

I created an issue for this problem, but also to prepare the full
support of unicode in the import machinery. I patched the import
machinery of Python 3 to support undecodable bytes, but Python 3 doesn't
support unencodable characters on Windows (see issue #3080).

http://bugs.python.org/issue10785

Victor


From stefan_ml at behnel.de  Tue Dec 28 12:20:36 2010
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Tue, 28 Dec 2010 12:20:36 +0100
Subject: [Python-Dev] [Python-checkins] r87504 - in
 python/branches/py3k: Doc/c-api/exceptions.rst Include/pyerrors.h
 Include/warnings.h
In-Reply-To: <4D19BE81.3070806@v.loewis.de>
References: <20101227014927.51BF5EE9A9@mail.python.org>	<20101228040740.E1AE0228FC8@kimball.webabinitio.net>	<1293532099.24502.70.camel@marge>
	<4D19BE81.3070806@v.loewis.de>
Message-ID: <ifch64$9eq$1@dough.gmane.org>

"Martin v. L?wis", 28.12.2010 11:40:
>> Le lundi 27 d?cembre 2010 ? 23:07 -0500, R. David Murray a ?crit :
>>> ASCII-encoded string, or ASCII encoding-string?
>>
> [...] apparently, you would put
> the hyphen after if it's "encoding", and before if it's "encoded";
> not sure why that is

I think David just put it there to point to the semantic difference. It's 
either a string that is "ASCII encoded", or an "encoding string" that 
refers to ASCII in some way. Whether you put a hyphen or not is a secondary 
concern.

Stefan


From vinay_sajip at yahoo.co.uk  Tue Dec 28 13:16:09 2010
From: vinay_sajip at yahoo.co.uk (Vinay Sajip)
Date: Tue, 28 Dec 2010 12:16:09 +0000 (UTC)
Subject: [Python-Dev] Loggers in the stdlib and logging configuration
	APIs
References: <loom.20101227T162842-639@post.gmane.org>
	<AANLkTinD=k=CqLpjfiPzCqBn=w+J_5CuUA+GJzRwegB_@mail.gmail.com>
	<4D199F00.4070503@g.nevcal.com>
Message-ID: <loom.20101228T122917-484@post.gmane.org>

Glenn Linderman <v+python <at> g.nevcal.com> writes:

>     I thought of the idea of a flag to make loggers immune, but it
>     seemed it could lead to an arms race, where the knee-jerk reaction
>     of discovering that a library's logger got disabled would be to set
>     the flag, and, of course, the knee-jerk reaction of discovering that
>     a library's logger produced an unwanted message would cause the
>     application to use the new option, to "turn them all off again". 
>     Then a new library logger option, to become immune to the new
>     disable them all option, would be requested.  Etc.

Although an arms race is theoretically possible, I doubt it would come to
that. After all, people can just set a logger's disabled flag to False, and
its messages would start flowing again. That would be as easy for users as
making any new API call.

>     That's why I suggested a list, although perhaps your scheme would be
>     "more" backward compatible, as it defines a meaning for the default
>     behavior that sounds somewhat similar to the current default
>     behavior.... but isn't, really.
>     Interesting that you choose to disable all loggers not mentioned
>     explicitly (a logger white list), whereas I suggested listing the
>     ones to disable (a logger black list).

That's the backward compatible behaviour. In general, when the configuration
APIs are called, the design assumption was that you want to replace any
existing configuration with a new one;  that now needs to change because of
places where logging will be used for notifications where exceptions cannot be
usefully raised.

>     I suppose you suggested the white list, figuring that the
>     application could choose which messages it wishes to see.

An application can always choose not to see or not see messages via setting
verbosity levels on specific loggers and/or handlers to the desired values.

>     I chose a black list, figuring that new messages being introduced
>     might be useful to the application, but if they never see them
>     (because they are not in the white list), they wouldn't know about
>     them.  Of course, either scheme can be programmed around, given an

Don't forget that any loggers created after a fileConfig/dictConfig call
wouldn't be disabled; and for many many uses, logging will be set up
reasonably early in the application's execution, and the only loggers which
would be disabled by a configuration call would be loggers created before the
call, and which are not explicitly named in the configuration. There will be
relatively few of those.

>     messages.  Applications that use logging, really should be prepared
>     for new messages to be displayed.  Testing won't discover them all,
>     disabling them could result in useful messages being suppressed. 

Yes, because third party libraries can introduce new messages either by
themseleves or through adding a new dependency, but disabling via a config
call is unlikely if the config call is made early.
 
>     Should a module's documentation be expected to include lists of log
>     messages that it may produce at any level of logging?

That would never be kept up to date :-) The best we can hope for is for module
authors to document that they use logging, mention any logger names used, the
default level settings for at least the top-level logger, and any convenience
APIs provided for configuring handlers.

Suppose we just implement a "leave_enabled" flag on loggers and change the
config code to honour this. Then, there will be no backward incompatible
change until we modify stdlib loggers to set leave_enabled to True (I'm not
sure if this could be done for concurrent.futures now as it's a new package in
3.2; it depends on whether this would be regarded as a new feature).

I don't propose adding another optional argument to the configuration APIs
unless and until there seems to be a need for it; in the absence of this
argument, users would have to  explicitly turn off loggers they wanted to
silence in the normal way (i.e. by setting levels, and perhaps propagation
flags).

I would propose to document "leave_enabled" as part of the public API, but
with a recommendation that due care and consideration be given before choosing
to set it for a logger.

How does that sound?

Regards,


Vinay Sajip



From victor.stinner at haypocalc.com  Tue Dec 28 13:58:24 2010
From: victor.stinner at haypocalc.com (Victor Stinner)
Date: Tue, 28 Dec 2010 13:58:24 +0100
Subject: [Python-Dev] [Python-checkins] r87504 - in
 python/branches/py3k: Doc/c-api/exceptions.rst Include/pyerrors.h
 Include/warnings.h
In-Reply-To: <4D19BE81.3070806@v.loewis.de>
References: <20101227014927.51BF5EE9A9@mail.python.org>
	<20101228040740.E1AE0228FC8@kimball.webabinitio.net>
	<1293532099.24502.70.camel@marge>  <4D19BE81.3070806@v.loewis.de>
Message-ID: <1293541104.24502.89.camel@marge>

Le mardi 28 d?cembre 2010 ? 11:40 +0100, "Martin v. L?wis" a ?crit :
> Am 28.12.2010 11:28, schrieb Victor Stinner:
> > Le lundi 27 d?cembre 2010 ? 23:07 -0500, R. David Murray a ?crit :
> >> On Mon, 27 Dec 2010 02:49:27 +0100, victor.stinner <python-checkins at python.org> wrote:
> >>> Author: victor.stinner
> >>> Date: Mon Dec 27 02:49:26 2010
> >>> New Revision: 87504
> >>>
> >>> Log:
> >>> Issue #9738: Document encodings of error and warning functions
> > 
> >> ASCII-encoded string, or ASCII encoding-string?
> > 
> > Oh, I don't really care. Which one do you prefer? I used "ASCII-encoded
> > string" and "UTF-8 encoded string", because "UTF-8-encoded" has too many
> > "-". I should maybe be uniformized.
> 
> David's other concern was whether it should be "encod*ed*" or
> "encod*ing*".

Ooops, it's ASCII-encoded string (and not ASCII-encoding string): fixed
by r87529.

Victor


From rdmurray at bitdance.com  Tue Dec 28 18:10:12 2010
From: rdmurray at bitdance.com (R. David Murray)
Date: Tue, 28 Dec 2010 12:10:12 -0500
Subject: [Python-Dev] [Python-checkins] r87504 - in
	python/branches/py3k: Doc/c-api/exceptions.rst
	Include/pyerrors.h Include/warnings.h
In-Reply-To: <1293541104.24502.89.camel@marge>
References: <20101227014927.51BF5EE9A9@mail.python.org>
	<20101228040740.E1AE0228FC8@kimball.webabinitio.net>
	<1293532099.24502.70.camel@marge> <4D19BE81.3070806@v.loewis.de>
	<1293541104.24502.89.camel@marge>
Message-ID: <20101228171012.4A2312361E2@kimball.webabinitio.net>

On Tue, 28 Dec 2010 13:58:24 +0100, Victor Stinner <victor.stinner at haypocalc.com> wrote:
> Le mardi 28 d??cembre 2010 ?? 11:40 +0100, "Martin v. L??wis" a ??crit :
> > Am 28.12.2010 11:28, schrieb Victor Stinner:
> > > Le lundi 27 d??cembre 2010 ?? 23:07 -0500, R. David Murray a ??crit :
> > >> On Mon, 27 Dec 2010 02:49:27 +0100, victor.stinner <python-checkins at python.org> wrote:
> > >>> Author: victor.stinner
> > >>> Date: Mon Dec 27 02:49:26 2010
> > >>> New Revision: 87504
> > >>>
> > >>> Log:
> > >>> Issue #9738: Document encodings of error and warning functions
> > >
> > >> ASCII-encoded string, or ASCII encoding-string?
> > >
> > > Oh, I don't really care. Which one do you prefer? I used "ASCII-encoded
> > > string" and "UTF-8 encoded string", because "UTF-8-encoded" has too many
> > > "-". I should maybe be uniformized.
> >
> > David's other concern was whether it should be "encod*ed*" or
> > "encod*ing*".
> 
> Ooops, it's ASCII-encoded string (and not ASCII-encoding string): fixed
> by r87529.

Yes, exactly.  Sorry for being a bit too subtle in my error report.

--
R. David Murray                                      www.bitdance.com

From rdmurray at bitdance.com  Tue Dec 28 18:14:52 2010
From: rdmurray at bitdance.com (R. David Murray)
Date: Tue, 28 Dec 2010 12:14:52 -0500
Subject: [Python-Dev] [Python-checkins] r87505 - in
	python/branches/py3k: Doc/c-api/unicode.rst Include/unicodeobject.h
In-Reply-To: <1293528531.24502.10.camel@marge>
References: <20101227014929.D926EEEA85@mail.python.org>
	<20101228041340.CDF93200C26@kimball.webabinitio.net>
	<1293528531.24502.10.camel@marge>
Message-ID: <20101228171452.C3FA72361E2@kimball.webabinitio.net>

On Tue, 28 Dec 2010 10:28:51 +0100, Victor Stinner <victor.stinner at haypocalc.com> wrote:
> Le lundi 27 d??cembre 2010 ?? 23:13 -0500, R. David Murray a ??crit :
> > > Modified: python/branches/py3k/Doc/c-api/unicode.rst
> > > ==============================================================================
> > > --- python/branches/py3k/Doc/c-api/unicode.rst      (original)
> > > +++ python/branches/py3k/Doc/c-api/unicode.rst      Mon Dec 27 02:49:29 2010
> > > @@ -1063,7 +1063,8 @@
> > >  .. c:function:: int PyUnicode_CompareWithASCIIString(PyObject *uni, char *string)
> > >
> > >     Compare a unicode object, *uni*, with *string* and return -1, 0, 1 for less
> > > -   than, equal, and greater than, respectively.
> > > +   than, equal, and greater than, respectively. *string* is an ASCII-encoded
> > > +   string (it is interpreted as ISO-8859-1).
> >
> > Does it mean anything to say that an ASCII string is interpreted as
> > ISO-8859-1?  If it is ASCII-encoded it shouldn't have any bytes with
> > the 8th bit set, leaving no room for interpretation.  So presumably
> > you mean it is (treated as) an ISO-8859-1 encoded string, despite the
> > function name?
> 
> Oh. Someone noticed :-) I would like to say that it is better to pass
> only ASCII-encoded string, but the function supports ISO-8859-1.
> 
> Would it be more clear to say that the function expects ISO-8859-1
> encoded string?
> 
> But I don't want to patch the function.

I think your first paragraph is what you should put in the docs: "it is
best to pass only ASCII-encoded strings, but the function interprets
the input string as ISO-8859-1 if it contains non-ASCII characters".

A bit harder to compress that into an in-line comment in the code...

--
R. David Murray                                      www.bitdance.com

From v+python at g.nevcal.com  Tue Dec 28 18:14:41 2010
From: v+python at g.nevcal.com (Glenn Linderman)
Date: Tue, 28 Dec 2010 09:14:41 -0800
Subject: [Python-Dev] Loggers in the stdlib and logging configuration
 APIs
In-Reply-To: <loom.20101228T122917-484@post.gmane.org>
References: <loom.20101227T162842-639@post.gmane.org>	<AANLkTinD=k=CqLpjfiPzCqBn=w+J_5CuUA+GJzRwegB_@mail.gmail.com>	<4D199F00.4070503@g.nevcal.com>
	<loom.20101228T122917-484@post.gmane.org>
Message-ID: <4D1A1B01.7080508@g.nevcal.com>

On 12/28/2010 4:16 AM, Vinay Sajip wrote:
> How does that sound?

Sounds pretty rational, overall.

If the leave_enabled flag can be turned on/off by the application, then 
I agree the arms race is unlikely.

I didn't dig through the logging docs to discover if there is an API 
that returns a list of currently known loggers.... such that an 
application could easily discover the current set.  It would be nice to 
have such a thing, in any case, but especially in the presence of the 
leave_enabled flag.

I also wasn't sure whether or not the "default disable" applied to 
future loggers or only preexisting loggers, but you clarified that it 
only applies to preexisting ones, so such an API as above would suffice 
to discover any that might need special handling at xxxConfig time.

I hope somewhere in the parts of the documentation I haven't read yet, 
that there is some verbiage that discusses the fact that loggers created 
after xxxConfig time and not configured by it, are enabled by default?  
And the concept that applications that use logging should be prepared to 
handle messages that they haven't seen before in some manner.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20101228/69e478c2/attachment.html>

From lukas.lueg at googlemail.com  Tue Dec 28 18:08:58 2010
From: lukas.lueg at googlemail.com (Lukas Lueg)
Date: Tue, 28 Dec 2010 18:08:58 +0100
Subject: [Python-Dev] Possible optimization for LOAD_FAST ?
Message-ID: <AANLkTin05vNDnuP53OefqYvpq6S5cv2aEF5MdXkMshbD@mail.gmail.com>

Consider the following code:

def foobar(x):
    for i in range(5):
        x[i] = i

The bytecode in python 2.7 is the following:

  2           0 SETUP_LOOP              30 (to 33)
              3 LOAD_GLOBAL              0 (range)
              6 LOAD_CONST               1 (5)
              9 CALL_FUNCTION            1
             12 GET_ITER
        >>   13 FOR_ITER                16 (to 32)
             16 STORE_FAST               1 (i)

  3          19 LOAD_FAST                1 (i)
             22 LOAD_FAST                0 (x)
             25 LOAD_FAST                1 (i)
             28 STORE_SUBSCR
             29 JUMP_ABSOLUTE           13
        >>   32 POP_BLOCK
        >>   33 LOAD_CONST               0 (None)
             36 RETURN_VALUE

Can't we optimize the LOAD_FAST in lines 19 and 25 to a single load
and put the reference twice on the stack? There is no way that the
reference of i might change in between the two lines. Also, the
load_fast in lne 22 to reference x could be taken out of the loop as x
will always point to the same object....

From benjamin at python.org  Tue Dec 28 18:24:36 2010
From: benjamin at python.org (Benjamin Peterson)
Date: Tue, 28 Dec 2010 11:24:36 -0600
Subject: [Python-Dev] Possible optimization for LOAD_FAST ?
In-Reply-To: <AANLkTin05vNDnuP53OefqYvpq6S5cv2aEF5MdXkMshbD@mail.gmail.com>
References: <AANLkTin05vNDnuP53OefqYvpq6S5cv2aEF5MdXkMshbD@mail.gmail.com>
Message-ID: <AANLkTikzhWkK+QVeXvJo1s6Ge48DXzUzQ2P0+J3F5Q-1@mail.gmail.com>

2010/12/28 Lukas Lueg <lukas.lueg at googlemail.com>:
> Consider the following code:
>
> def foobar(x):
> ? ?for i in range(5):
> ? ? ? ?x[i] = i
>
> The bytecode in python 2.7 is the following:
>
> ?2 ? ? ? ? ? 0 SETUP_LOOP ? ? ? ? ? ? ?30 (to 33)
> ? ? ? ? ? ? ?3 LOAD_GLOBAL ? ? ? ? ? ? ?0 (range)
> ? ? ? ? ? ? ?6 LOAD_CONST ? ? ? ? ? ? ? 1 (5)
> ? ? ? ? ? ? ?9 CALL_FUNCTION ? ? ? ? ? ?1
> ? ? ? ? ? ? 12 GET_ITER
> ? ? ? ?>> ? 13 FOR_ITER ? ? ? ? ? ? ? ?16 (to 32)
> ? ? ? ? ? ? 16 STORE_FAST ? ? ? ? ? ? ? 1 (i)
>
> ?3 ? ? ? ? ?19 LOAD_FAST ? ? ? ? ? ? ? ?1 (i)
> ? ? ? ? ? ? 22 LOAD_FAST ? ? ? ? ? ? ? ?0 (x)
> ? ? ? ? ? ? 25 LOAD_FAST ? ? ? ? ? ? ? ?1 (i)
> ? ? ? ? ? ? 28 STORE_SUBSCR
> ? ? ? ? ? ? 29 JUMP_ABSOLUTE ? ? ? ? ? 13
> ? ? ? ?>> ? 32 POP_BLOCK
> ? ? ? ?>> ? 33 LOAD_CONST ? ? ? ? ? ? ? 0 (None)
> ? ? ? ? ? ? 36 RETURN_VALUE
>
> Can't we optimize the LOAD_FAST in lines 19 and 25 to a single load
> and put the reference twice on the stack?

Yes. Would it be useful? Unlikely.

-- 
Regards,
Benjamin

From vinay_sajip at yahoo.co.uk  Tue Dec 28 19:28:07 2010
From: vinay_sajip at yahoo.co.uk (Vinay Sajip)
Date: Tue, 28 Dec 2010 18:28:07 +0000 (UTC)
Subject: [Python-Dev] Loggers in the stdlib and logging configuration
	APIs
References: <loom.20101227T162842-639@post.gmane.org>	<AANLkTinD=k=CqLpjfiPzCqBn=w+J_5CuUA+GJzRwegB_@mail.gmail.com>	<4D199F00.4070503@g.nevcal.com>
	<loom.20101228T122917-484@post.gmane.org>
	<4D1A1B01.7080508@g.nevcal.com>
Message-ID: <loom.20101228T192253-520@post.gmane.org>

Glenn Linderman <v+python <at> g.nevcal.com> writes:

>     I didn't dig through the logging docs to discover if there is an API
>     that returns a list of currently known loggers.... such that an
>     application could easily discover the current set.? It would be nice
>     to have such a thing, in any case, but especially in the presence of
>     the leave_enabled flag.

There isn't one, though I'll see about providing it in 3.3.

>     I hope somewhere in the parts of the documentation I haven't read
>     yet, that there is some verbiage that discusses the fact that
>     loggers created after xxxConfig time and not configured by it, are

Not quite; the fact that existing loggers are disabled by the configuration
calls is documented, as that would be the thing that might be unexpected. That
newly-created loggers would be enabled isn't documented specifically, as it is
what one would expect.

>     enabled by default?? And the concept that applications that use
>     logging should be prepared to handle messages that they haven't seen
>     before in some manner.

That concept is also not mentioned explicitly, because the rationale given for
having a logging module in the stdlib is that it can be used by modules from
different sources (standard library, 3rd party libs, application code) and
provide logs in an integrated manner. So the expectation you describe would
follow naturally.

Thanks for the feedback,


Vinay Sajip


From g.brandl at gmx.net  Tue Dec 28 19:30:48 2010
From: g.brandl at gmx.net (Georg Brandl)
Date: Tue, 28 Dec 2010 19:30:48 +0100
Subject: [Python-Dev] Possible optimization for LOAD_FAST ?
In-Reply-To: <AANLkTikzhWkK+QVeXvJo1s6Ge48DXzUzQ2P0+J3F5Q-1@mail.gmail.com>
References: <AANLkTin05vNDnuP53OefqYvpq6S5cv2aEF5MdXkMshbD@mail.gmail.com>
	<AANLkTikzhWkK+QVeXvJo1s6Ge48DXzUzQ2P0+J3F5Q-1@mail.gmail.com>
Message-ID: <ifdaej$q2b$1@dough.gmane.org>

Am 28.12.2010 18:24, schrieb Benjamin Peterson:
> 2010/12/28 Lukas Lueg <lukas.lueg at googlemail.com>:
>> Consider the following code:
>>
>> def foobar(x):
>>    for i in range(5):
>>        x[i] = i
>>
>> The bytecode in python 2.7 is the following:
>>
>>  2           0 SETUP_LOOP              30 (to 33)
>>              3 LOAD_GLOBAL              0 (range)
>>              6 LOAD_CONST               1 (5)
>>              9 CALL_FUNCTION            1
>>             12 GET_ITER
>>        >>   13 FOR_ITER                16 (to 32)
>>             16 STORE_FAST               1 (i)
>>
>>  3          19 LOAD_FAST                1 (i)
>>             22 LOAD_FAST                0 (x)
>>             25 LOAD_FAST                1 (i)
>>             28 STORE_SUBSCR
>>             29 JUMP_ABSOLUTE           13
>>        >>   32 POP_BLOCK
>>        >>   33 LOAD_CONST               0 (None)
>>             36 RETURN_VALUE
>>
>> Can't we optimize the LOAD_FAST in lines 19 and 25 to a single load
>> and put the reference twice on the stack?
> 
> Yes. Would it be useful? Unlikely.

Is it tricky to get all the corner cases right? Very probably :)

Georg


From martin at v.loewis.de  Tue Dec 28 21:11:56 2010
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Tue, 28 Dec 2010 21:11:56 +0100
Subject: [Python-Dev] os.getpriority() and os.setpriority()
In-Reply-To: <AANLkTikRdHNxooR1KLyvqdMRyKSCbC3qV1XKMff9O9BX@mail.gmail.com>
References: <AANLkTikRdHNxooR1KLyvqdMRyKSCbC3qV1XKMff9O9BX@mail.gmail.com>
Message-ID: <4D1A448C.40102@v.loewis.de>


> I think it would be nice to have Windows equivalents (GetPriorityClass
> / SetPriorityClass) as well but I'm not sure how since their notation
> is different.

I don't mind exposing it (somewhere), however, it should keep its
original name. In addition, you probably need OpenProcess, as well
as Get/SetThreadPriority along with it.

I think Python should incorporate something like pywin32, to expose
all APIs whole-sale, in a type-safe manner (unlike ctypes).

Regards,
Martin

From tjreedy at udel.edu  Tue Dec 28 21:22:07 2010
From: tjreedy at udel.edu (Terry Reedy)
Date: Tue, 28 Dec 2010 15:22:07 -0500
Subject: [Python-Dev] [Python-checkins] r87537 - in
	python/branches/py3k:	Doc/library/struct.rst
	Doc/whatsnew/3.2.rst	Lib/test/test_struct.py Lib/wave.py
	Misc/NEWS Modules/_struct.c
In-Reply-To: <20101228132642.83E95EE9B6@mail.python.org>
References: <20101228132642.83E95EE9B6@mail.python.org>
Message-ID: <4D1A46EF.5030509@udel.edu>

On 12/28/2010 8:26 AM, victor.stinner wrote:
> Author: victor.stinner
> Date: Tue Dec 28 14:26:42 2010
> New Revision: 87537
>
> Log:
> Issue #10783: struct.pack() doesn't encode implicitly unicode to UTF-8

> Modified: python/branches/py3k/Doc/whatsnew/3.2.rst
> ==============================================================================
> --- python/branches/py3k/Doc/whatsnew/3.2.rst	(original)
> +++ python/branches/py3k/Doc/whatsnew/3.2.rst	Tue Dec 28 14:26:42 2010
> @@ -1705,3 +1705,7 @@
>
>     (Contributed by Georg Brandl and Mattias Br?ndstr?m;
>     `appspot issue 53094<http://codereview.appspot.com/53094>`_.)
> +
> +* :func:`struct.pack` doesn't encode implicitly unicode to UTF-8 anymore:

That phrasing is awkward at best. I believe you mean
"struct.pack no longer implicitly encodes unicode to UTF-8",
which will be clearer to most people.

Terry



From victor.stinner at haypocalc.com  Wed Dec 29 00:38:13 2010
From: victor.stinner at haypocalc.com (Victor Stinner)
Date: Wed, 29 Dec 2010 00:38:13 +0100
Subject: [Python-Dev] [Python-checkins] r87537 - in
 python/branches/py3k:	Doc/library/struct.rst Doc/whatsnew/3.2.rst
 Lib/test/test_struct.py Lib/wave.py Misc/NEWS Modules/_struct.c
In-Reply-To: <4D1A46EF.5030509@udel.edu>
References: <20101228132642.83E95EE9B6@mail.python.org>
	<4D1A46EF.5030509@udel.edu>
Message-ID: <1293579493.29958.1.camel@marge>

Le mardi 28 d?cembre 2010 ? 15:22 -0500, Terry Reedy a ?crit :
> On 12/28/2010 8:26 AM, victor.stinner wrote:
> > Author: victor.stinner
> > Date: Tue Dec 28 14:26:42 2010
> > New Revision: 87537
> >
> > Log:
> > Issue #10783: struct.pack() doesn't encode implicitly unicode to UTF-8
> 
> > Modified: python/branches/py3k/Doc/whatsnew/3.2.rst
> > ==============================================================================
> > --- python/branches/py3k/Doc/whatsnew/3.2.rst	(original)
> > +++ python/branches/py3k/Doc/whatsnew/3.2.rst	Tue Dec 28 14:26:42 2010
> > @@ -1705,3 +1705,7 @@
> >
> >     (Contributed by Georg Brandl and Mattias Br?ndstr?m;
> >     `appspot issue 53094<http://codereview.appspot.com/53094>`_.)
> > +
> > +* :func:`struct.pack` doesn't encode implicitly unicode to UTF-8 anymore:
> 
> That phrasing is awkward at best.

Sorry, I am still learning english :-)

>  I believe you mean
> "struct.pack no longer implicitly encodes unicode to UTF-8",
> which will be clearer to most people.

Yes, done in r87559

Victor


From victor.stinner at haypocalc.com  Wed Dec 29 00:41:17 2010
From: victor.stinner at haypocalc.com (Victor Stinner)
Date: Wed, 29 Dec 2010 00:41:17 +0100
Subject: [Python-Dev] [Python-checkins] r87505 - in
 python/branches/py3k: Doc/c-api/unicode.rst Include/unicodeobject.h
In-Reply-To: <20101228171452.C3FA72361E2@kimball.webabinitio.net>
References: <20101227014929.D926EEEA85@mail.python.org>
	<20101228041340.CDF93200C26@kimball.webabinitio.net>
	<1293528531.24502.10.camel@marge>
	<20101228171452.C3FA72361E2@kimball.webabinitio.net>
Message-ID: <1293579677.29958.2.camel@marge>

Le mardi 28 d?cembre 2010 ? 12:14 -0500, R. David Murray a ?crit :
> I think your first paragraph is what you should put in the docs: "it is
> best to pass only ASCII-encoded strings, but the function interprets
> the input string as ISO-8859-1 if it contains non-ASCII characters".

Nice, done in r87560

Victor


From amauryfa at gmail.com  Wed Dec 29 01:13:13 2010
From: amauryfa at gmail.com (Amaury Forgeot d'Arc)
Date: Wed, 29 Dec 2010 01:13:13 +0100
Subject: [Python-Dev] Backport troubles with mercurial
Message-ID: <AANLkTi=+_qOSRo2tj12sec9UKb=hjJar+-8VQEr8_7BB@mail.gmail.com>

Hello,

The PyPy project recently switched from svn to mercurial. Since this day I
have some
difficulties to perform simple tasks, and my questions did not receive
satisfying answers.

I was sure the Python project would have the same issues;
so I started from the repositories in http://code.python.org/hg/ and
tried simple
merges between Python versions.
I would like several people to try the same example, and tell how they
handle it.
I'm new to mercurial, and I may have missed something important!


Let's say you have to do the simple change shown in the diff below,
and want to "fix" the the 3 usual versions: py3k, release31-maint,
release27-maint.
The diff was made against py3k.

How would you do it with Mercurial? Please try to do it for real!

"hg merge" is not the correct command: it merges whole branches, a change
comes with all its ancestors. What we want is to "cherry-pick" a single
change.

"hg transplant" fails to apply the change to release31-maint because
of a tiny conflict (in the diff context!) that leaves you with an ugly
"hunks FAILED" and a .rej file you have to parse and apply by hand.

"hg transplant" seems to succeed in release27-maint,
but the test fails to run because "support" should be changed to
"test_support".
The change is already committed - to fix it another changeset is needed.
This does not look clean to me: both changesets will be pushed to the
repository,
and diff review (on the python-checkins mailing list) is almost impossible.

Furthermore, "hg transplant" does not keep track of integrations.
There is a "transplants" file, but it stays in my local clone.

Faced with a similar case in pypy, we finally manually copied the files
between directories... and the fastest with our example is probably
to do the change manually in all three directories.

There is surely a better way to work with maintenance branches!
PEP374 suggests to first modify the oldest release, but this does not
seems right to me (I have three reasons in mind)
At the moment, I feel that mercurial just cannot work for core Python
development.

At the very least before the migration we need precise use cases like this
and recipes for common cases.

Thanks for your help,

-- 
Amaury Forgeot d'Arc


diff -r 2777ae4d10d9 Lib/test/test_contextlib.py
--- a/Lib/test/test_contextlib.py       Tue Dec 28 22:14:34 2010 +0100
+++ b/Lib/test/test_contextlib.py       Wed Dec 29 00:08:18 2010 +0100
@@ -26,6 +26,7 @@
             state.append(x)
         self.assertEqual(state, [1, 42, 999])

+    @support.cpython_only
     def test_contextmanager_finally(self):
         state = []
         @contextmanager
@@ -36,10 +37,10 @@
             finally:
                 state.append(999)
         with self.assertRaises(ZeroDivisionError):
-            with woohoo() as x:
+            with woohoo() as y:
                 self.assertEqual(state, [1])
-                self.assertEqual(x, 42)
-                state.append(x)
+                self.assertEqual(y, 42)
+                state.append(y)
                 raise ZeroDivisionError()
         self.assertEqual(state, [1, 42, 999])
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20101229/17b081a4/attachment.html>

From jimjjewett at gmail.com  Wed Dec 29 03:35:09 2010
From: jimjjewett at gmail.com (Jim Jewett)
Date: Tue, 28 Dec 2010 21:35:09 -0500
Subject: [Python-Dev] [Python-checkins] r87523 -
	python/branches/py3k/Doc/tutorial/interpreter.rst
In-Reply-To: <20101228091824.4FE70F4E4@mail.python.org>
References: <20101228091824.4FE70F4E4@mail.python.org>
Message-ID: <AANLkTikok21AMYN24XK5i5TGMz_SEYGk2_iAcnVVsqkh@mail.gmail.com>

It might still be worth saying something like:

Note that this "python <file" does something subtly different; the
details are not included in this tutorial.

On Tue, Dec 28, 2010 at 4:18 AM, georg.brandl
<python-checkins at python.org> wrote:
> Author: georg.brandl
> Date: Tue Dec 28 10:18:24 2010
> New Revision: 87523
>
> Log:
> Remove confusing paragraph -- this is relevant only to advanced users anyway and does not belong into the tutorial.
>
> Modified:
> ? python/branches/py3k/Doc/tutorial/interpreter.rst
>
> Modified: python/branches/py3k/Doc/tutorial/interpreter.rst
> ==============================================================================
> --- python/branches/py3k/Doc/tutorial/interpreter.rst ? (original)
> +++ python/branches/py3k/Doc/tutorial/interpreter.rst ? Tue Dec 28 10:18:24 2010
> @@ -58,14 +58,6 @@
> ?``python -m module [arg] ...``, which executes the source file for *module* as
> ?if you had spelled out its full name on the command line.
>
> -Note that there is a difference between ``python file`` and ``python
> -<file``. ?In the latter case, input requests from the program, such as calling
> -``sys.stdin.read()``, are satisfied from *file*. ?Since this file has already
> -been read until the end by the parser before the program starts executing, the
> -program will encounter end-of-file immediately. ?In the former case (which is
> -usually what you want) they are satisfied from whatever file or device is
> -connected to standard input of the Python interpreter.
> -
> ?When a script file is used, it is sometimes useful to be able to run the script
> ?and enter interactive mode afterwards. ?This can be done by passing :option:`-i`
> ?before the script. ?(This does not work if the script is read from standard
> _______________________________________________
> Python-checkins mailing list
> Python-checkins at python.org
> http://mail.python.org/mailman/listinfo/python-checkins
>

From rdmurray at bitdance.com  Wed Dec 29 03:33:43 2010
From: rdmurray at bitdance.com (R. David Murray)
Date: Tue, 28 Dec 2010 21:33:43 -0500
Subject: [Python-Dev] [Python-checkins] r87537 - in
	python/branches/py3k: Doc/library/struct.rst
	Doc/whatsnew/3.2.rst Lib/test/test_struct.py Lib/wave.py
	Misc/NEWS Modules/_struct.c
In-Reply-To: <1293579493.29958.1.camel@marge>
References: <20101228132642.83E95EE9B6@mail.python.org>
	<4D1A46EF.5030509@udel.edu> <1293579493.29958.1.camel@marge>
Message-ID: <20101229023343.C6B14238BCA@kimball.webabinitio.net>

On Wed, 29 Dec 2010 00:38:13 +0100, Victor Stinner <victor.stinner at haypocalc.com> wrote:
>Le mardi 28 d??cembre 2010 ?? 15:22 -0500, Terry Reedy a ??crit :
>> On 12/28/2010 8:26 AM, victor.stinner wrote:
>> > Author: victor.stinner
>> > Date: Tue Dec 28 14:26:42 2010
>> > New Revision: 87537
>> >
>> > Log:
>> > Issue #10783: struct.pack() doesn't encode implicitly unicode to UTF-8
>>
>> > Modified: python/branches/py3k/Doc/whatsnew/3.2.rst
>> >
>==============================================================================
>> > --- python/branches/py3k/Doc/whatsnew/3.2.rst       (original)
>> > +++ python/branches/py3k/Doc/whatsnew/3.2.rst       Tue Dec 28 14:26:42 2010
>> > @@ -1705,3 +1705,7 @@
>> >
>> >     (Contributed by Georg Brandl and Mattias Br??ndstr??m;
>> >     `appspot issue 53094<http://codereview.appspot.com/53094>`_.)
>> > +
>> > +* :func:`struct.pack` doesn't encode implicitly unicode to UTF-8 anymore:
>>
>> That phrasing is awkward at best.
>
>Sorry, I am still learning english :-)
>
>>  I believe you mean
>> "struct.pack no longer implicitly encodes unicode to UTF-8",
>> which will be clearer to most people.
>
>Yes, done in r87559

You still have two words swapped.  You changed it to

    no longer encodes implicitly unicode

but it should be

    no longer implicitly encodes unicode

--
R. David Murray                                      www.bitdance.com

From orsenthil at gmail.com  Wed Dec 29 04:29:01 2010
From: orsenthil at gmail.com (Senthil Kumaran)
Date: Wed, 29 Dec 2010 11:29:01 +0800
Subject: [Python-Dev] [Python-checkins] r87523 -
	python/branches/py3k/Doc/tutorial/interpreter.rst
In-Reply-To: <AANLkTikok21AMYN24XK5i5TGMz_SEYGk2_iAcnVVsqkh@mail.gmail.com>
References: <20101228091824.4FE70F4E4@mail.python.org>
	<AANLkTikok21AMYN24XK5i5TGMz_SEYGk2_iAcnVVsqkh@mail.gmail.com>
Message-ID: <AANLkTi=EqSfPyvZPEFRrzH9JAqofY4zd9xdG8FjW2Kiz@mail.gmail.com>

On Wed, Dec 29, 2010 at 10:35 AM, Jim Jewett <jimjjewett at gmail.com> wrote:
> It might still be worth saying something like:
>
> Note that this "python <file" does something subtly different; the
> details are not included in this tutorial.

No, it this in fact puzzling. I was fine with the previous paragraph
and if someone found it confusing, removing it is okay.
But the above suggested sentence would leave the reader with an
incomplete information.

-- 
Senthil

From cournape at gmail.com  Wed Dec 29 05:13:25 2010
From: cournape at gmail.com (David Cournapeau)
Date: Wed, 29 Dec 2010 13:13:25 +0900
Subject: [Python-Dev] Backport troubles with mercurial
In-Reply-To: <AANLkTi=+_qOSRo2tj12sec9UKb=hjJar+-8VQEr8_7BB@mail.gmail.com>
References: <AANLkTi=+_qOSRo2tj12sec9UKb=hjJar+-8VQEr8_7BB@mail.gmail.com>
Message-ID: <AANLkTik-3LiS8QWhg2J26+gRkjx49YpXh2rnwq90aJcE@mail.gmail.com>

On Wed, Dec 29, 2010 at 9:13 AM, Amaury Forgeot d'Arc
<amauryfa at gmail.com> wrote:
> Hello,
> The PyPy project recently switched from svn to mercurial. Since this day I
> have some
> difficulties to perform simple tasks, and my questions did not receive
> satisfying answers.
> I was sure the Python project would have the same issues;
> so I started from the repositories in?http://code.python.org/hg/?and
> tried?simple
> merges between Python versions.
> I would like several people to try the same example, and tell how they
> handle it.
> I'm new to mercurial, and I may have missed something important!
>
> Let's say you have to do the simple change shown in the diff below,
> and want to "fix" the the 3 usual versions: py3k, release31-maint,
> release27-maint.
> The diff was made against py3k.
> How would you do it with Mercurial??Please try to do it for real!

The easiest way I found to emulate git cherry-pick (which does exactly
what you want) with hg is to use import/export commands:
http://mercurial.selenic.com/wiki/CommunicatingChanges

It is indeed quite a pain to use in my experience, because you cannot
easily refer to the original commit the cherry pick is coming from
(i.e. no equivalent to git cherry-pick -x), and the conflict
resolution is quite dumb. One alternative is to be careful about where
you apply your patch
(http://stackoverflow.com/questions/3926906/what-is-the-most-efficient-way-to-handle-hg-import-rejects),
but that's not very convenient either.

cheers,

David

From stephen at xemacs.org  Wed Dec 29 07:18:56 2010
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Wed, 29 Dec 2010 15:18:56 +0900
Subject: [Python-Dev] [Python-checkins] r87537 - in
 python/branches/py3k:	Doc/library/struct.rst Doc/whatsnew/3.2.rst
 Lib/test/test_struct.py Lib/wave.py Misc/NEWS Modules/_struct.c
In-Reply-To: <1293579493.29958.1.camel@marge>
References: <20101228132642.83E95EE9B6@mail.python.org>
	<4D1A46EF.5030509@udel.edu> <1293579493.29958.1.camel@marge>
Message-ID: <87zkrpje3z.fsf@uwakimon.sk.tsukuba.ac.jp>

Victor Stinner writes:

 > >Sorry, I am still learning english :-)

Aren't we all!  :-)


From g.brandl at gmx.net  Wed Dec 29 09:01:12 2010
From: g.brandl at gmx.net (Georg Brandl)
Date: Wed, 29 Dec 2010 09:01:12 +0100
Subject: [Python-Dev] Backport troubles with mercurial
In-Reply-To: <AANLkTi=+_qOSRo2tj12sec9UKb=hjJar+-8VQEr8_7BB@mail.gmail.com>
References: <AANLkTi=+_qOSRo2tj12sec9UKb=hjJar+-8VQEr8_7BB@mail.gmail.com>
Message-ID: <ifepu0$748$1@dough.gmane.org>

Am 29.12.2010 01:13, schrieb Amaury Forgeot d'Arc:
> Hello,
> 
> The PyPy project recently switched from svn to mercurial. Since this day I have some
> difficulties to perform simple tasks, and my questions did not receive
> satisfying answers.
> 
> I was sure the Python project would have the same issues;
> so I started from the repositories in http://code.python.org/hg/ and tried simple
> merges between Python versions.
> I would like several people to try the same example, and tell how they handle it.
> I'm new to mercurial, and I may have missed something important!
> 
> 
> Let's say you have to do the simple change shown in the diff below,
> and want to "fix" the the 3 usual versions: py3k, release31-maint, release27-maint.
> The diff was made against py3k.
> 
> How would you do it with Mercurial? Please try to do it for real!
> 
> "hg merge" is not the correct command: it merges whole branches, a change
> comes with all its ancestors. What we want is to "cherry-pick" a single change.
> 
> "hg transplant" fails to apply the change to release31-maint because
> of a tiny conflict (in the diff context!) that leaves you with an ugly
> "hunks FAILED" and a .rej file you have to parse and apply by hand.

Yes, this has been an irritation to me as well.  It's probably not so hard for
a Mercurial coredev to change transplant into generating inline conflict markers
though.

BTW, we would apply the diff in release31-maint and then "hg merge" it to py3k.
transplant still applies to 2.7 though.

> "hg transplant" seems to succeed in release27-maint,
> but the test fails to run because "support" should be changed to "test_support".
> The change is already committed - to fix it another changeset is needed.
> This does not look clean to me: both changesets will be pushed to the repository,
> and diff review (on the python-checkins mailing list) is almost impossible.

Right.  Seems to me transplant should get a command-line switch that always goes
into break-and-continue mode, where the commit is only made after calling
"hg transplant --continue".

> Furthermore, "hg transplant" does not keep track of integrations.
> There is a "transplants" file, but it stays in my local clone.

You're wrong, it does keep track of integrations in the commit metadata.  The
only thing that's clone-local is the transplants cache file.  Sadly, transplant
only checks against this file, but it is very easy to write a hook that keeps
it up to date; I've already done that work.

> Faced with a similar case in pypy, we finally manually copied the files
> between directories... and the fastest with our example is probably
> to do the change manually in all three directories.
> 
> There is surely a better way to work with maintenance branches!
> PEP374 suggests to first modify the oldest release, but this does not
> seems right to me (I have three reasons in mind)

Would you care to explain those?

> At the very least before the migration we need precise use cases like this
> and recipes for common cases.

Which is what we'll certainly get once we have the test repo and Brett is
rewriting the dev guide.

Georg


From amauryfa at gmail.com  Wed Dec 29 09:02:23 2010
From: amauryfa at gmail.com (Amaury Forgeot d'Arc)
Date: Wed, 29 Dec 2010 09:02:23 +0100
Subject: [Python-Dev] Backport troubles with mercurial
In-Reply-To: <AANLkTik-3LiS8QWhg2J26+gRkjx49YpXh2rnwq90aJcE@mail.gmail.com>
References: <AANLkTi=+_qOSRo2tj12sec9UKb=hjJar+-8VQEr8_7BB@mail.gmail.com>
	<AANLkTik-3LiS8QWhg2J26+gRkjx49YpXh2rnwq90aJcE@mail.gmail.com>
Message-ID: <AANLkTin2yFJqD_OaRdTGOmYGry815bxzh_pUnHBZr_2x@mail.gmail.com>

2010/12/29 David Cournapeau <cournape at gmail.com>

> The easiest way I found to emulate git cherry-pick (which does exactly
> what you want) with hg is to use import/export commands:
> http://mercurial.selenic.com/wiki/CommunicatingChanges
>
> It is indeed quite a pain to use in my experience, because you cannot
> easily refer to the original commit the cherry pick is coming from
> (i.e. no equivalent to git cherry-pick -x), and the conflict
> resolution is quite dumb.
>

This is precisely why I proposed a specific example.
Which precise steps do you take?
How much typing or manual copy/paste is required for this very simple case?
Can you do the merge in less than 10 minutes?

And finally the biased question:
can you find one improvement over the current situation with svn?

One alternative is to be careful about where
> you apply your patch
> (
> http://stackoverflow.com/questions/3926906/what-is-the-most-efficient-way-to-handle-hg-import-rejects
> ),
> but that's not very convenient either.
>

I've read all this, and this method does not work, for several reasons:

- py3k / 2.7 / 3.1 cannot be ordered, there is no "earliest possible
parent"..
we usually consider py3k as a child of both 2.7 and 3.1, and there is no
common parent.

- even if there was one, there is the problem of changes specifically made
for 2.7
that make no sense in py3k. You'd have to perform a "dummy merge" to py3k
which has the disadvantage of cluttering the py3k change log. And think of
the horror
if someone forgets this dummy merge.

- in any case, the actual repositories in http://code.python.org/hg/ are not
clones
of each other, so "hg merge" won't work and "hg pull" will clone the whole
remote repository.
(btw, now that I have "hg pull" another repo, how can I remove it? is my
clone broken forever?)

-- 
Amaury Forgeot d'Arc
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20101229/a6876c77/attachment-0001.html>

From g.brandl at gmx.net  Wed Dec 29 09:20:18 2010
From: g.brandl at gmx.net (Georg Brandl)
Date: Wed, 29 Dec 2010 09:20:18 +0100
Subject: [Python-Dev] Backport troubles with mercurial
In-Reply-To: <AANLkTin2yFJqD_OaRdTGOmYGry815bxzh_pUnHBZr_2x@mail.gmail.com>
References: <AANLkTi=+_qOSRo2tj12sec9UKb=hjJar+-8VQEr8_7BB@mail.gmail.com>	<AANLkTik-3LiS8QWhg2J26+gRkjx49YpXh2rnwq90aJcE@mail.gmail.com>
	<AANLkTin2yFJqD_OaRdTGOmYGry815bxzh_pUnHBZr_2x@mail.gmail.com>
Message-ID: <ifer1q$b76$1@dough.gmane.org>

Am 29.12.2010 09:02, schrieb Amaury Forgeot d'Arc:

> I've read all this, and this method does not work, for several reasons:
> 
> - py3k / 2.7 / 3.1 cannot be ordered, there is no "earliest possible parent"..
> we usually consider py3k as a child of both 2.7 and 3.1, and there is no common
> parent.

Yes.  Think of 2.7 as the oddball one, where revisions must be cherry-picked to,
whereas the 3.x line can use the "commit in parent branch" system (and much more
easily, since there are no changes in syntax and much less changes in code
overall).

> - even if there was one, there is the problem of changes specifically made for 2.7
> that make no sense in py3k. You'd have to perform a "dummy merge" to py3k
> which has the disadvantage of cluttering the py3k change log. And think of the
> horror
> if someone forgets this dummy merge.

No horror at all: the next committer notices the extra changes in his merge and
removes them.  *Never* commit merges without reviewing the diff.  (The
situtation is similar, by the way, to someone typing the wrong revision number
when using svnmerge, and not noticing it because he does not review the diff.)

> - in any case, the actual repositories in http://code.python.org/hg/ are not clones
> of each other, so "hg merge" won't work and "hg pull" will clone the whole
> remote repository.

Note that the repos on code.python.org are not the results of our conversion
process.  Those will be on hg.python.org.  The former are repos contributed
by Antoine (I think) that he uses with hgsubversion to work on Python using
Mercurial right now; they will disappear after hg.python.org works.

> (btw, now that I have "hg pull" another repo, how can I remove it? is my clone
> broken forever?)

No, you can "hg strip" away the root of the wrongly pulled revisions (which
also strips all descendants), or use "hg clone -r XXX" to create a new clone
with only a specified revision and all its ancestors.

Georg


From amauryfa at gmail.com  Wed Dec 29 10:53:56 2010
From: amauryfa at gmail.com (Amaury Forgeot d'Arc)
Date: Wed, 29 Dec 2010 10:53:56 +0100
Subject: [Python-Dev] Backport troubles with mercurial
In-Reply-To: <ifer1q$b76$1@dough.gmane.org>
References: <AANLkTi=+_qOSRo2tj12sec9UKb=hjJar+-8VQEr8_7BB@mail.gmail.com>
	<AANLkTik-3LiS8QWhg2J26+gRkjx49YpXh2rnwq90aJcE@mail.gmail.com>
	<AANLkTin2yFJqD_OaRdTGOmYGry815bxzh_pUnHBZr_2x@mail.gmail.com>
	<ifer1q$b76$1@dough.gmane.org>
Message-ID: <AANLkTi=Y+nvzUTiGBTw5UOxgdn0oR7M54HdX++oo-ZBt@mail.gmail.com>

2010/12/29 Georg Brandl <g.brandl at gmx.net>
>
> Am 29.12.2010 09:02, schrieb Amaury Forgeot d'Arc:
>
> > - even if there was one, there is the problem of changes specifically made for 2.7
> > that make no sense in py3k. You'd have to perform a "dummy merge" to py3k
> > which has the disadvantage of cluttering the py3k change log. And think of the
> > horror
> > if someone forgets this dummy merge.
>
> No horror at all: the next committer notices the extra changes in his merge and
> removes them. ?*Never* commit merges without reviewing the diff. ?(The
> situtation is similar, by the way, to someone typing the wrong revision number
> when using svnmerge, and not noticing it because he does not review the diff.)

Except that it's easy to overlook a diff and not notice another small change
mixed in your merge. Checking "hg merge -P" is probably better.
But is it possible to "remove" the extra change?

What worries me more is the requirement to find the correct branch before I can
even edit the file.?How would you, Georg, deal with doc patches
(typos, bad markup)?
With subversion?it's easy to work on trunk, and then have svnmerge tell you
which chunk is in conflict?and does not apply to the maintenance branch.

>
> > - in any case, the actual repositories in http://code.python.org/hg/ are not clones
> > of each other, so "hg merge" won't work and "hg pull" will clone the whole
> > remote repository.
>
> Note that the repos on code.python.org are not the results of our conversion
> process. ?Those will be on hg.python.org. ?The former are repos contributed
> by Antoine (I think) that he uses with hgsubversion to work on Python using
> Mercurial right now; they will disappear after hg.python.org works.

ok. I used them because other the repos I found were really old.
And?http://hg.python.org/cpython/?probably needs an initial "dummy merge"
on every branch.
>
> > (btw, now that I have "hg pull" another repo, how can I remove it? is my clone
> > broken forever?)
>
> No, you can "hg strip" away the root of the wrongly pulled revisions (which
> also strips all descendants), or use "hg clone -r XXX" to create a new clone
> with only a specified revision and all its ancestors.

"hg strip" worked for me. The root revision to strip was simply the
first one (=oldest) in "hg outgoing".

--
Amaury Forgeot d'Arc

From dirkjan at ochtman.nl  Wed Dec 29 10:59:07 2010
From: dirkjan at ochtman.nl (Dirkjan Ochtman)
Date: Wed, 29 Dec 2010 10:59:07 +0100
Subject: [Python-Dev] Backport troubles with mercurial
In-Reply-To: <AANLkTi=Y+nvzUTiGBTw5UOxgdn0oR7M54HdX++oo-ZBt@mail.gmail.com>
References: <AANLkTi=+_qOSRo2tj12sec9UKb=hjJar+-8VQEr8_7BB@mail.gmail.com>
	<AANLkTik-3LiS8QWhg2J26+gRkjx49YpXh2rnwq90aJcE@mail.gmail.com>
	<AANLkTin2yFJqD_OaRdTGOmYGry815bxzh_pUnHBZr_2x@mail.gmail.com>
	<ifer1q$b76$1@dough.gmane.org>
	<AANLkTi=Y+nvzUTiGBTw5UOxgdn0oR7M54HdX++oo-ZBt@mail.gmail.com>
Message-ID: <AANLkTi=Y4O-R-rZ2fPx1+JRTegzGKHS_=+can0h1i2Le@mail.gmail.com>

On Wed, Dec 29, 2010 at 10:53, Amaury Forgeot d'Arc <amauryfa at gmail.com> wrote:
> And?http://hg.python.org/cpython/?probably needs an initial "dummy merge"
> on every branch.

Yes, the present delays are all about getting that right.

Cheers,

Dirkjan

From g.brandl at gmx.net  Wed Dec 29 11:00:59 2010
From: g.brandl at gmx.net (Georg Brandl)
Date: Wed, 29 Dec 2010 11:00:59 +0100
Subject: [Python-Dev] Backport troubles with mercurial
In-Reply-To: <AANLkTi=Y+nvzUTiGBTw5UOxgdn0oR7M54HdX++oo-ZBt@mail.gmail.com>
References: <AANLkTi=+_qOSRo2tj12sec9UKb=hjJar+-8VQEr8_7BB@mail.gmail.com>	<AANLkTik-3LiS8QWhg2J26+gRkjx49YpXh2rnwq90aJcE@mail.gmail.com>	<AANLkTin2yFJqD_OaRdTGOmYGry815bxzh_pUnHBZr_2x@mail.gmail.com>	<ifer1q$b76$1@dough.gmane.org>
	<AANLkTi=Y+nvzUTiGBTw5UOxgdn0oR7M54HdX++oo-ZBt@mail.gmail.com>
Message-ID: <iff0uj$1vm$1@dough.gmane.org>

Am 29.12.2010 10:53, schrieb Amaury Forgeot d'Arc:
> 2010/12/29 Georg Brandl <g.brandl at gmx.net>
>>
>> Am 29.12.2010 09:02, schrieb Amaury Forgeot d'Arc:
>>
>> > - even if there was one, there is the problem of changes specifically made for 2.7
>> > that make no sense in py3k. You'd have to perform a "dummy merge" to py3k
>> > which has the disadvantage of cluttering the py3k change log. And think of the
>> > horror
>> > if someone forgets this dummy merge.
>>
>> No horror at all: the next committer notices the extra changes in his merge and
>> removes them.  *Never* commit merges without reviewing the diff.  (The
>> situtation is similar, by the way, to someone typing the wrong revision number
>> when using svnmerge, and not noticing it because he does not review the diff.)
> 
> Except that it's easy to overlook a diff and not notice another small change
> mixed in your merge. Checking "hg merge -P" is probably better.
> But is it possible to "remove" the extra change?

No; hg merging is DAG-based, so you will always merge all ancestors.  The only
way to "remove" it is a null-merge.

> What worries me more is the requirement to find the correct branch before I can
> even edit the file. How would you, Georg, deal with doc patches
> (typos, bad markup)?

Finding the correct branch is not really hard.  Bugfixes go to maintenance,
features to trunk.  You need to think about which category your change is
right now too, when deciding what to backport/svnmerge.

Accordingly, I would apply doc patches in release31-maint and merge them
to trunk, probably all at once with one merge commit.

>> > - in any case, the actual repositories in http://code.python.org/hg/ are not clones
>> > of each other, so "hg merge" won't work and "hg pull" will clone the whole
>> > remote repository.
>>
>> Note that the repos on code.python.org are not the results of our conversion
>> process.  Those will be on hg.python.org.  The former are repos contributed
>> by Antoine (I think) that he uses with hgsubversion to work on Python using
>> Mercurial right now; they will disappear after hg.python.org works.
> 
> ok. I used them because other the repos I found were really old.
> And http://hg.python.org/cpython/ probably needs an initial "dummy merge"
> on every branch.

Yes, that is a very old conversion result.  We should get a new one in a few days.

Georg


From amauryfa at gmail.com  Wed Dec 29 11:09:21 2010
From: amauryfa at gmail.com (Amaury Forgeot d'Arc)
Date: Wed, 29 Dec 2010 11:09:21 +0100
Subject: [Python-Dev] Backport troubles with mercurial
In-Reply-To: <iff0uj$1vm$1@dough.gmane.org>
References: <AANLkTi=+_qOSRo2tj12sec9UKb=hjJar+-8VQEr8_7BB@mail.gmail.com>
	<AANLkTik-3LiS8QWhg2J26+gRkjx49YpXh2rnwq90aJcE@mail.gmail.com>
	<AANLkTin2yFJqD_OaRdTGOmYGry815bxzh_pUnHBZr_2x@mail.gmail.com>
	<ifer1q$b76$1@dough.gmane.org>
	<AANLkTi=Y+nvzUTiGBTw5UOxgdn0oR7M54HdX++oo-ZBt@mail.gmail.com>
	<iff0uj$1vm$1@dough.gmane.org>
Message-ID: <AANLkTin-hbx9sEiMuSC4Fkz+zSiyfmt3wMKBT-YBdPOG@mail.gmail.com>

2010/12/29 Georg Brandl <g.brandl at gmx.net>:
>> What worries me more is the requirement to find the correct branch before I can
>> even edit the file. How would you, Georg, deal with doc patches
>> (typos, bad markup)?
>
> Finding the correct branch is not really hard. ?Bugfixes go to maintenance,
> features to trunk.

This works well indeed, provided there is only one maintenance branch.

> You need to think about which category your change is
> right now too, when deciding what to backport/svnmerge.

No, today this decision can take place after the development,
some tickets got reopened because a backport was needed.

-- 
Amaury Forgeot d'Arc

From g.brandl at gmx.net  Wed Dec 29 11:16:44 2010
From: g.brandl at gmx.net (Georg Brandl)
Date: Wed, 29 Dec 2010 11:16:44 +0100
Subject: [Python-Dev] Backport troubles with mercurial
In-Reply-To: <AANLkTin-hbx9sEiMuSC4Fkz+zSiyfmt3wMKBT-YBdPOG@mail.gmail.com>
References: <AANLkTi=+_qOSRo2tj12sec9UKb=hjJar+-8VQEr8_7BB@mail.gmail.com>	<AANLkTik-3LiS8QWhg2J26+gRkjx49YpXh2rnwq90aJcE@mail.gmail.com>	<AANLkTin2yFJqD_OaRdTGOmYGry815bxzh_pUnHBZr_2x@mail.gmail.com>	<ifer1q$b76$1@dough.gmane.org>	<AANLkTi=Y+nvzUTiGBTw5UOxgdn0oR7M54HdX++oo-ZBt@mail.gmail.com>	<iff0uj$1vm$1@dough.gmane.org>
	<AANLkTin-hbx9sEiMuSC4Fkz+zSiyfmt3wMKBT-YBdPOG@mail.gmail.com>
Message-ID: <iff1s4$4vr$1@dough.gmane.org>

Am 29.12.2010 11:09, schrieb Amaury Forgeot d'Arc:
> 2010/12/29 Georg Brandl <g.brandl at gmx.net>:
>>> What worries me more is the requirement to find the correct branch before I can
>>> even edit the file. How would you, Georg, deal with doc patches
>>> (typos, bad markup)?
>>
>> Finding the correct branch is not really hard.  Bugfixes go to maintenance,
>> features to trunk.
> 
> This works well indeed, provided there is only one maintenance branch.

Which there is, except for security fixes (but they are rare and require release
manager intervention anyway).

>> You need to think about which category your change is
>> right now too, when deciding what to backport/svnmerge.
> 
> No, today this decision can take place after the development,
> some tickets got reopened because a backport was needed.

A change can of course also be transplanted after the fact.  It requires another
merge, but usually a trivial one.

Of course you may have to think a bit more about bugfix *before* plunging into
the code, but is that necessarily a bad thing?  We had quite a few bugfixes not
backported precisely because it is very easy not to care about them right now.

Georg


From victor.stinner at haypocalc.com  Wed Dec 29 14:17:33 2010
From: victor.stinner at haypocalc.com (Victor Stinner)
Date: Wed, 29 Dec 2010 14:17:33 +0100
Subject: [Python-Dev] Issue #10348: concurrent.futures doesn't work on BSD
Message-ID: <1293628653.16756.13.camel@marge>

Hi,

FreeBSD 7.2 3.x buildbot is red since some weeks (or months?) because of
a concurrent.futures failure. The problem is that
test_concurrent_futures uses many (multiprocessing) POSIX semaphores,
whereas POSIX semaphores support in FreeBSD is recent and limited. We
have to use SysV semaphores (ftok, semget, semop, semctl, ...) instead
of POSIX semaphores (sem_open, sem_getvalue, sem_unlink, ...). See:

 * http://bugs.python.org/issue10348
 * "Too many open files" errors on "x86 FreeBSD 7.2 3.x" buildbot
   ^-- thread in python-dev opened last month

I would like to know if it should be considered as a release blocker.
Georg Brandl said yes on IRC. Does anyone know SysV API? I tried to
write a patch, but I never used semaphores (POSIX or SysV).

There is a third party module which looks complete and stable:
http://semanchuk.com/philip/sysv_ipc/

It is released under the BSD license. It supports semaphores, but also
shared memory and message queues. We don't need all of those, semaphores
would be enough. I added its author (Philip Semanchuk) to this thread.

I don't know how we should decide to use POSIX or SysV semaphores. It
looks like SysV is preferred on FreeBSD and Darwin (and maybe all BSD
based OSes), and POSIX is preferred on Linux.

Victor


From martin at v.loewis.de  Wed Dec 29 14:20:18 2010
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Wed, 29 Dec 2010 14:20:18 +0100
Subject: [Python-Dev] Possible optimization for LOAD_FAST ?
In-Reply-To: <AANLkTin05vNDnuP53OefqYvpq6S5cv2aEF5MdXkMshbD@mail.gmail.com>
References: <AANLkTin05vNDnuP53OefqYvpq6S5cv2aEF5MdXkMshbD@mail.gmail.com>
Message-ID: <4D1B3592.3040403@v.loewis.de>

Am 28.12.2010 18:08, schrieb Lukas Lueg:
> Also, the
> load_fast in lne 22 to reference x could be taken out of the loop as x
> will always point to the same object....

That's not true; a debugger may change the value of x.

Regards,
Martin

From cournape at gmail.com  Wed Dec 29 14:40:50 2010
From: cournape at gmail.com (David Cournapeau)
Date: Wed, 29 Dec 2010 22:40:50 +0900
Subject: [Python-Dev] Backport troubles with mercurial
In-Reply-To: <AANLkTin2yFJqD_OaRdTGOmYGry815bxzh_pUnHBZr_2x@mail.gmail.com>
References: <AANLkTi=+_qOSRo2tj12sec9UKb=hjJar+-8VQEr8_7BB@mail.gmail.com>
	<AANLkTik-3LiS8QWhg2J26+gRkjx49YpXh2rnwq90aJcE@mail.gmail.com>
	<AANLkTin2yFJqD_OaRdTGOmYGry815bxzh_pUnHBZr_2x@mail.gmail.com>
Message-ID: <AANLkTi=nXGg6RMT1EjdLcpf_bGf52V4Ypx3Yty-txeVA@mail.gmail.com>

On Wed, Dec 29, 2010 at 5:02 PM, Amaury Forgeot d'Arc
<amauryfa at gmail.com> wrote:
> 2010/12/29 David Cournapeau <cournape at gmail.com>
>>
>> The easiest way I found to emulate git cherry-pick (which does exactly
>> what you want) with hg is to use import/export commands:
>> http://mercurial.selenic.com/wiki/CommunicatingChanges
>>
>> It is indeed quite a pain to use in my experience, because you cannot
>> easily refer to the original commit the cherry pick is coming from
>> (i.e. no equivalent to git cherry-pick -x), and the conflict
>> resolution is quite dumb.
>
> This is precisely why I proposed a specific example.
> Which precise steps do you take?
> How much typing or manual copy/paste is required for this very simple case?
> Can you do the merge in less than 10 minutes?

I don't know in this specific case. As I said, when I have to use hg,
that's the technique I use, and you get the issue you mention. That's
a hg limitation AFAICS.

> And finally the biased question:
> can you find one improvement over the current situation with svn?

I am not involved in the hg conversion process nor its decision (I am
not even a python committer). Cherry picking is actually easier to do
with svn by "accident", because its merge method, up to 1.5 at least,
was really dumb and never remembered the ancestors of a previous
merge.

As for a few data points which may or may not be relevant:  in numpy
we convereted from  svn -> git recently, and it has worked pretty
well, with numerous new contributions happening, and better, new
contributors appearing. I have been the release manager for numpy for
several years, and as such had to do the kind of tasks you mention
numerous times with svn, and the only words that comes to mind when
remember this period would not be appropriate on a public mailing
list: I always found svn to be horrible. I started using git to make
my life as release manager simpler, actually. I would be surprised if
python's situation did not end up being similar to numpy's one. Other
projects related to numpy made the changes to a DVCS (ipython, nipy,
lean scikit) before and none of them ever regretted it AFAIK, and
sometimes the people who become the most vocal proponents of the new
tool were the most sceptic ones before. Certainly, very few people if
any asked to revert the process.

cheers,

David

From stephen at xemacs.org  Wed Dec 29 15:12:28 2010
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Wed, 29 Dec 2010 23:12:28 +0900
Subject: [Python-Dev] Backport troubles with mercurial
In-Reply-To: <AANLkTin2yFJqD_OaRdTGOmYGry815bxzh_pUnHBZr_2x@mail.gmail.com>
References: <AANLkTi=+_qOSRo2tj12sec9UKb=hjJar+-8VQEr8_7BB@mail.gmail.com>
	<AANLkTik-3LiS8QWhg2J26+gRkjx49YpXh2rnwq90aJcE@mail.gmail.com>
	<AANLkTin2yFJqD_OaRdTGOmYGry815bxzh_pUnHBZr_2x@mail.gmail.com>
Message-ID: <87wrmsk6r7.fsf@uwakimon.sk.tsukuba.ac.jp>

Amaury Forgeot d'Arc writes:

 > Which precise steps do you take [to cherrypick with export/import]?

hg export -r $CHERRY .tmp.patch
xemacs .tmp.patch
;; Move to end of log message, type "Cherry-picked from: ", then
;; C-u M-! hg id -i -r $CHERRY RET C-x C-s C-x C-c>
hg import .tmp.patch

worked for me the one time I tried it, IIRC.  Now I use patch queues,
and avoid cherry-picking as much as possible.  (Avoiding cherry-pick
is not going to be possible for 3.x <-> 2.x porting, of course, but in
many cases there a simple patch application is not going to work at
all, anyway.  Should be relatively rare, no?)

CHERRY can be a tag, revno, or other alias for the revision.  You may
want to add other identifying information with additional arguments to
"hg id", but "hg id -i" give the canonical ID.  You may want to fix up
committer and date using the -u and -d arguments to hg import.

 > How much typing or manual copy/paste is required for this very simple case?

For the simple case, the above can be scripted, and the script takes
one argument, CHERRY.  It would not be hard to automate adding the
cherry-pick notice.

 > Can you do the merge in less than 10 minutes?

If no conflicts, sure, no problem.  Probably about 30 seconds,
including adding the cherrypicked revid to the log message.  With a
script, maybe 1 second.

If there are conflicts, it depends on the complexity of the conflicts.
I can imagine this could get hairy with the export | import method and
a megapatch, but I've never had to deal with that in practice.  This
is one reason why it's recommended that feature branches be
constructed of multiple, clearly separated patches.  (This may be
difficult to do for complex changes, but it's usually a reasonable
goal.)

 > And finally the biased question:
 > can you find one improvement over the current situation with svn?

No, I can't find just one.  How many are you willing to read?<wink>

svn has the advantage (and yes, it's a big one) that developers are
already used to it.  Otherwise, everything goes in favor of any of the
dVCSes.

 > - even if there was one, there is the problem of changes
 >   specifically made for 2.7 that make no sense in py3k.

This is a problem with the committer, not with the VCS.  Such changes
belong in a separate commit, with an appropriate log entry, in *any*
version control system.  If that is done, there's no problem, you just
don't cherrypick that change.

 >   You'd have to perform a "dummy merge" to py3k which has the
 >   disadvantage of cluttering the py3k change log.

I don't see why.  Such dummy merges can help only if you are going to
rely on hg merge.  But you are not going to be merging 2.x and 3.x.

True, some patches from the maintenance branch won't be applied to the
trunk.  This is expected, and it is a problem that needs to be solved,
probably with a tool like svnmerge, which will do a dummy merge or a
revert commit to exclude that revision from any future merge to trunk.
Therefore, if we change our minds, it's perfectly safe to cherrypick.
Any conflicts that occur are real conflicts, not merge artifacts.

 >   And think of the horror if someone forgets this dummy merge.

What horror?  By cherry-picking you have assumed responsibility for
managing future cherry-picks from that source to that target in any
case; you can no longer expect reliable help from Mercurial.  It's not
at all clear to me that a dummy merge could accomplish anything useful,
especially in the case of 2.x <-> 3.x cherry-picks.

If on the other hand you're thinking of the merge-from-maint-branch
case, as I say this is a problem that needs solving anyway, just as it
did with svn.



From amauryfa at gmail.com  Wed Dec 29 15:17:24 2010
From: amauryfa at gmail.com (Amaury Forgeot d'Arc)
Date: Wed, 29 Dec 2010 15:17:24 +0100
Subject: [Python-Dev] Backport troubles with mercurial
In-Reply-To: <iff1s4$4vr$1@dough.gmane.org>
References: <AANLkTi=+_qOSRo2tj12sec9UKb=hjJar+-8VQEr8_7BB@mail.gmail.com>
	<AANLkTik-3LiS8QWhg2J26+gRkjx49YpXh2rnwq90aJcE@mail.gmail.com>
	<AANLkTin2yFJqD_OaRdTGOmYGry815bxzh_pUnHBZr_2x@mail.gmail.com>
	<ifer1q$b76$1@dough.gmane.org>
	<AANLkTi=Y+nvzUTiGBTw5UOxgdn0oR7M54HdX++oo-ZBt@mail.gmail.com>
	<iff0uj$1vm$1@dough.gmane.org>
	<AANLkTin-hbx9sEiMuSC4Fkz+zSiyfmt3wMKBT-YBdPOG@mail.gmail.com>
	<iff1s4$4vr$1@dough.gmane.org>
Message-ID: <AANLkTi=weiwpLRvwJO8+rN4pqe8kt53_QOB6vi8hwoa0@mail.gmail.com>

2010/12/29 Georg Brandl <g.brandl at gmx.net>:
>>> You need to think about which category your change is
>>> right now too, when deciding what to backport/svnmerge.
>>
>> No, today this decision can take place after the development,
>> some tickets got reopened because a backport was needed.
>
> A change can of course also be transplanted after the fact. ?It requires another
> merge, but usually a trivial one.

No, it's not trivial with hg: this is the very subject of the thread!

-- 
Amaury Forgeot d'Arc

From amauryfa at gmail.com  Wed Dec 29 15:29:28 2010
From: amauryfa at gmail.com (Amaury Forgeot d'Arc)
Date: Wed, 29 Dec 2010 15:29:28 +0100
Subject: [Python-Dev] Backport troubles with mercurial
In-Reply-To: <AANLkTi=nXGg6RMT1EjdLcpf_bGf52V4Ypx3Yty-txeVA@mail.gmail.com>
References: <AANLkTi=+_qOSRo2tj12sec9UKb=hjJar+-8VQEr8_7BB@mail.gmail.com>
	<AANLkTik-3LiS8QWhg2J26+gRkjx49YpXh2rnwq90aJcE@mail.gmail.com>
	<AANLkTin2yFJqD_OaRdTGOmYGry815bxzh_pUnHBZr_2x@mail.gmail.com>
	<AANLkTi=nXGg6RMT1EjdLcpf_bGf52V4Ypx3Yty-txeVA@mail.gmail.com>
Message-ID: <AANLkTikR=G3gnzFsS8XLwJHL1HVrp9sibbaLCXkdTzxz@mail.gmail.com>

2010/12/29 David Cournapeau <cournape at gmail.com>:
> On Wed, Dec 29, 2010 at 5:02 PM, Amaury Forgeot d'Arc
> <amauryfa at gmail.com> wrote:
>> 2010/12/29 David Cournapeau <cournape at gmail.com>
>>>
>>> The easiest way I found to emulate git cherry-pick (which does exactly
>>> what you want) with hg is to use import/export commands:
>>> http://mercurial.selenic.com/wiki/CommunicatingChanges
>>>
>>> It is indeed quite a pain to use in my experience, because you cannot
>>> easily refer to the original commit the cherry pick is coming from
>>> (i.e. no equivalent to git cherry-pick -x), and the conflict
>>> resolution is quite dumb.
>>
>> This is precisely why I proposed a specific example.
>> Which precise steps do you take?
>> How much typing or manual copy/paste is required for this very simple case?
>> Can you do the merge in less than 10 minutes?
>
> I don't know in this specific case. As I said, when I have to use hg,
> that's the technique I use, and you get the issue you mention. That's
> a hg limitation AFAICS.

Yes, Georg identified three things that "hg transplant" should do better:
- an option to not commit
- a way to add conflict markers in the source instead of the .rej file
  (In this case, it may be just as easy to use the standard merge tools)
- somehow share the "transplants" cache file between clones.

> sometimes the people who become the most vocal proponents of the new
> tool were the most sceptic ones before.

I really was not sceptic before, and I certainly don't want to become one!
But yesterday I was blocked the whole afternoon by something
I still call an routine task with most other SCMs; and the only answer I
get is "that's right, it's a pain"

hg will certainly impose a change in the way we develop Python.
I'm not sure everybody understands the consequences.

-- 
Amaury Forgeot d'Arc

From g.brandl at gmx.net  Wed Dec 29 15:57:10 2010
From: g.brandl at gmx.net (Georg Brandl)
Date: Wed, 29 Dec 2010 15:57:10 +0100
Subject: [Python-Dev] Backport troubles with mercurial
In-Reply-To: <AANLkTi=weiwpLRvwJO8+rN4pqe8kt53_QOB6vi8hwoa0@mail.gmail.com>
References: <AANLkTi=+_qOSRo2tj12sec9UKb=hjJar+-8VQEr8_7BB@mail.gmail.com>	<AANLkTik-3LiS8QWhg2J26+gRkjx49YpXh2rnwq90aJcE@mail.gmail.com>	<AANLkTin2yFJqD_OaRdTGOmYGry815bxzh_pUnHBZr_2x@mail.gmail.com>	<ifer1q$b76$1@dough.gmane.org>	<AANLkTi=Y+nvzUTiGBTw5UOxgdn0oR7M54HdX++oo-ZBt@mail.gmail.com>	<iff0uj$1vm$1@dough.gmane.org>	<AANLkTin-hbx9sEiMuSC4Fkz+zSiyfmt3wMKBT-YBdPOG@mail.gmail.com>	<iff1s4$4vr$1@dough.gmane.org>
	<AANLkTi=weiwpLRvwJO8+rN4pqe8kt53_QOB6vi8hwoa0@mail.gmail.com>
Message-ID: <iffi9s$d6g$1@dough.gmane.org>

Am 29.12.2010 15:17, schrieb Amaury Forgeot d'Arc:
> 2010/12/29 Georg Brandl <g.brandl at gmx.net>:
>>>> You need to think about which category your change is
>>>> right now too, when deciding what to backport/svnmerge.
>>>
>>> No, today this decision can take place after the development,
>>> some tickets got reopened because a backport was needed.
>>
>> A change can of course also be transplanted after the fact.  It requires another
>> merge, but usually a trivial one.
> 
> No, it's not trivial with hg: this is the very subject of the thread!

I don't understand: if you make the same change in two branches, but
in different changesets, and then merge these branches, Mercurial will
usually notice that and not trouble you with conflicts.

Georg


From g.brandl at gmx.net  Wed Dec 29 16:01:38 2010
From: g.brandl at gmx.net (Georg Brandl)
Date: Wed, 29 Dec 2010 16:01:38 +0100
Subject: [Python-Dev] Issue #10348: concurrent.futures doesn't work on
	BSD
In-Reply-To: <1293628653.16756.13.camel@marge>
References: <1293628653.16756.13.camel@marge>
Message-ID: <iffii9$d6g$2@dough.gmane.org>

Am 29.12.2010 14:17, schrieb Victor Stinner:
> Hi,
> 
> FreeBSD 7.2 3.x buildbot is red since some weeks (or months?) because of
> a concurrent.futures failure. The problem is that
> test_concurrent_futures uses many (multiprocessing) POSIX semaphores,
> whereas POSIX semaphores support in FreeBSD is recent and limited. We
> have to use SysV semaphores (ftok, semget, semop, semctl, ...) instead
> of POSIX semaphores (sem_open, sem_getvalue, sem_unlink, ...). See:
> 
>  * http://bugs.python.org/issue10348
>  * "Too many open files" errors on "x86 FreeBSD 7.2 3.x" buildbot
>    ^-- thread in python-dev opened last month
> 
> I would like to know if it should be considered as a release blocker.
> Georg Brandl said yes on IRC.

Under the condition that it is within reason to fix it before the
release.

Georg


From amauryfa at gmail.com  Wed Dec 29 16:05:40 2010
From: amauryfa at gmail.com (Amaury Forgeot d'Arc)
Date: Wed, 29 Dec 2010 16:05:40 +0100
Subject: [Python-Dev] Backport troubles with mercurial
In-Reply-To: <iffi9s$d6g$1@dough.gmane.org>
References: <AANLkTi=+_qOSRo2tj12sec9UKb=hjJar+-8VQEr8_7BB@mail.gmail.com>
	<AANLkTik-3LiS8QWhg2J26+gRkjx49YpXh2rnwq90aJcE@mail.gmail.com>
	<AANLkTin2yFJqD_OaRdTGOmYGry815bxzh_pUnHBZr_2x@mail.gmail.com>
	<ifer1q$b76$1@dough.gmane.org>
	<AANLkTi=Y+nvzUTiGBTw5UOxgdn0oR7M54HdX++oo-ZBt@mail.gmail.com>
	<iff0uj$1vm$1@dough.gmane.org>
	<AANLkTin-hbx9sEiMuSC4Fkz+zSiyfmt3wMKBT-YBdPOG@mail.gmail.com>
	<iff1s4$4vr$1@dough.gmane.org>
	<AANLkTi=weiwpLRvwJO8+rN4pqe8kt53_QOB6vi8hwoa0@mail.gmail.com>
	<iffi9s$d6g$1@dough.gmane.org>
Message-ID: <AANLkTikdtUWjBLM1uD_vnrYbPSLkQ40Ua9vbHxAJ8GU-@mail.gmail.com>

2010/12/29 Georg Brandl <g.brandl at gmx.net>:
>>> A change can of course also be transplanted after the fact. ?It requires another
>>> merge, but usually a trivial one.
>>
>> No, it's not trivial with hg: this is the very subject of the thread!
>
> I don't understand: if you make the same change in two branches, but
> in different changesets, and then merge these branches, Mercurial will
> usually notice that and not trouble you with conflicts.

Actually I never passed the first step: make the same change to two branches.

-- 
Amaury Forgeot d'Arc

From philip at semanchuk.com  Wed Dec 29 15:24:07 2010
From: philip at semanchuk.com (Philip Semanchuk)
Date: Wed, 29 Dec 2010 09:24:07 -0500
Subject: [Python-Dev] Issue #10348: concurrent.futures doesn't work on
	BSD
In-Reply-To: <1293628653.16756.13.camel@marge>
References: <1293628653.16756.13.camel@marge>
Message-ID: <ACF12CD1-4F3C-4BE6-8687-8626C8A03E84@semanchuk.com>


On Dec 29, 2010, at 8:17 AM, Victor Stinner wrote:

> Hi,
> 
> FreeBSD 7.2 3.x buildbot is red since some weeks (or months?) because of
> a concurrent.futures failure. The problem is that
> test_concurrent_futures uses many (multiprocessing) POSIX semaphores,
> whereas POSIX semaphores support in FreeBSD is recent and limited. We
> have to use SysV semaphores (ftok, semget, semop, semctl, ...) instead
> of POSIX semaphores (sem_open, sem_getvalue, sem_unlink, ...). See:
> 
> * http://bugs.python.org/issue10348
> * "Too many open files" errors on "x86 FreeBSD 7.2 3.x" buildbot
>   ^-- thread in python-dev opened last month
> 
> I would like to know if it should be considered as a release blocker.
> Georg Brandl said yes on IRC. Does anyone know SysV API? I tried to
> write a patch, but I never used semaphores (POSIX or SysV).
> 
> There is a third party module which looks complete and stable:
> http://semanchuk.com/philip/sysv_ipc/
> 
> It is released under the BSD license. It supports semaphores, but also
> shared memory and message queues. We don't need all of those, semaphores
> would be enough. I added its author (Philip Semanchuk) to this thread.

Hi all,
What Victor says above is correct, although I wasn't aware that POSIX IPC under FreeBSD 7.2 was still having problems. Prior to 7.2 it was broken but 7.2 worked OK in my limited testing. In any case, the sysv_ipc module is mine and it's mature and you're welcome to pillage it in whole or in part.


> I don't know how we should decide to use POSIX or SysV semaphores. It
> looks like SysV is preferred on FreeBSD and Darwin (and maybe all BSD
> based OSes), and POSIX is preferred on Linux.

Hmmm, "preferred" is a tricky word here. I would phrase it slightly differently: POSIX IPC is preferred everywhere (by me, at least) because it's a more modern API. However, SysV IPC is fully supported everywhere while the same can't be said about POSIX IPC. 

Speaking of POSIX IPC, I also have a posix_ipc module that's quite similar to sysv_ipc and the platform notes in the documentation tell one what support to expect for POSIX IPC under various platforms:
http://semanchuk.com/philip/posix_ipc/#platforms

Cheers
Philip


From rdmurray at bitdance.com  Wed Dec 29 16:25:10 2010
From: rdmurray at bitdance.com (R. David Murray)
Date: Wed, 29 Dec 2010 10:25:10 -0500
Subject: [Python-Dev] Backport troubles with mercurial
In-Reply-To: <87wrmsk6r7.fsf@uwakimon.sk.tsukuba.ac.jp>
References: <AANLkTi=+_qOSRo2tj12sec9UKb=hjJar+-8VQEr8_7BB@mail.gmail.com>
	<AANLkTik-3LiS8QWhg2J26+gRkjx49YpXh2rnwq90aJcE@mail.gmail.com>
	<AANLkTin2yFJqD_OaRdTGOmYGry815bxzh_pUnHBZr_2x@mail.gmail.com>
	<87wrmsk6r7.fsf@uwakimon.sk.tsukuba.ac.jp>
Message-ID: <20101229152510.1E766239DB5@kimball.webabinitio.net>

On Wed, 29 Dec 2010 23:12:28 +0900, "Stephen J. Turnbull" <stephen at xemacs.org> wrote:
> worked for me the one time I tried it, IIRC.  Now I use patch queues,
> and avoid cherry-picking as much as possible.  (Avoiding cherry-pick
> is not going to be possible for 3.x <-> 2.x porting, of course, but in
> many cases there a simple patch application is not going to work at
> all, anyway.  Should be relatively rare, no?)

No.

We merge bug fixes to 2.7 on a routine basis.  It is the rule rather
than the exception, by a long shot.  (Features of course only go into
trunk...but those are irrelevant to this conversation, since there's no
requirement to merge them anywhere...as are bug fixes we choose not to
backport, since those don't go into the Python3 maint branch either.)

Some such merges apply cleanly, many more have only a few conflicts
requiring only trivial tweaks, which are made easy by the in-line merge
conflict markers.  A few require re-engineering of the patch, and of
course it is a bit of a pain with svnmerge to deal with the ones where
the target file names have changed (but again, there aren't that many
of those). [*]

So, since we are going to be maintaining 2.7 for a while, this is
a workflow problem that we *must* solve to make the hg transition
worthwhile.  I have no doubt that we will, but I also have no doubt we
need to *solve* it, not just wave our hands.  Many thanks to Georg and
Djirkan for working on the problem(s).

--
R. David Murray                                      www.bitdance.com

[*] I'm speaking of stdlib changes, which is what I do, but I suspect
the situation is similar for the C code, and certainly the fact that bug
fixes backported to Python3 maint are routinely backported to 2.7 is true.

From martin at v.loewis.de  Wed Dec 29 16:28:20 2010
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Wed, 29 Dec 2010 16:28:20 +0100
Subject: [Python-Dev] Issue #10348: concurrent.futures doesn't work on
 BSD
In-Reply-To: <iffii9$d6g$2@dough.gmane.org>
References: <1293628653.16756.13.camel@marge> <iffii9$d6g$2@dough.gmane.org>
Message-ID: <4D1B5394.10802@v.loewis.de>

>> I would like to know if it should be considered as a release blocker.
>> Georg Brandl said yes on IRC.
> 
> Under the condition that it is within reason to fix it before the
> release.

What *should* be possible is to disable building
SemLock/multiprocessing.synchronize on FreeBSD. As a consequence,
multiprocessing locks would stop working on FreeBSD, and concurrent
futures; the tests would recognize this lack of features and get
skipped.

Regards,
Martin

From martin at v.loewis.de  Wed Dec 29 16:43:11 2010
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Wed, 29 Dec 2010 16:43:11 +0100
Subject: [Python-Dev] Issue #10348: concurrent.futures doesn't work on
 BSD
In-Reply-To: <ACF12CD1-4F3C-4BE6-8687-8626C8A03E84@semanchuk.com>
References: <1293628653.16756.13.camel@marge>
	<ACF12CD1-4F3C-4BE6-8687-8626C8A03E84@semanchuk.com>
Message-ID: <4D1B570F.1000502@v.loewis.de>

> Hi all, What Victor says above is correct, although I wasn't aware
> that POSIX IPC under FreeBSD 7.2 was still having problems. Prior to
> 7.2 it was broken but 7.2 worked OK in my limited testing. In any
> case, the sysv_ipc module is mine and it's mature and you're welcome
> to pillage it in whole or in part.

The question really is whether you would be willing to port
Modules/_multiprocessing/semaphore.c to SysV IPC. I realize that
this would be practically new code, but it would be much appreciated.

One question is whether it's actually feasible to implement that API
with SysV semaphores.

>> I don't know how we should decide to use POSIX or SysV semaphores.
>> It looks like SysV is preferred on FreeBSD and Darwin (and maybe
>> all BSD based OSes), and POSIX is preferred on Linux.
> 
> Hmmm, "preferred" is a tricky word here. I would phrase it slightly
> differently: POSIX IPC is preferred everywhere (by me, at least)
> because it's a more modern API. However, SysV IPC is fully supported
> everywhere while the same can't be said about POSIX IPC.

If you can make the above change, the question then is what API
multiprocessing semaphores should be built upon. It seems that you
are saying that they should use SysV IPC, and only fall back to
POSIX IPC if SysV IPC doesn't work/exist (are there any platforms
where this would be the case?)

Alternatively, we could have a whitelist of systems on which
POSIX IPC is used (==['linux']), and use sysv ipc everywhere else.

Regards,
Martin

From jnoller at gmail.com  Wed Dec 29 18:54:57 2010
From: jnoller at gmail.com (Jesse Noller)
Date: Wed, 29 Dec 2010 12:54:57 -0500
Subject: [Python-Dev] Issue #10348: concurrent.futures doesn't work on
	BSD
In-Reply-To: <4D1B5394.10802@v.loewis.de>
References: <1293628653.16756.13.camel@marge> <iffii9$d6g$2@dough.gmane.org>
	<4D1B5394.10802@v.loewis.de>
Message-ID: <AANLkTin+SLFm7c9nyEmtGd4Ocgatkhindnc-APBfYvm6@mail.gmail.com>

On Wed, Dec 29, 2010 at 10:28 AM, "Martin v. L?wis" <martin at v.loewis.de> wrote:
>>> I would like to know if it should be considered as a release blocker.
>>> Georg Brandl said yes on IRC.
>>
>> Under the condition that it is within reason to fix it before the
>> release.
>
> What *should* be possible is to disable building
> SemLock/multiprocessing.synchronize on FreeBSD. As a consequence,
> multiprocessing locks would stop working on FreeBSD, and concurrent
> futures; the tests would recognize this lack of features and get
> skipped.
>
> Regards,
> Martin

The multiprocessing test suite already skips the tests which use the
(broken) functionality on BSD correctly. This logic needs to be added
to the concurrent.futures library.

jesse

From jnoller at gmail.com  Wed Dec 29 18:58:55 2010
From: jnoller at gmail.com (Jesse Noller)
Date: Wed, 29 Dec 2010 12:58:55 -0500
Subject: [Python-Dev] Issue #10348: concurrent.futures doesn't work on
	BSD
In-Reply-To: <1293628653.16756.13.camel@marge>
References: <1293628653.16756.13.camel@marge>
Message-ID: <AANLkTinXTEK1bm79Wef+2ZRa9nMyX0gS_viq+F+f3P3m@mail.gmail.com>

On Wed, Dec 29, 2010 at 8:17 AM, Victor Stinner
<victor.stinner at haypocalc.com> wrote:
> Hi,
>
> FreeBSD 7.2 3.x buildbot is red since some weeks (or months?) because of
> a concurrent.futures failure. The problem is that
> test_concurrent_futures uses many (multiprocessing) POSIX semaphores,
> whereas POSIX semaphores support in FreeBSD is recent and limited. We
> have to use SysV semaphores (ftok, semget, semop, semctl, ...) instead
> of POSIX semaphores (sem_open, sem_getvalue, sem_unlink, ...). See:
>
> ?* http://bugs.python.org/issue10348
> ?* "Too many open files" errors on "x86 FreeBSD 7.2 3.x" buildbot
> ? ^-- thread in python-dev opened last month
>
> I would like to know if it should be considered as a release blocker.
> Georg Brandl said yes on IRC. Does anyone know SysV API? I tried to
> write a patch, but I never used semaphores (POSIX or SysV).
>
> There is a third party module which looks complete and stable:
> http://semanchuk.com/philip/sysv_ipc/
>
> It is released under the BSD license. It supports semaphores, but also
> shared memory and message queues. We don't need all of those, semaphores
> would be enough. I added its author (Philip Semanchuk) to this thread.
>
> I don't know how we should decide to use POSIX or SysV semaphores. It
> looks like SysV is preferred on FreeBSD and Darwin (and maybe all BSD
> based OSes), and POSIX is preferred on Linux.
>
> Victor

The concurrent.futures tests should (like the multiprocessing test
suite) detect the lack of support and skip the tests on the broken
platforms. I'm sort of surprised FreeBSD support is still broken in
this way though (echoed by Philip) although it could be an issue on
that particular buildbot.

Moving from POSIX IPC to SYSV should *not* be on the plate for a
release blocker - that's a much larger task.

jesse

From martin at v.loewis.de  Wed Dec 29 19:34:31 2010
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Wed, 29 Dec 2010 19:34:31 +0100
Subject: [Python-Dev] Issue #10348: concurrent.futures doesn't work on
 BSD
In-Reply-To: <AANLkTin+SLFm7c9nyEmtGd4Ocgatkhindnc-APBfYvm6@mail.gmail.com>
References: <1293628653.16756.13.camel@marge> <iffii9$d6g$2@dough.gmane.org>
	<4D1B5394.10802@v.loewis.de>
	<AANLkTin+SLFm7c9nyEmtGd4Ocgatkhindnc-APBfYvm6@mail.gmail.com>
Message-ID: <4D1B7F37.3080201@v.loewis.de>

Am 29.12.2010 18:54, schrieb Jesse Noller:
> On Wed, Dec 29, 2010 at 10:28 AM, "Martin v. L?wis" <martin at v.loewis.de> wrote:
>>>> I would like to know if it should be considered as a release blocker.
>>>> Georg Brandl said yes on IRC.
>>>
>>> Under the condition that it is within reason to fix it before the
>>> release.
>>
>> What *should* be possible is to disable building
>> SemLock/multiprocessing.synchronize on FreeBSD. As a consequence,
>> multiprocessing locks would stop working on FreeBSD, and concurrent
>> futures; the tests would recognize this lack of features and get
>> skipped.
>>
>> Regards,
>> Martin
> 
> The multiprocessing test suite already skips the tests which use the
> (broken) functionality on BSD correctly. This logic needs to be added
> to the concurrent.futures library.

I'm not so sure that skipping the test is the right approach. Doesn't
that mean that the code will still fail at runtime with
difficult-to-explain messages? I'd rather prefer if the functionality
wasn't available in the first place.

Also, what specific test are you referring to?

Regards,
Martin

From rdmurray at bitdance.com  Wed Dec 29 20:14:03 2010
From: rdmurray at bitdance.com (R. David Murray)
Date: Wed, 29 Dec 2010 14:14:03 -0500
Subject: [Python-Dev] Issue #10348: concurrent.futures doesn't work on
	BSD
In-Reply-To: <AANLkTinXTEK1bm79Wef+2ZRa9nMyX0gS_viq+F+f3P3m@mail.gmail.com>
References: <1293628653.16756.13.camel@marge>
	<AANLkTinXTEK1bm79Wef+2ZRa9nMyX0gS_viq+F+f3P3m@mail.gmail.com>
Message-ID: <20101229191403.A166F239421@kimball.webabinitio.net>

On Wed, 29 Dec 2010 12:58:55 -0500, Jesse Noller <jnoller at gmail.com> wrote:
> The concurrent.futures tests should (like the multiprocessing test
> suite) detect the lack of support and skip the tests on the broken
> platforms. I'm sort of surprised FreeBSD support is still broken in
> this way though (echoed by Philip) although it could be an issue on
> that particular buildbot.

If I'm reading the issue correctly, it isn't that it doesn't work,
it's that the concurrent.futures tests fail because they create more
semaphores than the default FreeBSD install supports.  In other words,
a user of concurrent.futures really needs to tweak their FreeBSD install
to make in fully functional.

There should be a way (through sysctl, presumably) to query the maximum
number of semaphores and skip the relevant tests on that basis.

--
R. David Murray                                      www.bitdance.com

From tjreedy at udel.edu  Wed Dec 29 20:31:20 2010
From: tjreedy at udel.edu (Terry Reedy)
Date: Wed, 29 Dec 2010 14:31:20 -0500
Subject: [Python-Dev] Compile() and Windows/Mac newlines
Message-ID: <ifg2a9$ob1$1@dough.gmane.org>

In python-list thread "Does Python 3.1 accept \r\n in compile()?"
jmfauth notes that
compile('print(999)\r\n', '<in>', 'exec')
works in 2.7 but not 3.1 (and 3.2 not checked) because 3.1 sees '\r' as
SyntaxError.

I started to respond that this is part of Py3 cleanup with newlines 
converted on input and back-compatibility with ancient Python not 
needed. Then I saw in 3.2 manual

"Changed in version 3.2: Allowed use of Windows and Mac newlines. Also 
input in 'exec' mode does not have to end in a newline anymore. Added 
the optimize parameter."

I verified second statement ("print(999)" works) (and remember commit 
for third), but original above gives same error. Should "Allowed use of 
Windows and Mac newlines." be deleted? What else could it mean other 
than use of '\r' or '\r\n'?

-- 
Terry Jan Reedy


From philip at semanchuk.com  Wed Dec 29 20:33:29 2010
From: philip at semanchuk.com (Philip Semanchuk)
Date: Wed, 29 Dec 2010 14:33:29 -0500
Subject: [Python-Dev] Issue #10348: concurrent.futures doesn't work on
	BSD
In-Reply-To: <20101229191403.A166F239421@kimball.webabinitio.net>
References: <1293628653.16756.13.camel@marge>
	<AANLkTinXTEK1bm79Wef+2ZRa9nMyX0gS_viq+F+f3P3m@mail.gmail.com>
	<20101229191403.A166F239421@kimball.webabinitio.net>
Message-ID: <3C7FBFEC-ECB1-4D94-9233-261B135FE43C@semanchuk.com>


On Dec 29, 2010, at 2:14 PM, R. David Murray wrote:

> On Wed, 29 Dec 2010 12:58:55 -0500, Jesse Noller <jnoller at gmail.com> wrote:
>> The concurrent.futures tests should (like the multiprocessing test
>> suite) detect the lack of support and skip the tests on the broken
>> platforms. I'm sort of surprised FreeBSD support is still broken in
>> this way though (echoed by Philip) although it could be an issue on
>> that particular buildbot.
> 
> If I'm reading the issue correctly, it isn't that it doesn't work,
> it's that the concurrent.futures tests fail because they create more
> semaphores than the default FreeBSD install supports.  In other words,
> a user of concurrent.futures really needs to tweak their FreeBSD install
> to make in fully functional.


I think that's correct. Furthermore, the default 7.2 install doesn't have the necessary kernel modules loaded in order to use POSIX semaphores. See the notes here for FreeBSD:
http://semanchuk.com/philip/posix_ipc/#platforms

kldstat on my barely-customized 7.2 installation gives a list of three modules: kernel, acpi.ko and linux.ko (I assume because I asked for Linux executable format compatibility when I installed). 


bye
Philip

From philip at semanchuk.com  Wed Dec 29 20:37:29 2010
From: philip at semanchuk.com (Philip Semanchuk)
Date: Wed, 29 Dec 2010 14:37:29 -0500
Subject: [Python-Dev] Issue #10348: concurrent.futures doesn't work on
	BSD
In-Reply-To: <4D1B570F.1000502@v.loewis.de>
References: <1293628653.16756.13.camel@marge>
	<ACF12CD1-4F3C-4BE6-8687-8626C8A03E84@semanchuk.com>
	<4D1B570F.1000502@v.loewis.de>
Message-ID: <3C36C60B-717E-4550-AC2E-AB6E2B2E7ECB@semanchuk.com>


On Dec 29, 2010, at 10:43 AM, Martin v. L?wis wrote:

>> Hi all, What Victor says above is correct, although I wasn't aware
>> that POSIX IPC under FreeBSD 7.2 was still having problems. Prior to
>> 7.2 it was broken but 7.2 worked OK in my limited testing. In any
>> case, the sysv_ipc module is mine and it's mature and you're welcome
>> to pillage it in whole or in part.
> 
> The question really is whether you would be willing to port
> Modules/_multiprocessing/semaphore.c to SysV IPC. I realize that
> this would be practically new code, but it would be much appreciated.
> 
> One question is whether it's actually feasible to implement that API
> with SysV semaphores.
> 
>>> I don't know how we should decide to use POSIX or SysV semaphores.
>>> It looks like SysV is preferred on FreeBSD and Darwin (and maybe
>>> all BSD based OSes), and POSIX is preferred on Linux.
>> 
>> Hmmm, "preferred" is a tricky word here. I would phrase it slightly
>> differently: POSIX IPC is preferred everywhere (by me, at least)
>> because it's a more modern API. However, SysV IPC is fully supported
>> everywhere while the same can't be said about POSIX IPC.
> 
> If you can make the above change, the question then is what API
> multiprocessing semaphores should be built upon. It seems that you
> are saying that they should use SysV IPC, and only fall back to
> POSIX IPC if SysV IPC doesn't work/exist (are there any platforms
> where this would be the case?)

Actually, I'd probably recommend the opposite. IMO POSIX IPC is better designed and easier to work with so it would be my first choice. SysV would be a fallback for when POSIX IPC is unavailable, broken, incomplete, etc.


bye
Philip

From benjamin at python.org  Wed Dec 29 20:53:16 2010
From: benjamin at python.org (Benjamin Peterson)
Date: Wed, 29 Dec 2010 13:53:16 -0600
Subject: [Python-Dev] Compile() and Windows/Mac newlines
In-Reply-To: <ifg2a9$ob1$1@dough.gmane.org>
References: <ifg2a9$ob1$1@dough.gmane.org>
Message-ID: <AANLkTikGme+zNNxzmuBzwZrXaDd6fjiXDsqVCYw9MbWO@mail.gmail.com>

2010/12/29 Terry Reedy <tjreedy at udel.edu>:
> In python-list thread "Does Python 3.1 accept \r\n in compile()?"
> jmfauth notes that
> compile('print(999)\r\n', '<in>', 'exec')
> works in 2.7 but not 3.1 (and 3.2 not checked) because 3.1 sees '\r' as
> SyntaxError.
>
> I started to respond that this is part of Py3 cleanup with newlines
> converted on input and back-compatibility with ancient Python not needed.
> Then I saw in 3.2 manual
>
> "Changed in version 3.2: Allowed use of Windows and Mac newlines. Also input
> in 'exec' mode does not have to end in a newline anymore. Added the optimize
> parameter."
>
> I verified second statement ("print(999)" works) (and remember commit for
> third), but original above gives same error. Should "Allowed use of Windows
> and Mac newlines." be deleted? What else could it mean other than use of
> '\r' or '\r\n'?

$ ./python
Python 3.2b2 (py3k:87559, Dec 28 2010, 17:39:51)
[GCC 4.4.4] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> compile("print(999)\r\n", "blah", "exec")
<code object <module> at 0xb353e8, file "blah", line 1>




-- 
Regards,
Benjamin

From tjreedy at udel.edu  Wed Dec 29 21:02:34 2010
From: tjreedy at udel.edu (Terry Reedy)
Date: Wed, 29 Dec 2010 15:02:34 -0500
Subject: [Python-Dev] Compile() and Windows/Mac newlines
In-Reply-To: <ifg2a9$ob1$1@dough.gmane.org>
References: <ifg2a9$ob1$1@dough.gmane.org>
Message-ID: <ifg44r$100$1@dough.gmane.org>

On 12/29/2010 2:31 PM, Terry Reedy wrote:
> In python-list thread "Does Python 3.1 accept \r\n in compile()?"
> jmfauth notes that
> compile('print(999)\r\n', '<in>', 'exec')
> works in 2.7 but not 3.1 (and 3.2 not checked) because 3.1 sees '\r' as
> SyntaxError.
>
> I started to respond that this is part of Py3 cleanup with newlines
> converted on input and back-compatibility with ancient Python not
> needed. Then I saw in 3.2 manual
>
> "Changed in version 3.2: Allowed use of Windows and Mac newlines. Also
> input in 'exec' mode does not have to end in a newline anymore. Added
> the optimize parameter."
>
> I verified second statement ("print(999)" works) (and remember commit
> for third), but original above gives same error. Should "Allowed use of
> Windows and Mac newlines." be deleted? What else could it mean other
> than use of '\r' or '\r\n'?

After tracing the questioned comment to B.Peterson's r76232 merged from 
2.7 r76230 "fix several compile() issues by translating newlines in the 
tokenizer", I decided to open http://bugs.python.org/issue10792

-- 
Terry Jan Reedy


From solipsis at pitrou.net  Wed Dec 29 21:04:11 2010
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Wed, 29 Dec 2010 21:04:11 +0100
Subject: [Python-Dev] Issue #10348: concurrent.futures doesn't work on
	BSD
References: <1293628653.16756.13.camel@marge>
	<AANLkTinXTEK1bm79Wef+2ZRa9nMyX0gS_viq+F+f3P3m@mail.gmail.com>
	<20101229191403.A166F239421@kimball.webabinitio.net>
Message-ID: <20101229210411.4b562b59@pitrou.net>

On Wed, 29 Dec 2010 14:14:03 -0500
"R. David Murray" <rdmurray at bitdance.com> wrote:
> On Wed, 29 Dec 2010 12:58:55 -0500, Jesse Noller <jnoller at gmail.com> wrote:
> > The concurrent.futures tests should (like the multiprocessing test
> > suite) detect the lack of support and skip the tests on the broken
> > platforms. I'm sort of surprised FreeBSD support is still broken in
> > this way though (echoed by Philip) although it could be an issue on
> > that particular buildbot.
> 
> If I'm reading the issue correctly, it isn't that it doesn't work,
> it's that the concurrent.futures tests fail because they create more
> semaphores than the default FreeBSD install supports.

Doesn't it suggest a possible resource leak somewhere? Or do the
concurrent tests really rely on creating many semaphores? (as opposed
to say 5 or 10 of them)

Regards

Antoine.



From tjreedy at udel.edu  Wed Dec 29 21:09:07 2010
From: tjreedy at udel.edu (Terry Reedy)
Date: Wed, 29 Dec 2010 15:09:07 -0500
Subject: [Python-Dev] Compile() and Windows/Mac newlines
In-Reply-To: <AANLkTikGme+zNNxzmuBzwZrXaDd6fjiXDsqVCYw9MbWO@mail.gmail.com>
References: <ifg2a9$ob1$1@dough.gmane.org>
	<AANLkTikGme+zNNxzmuBzwZrXaDd6fjiXDsqVCYw9MbWO@mail.gmail.com>
Message-ID: <ifg4h3$2d2$1@dough.gmane.org>

On 12/29/2010 2:53 PM, Benjamin Peterson wrote:

> Type "help", "copyright", "credits" or "license" for more information.
>>>> compile("print(999)\r\n", "blah", "exec")
> <code object<module>  at 0xb353e8, file "blah", line 1>

I made a mistake in testing. Issue closed. Sorry for the noise.

-- 
Terry Jan Reedy


From jnoller at gmail.com  Wed Dec 29 21:17:32 2010
From: jnoller at gmail.com (Jesse Noller)
Date: Wed, 29 Dec 2010 15:17:32 -0500
Subject: [Python-Dev] Issue #10348: concurrent.futures doesn't work on
	BSD
In-Reply-To: <4D1B7F37.3080201@v.loewis.de>
References: <1293628653.16756.13.camel@marge> <iffii9$d6g$2@dough.gmane.org>
	<4D1B5394.10802@v.loewis.de>
	<AANLkTin+SLFm7c9nyEmtGd4Ocgatkhindnc-APBfYvm6@mail.gmail.com>
	<4D1B7F37.3080201@v.loewis.de>
Message-ID: <AANLkTim20h+UTLsSJVuhOPmCxQvB7RWFb=NZvh9dZNmF@mail.gmail.com>

On Wed, Dec 29, 2010 at 1:34 PM, "Martin v. L?wis" <martin at v.loewis.de> wrote:
> Am 29.12.2010 18:54, schrieb Jesse Noller:
>> On Wed, Dec 29, 2010 at 10:28 AM, "Martin v. L?wis" <martin at v.loewis.de> wrote:
>>>>> I would like to know if it should be considered as a release blocker.
>>>>> Georg Brandl said yes on IRC.
>>>>
>>>> Under the condition that it is within reason to fix it before the
>>>> release.
>>>
>>> What *should* be possible is to disable building
>>> SemLock/multiprocessing.synchronize on FreeBSD. As a consequence,
>>> multiprocessing locks would stop working on FreeBSD, and concurrent
>>> futures; the tests would recognize this lack of features and get
>>> skipped.
>>>
>>> Regards,
>>> Martin
>>
>> The multiprocessing test suite already skips the tests which use the
>> (broken) functionality on BSD correctly. This logic needs to be added
>> to the concurrent.futures library.
>
> I'm not so sure that skipping the test is the right approach. Doesn't
> that mean that the code will still fail at runtime with
> difficult-to-explain messages? I'd rather prefer if the functionality
> wasn't available in the first place.
>
> Also, what specific test are you referring to?
>
> Regards,
> Martin
>

If the functionality is not supported then users get an import error
(within multiprocessing). However, RDM's understanding is correct, and
the test is creating more than supported.

From victor.stinner at haypocalc.com  Wed Dec 29 21:28:26 2010
From: victor.stinner at haypocalc.com (Victor Stinner)
Date: Wed, 29 Dec 2010 21:28:26 +0100
Subject: [Python-Dev] Issue #10348: concurrent.futures doesn't work on
 BSD
In-Reply-To: <20101229210411.4b562b59@pitrou.net>
References: <1293628653.16756.13.camel@marge>
	<AANLkTinXTEK1bm79Wef+2ZRa9nMyX0gS_viq+F+f3P3m@mail.gmail.com>
	<20101229191403.A166F239421@kimball.webabinitio.net>
	<20101229210411.4b562b59@pitrou.net>
Message-ID: <1293654506.18690.15.camel@marge>

Le mercredi 29 d?cembre 2010 ? 21:04 +0100, Antoine Pitrou a ?crit :
> Doesn't it suggest a possible resource leak somewhere?

I already checked that: all locks are destroyed correctly on each test.

 - test_all_completed_some_already_completed() uses 51 SemLock objects
 - test_first_completed() uses 41 SemLock objects
 - test_processes_terminate() uses 41 SemLock objects
 - etc.

FreeBSD has an arbitrary limit of 30 semaphores.

Victor


From martin at v.loewis.de  Wed Dec 29 21:42:16 2010
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Wed, 29 Dec 2010 21:42:16 +0100
Subject: [Python-Dev] Issue #10348: concurrent.futures doesn't work on
 BSD
In-Reply-To: <3C36C60B-717E-4550-AC2E-AB6E2B2E7ECB@semanchuk.com>
References: <1293628653.16756.13.camel@marge>	<ACF12CD1-4F3C-4BE6-8687-8626C8A03E84@semanchuk.com>	<4D1B570F.1000502@v.loewis.de>
	<3C36C60B-717E-4550-AC2E-AB6E2B2E7ECB@semanchuk.com>
Message-ID: <4D1B9D28.9080801@v.loewis.de>

>> If you can make the above change, the question then is what API 
>> multiprocessing semaphores should be built upon. It seems that you 
>> are saying that they should use SysV IPC, and only fall back to 
>> POSIX IPC if SysV IPC doesn't work/exist (are there any platforms 
>> where this would be the case?)
> 
> Actually, I'd probably recommend the opposite. IMO POSIX IPC is
> better designed and easier to work with so it would be my first
> choice.

"better designed and easier to work with" don't count if you have to
do both, anyway. The code will be ugly - so what, once it is written?
It's not being exposed to the user.

> SysV would be a fallback for when POSIX IPC is unavailable,
> broken, incomplete, etc.

The problem seems to be that it is not possible to reliably determine
these cases. In particular, the ridiculous low limits seem to be
difficult to detect, and it's not clear what a required minimum level
should be.

Regards,
Martin

From martin at v.loewis.de  Wed Dec 29 21:43:38 2010
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Wed, 29 Dec 2010 21:43:38 +0100
Subject: [Python-Dev] Issue #10348: concurrent.futures doesn't work on
 BSD
In-Reply-To: <AANLkTim20h+UTLsSJVuhOPmCxQvB7RWFb=NZvh9dZNmF@mail.gmail.com>
References: <1293628653.16756.13.camel@marge>
	<iffii9$d6g$2@dough.gmane.org>	<4D1B5394.10802@v.loewis.de>	<AANLkTin+SLFm7c9nyEmtGd4Ocgatkhindnc-APBfYvm6@mail.gmail.com>	<4D1B7F37.3080201@v.loewis.de>
	<AANLkTim20h+UTLsSJVuhOPmCxQvB7RWFb=NZvh9dZNmF@mail.gmail.com>
Message-ID: <4D1B9D7A.9080003@v.loewis.de>

>>> The multiprocessing test suite already skips the tests which use the
>>> (broken) functionality on BSD correctly. This logic needs to be added
>>> to the concurrent.futures library.
>> Also, what specific test are you referring to?

Can you kindly point me to the test that skips if broken functionality
on BSD is detected?

Regards,
Martin

From martin at v.loewis.de  Wed Dec 29 21:49:38 2010
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Wed, 29 Dec 2010 21:49:38 +0100
Subject: [Python-Dev] Issue #10348: concurrent.futures doesn't work on
 BSD
In-Reply-To: <AANLkTim20h+UTLsSJVuhOPmCxQvB7RWFb=NZvh9dZNmF@mail.gmail.com>
References: <1293628653.16756.13.camel@marge>
	<iffii9$d6g$2@dough.gmane.org>	<4D1B5394.10802@v.loewis.de>	<AANLkTin+SLFm7c9nyEmtGd4Ocgatkhindnc-APBfYvm6@mail.gmail.com>	<4D1B7F37.3080201@v.loewis.de>
	<AANLkTim20h+UTLsSJVuhOPmCxQvB7RWFb=NZvh9dZNmF@mail.gmail.com>
Message-ID: <4D1B9EE2.70806@v.loewis.de>

> If the functionality is not supported then users get an import error
> (within multiprocessing). However, RDM's understanding is correct, and
> the test is creating more than supported.

Hmm. The tests do the absolute minimum stuff that exercises the code;
doing anything less, and they would be useless. Of course, one may
wonder why test_first_completed manages to create 41 SemLock objects,
when all it tries to do is two future calls.

So if the minimal test case fails, I'd claim that the module doesn't
work on FreeBSD, period. ISTM that Posix IPC is just not a feasible
approach to do IPC synchronization on FreeBSD, so it's better to say
that multiprocessing is not supported on FreeBSD (until SysV IPC is
getting used, hoping that this will fare better).

Regards,
Martin

From jnoller at gmail.com  Wed Dec 29 22:34:45 2010
From: jnoller at gmail.com (Jesse Noller)
Date: Wed, 29 Dec 2010 16:34:45 -0500
Subject: [Python-Dev] Issue #10348: concurrent.futures doesn't work on
	BSD
In-Reply-To: <4D1B9EE2.70806@v.loewis.de>
References: <1293628653.16756.13.camel@marge> <iffii9$d6g$2@dough.gmane.org>
	<4D1B5394.10802@v.loewis.de>
	<AANLkTin+SLFm7c9nyEmtGd4Ocgatkhindnc-APBfYvm6@mail.gmail.com>
	<4D1B7F37.3080201@v.loewis.de>
	<AANLkTim20h+UTLsSJVuhOPmCxQvB7RWFb=NZvh9dZNmF@mail.gmail.com>
	<4D1B9EE2.70806@v.loewis.de>
Message-ID: <22257F0D-AE3E-46F5-BE0D-1426D5F908CF@gmail.com>



On Dec 29, 2010, at 3:49 PM, "Martin v. L?wis" <martin at v.loewis.de> wrote:

>> If the functionality is not supported then users get an import error
>> (within multiprocessing). However, RDM's understanding is correct, and
>> the test is creating more than supported.
> 
> Hmm. The tests do the absolute minimum stuff that exercises the code;
> doing anything less, and they would be useless. Of course, one may
> wonder why test_first_completed manages to create 41 SemLock objects,
> when all it tries to do is two future calls.
> 
> So if the minimal test case fails, I'd claim that the module doesn't
> work on FreeBSD, period. ISTM that Posix IPC is just not a feasible
> approach to do IPC synchronization on FreeBSD, so it's better to say
> that multiprocessing is not supported on FreeBSD (until SysV IPC is
> getting used, hoping that this will fare better).
> 

Whatever you choose to say Martin. It does work, and is supported to the limitations that FreeBSD imposes.

From martin at v.loewis.de  Wed Dec 29 22:54:41 2010
From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=)
Date: Wed, 29 Dec 2010 22:54:41 +0100
Subject: [Python-Dev] Issue #10348: concurrent.futures doesn't work on
 BSD
In-Reply-To: <22257F0D-AE3E-46F5-BE0D-1426D5F908CF@gmail.com>
References: <1293628653.16756.13.camel@marge> <iffii9$d6g$2@dough.gmane.org>
	<4D1B5394.10802@v.loewis.de>
	<AANLkTin+SLFm7c9nyEmtGd4Ocgatkhindnc-APBfYvm6@mail.gmail.com>
	<4D1B7F37.3080201@v.loewis.de>
	<AANLkTim20h+UTLsSJVuhOPmCxQvB7RWFb=NZvh9dZNmF@mail.gmail.com>
	<4D1B9EE2.70806@v.loewis.de>
	<22257F0D-AE3E-46F5-BE0D-1426D5F908CF@gmail.com>
Message-ID: <4D1BAE21.5070001@v.loewis.de>

Am 29.12.2010 22:34, schrieb Jesse Noller:
> 
> 
> On Dec 29, 2010, at 3:49 PM, "Martin v. L?wis" <martin at v.loewis.de> wrote:
> 
>>> If the functionality is not supported then users get an import error
>>> (within multiprocessing). However, RDM's understanding is correct, and
>>> the test is creating more than supported.
>>
>> Hmm. The tests do the absolute minimum stuff that exercises the code;
>> doing anything less, and they would be useless. Of course, one may
>> wonder why test_first_completed manages to create 41 SemLock objects,
>> when all it tries to do is two future calls.
>>
>> So if the minimal test case fails, I'd claim that the module doesn't
>> work on FreeBSD, period. ISTM that Posix IPC is just not a feasible
>> approach to do IPC synchronization on FreeBSD, so it's better to say
>> that multiprocessing is not supported on FreeBSD (until SysV IPC is
>> getting used, hoping that this will fare better).
>>
> 
> Whatever you choose to say Martin. It does work, and is supported to the limitations that FreeBSD imposes.

So what do you propose to do?

Regards,
Martin

From jnoller at gmail.com  Wed Dec 29 23:13:43 2010
From: jnoller at gmail.com (Jesse Noller)
Date: Wed, 29 Dec 2010 17:13:43 -0500
Subject: [Python-Dev] Issue #10348: concurrent.futures doesn't work on
	BSD
In-Reply-To: <4D1BAE21.5070001@v.loewis.de>
References: <1293628653.16756.13.camel@marge> <iffii9$d6g$2@dough.gmane.org>
	<4D1B5394.10802@v.loewis.de>
	<AANLkTin+SLFm7c9nyEmtGd4Ocgatkhindnc-APBfYvm6@mail.gmail.com>
	<4D1B7F37.3080201@v.loewis.de>
	<AANLkTim20h+UTLsSJVuhOPmCxQvB7RWFb=NZvh9dZNmF@mail.gmail.com>
	<4D1B9EE2.70806@v.loewis.de>
	<22257F0D-AE3E-46F5-BE0D-1426D5F908CF@gmail.com>
	<4D1BAE21.5070001@v.loewis.de>
Message-ID: <681D6028-7898-4845-9941-1F8D48C02EEE@gmail.com>



On Dec 29, 2010, at 4:54 PM, "Martin v. L?wis" <martin at v.loewis.de> wrote:

> Am 29.12.2010 22:34, schrieb Jesse Noller:
>> 
>> 
>> On Dec 29, 2010, at 3:49 PM, "Martin v. L?wis" <martin at v.loewis.de> wrote:
>> 
>>>> If the functionality is not supported then users get an import error
>>>> (within multiprocessing). However, RDM's understanding is correct, and
>>>> the test is creating more than supported.
>>> 
>>> Hmm. The tests do the absolute minimum stuff that exercises the code;
>>> doing anything less, and they would be useless. Of course, one may
>>> wonder why test_first_completed manages to create 41 SemLock objects,
>>> when all it tries to do is two future calls.
>>> 
>>> So if the minimal test case fails, I'd claim that the module doesn't
>>> work on FreeBSD, period. ISTM that Posix IPC is just not a feasible
>>> approach to do IPC synchronization on FreeBSD, so it's better to say
>>> that multiprocessing is not supported on FreeBSD (until SysV IPC is
>>> getting used, hoping that this will fare better).
>>> 
>> 
>> Whatever you choose to say Martin. It does work, and is supported to the limitations that FreeBSD imposes.
> 
> So what do you propose to do?
> 

I don't have a good suggestion (or a computer with a keyboard anywhere near me) right now, but making a migration/fallback to SYSV style semaphores a release blocker seems like a mistake to me.

I would document the limitation in the futures/multiprocessing documentation, and skip that particular test for now on FreeBSD. FreeBSDs limitations seem arbitrary, but I'm not a FBSD expert. This isn't a bug in either futures or multiprocessing though.

From martin at v.loewis.de  Wed Dec 29 23:24:32 2010
From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=)
Date: Wed, 29 Dec 2010 23:24:32 +0100
Subject: [Python-Dev] Issue #10348: concurrent.futures doesn't work on
 BSD
In-Reply-To: <681D6028-7898-4845-9941-1F8D48C02EEE@gmail.com>
References: <1293628653.16756.13.camel@marge> <iffii9$d6g$2@dough.gmane.org>
	<4D1B5394.10802@v.loewis.de>
	<AANLkTin+SLFm7c9nyEmtGd4Ocgatkhindnc-APBfYvm6@mail.gmail.com>
	<4D1B7F37.3080201@v.loewis.de>
	<AANLkTim20h+UTLsSJVuhOPmCxQvB7RWFb=NZvh9dZNmF@mail.gmail.com>
	<4D1B9EE2.70806@v.loewis.de>
	<22257F0D-AE3E-46F5-BE0D-1426D5F908CF@gmail.com>
	<4D1BAE21.5070001@v.loewis.de>
	<681D6028-7898-4845-9941-1F8D48C02EEE@gmail.com>
Message-ID: <4D1BB520.6030608@v.loewis.de>

> I don't have a good suggestion (or a computer with a keyboard
> anywhere near me) right now, but making a migration/fallback to SYSV
> style semaphores a release blocker seems like a mistake to me.

And indeed, I don't propose to make that a release blocker. Instead,
I propose to disable support for the module (either multiprocessing
or concurrent.futures only) on FreeBSD, and make such disabling a
release blocker.

> I would document the limitation in the futures/multiprocessing
> documentation, and skip that particular test for now on FreeBSD.

Just skipping the test case seems inappropriate, when it really
tests the core functionality of the module in question. The failing
test really means that the module does not work on the system.

> FreeBSDs limitations seem arbitrary, but I'm not a FBSD expert. This
> isn't a bug in either futures or multiprocessing though.

I don't claim that it's a bug. I claim that FreeBSD is just not
supported (saying that FreeBSD doesn't support the multiprocessing
module might be more appropriate).

Somebody will have to correctly fix it - either the FreeBSD people,
or the multiprocessing authors, or the concurrent.future authors.
Meanwhile, we must admit that it just doesn't work (or else the
test case would not fail).

Regards,
Martin

From philip at semanchuk.com  Wed Dec 29 23:29:32 2010
From: philip at semanchuk.com (Philip Semanchuk)
Date: Wed, 29 Dec 2010 17:29:32 -0500
Subject: [Python-Dev] Issue #10348: concurrent.futures doesn't work on
	BSD
In-Reply-To: <4D1BB520.6030608@v.loewis.de>
References: <1293628653.16756.13.camel@marge> <iffii9$d6g$2@dough.gmane.org>
	<4D1B5394.10802@v.loewis.de>
	<AANLkTin+SLFm7c9nyEmtGd4Ocgatkhindnc-APBfYvm6@mail.gmail.com>
	<4D1B7F37.3080201@v.loewis.de>
	<AANLkTim20h+UTLsSJVuhOPmCxQvB7RWFb=NZvh9dZNmF@mail.gmail.com>
	<4D1B9EE2.70806@v.loewis.de>
	<22257F0D-AE3E-46F5-BE0D-1426D5F908CF@gmail.com>
	<4D1BAE21.5070001@v.loewis.de>
	<681D6028-7898-4845-9941-1F8D48C02EEE@gmail.com>
	<4D1BB520.6030608@v.loewis.de>
Message-ID: <092A9C27-CC75-4B5A-8731-7F8EF7E3DF68@semanchuk.com>


On Dec 29, 2010, at 5:24 PM, Martin v. L?wis wrote:

>> I don't have a good suggestion (or a computer with a keyboard
>> anywhere near me) right now, but making a migration/fallback to SYSV
>> style semaphores a release blocker seems like a mistake to me.
> 
> And indeed, I don't propose to make that a release blocker. Instead,
> I propose to disable support for the module (either multiprocessing
> or concurrent.futures only) on FreeBSD, and make such disabling a
> release blocker.

Note that we only know that the test fails on FreeBSD 7.2. The current release is FreeBSD 8.1. The 8.x series might work just fine. I don't know which test is failing under 7.2. Has anyone tested it under 7.3 or 8.x?

bye
Philip

From solipsis at pitrou.net  Wed Dec 29 23:33:17 2010
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Wed, 29 Dec 2010 23:33:17 +0100
Subject: [Python-Dev] Issue #10348: concurrent.futures doesn't work on
	BSD
References: <1293628653.16756.13.camel@marge> <iffii9$d6g$2@dough.gmane.org>
	<4D1B5394.10802@v.loewis.de>
	<AANLkTin+SLFm7c9nyEmtGd4Ocgatkhindnc-APBfYvm6@mail.gmail.com>
	<4D1B7F37.3080201@v.loewis.de>
	<AANLkTim20h+UTLsSJVuhOPmCxQvB7RWFb=NZvh9dZNmF@mail.gmail.com>
	<4D1B9EE2.70806@v.loewis.de>
	<22257F0D-AE3E-46F5-BE0D-1426D5F908CF@gmail.com>
	<4D1BAE21.5070001@v.loewis.de>
	<681D6028-7898-4845-9941-1F8D48C02EEE@gmail.com>
	<4D1BB520.6030608@v.loewis.de>
Message-ID: <20101229233317.11fc10e2@pitrou.net>

On Wed, 29 Dec 2010 23:24:32 +0100
"Martin v. L?wis" <martin at v.loewis.de> wrote:
> > I don't have a good suggestion (or a computer with a keyboard
> > anywhere near me) right now, but making a migration/fallback to SYSV
> > style semaphores a release blocker seems like a mistake to me.
> 
> And indeed, I don't propose to make that a release blocker. Instead,
> I propose to disable support for the module (either multiprocessing
> or concurrent.futures only) on FreeBSD, and make such disabling a
> release blocker.

I don't really agree with this. There's no need to explicitly forbid
use of multiprocessing from FreeBSD. First, it is not our task to
validate that each and every OS conforms to the APIs it claims to
implement. Second, such disabling would make life uselessly more
complicated for users the day FreeBSD actually fixes their stuff.

So, IMO, skipping tests is enough. At worse, multiprocessing can issue
a warning when imported under these OSes.

Regards

Antoine.



From db3l.net at gmail.com  Wed Dec 29 23:44:51 2010
From: db3l.net at gmail.com (David Bolen)
Date: Wed, 29 Dec 2010 17:44:51 -0500
Subject: [Python-Dev] Issue #10348: concurrent.futures doesn't work on
	BSD
References: <1293628653.16756.13.camel@marge> <iffii9$d6g$2@dough.gmane.org>
	<4D1B5394.10802@v.loewis.de>
	<AANLkTin+SLFm7c9nyEmtGd4Ocgatkhindnc-APBfYvm6@mail.gmail.com>
	<4D1B7F37.3080201@v.loewis.de>
	<AANLkTim20h+UTLsSJVuhOPmCxQvB7RWFb=NZvh9dZNmF@mail.gmail.com>
	<4D1B9EE2.70806@v.loewis.de>
	<22257F0D-AE3E-46F5-BE0D-1426D5F908CF@gmail.com>
	<4D1BAE21.5070001@v.loewis.de>
	<681D6028-7898-4845-9941-1F8D48C02EEE@gmail.com>
	<4D1BB520.6030608@v.loewis.de>
Message-ID: <m2y678jj18.fsf@valheru.db3l.homeip.net>

"Martin v. L?wis" <martin at v.loewis.de> writes:

>> I don't have a good suggestion (or a computer with a keyboard
>> anywhere near me) right now, but making a migration/fallback to SYSV
>> style semaphores a release blocker seems like a mistake to me.
>
> And indeed, I don't propose to make that a release blocker. Instead,
> I propose to disable support for the module (either multiprocessing
> or concurrent.futures only) on FreeBSD, and make such disabling a
> release blocker.

Or, I'll make the same offer I think I made in the multiprocessing
case, which is I can build a kernel on the buildbot with a higher
limit, if that's needed just to ensure test completion.  Yes, it would
also mean that users would need to do the same (or in later FreeBSD
releases it can be done dynamically via sysctl), but that didn't seem
to be considered a big hurdle in the prior discussion for
multiprocessing.

That would essentially switch this to a documentation issue, to document
that on older FreeBSD platforms a user needs to take some steps, either
as a startup configuration, or a kernel rebuild depending on release.

That assumes that normal use of the module will need as many
semaphores as the tests, but if not, presumably the documentation
comments could become more a caution than a requirement.

I've also been considering retiring the 6.x buildbot in favor of an
8.x (I only have resources for 2, and even that is slow).  Updating
the 7.x buildbot would also include dynamic adjustment of the limit,
which I think based on the prior discussion her was mentioned as being
in 7.3.  Though 7.4 releases late in January so could jump right to
that.

-- David

PS: In regards to another comment in this thread, while 7.2 doesn't load
    the POSIX support by default, the buildbot does have the appropriate
    loader configuration to include it.

PPS: I am about to be traveling on business through 1/10, so unlikely
     to make any buildbot changes before then.


From ncoghlan at gmail.com  Wed Dec 29 23:46:54 2010
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Thu, 30 Dec 2010 08:46:54 +1000
Subject: [Python-Dev] Issue #10348: concurrent.futures doesn't work on
	BSD
In-Reply-To: <20101229233317.11fc10e2@pitrou.net>
References: <1293628653.16756.13.camel@marge> <iffii9$d6g$2@dough.gmane.org>
	<4D1B5394.10802@v.loewis.de>
	<AANLkTin+SLFm7c9nyEmtGd4Ocgatkhindnc-APBfYvm6@mail.gmail.com>
	<4D1B7F37.3080201@v.loewis.de>
	<AANLkTim20h+UTLsSJVuhOPmCxQvB7RWFb=NZvh9dZNmF@mail.gmail.com>
	<4D1B9EE2.70806@v.loewis.de>
	<22257F0D-AE3E-46F5-BE0D-1426D5F908CF@gmail.com>
	<4D1BAE21.5070001@v.loewis.de>
	<681D6028-7898-4845-9941-1F8D48C02EEE@gmail.com>
	<4D1BB520.6030608@v.loewis.de> <20101229233317.11fc10e2@pitrou.net>
Message-ID: <AANLkTikRgb-UzJU5450bA=DusiNzz4xCn-XGivWMn+Vb@mail.gmail.com>

On Thu, Dec 30, 2010 at 8:33 AM, Antoine Pitrou <solipsis at pitrou.net> wrote:
> On Wed, 29 Dec 2010 23:24:32 +0100
> "Martin v. L?wis" <martin at v.loewis.de> wrote:
>> > I don't have a good suggestion (or a computer with a keyboard
>> > anywhere near me) right now, but making a migration/fallback to SYSV
>> > style semaphores a release blocker seems like a mistake to me.
>>
>> And indeed, I don't propose to make that a release blocker. Instead,
>> I propose to disable support for the module (either multiprocessing
>> or concurrent.futures only) on FreeBSD, and make such disabling a
>> release blocker.
>
> I don't really agree with this. There's no need to explicitly forbid
> use of multiprocessing from FreeBSD. First, it is not our task to
> validate that each and every OS conforms to the APIs it claims to
> implement. Second, such disabling would make life uselessly more
> complicated for users the day FreeBSD actually fixes their stuff.

Also, I believe you can get it to work on FreeBSD 7.2 by fiddling with
the arbitrary limits. So +1 for skipping the affected tests on FreeBSD
so the buildbot state can tell us something useful (conditionally
wrapping them with "expected failure" may be even better, as then
we'll find out when the FreeBSD out-of-the-box configuration actually
supports running them).

Cheers,
Nick.

-- 
Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia

From victor.stinner at haypocalc.com  Wed Dec 29 23:55:48 2010
From: victor.stinner at haypocalc.com (Victor Stinner)
Date: Wed, 29 Dec 2010 23:55:48 +0100
Subject: [Python-Dev] Issue #10348: concurrent.futures doesn't work on
 BSD
In-Reply-To: <4D1B9EE2.70806@v.loewis.de>
References: <1293628653.16756.13.camel@marge> <iffii9$d6g$2@dough.gmane.org>
	<4D1B5394.10802@v.loewis.de>
	<AANLkTin+SLFm7c9nyEmtGd4Ocgatkhindnc-APBfYvm6@mail.gmail.com>
	<4D1B7F37.3080201@v.loewis.de>
	<AANLkTim20h+UTLsSJVuhOPmCxQvB7RWFb=NZvh9dZNmF@mail.gmail.com>
	<4D1B9EE2.70806@v.loewis.de>
Message-ID: <1293663348.18690.163.camel@marge>

Le mercredi 29 d?cembre 2010 ? 21:49 +0100, "Martin v. L?wis" a ?crit :
> Of course, one may wonder why test_first_completed manages
> to create 41 SemLock objects, when all it tries to do is two future
> calls.

More numbers (on Linux):

 - Queue: 3 SemLock
 - Condition: 4 SemLock
 - Event: 5 SemLock
 - Call (test_concurrent_futures): 10 SemLock (2 Event)
 - ProcessPoolExecutor: 11 SemLock (2 Queue, 1 Condition)

FreeBSD 7.2 is limited to 30 semaphores, so with ProcessPoolExecutor,
you can only create *one* Call object, whereas some tests use 4 Call
objects or more.

Victor


From tjreedy at udel.edu  Thu Dec 30 00:23:44 2010
From: tjreedy at udel.edu (Terry Reedy)
Date: Wed, 29 Dec 2010 18:23:44 -0500
Subject: [Python-Dev] Issue #10348: concurrent.futures doesn't work on
	BSD
In-Reply-To: <m2y678jj18.fsf@valheru.db3l.homeip.net>
References: <1293628653.16756.13.camel@marge>
	<iffii9$d6g$2@dough.gmane.org>	<4D1B5394.10802@v.loewis.de>	<AANLkTin+SLFm7c9nyEmtGd4Ocgatkhindnc-APBfYvm6@mail.gmail.com>	<4D1B7F37.3080201@v.loewis.de>	<AANLkTim20h+UTLsSJVuhOPmCxQvB7RWFb=NZvh9dZNmF@mail.gmail.com>	<4D1B9EE2.70806@v.loewis.de>	<22257F0D-AE3E-46F5-BE0D-1426D5F908CF@gmail.com>	<4D1BAE21.5070001@v.loewis.de>	<681D6028-7898-4845-9941-1F8D48C02EEE@gmail.com>	<4D1BB520.6030608@v.loewis.de>
	<m2y678jj18.fsf@valheru.db3l.homeip.net>
Message-ID: <ifgfu0$j1b$1@dough.gmane.org>

On 12/29/2010 5:44 PM, David Bolen wrote:

> Or, I'll make the same offer I think I made in the multiprocessing
> case, which is I can build a kernel on the buildbot with a higher
> limit, if that's needed just to ensure test completion.  Yes, it would
> also mean that users would need to do the same (or in later FreeBSD
> releases it can be done dynamically via sysctl), but that didn't seem
> to be considered a big hurdle in the prior discussion for
> multiprocessing.
>
> That would essentially switch this to a documentation issue, to document
> that on older FreeBSD platforms a user needs to take some steps, either
> as a startup configuration, or a kernel rebuild depending on release.

Testing on updated buildbots and documenting older BSD limits and fixes 
seems like the best solution. Martin is right that we really do want the 
tests to run to catch any real bugs (on our side).

-- 
Terry Jan Reedy


From philip at semanchuk.com  Thu Dec 30 01:01:44 2010
From: philip at semanchuk.com (Philip Semanchuk)
Date: Wed, 29 Dec 2010 19:01:44 -0500
Subject: [Python-Dev] Issue #10348: concurrent.futures doesn't work on
	BSD
In-Reply-To: <AANLkTikRgb-UzJU5450bA=DusiNzz4xCn-XGivWMn+Vb@mail.gmail.com>
References: <1293628653.16756.13.camel@marge> <iffii9$d6g$2@dough.gmane.org>
	<4D1B5394.10802@v.loewis.de>
	<AANLkTin+SLFm7c9nyEmtGd4Ocgatkhindnc-APBfYvm6@mail.gmail.com>
	<4D1B7F37.3080201@v.loewis.de>
	<AANLkTim20h+UTLsSJVuhOPmCxQvB7RWFb=NZvh9dZNmF@mail.gmail.com>
	<4D1B9EE2.70806@v.loewis.de>
	<22257F0D-AE3E-46F5-BE0D-1426D5F908CF@gmail.com>
	<4D1BAE21.5070001@v.loewis.de>
	<681D6028-7898-4845-9941-1F8D48C02EEE@gmail.com>
	<4D1BB520.6030608@v.loewis.de> <20101229233317.11fc10e2@pitrou.net>
	<AANLkTikRgb-UzJU5450bA=DusiNzz4xCn-XGivWMn+Vb@mail.gmail.com>
Message-ID: <DABE9FBE-5B99-4E0E-8630-4DB015BB3804@semanchuk.com>


On Dec 29, 2010, at 5:46 PM, Nick Coghlan wrote:

> On Thu, Dec 30, 2010 at 8:33 AM, Antoine Pitrou <solipsis at pitrou.net> wrote:
>> On Wed, 29 Dec 2010 23:24:32 +0100
>> "Martin v. L?wis" <martin at v.loewis.de> wrote:
>>>> I don't have a good suggestion (or a computer with a keyboard
>>>> anywhere near me) right now, but making a migration/fallback to SYSV
>>>> style semaphores a release blocker seems like a mistake to me.
>>> 
>>> And indeed, I don't propose to make that a release blocker. Instead,
>>> I propose to disable support for the module (either multiprocessing
>>> or concurrent.futures only) on FreeBSD, and make such disabling a
>>> release blocker.
>> 
>> I don't really agree with this. There's no need to explicitly forbid
>> use of multiprocessing from FreeBSD. First, it is not our task to
>> validate that each and every OS conforms to the APIs it claims to
>> implement. Second, such disabling would make life uselessly more
>> complicated for users the day FreeBSD actually fixes their stuff.
> 
> Also, I believe you can get it to work on FreeBSD 7.2 by fiddling with
> the arbitrary limits. So +1 for skipping the affected tests on FreeBSD
> so the buildbot state can tell us something useful (conditionally
> wrapping them with "expected failure" may be even better, as then
> we'll find out when the FreeBSD out-of-the-box configuration actually
> supports running them).

A bit of info about FreeBSD 7.2 and 8.0 (the only versions I have installed). My installs are set up explicitly for testing and so they have had very little customization.

- `sysctl -a|grep p1003` displays the config values specific to POSIX IPC. I think `sysctl -a|grep ipc` shows values for SysV IPC. It looks like the POSIX 1003_1b names in sysctl are similar to the names here (e.g. SEM_NSEMS_MAX, SEM_VALUE_MAX, etc.) --
http://pubs.opengroup.org/onlinepubs/009695399/basedefs/limits.h.html

- Under my 7.2, sysctl.p1003_1b.sem_nsems_max == 0 and I can't create semaphores with my posix_ipc extension. After `kldload sem`, sysctl.p1003_1b.sem_nsems_max == 30 and my posix_ipc semaphore & shared mem demo works.

- Under my 8.0, sysctl.p1003_1b.sem_nsems_max == 30 and my posix_ipc semaphore & shared mem demo works even without `kldload sem`. `kldstat` doesn't show sem.ko as loaded, so I assume that POSIX semaphores are now a default part of the kernel.

- Under my 7.2 *and* 8.0, sysctl.p1003_1b.mq_open_max == 0 and I can't create a message queue with my posix_ipc extension. After `kldload mqueuefs`, sysctl.p1003_1b.mq_open_max remains 0 (huh?) but my posix_ipc message queue demo works.

- Under my 8.0, sysctl.p1003_1b.sem_nsems_max is read only even if I change it in /etc/sysctl.conf. According to this post, sem_nsems_max doesn't become dynamic until 8.1:
http://www.gossamer-threads.com/lists/python/dev/875319?do=post_view_threaded


Hope this helps a bit
Philip

From brian at sweetapp.com  Thu Dec 30 04:45:44 2010
From: brian at sweetapp.com (Brian Quinlan)
Date: Wed, 29 Dec 2010 19:45:44 -0800
Subject: [Python-Dev] Issue #10348: concurrent.futures doesn't work on
	BSD
In-Reply-To: <1293663348.18690.163.camel@marge>
References: <1293628653.16756.13.camel@marge> <iffii9$d6g$2@dough.gmane.org>
	<4D1B5394.10802@v.loewis.de>
	<AANLkTin+SLFm7c9nyEmtGd4Ocgatkhindnc-APBfYvm6@mail.gmail.com>
	<4D1B7F37.3080201@v.loewis.de>
	<AANLkTim20h+UTLsSJVuhOPmCxQvB7RWFb=NZvh9dZNmF@mail.gmail.com>
	<4D1B9EE2.70806@v.loewis.de> <1293663348.18690.163.camel@marge>
Message-ID: <1FC42F4E-BFAE-47E0-984F-0040AACD6804@sweetapp.com>

On Dec 29, 2010, at 2:55 PM, Victor Stinner wrote:

> Le mercredi 29 d?cembre 2010 ? 21:49 +0100, "Martin v. L?wis" a  
> ?crit :
>> Of course, one may wonder why test_first_completed manages
>> to create 41 SemLock objects, when all it tries to do is two future
>> calls.
>
> More numbers (on Linux):
>
> - Queue: 3 SemLock
> - Condition: 4 SemLock
> - Event: 5 SemLock
> - Call (test_concurrent_futures): 10 SemLock (2 Event)
> - ProcessPoolExecutor: 11 SemLock (2 Queue, 1 Condition)
>
> FreeBSD 7.2 is limited to 30 semaphores, so with ProcessPoolExecutor,
> you can only create *one* Call object, whereas some tests use 4 Call
> objects or more.

Great detective work!  This would suggest that ProcessPoolExecutors  
are useable on FreeBSD 7.2 so long as the user doesn't create more  
than two at once (which probably isn't a big deal for most apps).

So skipping the test is probably the way to go.

Cheers,
Brian

>
> Victor
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/brian%40sweetapp.com


From brian at sweetapp.com  Thu Dec 30 04:52:39 2010
From: brian at sweetapp.com (Brian Quinlan)
Date: Wed, 29 Dec 2010 19:52:39 -0800
Subject: [Python-Dev] Issue #10348: concurrent.futures doesn't work on
	BSD
In-Reply-To: <4D1B9EE2.70806@v.loewis.de>
References: <1293628653.16756.13.camel@marge>
	<iffii9$d6g$2@dough.gmane.org>	<4D1B5394.10802@v.loewis.de>	<AANLkTin+SLFm7c9nyEmtGd4Ocgatkhindnc-APBfYvm6@mail.gmail.com>	<4D1B7F37.3080201@v.loewis.de>
	<AANLkTim20h+UTLsSJVuhOPmCxQvB7RWFb=NZvh9dZNmF@mail.gmail.com>
	<4D1B9EE2.70806@v.loewis.de>
Message-ID: <A447E908-CA95-4A88-B9C0-A831223F5D14@sweetapp.com>

On Dec 29, 2010, at 12:49 PM, Martin v. L?wis wrote:

>> If the functionality is not supported then users get an import error
>> (within multiprocessing). However, RDM's understanding is correct,  
>> and
>> the test is creating more than supported.
>
> Hmm. The tests do the absolute minimum stuff that exercises the code;
> doing anything less, and they would be useless. Of course, one may
> wonder why test_first_completed manages to create 41 SemLock objects,
> when all it tries to do is two future calls.

I actually think that my tests may be overdone - in order to probe for  
specific race conditions they use a lot of locks to force calls to  
complete in a specific order.

I'm thinking about pairing the tests down to only demonstrate basic  
correctness. This should fix the tests on FreeBSD and Windows. Then,  
when Python 3.2 is released, I can gradually introduce more  
comprehensive tests while ensuring that I keep the buildbots green on  
all supported platforms.

Thoughts?

Cheers,
Brian

>
> So if the minimal test case fails, I'd claim that the module doesn't
> work on FreeBSD, period. ISTM that Posix IPC is just not a feasible
> approach to do IPC synchronization on FreeBSD, so it's better to say
> that multiprocessing is not supported on FreeBSD (until SysV IPC is
> getting used, hoping that this will fare better).
>
> Regards,
> Martin
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/brian%40sweetapp.com


From stephen at xemacs.org  Thu Dec 30 06:42:48 2010
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Thu, 30 Dec 2010 14:42:48 +0900
Subject: [Python-Dev] Backport troubles with mercurial
In-Reply-To: <20101229152510.1E766239DB5@kimball.webabinitio.net>
References: <AANLkTi=+_qOSRo2tj12sec9UKb=hjJar+-8VQEr8_7BB@mail.gmail.com>
	<AANLkTik-3LiS8QWhg2J26+gRkjx49YpXh2rnwq90aJcE@mail.gmail.com>
	<AANLkTin2yFJqD_OaRdTGOmYGry815bxzh_pUnHBZr_2x@mail.gmail.com>
	<87wrmsk6r7.fsf@uwakimon.sk.tsukuba.ac.jp>
	<20101229152510.1E766239DB5@kimball.webabinitio.net>
Message-ID: <87tyhvke93.fsf@uwakimon.sk.tsukuba.ac.jp>

R. David Murray writes:

 > We merge bug fixes to 2.7 on a routine basis.  It is the rule rather
 > than the exception, by a long shot.

For bugfixes, of course it's routine.  I understand that.  My point
was that the case Amaury fears, where the (new) VCS makes things
harder than patching or porting by hand, is likely to be relatively
uncommon, sandwiched between the "typo fixed in comment" conflicts
(aka "trivial tweaks") and those that require reengineering.

Also, while workflow helpers will make a big difference to the
non-VCS-geeks (ie, almost all Python developers), properly speaking
this isn't really an issue with Mercurial, because all of the methods
for this purpose are basically "diff | patch", although the
executables are called things like "svn" and "python".  They all
demand workflow helper scripts to regulate the flow of desired and
undesired patches.  The difference is that the tool for hg is a SMOP,
while that for svn is a SMOEP[1].

 > So, since we are going to be maintaining 2.7 for a while, this is
 > a workflow problem that we *must* solve to make the hg transition
 > worthwhile.  I have no doubt that we will, but I also have no doubt we
 > need to *solve* it, not just wave our hands.

Certainly.  I think I already said that, no?  My point is simply that
while Amaury's expression of his requirements is welcome, and his
experimenting with hg is extremely valuable, indeed a necessary part
of the transition, everything he describes so far is a known problem
that we basically know how to solve.  He talks about changes to the
workflow, but frankly, I don't see a *need* for that.[2]

IMO, changes to the workflow will be driven by kaizen, not by some
brave new world revolution (Guido inter alia insisted on that) nor by
thumb-in-dike disaster recovery (PEP 374 did a pretty good job on
that, if I do say so myself).

I wish I had more time to do real work on this (not to mention email,
thank *you*, David!), but it seems like every time I start
programming, I fall asleep ... and six hours later, it's back to day
job or family services. :-/


Footnotes: 
[1]  Simpler Matter Of Existing Program.

[2]  Aside for a need for establishing which hg commands correspond to
which parts of the existing workflow, and perhaps creating helper
scripts.  Ie, I think the chances are pretty good that most people who
have already tried hg are at least a little VCS-geeky, and probably
they adjust personal workflow to the new VCS.  That will not be so
transparent to the "the tool should work for me, not vice-versa"
majority.

From rdmurray at bitdance.com  Thu Dec 30 08:50:46 2010
From: rdmurray at bitdance.com (R. David Murray)
Date: Thu, 30 Dec 2010 02:50:46 -0500
Subject: [Python-Dev] Backport troubles with mercurial
In-Reply-To: <87tyhvke93.fsf@uwakimon.sk.tsukuba.ac.jp>
References: <AANLkTi=+_qOSRo2tj12sec9UKb=hjJar+-8VQEr8_7BB@mail.gmail.com>
	<AANLkTik-3LiS8QWhg2J26+gRkjx49YpXh2rnwq90aJcE@mail.gmail.com>
	<AANLkTin2yFJqD_OaRdTGOmYGry815bxzh_pUnHBZr_2x@mail.gmail.com>
	<87wrmsk6r7.fsf@uwakimon.sk.tsukuba.ac.jp>
	<20101229152510.1E766239DB5@kimball.webabinitio.net>
	<87tyhvke93.fsf@uwakimon.sk.tsukuba.ac.jp>
Message-ID: <20101230075046.B4DBD238F3D@kimball.webabinitio.net>

On Thu, 30 Dec 2010 14:42:48 +0900, "Stephen J. Turnbull" <stephen at xemacs.org> wrote:
> R. David Murray writes:
> 
>  > We merge bug fixes to 2.7 on a routine basis.  It is the rule rather
>  > than the exception, by a long shot.
> 
> For bugfixes, of course it's routine.  I understand that.  My point
> was that the case Amaury fears, where the (new) VCS makes things
> harder than patching or porting by hand, is likely to be relatively
> uncommon, sandwiched between the "typo fixed in comment" conflicts
> (aka "trivial tweaks") and those that require reengineering.

I thought Amaury was saying it was harder than svnmerge, not harder
than patching by hand (ie: that it *was* patching by hand, including
.rej files and the lack of tracking).  I also heard Georg say that this
should be fixable, and that he's partly fixed the tracking issue, but
not the patch/merge issue (and that doing so will require a change
in the hg core).

> Also, while workflow helpers will make a big difference to the
> non-VCS-geeks (ie, almost all Python developers), properly speaking
> this isn't really an issue with Mercurial, because all of the methods
> for this purpose are basically "diff | patch", although the
> executables are called things like "svn" and "python".  They all
> demand workflow helper scripts to regulate the flow of desired and
> undesired patches.  The difference is that the tool for hg is a SMOP,
> while that for svn is a SMOEP[1].

Well, considering that the transition is "soon", the fact that it
is a SMOP is a concern :)

>  > So, since we are going to be maintaining 2.7 for a while, this is
>  > a workflow problem that we *must* solve to make the hg transition
>  > worthwhile.  I have no doubt that we will, but I also have no doubt we
>  > need to *solve* it, not just wave our hands.
> 
> Certainly.  I think I already said that, no?  My point is simply that

Ah, I guess I did miss that, sorry.

> while Amaury's expression of his requirements is welcome, and his
> experimenting with hg is extremely valuable, indeed a necessary part
> of the transition, everything he describes so far is a known problem
> that we basically know how to solve.  He talks about changes to the
> workflow, but frankly, I don't see a *need* for that.[2]

Well, there will be *some* workflow change since we're talking about
committing to 3.2 maint before the new trunk instead of vice versa.
But you are right that that is, mostly, a detail.

I'm not as convinced that changes in workflow will be that minimal,
though.  I don't have much experience with the dvcses yet, but what
experience I have had leads me to think that understanding the DAG is
a non-trivial and necessary mental leap and does affect the workflow.
I don't feel as though I've made that leap yet.  Hopefully Brett
will be able to document this in the Python context so that the
*required* leap will be much smaller.

> IMO, changes to the workflow will be driven by kaizen, not by some
> brave new world revolution (Guido inter alia insisted on that) nor by
> thumb-in-dike disaster recovery (PEP 374 did a pretty good job on
> that, if I do say so myself).

Well, I hope you are right.  I'm looking forward to the new version of
the test repository and doing some real world experiments.

> I wish I had more time to do real work on this (not to mention email,
> thank *you*, David!), but it seems like every time I start

You are welcome; thanks for the feedback.  (I sometimes feel
like I'm working in a bit of a vacuum, though Barry does chime in
occasionally...but I do realize that people are busy; that's
why I inherited this job in the first place, after all :)

--
R. David Murray                                      www.bitdance.com

From martin at v.loewis.de  Thu Dec 30 10:16:43 2010
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Thu, 30 Dec 2010 10:16:43 +0100
Subject: [Python-Dev] Issue #10348: concurrent.futures doesn't work on
 BSD
In-Reply-To: <1FC42F4E-BFAE-47E0-984F-0040AACD6804@sweetapp.com>
References: <1293628653.16756.13.camel@marge>
	<iffii9$d6g$2@dough.gmane.org>	<4D1B5394.10802@v.loewis.de>	<AANLkTin+SLFm7c9nyEmtGd4Ocgatkhindnc-APBfYvm6@mail.gmail.com>	<4D1B7F37.3080201@v.loewis.de>	<AANLkTim20h+UTLsSJVuhOPmCxQvB7RWFb=NZvh9dZNmF@mail.gmail.com>	<4D1B9EE2.70806@v.loewis.de>
	<1293663348.18690.163.camel@marge>
	<1FC42F4E-BFAE-47E0-984F-0040AACD6804@sweetapp.com>
Message-ID: <4D1C4DFB.30302@v.loewis.de>

Am 30.12.2010 04:45, schrieb Brian Quinlan:
> On Dec 29, 2010, at 2:55 PM, Victor Stinner wrote:
> 
>> Le mercredi 29 d?cembre 2010 ? 21:49 +0100, "Martin v. L?wis" a ?crit :
>>> Of course, one may wonder why test_first_completed manages
>>> to create 41 SemLock objects, when all it tries to do is two future
>>> calls.
>>
>> More numbers (on Linux):
>>
>> - Queue: 3 SemLock
>> - Condition: 4 SemLock
>> - Event: 5 SemLock
>> - Call (test_concurrent_futures): 10 SemLock (2 Event)
>> - ProcessPoolExecutor: 11 SemLock (2 Queue, 1 Condition)
>>
>> FreeBSD 7.2 is limited to 30 semaphores, so with ProcessPoolExecutor,
>> you can only create *one* Call object, whereas some tests use 4 Call
>> objects or more.
> 
> Great detective work!  This would suggest that ProcessPoolExecutors are
> useable on FreeBSD 7.2 so long as the user doesn't create more than two
> at once (which probably isn't a big deal for most apps).

IIUC, this is a system-wide limit. So if you run any non-trivial app
a number of times will quickly cause failure. This is just not
practical.

> So skipping the test is probably the way to go.

I'm still -1 on that proposal.

Regards,
Martin


From lukasz at langa.pl  Thu Dec 30 16:04:16 2010
From: lukasz at langa.pl (=?utf-8?Q?=C5=81ukasz_Langa?=)
Date: Thu, 30 Dec 2010 16:04:16 +0100
Subject: [Python-Dev] Issue #10348: concurrent.futures doesn't work on
	BSD
In-Reply-To: <4D1C4DFB.30302@v.loewis.de>
References: <1293628653.16756.13.camel@marge>
	<iffii9$d6g$2@dough.gmane.org>	<4D1B5394.10802@v.loewis.de>	<AANLkTin+SLFm7c9nyEmtGd4Ocgatkhindnc-APBfYvm6@mail.gmail.com>	<4D1B7F37.3080201@v.loewis.de>	<AANLkTim20h+UTLsSJVuhOPmCxQvB7RWFb=NZvh9dZNmF@mail.gmail.com>	<4D1B9EE2.70806@v.loewis.de>
	<1293663348.18690.163.camel@marge>
	<1FC42F4E-BFAE-47E0-984F-0040AACD6804@sweetapp.com>
	<4D1C4DFB.30302@v.loewis.de>
Message-ID: <C277FE05-EE29-4A13-8142-7EA16AC55015@langa.pl>

Wiadomo?? napisana przez Martin v. L?wis w dniu 2010-12-30, o godz. 10:16:

> Am 30.12.2010 04:45, schrieb Brian Quinlan:
> 
>> So skipping the test is probably the way to go.
> 
> I'm still -1 on that proposal.

I agree with Martin, explanation follows.

In general, I'm trying to think as the user: someone wrote a program and said it requires Python 3.2. I expect it to work on Python 3.2 available on my platform. Doesn't matter if it's bundled with the OS or custom built.

When a module is not available, this should fail loudly and as fast as possible. This is currently the case with conditionally built modules like curses, sqlite3, tkinter and others. From the user's perspective: the import fails. From the administrator's perspective: the build reports skipped modules. The admin can then alter the environment to make the build behave as expected, and then run the tests to confirm it works. From an OS maintainer perspective it's even better: a module can be made available optionally with explicitly set dependencies and required configuration.

In our specific case a bunch of things should be determined first:
1. Does it still fail on FreeBSD 7.3+?
2. Why is the semaphore limit so low in the first place?
3. What is a reasonable number for that limit? That should include situations where people run multiple applications.
4. What other configuration should be done to ensure the module won't break on runtime.

Some answers Philip gave already. Knowing all these things would let us decide whether switching the module off on systems that don't meet the requirements is okay and can we get away with just documenting how to make it work. One problem is that the administrator can make the OS compatible/incompatible on runtime (with sysctl).

ISTM concurrent.futures was included a bit too early in the distribution. The module has problems not only on BSD [1]. Fortunately, there is visible action from Brian, Philip and David to make the problems go away. I'm sure it will be fixed pretty soon. Ultimately I also agree with Martin that an implementation using SysV IPC would probably be better from the user's perspective. As he said, even if the underlying code is not as pretty as with the POSIX APIs, once it's written and tested nobody cares because it's not exposed. However, it's far too late to change that now for 3.2.

[1] On RedHat/CentOS the test freezes badly (issue 10517).

-- 
Best regards,
?ukasz Langa
tel. +48 791 080 144
WWW http://lukasz.langa.pl/
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20101230/f04c45cf/attachment.html>

From solipsis at pitrou.net  Thu Dec 30 16:40:14 2010
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Thu, 30 Dec 2010 16:40:14 +0100
Subject: [Python-Dev] Issue #10348: concurrent.futures doesn't work on
	BSD
References: <1293628653.16756.13.camel@marge> <iffii9$d6g$2@dough.gmane.org>
	<4D1B5394.10802@v.loewis.de>
	<AANLkTin+SLFm7c9nyEmtGd4Ocgatkhindnc-APBfYvm6@mail.gmail.com>
	<4D1B7F37.3080201@v.loewis.de>
	<AANLkTim20h+UTLsSJVuhOPmCxQvB7RWFb=NZvh9dZNmF@mail.gmail.com>
	<4D1B9EE2.70806@v.loewis.de> <1293663348.18690.163.camel@marge>
	<1FC42F4E-BFAE-47E0-984F-0040AACD6804@sweetapp.com>
	<4D1C4DFB.30302@v.loewis.de>
	<C277FE05-EE29-4A13-8142-7EA16AC55015@langa.pl>
Message-ID: <20101230164014.218a1f27@pitrou.net>

On Thu, 30 Dec 2010 16:04:16 +0100
?ukasz Langa <lukasz at langa.pl> wrote:
> 
> Some answers Philip gave already. Knowing all these things would let us decide whether switching the module off on systems that don't meet the requirements is okay and can we get away with just documenting how to make it work.

I really don't think it is our job to maintain a list of OS/versions
which work and don't work.

At best, multiprocessing when imported could try to create a single
semaphore, and raise an ImportError if such creating fails (which it
does, IIUC, under some old FreeBSDs without a specific kernel tweak).

> ISTM concurrent.futures was included a bit too early in the distribution.
> The module has problems not only on BSD [1].

I'm not sure concurrent.futures is the culprit, rather than
multiprocessing itself (or perhaps even some core Python functionality).
Actually, the threading-based part of concurrent.futures probably works
fine.

Regards

Antoine.



From martin at v.loewis.de  Thu Dec 30 17:05:05 2010
From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=)
Date: Thu, 30 Dec 2010 17:05:05 +0100
Subject: [Python-Dev] Issue #10348: concurrent.futures doesn't work on
 BSD
In-Reply-To: <20101230164014.218a1f27@pitrou.net>
References: <1293628653.16756.13.camel@marge>
	<iffii9$d6g$2@dough.gmane.org>	<4D1B5394.10802@v.loewis.de>	<AANLkTin+SLFm7c9nyEmtGd4Ocgatkhindnc-APBfYvm6@mail.gmail.com>	<4D1B7F37.3080201@v.loewis.de>	<AANLkTim20h+UTLsSJVuhOPmCxQvB7RWFb=NZvh9dZNmF@mail.gmail.com>	<4D1B9EE2.70806@v.loewis.de>
	<1293663348.18690.163.camel@marge>	<1FC42F4E-BFAE-47E0-984F-0040AACD6804@sweetapp.com>	<4D1C4DFB.30302@v.loewis.de>	<C277FE05-EE29-4A13-8142-7EA16AC55015@langa.pl>
	<20101230164014.218a1f27@pitrou.net>
Message-ID: <4D1CADB1.6060803@v.loewis.de>

Am 30.12.2010 16:40, schrieb Antoine Pitrou:
> On Thu, 30 Dec 2010 16:04:16 +0100
> ?ukasz Langa <lukasz at langa.pl> wrote:
>>
>> Some answers Philip gave already. Knowing all these things would let us decide whether switching the module off on systems that don't meet the requirements is okay and can we get away with just documenting how to make it work.
> 
> I really don't think it is our job to maintain a list of OS/versions
> which work and don't work.

Of course not. I would propose a dynamic test: check how many POSIX
semaphores the installation supports, and fail if it's less than
200 (say).

>> ISTM concurrent.futures was included a bit too early in the distribution.
>> The module has problems not only on BSD [1].
> 
> I'm not sure concurrent.futures is the culprit, rather than
> multiprocessing itself (or perhaps even some core Python functionality).
> Actually, the threading-based part of concurrent.futures probably works
> fine.

Well, "the culprit" really is FreeBSD. However, concurrent.futures
apparently makes freehanded use of semaphores, in a way that the
FreeBSD authors didn't expect them to be used (if they expected them
to be used at all, that is).

Regards,
Martin

From solipsis at pitrou.net  Thu Dec 30 17:07:47 2010
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Thu, 30 Dec 2010 17:07:47 +0100
Subject: [Python-Dev] Issue #10348: concurrent.futures doesn't work on
 BSD
In-Reply-To: <4D1CADB1.6060803@v.loewis.de>
References: <1293628653.16756.13.camel@marge> <iffii9$d6g$2@dough.gmane.org>
	<4D1B5394.10802@v.loewis.de>
	<AANLkTin+SLFm7c9nyEmtGd4Ocgatkhindnc-APBfYvm6@mail.gmail.com>
	<4D1B7F37.3080201@v.loewis.de>
	<AANLkTim20h+UTLsSJVuhOPmCxQvB7RWFb=NZvh9dZNmF@mail.gmail.com>
	<4D1B9EE2.70806@v.loewis.de> <1293663348.18690.163.camel@marge>
	<1FC42F4E-BFAE-47E0-984F-0040AACD6804@sweetapp.com>
	<4D1C4DFB.30302@v.loewis.de>
	<C277FE05-EE29-4A13-8142-7EA16AC55015@langa.pl>
	<20101230164014.218a1f27@pitrou.net>  <4D1CADB1.6060803@v.loewis.de>
Message-ID: <1293725267.3660.3.camel@localhost.localdomain>


> > I'm not sure concurrent.futures is the culprit, rather than
> > multiprocessing itself (or perhaps even some core Python functionality).
> > Actually, the threading-based part of concurrent.futures probably works
> > fine.
> 
> Well, "the culprit" really is FreeBSD. However, concurrent.futures
> apparently makes freehanded use of semaphores, in a way that the
> FreeBSD authors didn't expect them to be used (if they expected them
> to be used at all, that is).

It seems to be multiprocessing as much as concurrent.futures itself.
Apparently a basic multiprocessing queue already uses two (interprocess)
locks and one semaphore.

And, again, the threading-based API in concurrent.futures should work
fine anyway. Do you suggest we selectively disable the process-based
API?

Regards

Antoine.



From martin at v.loewis.de  Thu Dec 30 17:50:23 2010
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Thu, 30 Dec 2010 17:50:23 +0100
Subject: [Python-Dev] Issue #10348: concurrent.futures doesn't work on
 BSD
In-Reply-To: <1293725267.3660.3.camel@localhost.localdomain>
References: <1293628653.16756.13.camel@marge>
	<iffii9$d6g$2@dough.gmane.org>	<4D1B5394.10802@v.loewis.de>	<AANLkTin+SLFm7c9nyEmtGd4Ocgatkhindnc-APBfYvm6@mail.gmail.com>	<4D1B7F37.3080201@v.loewis.de>	<AANLkTim20h+UTLsSJVuhOPmCxQvB7RWFb=NZvh9dZNmF@mail.gmail.com>	<4D1B9EE2.70806@v.loewis.de>
	<1293663348.18690.163.camel@marge>	<1FC42F4E-BFAE-47E0-984F-0040AACD6804@sweetapp.com>	<4D1C4DFB.30302@v.loewis.de>	<C277FE05-EE29-4A13-8142-7EA16AC55015@langa.pl>	<20101230164014.218a1f27@pitrou.net>
	<4D1CADB1.6060803@v.loewis.de>
	<1293725267.3660.3.camel@localhost.localdomain>
Message-ID: <4D1CB84F.403@v.loewis.de>

> And, again, the threading-based API in concurrent.futures should work
> fine anyway. Do you suggest we selectively disable the process-based
> API?

Yes. Importing concurrent.futures.process should fail. Unfortunately,
it's imported from __init__.py, so either we change the API to move
the executors to the submodules, or we let creation of process pools
fail (rather than the import).

Regards,
Martin

From stephen at xemacs.org  Thu Dec 30 18:54:26 2010
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Fri, 31 Dec 2010 02:54:26 +0900
Subject: [Python-Dev] Backport troubles with mercurial
In-Reply-To: <20101230075046.B4DBD238F3D@kimball.webabinitio.net>
References: <AANLkTi=+_qOSRo2tj12sec9UKb=hjJar+-8VQEr8_7BB@mail.gmail.com>
	<AANLkTik-3LiS8QWhg2J26+gRkjx49YpXh2rnwq90aJcE@mail.gmail.com>
	<AANLkTin2yFJqD_OaRdTGOmYGry815bxzh_pUnHBZr_2x@mail.gmail.com>
	<87wrmsk6r7.fsf@uwakimon.sk.tsukuba.ac.jp>
	<20101229152510.1E766239DB5@kimball.webabinitio.net>
	<87tyhvke93.fsf@uwakimon.sk.tsukuba.ac.jp>
	<20101230075046.B4DBD238F3D@kimball.webabinitio.net>
Message-ID: <87lj37jgdp.fsf@uwakimon.sk.tsukuba.ac.jp>

R. David Murray writes:

 > I thought Amaury was saying it was harder than svnmerge, not harder
 > than patching by hand (ie: that it *was* patching by hand, including
 > .rej files and the lack of tracking).  I also heard Georg say that this
 > should be fixable, and that he's partly fixed the tracking issue,

I don't see why the tracking issue is any different than it would be
for svn.  If you're actually merging, either a dummy merge (what git
calls an "ours merge") or reverting unwanted patches will "block"
them, and hg will keep track of the ones that have been merged.  If
you're actually cherry-picking, then you have to keep a separate
database of the patches that have been picked and those that are
blocked, but this has been done successfully for years with svnmerge,
right?

 > but not the patch/merge issue (and that doing so will require a change
 > in the hg core).

I don't think so.  For merges, there isn't a problem.

For cherypicking, I haven't thought carefully about this, but ISTM
that "hg export | hg import; merge post-patch /dev/null source" should
give the traditional conflict markers.  This will require a bit of
care to get the files to merge right, since there will in general be
multiple files that fail to patch, but the names can be fished out of
the .rej file(s).  Getting the source files will also be mildly
tricky, since they live in a different branch, and aren't available in
your local repository.  It will also require a bit of care to get the
commit log etc right.  But I don't think it's that hard, conceptually.

Of course it's preferable to get this feature in hg itself, but I
don't think we need to wait for hg/maintain a fork.

 > Well, considering that the transition is "soon", the fact that it
 > is a SMOP is a concern :)

Sure, but in this crowd, I wouldn't waste a good worry on this
particular SMOP.

 > Well, there will be *some* workflow change since we're talking about
 > committing to 3.2 maint before the new trunk instead of vice versa.
 > But you are right that that is, mostly, a detail.

AFAIK that's not hg-driven, though.

 > I'm not as convinced that changes in workflow will be that minimal,
 > though.  I don't have much experience with the dvcses yet, but what
 > experience I have had leads me to think that understanding the DAG is
 > a non-trivial and necessary mental leap and does affect the workflow.

Yes, no, and yes.  That is, although understanding it is nontrivial,
and once you do it will affect your workflow, it is unnecessary.  The
Emacs crew have proved that to my satisfaction; there are a bunch of
folks there who understand DAGs quite well, but there are also a bunch
who just don't want to know -- and they're doing fine.  True, making
the most of dVCS requires "understanding the DAG."  Adapting an
existing complex workflow to a new dVCS's commands also requires
understanding the DAG.

But using the adapted workflow simply requires learning new names for
old operations.  Annoying, but it will make a fair number of core devs
quite happy.

 > Well, I hope you are right.  I'm looking forward to the new version of
 > the test repository and doing some real world experiments.

Yup.  It's always risky to predict, especially the future<wink>, but
I'm pretty confident that things will work out.  We do need to (1)
make sure we do everything for hg that we've always done for svn, (2)
work out some features that hg doesn't have yet (Windows EOLs), and
(3) listen carefully to those who are testing out the new repository
-- there are always surprises in this kind of thing.  But Python does
that kind of thing very well.

From martin at v.loewis.de  Thu Dec 30 19:17:08 2010
From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=)
Date: Thu, 30 Dec 2010 19:17:08 +0100
Subject: [Python-Dev] Issue #10348: concurrent.futures doesn't work on
 BSD
In-Reply-To: <C277FE05-EE29-4A13-8142-7EA16AC55015@langa.pl>
References: <1293628653.16756.13.camel@marge>	<iffii9$d6g$2@dough.gmane.org>	<4D1B5394.10802@v.loewis.de>	<AANLkTin+SLFm7c9nyEmtGd4Ocgatkhindnc-APBfYvm6@mail.gmail.com>	<4D1B7F37.3080201@v.loewis.de>	<AANLkTim20h+UTLsSJVuhOPmCxQvB7RWFb=NZvh9dZNmF@mail.gmail.com>	<4D1B9EE2.70806@v.loewis.de>	<1293663348.18690.163.camel@marge>	<1FC42F4E-BFAE-47E0-984F-0040AACD6804@sweetapp.com>	<4D1C4DFB.30302@v.loewis.de>
	<C277FE05-EE29-4A13-8142-7EA16AC55015@langa.pl>
Message-ID: <4D1CCCA4.5000200@v.loewis.de>

> 1. Does it still fail on FreeBSD 7.3+?

Yes, it still fails. The limits (30 semaphores) haven't
changed. It also remains untunable.

> 2. Why is the semaphore limit so low in the first place?

I don't know - (Free)BSD is in the tradition of disliking
SysV inventions, and POSIX inventions unless they originate from
FreeBSD. This is polemics, of course :-)

In any case, SysV IPC doesn't seem much better. In SysV, you
allocate semaphores in sets. In FreeBSD 8.1, in the standard
setting, you can have up to 10 semaphore identifiers, with up
to 60 semaphores each. That may sound reasonable, except that
there is also a limit on the total number of semaphores of 60.

So it seems that switching to SysV IPC wouldn't actually
improve things (except that these are tunable parameters already,
so changing them requires only a reboot, not a recompile of the
kernel).

> 3. What is a reasonable number for that limit? That should include
> situations where people run multiple applications.

POSIX specifies that the minimum acceptable value for the
SEM_NSEMS_MAX setting is _POSIX_SEM_NSEMS_MAX, which in turn
must be at least 256. We could argue that we refuse to use
POSIX semaphores if the system doesn't conform to POSIX, i.e.
has semaphore limit of less than 256.

Regards,
Martin

From martin at v.loewis.de  Thu Dec 30 19:31:46 2010
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Thu, 30 Dec 2010 19:31:46 +0100
Subject: [Python-Dev] Backport troubles with mercurial
In-Reply-To: <87lj37jgdp.fsf@uwakimon.sk.tsukuba.ac.jp>
References: <AANLkTi=+_qOSRo2tj12sec9UKb=hjJar+-8VQEr8_7BB@mail.gmail.com>	<AANLkTik-3LiS8QWhg2J26+gRkjx49YpXh2rnwq90aJcE@mail.gmail.com>	<AANLkTin2yFJqD_OaRdTGOmYGry815bxzh_pUnHBZr_2x@mail.gmail.com>	<87wrmsk6r7.fsf@uwakimon.sk.tsukuba.ac.jp>	<20101229152510.1E766239DB5@kimball.webabinitio.net>	<87tyhvke93.fsf@uwakimon.sk.tsukuba.ac.jp>	<20101230075046.B4DBD238F3D@kimball.webabinitio.net>
	<87lj37jgdp.fsf@uwakimon.sk.tsukuba.ac.jp>
Message-ID: <4D1CD012.3010407@v.loewis.de>

Am 30.12.2010 18:54, schrieb Stephen J. Turnbull:
> R. David Murray writes:
> 
>  > I thought Amaury was saying it was harder than svnmerge, not harder
>  > than patching by hand (ie: that it *was* patching by hand, including
>  > .rej files and the lack of tracking).  I also heard Georg say that this
>  > should be fixable, and that he's partly fixed the tracking issue,
> 
> I don't see why the tracking issue is any different than it would be
> for svn.  If you're actually merging, either a dummy merge (what git
> calls an "ours merge") or reverting unwanted patches will "block"
> them, and hg will keep track of the ones that have been merged.

Are you still talking about merges from 3.x to 2.7? Then I don't think
what you say is actually true. hg will *not* be able to track the ones
that get merged, and will *not* be able to block by means of dummy
merges.

Perhaps we aren't "actually" merging, I suppose.

> If
> you're actually cherry-picking, then you have to keep a separate
> database of the patches that have been picked and those that are
> blocked, but this has been done successfully for years with svnmerge,
> right?

[Ok. So "cherry-picking" is not a special case of "merging", or
"actually merging", I presume]

Wrong. For subversion, merge tracking is built into the tools
that we use for merging (i.e. svnmerge). For hg (IIUC), the
standard procedure that people use for merging (or "cherry-picking")
apparently does not track what has been merged (or "cherry-picked").

>  > Well, considering that the transition is "soon", the fact that it
>  > is a SMOP is a concern :)
> 
> Sure, but in this crowd, I wouldn't waste a good worry on this
> particular SMOP.

Given how little enthusiasm this crowd has in actually helping the
migration, I doubt they show any enthusiasm in writing such tools.

>  > Well, there will be *some* workflow change since we're talking about
>  > committing to 3.2 maint before the new trunk instead of vice versa.
>  > But you are right that that is, mostly, a detail.
> 
> AFAIK that's not hg-driven, though.

It is. So far, we have *always* merged from the development branch
to the maintenance branch (i.e. "backported"), although there has
been opposition from a number of committers to this workflow in
recent years.

I personally consider this more appropriate: with backporting,
you can defer decision to backport until the original change
has been confirmed as correct. With constant forward-porting,
you risk breaking your maintenance branch with incorrect changes.

> 
>  > I'm not as convinced that changes in workflow will be that minimal,
>  > though.  I don't have much experience with the dvcses yet, but what
>  > experience I have had leads me to think that understanding the DAG is
>  > a non-trivial and necessary mental leap and does affect the workflow.
> 
> But using the adapted workflow simply requires learning new names for
> old operations.  Annoying, but it will make a fair number of core devs
> quite happy.

I think the new workflow will simply result in (even) less care for the
maintenance branches than currently. Some people just refuse to
patch multiple branches, and will continue to do so. While it was
previously possible to backport, it won't be that easy anymore in
the future, so it just won't be done.

This actually reduces workload, but also reduces quality.

Regards,
Martin

From rdmurray at bitdance.com  Thu Dec 30 20:16:14 2010
From: rdmurray at bitdance.com (R. David Murray)
Date: Thu, 30 Dec 2010 14:16:14 -0500
Subject: [Python-Dev] Backport troubles with mercurial
In-Reply-To: <87lj37jgdp.fsf@uwakimon.sk.tsukuba.ac.jp>
References: <AANLkTi=+_qOSRo2tj12sec9UKb=hjJar+-8VQEr8_7BB@mail.gmail.com>
	<AANLkTik-3LiS8QWhg2J26+gRkjx49YpXh2rnwq90aJcE@mail.gmail.com>
	<AANLkTin2yFJqD_OaRdTGOmYGry815bxzh_pUnHBZr_2x@mail.gmail.com>
	<87wrmsk6r7.fsf@uwakimon.sk.tsukuba.ac.jp>
	<20101229152510.1E766239DB5@kimball.webabinitio.net>
	<87tyhvke93.fsf@uwakimon.sk.tsukuba.ac.jp>
	<20101230075046.B4DBD238F3D@kimball.webabinitio.net>
	<87lj37jgdp.fsf@uwakimon.sk.tsukuba.ac.jp>
Message-ID: <20101230191614.9402E239FEA@kimball.webabinitio.net>

On Fri, 31 Dec 2010 02:54:26 +0900, "Stephen J. Turnbull" <stephen at xemacs.org> wrote:
> I don't see why the tracking issue is any different than it would be
> for svn.  If you're actually merging, either a dummy merge (what git

Martin already said most of what I would have in response to your
post.

> For cherypicking, I haven't thought carefully about this, but ISTM
> that "hg export | hg import; merge post-patch /dev/null source" should
> give the traditional conflict markers.  This will require a bit of
> care to get the files to merge right, since there will in general be
> multiple files that fail to patch, but the names can be fished out of
> the .rej file(s).  Getting the source files will also be mildly
> tricky, since they live in a different branch, and aren't available in
> your local repository.  It will also require a bit of care to get the
> commit log etc right.  But I don't think it's that hard, conceptually.
> 
> Of course it's preferable to get this feature in hg itself, but I
> don't think we need to wait for hg/maintain a fork.

The fact that I really haven't a clue what you are talking about here
means that I for one am not likely to be helping develop that tool,
at least not any time soon.  So I hope there are people who understand
this stuff who will make it work for the rest of us.

>  > Well, considering that the transition is "soon", the fact that it
>  > is a SMOP is a concern :)
> 
> Sure, but in this crowd, I wouldn't waste a good worry on this
> particular SMOP.

Talent is one thing, available time, as you pointed out about yourself,
is a different matter.

I'm confident we can make this all work.  The only question is when.

--
R. David Murray                                      www.bitdance.com

From merwok at netwok.org  Thu Dec 30 20:57:11 2010
From: merwok at netwok.org (=?UTF-8?B?w4lyaWMgQXJhdWpv?=)
Date: Thu, 30 Dec 2010 20:57:11 +0100
Subject: [Python-Dev] Backport troubles with mercurial
In-Reply-To: <AANLkTi=+_qOSRo2tj12sec9UKb=hjJar+-8VQEr8_7BB@mail.gmail.com>
References: <AANLkTi=+_qOSRo2tj12sec9UKb=hjJar+-8VQEr8_7BB@mail.gmail.com>
Message-ID: <4D1CE417.8070001@netwok.org>

Hi,

One thing confuses me in this thread: Do the problems come from merging
across different versions (i.e. different syntax and module names), or
are they regular problems that come up because people don?t want to use
a merge tool?  I for one immensely like Mercurial?s merge and utterly
dislike Subversion?s (certainly because I?ve learned a good merge tool
and don?t have a good editor to handle files with conflict markers).

> There is surely a better way to work with maintenance branches!
> PEP374 suggests to first modify the oldest release, but this does not
> seems right to me (I have three reasons in mind)
Can you give them?

Regards


From solipsis at pitrou.net  Thu Dec 30 21:14:31 2010
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Thu, 30 Dec 2010 21:14:31 +0100
Subject: [Python-Dev] Backport troubles with mercurial
References: <AANLkTi=+_qOSRo2tj12sec9UKb=hjJar+-8VQEr8_7BB@mail.gmail.com>
	<4D1CE417.8070001@netwok.org>
Message-ID: <20101230211431.67f5648e@pitrou.net>

On Thu, 30 Dec 2010 20:57:11 +0100
?ric Araujo <merwok at netwok.org> wrote:
> Hi,
> 
> One thing confuses me in this thread: Do the problems come from merging
> across different versions (i.e. different syntax and module names), or
> are they regular problems that come up because people don?t want to use
> a merge tool?  I for one immensely like Mercurial?s merge and utterly
> dislike Subversion?s (certainly because I?ve learned a good merge tool
> and don?t have a good editor to handle files with conflict markers).

IIUC this is not about merges, this is about transplants (or
cherry-picking). It would be unannoying to lose svnmerge's relative
convenience in that matter.

Regards

Antoine.



From benjamin at python.org  Thu Dec 30 21:24:31 2010
From: benjamin at python.org (Benjamin Peterson)
Date: Thu, 30 Dec 2010 14:24:31 -0600
Subject: [Python-Dev] Backport troubles with mercurial
In-Reply-To: <20101230211431.67f5648e@pitrou.net>
References: <AANLkTi=+_qOSRo2tj12sec9UKb=hjJar+-8VQEr8_7BB@mail.gmail.com>
	<4D1CE417.8070001@netwok.org> <20101230211431.67f5648e@pitrou.net>
Message-ID: <AANLkTi=mjTa=Y2+sGkLy8oRmSiFdpRJ_VnvgWrCzm20c@mail.gmail.com>

2010/12/30 Antoine Pitrou <solipsis at pitrou.net>:
> On Thu, 30 Dec 2010 20:57:11 +0100
> ?ric Araujo <merwok at netwok.org> wrote:
>> Hi,
>>
>> One thing confuses me in this thread: Do the problems come from merging
>> across different versions (i.e. different syntax and module names), or
>> are they regular problems that come up because people don?t want to use
>> a merge tool? ?I for one immensely like Mercurial?s merge and utterly
>> dislike Subversion?s (certainly because I?ve learned a good merge tool
>> and don?t have a good editor to handle files with conflict markers).
>
> IIUC this is not about merges, this is about transplants (or
> cherry-picking). It would be unannoying to lose svnmerge's relative
> convenience in that matter.

I assume you meant "annoying"?



-- 
Regards,
Benjamin

From solipsis at pitrou.net  Thu Dec 30 21:31:10 2010
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Thu, 30 Dec 2010 21:31:10 +0100
Subject: [Python-Dev] Backport troubles with mercurial
In-Reply-To: <AANLkTi=mjTa=Y2+sGkLy8oRmSiFdpRJ_VnvgWrCzm20c@mail.gmail.com>
References: <AANLkTi=+_qOSRo2tj12sec9UKb=hjJar+-8VQEr8_7BB@mail.gmail.com>
	<4D1CE417.8070001@netwok.org> <20101230211431.67f5648e@pitrou.net>
	<AANLkTi=mjTa=Y2+sGkLy8oRmSiFdpRJ_VnvgWrCzm20c@mail.gmail.com>
Message-ID: <1293741070.14809.3.camel@localhost.localdomain>

Le jeudi 30 d?cembre 2010 ? 14:24 -0600, Benjamin Peterson a ?crit :
> 2010/12/30 Antoine Pitrou <solipsis at pitrou.net>:
> > On Thu, 30 Dec 2010 20:57:11 +0100
> > ?ric Araujo <merwok at netwok.org> wrote:
> >> Hi,
> >>
> >> One thing confuses me in this thread: Do the problems come from merging
> >> across different versions (i.e. different syntax and module names), or
> >> are they regular problems that come up because people don?t want to use
> >> a merge tool?  I for one immensely like Mercurial?s merge and utterly
> >> dislike Subversion?s (certainly because I?ve learned a good merge tool
> >> and don?t have a good editor to handle files with conflict markers).
> >
> > IIUC this is not about merges, this is about transplants (or
> > cherry-picking). It would be unannoying to lose svnmerge's relative
> > convenience in that matter.
> 
> I assume you meant "annoying"?

Well, yes. I should stop twisting my negatives (whatever it means).

Antoine.



From ethan at stoneleaf.us  Thu Dec 30 21:36:47 2010
From: ethan at stoneleaf.us (Ethan Furman)
Date: Thu, 30 Dec 2010 12:36:47 -0800
Subject: [Python-Dev] Issue #10348: concurrent.futures doesn't work on
 BSD
In-Reply-To: <4D1CB84F.403@v.loewis.de>
References: <1293628653.16756.13.camel@marge>	<iffii9$d6g$2@dough.gmane.org>	<4D1B5394.10802@v.loewis.de>	<AANLkTin+SLFm7c9nyEmtGd4Ocgatkhindnc-APBfYvm6@mail.gmail.com>	<4D1B7F37.3080201@v.loewis.de>	<AANLkTim20h+UTLsSJVuhOPmCxQvB7RWFb=NZvh9dZNmF@mail.gmail.com>	<4D1B9EE2.70806@v.loewis.de>	<1293663348.18690.163.camel@marge>	<1FC42F4E-BFAE-47E0-984F-0040AACD6804@sweetapp.com>	<4D1C4DFB.30302@v.loewis.de>	<C277FE05-EE29-4A13-8142-7EA16AC55015@langa.pl>	<20101230164014.218a1f27@pitrou.net>	<4D1CADB1.6060803@v.loewis.de>	<1293725267.3660.3.camel@localhost.localdomain>
	<4D1CB84F.403@v.loewis.de>
Message-ID: <4D1CED5F.7090607@stoneleaf.us>

Martin v. L?wis wrote:
>> And, again, the threading-based API in concurrent.futures should work
>> fine anyway. Do you suggest we selectively disable the process-based
>> API?
> 
> Yes. Importing concurrent.futures.process should fail.

If I understand correctly, it is possible to adjust BSD so that this 
will work -- as a user I would much rather be informed of that, even as 
  just a caution, than to have Python not have the functionality even 
after I had fixed the OS problems.

Mind you, I have no idea how hard it would be to make this happen in the 
module in an intelligent way.

~Ethan~

From solipsis at pitrou.net  Thu Dec 30 21:43:13 2010
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Thu, 30 Dec 2010 21:43:13 +0100
Subject: [Python-Dev] Issue #10348: concurrent.futures doesn't work on
	BSD
References: <1293628653.16756.13.camel@marge> <iffii9$d6g$2@dough.gmane.org>
	<4D1B5394.10802@v.loewis.de>
	<AANLkTin+SLFm7c9nyEmtGd4Ocgatkhindnc-APBfYvm6@mail.gmail.com>
	<4D1B7F37.3080201@v.loewis.de>
	<AANLkTim20h+UTLsSJVuhOPmCxQvB7RWFb=NZvh9dZNmF@mail.gmail.com>
	<4D1B9EE2.70806@v.loewis.de> <1293663348.18690.163.camel@marge>
	<1FC42F4E-BFAE-47E0-984F-0040AACD6804@sweetapp.com>
	<4D1C4DFB.30302@v.loewis.de>
	<C277FE05-EE29-4A13-8142-7EA16AC55015@langa.pl>
	<20101230164014.218a1f27@pitrou.net> <4D1CADB1.6060803@v.loewis.de>
	<1293725267.3660.3.camel@localhost.localdomain>
	<4D1CB84F.403@v.loewis.de> <4D1CED5F.7090607@stoneleaf.us>
Message-ID: <20101230214313.710e6b55@pitrou.net>

On Thu, 30 Dec 2010 12:36:47 -0800
Ethan Furman <ethan at stoneleaf.us> wrote:

> Martin v. L?wis wrote:
> >> And, again, the threading-based API in concurrent.futures should work
> >> fine anyway. Do you suggest we selectively disable the process-based
> >> API?
> > 
> > Yes. Importing concurrent.futures.process should fail.
> 
> If I understand correctly, it is possible to adjust BSD so that this 
> will work -- as a user I would much rather be informed of that, even as 
>   just a caution, than to have Python not have the functionality even 
> after I had fixed the OS problems.
> 
> Mind you, I have no idea how hard it would be to make this happen in the 
> module in an intelligent way.

As I said, we can just emit a Warning instead of raising an exception.

Regards

Antoine.



From g.brandl at gmx.net  Thu Dec 30 22:05:52 2010
From: g.brandl at gmx.net (Georg Brandl)
Date: Thu, 30 Dec 2010 22:05:52 +0100
Subject: [Python-Dev] Backport troubles with mercurial
In-Reply-To: <4D1CD012.3010407@v.loewis.de>
References: <AANLkTi=+_qOSRo2tj12sec9UKb=hjJar+-8VQEr8_7BB@mail.gmail.com>	<AANLkTik-3LiS8QWhg2J26+gRkjx49YpXh2rnwq90aJcE@mail.gmail.com>	<AANLkTin2yFJqD_OaRdTGOmYGry815bxzh_pUnHBZr_2x@mail.gmail.com>	<87wrmsk6r7.fsf@uwakimon.sk.tsukuba.ac.jp>	<20101229152510.1E766239DB5@kimball.webabinitio.net>	<87tyhvke93.fsf@uwakimon.sk.tsukuba.ac.jp>	<20101230075046.B4DBD238F3D@kimball.webabinitio.net>	<87lj37jgdp.fsf@uwakimon.sk.tsukuba.ac.jp>
	<4D1CD012.3010407@v.loewis.de>
Message-ID: <ifis90$33p$1@dough.gmane.org>

Am 30.12.2010 19:31, schrieb "Martin v. L?wis":

>> But using the adapted workflow simply requires learning new names for
>> old operations.  Annoying, but it will make a fair number of core devs
>> quite happy.
> 
> I think the new workflow will simply result in (even) less care for the
> maintenance branches than currently. Some people just refuse to
> patch multiple branches, and will continue to do so. While it was
> previously possible to backport, it won't be that easy anymore in
> the future, so it just won't be done.

I refuse to believe that we cannot make our committers (in particular the active
ones, which is quite a small percentage) follow such simple rules, especially
when they have to learn a new VCS anyway...

Also, it's not really necessary for everyone to merge every change from
maintenance to trunk: a) they can do multiple commits on the same
subject and merge once, and b) if the change is small and no problems can
be expected from merging, merging can also be done by others, just like
the mass merging we had once from trunk -> py3k (just more convenient for
the one doing the merge).

Georg


From g.brandl at gmx.net  Thu Dec 30 22:07:51 2010
From: g.brandl at gmx.net (Georg Brandl)
Date: Thu, 30 Dec 2010 22:07:51 +0100
Subject: [Python-Dev] Issue #10348: concurrent.futures doesn't work on
	BSD
In-Reply-To: <4D1CCCA4.5000200@v.loewis.de>
References: <1293628653.16756.13.camel@marge>	<iffii9$d6g$2@dough.gmane.org>	<4D1B5394.10802@v.loewis.de>	<AANLkTin+SLFm7c9nyEmtGd4Ocgatkhindnc-APBfYvm6@mail.gmail.com>	<4D1B7F37.3080201@v.loewis.de>	<AANLkTim20h+UTLsSJVuhOPmCxQvB7RWFb=NZvh9dZNmF@mail.gmail.com>	<4D1B9EE2.70806@v.loewis.de>	<1293663348.18690.163.camel@marge>	<1FC42F4E-BFAE-47E0-984F-0040AACD6804@sweetapp.com>	<4D1C4DFB.30302@v.loewis.de>	<C277FE05-EE29-4A13-8142-7EA16AC55015@langa.pl>
	<4D1CCCA4.5000200@v.loewis.de>
Message-ID: <ifisco$33p$2@dough.gmane.org>

Am 30.12.2010 19:17, schrieb "Martin v. L?wis":
>> 1. Does it still fail on FreeBSD 7.3+?
> 
> Yes, it still fails. The limits (30 semaphores) haven't
> changed. It also remains untunable.
> 
>> 2. Why is the semaphore limit so low in the first place?
> 
> I don't know - (Free)BSD is in the tradition of disliking
> SysV inventions, and POSIX inventions unless they originate from
> FreeBSD. This is polemics, of course :-)

BTW - can anyone contribute data points from other *BSDs?

Georg


From martin at v.loewis.de  Thu Dec 30 22:38:29 2010
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Thu, 30 Dec 2010 22:38:29 +0100
Subject: [Python-Dev] Backport troubles with mercurial
In-Reply-To: <ifis90$33p$1@dough.gmane.org>
References: <AANLkTi=+_qOSRo2tj12sec9UKb=hjJar+-8VQEr8_7BB@mail.gmail.com>	<AANLkTik-3LiS8QWhg2J26+gRkjx49YpXh2rnwq90aJcE@mail.gmail.com>	<AANLkTin2yFJqD_OaRdTGOmYGry815bxzh_pUnHBZr_2x@mail.gmail.com>	<87wrmsk6r7.fsf@uwakimon.sk.tsukuba.ac.jp>	<20101229152510.1E766239DB5@kimball.webabinitio.net>	<87tyhvke93.fsf@uwakimon.sk.tsukuba.ac.jp>	<20101230075046.B4DBD238F3D@kimball.webabinitio.net>	<87lj37jgdp.fsf@uwakimon.sk.tsukuba.ac.jp>	<4D1CD012.3010407@v.loewis.de>
	<ifis90$33p$1@dough.gmane.org>
Message-ID: <4D1CFBD5.1010109@v.loewis.de>

> Also, it's not really necessary for everyone to merge every change from
> maintenance to trunk: a) they can do multiple commits on the same
> subject and merge once, and b) if the change is small and no problems can
> be expected from merging, merging can also be done by others, just like
> the mass merging we had once from trunk -> py3k (just more convenient for
> the one doing the merge).

But you can't use Mercurial's merge functionality for that, right?
You have to use some kind of transplant/cherry-picking to merge from the
3.3 branch to the 3.2 branch, right?

Regards,
Martin

From martin at v.loewis.de  Thu Dec 30 22:40:28 2010
From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=)
Date: Thu, 30 Dec 2010 22:40:28 +0100
Subject: [Python-Dev] Backport troubles with mercurial
In-Reply-To: <4D1CE417.8070001@netwok.org>
References: <AANLkTi=+_qOSRo2tj12sec9UKb=hjJar+-8VQEr8_7BB@mail.gmail.com>
	<4D1CE417.8070001@netwok.org>
Message-ID: <4D1CFC4C.60606@v.loewis.de>

Am 30.12.2010 20:57, schrieb ?ric Araujo:
> Hi,
> 
> One thing confuses me in this thread: Do the problems come from merging
> across different versions (i.e. different syntax and module names), or
> are they regular problems that come up because people don?t want to use
> a merge tool? 

Neither nor. They come from "hg merge" being useless when it comes
to having code that is meant to live both on 2.7, 3.2 (perhaps) and
3.3.

Regards,
Martin

From martin at v.loewis.de  Thu Dec 30 22:42:16 2010
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Thu, 30 Dec 2010 22:42:16 +0100
Subject: [Python-Dev] Issue #10348: concurrent.futures doesn't work on
 BSD
In-Reply-To: <4D1CED5F.7090607@stoneleaf.us>
References: <1293628653.16756.13.camel@marge>	<iffii9$d6g$2@dough.gmane.org>	<4D1B5394.10802@v.loewis.de>	<AANLkTin+SLFm7c9nyEmtGd4Ocgatkhindnc-APBfYvm6@mail.gmail.com>	<4D1B7F37.3080201@v.loewis.de>	<AANLkTim20h+UTLsSJVuhOPmCxQvB7RWFb=NZvh9dZNmF@mail.gmail.com>	<4D1B9EE2.70806@v.loewis.de>	<1293663348.18690.163.camel@marge>	<1FC42F4E-BFAE-47E0-984F-0040AACD6804@sweetapp.com>	<4D1C4DFB.30302@v.loewis.de>	<C277FE05-EE29-4A13-8142-7EA16AC55015@langa.pl>	<20101230164014.218a1f27@pitrou.net>	<4D1CADB1.6060803@v.loewis.de>	<1293725267.3660.3.camel@localhost.localdomain>	<4D1CB84F.403@v.loewis.de>
	<4D1CED5F.7090607@stoneleaf.us>
Message-ID: <4D1CFCB8.2020303@v.loewis.de>

Am 30.12.2010 21:36, schrieb Ethan Furman:
> Martin v. L?wis wrote:
>>> And, again, the threading-based API in concurrent.futures should work
>>> fine anyway. Do you suggest we selectively disable the process-based
>>> API?
>>
>> Yes. Importing concurrent.futures.process should fail.
> 
> If I understand correctly, it is possible to adjust BSD so that this
> will work -- as a user I would much rather be informed of that, even as
>  just a caution, than to have Python not have the functionality even
> after I had fixed the OS problems.

That doesn't exclude each other. The module will fail on default
(misconfigured) systems, and work on correctly-configured systems.
See my patch for details.

Regards,
Martin

From g.brandl at gmx.net  Thu Dec 30 22:44:47 2010
From: g.brandl at gmx.net (Georg Brandl)
Date: Thu, 30 Dec 2010 22:44:47 +0100
Subject: [Python-Dev] Backport troubles with mercurial
In-Reply-To: <4D1CFBD5.1010109@v.loewis.de>
References: <AANLkTi=+_qOSRo2tj12sec9UKb=hjJar+-8VQEr8_7BB@mail.gmail.com>	<AANLkTik-3LiS8QWhg2J26+gRkjx49YpXh2rnwq90aJcE@mail.gmail.com>	<AANLkTin2yFJqD_OaRdTGOmYGry815bxzh_pUnHBZr_2x@mail.gmail.com>	<87wrmsk6r7.fsf@uwakimon.sk.tsukuba.ac.jp>	<20101229152510.1E766239DB5@kimball.webabinitio.net>	<87tyhvke93.fsf@uwakimon.sk.tsukuba.ac.jp>	<20101230075046.B4DBD238F3D@kimball.webabinitio.net>	<87lj37jgdp.fsf@uwakimon.sk.tsukuba.ac.jp>	<4D1CD012.3010407@v.loewis.de>	<ifis90$33p$1@dough.gmane.org>
	<4D1CFBD5.1010109@v.loewis.de>
Message-ID: <ifiuhv$dad$1@dough.gmane.org>

Am 30.12.2010 22:38, schrieb "Martin v. L?wis":
>> Also, it's not really necessary for everyone to merge every change from
>> maintenance to trunk: a) they can do multiple commits on the same
>> subject and merge once, and b) if the change is small and no problems can
>> be expected from merging, merging can also be done by others, just like
>> the mass merging we had once from trunk -> py3k (just more convenient for
>> the one doing the merge).
> 
> But you can't use Mercurial's merge functionality for that, right?
> You have to use some kind of transplant/cherry-picking to merge from the
> 3.3 branch to the 3.2 branch, right?

Oh, I wrote the above assuming 3.2->3.3 merging.  For the other direction
you need cherry-picking, yes.

Georg


From martin at v.loewis.de  Thu Dec 30 23:19:00 2010
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Thu, 30 Dec 2010 23:19:00 +0100
Subject: [Python-Dev] Issue #10348: concurrent.futures doesn't work on
 BSD
In-Reply-To: <ifisco$33p$2@dough.gmane.org>
References: <1293628653.16756.13.camel@marge>	<iffii9$d6g$2@dough.gmane.org>	<4D1B5394.10802@v.loewis.de>	<AANLkTin+SLFm7c9nyEmtGd4Ocgatkhindnc-APBfYvm6@mail.gmail.com>	<4D1B7F37.3080201@v.loewis.de>	<AANLkTim20h+UTLsSJVuhOPmCxQvB7RWFb=NZvh9dZNmF@mail.gmail.com>	<4D1B9EE2.70806@v.loewis.de>	<1293663348.18690.163.camel@marge>	<1FC42F4E-BFAE-47E0-984F-0040AACD6804@sweetapp.com>	<4D1C4DFB.30302@v.loewis.de>	<C277FE05-EE29-4A13-8142-7EA16AC55015@langa.pl>	<4D1CCCA4.5000200@v.loewis.de>
	<ifisco$33p$2@dough.gmane.org>
Message-ID: <4D1D0554.1090002@v.loewis.de>

> BTW - can anyone contribute data points from other *BSDs?

I don't have an installation of OpenBSD, but...

In FreeBSD, POSIX semaphores are implemented in sys/kern/uipc_sem.c.
In

http://www.openbsd.org/cgi-bin/cvsweb/src/sys/kern/

that file doesn't exist. Also, in FreeBSD's limits.h,
_POSIX_SEM_NSEMS_MAX is defined (surprisingly to 256);
in

http://www.openbsd.org/cgi-bin/cvsweb/~checkout~/src/include/limits.h?rev=1.15;content-type=text/plain

this constant doesn't appear. So ISTM that OpenBSD doesn't implement
POSIX semaphores. IIUC, this means that the multiprocessing module
won't be fully functional, and its tests (and the concurrent.futures
tests) will be skipped.

NetBSD apparently supports sem_open since 2.0:

http://netbsd.gw.com/cgi-bin/man-cgi?sem_open++NetBSD-current

According to

http://cvsweb.netbsd.org/bsdweb.cgi/src/sys/kern/uipc_sem.c?rev=1.22&content-type=text/x-cvsweb-markup&only_with_tag=MAIN

SEM_MAX is 128 since 2007, and dynamically adjustable (no reboot).

Regards,
Martin

From tjreedy at udel.edu  Thu Dec 30 23:33:20 2010
From: tjreedy at udel.edu (Terry Reedy)
Date: Thu, 30 Dec 2010 17:33:20 -0500
Subject: [Python-Dev] Backport troubles with mercurial
In-Reply-To: <ifiuhv$dad$1@dough.gmane.org>
References: <AANLkTi=+_qOSRo2tj12sec9UKb=hjJar+-8VQEr8_7BB@mail.gmail.com>	<AANLkTik-3LiS8QWhg2J26+gRkjx49YpXh2rnwq90aJcE@mail.gmail.com>	<AANLkTin2yFJqD_OaRdTGOmYGry815bxzh_pUnHBZr_2x@mail.gmail.com>	<87wrmsk6r7.fsf@uwakimon.sk.tsukuba.ac.jp>	<20101229152510.1E766239DB5@kimball.webabinitio.net>	<87tyhvke93.fsf@uwakimon.sk.tsukuba.ac.jp>	<20101230075046.B4DBD238F3D@kimball.webabinitio.net>	<87lj37jgdp.fsf@uwakimon.sk.tsukuba.ac.jp>	<4D1CD012.3010407@v.loewis.de>	<ifis90$33p$1@dough.gmane.org>	<4D1CFBD5.1010109@v.loewis.de>
	<ifiuhv$dad$1@dough.gmane.org>
Message-ID: <ifj1bh$p4r$1@dough.gmane.org>

On 12/30/2010 4:44 PM, Georg Brandl wrote:

>> But you can't use Mercurial's merge functionality for that, right?
>> You have to use some kind of transplant/cherry-picking to merge from the
>> 3.3 branch to the 3.2 branch, right?
>
> Oh, I wrote the above assuming 3.2->3.3 merging.  For the other direction
> you need cherry-picking, yes.

I have the impression that Benjamin plans to continue 3.1 bug 
maintenance for months after 3.2 final rather than issue the last bug 
fix the traditional 1 week after. That would make the sequence 
3.1->3.2->3.3 or 3.1<-3.2<-3.3 or ... . The transition would be a lot 
easier, I think, if 3.1.4 was released along with 3.2, so most of us 
would never have to bother with 3.1 and hg together.

-- 
Terry Jan Reedy


From g.brandl at gmx.net  Thu Dec 30 23:42:19 2010
From: g.brandl at gmx.net (Georg Brandl)
Date: Thu, 30 Dec 2010 23:42:19 +0100
Subject: [Python-Dev] Demo is gone
Message-ID: <ifj1tr$qlp$1@dough.gmane.org>

According to the consensus (and loosely following the Google spreadsheet
I created for that purpose), I have removed the Demo directory from py3k.
Most demos have been deleted; some have been moved into Tools or into the
docs as an example.  If I removed something you think should have stayed,
please speak up.

I kept a few less-useful ones in a new directory called Tools/demo that
aims to give a broad overview of what you can do with a few lines of Python.
Please feel free to remove and/or add demos there, if you find better ones.

Georg


From benjamin at python.org  Thu Dec 30 23:49:41 2010
From: benjamin at python.org (Benjamin Peterson)
Date: Thu, 30 Dec 2010 16:49:41 -0600
Subject: [Python-Dev] Backport troubles with mercurial
In-Reply-To: <ifj1bh$p4r$1@dough.gmane.org>
References: <AANLkTi=+_qOSRo2tj12sec9UKb=hjJar+-8VQEr8_7BB@mail.gmail.com>
	<AANLkTik-3LiS8QWhg2J26+gRkjx49YpXh2rnwq90aJcE@mail.gmail.com>
	<AANLkTin2yFJqD_OaRdTGOmYGry815bxzh_pUnHBZr_2x@mail.gmail.com>
	<87wrmsk6r7.fsf@uwakimon.sk.tsukuba.ac.jp>
	<20101229152510.1E766239DB5@kimball.webabinitio.net>
	<87tyhvke93.fsf@uwakimon.sk.tsukuba.ac.jp>
	<20101230075046.B4DBD238F3D@kimball.webabinitio.net>
	<87lj37jgdp.fsf@uwakimon.sk.tsukuba.ac.jp>
	<4D1CD012.3010407@v.loewis.de> <ifis90$33p$1@dough.gmane.org>
	<4D1CFBD5.1010109@v.loewis.de> <ifiuhv$dad$1@dough.gmane.org>
	<ifj1bh$p4r$1@dough.gmane.org>
Message-ID: <AANLkTinTaq--wQq5TiQUWkHX7awTVynAsbT+-MMYHqqL@mail.gmail.com>

2010/12/30 Terry Reedy <tjreedy at udel.edu>:
> On 12/30/2010 4:44 PM, Georg Brandl wrote:
>
>>> But you can't use Mercurial's merge functionality for that, right?
>>> You have to use some kind of transplant/cherry-picking to merge from the
>>> 3.3 branch to the 3.2 branch, right?
>>
>> Oh, I wrote the above assuming 3.2->3.3 merging. ?For the other direction
>> you need cherry-picking, yes.
>
> I have the impression that Benjamin plans to continue 3.1 bug maintenance
> for months after 3.2 final rather than issue the last bug fix the
> traditional 1 week after. That would make the sequence 3.1->3.2->3.3 or
> 3.1<-3.2<-3.3 or ... . The transition would be a lot easier, I think, if
> 3.1.4 was released along with 3.2, so most of us would never have to bother
> with 3.1 and hg together.

I will try to release 3.1.4 soon after 3.2 is released, but February
and the spring in general will be quite busy for me.



-- 
Regards,
Benjamin

From db3l.net at gmail.com  Fri Dec 31 01:36:44 2010
From: db3l.net at gmail.com (David Bolen)
Date: Thu, 30 Dec 2010 19:36:44 -0500
Subject: [Python-Dev] Issue #10348: concurrent.futures doesn't work on
	BSD
References: <1293628653.16756.13.camel@marge> <iffii9$d6g$2@dough.gmane.org>
	<4D1B5394.10802@v.loewis.de>
	<AANLkTin+SLFm7c9nyEmtGd4Ocgatkhindnc-APBfYvm6@mail.gmail.com>
	<4D1B7F37.3080201@v.loewis.de>
	<AANLkTim20h+UTLsSJVuhOPmCxQvB7RWFb=NZvh9dZNmF@mail.gmail.com>
	<4D1B9EE2.70806@v.loewis.de> <1293663348.18690.163.camel@marge>
	<1FC42F4E-BFAE-47E0-984F-0040AACD6804@sweetapp.com>
	<4D1C4DFB.30302@v.loewis.de>
	<C277FE05-EE29-4A13-8142-7EA16AC55015@langa.pl>
	<4D1CCCA4.5000200@v.loewis.de>
Message-ID: <m2tyhukcbn.fsf@valheru.db3l.homeip.net>

"Martin v. L?wis" <martin at v.loewis.de> writes:

>> 1. Does it still fail on FreeBSD 7.3+?
>
> Yes, it still fails. The limits (30 semaphores) haven't
> changed. It also remains untunable.

Yeah, my recollection about 7.3 appears to have been remembering when
the kernel module was included by default as opposed to needing to be
explicitly loaded.

>From the older discussion
(http://mail.python.org/pipermail/python-dev/2010-November/105380.html)
it would appear that 7.x remains fixed sans a kernel build (not
necessarily a big deal for FreeBSD users), but 8.1+ can be tuned with
sysctl.

-- David


From cesare.di.mauro at gmail.com  Fri Dec 31 08:02:20 2010
From: cesare.di.mauro at gmail.com (Cesare Di Mauro)
Date: Fri, 31 Dec 2010 08:02:20 +0100
Subject: [Python-Dev] Possible optimization for LOAD_FAST ?
In-Reply-To: <AANLkTin05vNDnuP53OefqYvpq6S5cv2aEF5MdXkMshbD@mail.gmail.com>
References: <AANLkTin05vNDnuP53OefqYvpq6S5cv2aEF5MdXkMshbD@mail.gmail.com>
Message-ID: <AANLkTinoC0DBMojaL3W96drvAz1zd_sR2Zs84_4Ra=jG@mail.gmail.com>

2010/12/28 Lukas Lueg <lukas.lueg at googlemail.com>

> Consider the following code:
>
> def foobar(x):
>    for i in range(5):
>        x[i] = i
>
> The bytecode in python 2.7 is the following:
>
>  2           0 SETUP_LOOP              30 (to 33)
>              3 LOAD_GLOBAL              0 (range)
>              6 LOAD_CONST               1 (5)
>              9 CALL_FUNCTION            1
>             12 GET_ITER
>        >>   13 FOR_ITER                16 (to 32)
>             16 STORE_FAST               1 (i)
>
>  3          19 LOAD_FAST                1 (i)
>             22 LOAD_FAST                0 (x)
>             25 LOAD_FAST                1 (i)
>             28 STORE_SUBSCR
>             29 JUMP_ABSOLUTE           13
>        >>   32 POP_BLOCK
>        >>   33 LOAD_CONST               0 (None)
>             36 RETURN_VALUE
>
> Can't we optimize the LOAD_FAST in lines 19 and 25 to a single load
> and put the reference twice on the stack? There is no way that the
> reference of i might change in between the two lines. Also, the
> load_fast in lne 22 to reference x could be taken out of the loop as x
>  will always point to the same object....


Yes, you can, but you need:
- a better AST evaluator (to mark symbols/variables with proper attributes);
- a better optimizer (usually located on compile.c) which has a "global
vision" (not limited to single instructions and/or single expressions).

It's not that simple, and the results aren't guaranteed to be good.

Also, consider that Python, as a dynamic-and-not-statically-compiled
language need to find a good trade-off between compilation time and
execution.

Just to be clear, a C program is usually compiled once, then executed, so
you can spend even *hours* to better optimize the final binary code.

With a dynamic language, usually the code is compiled and the executed as
needed, in "realtime". So it isn't practical neither desirable having to
wait too much time before execution begins (the "startup" problem).

Python stays in a "gray area", because modules are usually compiled once
(when they are first used), and executed many times, but it isn't the only
case.

You cannot assume that optimization techniques used on other (static)
languages can be used/ported in Python.

Cesare
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20101231/cf8f3296/attachment.html>

From cesare.di.mauro at gmail.com  Fri Dec 31 08:17:21 2010
From: cesare.di.mauro at gmail.com (Cesare Di Mauro)
Date: Fri, 31 Dec 2010 08:17:21 +0100
Subject: [Python-Dev] Possible optimization for LOAD_FAST ?
In-Reply-To: <4D1B3592.3040403@v.loewis.de>
References: <AANLkTin05vNDnuP53OefqYvpq6S5cv2aEF5MdXkMshbD@mail.gmail.com>
	<4D1B3592.3040403@v.loewis.de>
Message-ID: <AANLkTinquPrf_1ZvV0SbfsOiKWKosiEo6Npk_wmXL6=p@mail.gmail.com>

2010/12/29 "Martin v. L?wis" <martin at v.loewis.de>

> Am 28.12.2010 18:08, schrieb Lukas Lueg:
> > Also, the
> > load_fast in lne 22 to reference x could be taken out of the loop as x
> > will always point to the same object....
>
> That's not true; a debugger may change the value of x.
>
> Regards,
> Martin


OK, but is it mandatory? For example, in the above code, I can unroll the
loop because I found that range is the usual built-in, 5 is a low-enough
constant, and the body is made by a simple statement.

Another example. I can totally remove the variable i, just using the stack,
so a debugger (or, in general, having the tracing enabled) cannot even find
something to change about it.

And so on with other optimization examples that can be possible.

Are they "legal" with Python? I think that we need to make it clear what
happens in such cases.

My idea is that it should be made implementation-specific. What happens with
local variables and the generated code must depend on the specific compiler
& virtual machine, in order to have a greater flexibility.

IMHO the most important thing should be that, under normal conditions, the
executed code have the expected behavior.

Cesare
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20101231/0ca7c890/attachment.html>

From fijall at gmail.com  Fri Dec 31 11:00:21 2010
From: fijall at gmail.com (Maciej Fijalkowski)
Date: Fri, 31 Dec 2010 12:00:21 +0200
Subject: [Python-Dev] Possible optimization for LOAD_FAST ?
In-Reply-To: <AANLkTinquPrf_1ZvV0SbfsOiKWKosiEo6Npk_wmXL6=p@mail.gmail.com>
References: <AANLkTin05vNDnuP53OefqYvpq6S5cv2aEF5MdXkMshbD@mail.gmail.com>
	<4D1B3592.3040403@v.loewis.de>
	<AANLkTinquPrf_1ZvV0SbfsOiKWKosiEo6Npk_wmXL6=p@mail.gmail.com>
Message-ID: <AANLkTi=SRzk8bZADY9CtXy16g0FddDussMuKoR=yBdye@mail.gmail.com>

> OK, but is it mandatory? For example, in the above code, I can unroll the
> loop because I found that range is the usual built-in, 5 is a low-enough
> constant,

How do you know xrange is xrange and not something else?

Cheers,
fijal

From fijall at gmail.com  Fri Dec 31 11:01:12 2010
From: fijall at gmail.com (Maciej Fijalkowski)
Date: Fri, 31 Dec 2010 12:01:12 +0200
Subject: [Python-Dev] Possible optimization for LOAD_FAST ?
In-Reply-To: <AANLkTi=SRzk8bZADY9CtXy16g0FddDussMuKoR=yBdye@mail.gmail.com>
References: <AANLkTin05vNDnuP53OefqYvpq6S5cv2aEF5MdXkMshbD@mail.gmail.com>
	<4D1B3592.3040403@v.loewis.de>
	<AANLkTinquPrf_1ZvV0SbfsOiKWKosiEo6Npk_wmXL6=p@mail.gmail.com>
	<AANLkTi=SRzk8bZADY9CtXy16g0FddDussMuKoR=yBdye@mail.gmail.com>
Message-ID: <AANLkTi=drEbS-MXwAv=O022v6kz4LHS80qqcavfQoqOM@mail.gmail.com>

On Fri, Dec 31, 2010 at 12:00 PM, Maciej Fijalkowski <fijall at gmail.com> wrote:
>> OK, but is it mandatory? For example, in the above code, I can unroll the
>> loop because I found that range is the usual built-in, 5 is a low-enough
>> constant,
>
> How do you know xrange is xrange and not something else?
>
> Cheers,
> fijal
>

Err, misread. How do you know that range is a builtin you're thinking
about and not some other object?

Cheers,
fijal

From cesare.di.mauro at gmail.com  Fri Dec 31 16:02:43 2010
From: cesare.di.mauro at gmail.com (Cesare Di Mauro)
Date: Fri, 31 Dec 2010 16:02:43 +0100
Subject: [Python-Dev] Possible optimization for LOAD_FAST ?
In-Reply-To: <AANLkTi=drEbS-MXwAv=O022v6kz4LHS80qqcavfQoqOM@mail.gmail.com>
References: <AANLkTin05vNDnuP53OefqYvpq6S5cv2aEF5MdXkMshbD@mail.gmail.com>
	<4D1B3592.3040403@v.loewis.de>
	<AANLkTinquPrf_1ZvV0SbfsOiKWKosiEo6Npk_wmXL6=p@mail.gmail.com>
	<AANLkTi=SRzk8bZADY9CtXy16g0FddDussMuKoR=yBdye@mail.gmail.com>
	<AANLkTi=drEbS-MXwAv=O022v6kz4LHS80qqcavfQoqOM@mail.gmail.com>
Message-ID: <AANLkTikk2Z=eKeRLO=MdBmFvS6rZrFkfhU=1f+m2UVon@mail.gmail.com>

2010/12/31 Maciej Fijalkowski <fijall at gmail.com>

> On Fri, Dec 31, 2010 at 12:00 PM, Maciej Fijalkowski <fijall at gmail.com>
> wrote:
> >> OK, but is it mandatory? For example, in the above code, I can unroll
> the
> >> loop because I found that range is the usual built-in, 5 is a low-enough
> >> constant,
> >
> > How do you know xrange is xrange and not something else?
> >
> > Cheers,
> > fijal
> >
>
> Err, misread. How do you know that range is a builtin you're thinking
> about and not some other object?
>
> Cheers,
> fijal
>

By a special opcode which could do this work. ]:-)

Cesare
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20101231/fff46624/attachment.html>

From ethan at stoneleaf.us  Fri Dec 31 16:55:36 2010
From: ethan at stoneleaf.us (Ethan Furman)
Date: Fri, 31 Dec 2010 07:55:36 -0800
Subject: [Python-Dev] Possible optimization for LOAD_FAST ?
In-Reply-To: <AANLkTinquPrf_1ZvV0SbfsOiKWKosiEo6Npk_wmXL6=p@mail.gmail.com>
References: <AANLkTin05vNDnuP53OefqYvpq6S5cv2aEF5MdXkMshbD@mail.gmail.com>	<4D1B3592.3040403@v.loewis.de>
	<AANLkTinquPrf_1ZvV0SbfsOiKWKosiEo6Npk_wmXL6=p@mail.gmail.com>
Message-ID: <4D1DFCF8.8080109@stoneleaf.us>

Cesare Di Mauro wrote:
 >
> 2010/12/29 "Martin v. L?wis" wrote:
 >>
>> Am 28.12.2010 18:08, schrieb Lukas Lueg:
>>> Also, the load_fast in lne 22 to reference x could be taken out of the
 >>> loop as x will always point to the same object....
>> 
>>  That's not true; a debugger may change the value of x.
> 
> Another example. I can totally remove the variable i, just using the 
> stack, so a debugger (or, in general, having the tracing enabled) cannot 
> even find something to change about it.

-1

Debugging is challenging enough as it is -- why would you want to make 
it even more difficult?

~Ethan~

From skip at pobox.com  Fri Dec 31 17:36:36 2010
From: skip at pobox.com (skip at pobox.com)
Date: Fri, 31 Dec 2010 10:36:36 -0600
Subject: [Python-Dev] Possible optimization for LOAD_FAST ?
In-Reply-To: <4D1DFCF8.8080109@stoneleaf.us>
References: <AANLkTin05vNDnuP53OefqYvpq6S5cv2aEF5MdXkMshbD@mail.gmail.com>
	<4D1B3592.3040403@v.loewis.de>
	<AANLkTinquPrf_1ZvV0SbfsOiKWKosiEo6Npk_wmXL6=p@mail.gmail.com>
	<4D1DFCF8.8080109@stoneleaf.us>
Message-ID: <19742.1684.846410.770977@montanaro.dyndns.org>


    >> Another example. I can totally remove the variable i, just using the
    >> stack, so a debugger (or, in general, having the tracing enabled)
    >> cannot even find something to change about it.

    Ethan> -1

    Ethan> Debugging is challenging enough as it is -- why would you want to
    Ethan> make it even more difficult?

<snarky>
I don't know.  Maybe he wants his program to run faster.
</snarky>

If you use print statements for the bulk of your debugging (many people do),
unrolling loops doesn't affect your debugging ability.

Skip

From status at bugs.python.org  Fri Dec 31 18:07:06 2010
From: status at bugs.python.org (Python tracker)
Date: Fri, 31 Dec 2010 18:07:06 +0100 (CET)
Subject: [Python-Dev] Summary of Python tracker Issues
Message-ID: <20101231170706.209AB1CBEB@psf.upfronthosting.co.za>


ACTIVITY SUMMARY (2010-12-24 - 2010-12-31)
Python tracker at http://bugs.python.org/

To view or respond to any of the issues listed below, click on the issue.
Do NOT respond to this message.

Issues counts and deltas:
  open    2525 (-17)
  closed 20058 (+49)
  total  22583 (+32)

Open issues with patches: 1063 


Issues opened (19)
==================

#10764: sysconfig and alternative implementations
http://bugs.python.org/issue10764  reopened by michael.foord

#10771: descriptor protocol documentation has two different definition
http://bugs.python.org/issue10771  opened by Devin Jeanpierre

#10772: Several actions for argparse arguments missing from docs
http://bugs.python.org/issue10772  opened by ipatrol

#10775: assertRaises as a context manager should accept a 'msg' keywor
http://bugs.python.org/issue10775  opened by r.david.murray

#10782: Not possible to cross-compile due to poor detection of %lld su
http://bugs.python.org/issue10782  opened by bgamari

#10784: os.getpriority() and os.setpriority()
http://bugs.python.org/issue10784  opened by giampaolo.rodola

#10785: parser: store the filename as an unicode object
http://bugs.python.org/issue10785  opened by haypo

#10786: unittest.TextTextRunner does not respect redirected stderr
http://bugs.python.org/issue10786  opened by cooyeah

#10787: [random.gammavariate] Add the expression of the distribution i
http://bugs.python.org/issue10787  opened by David.Kremer

#10788: test_logging failure
http://bugs.python.org/issue10788  opened by pitrou

#10789: Lock.acquire documentation is misleading
http://bugs.python.org/issue10789  opened by Jyrki.Pulliainen

#10790: Header.append's charset logic is bogus, 'shift_jis' and "euc_j
http://bugs.python.org/issue10790  opened by r.david.murray

#10791: Wrapping TextIOWrapper around gzip files
http://bugs.python.org/issue10791  opened by dabeaz

#10794: Infinite recursion while garbage collecting loops indefinitely
http://bugs.python.org/issue10794  opened by Mihai.Rusu

#10796: readline completion flaw
http://bugs.python.org/issue10796  opened by rheise

#10798: test_concurrent_futures fails on FreeBSD
http://bugs.python.org/issue10798  opened by loewis

#10799: Improve webbrowser.open doc (and, someday, behavior?)
http://bugs.python.org/issue10799  opened by terry.reedy

#10800: libffi build failure on HP-UX 11/PA
http://bugs.python.org/issue10800  opened by bugs-python at vendor.thewrittenword.com

#10801: zipfile.ZipFile().extractall() header mismatch for non-ASCII c
http://bugs.python.org/issue10801  opened by M..Z.



Most recent 15 issues with no replies (15)
==========================================

#10800: libffi build failure on HP-UX 11/PA
http://bugs.python.org/issue10800

#10799: Improve webbrowser.open doc (and, someday, behavior?)
http://bugs.python.org/issue10799

#10798: test_concurrent_futures fails on FreeBSD
http://bugs.python.org/issue10798

#10796: readline completion flaw
http://bugs.python.org/issue10796

#10789: Lock.acquire documentation is misleading
http://bugs.python.org/issue10789

#10787: [random.gammavariate] Add the expression of the distribution i
http://bugs.python.org/issue10787

#10775: assertRaises as a context manager should accept a 'msg' keywor
http://bugs.python.org/issue10775

#10772: Several actions for argparse arguments missing from docs
http://bugs.python.org/issue10772

#10761: tarfile.extractall fails to overwrite symlinks
http://bugs.python.org/issue10761

#10760: tarfile doesn't handle sysfs well
http://bugs.python.org/issue10760

#10752: build_ssl.py is relying on unreliable behaviour of os.popen
http://bugs.python.org/issue10752

#10751: WSGIREF - REMOTE_USER and REMOTE-USER collision
http://bugs.python.org/issue10751

#10747: Include version info in Windows shortcuts
http://bugs.python.org/issue10747

#10746: ctypes c_long & c_bool have incorrect PEP-3118 type codes
http://bugs.python.org/issue10746

#10745: setup.py install --user option undocumented
http://bugs.python.org/issue10745



Most recent 15 issues waiting for review (15)
=============================================

#10801: zipfile.ZipFile().extractall() header mismatch for non-ASCII c
http://bugs.python.org/issue10801

#10798: test_concurrent_futures fails on FreeBSD
http://bugs.python.org/issue10798

#10790: Header.append's charset logic is bogus, 'shift_jis' and "euc_j
http://bugs.python.org/issue10790

#10787: [random.gammavariate] Add the expression of the distribution i
http://bugs.python.org/issue10787

#10786: unittest.TextTextRunner does not respect redirected stderr
http://bugs.python.org/issue10786

#10785: parser: store the filename as an unicode object
http://bugs.python.org/issue10785

#10784: os.getpriority() and os.setpriority()
http://bugs.python.org/issue10784

#10766: optparse uses %s in gettext calls
http://bugs.python.org/issue10766

#10765: Build regression from automation changes on windows
http://bugs.python.org/issue10765

#10756: Error in atexit._run_exitfuncs [...]  Exception expected for v
http://bugs.python.org/issue10756

#10755: Add posix.fdlistdir
http://bugs.python.org/issue10755

#10746: ctypes c_long & c_bool have incorrect PEP-3118 type codes
http://bugs.python.org/issue10746

#10744: ctypes arrays have incorrect buffer information (PEP-3118)
http://bugs.python.org/issue10744

#10740: sqlite3 module should allow DDL statements in transactions
http://bugs.python.org/issue10740

#10735: platform.architecture() gives misleading results for OS X mult
http://bugs.python.org/issue10735



Top 10 most discussed issues (10)
=================================

#6210: Exception Chaining missing method for suppressing context
http://bugs.python.org/issue6210  23 msgs

#2636: Regexp 2.7 (modifications to current re 2.2.2)
http://bugs.python.org/issue2636  22 msgs

#10542: Py_UNICODE_NEXT and other macros for surrogates
http://bugs.python.org/issue10542  19 msgs

#10769: ast: provide more useful range information
http://bugs.python.org/issue10769  17 msgs

#10791: Wrapping TextIOWrapper around gzip files
http://bugs.python.org/issue10791  13 msgs

#6011: python doesn't build if prefix contains non-ascii characters
http://bugs.python.org/issue6011   5 msgs

#10348: multiprocessing: use SysV semaphores on FreeBSD
http://bugs.python.org/issue10348   5 msgs

#10764: sysconfig and alternative implementations
http://bugs.python.org/issue10764   5 msgs

#7995: On Mac / BSD sockets returned by accept inherit the parent's F
http://bugs.python.org/issue7995   4 msgs

#9893: Usefulness of the Misc/Vim/ files?
http://bugs.python.org/issue9893   4 msgs



Issues closed (45)
==================

#2708: IDLE subprocess error
http://bugs.python.org/issue2708  closed by terry.reedy

#3194: Demo/loop.c passing "char *" instead of "wchar_t *"
http://bugs.python.org/issue3194  closed by georg.brandl

#4212: email.LazyImporter does not use absolute imports
http://bugs.python.org/issue4212  closed by r.david.murray

#4685: IDLE will not open (2.6.1 on WinXP pro)
http://bugs.python.org/issue4685  closed by terry.reedy

#5258: addpackage in site.py fails hard on badly formed .pth files
http://bugs.python.org/issue5258  closed by r.david.murray

#5423: Exception raised when attempting to call set_charset on an ema
http://bugs.python.org/issue5423  closed by r.david.murray

#6533: Make test_xmlrpc_net functional in the absence of time.xmlrpc.
http://bugs.python.org/issue6533  closed by r.david.murray

#7056: regrtest runtest_inner calls findtestdir unnecessarily
http://bugs.python.org/issue7056  closed by r.david.murray

#7962: Demo and Tools need to be tested and pruned
http://bugs.python.org/issue7962  closed by georg.brandl

#8009: email.parser.Parser is inefficient with large strings
http://bugs.python.org/issue8009  closed by r.david.murray

#8889: test_support.transient_internet fails on Freebsd because socke
http://bugs.python.org/issue8889  closed by r.david.murray

#9153: Run tools and demos as part of the test suite
http://bugs.python.org/issue9153  closed by georg.brandl

#9824: SimpleCookie should escape commas and semi-colons
http://bugs.python.org/issue9824  closed by r.david.murray

#9891: Minor doc typo at datamodel.rst: "list" -> "alist"
http://bugs.python.org/issue9891  closed by eric.araujo

#10254: unicodedata.normalize('NFC', s) regression
http://bugs.python.org/issue10254  closed by belopolsky

#10494: Demo/comparisons/regextest.py needs some usage information.
http://bugs.python.org/issue10494  closed by georg.brandl

#10495: Demo/comparisons/sortingtest.py needs some usage information.
http://bugs.python.org/issue10495  closed by georg.brandl

#10609: dbm documentation example doesn't work (iteritems())
http://bugs.python.org/issue10609  closed by georg.brandl

#10626: Bad interaction between test_logging and test_concurrent_futur
http://bugs.python.org/issue10626  closed by bquinlan

#10679: "make altinstall" may clobber OS provided scripts
http://bugs.python.org/issue10679  closed by georg.brandl

#10738: webbrowser.py bug with Opera on Linux
http://bugs.python.org/issue10738  closed by terry.reedy

#10739: Subprocess behavior on Windows
http://bugs.python.org/issue10739  closed by georg.brandl

#10742: memoryview.readonly attribute is not documented
http://bugs.python.org/issue10742  closed by georg.brandl

#10753: request_uri method of wsgiref module does not support RFC1808 
http://bugs.python.org/issue10753  closed by orsenthil

#10759: HTMLParser.unescape() fails on HTML entities with incorrect sy
http://bugs.python.org/issue10759  closed by orsenthil

#10763: subprocess.communicate() doesn't close pipes on Windows
http://bugs.python.org/issue10763  closed by haypo

#10767: Lib/test/crashers/README is out of date
http://bugs.python.org/issue10767  closed by georg.brandl

#10768: Bug in scrolledtext
http://bugs.python.org/issue10768  closed by georg.brandl

#10770: zipinfo - fix of a typo in the doc
http://bugs.python.org/issue10770  closed by eric.araujo

#10773: "Building C and C++ Extensions on Windows" documentation shows
http://bugs.python.org/issue10773  closed by georg.brandl

#10774: test_logging leaves temp files
http://bugs.python.org/issue10774  closed by vinay.sajip

#10776: os.utime returns an error on NTFS-3G partition
http://bugs.python.org/issue10776  closed by Aaron.Masover

#10777: xml.etree.register_namespace dictionary changed size during it
http://bugs.python.org/issue10777  closed by georg.brandl

#10778: decoding_fgets() (tokenizer.c) decodes the filename from the w
http://bugs.python.org/issue10778  closed by haypo

#10779: Change filename encoding to FS encoding in PyErr_WarnExplicit(
http://bugs.python.org/issue10779  closed by haypo

#10780: Fix filename encoding in PyErr_SetFromWindowsErrWithFilename()
http://bugs.python.org/issue10780  closed by haypo

#10781: minidom Node.writexml method doesn't manage encoding parameter
http://bugs.python.org/issue10781  closed by georg.brandl

#10783: struct.pack() and Unicode strings
http://bugs.python.org/issue10783  closed by r.david.murray

#10792: Compile() and 'Windows/Mac newlines'
http://bugs.python.org/issue10792  closed by terry.reedy

#10793: hashlib.hash.digest() documentation incorrect re return type
http://bugs.python.org/issue10793  closed by orsenthil

#10795: standard library do not use ssl as recommended
http://bugs.python.org/issue10795  closed by pitrou

#10797: Wrong detection of lines in readlines() function
http://bugs.python.org/issue10797  closed by eric.smith

#1379416: email.Header encode() unicode P2.6
http://bugs.python.org/issue1379416  closed by r.david.murray

#1515839: socket timeout inheritance on accept
http://bugs.python.org/issue1515839  closed by ned.deily

#1685453: email package should work better with unicode
http://bugs.python.org/issue1685453  closed by r.david.murray

From cesare.di.mauro at gmail.com  Fri Dec 31 18:42:53 2010
From: cesare.di.mauro at gmail.com (Cesare Di Mauro)
Date: Fri, 31 Dec 2010 18:42:53 +0100
Subject: [Python-Dev] Possible optimization for LOAD_FAST ?
In-Reply-To: <4D1DFCF8.8080109@stoneleaf.us>
References: <AANLkTin05vNDnuP53OefqYvpq6S5cv2aEF5MdXkMshbD@mail.gmail.com>
	<4D1B3592.3040403@v.loewis.de>
	<AANLkTinquPrf_1ZvV0SbfsOiKWKosiEo6Npk_wmXL6=p@mail.gmail.com>
	<4D1DFCF8.8080109@stoneleaf.us>
Message-ID: <AANLkTikXhHa3vWFgC68CL2w+mz7H0m_W6tXErCozo=7J@mail.gmail.com>

2010/12/31 Ethan Furman <ethan at stoneleaf.us>

> Cesare Di Mauro wrote:
>
> >
>
>> 2010/12/29 "Martin v. L?wis" wrote:
>>
> >>
>
>> Am 28.12.2010 18:08, schrieb Lukas Lueg:
>>>
>>>> Also, the load_fast in lne 22 to reference x could be taken out of the
>>>>
>>> >>> loop as x will always point to the same object....
>
>>
>>>  That's not true; a debugger may change the value of x.
>>>
>>
>> Another example. I can totally remove the variable i, just using the
>> stack, so a debugger (or, in general, having the tracing enabled) cannot
>> even find something to change about it.
>>
>
> -1
>
> Debugging is challenging enough as it is -- why would you want to make it
> even more difficult?
>
> ~Ethan~


With a good test suite you can forget debuggers.

In more than 6 years of Python programming, I have used it only two times
(to debug an ANTLR generated parser).

Cesare
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20101231/91cb7a02/attachment.html>

From cesare.di.mauro at gmail.com  Fri Dec 31 18:51:39 2010
From: cesare.di.mauro at gmail.com (Cesare Di Mauro)
Date: Fri, 31 Dec 2010 18:51:39 +0100
Subject: [Python-Dev] Possible optimization for LOAD_FAST ?
In-Reply-To: <19742.1684.846410.770977@montanaro.dyndns.org>
References: <AANLkTin05vNDnuP53OefqYvpq6S5cv2aEF5MdXkMshbD@mail.gmail.com>
	<4D1B3592.3040403@v.loewis.de>
	<AANLkTinquPrf_1ZvV0SbfsOiKWKosiEo6Npk_wmXL6=p@mail.gmail.com>
	<4D1DFCF8.8080109@stoneleaf.us>
	<19742.1684.846410.770977@montanaro.dyndns.org>
Message-ID: <AANLkTimmTOSqc43+qhei=C=p7i8Ry7ufiOZziYMZRpsB@mail.gmail.com>

2010/12/31  <skip at pobox.com>

>
>    >> Another example. I can totally remove the variable i, just using the
>    >> stack, so a debugger (or, in general, having the tracing enabled)
>      >> cannot even find something to change about it.
>
   Ethan> -1
>
>    Ethan> Debugging is challenging enough as it is -- why would you want to
>    Ethan> make it even more difficult?
>
> <snarky>
> I don't know.  Maybe he wants his program to run faster.
>  </snarky>
>

:D

"Aggressive" optimizations can be enabled with explicit options, in order to
leave normal "debugger-prone" code.


> If you use print statements for the bulk of your debugging (many people
> do),
> unrolling loops doesn't affect your debugging ability.
>
>  Skip


It's a common practice. Also IDEs helps a lot, and advanced interactive
shells too (such as DreamPie).

Cesare
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20101231/21907f08/attachment.html>