From robert.kern at gmail.com  Mon Jun  1 18:44:06 2009
From: robert.kern at gmail.com (Robert Kern)
Date: Mon, 01 Jun 2009 11:44:06 -0500
Subject: [Python-Dev] Indentation oddness...
In-Reply-To: <4A21E541.1020003@canterbury.ac.nz>
References: <1A472770E042064698CB5ADC83A12ACD02915C13@TK5EX14MBXC118.redmond.corp.microsoft.com>	<gvpufg$77n$1@ger.gmane.org>
	<4A21E541.1020003@canterbury.ac.nz>
Message-ID: <h010gn$nb7$1@ger.gmane.org>

On 2009-05-30 21:02, Greg Ewing wrote:
> Robert Kern wrote:
>
>> The 'single' mode, which is used for the REPL, is a bit different than
>> 'exec', which is used for modules. This difference lets you insert
>> "blank" lines of whitespace into a function definition without exiting
>> the definition.
>
> All that means is that the REPL needs to keep reading
> lines until it gets a completely blank one. I don't
> see why the compiler has to treat the source any
> differently once the REPL has decided how much text
> to feed it.

Not true. The REPL will keep reading lines until 
compile(code,'<string>','single') passes without a SyntaxError. You can put in 
blank lines when line continuation is implicit, like in the middle of a list. 
This is the reason that there is a 'single' mode in the first place, to 
determine when you've stopped typing. It's easier to add the grammar rule that a 
block does not end with a line of whitespace to the compiler than to implement 
all of the context-specific special cases for pure empty lines outside of the 
compiler.

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
  that is made terrible by our own mad attempt to interpret it as though it had
  an underlying truth."
   -- Umberto Eco


From dinov at microsoft.com  Mon Jun  1 18:50:19 2009
From: dinov at microsoft.com (Dino Viehland)
Date: Mon, 1 Jun 2009 16:50:19 +0000
Subject: [Python-Dev] syntax warnings don't go through warnings.warn?
Message-ID: <1A472770E042064698CB5ADC83A12ACD0296CD4E@TK5EX14MBXC118.redmond.corp.microsoft.com>

I'm just a little surprised by this - Is there a reason why syntax warnings are special and untrappable via warnings.warn?

>>> import warnings
>>> def mywarn(*args): print 'xx', args
...
>>> warnings.warn = mywarn
>>> compile("def f():\n    a = 1\n    global a\n", "", "exec")
:3: SyntaxWarning: name 'a' is assigned to before global declaration
<code object <module> at 012DB410, file "", line 1>


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090601/1e0511ea/attachment.htm>

From matthew at matthewwilkes.co.uk  Mon Jun  1 19:29:08 2009
From: matthew at matthewwilkes.co.uk (Matthew Wilkes)
Date: Mon, 1 Jun 2009 18:29:08 +0100
Subject: [Python-Dev] syntax warnings don't go through warnings.warn?
In-Reply-To: <1A472770E042064698CB5ADC83A12ACD0296CD4E@TK5EX14MBXC118.redmond.corp.microsoft.com>
References: <1A472770E042064698CB5ADC83A12ACD0296CD4E@TK5EX14MBXC118.redmond.corp.microsoft.com>
Message-ID: <2BB78789-E545-4C3A-9652-5DF473C8A4A7@matthewwilkes.co.uk>


On 1 Jun 2009, at 17:50, Dino Viehland wrote:

> I?m just a little surprised by this - Is there a reason why syntax  
> warnings are special and untrappable via warnings.warn?

Why should this work?  From the docs... "Python programmers issue  
warnings by calling the warn() function defined in this module. (C  
programmers use PyErr_WarnEx; see Exception Handling for details)."

Check out the warnings.catch_warnings context manager, but if you have  
any further questions please direct them to the normal Python mailing  
list, this is for development _of_ Python only.

Matthew

From fuzzyman at voidspace.org.uk  Mon Jun  1 19:42:24 2009
From: fuzzyman at voidspace.org.uk (Michael Foord)
Date: Mon, 01 Jun 2009 18:42:24 +0100
Subject: [Python-Dev] syntax warnings don't go through warnings.warn?
In-Reply-To: <2BB78789-E545-4C3A-9652-5DF473C8A4A7@matthewwilkes.co.uk>
References: <1A472770E042064698CB5ADC83A12ACD0296CD4E@TK5EX14MBXC118.redmond.corp.microsoft.com>
	<2BB78789-E545-4C3A-9652-5DF473C8A4A7@matthewwilkes.co.uk>
Message-ID: <4A241300.4060200@voidspace.org.uk>

Matthew Wilkes wrote:
>
> On 1 Jun 2009, at 17:50, Dino Viehland wrote:
>
>> I?m just a little surprised by this - Is there a reason why syntax 
>> warnings are special and untrappable via warnings.warn?
>
> Why should this work?  From the docs... "Python programmers issue 
> warnings by calling the warn() function defined in this module. (C 
> programmers use PyErr_WarnEx; see Exception Handling for details)."
>
> Check out the warnings.catch_warnings context manager, but if you have 
> any further questions please direct them to the normal Python mailing 
> list, this is for development _of_ Python only.

Dino is developing Python - he's one of the core developers of 
IronPython and I suspect he is asking whether this is intentional, and 
IronPython should implement the same behaviour, or whether it is a bug.

Michael

>
> Matthew
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: 
> http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk 
>


-- 
http://www.ironpythoninaction.com/
http://www.voidspace.org.uk/blog



From robert.kern at gmail.com  Mon Jun  1 19:44:22 2009
From: robert.kern at gmail.com (Robert Kern)
Date: Mon, 01 Jun 2009 12:44:22 -0500
Subject: [Python-Dev] syntax warnings don't go through warnings.warn?
In-Reply-To: <1A472770E042064698CB5ADC83A12ACD0296CD4E@TK5EX14MBXC118.redmond.corp.microsoft.com>
References: <1A472770E042064698CB5ADC83A12ACD0296CD4E@TK5EX14MBXC118.redmond.corp.microsoft.com>
Message-ID: <h0141m$2q3$1@ger.gmane.org>

On 2009-06-01 11:50, Dino Viehland wrote:
> I?m just a little surprised by this - Is there a reason why syntax
> warnings are special and untrappable via warnings.warn?
>
>  >>> import warnings
>
>  >>> def mywarn(*args): print 'xx', args
>
> ...
>
>  >>> warnings.warn = mywarn
>
>  >>> compile("def f():\n a = 1\n global a\n", "", "exec")
>
> :3: SyntaxWarning: name 'a' is assigned to before global declaration
>
> <code object <module> at 012DB410, file "", line 1>

symtable.c uses PyErr_WarnExplicit() to provide file and line number 
information. You need to stub warnings.warn_explicit().

 >>> import warnings
 >>> def mywarn_explicit(*args):
...     print 'xx', args
...
 >>> warnings.warn_explicit = mywarn_explicit
 >>> compile("def f():\n a = 1\n global a\n", "", "exec")
xx ("name 'a' is assigned to before global declaration", <type 
'exceptions.SyntaxWarning'>, '', 3, None, None)
<code object <module> at 0x39e9f8, file "", line 1>

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
  that is made terrible by our own mad attempt to interpret it as though it had
  an underlying truth."
   -- Umberto Eco


From dinov at microsoft.com  Mon Jun  1 19:44:45 2009
From: dinov at microsoft.com (Dino Viehland)
Date: Mon, 1 Jun 2009 17:44:45 +0000
Subject: [Python-Dev] syntax warnings don't go through warnings.warn?
In-Reply-To: <2BB78789-E545-4C3A-9652-5DF473C8A4A7@matthewwilkes.co.uk>
References: <1A472770E042064698CB5ADC83A12ACD0296CD4E@TK5EX14MBXC118.redmond.corp.microsoft.com>
	<2BB78789-E545-4C3A-9652-5DF473C8A4A7@matthewwilkes.co.uk>
Message-ID: <1A472770E042064698CB5ADC83A12ACD0296DAA3@TK5EX14MBXC118.redmond.corp.microsoft.com>

I work on IronPython so I am asking a question about development of
Python.  In particular my goal is to make sure that IronPython is behaving
the same as CPython.  If the normal discussion list is more appropriate
for these questions I'd be happy to take it there.

But before I do that please consider nt.tempnam - a built-in function.  This
goes through the normal warning mechanism but the parser doesn't.  I
would assume that this would go through PyErr_WarnEx (sorry, I can't actually
look at the CPython code) given that tempnam is implemented in C.  Why
wouldn't the parser also go through PyErr_WarnEx?

And warnings.catch_warnings doesn't work w/ parse warnings either so I'm
not sure what the point of bringing that up is.


-----Original Message-----
From: Matthew Wilkes [mailto:matthew at matthewwilkes.co.uk]
Sent: Monday, June 01, 2009 10:29 AM
To: Dino Viehland
Cc: Python-Dev
Subject: Re: [Python-Dev] syntax warnings don't go through warnings.warn?


On 1 Jun 2009, at 17:50, Dino Viehland wrote:

> I'm just a little surprised by this - Is there a reason why syntax
> warnings are special and untrappable via warnings.warn?

Why should this work?  From the docs... "Python programmers issue
warnings by calling the warn() function defined in this module. (C
programmers use PyErr_WarnEx; see Exception Handling for details)."

Check out the warnings.catch_warnings context manager, but if you have
any further questions please direct them to the normal Python mailing
list, this is for development _of_ Python only.

Matthew

From matthew at matthewwilkes.co.uk  Mon Jun  1 19:49:04 2009
From: matthew at matthewwilkes.co.uk (Matthew Wilkes)
Date: Mon, 1 Jun 2009 18:49:04 +0100
Subject: [Python-Dev] syntax warnings don't go through warnings.warn?
In-Reply-To: <4A241300.4060200@voidspace.org.uk>
References: <1A472770E042064698CB5ADC83A12ACD0296CD4E@TK5EX14MBXC118.redmond.corp.microsoft.com>
	<2BB78789-E545-4C3A-9652-5DF473C8A4A7@matthewwilkes.co.uk>
	<4A241300.4060200@voidspace.org.uk>
Message-ID: <02BF8B44-5285-4574-8CE1-734BD579E3F1@matthewwilkes.co.uk>


On 1 Jun 2009, at 18:42, Michael Foord wrote:

> Dino is developing Python - he's one of the core developers of  
> IronPython

Ah, sorry, I'm bad with names, don't always pick up on who is who!

> and I suspect he is asking whether this is intentional, and  
> IronPython should implement the same behaviour, or whether it is a  
> bug.

The docs do seem to be clear though, warnings.warn creates a warning,  
but there are multiple other ways to do it, it's a convenience method.

Matt

From dinov at microsoft.com  Mon Jun  1 19:58:33 2009
From: dinov at microsoft.com (Dino Viehland)
Date: Mon, 1 Jun 2009 17:58:33 +0000
Subject: [Python-Dev] syntax warnings don't go through warnings.warn?
In-Reply-To: <02BF8B44-5285-4574-8CE1-734BD579E3F1@matthewwilkes.co.uk>
References: <1A472770E042064698CB5ADC83A12ACD0296CD4E@TK5EX14MBXC118.redmond.corp.microsoft.com>
	<2BB78789-E545-4C3A-9652-5DF473C8A4A7@matthewwilkes.co.uk>
	<4A241300.4060200@voidspace.org.uk>
	<02BF8B44-5285-4574-8CE1-734BD579E3F1@matthewwilkes.co.uk>
Message-ID: <1A472770E042064698CB5ADC83A12ACD0296DC13@TK5EX14MBXC118.redmond.corp.microsoft.com>

Ahh, ok, now I think I see...  there is clearly some other mechanism that
this warning is going through that we're not handling correctly.

I'll try and track it down.  Thanks!

-----Original Message-----
From: Matthew Wilkes [mailto:matthew at matthewwilkes.co.uk]
Sent: Monday, June 01, 2009 10:49 AM
To: Michael Foord
Cc: Dino Viehland; Python-Dev
Subject: Re: [Python-Dev] syntax warnings don't go through warnings.warn?


On 1 Jun 2009, at 18:42, Michael Foord wrote:

> Dino is developing Python - he's one of the core developers of
> IronPython

Ah, sorry, I'm bad with names, don't always pick up on who is who!

> and I suspect he is asking whether this is intentional, and
> IronPython should implement the same behaviour, or whether it is a
> bug.

The docs do seem to be clear though, warnings.warn creates a warning,
but there are multiple other ways to do it, it's a convenience method.

Matt


From python at rcn.com  Mon Jun  1 20:32:03 2009
From: python at rcn.com (Raymond Hettinger)
Date: Mon, 1 Jun 2009 11:32:03 -0700
Subject: [Python-Dev] Issues with Py3.1's new ipaddr
Message-ID: <2AD0D64C97C84D06AABB8C1292B5EE64@RaymondLaptop1>

In http://bugs.python.org/issue3959 , Clay McClure is raising some objections to the new ipaddr.py module.  JP Calderone shares his 
concerns.  I think they were the only commenters not directly affiliated with one of the competing projects.  The issues they've 
raised seem serious, but I don't know enough about the subject to make a meaningful comment.

Am hoping python-dev participants can provide some informed judgments.  At issue is whether the module has some deep conceptual 
flaws that would warrant pulling it out of the 3.1 release.  Also at issue is whether the addition was too rushed (see David Moss's 
comment on the tracker and later comments by Antoine Pitrou).

Does anyone here know if Clay's concern about subnets vs netmasks in accurate and whether it affects the usability of the module?


Raymond



From guido at python.org  Mon Jun  1 20:45:01 2009
From: guido at python.org (Guido van Rossum)
Date: Mon, 1 Jun 2009 11:45:01 -0700
Subject: [Python-Dev] Issues with Py3.1's new ipaddr
In-Reply-To: <2AD0D64C97C84D06AABB8C1292B5EE64@RaymondLaptop1>
References: <2AD0D64C97C84D06AABB8C1292B5EE64@RaymondLaptop1>
Message-ID: <ca471dc20906011145m232061ai6a149475dcb7a8d1@mail.gmail.com>

I haven't read the bug, but given the extensive real-life use that
ipaddr.py has seen at Google before inclusion into the stdlib, I think
"deep conceptual flaws" must be an overstatement. There may be real
differences of opinion about the politically correct way to view
subnets and netmasks, but I don't doubt that the module as it stands
is usable enough to keep it in the stdlib. Nothing's perfect.

I think we should just stick to "sorry, too late, try again for 3.2".
We've done that with plenty of more important flaws that were
discovered on the verge of a release, and I don't recall ever
regretting it. We can always add more features to the module in 3.2.

--Guido

On Mon, Jun 1, 2009 at 11:32 AM, Raymond Hettinger <python at rcn.com> wrote:
> In http://bugs.python.org/issue3959 , Clay McClure is raising some
> objections to the new ipaddr.py module. ?JP Calderone shares his concerns.
> ?I think they were the only commenters not directly affiliated with one of
> the competing projects. ?The issues they've raised seem serious, but I don't
> know enough about the subject to make a meaningful comment.
>
> Am hoping python-dev participants can provide some informed judgments. ?At
> issue is whether the module has some deep conceptual flaws that would
> warrant pulling it out of the 3.1 release. ?Also at issue is whether the
> addition was too rushed (see David Moss's comment on the tracker and later
> comments by Antoine Pitrou).
>
> Does anyone here know if Clay's concern about subnets vs netmasks in
> accurate and whether it affects the usability of the module?

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From rdmurray at bitdance.com  Mon Jun  1 21:11:45 2009
From: rdmurray at bitdance.com (R. David Murray)
Date: Mon, 1 Jun 2009 15:11:45 -0400 (EDT)
Subject: [Python-Dev] Issues with Py3.1's new ipaddr
In-Reply-To: <2AD0D64C97C84D06AABB8C1292B5EE64@RaymondLaptop1>
References: <2AD0D64C97C84D06AABB8C1292B5EE64@RaymondLaptop1>
Message-ID: <Pine.LNX.4.64.0906011457240.32513@kimball.webabinitio.net>

On Mon, 1 Jun 2009 at 11:32, Raymond Hettinger wrote:
> Does anyone here know if Clay's concern about subnets vs netmasks in accurate 
> and whether it affects the usability of the module?

I can't speak to usability of the module, not having looked at it yet,
but as far as I know from 10+ years of experience programming Cisco and
other routers, a host address can always also be considered as a subnet
consisting of one address.  The netmask is 255.255.255.255, and is called
a "host mask".  That said, address+hostmask is not always the way you
_think_ about an individual address; it depends on the context.

FWIW I hate dealing with non-subnet-aligned IP address ranges whenever
they come up.  But it is true that they do come up.

--David

From martin at v.loewis.de  Mon Jun  1 21:16:01 2009
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Mon, 01 Jun 2009 21:16:01 +0200
Subject: [Python-Dev] Issues with Py3.1's new ipaddr
In-Reply-To: <2AD0D64C97C84D06AABB8C1292B5EE64@RaymondLaptop1>
References: <2AD0D64C97C84D06AABB8C1292B5EE64@RaymondLaptop1>
Message-ID: <4A2428F1.7030708@v.loewis.de>

Raymond Hettinger wrote:
> Also at issue is whether the
> addition was too rushed (see David Moss's comment on the tracker and
> later comments by Antoine Pitrou).

That comment was from January 2009; it had two aspects
a) is ipaddr getting special treatment just because it was contributed
   by Google? and
b) shouldn't alternatives be considered?

For a), the answer was always a clear "no". That the code comes from
Google didn't affect inclusion at all.

For b), there have been several attempts to approach alternatives since.
For example, both netaddr people and ipaddr people tried to merge the
projects, unsuccessfully. Both Duncan McGreggor and David Moss
eventually spoke in favor of including ipaddr.

So I think this particular issue was resolved.

As for Clay McLure's issue: I feel it's primarily a matter of taste.
I see nothing morally wrong in using the same class for hosts and
networks, i.e. representing a host as a network of size 1. I can
understand why people dislike that, but I don't see why it would stop
people from doing with the library what they want to do.

A number of other features might be missing from the library, for
example, as Clay complains, there is no support for ranges of addresses
(or, more generally, arbitrary sets). While I understand that there
are good applications for such a data structure, I also think it can
be added when contributed (and I would think "range of addresses"
is still too narrow, for some applications I would envision - such
as computing consistency of BGP tables and whois databases).

Regards,
Martin

From mcguire at google.com  Tue Jun  2 03:54:21 2009
From: mcguire at google.com (Jake McGuire)
Date: Mon, 1 Jun 2009 18:54:21 -0700
Subject: [Python-Dev] Issues with Py3.1's new ipaddr
In-Reply-To: <4A2428F1.7030708@v.loewis.de>
References: <2AD0D64C97C84D06AABB8C1292B5EE64@RaymondLaptop1>
	<4A2428F1.7030708@v.loewis.de>
Message-ID: <77c780b40906011854t5afec054x718b2f59ee4e097c@mail.gmail.com>

On Mon, Jun 1, 2009 at 12:16 PM, "Martin v. L?wis" <martin at v.loewis.de>wrote:

> As for Clay McLure's issue: I feel it's primarily a matter of taste.
> I see nothing morally wrong in using the same class for hosts and
> networks, i.e. representing a host as a network of size 1. I can
> understand why people dislike that, but I don't see why it would stop
> people from doing with the library what they want to do.
>

To the extent that Clay is having issues, it's because ipaddr.py is poorly
documented, has potentially confusing comments, and he became confused.
 Lesser issues are that ipaddr.py doesn't work the way he wants and that ip
addressing is inherently subtle.

Looking at the code in detail shows that ipaddr.IP/IPv4/IPv6 objects always
represent *networks*.  He wants one particular address out of that network,
and that requires using __getitem__ or using IP.ip_ext. Neither is
particularly intuitive.

>>> import ipaddr
>>> ip = ipaddr.IPv4('10.33.11.17')
>>> ip
IPv4('10.33.11.17/32')
>>> ip[0]
'10.33.11.17'
>>> ip.ip_ext
'10.33.11.17'
>>>

This feels much more like poor documentation than wide-ranging conceptual
flaws.

I could put this in the tracker, but I'm not sure if that's appropriate.

-jake
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090601/48427223/attachment-0001.htm>

From jake at youtube.com  Tue Jun  2 04:47:46 2009
From: jake at youtube.com (Jake McGuire)
Date: Mon, 1 Jun 2009 19:47:46 -0700
Subject: [Python-Dev] Issues with Py3.1's new ipaddr
In-Reply-To: <77c780b40906011854t5afec054x718b2f59ee4e097c@mail.gmail.com>
References: <2AD0D64C97C84D06AABB8C1292B5EE64@RaymondLaptop1>
	<4A2428F1.7030708@v.loewis.de>
	<77c780b40906011854t5afec054x718b2f59ee4e097c@mail.gmail.com>
Message-ID: <77c780b40906011947x56ef79famb5b12891efe4963f@mail.gmail.com>

On Mon, Jun 1, 2009 at 6:54 PM, Jake McGuire <mcguire at google.com> wrote:

> On Mon, Jun 1, 2009 at 12:16 PM, "Martin v. L?wis" <martin at v.loewis.de>wrote:
>
>> As for Clay McLure's issue: I feel it's primarily a matter of taste.
>> I see nothing morally wrong in using the same class for hosts and
>> networks, i.e. representing a host as a network of size 1. I can
>> understand why people dislike that, but I don't see why it would stop
>> people from doing with the library what they want to do.
>>
>
> To the extent that Clay is having issues, it's because ipaddr.py is poorly
> documented, has potentially confusing comments, and he became confused.
>  Lesser issues are that ipaddr.py doesn't work the way he wants and that ip
> addressing is inherently subtle.
>

Sorry for the spam, I wrote my last message before reading the entire
discussion surrounding the two libraries and trying to imagine using both
ipaddr and netaddr.

Clay is basically correct.  The ipaddr.py API is missing important features,
and it would probably be a mistake to add it to the python standard library
if that means we'd have to maintain compatibility for the indefinite future.

Like all largeish python projects, we wrote a library to do IP address
manipulation.  In our case, it's a whopping 64 lines long.  While I wasn't
aware of ipaddr.py or netaddr at the time, the API we created is much closer
to netaddr's.  Migrating our code to ipaddr would require significant work
and is unlikely to happen.

In fact, if I was starting a new project from scratch with similar
requirements, I'd probably write my own library instead of using ipaddr.

-jake
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090601/5c2210db/attachment.htm>

From clay at daemons.net  Tue Jun  2 07:06:47 2009
From: clay at daemons.net (Clay McClure)
Date: Tue, 2 Jun 2009 01:06:47 -0400
Subject: [Python-Dev]  Issues with Py3.1's new ipaddr
Message-ID: <8657ee3f0906012206l11a82e9cl8af9fc4a7f587348@mail.gmail.com>

On Mon, Jun 1, 2009 at 11:32 AM, Guido van Rossum <guido at python.org > wrote:

> I haven't read the bug, but given the extensive real-life use that
> ipaddr.py has seen at Google before inclusion into the stdlib, I think
> "deep conceptual flaws" must be an overstatement.

When the users of the library are also the authors of the library, it
is not surprising that the library enjoys "extensive real-life use."
The real test of a library is not how well it succeeds at one
installation, but how well it meets the needs of the larger user base.

Having read the code and the author's comments, it seems to me that
networking concepts not employed at Google have been neglected. While
some of these features are easily added to ipaddr, their omission
exposes a narrow view of the problem domain that has resulted in more
fundamental problems in the library, such as the conflation of
addresses and networks.

> I think we should just stick to "sorry, too late, try again for 3.2".
> We've done that with plenty of more important flaws that were
> discovered on the verge of a release, and I don't recall ever
> regretting it. We can always add more features to the module in 3.2.

My concerns are not so much about adding features as they are about
changing the API. Addressing the concerns that I and others have
raised requires making backwards-incompatible changes. Doing that now,
before ipaddr enjoys widespread deployment as part of the stdlib,
seems prudent. Removing ipaddr from 3.1 appears to me less painful
than fixing apps when ipaddr's API changes in 3.2.

If this were a core feature that many developers were anxiously
awaiting, I could understand the desire to release and iterate. But is
there really a pressing need for an IP library in the stdlib? Until a
satisfactory library is available for inclusion in the stdlib, the few
developers that do require such functionality can easily enough
download a library that meets their needs.

Clay

From martin at v.loewis.de  Tue Jun  2 07:38:43 2009
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Tue, 02 Jun 2009 07:38:43 +0200
Subject: [Python-Dev] Issues with Py3.1's new ipaddr
In-Reply-To: <77c780b40906011947x56ef79famb5b12891efe4963f@mail.gmail.com>
References: <2AD0D64C97C84D06AABB8C1292B5EE64@RaymondLaptop1>	<4A2428F1.7030708@v.loewis.de>	<77c780b40906011854t5afec054x718b2f59ee4e097c@mail.gmail.com>
	<77c780b40906011947x56ef79famb5b12891efe4963f@mail.gmail.com>
Message-ID: <4A24BAE3.8070809@v.loewis.de>

> Clay is basically correct.  The ipaddr.py API is missing important
> features, and it would probably be a mistake to add it to the python
> standard library if that means we'd have to maintain compatibility for
> the indefinite future.

>From a maintenance point of view, these two statements don't really
relate. Yes, adding it to the standard library means that compatibility
must be maintained (not for the indefinite future, but in a very strong
sense). But no, that doesn't mean that it cannot have new features.

Adding new features would not have to break compatibility, and, in
many real-world cases of existing libraries, didn't.

For the net-vs-host issue, I think a backwards-compatible solution
is possible: just give the IP() function an option parameter that
makes it reject a netmask during parsing.

> Like all largeish python projects, we wrote a library to do IP address
> manipulation.  In our case, it's a whopping 64 lines long.
[...]
> In fact, if I was starting a new project from scratch with similar
> requirements, I'd probably write my own library instead of using ipaddr.

That was my feeling as well when ipaddr was first offered. It's just
not an important library, and people will continue to roll their own
for some time. OTOH, with ipaddr in the standard library, people will
also start contributing extensions that make it support their use cases,
so it should grow a wider application area than it currently supports.

Regards,
Martin

From martin at v.loewis.de  Tue Jun  2 07:47:28 2009
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Tue, 02 Jun 2009 07:47:28 +0200
Subject: [Python-Dev] Issues with Py3.1's new ipaddr
In-Reply-To: <8657ee3f0906012206l11a82e9cl8af9fc4a7f587348@mail.gmail.com>
References: <8657ee3f0906012206l11a82e9cl8af9fc4a7f587348@mail.gmail.com>
Message-ID: <4A24BCF0.5030800@v.loewis.de>

> If this were a core feature that many developers were anxiously
> awaiting, I could understand the desire to release and iterate. But is
> there really a pressing need for an IP library in the stdlib?

You should have asked this question a few month ago. Now, release
mechanics make it difficult to remove the library, except if a severe
problem was uncovered - which you haven't been able to demonstrate.

I don't believe that your issue "hosts and nets are represented with the
same class" is severe: it is very well possible to use the IP function
to represent individual hosts (including having a string representation
that doesn't print a netmask). The only possibly-missing feature is
support for rejecting host strings that include a mask on parsing; that
can be added in a backwards-compatible way as an optional parameter to
the IP function (as discussed on the tracker).

Regards,
Martin

From clay at daemons.net  Tue Jun  2 07:52:31 2009
From: clay at daemons.net (Clay McClure)
Date: Tue, 2 Jun 2009 01:52:31 -0400
Subject: [Python-Dev] Issues with Py3.1's new ipaddr
In-Reply-To: <4A24BAE3.8070809@v.loewis.de>
References: <2AD0D64C97C84D06AABB8C1292B5EE64@RaymondLaptop1>
	<4A2428F1.7030708@v.loewis.de>
	<77c780b40906011854t5afec054x718b2f59ee4e097c@mail.gmail.com>
	<77c780b40906011947x56ef79famb5b12891efe4963f@mail.gmail.com>
	<4A24BAE3.8070809@v.loewis.de>
Message-ID: <8657ee3f0906012252t7dd1ccefq79b56afcd3c57e55@mail.gmail.com>

On Tue, Jun 2, 2009 at 1:38 AM, "Martin v. L?wis" <martin at v.loewis.de> wrote:

> For the net-vs-host issue, I think a backwards-compatible solution
> is possible: just give the IP() function an option parameter that
> makes it reject a netmask during parsing.

That doesn't solve much. IPv4 objects still always use CIDR notation
when coerced to strings, meaning that IP addresses will always be
rendered with a trailing "/32". Such notation is unacceptable in
real-world applications that (correctly) distinguish between address
and network.

> That was my feeling as well when ipaddr was first offered. It's just
> not an important library, and people will continue to roll their own
> for some time. OTOH, with ipaddr in the standard library, people will
> also start contributing extensions that make it support their use cases,
> so it should grow a wider application area than it currently supports.

That being the case, why not delay its inclusion until we can be sure
that it in fact represents a good base upon which to build?

Clay

From clay at daemons.net  Tue Jun  2 08:01:28 2009
From: clay at daemons.net (Clay McClure)
Date: Tue, 2 Jun 2009 02:01:28 -0400
Subject: [Python-Dev] Issues with Py3.1's new ipaddr
In-Reply-To: <4A24BCF0.5030800@v.loewis.de>
References: <8657ee3f0906012206l11a82e9cl8af9fc4a7f587348@mail.gmail.com>
	<4A24BCF0.5030800@v.loewis.de>
Message-ID: <8657ee3f0906012301u39b70faat4a939251a0619486@mail.gmail.com>

On Tue, Jun 2, 2009 at 1:47 AM, "Martin v. L?wis" <martin at v.loewis.de> wrote:

>> If this were a core feature that many developers were anxiously
>> awaiting, I could understand the desire to release and iterate. But is
>> there really a pressing need for an IP library in the stdlib?
>
> You should have asked this question a few month ago. Now, release
> mechanics make it difficult to remove the library, except if a severe
> problem was uncovered - which you haven't been able to demonstrate.

This argument makes no sense. The feasibility of removing the library
does not depend on the severity of the issues found within it. Either
it is hard to remove the library, or it is easy. If it's hard to
remove, it doesn't get any easier because I discover a fatal flaw.

I've actually provided several examples of where the library fails
when used in common scenarios, yet your solution involves incremental
hacks that don't fix the underlying problems. You write as if you have
a vested interest in releasing the library -- why?

> I don't believe that your issue "hosts and nets are represented with the
> same class" is severe: it is very well possible to use the IP function
> to represent individual hosts (including having a string representation
> that doesn't print a netmask). The only possibly-missing feature is
> support for rejecting host strings that include a mask on parsing; that
> can be added in a backwards-compatible way as an optional parameter to
> the IP function (as discussed on the tracker).

There are other missing features, but again, my concern is not about
missing features: it's about a quirky API that conflates concepts in
the problem domain, leading to subtle bugs and breaking the principle
of least surprise.

Clay

From martin at v.loewis.de  Tue Jun  2 08:08:56 2009
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Tue, 02 Jun 2009 08:08:56 +0200
Subject: [Python-Dev] Issues with Py3.1's new ipaddr
In-Reply-To: <8657ee3f0906012252t7dd1ccefq79b56afcd3c57e55@mail.gmail.com>
References: <2AD0D64C97C84D06AABB8C1292B5EE64@RaymondLaptop1>	<4A2428F1.7030708@v.loewis.de>	<77c780b40906011854t5afec054x718b2f59ee4e097c@mail.gmail.com>	<77c780b40906011947x56ef79famb5b12891efe4963f@mail.gmail.com>	<4A24BAE3.8070809@v.loewis.de>
	<8657ee3f0906012252t7dd1ccefq79b56afcd3c57e55@mail.gmail.com>
Message-ID: <4A24C1F8.7080905@v.loewis.de>

Clay McClure wrote:
> On Tue, Jun 2, 2009 at 1:38 AM, "Martin v. L?wis" <martin at v.loewis.de> wrote:
> 
>> For the net-vs-host issue, I think a backwards-compatible solution
>> is possible: just give the IP() function an option parameter that
>> makes it reject a netmask during parsing.
> 
> That doesn't solve much. IPv4 objects still always use CIDR notation
> when coerced to strings, meaning that IP addresses will always be
> rendered with a trailing "/32".

That's not true:

py> x = ipaddr.IP("30.40.50.60")
py> print(x.ip_ext_full)
30.40.50.60

> Such notation is unacceptable in
> real-world applications that (correctly) distinguish between address
> and network.

So use a different notation that is also supported by the library.

>> That was my feeling as well when ipaddr was first offered. It's just
>> not an important library, and people will continue to roll their own
>> for some time. OTOH, with ipaddr in the standard library, people will
>> also start contributing extensions that make it support their use cases,
>> so it should grow a wider application area than it currently supports.
> 
> That being the case, why not delay its inclusion until we can be sure
> that it in fact represents a good base upon which to build?

Because we *are* sure that it in fact represents a good base upon which
to build.

Regards,
Martin

From martin at v.loewis.de  Tue Jun  2 08:15:42 2009
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Tue, 02 Jun 2009 08:15:42 +0200
Subject: [Python-Dev] Issues with Py3.1's new ipaddr
In-Reply-To: <8657ee3f0906012301u39b70faat4a939251a0619486@mail.gmail.com>
References: <8657ee3f0906012206l11a82e9cl8af9fc4a7f587348@mail.gmail.com>	<4A24BCF0.5030800@v.loewis.de>
	<8657ee3f0906012301u39b70faat4a939251a0619486@mail.gmail.com>
Message-ID: <4A24C38E.1050609@v.loewis.de>

> This argument makes no sense. The feasibility of removing the library
> does not depend on the severity of the issues found within it. Either
> it is hard to remove the library, or it is easy. If it's hard to
> remove, it doesn't get any easier because I discover a fatal flaw.

We could remove it, but then what we have wouldn't really be a release
candidate anymore, so the release would get delayed.

> I've actually provided several examples of where the library fails
> when used in common scenarios, yet your solution involves incremental
> hacks that don't fix the underlying problems. You write as if you have
> a vested interest in releasing the library -- why?

I have a vested interest in releasing Python 3.1, and in sticking to
decisions that have been made by other committers. I trust these fellow
committers, so I defend their decisions.

I personally have no plans for using this library, or any other IP
address library (at least not in any application I plan to write soon).
At the moment, I'm struggling more with IP address libraries in Perl.

> There are other missing features, but again, my concern is not about
> missing features: it's about a quirky API that conflates concepts in
> the problem domain, leading to subtle bugs and breaking the principle
> of least surprise.

I believe the API appears more quirky to you than it actually is,
probably because you don't have understood it fully (but I might
be wrong with that, and there might be a different reason).

Regards,
Martin

From rdmurray at bitdance.com  Tue Jun  2 15:22:33 2009
From: rdmurray at bitdance.com (R. David Murray)
Date: Tue, 2 Jun 2009 09:22:33 -0400 (EDT)
Subject: [Python-Dev] Issues with Py3.1's new ipaddr
In-Reply-To: <77c780b40906011854t5afec054x718b2f59ee4e097c@mail.gmail.com>
References: <2AD0D64C97C84D06AABB8C1292B5EE64@RaymondLaptop1>
	<4A2428F1.7030708@v.loewis.de>
	<77c780b40906011854t5afec054x718b2f59ee4e097c@mail.gmail.com>
Message-ID: <Pine.LNX.4.64.0906020919140.32513@kimball.webabinitio.net>

On Mon, 1 Jun 2009 at 18:54, Jake McGuire wrote:
> On Mon, Jun 1, 2009 at 12:16 PM, "Martin v. L?wis" <martin at v.loewis.de>wrote:
>> As for Clay McLure's issue: I feel it's primarily a matter of taste.
>> I see nothing morally wrong in using the same class for hosts and
>> networks, i.e. representing a host as a network of size 1. I can
>> understand why people dislike that, but I don't see why it would stop
>> people from doing with the library what they want to do.
>
> To the extent that Clay is having issues, it's because ipaddr.py is poorly
> documented, has potentially confusing comments, and he became confused.
> Lesser issues are that ipaddr.py doesn't work the way he wants and that ip
> addressing is inherently subtle.
>
> Looking at the code in detail shows that ipaddr.IP/IPv4/IPv6 objects always
> represent *networks*.  He wants one particular address out of that network,
> and that requires using __getitem__ or using IP.ip_ext. Neither is
> particularly intuitive.
>
>>>> import ipaddr
>>>> ip = ipaddr.IPv4('10.33.11.17')
>>>> ip
> IPv4('10.33.11.17/32')
>>>> ip[0]
> '10.33.11.17'

Actually I find that very intuitive if I understand that the objects
are always networks.

>>>> ip.ip_ext
> '10.33.11.17'

That's not intuitive :)

> This feels much more like poor documentation than wide-ranging conceptual
> flaws.
>
> I could put this in the tracker, but I'm not sure if that's appropriate.

I would say yes, put it in the tracker.

--David

From clay at daemons.net  Tue Jun  2 16:08:18 2009
From: clay at daemons.net (Clay McClure)
Date: Tue, 2 Jun 2009 10:08:18 -0400
Subject: [Python-Dev] Issues with Py3.1's new ipaddr
In-Reply-To: <Pine.LNX.4.64.0906020919140.32513@kimball.webabinitio.net>
References: <2AD0D64C97C84D06AABB8C1292B5EE64@RaymondLaptop1>
	<4A2428F1.7030708@v.loewis.de>
	<77c780b40906011854t5afec054x718b2f59ee4e097c@mail.gmail.com>
	<Pine.LNX.4.64.0906020919140.32513@kimball.webabinitio.net>
Message-ID: <8657ee3f0906020708i3d9330adx2c032d8521653153@mail.gmail.com>

On Tue, Jun 2, 2009 at 9:22 AM, R. David Murray <rdmurray at bitdance.com> wrote:

>>>>> ip[0]
>>
>> '10.33.11.17'
>
> Actually I find that very intuitive if I understand that the objects
> are always networks.

I suspect the authors would disagree with you on this point, since
they insist that host routes and host addresses are the same thing,
but assuming you are right, two flaws are immediately apparent:

* The ipaddr classes are poorly named. Since they model networks, they
should be named appropriately: IPNet or some such.

* The ipaddr authors have overlooked IP addresses, which also deserve
first-class treatment in any meaningful IP address library. It is
called ipaddr, after all, and not ipnet.

>>>>> ip.ip_ext
>>
>> '10.33.11.17'
>
> That's not intuitive :)

No, nor is the rest of the library intuitive. This is one of the
reasons I've called it "quirky".

Clay

From clay at daemons.net  Tue Jun  2 18:25:39 2009
From: clay at daemons.net (Clay McClure)
Date: Tue, 2 Jun 2009 12:25:39 -0400
Subject: [Python-Dev] Issues with Py3.1's new ipaddr
In-Reply-To: <4A24C38E.1050609@v.loewis.de>
References: <8657ee3f0906012206l11a82e9cl8af9fc4a7f587348@mail.gmail.com>
	<4A24BCF0.5030800@v.loewis.de>
	<8657ee3f0906012301u39b70faat4a939251a0619486@mail.gmail.com>
	<4A24C38E.1050609@v.loewis.de>
Message-ID: <8657ee3f0906020925i338f5376y949a0f6a68001db7@mail.gmail.com>

On Tue, Jun 2, 2009 at 2:15 AM, "Martin v. L?wis" <martin at v.loewis.de> wrote:

> We could remove it, but then what we have wouldn't really be a release
> candidate anymore, so the release would get delayed.

How long do release candidates soak in the field before being accepted?

I'm curious if an exception could be made in this case, given that you
have admitted that ipaddr is not an important library. The chances of
a problem being introduced due to its removal are vanishingly small.
No other components of the stdlib depend on ipaddr, and the few
(approximately zero?) developers who will have started writing
applications depending on ipaddr due to its inclusion in the release
candidate could simply download the library from Google.

> I believe the API appears more quirky to you than it actually is,
> probably because you don't have understood it fully (but I might
> be wrong with that, and there might be a different reason).

I believe the API is quirky because:

* It tries to represent distinct domain model concepts in a single class
* Its methods and properties are strangely named
* Methods and properties that should return domain model objects
return native types instead
* It cannot correctly parse output from netstat, nor can it correctly
pass values to ifconfig

You seem comfortable with these quirks, but then you're not planning
to write software with this library. Developers who do intend to write
meaningful network applications do seem concerned, yet we're ignored.
If you consult the issue notes, you'll see objections of the type I
just mentioned were raised months ago, yet no work has been done to
correct them. The only changes that I can see were related to pedantic
style issues: camel case and indentation.

Clay

From clay at daemons.net  Tue Jun  2 18:26:58 2009
From: clay at daemons.net (Clay McClure)
Date: Tue, 2 Jun 2009 12:26:58 -0400
Subject: [Python-Dev] Issues with Py3.1's new ipaddr
In-Reply-To: <4A24C1F8.7080905@v.loewis.de>
References: <2AD0D64C97C84D06AABB8C1292B5EE64@RaymondLaptop1>
	<4A2428F1.7030708@v.loewis.de>
	<77c780b40906011854t5afec054x718b2f59ee4e097c@mail.gmail.com>
	<77c780b40906011947x56ef79famb5b12891efe4963f@mail.gmail.com>
	<4A24BAE3.8070809@v.loewis.de>
	<8657ee3f0906012252t7dd1ccefq79b56afcd3c57e55@mail.gmail.com>
	<4A24C1F8.7080905@v.loewis.de>
Message-ID: <8657ee3f0906020926n42b41837k5afeb7d8333d063b@mail.gmail.com>

On Tue, Jun 2, 2009 at 2:08 AM, "Martin v. L?wis" <martin at v.loewis.de> wrote:

>> That doesn't solve much. IPv4 objects still always use CIDR notation
>> when coerced to strings, meaning that IP addresses will always be
>> rendered with a trailing "/32".
>
> That's not true:
>
> py> x = ipaddr.IP("30.40.50.60")
> py> print(x.ip_ext_full)
> 30.40.50.60

Thankfully the authors have provided this obscure and strangely-named
method to get at the correct string representation of an IP address,
but sadly their __str__ method -- which is the Pythonic way to get
string representations of objects -- fails in this regard because they
have only one class representing two distinct concepts.

I could probably make ipaddr work in my application; that is not the
issue (at least in my mind). The issue is that I shouldn't have to
work around design flaws in the library when they're simple enough to
fix before the library is bundled with the stdlib. I do not see the
utility of including ipaddr when my choice as a user is to either use
its strangely-named methods instead of obvious Pythonic conventions,
or to write my own library.

>> Such notation is unacceptable in
>> real-world applications that (correctly) distinguish between address
>> and network.
>
> So use a different notation that is also supported by the library.

I'm not referring to my software here -- I'm referring to applications
like ifconfig that expect addresses to be formatted properly. If the
default string representation produced by the ipaddr library does not
match the canonical representation expected by software that has
existed as long as IP itself, that indicates to me the library is
doing something wrong.

> Because we *are* sure that it in fact represents a good base upon which
> to build.

You certainly seem convinced. I could not disagree more.

Clay

From martin at v.loewis.de  Tue Jun  2 19:34:11 2009
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Tue, 02 Jun 2009 19:34:11 +0200
Subject: [Python-Dev] Issues with Py3.1's new ipaddr
In-Reply-To: <8657ee3f0906020925i338f5376y949a0f6a68001db7@mail.gmail.com>
References: <8657ee3f0906012206l11a82e9cl8af9fc4a7f587348@mail.gmail.com>	<4A24BCF0.5030800@v.loewis.de>	<8657ee3f0906012301u39b70faat4a939251a0619486@mail.gmail.com>	<4A24C38E.1050609@v.loewis.de>
	<8657ee3f0906020925i338f5376y949a0f6a68001db7@mail.gmail.com>
Message-ID: <4A256293.9000204@v.loewis.de>

>> We could remove it, but then what we have wouldn't really be a release
>> candidate anymore, so the release would get delayed.
> 
> How long do release candidates soak in the field before being accepted?

For this release, the release schedule is defined in PEP 375

> I'm curious if an exception could be made in this case, given that you
> have admitted that ipaddr is not an important library.

This would be need to be decided by the release manager. However, given
that Guido has already pronounced on this issue, Benjamin is unlikely to
change anything.

> You seem comfortable with these quirks, but then you're not planning
> to write software with this library. Developers who do intend to write
> meaningful network applications do seem concerned, yet we're ignored.

I don't hear a public outcry - only a single complainer.

Regards,
Martin

From exarkun at divmod.com  Tue Jun  2 19:56:00 2009
From: exarkun at divmod.com (Jean-Paul Calderone)
Date: Tue, 2 Jun 2009 13:56:00 -0400
Subject: [Python-Dev] Issues with Py3.1's new ipaddr
In-Reply-To: <4A256293.9000204@v.loewis.de>
Message-ID: <20090602175600.22176.619128085.divmod.quotient.1116@henry.divmod.com>

On Tue, 02 Jun 2009 19:34:11 +0200, "\"Martin v. L?wis\"" <martin at v.loewis.de> wrote:
> [snip]
>
>> You seem comfortable with these quirks, but then you're not planning
>> to write software with this library. Developers who do intend to write
>> meaningful network applications do seem concerned, yet we're ignored.
>
>I don't hear a public outcry - only a single complainer.

Clay repeatedly pointed out that other people have objected to ipaddr and
been ignored.  It's really, really disappointing to see you continue to
ignore not only them, but the repeated attempts Clay has made to point
them out.

I don't have time to argue this issue, but I agree with essentially
everything Clay has said in this thread, and I commented about these
problems on the ticket months ago, before ipaddr was added.

Jean-Paul

From python at rcn.com  Tue Jun  2 20:01:55 2009
From: python at rcn.com (Raymond Hettinger)
Date: Tue, 2 Jun 2009 11:01:55 -0700
Subject: [Python-Dev] Issues with Py3.1's new ipaddr
References: <8657ee3f0906012206l11a82e9cl8af9fc4a7f587348@mail.gmail.com><4A24BCF0.5030800@v.loewis.de><8657ee3f0906012301u39b70faat4a939251a0619486@mail.gmail.com><4A24C38E.1050609@v.loewis.de>
	<8657ee3f0906020925i338f5376y949a0f6a68001db7@mail.gmail.com>
Message-ID: <E46209EE56914F5ABE897E9AB635AEDA@RaymondLaptop1>

> We could remove it, but then what we have wouldn't really be a release
> candidate anymore, so the release would get delayed.

Not true.  There is a second release candidate scheduled on June 13th.
Removing the module involves doing "svn remove" on two files and
updating Misc/NEWS.  Yesterday, there was a conversation on IRC
(including the RM) where it was discussed.  So, in the unlikely event 
that Guido changes his mind, there is still time to do so.


Raymond






From casevh at gmail.com  Tue Jun  2 20:16:15 2009
From: casevh at gmail.com (Case Vanhorsen)
Date: Tue, 2 Jun 2009 11:16:15 -0700
Subject: [Python-Dev] Issues with Py3.1's new ipaddr
In-Reply-To: <4A256293.9000204@v.loewis.de>
References: <8657ee3f0906012206l11a82e9cl8af9fc4a7f587348@mail.gmail.com>
	<4A24BCF0.5030800@v.loewis.de>
	<8657ee3f0906012301u39b70faat4a939251a0619486@mail.gmail.com>
	<4A24C38E.1050609@v.loewis.de>
	<8657ee3f0906020925i338f5376y949a0f6a68001db7@mail.gmail.com>
	<4A256293.9000204@v.loewis.de>
Message-ID: <99e0b9530906021116m3af1107aya36d7a0998792ba6@mail.gmail.com>

On Tue, Jun 2, 2009 at 10:34 AM, "Martin v. L?wis" <martin at v.loewis.de> wrote:
>>> We could remove it, but then what we have wouldn't really be a release
>>> candidate anymore, so the release would get delayed.
>>
>> How long do release candidates soak in the field before being accepted?
>
> For this release, the release schedule is defined in PEP 375
>
>> I'm curious if an exception could be made in this case, given that you
>> have admitted that ipaddr is not an important library.
>
> This would be need to be decided by the release manager. However, given
> that Guido has already pronounced on this issue, Benjamin is unlikely to
> change anything.
>
>> You seem comfortable with these quirks, but then you're not planning
>> to write software with this library. Developers who do intend to write
>> meaningful network applications do seem concerned, yet we're ignored.
>
> I don't hear a public outcry - only a single complainer.

I normally just lurk on python-dev, but I will comment on this thread.
I manage several large IP address spaces and I've written my own IP
address tools in the past.

The comments on the thread motivated me to look at the ipaddr module.
I fully agree with Clay's comments. I would not use the module as it
stands.

I apologize for lurking too much and not commenting earlier.

casevh

>
> Regards,
> Martin
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/casevh%40gmail.com
>

From larry.bugbee at boeing.com  Tue Jun  2 20:25:38 2009
From: larry.bugbee at boeing.com (Bugbee, Larry)
Date: Tue, 2 Jun 2009 11:25:38 -0700
Subject: [Python-Dev] Issues with Py3.1's new ipaddr
In-Reply-To: <mailman.3955.1243965720.8014.python-dev@python.org>
References: <mailman.3955.1243965720.8014.python-dev@python.org>
Message-ID: <9418DB6C0B9D434190E54A78E931C3D1097E0E69@XCH-NW-7V1.nw.nos.boeing.com>


> The chances of a problem being introduced due to its removal 
> are vanishingly small.

But that provides little consolation to the user who sees it in the
standard library, is not aware to this discussion, and builds it into
his app.  Changes to the lib later may cause subtle but significant
effects.  ...perhaps undetected for a while.


> > > I don't hear a public outcry - only a single complainer.
> > 
> > Clay repeatedly pointed out that other people have objected 
> > to ipaddr and been ignored.  It's really, really disappointing 
> > to see you continue to ignore not only them, but the repeated 
> > attempts Clay has made to point them out.
> 
> I don't have time to argue this issue, but I agree with 
> essentially everything Clay has said in this thread....

I too agree.  If it is not ready, it is not ready.  Please don't create
problems for others.  Remove the lib until it is ready.

Larry

From p.f.moore at gmail.com  Tue Jun  2 22:02:19 2009
From: p.f.moore at gmail.com (Paul Moore)
Date: Tue, 2 Jun 2009 21:02:19 +0100
Subject: [Python-Dev] Issues with Py3.1's new ipaddr
In-Reply-To: <9418DB6C0B9D434190E54A78E931C3D1097E0E69@XCH-NW-7V1.nw.nos.boeing.com>
References: <mailman.3955.1243965720.8014.python-dev@python.org>
	<9418DB6C0B9D434190E54A78E931C3D1097E0E69@XCH-NW-7V1.nw.nos.boeing.com>
Message-ID: <79990c6b0906021302u8986b6fq6cdd4acb136947f9@mail.gmail.com>

2009/6/2 Bugbee, Larry <larry.bugbee at boeing.com>:
>> > > I don't hear a public outcry - only a single complainer.
>> >
>> > Clay repeatedly pointed out that other people have objected
>> > to ipaddr and been ignored. ?It's really, really disappointing
>> > to see you continue to ignore not only them, but the repeated
>> > attempts Clay has made to point them out.
>>
>> I don't have time to argue this issue, but I agree with
>> essentially everything Clay has said in this thread....
>
> I too agree. ?If it is not ready, it is not ready. ?Please don't create
> problems for others. ?Remove the lib until it is ready.

I don't write the sort of code that requires this module very often,
so I guess I class as an example of a casual user who would assume the
stdlib module was "best practice", and expect to use it naively to
avoid the more obvious "gotchas".

I've just now read the documentation for the first time (after reading
this thread) and for what it's worth here are my thoughts:

* I'd expect separate classes for "an IP address" and "a subnet" -
I've no problem with that expectation being wrong, but I'd like some
documentation as to *why* a single class is appropriate. (More
generally, the documentation seems pretty terse). Seeing "/32" all
over the place in the examples reinforces my feeling.

* I'm sorry, but ip_ext is a hopeless name. It makes no sense to me.
My first thought was "why not just use ip?" until I realised that
(contrary to the expectations of a naive user) an IP address is an
integer and that (uncommon???) usage gets to use the "ip" property
name. But still, why not "ip_str" at least?

* IP('1.2.3.4') vs IP('1.2.3.0/255.255.255.0') - it seems entirely
sane for the former to have a .ip/.ip_str/.ip_ext property, but
bizarre for the latter to. Explain the principles all you like, it
still seems peculiar.

* To be honest, I'd prefer to have the _ext versions be named without
a suffix, and the currently unsuffixed versions have an _int suffix.
That may reflect my naive expectations, and experts may well expect
the integer forms to be the more easily accessible, but is the library
intended for expert or non-expert users? (That's not a rhetorical
question). The people I work with (DBAs) deal with IP addresses all
the time, but almost certainly aren't even aware that they aren't
strings. If I did a poll, I guess most wouldn't even know that the
components were restricted to 0-255! I can't imagine them being
comfortable with this module.

Reading the documentation, I'd probably end up assuming it was for
experts, and writing my own code rather than using it - which is
probably a sign of failure for the module.

Simple example. If I want to scan all the IP addresses on my network
(my IP address is 192.168.1.101) I'd probably write:

    for i in range(253):
        ip = '192.168.1.' + str(i+1)
        ...

- and to heck with generality.

Even after reading the documentation, I've *no idea* how I would use
the ipaddr module to do this better. Let alone in as few lines.

My requirements certainly aren't important enough to drive the design
of the stdlib - and it's possible that these issues are *precisely*
the sort of things that can be fixed with documentation and
backward-compatible changes - but I also think that a bit more time to
address such things would be reasonable.

And I also apologise for not checking the module out sooner. Blame an
assumption that my trivial needs would "obviously" be covered...

Paul.

From rdmurray at bitdance.com  Tue Jun  2 22:53:16 2009
From: rdmurray at bitdance.com (R. David Murray)
Date: Tue, 2 Jun 2009 16:53:16 -0400 (EDT)
Subject: [Python-Dev] Issues with Py3.1's new ipaddr
In-Reply-To: <8657ee3f0906020926n42b41837k5afeb7d8333d063b@mail.gmail.com>
References: <2AD0D64C97C84D06AABB8C1292B5EE64@RaymondLaptop1>
	<4A2428F1.7030708@v.loewis.de>
	<77c780b40906011854t5afec054x718b2f59ee4e097c@mail.gmail.com>
	<77c780b40906011947x56ef79famb5b12891efe4963f@mail.gmail.com>
	<4A24BAE3.8070809@v.loewis.de>
	<8657ee3f0906012252t7dd1ccefq79b56afcd3c57e55@mail.gmail.com>
	<4A24C1F8.7080905@v.loewis.de>
	<8657ee3f0906020926n42b41837k5afeb7d8333d063b@mail.gmail.com>
Message-ID: <Pine.LNX.4.64.0906021350071.32513@kimball.webabinitio.net>

On Tue, 2 Jun 2009 at 12:26, Clay McClure wrote:
> On Tue, Jun 2, 2009 at 2:08 AM, "Martin v. L?wis" <martin at v.loewis.de> wrote:
>> py> x = ipaddr.IP("30.40.50.60")
>> py> print(x.ip_ext_full)
>> 30.40.50.60
>
> Thankfully the authors have provided this obscure and strangely-named
> method to get at the correct string representation of an IP address,
> but sadly their __str__ method -- which is the Pythonic way to get
> string representations of objects -- fails in this regard because they
> have only one class representing two distinct concepts.

Having thought more about this, I will agree with you that it would
be useful to have an address-without-netmask class. I'm thinking about
the cases where having an attached netmask is not particularly helpful,
for example in DNS entries.  But there are only two reasons I've so far
come up with why this would be useful:  to have a different default output
format, and to have a way to encode an IP so that I can have my code raise
an error if I try to use it in a context where a netmask is required.
(Note, however, that even a DNS entry can conceptually be considered to
be a host route.)

IMO, not having such a class is an inconvenience, not a show stopper,
especially since it seems like one could be added without breaking
backward compatibility.  I also don't particularly like the names of
the ipaddr attributes for accessing the IP address-without-netmask or
the netmask-without-ip-address, etc; but again, I don't consider that
a show stopper.

So I'm not in favor of pulling ipaddr from 3.1, and it's too late in
the release cycle to change anything.

I wish you had brought this energy to bear earlier, when changes could
have been made.  Reality is what it is, though, and now we should work
on making improvements for the next release.  I see in the ticket that
the netaddr folks were going to propose improvements so that they could
build netaddr on top of ipaddr, but I guess that didn't happen (yet?).

I have no association with Google, by the way, and I do intend to use
ipaddr in upcoming code, and have hacked my own address manipulation
stuff in previous code.

>>> Such notation is unacceptable in
>>> real-world applications that (correctly) distinguish between address
>>> and network.
>>
>> So use a different notation that is also supported by the library.
>
> I'm not referring to my software here -- I'm referring to applications
> like ifconfig that expect addresses to be formatted properly. If the
> default string representation produced by the ipaddr library does not
> match the canonical representation expected by software that has
> existed as long as IP itself, that indicates to me the library is
> doing something wrong.

I don't understand why you are trying to use ifconfig as an example.
It is actually a counter example to your thesis:  when working with an IP
address intended for consumption by ifconfig, you had best be using a
datatype that includes a netmask for that IP, or your code is going to
be broken.   What's more, modern versions of ifconfig _do_ accept the
default output format of ipaddr.  So this does the Right Thing:

     myip = ipaddr.IP('192.168.1.1/26')
     system('ifconfig eth0 {}'.format(myip))

If you use your ip-address-only class with ifconfig, you will wind up
using the classful default netmask, which is only going to be correct
by luck.

Hmm.  I think there is a conceptual divide here.  You have said you
think about IP addresses and networks as separate objects, so I wonder
if you would be pulling the netmask for ifconfig out of a separate
network object?

On the other hand I, a network professional, think about an IP address
paired with a netmask as a fundamental object.  Very rarely do I use
an IP address in isolation (without a netmask), and in many of _those_
cases there is an implied netmask of 255.255.255.255.  Networks to
me are closely related objects, defined by their "network number"
(the zero of the subnet, which is an IP address and normally not used
as a host address) and the mask.  So to me ipaddr's use of a single
datatype makes reasonable sense.  I would, as I said, above, find an
ip-without-netmask data type to be also useful (but a lot less important!)
I would also find a subclass that was network-only (rejects anything
but the zero of the subnet for the IP) useful.  But I think both of
those can be implemented fairly trivially as subclasses of the existing
ipaddr objects.

--David

PS: I've looked briefly at netaddr, and while I could probably work with
it by adding some equally trivial support code, I don't think it would
serve my needs as well as ipaddr will.  In particular, this is _very_
troubling:

>>> import netaddr
>>> ip = netaddr.IP('192.168.1.1/26')
>>> ip
IP('192.168.1.1/26')
>>> ip2 = netaddr.IP('192.168.1.1/27')
>>> ip2
IP('192.168.1.1/27')
>>> ip == ip2
True

The docs say the netmask is accepted only for "user convenience",
but to me the netmask is an integral part of the data entity I want to
manipulate.  Nor can I express ip-address-with-netmask using the CIDR
data type, since it will not accept anything but the zero of the network
as the IP.

In short, netaddr's object model does not match my desired model, while
ipaddr's is a lot closer to my desired model.

From mcguire at google.com  Tue Jun  2 22:11:22 2009
From: mcguire at google.com (Jake McGuire)
Date: Tue, 2 Jun 2009 13:11:22 -0700
Subject: [Python-Dev] Issues with Py3.1's new ipaddr
In-Reply-To: <8657ee3f0906020926n42b41837k5afeb7d8333d063b@mail.gmail.com>
References: <2AD0D64C97C84D06AABB8C1292B5EE64@RaymondLaptop1>
	<4A2428F1.7030708@v.loewis.de>
	<77c780b40906011854t5afec054x718b2f59ee4e097c@mail.gmail.com>
	<77c780b40906011947x56ef79famb5b12891efe4963f@mail.gmail.com>
	<4A24BAE3.8070809@v.loewis.de>
	<8657ee3f0906012252t7dd1ccefq79b56afcd3c57e55@mail.gmail.com>
	<4A24C1F8.7080905@v.loewis.de>
	<8657ee3f0906020926n42b41837k5afeb7d8333d063b@mail.gmail.com>
Message-ID: <77c780b40906021311y5520a7ffheb559f3f88f375f7@mail.gmail.com>

On Tue, Jun 2, 2009 at 9:26 AM, Clay McClure <clay at daemons.net> wrote:
> On Tue, Jun 2, 2009 at 2:08 AM, "Martin v. L?wis" <martin at v.loewis.de> wrote:
>
>>> That doesn't solve much. IPv4 objects still always use CIDR notation
>>> when coerced to strings, meaning that IP addresses will always be
>>> rendered with a trailing "/32".
>>
>> That's not true:
>>
>> py> x = ipaddr.IP("30.40.50.60")
>> py> print(x.ip_ext_full)
>> 30.40.50.60
>
> Thankfully the authors have provided this obscure and strangely-named
> method to get at the correct string representation of an IP address,
> but sadly their __str__ method -- which is the Pythonic way to get
> string representations of objects -- fails in this regard because they
> have only one class representing two distinct concepts.

The minimal demonstration of the problem of representing networks and
addresses using the same class:

>>> container = [1, 2, 3, 4]
>>> for item in container:
...   print "%s in %s: %s" % (item, container, item in container)
...
1 in [1, 2, 3, 4]: True
2 in [1, 2, 3, 4]: True
3 in [1, 2, 3, 4]: True
4 in [1, 2, 3, 4]: True
>>> import ipaddr
>>> container = ipaddr.IP('192.168.1.0/24')
>>> for item in container:
...   print "%s in %s: %s" % (item, container, item in container)
...
Traceback (most recent call last):
  File "<stdin>", line 2, in <module>
  File "ipaddr.py", line 438, in __contains__
    return self.network <= other.ip and self.broadcast >= other.broadcast
AttributeError: 'str' object has no attribute 'ip'
>>>

-jake

From rdmurray at bitdance.com  Tue Jun  2 23:27:47 2009
From: rdmurray at bitdance.com (R. David Murray)
Date: Tue, 2 Jun 2009 17:27:47 -0400 (EDT)
Subject: [Python-Dev] Issues with Py3.1's new ipaddr
In-Reply-To: <79990c6b0906021302u8986b6fq6cdd4acb136947f9@mail.gmail.com>
References: <mailman.3955.1243965720.8014.python-dev@python.org>
	<9418DB6C0B9D434190E54A78E931C3D1097E0E69@XCH-NW-7V1.nw.nos.boeing.com>
	<79990c6b0906021302u8986b6fq6cdd4acb136947f9@mail.gmail.com>
Message-ID: <Pine.LNX.4.64.0906021655160.32513@kimball.webabinitio.net>

On Tue, 2 Jun 2009 at 21:02, Paul Moore wrote:
> * I'd expect separate classes for "an IP address" and "a subnet" -
> I've no problem with that expectation being wrong, but I'd like some
> documentation as to *why* a single class is appropriate. (More
> generally, the documentation seems pretty terse). Seeing "/32" all
> over the place in the examples reinforces my feeling.

Yeah, the documentation needs a lot of work.  Since this is a
subject area I know something about, hopefully I can make time to
contribute something.

> * I'm sorry, but ip_ext is a hopeless name. It makes no sense to me.

My guess is that it is "ip external representation", as opposed
to the internal representation as an integer.  I agree that the
name is terrible.

> My first thought was "why not just use ip?" until I realised that
> (contrary to the expectations of a naive user) an IP address is an
> integer and that (uncommon???) usage gets to use the "ip" property
> name. But still, why not "ip_str" at least?

Agreed; unless you are talking to C code, I'd think you'd want the string
representation.  This seems like an odd design decision.

> * IP('1.2.3.4') vs IP('1.2.3.0/255.255.255.0') - it seems entirely
> sane for the former to have a .ip/.ip_str/.ip_ext property, but
> bizarre for the latter to. Explain the principles all you like, it
> still seems peculiar.

It may seem peculiar, but IMO it is correct.  The ip is the zero
of the network, and is just as valid an IP as an ip inside the
network with the same netmask.

> * To be honest, I'd prefer to have the _ext versions be named without
> a suffix, and the currently unsuffixed versions have an _int suffix.

I agree.  I wish I'd looked at this back when it was put in, but I was
a lot busier with other things then :)

> That may reflect my naive expectations, and experts may well expect
> the integer forms to be the more easily accessible, but is the library
> intended for expert or non-expert users? (That's not a rhetorical
> question). The people I work with (DBAs) deal with IP addresses all
> the time, but almost certainly aren't even aware that they aren't
> strings. If I did a poll, I guess most wouldn't even know that the
> components were restricted to 0-255! I can't imagine them being
> comfortable with this module.

If they don't know that, and they only work with IP addresses in the most
trivial form, then I don't think they would need any of the services this
library provides.  That doesn't mean the library is for "experts", but
it does mean it's for people who need to manipulate IP addresses within
the context of networks.  (If all you need to do is move IP addresses
around, just keep them as strings.)

Hmm.  Except I suppose they could use the input validation checking...

I think we've already agreed that adding a flag to IP saying 'no netmask'
is a good idea...if the object created that way would by default output
without a netmask, then the trivial needs of your DBA's would probably
be met.

> Reading the documentation, I'd probably end up assuming it was for
> experts, and writing my own code rather than using it - which is
> probably a sign of failure for the module.
>
> Simple example. If I want to scan all the IP addresses on my network
> (my IP address is 192.168.1.101) I'd probably write:
>
>    for i in range(253):
>        ip = '192.168.1.' + str(i+1)
>        ...
>
> - and to heck with generality.
>
> Even after reading the documentation, I've *no idea* how I would use
> the ipaddr module to do this better. Let alone in as few lines.

     net = ipaddr.IP('192.168.1.0/24'):
     for i in range(253):
         ip = net[i]
         ...

So, that's one example that needs to be added to the docs.

I'd have liked to write that as:

     for ip in ipaddr.IP('192.168.1.0/24')[:253]:
         ...

but apparently it doesn't support slicing (time for an RFE :)

> My requirements certainly aren't important enough to drive the design
> of the stdlib - and it's possible that these issues are *precisely*
> the sort of things that can be fixed with documentation and
> backward-compatible changes - but I also think that a bit more time to
> address such things would be reasonable.

If they are documentation and backward compatible changes (and I believe
they are), why not get the thing into the field so more people can
provide feedback and RFEs?

> And I also apologise for not checking the module out sooner. Blame an
> assumption that my trivial needs would "obviously" be covered...

Yeah, me too.

--David

From martin at v.loewis.de  Tue Jun  2 23:32:59 2009
From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=)
Date: Tue, 02 Jun 2009 23:32:59 +0200
Subject: [Python-Dev] Issues with Py3.1's new ipaddr
In-Reply-To: <20090602175600.22176.619128085.divmod.quotient.1116@henry.divmod.com>
References: <20090602175600.22176.619128085.divmod.quotient.1116@henry.divmod.com>
Message-ID: <4A259A8B.1090109@v.loewis.de>

> Clay repeatedly pointed out that other people have objected to ipaddr and
> been ignored.  It's really, really disappointing to see you continue to
> ignore not only them, but the repeated attempts Clay has made to point
> them out.

[I meant to stop discussing here, but I just want comment on this remark]

I had seen objections from Victor Stinner, which I did not fully
understand, but he seemed to be saying that he is ok with including
ipaddr. I had also seen objections from David Moss, which he then
seems to have withdrawn. I did not take your message (msg78675 in
the tracker) as an objection - you just seemed to express a preference
to use netaddr instead. You said it had minor quirks, and some of them
have to be removed - but I would not infer that the library should be
exclude because of these minor quirks.

> I don't have time to argue this issue, but I agree with essentially
> everything Clay has said in this thread, and I commented about these
> problems on the ticket months ago, before ipaddr was added.

I now understand (but honestly didn't understand before) that you
are objecting to ipaddr's inclusion, and that you would prefer its
removal at this point.

Regards,
Martin

From clay at daemons.net  Wed Jun  3 00:25:10 2009
From: clay at daemons.net (Clay McClure)
Date: Tue, 2 Jun 2009 18:25:10 -0400
Subject: [Python-Dev] Issues with Py3.1's new ipaddr
In-Reply-To: <Pine.LNX.4.64.0906021350071.32513@kimball.webabinitio.net>
References: <2AD0D64C97C84D06AABB8C1292B5EE64@RaymondLaptop1>
	<4A2428F1.7030708@v.loewis.de>
	<77c780b40906011854t5afec054x718b2f59ee4e097c@mail.gmail.com>
	<77c780b40906011947x56ef79famb5b12891efe4963f@mail.gmail.com>
	<4A24BAE3.8070809@v.loewis.de>
	<8657ee3f0906012252t7dd1ccefq79b56afcd3c57e55@mail.gmail.com>
	<4A24C1F8.7080905@v.loewis.de>
	<8657ee3f0906020926n42b41837k5afeb7d8333d063b@mail.gmail.com>
	<Pine.LNX.4.64.0906021350071.32513@kimball.webabinitio.net>
Message-ID: <8657ee3f0906021525h74d28aeet614cda2e4c481d9d@mail.gmail.com>

On Tue, Jun 2, 2009 at 4:53 PM, R. David Murray <rdmurray at bitdance.com> wrote:

> Having thought more about this, I will agree with you that it would
> be useful to have an address-without-netmask class.

Not only useful, but necessary. It seems there are few people on this
list who understand IP well enough to realize how distorted ipaddr
actually is.

> (Note, however, that even a DNS entry can conceptually be considered to
> be a host route.)

Not at all. A host route and a host address (as represented in DNS)
are fundamentally different concepts. Please see my recent post to
ipaddr-py-dev for a refresher on these concepts:

http://groups.google.com/group/ipaddr-py-dev/t/94d54fe581d24e72

> So I'm not in favor of pulling ipaddr from 3.1, and it's too late in
> the release cycle to change anything.

I'm not sure why you say that when others have said that another
release candidate is planned, and that removing ipaddr is essentially
trivial to do.

> I wish you had brought this energy to bear earlier, when changes could
> have been made. ?Reality is what it is, though, and now we should work
> on making improvements for the next release. ?I see in the ticket that
> the netaddr folks were going to propose improvements so that they could
> build netaddr on top of ipaddr, but I guess that didn't happen (yet?).

I don't think that can happen, actually. If I was a netaddr committer
(which I'm not), I would find it hard to salvage anything reusable
from ipaddr. It is certainly simpler and clearer to start over with an
object model that actually makes sense.

> I have no association with Google, by the way, and I do intend to use
> ipaddr in upcoming code, and have hacked my own address manipulation
> stuff in previous code.

Sorry, I wasn't aware of that. My mistake. Regardless, I find that
your understanding of IP is similar to that of ipaddr's authors, which
is to say imprecise.

> I don't understand why you are trying to use ifconfig as an example.

Because it's an obvious real world example that explains why these two
strings are not equivalent:

192.168.1.1

and

192.168.1.1/32

You and others continue to suggest that those strings are equivalent,
yet ifconfig is a tool that has been around for thirty years that
clearly demonstrates that those strings are not equivalent. If what
you say is true, I should be able to pass either string to ifconfig
and get the same result. That is not the case, because the strings are
not equivalent, because a host route is not the same thing as a host
address.

> So this does the Right Thing:
>
> ? ?myip = ipaddr.IP('192.168.1.1/26')
> ? ?system('ifconfig eth0 {}'.format(myip))

Sure, but shouldn't this also do the right thing?

address = ipaddr.IP('192.168.1.1')
netmask = ipaddr.IP('255.255.255.192')
system("ifconfig eth0 %s/%s" % (address, netmask))

It doesn't.

> Hmm. ?I think there is a conceptual divide here. ?You have said you
> think about IP addresses and networks as separate objects, so I wonder
> if you would be pulling the netmask for ifconfig out of a separate
> network object?

Of course, because addresses don't have masks; networks do. This command:

ifconfig en0 192.168.1.1/24

is shorthand for operator convenience. What's going on behind the
scenes is quite a lot different than it looks. First, ifconfig
computes a network address by masking the supplied interface address
with the supplied network mask. It is then able to configure a route
for the proper network: 192.168.1.0/24.

The fact that "192.168.1.1/24" appears in the command does *not* mean
that the address 192.168.1.1 has a mask of /24. That is absurd.
Addresses don't have masks; networks do. That's why they're called
netmasks.

> On the other hand I, a network professional, think about an IP address
> paired with a netmask as a fundamental object.

The IT industry is unique among engineering disciplines in that formal
training and licensing are typically not required for IT
professionals. Whereas concepts like resistance, current, and voltage
have very specific meanings to electrical engineers, the IT vernacular
is not so precise. Since formal training is rare, and what little is
available is often high-priced and vendor-specific, IT professionals
tend to learn their trade from trade books, word of mouth, and
hands-on experience. As a result, IT professionals tend to have a good
working knowledge of how technology applies to their particular job,
but may not have an appreciation of the more theoretical aspects of
the discipline.

What this means in practice is that your experience as a network
professional may not resemble the experiences of other network
professionals. That you tend to think of addresses as having masks is
probably not universal, or even particularly common, among network
professionals. Some electrical engineers probably think of voltage as
pressure, and that may be a useful abstraction, but I would be
surprised to see a voltmeter calibrated in pascals.

What are we to do? How do we arrive at a common understanding of our
domain? We should consult the canonical sources of truth: RFC-791, and
the BSD IP implementation. In neither of those will you see that an IP
address has a mask.

If this were any other problem domain, I think it should be obvious
that the design of the library is flawed. But given that this is IP, a
subject that many people think they understand but actually don't, the
design flaws are obscured by ambiguity.

Guido's earlier comment about "political correctness" underscores this
point. This is not simply a case of me preferring my way of thinking
about IP addresses to the way the ipaddr authors think about IP
addresses. I'm simply stating a fact that, were it not true, the
Internet would not function: addresses and networks are not the same
thing. To represent them in the same class is a mistake.

> But I think both of
> those can be implemented fairly trivially as subclasses of the existing
> ipaddr objects.

Yes, I could certainly see BaseIPAddress and BaseIPNetwork classes
inheriting from BaseIP, with the ipaddr.IP function selectively return
an object of either type depending on what the user passes in the
constructor. If they include a mask, it's a network. If they don't,
it's an address. I think that particular change might be backwards
compatible, but I'm not sure. None the less, other changes that I
think are important (returning IP objects instead of strings and ints
for some properties; renaming poorly named methods) do change the API
and thus are not backwards compatible without adding lots of cruft.
Better to fix those now, in my opinion.

> In short, netaddr's object model does not match my desired model, while
> ipaddr's is a lot closer to my desired model.

I'm not advocating netaddr. The decision of which library to use has
already been made; I'm not debating that point. I'm merely suggesting
that we pull ipaddr from the release until such time that it can be
evolved to have a more agreeable API. ipaddr might be usable for you
(probably because your limited understanding of IP matches the ipaddr
developers'), but we've already heard from a handful of others on this
list who would rather roll their own library than suffer through the
quirks in ipaddr's current implementation.

Clay

From solipsis at pitrou.net  Wed Jun  3 01:18:09 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Tue, 2 Jun 2009 23:18:09 +0000 (UTC)
Subject: [Python-Dev] Issues with Py3.1's new ipaddr
References: <2AD0D64C97C84D06AABB8C1292B5EE64@RaymondLaptop1>
	<4A2428F1.7030708@v.loewis.de>
	<77c780b40906011854t5afec054x718b2f59ee4e097c@mail.gmail.com>
	<77c780b40906011947x56ef79famb5b12891efe4963f@mail.gmail.com>
	<4A24BAE3.8070809@v.loewis.de>
	<8657ee3f0906012252t7dd1ccefq79b56afcd3c57e55@mail.gmail.com>
	<4A24C1F8.7080905@v.loewis.de>
	<8657ee3f0906020926n42b41837k5afeb7d8333d063b@mail.gmail.com>
	<Pine.LNX.4.64.0906021350071.32513@kimball.webabinitio.net>
	<8657ee3f0906021525h74d28aeet614cda2e4c481d9d@mail.gmail.com>
Message-ID: <loom.20090602T230904-345@post.gmane.org>

Clay McClure <clay <at> daemons.net> writes:
> 
> Not only useful, but necessary. It seems there are few people on this
> list who understand IP well enough to realize how distorted ipaddr
> actually is.

Not having myself enough knowledge about IP routing and administration (although
I did happen to dissect BGP and IS-IS packets in an earlier life), I'm, however,
inclined to trust Jean-Paul's word when it comes to network programming. IMHO,
we should delay ipaddr's official inclusion in the stdlib until the conceptual
issues are solved.

Regards

Antoine.



From drkjam at gmail.com  Wed Jun  3 01:50:53 2009
From: drkjam at gmail.com (DrKJam)
Date: Wed, 3 Jun 2009 00:50:53 +0100
Subject: [Python-Dev] Issues with Py3.1's new ipaddr
In-Reply-To: <loom.20090602T230904-345@post.gmane.org>
References: <2AD0D64C97C84D06AABB8C1292B5EE64@RaymondLaptop1>
	<77c780b40906011854t5afec054x718b2f59ee4e097c@mail.gmail.com>
	<77c780b40906011947x56ef79famb5b12891efe4963f@mail.gmail.com>
	<4A24BAE3.8070809@v.loewis.de>
	<8657ee3f0906012252t7dd1ccefq79b56afcd3c57e55@mail.gmail.com>
	<4A24C1F8.7080905@v.loewis.de>
	<8657ee3f0906020926n42b41837k5afeb7d8333d063b@mail.gmail.com>
	<Pine.LNX.4.64.0906021350071.32513@kimball.webabinitio.net>
	<8657ee3f0906021525h74d28aeet614cda2e4c481d9d@mail.gmail.com>
	<loom.20090602T230904-345@post.gmane.org>
Message-ID: <538a660a0906021650p7a5bfa65j53194db40d023c86@mail.gmail.com>

I've just subscribed to python-dev again after being pointed towards this
thread (thanks Raymond).

I'd be the first to accept failings and oddities in the interface of my own
library. Since its release netaddr has taken its own interesting set of
twists and turns. However, all along I have tried to be responsive and
accommodating with regard to my users needs and requests for new features
and bug fixes. There has been a lot of useful feedback which I have
faithfully incorporated into netaddr's codebase.

It would be a shame to see all the hard work go to waste on Y.A.P.I.M. :-

    http://code.google.com/p/netaddr/wiki/YetAnotherPythonIPModule

There is a veritable graveyard of stuff out there! Some good, some not so
good.

The netaddr/ipaddr-py merge went awry and our projects decided to remain
separate. I don't see any benefit in raking over those old coals; it's all
water under the bridge.

That being said, based on the passion in this thread, the decision to
include ipaddr-py into the stdlib seems to be proving too hasty and
contentious for some. It really might be worth taking a step back and taking
heed of those concerns.

Having had quite a while to think about this over the last few months, this
is what I would ideally like to see.

A sensible discussion from a *broad* base of network admins, sysadmins and
developers using Python on the formulation of a reasonable functional
specification and design for a brand new library (without the associated
baggage and vested interests of any particular implementation). This would
be an opportunity for us to nail our respective problems and differences
while simultaneously bringing together most of the Python community behind
code that WE WILL ACTUALLY USE. If this is going in the stdlib surely that
is doubly important?

If possible, I would particularly like to see input from Victor Stinner, the
current IPy maintainer. There as some wrinkles and failings in IPy's
interface and implementation but its actually not as bad as I first thought
having actually spent some time implementing its interface (mostly
successfully) as a facade on top of netaddr. See the netaddr repo if you are
interested and *no* it is not supported code ;-)

Would this actually be feasible or am I just a hopeless optimist? Should we
formulate a PEP even though calls for that have so far been rejected?
Possibly PEPs don't apply to libraries?

Whoever overseas this would need to be someone with a fairly neutral
perspective on all of this.

Dave M.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090603/974cf544/attachment.htm>

From andrewm at object-craft.com.au  Wed Jun  3 02:13:08 2009
From: andrewm at object-craft.com.au (Andrew McNamara)
Date: Wed, 3 Jun 2009 10:13:08 +1000
Subject: [Python-Dev] Issues with Py3.1's new ipaddr
In-Reply-To: <20090602175600.22176.619128085.divmod.quotient.1116@henry.divmod.com>
References: <20090602175600.22176.619128085.divmod.quotient.1116@henry.divmod.com>
Message-ID: <F6D076DA-5542-4532-8D99-D44B588E8F3E@object-craft.com.au>


On 03/06/2009, at 3:56 AM, Jean-Paul Calderone wrote:

> On Tue, 02 Jun 2009 19:34:11 +0200, "\"Martin v. L?wis\"" <martin at v.loewis.de 
> > wrote:
>> [snip]
>>
>>> You seem comfortable with these quirks, but then you're not planning
>>> to write software with this library. Developers who do intend to  
>>> write
>>> meaningful network applications do seem concerned, yet we're  
>>> ignored.
>>
>> I don't hear a public outcry - only a single complainer.
>
> Clay repeatedly pointed out that other people have objected to  
> ipaddr and
> been ignored.  It's really, really disappointing to see you continue  
> to
> ignore not only them, but the repeated attempts Clay has made to point
> them out.
>
> I don't have time to argue this issue, but I agree with essentially
> everything Clay has said in this thread, and I commented about these
> problems on the ticket months ago, before ipaddr was added.

Indeed... "Me too" - I've been quietly concerned with these issues,  
but have have not said anything as Clay's postings pretty much cover  
it (and swine flu response is trumping all my other priorities).

From guido at python.org  Wed Jun  3 04:39:36 2009
From: guido at python.org (Guido van Rossum)
Date: Tue, 2 Jun 2009 19:39:36 -0700
Subject: [Python-Dev] Issues with Py3.1's new ipaddr
In-Reply-To: <F6D076DA-5542-4532-8D99-D44B588E8F3E@object-craft.com.au>
References: <20090602175600.22176.619128085.divmod.quotient.1116@henry.divmod.com>
	<F6D076DA-5542-4532-8D99-D44B588E8F3E@object-craft.com.au>
Message-ID: <ca471dc20906021939wa12e7b6y4e0f7f1a9349d9ce@mail.gmail.com>

Well, it sounds like the current incarnation of the ipaddr module is
widely loathed, even if it also has some fans. Since it is still
available as a 3rd party module, and hasn't been available in other
releases except 3.1 beta and rc1, I expect few users will be
inconvenienced if we withdraw it at this point.

Benjamin, what would be involved in removing it? I suppose there's the
module itself, some unit tests, and some docs. (I'm not asking you to
remove it yet -- but I'm asking to look into the consequences, so that
we can be sure to do the right thing before releasing 3.1 final.)

I'm disappointed in the process -- it's as if nobody really reviewed
the API until it was released with rc1, and this despite there being a
significant discussion about its inclusion and alternatives months
ago. (Don't look at me -- I wouldn't recognize a netmask if it bit me
in the behind, and I can honestly say that I don't know whether /8
means to look only at the first 8 bits or whether it means to mask off
the last 8 bits.)

I hope we can learn from this.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From greg at krypto.org  Wed Jun  3 04:41:20 2009
From: greg at krypto.org (Gregory P. Smith)
Date: Tue, 2 Jun 2009 19:41:20 -0700
Subject: [Python-Dev] Issues with Py3.1's new ipaddr
In-Reply-To: <77c780b40906021311y5520a7ffheb559f3f88f375f7@mail.gmail.com>
References: <2AD0D64C97C84D06AABB8C1292B5EE64@RaymondLaptop1> 
	<4A2428F1.7030708@v.loewis.de>
	<77c780b40906011854t5afec054x718b2f59ee4e097c@mail.gmail.com> 
	<77c780b40906011947x56ef79famb5b12891efe4963f@mail.gmail.com> 
	<4A24BAE3.8070809@v.loewis.de>
	<8657ee3f0906012252t7dd1ccefq79b56afcd3c57e55@mail.gmail.com> 
	<4A24C1F8.7080905@v.loewis.de>
	<8657ee3f0906020926n42b41837k5afeb7d8333d063b@mail.gmail.com> 
	<77c780b40906021311y5520a7ffheb559f3f88f375f7@mail.gmail.com>
Message-ID: <52dc1c820906021941r7bf5ad25l1433dc9ed33ec3a4@mail.gmail.com>

On Tue, Jun 2, 2009 at 1:11 PM, Jake McGuire <mcguire at google.com> wrote:
> The minimal demonstration of the problem of representing networks and
> addresses using the same class:

fwiw, that (hosts vs networks in the same class) is not what you are
demonstrating below.  What you demonstrate is that its silly for
iteration over a network to return strings rather than IP objects.
Regardless, it is a good example of a problem with the API.

ipaddr could be changed to yield IPv4 /32 or IPv6 /128 objects when
iterating over it instead of strings and this example would work
properly.

>>>> container = [1, 2, 3, 4]
>>>> for item in container:
> ... ? print "%s in %s: %s" % (item, container, item in container)
> ...
> 1 in [1, 2, 3, 4]: True
> 2 in [1, 2, 3, 4]: True
> 3 in [1, 2, 3, 4]: True
> 4 in [1, 2, 3, 4]: True
>>>> import ipaddr
>>>> container = ipaddr.IP('192.168.1.0/24')
>>>> for item in container:
> ... ? print "%s in %s: %s" % (item, container, item in container)
> ...
> Traceback (most recent call last):
> ?File "<stdin>", line 2, in <module>
> ?File "ipaddr.py", line 438, in __contains__
> ? ?return self.network <= other.ip and self.broadcast >= other.broadcast
> AttributeError: 'str' object has no attribute 'ip'
>>>>

Regardless, after reading the many different opinions on this thread
especially including those of other python developers it sounds like
we should not include ipaddr in 3.1.

This example is IMHO one good reason to not include ipaddr in the
standard library as it stands.

It sounds like we have a 3.1rc2 scheduled for June 13th.  At this
point based on the multitude of other developer comments in these
threads and barring strong arguments in favor of including it as it
stands I will remove it because there seem to be quite a number of
people with objections to it as an API in its current form.

I've filed http://bugs.python.org/issue6182 as a release blocker to
track its removal (or not) based on any further discussion in these
python-dev threads.

I do believe an API for an ip and network address manipulation library
can be worked out but during the 3.1 release candidate phase is not
the time to hastily do that and choose one so removal due to
disagreement seems the best option.

Would someone volunteer write up a proposal (PEP) for such with a goal
of having it completed within the next few months?  (I know people
have suggested various other libraries, that counts; i have not taken
the time to look at them).

-gps

From greg at krypto.org  Wed Jun  3 04:43:49 2009
From: greg at krypto.org (Gregory P. Smith)
Date: Tue, 2 Jun 2009 19:43:49 -0700
Subject: [Python-Dev] Issues with Py3.1's new ipaddr
In-Reply-To: <ca471dc20906021939wa12e7b6y4e0f7f1a9349d9ce@mail.gmail.com>
References: <20090602175600.22176.619128085.divmod.quotient.1116@henry.divmod.com>
	<F6D076DA-5542-4532-8D99-D44B588E8F3E@object-craft.com.au> 
	<ca471dc20906021939wa12e7b6y4e0f7f1a9349d9ce@mail.gmail.com>
Message-ID: <52dc1c820906021943o390bffefk74cbcbc1e760b387@mail.gmail.com>

On Tue, Jun 2, 2009 at 7:39 PM, Guido van Rossum <guido at python.org> wrote:
>
> I'm disappointed in the process -- it's as if nobody really reviewed
> the API until it was released with rc1, and this despite there being a
> significant discussion about its inclusion and alternatives months
> ago. (Don't look at me -- I wouldn't recognize a netmask if it bit me
> in the behind, and I can honestly say that I don't know whether /8
> means to look only at the first 8 bits or whether it means to mask off
> the last 8 bits.)
>
> I hope we can learn from this.

Yep, thats a fair summary.  Removal will be trivial.

Should it only be removed from py3k branch or also from trunk pending
a decision as to if the library is reworked or if something else
entirely is adopted?

From aahz at pythoncraft.com  Wed Jun  3 04:44:58 2009
From: aahz at pythoncraft.com (Aahz)
Date: Tue, 2 Jun 2009 19:44:58 -0700
Subject: [Python-Dev] Issues with Py3.1's new ipaddr
In-Reply-To: <ca471dc20906021939wa12e7b6y4e0f7f1a9349d9ce@mail.gmail.com>
References: <20090602175600.22176.619128085.divmod.quotient.1116@henry.divmod.com>
	<F6D076DA-5542-4532-8D99-D44B588E8F3E@object-craft.com.au>
	<ca471dc20906021939wa12e7b6y4e0f7f1a9349d9ce@mail.gmail.com>
Message-ID: <20090603024458.GA22308@panix.com>

On Tue, Jun 02, 2009, Guido van Rossum wrote:
>
> I hope we can learn from this.

I'm not thrilled with adding more process just because we had a problem
here, and the only obvious solution I see is to require a PEP every time
a module is added.  Based on what I've seen of this discussion so far, I
think that cure would at this time be worse than the disease.
-- 
Aahz (aahz at pythoncraft.com)           <*>         http://www.pythoncraft.com/

my-python-code-runs-5x-faster-this-month-thanks-to-dumping-$2K-
    on-a-new-machine-ly y'rs  - tim

From benjamin at python.org  Wed Jun  3 04:53:36 2009
From: benjamin at python.org (Benjamin Peterson)
Date: Tue, 2 Jun 2009 21:53:36 -0500
Subject: [Python-Dev] Issues with Py3.1's new ipaddr
In-Reply-To: <ca471dc20906021939wa12e7b6y4e0f7f1a9349d9ce@mail.gmail.com>
References: <20090602175600.22176.619128085.divmod.quotient.1116@henry.divmod.com>
	<F6D076DA-5542-4532-8D99-D44B588E8F3E@object-craft.com.au>
	<ca471dc20906021939wa12e7b6y4e0f7f1a9349d9ce@mail.gmail.com>
Message-ID: <1afaf6160906021953l3157a361oa0956514ac9f0ae2@mail.gmail.com>

2009/6/2 Guido van Rossum <guido at python.org>:
> Benjamin, what would be involved in removing it? I suppose there's the
> module itself, some unit tests, and some docs. (I'm not asking you to
> remove it yet -- but I'm asking to look into the consequences, so that
> we can be sure to do the right thing before releasing 3.1 final.)

As Raymond and Gregory have pointed out in this thread, the library is
quite independent as it stands now in the stlib, so should be trivial
to remove. Nothing else should be affected.



-- 
Regards,
Benjamin

From benjamin at python.org  Wed Jun  3 04:55:44 2009
From: benjamin at python.org (Benjamin Peterson)
Date: Tue, 2 Jun 2009 21:55:44 -0500
Subject: [Python-Dev] Issues with Py3.1's new ipaddr
In-Reply-To: <20090603024458.GA22308@panix.com>
References: <20090602175600.22176.619128085.divmod.quotient.1116@henry.divmod.com>
	<F6D076DA-5542-4532-8D99-D44B588E8F3E@object-craft.com.au>
	<ca471dc20906021939wa12e7b6y4e0f7f1a9349d9ce@mail.gmail.com>
	<20090603024458.GA22308@panix.com>
Message-ID: <1afaf6160906021955x7f0c6dd5x490f85e4b2701617@mail.gmail.com>

2009/6/2 Aahz <aahz at pythoncraft.com>:
> On Tue, Jun 02, 2009, Guido van Rossum wrote:
>>
>> I hope we can learn from this.
>
> I'm not thrilled with adding more process just because we had a problem
> here, and the only obvious solution I see is to require a PEP every time
> a module is added. ?Based on what I've seen of this discussion so far, I
> think that cure would at this time be worse than the disease.

I don't think he's suggesting adding more process yet, just in the
diagnosis phase.



-- 
Regards,
Benjamin

From clay at daemons.net  Wed Jun  3 04:58:09 2009
From: clay at daemons.net (Clay McClure)
Date: Tue, 2 Jun 2009 22:58:09 -0400
Subject: [Python-Dev] Issues with Py3.1's new ipaddr
In-Reply-To: <52dc1c820906021941r7bf5ad25l1433dc9ed33ec3a4@mail.gmail.com>
References: <2AD0D64C97C84D06AABB8C1292B5EE64@RaymondLaptop1>
	<4A2428F1.7030708@v.loewis.de>
	<77c780b40906011854t5afec054x718b2f59ee4e097c@mail.gmail.com>
	<77c780b40906011947x56ef79famb5b12891efe4963f@mail.gmail.com>
	<4A24BAE3.8070809@v.loewis.de>
	<8657ee3f0906012252t7dd1ccefq79b56afcd3c57e55@mail.gmail.com>
	<4A24C1F8.7080905@v.loewis.de>
	<8657ee3f0906020926n42b41837k5afeb7d8333d063b@mail.gmail.com>
	<77c780b40906021311y5520a7ffheb559f3f88f375f7@mail.gmail.com>
	<52dc1c820906021941r7bf5ad25l1433dc9ed33ec3a4@mail.gmail.com>
Message-ID: <8657ee3f0906021958x6dcbf40ap3c9ba6ced09651c8@mail.gmail.com>

On Tue, Jun 2, 2009 at 10:41 PM, Gregory P. Smith <greg at krypto.org> wrote:

> ipaddr could be changed to yield IPv4 /32 or IPv6 /128 objects when
> iterating over it instead of strings and this example would work
> properly.

I have opened an issue in the ipaddr bug tracker that I think
addresses this issue. There are a few methods and properties in the
ipaddr.IP classes that return native types but should return IP
objects:

http://code.google.com/p/ipaddr-py/issues/detail?id=21

> It sounds like we have a 3.1rc2 scheduled for June 13th. ?At this
> point based on the multitude of other developer comments in these
> threads and barring strong arguments in favor of including it as it
> stands I will remove it because there seem to be quite a number of
> people with objections to it as an API in its current form.

Thank you.

> Would someone volunteer write up a proposal (PEP) for such with a goal
> of having it completed within the next few months? ?(I know people
> have suggested various other libraries, that counts; i have not taken
> the time to look at them).

I am happy to take a stab at that.

Thanks,

Clay

From benjamin at python.org  Wed Jun  3 04:58:33 2009
From: benjamin at python.org (Benjamin Peterson)
Date: Tue, 2 Jun 2009 21:58:33 -0500
Subject: [Python-Dev] Issues with Py3.1's new ipaddr
In-Reply-To: <52dc1c820906021943o390bffefk74cbcbc1e760b387@mail.gmail.com>
References: <20090602175600.22176.619128085.divmod.quotient.1116@henry.divmod.com>
	<F6D076DA-5542-4532-8D99-D44B588E8F3E@object-craft.com.au>
	<ca471dc20906021939wa12e7b6y4e0f7f1a9349d9ce@mail.gmail.com>
	<52dc1c820906021943o390bffefk74cbcbc1e760b387@mail.gmail.com>
Message-ID: <1afaf6160906021958r623f12fbjebefa963ba639a9@mail.gmail.com>

2009/6/2 Gregory P. Smith <greg at krypto.org>:
> Should it only be removed from py3k branch or also from trunk pending
> a decision as to if the library is reworked or if something else
> entirely is adopted?

I think it should disappear from the whole python tree for the moment.
Even the unstable trunk is not a good place for a module pending
uncertain, future changes.



-- 
Regards,
Benjamin

From andrewm at object-craft.com.au  Wed Jun  3 06:55:49 2009
From: andrewm at object-craft.com.au (Andrew McNamara)
Date: Wed, 3 Jun 2009 14:55:49 +1000
Subject: [Python-Dev] Issues with Py3.1's new ipaddr
In-Reply-To: <ca471dc20906021939wa12e7b6y4e0f7f1a9349d9ce@mail.gmail.com>
References: <20090602175600.22176.619128085.divmod.quotient.1116@henry.divmod.com>
	<F6D076DA-5542-4532-8D99-D44B588E8F3E@object-craft.com.au>
	<ca471dc20906021939wa12e7b6y4e0f7f1a9349d9ce@mail.gmail.com>
Message-ID: <AC456743-4681-4A0F-B109-F80781C3E743@object-craft.com.au>


On 03/06/2009, at 12:39 PM, Guido van Rossum wrote:

> I'm disappointed in the process -- it's as if nobody really reviewed
> the API until it was released with rc1, and this despite there being a
> significant discussion about its inclusion and alternatives months
> ago. (Don't look at me -- I wouldn't recognize a netmask if it bit me
> in the behind, and I can honestly say that I don't know whether /8
> means to look only at the first 8 bits or whether it means to mask off
> the last 8 bits.)
>
> I hope we can learn from this.

When including third-party modules into the standard library, we've  
generally only included them after they have broad acceptance in the  
community. In this case, however, it seems that while the ipaddr  
module had acceptance within Google, it had not had much exposure to  
the broader python community. I think if anyone other than Guido had  
proposed adding the module to the standard library, we would not have  
even considered it until it had spent some time standing on it's own  
two feet.

From stephen at xemacs.org  Wed Jun  3 07:31:11 2009
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Wed, 03 Jun 2009 14:31:11 +0900
Subject: [Python-Dev] Issues with Py3.1's new ipaddr
In-Reply-To: <20090603024458.GA22308@panix.com>
References: <20090602175600.22176.619128085.divmod.quotient.1116@henry.divmod.com>
	<F6D076DA-5542-4532-8D99-D44B588E8F3E@object-craft.com.au>
	<ca471dc20906021939wa12e7b6y4e0f7f1a9349d9ce@mail.gmail.com>
	<20090603024458.GA22308@panix.com>
Message-ID: <87ws7thoio.fsf@uwakimon.sk.tsukuba.ac.jp>

Aahz writes:

 > On Tue, Jun 02, 2009, Guido van Rossum wrote:
 > >
 > > I hope we can learn from this.
 > 
 > I'm not thrilled with adding more process just because we had a problem
 > here, and the only obvious solution I see is to require a PEP every time
 > a module is added.  Based on what I've seen of this discussion so far, I
 > think that cure would at this time be worse than the disease.

+1

A couple of people commented that they didn't say anything because
they were really busy.  I don't think there's much you can do about
that, unless the time machine can be used to unmutate the swine flu!
FWIW I also agree with Martin's assessment of the thread in the
tracker that it looked like there was only one person strongly
opposed.  Mostly an unfortunate combination of circumstances.

One thing I would recommend is that while inclusion is not a matter of
voting, people who are recognized as domain experts on Python-Dev
*should* try to add their "+1" or "-1" early.  Especially if they
don't expect to have time to participate actively in discussion.

After all, they can always change their assessment based on either
changes or as a response to a persuasive lobby, precisely because it's
not a vote.

From stefan_ml at behnel.de  Wed Jun  3 09:08:51 2009
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Wed, 03 Jun 2009 09:08:51 +0200
Subject: [Python-Dev] Serious regression in doctest in Py3.1rc1
Message-ID: <h057jb$bg3$1@ger.gmane.org>

Hi,

I can't currently file a bug report on this, but I was told by Lisandro
Dalc?n that there is a serious problem with the doctest module in Py3.1rc1.
In Cython, we use doctests to test the compiler in that we compile a
Python/Cython module with doctests into a C module and then run doctest on
the imported extension module.

>From the error report it seems to me that doctest is now trying to read the
module itself through linecache for some reason, which horribly fails for a
binary module.

Could someone please look into this? I'll open up a bug report tomorrow
unless someone beats me to it.

Thanks,

Stefan


From p.f.moore at gmail.com  Wed Jun  3 09:51:54 2009
From: p.f.moore at gmail.com (Paul Moore)
Date: Wed, 3 Jun 2009 08:51:54 +0100
Subject: [Python-Dev] Issues with Py3.1's new ipaddr
In-Reply-To: <87ws7thoio.fsf@uwakimon.sk.tsukuba.ac.jp>
References: <20090602175600.22176.619128085.divmod.quotient.1116@henry.divmod.com>
	<F6D076DA-5542-4532-8D99-D44B588E8F3E@object-craft.com.au>
	<ca471dc20906021939wa12e7b6y4e0f7f1a9349d9ce@mail.gmail.com>
	<20090603024458.GA22308@panix.com>
	<87ws7thoio.fsf@uwakimon.sk.tsukuba.ac.jp>
Message-ID: <79990c6b0906030051l3aac84c6v5f68803096554d2d@mail.gmail.com>

2009/6/3 Stephen J. Turnbull <stephen at xemacs.org>:
> Aahz writes:
>
> ?> On Tue, Jun 02, 2009, Guido van Rossum wrote:
> ?> >
> ?> > I hope we can learn from this.
> ?>
> ?> I'm not thrilled with adding more process just because we had a problem
> ?> here, and the only obvious solution I see is to require a PEP every time
> ?> a module is added. ?Based on what I've seen of this discussion so far, I
> ?> think that cure would at this time be worse than the disease.
>
> +1
[...]
>
> One thing I would recommend is that while inclusion is not a matter of
> voting, people who are recognized as domain experts on Python-Dev
> *should* try to add their "+1" or "-1" early. ?Especially if they
> don't expect to have time to participate actively in discussion.
>
> After all, they can always change their assessment based on either
> changes or as a response to a persuasive lobby, precisely because it's
> not a vote.

FWIW, I'd add some points:

1. Publishing the documentation somewhere prominent would help. I
don't know if that happened here, but it makes it a *lot* easier for
people to "have a quick look". Downloading a zip file, unpacking the
docs and opening a HTML file (or worse still, building it first!) can
be enough of a barrier to stop people who are pressed for time from
commenting. (Once the module was included, it gets into the online
Python docs, hence I could read them and comment).

2. Encouraging a clear +1/-1 from people, in addition to discussion on
specific points, would clarify things. I believe Martin commented that
he hadn't realised that one of the opposing comments was a strong
enough objection to count as a -1.

3. Discussion should happen on python-dev, not on the tracker. (Some
people may object to this, I know). I saw the call for input to the
tracker item, and thought "that's not a module I'm likely to need,
I'll leave it to the experts" and did nothing more. When the
discussion flared up on python-dev, on the other hand, I kept skimming
the discussion, and when I saw something that seemed at my level, I
felt encouraged to comment. Also, seeing that there *was* disagreement
encouraged me to comment - if the experts aren't agreeing, maybe my
non-expert view might be helpful input on usability/intuitiveness.

But I agree, let's not add more process if a bit of focus in the
discussion is enough.

Paul.

From p.f.moore at gmail.com  Wed Jun  3 10:02:25 2009
From: p.f.moore at gmail.com (Paul Moore)
Date: Wed, 3 Jun 2009 09:02:25 +0100
Subject: [Python-Dev] Issues with Py3.1's new ipaddr
In-Reply-To: <Pine.LNX.4.64.0906021655160.32513@kimball.webabinitio.net>
References: <mailman.3955.1243965720.8014.python-dev@python.org>
	<9418DB6C0B9D434190E54A78E931C3D1097E0E69@XCH-NW-7V1.nw.nos.boeing.com>
	<79990c6b0906021302u8986b6fq6cdd4acb136947f9@mail.gmail.com>
	<Pine.LNX.4.64.0906021655160.32513@kimball.webabinitio.net>
Message-ID: <79990c6b0906030102m5549f543j8bd75592d26e82c4@mail.gmail.com>

2009/6/2 R. David Murray <rdmurray at bitdance.com>:
> On Tue, 2 Jun 2009 at 21:02, Paul Moore wrote:
>> Simple example. If I want to scan all the IP addresses on my network
>> (my IP address is 192.168.1.101) I'd probably write:
>>
>> ? for i in range(253):
>> ? ? ? ip = '192.168.1.' + str(i+1)
>> ? ? ? ...
>>
>> - and to heck with generality.
>>
>> Even after reading the documentation, I've *no idea* how I would use
>> the ipaddr module to do this better. Let alone in as few lines.
>
> ? ?net = ipaddr.IP('192.168.1.0/24'):
> ? ?for i in range(253):
> ? ? ? ?ip = net[i]
> ? ? ? ?...
>
> So, that's one example that needs to be added to the docs.
>
> I'd have liked to write that as:
>
> ? ?for ip in ipaddr.IP('192.168.1.0/24')[:253]:
> ? ? ? ?...
>
> but apparently it doesn't support slicing (time for an RFE :)

Given that what I *want* to do is to skip the "special" cases of 0 and
255, would the following work?

net = ipaddr.IP('192.168.1.101/255.255.255.0') # Note 1
for ip in net:
    if ip.ip = ip.broadcast or ip.is_multicast():
        continue
    ...

That would be what I mean by the module helping me to avoid "gotchas"
- like assuming 0 and 255 are the "special" multicast and broadcast
(or whatever they are) addresses that I shouldn't be testing in my
port scanner.

I'd like a .is_broadcast() method. Does that expose my lack of
understanding, or is it a sensible thing to add?

Note 1 - by the way, I use this form because I don't understand how
the /24 notation works. I can get the subnet mask from ipconfig, so I
use that. Ideally, I'd rather just put my IP address and have the
module work out the "obvious" subnet (at my level of use, it's always
255.255.255.0 for 192.168 addresses) but I guess that's not actually a
well-defined idea.

Paul.

From mike at pennington.net  Wed Jun  3 10:42:47 2009
From: mike at pennington.net (Mike Pennington)
Date: Wed, 03 Jun 2009 03:42:47 -0500
Subject: [Python-Dev] Issues with Py3.1's new ipaddr
Message-ID: <4A263787.5010203@pennington.net>

Raymond solicited a comment from me about the design of ipaddr.  By way 
of full disclosure, I have a small competing project called pynet.

That said, I test drove ipaddr for about 30 minutes and so far like the 
big-picture API design quite a bit.  I'll specifically address Clay's 
concern about hosts vs networks, because this issue is important to me; 
I've been in the network engineering field for over 15 years, worked on 
Cisco's product development team, and held a CCIE (consider it the 
equivalent of a CPA for network engineers) for 10 years...

Clay seems to object to ipaddr's IP object because it is not the same as 
the object model used in the BSD ip stack.  Indeed, I'm one of the 
raving fans of what BSD has done for the quality of ip networking, but 
let's also consider their requirements.  BSD must approach ip networking 
from a host perspective, it is the consumer of individual IP packets and 
their payloads. ipaddr's whole point of existence is really driven 
towards the manipulation of potentially massive lists of ip addresses. 
This is no small difference in requirements, and I believe ipaddr's 
different approach makes their code much simpler for the tasks it needs 
to do.  Incorporating host addresses as a special case of a /32 IPv4 
network or /128 IPv6 network makes a lot of sense to me, in fact, I also 
chose this same object model.  Perl's NetAddr::IP does this too, it is 
considered the gold standard for perl's address manipulation module.

Whether python includes ipaddr now, later, or uses another module 
entirely does not bother me.  Whatever is included should have a very 
stable API, and major bugs should be worked out.  Documentation should 
be good enough for the average consumer, and if anything this is where 
ipaddr to be lacking a bit.

I hope that python does include something to manipulate IPv4 and IPv6 
address blocks in the future, since this is a big hole is python's 
batteries-included philosophy.  However, I'd need more time at the wheel 
of ipaddr before I could comment whether this truly is ready for 
inclusion in stdlib.

All the best,
\m

From amauryfa at gmail.com  Wed Jun  3 13:26:13 2009
From: amauryfa at gmail.com (Amaury Forgeot d'Arc)
Date: Wed, 3 Jun 2009 13:26:13 +0200
Subject: [Python-Dev] Serious regression in doctest in Py3.1rc1
In-Reply-To: <h057jb$bg3$1@ger.gmane.org>
References: <h057jb$bg3$1@ger.gmane.org>
Message-ID: <e27efe130906030426o45081fd5v9bbb877ffd76503d@mail.gmail.com>

Hello,

2009/6/3 Stefan Behnel <stefan_ml at behnel.de>:
> Hi,
>
> I can't currently file a bug report on this, but I was told by Lisandro
> Dalc?n that there is a serious problem with the doctest module in Py3.1rc1.
> In Cython, we use doctests to test the compiler in that we compile a
> Python/Cython module with doctests into a C module and then run doctest on
> the imported extension module.
>
> >From the error report it seems to me that doctest is now trying to read the
> module itself through linecache for some reason, which horribly fails for a
> binary module.
>
> Could someone please look into this? I'll open up a bug report tomorrow
> unless someone beats me to it.

I don't have the time either, but the problem looks very similar to
http://bugs.python.org/issue4050
The fix was to replace:
     file = inspect.getsourcefile(object) or inspect.getfile(object)
was replaced with
     file = inspect.getsourcefile(object)

-- 
Amaury Forgeot d'Arc

From barry at python.org  Wed Jun  3 14:31:50 2009
From: barry at python.org (Barry Warsaw)
Date: Wed, 3 Jun 2009 08:31:50 -0400
Subject: [Python-Dev] Issues with Py3.1's new ipaddr
In-Reply-To: <52dc1c820906021943o390bffefk74cbcbc1e760b387@mail.gmail.com>
References: <20090602175600.22176.619128085.divmod.quotient.1116@henry.divmod.com>
	<F6D076DA-5542-4532-8D99-D44B588E8F3E@object-craft.com.au>
	<ca471dc20906021939wa12e7b6y4e0f7f1a9349d9ce@mail.gmail.com>
	<52dc1c820906021943o390bffefk74cbcbc1e760b387@mail.gmail.com>
Message-ID: <C7DFDA96-4170-4A8A-B377-24544D2C100D@python.org>

On Jun 2, 2009, at 10:43 PM, Gregory P. Smith wrote:

> Should it only be removed from py3k branch or also from trunk pending
> a decision as to if the library is reworked or if something else
> entirely is adopted?

I think it should be removed from trunk if it's removed from the py3k  
branch.  Nothing goes into Python 2.7 that isn't already in Python 3.1.

-Barry

-------------- next part --------------
A non-text attachment was scrubbed...
Name: PGP.sig
Type: application/pgp-signature
Size: 832 bytes
Desc: This is a digitally signed message part
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090603/a58148ce/attachment.pgp>

From barry at python.org  Wed Jun  3 14:37:00 2009
From: barry at python.org (Barry Warsaw)
Date: Wed, 3 Jun 2009 08:37:00 -0400
Subject: [Python-Dev] Issues with Py3.1's new ipaddr
In-Reply-To: <ca471dc20906021939wa12e7b6y4e0f7f1a9349d9ce@mail.gmail.com>
References: <20090602175600.22176.619128085.divmod.quotient.1116@henry.divmod.com>
	<F6D076DA-5542-4532-8D99-D44B588E8F3E@object-craft.com.au>
	<ca471dc20906021939wa12e7b6y4e0f7f1a9349d9ce@mail.gmail.com>
Message-ID: <15494D8B-AEF9-4204-9B85-19AD23325F95@python.org>

On Jun 2, 2009, at 10:39 PM, Guido van Rossum wrote:

> I hope we can learn from this.

One crazy thought: let's use the Cheeseshop.

When I search for 'ipaddr' I get three hits, with Google's module at  
the top with a score of '8'.  I really don't know what that means but  
I'm guessing it means that module is "two times better" than the next  
highest score of 4 for ipaddresslib.

It would be really nice if say the Cheeseshop had a voting feature.   
Use PEP 10 voting to get a rough estimate of a module's popularity  
(download counts alone might not tell you everything).  Then at least  
you can get a rough idea of how generally popular a module is in the  
wider community.  Also, a module should have to live on its own two  
feet for while on Cheeseshop before being considered for inclusion in  
the stdlib.

-Barry

-------------- next part --------------
A non-text attachment was scrubbed...
Name: PGP.sig
Type: application/pgp-signature
Size: 832 bytes
Desc: This is a digitally signed message part
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090603/c510577d/attachment.pgp>

From ncoghlan at gmail.com  Wed Jun  3 14:49:18 2009
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Wed, 03 Jun 2009 22:49:18 +1000
Subject: [Python-Dev] Issues with Py3.1's new ipaddr
In-Reply-To: <79990c6b0906030102m5549f543j8bd75592d26e82c4@mail.gmail.com>
References: <mailman.3955.1243965720.8014.python-dev@python.org>	<9418DB6C0B9D434190E54A78E931C3D1097E0E69@XCH-NW-7V1.nw.nos.boeing.com>	<79990c6b0906021302u8986b6fq6cdd4acb136947f9@mail.gmail.com>	<Pine.LNX.4.64.0906021655160.32513@kimball.webabinitio.net>
	<79990c6b0906030102m5549f543j8bd75592d26e82c4@mail.gmail.com>
Message-ID: <4A26714E.2000209@gmail.com>

Paul Moore wrote:

> Note 1 - by the way, I use this form because I don't understand how
> the /24 notation works. I can get the subnet mask from ipconfig, so I
> use that.

<OT>

It's just a shorthand way of writing IPv4 net masks based on their
binary form:

/8 = 8 ones followed by 24 zeroes = 255.0.0.0
/16 = 16 ones followed by 16 zeroes = 255.255.0.0
/24 = 24 ones followed by 8 zeroes = 255.255.255.0
/30 = 30 ones followed by 2 zeroes = 255.255.255.252
/32 = 32 ones followed by no zeroes = 255.255.255.255

It's particularly convenient when you're dividing subnets up into chunks
that don't align with a byte boundary in the IPv4 address (e.g. /27 can
easily be recognised as giving a subnet containing 2**5 = 32 hosts, but
the subnet size is significantly less obvious when written using the
equivalent 255.255.255.224 netmask).

</OT>

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------

From davide.alberani at gmail.com  Wed Jun  3 16:35:39 2009
From: davide.alberani at gmail.com (Davide Alberani)
Date: Wed, 3 Jun 2009 16:35:39 +0200
Subject: [Python-Dev]  Issues with Py3.1's new ipaddr
In-Reply-To: <538a660a0906021650p7a5bfa65j53194db40d023c86@mail.gmail.com>
Message-ID: <20090603143539.GA3657@gmail.com>

> I've just subscribed to python-dev again after being pointed towards
> this thread (thanks Raymond).

The same for me, thanks. :-)
I'm the author of IPLib [1]; I don't consider myself an expert on the
subject and my code dates back to Python 1.6 times (last updated in 2005).
Moreover, while it works for me and its almost non-existent user base,
I assume it can't be used for anything but as a source of ideas.

> There is a veritable graveyard of stuff out there! Some good,
> some not so good.

The fact is that most of the times a programmer writes yet another
Python IP module to cover only a very limited and usually simple
aspect of IP manipulation that he needs at the given time (a conversion
to/from decimal, a check for inclusion in a CIDR, ...)

> A sensible discussion from a *broad* base of network admins, 
> sysadmins and developers using Python on the formulation of a
> reasonable functional specification and design for a brand new
> library

To me, this makes a lot of sense: check which are the most used
modules and ask the users.
Generically speaking, I tend to agree with Clay, as I always had
looked at IP addresses, netmasks and CIDR blocks as separate
concepts, but again: I'm not an expert.


+++
[1] http://erlug.linux.it/~da/soft/iplib/
    Only supports IPv4; its main use is to convert amongst notations,
    but can be used to check if an IP (or another subnet) is included
    in a CIDR block and to gather some basic information about a CIDR.

-- 
Davide Alberani <davide.alberani at gmail.com> [GPG KeyID: 0x465BFD47]
http://erlug.linux.it/~da/

From rdmurray at bitdance.com  Wed Jun  3 17:16:01 2009
From: rdmurray at bitdance.com (R. David Murray)
Date: Wed, 3 Jun 2009 11:16:01 -0400 (EDT)
Subject: [Python-Dev] Issues with Py3.1's new ipaddr
In-Reply-To: <4A263787.5010203@pennington.net>
References: <4A263787.5010203@pennington.net>
Message-ID: <Pine.LNX.4.64.0906031103380.32513@kimball.webabinitio.net>

On Wed, 3 Jun 2009 at 03:42, Mike Pennington wrote:
> That said, I test drove ipaddr for about 30 minutes and so far like the 
> big-picture API design quite a bit.  I'll specifically address Clay's concern 
> about hosts vs networks, because this issue is important to me; I've been in 
> the network engineering field for over 15 years, worked on Cisco's product 
> development team, and held a CCIE (consider it the equivalent of a CPA for 
> network engineers) for 10 years...
>
> Clay seems to object to ipaddr's IP object because it is not the same as the 
> object model used in the BSD ip stack.  Indeed, I'm one of the raving fans of 
> what BSD has done for the quality of ip networking, but let's also consider 
> their requirements.  BSD must approach ip networking from a host perspective, 
> it is the consumer of individual IP packets and their payloads. ipaddr's 
> whole point of existence is really driven towards the manipulation of 
> potentially massive lists of ip addresses. This is no small difference in 
> requirements, and I believe ipaddr's different approach makes their code much 
> simpler for the tasks it needs to do.  Incorporating host addresses as a 
> special case of a /32 IPv4 network or /128 IPv6 network makes a lot of sense 
> to me, in fact, I also chose this same object model.  Perl's NetAddr::IP does 
> this too, it is considered the gold standard for perl's address manipulation 
> module.

I think this hits the nail on the head.  Rather than network engineers
having a less precise understanding of IP, what we have is two different
sets of domain requirements.  Network engineers deal with networks, with
IPs being a necessary special case.  Others deal with host addresses,
with networks as an additional data type.

Both approaches are valid, but lead to different design decisions.
I don't see any reason why both needs cannot be met by a common API,
but I'm wondering if any existing package is going to incorporate both
approaches satisfactorily.  As another poster said, each package that gets
written solves the problems that the particular author(s) needed solved.

Since it seems clear that ipaddr does not address the needs of the
"other half" of the user base, and that its API considered on its own
does have design flaws that would be difficult to fix in a backward
compatible fashion (eg: returning strings from __getitem__), I withdraw
my support for keeping it in 3.1.

--David

From janssen at parc.com  Wed Jun  3 17:31:07 2009
From: janssen at parc.com (Bill Janssen)
Date: Wed, 3 Jun 2009 08:31:07 PDT
Subject: [Python-Dev] Issues with Py3.1's new ipaddr
In-Reply-To: <Pine.LNX.4.64.0906031103380.32513@kimball.webabinitio.net>
References: <4A263787.5010203@pennington.net>
	<Pine.LNX.4.64.0906031103380.32513@kimball.webabinitio.net>
Message-ID: <31390.1244043067@parc.com>

R. David Murray <rdmurray at bitdance.com> wrote:

> Both approaches are valid, but lead to different design decisions.
> I don't see any reason why both needs cannot be met by a common API,
> but I'm wondering if any existing package is going to incorporate both
> approaches satisfactorily.  As another poster said, each package that gets
> written solves the problems that the particular author(s) needed solved.

I wonder if part of the problem is the name of the module.  Just from
"ipaddr", I'd expect it to deal with host addresses (what I think of as
an IP address) and would probably approach its use with the wrong
expectations.  I could see frustration and irritation following from
that.  If the module was called "networks" instead of "ipaddr", it might
help.

Bill

From nas at arctrix.com  Wed Jun  3 19:13:19 2009
From: nas at arctrix.com (Neil Schemenauer)
Date: Wed, 3 Jun 2009 17:13:19 +0000 (UTC)
Subject: [Python-Dev] Issues with Py3.1's new ipaddr
References: <20090602175600.22176.619128085.divmod.quotient.1116@henry.divmod.com>
	<F6D076DA-5542-4532-8D99-D44B588E8F3E@object-craft.com.au>
	<ca471dc20906021939wa12e7b6y4e0f7f1a9349d9ce@mail.gmail.com>
	<15494D8B-AEF9-4204-9B85-19AD23325F95@python.org>
Message-ID: <h06avf$uej$1@ger.gmane.org>

Barry Warsaw <barry at python.org> wrote:
> It would be really nice if say the Cheeseshop had a voting feature.   
> Use PEP 10 voting to get a rough estimate of a module's popularity  
> (download counts alone might not tell you everything).  Then at least  
> you can get a rough idea of how generally popular a module is in the  
> wider community.  Also, a module should have to live on its own two  
> feet for while on Cheeseshop before being considered for inclusion in  
> the stdlib.

Better yet would be something like Debian's popularity-contest
mechanism (it provides opt-in voting of packages based on what is
installed on your machine).  popularity-contest runs from a cron
job.  Maybe when Python is installed it could ask if you want to
submit package statistics.  If so, installing a package with
distutils would submit the name and version number to a central
server.

If we knew which batteries were most popular we could make sure they
are included in the package. ;-)

  Neil


From guido at python.org  Wed Jun  3 19:19:25 2009
From: guido at python.org (Guido van Rossum)
Date: Wed, 3 Jun 2009 10:19:25 -0700
Subject: [Python-Dev] Issues with Py3.1's new ipaddr
In-Reply-To: <h06avf$uej$1@ger.gmane.org>
References: <20090602175600.22176.619128085.divmod.quotient.1116@henry.divmod.com>
	<F6D076DA-5542-4532-8D99-D44B588E8F3E@object-craft.com.au> 
	<ca471dc20906021939wa12e7b6y4e0f7f1a9349d9ce@mail.gmail.com> 
	<15494D8B-AEF9-4204-9B85-19AD23325F95@python.org>
	<h06avf$uej$1@ger.gmane.org>
Message-ID: <ca471dc20906031019q30e08bf3v958de93a590a2348@mail.gmail.com>

On Wed, Jun 3, 2009 at 10:13 AM, Neil Schemenauer <nas at arctrix.com> wrote:
> Barry Warsaw <barry at python.org> wrote:
>> It would be really nice if say the Cheeseshop had a voting feature.
>> Use PEP 10 voting to get a rough estimate of a module's popularity
>> (download counts alone might not tell you everything). ?Then at least
>> you can get a rough idea of how generally popular a module is in the
>> wider community. ?Also, a module should have to live on its own two
>> feet for while on Cheeseshop before being considered for inclusion in
>> the stdlib.
>
> Better yet would be something like Debian's popularity-contest
> mechanism (it provides opt-in voting of packages based on what is
> installed on your machine). ?popularity-contest runs from a cron
> job. ?Maybe when Python is installed it could ask if you want to
> submit package statistics. ?If so, installing a package with
> distutils would submit the name and version number to a central
> server.
>
> If we knew which batteries were most popular we could make sure they
> are included in the package. ;-)

Whoa. Are you all suddenly trying to turn Python into a democracy? I'm
outta here if that ever happens (even if I were voted BDFL by a
majority :-).

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From glyph at divmod.com  Wed Jun  3 19:41:51 2009
From: glyph at divmod.com (glyph at divmod.com)
Date: Wed, 03 Jun 2009 17:41:51 -0000
Subject: [Python-Dev] Issues with process and discussions (Re: Issues
	with	Py3.1's new ipaddr)
In-Reply-To: <ca471dc20906021939wa12e7b6y4e0f7f1a9349d9ce@mail.gmail.com>
References: <20090602175600.22176.619128085.divmod.quotient.1116@henry.divmod.com>
	<F6D076DA-5542-4532-8D99-D44B588E8F3E@object-craft.com.au>
	<ca471dc20906021939wa12e7b6y4e0f7f1a9349d9ce@mail.gmail.com>
Message-ID: <20090603174151.12555.1359697426.divmod.xquotient.12290@weber.divmod.com>


On 02:39 am, guido at python.org wrote:
>I'm disappointed in the process -- it's as if nobody really reviewed
>the API until it was released with rc1, and this despite there being a
>significant discussion about its inclusion and alternatives months
>ago. (Don't look at me -- I wouldn't recognize a netmask if it bit me
>in the behind, and I can honestly say that I don't know whether /8
>means to look only at the first 8 bits or whether it means to mask off
>the last 8 bits.)
>
>I hope we can learn from this.

As he pointed out to Martin, Jean-Paul voiced objections several months 
ago which are similar to the ones which are now being discussed.  To be 
fair, he didn't unambiguously say "... and therefore don't include this 
library"; he simply suggested that netaddr was superior in some ways and 
that perhaps some documentation could illuminate why ipaddr was better.

I've been frustrated with similar aspects of Python's development 
process in the past.  The biggest problem I can see is that it's too 
hard to follow the discussion, and catch oneself up on the discussion 
thus far.

It's also difficult to refer back to posts much earlier in the history 
of an email discussion, and that frequently needs to be done when 
someone jumps into the middle of a discussion.

The way Twisted dealt with this particular issue was to move *all* 
discussions relevant to a particular feature into the ticket for that 
feature.  If discussion starts up on the mailing list, within a few 
messages of it starting, someone on the dev team will pipe up and say 
"Hey, here's the ticket for this, could you add a link to this 
discussion and a summary".

Once on a ticket, the phraseology and typesetting used by our core team 
has reached a very precise consensus.  Like the feature?  "Merge this 
patch" or "Land this branch".  Don't like it?  "Thanks for your patch, 
but:", followed by a list of enumerated feedback.  The required 
reactions to such feedback are equally well understood.  Even if a 
comment isn't a full, formal code review, it still follows a similar 
style.

This system is possibly too simplistic for the more wide-ranging 
development of Python; although Twisted has its share of enthusiastic 
discussions, there is rarely the level of controversy I see here on 
python-dev, so I can't recommend it as such.  I can say that keeping 
ticket discussions on tickets is generally a good idea, though, 
especially in a system like roundup or trac where it's easy to say "see 
message 7" rather than repeating yourself.

It seems that there is a habit of occasionally using ASF-style 
+1/+0/-0/-1 markers, but it's inconsistently applied.  More importantly, 
nobody ever actually adds them up, so it's not entirely clear when they 
should be used.

To go back to JP's original comments though: what was the right thing 
for him to do, back in January, when he had these concerns?  Should he 
have said "I am therefore -1 on this inclusion"?  Should he have been 
discussing this on the mailing list rather than the tracker?  Should he 
have kept coming back to the ticket and answering every single message 
reinforcing his original conclusions?  I honestly don't think it's very 
clear what one is "officially" supposed to do.  Without a clear 
expectation that one should say "No" to features that are problematic, 
it seems gratuitously mean to do so; so, it's nicer to just say "here's 
what I found" with the hopes that someone will evaluate that feedback.

Unfortunately it seems like the winning strategy here is just to keep 
flogging a dead horse until it's a dead horse hamburger; reply and reply 
and reply until everybody gets sick of talking about it.  Repeat your 
original points in every post so that nobody will miss them.  I think 
everyone is ill-served by this discussion format.  Certainly when I 
voice my own objections or support for something, I'd like to just stop 
by, add a note for the committers to take into account when considering 
the issue, and then go away.

So, here are my recommendations:

  1. Use the tracker for discussing tickets, so that it's easy to refer 
back to a previous point in the discussion, and so that people working 
on those tickets can easily find your commentary.
  2. Use the mailing list for drawing attention to these discussions if 
they are of general interest, especially if the discussion is time- 
critical.  In this case, an announcement "You have six weeks to review 
ipaddr now until its inclusion is permanent, anyone interested please 
look at issue 3959."
  3. If you have an opinion, put your +1/+0/-0/-1 on a line by itself at 
the top of your message, so that it's easy for newcomers to the 
discussion to get a general feel.

Of course, this won't prevent all meandering discussions, or discussions 
that are too late to the party, but I do think it will help.

However, I think before everyone just starts doing this, even *more* 
important is my meta-suggestion: let's agree on how and when feedback is 
useful, and when it will be considered, so that it's not just a contest 
to overflow Guido's inbox.  The opinion of the committers who will be 
considering feedback is much more important than mine :).

From glyph at divmod.com  Wed Jun  3 19:45:07 2009
From: glyph at divmod.com (glyph at divmod.com)
Date: Wed, 03 Jun 2009 17:45:07 -0000
Subject: [Python-Dev] Issues with Py3.1's new ipaddr
In-Reply-To: <20090603024458.GA22308@panix.com>
References: <20090602175600.22176.619128085.divmod.quotient.1116@henry.divmod.com>
	<F6D076DA-5542-4532-8D99-D44B588E8F3E@object-craft.com.au>
	<ca471dc20906021939wa12e7b6y4e0f7f1a9349d9ce@mail.gmail.com>
	<20090603024458.GA22308@panix.com>
Message-ID: <20090603174507.12555.1277667142.divmod.xquotient.12295@weber.divmod.com>


On 02:44 am, aahz at pythoncraft.com wrote:
>On Tue, Jun 02, 2009, Guido van Rossum wrote:
>>
>>I hope we can learn from this.
>
>I'm not thrilled with adding more process just because we had a problem
>here, and the only obvious solution I see is to require a PEP every 
>time
>a module is added.  Based on what I've seen of this discussion so far, 
>I
>think that cure would at this time be worse than the disease.

I thought the solution that I just proposed was pretty obvious ;-).

But in all seriousness, even if an improvement looks nothing like what I 
just proposed, it seems like a simple failure of imagination to say that 
nothing could make this situation better.

From glyph at divmod.com  Wed Jun  3 19:51:35 2009
From: glyph at divmod.com (glyph at divmod.com)
Date: Wed, 03 Jun 2009 17:51:35 -0000
Subject: [Python-Dev] Issues with Py3.1's new ipaddr
In-Reply-To: <79990c6b0906030051l3aac84c6v5f68803096554d2d@mail.gmail.com>
References: <20090602175600.22176.619128085.divmod.quotient.1116@henry.divmod.com>
	<F6D076DA-5542-4532-8D99-D44B588E8F3E@object-craft.com.au>
	<ca471dc20906021939wa12e7b6y4e0f7f1a9349d9ce@mail.gmail.com>
	<20090603024458.GA22308@panix.com>
	<87ws7thoio.fsf@uwakimon.sk.tsukuba.ac.jp>
	<79990c6b0906030051l3aac84c6v5f68803096554d2d@mail.gmail.com>
Message-ID: <20090603175135.12555.1469649106.divmod.xquotient.12300@weber.divmod.com>

On 07:51 am, p.f.moore at gmail.com wrote:
>2009/6/3 Stephen J. Turnbull <stephen at xemacs.org>:

>>One thing I would recommend is that while inclusion is not a matter of
>>voting, people who are recognized as domain experts on Python-Dev
>>*should* try to add their "+1" or "-1" early. ?Especially if they
>>don't expect to have time to participate actively in discussion.

>2. Encouraging a clear +1/-1 from people, in addition to discussion on
>specific points, would clarify things. I believe Martin commented that
>he hadn't realised that one of the opposing comments was a strong
>enough objection to count as a -1.

+1 :-)

From p.f.moore at gmail.com  Wed Jun  3 19:42:48 2009
From: p.f.moore at gmail.com (Paul Moore)
Date: Wed, 3 Jun 2009 18:42:48 +0100
Subject: [Python-Dev] Issues with process and discussions (Re: Issues
	with Py3.1's new ipaddr)
In-Reply-To: <20090603174151.12555.1359697426.divmod.xquotient.12290@weber.divmod.com>
References: <20090602175600.22176.619128085.divmod.quotient.1116@henry.divmod.com>
	<F6D076DA-5542-4532-8D99-D44B588E8F3E@object-craft.com.au>
	<ca471dc20906021939wa12e7b6y4e0f7f1a9349d9ce@mail.gmail.com>
	<20090603174151.12555.1359697426.divmod.xquotient.12290@weber.divmod.com>
Message-ID: <79990c6b0906031042y190e05d6obbf675200b171666@mail.gmail.com>

2009/6/3  <glyph at divmod.com>:
> So, here are my recommendations:
>
> ?1. Use the tracker for discussing tickets, so that it's easy to refer back
> to a previous point in the discussion, and so that people working on those
> tickets can easily find your commentary.
> ?2. Use the mailing list for drawing attention to these discussions if they
> are of general interest, especially if the discussion is time- critical. ?In
> this case, an announcement "You have six weeks to review ipaddr now until
> its inclusion is permanent, anyone interested please look at issue 3959."
> ?3. If you have an opinion, put your +1/+0/-0/-1 on a line by itself at the
> top of your message, so that it's easy for newcomers to the discussion to
> get a general feel.

Mostly, I agree, but I definitely disagree, I'm afraid, on the use of
the tracker for discussions. To keep track of discussions on a ticket,
I have to personally keep a list of the tickets I'm interested in,
check back regularly to see if there's anything new, and keep a mental
note of where I've read up to so I know what's new. RSS would make
this simpler, certainly, but I'm not sure about how I'd use it (it's
not how I currently use RSS, so I'd have to mess round with my current
setup to make it appropriate).

Email is delivered to me by default - I get anything new in my
python-dev folder, and I can skip or read the discussion as I choose.
I don't have to take action just to monitor things. (In other words,
the default is for people to see the discussions, rather than the
other way around.

Paul.

From glyph at divmod.com  Wed Jun  3 20:04:53 2009
From: glyph at divmod.com (glyph at divmod.com)
Date: Wed, 03 Jun 2009 18:04:53 -0000
Subject: [Python-Dev] Issues with Py3.1's new ipaddr
In-Reply-To: <ca471dc20906031019q30e08bf3v958de93a590a2348@mail.gmail.com>
References: <20090602175600.22176.619128085.divmod.quotient.1116@henry.divmod.com>
	<F6D076DA-5542-4532-8D99-D44B588E8F3E@object-craft.com.au>
	<ca471dc20906021939wa12e7b6y4e0f7f1a9349d9ce@mail.gmail.com>
	<15494D8B-AEF9-4204-9B85-19AD23325F95@python.org>
	<h06avf$uej$1@ger.gmane.org>
	<ca471dc20906031019q30e08bf3v958de93a590a2348@mail.gmail.com>
Message-ID: <20090603180453.12555.597041055.divmod.xquotient.12306@weber.divmod.com>

On 05:19 pm, guido at python.org wrote:
>On Wed, Jun 3, 2009 at 10:13 AM, Neil Schemenauer <nas at arctrix.com> 
>wrote:
>>Barry Warsaw <barry at python.org> wrote:
>>>It would be really nice if say the Cheeseshop had a voting feature.
>>>Use PEP 10 voting to get a rough estimate of a module's popularity
>>>(download counts alone might not tell you everything). ?Then at least
>>>you can get a rough idea of how generally popular a module is in the
>>>wider community. ?Also, a module should have to live on its own two
>>>feet for while on Cheeseshop before being considered for inclusion in
>>>the stdlib.
>>
>>Better yet would be something like Debian's popularity-contest
>>mechanism (it provides opt-in voting of packages based on what is
>>installed on your machine). ?popularity-contest runs from a cron
>>job. ?Maybe when Python is installed it could ask if you want to
>>submit package statistics. ?If so, installing a package with
>>distutils would submit the name and version number to a central
>>server.
>>
>>If we knew which batteries were most popular we could make sure they
>>are included in the package. ;-)
>
>Whoa. Are you all suddenly trying to turn Python into a democracy? I'm
>outta here if that ever happens (even if I were voted BDFL by a
>majority :-).

I'm sure that what Barry and Neil are recommending is the same as what I 
did later: a way to give our most beloved regent accurate data on the 
populace's current mood.

Certainly we wouldn't be discussing our plans for a democratic coup out 
in the open like this!  Clearly, that would be foolish ;-).

From glyph at divmod.com  Wed Jun  3 20:12:36 2009
From: glyph at divmod.com (glyph at divmod.com)
Date: Wed, 03 Jun 2009 18:12:36 -0000
Subject: [Python-Dev] Issues with process and discussions (Re: Issues
	with	Py3.1's new ipaddr)
In-Reply-To: <79990c6b0906031042y190e05d6obbf675200b171666@mail.gmail.com>
References: <20090602175600.22176.619128085.divmod.quotient.1116@henry.divmod.com>
	<F6D076DA-5542-4532-8D99-D44B588E8F3E@object-craft.com.au>
	<ca471dc20906021939wa12e7b6y4e0f7f1a9349d9ce@mail.gmail.com>
	<20090603174151.12555.1359697426.divmod.xquotient.12290@weber.divmod.com>
	<79990c6b0906031042y190e05d6obbf675200b171666@mail.gmail.com>
Message-ID: <20090603181236.12555.1355794514.divmod.xquotient.12321@weber.divmod.com>


On 05:42 pm, p.f.moore at gmail.com wrote:
>2009/6/3  <glyph at divmod.com>:
>>So, here are my recommendations:
>>
>>?1. Use the tracker for discussing tickets, so that it's easy to refer 
>>back
>>to a previous point in the discussion, and so that people working on 
>>those
>>tickets can easily find your commentary.
>>?2. Use the mailing list for drawing attention to these discussions if 
>>they
>>are of general interest, especially if the discussion is time- 
>>critical. ?In
>>this case, an announcement "You have six weeks to review ipaddr now 
>>until
>>its inclusion is permanent, anyone interested please look at issue 
>>3959."
>>?3. If you have an opinion, put your +1/+0/-0/-1 on a line by itself 
>>at the
>>top of your message, so that it's easy for newcomers to the discussion 
>>to
>>get a general feel.
>
>Mostly, I agree, but I definitely disagree, I'm afraid, on the use of
>the tracker for discussions. To keep track of discussions on a ticket,
>I have to personally keep a list of the tickets I'm interested in,
>check back regularly to see if there's anything new, and keep a mental
>note of where I've read up to so I know what's new. RSS would make
>this simpler, certainly, but I'm not sure about how I'd use it (it's
>not how I currently use RSS, so I'd have to mess round with my current
>setup to make it appropriate).
>
>Email is delivered to me by default - I get anything new in my
>python-dev folder, and I can skip or read the discussion as I choose.
>I don't have to take action just to monitor things. (In other words,
>the default is for people to see the discussions, rather than the
>other way around.

A good point, but there are a couple of technical solutions to this 
problem, which, according to http://wiki.python.org/moin/TrackerDocs/, 
have already been implemented.

If you want to get email about new issues, subscribe to new-bugs- 
announce at mail.python.org.  If you want to know about every message on 
every issue, subscribe to python-bugs-list at mail.python.org.

But, frankly, I think it's a bad idea to subscribe to python-bugs-list 
for announcements.  The whole point here is that there is simply too 
much going on in python development for anyone to reasonably keep track 
of at a low level.  Guido himself has complained on numerous occasions 
of being too busy to monitor things closely.  A better model is to 
subscribe to new-bugs-announce and selectively pay attention to the bugs 
which are interesting to you; and, when a discussion you're involved in 
gets interesting and becomes of more general interest, raise it on 
python-dev.

(On the other hand, if you want to subscribe to get your own personal 
searchable archive, then by all means.)

From python at rcn.com  Wed Jun  3 19:39:26 2009
From: python at rcn.com (Raymond Hettinger)
Date: Wed, 3 Jun 2009 10:39:26 -0700
Subject: [Python-Dev] Issues with Py3.1's new ipaddr
References: <20090602175600.22176.619128085.divmod.quotient.1116@henry.divmod.com><F6D076DA-5542-4532-8D99-D44B588E8F3E@object-craft.com.au>
	<ca471dc20906021939wa12e7b6y4e0f7f1a9349d9ce@mail.gmail.com>
	<15494D8B-AEF9-4204-9B85-19AD23325F95@python.org><h06avf$uej$1@ger.gmane.org>
	<ca471dc20906031019q30e08bf3v958de93a590a2348@mail.gmail.com>
Message-ID: <AB5A2A7E8C2A4CC39864CB90EDC625FC@RaymondLaptop1>


[GvR]
> Whoa. Are you all suddenly trying to turn Python into a democracy? 

Arthur: I am your king! 
Woman: Well I didn't vote for you! 
Arthur: You don't vote for kings. 
Woman: Well how'd you become king then? 
[Angelic music plays...] 
Arthur: The Lady of the Lake, her arm clad in the purest shimmering silmite 
held aloft Excalibur from the bosom of the water, signifying by divine 
providence that I, Arthur, was to carry Excalibur. THAT is why I am your 
king! 
Dennis interrupting: Listen, strange women lyin' in ponds distributin' 
swords is no basis for a system of government! Supreme executive power 
derives from a mandate from the masses, not from some farcical aquatic 
ceremony! 
                           ----------------------- 
Dennis: Oh, but you can't expect to wield supreme executive power just 
because some watery tart threw a sword at you! 
                           ----------------------- 
Dennis: Oh but if I went 'round sayin' I was Emperor, just because some 
moistened bint lobbed a scimitar at me, they'd put me away! 
                           ----------------------- 
Dennis: Help! Help! I'm being oppressed! Violence inherent in the system! 
Violence inherent in the system! 

From clay at daemons.net  Wed Jun  3 20:25:28 2009
From: clay at daemons.net (Clay McClure)
Date: Wed, 3 Jun 2009 14:25:28 -0400
Subject: [Python-Dev] Issues with Py3.1's new ipaddr
In-Reply-To: <Pine.LNX.4.64.0906031103380.32513@kimball.webabinitio.net>
References: <4A263787.5010203@pennington.net>
	<Pine.LNX.4.64.0906031103380.32513@kimball.webabinitio.net>
Message-ID: <8657ee3f0906031125v31d2c10ev25b64675dea9fc13@mail.gmail.com>

On Wed, Jun 3, 2009 at 11:16 AM, R. David Murray <rdmurray at bitdance.com> wrote:

> I think this hits the nail on the head. ?Rather than network engineers
> having a less precise understanding of IP, what we have is two different
> sets of domain requirements. ?Network engineers deal with networks, with
> IPs being a necessary special case. ?Others deal with host addresses,
> with networks as an additional data type.

It has been brought to my attention that my use of the word
"imprecise" may have been construed as an insult. For that I
apologize. That was not my intent. I am sure that your understanding
of IP in your domain (network engineering) is as good or better than
mine in my domain (UNIX administration). That is not the point I was
arguing.

Had you said that thinking of addresses as having netmasks is a useful
mental model, I would have agreed with you wholeheartedly. To me, this
is similar to thinking of voltage as pressure. Instead, you said (or
at least implied) that addresses in fact do have netmasks, which I
think is technically an imprecise way of describing how the technology
works. To me, that would be like building a voltmeter calibrated in
pascals.

As another poster has commented, I think the name of the module is the
source of some confusion. While I see the validity of your use case,
that is not the use case I had in mind for a module named "ipaddr". In
any case, I think with some enhanced documentation and maybe some
class name changes, we can clarify ipaddr's API and strive to make it
support both use cases. Your continued input will be invaluable as we
move forward with the PEP.

Clay

From mcguire at google.com  Wed Jun  3 20:05:36 2009
From: mcguire at google.com (Jake McGuire)
Date: Wed, 3 Jun 2009 11:05:36 -0700
Subject: [Python-Dev] Issues with process and discussions (Re: Issues
	with Py3.1's new ipaddr)
In-Reply-To: <20090603174151.12555.1359697426.divmod.xquotient.12290@weber.divmod.com>
References: <20090602175600.22176.619128085.divmod.quotient.1116@henry.divmod.com>
	<F6D076DA-5542-4532-8D99-D44B588E8F3E@object-craft.com.au>
	<ca471dc20906021939wa12e7b6y4e0f7f1a9349d9ce@mail.gmail.com>
	<20090603174151.12555.1359697426.divmod.xquotient.12290@weber.divmod.com>
Message-ID: <77c780b40906031105o63b9131n3609d29a5bc7f189@mail.gmail.com>

On Wed, Jun 3, 2009 at 10:41 AM,  <glyph at divmod.com> wrote:
>
> On 02:39 am, guido at python.org wrote:
>>
>> I'm disappointed in the process -- it's as if nobody really reviewed
>> the API until it was released with rc1, and this despite there being a
>> significant discussion about its inclusion and alternatives months
>> ago. (Don't look at me -- I wouldn't recognize a netmask if it bit me
>> in the behind, and I can honestly say that I don't know whether /8
>> means to look only at the first 8 bits or whether it means to mask off
>> the last 8 bits.)
>>
>> I hope we can learn from this.
>
> As he pointed out to Martin, Jean-Paul voiced objections several months ago
> which are similar to the ones which are now being discussed. ?To be fair, he
> didn't unambiguously say "... and therefore don't include this library"; he
> simply suggested that netaddr was superior in some ways and that perhaps
> some documentation could illuminate why ipaddr was better.

The thing that stands out about the earlier tracker/mailing list
discussions is how very few people affirmatively wanted ipaddr added
to the standard library.  Most people thought it sounded ok in
principle, didn't care, or thought it was not a great idea but didn't
feel like arguing about it.

-jake

From nad at acm.org  Wed Jun  3 20:53:31 2009
From: nad at acm.org (Ned Deily)
Date: Wed, 03 Jun 2009 11:53:31 -0700
Subject: [Python-Dev] Issues with process and discussions (Re: Issues
	with	Py3.1's new ipaddr)
References: <20090602175600.22176.619128085.divmod.quotient.1116@henry.divmod.com>
	<F6D076DA-5542-4532-8D99-D44B588E8F3E@object-craft.com.au>
	<ca471dc20906021939wa12e7b6y4e0f7f1a9349d9ce@mail.gmail.com>
	<20090603174151.12555.1359697426.divmod.xquotient.12290@weber.divmod.com>
	<79990c6b0906031042y190e05d6obbf675200b171666@mail.gmail.com>
	<20090603181236.12555.1355794514.divmod.xquotient.12321@weber.divmod.com>
Message-ID: <nad-32ED37.11533003062009@ger.gmane.org>

In article 
<20090603181236.12555.1355794514.divmod.xquotient.12321 at weber.divmod.com
>glyph at divmod.com wrote:
> If you want to get email about new issues, subscribe to new-bugs- 
> announce at mail.python.org.  If you want to know about every message on 
> every issue, subscribe to python-bugs-list at mail.python.org.
> 
> But, frankly, I think it's a bad idea to subscribe to python-bugs-list 
> for announcements.  The whole point here is that there is simply too 
> much going on in python development for anyone to reasonably keep track 
> of at a low level.  Guido himself has complained on numerous occasions 
> of being too busy to monitor things closely.  A better model is to 
> subscribe to new-bugs-announce and selectively pay attention to the bugs 
> which are interesting to you; and, when a discussion you're involved in 
> gets interesting and becomes of more general interest, raise it on 
> python-dev.

Another option: if you are more comfortable with managing information 
flow via usenet newsgroups than via email lists, gmane.org provides 
mail-to-nntp gateways and archiving of most of the major python mailing 
lists, including this one, the bugs list, and code checkins.   It also 
supports posting via your news reader, rss feeds, searching, and several 
web interfaces.

http://dir.gmane.org/gmane.comp.python.bugs
http://dir.gmane.org/gmane.comp.python.cvs
http://dir.gmane.org/gmane.comp.python.devel
http://dir.gmane.org/index.php?prefix=gmane.comp.python

-- 
 Ned Deily,
 nad at acm.org


From fuzzyman at voidspace.org.uk  Wed Jun  3 20:56:44 2009
From: fuzzyman at voidspace.org.uk (Michael Foord)
Date: Wed, 03 Jun 2009 19:56:44 +0100
Subject: [Python-Dev] Issues with process and discussions (Re: Issues
 with Py3.1's new ipaddr)
In-Reply-To: <79990c6b0906031042y190e05d6obbf675200b171666@mail.gmail.com>
References: <20090602175600.22176.619128085.divmod.quotient.1116@henry.divmod.com>	<F6D076DA-5542-4532-8D99-D44B588E8F3E@object-craft.com.au>	<ca471dc20906021939wa12e7b6y4e0f7f1a9349d9ce@mail.gmail.com>	<20090603174151.12555.1359697426.divmod.xquotient.12290@weber.divmod.com>
	<79990c6b0906031042y190e05d6obbf675200b171666@mail.gmail.com>
Message-ID: <4A26C76C.9020506@voidspace.org.uk>

Paul Moore wrote:
> 2009/6/3  <glyph at divmod.com>:
>   
>> So, here are my recommendations:
>>
>>  1. Use the tracker for discussing tickets, so that it's easy to refer back
>> to a previous point in the discussion, and so that people working on those
>> tickets can easily find your commentary.
>>  2. Use the mailing list for drawing attention to these discussions if they
>> are of general interest, especially if the discussion is time- critical.  In
>> this case, an announcement "You have six weeks to review ipaddr now until
>> its inclusion is permanent, anyone interested please look at issue 3959."
>>  3. If you have an opinion, put your +1/+0/-0/-1 on a line by itself at the
>> top of your message, so that it's easy for newcomers to the discussion to
>> get a general feel.
>>     
>
> Mostly, I agree, but I definitely disagree, I'm afraid, on the use of
> the tracker for discussions. To keep track of discussions on a ticket,
> I have to personally keep a list of the tickets I'm interested in,
> check back regularly to see if there's anything new,

Not true - if you are added as nosy on a tracker item (which happens 
when you make a comment or you can do yourself) then you get emailed 
about new comments. The email contains the body of the comment so you 
can follow discussions completely by email only going to the tracker to 
add responses.

Michael

>  and keep a mental
> note of where I've read up to so I know what's new. RSS would make
> this simpler, certainly, but I'm not sure about how I'd use it (it's
> not how I currently use RSS, so I'd have to mess round with my current
> setup to make it appropriate).
>
> Email is delivered to me by default - I get anything new in my
> python-dev folder, and I can skip or read the discussion as I choose.
> I don't have to take action just to monitor things. (In other words,
> the default is for people to see the discussions, rather than the
> other way around.
>
> Paul.
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk
>   


-- 
http://www.ironpythoninaction.com/
http://www.voidspace.org.uk/blog



From martin at v.loewis.de  Wed Jun  3 21:08:07 2009
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Wed, 03 Jun 2009 21:08:07 +0200
Subject: [Python-Dev] Issues with process and discussions (Re: Issues
 with	Py3.1's new ipaddr)
In-Reply-To: <20090603174151.12555.1359697426.divmod.xquotient.12290@weber.divmod.com>
References: <20090602175600.22176.619128085.divmod.quotient.1116@henry.divmod.com>	<F6D076DA-5542-4532-8D99-D44B588E8F3E@object-craft.com.au>	<ca471dc20906021939wa12e7b6y4e0f7f1a9349d9ce@mail.gmail.com>
	<20090603174151.12555.1359697426.divmod.xquotient.12290@weber.divmod.com>
Message-ID: <4A26CA17.901@v.loewis.de>

> To go back to JP's original comments though: what was the right thing
> for him to do, back in January, when he had these concerns?  Should he
> have said "I am therefore -1 on this inclusion"?  Should he have been
> discussing this on the mailing list rather than the tracker?  Should he
> have kept coming back to the ticket and answering every single message
> reinforcing his original conclusions?  I honestly don't think it's very
> clear what one is "officially" supposed to do.

To me, it's fairly clear: what the committer needs to get is guidance in
any action to take. In most cases, the set of possible actions comes
down to three:
a) reject-as-is
b) commit-as-is
c) commit-with-changes (specify changes to make)
[d) take no action at this point, until certain preconditions are met]

For d), it is common to request, to the submitter,
resubmit-with-changes, then the code needs to be reevaluated when the
submitter claims to have implemented the requested changes.

In the specific case, JP didn't propose an action to take, hence it
wasn't clear (to me) whom his comment was directed to; I understood
it as "the module has these minor flaws, they should be fixed at some
point", which means "commit, then change later". This is what happened.

Regards,
Martin

From martin at v.loewis.de  Wed Jun  3 21:12:49 2009
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Wed, 03 Jun 2009 21:12:49 +0200
Subject: [Python-Dev] Issues with process and discussions (Re: Issues
 with Py3.1's new ipaddr)
In-Reply-To: <77c780b40906031105o63b9131n3609d29a5bc7f189@mail.gmail.com>
References: <20090602175600.22176.619128085.divmod.quotient.1116@henry.divmod.com>	<F6D076DA-5542-4532-8D99-D44B588E8F3E@object-craft.com.au>	<ca471dc20906021939wa12e7b6y4e0f7f1a9349d9ce@mail.gmail.com>	<20090603174151.12555.1359697426.divmod.xquotient.12290@weber.divmod.com>
	<77c780b40906031105o63b9131n3609d29a5bc7f189@mail.gmail.com>
Message-ID: <4A26CB31.20804@v.loewis.de>

> The thing that stands out about the earlier tracker/mailing list
> discussions is how very few people affirmatively wanted ipaddr added
> to the standard library.  Most people thought it sounded ok in
> principle, didn't care, or thought it was not a great idea but didn't
> feel like arguing about it.

I specifically thought "manipulating IP addresses is a frequent task,
and probably everybody's requirements will be different. So starting
with this library is as good as starting with any other - we can add
use cases as we go". I was in favor of ipaddr because his author
offered to maintain it. I then didn't have the time to review it
myself, and was happy that others picked it up.

Regards,
Martin


From jnoller at gmail.com  Wed Jun  3 21:08:33 2009
From: jnoller at gmail.com (Jesse Noller)
Date: Wed, 3 Jun 2009 15:08:33 -0400
Subject: [Python-Dev] Issues with process and discussions (Re: Issues
	with Py3.1's new ipaddr)
In-Reply-To: <4A26C76C.9020506@voidspace.org.uk>
References: <20090602175600.22176.619128085.divmod.quotient.1116@henry.divmod.com>
	<F6D076DA-5542-4532-8D99-D44B588E8F3E@object-craft.com.au>
	<ca471dc20906021939wa12e7b6y4e0f7f1a9349d9ce@mail.gmail.com>
	<20090603174151.12555.1359697426.divmod.xquotient.12290@weber.divmod.com>
	<79990c6b0906031042y190e05d6obbf675200b171666@mail.gmail.com>
	<4A26C76C.9020506@voidspace.org.uk>
Message-ID: <4222a8490906031208j1ee7cea6p404eec07457c1497@mail.gmail.com>

On Wed, Jun 3, 2009 at 2:56 PM, Michael Foord <fuzzyman at voidspace.org.uk> wrote:
> Paul Moore wrote:
>>
>> 2009/6/3 ?<glyph at divmod.com>:
>>
>>>
>>> So, here are my recommendations:
>>>
>>> ?1. Use the tracker for discussing tickets, so that it's easy to refer
>>> back
>>> to a previous point in the discussion, and so that people working on
>>> those
>>> tickets can easily find your commentary.
>>> ?2. Use the mailing list for drawing attention to these discussions if
>>> they
>>> are of general interest, especially if the discussion is time- critical.
>>> ?In
>>> this case, an announcement "You have six weeks to review ipaddr now until
>>> its inclusion is permanent, anyone interested please look at issue 3959."
>>> ?3. If you have an opinion, put your +1/+0/-0/-1 on a line by itself at
>>> the
>>> top of your message, so that it's easy for newcomers to the discussion to
>>> get a general feel.
>>>
>>
>> Mostly, I agree, but I definitely disagree, I'm afraid, on the use of
>> the tracker for discussions. To keep track of discussions on a ticket,
>> I have to personally keep a list of the tickets I'm interested in,
>> check back regularly to see if there's anything new,
>
> Not true - if you are added as nosy on a tracker item (which happens when
> you make a comment or you can do yourself) then you get emailed about new
> comments. The email contains the body of the comment so you can follow
> discussions completely by email only going to the tracker to add responses.
>
> Michael
>

You can also directly reply to tracker issues via email, which is how
I do most of my responses. It's cool.

From martin at v.loewis.de  Wed Jun  3 21:15:18 2009
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Wed, 03 Jun 2009 21:15:18 +0200
Subject: [Python-Dev] Issues with process and discussions (Re: Issues
 with Py3.1's new ipaddr)
In-Reply-To: <4A26C76C.9020506@voidspace.org.uk>
References: <20090602175600.22176.619128085.divmod.quotient.1116@henry.divmod.com>	<F6D076DA-5542-4532-8D99-D44B588E8F3E@object-craft.com.au>	<ca471dc20906021939wa12e7b6y4e0f7f1a9349d9ce@mail.gmail.com>	<20090603174151.12555.1359697426.divmod.xquotient.12290@weber.divmod.com>	<79990c6b0906031042y190e05d6obbf675200b171666@mail.gmail.com>
	<4A26C76C.9020506@voidspace.org.uk>
Message-ID: <4A26CBC6.6000207@v.loewis.de>

> Not true - if you are added as nosy on a tracker item (which happens
> when you make a comment or you can do yourself) then you get emailed
> about new comments. The email contains the body of the comment so you
> can follow discussions completely by email only going to the tracker to
> add responses.

Actually, one of roundup's big advantages over many competitors is that
you can also respond by email; many contributors actually do that.

You only have to remember that this isn't really email, so you can
usually omit salute and signature.

Regards,
Martin

From tjreedy at udel.edu  Wed Jun  3 22:37:19 2009
From: tjreedy at udel.edu (Terry Reedy)
Date: Wed, 03 Jun 2009 16:37:19 -0400
Subject: [Python-Dev] Issues with process and discussions (Re: Issues
 with Py3.1's new ipaddr)
In-Reply-To: <20090603174151.12555.1359697426.divmod.xquotient.12290@weber.divmod.com>
References: <20090602175600.22176.619128085.divmod.quotient.1116@henry.divmod.com>	<F6D076DA-5542-4532-8D99-D44B588E8F3E@object-craft.com.au>	<ca471dc20906021939wa12e7b6y4e0f7f1a9349d9ce@mail.gmail.com>
	<20090603174151.12555.1359697426.divmod.xquotient.12290@weber.divmod.com>
Message-ID: <h06mtt$dci$1@ger.gmane.org>

glyph at divmod.com wrote:

> So, here are my recommendations:
> 
>  1. Use the tracker for discussing tickets, so that it's easy to refer 
> back to a previous point in the discussion, and so that people working 
> on those tickets can easily find your commentary.
>  2. Use the mailing list for drawing attention to these discussions if 
> they are of general interest, especially if the discussion is time- 
> critical.  In this case, an announcement "You have six weeks to review 
> ipaddr now until its inclusion is permanent, anyone interested please 
> look at issue 3959."
>  3. If you have an opinion, put your +1/+0/-0/-1 on a line by itself at 
> the top of your message, so that it's easy for newcomers to the 
> discussion to get a general feel.

I watched and was greatly impressed by the video demo of Google's new 
Wave collaborative communication system.  I believe it would/will help 
with some of the chronic problems we (and others) have.  Someone already 
added a click-on Yes/No/Maybe client-side widget (for group outings) 
that tabulates on the wave who made each vote, with votes changeable.


From solipsis at pitrou.net  Wed Jun  3 22:40:55 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Wed, 3 Jun 2009 20:40:55 +0000 (UTC)
Subject: [Python-Dev] Issues with process and discussions (Re: Issues
	with Py3.1's new ipaddr)
References: <20090602175600.22176.619128085.divmod.quotient.1116@henry.divmod.com>	<F6D076DA-5542-4532-8D99-D44B588E8F3E@object-craft.com.au>	<ca471dc20906021939wa12e7b6y4e0f7f1a9349d9ce@mail.gmail.com>
	<20090603174151.12555.1359697426.divmod.xquotient.12290@weber.divmod.com>
	<h06mtt$dci$1@ger.gmane.org>
Message-ID: <loom.20090603T203956-109@post.gmane.org>

Terry Reedy <tjreedy <at> udel.edu> writes:
> 
> I watched and was greatly impressed by the video demo of Google's new 
> Wave collaborative communication system.  I believe it would/will help 
> with some of the chronic problems we (and others) have.

I really don't think technical systems are an answer to social issues. It's a
flaw of the engineering mindset.
(even when there's "Google" engraved on it ;-))

cheers

Antoine.



From g.brandl at gmx.net  Wed Jun  3 23:08:20 2009
From: g.brandl at gmx.net (Georg Brandl)
Date: Wed, 03 Jun 2009 23:08:20 +0200
Subject: [Python-Dev] Issues with Py3.1's new ipaddr
In-Reply-To: <1afaf6160906021953l3157a361oa0956514ac9f0ae2@mail.gmail.com>
References: <20090602175600.22176.619128085.divmod.quotient.1116@henry.divmod.com>	<F6D076DA-5542-4532-8D99-D44B588E8F3E@object-craft.com.au>	<ca471dc20906021939wa12e7b6y4e0f7f1a9349d9ce@mail.gmail.com>
	<1afaf6160906021953l3157a361oa0956514ac9f0ae2@mail.gmail.com>
Message-ID: <h06onr$jpd$1@ger.gmane.org>

Benjamin Peterson schrieb:
> 2009/6/2 Guido van Rossum <guido at python.org>:
>> Benjamin, what would be involved in removing it? I suppose there's the
>> module itself, some unit tests, and some docs. (I'm not asking you to
>> remove it yet -- but I'm asking to look into the consequences, so that
>> we can be sure to do the right thing before releasing 3.1 final.)
> 
> As Raymond and Gregory have pointed out in this thread, the library is
> quite independent as it stands now in the stlib, so should be trivial
> to remove. Nothing else should be affected.

Don't forget the what's new :)

Georg


From python at rcn.com  Wed Jun  3 23:58:29 2009
From: python at rcn.com (Raymond Hettinger)
Date: Wed, 3 Jun 2009 14:58:29 -0700
Subject: [Python-Dev] Issues with Py3.1's new ipaddr
References: <20090602175600.22176.619128085.divmod.quotient.1116@henry.divmod.com>	<F6D076DA-5542-4532-8D99-D44B588E8F3E@object-craft.com.au>	<ca471dc20906021939wa12e7b6y4e0f7f1a9349d9ce@mail.gmail.com><1afaf6160906021953l3157a361oa0956514ac9f0ae2@mail.gmail.com>
	<h06onr$jpd$1@ger.gmane.org>
Message-ID: <7796C6C0FA624BBE83F5DDBB31D1E3A0@RaymondLaptop1>

[GvR]
>>> Benjamin, what would be involved in removing it? I suppose there's the
>>> module itself, some unit tests, and some docs. (I'm not asking you to
>>> remove it yet -- but I'm asking to look into the consequences, so that
>>> we can be sure to do the right thing before releasing 3.1 final.)

[Benjamin Peterson]
>> As Raymond and Gregory have pointed out in this thread, the library is
>> quite independent as it stands now in the stlib, so should be trivial
>> to remove. Nothing else should be affected.

Guido, have you made a firm decision to remove ipaddr.py from 3.1?
The guys on IRC are chomping at the bit.


Raymond

From guido at python.org  Thu Jun  4 01:31:05 2009
From: guido at python.org (Guido van Rossum)
Date: Wed, 3 Jun 2009 16:31:05 -0700
Subject: [Python-Dev] Issues with Py3.1's new ipaddr
In-Reply-To: <7796C6C0FA624BBE83F5DDBB31D1E3A0@RaymondLaptop1>
References: <20090602175600.22176.619128085.divmod.quotient.1116@henry.divmod.com>
	<F6D076DA-5542-4532-8D99-D44B588E8F3E@object-craft.com.au> 
	<ca471dc20906021939wa12e7b6y4e0f7f1a9349d9ce@mail.gmail.com> 
	<1afaf6160906021953l3157a361oa0956514ac9f0ae2@mail.gmail.com> 
	<h06onr$jpd$1@ger.gmane.org>
	<7796C6C0FA624BBE83F5DDBB31D1E3A0@RaymondLaptop1>
Message-ID: <ca471dc20906031631m79076537j5feba4516a44c35c@mail.gmail.com>

On Wed, Jun 3, 2009 at 2:58 PM, Raymond Hettinger <python at rcn.com> wrote:
> [GvR]
>>>>
>>>> Benjamin, what would be involved in removing it? I suppose there's the
>>>> module itself, some unit tests, and some docs. (I'm not asking you to
>>>> remove it yet -- but I'm asking to look into the consequences, so that
>>>> we can be sure to do the right thing before releasing 3.1 final.)
>
> [Benjamin Peterson]
>>>
>>> As Raymond and Gregory have pointed out in this thread, the library is
>>> quite independent as it stands now in the stlib, so should be trivial
>>> to remove. Nothing else should be affected.
>
> Guido, have you made a firm decision to remove ipaddr.py from 3.1?
> The guys on IRC are chomping at the bit.

I don't make firm decisions any more (this is something that comes
with old age).

Let them remove it. If it means Python will never have such
functionality, because the users can't agree on the correct semantic
model, so be it. It's pretty advanced stuff, some people are happy to
write their own based on the RFCs, others will gladly download one of
the existing contenders (ipaddr included).

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From ben+python at benfinney.id.au  Thu Jun  4 01:41:08 2009
From: ben+python at benfinney.id.au (Ben Finney)
Date: Thu, 04 Jun 2009 09:41:08 +1000
Subject: [Python-Dev] Google Wave as a developer communication tool (was:
	Issues with process and discussions (Re: Issues with Py3.1's
	new ipaddr))
References: <20090602175600.22176.619128085.divmod.quotient.1116@henry.divmod.com>
	<F6D076DA-5542-4532-8D99-D44B588E8F3E@object-craft.com.au>
	<ca471dc20906021939wa12e7b6y4e0f7f1a9349d9ce@mail.gmail.com>
	<20090603174151.12555.1359697426.divmod.xquotient.12290@weber.divmod.com>
	<h06mtt$dci$1@ger.gmane.org>
Message-ID: <87vdncq417.fsf_-_@benfinney.id.au>

Terry Reedy <tjreedy at udel.edu> writes:

> I watched and was greatly impressed by the video demo of Google's new
> Wave collaborative communication system. I believe it would/will help
> with some of the chronic problems we (and others) have.

I watched that too. It appears to be heavily reliant on *very* fast
internet access for participants in a wave. That's far from universal in
the Python community, let alone the internet at large.

It also appears to be heavily reliant on the wave's existence at a
single point of failure (the hosting server): if that one point becomes
unreliable, all participants are hosed.

Neither of these problems exist with email (or NNTP).

-- 
 \       ?Well, my brother says Hello. So, hooray for speech therapy.? |
  `\                                                      ?Emo Philips |
_o__)                                                                  |
Ben Finney


From janssen at parc.com  Thu Jun  4 02:17:38 2009
From: janssen at parc.com (Bill Janssen)
Date: Wed, 3 Jun 2009 17:17:38 PDT
Subject: [Python-Dev] Google Wave as a developer communication tool
	(was: Issues with process and discussions (Re: Issues with
	Py3.1's new ipaddr))
In-Reply-To: <87vdncq417.fsf_-_@benfinney.id.au>
References: <20090602175600.22176.619128085.divmod.quotient.1116@henry.divmod.com>
	<F6D076DA-5542-4532-8D99-D44B588E8F3E@object-craft.com.au>
	<ca471dc20906021939wa12e7b6y4e0f7f1a9349d9ce@mail.gmail.com>
	<20090603174151.12555.1359697426.divmod.xquotient.12290@weber.divmod.com>
	<h06mtt$dci$1@ger.gmane.org> <87vdncq417.fsf_-_@benfinney.id.au>
Message-ID: <44142.1244074658@parc.com>

Ben Finney <ben+python at benfinney.id.au> wrote:

> Terry Reedy <tjreedy at udel.edu> writes:
> 
> > I watched and was greatly impressed by the video demo of Google's new
> > Wave collaborative communication system. I believe it would/will help
> > with some of the chronic problems we (and others) have.
> 
> I watched that too. It appears to be heavily reliant on *very* fast
> internet access for participants in a wave. That's far from universal in
> the Python community, let alone the internet at large.

And on HTML 5.  Those of us using IE might be hosed, but perhaps Python
developers all use Firefox?  (I use Camino.)

> It also appears to be heavily reliant on the wave's existence at a
> single point of failure (the hosting server): if that one point becomes
> unreliable, all participants are hosed.

Well, they've got this server synchronization protocol.  Might mitigate
both problems.  I'm thinking about building a Python Wave server to run
in UpLib, if I can find the time.

Bill

From greg.ewing at canterbury.ac.nz  Thu Jun  4 02:23:19 2009
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Thu, 04 Jun 2009 12:23:19 +1200
Subject: [Python-Dev] Issues with process and discussions (Re: Issues
 with Py3.1's new ipaddr)
In-Reply-To: <4A26C76C.9020506@voidspace.org.uk>
References: <20090602175600.22176.619128085.divmod.quotient.1116@henry.divmod.com>
	<F6D076DA-5542-4532-8D99-D44B588E8F3E@object-craft.com.au>
	<ca471dc20906021939wa12e7b6y4e0f7f1a9349d9ce@mail.gmail.com>
	<20090603174151.12555.1359697426.divmod.xquotient.12290@weber.divmod.com>
	<79990c6b0906031042y190e05d6obbf675200b171666@mail.gmail.com>
	<4A26C76C.9020506@voidspace.org.uk>
Message-ID: <4A2713F7.3090904@canterbury.ac.nz>

Michael Foord wrote:

> if you are added as nosy on a tracker item (which happens 
> when you make a comment or you can do yourself) then you get emailed 
> about new comments.

That's good, but...

> only going to the tracker to add responses.

is not so good. If the goal is to ensure that all previous
discussion on a given issue is reliably recorded, then
email replies ought to be archived under the relevant
ticket automatically.

For me at least, having to go somewhere special to
post a reply would be too error-prone. I get a large
number of Python-related messages in my inbox from
a number of sources, and I don't usually pay much
attention to exactly where they're coming from. I just
hit reply-all and trust that it will go somewhere
sensible.

-- 
Greg

From stephen at xemacs.org  Thu Jun  4 02:46:46 2009
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Thu, 04 Jun 2009 09:46:46 +0900
Subject: [Python-Dev] Issues with process and discussions (Re:
	Issues	with Py3.1's new ipaddr)
In-Reply-To: <loom.20090603T203956-109@post.gmane.org>
References: <20090602175600.22176.619128085.divmod.quotient.1116@henry.divmod.com>
	<F6D076DA-5542-4532-8D99-D44B588E8F3E@object-craft.com.au>
	<ca471dc20906021939wa12e7b6y4e0f7f1a9349d9ce@mail.gmail.com>
	<20090603174151.12555.1359697426.divmod.xquotient.12290@weber.divmod.com>
	<h06mtt$dci$1@ger.gmane.org>
	<loom.20090603T203956-109@post.gmane.org>
Message-ID: <87skighll5.fsf@uwakimon.sk.tsukuba.ac.jp>

Antoine Pitrou writes:
 > Terry Reedy <tjreedy <at> udel.edu> writes:
 > > 
 > > I watched and was greatly impressed by the video demo of Google's new 
 > > Wave collaborative communication system.  I believe it would/will help 
 > > with some of the chronic problems we (and others) have.
 > 
 > I really don't think technical systems are an answer to social
 > issues. It's a flaw of the engineering mindset.

That depends on the definition of "problem".  For example, if the
problem is "half our people detest web interfaces and want to discuss
issues by email", then "Roundup: nosy list" *is* a technical system
that is an answer.

I agree that Terry wasn't particularly specific about what impressed
him and how it would help for what problem.  But rather than just say
"technology is not a universal answer", we should ask "what problem
does this address?"  (Personally, I'm satisfied from the example Terry
gave that he had the "summarizing the opinions of those whose opinions
we respect in this domain" problem in mind, and I think there *are*
technical solutions to that.  Terry?)

From rdmurray at bitdance.com  Thu Jun  4 03:02:09 2009
From: rdmurray at bitdance.com (R. David Murray)
Date: Wed, 3 Jun 2009 21:02:09 -0400 (EDT)
Subject: [Python-Dev] Issues with process and discussions (Re: Issues
 with Py3.1's new ipaddr)
In-Reply-To: <4A2713F7.3090904@canterbury.ac.nz>
References: <20090602175600.22176.619128085.divmod.quotient.1116@henry.divmod.com>
	<F6D076DA-5542-4532-8D99-D44B588E8F3E@object-craft.com.au>
	<ca471dc20906021939wa12e7b6y4e0f7f1a9349d9ce@mail.gmail.com>
	<20090603174151.12555.1359697426.divmod.xquotient.12290@weber.divmod.com>
	<79990c6b0906031042y190e05d6obbf675200b171666@mail.gmail.com>
	<4A26C76C.9020506@voidspace.org.uk> <4A2713F7.3090904@canterbury.ac.nz>
Message-ID: <Pine.LNX.4.64.0906032059140.32513@kimball.webabinitio.net>

On Thu, 4 Jun 2009 at 12:23, Greg Ewing wrote:
> Michael Foord wrote:
>
>>  if you are added as nosy on a tracker item (which happens when you make a
>>  comment or you can do yourself) then you get emailed about new comments.
>
> That's good, but...
>
>>  only going to the tracker to add responses.
>
> is not so good. If the goal is to ensure that all previous
> discussion on a given issue is reliably recorded, then
> email replies ought to be archived under the relevant
> ticket automatically.

As Martin sad, replies work.  Michael was mistaken in thinking you have
to go to the tracker to post a followup (I initially did not know I
could just hit reply either).

You can even open new tickets via email, but I've never tried that.

--David

From glyph at divmod.com  Thu Jun  4 04:48:02 2009
From: glyph at divmod.com (glyph at divmod.com)
Date: Thu, 04 Jun 2009 02:48:02 -0000
Subject: [Python-Dev] Issues with process and discussions (Re: Issues
	with	Py3.1's new ipaddr)
In-Reply-To: <4A26CA17.901@v.loewis.de>
References: <20090602175600.22176.619128085.divmod.quotient.1116@henry.divmod.com>
	<F6D076DA-5542-4532-8D99-D44B588E8F3E@object-craft.com.au>
	<ca471dc20906021939wa12e7b6y4e0f7f1a9349d9ce@mail.gmail.com>
	<20090603174151.12555.1359697426.divmod.xquotient.12290@weber.divmod.com>
	<4A26CA17.901@v.loewis.de>
Message-ID: <20090604024802.12555.604416129.divmod.xquotient.12376@weber.divmod.com>


On 3 Jun, 07:08 pm, martin at v.loewis.de wrote:
>>To go back to JP's original comments though: what was the right thing
>>for him to do, back in January, when he had these concerns?

>To me, it's fairly clear: what the committer needs to get is guidance 
>in
>any action to take. In most cases, the set of possible actions comes
>down to three:
>a) reject-as-is
>b) commit-as-is
>c) commit-with-changes (specify changes to make)
>[d) take no action at this point, until certain preconditions are met]
>
>For d), it is common to request, to the submitter,
>resubmit-with-changes, then the code needs to be reevaluated when the
>submitter claims to have implemented the requested changes.

Is there a document which lists these things, and explains how it is 
desirable to communicate them?  I recently updated Twisted's equivalent 
document, adding minutae like which buttons to click on in our issue 
tracker, since that seems obvious to me but apparently wasn't obvious to 
a lot of new contributors.
>In the specific case, JP didn't propose an action to take, hence it
>wasn't clear (to me) whom his comment was directed to; I understood
>it as "the module has these minor flaws, they should be fixed at some
>point", which means "commit, then change later". This is what happened.

My reading of it suggests that he was saying "netaddr appears to be 
superior in every way, so python should include that instead.  But, if 
someone is insisting on ipaddr here are the things that could change 
about it".  The important thing here is that interpretation of the 
comment is required, so I can definitely see how you saw it the way you 
did.  There is no "-1" in his comment, and there's no documentation 
(that I'm aware of) which says that a "-1" is required, or how it will 
be used or interpreted.

From tjreedy at udel.edu  Thu Jun  4 07:43:32 2009
From: tjreedy at udel.edu (Terry Reedy)
Date: Thu, 04 Jun 2009 01:43:32 -0400
Subject: [Python-Dev] Google Wave as a developer communication tool
In-Reply-To: <87vdncq417.fsf_-_@benfinney.id.au>
References: <20090602175600.22176.619128085.divmod.quotient.1116@henry.divmod.com>	<F6D076DA-5542-4532-8D99-D44B588E8F3E@object-craft.com.au>	<ca471dc20906021939wa12e7b6y4e0f7f1a9349d9ce@mail.gmail.com>	<20090603174151.12555.1359697426.divmod.xquotient.12290@weber.divmod.com>	<h06mtt$dci$1@ger.gmane.org>
	<87vdncq417.fsf_-_@benfinney.id.au>
Message-ID: <h07mu5$qct$1@ger.gmane.org>

Ben Finney wrote:
> Terry Reedy <tjreedy at udel.edu> writes:
> 
>> I watched and was greatly impressed by the video demo of Google's new
>> Wave collaborative communication system. I believe it would/will help
>> with some of the chronic problems we (and others) have.

Example: if PEPs were waves, then responses could either be entered as 
live edits (with permission) or comments immediately following the 
relevant text (as with email/newsgroups) visible to all.  Much easier 
than current situation.  Edits are marked in red shading for those who 
have previously seen document.  Many have complained that it is hard to 
read multiple versions of a PEP since so much is new and there is no 
indication of what is new to the particular reader.

> I watched that too. It appears to be heavily reliant on *very* fast
> internet access for participants in a wave. That's far from universal in
> the Python community, let alone the internet at large.

Even a slow connection would make participation in PEPs better than today.

And being able to get a version of the docs that hi-lites "what's new 
for you" would also be very nice.

> It also appears to be heavily reliant on the wave's existence at a
> single point of failure (the hosting server): if that one point becomes
> unreliable, all participants are hosed.

We have that problem already with the tracker, which does occasionally 
go down for a bit.  And the svn host? (One reason to move to distributed 
system.)

> Neither of these problems exist with email (or NNTP).

But do for an email list, like this one.  Or a wiki.

Terry Jan Reedy


From tjreedy at udel.edu  Thu Jun  4 09:12:37 2009
From: tjreedy at udel.edu (Terry Reedy)
Date: Thu, 04 Jun 2009 03:12:37 -0400
Subject: [Python-Dev] Issues with process and discussions (Re: Issues
 with Py3.1's new ipaddr)
In-Reply-To: <87skighll5.fsf@uwakimon.sk.tsukuba.ac.jp>
References: <20090602175600.22176.619128085.divmod.quotient.1116@henry.divmod.com>	<F6D076DA-5542-4532-8D99-D44B588E8F3E@object-craft.com.au>	<ca471dc20906021939wa12e7b6y4e0f7f1a9349d9ce@mail.gmail.com>	<20090603174151.12555.1359697426.divmod.xquotient.12290@weber.divmod.com>	<h06mtt$dci$1@ger.gmane.org>	<loom.20090603T203956-109@post.gmane.org>
	<87skighll5.fsf@uwakimon.sk.tsukuba.ac.jp>
Message-ID: <h07s56$715$1@ger.gmane.org>

Stephen J. Turnbull wrote:
> Antoine Pitrou writes:
>  > Terry Reedy <tjreedy <at> udel.edu> writes:
>  > > 
>  > > I watched and was greatly impressed by the video demo of Google's new 
>  > > Wave collaborative communication system.  I believe it would/will help 
>  > > with some of the chronic problems we (and others) have.
>  > 
>  > I really don't think technical systems are an answer to social
>  > issues. It's a flaw of the engineering mindset.

Well, its true that telephones did not cure people of dishonesty, but it 
did make communication-at-a-distance, truthful or otherwise, easier.

> That depends on the definition of "problem".  For example, if the
> problem is "half our people detest web interfaces and want to discuss
> issues by email", then "Roundup: nosy list" *is* a technical system
> that is an answer.

And for me, it is very helpful.

> I agree that Terry wasn't particularly specific about what impressed
> him and how it would help for what problem.

Right.  A quick three lines plus a few more.  There were several things 
that Wave brings together.  I will suggest specific experiments when it 
is actually available for use.

 > But rather than just say
> "technology is not a universal answer", we should ask "what problem
> does this address?" 

As I said in my response to Ben Finney, two problems addressed are

1. Reading a document that has been edited since you read it last.

The old method of in-place deltas -- typically strikeout for deletions 
and some other special marking for additions -- has stood that test of 
time. Many word processors do this, but Waves improve on them by 
individualizing the markings for the particular reader and then removing 
them once read.

2. Multiple people editing a document.

One solution is the "You're it" method, whether informal by passing a 
doc around either on paper or electronicly or formal by VCS checkout. 
Another is edit in isolation, merge, and resolve conflicts.  Waves allow 
real-time simultaneous editing and merging via micro-deltas, so 
conflicts are immediately apparent.  This would generally work better 
with text docs than with code, which needs to be frequently frozen to 
run, but code might even be workable for side-by-side pair or sprint 
programming.

 > (Personally, I'm satisfied from the example Terry
> gave that he had the "summarizing the opinions of those whose opinions
> we respect in this domain" problem in mind, and I think there *are*
> technical solutions to that.  Terry?)

The current 'summarize opinion' system is somewhat haphazard and 
informal, with no auto-tabulation.  It seems to have broken-down a bit 
for the ipaddr issue, at least until the last few days when the best 
solution was to remove and defer.  Better would have been either not add 
and defer or add with generally supported revision.  I speculate that if 
there had been a proto-pep wave (though not then possible) with a vote 
widget with buttons such as Accept as is, Probably accept after 
revision, Don't know, and Reject, we might have reached a better outcome 
sooner.

Terry Jan Reedy


From ben+python at benfinney.id.au  Thu Jun  4 10:21:14 2009
From: ben+python at benfinney.id.au (Ben Finney)
Date: Thu, 04 Jun 2009 18:21:14 +1000
Subject: [Python-Dev] Google Wave as a developer communication tool
References: <20090602175600.22176.619128085.divmod.quotient.1116@henry.divmod.com>
	<F6D076DA-5542-4532-8D99-D44B588E8F3E@object-craft.com.au>
	<ca471dc20906021939wa12e7b6y4e0f7f1a9349d9ce@mail.gmail.com>
	<20090603174151.12555.1359697426.divmod.xquotient.12290@weber.divmod.com>
	<h06mtt$dci$1@ger.gmane.org> <87vdncq417.fsf_-_@benfinney.id.au>
	<h07mu5$qct$1@ger.gmane.org>
Message-ID: <87tz2wo1dx.fsf@benfinney.id.au>

Terry Reedy <tjreedy at udel.edu> writes:

> Ben Finney wrote:
> > I watched [the Google Wave presentation] too. It appears to be
> > heavily reliant on *very* fast internet access for participants in a
> > wave. That's far from universal in the Python community, let alone
> > the internet at large.
> 
> Even a slow connection would make participation in PEPs better than
> today.

How can you know that? A slow link doesn't punish email or NNTP
communication the way an interactive web application does. Why would
Google Wave be any less punitive to low-bandwidth users than existing
live web applications?

I would not want to put money against Google technologists giving lower
priority to the needs of the majority of internet users without fast
connections.

> > It also appears to be heavily reliant on the wave's existence at a
> > single point of failure (the hosting server): if that one point
> > becomes unreliable, all participants are hosed.
> 
> We have that problem already with the tracker, which does occasionally
> go down for a bit. And the svn host? (One reason to move to
> distributed system.)

Right. These are all reasons for moving toward distributed systems;
Python has chosen to do so already with its VCS. Why would the choice of
a new communications technology not take this into consideration?

> > Neither of these problems exist with email (or NNTP).
> 
> But do for an email list, like this one.  Or a wiki.

No. None of mailing list, NNTP, or wiki are heavily punitive to
low-bandwidth links.

-- 
 \     ?Creativity can be a social contribution, but only in so far as |
  `\            society is free to use the results.? ?Richard Stallman |
_o__)                                                                  |
Ben Finney


From ben+python at benfinney.id.au  Thu Jun  4 10:50:27 2009
From: ben+python at benfinney.id.au (Ben Finney)
Date: Thu, 04 Jun 2009 18:50:27 +1000
Subject: [Python-Dev] Google Wave as a developer communication tool
References: <20090602175600.22176.619128085.divmod.quotient.1116@henry.divmod.com>
	<F6D076DA-5542-4532-8D99-D44B588E8F3E@object-craft.com.au>
	<ca471dc20906021939wa12e7b6y4e0f7f1a9349d9ce@mail.gmail.com>
	<20090603174151.12555.1359697426.divmod.xquotient.12290@weber.divmod.com>
	<h06mtt$dci$1@ger.gmane.org> <87vdncq417.fsf_-_@benfinney.id.au>
	<h07mu5$qct$1@ger.gmane.org> <87tz2wo1dx.fsf@benfinney.id.au>
Message-ID: <87ljo8o018.fsf@benfinney.id.au>

Ben Finney <ben+python at benfinney.id.au> writes:

> Terry Reedy <tjreedy at udel.edu> writes:
> > Even a slow connection would make participation in PEPs better than
> > today.
> 
> How can you know that? A slow link doesn't punish email or NNTP
> communication the way an interactive web application does.

Strike that; reverse it. Should be ?A slow link is not punished by email
or NNTP communication the way it is by an interactive web application?.

-- 
 \         ?I went over to the neighbor's and asked to borrow a cup of |
  `\       salt. ?What are you making?? ?A salt lick.?? ?Steven Wright |
_o__)                                                                  |
Ben Finney


From ncoghlan at gmail.com  Thu Jun  4 12:07:05 2009
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Thu, 04 Jun 2009 20:07:05 +1000
Subject: [Python-Dev] Issues with process and discussions (Re: Issues
 with Py3.1's new ipaddr)
In-Reply-To: <79990c6b0906031042y190e05d6obbf675200b171666@mail.gmail.com>
References: <20090602175600.22176.619128085.divmod.quotient.1116@henry.divmod.com>	<F6D076DA-5542-4532-8D99-D44B588E8F3E@object-craft.com.au>	<ca471dc20906021939wa12e7b6y4e0f7f1a9349d9ce@mail.gmail.com>	<20090603174151.12555.1359697426.divmod.xquotient.12290@weber.divmod.com>
	<79990c6b0906031042y190e05d6obbf675200b171666@mail.gmail.com>
Message-ID: <4A279CC9.7080507@gmail.com>

Paul Moore wrote:
> Mostly, I agree, but I definitely disagree, I'm afraid, on the use of
> the tracker for discussions. To keep track of discussions on a ticket,
> I have to personally keep a list of the tickets I'm interested in,
> check back regularly to see if there's anything new, and keep a mental
> note of where I've read up to so I know what's new.

Others have already noted that Roundup not only emails you with new
comments, but also lets you reply via email. That happens for anyone on
the nosy list for an issue (commenting on an issue automatically adds
you to the nosy list).

In addition, you can fairly easily create a saved query to show you all
the open tickets that you are on the nosy list for. (Although I created
and saved my query for that so long ago that I don't recall the exact
details on how to go about doing that).

Regards,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------

From p.f.moore at gmail.com  Thu Jun  4 12:20:56 2009
From: p.f.moore at gmail.com (Paul Moore)
Date: Thu, 4 Jun 2009 11:20:56 +0100
Subject: [Python-Dev] Issues with process and discussions (Re: Issues
	with Py3.1's new ipaddr)
In-Reply-To: <4A279CC9.7080507@gmail.com>
References: <20090602175600.22176.619128085.divmod.quotient.1116@henry.divmod.com>
	<F6D076DA-5542-4532-8D99-D44B588E8F3E@object-craft.com.au>
	<ca471dc20906021939wa12e7b6y4e0f7f1a9349d9ce@mail.gmail.com>
	<20090603174151.12555.1359697426.divmod.xquotient.12290@weber.divmod.com>
	<79990c6b0906031042y190e05d6obbf675200b171666@mail.gmail.com>
	<4A279CC9.7080507@gmail.com>
Message-ID: <79990c6b0906040320m22c79a56jb3d3286f330bcb68@mail.gmail.com>

2009/6/4 Nick Coghlan <ncoghlan at gmail.com>:
> Paul Moore wrote:
>> Mostly, I agree, but I definitely disagree, I'm afraid, on the use of
>> the tracker for discussions. To keep track of discussions on a ticket,
>> I have to personally keep a list of the tickets I'm interested in,
>> check back regularly to see if there's anything new, and keep a mental
>> note of where I've read up to so I know what's new.
>
> Others have already noted that Roundup not only emails you with new
> comments, but also lets you reply via email. That happens for anyone on
> the nosy list for an issue (commenting on an issue automatically adds
> you to the nosy list).
>
> In addition, you can fairly easily create a saved query to show you all
> the open tickets that you are on the nosy list for. (Although I created
> and saved my query for that so long ago that I don't recall the exact
> details on how to go about doing that).

Agreed, and thanks to all for the pointers to features I wasn't aware of.

I could still argue that there are downsides (need to take action to
set myself as nosy on an issue, possibly setting up a new mail filter,
housekeeping cruft, the fact that people don't quote in the same way
on tracker items) that make tracker discussions less attractive to me,
but it's very personal things.

Let's leave it as merely that I wouldn't have contributed to the
ipaddr discussion if it had been solely on the tracker (not even if
pointers to and reminders of the tracker item were posted to
python-dev). My input wasn't very valuable, so maybe that doesn't
matter much. Maybe the key point is that keeping discussion on the
tracker introduces a barrier to participation by lurkers, FWIW.

Paul.

From aahz at pythoncraft.com  Thu Jun  4 14:23:00 2009
From: aahz at pythoncraft.com (Aahz)
Date: Thu, 4 Jun 2009 05:23:00 -0700
Subject: [Python-Dev] Google Wave as a developer communication tool
In-Reply-To: <h07mu5$qct$1@ger.gmane.org>
References: <20090602175600.22176.619128085.divmod.quotient.1116@henry.divmod.com>
	<F6D076DA-5542-4532-8D99-D44B588E8F3E@object-craft.com.au>
	<ca471dc20906021939wa12e7b6y4e0f7f1a9349d9ce@mail.gmail.com>
	<20090603174151.12555.1359697426.divmod.xquotient.12290@weber.divmod.com>
	<h06mtt$dci$1@ger.gmane.org> <87vdncq417.fsf_-_@benfinney.id.au>
	<h07mu5$qct$1@ger.gmane.org>
Message-ID: <20090604122300.GA3768@panix.com>

On Thu, Jun 04, 2009, Terry Reedy wrote:
>
> Example: if PEPs were waves, then responses could either be entered as  
> live edits (with permission) or comments immediately following the  
> relevant text (as with email/newsgroups) visible to all.  Much easier  
> than current situation.  Edits are marked in red shading for those who  
> have previously seen document.  Many have complained that it is hard to  
> read multiple versions of a PEP since so much is new and there is no  
> indication of what is new to the particular reader.

It sounds like Wave requires a high-powered browser, similar to Google
Maps.  That makes me -1 because I want to continue using Lynx.
-- 
Aahz (aahz at pythoncraft.com)           <*>         http://www.pythoncraft.com/

"Given that C++ has pointers and typecasts, it's really hard to have a
serious conversation about type safety with a C++ programmer and keep a
straight face.  It's kind of like having a guy who juggles chainsaws
wearing body armor arguing with a guy who juggles rubber chickens wearing
a T-shirt about who's in more danger."  --Roy Smith, c.l.py, 2004.05.23

From jaraco at jaraco.com  Thu Jun  4 15:02:53 2009
From: jaraco at jaraco.com (Jason R. Coombs)
Date: Thu, 4 Jun 2009 09:02:53 -0400
Subject: [Python-Dev] Mercurial, linefeeds, and Visual Studio
Message-ID: <8B473FAE8A08C34C9F5666FD4B0A87B67E2DDA50B9@hornigold.jaraco.com>

I just wanted to share my experience with the mercurial checkout.  I cloned http://code.python.org/hg/branches/py3k to continue work on http://bugs.python.org/issue1578269 but I found that when I click on PC/VS8.0/pcbuild.sln, nothing happens.

This appears to be due to a bug/limitation in vslauncher in that it doesn't recognize LF as a line separator.  vslauncher is the default association for sln files and its purpose is to parse out the .sln file and launch it with the appropriate Visual Studio version based on the header.  What makes matters worse is if vslauncher fails to recognize the format, it does nothing, so it just appears as if the file fails to launch anything.

It seems that within the hg repository, everything has been converted to LF for line endings.  I suspect this is because HG provides no integrated support for line-ending conversions and because the hg to svn bridge is probably running on a Unix OS.

So converting the pcbuild.sln file to CRLF line endings resolved the problem and the file would launch normally.  Also, without conversion, it was possible to open the .sln file in Visual Studio explicitly.

I wanted to share this with the community in case anyone else runs into this issue.  Also, if there's a recommended procedure for addressing this issue (and others that might arise due to non-native line endings), I'd be interested to hear it.

Regards,
Jason

From phd at phd.pp.ru  Thu Jun  4 15:30:31 2009
From: phd at phd.pp.ru (Oleg Broytmann)
Date: Thu, 4 Jun 2009 17:30:31 +0400
Subject: [Python-Dev] Mercurial and linefeeds
In-Reply-To: <8B473FAE8A08C34C9F5666FD4B0A87B67E2DDA50B9@hornigold.jaraco.com>
References: <8B473FAE8A08C34C9F5666FD4B0A87B67E2DDA50B9@hornigold.jaraco.com>
Message-ID: <20090604133031.GH30429@phd.pp.ru>

On Thu, Jun 04, 2009 at 09:02:53AM -0400, Jason R. Coombs wrote:
> It seems that within the hg repository, everything has been converted to LF for line endings.  I suspect this is because HG provides no integrated support for line-ending conversions and because the hg to svn bridge is probably running on a Unix OS.

http://www.selenic.com/mercurial/wiki/FAQ#FAQ.2BAC8-TechnicalDetails.What_about_Windows_line_endings_vs._Unix_line_endings.3F

Oleg.
-- 
     Oleg Broytmann            http://phd.pp.ru/            phd at phd.pp.ru
           Programmers don't die, they just GOSUB without RETURN.

From dirkjan at ochtman.nl  Thu Jun  4 15:30:51 2009
From: dirkjan at ochtman.nl (Dirkjan Ochtman)
Date: Thu, 4 Jun 2009 15:30:51 +0200
Subject: [Python-Dev] Mercurial, linefeeds, and Visual Studio
In-Reply-To: <8B473FAE8A08C34C9F5666FD4B0A87B67E2DDA50B9@hornigold.jaraco.com>
References: <8B473FAE8A08C34C9F5666FD4B0A87B67E2DDA50B9@hornigold.jaraco.com>
Message-ID: <ea2499da0906040630v926e477g7cdd176ce70d8a56@mail.gmail.com>

On Thu, Jun 4, 2009 at 3:02 PM, Jason R. Coombs <jaraco at jaraco.com> wrote:
> I wanted to share this with the community in case anyone else runs into this issue. ?Also, if there's a recommended procedure for addressing this issue (and others that might arise due to non-native line endings), I'd be interested to hear it.

Mercurial comes with a win32text extension that helps resolve issues
like these. It allows you to specify in the .hg/hgrc that .sln files
should always be expanded to CRLF.

Cheers,

Dirkjan

From rdmurray at bitdance.com  Thu Jun  4 15:49:46 2009
From: rdmurray at bitdance.com (R. David Murray)
Date: Thu, 4 Jun 2009 09:49:46 -0400 (EDT)
Subject: [Python-Dev] Mercurial and linefeeds
In-Reply-To: <20090604133031.GH30429@phd.pp.ru>
References: <8B473FAE8A08C34C9F5666FD4B0A87B67E2DDA50B9@hornigold.jaraco.com>
	<20090604133031.GH30429@phd.pp.ru>
Message-ID: <Pine.LNX.4.64.0906040944430.32513@kimball.webabinitio.net>

On Thu, 4 Jun 2009 at 17:30, Oleg Broytmann wrote:
> On Thu, Jun 04, 2009 at 09:02:53AM -0400, Jason R. Coombs wrote:
>> It seems that within the hg repository, everything has been converted to LF for line endings.  I suspect this is because HG provides no integrated support for line-ending conversions and because the hg to svn bridge is probably running on a Unix OS.
>
> http://www.selenic.com/mercurial/wiki/FAQ#FAQ.2BAC8-TechnicalDetails.What_about_Windows_line_endings_vs._Unix_line_endings.3F

Linked from that is:

     http://www.selenic.com/mercurial/wiki/Win32TextExtension

which says:

     "The filter specification can however not be managed by the
     repository..."

(I'm not clear what is being filtered on, but it looks like it might
be the _content_ of the files.)

The above statement seems like a really odd design choice, with the
potential for causing considerable developer coordination headaches.

Any Mercurial boffins want to talk about how this works in practice?

--David

From dirkjan at ochtman.nl  Thu Jun  4 16:12:05 2009
From: dirkjan at ochtman.nl (Dirkjan Ochtman)
Date: Thu, 4 Jun 2009 16:12:05 +0200
Subject: [Python-Dev] Mercurial and linefeeds
In-Reply-To: <Pine.LNX.4.64.0906040944430.32513@kimball.webabinitio.net>
References: <8B473FAE8A08C34C9F5666FD4B0A87B67E2DDA50B9@hornigold.jaraco.com>
	<20090604133031.GH30429@phd.pp.ru>
	<Pine.LNX.4.64.0906040944430.32513@kimball.webabinitio.net>
Message-ID: <ea2499da0906040712h648db20bh4aafeae6f1f8d3bf@mail.gmail.com>

On Thu, Jun 4, 2009 at 3:49 PM, R. David Murray <rdmurray at bitdance.com> wrote:
> The above statement seems like a really odd design choice, with the
> potential for causing considerable developer coordination headaches.
>
> Any Mercurial boffins want to talk about how this works in practice?

I'm guessing that's just because it uses Mercurial's built-in
configuration file chain (.hg/hgrc, ~/.hgrc and /etc/mercurial/hgrc).
Those aren't transmitted with repositories because they can contain
potentially security-sensitive information (like triggers for
executing interesting hooks and information on what users to trust).

It would certainly be possible to write a modified win32text-like
extension that takes its cues from a versioned file (like the .hgtags
and .hgsigs files currently supported). I'm not sure why the original
author didn't go that way, though.

Cheers,

Dirkjan

From p.f.moore at gmail.com  Thu Jun  4 16:20:56 2009
From: p.f.moore at gmail.com (Paul Moore)
Date: Thu, 4 Jun 2009 15:20:56 +0100
Subject: [Python-Dev] Mercurial, linefeeds, and Visual Studio
In-Reply-To: <ea2499da0906040630v926e477g7cdd176ce70d8a56@mail.gmail.com>
References: <8B473FAE8A08C34C9F5666FD4B0A87B67E2DDA50B9@hornigold.jaraco.com>
	<ea2499da0906040630v926e477g7cdd176ce70d8a56@mail.gmail.com>
Message-ID: <79990c6b0906040720m1d31f2fuff863262b891928a@mail.gmail.com>

2009/6/4 Dirkjan Ochtman <dirkjan at ochtman.nl>:
> On Thu, Jun 4, 2009 at 3:02 PM, Jason R. Coombs <jaraco at jaraco.com> wrote:
>> I wanted to share this with the community in case anyone else runs into this issue. ?Also, if there's a recommended procedure for addressing this issue (and others that might arise due to non-native line endings), I'd be interested to hear it.
>
> Mercurial comes with a win32text extension that helps resolve issues
> like these. It allows you to specify in the .hg/hgrc that .sln files
> should always be expanded to CRLF.

Silly question, but given that Mercurial itself does *no* conversion,
and CRLF is the only valid form for that file, why shouldn't the file
be checked into the repository with CRLF endings, and that's the end
of it (apart from misconfigured win32text setups on the client)?

Ideally, the history should be imported with CRLF throughout, which is
a task for the migration process. If that's too hard because of
limitations in the available conversion tools, then maybe the
limitation has to be accepted that pre-conversion copies of the file
have incorrect line endings and a manual fiox is committed right after
the switchover.

Doesn't help the current Mercurial mirrors, of course, but at least
this report alerts us to a consideration for the final switchover.

Essentially, pcbuild.sln is a binary file, and should be treated as
such. Maybe it's an error in the Subversion setup that it's treated as
text at all...

Paul.

From phd at phd.pp.ru  Thu Jun  4 16:29:40 2009
From: phd at phd.pp.ru (Oleg Broytmann)
Date: Thu, 4 Jun 2009 18:29:40 +0400
Subject: [Python-Dev] Mercurial, linefeeds, and Visual Studio
In-Reply-To: <79990c6b0906040720m1d31f2fuff863262b891928a@mail.gmail.com>
References: <8B473FAE8A08C34C9F5666FD4B0A87B67E2DDA50B9@hornigold.jaraco.com>
	<ea2499da0906040630v926e477g7cdd176ce70d8a56@mail.gmail.com>
	<79990c6b0906040720m1d31f2fuff863262b891928a@mail.gmail.com>
Message-ID: <20090604142940.GA15804@phd.pp.ru>

On Thu, Jun 04, 2009 at 03:20:56PM +0100, Paul Moore wrote:
> Essentially, pcbuild.sln is a binary file, and should be treated as
> such. Maybe it's an error in the Subversion setup that it's treated as
> text at all...

   Subversion has a built-in notion of eol-conversion (don't know if it was
used for this particular file).

Oleg.
-- 
     Oleg Broytmann            http://phd.pp.ru/            phd at phd.pp.ru
           Programmers don't die, they just GOSUB without RETURN.

From alexander.belopolsky at gmail.com  Thu Jun  4 17:03:22 2009
From: alexander.belopolsky at gmail.com (Alexander Belopolsky)
Date: Thu, 4 Jun 2009 11:03:22 -0400
Subject: [Python-Dev] Issues with process and discussions (Re: Issues
	with Py3.1's new ipaddr)
In-Reply-To: <79990c6b0906040320m22c79a56jb3d3286f330bcb68@mail.gmail.com>
References: <20090602175600.22176.619128085.divmod.quotient.1116@henry.divmod.com>
	<F6D076DA-5542-4532-8D99-D44B588E8F3E@object-craft.com.au>
	<ca471dc20906021939wa12e7b6y4e0f7f1a9349d9ce@mail.gmail.com>
	<20090603174151.12555.1359697426.divmod.xquotient.12290@weber.divmod.com>
	<79990c6b0906031042y190e05d6obbf675200b171666@mail.gmail.com>
	<4A279CC9.7080507@gmail.com>
	<79990c6b0906040320m22c79a56jb3d3286f330bcb68@mail.gmail.com>
Message-ID: <d38f5330906040803n3c120897t346439a45ea4ea14@mail.gmail.com>

On Thu, Jun 4, 2009 at 6:20 AM, Paul Moore <p.f.moore at gmail.com> wrote:
...
> I could still argue that there are downsides (need to take action to
> set myself as nosy on an issue, possibly setting up a new mail filter,
> housekeeping cruft, the fact that people don't quote in the same way
> on tracker items) that make tracker discussions less attractive to me,
> but it's very personal things.

How hard would it be to set up a bot that will subscribe to python-dev
and archive messages that have [issueXXXX] in the subject under
appropriate roundup ticket?  This way if discussion started on
roundup, it would only take adding python-dev in CC to bring it to the
larger python-dev audience and adding issue number to the python-dev
thread subject will be all it takes to archive new messages on
roundup.  This would not solve a problem of linking threads that start
on python-dev before a ticket is opened, but with some proper
conventions this problem can be solved as well.

From dirkjan at ochtman.nl  Thu Jun  4 17:03:28 2009
From: dirkjan at ochtman.nl (Dirkjan Ochtman)
Date: Thu, 4 Jun 2009 17:03:28 +0200
Subject: [Python-Dev] Mercurial, linefeeds, and Visual Studio
In-Reply-To: <79990c6b0906040720m1d31f2fuff863262b891928a@mail.gmail.com>
References: <8B473FAE8A08C34C9F5666FD4B0A87B67E2DDA50B9@hornigold.jaraco.com>
	<ea2499da0906040630v926e477g7cdd176ce70d8a56@mail.gmail.com>
	<79990c6b0906040720m1d31f2fuff863262b891928a@mail.gmail.com>
Message-ID: <ea2499da0906040803t4d04267eua40e205ef897042e@mail.gmail.com>

On Thu, Jun 4, 2009 at 4:20 PM, Paul Moore <p.f.moore at gmail.com> wrote:
> Silly question, but given that Mercurial itself does *no* conversion,
> and CRLF is the only valid form for that file, why shouldn't the file
> be checked into the repository with CRLF endings, and that's the end
> of it (apart from misconfigured win32text setups on the client)?

That's certainly a valid setup, and I wasn't suggesting otherwise. I
have no experience with .sln files, and I haven't really used
win32text either; I was just commenting on the general facilities of
Mercurial wrt these kinds of issues.

> Ideally, the history should be imported with CRLF throughout, which is
> a task for the migration process. If that's too hard because of
> limitations in the available conversion tools, then maybe the
> limitation has to be accepted that pre-conversion copies of the file
> have incorrect line endings and a manual fiox is committed right after
> the switchover.

Importing it with CRLF throughout would certainly be possible. I'll
add a section to my draft PEP.

> Essentially, pcbuild.sln is a binary file, and should be treated as
> such. Maybe it's an error in the Subversion setup that it's treated as
> text at all...

Yes, it certainly seems that way.

Cheers,

Dirkjan

From alexander.belopolsky at gmail.com  Thu Jun  4 17:17:11 2009
From: alexander.belopolsky at gmail.com (Alexander Belopolsky)
Date: Thu, 4 Jun 2009 11:17:11 -0400
Subject: [Python-Dev] Issues with process and discussions (Re: Issues
	with Py3.1's new ipaddr)
In-Reply-To: <20090604150814.GB17981@idyll.org>
References: <20090602175600.22176.619128085.divmod.quotient.1116@henry.divmod.com>
	<F6D076DA-5542-4532-8D99-D44B588E8F3E@object-craft.com.au>
	<ca471dc20906021939wa12e7b6y4e0f7f1a9349d9ce@mail.gmail.com>
	<20090603174151.12555.1359697426.divmod.xquotient.12290@weber.divmod.com>
	<79990c6b0906031042y190e05d6obbf675200b171666@mail.gmail.com>
	<4A279CC9.7080507@gmail.com>
	<79990c6b0906040320m22c79a56jb3d3286f330bcb68@mail.gmail.com>
	<d38f5330906040803n3c120897t346439a45ea4ea14@mail.gmail.com>
	<20090604150814.GB17981@idyll.org>
Message-ID: <d38f5330906040817r7127c0a8g36b102b56428f5a4@mail.gmail.com>

Alternatively, one could create a python-dev profile on roundup and
encourage people to add python-dev to the nosy list when discussion
needs to be broaden. This way replies to all will be posted in both
places.  Once discussion gets too specialized, someone can remove
python-dev from the "nosies." No new code is required to set this up
as far as I can tell.

On Thu, Jun 4, 2009 at 11:08 AM, C. Titus Brown <ctb at msu.edu> wrote:
> On Thu, Jun 04, 2009 at 11:03:22AM -0400, Alexander Belopolsky wrote:
> -> On Thu, Jun 4, 2009 at 6:20 AM, Paul Moore <p.f.moore at gmail.com> wrote:
> -> ...
> -> > I could still argue that there are downsides (need to take action to
> -> > set myself as nosy on an issue, possibly setting up a new mail filter,
> -> > housekeeping cruft, the fact that people don't quote in the same way
> -> > on tracker items) that make tracker discussions less attractive to me,
> -> > but it's very personal things.
> ->
> -> How hard would it be to set up a bot that will subscribe to python-dev
> -> and archive messages that have [issueXXXX] in the subject under
> -> appropriate roundup ticket? ?This way if discussion started on
> -> roundup, it would only take adding python-dev in CC to bring it to the
> -> larger python-dev audience and adding issue number to the python-dev
> -> thread subject will be all it takes to archive new messages on
> -> roundup. ?This would not solve a problem of linking threads that start
> -> on python-dev before a ticket is opened, but with some proper
> -> conventions this problem can be solved as well.
>
> In my experience this kind automated stuff is too fragile to work
> reliably & specifically over time, which is what we would want -- "fire
> and forget".
>
> Is there a good python-dev archive search mechanism (other than to
> google "python-dev <term>" ;) out there? ?Wouldn't that help?
>
> cheers,
> --titus
> --
> C. Titus Brown, ctb at msu.edu
>

From ctb at msu.edu  Thu Jun  4 17:08:14 2009
From: ctb at msu.edu (C. Titus Brown)
Date: Thu, 4 Jun 2009 08:08:14 -0700
Subject: [Python-Dev] Issues with process and discussions (Re: Issues
	with Py3.1's new ipaddr)
In-Reply-To: <d38f5330906040803n3c120897t346439a45ea4ea14@mail.gmail.com>
References: <20090602175600.22176.619128085.divmod.quotient.1116@henry.divmod.com>
	<F6D076DA-5542-4532-8D99-D44B588E8F3E@object-craft.com.au>
	<ca471dc20906021939wa12e7b6y4e0f7f1a9349d9ce@mail.gmail.com>
	<20090603174151.12555.1359697426.divmod.xquotient.12290@weber.divmod.com>
	<79990c6b0906031042y190e05d6obbf675200b171666@mail.gmail.com>
	<4A279CC9.7080507@gmail.com>
	<79990c6b0906040320m22c79a56jb3d3286f330bcb68@mail.gmail.com>
	<d38f5330906040803n3c120897t346439a45ea4ea14@mail.gmail.com>
Message-ID: <20090604150814.GB17981@idyll.org>

On Thu, Jun 04, 2009 at 11:03:22AM -0400, Alexander Belopolsky wrote:
-> On Thu, Jun 4, 2009 at 6:20 AM, Paul Moore <p.f.moore at gmail.com> wrote:
-> ...
-> > I could still argue that there are downsides (need to take action to
-> > set myself as nosy on an issue, possibly setting up a new mail filter,
-> > housekeeping cruft, the fact that people don't quote in the same way
-> > on tracker items) that make tracker discussions less attractive to me,
-> > but it's very personal things.
-> 
-> How hard would it be to set up a bot that will subscribe to python-dev
-> and archive messages that have [issueXXXX] in the subject under
-> appropriate roundup ticket?  This way if discussion started on
-> roundup, it would only take adding python-dev in CC to bring it to the
-> larger python-dev audience and adding issue number to the python-dev
-> thread subject will be all it takes to archive new messages on
-> roundup.  This would not solve a problem of linking threads that start
-> on python-dev before a ticket is opened, but with some proper
-> conventions this problem can be solved as well.

In my experience this kind automated stuff is too fragile to work
reliably & specifically over time, which is what we would want -- "fire
and forget".

Is there a good python-dev archive search mechanism (other than to
google "python-dev <term>" ;) out there?  Wouldn't that help?

cheers,
--titus
-- 
C. Titus Brown, ctb at msu.edu

From martin at v.loewis.de  Thu Jun  4 17:50:34 2009
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Thu, 04 Jun 2009 17:50:34 +0200
Subject: [Python-Dev] Issues with process and discussions (Re: Issues
 with	Py3.1's new ipaddr)
In-Reply-To: <20090604024802.12555.604416129.divmod.xquotient.12376@weber.divmod.com>
References: <20090602175600.22176.619128085.divmod.quotient.1116@henry.divmod.com>	<F6D076DA-5542-4532-8D99-D44B588E8F3E@object-craft.com.au>	<ca471dc20906021939wa12e7b6y4e0f7f1a9349d9ce@mail.gmail.com>	<20090603174151.12555.1359697426.divmod.xquotient.12290@weber.divmod.com>	<4A26CA17.901@v.loewis.de>
	<20090604024802.12555.604416129.divmod.xquotient.12376@weber.divmod.com>
Message-ID: <4A27ED4A.6080002@v.loewis.de>

> Is there a document which lists these things, and explains how it is
> desirable to communicate them?  I recently updated Twisted's equivalent
> document, adding minutae like which buttons to click on in our issue
> tracker, since that seems obvious to me but apparently wasn't obvious to
> a lot of new contributors.

I'm not quite sure whether this is what you are asking for, but there is

http://www.python.org/dev/workflow/

> My reading of it suggests that he was saying "netaddr appears to be
> superior in every way, so python should include that instead.  But, if
> someone is insisting on ipaddr here are the things that could change
> about it".  The important thing here is that interpretation of the
> comment is required, so I can definitely see how you saw it the way you
> did.  There is no "-1" in his comment, and there's no documentation
> (that I'm aware of) which says that a "-1" is required, or how it will
> be used or interpreted.

Well, there is

http://www.python.org/dev/peps/pep-0010/

I personally use them sometimes, but wouldn't insist on others using them.

Regards,
Martin


From dirkjan at ochtman.nl  Thu Jun  4 17:53:39 2009
From: dirkjan at ochtman.nl (Dirkjan Ochtman)
Date: Thu, 4 Jun 2009 17:53:39 +0200
Subject: [Python-Dev] Issues with process and discussions (Re: Issues
	with Py3.1's new ipaddr)
In-Reply-To: <20090604150814.GB17981@idyll.org>
References: <20090602175600.22176.619128085.divmod.quotient.1116@henry.divmod.com>
	<F6D076DA-5542-4532-8D99-D44B588E8F3E@object-craft.com.au>
	<ca471dc20906021939wa12e7b6y4e0f7f1a9349d9ce@mail.gmail.com>
	<20090603174151.12555.1359697426.divmod.xquotient.12290@weber.divmod.com>
	<79990c6b0906031042y190e05d6obbf675200b171666@mail.gmail.com>
	<4A279CC9.7080507@gmail.com>
	<79990c6b0906040320m22c79a56jb3d3286f330bcb68@mail.gmail.com>
	<d38f5330906040803n3c120897t346439a45ea4ea14@mail.gmail.com>
	<20090604150814.GB17981@idyll.org>
Message-ID: <ea2499da0906040853x170dbdfpfb16c8ce8a454aea@mail.gmail.com>

On Thu, Jun 4, 2009 at 5:08 PM, C. Titus Brown <ctb at msu.edu> wrote:
> In my experience this kind automated stuff is too fragile to work
> reliably & specifically over time, which is what we would want -- "fire
> and forget".

You could a python-dev account in the tracker and allow people to nosy it.

> Is there a good python-dev archive search mechanism (other than to
> google "python-dev <term>" ;) out there? ?Wouldn't that help?

Like http://python.markmail.org/?

Cheers,

Dirkjan

From martin at v.loewis.de  Thu Jun  4 17:54:23 2009
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Thu, 04 Jun 2009 17:54:23 +0200
Subject: [Python-Dev] Issues with process and discussions (Re: Issues
 with Py3.1's new ipaddr)
In-Reply-To: <20090604150814.GB17981@idyll.org>
References: <20090602175600.22176.619128085.divmod.quotient.1116@henry.divmod.com>	<F6D076DA-5542-4532-8D99-D44B588E8F3E@object-craft.com.au>	<ca471dc20906021939wa12e7b6y4e0f7f1a9349d9ce@mail.gmail.com>	<20090603174151.12555.1359697426.divmod.xquotient.12290@weber.divmod.com>	<79990c6b0906031042y190e05d6obbf675200b171666@mail.gmail.com>	<4A279CC9.7080507@gmail.com>	<79990c6b0906040320m22c79a56jb3d3286f330bcb68@mail.gmail.com>	<d38f5330906040803n3c120897t346439a45ea4ea14@mail.gmail.com>
	<20090604150814.GB17981@idyll.org>
Message-ID: <4A27EE2F.4080600@v.loewis.de>

> Is there a good python-dev archive search mechanism (other than to
> google "python-dev <term>" ;) out there?  Wouldn't that help?

I would add site:mail.python.org into the google query, but apart from
that: what's wrong with google search?

Regards,
Martin

From martin at v.loewis.de  Thu Jun  4 17:55:52 2009
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Thu, 04 Jun 2009 17:55:52 +0200
Subject: [Python-Dev] Issues with process and discussions (Re: Issues
 with Py3.1's new ipaddr)
In-Reply-To: <d38f5330906040817r7127c0a8g36b102b56428f5a4@mail.gmail.com>
References: <20090602175600.22176.619128085.divmod.quotient.1116@henry.divmod.com>	<F6D076DA-5542-4532-8D99-D44B588E8F3E@object-craft.com.au>	<ca471dc20906021939wa12e7b6y4e0f7f1a9349d9ce@mail.gmail.com>	<20090603174151.12555.1359697426.divmod.xquotient.12290@weber.divmod.com>	<79990c6b0906031042y190e05d6obbf675200b171666@mail.gmail.com>	<4A279CC9.7080507@gmail.com>	<79990c6b0906040320m22c79a56jb3d3286f330bcb68@mail.gmail.com>	<d38f5330906040803n3c120897t346439a45ea4ea14@mail.gmail.com>	<20090604150814.GB17981@idyll.org>
	<d38f5330906040817r7127c0a8g36b102b56428f5a4@mail.gmail.com>
Message-ID: <4A27EE88.7080208@v.loewis.de>

Alexander Belopolsky wrote:
> Alternatively, one could create a python-dev profile on roundup and
> encourage people to add python-dev to the nosy list when discussion
> needs to be broaden. This way replies to all will be posted in both
> places.  Once discussion gets too specialized, someone can remove
> python-dev from the "nosies." No new code is required to set this up
> as far as I can tell.

That would certainly be possible. I'm not sure whether it *should*
be done, though. If anybody actually wants that feature, please create
a ticket in the meta tracker.

Regards,
Martin


From martin at v.loewis.de  Thu Jun  4 18:01:03 2009
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Thu, 04 Jun 2009 18:01:03 +0200
Subject: [Python-Dev] Issues with process and discussions (Re: Issues
 with Py3.1's new ipaddr)
In-Reply-To: <4A279CC9.7080507@gmail.com>
References: <20090602175600.22176.619128085.divmod.quotient.1116@henry.divmod.com>	<F6D076DA-5542-4532-8D99-D44B588E8F3E@object-craft.com.au>	<ca471dc20906021939wa12e7b6y4e0f7f1a9349d9ce@mail.gmail.com>	<20090603174151.12555.1359697426.divmod.xquotient.12290@weber.divmod.com>	<79990c6b0906031042y190e05d6obbf675200b171666@mail.gmail.com>
	<4A279CC9.7080507@gmail.com>
Message-ID: <4A27EFBF.7070006@v.loewis.de>

> In addition, you can fairly easily create a saved query to show you all
> the open tickets that you are on the nosy list for. (Although I created
> and saved my query for that so long ago that I don't recall the exact
> details on how to go about doing that).

It's fairly easy. Start a search, put your account name into "Nosy list
member", and put "My nosy" (or some such) into "Query name**". Then hit
search, and watch "My nosy" appear under "Your queries".

Regards,
Martin

From tjreedy at udel.edu  Thu Jun  4 19:15:16 2009
From: tjreedy at udel.edu (Terry Reedy)
Date: Thu, 04 Jun 2009 13:15:16 -0400
Subject: [Python-Dev] Google Wave as a developer communication tool
In-Reply-To: <87tz2wo1dx.fsf@benfinney.id.au>
References: <20090602175600.22176.619128085.divmod.quotient.1116@henry.divmod.com>	<F6D076DA-5542-4532-8D99-D44B588E8F3E@object-craft.com.au>	<ca471dc20906021939wa12e7b6y4e0f7f1a9349d9ce@mail.gmail.com>	<20090603174151.12555.1359697426.divmod.xquotient.12290@weber.divmod.com>	<h06mtt$dci$1@ger.gmane.org>
	<87vdncq417.fsf_-_@benfinney.id.au>	<h07mu5$qct$1@ger.gmane.org>
	<87tz2wo1dx.fsf@benfinney.id.au>
Message-ID: <h08vf6$tgk$1@ger.gmane.org>

Ben Finney wrote:
> Terry Reedy <tjreedy at udel.edu> writes:
> 
>> Ben Finney wrote:
>>> I watched [the Google Wave presentation] too. It appears to be
>>> heavily reliant on *very* fast internet access for participants in a
>>> wave. That's far from universal in the Python community, let alone
>>> the internet at large.
>> Even a slow connection would make participation in PEPs better than
>> today.
> 
> How can you know that? A slow link doesn't punish email or NNTP
> communication the way an interactive web application does. Why would
> Google Wave be any less punitive to low-bandwidth users than existing
> live web applications?

I am not sure of what 'live web application' you have in mind to compare 
to ;-).  A general answer is text stream versus binary blob stream.  A 
wave is more like a souped-up mailing-list or newsgroup thread or wiki 
page than an online game.  You open your wave client and PEPxxx.wave is 
marked as having new content.  At your leisure, you open it (or perhaps 
you have marked it 'download updates in background').  That that takes 
longer with a slow connection is no different than with other text 
streams.  If you type in a comment, even 1200 baud upsteam is fast 
enough. Where do you get 'punitive' in this?

>>> It also appears to be heavily reliant on the wave's existence at a
>>> single point of failure (the hosting server): if that one point
>>> becomes unreliable, all participants are hosed.
>> We have that problem already with the tracker, which does occasionally
>> go down for a bit. And the svn host? (One reason to move to
>> distributed system.)
> 
> Right. These are all reasons for moving toward distributed systems;
> Python has chosen to do so already with its VCS. Why would the choice of
> a new communications technology not take this into consideration?

It should.  The wave protocol includes server-to-server mirroring.  A 
wave can be shared across multiple servers.

>>> Neither of these problems exist with email (or NNTP).
>> But do for an email list, like this one.  Or a wiki.

The second of 'these problems' was 'single point of failure'.
That *can* apply to email and wiki.

> No. None of mailing list, NNTP, or wiki are heavily punitive to
> low-bandwidth links.

I would not expect that to be much more true of a text-only wave.  It is 
binary content (and embeded ad links) that made broadband really 
necessary for web surfing.

Terry Jan Reedy


From mario.danic at gmail.com  Thu Jun  4 19:17:23 2009
From: mario.danic at gmail.com (Mario)
Date: Thu, 4 Jun 2009 19:17:23 +0200
Subject: [Python-Dev] Roundup improvements use-cases
Message-ID: <79957db20906041017u67232014r21f87ddea5304d50@mail.gmail.com>

Hello,

in the last few weeks I have been working on defining use-cases which
will lead the improvement of the Roundup tracker. As this is very important,
I would like your valuable input in form of comments, criticism and advices.

Use-cases in-lined:
==============
Daniel is a great hacker, but the only time he has time to hack is
when his child is asleep. And his child wakes up every now and then,
pretty often actually. So when it starts crying, Daniel commits fix to
that silly bug that has been hunting us for ages, but forgets to close
down the bug so it stays open.

Looking over bug tracker, Jane finds the same bug and starts working
on the same bug. Wasted time. Daniel should be able to format commit
message in such a way that would automatically close the bug.

Maria is a student, and got involved in FOSS recently. With all the
studies she has to do, she would prefer not to spend her time changing
bug status. She should be able to format commit message in such a way
that would change bug status to something other then the current
status.

Technical talk: USE CASE A: Integrate issue property editing with
Mercurial, including:

    * USE CASE A1: Allow users to close issues via tokens embedded
into commit messages.
    * USE CASE A2: Allow users to change issue status via tokens
embedded into commit messages.

Ronny is about to fix a torny bug, so he has a public branch for that.
He is making great progress, but the issue that tracks the bug only
contains a very out-of-date patch, prompting other developers to try
to fix the same bug. Ronny should be able to tell Roundup about where
his code lives, so users can get up-to-date patches/diffs
automatically. This also allows other users to know all the code that
changed to fix a given bug.

Technical talk: Integrate branch and rich patch handling into Roundup

    * USE CASE B: Track all changesets that relate to a given issue:
          o USE CASE B1: Using tokens embedded into commit messages.
(Post commit)
          o USE CASE B2: Using named branches, bookmarks. (Pre or post commit)
          o USE CASE B3: Using patchsets, bundles or whatchacallit for
fat Mercurial patches. (Pre or post commit)

Brett wants to fix a couple of related issues and has a local
Mercurial branch for that. He would like his commit messages to
include useful information for when his patch/branch lands in the
Python repository. Besides the Mercurial->Roundup integration, a
Roundup->Mercurial one that would allow one to fetch issue details and
existing patches/branches with metadata would make Brett's work more
valuable.

Technical talk: USE CASE C: Add a CLI interface for Roundup so VCSs
can query the tracker.

    * USE CASE C1: Automatically fetch issue data.
    * USE CASE C2: Pre-format output for greater usefulness in commit messages:
          o USE CASE C2.1: Setting issue properties.
          o USE CASE C2.2: Grouping changesets.
    * USE CASE C3: Fetch branch information for local cloning.
    * USE CASE C4: Add a Mercurial extension to exercize the CLI client.

USE CASE D:

Antoine is merging lots of branches and applying lots of patches to
his local branch of the Python repository. He goes to each issue with
a patch/branch and tells people whether their patches apply cleanly,
just to avoid merge issues in the main branch. While he could use
Mercurial Patch Queues (mq), Roundup would need to learn to both
listen to and to submit patches to mq in order to completely replace
Antoine's work with automated tools. Having a quick 'check if this
patch applies cleanly' button would make triaging issues much easier.

USE CASE E:

David is checking the python-commits list for avoiding bad code from
landing and nesting in the Python code base. He only sees the patches:
while informative, it requires a bit of mental gymanstics to see how
it would merge with the surrounding code. Whenever he thinks some code
has tabs and spaces or lines that are too long, he runs pylint and
reindent.py locally. He can only raise concerns about the code after
it lands on the main repository. It should be easier to see the code
changes in context. Having a way to check the code for mistakes in the
tracker would make David's life a lot easier.

USE CASE F:

Van Lindberg is concerned about code submissions from non-core
developers: how can the PSF re-license this code in the future without
talking to each contributor, whether the PSF is safe from litigation
based on copyrights of these contributions and related questions are
always bugging him. While the PSF has Contributor Agreements exactly
for these cases, it would be great to have the issue tracker and the
VCS talk to each other, so they could ask contributors to sign (or
declare to have already signed) the CAs when necessary.

USE CASE G: Use Transplant/Patch Branch to generate patches from
branches linked from Roundup.

USE CASE J:

Integrate the code/commits navigation interface with Roundup, so
changesets, branches, etc., can be easily linked/browsed (starting)
from the Roundup UI and issues can be created/linked to commits
(starting) from the navigation tool UI.

USE CASE K: For a given issue, add per patched file links for RSS logs
(see http://selenic.com/hg/rss-log/tip/mercurial/hgweb/hgweb_mod.py).

USE CASE M: Besides links to files, allow adding links to files at
given versions/tags/branchs, links to tarballs and easy to clone links
to branches and repositories.

USE CASE V:

Handle small branches (and maybe suggest using them for small
patches?) generated using the convert extension with --filemap.
==============

Improvements to the use cases will follow on the wiki page:
http://www.roundup-tracker.org/cgi-bin/moin.cgi/GSoC09

Thanks for your attention and time.

Respectfully,
Mario

From tjreedy at udel.edu  Thu Jun  4 19:24:26 2009
From: tjreedy at udel.edu (Terry Reedy)
Date: Thu, 04 Jun 2009 13:24:26 -0400
Subject: [Python-Dev] Issues with process and discussions (Re: Issues
 with Py3.1's new ipaddr)
In-Reply-To: <4A27EFBF.7070006@v.loewis.de>
References: <20090602175600.22176.619128085.divmod.quotient.1116@henry.divmod.com>	<F6D076DA-5542-4532-8D99-D44B588E8F3E@object-craft.com.au>	<ca471dc20906021939wa12e7b6y4e0f7f1a9349d9ce@mail.gmail.com>	<20090603174151.12555.1359697426.divmod.xquotient.12290@weber.divmod.com>	<79990c6b0906031042y190e05d6obbf675200b171666@mail.gmail.com>	<4A279CC9.7080507@gmail.com>
	<4A27EFBF.7070006@v.loewis.de>
Message-ID: <h0900a$v17$1@ger.gmane.org>

Martin v. L?wis wrote:
>> In addition, you can fairly easily create a saved query to show you all
>> the open tickets that you are on the nosy list for. (Although I created
>> and saved my query for that so long ago that I don't recall the exact
>> details on how to go about doing that).
> 
> It's fairly easy. Start a search, put your account name into "Nosy list
> member", and put "My nosy" (or some such) into "Query name**". Then hit
> search, and watch "My nosy" appear under "Your queries".

Done. Wow. Thanks.

tjr


From ctb at msu.edu  Thu Jun  4 19:34:00 2009
From: ctb at msu.edu (C. Titus Brown)
Date: Thu, 4 Jun 2009 10:34:00 -0700
Subject: [Python-Dev] Issues with process and discussions (Re: Issues
	with Py3.1's new ipaddr)
In-Reply-To: <4A27EE2F.4080600@v.loewis.de>
References: <20090602175600.22176.619128085.divmod.quotient.1116@henry.divmod.com>
	<F6D076DA-5542-4532-8D99-D44B588E8F3E@object-craft.com.au>
	<ca471dc20906021939wa12e7b6y4e0f7f1a9349d9ce@mail.gmail.com>
	<20090603174151.12555.1359697426.divmod.xquotient.12290@weber.divmod.com>
	<79990c6b0906031042y190e05d6obbf675200b171666@mail.gmail.com>
	<4A279CC9.7080507@gmail.com>
	<79990c6b0906040320m22c79a56jb3d3286f330bcb68@mail.gmail.com>
	<d38f5330906040803n3c120897t346439a45ea4ea14@mail.gmail.com>
	<20090604150814.GB17981@idyll.org> <4A27EE2F.4080600@v.loewis.de>
Message-ID: <20090604173400.GB25702@idyll.org>

On Thu, Jun 04, 2009 at 05:54:23PM +0200, "Martin v. L?wis" wrote:
-> > Is there a good python-dev archive search mechanism (other than to
-> > google "python-dev <term>" ;) out there?  Wouldn't that help?
-> 
-> I would add site:mail.python.org into the google query, but apart from
-> that: what's wrong with google search?

It generally points you towards the right messages in the python-dev
archives, but those are kept by month and the threading doesn't always
work well.  Sometimes conversations span many months, and threading can
be an immense help for understanding the discussion.

Something like MarkMail (as Dirkjan mentioned) may have a better
interface.  I'll give it a try.

thanks,
--titus
-- 
C. Titus Brown, ctb at msu.edu

From nad at acm.org  Thu Jun  4 20:48:07 2009
From: nad at acm.org (Ned Deily)
Date: Thu, 04 Jun 2009 11:48:07 -0700
Subject: [Python-Dev] Issues with process and discussions (Re: Issues
	with Py3.1's new ipaddr)
References: <20090602175600.22176.619128085.divmod.quotient.1116@henry.divmod.com>
	<F6D076DA-5542-4532-8D99-D44B588E8F3E@object-craft.com.au>
	<ca471dc20906021939wa12e7b6y4e0f7f1a9349d9ce@mail.gmail.com>
	<20090603174151.12555.1359697426.divmod.xquotient.12290@weber.divmod.com>
	<79990c6b0906031042y190e05d6obbf675200b171666@mail.gmail.com>
	<4A279CC9.7080507@gmail.com>
	<79990c6b0906040320m22c79a56jb3d3286f330bcb68@mail.gmail.com>
	<d38f5330906040803n3c120897t346439a45ea4ea14@mail.gmail.com>
	<20090604150814.GB17981@idyll.org> <4A27EE2F.4080600@v.loewis.de>
	<20090604173400.GB25702@idyll.org>
Message-ID: <nad-54C592.11480704062009@ger.gmane.org>

In article <20090604173400.GB25702 at idyll.org>,
 "C. Titus Brown" <ctb at msu.edu> wrote:
> Something like MarkMail (as Dirkjan mentioned) may have a better
> interface.  I'll give it a try.

Or http://search.gmane.org/  with group gmane.comp.python.devel

-- 
 Ned Deily,
 nad at acm.org


From van.lindberg at gmail.com  Thu Jun  4 20:52:12 2009
From: van.lindberg at gmail.com (VanL)
Date: Thu, 04 Jun 2009 13:52:12 -0500
Subject: [Python-Dev] Roundup improvements use-cases
In-Reply-To: <79957db20906041017u67232014r21f87ddea5304d50@mail.gmail.com>
References: <79957db20906041017u67232014r21f87ddea5304d50@mail.gmail.com>
Message-ID: <h0954s$dak$1@ger.gmane.org>

Mario wrote:
> USE CASE F:
> 
> Van Lindberg is concerned about code submissions from non-core
> developers: how can the PSF re-license this code in the future without
> talking to each contributor, whether the PSF is safe from litigation
> based on copyrights of these contributions and related questions are
> always bugging him. While the PSF has Contributor Agreements exactly
> for these cases, it would be great to have the issue tracker and the
> VCS talk to each other, so they could ask contributors to sign (or
> declare to have already signed) the CAs when necessary.

+1. This is exactly right.


From seefeld at sympatico.ca  Thu Jun  4 22:10:41 2009
From: seefeld at sympatico.ca (Stefan Seefeld)
Date: Thu, 04 Jun 2009 16:10:41 -0400
Subject: [Python-Dev] [Roundup-devel] Roundup improvements use-cases
In-Reply-To: <79957db20906041017u67232014r21f87ddea5304d50@mail.gmail.com>
References: <79957db20906041017u67232014r21f87ddea5304d50@mail.gmail.com>
Message-ID: <4A282A41.8040701@sympatico.ca>

Hi Mario,

thanks for sending these use-cases out. Let me give some feedback:

Mario wrote:
> Technical talk: USE CASE A: Integrate issue property editing with
> Mercurial, including:
>
>     * USE CASE A1: Allow users to close issues via tokens embedded
> into commit messages.
>     * USE CASE A2: Allow users to change issue status via tokens
> embedded into commit messages.
>   

This is the most fundamental one, and it is implementable without even 
touching Roundup itself. All that is required is a 'post-commit-hook' 
(whatever that is spelled for a particular VCS) that sends a mail to a 
tracker, if some tokens were found in the commit message. While in 
theory this is all up for customization, I think it would be nice to set 
up some conventions to guide users in how to do this (the 'token 
grammar', mostly), just for convenience.



> Ronny is about to fix a torny bug, so he has a public branch for that.
> He is making great progress, but the issue that tracks the bug only
> contains a very out-of-date patch, prompting other developers to try
> to fix the same bug. Ronny should be able to tell Roundup about where
> his code lives, so users can get up-to-date patches/diffs
> automatically. This also allows other users to know all the code that
> changed to fix a given bug.
>
> Technical talk: Integrate branch and rich patch handling into Roundup
>
>     * USE CASE B: Track all changesets that relate to a given issue:
>           o USE CASE B1: Using tokens embedded into commit messages.
> (Post commit)
>           o USE CASE B2: Using named branches, bookmarks. (Pre or post commit)
>           o USE CASE B3: Using patchsets, bundles or whatchacallit for
> fat Mercurial patches. (Pre or post commit)
>   

Hmm, I have difficulties mapping your prosaic description of the use 
case above to the 'technical talk'. The latter almost sounds like an 
implementation strategy to me.

This is a very interesting use case, driving us at the core of the 'VCS 
integration'. We need to customize a schema to provide a notion for 
'repository' and 'branch', and also for 'patch'. Then we need to somehow 
endow the patch type with knowledge of what it means to be 'applied' to 
a branch, etc. (more on this below).

> Brett wants to fix a couple of related issues and has a local
> Mercurial branch for that. He would like his commit messages to
> include useful information for when his patch/branch lands in the
> Python repository. Besides the Mercurial->Roundup integration, a
> Roundup->Mercurial one that would allow one to fetch issue details and
> existing patches/branches with metadata would make Brett's work more
> valuable.
>
> Technical talk: USE CASE C: Add a CLI interface for Roundup so VCSs
> can query the tracker.
>
>     * USE CASE C1: Automatically fetch issue data.
>     * USE CASE C2: Pre-format output for greater usefulness in commit messages:
>           o USE CASE C2.1: Setting issue properties.
>           o USE CASE C2.2: Grouping changesets.
>     * USE CASE C3: Fetch branch information for local cloning.
>     * USE CASE C4: Add a Mercurial extension to exercize the CLI client.
>   

This set of use cases ('C') is what I have most difficulties with. It 
clearly is about 'VCS <-> Roundup integration', but it's in the other 
direction. And thus, this doesn't seem to concern Roundup itself, or 
does it ? Presumably, everything needed already exists. We now have an 
XMLRPC interface, so it is possible to write (remote) Roundup clients. I 
think writing such a 'Roundup client library' would be useful, but I 
think somewhat out of scope with respect to this GSoC project.


> USE CASE D:
>
> Antoine is merging lots of branches and applying lots of patches to
> his local branch of the Python repository. He goes to each issue with
> a patch/branch and tells people whether their patches apply cleanly,
> just to avoid merge issues in the main branch. While he could use
> Mercurial Patch Queues (mq), Roundup would need to learn to both
> listen to and to submit patches to mq in order to completely replace
> Antoine's work with automated tools. Having a quick 'check if this
> patch applies cleanly' button would make triaging issues much easier.
>   

Let's drill down here a little, as this may be more than one use-case:

D1: provide an interface through which users can attempt to apply a 
submitted patch to a given branch.
       This requires a 'working copy' of the code (Roundup may be 
configured to create that if it doesn't exist yet), and a simple 
interface to the 'patch' tool. If we create a new 'Patch' type, it may 
include metadata that help Roundup to figure out what repo / branch URL 
to use to fetch the code from against which the patch is to be applied 
against.

D2: While D1 is about applying patches, this may not be enough to figure 
out whether the patch is good. The user may also want to attempt to 
build the code (and even run tests). While I believe this to be outside 
the realm of a bug / patch tracker, I can see good use in integration 
with external build tools. In fact, it is a use case such as this that I 
had in mind when alluding to bridging with buildbot. I think it would be 
nice to think a little more about it, without actually implementing it. 
Just making it possible for future work in this direction ought to be 
enough.

> USE CASE E:
>
> David is checking the python-commits list for avoiding bad code from
> landing and nesting in the Python code base. He only sees the patches:
> while informative, it requires a bit of mental gymanstics to see how
> it would merge with the surrounding code. Whenever he thinks some code
> has tabs and spaces or lines that are too long, he runs pylint and
> reindent.py locally. He can only raise concerns about the code after
> it lands on the main repository. It should be easier to see the code
> changes in context. Having a way to check the code for mistakes in the
> tracker would make David's life a lot easier.
>   

Yes, if we create a new 'Patch' type, we also want to add to the 
templating module to provide rendering functions that can render the 
patch in different ways (diffs between the target branch, with and 
without the patch applied, etc.)

Again, 'checking the code for mistakes' can be a big task, and so goes 
into the D2 case above.


> USE CASE F:
>
> Van Lindberg is concerned about code submissions from non-core
> developers: how can the PSF re-license this code in the future without
> talking to each contributor, whether the PSF is safe from litigation
> based on copyrights of these contributions and related questions are
> always bugging him. While the PSF has Contributor Agreements exactly
> for these cases, it would be great to have the issue tracker and the
> VCS talk to each other, so they could ask contributors to sign (or
> declare to have already signed) the CAs when necessary.
>   

Sounds good. Nothing new here (besides what we already described in the 
use cases above).

> USE CASE G: Use Transplant/Patch Branch to generate patches from
> branches linked from Roundup.
>   

This sounds like accessing a VCS-specific GUI through Roundup. While I 
can see this being useful, I'm not sure how much this relates directly 
to Roundup.

> USE CASE J:
>
> Integrate the code/commits navigation interface with Roundup, so
> changesets, branches, etc., can be easily linked/browsed (starting)
> from the Roundup UI and issues can be created/linked to commits
> (starting) from the navigation tool UI.
>   

Similarly here.


> USE CASE K: For a given issue, add per patched file links for RSS logs
> (see http://selenic.com/hg/rss-log/tip/mercurial/hgweb/hgweb_mod.py).
>   

This sounds like a small addition to case A above: Once I collect 
(references to) changesets in an issue, I can use them to backlink into 
a repository UI.

> USE CASE M: Besides links to files, allow adding links to files at
> given versions/tags/branchs, links to tarballs and easy to clone links
> to branches and repositories.
>   
Right. Easy enough once our schema knows about 'repo', 'branch' and 
'revision' (We still need a per VCS definition of how to construct such 
a link from this tuple.

> USE CASE V:
>
> Handle small branches (and maybe suggest using them for small
> patches?) generated using the convert extension with --filemap.
>   
I have no idea what this is about, sorry. :-)

> ==============
>
> Improvements to the use cases will follow on the wiki page:
> http://www.roundup-tracker.org/cgi-bin/moin.cgi/GSoC09
>   

Great, thanks. I suggest that we keep discussions here on the list(s), 
and then capture any outcome on that (and related) wiki page(s).

> Thanks for your attention and time.
>   

Thanks for working on this !

       Stefan


-- 

      ...ich hab' noch einen Koffer in Berlin...


From amk at amk.ca  Thu Jun  4 22:44:09 2009
From: amk at amk.ca (A.M. Kuchling)
Date: Thu, 4 Jun 2009 16:44:09 -0400
Subject: [Python-Dev] Google Wave as a developer communication tool
In-Reply-To: <h08vf6$tgk$1@ger.gmane.org>
References: <20090602175600.22176.619128085.divmod.quotient.1116@henry.divmod.com>
	<F6D076DA-5542-4532-8D99-D44B588E8F3E@object-craft.com.au>
	<ca471dc20906021939wa12e7b6y4e0f7f1a9349d9ce@mail.gmail.com>
	<20090603174151.12555.1359697426.divmod.xquotient.12290@weber.divmod.com>
	<h06mtt$dci$1@ger.gmane.org> <87vdncq417.fsf_-_@benfinney.id.au>
	<h07mu5$qct$1@ger.gmane.org> <87tz2wo1dx.fsf@benfinney.id.au>
	<h08vf6$tgk$1@ger.gmane.org>
Message-ID: <20090604204409.GA15415@amk-desktop.matrixgroup.net>

On Thu, Jun 04, 2009 at 01:15:16PM -0400, Terry Reedy wrote:
> marked as having new content.  At your leisure, you open it (or perhaps  
> you have marked it 'download updates in background').  That that takes  
> longer with a slow connection is no different than with other text  
> streams.  If you type in a comment, even 1200 baud upsteam is fast  
> enough. Where do you get 'punitive' in this?

I visited my parents at Christmas.  They live in a rural area, don't
have great phone lines, and my dad's computer ends up getting about a
21Kb modem connection (not even 38.4!).  I discovered that great
swaths of the Web 2.0 world were effectively unusable for me; after
login, the Twitter home page took 3 minutes to display.  Logging into
my bank was a tortuous 10 minute process.  I never succeeded in
getting into Facebook at all.  Many pages don't render until they're
completely downloaded.

The little AJAXy update that adds an extra second or two on a fast
connection becomes shockingly painful on a slow connection.  The JS
files are organized Unless Google Wave is written with attention to
such slow connections (which I doubt -- such users are pretty rare,
after all), I would assume it will be unusable.  The bandwidth is
enough for a character by character stream, but web browser apps
impose many overheads atop that stream.

--amk

From jake at youtube.com  Thu Jun  4 23:04:00 2009
From: jake at youtube.com (Jake McGuire)
Date: Thu, 4 Jun 2009 14:04:00 -0700
Subject: [Python-Dev] Google Wave as a developer communication tool
In-Reply-To: <20090604204409.GA15415@amk-desktop.matrixgroup.net>
References: <20090602175600.22176.619128085.divmod.quotient.1116@henry.divmod.com>
	<F6D076DA-5542-4532-8D99-D44B588E8F3E@object-craft.com.au>
	<ca471dc20906021939wa12e7b6y4e0f7f1a9349d9ce@mail.gmail.com>
	<20090603174151.12555.1359697426.divmod.xquotient.12290@weber.divmod.com>
	<h06mtt$dci$1@ger.gmane.org> <87vdncq417.fsf_-_@benfinney.id.au>
	<h07mu5$qct$1@ger.gmane.org> <87tz2wo1dx.fsf@benfinney.id.au>
	<h08vf6$tgk$1@ger.gmane.org>
	<20090604204409.GA15415@amk-desktop.matrixgroup.net>
Message-ID: <77c780b40906041404k737802few774556ca294a3646@mail.gmail.com>

Google Wave is also still in tightly restricted beta.  Gmail went
through a long invite-only period.  Until we have an idea of how long
it will be until basically all python developers who want a Wave
account can get one, it doesn't make sense to talk about using it for
python development, IMO.

-jake

From db3l.net at gmail.com  Thu Jun  4 23:23:24 2009
From: db3l.net at gmail.com (David Bolen)
Date: Thu, 04 Jun 2009 17:23:24 -0400
Subject: [Python-Dev] Google Wave as a developer communication tool
References: <20090602175600.22176.619128085.divmod.quotient.1116@henry.divmod.com>
	<F6D076DA-5542-4532-8D99-D44B588E8F3E@object-craft.com.au>
	<ca471dc20906021939wa12e7b6y4e0f7f1a9349d9ce@mail.gmail.com>
	<20090603174151.12555.1359697426.divmod.xquotient.12290@weber.divmod.com>
	<h06mtt$dci$1@ger.gmane.org> <87vdncq417.fsf_-_@benfinney.id.au>
	<h07mu5$qct$1@ger.gmane.org> <20090604122300.GA3768@panix.com>
Message-ID: <m2bpp3k81f.fsf@valheru.db3l.homeip.net>

Aahz <aahz at pythoncraft.com> writes:

> It sounds like Wave requires a high-powered browser, similar to Google
> Maps.  That makes me -1 because I want to continue using Lynx.

I'm not sure - I think you can implement your own choices at different
points.

What's interesting to me so far is less the current UI/flashiness
(drag 'n drop photos, real-time keystrokes, etc...) they are showing,
than the openness of the protocol and implementation, opening the door
to all sorts of potential uses and/or integrations with existing
systems.  Other than convenience for interactive use, and as a
reference client, I'm not sure how critical a browser really is to the
system.

Fundamentally, wave seems to be rich, versioned (or at least time
tracked), content management system with well defined server to server
and server to client communication protocols, including conflict
resolution.  They even brought up VCS-like concepts (currently
unimplemented) of having waves branch and evolve independently, fold
changes back into a parent wave and then re-distribute down to the
other children over time, which points to a reasonable central data
structure.

For example, if you jump to about 1:08:13 in the presentation you can
see a more textual display implemented by an independent wave
server/client - almost newsreader like.  I presume such a client has
to make similar decisions as to display of non-textual data as Lynx
has to with the web.

Or if you embed the wave in a web page, it could still be handled by
text browsers (the blog integration at 20:43 for example) or perhaps
with RSS.

Server robots could help too - a robot participating in the wave could
produce a more text-friendly reader version, perhaps published
elsewhere on the web or in a different system.  They demoed, for
example, integrating with Google Code's bug tracking back end at
1:02:30 where comments in the wave were reflected into the bug
comments (though the reverse direction wasn't implemented yet), or
even the twitter integration at 57:53.

-- David


From dalcinl at gmail.com  Thu Jun  4 23:44:06 2009
From: dalcinl at gmail.com (Lisandro Dalcin)
Date: Thu, 4 Jun 2009 18:44:06 -0300
Subject: [Python-Dev] Serious regression in doctest in Py3.1rc1
In-Reply-To: <e27efe130906030426o45081fd5v9bbb877ffd76503d@mail.gmail.com>
References: <h057jb$bg3$1@ger.gmane.org>
	<e27efe130906030426o45081fd5v9bbb877ffd76503d@mail.gmail.com>
Message-ID: <e7ba66e40906041444q285fcc1cg9cd433bef008e687@mail.gmail.com>

http://bugs.python.org/issue6195 (with patch)

On Wed, Jun 3, 2009 at 8:26 AM, Amaury Forgeot d'Arc <amauryfa at gmail.com> wrote:
> Hello,
>
> 2009/6/3 Stefan Behnel <stefan_ml at behnel.de>:
>> Hi,
>>
>> I can't currently file a bug report on this, but I was told by Lisandro
>> Dalc?n that there is a serious problem with the doctest module in Py3.1rc1.
>> In Cython, we use doctests to test the compiler in that we compile a
>> Python/Cython module with doctests into a C module and then run doctest on
>> the imported extension module.
>>
>> >From the error report it seems to me that doctest is now trying to read the
>> module itself through linecache for some reason, which horribly fails for a
>> binary module.
>>
>> Could someone please look into this? I'll open up a bug report tomorrow
>> unless someone beats me to it.
>
> I don't have the time either, but the problem looks very similar to
> http://bugs.python.org/issue4050
> The fix was to replace:
> ? ? file = inspect.getsourcefile(object) or inspect.getfile(object)
> was replaced with
> ? ? file = inspect.getsourcefile(object)
>
> --
> Amaury Forgeot d'Arc
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/dalcinl%40gmail.com
>



-- 
Lisandro Dalc?n
---------------
Centro Internacional de M?todos Computacionales en Ingenier?a (CIMEC)
Instituto de Desarrollo Tecnol?gico para la Industria Qu?mica (INTEC)
Consejo Nacional de Investigaciones Cient?ficas y T?cnicas (CONICET)
PTLC - G?emes 3450, (3000) Santa Fe, Argentina
Tel/Fax: +54-(0)342-451.1594

From ben+python at benfinney.id.au  Fri Jun  5 01:52:39 2009
From: ben+python at benfinney.id.au (Ben Finney)
Date: Fri, 05 Jun 2009 09:52:39 +1000
Subject: [Python-Dev] Roundup improvements use-cases
References: <79957db20906041017u67232014r21f87ddea5304d50@mail.gmail.com>
	<4A282A41.8040701@sympatico.ca>
Message-ID: <874ouvo8u0.fsf@benfinney.id.au>

Stefan Seefeld <seefeld at sympatico.ca> writes:

> While in theory [implementing a hook for closing bugs via VCS commit
> messages] is all up for customization, I think it would be nice to set
> up some conventions to guide users in how to do this (the 'token
> grammar', mostly), just for convenience.

A reference implementation that I think works quite well is the system
used by the Debian project for detecting, from the changelog entries,
which bugs to close when a particular release of a package is uploaded.

    <URL:http://www.debian.org/doc/manuals/developers-reference/pkgs.html#upload-bugfix>

Note also that this system doesn't just close the bug, it makrs it as
fixed in a specific version of the package: the package version that
included the close request in its changelog entry.

It would be valuable if analogous behaviour could occur with Roundup,
which requires allowing for it in the specification of the workflow.

> This is a very interesting use case, driving us at the core of the
> 'VCS integration'. We need to customize a schema to provide a notion
> for 'repository' and 'branch', and also for 'patch'. Then we need to
> somehow endow the patch type with knowledge of what it means to be
> 'applied' to a branch, etc. (more on this below).

All this would apply to the above workflow for ?mark that bug as fixed
by this specific branch?.

(good sigmonster, have a cookie)

-- 
 \          ?That's all very good in practice, but how does it work in |
  `\                                             *theory*?? ?anonymous |
_o__)                                                                  |
Ben Finney


From ben+python at benfinney.id.au  Fri Jun  5 02:06:07 2009
From: ben+python at benfinney.id.au (Ben Finney)
Date: Fri, 05 Jun 2009 10:06:07 +1000
Subject: [Python-Dev] Roundup improvements use-cases
References: <79957db20906041017u67232014r21f87ddea5304d50@mail.gmail.com>
	<4A282A41.8040701@sympatico.ca>
Message-ID: <87zlcnmtn4.fsf@benfinney.id.au>

Stefan Seefeld <seefeld at sympatico.ca> writes:

> This set of use cases ('C') is what I have most difficulties with. It 
> clearly is about 'VCS <-> Roundup integration', but it's in the other 
> direction. And thus, this doesn't seem to concern Roundup itself, or 
> does it ? Presumably, everything needed already exists. We now have an 
> XMLRPC interface, so it is possible to write (remote) Roundup clients. I 
> think writing such a 'Roundup client library' would be useful, but I 
> think somewhat out of scope with respect to this GSoC project.

Note that there are several attempts in the wild at ?BTS database as
versioned data in the VCS repository?, allowing the tightest
integration between the two and ensuring that VCS commits and BTS
updates are inherently tied to each other.

An article at Linux Weekly News <URL:http://lwn.net/Articles/281849/>
describes these systems. It might be instructive for those looking to
tightly integrate VCS and BTS to seek inspiration from these designs.

-- 
 \         ?The userbase for strong cryptography declines by half with |
  `\      every additional keystroke or mouseclick required to make it |
_o__)                                             work.? ?Carl Ellison |
Ben Finney


From solipsis at pitrou.net  Fri Jun  5 09:19:51 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Fri, 5 Jun 2009 07:19:51 +0000 (UTC)
Subject: [Python-Dev] py3k buildbots
Message-ID: <loom.20090605T071922-555@post.gmane.org>

Hello

Only one of the py3k buildbots seems up:
http://www.python.org/dev/buildbot/3.x.stable/

cheers

Antoine.



From theller at ctypes.org  Fri Jun  5 11:26:34 2009
From: theller at ctypes.org (Thomas Heller)
Date: Fri, 05 Jun 2009 11:26:34 +0200
Subject: [Python-Dev] py3k buildbots
In-Reply-To: <loom.20090605T071922-555@post.gmane.org>
References: <loom.20090605T071922-555@post.gmane.org>
Message-ID: <h0aocb$8rr$1@ger.gmane.org>

Antoine Pitrou schrieb:
> Hello
> 
> Only one of the py3k buildbots seems up:
> http://www.python.org/dev/buildbot/3.x.stable/

Maybe they are waiting for the snakebite network ;-) (what's up with it, anyway?).

I've restarted mine (x86 osx.5), but it isn't in the stable list...

-- 
Thanks,
Thomas


From ncoghlan at gmail.com  Fri Jun  5 11:31:03 2009
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Fri, 05 Jun 2009 19:31:03 +1000
Subject: [Python-Dev] Mercurial, linefeeds, and Visual Studio
In-Reply-To: <ea2499da0906040803t4d04267eua40e205ef897042e@mail.gmail.com>
References: <8B473FAE8A08C34C9F5666FD4B0A87B67E2DDA50B9@hornigold.jaraco.com>	<ea2499da0906040630v926e477g7cdd176ce70d8a56@mail.gmail.com>	<79990c6b0906040720m1d31f2fuff863262b891928a@mail.gmail.com>
	<ea2499da0906040803t4d04267eua40e205ef897042e@mail.gmail.com>
Message-ID: <4A28E5D7.2030402@gmail.com>

Dirkjan Ochtman wrote:
>> Essentially, pcbuild.sln is a binary file, and should be treated as
>> such. Maybe it's an error in the Subversion setup that it's treated as
>> text at all...
> 
> Yes, it certainly seems that way.

Except it isn't a binary file - it's a text file with CRLF line endings.
Why would we throw away the chance to get meaningful diffs when
Subversion can easily get the line endings correct?

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------

From p.f.moore at gmail.com  Fri Jun  5 15:14:52 2009
From: p.f.moore at gmail.com (Paul Moore)
Date: Fri, 5 Jun 2009 14:14:52 +0100
Subject: [Python-Dev] Mercurial, linefeeds, and Visual Studio
In-Reply-To: <4A28E5D7.2030402@gmail.com>
References: <8B473FAE8A08C34C9F5666FD4B0A87B67E2DDA50B9@hornigold.jaraco.com>
	<ea2499da0906040630v926e477g7cdd176ce70d8a56@mail.gmail.com>
	<79990c6b0906040720m1d31f2fuff863262b891928a@mail.gmail.com>
	<ea2499da0906040803t4d04267eua40e205ef897042e@mail.gmail.com>
	<4A28E5D7.2030402@gmail.com>
Message-ID: <79990c6b0906050614k6cfc4fdbmf4285367c8e447fa@mail.gmail.com>

2009/6/5 Nick Coghlan <ncoghlan at gmail.com>:
> Dirkjan Ochtman wrote:
>>> Essentially, pcbuild.sln is a binary file, and should be treated as
>>> such. Maybe it's an error in the Subversion setup that it's treated as
>>> text at all...
>>
>> Yes, it certainly seems that way.
>
> Except it isn't a binary file - it's a text file with CRLF line endings.
> Why would we throw away the chance to get meaningful diffs when
> Subversion can easily get the line endings correct?

Fair point - I hadn't thought of it in that context. I wonder if
Mercurial can treat it as a specialised form of binary file with
custom diff/merge behaviour (which just happens to be "treat as
text")? Of course, if that also requires client-side configuration,
it's no better solution than win32text.

I've a feeling I've seen discussions of versioned metadata (things
like file properties) in Mercurial at some point in the past, but my
Google skills are failing me at the moment...

Paul.

From status at bugs.python.org  Fri Jun  5 18:07:16 2009
From: status at bugs.python.org (Python tracker)
Date: Fri,  5 Jun 2009 18:07:16 +0200 (CEST)
Subject: [Python-Dev] Summary of Python tracker Issues
Message-ID: <20090605160716.09A8578632@psf.upfronthosting.co.za>


ACTIVITY SUMMARY (05/29/09 - 06/05/09)
Python tracker at http://bugs.python.org/

To view or respond to any of the issues listed below, click on the issue 
number.  Do NOT respond to this message.


 2207 open (+37) / 15822 closed (+27) / 18029 total (+64)

Open issues with patches:   868

Average duration of open issues: 653 days.
Median duration of open issues: 403 days.

Open Issues Breakdown
   open  2178 (+37)
pending    29 ( +0)

Issues Created Or Reopened (66)
_______________________________

string module requires bytes type for maketrans, but calling met 05/31/09
CLOSED http://bugs.python.org/issue5675    reopened rhettinger                    
                                                                               

IDLE - an extension to clear the shell window                    05/29/09
       http://bugs.python.org/issue6143    created  serwy                         
                                                                               

[IDLE] UnicodeDecodeError when invoking force-open-completions   05/29/09
       http://bugs.python.org/issue6144    created  jamesie                       
                                                                               

distutils.extension.read_setup_file misinterprets -C switch      05/29/09
       http://bugs.python.org/issue6145    created  donlorenzo                    
       patch                                                                   

markup error in Doc/library/rlcompleter.rst                      05/29/09
CLOSED http://bugs.python.org/issue6146    created  Trundle                       
       patch                                                                   

multithreading.Pool.map() crashes Windows computer               05/30/09
CLOSED http://bugs.python.org/issue6147    created  ac.james                      
                                                                               

Help well execute code it is called on                           05/30/09
CLOSED http://bugs.python.org/issue6148    created  imiers1                       
                                                                               

WeakValueDictionary constructor ported to Python 3.0 incorrectly 05/30/09
CLOSED http://bugs.python.org/issue6149    created  zzzeek                        
       patch                                                                   

test_unicode fails in wide unicode build                         05/30/09
CLOSED http://bugs.python.org/issue6150    created  hagen                         
                                                                               

Make PyDescr_COMMON conform to standard C                        05/30/09
       http://bugs.python.org/issue6151    created  alexandre.vassalotti          
       patch, patch, needs review                                              

Parallel regression testing                                      05/31/09
       http://bugs.python.org/issue6152    created  pitrou                        
       patch                                                                   

email parsing - Rare Failure                                     05/31/09
CLOSED http://bugs.python.org/issue6153    created  WinstonEwert                  
                                                                               

Python 3.1 rc1 will not build on OS X 10.5.7 with macports libin 05/31/09
       http://bugs.python.org/issue6154    created  brett.cannon                  
       patch                                                                   

"logging" example uses unavailable cPickle module                05/31/09
CLOSED http://bugs.python.org/issue6155    created  mnewman                       
                                                                               

Error compiling valid regex                                      05/31/09
       http://bugs.python.org/issue6156    created  Eloff                         
       patch                                                                   

Tkinter.Text: changes for bbox, debug, and edit methods.         05/31/09
       http://bugs.python.org/issue6157    created  gpolo                         
       patch                                                                   

test_aifc failing on 32bit windows in python 3.1 rc 1            05/31/09
CLOSED http://bugs.python.org/issue6158    created  rhettinger                    
       patch                                                                   

Tkinter.PanedWindow: docstring fixes, change in paneconfigure an 05/31/09
       http://bugs.python.org/issue6159    created  gpolo                         
       patch                                                                   

Tkinter.Spinbox: fix for bbox and removed some uninteresting ret 06/01/09
       http://bugs.python.org/issue6160    created  gpolo                         
       patch                                                                   

httplib: HTTPResponse not storing response body                  06/01/09
CLOSED http://bugs.python.org/issue6161    created  Guthur                        
                                                                               

What should happen to signals when the main thread dies?         06/01/09
       http://bugs.python.org/issue6162    created  eggy                          
                                                                               

[HP-UX] ld: Unrecognized argument: +s -L<dir>                    06/01/09
       http://bugs.python.org/issue6163    created  srid                          
       patch                                                                   

[AIX] Patch to correct the AIX C/C++ linker argument used for 'r 06/01/09
       http://bugs.python.org/issue6164    created  srid                          
       patch                                                                   

strftime incorrectly processes DST flag when passed time touples 06/01/09
       http://bugs.python.org/issue6165    created  jonathan.cervidae             
                                                                               

encoding error for 'setup.py --author' when read via subprocess  06/01/09
       http://bugs.python.org/issue6166    created  srid                          
                                                                               

Tkinter.Scrollbar: the activate method needs to return a value,  06/01/09
       http://bugs.python.org/issue6167    created  gpolo                         
       patch                                                                   

Missing Shell menu in Linux IDLE                                 06/01/09
       http://bugs.python.org/issue6168    created  sirgimp                       
                                                                               

Important comparison bug in ipaddr                               06/01/09
CLOSED http://bugs.python.org/issue6169    created  shields                       
                                                                               

Mac 'make frameworkinstall' error: [...]/Python.framework/Versio 06/02/09
       http://bugs.python.org/issue6170    created  srid                          
                                                                               

Class Browser selection in Ubuntu                                06/02/09
       http://bugs.python.org/issue6171    created  sirgimp                       
                                                                               

'make framework...' fails on Mac ([...]/bin/pythonw3.1: No such  06/02/09
CLOSED http://bugs.python.org/issue6172    created  srid                          
                                                                               

Minor typo in socket.py                                          06/02/09
CLOSED http://bugs.python.org/issue6173    created  ptn                           
       patch                                                                   

What's new in 2.6, wrong indentation in code sample              06/02/09
CLOSED http://bugs.python.org/issue6174    created  joe.amenta                    
       patch                                                                   

inet_aton documentation kind of lies                             06/02/09
CLOSED http://bugs.python.org/issue6175    created  dsvensson                     
                                                                               

Reference to a wrong version of flock's documentation            06/02/09
CLOSED http://bugs.python.org/issue6176    created  pakal                         
                                                                               

unittest.main testRunner default argument changed from None      06/02/09
CLOSED http://bugs.python.org/issue6177    created  michael.foord                 
       patch, easy                                                             

Core error in Py_EvalFrameEx 2.6.2                               06/02/09
       http://bugs.python.org/issue6178    created  tsavannah                     
                                                                               

Documentation of for statement in Reference says that range() re 06/02/09
CLOSED http://bugs.python.org/issue6179    created  MLModel                       
                                                                               

Tkinter.Entry: fix for xview and some doc clarifications         06/02/09
       http://bugs.python.org/issue6180    created  gpolo                         
       patch                                                                   

Tkinter.Listbox several minor issues                             06/02/09
       http://bugs.python.org/issue6181    created  gpolo                         
       patch                                                                   

Remove ipaddr library from py3k before 3.1rc2                    06/03/09
CLOSED http://bugs.python.org/issue6182    created  gregory.p.smith               
                                                                               

test_time fails on VC6                                           06/03/09
CLOSED http://bugs.python.org/issue6183    created  ocean-city                    
       patch                                                                   

py3k build gets spurious errors from libinstall target compile o 06/03/09
CLOSED http://bugs.python.org/issue6184    reopened nad                           
                                                                               

2to3 misses buffer slicing                                       06/03/09
CLOSED http://bugs.python.org/issue6185    created  joe.amenta                    
       patch                                                                   

test_thread occasionally reports unhandled exceptions on OS X    06/03/09
       http://bugs.python.org/issue6186    created  nad                           
                                                                               

Improvement in doc of "Extending and Embedding the Python Interp 06/03/09
       http://bugs.python.org/issue6187    created  blubdiebla                    
                                                                               

Error Evaluating float(x) ** float(y)                            06/03/09
CLOSED http://bugs.python.org/issue6188    created  Zero                          
                                                                               

subprocess module not compatible with python 2.5                 06/03/09
       http://bugs.python.org/issue6189    created  itsadok                       
       patch                                                                   

Tutorial 3.1.2 (Strings) has duplicate lines                     06/03/09
CLOSED http://bugs.python.org/issue6190    created  cofi                          
       patch                                                                   

HTMLParser attribute parsing - 2 test cases when it fails        06/04/09
       http://bugs.python.org/issue6191    reopened momat                         
                                                                               

add disable_nagle_algorithm to SocketServer.TCPServer            06/04/09
       http://bugs.python.org/issue6192    created  krisvale                      
       patch, patch, easy                                                      

urllib: ... IOError: ... unknown url type: 'c'                   06/04/09
CLOSED http://bugs.python.org/issue6193    created  srid                          
                                                                               

fcntl footnote about O_SHLOCK and O_EXLOCK is misleading         06/04/09
       http://bugs.python.org/issue6194    created  r.david.murray                
       easy                                                                    

Serious regression in doctest in Py3.1rc1                        06/04/09
       http://bugs.python.org/issue6195    created  dalcinl                       
       patch                                                                   

tarfile.extractall(readaccess=True)                              06/04/09
       http://bugs.python.org/issue6196    created  srid                          
                                                                               

test__locale fails with RADIXCHAR on Windows                     06/05/09
CLOSED http://bugs.python.org/issue6197    created  abbeyj                        
       patch                                                                   

test_float fails on Windows                                      06/05/09
CLOSED http://bugs.python.org/issue6198    created  abbeyj                        
                                                                               

test_unittest fails on Windows                                   06/05/09
CLOSED http://bugs.python.org/issue6199    created  abbeyj                        
       patch                                                                   

test_winreg fails                                                06/05/09
       http://bugs.python.org/issue6201    created  abbeyj                        
                                                                               

Obsolete default file encoding "mac-roman" on OS X, not influenc 06/05/09
       http://bugs.python.org/issue6202    created  nad                           
                                                                               

3.x locale does not default to C, contrary to the documentation  06/05/09
       http://bugs.python.org/issue6203    created  nad                           
                                                                               

Missing reference in section 4.6 to chapter on classes           06/05/09
       http://bugs.python.org/issue6204    created  cofi                          
       patch                                                                   

sdist doesn't include data_files                                 06/05/09
       http://bugs.python.org/issue6205    created  purpleidea                    
                                                                               

Correct a trivial typo introduced by r73238.                     06/05/09
       http://bugs.python.org/issue6206    created  vshenoy                       
       patch                                                                   

Simple For-Loops                                                 06/05/09
       http://bugs.python.org/issue6207    created  gabrielkfl                    
                                                                               

Function-level import in os triggering an threaded import deadlo 06/04/09
       http://bugs.python.org/issue1590864 reopened brett.cannon                  
                                                                               



Issues Now Closed (67)
______________________

Load tests from path (patch included)                             615 days
       http://bugs.python.org/issue1207    michael.foord                 
       patch                                                                   

S_unpack_from() Read Access Violation                             421 days
       http://bugs.python.org/issue2590    amaury.forgeotdarc            
                                                                               

Py3k warn on unicode escapes in raw strings                       401 days
       http://bugs.python.org/issue2713    amaury.forgeotdarc            
       patch                                                                   

example code does not work                                        392 days
       http://bugs.python.org/issue2742    fogus                         
       patch                                                                   

time.strftime() always decodes result with UTF-8                  356 days
       http://bugs.python.org/issue3061    loewis                        
       patch                                                                   

pkg-config support                                                285 days
       http://bugs.python.org/issue3585    loewis                        
       patch, needs review                                                     

base64.encodestring does not actually accept strings              288 days
       http://bugs.python.org/issue3613    georg.brandl                  
       patch                                                                   

Python 2.6 can't read sets pickled with Python 3.0                283 days
       http://bugs.python.org/issue3675    alexandre.vassalotti          
       patch                                                                   

traceback.format_exception_only() misplaces the caret for certai  283 days
       http://bugs.python.org/issue3684    georg.brandl                  
       patch                                                                   

bsddb not completely removed                                      271 days
       http://bugs.python.org/issue3791    georg.brandl                  
       patch, patch                                                            

select.epoll calling register with the same fd fails              261 days
       http://bugs.python.org/issue3848    r.david.murray                
       patch                                                                   

Add Google's ipaddr.py to the stdlib                              251 days
       http://bugs.python.org/issue3959    belopolsky                    
                                                                               

Long jumps with frame_setlineno                                   178 days
       http://bugs.python.org/issue4547    amaury.forgeotdarc            
       patch, needs review                                                     

Refcount error and file descriptor leaks in pwd, grp modules      142 days
       http://bugs.python.org/issue4873    loewis                        
       patch                                                                   

Subversion problem with PythonLauncher after building 3.0 or 3.1  109 days
       http://bugs.python.org/issue5267    ronaldoussoren                
                                                                               

profile and cProfile do not report C functions called with keywo   99 days
       http://bugs.python.org/issue5330    pitrou                        
       patch                                                                   

TypeError when '\x00' in docstring                                 98 days
       http://bugs.python.org/issue5373    alexandre.vassalotti          
       patch                                                                   

Locale-based date formatting crashes on non-ASCII data             65 days
       http://bugs.python.org/issue5562    loewis                        
                                                                               

test_memoryio fails for py3k on windows                            65 days
       http://bugs.python.org/issue5645    jaraco                        
       patch                                                                   

OS X Installer: do not install obsolete documentation within Pyt   64 days
       http://bugs.python.org/issue5648    ronaldoussoren                
                                                                               

string module requires bytes type for maketrans, but calling met    0 days
       http://bugs.python.org/issue5675    georg.brandl                  
                                                                               

Segfault when loading not recompiled module                        51 days
       http://bugs.python.org/issue5735    amaury.forgeotdarc            
       patch, needs review                                                     

xmlrpclib loads invalid documents                                  49 days
       http://bugs.python.org/issue5767    georg.brandl                  
       patch                                                                   

test_asynchat fails on Mac OSX                                     46 days
       http://bugs.python.org/issue5798    MrJean1                       
       patch                                                                   

strftime fails in non UTF-8 locale                                 27 days
       http://bugs.python.org/issue5903    loewis                        
                                                                               

Multi-with patch                                                   26 days
       http://bugs.python.org/issue5922    Merwok                        
       patch                                                                   

PyList_GetSlice does not indicate negative ranges dont work as i   27 days
       http://bugs.python.org/issue5967    georg.brandl                  
       patch                                                                   

Test discovery for unittest                                        18 days
       http://bugs.python.org/issue6001    michael.foord                 
       patch, patch, needs review                                              

test_distutils leaves a 'foo' file behind in the cwd               16 days
       http://bugs.python.org/issue6022    rpetrov                       
                                                                               

Python fails to build with Subversion 1.7                           7 days
       http://bugs.python.org/issue6094    benjamin.peterson             
       patch                                                                   

Encoded surrogate characters on command line not escaped in sys.    5 days
       http://bugs.python.org/issue6097    loewis                        
       patch                                                                   

Expanding arrays inside other arrays                                5 days
       http://bugs.python.org/issue6100    tjreedy                       
                                                                               

When the package has non-ascii path and .pyc file, we cannot imp    5 days
       http://bugs.python.org/issue6102    tjreedy                       
                                                                               

IDLE rendering issue with oriental characters on OSX                7 days
       http://bugs.python.org/issue6109    benjamin.peterson             
                                                                               

Fix O(n**2) performance problem in socket._fileobject               7 days
       http://bugs.python.org/issue6117    krisvale                      
       patch, patch, easy, needs review                                        

Typo in email.base64mime                                            1 days
       http://bugs.python.org/issue6139    georg.brandl                  
       patch                                                                   

missing first argument on subprocess.Popen w/ executable            0 days
       http://bugs.python.org/issue6141    r.david.murray                
       patch                                                                   

markup error in Doc/library/rlcompleter.rst                         0 days
       http://bugs.python.org/issue6146    georg.brandl                  
       patch                                                                   

multithreading.Pool.map() crashes Windows computer                  2 days
       http://bugs.python.org/issue6147    benjamin.peterson             
                                                                               

Help well execute code it is called on                              0 days
       http://bugs.python.org/issue6148    georg.brandl                  
                                                                               

WeakValueDictionary constructor ported to Python 3.0 incorrectly    0 days
       http://bugs.python.org/issue6149    pitrou                        
       patch                                                                   

test_unicode fails in wide unicode build                            1 days
       http://bugs.python.org/issue6150    loewis                        
                                                                               

email parsing - Rare Failure                                        2 days
       http://bugs.python.org/issue6153    amaury.forgeotdarc            
                                                                               

"logging" example uses unavailable cPickle module                   0 days
       http://bugs.python.org/issue6155    georg.brandl                  
                                                                               

test_aifc failing on 32bit windows in python 3.1 rc 1               0 days
       http://bugs.python.org/issue6158    loewis                        
       patch                                                                   

httplib: HTTPResponse not storing response body                     0 days
       http://bugs.python.org/issue6161    Guthur                        
                                                                               

Important comparison bug in ipaddr                                  0 days
       http://bugs.python.org/issue6169    gregory.p.smith               
                                                                               

'make framework...' fails on Mac ([...]/bin/pythonw3.1: No such     0 days
       http://bugs.python.org/issue6172    ronaldoussoren                
                                                                               

Minor typo in socket.py                                             0 days
       http://bugs.python.org/issue6173    marketdickinson               
       patch                                                                   

What's new in 2.6, wrong indentation in code sample                 2 days
       http://bugs.python.org/issue6174    georg.brandl                  
       patch                                                                   

inet_aton documentation kind of lies                                2 days
       http://bugs.python.org/issue6175    georg.brandl                  
                                                                               

Reference to a wrong version of flock's documentation               2 days
       http://bugs.python.org/issue6176    georg.brandl                  
                                                                               

unittest.main testRunner default argument changed from None         0 days
       http://bugs.python.org/issue6177    michael.foord                 
       patch, easy                                                             

Documentation of for statement in Reference says that range() re    0 days
       http://bugs.python.org/issue6179    benjamin.peterson             
                                                                               

Remove ipaddr library from py3k before 3.1rc2                       2 days
       http://bugs.python.org/issue6182    rhettinger                    
                                                                               

test_time fails on VC6                                              0 days
       http://bugs.python.org/issue6183    ocean-city                    
       patch                                                                   

py3k build gets spurious errors from libinstall target compile o    1 days
       http://bugs.python.org/issue6184    benjamin.peterson             
                                                                               

2to3 misses buffer slicing                                          1 days
       http://bugs.python.org/issue6185    benjamin.peterson             
       patch                                                                   

Error Evaluating float(x) ** float(y)                               0 days
       http://bugs.python.org/issue6188    marketdickinson               
                                                                               

Tutorial 3.1.2 (Strings) has duplicate lines                        0 days
       http://bugs.python.org/issue6190    georg.brandl                  
       patch                                                                   

urllib: ... IOError: ... unknown url type: 'c'                      0 days
       http://bugs.python.org/issue6193    r.david.murray                
                                                                               

test__locale fails with RADIXCHAR on Windows                        0 days
       http://bugs.python.org/issue6197    ocean-city                    
       patch                                                                   

test_float fails on Windows                                         0 days
       http://bugs.python.org/issue6198    eric.smith                    
                                                                               

test_unittest fails on Windows                                      0 days
       http://bugs.python.org/issue6199    michael.foord                 
       patch                                                                   

SRE: (?flag:...) is not supported                                2912 days
       http://bugs.python.org/issue433028  georg.brandl                  
                                                                               

Update pickle docs to describe format of persistent IDs          1838 days
       http://bugs.python.org/issue956303  alexandre.vassalotti          
       easy                                                                    

hide tests from TestProgram                                      1408 days
       http://bugs.python.org/issue1244929 michael.foord                 
       patch                                                                   



Top Issues Most Discussed (10)
______________________________

 33 Python 3.1 rc1 will not build on OS X 10.5.7 with macports libi    5 days
open    http://bugs.python.org/issue6154   

 32 improved allocation of PyUnicode objects                          12 days
open    http://bugs.python.org/issue1943   

 27 Add Google's ipaddr.py to the stdlib                             251 days
closed  http://bugs.python.org/issue3959   

 17 Mac 'make frameworkinstall' error: [...]/Python.framework/Versi    4 days
open    http://bugs.python.org/issue6170   

 15 pydoc reports misleading failure if target module raises an Imp  113 days
open    http://bugs.python.org/issue5230   

 14 Make pickle generated by Python 3.x compatible with 2.x and vic    8 days
open    http://bugs.python.org/issue6137   

 10 Distutils doesn't remove .pyc files                                7 days
open    http://bugs.python.org/issue6142   

 10 LOAD_CONST followed by LOAD_ATTR can be optimized to just	be a     9 days
open    http://bugs.python.org/issue6133   

  9 test_float fails on Windows                                        0 days
closed  http://bugs.python.org/issue6198   

  8 test_asynchat fails on Mac OSX                                    46 days
closed  http://bugs.python.org/issue5798   




From db3l.net at gmail.com  Fri Jun  5 21:38:32 2009
From: db3l.net at gmail.com (David Bolen)
Date: Fri, 05 Jun 2009 15:38:32 -0400
Subject: [Python-Dev] py3k buildbots
References: <loom.20090605T071922-555@post.gmane.org>
Message-ID: <m2r5xyii87.fsf@valheru.db3l.homeip.net>

Antoine Pitrou <solipsis at pitrou.net> writes:

> Only one of the py3k buildbots seems up:
> http://www.python.org/dev/buildbot/3.x.stable/

Strange - everything looks good on my buildbot end (XP-4), including
an established TCP session back to dinsdale.  Not sure why the master
thinks it's offline.  Although I'm pretty sure I've seen this a handful
of times in the past too.

The fact that it also shows my FreeBSD buildbot offline (which I
rarely have to touch compared to the Windows version and they're on
the same host machine) seems to imply some wierd communication
breakdown.

I've restarted my buildbots.

-- David


From theller at ctypes.org  Fri Jun  5 21:52:46 2009
From: theller at ctypes.org (Thomas Heller)
Date: Fri, 05 Jun 2009 21:52:46 +0200
Subject: [Python-Dev] py3k buildbots
In-Reply-To: <m2r5xyii87.fsf@valheru.db3l.homeip.net>
References: <loom.20090605T071922-555@post.gmane.org>
	<m2r5xyii87.fsf@valheru.db3l.homeip.net>
Message-ID: <h0bt2e$1g3$1@ger.gmane.org>

David Bolen schrieb:
> Antoine Pitrou <solipsis at pitrou.net> writes:
> 
>> Only one of the py3k buildbots seems up:
>> http://www.python.org/dev/buildbot/3.x.stable/
> 
> Strange - everything looks good on my buildbot end (XP-4), including
> an established TCP session back to dinsdale.  Not sure why the master
> thinks it's offline.  Although I'm pretty sure I've seen this a handful
> of times in the past too.
> 
> The fact that it also shows my FreeBSD buildbot offline (which I
> rarely have to touch compared to the Windows version and they're on
> the same host machine) seems to imply some wierd communication
> breakdown.
> 
> I've restarted my buildbots.

Well, here is an excerpt from the twistd.log of my osx buildbot, in case someone
want to look at it.  I restarted it this morning, at 2009/06/05 11:21 +0200:

2009/06/01 20:20 +0200 [Broker,client] SlaveBuilder._ackFailed: SlaveBuilder.sendUpdate
2009/06/01 20:20 +0200 [Broker,client] SlaveBuilder._ackFailed: SlaveBuilder.sendUpdate
2009/06/01 20:20 +0200 [Broker,client] SlaveBuilder._ackFailed: SlaveBuilder.sendUpdate
2009/06/01 20:20 +0200 [Broker,client] lost remote
2009/06/01 20:20 +0200 [Broker,client] lost remote
2009/06/01 20:20 +0200 [Broker,client] lost remote
2009/06/01 20:20 +0200 [Broker,client] lost remote
2009/06/01 20:20 +0200 [Broker,client] lost remote step
2009/06/01 20:20 +0200 [Broker,client] stopCommand: halting current command <buildbot.slave.commands.SlaveShellCommand instance at 0x10820d0>
2009/06/01 20:20 +0200 [Broker,client] command interrupted, killing pid 21579
2009/06/01 20:20 +0200 [Broker,client] trying os.kill(-pid, 9)
2009/06/01 20:20 +0200 [Broker,client]  signal 9 sent successfully
2009/06/01 20:20 +0200 [Broker,client] <twisted.internet.tcp.Connector instance at 0x7930a8> will retry in 3 seconds
2009/06/01 20:20 +0200 [Broker,client] Stopping factory <buildbot.slave.bot.BotFactory instance at 0x10080f8>
2009/06/01 20:20 +0200 [-] Unhandled Error
        Traceback (most recent call last):
          File "/System/Library/Frameworks/Python.framework/Versions/2.5/Extras/lib/python/twisted/python/log.py", line 48, in callWithLogger
            return callWithContext({"system": lp}, func, *args, **kw)
          File "/System/Library/Frameworks/Python.framework/Versions/2.5/Extras/lib/python/twisted/python/log.py", line 33, in callWithContext
            return context.call({ILogContext: newCtx}, func, *args, **kw)
          File "/System/Library/Frameworks/Python.framework/Versions/2.5/Extras/lib/python/twisted/python/context.py", line 59, in callWithContext
            return self.currentContext().callWithContext(ctx, func, *args, **kw)
          File "/System/Library/Frameworks/Python.framework/Versions/2.5/Extras/lib/python/twisted/python/context.py", line 37, in callWithContext
            return func(*args,**kw)
        --- <exception caught here> ---
          File "/System/Library/Frameworks/Python.framework/Versions/2.5/Extras/lib/python/twisted/internet/selectreactor.py", line 139, in _doReadOrWrite
            why = getattr(selectable, method)()
          File "/System/Library/Frameworks/Python.framework/Versions/2.5/Extras/lib/python/twisted/internet/abstract.py", line 113, in doWrite
            l = self.writeSomeData(self.dataBuffer)
          File "/System/Library/Frameworks/Python.framework/Versions/2.5/Extras/lib/python/twisted/internet/process.py", line 879, in writeSomeData
            return os.write(self.fd, data)
        exceptions.OSError: [Errno 5] Input/output error

2009/06/01 20:20 +0200 [-] command finished with signal 9, exit code None
2009/06/01 20:20 +0200 [-] SlaveBuilder.commandComplete None
2009/06/01 20:20 +0200 [-] Starting factory <buildbot.slave.bot.BotFactory instance at 0x10080f8>
2009/06/05 11:21 +0200 [-] Received SIGTERM, shutting down.
2009/06/05 11:21 +0200 [Broker,client] Stopping factory <buildbot.slave.bot.BotFactory instance at 0x10080f8>
2009/06/05 11:21 +0200 [-] Main loop terminated.
2009/06/05 11:21 +0200 [-] Server Shut Down.
2009/06/05 11:21 +0200 [-] Log opened.
2009/06/05 11:21 +0200 [-] twistd 2.5.0 (/System/Library/Frameworks/Python.framework/Versions/2.5/Resources/Python.app/Contents/MacOS/Python 2.5.1) starting up
2009/06/05 11:21 +0200 [-] reactor class: <class 'twisted.internet.selectreactor.SelectReactor'>
2009/06/05 11:21 +0200 [-] Loading buildbot.tac...


-- 
Thanks,
Thomas


From dirkjan at ochtman.nl  Fri Jun  5 22:48:15 2009
From: dirkjan at ochtman.nl (Dirkjan Ochtman)
Date: Fri, 5 Jun 2009 22:48:15 +0200
Subject: [Python-Dev] Draft PEP 385: Migrating from svn to Mercurial
Message-ID: <ea2499da0906051348g20c204d1rdec62c4b95b1697e@mail.gmail.com>

So, a while ago Martin von L?wis asked who would champion the
migration to Mercurial, and I volunteered. He asked me to produce a
PEP outlining which steps to take, which I've been working on. The PEP
is numbered 385, and is just about ready for your review. I'd really
welcome any sort of feedback or questions about Mercurial and
surrounding tools relevant to the migration project (I'm happy to
answer other questions as well, but maybe python-dev is not the
appropriate forum for that -- instead, try mercurial at selenic.com
and/or #mercurial on freenode).

http://www.python.org/dev/peps/pep-0385/

In particular, you may want to review (a) branches that you care about
and whether they would be migrated under the current proposal, (b)
non-release tags that you think are useful and (c) the author map
(check if it has a correct email address for you and people you know).

Thanks to Antoine, Benjamin, Georg and Martin for early feedback.

Cheers,

Dirkjan

From benjamin at python.org  Fri Jun  5 22:58:19 2009
From: benjamin at python.org (Benjamin Peterson)
Date: Fri, 5 Jun 2009 15:58:19 -0500
Subject: [Python-Dev] Draft PEP 385: Migrating from svn to Mercurial
In-Reply-To: <ea2499da0906051348g20c204d1rdec62c4b95b1697e@mail.gmail.com>
References: <ea2499da0906051348g20c204d1rdec62c4b95b1697e@mail.gmail.com>
Message-ID: <1afaf6160906051358n25a833f0vc1be58a3598be88d@mail.gmail.com>

2009/6/5 Dirkjan Ochtman <dirkjan at ochtman.nl>:
>
> In particular, you may want to review (a) branches that you care about
> and whether they would be migrated under the current proposal, (b)
> non-release tags that you think are useful and (c) the author map
> (check if it has a correct email address for you and people you know).

Thanks very much for doing this!

The PEP should talk about tracker migration. ie. how to handle
specifying revisions on multiple clones if we choose that path.


-- 
Regards,
Benjamin

From solipsis at pitrou.net  Sat Jun  6 00:35:59 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Fri, 5 Jun 2009 22:35:59 +0000 (UTC)
Subject: [Python-Dev] Draft PEP 385: Migrating from svn to Mercurial
References: <ea2499da0906051348g20c204d1rdec62c4b95b1697e@mail.gmail.com>
Message-ID: <loom.20090605T222758-438@post.gmane.org>

Dirkjan Ochtman <dirkjan <at> ochtman.nl> writes:
> 
> http://www.python.org/dev/peps/pep-0385/

? [cloned branches] makes it easier to distinguish branches, at the expense of
requiring more disk space on the client. ?

This is a bit misleading. Actually, by separating branches into distinct
repositories, you can save quite a bit of repository space. It really depends on
the structure of the project and your own workflow.

? This reflects that the default branch in hg is called 'default' instead of
Subversion's 'trunk', and reflects the proposed new tag format. ?

If we use several distinct repositories, the default branch should instead
mirror the repository name (like what is done in the current hg mirrors:
"trunk", "py3k", etc.).

Regards

Antoine.



From techtonik at gmail.com  Sat Jun  6 01:59:24 2009
From: techtonik at gmail.com (anatoly techtonik)
Date: Sat, 6 Jun 2009 02:59:24 +0300
Subject: [Python-Dev] Roundup keywords for bug tracking
Message-ID: <d34314100906051659r4e89788bybfeeed8f6e854366@mail.gmail.com>

It is impossible to edit roundup keywords and this takes away the
flexibility in selecting bugs related to a module/function/test or
some other aspect of development. For example, I need to gather all
subprocess bugs in one query and things that won't be fixed in
deprecated os.popen() into another. In Trac I would use "subprocess"
and "os.popen" keywords. On ohloh I would add similar tags (if bugs
were software) without, but I can't do anything about Python roundup.
Is there any reason for such restriction?

--anatoly t.

From tjreedy at udel.edu  Sat Jun  6 02:42:14 2009
From: tjreedy at udel.edu (Terry Reedy)
Date: Fri, 05 Jun 2009 20:42:14 -0400
Subject: [Python-Dev] Roundup keywords for bug tracking
In-Reply-To: <d34314100906051659r4e89788bybfeeed8f6e854366@mail.gmail.com>
References: <d34314100906051659r4e89788bybfeeed8f6e854366@mail.gmail.com>
Message-ID: <h0ce16$chh$1@ger.gmane.org>

anatoly techtonik wrote:
> It is impossible to edit roundup keywords and this takes away the
> flexibility in selecting bugs related to a module/function/test or
> some other aspect of development. For example, I need to gather all
> subprocess bugs in one query 

At the moment, search for 'subprocess' in text and component = library 
returns 53 open issues.  A quick scan of the titles suggests that about 
40 are for the subprocess module.  So changing the search to 
'subprocess' in title might be better.


and things that won't be fixed in
> deprecated os.popen() into another. In Trac I would use "subprocess"
> and "os.popen" keywords. On ohloh I would add similar tags (if bugs
> were software) without, but I can't do anything about Python roundup.
> Is there any reason for such restriction?

If every module and built-in and keyword were added to the drop-down 
list, it would be pretty long.  But I agree that better sorting could 
help. G. Polo, for one, made his own app to download canned search 
results (for tk and idle) and store them in a local db, where he could 
add annotations.

tjr


From ajaksu at gmail.com  Sat Jun  6 03:07:44 2009
From: ajaksu at gmail.com (Daniel Diniz)
Date: Fri, 5 Jun 2009 22:07:44 -0300
Subject: [Python-Dev] Roundup keywords for bug tracking
In-Reply-To: <d34314100906051659r4e89788bybfeeed8f6e854366@mail.gmail.com>
References: <d34314100906051659r4e89788bybfeeed8f6e854366@mail.gmail.com>
Message-ID: <2d75d7660906051807s3ca051fcw5af5841e99cd707a@mail.gmail.com>

anatoly techtonik wrote:
>
> It is impossible to edit roundup keywords and this takes away the
> flexibility in selecting bugs related to a module/function/test or
> some other aspect of development. For example, I need to gather all
> subprocess bugs in one query and things that won't be fixed in
> deprecated os.popen() into another. In Trac I would use "subprocess"
> and "os.popen" keywords. On ohloh I would add similar tags (if bugs
> were software) without, but I can't do anything about Python roundup.
> Is there any reason for such restriction?

Well, keywords are used as a very restricted set of tags, so only
users in the Developer group can create them. We've discussed free
form issue tags that any user can create or edit in #python-dev and
tracker-discuss[0]. I'm pretty sure they'd cover your use-case. I've
submitted a patch to Rietveld[1], but it seems I never filled it in
the meta-tracker, oopsie.

If you (or anyone else) want to test-drive the tags feature, I can
create an account in the experimental tracker[2] (which needs some
attention anyway). I should be able to submit the patch to the
meta-tracker during the weekend.

Also, if you would like to bookmark arbitrary sets of issues, the
bookmarklet and form in http://static.bot.bio.br/tool.html may be of
help. You can paste the ids into the search page's ID field and create
a query for a given (static) set of issues.

Regards,
Daniel

[0] http://mail.python.org/pipermail/tracker-discuss/2009-April/002099.html
[1] http://codereview.appspot.com/40100/show
[2] http://bot.bio.br/python-dev-exp/issue5

From skippy.hammond at gmail.com  Sat Jun  6 03:20:54 2009
From: skippy.hammond at gmail.com (Mark Hammond)
Date: Sat, 06 Jun 2009 11:20:54 +1000
Subject: [Python-Dev] Mercurial, linefeeds, and Visual Studio
In-Reply-To: <79990c6b0906050614k6cfc4fdbmf4285367c8e447fa@mail.gmail.com>
References: <8B473FAE8A08C34C9F5666FD4B0A87B67E2DDA50B9@hornigold.jaraco.com>	<ea2499da0906040630v926e477g7cdd176ce70d8a56@mail.gmail.com>	<79990c6b0906040720m1d31f2fuff863262b891928a@mail.gmail.com>	<ea2499da0906040803t4d04267eua40e205ef897042e@mail.gmail.com>	<4A28E5D7.2030402@gmail.com>
	<79990c6b0906050614k6cfc4fdbmf4285367c8e447fa@mail.gmail.com>
Message-ID: <4A29C476.6060101@gmail.com>

Paul Moore wrote:
> 2009/6/5 Nick Coghlan <ncoghlan at gmail.com>:
>> Dirkjan Ochtman wrote:
>>>> Essentially, pcbuild.sln is a binary file, and should be treated as
>>>> such. Maybe it's an error in the Subversion setup that it's treated as
>>>> text at all...
>>> Yes, it certainly seems that way.
>> Except it isn't a binary file - it's a text file with CRLF line endings.
>> Why would we throw away the chance to get meaningful diffs when
>> Subversion can easily get the line endings correct?
> 
> Fair point - I hadn't thought of it in that context. I wonder if
> Mercurial can treat it as a specialised form of binary file with
> custom diff/merge behaviour (which just happens to be "treat as
> text")? Of course, if that also requires client-side configuration,
> it's no better solution than win32text.
> 
> I've a feeling I've seen discussions of versioned metadata (things
> like file properties) in Mercurial at some point in the past, but my
> Google skills are failing me at the moment...

I've looked into this recently.  IIUC:

* There has been some limited support expressed for having win32text 
look in a normal versioned file for hints about the current repo.

* Once concern here was the fact that such config files are somewhat 
'untrusted', but the hg config parsing code has recently been 
reimplemented to support the concept of 'trusted' versus 'untrusted' 
options.  I'm not sure what this means in practice though.

* There isn't currently much activity on win32text.  None of the core hg 
devs seem to use Windows, so while they are receptive to Windows 
patches, it might be necessary to be quite noisy to get attention.

I recently submitted some changes to the filtering for Windows which 
were accepted without any angst, and the ability to use a versioned file 
for per-repo rules is something I'd like too.  I believe that if a few 
Windows users got together and agreed on both semantics and 
implementation we could get such an enhancement landed in the core of hg...

Cheers,

Mark


From p.f.moore at gmail.com  Sat Jun  6 14:25:11 2009
From: p.f.moore at gmail.com (Paul Moore)
Date: Sat, 6 Jun 2009 13:25:11 +0100
Subject: [Python-Dev] Mercurial, linefeeds, and Visual Studio
In-Reply-To: <4A29C476.6060101@gmail.com>
References: <8B473FAE8A08C34C9F5666FD4B0A87B67E2DDA50B9@hornigold.jaraco.com>
	<ea2499da0906040630v926e477g7cdd176ce70d8a56@mail.gmail.com>
	<79990c6b0906040720m1d31f2fuff863262b891928a@mail.gmail.com>
	<ea2499da0906040803t4d04267eua40e205ef897042e@mail.gmail.com>
	<4A28E5D7.2030402@gmail.com>
	<79990c6b0906050614k6cfc4fdbmf4285367c8e447fa@mail.gmail.com>
	<4A29C476.6060101@gmail.com>
Message-ID: <79990c6b0906060525ga75ab3bmf2220169b92c34ba@mail.gmail.com>

2009/6/6 Mark Hammond <skippy.hammond at gmail.com>:
> Paul Moore wrote:
>>
>> 2009/6/5 Nick Coghlan <ncoghlan at gmail.com>:
>>>
>>> Dirkjan Ochtman wrote:
>>>>>
>>>>> Essentially, pcbuild.sln is a binary file, and should be treated as
>>>>> such. Maybe it's an error in the Subversion setup that it's treated as
>>>>> text at all...
>>>>
>>>> Yes, it certainly seems that way.
>>>
>>> Except it isn't a binary file - it's a text file with CRLF line endings.
>>> Why would we throw away the chance to get meaningful diffs when
>>> Subversion can easily get the line endings correct?
>>
>> Fair point - I hadn't thought of it in that context. I wonder if
>> Mercurial can treat it as a specialised form of binary file with
>> custom diff/merge behaviour (which just happens to be "treat as
>> text")? Of course, if that also requires client-side configuration,
>> it's no better solution than win32text.
>>
>> I've a feeling I've seen discussions of versioned metadata (things
>> like file properties) in Mercurial at some point in the past, but my
>> Google skills are failing me at the moment...
>
> I've looked into this recently. ?IIUC:
>
> * There has been some limited support expressed for having win32text look in
> a normal versioned file for hints about the current repo.
>
> * Once concern here was the fact that such config files are somewhat
> 'untrusted', but the hg config parsing code has recently been reimplemented
> to support the concept of 'trusted' versus 'untrusted' options. ?I'm not
> sure what this means in practice though.
>
> * There isn't currently much activity on win32text. ?None of the core hg
> devs seem to use Windows, so while they are receptive to Windows patches, it
> might be necessary to be quite noisy to get attention.
>
> I recently submitted some changes to the filtering for Windows which were
> accepted without any angst, and the ability to use a versioned file for
> per-repo rules is something I'd like too. ?I believe that if a few Windows
> users got together and agreed on both semantics and implementation we could
> get such an enhancement landed in the core of hg...

I'm willing to help on this, but I don't personally use win32text (my
approach is to have all files always in Unix line-ending format, which
every tool I use handles fine). So take any design suggestions I might
make with a pinch of salt :-)

Paul.

From g.brandl at gmx.net  Sat Jun  6 19:43:44 2009
From: g.brandl at gmx.net (Georg Brandl)
Date: Sat, 06 Jun 2009 19:43:44 +0200
Subject: [Python-Dev] Draft PEP 385: Migrating from svn to Mercurial
In-Reply-To: <loom.20090605T222758-438@post.gmane.org>
References: <ea2499da0906051348g20c204d1rdec62c4b95b1697e@mail.gmail.com>
	<loom.20090605T222758-438@post.gmane.org>
Message-ID: <h0e9u8$8n8$1@ger.gmane.org>

Antoine Pitrou schrieb:
> Dirkjan Ochtman <dirkjan <at> ochtman.nl> writes:
>> 
>> http://www.python.org/dev/peps/pep-0385/
> 
> ? [cloned branches] makes it easier to distinguish branches, at the expense of
> requiring more disk space on the client. ?
> 
> This is a bit misleading. Actually, by separating branches into distinct
> repositories, you can save quite a bit of repository space. It really depends on
> the structure of the project and your own workflow.

If you follow the "strict-subset" policy (which I would strongly recommend, from
my experience) you'll end up with both branches in the trunk repo anyway.

> ? This reflects that the default branch in hg is called 'default' instead of
> Subversion's 'trunk', and reflects the proposed new tag format. ?
>
> If we use several distinct repositories, the default branch should instead
> mirror the repository name (like what is done in the current hg mirrors:
> "trunk", "py3k", etc.).

+1.

Georg

-- 
Thus spake the Lord: Thou shalt indent with four spaces. No more, no less.
Four shall be the number of spaces thou shalt indent, and the number of thy
indenting shall be four. Eight shalt thou not indent, nor either indent thou
two, excepting that thou then proceed to four. Tabs are right out.


From g.brandl at gmx.net  Sat Jun  6 19:45:11 2009
From: g.brandl at gmx.net (Georg Brandl)
Date: Sat, 06 Jun 2009 19:45:11 +0200
Subject: [Python-Dev] Roundup keywords for bug tracking
In-Reply-To: <2d75d7660906051807s3ca051fcw5af5841e99cd707a@mail.gmail.com>
References: <d34314100906051659r4e89788bybfeeed8f6e854366@mail.gmail.com>
	<2d75d7660906051807s3ca051fcw5af5841e99cd707a@mail.gmail.com>
Message-ID: <h0ea10$8n8$2@ger.gmane.org>

Daniel Diniz schrieb:
> anatoly techtonik wrote:
>>
>> It is impossible to edit roundup keywords and this takes away the
>> flexibility in selecting bugs related to a module/function/test or
>> some other aspect of development. For example, I need to gather all
>> subprocess bugs in one query and things that won't be fixed in
>> deprecated os.popen() into another. In Trac I would use "subprocess"
>> and "os.popen" keywords. On ohloh I would add similar tags (if bugs
>> were software) without, but I can't do anything about Python roundup.
>> Is there any reason for such restriction?
> 
> Well, keywords are used as a very restricted set of tags, so only
> users in the Developer group can create them. We've discussed free
> form issue tags that any user can create or edit in #python-dev and
> tracker-discuss[0]. I'm pretty sure they'd cover your use-case. I've
> submitted a patch to Rietveld[1], but it seems I never filled it in
> the meta-tracker, oopsie.
> 
> If you (or anyone else) want to test-drive the tags feature, I can
> create an account in the experimental tracker[2] (which needs some
> attention anyway). I should be able to submit the patch to the
> meta-tracker during the weekend.

I'm not sure it will get well tested on the experimental tracker.  If it
basically works, and no one has any real objections, I'd just add it to
the live tracker and try out how and if it is used.

Georg

-- 
Thus spake the Lord: Thou shalt indent with four spaces. No more, no less.
Four shall be the number of spaces thou shalt indent, and the number of thy
indenting shall be four. Eight shalt thou not indent, nor either indent thou
two, excepting that thou then proceed to four. Tabs are right out.


From ctb at msu.edu  Sun Jun  7 05:16:09 2009
From: ctb at msu.edu (C. Titus Brown)
Date: Sat, 6 Jun 2009 20:16:09 -0700
Subject: [Python-Dev] Snakebite (was: Re:  py3k buildbots)
In-Reply-To: <h0aocb$8rr$1@ger.gmane.org>
References: <loom.20090605T071922-555@post.gmane.org>
	<h0aocb$8rr$1@ger.gmane.org>
Message-ID: <20090607031609.GC8905@idyll.org>

On Fri, Jun 05, 2009 at 11:26:34AM +0200, Thomas Heller wrote:
-> Antoine Pitrou schrieb:
-> > Hello
-> > 
-> > Only one of the py3k buildbots seems up:
-> > http://www.python.org/dev/buildbot/3.x.stable/
-> 
-> Maybe they are waiting for the snakebite network ;-) (what's up with it, anyway?).

We're still getting the machines set up.  It turns out delivering power
and A/C to a wide variety of architectures is more complicated than it
may sound ;)

--titus
-- 
C. Titus Brown, ctb at msu.edu

From tjreedy at udel.edu  Sun Jun  7 05:21:06 2009
From: tjreedy at udel.edu (Terry Reedy)
Date: Sat, 06 Jun 2009 23:21:06 -0400
Subject: [Python-Dev] Status of 2.7 and 3.2
Message-ID: <h0fbn2$gop$1@ger.gmane.org>

2.7

Once upon a time, the plan was to come out with 2.6 and 3.0, and then 
after the usual interval, 2.7 and 3.1.  As it turned out, 3.0 came out 3 
months after 2.6, but, as it typical of x.0 releases, had some flaws 
leading to 3.1 now just 6 month later.

I have thought that 2.7 was now to come out instead with 3.2 and would 
include backported 3.2 new features.  Others expect 2.7 to come out soon 
after 3.1 and to only contain new 3.1 features.  So Guido or someone, 
please clarify: is 2.7 to be the counterpart of 3.1 or 3.2?

Data:
On the tracker, new features are being assigned to 2.7 and 3.2.
PEP 373 Python 2.7 Release Schedule says zilch:
Release Schedule: Not yet finalized
Possible features for 2.7: Nothing here


3.2

At some point, 3.x will become the "trunk" branch.  It seems to me that 
this should be done with 3.2 as part of the transition to Mercurial.

A. As long as 2.x is 'trunk', some people will view 3.x as 
'experimental'.  That was true for 3.0, but (much?) less so for 3.1.  Is 
there any known reason why 3.2 should not soon be considered and treated 
as the main development version, to become the main production version?

B. All new features will go into 3.2.  Only some will be backported to 
2.x.  So it seems that the flow should be to develop for 3.2 and maybe 
backport thereafter.


2.final

Is there any thought of making 2.7 be 2.final?

A. To my mind, the main reason to add features to 2.x is to make 
transition to 3.x easier, rather than to discourage transition to 3.x.

B. Do we really want to encourage library developers to put their 
'upgrade to a new version' energy into 2.x to 2.x+1 upgrades, rather 
than a 2.x to 3.y upgrades?

C. Some people are sticking with some version of 2.x because they want a 
stable version with minimal disturbance.  Such people might have 
preferred, for instance, getting 2.5.5 in April instead of the latest 
2.6 release.  Instead people 2.5 fixes are being told "Sorry. 2.5 is out 
of Maintenance phase and into SecurityFix only phase."  Once 2.x is put 
in feature freeze, micro bugfix releases can appear for years, as long 
as bugs are found and patches submitted and committed.  It should 
gradually become truly rock solid.


Terry Jan Reedy



From martin at v.loewis.de  Sun Jun  7 09:37:29 2009
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sun, 07 Jun 2009 09:37:29 +0200
Subject: [Python-Dev] Status of 2.7 and 3.2
In-Reply-To: <h0fbn2$gop$1@ger.gmane.org>
References: <h0fbn2$gop$1@ger.gmane.org>
Message-ID: <4A2B6E39.3090405@v.loewis.de>

> I have thought that 2.7 was now to come out instead with 3.2 and would
> include backported 3.2 new features.  Others expect 2.7 to come out soon
> after 3.1 and to only contain new 3.1 features.  So Guido or someone,
> please clarify: is 2.7 to be the counterpart of 3.1 or 3.2?

Neither, nor. 2.7 will be released 18..24 months after 2.6, i.e. between
April 2010 and October 2010. I think it's too early to speculate about
a release schedule for 3.2.

> 3.2
> 
> At some point, 3.x will become the "trunk" branch.  It seems to me that
> this should be done with 3.2 as part of the transition to Mercurial.

I'm not sure that the concept of a "trunk" branch still exists in
Mercurial. PEP 385 apparently doesn't have resolved the branch strategy
for Mercurial yet. With cloned branches, I think the concept of a
"trunk" becomes irrelevant.

> A. As long as 2.x is 'trunk', some people will view 3.x as
> 'experimental'.  That was true for 3.0, but (much?) less so for 3.1.  Is
> there any known reason why 3.2 should not soon be considered and treated
> as the main development version, to become the main production version?
> 
> B. All new features will go into 3.2.  Only some will be backported to
> 2.x.  So it seems that the flow should be to develop for 3.2 and maybe
> backport thereafter.

What about bug fixes? How will they flow?

> 2.final
> 
> Is there any thought of making 2.7 be 2.final?

Yes. Whether or not a 2.8 release will happen hasn't been decided yet,
but 2.7 may well be the last 2.x release.

> B. Do we really want to encourage library developers to put their
> 'upgrade to a new version' energy into 2.x to 2.x+1 upgrades, rather
> than a 2.x to 3.y upgrades?

IMO, it's much up to the contributors. If the regular committers
want to continue to work on 2.x, and a release manager is found
to create releases, we can continue to release 2.8, and perhaps
2.9. However, I think at this point, it is too early to discuss
that - 2.7 is still many months ahead.

Regards,
Martin


From stephen at xemacs.org  Sun Jun  7 14:25:05 2009
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Sun, 07 Jun 2009 21:25:05 +0900
Subject: [Python-Dev] Status of 2.7 and 3.2
In-Reply-To: <4A2B6E39.3090405@v.loewis.de>
References: <h0fbn2$gop$1@ger.gmane.org>
	<4A2B6E39.3090405@v.loewis.de>
Message-ID: <87hbysfcym.fsf@uwakimon.sk.tsukuba.ac.jp>

"Martin v. L?wis" writes:

 > I'm not sure that the concept of a "trunk" branch still exists in
 > Mercurial. PEP 385 apparently doesn't have resolved the branch strategy
 > for Mercurial yet. With cloned branches, I think the concept of a
 > "trunk" becomes irrelevant.

"Trunk" exists as a technical restriction in CVS, and maybe to some
extent in Subversion.  Of course that restriction is lifted by DVCSes.

But really "trunk" is a social concept.  Most projects have a trunk.
Even the anarchy known as GNU Arch converged on Tom Lord's version,
just as the storm of turbulance known as "Linux kernel development"
does on Linus's.

Python's 2.x/py3k division is a tour de force; I still can't believe
my eyes that you've pulled it off.  Consider Perl 6, LaTeX 3, and Zope
3, or Linux 3, for that matter.  The first 3 are all facing the "what
is trunk?" issue, in the case of Zope several years after the "point
oh" release, and AFAIK there are no current plans for Linux 3 (a
microkernel architecture, maybe?<1.01 wink>).  Of course the issues
faced by LaTeX, Zope, and the kernel are quite different from
Python's, and I don't know enough about Perl internals to compare.

So I think "trunk" does matter.  But it's not entirely in the power of
the Python committers, not even the BDFL, to determine what "trunk" is.

>>>>> Terry Reedy writes:

 > > A. As long as 2.x is 'trunk', some people will view 3.x as
 > > 'experimental'.  That was true for 3.0, but (much?) less so for 3.1.  Is
 > > there any known reason why 3.2 should not soon be considered and treated
 > > as the main development version, to become the main production version?

Yes.  User/developers may prefer strongly prefer the stability of
2.x.  That's the problem of being a successful product, you lose
agility almost by definition.

 > > B. All new features will go into 3.2.  Only some will be backported to
 > > 2.x.  So it seems that the flow should be to develop for 3.2 and maybe
 > > backport thereafter.
 > 
 > What about bug fixes? How will they flow?

This has to be ad hoc, at least at first.  Some bugs will be uncovered
in each version.  The solutions will often not be the same in the
different versions.  Many developers will be downstream, and only
willing to contribute the patch for the version they use.

 > > B. Do we really want to encourage library developers to put their
 > > 'upgrade to a new version' energy into 2.x to 2.x+1 upgrades, rather
 > > than a 2.x to 3.y upgrades?
 > 
 > IMO, it's much up to the contributors.

I think so, too.  Terry's word "encourage" is appropriate here,
though, and at some point that question will need to be answered.  I
think he's right to raise it early.



From tseaver at palladion.com  Sun Jun  7 15:02:41 2009
From: tseaver at palladion.com (Tres Seaver)
Date: Sun, 07 Jun 2009 09:02:41 -0400
Subject: [Python-Dev] Status of 2.7 and 3.2
In-Reply-To: <87hbysfcym.fsf@uwakimon.sk.tsukuba.ac.jp>
References: <h0fbn2$gop$1@ger.gmane.org>	<4A2B6E39.3090405@v.loewis.de>
	<87hbysfcym.fsf@uwakimon.sk.tsukuba.ac.jp>
Message-ID: <h0gdrg$ir9$1@ger.gmane.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Stephen J. Turnbull wrote:
> "Martin v. L?wis" writes:
> 
>  > I'm not sure that the concept of a "trunk" branch still exists in
>  > Mercurial. PEP 385 apparently doesn't have resolved the branch strategy
>  > for Mercurial yet. With cloned branches, I think the concept of a
>  > "trunk" becomes irrelevant.
> 
> "Trunk" exists as a technical restriction in CVS, and maybe to some
> extent in Subversion.  Of course that restriction is lifted by DVCSes.
> 
> But really "trunk" is a social concept.  Most projects have a trunk.
> Even the anarchy known as GNU Arch converged on Tom Lord's version,
> just as the storm of turbulance known as "Linux kernel development"
> does on Linus's.
> 
> Python's 2.x/py3k division is a tour de force; I still can't believe
> my eyes that you've pulled it off.  Consider Perl 6, LaTeX 3, and Zope
> 3, or Linux 3, for that matter.  The first 3 are all facing the "what
> is trunk?" issue, in the case of Zope several years after the "point
> oh" release, and AFAIK there are no current plans for Linux 3 (a
> microkernel architecture, maybe?<1.01 wink>).  Of course the issues
> faced by LaTeX, Zope, and the kernel are quite different from
> Python's, and I don't know enough about Perl internals to compare.

FWIW, it seems to be consensus in the Zope community that "Zope3" was a
misnomer, reflecting a vision for the software (serving as the
replacement / successor to Zope2) which it didn't grow to fulfill.

At this point, the "reusable" bits now form a collection of
separately-released packages referred to as the "Zope Toolkit,"[1] which
is intended to be used by Zope2, Grok, and the folks still running the
"pure" Zope3 appserver.

A lot of the work involved in this effort has been in disentagling the
dependencies among the various packages, precisely so that they can be
used piecemeal, without pulling in all of the stack.

[1] http://docs.zope.org/zopetoolkit/



Tres.
- --
===================================================================
Tres Seaver          +1 540-429-0999          tseaver at palladion.com
Palladion Software   "Excellence by Design"    http://palladion.com
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFKK7px+gerLs4ltQ4RAoMnAJ9WTL8zo3+bSop+SAXr+akp85tMZQCfS5oC
y14j64VJPjgxDQzCb1p/iZo=
=RCZa
-----END PGP SIGNATURE-----


From ncoghlan at gmail.com  Sun Jun  7 15:32:20 2009
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sun, 07 Jun 2009 23:32:20 +1000
Subject: [Python-Dev] Status of 2.7 and 3.2
In-Reply-To: <4A2B6E39.3090405@v.loewis.de>
References: <h0fbn2$gop$1@ger.gmane.org> <4A2B6E39.3090405@v.loewis.de>
Message-ID: <4A2BC164.6050201@gmail.com>

Martin v. L?wis wrote:
>> I have thought that 2.7 was now to come out instead with 3.2 and would
>> include backported 3.2 new features.  Others expect 2.7 to come out soon
>> after 3.1 and to only contain new 3.1 features.  So Guido or someone,
>> please clarify: is 2.7 to be the counterpart of 3.1 or 3.2?
> 
> Neither, nor. 2.7 will be released 18..24 months after 2.6, i.e. between
> April 2010 and October 2010. I think it's too early to speculate about
> a release schedule for 3.2.

I think Terry's underlying question is more basic than that. It can also
be phrased as:

"Is 2.7 allowed to have new features which did not appear in 3.1?"

In previous discussions, a general policy has been articulated that
having released Python 3.0, any new features should appear in a 3.x
release before they appear in a 2.x release. Following that policy
(which I think is actually a good one) means there are certain
consequences for the two possible answers to the above question:

A. "No."

  This answer means that 2.7 will only contain new features that are
part of 3.1. If 2.7 sticks to a normal 18-24 month release cycle the
only activities on the trunk for the next 12 months or so should be
backports from 3.1 and bug fixes. New features added to the py3k branch
for 3.2 should not be backported until after 2.7 is released. The other
alternative in this case is to release 2.7 earlier than normal, but that
creates problems in terms of the absolute duration of maintenance branch
support for 2.6.

B. "Yes."

  This answer means that the 3.1 to 3.2 development cycle will need to
be truncated by roughly 6 months so that 3.2 can be released before 2.7
with any new features of interest. The 3.2 and 2.7 releases should then
occur within a few months of each other.

Releasing 2.7 early doesn't seem like a good idea to me and neither does
putting the trunk into the equivalent of a feature freeze for 12 months
or more. So I'm in favour of the idea of a paired 3.2/2.7 release late
next year.

I don't think that's a novel idea though - I'm pretty sure it was
suggested (and met with general approval) when the idea of a short
release cycle for 3.1 was first brought up.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------

From martin at v.loewis.de  Sun Jun  7 18:55:43 2009
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sun, 07 Jun 2009 18:55:43 +0200
Subject: [Python-Dev] Status of 2.7 and 3.2
In-Reply-To: <4A2BC164.6050201@gmail.com>
References: <h0fbn2$gop$1@ger.gmane.org> <4A2B6E39.3090405@v.loewis.de>
	<4A2BC164.6050201@gmail.com>
Message-ID: <4A2BF10F.3040105@v.loewis.de>

Nick Coghlan wrote:
> Martin v. L?wis wrote:
>>> I have thought that 2.7 was now to come out instead with 3.2 and would
>>> include backported 3.2 new features.  Others expect 2.7 to come out soon
>>> after 3.1 and to only contain new 3.1 features.  So Guido or someone,
>>> please clarify: is 2.7 to be the counterpart of 3.1 or 3.2?
>> Neither, nor. 2.7 will be released 18..24 months after 2.6, i.e. between
>> April 2010 and October 2010. I think it's too early to speculate about
>> a release schedule for 3.2.
> 
> I think Terry's underlying question is more basic than that. It can also
> be phrased as:
> 
> "Is 2.7 allowed to have new features which did not appear in 3.1?"

I don't think this is the question Terry meant to ask, and the answer
to this question is a clear "Certainly, but you are asking the wrong
question."

The question that Terry really meant to ask is "why is there no release
candidate for 2.7 already, when 3.1 is going to be released RSN?", and
that question arises from the (incorrect) understanding that 2.x
releases and 3.x releases will be lock-step.

The question that you posed should have been phrased as
"Is 2.7 allowed to have new features which don't also appear on
the 3k branch?", to which the answer is a clear "No."

> In previous discussions, a general policy has been articulated that
> having released Python 3.0, any new features should appear in a 3.x
> release before they appear in a 2.x release.

I don't think that the policy has been articulated in exactly that
way, and it certainly wasn't implemented for 2.6/3.0. 2.6 was released
with features that had not been released in any 3.x release, just
because 3.x had not been released at all.

> A. "No."
> 
>   This answer means that 2.7 will only contain new features that are
> part of 3.1. If 2.7 sticks to a normal 18-24 month release cycle the
> only activities on the trunk for the next 12 months or so should be
> backports from 3.1 and bug fixes. New features added to the py3k branch
> for 3.2 should not be backported until after 2.7 is released. The other
> alternative in this case is to release 2.7 earlier than normal, but that
> creates problems in terms of the absolute duration of maintenance branch
> support for 2.6.

I think this makes it clear that the answer can't possibly be "No".
Not allowing addition of new features even though the release is still
12 months away would be unreasonable, IMO - what's the point of freezing
development now, and where should new features to 2.x be added if not
to the trunk?

Likewise, releasing 2.7 now is completely unacceptable.

> B. "Yes."
> 
>   This answer means that the 3.1 to 3.2 development cycle will need to
> be truncated by roughly 6 months so that 3.2 can be released before 2.7
> with any new features of interest. The 3.2 and 2.7 releases should then
> occur within a few months of each other.

Assuming the policy "release new features for 2.x only after they got
released for 3.x". I don't think such a policy actually exists.

Regards,
Martin

From rdmurray at bitdance.com  Sun Jun  7 19:51:04 2009
From: rdmurray at bitdance.com (R. David Murray)
Date: Sun, 7 Jun 2009 13:51:04 -0400 (EDT)
Subject: [Python-Dev] Status of 2.7 and 3.2
In-Reply-To: <4A2BF10F.3040105@v.loewis.de>
References: <h0fbn2$gop$1@ger.gmane.org> <4A2B6E39.3090405@v.loewis.de>
	<4A2BC164.6050201@gmail.com> <4A2BF10F.3040105@v.loewis.de>
Message-ID: <Pine.LNX.4.64.0906071344230.32513@kimball.webabinitio.net>

On Sun, 7 Jun 2009 at 18:55, "Martin v. L?wis" wrote:
>> B. "Yes."
>>
>>   This answer means that the 3.1 to 3.2 development cycle will need to
>> be truncated by roughly 6 months so that 3.2 can be released before 2.7
>> with any new features of interest. The 3.2 and 2.7 releases should then
>> occur within a few months of each other.
>
> Assuming the policy "release new features for 2.x only after they got
> released for 3.x". I don't think such a policy actually exists.

I'm neutral on time frames, but I think that it _should_ be a policy
that new features only get released to the 2.x branch after they have
been released in the 3.x branch.  Or, rather, I though that policy was
implicit in the idea that we weren't _automatically_ backporting features,
specifically in order to encourage 3.x adoption.  So if 2.7 is going to
come out before 3.2, you'll have to convince me that having new features
in 2.7 that aren't in 3.1 is a good idea :)

Note that I'm not advocating feature freezing the trunk, but rather
having 3.2 come out before or in sync with 2.7.

--David

From martin at v.loewis.de  Sun Jun  7 21:21:11 2009
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sun, 07 Jun 2009 21:21:11 +0200
Subject: [Python-Dev] Status of 2.7 and 3.2
In-Reply-To: <Pine.LNX.4.64.0906071344230.32513@kimball.webabinitio.net>
References: <h0fbn2$gop$1@ger.gmane.org>
	<4A2B6E39.3090405@v.loewis.de>	<4A2BC164.6050201@gmail.com>
	<4A2BF10F.3040105@v.loewis.de>
	<Pine.LNX.4.64.0906071344230.32513@kimball.webabinitio.net>
Message-ID: <4A2C1327.9020909@v.loewis.de>

> I'm neutral on time frames, but I think that it _should_ be a policy
> that new features only get released to the 2.x branch after they have
> been released in the 3.x branch.  Or, rather, I though that policy was
> implicit in the idea that we weren't _automatically_ backporting features,
> specifically in order to encourage 3.x adoption.

I think "backporting" is an entirely different issue, let alone
"automatic" backporting.

What *is* policy (AFAIU) is "any feature in the trunk must also exist in
the py3k branch". IMO, this is sufficient to encourage 3.x adoption, and
it is a policy that is silent wrt. to release date ordering.

By the policy you propose, we could not have released 2.6 in October
2008, which we really really wanted to because Apple wanted us to.

> So if 2.7 is going to
> come out before 3.2, you'll have to convince me that having new features
> in 2.7 that aren't in 3.1 is a good idea :)

I don't see why the precise ordering of release dates matters at all.
It seems you are happy with having 3.2 be released "around the same
time" as 2.7. What is "the same time"? 3 months difference? 6 months
difference? It certainly wouldn't be a year.

Regards,
Martin

From barry at python.org  Sun Jun  7 22:31:44 2009
From: barry at python.org (Barry Warsaw)
Date: Sun, 7 Jun 2009 16:31:44 -0400
Subject: [Python-Dev] Status of 2.7 and 3.2
In-Reply-To: <4A2BF10F.3040105@v.loewis.de>
References: <h0fbn2$gop$1@ger.gmane.org> <4A2B6E39.3090405@v.loewis.de>
	<4A2BC164.6050201@gmail.com> <4A2BF10F.3040105@v.loewis.de>
Message-ID: <CCB2DAB9-A2ED-483D-98AF-3CEE3E9763B5@python.org>

On Jun 7, 2009, at 12:55 PM, Martin v. L?wis wrote:

> Assuming the policy "release new features for 2.x only after they got
> released for 3.x". I don't think such a policy actually exists.

The policy, as I remember it, can be summed up: don't innovate new  
features in the 2.x branch.  Meaning, add it to 3.x first and then  
backport if you want.  I don't believe a new feature has to be in a / 
released/ version of 3.x first for it to show up in the next 2.x  
release.

-Barry

-------------- next part --------------
A non-text attachment was scrubbed...
Name: PGP.sig
Type: application/pgp-signature
Size: 832 bytes
Desc: This is a digitally signed message part
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090607/d1aac55f/attachment.pgp>

From rdmurray at bitdance.com  Sun Jun  7 22:40:31 2009
From: rdmurray at bitdance.com (R. David Murray)
Date: Sun, 7 Jun 2009 16:40:31 -0400 (EDT)
Subject: [Python-Dev] Status of 2.7 and 3.2
In-Reply-To: <4A2C1327.9020909@v.loewis.de>
References: <h0fbn2$gop$1@ger.gmane.org> <4A2B6E39.3090405@v.loewis.de>
	<4A2BC164.6050201@gmail.com> <4A2BF10F.3040105@v.loewis.de>
	<Pine.LNX.4.64.0906071344230.32513@kimball.webabinitio.net>
	<4A2C1327.9020909@v.loewis.de>
Message-ID: <Pine.LNX.4.64.0906071611150.32513@kimball.webabinitio.net>

On Sun, 7 Jun 2009 at 21:21, "Martin v. L?wis" wrote:
>> I'm neutral on time frames, but I think that it _should_ be a policy
>> that new features only get released to the 2.x branch after they have
>> been released in the 3.x branch.  Or, rather, I though that policy was
>> implicit in the idea that we weren't _automatically_ backporting features,
>> specifically in order to encourage 3.x adoption.
>
> I think "backporting" is an entirely different issue, let alone
> "automatic" backporting.
>
> What *is* policy (AFAIU) is "any feature in the trunk must also exist in
> the py3k branch". IMO, this is sufficient to encourage 3.x adoption, and
> it is a policy that is silent wrt. to release date ordering.

You are right, my use of the term backport was imprecise.

My impression at pycon was that Guido (and others) wanted a stronger
policy than "make sure new features in trunk are also in 3.x".  I heard
this as "put new features in 3.x, not 2.x, to encourage 3.x adoption,"
but leaving the decision to the individual developers on whether or not
to also "backport" them to 2.x.  (The quotes around backport being that
if you know you are going to put it into both it is currently easier to
do trunk first.)  As we have discovered since, this tends to get modified
by wanting to ease transition from 2.x to 3.x by providing some of the
features in 2.x (I'm thinking specifically of the distutils discussion.)

How I got from that to release date ordering was by hearing that as
"new features should be in 3.x first, and only maybe in 2.x".  Clearly
that was just my interpretation :)

> By the policy you propose, we could not have released 2.6 in October
> 2008, which we really really wanted to because Apple wanted us to.

I don't think the 2.6 release date is relevant to this discussion,
since 3.x hadn't been released at all at that point.  What I want to
avoid is people writing new code for 2.7 instead of 3.1 because they
want to take advantage of a nifty new feature that 3.1 doesn't have.

>> So if 2.7 is going to
>> come out before 3.2, you'll have to convince me that having new features
>> in 2.7 that aren't in 3.1 is a good idea :)
>
> I don't see why the precise ordering of release dates matters at all.
> It seems you are happy with having 3.2 be released "around the same
> time" as 2.7. What is "the same time"? 3 months difference? 6 months
> difference? It certainly wouldn't be a year.

No more than three months, I'd say, but that's just a gut level feeling,
not backed by a specific argument.

That said, I also have a gut level feeling that it is better to have
the new features come out in 3.2 _first_.  But I'm not presenting that
as anything more than my feeling, and I'm happy to go with whatever
consensus develops.  FWIW, Benjamin has said the he personally has no
problem with 3.2's release cycle being shorter than "normal".

--David

From fuzzyman at voidspace.org.uk  Sun Jun  7 22:55:19 2009
From: fuzzyman at voidspace.org.uk (Michael Foord)
Date: Sun, 07 Jun 2009 21:55:19 +0100
Subject: [Python-Dev] Status of 2.7 and 3.2
In-Reply-To: <Pine.LNX.4.64.0906071611150.32513@kimball.webabinitio.net>
References: <h0fbn2$gop$1@ger.gmane.org>
	<4A2B6E39.3090405@v.loewis.de>	<4A2BC164.6050201@gmail.com>
	<4A2BF10F.3040105@v.loewis.de>	<Pine.LNX.4.64.0906071344230.32513@kimball.webabinitio.net>	<4A2C1327.9020909@v.loewis.de>
	<Pine.LNX.4.64.0906071611150.32513@kimball.webabinitio.net>
Message-ID: <4A2C2937.8080502@voidspace.org.uk>

R. David Murray wrote:
> [snip...]
>> By the policy you propose, we could not have released 2.6 in October
>> 2008, which we really really wanted to because Apple wanted us to.
>
> I don't think the 2.6 release date is relevant to this discussion,
> since 3.x hadn't been released at all at that point.  What I want to
> avoid is people writing new code for 2.7 instead of 3.1 because they
> want to take advantage of a nifty new feature that 3.1 doesn't have.
>
But 3.1 is in feature freeze (py3k trunk) and it is not possible to 
check-in code for 3.2. Following this policy it would mean a feature 
freeze on trunk for an indefinite period of time.


Michael



-- 
http://www.ironpythoninaction.com/
http://www.voidspace.org.uk/blog



From benjamin at python.org  Sun Jun  7 22:57:25 2009
From: benjamin at python.org (Benjamin Peterson)
Date: Sun, 7 Jun 2009 15:57:25 -0500
Subject: [Python-Dev] Status of 2.7 and 3.2
In-Reply-To: <4A2C2937.8080502@voidspace.org.uk>
References: <h0fbn2$gop$1@ger.gmane.org> <4A2B6E39.3090405@v.loewis.de>
	<4A2BC164.6050201@gmail.com> <4A2BF10F.3040105@v.loewis.de>
	<Pine.LNX.4.64.0906071344230.32513@kimball.webabinitio.net>
	<4A2C1327.9020909@v.loewis.de>
	<Pine.LNX.4.64.0906071611150.32513@kimball.webabinitio.net>
	<4A2C2937.8080502@voidspace.org.uk>
Message-ID: <1afaf6160906071357s293cb591n629b8f41c4b5f2ac@mail.gmail.com>

2009/6/7 Michael Foord <fuzzyman at voidspace.org.uk>:
> R. David Murray wrote:
>>
>> [snip...]
>>>
>>> By the policy you propose, we could not have released 2.6 in October
>>> 2008, which we really really wanted to because Apple wanted us to.
>>
>> I don't think the 2.6 release date is relevant to this discussion,
>> since 3.x hadn't been released at all at that point. ?What I want to
>> avoid is people writing new code for 2.7 instead of 3.1 because they
>> want to take advantage of a nifty new feature that 3.1 doesn't have.
>>
> But 3.1 is in feature freeze (py3k trunk) and it is not possible to check-in
> code for 3.2. Following this policy it would mean a feature freeze on trunk
> for an indefinite period of time.

And that's what we want to avoid, so we are discussing 3.2.



-- 
Regards,
Benjamin

From tjreedy at udel.edu  Sun Jun  7 23:30:23 2009
From: tjreedy at udel.edu (Terry Reedy)
Date: Sun, 07 Jun 2009 17:30:23 -0400
Subject: [Python-Dev] Status of 2.7 and 3.2
In-Reply-To: <4A2BC164.6050201@gmail.com>
References: <h0fbn2$gop$1@ger.gmane.org> <4A2B6E39.3090405@v.loewis.de>
	<4A2BC164.6050201@gmail.com>
Message-ID: <h0hbhe$3jg$1@ger.gmane.org>

Nick Coghlan wrote:
> Martin v. L?wis wrote:
>>> I have thought that 2.7 was now to come out instead with 3.2 and would
>>> include backported 3.2 new features.  Others expect 2.7 to come out soon
>>> after 3.1 and to only contain new 3.1 features.  So Guido or someone,
>>> please clarify: is 2.7 to be the counterpart of 3.1 or 3.2?
>> Neither, nor. 2.7 will be released 18..24 months after 2.6, i.e. between
>> April 2010 and October 2010. I think it's too early to speculate about
>> a release schedule for 3.2.
> 
> I think Terry's underlying question is more basic than that. It can also
> be phrased as:
> 
> "Is 2.7 allowed to have new features which did not appear in 3.1?"

That is the issue that came up in a Python list discussion when two 
people, including one who I expect to know as much as me about this, 
'corrected' my 'yes' to "No, 2.7 will come out in a few months after 
3.1."  So there clearly is confusion on this and related issues (as I 
already see in the responses on this thread ;-).  The answer makes some 
difference in how issues are handled on the tracker, which I 
occasionally help with.

> In previous discussions, a general policy has been articulated that
> having released Python 3.0, any new features should appear in a 3.x
> release before they appear in a 2.x release. Following that policy
> (which I think is actually a good one) means there are certain
> consequences for the two possible answers to the above question:
> 
> A. "No."
> 
>   This answer means that 2.7 will only contain new features that are
> part of 3.1. If 2.7 sticks to a normal 18-24 month release cycle the
> only activities on the trunk for the next 12 months or so should be
> backports from 3.1 and bug fixes. New features added to the py3k branch
> for 3.2 should not be backported until after 2.7 is released. The other
> alternative in this case is to release 2.7 earlier than normal, but that
> creates problems in terms of the absolute duration of maintenance branch
> support for 2.6.

I agree, but this is what two people are expecting.

> B. "Yes."
> 
>   This answer means that the 3.1 to 3.2 development cycle will need to
> be truncated by roughly 6 months so that 3.2 can be released before 2.7
> with any new features of interest. The 3.2 and 2.7 releases should then
> occur within a few months of each other.
> 
> Releasing 2.7 early doesn't seem like a good idea to me and neither does
> putting the trunk into the equivalent of a feature freeze for 12 months
> or more. So I'm in favour of the idea of a paired 3.2/2.7 release late
> next year.

This is what I have been expecting,
> 
> I don't think that's a novel idea though - I'm pretty sure it was
> suggested (and met with general approval) when the idea of a short
> release cycle for 3.1 was first brought up.

I presume because it has been stated before.

In addition to the question above, I am also trying to provoke thought 
on the nature and purpose of 2.7.  Backporting features 'if someone 
feels like it' seems pretty haphazard.  For someone wanting to maintain 
compatibility across multiple 2.x releases, a random new features may be 
nearly useless.

Terry Jan Reedy



From fuzzyman at voidspace.org.uk  Sun Jun  7 23:32:49 2009
From: fuzzyman at voidspace.org.uk (Michael Foord)
Date: Sun, 07 Jun 2009 22:32:49 +0100
Subject: [Python-Dev] Status of 2.7 and 3.2
In-Reply-To: <h0hbhe$3jg$1@ger.gmane.org>
References: <h0fbn2$gop$1@ger.gmane.org>
	<4A2B6E39.3090405@v.loewis.de>	<4A2BC164.6050201@gmail.com>
	<h0hbhe$3jg$1@ger.gmane.org>
Message-ID: <4A2C3201.4060007@voidspace.org.uk>

Terry Reedy wrote:
> [snip...]
>>
>> I don't think that's a novel idea though - I'm pretty sure it was
>> suggested (and met with general approval) when the idea of a short
>> release cycle for 3.1 was first brought up.
>
> I presume because it has been stated before.
>
> In addition to the question above, I am also trying to provoke thought 
> on the nature and purpose of 2.7.  Backporting features 'if someone 
> feels like it' seems pretty haphazard.  For someone wanting to 
> maintain compatibility across multiple 2.x releases, a random new 
> features may be nearly useless.
>

The "What's new in Python 2.7" list is already very impressive. :-)

Michael

> Terry Jan Reedy
>
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: 
> http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk 
>


-- 
http://www.ironpythoninaction.com/
http://www.voidspace.org.uk/blog



From rdmurray at bitdance.com  Sun Jun  7 23:37:35 2009
From: rdmurray at bitdance.com (R. David Murray)
Date: Sun, 7 Jun 2009 17:37:35 -0400 (EDT)
Subject: [Python-Dev] Status of 2.7 and 3.2
In-Reply-To: <4A2C2937.8080502@voidspace.org.uk>
References: <h0fbn2$gop$1@ger.gmane.org> <4A2B6E39.3090405@v.loewis.de>
	<4A2BC164.6050201@gmail.com> <4A2BF10F.3040105@v.loewis.de>
	<Pine.LNX.4.64.0906071344230.32513@kimball.webabinitio.net>
	<4A2C1327.9020909@v.loewis.de>
	<Pine.LNX.4.64.0906071611150.32513@kimball.webabinitio.net>
	<4A2C2937.8080502@voidspace.org.uk>
Message-ID: <Pine.LNX.4.64.0906071731250.32513@kimball.webabinitio.net>

On Sun, 7 Jun 2009 at 21:55, Michael Foord wrote:
> R. David Murray wrote:
>>  [snip...]
>> >  By the policy you propose, we could not have released 2.6 in October
>> >  2008, which we really really wanted to because Apple wanted us to.
>>
>>  I don't think the 2.6 release date is relevant to this discussion,
>>  since 3.x hadn't been released at all at that point.  What I want to
>>  avoid is people writing new code for 2.7 instead of 3.1 because they
>>  want to take advantage of a nifty new feature that 3.1 doesn't have.
>> 
> But 3.1 is in feature freeze (py3k trunk) and it is not possible to check-in 
> code for 3.2. Following this policy it would mean a feature freeze on trunk 
> for an indefinite period of time.

As I said on IRC, that's not what I'm advocating.  I'm advocating having
3.2 come out before 2.7, so that the new post-3.1 features get released to
the public in 3.2 first, before appearing in 2.7.  (Although "first" could
alternatively mean the same day, if the community were up to for that).

But like I said I'm not attached to the idea, just advocating it ;)

--David

From python at rcn.com  Mon Jun  8 00:23:49 2009
From: python at rcn.com (Raymond Hettinger)
Date: Sun,  7 Jun 2009 18:23:49 -0400 (EDT)
Subject: [Python-Dev] Status of 2.7 and 3.2
Message-ID: <20090607182349.BID99621@ms19.lnh.mail.rcn.net>

How about we just continue to improve both branches, doing forward or backports as appropriate.  No need to develop a policy of crippling one branch on the theory that it will make the other seem more attractive.

Besides, if 2.7 and 3.2 get released within a few months of each other, any  inversion of incentives will be temporary and fleeting.  Most likely people's decisions on switching to 3.x will be dominated by other factors such as the availability of third-party modules or other dependencies.

IIRC, Benjamin's current merge procedures flow from the trunk to the py3k branch.  Probably, it is best to continue with that practice lest we muck-up his merge/block entries.


Raymond










From ncoghlan at gmail.com  Mon Jun  8 04:21:11 2009
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Mon, 08 Jun 2009 12:21:11 +1000
Subject: [Python-Dev] Status of 2.7 and 3.2
In-Reply-To: <4A2BF10F.3040105@v.loewis.de>
References: <h0fbn2$gop$1@ger.gmane.org> <4A2B6E39.3090405@v.loewis.de>
	<4A2BC164.6050201@gmail.com> <4A2BF10F.3040105@v.loewis.de>
Message-ID: <4A2C7597.1050103@gmail.com>

Martin v. L?wis wrote:
> I don't think that the policy has been articulated in exactly that
> way, and it certainly wasn't implemented for 2.6/3.0. 2.6 was released
> with features that had not been released in any 3.x release, just
> because 3.x had not been released at all.

I don't think the idea of "3.x first" was raised explicitly until the
discussions about having a short release cycle for 3.1 came up. Before
then there was an assumption that 3.1 and 2.7 would come out at roughly
the same time.

>> B. "Yes."
>>
>>   This answer means that the 3.1 to 3.2 development cycle will need to
>> be truncated by roughly 6 months so that 3.2 can be released before 2.7
>> with any new features of interest. The 3.2 and 2.7 releases should then
>> occur within a few months of each other.
> 
> Assuming the policy "release new features for 2.x only after they got
> released for 3.x". I don't think such a policy actually exists.

As RDM already mentioned, the potential problem such a policy is
intended to address is the idea of being able to write 2.7 code that
can't be forward ported to 3.x because no version with a corresponding
feature set has been released yet.

My last sentence above was actually deliberately ambiguous as to whether
2.7 or 3.2 ends up being released first - so long as the releases happen
within a few months of each other and any new features that appear in
2.7 also appear in 3.2, then the migration path for different 2.x
versions remains clear (i.e 2.6-only code can be migrated to 3.1, but
2.7 code may need to be migrated to 3.2 instead if it uses features that
aren't in 3.1).

The exact order and timing of 3.2/2.7 will no doubt depend on any
specific issues relating to the two releases around the time that they
come out. But in the interests of maintaining a coherent migration path
from 2.x to 3.x, it makes sense to plan to release them at roughly the
same time.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------

From ncoghlan at gmail.com  Mon Jun  8 04:28:45 2009
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Mon, 08 Jun 2009 12:28:45 +1000
Subject: [Python-Dev] Status of 2.7 and 3.2
In-Reply-To: <20090607182349.BID99621@ms19.lnh.mail.rcn.net>
References: <20090607182349.BID99621@ms19.lnh.mail.rcn.net>
Message-ID: <4A2C775D.7030205@gmail.com>

Raymond Hettinger wrote:
> How about we just continue to improve both branches, doing forward or
> backports as appropriate.  No need to develop a policy of crippling
> one branch on the theory that it will make the other seem more
> attractive.
> 
> Besides, if 2.7 and 3.2 get released within a few months of each
> other, any  inversion of incentives will be temporary and fleeting.
> Most likely people's decisions on switching to 3.x will be dominated
> by other factors such as the availability of third-party modules or
> other dependencies.
> 
> IIRC, Benjamin's current merge procedures flow from the trunk to the
> py3k branch.  Probably, it is best to continue with that practice
> lest we muck-up his merge/block entries.

Agreed.

>From Terry's later response, I think the main confusion here was whether
or not 2.7 was going to be getting released hot on the heels of 3.1, and
the answer to that is "No, 2.7 won't be released until 18-24 months have
passed since the release of 2.6, as with previous 2.x releases".

Releasing 3.2 around the same time as 2.7 to allow for easier migration
from the latest 2.x release to the latest 3.x release will likely make
sense, but doesn't actually need to be a 100% definite plan at this point.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------

From fwierzbicki at gmail.com  Mon Jun  8 16:29:14 2009
From: fwierzbicki at gmail.com (Frank Wierzbicki)
Date: Mon, 8 Jun 2009 10:29:14 -0400
Subject: [Python-Dev] Draft PEP 385: Migrating from svn to Mercurial
In-Reply-To: <ea2499da0906051348g20c204d1rdec62c4b95b1697e@mail.gmail.com>
References: <ea2499da0906051348g20c204d1rdec62c4b95b1697e@mail.gmail.com>
Message-ID: <4dab5f760906080729gfb4ff90tdba68d800ac32ab6@mail.gmail.com>

At PyCon, we discussed moving Jython's svn repository to Python's with
Martin von L?wis.  I would think that Jython would live in Python's hg
repository in the same way as stackless and distutils.  Has the
parallel project strategy been determined?  Will they be separate
repositories, separate "forests", something else?

Also, Martin suggested we migrate to Python's svn and then go along
for the svn->hg ride.  Does that still make sense now that some
planning has been done?

-Frank

From techtonik at gmail.com  Mon Jun  8 17:07:13 2009
From: techtonik at gmail.com (anatoly techtonik)
Date: Mon, 8 Jun 2009 18:07:13 +0300
Subject: [Python-Dev] Roundup keywords for bug tracking
In-Reply-To: <2d75d7660906051807s3ca051fcw5af5841e99cd707a@mail.gmail.com>
References: <d34314100906051659r4e89788bybfeeed8f6e854366@mail.gmail.com> 
	<2d75d7660906051807s3ca051fcw5af5841e99cd707a@mail.gmail.com>
Message-ID: <d34314100906080807x6fa462fcs4565d4cdc00195b0@mail.gmail.com>

On Sat, Jun 6, 2009 at 4:07 AM, Daniel Diniz <ajaksu at gmail.com> wrote:
> anatoly techtonik wrote:
>>
>> It is impossible to edit roundup keywords and this takes away the
>> flexibility in selecting bugs related to a module/function/test or
>> some other aspect of development. For example, I need to gather all
>> subprocess bugs in one query and things that won't be fixed in
>> deprecated os.popen() into another. In Trac I would use "subprocess"
>> and "os.popen" keywords. On ohloh I would add similar tags (if bugs
>> were software) without, but I can't do anything about Python roundup.
>> Is there any reason for such restriction?
>
> Well, keywords are used as a very restricted set of tags, so only
> users in the Developer group can create them. We've discussed free
> form issue tags that any user can create or edit in #python-dev and
> tracker-discuss[0]. I'm pretty sure they'd cover your use-case. I've
> submitted a patch to Rietveld[1], but it seems I never filled it in
> the meta-tracker, oopsie.

>From [0] discussion it seems that tags are planned to be a replacement
for component or keywords field, but in my vision they should be just
tags that doesn't have any specific meaning or administration
interface. Autocomplete with ajax lookup is nice, but no drop-down
lists etc.

I made some comments in Rietveld at [1], but was unable to test it
live, because [2] is offline.

> If you (or anyone else) want to test-drive the tags feature, I can
> create an account in the experimental tracker[2] (which needs some
> attention anyway). I should be able to submit the patch to the
> meta-tracker during the weekend.

Hope this went well. I would definitely like to see how far this
feature from how I imagine it, but b.p.o. deployment could be a better
alternative for a real testing.

> Also, if you would like to bookmark arbitrary sets of issues, the
> bookmarklet and form in http://static.bot.bio.br/tool.html may be of
> help. You can paste the ids into the search page's ID field and create
> a query for a given (static) set of issues.

Seems like it can come in handy. Thanks.

-- 
anatoly t.

From g.brandl at gmx.net  Mon Jun  8 17:27:35 2009
From: g.brandl at gmx.net (Georg Brandl)
Date: Mon, 08 Jun 2009 17:27:35 +0200
Subject: [Python-Dev] Draft PEP 385: Migrating from svn to Mercurial
In-Reply-To: <4dab5f760906080729gfb4ff90tdba68d800ac32ab6@mail.gmail.com>
References: <ea2499da0906051348g20c204d1rdec62c4b95b1697e@mail.gmail.com>
	<4dab5f760906080729gfb4ff90tdba68d800ac32ab6@mail.gmail.com>
Message-ID: <h0jan2$ueb$1@ger.gmane.org>

Frank Wierzbicki schrieb:
> At PyCon, we discussed moving Jython's svn repository to Python's with
> Martin von L?wis.  I would think that Jython would live in Python's hg
> repository in the same way as stackless and distutils.  Has the
> parallel project strategy been determined?  Will they be separate
> repositories, separate "forests", something else?

It should definitely be a separate repository, repositories tend to be a
smaller unit in hg than in SVN, due to the fact that partial clones/checkouts
are not supported.

> Also, Martin suggested we migrate to Python's svn and then go along
> for the svn->hg ride.  Does that still make sense now that some
> planning has been done?

I don't think so.  It should not matter from which repository the conversion
is done, especially since there is no shared history at all.

Georg

-- 
Thus spake the Lord: Thou shalt indent with four spaces. No more, no less.
Four shall be the number of spaces thou shalt indent, and the number of thy
indenting shall be four. Eight shalt thou not indent, nor either indent thou
two, excepting that thou then proceed to four. Tabs are right out.


From dirkjan at ochtman.nl  Mon Jun  8 17:32:46 2009
From: dirkjan at ochtman.nl (Dirkjan Ochtman)
Date: Mon, 8 Jun 2009 17:32:46 +0200
Subject: [Python-Dev] Draft PEP 385: Migrating from svn to Mercurial
In-Reply-To: <4dab5f760906080729gfb4ff90tdba68d800ac32ab6@mail.gmail.com>
References: <ea2499da0906051348g20c204d1rdec62c4b95b1697e@mail.gmail.com>
	<4dab5f760906080729gfb4ff90tdba68d800ac32ab6@mail.gmail.com>
Message-ID: <ea2499da0906080832w3f232721sfcf2cb6a12441965@mail.gmail.com>

On Mon, Jun 8, 2009 at 16:29, Frank Wierzbicki<fwierzbicki at gmail.com> wrote:
> At PyCon, we discussed moving Jython's svn repository to Python's with
> Martin von L?wis. ?I would think that Jython would live in Python's hg
> repository in the same way as stackless and distutils. ?Has the
> parallel project strategy been determined? ?Will they be separate
> repositories, separate "forests", something else?

I think they should just be separate repositories. The svn.python.org
"repository" is more like a collection of actual repositories than a
repository in itself.

> Also, Martin suggested we migrate to Python's svn and then go along
> for the svn->hg ride. ?Does that still make sense now that some
> planning has been done?

I'd say migrating to Python's svn doesn't make a whole lot of sense at
this point, but I'll leave that to Martin (since he has to do the
work). For the conversion, I can just as well take the Jython repo
from your current server. I've started a svnsync job with your repo so
I can run some test conversions. It's a relatively small repository,
so it shouldn't be much of a problem.

Cheers,

Dirkjan

From techtonik at gmail.com  Mon Jun  8 19:50:26 2009
From: techtonik at gmail.com (anatoly techtonik)
Date: Mon, 8 Jun 2009 20:50:26 +0300
Subject: [Python-Dev] Status of 2.7 and 3.2
In-Reply-To: <h0fbn2$gop$1@ger.gmane.org>
References: <h0fbn2$gop$1@ger.gmane.org>
Message-ID: <d34314100906081050x5cf50e57r37036a08bf8166e8@mail.gmail.com>

On Sun, Jun 7, 2009 at 6:21 AM, Terry Reedy <tjreedy at udel.edu> wrote:
>
> I have thought that 2.7 was now to come out instead with 3.2 and would
> include backported 3.2 new features. ?Others expect 2.7 to come out soon
> after 3.1 and to only contain new 3.1 features. ?So Guido or someone, please
> clarify: is 2.7 to be the counterpart of 3.1 or 3.2?

Just my 0.02 cents, but struggling with all warts of 2.5 subprocessing
in Windows I would vote for more time for stabilizating things - not
adding new features. Long awaited subprocess as replacement for
os.popen() AFAIK is still incapable to asynchronously communicate with
spawned processes on Windows. I would call this feature as critical
even on 2.6  As a release testcase - try porting pyexpect module to
this platform. Absence of native curses/console/readline module also
makes Python one-way unix shell language while many users expect it to
be crossplatform.

-- 
anatoly t.

From solipsis at pitrou.net  Mon Jun  8 20:17:32 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Mon, 8 Jun 2009 18:17:32 +0000 (UTC)
Subject: [Python-Dev] Status of 2.7 and 3.2
References: <h0fbn2$gop$1@ger.gmane.org>
	<d34314100906081050x5cf50e57r37036a08bf8166e8@mail.gmail.com>
Message-ID: <loom.20090608T181051-839@post.gmane.org>

anatoly techtonik <techtonik <at> gmail.com> writes:
> 
> Just my 0.02 cents, but struggling with all warts of 2.5 subprocessing
> in Windows I would vote for more time for stabilizating things - not
> adding new features. Long awaited subprocess as replacement for
> os.popen() AFAIK is still incapable to asynchronously communicate with
> spawned processes on Windows. I would call this feature as critical
> even on 2.6  As a release testcase - try porting pyexpect module to
> this platform. Absence of native curses/console/readline module also
> makes Python one-way unix shell language while many users expect it to
> be crossplatform.

As always, patches and proposals are welcome!
However, as far as the above issues are concerned, it seems to be less a matter
of time between releases than of motivation to get things done (tm). I don't
think a knowledgeable and determined Windows programmer would have much trouble
solving each of them.

Regards

Antoine.



From fwierzbicki at gmail.com  Mon Jun  8 20:19:10 2009
From: fwierzbicki at gmail.com (Frank Wierzbicki)
Date: Mon, 8 Jun 2009 14:19:10 -0400
Subject: [Python-Dev] Draft PEP 385: Migrating from svn to Mercurial
In-Reply-To: <ea2499da0906080832w3f232721sfcf2cb6a12441965@mail.gmail.com>
References: <ea2499da0906051348g20c204d1rdec62c4b95b1697e@mail.gmail.com>
	<4dab5f760906080729gfb4ff90tdba68d800ac32ab6@mail.gmail.com>
	<ea2499da0906080832w3f232721sfcf2cb6a12441965@mail.gmail.com>
Message-ID: <4dab5f760906081119nfc13950p1201c54dbfc70a8d@mail.gmail.com>

On Mon, Jun 8, 2009 at 11:32 AM, Dirkjan Ochtman<dirkjan at ochtman.nl> wrote:
> I'd say migrating to Python's svn doesn't make a whole lot of sense at
> this point, but I'll leave that to Martin (since he has to do the
> work). For the conversion, I can just as well take the Jython repo
> from your current server. I've started a svnsync job with your repo so
> I can run some test conversions. It's a relatively small repository,
> so it shouldn't be much of a problem.
Great!  Thanks for giving this a try!

For my part I'm fine either way (and I agree that Martin should decide
based on what is best  for him).

-Frank

From g.brandl at gmx.net  Mon Jun  8 20:46:16 2009
From: g.brandl at gmx.net (Georg Brandl)
Date: Mon, 08 Jun 2009 20:46:16 +0200
Subject: [Python-Dev] Draft PEP 385: Migrating from svn to Mercurial
In-Reply-To: <4dab5f760906081119nfc13950p1201c54dbfc70a8d@mail.gmail.com>
References: <ea2499da0906051348g20c204d1rdec62c4b95b1697e@mail.gmail.com>	<4dab5f760906080729gfb4ff90tdba68d800ac32ab6@mail.gmail.com>	<ea2499da0906080832w3f232721sfcf2cb6a12441965@mail.gmail.com>
	<4dab5f760906081119nfc13950p1201c54dbfc70a8d@mail.gmail.com>
Message-ID: <h0jmbl$8fk$1@ger.gmane.org>

Frank Wierzbicki schrieb:
> On Mon, Jun 8, 2009 at 11:32 AM, Dirkjan Ochtman<dirkjan at ochtman.nl> wrote:
>> I'd say migrating to Python's svn doesn't make a whole lot of sense at
>> this point, but I'll leave that to Martin (since he has to do the
>> work). For the conversion, I can just as well take the Jython repo
>> from your current server. I've started a svnsync job with your repo so
>> I can run some test conversions. It's a relatively small repository,
>> so it shouldn't be much of a problem.
> Great!  Thanks for giving this a try!
> 
> For my part I'm fine either way (and I agree that Martin should decide
> based on what is best  for him).

For CPython, we still need SVN, e.g. for the website repo, and several others
like it that probably won't get converted (yet), and also for reference since
we won't convert every old branch to hg.

If that isn't necessary for Jython, I see no reason for moving into
svn.python.org first.

Georg

-- 
Thus spake the Lord: Thou shalt indent with four spaces. No more, no less.
Four shall be the number of spaces thou shalt indent, and the number of thy
indenting shall be four. Eight shalt thou not indent, nor either indent thou
two, excepting that thou then proceed to four. Tabs are right out.


From martin at v.loewis.de  Mon Jun  8 21:51:04 2009
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Mon, 08 Jun 2009 21:51:04 +0200
Subject: [Python-Dev] Draft PEP 385: Migrating from svn to Mercurial
In-Reply-To: <h0jan2$ueb$1@ger.gmane.org>
References: <ea2499da0906051348g20c204d1rdec62c4b95b1697e@mail.gmail.com>	<4dab5f760906080729gfb4ff90tdba68d800ac32ab6@mail.gmail.com>
	<h0jan2$ueb$1@ger.gmane.org>
Message-ID: <4A2D6BA8.2060906@v.loewis.de>

>> Also, Martin suggested we migrate to Python's svn and then go along
>> for the svn->hg ride.  Does that still make sense now that some
>> planning has been done?
> 
> I don't think so.  It should not matter from which repository the conversion
> is done, especially since there is no shared history at all.

Also, I would not merge the Jython SVN into the /projects repository,
but instead set up a separate repository (http://svn.python.org/jython,
and svn+ssh://jython at svn.python.org/).

As discussed at PyCon, we should collect SSH keys for Jython committers
in advance of the svn switchover; we would have to do that either way
(also for hg), so now might be the right time to start doing it.

As for the repository organization: I have now opinion (or, rather,
no experience with hg).

Regards,
Martin


From martin at v.loewis.de  Mon Jun  8 21:57:05 2009
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Mon, 08 Jun 2009 21:57:05 +0200
Subject: [Python-Dev] Draft PEP 385: Migrating from svn to Mercurial
In-Reply-To: <4dab5f760906081119nfc13950p1201c54dbfc70a8d@mail.gmail.com>
References: <ea2499da0906051348g20c204d1rdec62c4b95b1697e@mail.gmail.com>	<4dab5f760906080729gfb4ff90tdba68d800ac32ab6@mail.gmail.com>	<ea2499da0906080832w3f232721sfcf2cb6a12441965@mail.gmail.com>
	<4dab5f760906081119nfc13950p1201c54dbfc70a8d@mail.gmail.com>
Message-ID: <4A2D6D11.4090707@v.loewis.de>

> For my part I'm fine either way (and I agree that Martin should decide
> based on what is best  for him).

See my other message. We need to collect SSH keys, and I don't mind
moving the Jython repository. OTOH, if the Jython repository gets
converted into hg right away, it's certainly (a little) less work.

Of course, the Jython committers would need to relocate their
svn sandboxes now, and then completely abandon them and re-checkout
from hg in a few months, so if you can wait a few months until the
hg conversion is ready, it's also less work for your committers.

FWIW, I really think that PEP 385 should really grow a timeline
pretty soon. Are we going to switch this year, next year, or 2011?

Regards,
Martin

From martin at v.loewis.de  Mon Jun  8 22:01:17 2009
From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=)
Date: Mon, 08 Jun 2009 22:01:17 +0200
Subject: [Python-Dev] Status of 2.7 and 3.2
In-Reply-To: <d34314100906081050x5cf50e57r37036a08bf8166e8@mail.gmail.com>
References: <h0fbn2$gop$1@ger.gmane.org>
	<d34314100906081050x5cf50e57r37036a08bf8166e8@mail.gmail.com>
Message-ID: <4A2D6E0D.6050009@v.loewis.de>

> Just my 0.02 cents, but struggling with all warts of 2.5 subprocessing
> in Windows I would vote for more time for stabilizating things - not
> adding new features. Long awaited subprocess as replacement for
> os.popen() AFAIK is still incapable to asynchronously communicate with
> spawned processes on Windows. I would call this feature as critical
> even on 2.6  As a release testcase - try porting pyexpect module to
> this platform. Absence of native curses/console/readline module also
> makes Python one-way unix shell language while many users expect it to
> be crossplatform.

I am not quite sure whether you are for new features or not. Your
first sentence ("vote for ... not adding new features") seems to
suggest that you would not like to see new features, and your last
sentence ("absence of native curses/console/readline module")
suggests that you *do* want to see new features (namely, a native
curses module, and a native readline module).

Which one is it?

Regards,
Martin

From alexandre at peadrop.com  Mon Jun  8 22:04:27 2009
From: alexandre at peadrop.com (Alexandre Vassalotti)
Date: Mon, 8 Jun 2009 16:04:27 -0400
Subject: [Python-Dev] Draft PEP 385: Migrating from svn to Mercurial
In-Reply-To: <4A2D6D11.4090707@v.loewis.de>
References: <ea2499da0906051348g20c204d1rdec62c4b95b1697e@mail.gmail.com> 
	<4dab5f760906080729gfb4ff90tdba68d800ac32ab6@mail.gmail.com> 
	<ea2499da0906080832w3f232721sfcf2cb6a12441965@mail.gmail.com> 
	<4dab5f760906081119nfc13950p1201c54dbfc70a8d@mail.gmail.com> 
	<4A2D6D11.4090707@v.loewis.de>
Message-ID: <acd65fa20906081304o1fada23ele3854fc856648c8d@mail.gmail.com>

On Mon, Jun 8, 2009 at 3:57 PM, "Martin v. L?wis"<martin at v.loewis.de> wrote:
> FWIW, I really think that PEP 385 should really grow a timeline
> pretty soon. Are we going to switch this year, next year, or 2011?
>

+1

-- Alexandre

From dirkjan at ochtman.nl  Mon Jun  8 22:29:32 2009
From: dirkjan at ochtman.nl (Dirkjan Ochtman)
Date: Mon, 8 Jun 2009 22:29:32 +0200
Subject: [Python-Dev] Draft PEP 385: Migrating from svn to Mercurial
In-Reply-To: <4A2D6D11.4090707@v.loewis.de>
References: <ea2499da0906051348g20c204d1rdec62c4b95b1697e@mail.gmail.com>
	<4dab5f760906080729gfb4ff90tdba68d800ac32ab6@mail.gmail.com>
	<ea2499da0906080832w3f232721sfcf2cb6a12441965@mail.gmail.com>
	<4dab5f760906081119nfc13950p1201c54dbfc70a8d@mail.gmail.com>
	<4A2D6D11.4090707@v.loewis.de>
Message-ID: <ea2499da0906081329r2cbf1a01n28b35f0d552904ec@mail.gmail.com>

On Mon, Jun 8, 2009 at 21:57, "Martin v. L?wis"<martin at v.loewis.de> wrote:
> See my other message. We need to collect SSH keys, and I don't mind
> moving the Jython repository. OTOH, if the Jython repository gets
> converted into hg right away, it's certainly (a little) less work.

Yeah, I guess if you move it to a separate repo on the svn.python.org
server that might be nice. But it's not a big deal either way.

> FWIW, I really think that PEP 385 should really grow a timeline
> pretty soon. Are we going to switch this year, next year, or 2011?

Sorry, I should've been clearer. I fully intend to complete the
conversion in a few months, say by October 1st or sooner. That would
be between 3.1 and whatever the next release will end up being, I
hope? It's just hard to be very specific at this point.

BTW, I tried my hand at rewriting the revlog for the manifest, as
described in the PEP, and it made the full conversion *much* (as in
70%) smaller. I've also been looking at what branches we should keep
and will ask for definitive feedback on that point soonish.

Cheers,

Dirkjan

From fabiofz at gmail.com  Mon Jun  8 23:59:19 2009
From: fabiofz at gmail.com (Fabio Zadrozny)
Date: Mon, 8 Jun 2009 18:59:19 -0300
Subject: [Python-Dev] Cannot set PYTHONPATH with big paths with Python 3.0
	and 3.1
Message-ID: <cfb578b20906081459k3f39ebc4s1dae386343bd1be9@mail.gmail.com>

Hi all,

I've reported bug http://bugs.python.org/issue5924 some time ago and I
think it's a release blocker -- it seems easy to fix, but I don't have
time to actually submit a patch, so, I'd like to draw attention to it,
especially as a release candidate is already out.

Cheers,

Fabio

From martin at v.loewis.de  Tue Jun  9 07:13:41 2009
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Tue, 09 Jun 2009 07:13:41 +0200
Subject: [Python-Dev] Cannot set PYTHONPATH with big paths with Python
 3.0	and 3.1
In-Reply-To: <cfb578b20906081459k3f39ebc4s1dae386343bd1be9@mail.gmail.com>
References: <cfb578b20906081459k3f39ebc4s1dae386343bd1be9@mail.gmail.com>
Message-ID: <4A2DEF85.4070108@v.loewis.de>

> I've reported bug http://bugs.python.org/issue5924 some time ago and I
> think it's a release blocker -- it seems easy to fix, but I don't have
> time to actually submit a patch, so, I'd like to draw attention to it,
> especially as a release candidate is already out.

In absence of a patch, it can't be a release blocker, IMO. It's not a
really critical bug (just an annoying one), as it is possible to work
around it (e.g. by setting sys.path in sitecustomize, or by moving
all modules into a single directory so that sys.path doesn't have to
be that long).

If nobody else has time to submit a patch, either, Python 3.1 will
get released with this bug.

Regards,
Martin

From solipsis at pitrou.net  Tue Jun  9 12:48:14 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Tue, 9 Jun 2009 10:48:14 +0000 (UTC)
Subject: [Python-Dev]
	=?utf-8?q?Cannot_set_PYTHONPATH_with_big_paths_with_?=
	=?utf-8?q?Python_3=2E0=09and_3=2E1?=
References: <cfb578b20906081459k3f39ebc4s1dae386343bd1be9@mail.gmail.com>
	<4A2DEF85.4070108@v.loewis.de>
Message-ID: <loom.20090609T104649-386@post.gmane.org>

Martin v. L?wis <martin <at> v.loewis.de> writes:
> 
> > I've reported bug http://bugs.python.org/issue5924 some time ago and I
> > think it's a release blocker -- it seems easy to fix, but I don't have
> > time to actually submit a patch, so, I'd like to draw attention to it,
> > especially as a release candidate is already out.
> 
> In absence of a patch, it can't be a release blocker, IMO.

I think we've had lots of issues filed as release blockers while they still
didn't have a patch.
As for whether this particular bug deserves to be a blocker, I don't know. It is
certainly annoying, however.

Regards

Antoine.



From amauryfa at gmail.com  Tue Jun  9 12:58:08 2009
From: amauryfa at gmail.com (Amaury Forgeot d'Arc)
Date: Tue, 9 Jun 2009 12:58:08 +0200
Subject: [Python-Dev] Cannot set PYTHONPATH with big paths with Python
	3.0 and 3.1
In-Reply-To: <loom.20090609T104649-386@post.gmane.org>
References: <cfb578b20906081459k3f39ebc4s1dae386343bd1be9@mail.gmail.com>
	<4A2DEF85.4070108@v.loewis.de>
	<loom.20090609T104649-386@post.gmane.org>
Message-ID: <e27efe130906090358p52adc117m73a5619a1ee13ce3@mail.gmail.com>

Hi,

2009/6/9 Antoine Pitrou <solipsis at pitrou.net>:
> Martin v. L?wis <martin <at> v.loewis.de> writes:
>> In absence of a patch, it can't be a release blocker, IMO.
>
> I think we've had lots of issues filed as release blockers while they still
> didn't have a patch.
> As for whether this particular bug deserves to be a blocker, I don't know. It is
> certainly annoying, however.

FYI, I just submitted a patch. It is simple enough to be considered
for inclusion for rc2.

-- 
Amaury Forgeot d'Arc

From techtonik at gmail.com  Tue Jun  9 12:58:50 2009
From: techtonik at gmail.com (anatoly techtonik)
Date: Tue, 9 Jun 2009 13:58:50 +0300
Subject: [Python-Dev] Status of 2.7 and 3.2
In-Reply-To: <4A2D6E0D.6050009@v.loewis.de>
References: <h0fbn2$gop$1@ger.gmane.org>
	<d34314100906081050x5cf50e57r37036a08bf8166e8@mail.gmail.com> 
	<4A2D6E0D.6050009@v.loewis.de>
Message-ID: <d34314100906090358l22a04307w5d1b31b3be517480@mail.gmail.com>

On Mon, Jun 8, 2009 at 11:01 PM, "Martin v. L?wis" <martin at v.loewis.de> wrote:
>
> I am not quite sure whether you are for new features or not. Your
> first sentence ("vote for ... not adding new features") seems to
> suggest that you would not like to see new features, and your last
> sentence ("absence of native curses/console/readline module")
> suggests that you *do* want to see new features (namely, a native
> curses module, and a native readline module).
>
> Which one is it?

I would like to see new features in Python, but only if they are
cross-platform. Unfortunately, I do not possess C skills to make this
happen, nor do I have deep understanding of Microsoft Visual Studio
.project files to port Makefiles and GCC options even when codebase is
available for windows.

The level to make a contribution in this case is too high. The lack of
free time makes it impossible to close the gap in one step leaving
remnants of work-in-process that will make it harder to continue in
future than to start from scratch.

Perhaps the necessity to make it in one huge step could be compensated
by incremental solution approach and development process if there was
obvious centralized place to organize efforts AND provide clear
visibility into progress made so far, initial plan, plan deviation and
current status. Mailing lists are good for discussions, but that's all
- information becomes outdated, text-too-much, no prompt reply often
stops the progress. Perhaps I shift my problem from lack-of-skill into
lack-of-tools direction, but being amazed by efforts people put into
supporting this mailing list I most of the time unable to reply to
emails I get mostly because replies require time for testing and
proving facts.

There is no definite proposal to solve problems of enabling OpenID or
SSO for python.org, of porting curses to windows, of testing
subprocessing etc., but there is an idea that some things could be
given more visibility AND priority to allow people to see the big
picture and focus on outstanding problems. Even though most people
here know about big-things-to-fix, these things doesn't standout from
the pile of issues in roundup.

The thing I miss the most is ability to gather all information
relevant to one problem in one place. This includes timeline with
commits, branches, relevant issues, issue updates, relevant wiki
edits, current focus URLs, _filtered_ threads and refactored comments.
The problem is to ensure that this information is up to date and
provide easy way/instruction how to bring it up to date in case
something is broken. It is not necessary to meet the bus to experience
the effect of bus factor.

-- 
anatoly t.

From kristjan at ccpgames.com  Tue Jun  9 15:49:42 2009
From: kristjan at ccpgames.com (=?iso-8859-1?Q?Kristj=E1n_Valur_J=F3nsson?=)
Date: Tue, 9 Jun 2009 13:49:42 +0000
Subject: [Python-Dev] xmlrpc and http/1.1
Message-ID: <930F189C8A437347B80DF2C156F7EC7F057F9DD5D4@exchis.ccp.ad.local>

Hello there.
I've been doing some work on xmlrpc for CCP in the last weeks.  I'm trying to communicate the results of this back to the python trunk.
I've had the following issues open for a while now:
http://bugs.python.org/issue6096
http://bugs.python.org/issue6099

I would appreciate some comments on this.
Moving forward, I'd like to add support for the "gzip" content encoding for XMLRPC requests and responses.

K
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090609/8ffc5bfe/attachment.htm>

From dirkjan at ochtman.nl  Tue Jun  9 16:52:19 2009
From: dirkjan at ochtman.nl (Dirkjan Ochtman)
Date: Tue, 9 Jun 2009 16:52:19 +0200
Subject: [Python-Dev] Tags from closed branch heads?
Message-ID: <ea2499da0906090752g2705761ch261341b842eac5d1@mail.gmail.com>

So ca8d05e1f1d1 changed the default for repo.heads() to closed=False,
so that calls to heads() by default will not return closed heads.
Unfortunately, this also means that any tags from those heads will not
be considered anymore. That seems inadvertent at best, and may be
should be reverted. Conceptually, it seems wrong that deleting a
branch would also delete all the tags from it. I'd like to propose
this change:

diff --git a/mercurial/localrepo.py b/mercurial/localrepo.py
--- a/mercurial/localrepo.py
+++ b/mercurial/localrepo.py
@@ -320,7 +320,7 @@
     def _hgtagsnodes(self):
         last = {}
         ret = []
-        for node in reversed(self.heads()):
+        for node in reversed(self.heads(closed=True)):
             c = self[node]
             rev = c.rev()
             try:

Is that something we can agree on?

Cheers,

Dirkjan

From kalman.gergely at duodecad.hu  Tue Jun  9 16:46:54 2009
From: kalman.gergely at duodecad.hu (=?ISO-8859-1?Q?K=E1lm=E1n_Gergely?=)
Date: Tue, 09 Jun 2009 16:46:54 +0200
Subject: [Python-Dev] python sendmsg()/recvmsg() implementation
Message-ID: <4A2E75DE.9060508@duodecad.hu>

Hello, my name is Greg.

I've just started using python after many years of C programming, and 
I'm also new to the list. I wanted to clarify this
first, so that maybe I will get a little less beating for my stupidity :)

I use python3 and Linux on arch x86 (production will be on x86_64, 
though this shouldn't matter much).

The application that I'm presently working on is a network server. It 
would use separate processes to accept the
connections, and to do the work (much like how apache prefork does). One 
process accept()s on the original socket and
the received socket (client socket) will be read for a request. After 
the request is received and parsed this process (the
controller) will choose one from its' children that is most capable of 
handling the said request. It would then pass the
file descriptor through a socketpair to the appropriate children and go 
handle the next client. All works fine and smooth,
but I realized that I need sendmsg()/recvmsg() to pass the FD. Since 
these are not implemented in the python socket
module, and Linux has no other way to do this, I'm stuck. Fell flat on 
my face, too :)

Browsing the net I've found a patch to the python core 
(http://bugs.python.org/issue1194378), dated 2005.

First of all, I would like to ask you guys, whether you know of any way 
of doing this FD passing magic, or that you know
of any 3rd party module / patch / anything that can do this for me.

Since I'm fairly familiar with C and (not that much, but I feel the 
power) python, I would take the challenge of writing
it, given that the above code is still somewhat usable. If all else 
fails I would like to have your help to guide me through
this process.

Thanks

Kalman Gergely

From exarkun at divmod.com  Tue Jun  9 17:10:26 2009
From: exarkun at divmod.com (Jean-Paul Calderone)
Date: Tue, 9 Jun 2009 11:10:26 -0400
Subject: [Python-Dev] python sendmsg()/recvmsg() implementation
In-Reply-To: <4A2E75DE.9060508@duodecad.hu>
Message-ID: <20090609151026.22176.614294214.divmod.quotient.3795@henry.divmod.com>

On Tue, 09 Jun 2009 16:46:54 +0200, K?lm?n Gergely <kalman.gergely at duodecad.hu> wrote:
>Hello, my name is Greg.
>
>I've just started using python after many years of C programming, and I'm 
>also new to the list. I wanted to clarify this
>first, so that maybe I will get a little less beating for my stupidity :)
>

Welcome!

>
> [snip]
>
>Browsing the net I've found a patch to the python core 
>(http://bugs.python.org/issue1194378), dated 2005.
>
>First of all, I would like to ask you guys, whether you know of any way of 
>doing this FD passing magic, or that you know
>of any 3rd party module / patch / anything that can do this for me.

Aside from the patch in the tracker, there are several implementations of
these APIs as third-party extension modules.

>
>Since I'm fairly familiar with C and (not that much, but I feel the power) 
>python, I would take the challenge of writing
>it, given that the above code is still somewhat usable. If all else fails I 
>would like to have your help to guide me through
>this process.
>

What would be great is if you could take the patch in the tracker and get
it into shape so that it is suitable for inclusion.  This would involve
three things, I think:

  1. Write unit tests for the functionality (since the patch itself provides
     none)
  2. Update the patch so that it again applies cleanly against trunk
  3. Add documentation for the new APIs

Once this is done, you can get a committer to look at it and either provide
more specific feedback or apply it.

Thanks,

Jean-Paul

From dirkjan at ochtman.nl  Tue Jun  9 17:15:58 2009
From: dirkjan at ochtman.nl (Dirkjan Ochtman)
Date: Tue, 9 Jun 2009 17:15:58 +0200
Subject: [Python-Dev] Tags from closed branch heads?
In-Reply-To: <ea2499da0906090752g2705761ch261341b842eac5d1@mail.gmail.com>
References: <ea2499da0906090752g2705761ch261341b842eac5d1@mail.gmail.com>
Message-ID: <ea2499da0906090815t18c2d1aco4877c4ebd5817c3@mail.gmail.com>

Sorry about that, got dev-lists mixed up in my head somewhere...

On Tue, Jun 9, 2009 at 16:52, Dirkjan Ochtman<dirkjan at ochtman.nl> wrote:
> So ca8d05e1f1d1 changed the default for repo.heads() to closed=False,
> so that calls to heads() by default will not return closed heads.
> Unfortunately, this also means that any tags from those heads will not
> be considered anymore. That seems inadvertent at best, and may be
> should be reverted. Conceptually, it seems wrong that deleting a
> branch would also delete all the tags from it. I'd like to propose
> this change:
>
> diff --git a/mercurial/localrepo.py b/mercurial/localrepo.py
> --- a/mercurial/localrepo.py
> +++ b/mercurial/localrepo.py
> @@ -320,7 +320,7 @@
> ? ? def _hgtagsnodes(self):
> ? ? ? ? last = {}
> ? ? ? ? ret = []
> - ? ? ? ?for node in reversed(self.heads()):
> + ? ? ? ?for node in reversed(self.heads(closed=True)):
> ? ? ? ? ? ? c = self[node]
> ? ? ? ? ? ? rev = c.rev()
> ? ? ? ? ? ? try:
>
> Is that something we can agree on?
>
> Cheers,
>
> Dirkjan
>

From nestornissen at gmail.com  Wed Jun 10 04:58:47 2009
From: nestornissen at gmail.com (Nestor)
Date: Tue, 9 Jun 2009 22:58:47 -0400
Subject: [Python-Dev] Rewrite os.popen using subprocess.Popen?
Message-ID: <4b57b0700906091958k294e9ea7u65fc45a94e272f31@mail.gmail.com>

pydoc.py uses os.popen once on line 1359. According to the
documentation that function is deprecated since Python 2.6. Does it
make sense to rewrite that line using the newer subprocess instead?

I am asking because os.popen stopped working for me in Python 3.1 but
I don't know if it is worth investing time in a module that is
deprecated when the newer one does work.

http://bugs.python.org/issue6236

From josiah.carlson at gmail.com  Wed Jun 10 05:36:19 2009
From: josiah.carlson at gmail.com (Josiah Carlson)
Date: Tue, 9 Jun 2009 20:36:19 -0700
Subject: [Python-Dev] python sendmsg()/recvmsg() implementation
In-Reply-To: <4A2E75DE.9060508@duodecad.hu>
References: <4A2E75DE.9060508@duodecad.hu>
Message-ID: <e6511dbf0906092036o1a484095kbb23033d3f68d809@mail.gmail.com>

On Tue, Jun 9, 2009 at 7:46 AM, K?lm?n
Gergely<kalman.gergely at duodecad.hu> wrote:
> Hello, my name is Greg.
>
> I've just started using python after many years of C programming, and I'm
> also new to the list. I wanted to clarify this
> first, so that maybe I will get a little less beating for my stupidity :)
>
> I use python3 and Linux on arch x86 (production will be on x86_64, though
> this shouldn't matter much).
>
> The application that I'm presently working on is a network server. It would
> use separate processes to accept the
> connections, and to do the work (much like how apache prefork does). One
> process accept()s on the original socket and
> the received socket (client socket) will be read for a request. After the
> request is received and parsed this process (the
> controller) will choose one from its' children that is most capable of
> handling the said request. It would then pass the
> file descriptor through a socketpair to the appropriate children and go
> handle the next client. All works fine and smooth,
> but I realized that I need sendmsg()/recvmsg() to pass the FD. Since these
> are not implemented in the python socket
> module, and Linux has no other way to do this, I'm stuck. Fell flat on my
> face, too :)
>
> Browsing the net I've found a patch to the python core
> (http://bugs.python.org/issue1194378), dated 2005.
>
> First of all, I would like to ask you guys, whether you know of any way of
> doing this FD passing magic, or that you know
> of any 3rd party module / patch / anything that can do this for me.

IIRC, this is already implemented in the multiprocessing package,
which comes standard with Python 2.6 and 3.0 .

It looks like the test is disabled in test/test_multiprocessing.py ,
and re-enabling it (on Windows) produces errors that make me think
it's more an issue with the tests than with multiprocessing itself.

Dig into it, see if you can get the tests to pass :)

 - Josiah

> Since I'm fairly familiar with C and (not that much, but I feel the power)
> python, I would take the challenge of writing
> it, given that the above code is still somewhat usable. If all else fails I
> would like to have your help to guide me through
> this process.
>
> Thanks
>
> Kalman Gergely
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> http://mail.python.org/mailman/options/python-dev/josiah.carlson%40gmail.com
>

From cylix at solace.info  Thu Jun 11 03:38:57 2009
From: cylix at solace.info (Frederick Reeve)
Date: Wed, 10 Jun 2009 20:38:57 -0500
Subject: [Python-Dev] io.BufferedReader.peek() Behaviour in python3.1
Message-ID: <20090610203857.00a019fa@cylix>

Greetings,

As I'm sure you all know there are currently two implementations of
the io module one in python and one much faster implementation in C.
As I recall the python version was used in python3 and the C version is
now used by default in python3.1x.  The behavior of the two is
different in some ways especially regarding io.BufferedReader.peek().

I wrote an email to the authors of the new C code last Friday.  I also
sent a copy of it to the python list for comments.  I was directed by
Antoine Pitrou that I should possibly bring up what I had asked there
here or as a bug report.  I elected to write here because I am not sure
it constitutes a bug.  In my former email I stated I was willing to
submit patches if the old behavior was desired back and the code author
was fine with the changes but didn't want to implement them.  Antoine
said this, "If people need more sophisticated semantics, I'm open to
changing peek() to accommodate it."

Antoine: If I do wrong quoting you are free to chastise me.

So my basic question is:  The behavior of io.BufferedReader.peek() has
changed; is that change something that should: stay
as is, revert or be different entirely?

Here are the two behaviors:

The python version of io.BufferedReader.peek() behaves as:
If the buffer holds less than requested (upto buffersize) read from the
raw stream the difference or up to EOF into the buffer.  Return
requested number of bytes from the start of the buffer.  This may
advance the raw stream but not the local stream.  This version can
guarantee a peek of one chunk (4096 bytes here).

The C version behaves as:
If the buffer holds 0 bytes fill from the raw stream or up to EOF.
Return what is in the buffer.  This may advance the raw stream but not
the local stream.  This version cannot guarantee a peek of over 1 byte
if random length reads are being used at all and not tracked.

Neither case limits what is possible, though, in my opinion, one makes
it easier to accomplish certain things and is more efficient in those
cases.  Take the following two parser examples:

s = io.BufferedReader wrapped stream with no negative seek in most
cases. f = output file handler or such.

python version work flow:

are = re.compile(b'(\r\n|\r|\n)')
while True:
    d = s.peek(4096) # chunk size or so.
    found = are.search(d)
    if found:
        w = d[:found.start()]
        s.seek(f.write(w))
        p = s.peek(74)
        if p.startswith(multipart_boundary):
            s.seek(len(multipart_boundary))
            # other code containing more possible splits
            # across boundaries
            continue
        w  = d[found.start():found.end()]
        s.seek(f.write(w))
        continue
    f.write(d)
    #more code
    continue


C version work flow:

old = b''
are = re.compile(b'(\r\n|\r|\n)')
while True:
    d = old if old != b'' else s.read1(4096)
    found = are.search(d)
    if found:
        w = d[:found.start()]
        f.write(w)
        w = d[found.start():]
        p = w if len(w) >= 74 else w + s.read(73)
        if p.startswith(multipart_boundary):
            # Other code containing more possible splits
            # across boundaries and joins to p.
            old = ???
            continue
        f.write(d[found.start():found.end()])
        old = dd[found.end():] + p
        continue
    old = b''
    f.write(d)
    #more code
    continue

These two examples are not real code but get the point across and are
based off code I put into a multipart parser.  The former written for
python3. I later tried running that parser on 3.1 after the new io
layer and found it broken.  Then rewrote it to the new interface.  That
rewrite is represented in the latter some what.  This is only one
example.  Others may vary, of course. Peek seems to me to have little
use outside of parsers.  Thus I used parsers as an example.

My opinion is that it would be better to have a peek function similar
to the the python implementation in C like as follows:

peek(n):
If n is less than 0, None, or not set; return buffer contents with out
advancing stream position. If the buffer is empty read a full chunk and
return the buffer.  Otherwise return exactly n bytes up to _chunk
size_(not contents) with out advancing the stream position.  If the
buffer contents is less than n, buffer an additional chunk from the
"raw" stream before hand.  If EOF is encountered during any raw read
then return as much as we can up to n. (maybe I should write that in
code form??)

This allows us to obtain the behavior of the current C io
implementation easily and would give us the old python
implementation's behavior when n is given.

The basis for this is:

1. Code reduction and Simplicity

Looking at the examples, the code reduction should be obvious.  The
logic needed to maintain a bytestring of the variously required lengths,
so that it may be checked, would not be necessary. The need to
hold a bytestring to the next iteration would be done away with as well.
Other pieces of data handling would also be simpler.

2. Speed

It would require less handling in the "slower" interpreter if we would
use the buffer in the buffered reader.  Also, all that logic mentioned
in 1 is moved to the faster C code or done away with.  There is very
little necessity for peek outside of parsers, so speed in read-through
and random reads would not have to be affected.

I have other reasons and arguments, but I want to know what every one
else thinks.  This will most likely show me what I have missed or am
not seeing, if anything.  Please I have babbled enough.

Thanks so much for the consideration.

Frederick Reeve


From kristjan at ccpgames.com  Thu Jun 11 17:33:22 2009
From: kristjan at ccpgames.com (=?iso-8859-1?Q?Kristj=E1n_Valur_J=F3nsson?=)
Date: Thu, 11 Jun 2009 15:33:22 +0000
Subject: [Python-Dev] xmlrpc improvements
Message-ID: <930F189C8A437347B80DF2C156F7EC7F057F9DDAF3@exchis.ccp.ad.local>

Hello there.
I've been trying to, in the last weeks, to pass on to the trunk the improvements I've made to XMLRPC.
I've created several issues in order to do these changes incrementally but have got no comments.
Perhaps it is best to show the whole thing in context, then.
I?ve gathered all the changes in a single patch, here:
http://codereview.appspot.com/73041/show

I've also created a corresponding issue in the issue tracker:
http://bugs.python.org/issue6267

I'd really  like to get this stuff in.  The performance gains allowing http1.1 and gzip for xmlrpc are quite significant.
Also, there are bugfixes in there.

Cheers,
Kristj?n


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090611/b58b2de7/attachment.htm>

From david.barrett at asgard.com  Thu Jun 11 22:48:09 2009
From: david.barrett at asgard.com (David A. Barrett)
Date: Thu, 11 Jun 2009 16:48:09 -0400
Subject: [Python-Dev] Adding key and reverse args to bisect
Message-ID: <4A316D89.80702@Asgard.COM>

I propose adding the key parameter
to the bisect.bisect routines.  This
would allow it to be used on lists with
an ordering other than the one "natural"
to the contained objects.

(and anywhere else it makes sense in the
bisect module).

Would this be easy enough to do?

It looks like the main difficulty may have to
do with the C implementation.

I've also noticed that sort appears far faster;
is the C implementation working as expected?

It may be nice to have the reverse parameter
as well, but I'm not sure how that could be
implemented right off the bat.

David A. Barrett

- - - -
Here's an example I've coded up for my own use
(reverse hasn't been implemented here):

def bisect_right(a, x, lo=0, hi=None, key=lambda x: x, reverse=False):
    """A keyed version of bisect.bisect_bisect_right:

    return i: for all e in a[:i] e <= x < f, for all f in a[i:]

    """
    if hi is None: hi = len(a)
    if reverse:
      while lo < hi:
	  mid = (lo+hi)//2
	  if key(a[mid]) < key(x): hi = mid
	  else: lo = mid+1
    else:
      while lo < hi:
	  mid = (lo+hi)//2
	  if key(x) < key(a[mid]): hi = mid
	  else: lo = mid+1
    return lo

def bisect_left(a, x, lo=0, hi=None, key=lambda x: x, reverse=False):
    """A keyed version of bisect.bisect_bisect_left:

    return i: for all e in a[:i] e < x <= f, for all f in a[i:]
    """
    if hi is None: hi = len(a)
    if reverse:
      while lo < hi:
	  mid = (lo+hi)//2
	  if key(x) < key(a[mid]): lo = mid+1
	  else: hi = mid
    else:
      while lo < hi:
	  mid = (lo+hi)//2
	  if key(a[mid]) < key(x): lo = mid+1
	  else: hi = mid
    return lo


From python at rcn.com  Thu Jun 11 23:33:03 2009
From: python at rcn.com (Raymond Hettinger)
Date: Thu, 11 Jun 2009 14:33:03 -0700
Subject: [Python-Dev] Adding key and reverse args to bisect
References: <4A316D89.80702@Asgard.COM>
Message-ID: <AE409B6974E745B1B79291F916C14480@RaymondLaptop1>

[David A. Barrett]
>I propose adding the key parameter
> to the bisect.bisect routines.  This
> would allow it to be used on lists with
> an ordering other than the one "natural"
> to the contained objects.

Algorithmically, the bisect routines are the wrong place to do key lookups.
If you do many calls to bisect(), each one will make multiple calls to the key
function, potentially repeating calls that were made on previous searches.
Instead, it is better to search a list of precomputed keys to find the index
of the record in question.

>>> data = [('red', 5), ('blue', 1), ('yellow', 8), ('black', 0)]
>>> data.sort(key=lambda r: r[1])    # key function called exactly len(data) times
>>> keys = [r[1] for r in data]
>>> data[bisect_left(keys, 0)]
('black', 0)
>>> data[bisect_left(keys, 1)]
('blue', 1)
>>> data[bisect_left(keys, 5)]
('red', 5)
>>> data[bisect_left(keys, 8)]
('yellow', 8)


Raymond

From aahz at pythoncraft.com  Fri Jun 12 01:01:54 2009
From: aahz at pythoncraft.com (Aahz)
Date: Thu, 11 Jun 2009 16:01:54 -0700
Subject: [Python-Dev] Adding key and reverse args to bisect
In-Reply-To: <4A316D89.80702@Asgard.COM>
References: <4A316D89.80702@Asgard.COM>
Message-ID: <20090611230154.GA20007@panix.com>

On Thu, Jun 11, 2009, David A. Barrett wrote:
>
> I propose adding the key parameter to the bisect.bisect routines.
> This would allow it to be used on lists with an ordering other than
> the one "natural" to the contained objects.

Raymond addressed your actual question, but please post suggestions like
this to python-ideas, that's the best place for them.
-- 
Aahz (aahz at pythoncraft.com)           <*>         http://www.pythoncraft.com/

"If you don't know what your program is supposed to do, you'd better not
start writing it."  --Dijkstra

From hfuerstenau at gmx.net  Fri Jun 12 17:18:18 2009
From: hfuerstenau at gmx.net (=?UTF-8?B?SGFnZW4gRsO8cnN0ZW5hdQ==?=)
Date: Fri, 12 Jun 2009 16:18:18 +0100
Subject: [Python-Dev] Iterator version of contextlib.nested
Message-ID: <4A3271BA.2070904@gmx.net>

contextlib.nested has recently been deprecated on grounds of being 
unnecessary now that the with statement accepts multiple context 
managers. However, as has been mentioned before 
(http://mail.python.org/pipermail/python-dev/2009-May/089359.html), that 
doesn't cover the case of a variable number of context managers, i.e.

with contextlib.nested(*list_of_managers) as list_of_results:

or

with contexlib.nested(*iterator_of_managers):

It was suggested that in these use cases a custom context manager should 
be implemented. However, it seems that such an implementation would be 
an almost exact copy of the present code for "nested".

I'm proposing to add an iterator version of "nested" to contextlib 
(possibly called "inested"), which takes an iterable of context managers 
instead of a variable number of parameters. The implementation could be 
taken over from the present "nested", only changing "def 
nested(*managers)" to "def inested(managers)".

This has the advantage that an iterator can be passed to "inested", so 
that each context managers is created in the context of all previous 
ones, which was one of the reasons for introducing the multi-with 
statement in the first place. "contextlib.inested" would therefore be 
the generalization of the multi-with statement to a variable number of 
managers (and "contextlib.nested" would stay deprecated).

- Hagen

From status at bugs.python.org  Fri Jun 12 18:07:24 2009
From: status at bugs.python.org (Python tracker)
Date: Fri, 12 Jun 2009 18:07:24 +0200 (CEST)
Subject: [Python-Dev] Summary of Python tracker Issues
Message-ID: <20090612160724.26F897839F@psf.upfronthosting.co.za>


ACTIVITY SUMMARY (06/05/09 - 06/12/09)
Python tracker at http://bugs.python.org/

To view or respond to any of the issues listed below, click on the issue 
number.  Do NOT respond to this message.


 2225 open (+39) / 15869 closed (+26) / 18094 total (+65)

Open issues with patches:   882

Average duration of open issues: 652 days.
Median duration of open issues: 402 days.

Open Issues Breakdown
   open  2197 (+39)
pending    28 ( +0)

Issues Created Or Reopened (68)
_______________________________

Issue with RotatingFileHandler logging handler on Windows        06/08/09
CLOSED http://bugs.python.org/issue4749    reopened r.david.murray                
                                                                               

add disable_nagle_algorithm to SocketServer.TCPServer            06/08/09
       http://bugs.python.org/issue6192    reopened pitrou                        
       patch, patch, easy                                                      

test_float fails on Windows                                      06/05/09
CLOSED http://bugs.python.org/issue6198    reopened eric.smith                    
                                                                               

path separator output ignores shell's path separator: / instead  06/05/09
       http://bugs.python.org/issue6208    created  ThurnerRupert                 
                                                                               

compilation error in std. lib. module shutil (Python 3.1rc1, pla 06/05/09
CLOSED http://bugs.python.org/issue6209    created  dpodbori                      
                                                                               

Exception Chaining missing method for suppressing context        06/05/09
       http://bugs.python.org/issue6210    created  poke                          
                                                                               

[Tutorial] Section 4.7.2 has a wrong description of an example   06/05/09
CLOSED http://bugs.python.org/issue6211    created  cofi                          
       patch                                                                   

piped input                                                      06/05/09
CLOSED http://bugs.python.org/issue6212    created  rtmq                          
                                                                               

Incremental encoder incompatibility between 2.x and py3k         06/05/09
       http://bugs.python.org/issue6213    created  pitrou                        
                                                                               

test__locale broken on trunk                                     06/05/09
CLOSED http://bugs.python.org/issue6214    created  pitrou                        
                                                                               

Backport the IO lib to trunk                                     06/05/09
       http://bugs.python.org/issue6215    created  pitrou                        
       patch                                                                   

Raise Unicode KEEPALIVE_SIZE_LIMIT from 9 to 32?                 06/06/09
       http://bugs.python.org/issue6216    created  tjreedy                       
                                                                               

Add _io._TextIOWrapper.errors                                    06/06/09
CLOSED http://bugs.python.org/issue6217    created  pjenvey                       
       patch                                                                   

Make io.BytesIO and io.StringIO picklable.                       06/06/09
       http://bugs.python.org/issue6218    created  alexandre.vassalotti          
       patch, patch                                                            

nested list value change                                         06/06/09
CLOSED http://bugs.python.org/issue6219    created  pushkarparanjpe               
                                                                               

typo: opne in "doanddont"                                        06/06/09
CLOSED http://bugs.python.org/issue6220    created  cate                          
                                                                               

Windows buildbot failure in test_winreg                          06/06/09
CLOSED http://bugs.python.org/issue6221    created  pitrou                        
                                                                               

2to3 except fixer failed in certain case                         06/06/09
CLOSED http://bugs.python.org/issue6222    created  bhy                           
                                                                               

Make _PyUnicode_AsString as public API                           06/06/09
CLOSED http://bugs.python.org/issue6223    created  bhy                           
                                                                               

References to JPython                                            06/06/09
CLOSED http://bugs.python.org/issue6224    created  thijs                         
                                                                               

Fixing several minor bugs in Tkinter.Canvas and one in Misc._con 06/06/09
       http://bugs.python.org/issue6225    created  gpolo                         
       patch                                                                   

Inconsistent 'file' vs 'stream' kwarg in pprint, other stdlibs   06/06/09
       http://bugs.python.org/issue6226    created  pjenvey                       
                                                                               

doctest_aliases doesn't test duplicate removal                   06/07/09
       http://bugs.python.org/issue6227    created  abbeyj                        
       patch                                                                   

round() error                                                    06/07/09
CLOSED http://bugs.python.org/issue6228    created  steve21                       
                                                                               

Installation python on mac                                       06/07/09
CLOSED http://bugs.python.org/issue6229    created  kostonstyle                   
                                                                               

ElementTree.Element and cElementTree.Element have slightly diffe 06/07/09
       http://bugs.python.org/issue6230    created  Neil Muller                   
       patch                                                                   

ElementInclude may drop text                                     06/07/09
       http://bugs.python.org/issue6231    created  Neil Muller                   
       patch                                                                   

Improve test coverage of ElementTree and cElementTree            06/07/09
       http://bugs.python.org/issue6232    created  Neil Muller                   
                                                                               

ElementTree (py3k) doesn't properly encode characters that can't 06/07/09
       http://bugs.python.org/issue6233    created  Neil Muller                   
       patch                                                                   

cgi.FieldStorage is broken when given POST data                  06/07/09
       http://bugs.python.org/issue6234    created  efosmark                      
                                                                               

\d missing from effects of re.ASCII flag                         06/08/09
CLOSED http://bugs.python.org/issue6235    created  MLModel                       
                                                                               

os.popen causes illegal seek on AIX in Python 3.1rc              06/08/09
       http://bugs.python.org/issue6236    created  nestor                        
                                                                               

Build errors when using LDFLAGS="-Wl,--no-undefined"             06/08/09
       http://bugs.python.org/issue6237    created  fundawang                     
                                                                               

string.ljust documentation is missing optional fillchar descript 06/08/09
CLOSED http://bugs.python.org/issue6238    created  jaraco                        
                                                                               

c_char_p return value returns string, not bytes                  06/08/09
       http://bugs.python.org/issue6239    created  georg.brandl                  
                                                                               

API to get source encoding as defined by PEP 263                 06/08/09
CLOSED http://bugs.python.org/issue6240    created  srid                          
                                                                               

Better type checking for the arguments of io.StringIO            06/08/09
       http://bugs.python.org/issue6241    created  alexandre.vassalotti          
       patch                                                                   

Fix reference leak in io.StringIO                                06/08/09
       http://bugs.python.org/issue6242    created  alexandre.vassalotti          
       patch                                                                   

getkey() can segfault in combination with curses.ungetch()       06/08/09
       http://bugs.python.org/issue6243    created  Trundle                       
       patch                                                                   

Support for tcl 8.6                                              06/08/09
       http://bugs.python.org/issue6244    created  misc                          
       patch                                                                   

Add "intel" universal architecture on OSX                        06/08/09
       http://bugs.python.org/issue6245    created  ronaldoussoren                
       patch, needs review                                                     

Python debugger ignores exception if instructed to return from g 06/08/09
       http://bugs.python.org/issue6246    created  inducer                       
                                                                               

should we include argparse                                       06/09/09
CLOSED http://bugs.python.org/issue6247    reopened rickysarraf                   
                                                                               

TCP Sockets not closed by TCPServer and StreamRequestHandler     06/09/09
       http://bugs.python.org/issue6248    created  krisvale                      
       patch, patch, needs review                                              

Error Prompt                                                     06/10/09
CLOSED http://bugs.python.org/issue6249    created  SonMarvin                     
                                                                               

Python compiles dead code                                        06/10/09
       http://bugs.python.org/issue6250    created  abbeyj                        
       patch                                                                   

c++ extension module implementation guide/example in extending/e 06/10/09
       http://bugs.python.org/issue6251    created  subgeometer                   
                                                                               

logging midnight rotation                                        06/10/09
CLOSED http://bugs.python.org/issue6252    created  SanityIO                      
                                                                               

optparse.OptionParser.get_usage uses wrong formatter             06/10/09
       http://bugs.python.org/issue6253    created  ccx                           
       patch                                                                   

tarfile unnecessarily requires seekable files                    06/10/09
CLOSED http://bugs.python.org/issue6254    created  johnsonm                      
                                                                               

PyInt_FromSize_t is undocumented.                                06/10/09
       http://bugs.python.org/issue6255    created  naoki                         
                                                                               

Wrong stacklevel in warning for contextlib.nested                06/10/09
CLOSED http://bugs.python.org/issue6256    created  hagen                         
       patch                                                                   

Idle terminates on source save while debugging                   06/10/09
       http://bugs.python.org/issue6257    created  andyharrington                
                                                                               

distributions built with bdist_msi on 64-bit Windows fail	to ins 06/10/09
       http://bugs.python.org/issue6258    created  jaraco                        
                                                                               

ctypes pointer arithmetic                                        06/10/09
       http://bugs.python.org/issue6259    created  theller                       
       patch                                                                   

os.utime should allow None values for ATIME or MTIME             06/10/09
CLOSED http://bugs.python.org/issue6260    created  purpleidea                    
                                                                               

Clarify behaviour of random.uniform                              06/10/09
CLOSED http://bugs.python.org/issue6261    created  marketdickinson               
                                                                               

VS 2008 binaries                                                 06/11/09
CLOSED http://bugs.python.org/issue6262    created  bellenot                      
                                                                               

syntax error in get_msvcr                                        06/11/09
CLOSED http://bugs.python.org/issue6263    created  tarek                         
                                                                               

Misleading instructions for building extensions with mingw       06/11/09
CLOSED http://bugs.python.org/issue6264    created  bdew                          
                                                                               

cElementTree & ElementTree use different exceptions for XML Erro 06/11/09
       http://bugs.python.org/issue6265    created  Neil Muller                   
                                                                               

cElementTree.iterparse & ElementTree.iterparse return differentl 06/11/09
       http://bugs.python.org/issue6266    created  Neil Muller                   
                                                                               

Cumulative patch to http and xmlrpc                              06/11/09
       http://bugs.python.org/issue6267    created  krisvale                      
                                                                               

Seeking to the beginning of a text file a second time will retur 06/11/09
       http://bugs.python.org/issue6268    created  eggy                          
                                                                               

threading documentation makes no mention of the GIL              06/11/09
       http://bugs.python.org/issue6269    created  segfaulthunter                
       patch                                                                   

Menu deletecommand fails if command is already deleted           06/11/09
       http://bugs.python.org/issue6270    created  gregcouch                     
       patch                                                                   

mmap: don't close file description if fd=-1                      06/12/09
       http://bugs.python.org/issue6271    created  haypo                         
       patch                                                                   

Upgrading xml.etree to ElementTree 1.3                           06/12/09
       http://bugs.python.org/issue6272    created  orsenthil                     
                                                                               



Issues Now Closed (52)
______________________

Improve the hackish runtime_library_dirs support for gcc          653 days
       http://bugs.python.org/issue1032    alexandre.vassalotti          
       patch                                                                   

Builtin round function is sometimes inaccurate for floats          50 days
       http://bugs.python.org/issue1869    marketdickinson               
       patch                                                                   

Set up "supported"-only buildbot waterfall view                   452 days
       http://bugs.python.org/issue2376    exarkun                       
                                                                               

"No windows home dir" doc error                                   384 days
       http://bugs.python.org/issue2922    r.david.murray                
                                                                               

subprocess (Replacing popen) - add a warning / hint               382 days
       http://bugs.python.org/issue2947    r.david.murray                
       patch, easy                                                             

__newobj__ pickle feature is not documented                       272 days
       http://bugs.python.org/issue3816    alexandre.vassalotti          
       easy                                                                    

ctypes documentation                                              208 days
       http://bugs.python.org/issue4309    mnewman                       
                                                                               

Issue with RotatingFileHandler logging handler on Windows           3 days
       http://bugs.python.org/issue4749    vsajip                        
                                                                               

Using LDFLAGS='-rpath=\$$LIB:/some/other/path' ./configure break  121 days
       http://bugs.python.org/issue5201    tarek                         
       patch                                                                   

test_fdopen fails with vs2005,	release build on Windows 2000       70 days
       http://bugs.python.org/issue5623    krisvale                      
       patch                                                                   

"No such file or directory" with framework build under MacOS 10.   48 days
       http://bugs.python.org/issue5809    ronaldoussoren                
                                                                               

When setting complete PYTHONPATH on Python 3.x, paths in the PYT   36 days
       http://bugs.python.org/issue5924    amaury.forgeotdarc            
       patch                                                                   

Make pickle generated by Python 3.x compatible with 2.x and vice   12 days
       http://bugs.python.org/issue6137    rhettinger                    
       patch                                                                   

Python 3.1 rc1 will not build on OS X 10.5.7 with macports libin    8 days
       http://bugs.python.org/issue6154    brett.cannon                  
       patch                                                                   

fcntl footnote about O_SHLOCK and O_EXLOCK is misleading            4 days
       http://bugs.python.org/issue6194    georg.brandl                  
       easy                                                                    

Serious regression in doctest in Py3.1rc1                           8 days
       http://bugs.python.org/issue6195    r.david.murray                
       patch                                                                   

tarfile.extractall(readaccess=True)                                 1 days
       http://bugs.python.org/issue6196    lars.gustaebel                
                                                                               

test_float fails on Windows                                         4 days
       http://bugs.python.org/issue6198    eric.smith                    
                                                                               

test_winreg fails                                                   5 days
       http://bugs.python.org/issue6201    amaury.forgeotdarc            
                                                                               

Obsolete default file encoding "mac-roman" on OS X, not influenc    2 days
       http://bugs.python.org/issue6202    nad                           
       patch                                                                   

Missing reference in section 4.6 to chapter on classes              1 days
       http://bugs.python.org/issue6204    georg.brandl                  
       patch                                                                   

Correct a trivial typo introduced by r73238.                        1 days
       http://bugs.python.org/issue6206    georg.brandl                  
       patch                                                                   

Simple For-Loops                                                    0 days
       http://bugs.python.org/issue6207    pitrou                        
                                                                               

compilation error in std. lib. module shutil (Python 3.1rc1, pla    0 days
       http://bugs.python.org/issue6209    benjamin.peterson             
                                                                               

[Tutorial] Section 4.7.2 has a wrong description of an example      1 days
       http://bugs.python.org/issue6211    georg.brandl                  
       patch                                                                   

piped input                                                         0 days
       http://bugs.python.org/issue6212    georg.brandl                  
                                                                               

test__locale broken on trunk                                        1 days
       http://bugs.python.org/issue6214    ezio.melotti                  
                                                                               

Add _io._TextIOWrapper.errors                                       1 days
       http://bugs.python.org/issue6217    benjamin.peterson             
       patch                                                                   

nested list value change                                            0 days
       http://bugs.python.org/issue6219    rhettinger                    
                                                                               

typo: opne in "doanddont"                                           0 days
       http://bugs.python.org/issue6220    ezio.melotti                  
                                                                               

Windows buildbot failure in test_winreg                             1 days
       http://bugs.python.org/issue6221    benjamin.peterson             
                                                                               

2to3 except fixer failed in certain case                            0 days
       http://bugs.python.org/issue6222    benjamin.peterson             
                                                                               

Make _PyUnicode_AsString as public API                              0 days
       http://bugs.python.org/issue6223    benjamin.peterson             
                                                                               

References to JPython                                               0 days
       http://bugs.python.org/issue6224    georg.brandl                  
                                                                               

round() error                                                       0 days
       http://bugs.python.org/issue6228    marketdickinson               
                                                                               

Installation python on mac                                          0 days
       http://bugs.python.org/issue6229    benjamin.peterson             
                                                                               

\d missing from effects of re.ASCII flag                            0 days
       http://bugs.python.org/issue6235    georg.brandl                  
                                                                               

string.ljust documentation is missing optional fillchar descript    0 days
       http://bugs.python.org/issue6238    georg.brandl                  
                                                                               

API to get source encoding as defined by PEP 263                    0 days
       http://bugs.python.org/issue6240    benjamin.peterson             
                                                                               

should we include argparse                                          3 days
       http://bugs.python.org/issue6247    bethard                       
                                                                               

Error Prompt                                                        0 days
       http://bugs.python.org/issue6249    georg.brandl                  
                                                                               

logging midnight rotation                                           1 days
       http://bugs.python.org/issue6252    vsajip                        
                                                                               

tarfile unnecessarily requires seekable files                       2 days
       http://bugs.python.org/issue6254    lars.gustaebel                
                                                                               

Wrong stacklevel in warning for contextlib.nested                   0 days
       http://bugs.python.org/issue6256    rhettinger                    
       patch                                                                   

os.utime should allow None values for ATIME or MTIME                2 days
       http://bugs.python.org/issue6260    purpleidea                    
                                                                               

Clarify behaviour of random.uniform                                 1 days
       http://bugs.python.org/issue6261    rhettinger                    
                                                                               

VS 2008 binaries                                                    1 days
       http://bugs.python.org/issue6262    loewis                        
                                                                               

syntax error in get_msvcr                                           0 days
       http://bugs.python.org/issue6263    tarek                         
                                                                               

Misleading instructions for building extensions with mingw          1 days
       http://bugs.python.org/issue6264    bdew                          
                                                                               

IDLE / PyOS_InputHook                                            2108 days
       http://bugs.python.org/issue798058  gpolo                         
                                                                               

import on Windows: please call SetErrorMode first                1664 days
       http://bugs.python.org/issue1069410 srid                          
                                                                               

logging module / wrong bytecode?                                  876 days
       http://bugs.python.org/issue1633605 annacoder                     
                                                                               



Top Issues Most Discussed (10)
______________________________

 31 Issue with RotatingFileHandler logging handler on Windows          3 days
closed  http://bugs.python.org/issue4749   

 12 should we include argparse                                         3 days
closed  http://bugs.python.org/issue6247   

 10 3.x locale does not default to C, contrary to the documentation    7 days
open    http://bugs.python.org/issue6203   

  8 Make pickle generated by Python 3.x compatible with 2.x and vic   12 days
closed  http://bugs.python.org/issue6137   

  8 Make logging configuration files easier to use                    15 days
open    http://bugs.python.org/issue6136   

  7 Serious regression in doctest in Py3.1rc1                          8 days
closed  http://bugs.python.org/issue6195   

  7 GCC detection for runtime_library_dirs when ccache is used      1403 days
open    http://bugs.python.org/issue1254718

  5 tarfile unnecessarily requires seekable files                      2 days
closed  http://bugs.python.org/issue6254   

  5 Add "intel" universal architecture on OSX                          4 days
open    http://bugs.python.org/issue6245   

  5 typo: opne in "doanddont"                                          0 days
closed  http://bugs.python.org/issue6220   




From facundobatista at gmail.com  Fri Jun 12 23:25:30 2009
From: facundobatista at gmail.com (Facundo Batista)
Date: Fri, 12 Jun 2009 18:25:30 -0300
Subject: [Python-Dev] Avoiding file descriptors leakage in subprocess.Popen()
Message-ID: <e04bdf310906121425x72a7c033wec71500712759375@mail.gmail.com>

In a long lived process at work, we started leaking file descriptors.

The problem were that subprocess.Popen._execute_child() method creates
two files descriptors through a call to os.pipe(), and after some work
it closes them. But an os.read() on the descriptor was interrupted
(EINTR), so an exception was raised, and the descriptors were not
closed... leakage!

This problem is easy to fix (I have a patch that fixes it, all tests
pass ok, see http://bugs.python.org/issue6274).

So, why this mail? I just think that the fix is ugly... it's not
elegant. It has the following structure:


errpipe_read, errpipe_write = os.pipe()
try:
    try:
        .....
        .....
        .....
        .....
        .....
        .....
    finally:
        os.close(errpipe_write)
    .....
    .....
    .....
finally:
    os.close(errpipe_read)


I just don't like a huge try/finally... but as FDs are just ints, do
you think is there a better way to handle it?

Thank you!!

Regards,

--
. ? ?Facundo

Blog: http://www.taniquetil.com.ar/plog/
PyAr: http://www.python.org/ar/

From omega_force2003 at yahoo.com  Sat Jun 13 01:01:43 2009
From: omega_force2003 at yahoo.com (OMEGA RED)
Date: Fri, 12 Jun 2009 16:01:43 -0700 (PDT)
Subject: [Python-Dev] FINAL PROPULSION OPEN SOURCE ENGINE VARIANT FOR THE
	F-35 JOINT STRIKE FIGHTER
Message-ID: <958091.40012.qm@web34401.mail.mud.yahoo.com>


?







TO ALL,
?
?
I AM HAPPY TO SAY THAT I AM NOW IN THE PROCESS OF DEVELOPING A WEBSITE FOR THE STATED PURPOSE OF COMBINING SOME OF THE BEST MINDS IN THE WORLD.? THIS PROJECT WILL BE A MEANS TO DEVELOP THE FIRST AND ONLY COMPLETELY OPEN SOURCE VARIANT OF THE PROPULSION ENGINE FOR THE F-35 JOINT STRIKE FIGHTER.? WHICH MEANS THAT EVERYTHING ABOUT THE DESIGN, DEVELOPMENT, AND CONSTRUCTION OF THE ENGINE WILL BE COMPLETELY AVAILABLE TO THE PUBLIC AND TO ANY ENGINE MANUFACTURE FREE OF ANY REQUIREMENTS.? 
?
THIS ENGINE WILL BE CALLED THE PHOENIX NexT F-200 ENGINE.? 
?
THIS ENGINE WILL ALSO BE A POTENTIAL THIRD AND FINAL ALTERNATIVE ENGINE FOR THE JOINT STRIKE FIGHTER.? 
?
THIS ENGINE WILL BE STATED AS BEING A COMPETITOR WITH BOTH THE F-135 AND THE F-136 ENGINE, WHICH ARE PROVIDED BY BOTH PRATT-WHITNEY AND GENERAL ELECTRIC/ ROLLS ROYCE.? ? 
?
THIS ENGINE WILL UTILIZE PYTHON FOR THE MAJOR PORTION OF THE SIMULATION, RESEARCH AND DEVELOPMENT STAGES FOR THE DEVELOPMENT OF THE ENGINE.? THE CONSTRUCTION OF THE MATHEMATICAL PROCESSING OF THE REQUIRED NONLINEAR CONTROLS WILL BE BASED ON A SPECIALIZED VERSION OF GROUP THEORY FOR THE DESIGN AND CONSTRUCTION OF THE NONLINEAR CONTROL SYSTEMS.? 
?
?
THE PURPOSE OF THE DEVELOPMENT OF THE ENGINE WILL BE TO CONSTRUCT AN ENGINE FOR THE F-35 THAT WILL BE AS RELIABLE AND CAPABLE AS THE F-135 AND THE F-136.? 
?
HOWEVER, THE PHOENIX NexT ENGINE WILL ALSO BE STATED AS BEING ONLY AROUND $20,000 TO $110,000 DOLLARS TO BEING PURCHASED BY AN OPERATOR OF THE F-35, WHICH WILL BE AN INSIGNIFICANT COST COMPARED TO THE COST OF BOTH THE F-135 AND THE F-136 ENGINE (10 MILLION DOLLARS PER UNIT).? 
?
HOW MANY OUT THERE WANT TO HELP IN THIS ENDEAVOR ?? I BELIEVE THAT WE ALL?CAN SUCCEED WHERE BOTH PRATT-WHITNEY AND GE/ROLLS ROYCE HAVE FAILED TO DEVELOP AN INEXPENSIVE AND RELIABLE ENGINE FOR THE F-35.
?
THANKS,? 
?
W108DAB


      
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090612/4c1b6cf5/attachment.htm>

From matthew at matthewwilkes.co.uk  Sat Jun 13 01:53:44 2009
From: matthew at matthewwilkes.co.uk (Matthew Wilkes)
Date: Sat, 13 Jun 2009 00:53:44 +0100
Subject: [Python-Dev] FINAL PROPULSION OPEN SOURCE ENGINE VARIANT FOR
	THE F-35 JOINT STRIKE FIGHTER
In-Reply-To: <958091.40012.qm@web34401.mail.mud.yahoo.com>
References: <958091.40012.qm@web34401.mail.mud.yahoo.com>
Message-ID: <4B23F475-2DDE-4929-8993-B568922F5A5B@matthewwilkes.co.uk>


On 13 Jun 2009, at 00:01, OMEGA RED wrote:

> DEVELOP THE FIRST AND ONLY COMPLETELY OPEN SOURCE VARIANT OF THE  
> PROPULSION ENGINE FOR THE F-35 JOINT STRIKE FIGHTER

You're unlikely to find many people who want to help use open-source  
to facilitate murder.

> HOW MANY OUT THERE WANT TO HELP IN THIS ENDEAVOR ?

Nobody here.  You're off topic, this list is for development of  
Python, not pet projects.

Matt

From guido at python.org  Sat Jun 13 02:00:52 2009
From: guido at python.org (Guido van Rossum)
Date: Fri, 12 Jun 2009 17:00:52 -0700
Subject: [Python-Dev] FINAL PROPULSION OPEN SOURCE ENGINE VARIANT FOR
	THE F-35 JOINT STRIKE FIGHTER
In-Reply-To: <4B23F475-2DDE-4929-8993-B568922F5A5B@matthewwilkes.co.uk>
References: <958091.40012.qm@web34401.mail.mud.yahoo.com>
	<4B23F475-2DDE-4929-8993-B568922F5A5B@matthewwilkes.co.uk>
Message-ID: <ca471dc20906121700h14133f0aj9b39ed75ed2f37b3@mail.gmail.com>

On Fri, Jun 12, 2009 at 4:53 PM, Matthew
Wilkes<matthew at matthewwilkes.co.uk> wrote:
>
> On 13 Jun 2009, at 00:01, OMEGA RED wrote:
>
>> DEVELOP THE FIRST AND ONLY COMPLETELY OPEN SOURCE VARIANT OF THE
>> PROPULSION ENGINE FOR THE F-35 JOINT STRIKE FIGHTER
>
> You're unlikely to find many people who want to help use open-source to
> facilitate murder.

That's a rather presumptuous statement. Despite the poster's use of
SHOUTING I don't see a reason to tell them they should use proprietary
software just because you disagree with their politics. (Hey, I
disagree with Eric Raymond's *and* Richard Stallman's politics. :-)

>> HOW MANY OUT THERE WANT TO HELP IN THIS ENDEAVOR ?
>
> Nobody here. ?You're off topic, this list is for development of Python, not
> pet projects.

True. Comp.lang.python might be a better place.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From benjamin at python.org  Sat Jun 13 02:05:06 2009
From: benjamin at python.org (Benjamin Peterson)
Date: Fri, 12 Jun 2009 19:05:06 -0500
Subject: [Python-Dev] FINAL PROPULSION OPEN SOURCE ENGINE VARIANT FOR
	THE F-35 JOINT STRIKE FIGHTER
In-Reply-To: <ca471dc20906121700h14133f0aj9b39ed75ed2f37b3@mail.gmail.com>
References: <958091.40012.qm@web34401.mail.mud.yahoo.com>
	<4B23F475-2DDE-4929-8993-B568922F5A5B@matthewwilkes.co.uk>
	<ca471dc20906121700h14133f0aj9b39ed75ed2f37b3@mail.gmail.com>
Message-ID: <1afaf6160906121705r2e2acac2n3568a78469317f51@mail.gmail.com>

2009/6/12 Guido van Rossum <guido at python.org>:
> On Fri, Jun 12, 2009 at 4:53 PM, Matthew
> Wilkes<matthew at matthewwilkes.co.uk> wrote:
>> On 13 Jun 2009, at 00:01, OMEGA RED wrote:
>>> HOW MANY OUT THERE WANT TO HELP IN THIS ENDEAVOR ?
>>
>> Nobody here. ?You're off topic, this list is for development of Python, not
>> pet projects.
>
> True. Comp.lang.python might be a better place.

Actually, I think the Python community might be better off if he went
to comp.lang.perl.



-- 
Regards,
Benjamin

From matthew at matthewwilkes.co.uk  Sat Jun 13 02:09:24 2009
From: matthew at matthewwilkes.co.uk (Matthew Wilkes)
Date: Sat, 13 Jun 2009 01:09:24 +0100
Subject: [Python-Dev] FINAL PROPULSION OPEN SOURCE ENGINE VARIANT FOR
	THE F-35 JOINT STRIKE FIGHTER
In-Reply-To: <ca471dc20906121700h14133f0aj9b39ed75ed2f37b3@mail.gmail.com>
References: <958091.40012.qm@web34401.mail.mud.yahoo.com>
	<4B23F475-2DDE-4929-8993-B568922F5A5B@matthewwilkes.co.uk>
	<ca471dc20906121700h14133f0aj9b39ed75ed2f37b3@mail.gmail.com>
Message-ID: <7CFD8BD5-784E-4927-8A7C-8422472B441E@matthewwilkes.co.uk>


On 13 Jun 2009, at 01:00, Guido van Rossum wrote:

> That's a rather presumptuous statement. Despite the poster's use of
> SHOUTING I don't see a reason to tell them they should use proprietary
> software just because you disagree with their politics

Oh, I didn't mean they should use proprietary software, just that in  
my experience the kind of people who are active in open source are  
quite anti-war, green, etc.  There are notable exceptions, but I know  
people who worry that their work will have military applications, and  
who turn down projects because of it.

You're more likely to get people who are interested in aviation with  
some programming background (universities do teach coding to  
engineers, afterall), than the other way around.

Matt


From lists at cheimes.de  Sat Jun 13 02:06:22 2009
From: lists at cheimes.de (Christian Heimes)
Date: Sat, 13 Jun 2009 02:06:22 +0200
Subject: [Python-Dev] Avoiding file descriptors leakage in
	subprocess.Popen()
In-Reply-To: <e04bdf310906121425x72a7c033wec71500712759375@mail.gmail.com>
References: <e04bdf310906121425x72a7c033wec71500712759375@mail.gmail.com>
Message-ID: <4A32ED7E.4070504@cheimes.de>

Facundo Batista wrote:
> I just don't like a huge try/finally... but as FDs are just ints, do
> you think is there a better way to handle it?

How about a nice 'n shiny context wrapper for the pipe:


import os

class Pipe(object):
    def __enter__(self):
        self.read, self.write = os.pipe()
        return self.read, self.write

    def __exit__(self, *args):
        try:
            os.close(self.read)
        finally:
            # make sure that write is closed even if
            # self.read can't be closed
            os.close(self.write)


with Pipe() as (read, write):
    print read, write

Christian

PS and nit pick:
File descriptor are opaque resource handlers which just happened to be
ints. They should be treated as magic cookies.


From lists at cheimes.de  Sat Jun 13 02:06:13 2009
From: lists at cheimes.de (Christian Heimes)
Date: Sat, 13 Jun 2009 02:06:13 +0200
Subject: [Python-Dev] Avoiding file descriptors leakage in
	subprocess.Popen()
In-Reply-To: <e04bdf310906121425x72a7c033wec71500712759375@mail.gmail.com>
References: <e04bdf310906121425x72a7c033wec71500712759375@mail.gmail.com>
Message-ID: <4A32ED75.7060803@cheimes.de>

Facundo Batista wrote:
> I just don't like a huge try/finally... but as FDs are just ints, do
> you think is there a better way to handle it?

How about a nice 'n shiny context wrapper for the pipe:


import os

class Pipe(object):
    def __enter__(self):
        self.read, self.write = os.pipe()
        return self.read, self.write

    def __exit__(self, *args):
        try:
            os.close(self.read)
        finally:
            # make sure that write is closed even if
            # self.read can't be closed
            os.close(self.write)


with Pipe() as (read, write):
    print read, write

Christian

PS and nit pick:
File descriptor are opaque resource handlers which just happened to be
ints. They should be treated as magic cookies.


From python at rcn.com  Sat Jun 13 02:17:50 2009
From: python at rcn.com (Raymond Hettinger)
Date: Fri, 12 Jun 2009 17:17:50 -0700
Subject: [Python-Dev] FINAL PROPULSION OPEN SOURCE ENGINE VARIANT FORTHE
	F-35 JOINT STRIKE FIGHTER
References: <958091.40012.qm@web34401.mail.mud.yahoo.com><4B23F475-2DDE-4929-8993-B568922F5A5B@matthewwilkes.co.uk><ca471dc20906121700h14133f0aj9b39ed75ed2f37b3@mail.gmail.com>
	<7CFD8BD5-784E-4927-8A7C-8422472B441E@matthewwilkes.co.uk>
Message-ID: <73492FDFAB444938B5185B04F8E39DF4@RaymondLaptop1>


[Matthew Wilkes]
> Oh, I didn't mean they should use proprietary software, just that in  
> my experience the kind of people who are active in open source are  
> quite anti-war, green, etc.  There are notable exceptions, but I know  
> people who worry that their work will have military applications, and  
> who turn down projects because of it.

I question the whole notion of using open source in military weapons.
It seems like a rather basic violation of operational security.  Perhaps
your enemies will exploit your bugs instead of nicely reporting them
and submitting patches on SourceForge ;-)


Raymond

From guido at python.org  Sat Jun 13 02:19:30 2009
From: guido at python.org (Guido van Rossum)
Date: Fri, 12 Jun 2009 17:19:30 -0700
Subject: [Python-Dev] FINAL PROPULSION OPEN SOURCE ENGINE VARIANT FORTHE
	F-35 JOINT STRIKE FIGHTER
In-Reply-To: <73492FDFAB444938B5185B04F8E39DF4@RaymondLaptop1>
References: <958091.40012.qm@web34401.mail.mud.yahoo.com>
	<4B23F475-2DDE-4929-8993-B568922F5A5B@matthewwilkes.co.uk> 
	<ca471dc20906121700h14133f0aj9b39ed75ed2f37b3@mail.gmail.com> 
	<7CFD8BD5-784E-4927-8A7C-8422472B441E@matthewwilkes.co.uk> 
	<73492FDFAB444938B5185B04F8E39DF4@RaymondLaptop1>
Message-ID: <ca471dc20906121719x3ec571ddsedab775fa3a54fea@mail.gmail.com>

On Fri, Jun 12, 2009 at 5:17 PM, Raymond Hettinger<python at rcn.com> wrote:
>
> [Matthew Wilkes]
>>
>> Oh, I didn't mean they should use proprietary software, just that in ?my
>> experience the kind of people who are active in open source are ?quite
>> anti-war, green, etc. ?There are notable exceptions, but I know ?people who
>> worry that their work will have military applications, and ?who turn down
>> projects because of it.
>
> I question the whole notion of using open source in military weapons.
> It seems like a rather basic violation of operational security. ?Perhaps
> your enemies will exploit your bugs instead of nicely reporting them
> and submitting patches on SourceForge ;-)

Eric Raymond would argue that it's probably the other way around --
proprietary software doesn't have enough eyeballs to make it safe. :-)

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From cylix at solace.info  Sat Jun 13 02:37:07 2009
From: cylix at solace.info (Frederick Reeve)
Date: Fri, 12 Jun 2009 19:37:07 -0500
Subject: [Python-Dev] io.BufferedReader.peek() Behaviour in python3.1
In-Reply-To: <20090610203857.00a019fa@cylix>
References: <20090610203857.00a019fa@cylix>
Message-ID: <20090612193707.24f5563c@cylix>

Greetings,

I feel the need to point out I made a mistake.  When I wrote my last email I said the behavior had changed python3-3.1.  This seems not to be the case.. I had made that assumption because I had written code based on the looking at the code in _pyio.py as well as the python3 documentation (http://docs.python.org/3.0/library/io.html#io.BufferedReader) which seems to be wrong on that point or I miss understand.  Anyway I'm sorry about that.

The other point still stands though.  I would like to see peek changed.  I am willing to write and submit changes but don't want to unless others agree this is a good idea.  So I put forth the implementation at the bottom of this email.  If its bad or you don't see the point I may try to clarify but if nobody things its good, please just tell me I'm waisting your time, and I will go away.  I also apologize my last email was so long.

peek(n):
If n is less than 0, None, or not set; return buffer contents with out
advancing stream position. If the buffer is empty read a full chunk and
return the buffer.  Otherwise return exactly n bytes up to _chunk
size_(not contents) with out advancing the stream position.  If the
buffer contents is less than n, buffer an additional chunk from the
"raw" stream before hand.  If EOF is encountered during any raw read
then return as much as we can up to n. (maybe I should write that in
code form??)

Thanks

Frederick Reeve

From ncoghlan at gmail.com  Sat Jun 13 03:01:49 2009
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sat, 13 Jun 2009 11:01:49 +1000
Subject: [Python-Dev] Iterator version of contextlib.nested
In-Reply-To: <4A3271BA.2070904@gmx.net>
References: <4A3271BA.2070904@gmx.net>
Message-ID: <4A32FA7D.1060906@gmail.com>

Hagen F?rstenau wrote:
> I'm proposing to add an iterator version of "nested" to contextlib
> (possibly called "inested"), which takes an iterable of context managers
> instead of a variable number of parameters. The implementation could be
> taken over from the present "nested", only changing "def
> nested(*managers)" to "def inested(managers)".
> 
> This has the advantage that an iterator can be passed to "inested", so
> that each context managers is created in the context of all previous
> ones, which was one of the reasons for introducing the multi-with
> statement in the first place. "contextlib.inested" would therefore be
> the generalization of the multi-with statement to a variable number of
> managers (and "contextlib.nested" would stay deprecated).

The semantic change actually needed to make nested() more equivalent to
the multi-with statement is for it to accept zero-argument callables
that create context managers as arguments rather than pre-created
context managers.

Rather than changing the name of the function, this could be done by
inspecting the first argument for an "__enter__" method. If it has one,
use the old semantics (and issue a DeprecationWarning as in 3.1).
Otherwise, use the proposed new semantics.

However, the semantic equivalence still won't be complete as nested()
currently has no way of matching the multi-with behaviour of allowing
outer context managers to suppress exceptions raised by the constructors
or __enter__ methods of inner context managers. Attempting to do so will
result in a RuntimeError as the contextlib.contextmanager wrapper
complains that the generator implementing nested() didn't yield.

The further enhancement needed to address that would be to tweak
nested() to raise a custom exception in that case (e.g. ContextSkipped)
and provide an "allowskip" context manager that just catches and
suppresses that specific exception:

i.e.
  @contextmanager
  def allowskip():
    try:
      yield
    except ContextSkipped:
      pass

  with allowskip(), nested(*cm_factories):
    # Do something

I suggest putting an RFE on the tracker for this.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------

From phillip.sitbon+python-dev at gmail.com  Sat Jun 13 03:10:15 2009
From: phillip.sitbon+python-dev at gmail.com (Phillip Sitbon)
Date: Fri, 12 Jun 2009 18:10:15 -0700
Subject: [Python-Dev] FINAL PROPULSION OPEN SOURCE ENGINE VARIANT FORTHE
	F-35 JOINT STRIKE FIGHTER
In-Reply-To: <ca471dc20906121719x3ec571ddsedab775fa3a54fea@mail.gmail.com>
References: <958091.40012.qm@web34401.mail.mud.yahoo.com>
	<4B23F475-2DDE-4929-8993-B568922F5A5B@matthewwilkes.co.uk>
	<ca471dc20906121700h14133f0aj9b39ed75ed2f37b3@mail.gmail.com>
	<7CFD8BD5-784E-4927-8A7C-8422472B441E@matthewwilkes.co.uk>
	<73492FDFAB444938B5185B04F8E39DF4@RaymondLaptop1>
	<ca471dc20906121719x3ec571ddsedab775fa3a54fea@mail.gmail.com>
Message-ID: <536685ea0906121810m367ab103tf8b1b3e4980b0e71@mail.gmail.com>

>> I question the whole notion of using open source in military weapons.
>> It seems like a rather basic violation of operational security. ?Perhaps
>> your enemies will exploit your bugs instead of nicely reporting them
>> and submitting patches on SourceForge ;-)
>
> Eric Raymond would argue that it's probably the other way around --
> proprietary software doesn't have enough eyeballs to make it safe. :-)
>

I guess in some cases it wouldn't matter if it were open source:

http://online.wsj.com/article/SB124027491029837401.html

I'm not sure of the seriousness (or mental stability) of Mr. "OMEGA
RED", but I definitely got a chuckle from this :)

 - Phillip

From ncoghlan at gmail.com  Sat Jun 13 03:30:29 2009
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sat, 13 Jun 2009 11:30:29 +1000
Subject: [Python-Dev] FINAL PROPULSION OPEN SOURCE ENGINE VARIANT FORTHE
 F-35 JOINT STRIKE FIGHTER
In-Reply-To: <73492FDFAB444938B5185B04F8E39DF4@RaymondLaptop1>
References: <958091.40012.qm@web34401.mail.mud.yahoo.com><4B23F475-2DDE-4929-8993-B568922F5A5B@matthewwilkes.co.uk><ca471dc20906121700h14133f0aj9b39ed75ed2f37b3@mail.gmail.com>	<7CFD8BD5-784E-4927-8A7C-8422472B441E@matthewwilkes.co.uk>
	<73492FDFAB444938B5185B04F8E39DF4@RaymondLaptop1>
Message-ID: <4A330135.8060803@gmail.com>

Raymond Hettinger wrote:
> 
> [Matthew Wilkes]
>> Oh, I didn't mean they should use proprietary software, just that in 
>> my experience the kind of people who are active in open source are 
>> quite anti-war, green, etc.  There are notable exceptions, but I know 
>> people who worry that their work will have military applications, and 
>> who turn down projects because of it.
> 
> I question the whole notion of using open source in military weapons.
> It seems like a rather basic violation of operational security.  Perhaps
> your enemies will exploit your bugs instead of nicely reporting them
> and submitting patches on SourceForge ;-)

As Guido said, even the military are aware that there are major problems
with the idea of security through obscurity. Plus most defence forces
around the world are just as interested in saving a few bucks on
software costs as any other organisation, particularly if they can
reduce their reliance on a foreign software vendor in the process.

As to the existence of open source developers that are willing to work
with the military... as Matthew said, while they may not be particularly
common, I can definitely say that they're around ;)

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------

From pjenvey at underboss.org  Sat Jun 13 03:15:09 2009
From: pjenvey at underboss.org (Philip Jenvey)
Date: Fri, 12 Jun 2009 18:15:09 -0700
Subject: [Python-Dev] FINAL PROPULSION OPEN SOURCE ENGINE VARIANT FORTHE
	F-35 JOINT STRIKE FIGHTER
In-Reply-To: <73492FDFAB444938B5185B04F8E39DF4@RaymondLaptop1>
References: <958091.40012.qm@web34401.mail.mud.yahoo.com><4B23F475-2DDE-4929-8993-B568922F5A5B@matthewwilkes.co.uk><ca471dc20906121700h14133f0aj9b39ed75ed2f37b3@mail.gmail.com>
	<7CFD8BD5-784E-4927-8A7C-8422472B441E@matthewwilkes.co.uk>
	<73492FDFAB444938B5185B04F8E39DF4@RaymondLaptop1>
Message-ID: <F3809A98-43DE-4F44-906D-4C7768049235@underboss.org>


On Jun 12, 2009, at 5:17 PM, Raymond Hettinger wrote:

>
> [Matthew Wilkes]
>> Oh, I didn't mean they should use proprietary software, just that  
>> in  my experience the kind of people who are active in open source  
>> are  quite anti-war, green, etc.  There are notable exceptions, but  
>> I know  people who worry that their work will have military  
>> applications, and  who turn down projects because of it.
>
> I question the whole notion of using open source in military weapons.
> It seems like a rather basic violation of operational security.   
> Perhaps
> your enemies will exploit your bugs instead of nicely reporting them
> and submitting patches on SourceForge ;-)

FYI Python (Jython) is already used in production of the F-35. There  
was a talk @ PyCon '08 about it:

http://us.pycon.org/2008/conference/schedule/event/27/
http://www.youtube.com/watch?v=zgE55z_RNgQ

--
Philip Jenvey

From ncoghlan at gmail.com  Sat Jun 13 04:24:13 2009
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sat, 13 Jun 2009 12:24:13 +1000
Subject: [Python-Dev] io.BufferedReader.peek() Behaviour in python3.1
In-Reply-To: <20090612193707.24f5563c@cylix>
References: <20090610203857.00a019fa@cylix> <20090612193707.24f5563c@cylix>
Message-ID: <4A330DCD.4080608@gmail.com>

Frederick Reeve wrote:
> The other point still stands though.  I would like to see peek
> changed.  I am willing to write and submit changes but don't want to
> unless others agree this is a good idea.  So I put forth the
> implementation at the bottom of this email.  If its bad or you don't
> see the point I may try to clarify but if nobody things its good,
> please just tell me I'm waisting your time, and I will go away.  I
> also apologize my last email was so long.
> 
> peek(n): If n is less than 0, None, or not set; return buffer
> contents with out advancing stream position. If the buffer is empty
> read a full chunk and return the buffer.  Otherwise return exactly n
> bytes up to _chunk size_(not contents) with out advancing the stream
> position.  If the buffer contents is less than n, buffer an
> additional chunk from the "raw" stream before hand.  If EOF is
> encountered during any raw read then return as much as we can up to
> n. (maybe I should write that in code form??)

I would phrase this suggestion as users having a reasonable expectation
that the following invariant should hold for a buffered stream:

  f.peek(n) == f.read(n)

Since the latter method will perform as many reads of the underlying
stream as necessary to reach the requested number of bytes (or EOF),
then so should the former.

However, the default value for n for peek() should remain at 1 to remain
consistent with the current documented behaviour.

If this invariant was implemented, I would also suggest adding a "peek1"
method to parallel "read1".

Note that the current behaviour I get from Python 3.1 is for it to
return the *entire* buffer, no matter what number I pass to it:

(current Py3k head)

>>> f = open('setup.py', 'rb')
>>> len(f.peek(10))
4096
>>> len(f.peek(1))
4096
>>> len(f.peek(4095))
4096
>>> len(f.peek(10095))
4096

That's an outright bug - I've promoted an existing issue about this [1]
to a release blocker and sent it to Benjamin to have another look at.

Cheers,
Nick.

[1] http://bugs.python.org/issue5811

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------

From cs at zip.com.au  Sat Jun 13 07:12:22 2009
From: cs at zip.com.au (Cameron Simpson)
Date: Sat, 13 Jun 2009 15:12:22 +1000
Subject: [Python-Dev] io.BufferedReader.peek() Behaviour in python3.1
In-Reply-To: <4A330DCD.4080608@gmail.com>
Message-ID: <20090613051222.GA23544@cskk.homeip.net>

On 13Jun2009 12:24, Nick Coghlan <ncoghlan at gmail.com> wrote:
| Frederick Reeve wrote:
| > The other point still stands though.  I would like to see peek
| > changed.  I am willing to write and submit changes but don't want to
| > unless others agree this is a good idea.  So I put forth the
| > implementation at the bottom of this email.  If its bad or you don't
| > see the point I may try to clarify but if nobody things its good,
| > please just tell me I'm waisting your time, and I will go away.  I
| > also apologize my last email was so long.
| > 
| > peek(n): If n is less than 0, None, or not set; return buffer
| > contents with out advancing stream position. If the buffer is empty
| > read a full chunk and return the buffer.  Otherwise return exactly n
| > bytes up to _chunk size_(not contents) with out advancing the stream
| > position.  If the buffer contents is less than n, buffer an
| > additional chunk from the "raw" stream before hand.  If EOF is
| > encountered during any raw read then return as much as we can up to
| > n. (maybe I should write that in code form??)
| 
| I would phrase this suggestion as users having a reasonable expectation
| that the following invariant should hold for a buffered stream:
| 
|   f.peek(n) == f.read(n)
| 
| Since the latter method will perform as many reads of the underlying
| stream as necessary to reach the requested number of bytes (or EOF),
| then so should the former.

I disagree. If that were that case, why have peek() at all? I realise
that it doesn't move the logical position, but it does mean that
peek(huge_number) imposes a requirement to grow the stream buffer
arbitrarily.

A peek that does at most one raw read has the advantage that it can pick up
data outside the buffer but lurking in the OS buffer, yet to be obtained.
Those data are free, if they're present. (Of course, if they're absent
peek() wil still block).

Since (if the OS buffer is also empty) even a peek(1) can block, maybe
we should canvas peek()'s common use cases. Naively (never having used
peek()), my own desire would normally be for a peek(n, block=False) a
bit like Queue.get(). Then I could be sure not to block if I wanted to
avoid it, even on a blocking stream, yet still obtain unread buffered
data if present.

So: what do people use peek() for, mostly?

Cheers,
-- 
Cameron Simpson <cs at zip.com.au> DoD#743
http://www.cskk.ezoshosting.com/cs/

We're in the business of putting goo on a substrate.
- overhead by WIRED at the Intelligent Printing conference Oct2006

From ncoghlan at gmail.com  Sat Jun 13 08:59:02 2009
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sat, 13 Jun 2009 16:59:02 +1000
Subject: [Python-Dev] io.BufferedReader.peek() Behaviour in python3.1
In-Reply-To: <20090613051222.GA23544@cskk.homeip.net>
References: <20090613051222.GA23544@cskk.homeip.net>
Message-ID: <4A334E36.6000705@gmail.com>

Cameron Simpson wrote:
> On 13Jun2009 12:24, Nick Coghlan <ncoghlan at gmail.com> wrote:
> | I would phrase this suggestion as users having a reasonable expectation
> | that the following invariant should hold for a buffered stream:
> | 
> |   f.peek(n) == f.read(n)
> | 
> | Since the latter method will perform as many reads of the underlying
> | stream as necessary to reach the requested number of bytes (or EOF),
> | then so should the former.
> 
> I disagree. If that were that case, why have peek() at all? I realise
> that it doesn't move the logical position, but it does mean that
> peek(huge_number) imposes a requirement to grow the stream buffer
> arbitrarily.
> 
> A peek that does at most one raw read has the advantage that it can pick up
> data outside the buffer but lurking in the OS buffer, yet to be obtained.
> Those data are free, if they're present. (Of course, if they're absent
> peek() wil still block).

Note my suggestion later that if the above invariant were to be adopted
then a peek1() method should be added to parallel read1().

However, from what Benjamin has said, a more likely invariant is going
to be:

  preview = f.peek(n)
  f.read(n).startswith(preview)

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------

From eric.pruitt at gmail.com  Sat Jun 13 03:55:30 2009
From: eric.pruitt at gmail.com (Eric Pruitt)
Date: Fri, 12 Jun 2009 20:55:30 -0500
Subject: [Python-Dev] Status of 2.7 and 3.2
Message-ID: <171e8a410906121855v17f211dred8f1c4cbdf96f3c@mail.gmail.com>

I am in the process of implementing a number of often requested features and
proposed patches in the subprocess module for my Google Summer of Code 2009
project. For information on my progress, check out my blog located at *
http://subdev.blogspot.com/* <http://subdev.blogspot.com/>. Any comments and
suggestions are greatly appreciated.

> Just my 0.02 cents, but struggling with all warts of 2.5 subprocessing
> > in Windows I would vote for more time for stabilizating things - not
> > adding new features. Long awaited subprocess as replacement for
> > os.popen() AFAIK is still incapable to asynchronously communicate with
> > spawned processes on Windows. I would call this feature as critical
> > even on 2.6  As a release testcase - try porting pyexpect module to
> > this platform. Absence of native curses/console/readline module also
> > makes Python one-way unix shell language while many users expect it to
> > be crossplatform.
>
> I am not quite sure whether you are for new features or not. Your
> first sentence ("vote for ... not adding new features") seems to
> suggest that you would not like to see new features, and your last
> sentence ("absence of native curses/console/readline module")
> suggests that you *do* want to see new features (namely, a native
> curses module, and a native readline module).
>
> Which one is it?
>
> Regards,
> Martin
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090612/1a5a3214/attachment.htm>

From regebro at gmail.com  Sat Jun 13 11:05:12 2009
From: regebro at gmail.com (Lennart Regebro)
Date: Sat, 13 Jun 2009 11:05:12 +0200
Subject: [Python-Dev] Status of 2.7 and 3.2
In-Reply-To: <87hbysfcym.fsf@uwakimon.sk.tsukuba.ac.jp>
References: <h0fbn2$gop$1@ger.gmane.org> <4A2B6E39.3090405@v.loewis.de>
	<87hbysfcym.fsf@uwakimon.sk.tsukuba.ac.jp>
Message-ID: <319e029f0906130205s337b77f4t50ef992cf8d75412@mail.gmail.com>

2009/6/7 Stephen J. Turnbull <stephen at xemacs.org>:
> Python's 2.x/py3k division is a tour de force; I still can't believe
> my eyes that you've pulled it off.

Well, It's not pulled off until Python 3 has surpassed Python 2 in
usage. That's still a long way away. I'm not familiar with the other
examples except Zope 3, which is a completely different thing, so it
may very well be that Python did it way better. But still, it's not
pulled off yet.

-- 
Lennart Regebro: Python, Zope, Plone, Grok
http://regebro.wordpress.com/
+33 661 58 14 64

From stephen at xemacs.org  Sat Jun 13 13:58:13 2009
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Sat, 13 Jun 2009 20:58:13 +0900
Subject: [Python-Dev] Status of 2.7 and 3.2
In-Reply-To: <319e029f0906130205s337b77f4t50ef992cf8d75412@mail.gmail.com>
References: <h0fbn2$gop$1@ger.gmane.org> <4A2B6E39.3090405@v.loewis.de>
	<87hbysfcym.fsf@uwakimon.sk.tsukuba.ac.jp>
	<319e029f0906130205s337b77f4t50ef992cf8d75412@mail.gmail.com>
Message-ID: <87fxe4wdka.fsf@uwakimon.sk.tsukuba.ac.jp>

Lennart Regebro writes:
 > 2009/6/7 Stephen J. Turnbull <stephen at xemacs.org>:
 > > Python's 2.x/py3k division is a tour de force; I still can't believe
 > > my eyes that you've pulled it off.
 > 
 > Well, It's not pulled off until Python 3 has surpassed Python 2 in
 > usage.

I'm referring only to the management of the dual-trunk workflow, not
to whether Python3 will be the preferred path for the future.  That
workflow has its problems, but still it seems very successful in
keeping what needs to be synced, synced, and what needs to be
different, different.


From solipsis at pitrou.net  Sat Jun 13 14:16:14 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Sat, 13 Jun 2009 12:16:14 +0000 (UTC)
Subject: [Python-Dev] io.BufferedReader.peek() Behaviour in python3.1
References: <20090610203857.00a019fa@cylix> <20090612193707.24f5563c@cylix>
	<4A330DCD.4080608@gmail.com>
Message-ID: <loom.20090613T121150-565@post.gmane.org>

Nick Coghlan <ncoghlan <at> gmail.com> writes:
> 
> Since the latter method will perform as many reads of the underlying
> stream as necessary to reach the requested number of bytes (or EOF),
> then so should the former.

How do you propose to implement this while staying compatible with
1) unseekable raw streams
2) the expectation that peek() doesn't advance the logical file pointer?

> Note that the current behaviour I get from Python 3.1 is for it to
> return the *entire* buffer, no matter what number I pass to it:
> 
[...]
> 
> That's an outright bug - I've promoted an existing issue about this [1]
> to a release blocker and sent it to Benjamin to have another look at.

The original docstring for peek() says:

        """Returns buffered bytes without advancing the position.

        The argument indicates a desired minimal number of bytes; we
        do at most one raw read to satisfy it.  We never return more
        than self.buffer_size.
        """

In that light, I'm not sure it's a bug -- although it can certainly look
unexpected at first sight.

Regards

Antoine.



From solipsis at pitrou.net  Sat Jun 13 14:33:46 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Sat, 13 Jun 2009 12:33:46 +0000 (UTC)
Subject: [Python-Dev] io.BufferedReader.peek() Behaviour in python3.1
References: <20090610203857.00a019fa@cylix> <20090612193707.24f5563c@cylix>
Message-ID: <loom.20090613T123207-984@post.gmane.org>

Frederick Reeve <cylix <at> solace.info> writes:
> 
> peek(n):
> If n is less than 0, None, or not set; return buffer contents with out
> advancing stream position. If the buffer is empty read a full chunk and
> return the buffer.  Otherwise return exactly n bytes up to _chunk
> size_(not contents) with out advancing the stream position.  If the
> buffer contents is less than n, buffer an additional chunk from the
> "raw" stream before hand.  If EOF is encountered during any raw read
> then return as much as we can up to n. (maybe I should write that in
> code form??)

This proposal looks reasonable to me. Please note that it's too late for 3.1
anyway - we're in release candidate phase. Once you have a patch, you can post
it on the bug tracker.

Regards

Antoine.



From solipsis at pitrou.net  Sat Jun 13 14:35:12 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Sat, 13 Jun 2009 12:35:12 +0000 (UTC)
Subject: [Python-Dev] Status of 2.7 and 3.2
References: <171e8a410906121855v17f211dred8f1c4cbdf96f3c@mail.gmail.com>
Message-ID: <loom.20090613T123427-349@post.gmane.org>

Eric Pruitt <eric.pruitt <at> gmail.com> writes:
> 
> I am in the process of implementing a 
> number of often requested features and proposed patches in the subprocess 
> module for my Google Summer of Code 2009 project. For information on 
> my progress, check out my blog located at?http://subdev.blogspot.com/.

Please note there's already a Mercurial mirror of the Python trunk:
http://code.python.org/hg/trunk/

(I'm replying here since your blog disallows anonymous comments)


Antoine.



From facundobatista at gmail.com  Sat Jun 13 14:40:49 2009
From: facundobatista at gmail.com (Facundo Batista)
Date: Sat, 13 Jun 2009 09:40:49 -0300
Subject: [Python-Dev] Avoiding file descriptors leakage in
	subprocess.Popen()
In-Reply-To: <4A32ED75.7060803@cheimes.de>
References: <e04bdf310906121425x72a7c033wec71500712759375@mail.gmail.com>
	<4A32ED75.7060803@cheimes.de>
Message-ID: <e04bdf310906130540t23c4b78r68a01a95a3e9f621@mail.gmail.com>

On Fri, Jun 12, 2009 at 9:06 PM, Christian Heimes<lists at cheimes.de> wrote:

> How about a nice 'n shiny context wrapper for the pipe:

I'll do this!

Thank you for the suggestion!

BTW, as this is a good way of avoiding the FD leakage, should this
context wrapper for pipe() be in the stdlib?

Regards,

-- 
.    Facundo

Blog: http://www.taniquetil.com.ar/plog/
PyAr: http://www.python.org/ar/

From hfuerstenau at gmx.net  Sat Jun 13 14:45:56 2009
From: hfuerstenau at gmx.net (=?ISO-8859-1?Q?Hagen_F=FCrstenau?=)
Date: Sat, 13 Jun 2009 13:45:56 +0100
Subject: [Python-Dev] Iterator version of contextlib.nested
In-Reply-To: <4A32FA7D.1060906@gmail.com>
References: <4A3271BA.2070904@gmx.net> <4A32FA7D.1060906@gmail.com>
Message-ID: <4A339F84.2070400@gmx.net>

> The semantic change actually needed to make nested() more equivalent to
> the multi-with statement is for it to accept zero-argument callables
> that create context managers as arguments rather than pre-created
> context managers.

It seems to me that both passing callables which return managers and
passing a generator which yields managers achieve about the same thing.
Are you proposing the former just to avoid introducing a new interface?

> Rather than changing the name of the function, this could be done by
> inspecting the first argument for an "__enter__" method. If it has one,
> use the old semantics (and issue a DeprecationWarning as in 3.1).
> Otherwise, use the proposed new semantics.

I guess this is much too late for 3.1, but could we then at least
un-deprecate "contextlib.nested" for now? As it is, you get a
DeprecationWarning for something like

with contextlib.nested(*my_managers):

without any good way to get rid of it.

- Hagen

From benjamin at python.org  Sat Jun 13 16:46:28 2009
From: benjamin at python.org (Benjamin Peterson)
Date: Sat, 13 Jun 2009 09:46:28 -0500
Subject: [Python-Dev] [RELEASED] Python 3.1 Release Candidate 2
Message-ID: <1afaf6160906130746v2d951486v7a52d7b838070189@mail.gmail.com>

On behalf of the Python development team, I'm happy to announce the second
release candidate of Python 3.1.

Python 3.1 focuses on the stabilization and optimization of the features and
changes that Python 3.0 introduced.  For example, the new I/O system has been
rewritten in C for speed.  File system APIs that use unicode strings now handle
paths with undecodable bytes in them. Other features include an ordered
dictionary implementation, a condensed syntax for nested with statements, and
support for ttk Tile in Tkinter.  For a more extensive list of changes in 3.1,
see http://doc.python.org/dev/py3k/whatsnew/3.1.html or Misc/NEWS in the Python
distribution.

This is a release candidate, and as such, we do not recommend use in production
environments.  However, please take this opportunity to test the release with
your libraries or applications.  This will hopefully discover bugs before the
final release and allow you to determine how changes in 3.1 might impact you.
If you find things broken or incorrect, please submit a bug report at

     http://bugs.python.org

For more information and downloadable distributions, see the Python 3.1 website:

     http://www.python.org/download/releases/3.1/

See PEP 375 for release schedule details:

     http://www.python.org/dev/peps/pep-0375/



Enjoy,
-- Benjamin

Benjamin Peterson
benjamin at python.org
Release Manager
(on behalf of the entire python-dev team and 3.1's contributors)

From pebarrett at gmail.com  Sat Jun 13 19:02:47 2009
From: pebarrett at gmail.com (Paul Barrett)
Date: Sat, 13 Jun 2009 13:02:47 -0400
Subject: [Python-Dev] FINAL PROPULSION OPEN SOURCE ENGINE VARIANT FORTHE
	F-35 JOINT STRIKE FIGHTER
In-Reply-To: <4A330135.8060803@gmail.com>
References: <958091.40012.qm@web34401.mail.mud.yahoo.com>
	<4B23F475-2DDE-4929-8993-B568922F5A5B@matthewwilkes.co.uk>
	<ca471dc20906121700h14133f0aj9b39ed75ed2f37b3@mail.gmail.com>
	<7CFD8BD5-784E-4927-8A7C-8422472B441E@matthewwilkes.co.uk>
	<73492FDFAB444938B5185B04F8E39DF4@RaymondLaptop1>
	<4A330135.8060803@gmail.com>
Message-ID: <40e64fa20906131002i61c44668of03092456828b830@mail.gmail.com>

The other benefit of the military using open source software is that
is can save the taxpayers money over the short and long term. For some
projects it is a small percentage of the total cost.  However, for
others it can be significant portion of the cost, so don't discount
its use or its benefit.  Saving time and money is a good thing.

Cheers,
Paul

On Fri, Jun 12, 2009 at 9:30 PM, Nick Coghlan<ncoghlan at gmail.com> wrote:
> Raymond Hettinger wrote:
>>
>> [Matthew Wilkes]
>>> Oh, I didn't mean they should use proprietary software, just that in
>>> my experience the kind of people who are active in open source are
>>> quite anti-war, green, etc. ?There are notable exceptions, but I know
>>> people who worry that their work will have military applications, and
>>> who turn down projects because of it.
>>
>> I question the whole notion of using open source in military weapons.
>> It seems like a rather basic violation of operational security. ?Perhaps
>> your enemies will exploit your bugs instead of nicely reporting them
>> and submitting patches on SourceForge ;-)
>
> As Guido said, even the military are aware that there are major problems
> with the idea of security through obscurity. Plus most defence forces
> around the world are just as interested in saving a few bucks on
> software costs as any other organisation, particularly if they can
> reduce their reliance on a foreign software vendor in the process.
>
> As to the existence of open source developers that are willing to work
> with the military... as Matthew said, while they may not be particularly
> common, I can definitely say that they're around ;)
>
> Cheers,
> Nick.
>
> --
> Nick Coghlan ? | ? ncoghlan at gmail.com ? | ? Brisbane, Australia
> ---------------------------------------------------------------
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/pebarrett%40gmail.com
>

From zooko at zooko.com  Sat Jun 13 18:58:16 2009
From: zooko at zooko.com (Zooko Wilcox-O'Hearn)
Date: Sat, 13 Jun 2009 10:58:16 -0600
Subject: [Python-Dev] ctime: I don't think that word means what you think it
	means.
Message-ID: <69132116-861D-4362-A30D-35BCC45A6772@zooko.com>

The stat module uses the "st_ctime" slot to hold two kinds of values
which are semantically different and which are frequently
confused with one another.  It chooses which kind of value to put in
there based on platform -- Windows gets the file creation time and all
other platforms get the "ctime".  The only sane way to use this API is
then to switch on platform:

if platform.system() == "Windows":
     metadata["creation time"] = s.st_ctime
else:
     metadata["unix ctime"] = s.st_ctime

(That is an actual code snippet from the Allmydata-Tahoe project.)

Many or even most programmers incorrectly think that unix ctime is file
creation time, so instead of using the sane idiom above, they write the
following:

metadata["ctime"] = s.st_ctime

thus passing on the confusion to the users of their metadata, who may
not be able to tell on which platform this metadata was created.   
This is
the situation we have found ourselves in for the Allmydata-Tahoe
project -- we now have a bunch of "ctime" values stored in our
filesystem and no way to tell which kind they were.

More and more filesystems such as ZFS and Mac HFS+ apparently offer
creation time nowadays.

I propose the following changes:

1.  Add a "st_crtime" field which gets populated on filesystems
(Windows, ZFS, Mac) which can do so.

That is hopefully not too controversial and we could proceed to do so
even if the next proposal gets bogged down:

2.  Add a "st_unixctime" field which gets populated *only* by the unix
ctime and never by any other value (even on Windows, where the unix
ctime *is* available even though nobody cares about it), and deprecate
the hopelessly ambiguous "st_ctime" field.

You may be interested in http://allmydata.org/trac/tahoe/ticket/628
("mtime" and "ctime": I don't think that word means what you think it
means.) where the Allmydata-Tahoe project is carefully unpicking the
mess we made for ourselves by confusing ctime with file-creation time.

This is ticket http://bugs.python.org/issue5720 .

Regards,

Zooko


From omega_force2003 at yahoo.com  Sat Jun 13 21:35:57 2009
From: omega_force2003 at yahoo.com (OMEGA RED)
Date: Sat, 13 Jun 2009 12:35:57 -0700 (PDT)
Subject: [Python-Dev] FINAL PROPULSION OPEN SOURCE ENGINE VARIANT FOR
	THE F-35 JOINT STRIKE FIGHTER
Message-ID: <331154.6420.qm@web34401.mail.mud.yahoo.com>

Guido,
?
?
The reason why I use a code name is that it would be dangerous to reveal my true name to the world since there are those who are radical liberals who would like to think that there is no need for a U.S. military and that people such as myself should be eliminated.? Think of the U.S. military as the heroes that they truly are?to protect its citizens from attack from other entities.? The U.S. military is here to serve and protect people such as yourself, not to act as the terrorist organization as?bin laden would like?you to think.? ? If any of you need any further evidence of the need of the U.S. military or the F-35 for that matter, then I would suggest that you all take a nice long trip to Manhattan island in New York City and take some time to notice the fact that there appears to be a large hole where many?buildings used to be located.? 
?
?
Second, the biggest concern that I see in utilizing the python language in safety-critical systems is that it is a dynamically type language.? Unlike Ada which is safety statically type language.? These were just brainstorming ideas, but if you do not like those that brainstorm regarding the further development of python, then I guess I will be moving forward to some other language that is more capable to assist in the development of safety?critical systems.? These were just brainstorming ideas.? 
?
The idea being is to research onto the idea of developing a statically?type variant of the python language?for the development of software for safety critical systems.? Is this possible?? I am also wondering if it was possible to interface Ada programs with Python, similar to the method that has been utilized to interface FORTRAN with Python.? Is this also possible?? 
?
?
?
Thank You.? 
?
?
W108dab
?
?
?
?
?


--- On Sat, 6/13/09, Guido van Rossum <guido at python.org> wrote:


From: Guido van Rossum <guido at python.org>
Subject: Re: [Python-Dev] FINAL PROPULSION OPEN SOURCE ENGINE VARIANT FOR THE F-35 JOINT STRIKE FIGHTER
To: "OMEGA RED" omega_force2003 at yahoo.com
Date: Saturday, June 13, 2009, 11:58 AM


I'm sorry, but I have so far three problems with your proposals.

(1) You use a code name.

(2) Your first message was ALL CAPS.

(3) You don't seem to know much about programming language design, or
you wouldn't propose such a vague non-starter as "adopt the
safety-critical charasteristics of the Ada programming language an
incorporate them into Python."

This is my last message to you.

--Guido

On Sat, Jun 13, 2009 at 7:27 AM, OMEGA RED<omega_force2003 at yahoo.com> wrote:
> Yes,
>
> I think that python developers would be great in the development of the
> FADEC controller as an open-source alternative to the propreitary FADEC
> controller software to the propulsion system for the F-35.? The FADEC
> controller is Full Authority Flight Digital Electronic Controller.
>
> I know that the python community could never possibly create the mechanical
> aspects of this proposed engine, but that does not mean that we cannot
> create something else.
>
> I believe that what could be done is to create a safety-critical version of
> python.? Such as adopt the safety-critical charasteristics of the Ada
> programming language an incorporate them into python.
>
> How about that as a project?? Create the python equivalent of Ada?
>
> Thank You,
>
> w108dab
>
>

> Subject: Re: [Python-Dev] FINAL PROPULSION OPEN SOURCE ENGINE VARIANT FOR
> THE F-35 JOINT STRIKE FIGHTER
> To: "OMEGA RED" <omega_force2003 at yahoo.com>
> Date: Saturday, June 13, 2009, 6:16 AM
>
>
> I'm reasonably confused here,?are you searching for Python developers?
>
>
>
>
> On Fri, Jun 12, 2009 at 7:01 PM, OMEGA RED <omega_force2003 at yahoo.com>
> wrote:
>>
>>
>> TO ALL,
>>
>>
>> I AM HAPPY TO SAY THAT I AM NOW IN THE PROCESS OF DEVELOPING A WEBSITE FOR
>> THE STATED PURPOSE OF COMBINING SOME OF THE BEST MINDS IN THE WORLD.? THIS
>> PROJECT WILL BE A MEANS TO DEVELOP THE FIRST AND ONLY COMPLETELY OPEN SOURCE
>> VARIANT OF THE PROPULSION ENGINE FOR THE F-35 JOINT STRIKE FIGHTER.? WHICH
>> MEANS THAT EVERYTHING ABOUT THE DESIGN, DEVELOPMENT, AND CONSTRUCTION OF THE
>> ENGINE WILL BE COMPLETELY AVAILABLE TO THE PUBLIC AND TO ANY ENGINE
>> MANUFACTURE FREE OF ANY REQUIREMENTS.
>>
>> THIS ENGINE WILL BE CALLED THE PHOENIX NexT F-200 ENGINE.
>>
>> THIS ENGINE WILL ALSO BE A POTENTIAL THIRD AND FINAL ALTERNATIVE ENGINE
>> FOR THE JOINT STRIKE FIGHTER.
>>
>> THIS ENGINE WILL BE STATED AS BEING A COMPETITOR WITH BOTH THE F-135 AND
>> THE F-136 ENGINE, WHICH ARE PROVIDED BY BOTH PRATT-WHITNEY AND GENERAL
>> ELECTRIC/ ROLLS ROYCE.
>>
>> THIS ENGINE WILL UTILIZE PYTHON FOR THE MAJOR PORTION OF THE SIMULATION,
>> RESEARCH AND DEVELOPMENT STAGES FOR THE DEVELOPMENT OF THE ENGINE.? THE
>> CONSTRUCTION OF THE MATHEMATICAL PROCESSING OF THE REQUIRED NONLINEAR
>> CONTROLS WILL BE BASED ON A SPECIALIZED VERSION OF GROUP THEORY FOR THE
>> DESIGN AND CONSTRUCTION OF THE NONLINEAR CONTROL SYSTEMS.
>>
>>
>> THE PURPOSE OF THE DEVELOPMENT OF THE ENGINE WILL BE TO CONSTRUCT AN
>> ENGINE FOR THE F-35 THAT WILL BE AS RELIABLE AND CAPABLE AS THE F-135 AND
>> THE F-136.
>>
>> HOWEVER, THE PHOENIX NexT ENGINE WILL ALSO BE STATED AS BEING ONLY AROUND
>> $20,000 TO $110,000 DOLLARS TO BEING PURCHASED BY AN OPERATOR OF THE F-35,
>> WHICH WILL BE AN INSIGNIFICANT COST COMPARED TO THE COST OF BOTH THE F-135
>> AND THE F-136 ENGINE (10 MILLION DOLLARS PER UNIT).
>>
>> HOW MANY OUT THERE WANT TO HELP IN THIS ENDEAVOR ?? I BELIEVE THAT WE
>> ALL?CAN SUCCEED WHERE BOTH PRATT-WHITNEY AND GE/ROLLS ROYCE HAVE FAILED TO
>> DEVELOP AN INEXPENSIVE AND RELIABLE ENGINE FOR THE F-35.
>>
>> THANKS,
>>
>> W108DAB
>>
>> _______________________________________________
>> Python-Dev mailing list
>> Python-Dev at python.org
>> http://mail.python.org/mailman/listinfo/python-dev
>> Unsubscribe:
>> http://mail.python.org/mailman/options/python-dev/josepharmbruster%40gmail.com
>>
>
>
>



-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)



      
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090613/5c7ad178/attachment-0001.htm>

From arcriley at gmail.com  Sat Jun 13 21:52:53 2009
From: arcriley at gmail.com (Arc Riley)
Date: Sat, 13 Jun 2009 15:52:53 -0400
Subject: [Python-Dev] FINAL PROPULSION OPEN SOURCE ENGINE VARIANT FOR
	THE F-35 JOINT STRIKE FIGHTER
In-Reply-To: <331154.6420.qm@web34401.mail.mud.yahoo.com>
References: <331154.6420.qm@web34401.mail.mud.yahoo.com>
Message-ID: <bad82a80906131252g600af38bw414f3728e092dcdc@mail.gmail.com>

Enough is enough guys.  As entertaining as this thread has been, shouldn't
we be focused on the 3.1 release?

Don't feed the trolls.  Ok so one wandered in, but nobody needed to respond
and it can only get worse from here.

Please just flag the offending address(es) for moderation and ask them
politely to keep their posts to this list on-topic for core Python
development.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090613/a65a72a4/attachment.htm>

From greg.ewing at canterbury.ac.nz  Sun Jun 14 02:33:35 2009
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Sun, 14 Jun 2009 12:33:35 +1200
Subject: [Python-Dev] io.BufferedReader.peek() Behaviour in python3.1
In-Reply-To: <loom.20090613T121150-565@post.gmane.org>
References: <20090610203857.00a019fa@cylix> <20090612193707.24f5563c@cylix>
	<4A330DCD.4080608@gmail.com> <loom.20090613T121150-565@post.gmane.org>
Message-ID: <4A34455F.1030106@canterbury.ac.nz>

Antoine Pitrou wrote:
> The original docstring for peek() says:
> 
>         ...we
>         do at most one raw read to satisfy it.
> 
> In that light, I'm not sure it's a bug

It may be behaving according to the docs, but is that
behaviour useful?

Seems to me that if you're asking for n bytes, then it's
because you're doing some kind of parsing that requires
lookahead, and nothing less than n bytes will do.

I think it would be more useful if the "at most one
raw read" part were dropped. That would give it the
kind of deterministic behaviour generally expected
when dealing with buffered streams.

-- 
Greg

From solipsis at pitrou.net  Sun Jun 14 02:35:16 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Sun, 14 Jun 2009 00:35:16 +0000 (UTC)
Subject: [Python-Dev] io.BufferedReader.peek() Behaviour in python3.1
References: <20090610203857.00a019fa@cylix> <20090612193707.24f5563c@cylix>
	<4A330DCD.4080608@gmail.com>
	<loom.20090613T121150-565@post.gmane.org>
	<4A34455F.1030106@canterbury.ac.nz>
Message-ID: <loom.20090614T003419-836@post.gmane.org>

Greg Ewing <greg.ewing <at> canterbury.ac.nz> writes:
> 
> I think it would be more useful if the "at most one
> raw read" part were dropped. That would give it the
> kind of deterministic behaviour generally expected
> when dealing with buffered streams.

As I already told Nick: please propose an implementation scheme.

Antoine.



From greg.ewing at canterbury.ac.nz  Sun Jun 14 02:40:53 2009
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Sun, 14 Jun 2009 12:40:53 +1200
Subject: [Python-Dev] ctime: I don't think that word means what you
 think it means.
In-Reply-To: <69132116-861D-4362-A30D-35BCC45A6772@zooko.com>
References: <69132116-861D-4362-A30D-35BCC45A6772@zooko.com>
Message-ID: <4A344715.7010604@canterbury.ac.nz>

Zooko Wilcox-O'Hearn wrote:

> 1.  Add a "st_crtime" field which gets populated on filesystems
> (Windows, ZFS, Mac) which can do so.

"crtime" looks rather too similar to "ctime" for my
liking. People who think that the "c" in "ctime"
means "creation" are still likely to confuse them.

Why not give it a more explicit name, such
as "st_creationtime"?

-- 
Greg

From ncoghlan at gmail.com  Sun Jun 14 03:22:37 2009
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sun, 14 Jun 2009 11:22:37 +1000
Subject: [Python-Dev] Iterator version of contextlib.nested
In-Reply-To: <4A339F84.2070400@gmx.net>
References: <4A3271BA.2070904@gmx.net> <4A32FA7D.1060906@gmail.com>
	<4A339F84.2070400@gmx.net>
Message-ID: <4A3450DD.6040905@gmail.com>

Hagen F?rstenau wrote:
> I guess this is much too late for 3.1, but could we then at least
> un-deprecate "contextlib.nested" for now? As it is, you get a
> DeprecationWarning for something like
> 
> with contextlib.nested(*my_managers):
> 
> without any good way to get rid of it.

I actually almost asked for that to be changed to a
PendingDeprecationWarning when it was first added - Benjamin, do you
mind if I downgrade this warning to a pending one post rc2?

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------

From jeremy.cowles at gmail.com  Sun Jun 14 04:45:06 2009
From: jeremy.cowles at gmail.com (Jeremy Cowles)
Date: Sat, 13 Jun 2009 19:45:06 -0700
Subject: [Python-Dev] Distributed computing & sending the interpreter
Message-ID: <373cf0740906131945o6f17d70dnaa54b2d2b6c85833@mail.gmail.com>

Hi all,
I apologize if this question is misplaced, I wasn't sure which list to post
it in.

I'm working on a distributed computing project (PyMW and BOINC) where we are
sending Python scripts to client machines. Currently, we make two very
unlikely assumptions: that Python 2.5 is installed and that the interpreter
is available on the PATH.

We are looking at our options to remove this assumption and the two most
likely are redistributing the entire Python installation (for each supported
platform) and embedding the interpreter in a custom executable and sending
select parts of the standard library with it (to reduce size).

It seems like we would have to pre-compile for each different platform,
which is a pain and sending the entire installation could be tricky.

I am looking for alternatives, comments and suggestions. Any help would be
greatly appreciated.

Thanks,
Jeremy
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090613/c90a2daa/attachment.htm>

From aahz at pythoncraft.com  Sun Jun 14 05:44:23 2009
From: aahz at pythoncraft.com (Aahz)
Date: Sat, 13 Jun 2009 20:44:23 -0700
Subject: [Python-Dev] Distributed computing & sending the interpreter
In-Reply-To: <373cf0740906131945o6f17d70dnaa54b2d2b6c85833@mail.gmail.com>
References: <373cf0740906131945o6f17d70dnaa54b2d2b6c85833@mail.gmail.com>
Message-ID: <20090614034422.GA15064@panix.com>

On Sat, Jun 13, 2009, Jeremy Cowles wrote:
>
> I apologize if this question is misplaced, I wasn't sure which list to
> post it in.
>
> I'm working on a distributed computing project (PyMW and BOINC) where
> we are sending Python scripts to client machines. Currently, we make
> two very unlikely assumptions: that Python 2.5 is installed and that
> the interpreter is available on the PATH.

Definitely the wrong list -- python-dev is for people working on the core
interpreter and libraries.  comp.lang.python would be the standard place,
but you might also find some good advice on the new list concurrency-sig.
-- 
Aahz (aahz at pythoncraft.com)           <*>         http://www.pythoncraft.com/

"Many customs in this life persist because they ease friction and promote
productivity as a result of universal agreement, and whether they are
precisely the optimal choices is much less important." --Henry Spencer

From jeremy.cowles at gmail.com  Sun Jun 14 05:48:35 2009
From: jeremy.cowles at gmail.com (Jeremy Cowles)
Date: Sat, 13 Jun 2009 20:48:35 -0700
Subject: [Python-Dev] Distributed computing & sending the interpreter
In-Reply-To: <20090614034422.GA15064@panix.com>
References: <373cf0740906131945o6f17d70dnaa54b2d2b6c85833@mail.gmail.com>
	<20090614034422.GA15064@panix.com>
Message-ID: <373cf0740906132048u3f0989dfx546b689ddf5ca919@mail.gmail.com>

>
> Definitely the wrong list -- python-dev is for people working on the core
> interpreter and libraries.  comp.lang.python would be the standard place,
> but you might also find some good advice on the new list concurrency-sig.


Sorry for the list pollution, I will repost it there.

Thanks,
Jeremy
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090613/60777cc0/attachment.htm>

From cs at zip.com.au  Sun Jun 14 07:16:18 2009
From: cs at zip.com.au (Cameron Simpson)
Date: Sun, 14 Jun 2009 15:16:18 +1000
Subject: [Python-Dev] io.BufferedReader.peek() Behaviour in python3.1
In-Reply-To: <4A34455F.1030106@canterbury.ac.nz>
Message-ID: <20090614051618.GA19302@cskk.homeip.net>

On 14Jun2009 12:33, Greg Ewing <greg.ewing at canterbury.ac.nz> wrote:
> Antoine Pitrou wrote:
>> The original docstring for peek() says:
>>
>>         ...we
>>         do at most one raw read to satisfy it.
>>
>> In that light, I'm not sure it's a bug
>
> It may be behaving according to the docs, but is that
> behaviour useful?
>
> Seems to me that if you're asking for n bytes, then it's
> because you're doing some kind of parsing that requires
> lookahead, and nothing less than n bytes will do.
>
> I think it would be more useful if the "at most one
> raw read" part were dropped. That would give it the
> kind of deterministic behaviour generally expected
> when dealing with buffered streams.

Is it possible to access the buffer? I see nothing in the docs.

People seem to want peek() to be "read() without moving the read
offset", which it almost seems to be. Nick and Greg both want it to
really be that, and thus do enough raw reads to get "n" bytes; Nick
wants a peek1() like read1(), too.

It has a pleasing feel to me, too. But ...

For myself, I'd expect more often to want to see if there's stuff in the
buffer _without_ doing any raw reads at all. A peek0(n), if you will:

  Read and return up to n bytes without calling on the raw stream.

It feels like peek is trying to span both extremes and doesn't satisfy
either really well.

If peek gets enhanced to act like read in terms of the amount of data
returned, should there not be a facility to examine buffered data
without raw reads?

Cheers,
-- 
Cameron Simpson <cs at zip.com.au> DoD#743
http://www.cskk.ezoshosting.com/cs/

Being on a Beemer and not having a wave returned by a Sportster is like
having a clipper ship's hailing not returned by an orphaned New Jersey
solid waste barge.   - OTL

From cs at zip.com.au  Sun Jun 14 07:26:47 2009
From: cs at zip.com.au (Cameron Simpson)
Date: Sun, 14 Jun 2009 15:26:47 +1000
Subject: [Python-Dev] io.BufferedReader.peek() Behaviour in python3.1
In-Reply-To: <20090614051618.GA19302@cskk.homeip.net>
Message-ID: <20090614052647.GA23072@cskk.homeip.net>

On 14Jun2009 15:16, I wrote:
| Is it possible to access the buffer? I see nothing in the docs.

I've just found getvalue() in IOBase. Forget I said anything.
It seems to be my day for that kind of post:-(
-- 
Cameron Simpson <cs at zip.com.au> DoD#743
http://www.cskk.ezoshosting.com/cs/

These are but a few examples of what can happen when the human mind is
employed to learn, to probe, to question as opposed to merely keeping the
ears from touching. - rec.humor.funny 90.07.16

From hfuerstenau at gmx.net  Sun Jun 14 10:18:28 2009
From: hfuerstenau at gmx.net (=?ISO-8859-1?Q?Hagen_F=FCrstenau?=)
Date: Sun, 14 Jun 2009 09:18:28 +0100
Subject: [Python-Dev] Iterator version of contextlib.nested
In-Reply-To: <4A3450DD.6040905@gmail.com>
References: <4A3271BA.2070904@gmx.net> <4A32FA7D.1060906@gmail.com>
	<4A339F84.2070400@gmx.net> <4A3450DD.6040905@gmail.com>
Message-ID: <4A34B254.6050907@gmx.net>

> I actually almost asked for that to be changed to a
> PendingDeprecationWarning when it was first added - Benjamin, do you
> mind if I downgrade this warning to a pending one post rc2?

I'm not sure what that would buy us. For the use case I mentioned it
would be just as annoying to get a PendingDeprecationWarning. But if the
warning was completely removed now, nested could still get deprecated in
3.2 as soon as some better mechanism for a variable number of managers
has been provided.

- Hagen

From steve at pearwood.info  Sun Jun 14 10:59:41 2009
From: steve at pearwood.info (Steven D'Aprano)
Date: Sun, 14 Jun 2009 18:59:41 +1000
Subject: [Python-Dev] ctime: I don't think that word means what you
	think it means.
In-Reply-To: <4A344715.7010604@canterbury.ac.nz>
References: <69132116-861D-4362-A30D-35BCC45A6772@zooko.com>
	<4A344715.7010604@canterbury.ac.nz>
Message-ID: <200906141859.42328.steve@pearwood.info>

On Sun, 14 Jun 2009 10:40:53 am Greg Ewing wrote:
> Zooko Wilcox-O'Hearn wrote:
> > 1.  Add a "st_crtime" field which gets populated on filesystems
> > (Windows, ZFS, Mac) which can do so.
>
> "crtime" looks rather too similar to "ctime" for my
> liking. People who think that the "c" in "ctime"
> means "creation" are still likely to confuse them.
>
> Why not give it a more explicit name, such
> as "st_creationtime"?

Speaking as somebody who thought the c in ctime meant creation, I'm +1 
on this proposal with Greg's modification.



-- 
Steven D'Aprano

From Scott.Daniels at Acm.Org  Sun Jun 14 14:49:12 2009
From: Scott.Daniels at Acm.Org (Scott David Daniels)
Date: Sun, 14 Jun 2009 05:49:12 -0700
Subject: [Python-Dev] ctime: I don't think that word means what you
	think it means.
In-Reply-To: <200906141859.42328.steve@pearwood.info>
References: <69132116-861D-4362-A30D-35BCC45A6772@zooko.com>	<4A344715.7010604@canterbury.ac.nz>
	<200906141859.42328.steve@pearwood.info>
Message-ID: <h12r86$67e$1@ger.gmane.org>

Steven D'Aprano wrote:
> On Sun, 14 Jun 2009 10:40:53 am Greg Ewing wrote:
>> Zooko Wilcox-O'Hearn wrote:
>>> 1.  Add a "st_crtime" field which gets populated on filesystems
>>> (Windows, ZFS, Mac) which can do so.
>> "crtime" looks rather too similar to "ctime" for my
>> liking. People who think that the "c" in "ctime"
>> means "creation" are still likely to confuse them.
>>
>> Why not give it a more explicit name, such
>> as "st_creationtime"?

My this bike-shed looks a bit greenish.  How about:
          creation_time
or, at least,
          st_creation_time

> Speaking as somebody who thought the c in ctime meant creation, I'm +1 
> on this proposal with Greg's modification.

--Scott David Daniels
Scott.Daniels at Acm.Org


From benjamin at python.org  Sun Jun 14 16:21:32 2009
From: benjamin at python.org (Benjamin Peterson)
Date: Sun, 14 Jun 2009 09:21:32 -0500
Subject: [Python-Dev] io.BufferedReader.peek() Behaviour in python3.1
In-Reply-To: <20090614052647.GA23072@cskk.homeip.net>
References: <20090614051618.GA19302@cskk.homeip.net>
	<20090614052647.GA23072@cskk.homeip.net>
Message-ID: <1afaf6160906140721i27610ab5j245a1d8678a73c21@mail.gmail.com>

2009/6/14 Cameron Simpson <cs at zip.com.au>:
> On 14Jun2009 15:16, I wrote:
> | Is it possible to access the buffer? I see nothing in the docs.
>
> I've just found getvalue() in IOBase. Forget I said anything.
> It seems to be my day for that kind of post:-(

Where are you seeing this? Only BytesIO and StringIO have a getvalue() method.


-- 
Regards,
Benjamin

From benjamin at python.org  Sun Jun 14 16:24:46 2009
From: benjamin at python.org (Benjamin Peterson)
Date: Sun, 14 Jun 2009 09:24:46 -0500
Subject: [Python-Dev] Iterator version of contextlib.nested
In-Reply-To: <4A3450DD.6040905@gmail.com>
References: <4A3271BA.2070904@gmx.net> <4A32FA7D.1060906@gmail.com>
	<4A339F84.2070400@gmx.net> <4A3450DD.6040905@gmail.com>
Message-ID: <1afaf6160906140724i72aa9d47mdffa5b24a052a67f@mail.gmail.com>

2009/6/13 Nick Coghlan <ncoghlan at gmail.com>:
> Hagen F?rstenau wrote:
>> I guess this is much too late for 3.1, but could we then at least
>> un-deprecate "contextlib.nested" for now? As it is, you get a
>> DeprecationWarning for something like
>>
>> with contextlib.nested(*my_managers):
>>
>> without any good way to get rid of it.
>
> I actually almost asked for that to be changed to a
> PendingDeprecationWarning when it was first added - Benjamin, do you
> mind if I downgrade this warning to a pending one post rc2?

Yes, I think that's a good idea. It will also help people who have to
use contextlib.nested() for backwards compatibility.



-- 
Regards,
Benjamin

From mrs at mythic-beasts.com  Sun Jun 14 18:09:36 2009
From: mrs at mythic-beasts.com (Mark Seaborn)
Date: Sun, 14 Jun 2009 17:09:36 +0100 (BST)
Subject: [Python-Dev] CPython in the web browser under Native Client
Message-ID: <20090614.170936.846958184.mrs@localhost.localdomain>

I have been doing some work to extend Google's Native Client [1] to
support dynamic linking [2].  For those who haven't heard of it,
Native Client is a sandboxing system for running a subset of x86 code.
It is proposed as a way of running native code inside web apps.

One of my aims has been to get CPython working in the web browser
under Native Client without having to modify CPython.

I recently got to the point where modules from the Python standard
library are importable under Native Client, including (as a
demonstration) the Sqlite extension module.  Sqlite also requires no
modification - it builds straight from the Debian package.

I've written a simple REPL to demonstrate Python running in the
browser.  There are some screenshots on my blog [3].  I haven't
implemented accessing the DOM from Python yet - that's another project
for later. :-)

Mark

[1] http://code.google.com/p/nativeclient/
[2] http://plash.beasts.org/wiki/NativeClient
[3] http://lackingrhoticity.blogspot.com/2009/06/python-standard-library-in-native.html

From mrs at mythic-beasts.com  Sun Jun 14 17:42:55 2009
From: mrs at mythic-beasts.com (Mark Seaborn)
Date: Sun, 14 Jun 2009 16:42:55 +0100 (BST)
Subject: [Python-Dev] Avoiding file descriptors leakage in
 subprocess.Popen()
In-Reply-To: <e04bdf310906121425x72a7c033wec71500712759375@mail.gmail.com>
References: <e04bdf310906121425x72a7c033wec71500712759375@mail.gmail.com>
Message-ID: <20090614.164255.343165597.mrs@localhost.localdomain>

Facundo Batista <facundobatista at gmail.com> wrote:

> errpipe_read, errpipe_write = os.pipe()
> try:
>     try:
>         .....
>         .....
>         .....
>         .....
>         .....
>         .....
>     finally:
>         os.close(errpipe_write)
>     .....
>     .....
>     .....
> finally:
>     os.close(errpipe_read)
> 
> I just don't like a huge try/finally... but as FDs are just ints, do
> you think is there a better way to handle it?

I use a convenience function like this, so that GC takes care of the FDs:

def make_pipe():
    read_fd, write_fd = os.pipe()
    return os.fdopen(read_fd, "r"), os.fdopen(write_fd, "w")

Mark

From sanxiyn at gmail.com  Sun Jun 14 21:09:25 2009
From: sanxiyn at gmail.com (Seo Sanghyeon)
Date: Mon, 15 Jun 2009 04:09:25 +0900
Subject: [Python-Dev] Exception for setting attributes of built-in type
Message-ID: <5b0248170906141209y4398defanacfd0bb62504e550@mail.gmail.com>

Exception for setting attributes of built-in type differs between
CPython and IronPython. This is not purely theoretical, as
zope.interface tries to set Implements declaration as __implemented__
attribute of built-in type object, and excepts TypeError.

Python 2.6.1
>>> object.flag = True
TypeError: can't set attributes of built-in/extension type 'object'

IronPython 2.6
>>> object.flag = True
AttributeError: 'object' object has no attribute 'flag'

I was surprised that CPython raises TypeError. Library Reference seems
to mention it here:

exception AttributeError
Raised when an attribute reference or assignment fails. (When an
object does not support attribute references or attribute assignments
at all, TypeError is raised.)
http://docs.python.org/library/exceptions.html

What does it mean that "an object does not support attribute
references or attribute assignments at all"?

-- 
Seo Sanghyeon

From tjreedy at udel.edu  Mon Jun 15 00:40:56 2009
From: tjreedy at udel.edu (Terry Reedy)
Date: Sun, 14 Jun 2009 18:40:56 -0400
Subject: [Python-Dev] Exception for setting attributes of built-in type
In-Reply-To: <5b0248170906141209y4398defanacfd0bb62504e550@mail.gmail.com>
References: <5b0248170906141209y4398defanacfd0bb62504e550@mail.gmail.com>
Message-ID: <h13u9n$hs$1@ger.gmane.org>

Seo Sanghyeon wrote:
> Exception for setting attributes of built-in type differs between
> CPython and IronPython. This is not purely theoretical, as
> zope.interface tries to set Implements declaration as __implemented__
> attribute of built-in type object, and excepts TypeError.
> 
> Python 2.6.1
>>>> object.flag = True
> TypeError: can't set attributes of built-in/extension type 'object'
> 
> IronPython 2.6
>>>> object.flag = True
> AttributeError: 'object' object has no attribute 'flag'
> 
> I was surprised that CPython raises TypeError. Library Reference seems
> to mention it here:
> 
> exception AttributeError
> Raised when an attribute reference or assignment fails. (When an
> object does not support attribute references or attribute assignments
> at all, TypeError is raised.)
> http://docs.python.org/library/exceptions.html
> 
> What does it mean that "an object does not support attribute
> references or attribute assignments at all"?

I see it as slightly ambiguous:
1. It neither supports references nor assignments.
2. It either does not support reference or it does not support assignments.

1 could hardly apply any more, certainly to built-ins since everything 
now has attributes.  2 covers object, so if that is meant, then 
TypeError is appropriate.

Or were you unclear about 'at all', which means 'never'?

Terry Jan Reedy



From python at mrabarnett.plus.com  Mon Jun 15 00:42:56 2009
From: python at mrabarnett.plus.com (MRAB)
Date: Sun, 14 Jun 2009 23:42:56 +0100
Subject: [Python-Dev] Exception for setting attributes of built-in type
In-Reply-To: <5b0248170906141209y4398defanacfd0bb62504e550@mail.gmail.com>
References: <5b0248170906141209y4398defanacfd0bb62504e550@mail.gmail.com>
Message-ID: <4A357CF0.2050604@mrabarnett.plus.com>

Seo Sanghyeon wrote:
> Exception for setting attributes of built-in type differs between
> CPython and IronPython. This is not purely theoretical, as
> zope.interface tries to set Implements declaration as __implemented__
> attribute of built-in type object, and excepts TypeError.
> 
> Python 2.6.1
>>>> object.flag = True
> TypeError: can't set attributes of built-in/extension type 'object'
> 
> IronPython 2.6
>>>> object.flag = True
> AttributeError: 'object' object has no attribute 'flag'
> 
> I was surprised that CPython raises TypeError. Library Reference seems
> to mention it here:
> 
> exception AttributeError
> Raised when an attribute reference or assignment fails. (When an
> object does not support attribute references or attribute assignments
> at all, TypeError is raised.)
> http://docs.python.org/library/exceptions.html
> 
> What does it mean that "an object does not support attribute
> references or attribute assignments at all"?
> 
Here's my guess:

Some objects have a fixed (and possibly empty) set of attributes.
Attempting to add a new attribute or get a non-existent attribute raises
an AttributeError.

Other objects, such as types (are there any that aren't types?), don't
have attributes at all. Attempting to add or get an attribute raises a
TypeError.

From guido at python.org  Mon Jun 15 01:04:21 2009
From: guido at python.org (Guido van Rossum)
Date: Sun, 14 Jun 2009 16:04:21 -0700
Subject: [Python-Dev] CPython in the web browser under Native Client
In-Reply-To: <20090614.170936.846958184.mrs@localhost.localdomain>
References: <20090614.170936.846958184.mrs@localhost.localdomain>
Message-ID: <ca471dc20906141604y4709c82gff9c049016a8d69a@mail.gmail.com>

Wow. I'm impressed.

On Sun, Jun 14, 2009 at 9:09 AM, Mark Seaborn<mrs at mythic-beasts.com> wrote:
> I have been doing some work to extend Google's Native Client [1] to
> support dynamic linking [2]. ?For those who haven't heard of it,
> Native Client is a sandboxing system for running a subset of x86 code.
> It is proposed as a way of running native code inside web apps.
>
> One of my aims has been to get CPython working in the web browser
> under Native Client without having to modify CPython.
>
> I recently got to the point where modules from the Python standard
> library are importable under Native Client, including (as a
> demonstration) the Sqlite extension module. ?Sqlite also requires no
> modification - it builds straight from the Debian package.
>
> I've written a simple REPL to demonstrate Python running in the
> browser. ?There are some screenshots on my blog [3]. ?I haven't
> implemented accessing the DOM from Python yet - that's another project
> for later. :-)
>
> Mark
>
> [1] http://code.google.com/p/nativeclient/
> [2] http://plash.beasts.org/wiki/NativeClient
> [3] http://lackingrhoticity.blogspot.com/2009/06/python-standard-library-in-native.html
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/guido%40python.org
>



-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From guido at python.org  Mon Jun 15 01:19:18 2009
From: guido at python.org (Guido van Rossum)
Date: Sun, 14 Jun 2009 16:19:18 -0700
Subject: [Python-Dev] Exception for setting attributes of built-in type
In-Reply-To: <4A357CF0.2050604@mrabarnett.plus.com>
References: <5b0248170906141209y4398defanacfd0bb62504e550@mail.gmail.com> 
	<4A357CF0.2050604@mrabarnett.plus.com>
Message-ID: <ca471dc20906141619i6c5d1679x7d99616c2b1fc18e@mail.gmail.com>

On Sun, Jun 14, 2009 at 3:42 PM, MRAB<python at mrabarnett.plus.com> wrote:
> Seo Sanghyeon wrote:
>>
>> Exception for setting attributes of built-in type differs between
>> CPython and IronPython. This is not purely theoretical, as
>> zope.interface tries to set Implements declaration as __implemented__
>> attribute of built-in type object, and excepts TypeError.
>>
>> Python 2.6.1
>>>>>
>>>>> object.flag = True
>>
>> TypeError: can't set attributes of built-in/extension type 'object'
>>
>> IronPython 2.6
>>>>>
>>>>> object.flag = True
>>
>> AttributeError: 'object' object has no attribute 'flag'
>>
>> I was surprised that CPython raises TypeError. Library Reference seems
>> to mention it here:
>>
>> exception AttributeError
>> Raised when an attribute reference or assignment fails. (When an
>> object does not support attribute references or attribute assignments
>> at all, TypeError is raised.)
>> http://docs.python.org/library/exceptions.html
>>
>> What does it mean that "an object does not support attribute
>> references or attribute assignments at all"?
>>
> Here's my guess:
>
> Some objects have a fixed (and possibly empty) set of attributes.
> Attempting to add a new attribute or get a non-existent attribute raises
> an AttributeError.
>
> Other objects, such as types (are there any that aren't types?), don't
> have attributes at all. Attempting to add or get an attribute raises a
> TypeError.

This particular error comes (grep tells me :-) from type_setattro() in
Objects/typeobject.c. It makes a specific check whether the type
object whose attribute you want to set is a "built-in type" (this is
done by checking the HEAPTYPE flag, which is never set for built-in
types and always for user-defined types). For built-in types it
disallows setting attributes. This is a pure policy issue: it prevents
different 3rd party modules to make incompatible modifications of
built-in types.

In general, CPython isn't always consistent in raising AttributeError
and TypeError when it comes to such policy issues: there are various
places that raise TypeError in typeobject.c (and probably elsewhere)
that simply forbid setting a specific attribute (another example is
__name__).

Given how poorly Python exceptions are specified in general, I don't
want to hold IronPython to this standard (nor CPython :-). The reason
this breaks zope.interfaces in IronPython is probably just that
zope.interfaces was primarily written/tested against CPython.

Going back to the phrase quoted from the reference manual, it's
probably referring to the possibility that a type's tp_getattro slot
is NULL; but even so it's wrong: PyObject_GetAttr() raises
AttributeError in this case.

Uselessly y'rs,

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From greg.ewing at canterbury.ac.nz  Mon Jun 15 01:48:39 2009
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Mon, 15 Jun 2009 11:48:39 +1200
Subject: [Python-Dev] io.BufferedReader.peek() Behaviour in python3.1
In-Reply-To: <20090614051618.GA19302@cskk.homeip.net>
References: <20090614051618.GA19302@cskk.homeip.net>
Message-ID: <4A358C57.3060004@canterbury.ac.nz>

Cameron Simpson wrote:

> For myself, I'd expect more often to want to see if there's stuff in the
> buffer _without_ doing any raw reads at all.

What uses do you have in mind for that?

-- 
Greg

From cs at zip.com.au  Mon Jun 15 02:34:07 2009
From: cs at zip.com.au (Cameron Simpson)
Date: Mon, 15 Jun 2009 10:34:07 +1000
Subject: [Python-Dev] io.BufferedReader.peek() Behaviour in python3.1
In-Reply-To: <1afaf6160906140721i27610ab5j245a1d8678a73c21@mail.gmail.com>
Message-ID: <20090615003407.GA12475@cskk.homeip.net>

On 14Jun2009 09:21, Benjamin Peterson <benjamin at python.org> wrote:
| 2009/6/14 Cameron Simpson <cs at zip.com.au>:
| > On 14Jun2009 15:16, I wrote:
| > | Is it possible to access the buffer? I see nothing in the docs.
| >
| > I've just found getvalue() in IOBase. Forget I said anything.
| > It seems to be my day for that kind of post:-(
| 
| Where are you seeing this? Only BytesIO and StringIO have a getvalue() method.

I had thought I'd traced it by class inheritance. But I got BytesIO and
IOBase confused. So: no getvalue then. So probably there is a case for
peek0(), which never does a raw read.

Thoughts?
-- 
Cameron Simpson <cs at zip.com.au> DoD#743
http://www.cskk.ezoshosting.com/cs/

I was gratified to be able to answer promptly and I did.
I said I didn't know.   - Mark Twain

From cs at zip.com.au  Mon Jun 15 03:00:46 2009
From: cs at zip.com.au (Cameron Simpson)
Date: Mon, 15 Jun 2009 11:00:46 +1000
Subject: [Python-Dev] io.BufferedReader.peek() Behaviour in python3.1
In-Reply-To: <4A358C57.3060004@canterbury.ac.nz>
Message-ID: <20090615010046.GA30997@cskk.homeip.net>

On 15Jun2009 11:48, Greg Ewing <greg.ewing at canterbury.ac.nz> wrote:
>> For myself, I'd expect more often to want to see if there's stuff in the
>> buffer _without_ doing any raw reads at all.
>
> What uses do you have in mind for that?

It seems like whenever I want to do some kind of opportunistic but
non-blocking stuff with a remote service (eg something I have a
packetising connection to, such as a TCP stream) I want to be able
to see if there's "more stuff" to gather up before issuing a "flush"
operation. And I want to be able to do that in a non-blocking way,
much as a Queue has a non-blocking get() method.

As an example, I've got the occasional protocol handler where it has to
make a remote query. To avoid deadlock, the stream must be flushed after
write()ing the query packet. However, flushing on every such packet is
horribly wasteful if you know you have a bunch of them (for example,
the caller is asking a bunch of questions). It is far far more efficient
to write() each packet without flushes, keep the knowledge that a flush
is needed, and flush when there's nothing more pending. That way the
lower layer has maximum opportunity to pack data into packets.

All that presumes another thread reading responses, which is how I generally
write this stuff anyway, otherwise a full buffer will deadlock too.

So your dispatch thread inner loop looks like this:

  # single consumer, so Q.empty() implies ok to Q.get()
  needFlush = False
  while not Q.empty():
    P=Q.get()
    if P.needFlush:
      needFlush = True
    out.write(P.serialise())
  if needFlush:
    out.flush()

In this scheme, there _are_ packets that don't need a flush, because nobody
is waiting on their response.

Anyway, if I were reading from an IO object instead of a Queue I'd want
to poll for "buffer empty". If there isn't an empty buffer I know there
will be a packet worth of data coming immediately and I can pick it up
with regular read()s, just as I'm doing with Q.get() above. But if the
buffer is empty I can drop out of the "pick it all up now" loop and
flush().

Cheers,
-- 
Cameron Simpson <cs at zip.com.au> DoD#743
http://www.cskk.ezoshosting.com/cs/

If you take something apart and put it back together again enough times, you
will eventually have enough parts left over to build a second one.
        - Ayse Sercan <ayse at netcom.com>

From python at rcn.com  Mon Jun 15 04:48:18 2009
From: python at rcn.com (Raymond Hettinger)
Date: Sun, 14 Jun 2009 19:48:18 -0700
Subject: [Python-Dev] Iterator version of contextlib.nested
References: <4A3271BA.2070904@gmx.net>
	<4A32FA7D.1060906@gmail.com><4A339F84.2070400@gmx.net>
	<4A3450DD.6040905@gmail.com> <4A34B254.6050907@gmx.net>
Message-ID: <022349E0DE2F46F3B68B0050A7608EF4@RaymondLaptop1>

FWIW, I think resurrecting contextlib.nested() is a bad idea.
Part of the justification for the new with-statement syntax was
that nested() doesn't have a way to finalize the constructors
if one of them fails.   It is a pitfall for the unwary.  And now
that we have the new with-statement syntax, it mostly just
represents a second-way-to-do-it (a second way that has
has the stated pitfall).

The new statement was not designed to support passing in
tuples of context-managers.  This issue was raised while
the new with-statement was being designed and it was
intentionally left-out (in part, because the use cases were
questionable and in-part because there were other ways
to do it such as adding __enter__ and __exit__ to tuple).

I suggest a PEP for 2.7 and 3.2 for building-out the
with-statement to support tuples of context managers
(perhaps modeled after the syntax for except-statements
which allows either individual exceptions or tuples of
exceptions).  The reason I suggest a PEP is that use
cases need to be fully thought out and the failing
constructor problem needs to be dealt with.  

IMO, this represents doing-it-the-right-way instead of 
preserving a construct that is known to be problematic.
Leaving it in will enshrine it.  Better to  just provide our new
syntax that correctly handles the common case and then
wait to carefully think through how to add support for passed-in
nested cm's if in-fact those turn-out to have reasonable
use cases.


Raymond

From benjamin at python.org  Mon Jun 15 04:58:30 2009
From: benjamin at python.org (Benjamin Peterson)
Date: Sun, 14 Jun 2009 21:58:30 -0500
Subject: [Python-Dev] Iterator version of contextlib.nested
In-Reply-To: <022349E0DE2F46F3B68B0050A7608EF4@RaymondLaptop1>
References: <4A3271BA.2070904@gmx.net> <4A32FA7D.1060906@gmail.com>
	<4A339F84.2070400@gmx.net> <4A3450DD.6040905@gmail.com>
	<4A34B254.6050907@gmx.net>
	<022349E0DE2F46F3B68B0050A7608EF4@RaymondLaptop1>
Message-ID: <1afaf6160906141958s3bb76c11o7c60c3a2ceb98e9a@mail.gmail.com>

2009/6/14 Raymond Hettinger <python at rcn.com>:
> FWIW, I think resurrecting contextlib.nested() is a bad idea.
> Part of the justification for the new with-statement syntax was
> that nested() doesn't have a way to finalize the constructors
> if one of them fails. ? It is a pitfall for the unwary. ?And now
> that we have the new with-statement syntax, it mostly just
> represents a second-way-to-do-it (a second way that has
> has the stated pitfall).

I don't consider changing a DeprecationWarning to a
PendingDeprecationWarning "resurrecting" its target. Fully deprecating
a feature in the same version that we add its replacement will just
make more difficulties for cross-version libraries.

....

> I suggest a PEP for 2.7 and 3.2 for building-out the
> with-statement to support tuples of context managers
> (perhaps modeled after the syntax for except-statements
> which allows either individual exceptions or tuples of
> exceptions).

I think the question of extending the syntax later is orthogonal to
the issue of the DeprecationWarning.



-- 
Regards,
Benjamin

From hfuerstenau at gmx.net  Mon Jun 15 10:11:51 2009
From: hfuerstenau at gmx.net (=?ISO-8859-1?Q?Hagen_F=FCrstenau?=)
Date: Mon, 15 Jun 2009 09:11:51 +0100
Subject: [Python-Dev] Iterator version of contextlib.nested
In-Reply-To: <022349E0DE2F46F3B68B0050A7608EF4@RaymondLaptop1>
References: <4A3271BA.2070904@gmx.net>
	<4A32FA7D.1060906@gmail.com><4A339F84.2070400@gmx.net>
	<4A3450DD.6040905@gmail.com> <4A34B254.6050907@gmx.net>
	<022349E0DE2F46F3B68B0050A7608EF4@RaymondLaptop1>
Message-ID: <4A360247.5060407@gmx.net>

> Part of the justification for the new with-statement syntax was
> that nested() doesn't have a way to finalize the constructors
> if one of them fails.

I think the problem was a little bit more subtle: nested() gets passed
managers, so their __init__()s should all have run when the first
context is entered. The only problem comes up when the __exit__() of an
outer manager tries to suppress an exception raised by the __enter__()
of an inner one. This is a limited defect in that it doesn't affect the
common situation where no __exit__() tries to suppress any exceptions.
(In a quick glance over the std library I couldn't find a single
instance of an exception-suppressing __exit__().).

> And now
> that we have the new with-statement syntax, it mostly just
> represents a second-way-to-do-it (a second way that has
> has the stated pitfall).

So the functionalities of nested() and multi-with overlap in the common
use cases, and each has its own limitation in an uncommon one. I agree
that this situation is unfortunate, but I think introducing support for
one uncommon case and removing it for another is not the way to go in
3.1. That's why I think nested() should stay un-deprecated until there
is a replacement which handles a superset of its use cases.

> The new statement was not designed to support passing in
> tuples of context-managers.  This issue was raised while
> the new with-statement was being designed and it was
> intentionally left-out (in part, because the use cases were
> questionable

FWIW, my use case (which made me notice the DeprecationWarning in the
first place) is in a command dispatch function, which looks at the
command to be executed and pre-processes its arguments in a uniform way.
Part of that pre-processing is entering contexts of context manager
before passing them along (and exiting them when the command finishes or
raises an exception).

> and in-part because there were other ways
> to do it such as adding __enter__ and __exit__ to tuple).

Which has not been done for 3.1. Granted, you could subclass tuple and
add them yourself, but then you would mostly be copying what's already
implemented in nested().

> I suggest a PEP for 2.7 and 3.2 for building-out the
> with-statement to support tuples of context managers

That sounds like a good idea.

> IMO, this represents doing-it-the-right-way instead of preserving a
> construct that is known to be problematic.
> Leaving it in will enshrine it.

I don't see the problem with deprecating it only after a completely
suitable replacement is found. Why would it be any harder to deprecate
nested() in 3.2?

- Hagen


From ncoghlan at gmail.com  Mon Jun 15 12:47:45 2009
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Mon, 15 Jun 2009 20:47:45 +1000
Subject: [Python-Dev] Iterator version of contextlib.nested
In-Reply-To: <4A34B254.6050907@gmx.net>
References: <4A3271BA.2070904@gmx.net> <4A32FA7D.1060906@gmail.com>
	<4A339F84.2070400@gmx.net> <4A3450DD.6040905@gmail.com>
	<4A34B254.6050907@gmx.net>
Message-ID: <4A3626D1.5000402@gmail.com>

Hagen F?rstenau wrote:
>> I actually almost asked for that to be changed to a
>> PendingDeprecationWarning when it was first added - Benjamin, do you
>> mind if I downgrade this warning to a pending one post rc2?
> 
> I'm not sure what that would buy us. For the use case I mentioned it
> would be just as annoying to get a PendingDeprecationWarning. But if the
> warning was completely removed now, nested could still get deprecated in
> 3.2 as soon as some better mechanism for a variable number of managers
> has been provided.

Unlike a full DeprecationWarning, a PendingDeprecationWarning is ignored
by default. You have to switch them on explicitly via code or a command
line switch in order to see them.

Pending warnings give the hint that we intend to get rid of something,
but either don't have clear replacements for some legitimate use cases
(as in the case of contextlib.nested) or have some other reason for not
generating a warning by default (e.g. we may not have cleared all uses
out of the standard library yet).

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------

From hfuerstenau at gmx.net  Mon Jun 15 12:56:47 2009
From: hfuerstenau at gmx.net (=?ISO-8859-1?Q?Hagen_F=FCrstenau?=)
Date: Mon, 15 Jun 2009 11:56:47 +0100
Subject: [Python-Dev] Iterator version of contextlib.nested
In-Reply-To: <4A3626D1.5000402@gmail.com>
References: <4A3271BA.2070904@gmx.net> <4A32FA7D.1060906@gmail.com>
	<4A339F84.2070400@gmx.net> <4A3450DD.6040905@gmail.com>
	<4A34B254.6050907@gmx.net> <4A3626D1.5000402@gmail.com>
Message-ID: <4A3628EF.4010405@gmx.net>

> Unlike a full DeprecationWarning, a PendingDeprecationWarning is ignored
> by default. You have to switch them on explicitly via code or a command
> line switch in order to see them.

Sorry, I should have made myself more familiar with the warnings
mechanism before writing. In that case I'm fine with a
PendingDeprecationWarning. :-)

- Hagen


From ncoghlan at gmail.com  Mon Jun 15 13:13:03 2009
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Mon, 15 Jun 2009 21:13:03 +1000
Subject: [Python-Dev] Iterator version of contextlib.nested
In-Reply-To: <022349E0DE2F46F3B68B0050A7608EF4@RaymondLaptop1>
References: <4A3271BA.2070904@gmx.net>
	<4A32FA7D.1060906@gmail.com><4A339F84.2070400@gmx.net>
	<4A3450DD.6040905@gmail.com> <4A34B254.6050907@gmx.net>
	<022349E0DE2F46F3B68B0050A7608EF4@RaymondLaptop1>
Message-ID: <4A362CBF.7070609@gmail.com>

Raymond Hettinger wrote:
> I suggest a PEP for 2.7 and 3.2 for building-out the
> with-statement to support tuples of context managers
> (perhaps modeled after the syntax for except-statements
> which allows either individual exceptions or tuples of
> exceptions).  The reason I suggest a PEP is that use
> cases need to be fully thought out and the failing
> constructor problem needs to be dealt with. 
> IMO, this represents doing-it-the-right-way instead of preserving a
> construct that is known to be problematic.
> Leaving it in will enshrine it.  Better to  just provide our new
> syntax that correctly handles the common case and then
> wait to carefully think through how to add support for passed-in
> nested cm's if in-fact those turn-out to have reasonable
> use cases.

I agree that the current incarnation of contextlib.nested needs to go -
it isn't really salvagable in its current form.

However, I don't think we should generate a warning for it by default
until we provide a new mechanism for handling a variable number of
context managers - PendingDeprecationWarning seems a much better fit.

A 2.7/3.2 PEP can then address the two main issues with the current
approach:

1. The __init__ calls for the inner context managers occur outside the
scope of the outer context managers. Some form of lazy evaluation would
be needed to deal with that.

2. If an inner __enter__ call raises an exception that is suppressed by
an outer __exit__ call then the body of with statement should be skipped
rather than raising RuntimeError. This means either new syntax with
semantics along the lines of the previously rejected PEP 377 or else a
custom exception and a second context manager that suppresses it.

Personally, I don't think new syntax for the PEP 377 semantics is
warranted for the same reason that PEP 377 itself was rejected - it
complicates the statement definition significantly for a really obscure
corner case. Better to come up with a new and improved version of nested
that eliminates (or better handles) the quirks and leave the statement
definition alone.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------

From ncoghlan at gmail.com  Mon Jun 15 13:18:54 2009
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Mon, 15 Jun 2009 21:18:54 +1000
Subject: [Python-Dev] CPython in the web browser under Native Client
In-Reply-To: <20090614.170936.846958184.mrs@localhost.localdomain>
References: <20090614.170936.846958184.mrs@localhost.localdomain>
Message-ID: <4A362E1E.3060007@gmail.com>

Mark Seaborn wrote:
> [3] http://lackingrhoticity.blogspot.com/2009/06/python-standard-library-in-native.html

Very cool!

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------

From ncoghlan at gmail.com  Mon Jun 15 13:27:24 2009
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Mon, 15 Jun 2009 21:27:24 +1000
Subject: [Python-Dev] Exception for setting attributes of built-in type
In-Reply-To: <ca471dc20906141619i6c5d1679x7d99616c2b1fc18e@mail.gmail.com>
References: <5b0248170906141209y4398defanacfd0bb62504e550@mail.gmail.com>
	<4A357CF0.2050604@mrabarnett.plus.com>
	<ca471dc20906141619i6c5d1679x7d99616c2b1fc18e@mail.gmail.com>
Message-ID: <4A36301C.2050100@gmail.com>

Guido van Rossum wrote:
> In general, CPython isn't always consistent in raising AttributeError
> and TypeError when it comes to such policy issues: there are various
> places that raise TypeError in typeobject.c (and probably elsewhere)
> that simply forbid setting a specific attribute (another example is
> __name__).

We're pretty inconsistent when it comes to looking up special methods as
well - those that are looked up through dedicated slots in abstract.c
usually raise TypeError, while those that are looked up via a PyType
method usually raise AttributeError.

I'm not sure that there is any practical way to handle that other than
for applications that care about cross-implementation compatibility in
this area to catch a (TypeError, AttributeError) tuple rather than one
or the other.

CHeers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------

From kalman.gergely at duodecad.hu  Mon Jun 15 13:47:55 2009
From: kalman.gergely at duodecad.hu (=?UTF-8?B?S8OhbG3DoW4gR2VyZ2VseQ==?=)
Date: Mon, 15 Jun 2009 13:47:55 +0200
Subject: [Python-Dev] python sendmsg()/recvmsg() implementation
In-Reply-To: <20090609151026.22176.614294214.divmod.quotient.3795@henry.divmod.com>
References: <20090609151026.22176.614294214.divmod.quotient.3795@henry.divmod.com>
Message-ID: <4A3634EB.2020800@duodecad.hu>

Jean-Paul Calderone wrote:
> On Tue, 09 Jun 2009 16:46:54 +0200, K?lm?n Gergely 
> <kalman.gergely at duodecad.hu> wrote:
>> Hello, my name is Greg.
>>
>> I've just started using python after many years of C programming, and 
>> I'm also new to the list. I wanted to clarify this
>> first, so that maybe I will get a little less beating for my 
>> stupidity :)
>>
>
> Welcome!
>
>>
>> [snip]
>>
>> Browsing the net I've found a patch to the python core 
>> (http://bugs.python.org/issue1194378), dated 2005.
>>
>> First of all, I would like to ask you guys, whether you know of any 
>> way of doing this FD passing magic, or that you know
>> of any 3rd party module / patch / anything that can do this for me.
>
> Aside from the patch in the tracker, there are several implementations of
> these APIs as third-party extension modules.
>
>>
>> Since I'm fairly familiar with C and (not that much, but I feel the 
>> power) python, I would take the challenge of writing
>> it, given that the above code is still somewhat usable. If all else 
>> fails I would like to have your help to guide me through
>> this process.
>>
>
> What would be great is if you could take the patch in the tracker and get
> it into shape so that it is suitable for inclusion.  This would involve
> three things, I think:
>
>  1. Write unit tests for the functionality (since the patch itself 
> provides
>     none)
>  2. Update the patch so that it again applies cleanly against trunk
>  3. Add documentation for the new APIs
>
> Once this is done, you can get a committer to look at it and either 
> provide
> more specific feedback or apply it.
>
> Thanks,
>
> Jean-Paul
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: 
> http://mail.python.org/mailman/options/python-dev/synapse%40jasmin.hu

Hello again

So, after a little cleanup I've managed to integrate the code into 
socketmodule.c/h. It works fine now, though I needed to
add it to Lib/socket.py, otherwise it wouldn't show up in the socket 
module (I've searched for recvfrom and added it).
I've also cleaned up the code a little, fixed some codingstyle issues 
(which might still exist).

Since I am not a python core developer the patch might still be in a 
pretty outdated state. I'd like someone to look it over
and direct me to some documentation (the ones I've found so far were 
pretty sketchy), for it to be acceptable for inclusion.
The sanity of the code is what matters to me the most. I've looked it 
over though and found it in a sound state, but I guess
you guys might have a different opinion about that ;)

With writing the test cases, some documentation would be nice.

I've attached the patch generated with svn diff for revision 73434, and 
the test code that I use to pass a file descriptor
between processes. It works just fine :).

Thanks

Kalman Gergely

-------------- next part --------------
A non-text attachment was scrubbed...
Name: sendmsg.diff
Type: text/x-patch
Size: 19805 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090615/588e82e4/attachment-0001.bin>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: sendfd.py
Type: text/x-python
Size: 1353 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090615/588e82e4/attachment-0001.py>

From ncoghlan at gmail.com  Mon Jun 15 16:32:51 2009
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Tue, 16 Jun 2009 00:32:51 +1000
Subject: [Python-Dev] python sendmsg()/recvmsg() implementation
In-Reply-To: <4A3634EB.2020800@duodecad.hu>
References: <20090609151026.22176.614294214.divmod.quotient.3795@henry.divmod.com>
	<4A3634EB.2020800@duodecad.hu>
Message-ID: <4A365B93.7020501@gmail.com>

K?lm?n Gergely wrote:
> Since I am not a python core developer the patch might still be in a
> pretty outdated state. I'd like someone to look it over
> and direct me to some documentation (the ones I've found so far were
> pretty sketchy), for it to be acceptable for inclusion.
> The sanity of the code is what matters to me the most. I've looked it
> over though and found it in a sound state, but I guess
> you guys might have a different opinion about that ;)
> 
> With writing the test cases, some documentation would be nice.

Most unit tests these days are written based on either doctest or
unittest. When adding new features to an existing module, it is usually
best to follow the testing style already used for the rest of that
module (in this case, that should be test/test_socket.py).

The relevant question in the dev FAQ gives good pointers:
http://www.python.org/dev/faq/#how-to-test-a-patch

> I've attached the patch generated with svn diff for revision 73434, and
> the test code that I use to pass a file descriptor
> between processes. It works just fine :).

Uploading files to the tracker is generally the best option - patches
tend to get lost if they're just sent to the mailing list.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------

From guido at python.org  Mon Jun 15 16:48:33 2009
From: guido at python.org (Guido van Rossum)
Date: Mon, 15 Jun 2009 07:48:33 -0700
Subject: [Python-Dev] Exception for setting attributes of built-in type
In-Reply-To: <ca471dc20906141619i6c5d1679x7d99616c2b1fc18e@mail.gmail.com>
References: <5b0248170906141209y4398defanacfd0bb62504e550@mail.gmail.com> 
	<4A357CF0.2050604@mrabarnett.plus.com>
	<ca471dc20906141619i6c5d1679x7d99616c2b1fc18e@mail.gmail.com>
Message-ID: <ca471dc20906150748t2b98e999re6505776c8efe2c1@mail.gmail.com>

On Sun, Jun 14, 2009 at 4:19 PM, Guido van Rossum<guido at python.org> wrote:
> In general, CPython isn't always consistent in raising AttributeError
> and TypeError when it comes to such policy issues: there are various
> places that raise TypeError in typeobject.c (and probably elsewhere)
> that simply forbid setting a specific attribute (another example is
> __name__).

I should add that this policy is also forced somewhat by the existence
of the "multiple interpreters in one address space" feature, which is
used e.g. by mod_python. This feature attempts to provide isolation
between interpreters to the point that each one can have a completely
different set of modules loaded and can be working on a totally
different application. The implementation of CPython shares built-in
types between multiple interpreters (and it wouldn't be easy to change
this); if you were able to modify a built-in type from one
interpreter, all other interpreters would see that same modification.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From dinov at microsoft.com  Mon Jun 15 17:54:51 2009
From: dinov at microsoft.com (Dino Viehland)
Date: Mon, 15 Jun 2009 15:54:51 +0000
Subject: [Python-Dev] [IronPython] Exception for setting attributes
	of	built-in type
In-Reply-To: <ca471dc20906150748t2b98e999re6505776c8efe2c1@mail.gmail.com>
References: <5b0248170906141209y4398defanacfd0bb62504e550@mail.gmail.com>
	<4A357CF0.2050604@mrabarnett.plus.com>
	<ca471dc20906141619i6c5d1679x7d99616c2b1fc18e@mail.gmail.com>
	<ca471dc20906150748t2b98e999re6505776c8efe2c1@mail.gmail.com>
Message-ID: <1A472770E042064698CB5ADC83A12ACD0324503A@TK5EX14MBXC116.redmond.corp.microsoft.com>


Guido wrote:
> I should add that this policy is also forced somewhat by the existence
> of the "multiple interpreters in one address space" feature, which is
> used e.g. by mod_python. This feature attempts to provide isolation
> between interpreters to the point that each one can have a completely
> different set of modules loaded and can be working on a totally
> different application. The implementation of CPython shares built-in
> types between multiple interpreters (and it wouldn't be easy to change
> this); if you were able to modify a built-in type from one
> interpreter, all other interpreters would see that same modification.

IronPython is in the exact same boat here - we share built-in types
Across multiple Python engines as well.



From fuzzyman at voidspace.org.uk  Mon Jun 15 18:10:08 2009
From: fuzzyman at voidspace.org.uk (Michael Foord)
Date: Mon, 15 Jun 2009 17:10:08 +0100
Subject: [Python-Dev] [IronPython] Exception for setting attributes	of
 built-in type
In-Reply-To: <1A472770E042064698CB5ADC83A12ACD0324503A@TK5EX14MBXC116.redmond.corp.microsoft.com>
References: <5b0248170906141209y4398defanacfd0bb62504e550@mail.gmail.com>	<4A357CF0.2050604@mrabarnett.plus.com>	<ca471dc20906141619i6c5d1679x7d99616c2b1fc18e@mail.gmail.com>	<ca471dc20906150748t2b98e999re6505776c8efe2c1@mail.gmail.com>
	<1A472770E042064698CB5ADC83A12ACD0324503A@TK5EX14MBXC116.redmond.corp.microsoft.com>
Message-ID: <4A367260.2010400@voidspace.org.uk>

Dino Viehland wrote:
> Guido wrote:
>   
>> I should add that this policy is also forced somewhat by the existence
>> of the "multiple interpreters in one address space" feature, which is
>> used e.g. by mod_python. This feature attempts to provide isolation
>> between interpreters to the point that each one can have a completely
>> different set of modules loaded and can be working on a totally
>> different application. The implementation of CPython shares built-in
>> types between multiple interpreters (and it wouldn't be easy to change
>> this); if you were able to modify a built-in type from one
>> interpreter, all other interpreters would see that same modification.
>>     
>
> IronPython is in the exact same boat here - we share built-in types
> Across multiple Python engines as well.
>
>
>   

And indeed it is needed - if you are working with multiple interpreters 
(engines in IronPython) you don't want isinstance(something, dict) to 
fail because it is a dictionary from a different interpreter...

Michael

> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk
>   


-- 
http://www.ironpythoninaction.com/
http://www.voidspace.org.uk/blog



From guido at python.org  Mon Jun 15 19:10:59 2009
From: guido at python.org (Guido van Rossum)
Date: Mon, 15 Jun 2009 10:10:59 -0700
Subject: [Python-Dev] [IronPython] Exception for setting attributes of
	built-in type
In-Reply-To: <4A367260.2010400@voidspace.org.uk>
References: <5b0248170906141209y4398defanacfd0bb62504e550@mail.gmail.com> 
	<4A357CF0.2050604@mrabarnett.plus.com>
	<ca471dc20906141619i6c5d1679x7d99616c2b1fc18e@mail.gmail.com> 
	<ca471dc20906150748t2b98e999re6505776c8efe2c1@mail.gmail.com> 
	<1A472770E042064698CB5ADC83A12ACD0324503A@TK5EX14MBXC116.redmond.corp.microsoft.com>
	<4A367260.2010400@voidspace.org.uk>
Message-ID: <ca471dc20906151010y219cfe05va419b217c9504d22@mail.gmail.com>

On Mon, Jun 15, 2009 at 9:10 AM, Michael Foord<fuzzyman at voidspace.org.uk> wrote:
> Dino Viehland wrote:
>> Guido wrote:
>>> I should add that this policy is also forced somewhat by the existence
>>> of the "multiple interpreters in one address space" feature, which is
>>> used e.g. by mod_python. This feature attempts to provide isolation
>>> between interpreters to the point that each one can have a completely
>>> different set of modules loaded and can be working on a totally
>>> different application. The implementation of CPython shares built-in
>>> types between multiple interpreters (and it wouldn't be easy to change
>>> this); if you were able to modify a built-in type from one
>>> interpreter, all other interpreters would see that same modification.

>> IronPython is in the exact same boat here - we share built-in types
>> Across multiple Python engines as well.

> And indeed it is needed - if you are working with multiple interpreters
> (engines in IronPython) you don't want isinstance(something, dict) to fail
> because it is a dictionary from a different interpreter...

Ah, but that suggests you have sharing between different interpreters.
If you're doing that, perhaps you shouldn't be using multiple
interpreters, but instead multiple threads?

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From fuzzyman at voidspace.org.uk  Mon Jun 15 19:20:41 2009
From: fuzzyman at voidspace.org.uk (Michael Foord)
Date: Mon, 15 Jun 2009 18:20:41 +0100
Subject: [Python-Dev] [IronPython] Exception for setting attributes of
 built-in type
In-Reply-To: <ca471dc20906151010y219cfe05va419b217c9504d22@mail.gmail.com>
References: <5b0248170906141209y4398defanacfd0bb62504e550@mail.gmail.com>
	<4A357CF0.2050604@mrabarnett.plus.com>
	<ca471dc20906141619i6c5d1679x7d99616c2b1fc18e@mail.gmail.com>
	<ca471dc20906150748t2b98e999re6505776c8efe2c1@mail.gmail.com>
	<1A472770E042064698CB5ADC83A12ACD0324503A@TK5EX14MBXC116.redmond.corp.microsoft.com>
	<4A367260.2010400@voidspace.org.uk>
	<ca471dc20906151010y219cfe05va419b217c9504d22@mail.gmail.com>
Message-ID: <4A3682E9.3000405@voidspace.org.uk>

Guido van Rossum wrote:
> On Mon, Jun 15, 2009 at 9:10 AM, Michael Foord<fuzzyman at voidspace.org.uk> wrote:
>   
>> Dino Viehland wrote:
>>     
>>> Guido wrote:
>>>       
>>>> I should add that this policy is also forced somewhat by the existence
>>>> of the "multiple interpreters in one address space" feature, which is
>>>> used e.g. by mod_python. This feature attempts to provide isolation
>>>> between interpreters to the point that each one can have a completely
>>>> different set of modules loaded and can be working on a totally
>>>> different application. The implementation of CPython shares built-in
>>>> types between multiple interpreters (and it wouldn't be easy to change
>>>> this); if you were able to modify a built-in type from one
>>>> interpreter, all other interpreters would see that same modification.
>>>>         
>
>   
>>> IronPython is in the exact same boat here - we share built-in types
>>> Across multiple Python engines as well.
>>>       
>
>   
>> And indeed it is needed - if you are working with multiple interpreters
>> (engines in IronPython) you don't want isinstance(something, dict) to fail
>> because it is a dictionary from a different interpreter...
>>     
>
> Ah, but that suggests you have sharing between different interpreters.
> If you're doing that, perhaps you shouldn't be using multiple
> interpreters, but instead multiple threads?
>
>   
Well, in our use case we use multiple engines to provide an isolated 
execution context for every document (the Resolver One spreadsheet 
written in IronPython). Each of these has their own calculation thread 
as well - but the engine per document structure is nice and clean and 
means each document can have its own set of modules loaded without 
affecting the other documents (although they share a core set of modules).

Once we move these engines into their own app domains we can completely 
isolate each document and apply separate security permissions to each 
one. That might mean each document effectively paying the 
not-insubstantial startup time hit and we haven't begun to look at how 
to mitigate that.

Michael

-- 
http://www.ironpythoninaction.com/
http://www.voidspace.org.uk/blog



From benjamin at python.org  Mon Jun 15 21:09:00 2009
From: benjamin at python.org (Benjamin Peterson)
Date: Mon, 15 Jun 2009 14:09:00 -0500
Subject: [Python-Dev] Exception for setting attributes of built-in type
In-Reply-To: <4A36301C.2050100@gmail.com>
References: <5b0248170906141209y4398defanacfd0bb62504e550@mail.gmail.com>
	<4A357CF0.2050604@mrabarnett.plus.com>
	<ca471dc20906141619i6c5d1679x7d99616c2b1fc18e@mail.gmail.com>
	<4A36301C.2050100@gmail.com>
Message-ID: <1afaf6160906151209h10138643ob957e1235176ba45@mail.gmail.com>

2009/6/15 Nick Coghlan <ncoghlan at gmail.com>:
> Guido van Rossum wrote:
>> In general, CPython isn't always consistent in raising AttributeError
>> and TypeError when it comes to such policy issues: there are various
>> places that raise TypeError in typeobject.c (and probably elsewhere)
>> that simply forbid setting a specific attribute (another example is
>> __name__).
>
> We're pretty inconsistent when it comes to looking up special methods as
> well - those that are looked up through dedicated slots in abstract.c
> usually raise TypeError, while those that are looked up via a PyType
> method usually raise AttributeError.

What's a PyType method?



-- 
Regards,
Benjamin

From python at rcn.com  Mon Jun 15 21:17:55 2009
From: python at rcn.com (Raymond Hettinger)
Date: Mon, 15 Jun 2009 12:17:55 -0700
Subject: [Python-Dev] Iterator version of contextlib.nested
References: <4A3271BA.2070904@gmx.net> <4A32FA7D.1060906@gmail.com>
	<4A339F84.2070400@gmx.net> <4A3450DD.6040905@gmail.com>
	<4A34B254.6050907@gmx.net>
	<022349E0DE2F46F3B68B0050A7608EF4@RaymondLaptop1>
	<1afaf6160906141958s3bb76c11o7c60c3a2ceb98e9a@mail.gmail.com>
Message-ID: <ACD6D03AF454484094BF805AAC4F9783@RaymondLaptop1>

> I don't consider changing a DeprecationWarning to a
> PendingDeprecationWarning "resurrecting" its target.

Seems like resurrection to me.  Pending warnings are hidden
by default, so someone has to go look for it (and no one does this).

The problem with the nested() construct is not so much that
it duplicates the new with-statement; the problem is that it
is a bug factory when used as advertised.  The sole justification
for keeping it around is that it handles an obscure use case
(one that isn't even shown in its documentation or examples).

I'm not opposing the idea to change the DeprecationWarning
to a PendingDeprecationWarning, but I don't think we're doing
the users any favors by hiding the warning message.


Raymond


P.S.  If you switch to PendingDeprecationWarning, the example
in the docs should probably be switched to show the one valid
use case (passing in a prepackaged nest of context managers).
Right now, the current example just shows the hazardous pattern
that is much better served by the new with-statement syntax.


From benjamin at python.org  Mon Jun 15 21:29:22 2009
From: benjamin at python.org (Benjamin Peterson)
Date: Mon, 15 Jun 2009 14:29:22 -0500
Subject: [Python-Dev] Iterator version of contextlib.nested
In-Reply-To: <ACD6D03AF454484094BF805AAC4F9783@RaymondLaptop1>
References: <4A3271BA.2070904@gmx.net> <4A32FA7D.1060906@gmail.com>
	<4A339F84.2070400@gmx.net> <4A3450DD.6040905@gmail.com>
	<4A34B254.6050907@gmx.net>
	<022349E0DE2F46F3B68B0050A7608EF4@RaymondLaptop1>
	<1afaf6160906141958s3bb76c11o7c60c3a2ceb98e9a@mail.gmail.com>
	<ACD6D03AF454484094BF805AAC4F9783@RaymondLaptop1>
Message-ID: <1afaf6160906151229g226c67acpa957ed4087a2dfde@mail.gmail.com>

2009/6/15 Raymond Hettinger <python at rcn.com>:
> P.S. ?If you switch to PendingDeprecationWarning, the example
> in the docs should probably be switched to show the one valid
> use case (passing in a prepackaged nest of context managers).
> Right now, the current example just shows the hazardous pattern
> that is much better served by the new with-statement syntax.


+1 I think this should be done anyway.



-- 
Regards,
Benjamin

From tjreedy at udel.edu  Mon Jun 15 22:36:13 2009
From: tjreedy at udel.edu (Terry Reedy)
Date: Mon, 15 Jun 2009 16:36:13 -0400
Subject: [Python-Dev] Iterator version of contextlib.nested
In-Reply-To: <ACD6D03AF454484094BF805AAC4F9783@RaymondLaptop1>
References: <4A3271BA.2070904@gmx.net>
	<4A32FA7D.1060906@gmail.com>	<4A339F84.2070400@gmx.net>
	<4A3450DD.6040905@gmail.com>	<4A34B254.6050907@gmx.net>	<022349E0DE2F46F3B68B0050A7608EF4@RaymondLaptop1>	<1afaf6160906141958s3bb76c11o7c60c3a2ceb98e9a@mail.gmail.com>
	<ACD6D03AF454484094BF805AAC4F9783@RaymondLaptop1>
Message-ID: <h16bbr$fs6$2@ger.gmane.org>

Raymond Hettinger wrote:

> P.S.  If you switch to PendingDeprecationWarning, the example
> in the docs should probably be switched to show the one valid
> use case (passing in a prepackaged nest of context managers).

It could even suggest that it only be used for this, since it may 
disappear, and that other uses should use the new syntax.  That would 
give people the best chance of writing future-proof code when they can.


From p.f.moore at gmail.com  Mon Jun 15 22:41:11 2009
From: p.f.moore at gmail.com (Paul Moore)
Date: Mon, 15 Jun 2009 21:41:11 +0100
Subject: [Python-Dev] Iterator version of contextlib.nested
In-Reply-To: <h16bbr$fs6$2@ger.gmane.org>
References: <4A3271BA.2070904@gmx.net> <4A32FA7D.1060906@gmail.com>
	<4A339F84.2070400@gmx.net> <4A3450DD.6040905@gmail.com>
	<4A34B254.6050907@gmx.net>
	<022349E0DE2F46F3B68B0050A7608EF4@RaymondLaptop1>
	<1afaf6160906141958s3bb76c11o7c60c3a2ceb98e9a@mail.gmail.com>
	<ACD6D03AF454484094BF805AAC4F9783@RaymondLaptop1>
	<h16bbr$fs6$2@ger.gmane.org>
Message-ID: <79990c6b0906151341k2224a02eu202ad7bac6f90774@mail.gmail.com>

2009/6/15 Terry Reedy <tjreedy at udel.edu>:
> Raymond Hettinger wrote:
>
>> P.S. ?If you switch to PendingDeprecationWarning, the example
>> in the docs should probably be switched to show the one valid
>> use case (passing in a prepackaged nest of context managers).
>
> It could even suggest that it only be used for this, since it may disappear,
> and that other uses should use the new syntax. ?That would give people the
> best chance of writing future-proof code when they can.

And if the warning is *not* changed to a PendingDeprecationWarning,
the documentation could also note the necessary warnings call needed
to let the example code run warning-free.

Paul.

From p.f.moore at gmail.com  Mon Jun 15 22:46:40 2009
From: p.f.moore at gmail.com (Paul Moore)
Date: Mon, 15 Jun 2009 21:46:40 +0100
Subject: [Python-Dev] Is the py3k mercurial mirror broken?
Message-ID: <79990c6b0906151346w7e104b2el59280d4dc529c391@mail.gmail.com>

I get the following:

>hg version
Mercurial Distributed SCM (version 1.2)

Copyright (C) 2005-2008 Matt Mackall <mpm at selenic.com> and others
This is free software; see the source for copying conditions. There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

>hg clone http://code.python.org/hg/branches/py3k python-3k
requesting all changes
adding changesets
transaction abort!
rollback completed
** unknown exception encountered, details follow
** report bug details to http://www.selenic.com/mercurial/bts
** or mercurial at selenic.com
** Mercurial Distributed SCM (version 1.2)
** Extensions loaded:
Traceback (most recent call last):
  File "hg", line 27, in <module>
  File "mercurial\dispatch.pyc", line 16, in run
  File "mercurial\dispatch.pyc", line 25, in dispatch
  File "mercurial\dispatch.pyc", line 41, in _runcatch
  File "mercurial\dispatch.pyc", line 372, in _dispatch
  File "mercurial\dispatch.pyc", line 247, in runcommand
  File "mercurial\dispatch.pyc", line 417, in _runcommand
  File "mercurial\dispatch.pyc", line 377, in checkargs
  File "mercurial\dispatch.pyc", line 371, in <lambda>
  File "mercurial\util.pyc", line 718, in check
  File "mercurial\commands.pyc", line 603, in clone
  File "mercurial\hg.pyc", line 215, in clone
  File "mercurial\localrepo.pyc", line 2149, in clone
  File "mercurial\localrepo.pyc", line 1492, in pull
  File "mercurial\localrepo.pyc", line 2016, in addchangegroup
  File "mercurial\revlog.pyc", line 1192, in addgroup
  File "mercurial\changegroup.pyc", line 31, in chunkiter
  File "mercurial\changegroup.pyc", line 15, in getchunk
  File "mercurial\util.pyc", line 1674, in read
  File "mercurial\httprepo.pyc", line 18, in zgenerator
zlib.error: Error -3 while decompressing: incorrect header check

Paul.

From benjamin at python.org  Mon Jun 15 22:49:11 2009
From: benjamin at python.org (Benjamin Peterson)
Date: Mon, 15 Jun 2009 15:49:11 -0500
Subject: [Python-Dev] Is the py3k mercurial mirror broken?
In-Reply-To: <79990c6b0906151346w7e104b2el59280d4dc529c391@mail.gmail.com>
References: <79990c6b0906151346w7e104b2el59280d4dc529c391@mail.gmail.com>
Message-ID: <1afaf6160906151349t5328bd67lcf25bee88a4eeb99@mail.gmail.com>

There seem to be some issues with Subversion, so that's probably affecting hg.

2009/6/15 Paul Moore <p.f.moore at gmail.com>:
> I get the following:
>
>>hg version
> Mercurial Distributed SCM (version 1.2)
>
> Copyright (C) 2005-2008 Matt Mackall <mpm at selenic.com> and others
> This is free software; see the source for copying conditions. There is NO
> warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
>
>>hg clone http://code.python.org/hg/branches/py3k python-3k
> requesting all changes
> adding changesets
> transaction abort!
> rollback completed
> ** unknown exception encountered, details follow
> ** report bug details to http://www.selenic.com/mercurial/bts
> ** or mercurial at selenic.com
> ** Mercurial Distributed SCM (version 1.2)
> ** Extensions loaded:
> Traceback (most recent call last):
> ?File "hg", line 27, in <module>
> ?File "mercurial\dispatch.pyc", line 16, in run
> ?File "mercurial\dispatch.pyc", line 25, in dispatch
> ?File "mercurial\dispatch.pyc", line 41, in _runcatch
> ?File "mercurial\dispatch.pyc", line 372, in _dispatch
> ?File "mercurial\dispatch.pyc", line 247, in runcommand
> ?File "mercurial\dispatch.pyc", line 417, in _runcommand
> ?File "mercurial\dispatch.pyc", line 377, in checkargs
> ?File "mercurial\dispatch.pyc", line 371, in <lambda>
> ?File "mercurial\util.pyc", line 718, in check
> ?File "mercurial\commands.pyc", line 603, in clone
> ?File "mercurial\hg.pyc", line 215, in clone
> ?File "mercurial\localrepo.pyc", line 2149, in clone
> ?File "mercurial\localrepo.pyc", line 1492, in pull
> ?File "mercurial\localrepo.pyc", line 2016, in addchangegroup
> ?File "mercurial\revlog.pyc", line 1192, in addgroup
> ?File "mercurial\changegroup.pyc", line 31, in chunkiter
> ?File "mercurial\changegroup.pyc", line 15, in getchunk
> ?File "mercurial\util.pyc", line 1674, in read
> ?File "mercurial\httprepo.pyc", line 18, in zgenerator
> zlib.error: Error -3 while decompressing: incorrect header check
>
> Paul.
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/benjamin%40python.org
>



-- 
Regards,
Benjamin

From ncoghlan at gmail.com  Mon Jun 15 23:31:06 2009
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Tue, 16 Jun 2009 07:31:06 +1000
Subject: [Python-Dev] Exception for setting attributes of built-in type
In-Reply-To: <1afaf6160906151209h10138643ob957e1235176ba45@mail.gmail.com>
References: <5b0248170906141209y4398defanacfd0bb62504e550@mail.gmail.com>	
	<4A357CF0.2050604@mrabarnett.plus.com>	
	<ca471dc20906141619i6c5d1679x7d99616c2b1fc18e@mail.gmail.com>	
	<4A36301C.2050100@gmail.com>
	<1afaf6160906151209h10138643ob957e1235176ba45@mail.gmail.com>
Message-ID: <4A36BD9A.3010301@gmail.com>

Benjamin Peterson wrote:
> 2009/6/15 Nick Coghlan <ncoghlan at gmail.com>:
>> Guido van Rossum wrote:
>>> In general, CPython isn't always consistent in raising AttributeError
>>> and TypeError when it comes to such policy issues: there are various
>>> places that raise TypeError in typeobject.c (and probably elsewhere)
>>> that simply forbid setting a specific attribute (another example is
>>> __name__).
>> We're pretty inconsistent when it comes to looking up special methods as
>> well - those that are looked up through dedicated slots in abstract.c
>> usually raise TypeError, while those that are looked up via a PyType
>> method usually raise AttributeError.
> 
> What's a PyType method?

I was misremembering - for some reason I thought:
a) _PyObject_LookupSpecial was a PyType method
b) That it raised AttributeError itself instead of letting the caller
decide what error to raise

It's still the case that (e.g.) special_lookup() in ceval.c raises an
AttributeError, as do special method lookups from Python of the form
"type(obj).__method__(obj)".

Whether CPython raises TypeError or AttributeError when it comes to
special methods is really pretty arbitrary rather than a matter of any
grand design.

Cheers,
Nick.

P.S. If anyone feels like digging into the archives, the last time I can
recall this topic coming up is when I was concerned about the original
with statement implementation raising AttributeError rather than
TypeError when it couldn't find an __enter__ or __exit__ method. Guido
chimed in then (as now) to say that either exception was fine.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------

From ncoghlan at gmail.com  Mon Jun 15 23:44:32 2009
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Tue, 16 Jun 2009 07:44:32 +1000
Subject: [Python-Dev] Iterator version of contextlib.nested
In-Reply-To: <79990c6b0906151341k2224a02eu202ad7bac6f90774@mail.gmail.com>
References: <4A3271BA.2070904@gmx.net>
	<4A32FA7D.1060906@gmail.com>	<4A339F84.2070400@gmx.net>
	<4A3450DD.6040905@gmail.com>	<4A34B254.6050907@gmx.net>	<022349E0DE2F46F3B68B0050A7608EF4@RaymondLaptop1>	<1afaf6160906141958s3bb76c11o7c60c3a2ceb98e9a@mail.gmail.com>	<ACD6D03AF454484094BF805AAC4F9783@RaymondLaptop1>	<h16bbr$fs6$2@ger.gmane.org>
	<79990c6b0906151341k2224a02eu202ad7bac6f90774@mail.gmail.com>
Message-ID: <4A36C0C0.4090704@gmail.com>

Paul Moore wrote:
> 2009/6/15 Terry Reedy <tjreedy at udel.edu>:
>> Raymond Hettinger wrote:
>>
>>> P.S.  If you switch to PendingDeprecationWarning, the example
>>> in the docs should probably be switched to show the one valid
>>> use case (passing in a prepackaged nest of context managers).
>> It could even suggest that it only be used for this, since it may disappear,
>> and that other uses should use the new syntax.  That would give people the
>> best chance of writing future-proof code when they can.
> 
> And if the warning is *not* changed to a PendingDeprecationWarning,
> the documentation could also note the necessary warnings call needed
> to let the example code run warning-free.

I think I like that option even better than downgrading the warning.

I created http://bugs.python.org/issue6288 to make it clear to Benjamin
when I have finished updating the docs.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------

From greg.ewing at canterbury.ac.nz  Tue Jun 16 01:21:03 2009
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Tue, 16 Jun 2009 11:21:03 +1200
Subject: [Python-Dev] io.BufferedReader.peek() Behaviour in python3.1
In-Reply-To: <20090615010046.GA30997@cskk.homeip.net>
References: <20090615010046.GA30997@cskk.homeip.net>
Message-ID: <4A36D75F.5070805@canterbury.ac.nz>

Cameron Simpson wrote:

> It seems like whenever I want to do some kind of opportunistic but
> non-blocking stuff with a remote service

Do you actually do this with buffered streams? I find
it's better to steer well clear of buffered I/O objects
when doing non-blocking stuff, because they don't play
well with other things like select().

Anyhow, I wouldn't be opposed to having a way of looking
into the buffer, but it should be a separate API  --
preferably with a better name than peek0(), which gives
no clue at all about what it does differently from peek().

-- 
Greg

From cs at zip.com.au  Tue Jun 16 02:43:27 2009
From: cs at zip.com.au (Cameron Simpson)
Date: Tue, 16 Jun 2009 10:43:27 +1000
Subject: [Python-Dev] io.BufferedReader.peek() Behaviour in python3.1
In-Reply-To: <4A36D75F.5070805@canterbury.ac.nz>
Message-ID: <20090616004327.GA15853@cskk.homeip.net>

On 16Jun2009 11:21, Greg Ewing <greg.ewing at canterbury.ac.nz> wrote:
> Cameron Simpson wrote:
>> It seems like whenever I want to do some kind of opportunistic but
>> non-blocking stuff with a remote service
>
> Do you actually do this with buffered streams?

Sure, in C, python and perl quite happily. In some circumstances.
Provided you're careful about when to fflush() it can all go quite
smoothly. It's certainly not applicable to everything.

> I find
> it's better to steer well clear of buffered I/O objects
> when doing non-blocking stuff, because they don't play
> well with other things like select().

Ah, the non-blockingness. Well that's the rub. I normally avoid
non-blocking requirements by using threads, so that the thread gathering
from the stream can block. Then the consumer can poll a Queue from the
worker thread, etc.

I really don't like select(/poll/epoll etc) much; aside from select's
scaling issues with lots of files (hence poll/epoll) there are high
performance things where having an event loop using select is a good
way to go, but I generally prefer using threads and/or generators
to make the code clear (single flow of control, single task for the
block of code, etc) if there's no reason not to.

> Anyhow, I wouldn't be opposed to having a way of looking
> into the buffer, but it should be a separate API  --
> preferably with a better name than peek0(), which gives
> no clue at all about what it does differently from peek().

Indeed, though arguably read1() is a lousy name too, on the same basis.
My itch is that peek() _feels_ like it should be "look into the buffer"
but actually can block and/or change the buffer.

Cheers,
-- 
Cameron Simpson <cs at zip.com.au> DoD#743
http://www.cskk.ezoshosting.com/cs/

You can't wait for inspiration.  You have to go after it with a club.
        - Jack London

From python at mrabarnett.plus.com  Tue Jun 16 03:18:22 2009
From: python at mrabarnett.plus.com (MRAB)
Date: Tue, 16 Jun 2009 02:18:22 +0100
Subject: [Python-Dev] io.BufferedReader.peek() Behaviour in python3.1
In-Reply-To: <20090616004327.GA15853@cskk.homeip.net>
References: <20090616004327.GA15853@cskk.homeip.net>
Message-ID: <4A36F2DE.3010007@mrabarnett.plus.com>

Cameron Simpson wrote:
> On 16Jun2009 11:21, Greg Ewing <greg.ewing at canterbury.ac.nz> wrote:
>> Cameron Simpson wrote:
>>> It seems like whenever I want to do some kind of opportunistic but
>>> non-blocking stuff with a remote service
>> Do you actually do this with buffered streams?
> 
> Sure, in C, python and perl quite happily. In some circumstances.
> Provided you're careful about when to fflush() it can all go quite
> smoothly. It's certainly not applicable to everything.
> 
>> I find
>> it's better to steer well clear of buffered I/O objects
>> when doing non-blocking stuff, because they don't play
>> well with other things like select().
> 
> Ah, the non-blockingness. Well that's the rub. I normally avoid
> non-blocking requirements by using threads, so that the thread gathering
> from the stream can block. Then the consumer can poll a Queue from the
> worker thread, etc.
> 
> I really don't like select(/poll/epoll etc) much; aside from select's
> scaling issues with lots of files (hence poll/epoll) there are high
> performance things where having an event loop using select is a good
> way to go, but I generally prefer using threads and/or generators
> to make the code clear (single flow of control, single task for the
> block of code, etc) if there's no reason not to.
> 
>> Anyhow, I wouldn't be opposed to having a way of looking
>> into the buffer, but it should be a separate API  --
>> preferably with a better name than peek0(), which gives
>> no clue at all about what it does differently from peek().
> 
> Indeed, though arguably read1() is a lousy name too, on the same basis.
> My itch is that peek() _feels_ like it should be "look into the buffer"
> but actually can block and/or change the buffer.
> 
Can block, but not if you don't want it too. You might just want to see
what, if anything, is currently available, up to n bytes.

From cs at zip.com.au  Tue Jun 16 04:03:11 2009
From: cs at zip.com.au (Cameron Simpson)
Date: Tue, 16 Jun 2009 12:03:11 +1000
Subject: [Python-Dev] io.BufferedReader.peek() Behaviour in python3.1
In-Reply-To: <4A36F2DE.3010007@mrabarnett.plus.com>
Message-ID: <20090616020311.GA31250@cskk.homeip.net>

On 16Jun2009 02:18, MRAB <python at mrabarnett.plus.com> wrote:
>> My itch is that peek() _feels_ like it should be "look into the buffer"
>> but actually can block and/or change the buffer.
>>
> Can block, but not if you don't want it too. You might just want to see
> what, if anything, is currently available, up to n bytes.

Am I missing something?

In the face of an _empty_ buffer (which I can't tell from outside) how
do I prevent peek() blocking? More generally, if I go peek(n) and if n >
bytes_in_buffer_right_now and the raw stream would block if a raw read
is done?

My concerns would go away if I could probe the buffer content size;
then I could ensure peek(n) chose n <= the content size. If that's not
enough, my problem - I can choose to read-and-block or go away and come
back later.
-- 
Cameron Simpson <cs at zip.com.au> DoD#743
http://www.cskk.ezoshosting.com/cs/

If all around you is darkness and you feel you're contending in vain,
then the light at the end of the tunnel is the front of an oncoming train.

From cs at zip.com.au  Tue Jun 16 05:20:53 2009
From: cs at zip.com.au (Cameron Simpson)
Date: Tue, 16 Jun 2009 13:20:53 +1000
Subject: [Python-Dev] Avoiding file descriptors leakage in
	subprocess.Popen()
In-Reply-To: <20090614.164255.343165597.mrs@localhost.localdomain>
Message-ID: <20090616032053.GA15938@cskk.homeip.net>

On 14Jun2009 16:42, Mark Seaborn <mrs at mythic-beasts.com> wrote:
| I use a convenience function like this, so that GC takes care of the FDs:
| 
| def make_pipe():
|     read_fd, write_fd = os.pipe()
|     return os.fdopen(read_fd, "r"), os.fdopen(write_fd, "w")

Not guarrenteed to be timely. The try/except at least closes things as
control passes out of the relevant scope. I don't think all pythons
do immediate ref-counted GC.

But it's very neat!
-- 
Cameron Simpson <cs at zip.com.au> DoD#743
http://www.cskk.ezoshosting.com/cs/

Trust the computer industry to shorten Year 2000 to Y2K. It was this
thinking that caused the problem in the first place.
- Mark Ovens <marko at uk.radan.com>

From python at mrabarnett.plus.com  Tue Jun 16 13:14:45 2009
From: python at mrabarnett.plus.com (MRAB)
Date: Tue, 16 Jun 2009 12:14:45 +0100
Subject: [Python-Dev] io.BufferedReader.peek() Behaviour in python3.1
In-Reply-To: <20090616020311.GA31250@cskk.homeip.net>
References: <20090616020311.GA31250@cskk.homeip.net>
Message-ID: <4A377EA5.4090804@mrabarnett.plus.com>

Cameron Simpson wrote:
> On 16Jun2009 02:18, MRAB <python at mrabarnett.plus.com> wrote:
>>> My itch is that peek() _feels_ like it should be "look into the buffer"
>>> but actually can block and/or change the buffer.
>>>
>> Can block, but not if you don't want it too. You might just want to see
>> what, if anything, is currently available, up to n bytes.
> 
> Am I missing something?
> 
> In the face of an _empty_ buffer (which I can't tell from outside) how
> do I prevent peek() blocking? More generally, if I go peek(n) and if n >
> bytes_in_buffer_right_now and the raw stream would block if a raw read
> is done?
> 
> My concerns would go away if I could probe the buffer content size;
> then I could ensure peek(n) chose n <= the content size. If that's not
> enough, my problem - I can choose to read-and-block or go away and come
> back later.

I was thinking along the lines of:

     def peek(self, size=None, block=True)

If 'block' is True then return 'size' bytes, unless the end of the
file/stream is reached; if 'block' is False then return up to 'size'
bytes, without blocking. The blocking form might impose a limit to how 
much can be peeked (the maximum size of the buffer), or it might enlarge
the buffer as necessary.

From steve at pearwood.info  Tue Jun 16 13:19:38 2009
From: steve at pearwood.info (Steven D'Aprano)
Date: Tue, 16 Jun 2009 21:19:38 +1000
Subject: [Python-Dev] Avoiding file descriptors leakage in
	subprocess.Popen()
In-Reply-To: <20090616032053.GA15938@cskk.homeip.net>
References: <20090616032053.GA15938@cskk.homeip.net>
Message-ID: <200906162119.39786.steve@pearwood.info>

On Tue, 16 Jun 2009 01:20:53 pm Cameron Simpson wrote:

> I don't think all
> pythons do immediate ref-counted GC.

Jython and IronPython don't. I don't know about PyPy, CLPython, Unladen 
Swallow, etc.


-- 
Steven D'Aprano

From fuzzyman at voidspace.org.uk  Tue Jun 16 13:21:29 2009
From: fuzzyman at voidspace.org.uk (Michael Foord)
Date: Tue, 16 Jun 2009 12:21:29 +0100
Subject: [Python-Dev] Avoiding file descriptors leakage
	in	subprocess.Popen()
In-Reply-To: <200906162119.39786.steve@pearwood.info>
References: <20090616032053.GA15938@cskk.homeip.net>
	<200906162119.39786.steve@pearwood.info>
Message-ID: <4A378039.5050904@voidspace.org.uk>

Steven D'Aprano wrote:
> On Tue, 16 Jun 2009 01:20:53 pm Cameron Simpson wrote:
>
>   
>> I don't think all
>> pythons do immediate ref-counted GC.
>>     
>
> Jython and IronPython don't. I don't know about PyPy, CLPython, Unladen 
> Swallow, etc.
>
>
>   
PyPy doesn't, Unladen Swallow won't.

Michael

-- 
http://www.ironpythoninaction.com/


From lukepadawan at gmail.com  Tue Jun 16 15:10:40 2009
From: lukepadawan at gmail.com (Lucas P Melo)
Date: Tue, 16 Jun 2009 10:10:40 -0300
Subject: [Python-Dev] io.BufferedReader.peek() Behaviour in python3.1
In-Reply-To: <20090616004327.GA15853@cskk.homeip.net>
References: <20090616004327.GA15853@cskk.homeip.net>
Message-ID: <4A3799D0.6010001@gmail.com>

Cameron Simpson wrote:
> Indeed, though arguably read1() is a lousy name too, on the same basis.
> My itch is that peek() _feels_ like it should be "look into the buffer"
> but actually can block and/or change the buffer.
>   
I guess all the buffer operations should be transparent to the user if 
he wants it to be like that, since not so many people want to have a 
tight control over this kind of detail.
I think of peek() as an operation that allows me to peek what's going to 
show up in the future without affecting further read()s. This kind of 
behavior is expected by users without prior knowledge of the inner 
workings of buffered IO.

So, if an user _really_ wants to take a look at what's to come without 
affecting the buffer, we could allow that by doing something like this:
peek(5, change_buffer=False)
This is an alternative to the peek0(). But I am ok wih the peek0() too.

From lukepadawan at gmail.com  Tue Jun 16 15:14:08 2009
From: lukepadawan at gmail.com (Lucas P Melo)
Date: Tue, 16 Jun 2009 10:14:08 -0300
Subject: [Python-Dev] io.BufferedReader.peek() Behaviour in python3.1
In-Reply-To: <4A377EA5.4090804@mrabarnett.plus.com>
References: <20090616020311.GA31250@cskk.homeip.net>
	<4A377EA5.4090804@mrabarnett.plus.com>
Message-ID: <4A379AA0.7030409@gmail.com>

MRAB wrote:
> I was thinking along the lines of:
>
>     def peek(self, size=None, block=True)
I think this is fine too. :)

>
> If 'block' is True then return 'size' bytes, unless the end of the
> file/stream is reached; if 'block' is False then return up to 'size'
> bytes, without blocking. The blocking form might impose a limit to how 
> much can be peeked (the maximum size of the buffer), or it might enlarge
> the buffer as necessary.
I guess the limit wouldn't be a problem to someone that chose to block 
further reads.


From metawilm at gmail.com  Tue Jun 16 16:19:09 2009
From: metawilm at gmail.com (Willem Broekema)
Date: Tue, 16 Jun 2009 16:19:09 +0200
Subject: [Python-Dev] Avoiding file descriptors leakage in
	subprocess.Popen()
In-Reply-To: <4A378039.5050904@voidspace.org.uk>
References: <20090616032053.GA15938@cskk.homeip.net>
	<200906162119.39786.steve@pearwood.info>
	<4A378039.5050904@voidspace.org.uk>
Message-ID: <f6bc9b490906160719u64a732c9v563e4c98e85a6d92@mail.gmail.com>

On Tue, Jun 16, 2009 at 1:21 PM, Michael Foord<fuzzyman at voidspace.org.uk> wrote:
> Steven D'Aprano wrote:
>> On Tue, 16 Jun 2009 01:20:53 pm Cameron Simpson wrote:
>>> I don't think all pythons do immediate ref-counted GC.
>>
>> Jython and IronPython don't. I don't know about PyPy, CLPython, Unladen
>> Swallow, etc.
>
> PyPy doesn't, Unladen Swallow won't.

Also most Lisp implementations, thus CLPython, use a tracing GC.

- Willem

From devin.c.cook at gmail.com  Tue Jun 16 16:09:11 2009
From: devin.c.cook at gmail.com (Devin Cook)
Date: Tue, 16 Jun 2009 10:09:11 -0400
Subject: [Python-Dev] SSL Certificate Validation
Message-ID: <8847f980906160709q405ad950qac0c56b304562d3@mail.gmail.com>

Hi all,

I have a few questions about validating SSL certificates. From what I
gather, this validation occurs in the OpenSSL code called from _ssl.c. Is
this correct?

Also, I have looked through the docs and code, but haven't been able to
figure out exactly what is included in certificate "validation". Is it just
validating the chain? Does it check the NotBefore and NotAfter dates? Does
it check that the host the socket is connected to is the same as what's
given in the CN field in the certificate?

Where I'm going with this is I think all this checking needs to be part of
certificate validation in the ssl module. If it isn't yet, I'd be happy to
work on a patch for it. Please let me know what you think.

Thanks!
-Devin Cook
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090616/761bb87f/attachment.htm>

From mrs at mythic-beasts.com  Tue Jun 16 20:04:44 2009
From: mrs at mythic-beasts.com (Mark Seaborn)
Date: Tue, 16 Jun 2009 19:04:44 +0100 (BST)
Subject: [Python-Dev] Avoiding file descriptors leakage in
	subprocess.Popen()
In-Reply-To: <20090616032053.GA15938@cskk.homeip.net>
References: <20090614.164255.343165597.mrs@localhost.localdomain>
	<20090616032053.GA15938@cskk.homeip.net>
Message-ID: <20090616.190444.343167081.mrs@localhost.localdomain>

Cameron Simpson <cs at zip.com.au> wrote:

> On 14Jun2009 16:42, Mark Seaborn <mrs at mythic-beasts.com> wrote:
> | I use a convenience function like this, so that GC takes care of the FDs:
> | 
> | def make_pipe():
> |     read_fd, write_fd = os.pipe()
> |     return os.fdopen(read_fd, "r"), os.fdopen(write_fd, "w")
> 
> Not guarrenteed to be timely. The try/except at least closes things as
> control passes out of the relevant scope. I don't think all pythons
> do immediate ref-counted GC.

Yep.  I don't mind closing FDs explicitly when it's easy to do so in a
try..finally, but it's not always simple.

There are two different problems with non-prompt closing of FDs:

 * Whether an FD has been closed is sometimes externally observable.
   e.g. Pipes and sockets notify the other end of the connection.
   Open file and directory FDs prevent filesystems from being
   unmounted.
 * FDs use up space in the process's FD table.

The second problem could be dealt with by running the GC when we get
EMFILE, or before any calls that allocate FDs when the FD table is
almost full, just as the GC runs when we "run out" of memory.

I wonder if this proposal could help:
'GC & The "Expensive Object" Problem'
http://www.eros-os.org/pipermail/e-lang/1999-May/002590.html

Mark

From Scott.Daniels at Acm.Org  Tue Jun 16 20:20:48 2009
From: Scott.Daniels at Acm.Org (Scott David Daniels)
Date: Tue, 16 Jun 2009 11:20:48 -0700
Subject: [Python-Dev] io.BufferedReader.peek() Behaviour in python3.1
In-Reply-To: <4A377EA5.4090804@mrabarnett.plus.com>
References: <20090616020311.GA31250@cskk.homeip.net>
	<4A377EA5.4090804@mrabarnett.plus.com>
Message-ID: <h18ndq$l3d$1@ger.gmane.org>

MRAB wrote:
> I was thinking along the lines of:
>     def peek(self, size=None, block=True)
> If 'block' is True then return 'size' bytes, unless the end of the
> file/stream is reached; if 'block' is False then return up to 'size'
> bytes, without blocking....

I tend to prefer zero-ish defaults, how about:
     def peek(self, size=None, nonblocking=False)

We still don't have "at most one read" code, but something a bit like

     data = obj.peek(size=desired, nonblocking=True)
     if len(data) < desired:
         data = obj.peek(size=wanted, nonblocking=False)

might suffice.

--Scott David Daniels
Scott.Daniels at Acm.Org



From v+python at g.nevcal.com  Tue Jun 16 21:08:04 2009
From: v+python at g.nevcal.com (Glenn Linderman)
Date: Tue, 16 Jun 2009 12:08:04 -0700
Subject: [Python-Dev] io.BufferedReader.peek() Behaviour in python3.1
In-Reply-To: <h18ndq$l3d$1@ger.gmane.org>
References: <20090616020311.GA31250@cskk.homeip.net>	<4A377EA5.4090804@mrabarnett.plus.com>
	<h18ndq$l3d$1@ger.gmane.org>
Message-ID: <4A37ED94.4060403@g.nevcal.com>

On approximately 6/16/2009 11:20 AM, came the following characters from 
the keyboard of Scott David Daniels:
> MRAB wrote:
>> I was thinking along the lines of:
>>     def peek(self, size=None, block=True)
>> If 'block' is True then return 'size' bytes, unless the end of the
>> file/stream is reached; if 'block' is False then return up to 'size'
>> bytes, without blocking....
> 
> I tend to prefer zero-ish defaults, how about:
>     def peek(self, size=None, nonblocking=False)


No, no, no!  Double negatives are extremely easy to not code correctly. 
  The lack of ease of not understanding of code containing double 
negatives quadruples, at least!  Not so differently, I'm sure my 
sentences here are not easy to understand because I put the inverse 
logic in them in the places that are not the usual.

-- 
Glenn -- http://nevcal.com/
===========================
A protocol is complete when there is nothing left to remove.
-- Stuart Cheshire, Apple Computer, regarding Zero Configuration Networking

From solipsis at pitrou.net  Tue Jun 16 21:17:09 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Tue, 16 Jun 2009 19:17:09 +0000 (UTC)
Subject: [Python-Dev] io.BufferedReader.peek() Behaviour in python3.1
References: <20090616020311.GA31250@cskk.homeip.net>
	<4A377EA5.4090804@mrabarnett.plus.com> <h18ndq$l3d$1@ger.gmane.org>
Message-ID: <loom.20090616T191404-734@post.gmane.org>

Scott David Daniels <Scott.Daniels <at> Acm.Org> writes:
> 
> MRAB wrote:
> > I was thinking along the lines of:
> >     def peek(self, size=None, block=True)
> > If 'block' is True then return 'size' bytes, unless the end of the
> > file/stream is reached; if 'block' is False then return up to 'size'
> > bytes, without blocking....
> 
> I tend to prefer zero-ish defaults, how about:
>      def peek(self, size=None, nonblocking=False)

Since blocking and non-blocking are already used to refer to different types of
raw streams, another verb should be found for this option.


Antoine.



From martin at v.loewis.de  Tue Jun 16 21:23:18 2009
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Tue, 16 Jun 2009 21:23:18 +0200
Subject: [Python-Dev] SSL Certificate Validation
In-Reply-To: <8847f980906160709q405ad950qac0c56b304562d3@mail.gmail.com>
References: <8847f980906160709q405ad950qac0c56b304562d3@mail.gmail.com>
Message-ID: <4A37F126.5010008@v.loewis.de>

> I have a few questions about validating SSL certificates. From what I
> gather, this validation occurs in the OpenSSL code called from _ssl.c.
> Is this correct?

This question is really off-topic for python-dev. As a python-dev
poster, you should do research upfront, and only post on what you
consider facts.

> Where I'm going with this is I think all this checking needs to be part
> of certificate validation in the ssl module. If it isn't yet, I'd be
> happy to work on a patch for it. Please let me know what you think.

I think you need to familiarize yourself much more with OpenSSL.

Regards,
Martin

From jnoller at gmail.com  Tue Jun 16 21:42:59 2009
From: jnoller at gmail.com (Jesse Noller)
Date: Tue, 16 Jun 2009 15:42:59 -0400
Subject: [Python-Dev] SSL Certificate Validation
In-Reply-To: <4A37F126.5010008@v.loewis.de>
References: <8847f980906160709q405ad950qac0c56b304562d3@mail.gmail.com>
	<4A37F126.5010008@v.loewis.de>
Message-ID: <4222a8490906161242p305e2268q550cf48d1d82f884@mail.gmail.com>

On Tue, Jun 16, 2009 at 3:23 PM, "Martin v. L?wis"<martin at v.loewis.de> wrote:
>> I have a few questions about validating SSL certificates. From what I
>> gather, this validation occurs in the OpenSSL code called from _ssl.c.
>> Is this correct?
>
> This question is really off-topic for python-dev. As a python-dev
> poster, you should do research upfront, and only post on what you
> consider facts.

Martin, I told him to ask his question about _ssl internals on
python-dev as he is new, and looking to work on some of the
internals/make a patch for core. I didn't think that asking internals
questions was a faux pas for the list, especially as he's looking to
submit a patch to core.

>> Where I'm going with this is I think all this checking needs to be part
>> of certificate validation in the ssl module. If it isn't yet, I'd be
>> happy to work on a patch for it. Please let me know what you think.
>
> I think you need to familiarize yourself much more with OpenSSL.

I don't think that's called for, he is attempting to familiarize
himself and simply inquiring about some of the internals. I'm sure
he'll know plenty by the time the patch is more fully formed.

-jesse

From martin at v.loewis.de  Tue Jun 16 22:14:35 2009
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Tue, 16 Jun 2009 22:14:35 +0200
Subject: [Python-Dev] SSL Certificate Validation
In-Reply-To: <4222a8490906161242p305e2268q550cf48d1d82f884@mail.gmail.com>
References: <8847f980906160709q405ad950qac0c56b304562d3@mail.gmail.com>	
	<4A37F126.5010008@v.loewis.de>
	<4222a8490906161242p305e2268q550cf48d1d82f884@mail.gmail.com>
Message-ID: <4A37FD2B.1000904@v.loewis.de>

>> This question is really off-topic for python-dev. As a python-dev
>> poster, you should do research upfront, and only post on what you
>> consider facts.
> 
> Martin, I told him to ask his question about _ssl internals on
> python-dev as he is new, and looking to work on some of the
> internals/make a patch for core. I didn't think that asking internals
> questions was a faux pas for the list, especially as he's looking to
> submit a patch to core.

Hmm. For somebody new to Python, I'm fairly skeptical that the SSL
module is the best starting point.

>>> Where I'm going with this is I think all this checking needs to be part
>>> of certificate validation in the ssl module. If it isn't yet, I'd be
>>> happy to work on a patch for it. Please let me know what you think.
>> I think you need to familiarize yourself much more with OpenSSL.
> 
> I don't think that's called for, he is attempting to familiarize
> himself and simply inquiring about some of the internals. I'm sure
> he'll know plenty by the time the patch is more fully formed.

But I really do believe that this is what he need to do next:
familiarize himself with OpenSSL. There is a lot of APIs in that
library, and it takes a while (i.e.: several months) to get
productive, in particular since OpenSSL doesn't have the most
intuitive API.

>From "I want to know what features it currently has" to "I can
contribute new features" is really a looong way here.

To give a little more guidance: find out what
SSL_CTX_use_certificate_chain_file and SSL_CTX_set_verify do.
Finding that out is really out of scope of python-dev, since
it has nothing to do with Python.

Regards,
Martin

From devin.c.cook at gmail.com  Tue Jun 16 23:31:23 2009
From: devin.c.cook at gmail.com (Devin Cook)
Date: Tue, 16 Jun 2009 17:31:23 -0400
Subject: [Python-Dev] SSL Certificate Validation
In-Reply-To: <4A37FD2B.1000904@v.loewis.de>
References: <8847f980906160709q405ad950qac0c56b304562d3@mail.gmail.com>
	<4A37F126.5010008@v.loewis.de>
	<4222a8490906161242p305e2268q550cf48d1d82f884@mail.gmail.com>
	<4A37FD2B.1000904@v.loewis.de>
Message-ID: <8847f980906161431n2f9043afk27a43194ac3721f5@mail.gmail.com>

> But I really do believe that this is what he need to do next:
> familiarize himself with OpenSSL. There is a lot of APIs in that
> library, and it takes a while (i.e.: several months) to get
> productive, in particular since OpenSSL doesn't have the most
> intuitive API.

Well, I realized this as soon as I looked at the _ssl.c code... I was
just hoping?that someone would be able to give me a quick
clarification on exactly what gets validated. If it's just the chain
(which is what I suspect), I would like to submit a patch that does
the rest of the validation (that a browser typically does:
CN/hostname, NotBefore, NotAfter, etc.) in the ssl module. I was also
hoping to find out what the consensus is about this: mainly, *should*
that verification be done in the ssl module? Maybe this verification
should somehow be done in OpenSSL, which would mean that I need to do
a LOT more reading and go pester their mailing list instead.

This is for issue 6273 ( http://bugs.python.org/issue6273 ). In your
reply to that issue, it seemed to me like you were saying that these
things were not getting checked in the ssl module (and, therefore, not
in OpenSSL either):

> I find the patch incomplete, for formal and semantical reasons:
> a) it doesn't come with documentation or test suite changes, and
> b) it doesn't implement the typical certificate checks that browsers
>    do, beyond validating that the certificate is valid - e.g. also
>    validating that the certificate is issued to the host you are trying
>    to connect to.

I would like to do validation of server certificates in a project I'm
working on, and I figured it would be better to be proactive and try
to help create a patch than to just sit back and complain about it. It
seems to me that this is a bug that you can't do peer certificate
validation in httplib.

If this isn't the place to ask these kinds of questions, I apologise.
I can take the discussion elsewhere if I need to.

Thanks,
-Devin

From janssen at parc.com  Tue Jun 16 23:45:14 2009
From: janssen at parc.com (Bill Janssen)
Date: Tue, 16 Jun 2009 14:45:14 PDT
Subject: [Python-Dev] SSL Certificate Validation
In-Reply-To: <8847f980906160709q405ad950qac0c56b304562d3@mail.gmail.com>
References: <8847f980906160709q405ad950qac0c56b304562d3@mail.gmail.com>
Message-ID: <26228.1245188714@parc.com>

Devin Cook <devin.c.cook at gmail.com> wrote:

> Also, I have looked through the docs and code, but haven't been able to
> figure out exactly what is included in certificate "validation". Is it just
> validating the chain? Does it check the NotBefore and NotAfter dates?

I believe so, but you'll have to check the OpenSSL code.

> Does it check that the host the socket is connected to is the same as
> what's given in the CN field in the certificate?

No.  That, in general, doesn't work very well.  The IETF working group
on this is considering deprecating putting a hostname in the CN field at
all, and just adding hostnames via the subjectAltName extension.  The
problem that's come up is that many computers don't have fixed IP
addresses, and even with that the hostname is part of a different
mapping of hostnames to IP addresses, which can also vary.

I think that when the https: protocol scheme was written, it seemed like
a good idea, but conventions on the Internet have changed a lot since
then.

> Where I'm going with this is I think all this checking needs to be part of
> certificate validation in the ssl module.

I don't think so.  I put in hooks to let you do this in user code if you
need to.  See the archives for more discussion on this -- I'm not going to
rehash it again.

(This is really a question for OpenSSL mailing lists, or perhaps python-list.)

Bill

From jnoller at gmail.com  Wed Jun 17 00:09:58 2009
From: jnoller at gmail.com (Jesse Noller)
Date: Tue, 16 Jun 2009 18:09:58 -0400
Subject: [Python-Dev] SSL Certificate Validation
In-Reply-To: <8847f980906161431n2f9043afk27a43194ac3721f5@mail.gmail.com>
References: <8847f980906160709q405ad950qac0c56b304562d3@mail.gmail.com>
	<4A37F126.5010008@v.loewis.de>
	<4222a8490906161242p305e2268q550cf48d1d82f884@mail.gmail.com>
	<4A37FD2B.1000904@v.loewis.de>
	<8847f980906161431n2f9043afk27a43194ac3721f5@mail.gmail.com>
Message-ID: <4222a8490906161509x44c843bel9ca8fb14814ce6a8@mail.gmail.com>

On Tue, Jun 16, 2009 at 5:31 PM, Devin Cook<devin.c.cook at gmail.com> wrote:
>> But I really do believe that this is what he need to do next:
>> familiarize himself with OpenSSL. There is a lot of APIs in that
>> library, and it takes a while (i.e.: several months) to get
>> productive, in particular since OpenSSL doesn't have the most
>> intuitive API.
>
> Well, I realized this as soon as I looked at the _ssl.c code... I was
> just hoping?that someone would be able to give me a quick
> clarification on exactly what gets validated. If it's just the chain
> (which is what I suspect), I would like to submit a patch that does
> the rest of the validation (that a browser typically does:
> CN/hostname, NotBefore, NotAfter, etc.) in the ssl module. I was also
> hoping to find out what the consensus is about this: mainly, *should*
> that verification be done in the ssl module? Maybe this verification
> should somehow be done in OpenSSL, which would mean that I need to do
> a LOT more reading and go pester their mailing list instead.
>
> This is for issue 6273 ( http://bugs.python.org/issue6273 ). In your
> reply to that issue, it seemed to me like you were saying that these
> things were not getting checked in the ssl module (and, therefore, not
> in OpenSSL either):
>

Also my initial bug report "client-side cert support" was a big fat
typo on my part.

face-palm'dly yours,
jesse

From martin at v.loewis.de  Wed Jun 17 00:16:29 2009
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Wed, 17 Jun 2009 00:16:29 +0200
Subject: [Python-Dev] SSL Certificate Validation
In-Reply-To: <8847f980906161431n2f9043afk27a43194ac3721f5@mail.gmail.com>
References: <8847f980906160709q405ad950qac0c56b304562d3@mail.gmail.com>	
	<4A37F126.5010008@v.loewis.de>	
	<4222a8490906161242p305e2268q550cf48d1d82f884@mail.gmail.com>	
	<4A37FD2B.1000904@v.loewis.de>
	<8847f980906161431n2f9043afk27a43194ac3721f5@mail.gmail.com>
Message-ID: <4A3819BD.4000301@v.loewis.de>

> If this isn't the place to ask these kinds of questions, I apologise.
> I can take the discussion elsewhere if I need to.

It really depends on what "these questions" are. If your question is
"I have this patch, is it correct?", then the question is entirely
appropriate. If it is "I just have barely looked at the API, can
somebody please explain it all to me?", then this isn't appropriate
for this list, and probably not appropriate elsewhere: anybody answering
this question could just as well fix the original problem right away.

So please do try to find the answer for yourself, with the (little)
direction I gave. If you find that it takes a lot of effort, then you'll
probably have to accept the bug as-is, and live with it.

FWIW, I actually don't know the answer for sure, either, so I would have
to research this myself, too. In any case, _ssl.c is *not* the place
where any of the certificate validation actually happens - nor does it
happen elsewhere in the Python source code, IIUC.

Regards,
Martin

From greg.ewing at canterbury.ac.nz  Wed Jun 17 00:55:26 2009
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Wed, 17 Jun 2009 10:55:26 +1200
Subject: [Python-Dev] io.BufferedReader.peek() Behaviour in python3.1
In-Reply-To: <20090616004327.GA15853@cskk.homeip.net>
References: <20090616004327.GA15853@cskk.homeip.net>
Message-ID: <4A3822DE.2040503@canterbury.ac.nz>

Cameron Simpson wrote:
> I normally avoid
> non-blocking requirements by using threads, so that the thread gathering
> from the stream can block.

If you have a thread dedicated to reading from that
stream, then I don't see why you need to peek into
the buffer. Just have it loop reading a packet at a
time and put each completed packet in the queue.
If several packets arrive at once, it'll loop around
that many times before blocking.

> arguably read1() is a lousy name too, on the same basis.

Certainly.

> My itch is that peek() _feels_ like it should be "look into the buffer"
> but actually can block and/or change the buffer.

My problem with the idea of looking into the buffer
is that it crosses levels of abstraction. A buffered
stream is supposed to behave the same way as a
deterministic non-buffered stream, with the buffer
being an internal optimisation detail that doesn't
exist as far as the outside world is concerned.

Sometimes it's pragmatic to break the abstraction,
but it should be made very obvious when you're doing
that. So I'd prefer something like peek_buffer() to
make it perfectly clear what's being done.

Anything else such as peek() that doesn't explicitly
mention the buffer should fit into the abstraction
properly.

-- 
Greg

From greg.ewing at canterbury.ac.nz  Wed Jun 17 01:04:37 2009
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Wed, 17 Jun 2009 11:04:37 +1200
Subject: [Python-Dev] Functions that steal references (Re: [pygame] [patch]
 minor memory leaks...)
In-Reply-To: <4A36F1EE.9040704@telus.net>
References: <4A36AF6C.2000503@amberfisharts.com> <4A36F1EE.9040704@telus.net>
Message-ID: <4A382505.8010101@canterbury.ac.nz>

Lenard Lindstrom wrote:

> I assumed that since PyModule_AddObject is documented as 
> stealing a reference, it always stole a reference. But in reality it 
> only does so conditionally, when it succeeds.

As an aside, is this a general feature of functions
that steal references, or is PyModule_AddObject an
oddity?

I ask because I've been thinking about adding features
to Pyrex for dealing with stolen references, and it
could be important to know things like this.

Also, if it's an oddity, it would be a good idea
to mention this behaviour in the docs.

-- 
Greg

From benjamin at python.org  Wed Jun 17 01:05:08 2009
From: benjamin at python.org (Benjamin Peterson)
Date: Tue, 16 Jun 2009 18:05:08 -0500
Subject: [Python-Dev] Functions that steal references (Re: [pygame]
	[patch] minor memory leaks...)
In-Reply-To: <4A382505.8010101@canterbury.ac.nz>
References: <4A36AF6C.2000503@amberfisharts.com> <4A36F1EE.9040704@telus.net>
	<4A382505.8010101@canterbury.ac.nz>
Message-ID: <1afaf6160906161605j5e8ec66dgb5ae740abf21de0a@mail.gmail.com>

2009/6/16 Greg Ewing <greg.ewing at canterbury.ac.nz>:
> Lenard Lindstrom wrote:
>
>> I assumed that since PyModule_AddObject is documented as stealing a
>> reference, it always stole a reference. But in reality it only does so
>> conditionally, when it succeeds.
>
> As an aside, is this a general feature of functions
> that steal references, or is PyModule_AddObject an
> oddity?

IIRC, It's an oddity.
>
> I ask because I've been thinking about adding features
> to Pyrex for dealing with stolen references, and it
> could be important to know things like this.
>
> Also, if it's an oddity, it would be a good idea
> to mention this behaviour in the docs.

Yes, indeed.



-- 
Regards,
Benjamin

From lists at cheimes.de  Wed Jun 17 01:24:27 2009
From: lists at cheimes.de (Christian Heimes)
Date: Wed, 17 Jun 2009 01:24:27 +0200
Subject: [Python-Dev] Functions that steal references (Re: [pygame]
 [patch] minor memory leaks...)
In-Reply-To: <1afaf6160906161605j5e8ec66dgb5ae740abf21de0a@mail.gmail.com>
References: <4A36AF6C.2000503@amberfisharts.com>
	<4A36F1EE.9040704@telus.net>	<4A382505.8010101@canterbury.ac.nz>
	<1afaf6160906161605j5e8ec66dgb5ae740abf21de0a@mail.gmail.com>
Message-ID: <h199jd$fkg$1@ger.gmane.org>

Benjamin Peterson schrieb:
> 2009/6/16 Greg Ewing <greg.ewing at canterbury.ac.nz>:
>> Lenard Lindstrom wrote:
>>
>>> I assumed that since PyModule_AddObject is documented as stealing a
>>> reference, it always stole a reference. But in reality it only does so
>>> conditionally, when it succeeds.
>> As an aside, is this a general feature of functions
>> that steal references, or is PyModule_AddObject an
>> oddity?
> 
> IIRC, It's an oddity.

But it is a convenient oddity nonetheless.

Christian


From solipsis at pitrou.net  Wed Jun 17 01:38:49 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Tue, 16 Jun 2009 23:38:49 +0000 (UTC)
Subject: [Python-Dev] io.BufferedReader.peek() Behaviour in python3.1
References: <20090616004327.GA15853@cskk.homeip.net>
	<4A3822DE.2040503@canterbury.ac.nz>
Message-ID: <loom.20090616T233518-558@post.gmane.org>

Greg Ewing <greg.ewing <at> canterbury.ac.nz> writes:
> 
> Anything else such as peek() that doesn't explicitly
> mention the buffer should fit into the abstraction
> properly.

peek() doesn't "fit into the abstraction" since it doesn't even exist on raw
streams.

While buffered and non-buffered streams have a reasonably similar API, expecting
them to behave the same in all circumstances is IMO unrealistic.


Antoine.



From greg.ewing at canterbury.ac.nz  Wed Jun 17 02:25:22 2009
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Wed, 17 Jun 2009 12:25:22 +1200
Subject: [Python-Dev] Functions that steal references (Re: [pygame]
 [patch] minor memory leaks...)
In-Reply-To: <h199jd$fkg$1@ger.gmane.org>
References: <4A36AF6C.2000503@amberfisharts.com> <4A36F1EE.9040704@telus.net>
	<4A382505.8010101@canterbury.ac.nz>
	<1afaf6160906161605j5e8ec66dgb5ae740abf21de0a@mail.gmail.com>
	<h199jd$fkg$1@ger.gmane.org>
Message-ID: <4A3837F2.4070802@canterbury.ac.nz>

Christian Heimes wrote:

> But it is a convenient oddity nonetheless.

What's convenient about it? Seems to me it's the
opposite, since you can't just bail out if it
fails, but have to decref the reference you
thought it was going to take care of for you.

-- 
Greg

From cs at zip.com.au  Wed Jun 17 03:27:19 2009
From: cs at zip.com.au (Cameron Simpson)
Date: Wed, 17 Jun 2009 11:27:19 +1000
Subject: [Python-Dev] io.BufferedReader.peek() Behaviour in python3.1
In-Reply-To: <4A3822DE.2040503@canterbury.ac.nz>
Message-ID: <20090617012719.GA19974@cskk.homeip.net>

On 17Jun2009 10:55, Greg Ewing <greg.ewing at canterbury.ac.nz> wrote:
> Cameron Simpson wrote:
>> I normally avoid
>> non-blocking requirements by using threads, so that the thread gathering
>> from the stream can block.
>
> If you have a thread dedicated to reading from that
> stream, then I don't see why you need to peek into
> the buffer. Just have it loop reading a packet at a
> time and put each completed packet in the queue.
> If several packets arrive at once, it'll loop around
> that many times before blocking.

Yes, this is true. But people not using threads, or at any rate not
dedicating a thread to the reading task, don't have such luxury.

Are we disputing the utility of being able to ask "how much might I
read/peek without blocking"? Or disputing the purpose of peek, which
feels to me like it should/might ask that question, but doesn't.

[...]
>> My itch is that peek() _feels_ like it should be "look into the buffer"
>> but actually can block and/or change the buffer.
>
> My problem with the idea of looking into the buffer
> is that it crosses levels of abstraction. A buffered
> stream is supposed to behave the same way as a
> deterministic non-buffered stream, with the buffer
> being an internal optimisation detail that doesn't
> exist as far as the outside world is concerned.
>
> Sometimes it's pragmatic to break the abstraction,
> but it should be made very obvious when you're doing
> that. So I'd prefer something like peek_buffer() to
> make it perfectly clear what's being done.
>
> Anything else such as peek() that doesn't explicitly
> mention the buffer should fit into the abstraction
> properly.

It's perfectly possible, even reasonable, to avoid talking about the
buffer at all. I'd be happy not to mention the buffer.

For example, one can readily imagine the buffered stream being capable
of querying its input raw stream if there's "available now" data.

The raw stream can sometimes know if a read of a given size would
block, or be able to ask what size read will not block. As a concrete
example, the UNIX FIONREAD ioctl can generally query a file descriptor
for instantly-available  data (== in the OS buffer).  I've also used
UNIXen where your can fstat() a pipe and use the st_size field to test
for available unread data in the pipe buffer. Raw streams which can't
do that would return 0 (== can't guarentee any non-blocking data) unless
the stream itself also had a buffer of its own and it wasn't empty.

So I would _want_ the spec for available_data() (new lousy name) to talk
about "data available without blocking", allowing the implementation to
use data in the IO buffer and/or to query the raw stream, etc.

Cheers,
-- 
Cameron Simpson <cs at zip.com.au> DoD#743
http://www.cskk.ezoshosting.com/cs/

For those who understand, NO explanation is needed,
for those who don't understand, NO explanation will be given!
        - Davey D <decoster at vnet.ibm.com>

From cylix at solace.info  Wed Jun 17 04:21:40 2009
From: cylix at solace.info (Frederick Reeve)
Date: Tue, 16 Jun 2009 21:21:40 -0500
Subject: [Python-Dev] io.BufferedReader.peek() Behaviour in python3.1
In-Reply-To: <loom.20090613T123207-984@post.gmane.org>
References: <20090610203857.00a019fa@cylix> <20090612193707.24f5563c@cylix>
	<loom.20090613T123207-984@post.gmane.org>
Message-ID: <20090616212140.5f291873@cylix>

On Sat, 13 Jun 2009 12:33:46 +0000 (UTC)
Antoine Pitrou <solipsis at pitrou.net> wrote:

> This proposal looks reasonable to me. Please note that it's too late for 3.1
> anyway - we're in release candidate phase. Once you have a patch, you can post
> it on the bug tracker.

Thanks I will do that.  Sometime in the next couple of weeks.

Gratefully

Frederick

From hrvoje.niksic at avl.com  Wed Jun 17 10:16:15 2009
From: hrvoje.niksic at avl.com (Hrvoje Niksic)
Date: Wed, 17 Jun 2009 10:16:15 +0200
Subject: [Python-Dev] Functions that steal references (Re: [pygame]
 [patch] minor memory leaks...)
In-Reply-To: <31501613.234006.1245194699078.JavaMail.xicrypt@atgrzls001>
References: <4A36AF6C.2000503@amberfisharts.com>	<4A36F1EE.9040704@telus.net>	<4A382505.8010101@canterbury.ac.nz>	<1afaf6160906161605j5e8ec66dgb5ae740abf21de0a@mail.gmail.com>
	<31501613.234006.1245194699078.JavaMail.xicrypt@atgrzls001>
Message-ID: <4A38A64F.1080005@avl.com>

Christian Heimes wrote:
>>>> I assumed that since PyModule_AddObject is documented as stealing a
>>>> reference, it always stole a reference. But in reality it only does so
>>>> conditionally, when it succeeds.
>>> As an aside, is this a general feature of functions
>>> that steal references, or is PyModule_AddObject an
>>> oddity?
>> 
>> IIRC, It's an oddity.
> 
> But it is a convenient oddity nonetheless.

Stealing references is sometimes convenient, but Greg was referring to 
functions that steal references *conditionally*, which is indeed an 
oddity.  Most functions and macros that steal references do so 
unconditionally, typically because they can't fail anyway.  Conditional 
stealing of references requires very careful thinking on the side of 
callers that care about not leaking references in the face of 
exceptions.  See http://bugs.python.org/issue1782 for an example.

From zeal_goswami at yahoo.com  Wed Jun 17 12:27:13 2009
From: zeal_goswami at yahoo.com (abhishek goswami)
Date: Wed, 17 Jun 2009 15:57:13 +0530 (IST)
Subject: [Python-Dev] Python script language or not
Message-ID: <350154.15071.qm@web94907.mail.in2.yahoo.com>


Hi,
I have very basic question about Python that do we consider pyhton as script language.
I searched in google but it becomes more confusion for me. After some analysis I came to know that Python support oops .

Can anyone clarify me. Please let me know also it is right forum or not. 

I was looking for Python forum but i did not find any forum. Do we have forum like
linux forum so it will be very easy for me to ask this question in forum and i will not ask
repeated question becuase i am sure this question may be raised before

Abhishek Goswami

Chennai

Phone No -0996227099


      Love Cricket? Check out live scores, photos, video highlights and more. Click here http://cricket.yahoo.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090617/cb69166d/attachment.htm>

From dirkjan at ochtman.nl  Wed Jun 17 12:52:05 2009
From: dirkjan at ochtman.nl (Dirkjan Ochtman)
Date: Wed, 17 Jun 2009 12:52:05 +0200
Subject: [Python-Dev] Python script language or not
In-Reply-To: <350154.15071.qm@web94907.mail.in2.yahoo.com>
References: <350154.15071.qm@web94907.mail.in2.yahoo.com>
Message-ID: <ea2499da0906170352v55870c40s57c9ab9b91f4d7e7@mail.gmail.com>

On Wed, Jun 17, 2009 at 12:27, abhishek goswami<zeal_goswami at yahoo.com> wrote:
> Can anyone clarify me. Please let me know also it is right forum or not.

This is not the right forum. This mailing list is about developing the
CPython interpreter.

For general questions, you may want to try the comp.lang.python newsgroup.

Cheers,

Dirkjan

From tav at espians.com  Wed Jun 17 16:13:29 2009
From: tav at espians.com (tav)
Date: Wed, 17 Jun 2009 15:13:29 +0100
Subject: [Python-Dev] CPython in the web browser under Native Client
In-Reply-To: <20090614.170936.846958184.mrs@localhost.localdomain>
References: <20090614.170936.846958184.mrs@localhost.localdomain>
Message-ID: <eb24b25b0906170713x562d537ax7101bc993854672a@mail.gmail.com>

Hey Mark,

> http://lackingrhoticity.blogspot.com/2009/06/python-standard-library-in-native.html

Really glad to see that you carried on with this -- great work!!

I guess the elders will simply say that it's history repeating itself,
but Grails 2.0 is looking promising! We can finally give that upstart
Javascript a run for its money (despite ES5 looking nice...) -- not to
mention Google Wave/Opera Unite/etc.

Now, the real blocker is accessing the DOM from Python. I haven't
looked into the source code deeply, but perhaps Kroll could be of
help:

* http://github.com/jhaynie/kroll/tree/master

I was able to do the following with Titanium Desktop (which uses it)
to access the DOM from Python:

<script type="text/python">
  def hello(s):
      document.getElementById('foo').innerHTML = s
</script>
<div> Hello <div id="foo">World</div></div>
<script>hello("tav")</script>

More relevant links:

* http://www.appcelerator.com/
* http://github.com/appcelerator/

Anyways, I'm sure others might have a better idea of integrating
Python and the DOM.

Well done again!

-- 
love, tav

plex:espians/tav | tav at espians.com | +44 (0) 7809 569 369
http://tav.espians.com | http://twitter.com/tav | skype:tavespian

From tjreedy at udel.edu  Wed Jun 17 19:26:50 2009
From: tjreedy at udel.edu (Terry Reedy)
Date: Wed, 17 Jun 2009 13:26:50 -0400
Subject: [Python-Dev] Python script language or not
In-Reply-To: <ea2499da0906170352v55870c40s57c9ab9b91f4d7e7@mail.gmail.com>
References: <350154.15071.qm@web94907.mail.in2.yahoo.com>
	<ea2499da0906170352v55870c40s57c9ab9b91f4d7e7@mail.gmail.com>
Message-ID: <h1b90o$et$1@ger.gmane.org>

Dirkjan Ochtman wrote:
> On Wed, Jun 17, 2009 at 12:27, abhishek goswami<zeal_goswami at yahoo.com> wrote:
>> Can anyone clarify me. Please let me know also it is right forum or not.
> 
> This is not the right forum. This mailing list is about developing the
> CPython interpreter.
> 
> For general questions, you may want to try the comp.lang.python newsgroup.

Or the python-list mailing list, or the gmane.comp.python.general 
newsgroup mirror at gmane.org (free).


From janssen at parc.com  Wed Jun 17 21:14:35 2009
From: janssen at parc.com (Bill Janssen)
Date: Wed, 17 Jun 2009 12:14:35 PDT
Subject: [Python-Dev] SSL Certificate Validation
In-Reply-To: <26228.1245188714@parc.com>
References: <8847f980906160709q405ad950qac0c56b304562d3@mail.gmail.com>
	<26228.1245188714@parc.com>
Message-ID: <36924.1245266075@parc.com>

Bill Janssen <janssen at parc.com> wrote:

> > Does it check that the host the socket is connected to is the same as
> > what's given in the CN field in the certificate?
> 
> No.  That, in general, doesn't work very well.  The IETF working group
> on this is considering deprecating putting a hostname in the CN field at
> all, and just adding hostnames via the subjectAltName extension.  The
> problem that's come up is that many computers don't have fixed IP
> addresses, and even with that the hostname is part of a different
> mapping of hostnames to IP addresses, which can also vary.

Incidentally, the current working draft on this seems to be at
<http://tools.ietf.org/html/draft-saintandre-tls-server-id-check-00>.

Bill

From janssen at parc.com  Wed Jun 17 21:15:57 2009
From: janssen at parc.com (Bill Janssen)
Date: Wed, 17 Jun 2009 12:15:57 PDT
Subject: [Python-Dev] SSL Certificate Validation
In-Reply-To: <4A3819BD.4000301@v.loewis.de>
References: <8847f980906160709q405ad950qac0c56b304562d3@mail.gmail.com>
	<4A37F126.5010008@v.loewis.de>
	<4222a8490906161242p305e2268q550cf48d1d82f884@mail.gmail.com>
	<4A37FD2B.1000904@v.loewis.de>
	<8847f980906161431n2f9043afk27a43194ac3721f5@mail.gmail.com>
	<4A3819BD.4000301@v.loewis.de>
Message-ID: <36941.1245266157@parc.com>

Martin v. L?wis <martin at v.loewis.de> wrote:

> FWIW, I actually don't know the answer for sure, either, so I would have
> to research this myself, too. In any case, _ssl.c is *not* the place
> where any of the certificate validation actually happens - nor does it
> happen elsewhere in the Python source code, IIUC.

Strictly speaking, that's right.  It's all done by OpenSSL.

Bill

From devin.c.cook at gmail.com  Wed Jun 17 23:00:01 2009
From: devin.c.cook at gmail.com (Devin Cook)
Date: Wed, 17 Jun 2009 17:00:01 -0400
Subject: [Python-Dev] SSL Certificate Validation
In-Reply-To: <36924.1245266075@parc.com>
References: <8847f980906160709q405ad950qac0c56b304562d3@mail.gmail.com>
	<26228.1245188714@parc.com> <36924.1245266075@parc.com>
Message-ID: <8847f980906171400p3656304o70559dc2a946c789@mail.gmail.com>

Ok, thanks for all the feedback. Just for clarity, I'll summarize
everything as I understand it:

* OpenSSL does the all validation of the certificate itself.
(http://openssl.org/docs/apps/verify.html)
* httplib should have a way to enable validation of the certificate.
* httplib should have a way to enable checking of the reference
identity. (that complies with section 3 of this draft:
http://tools.ietf.org/html/draft-saintandre-tls-server-id-check-00)
* The reference identity checking (and cert validation, I assume)
shouldn't be automatic. (per Bill)

Does that sound about right? I'll try to work up a patch tonight
implementing this.

-Devin

From janssen at parc.com  Wed Jun 17 23:57:48 2009
From: janssen at parc.com (Bill Janssen)
Date: Wed, 17 Jun 2009 14:57:48 PDT
Subject: [Python-Dev] SSL Certificate Validation
In-Reply-To: <8847f980906171400p3656304o70559dc2a946c789@mail.gmail.com>
References: <8847f980906160709q405ad950qac0c56b304562d3@mail.gmail.com>
	<26228.1245188714@parc.com> <36924.1245266075@parc.com>
	<8847f980906171400p3656304o70559dc2a946c789@mail.gmail.com>
Message-ID: <40972.1245275868@parc.com>

I think if you check the issue tracker, there's already a patch for this
somewhere, IIRC.

Bill

Devin Cook <devin.c.cook at gmail.com> wrote:

> Ok, thanks for all the feedback. Just for clarity, I'll summarize
> everything as I understand it:
> 
> * OpenSSL does the all validation of the certificate itself.
> (http://openssl.org/docs/apps/verify.html)
> * httplib should have a way to enable validation of the certificate.
> * httplib should have a way to enable checking of the reference
> identity. (that complies with section 3 of this draft:
> http://tools.ietf.org/html/draft-saintandre-tls-server-id-check-00)
> * The reference identity checking (and cert validation, I assume)
> shouldn't be automatic. (per Bill)
> 
> Does that sound about right? I'll try to work up a patch tonight
> implementing this.
> 
> -Devin

From greg.ewing at canterbury.ac.nz  Thu Jun 18 03:48:31 2009
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Thu, 18 Jun 2009 13:48:31 +1200
Subject: [Python-Dev] io.BufferedReader.peek() Behaviour in python3.1
In-Reply-To: <20090617012719.GA19974@cskk.homeip.net>
References: <20090617012719.GA19974@cskk.homeip.net>
Message-ID: <4A399CEF.2070202@canterbury.ac.nz>

Cameron Simpson wrote:

> But people not using threads, or at any rate not
> dedicating a thread to the reading task, don't have such luxury.

But without a dedicated thread you need to use
select() or poll(), and then buffering causes other
headaches.

> Are we disputing the utility of being able to ask "how much might I
> read/peek without blocking"?

I'm saying that I don't see how I would make use
of such a thing, so I probably wouldn't mind if
it didn't exist.

> Or disputing the purpose of peek, which
> feels to me like it should/might ask that question, but doesn't.

I think what I'm saying is that there are two
distinct use cases being talked about for a peek-like
operation, and different people seem to have different
ideas on which one should be mapped to the name "peek".
So perhaps they should both be given more-explicit
names.

> It's perfectly possible, even reasonable, to avoid talking about the
> buffer at all. I'd be happy not to mention the buffer.

Even if you don't mention it explicitly, its
existence shows through in the fact that there
is an arbitrary limit on the amount you can
peek ahead, and that limit needs to be documented
so that people can write correct programs.

This is true of both kinds of peeking, so I
concede that they both break the abstraction.

However I think the non-blocking peek breaks it
more than the blocking one, because it also
brings non-deterministic behaviour.

-- 
Greg

From drnlmuller+python at gmail.com  Thu Jun 18 15:22:13 2009
From: drnlmuller+python at gmail.com (Neil Muller)
Date: Thu, 18 Jun 2009 15:22:13 +0200
Subject: [Python-Dev] ElementTree issues in python 3.1 rc2
Message-ID: <a4cf72520906180622m6a2cc76cr89309f0a4c56fbe7@mail.gmail.com>

The following issues in ElementTree still exist in the latest release
candidate & all have small patches to fix the issue. Any chance of
getting these in before the final release?

http://bugs.python.org/issue6231
http://bugs.python.org/issue6233
http://bugs.python.org/issue2746

-- 
Neil Muller
drnlmuller at gmail.com

From solipsis at pitrou.net  Thu Jun 18 15:27:26 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Thu, 18 Jun 2009 13:27:26 +0000 (UTC)
Subject: [Python-Dev] ElementTree issues in python 3.1 rc2
References: <a4cf72520906180622m6a2cc76cr89309f0a4c56fbe7@mail.gmail.com>
Message-ID: <loom.20090618T132529-859@post.gmane.org>

Neil Muller <drnlmuller+python <at> gmail.com> writes:
> 
> The following issues in ElementTree still exist in the latest release
> candidate & all have small patches to fix the issue. Any chance of
> getting these in before the final release?
> 
> http://bugs.python.org/issue6231
> http://bugs.python.org/issue6233
> http://bugs.python.org/issue2746

I've set issue 6233 as a release blocker, since it's an annoying regression
compared to trunk.

As a side note, I think we have a process problem with externally maintained
modules such as etree and json, especially when the maintainer isn't very
reactive to bug reports.

Regards

Antoine.



From lukepadawan at gmail.com  Thu Jun 18 16:54:18 2009
From: lukepadawan at gmail.com (Lucas P Melo)
Date: Thu, 18 Jun 2009 11:54:18 -0300
Subject: [Python-Dev] io.BufferedReader.peek() Behaviour in python3.1
In-Reply-To: <4A399CEF.2070202@canterbury.ac.nz>
References: <20090617012719.GA19974@cskk.homeip.net>
	<4A399CEF.2070202@canterbury.ac.nz>
Message-ID: <4A3A551A.6060304@gmail.com>

Greg Ewing wrote:
> Even if you don't mention it explicitly, its
> existence shows through in the fact that there
> is an arbitrary limit on the amount you can
> peek ahead, and that limit needs to be documented
> so that people can write correct programs.
>
> This is true of both kinds of peeking, so I
> concede that they both break the abstraction.
>
> However I think the non-blocking peek breaks it
> more than the blocking one, because it also
> brings non-deterministic behaviour.
It depends on the point of view.
For example, someone is writing a program that must read from any kind 
of file descriptor and generate the derivation tree of the text read 
based on some context-free grammar. The problem is that the chosen 
method to accomplish it would read 2 symbols (bytes) ahead and this guy 
is using peek() to grab these 2 bytes. The program will seem to work 
correctly most of the time, but on the 4095th byte read, he would grab 1 
byte at most using peek() (despite the existence of say 10k bytes 
ahead). The blocking definition of peek() would create this hard-to-spot 
bug. Thus, for him, this behavior would seem non-deterministic.
On the other hand, if someone is conscious about the number of raw 
reads, he would definitely be willing to look into the documentation for 
any parameters that match his special needs. That's why the non-blocking 
behavior should be the default one while the blocking behavior should be 
accessible by choice.

From eric.pruitt at gmail.com  Thu Jun 18 17:30:29 2009
From: eric.pruitt at gmail.com (Eric Pruitt)
Date: Thu, 18 Jun 2009 10:30:29 -0500
Subject: [Python-Dev] Popen asynchronous input for Windows
Message-ID: <171e8a410906180830u3ef49ba8j3dc384f329763167@mail.gmail.com>

Hello,

I am looking for alternatives to Josiah Carlson's asynchronous I/O patch for
subprocess.Popen. While his patch seems to work well, it relies on pywin32
which is not part of the standard Python library. If I cannot find an
alternative, I will be using cTypes with the parts of Mark Hammond's code
that I need, license permitting. Any suggestions are greatly appreciated.

Eric
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090618/eb14d103/attachment.htm>

From lists at cheimes.de  Thu Jun 18 20:32:27 2009
From: lists at cheimes.de (Christian Heimes)
Date: Thu, 18 Jun 2009 20:32:27 +0200
Subject: [Python-Dev] Popen asynchronous input for Windows
In-Reply-To: <171e8a410906180830u3ef49ba8j3dc384f329763167@mail.gmail.com>
References: <171e8a410906180830u3ef49ba8j3dc384f329763167@mail.gmail.com>
Message-ID: <4A3A883B.5040508@cheimes.de>

Eric Pruitt schrieb:
> Hello,
> 
> I am looking for alternatives to Josiah Carlson's asynchronous I/O patch for
> subprocess.Popen. While his patch seems to work well, it relies on pywin32
> which is not part of the standard Python library. If I cannot find an
> alternative, I will be using cTypes with the parts of Mark Hammond's code
> that I need, license permitting. Any suggestions are greatly appreciated.

The subprocess module several wrappers for win32 APIs in
Modules/_subprocess.c. You could add the necessary functions.

Christian

From facundobatista at gmail.com  Thu Jun 18 22:07:02 2009
From: facundobatista at gmail.com (Facundo Batista)
Date: Thu, 18 Jun 2009 17:07:02 -0300
Subject: [Python-Dev] Avoiding file descriptors leakage in
	subprocess.Popen()
In-Reply-To: <e04bdf310906130540t23c4b78r68a01a95a3e9f621@mail.gmail.com>
References: <e04bdf310906121425x72a7c033wec71500712759375@mail.gmail.com>
	<4A32ED75.7060803@cheimes.de>
	<e04bdf310906130540t23c4b78r68a01a95a3e9f621@mail.gmail.com>
Message-ID: <e04bdf310906181307t10b08c9t832bef4770151799@mail.gmail.com>

On Sat, Jun 13, 2009 at 9:40 AM, Facundo
Batista<facundobatista at gmail.com> wrote:

>> How about a nice 'n shiny context wrapper for the pipe:
>
> I'll do this!
>
> Thank you for the suggestion!

Boo, I can not take this approach, neither the previous one.

The reason is very specific for subprocess.py... after creating the
FDs, it forks(), and the behaviour of when closing which descriptors
are different for the parent and child processes.  If I take Christian
approach, the test just hangs. One drawback of the "with" usage is
that it closes both FDs at the same time... and in this case this may
be a problem, as they're used and closed by different processes in
different times... don't know.

I also tried the approach of wrapping the FDs with a file object...
this *almost* work... but in the case of a problem in the child
process, when a traceback should be written through the pipe to the
parent, nothing happens (it seems that the GC just closes the object
and the info is not transferred).

So, I'll stick to the code I submitted to the bug, because even if
it's ugly it works.

-- 
.    Facundo

Blog: http://www.taniquetil.com.ar/plog/
PyAr: http://www.python.org/ar/

From eric.pruitt at gmail.com  Thu Jun 18 22:29:44 2009
From: eric.pruitt at gmail.com (Eric Pruitt)
Date: Thu, 18 Jun 2009 15:29:44 -0500
Subject: [Python-Dev] Popen asynchronous input for Windows
In-Reply-To: <4A3A883B.5040508@cheimes.de>
References: <171e8a410906180830u3ef49ba8j3dc384f329763167@mail.gmail.com> 
	<4A3A883B.5040508@cheimes.de>
Message-ID: <171e8a410906181329j5f981266p5a61b4a9f4008357@mail.gmail.com>

Thanks for the lead. I have the pywin32 source code and have found the files
that appear to contain the code I need inside of some "*.i" files. After a
bit of Googling and paying attention to the blatantly obvious *.cpp files, I
realized Hammond's code is written in C++ whereas Python uses C. I am
familiar with neither language very well but if I can't find a work-around
for Windows asynchronous I/O, I will be learning a bit of both of them.

Please feel free to suggest any other ideas that don't rely on my sparse
knowledge of C and C++.

Eric

On Thu, Jun 18, 2009 at 13:32, Christian Heimes <lists at cheimes.de> wrote:

> Eric Pruitt schrieb:
> > Hello,
> >
> > I am looking for alternatives to Josiah Carlson's asynchronous I/O patch
> for
> > subprocess.Popen. While his patch seems to work well, it relies on
> pywin32
> > which is not part of the standard Python library. If I cannot find an
> > alternative, I will be using cTypes with the parts of Mark Hammond's code
> > that I need, license permitting. Any suggestions are greatly appreciated.
>
> The subprocess module several wrappers for win32 APIs in
> Modules/_subprocess.c. You could add the necessary functions.
>
> Christian
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090618/c85c875b/attachment-0001.htm>

From greg.ewing at canterbury.ac.nz  Fri Jun 19 02:50:27 2009
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Fri, 19 Jun 2009 12:50:27 +1200
Subject: [Python-Dev] io.BufferedReader.peek() Behaviour in python3.1
In-Reply-To: <4A3A551A.6060304@gmail.com>
References: <20090617012719.GA19974@cskk.homeip.net>
	<4A399CEF.2070202@canterbury.ac.nz> <4A3A551A.6060304@gmail.com>
Message-ID: <4A3AE0D3.2090708@canterbury.ac.nz>

Lucas P Melo wrote:

> The problem is that the chosen 
> method to accomplish it would read 2 symbols (bytes) ahead and this guy 
> is using peek() to grab these 2 bytes. The program will seem to work 
> correctly most of the time, but on the 4095th byte read, he would grab 1 
> byte at most using peek()

That's exactly why I think the blocking version should
keep reading until the requested number of bytes is
available (or the buffer is full or EOF occurs).

In other words, the blocking version should be fully
deterministic given knowledge of the buffer size.

-- 
Greg

From benjamin at python.org  Fri Jun 19 04:17:44 2009
From: benjamin at python.org (Benjamin Peterson)
Date: Thu, 18 Jun 2009 21:17:44 -0500
Subject: [Python-Dev] draft pep: backwards compatibility
Message-ID: <1afaf6160906181917m670663f5ucb21b8bc00c07735@mail.gmail.com>

Backwards compatibility seems to be an issue that arises a lot here. I
think we all have an idea of it is, but we need some hard place to
point to. So here's my attempt:


PEP: 387
Title: Backwards Compatibility Policy
Version: $Revision$
Last-Modified: $Date$
Author: Benjamin Peterson <benjamin at python.org>
Status: Draft
Type: Process
Content-Type: text/x-rst
Created: 18-Jun-2009


Abstract
========

This PEP outlines Python's backwards compatibility policy.


Rationale
=========

As one of the most used programming languages today [#tiobe]_, the Python core
language and its standard library play a critcal role in thousands of
applications and libraries.  This is fantastic; it is probably one of a language
designer's most wishful dreams.  However, it means the development team must be
very careful not to break this existing 3rd party code with new releases.


Backwards Compatibility Rules
=============================

This policy applys to all public APIs.  These include the C-API, the standard
library, and the core language including syntax and operation as defined by the
reference manual.

This is the basic policy for backwards compatibility:

* The behavior of an API *must* not change between any two consecutive releases.

* A feature cannot be removed without notice between any two consecutive
  releases.

* Addition of a feature which breaks 3rd party libraries or applications should
  have a large benefit to breakage ratio, and/or the incompatibility should be
  trival to fix in broken code.


Making Incompatible Changes
===========================

It's a fact: design mistakes happen.  Thus it is important to be able to change
APIs or remove misguided features.  This is accomplished through a gradual
process over several releases:

1. Discuss the change.  Depending on the size of the incompatibility, this could
   be on the bug tracker, python-dev, python-list, or the appropriate SIG.  A
   PEP or similar document may be written.  Hopefully users of the affected API
   will pipe up to comment.

2. Add a warning [#warnings_]_.  If behavior is changing, a the API may gain a
   new function or method to perform the new behavior; old usage should raise
   the warning.  If an API is being removed, simply warn whenever it is entered.
   DeprecationWarning is the usual warning category to use, but
   PendingDeprecationWarning may be used in special cases were the old and new
   versions of the API will coexist for many releases.

3. Wait for a release.

4. See if there's any feedback.  Users not involved in the original discussions
   may comment now after seeing the warning.  Perhaps reconsider.

5. The behavior change or feature removal may now be made default or permanent
   in the next release.  Remove the old version and warning.


References
==========

.. [#tiobe] TIOBE Programming Community Index

   http://www.tiobe.com/index.php/content/paperinfo/tpci/index.html

.. [#warnings] The warnings module

   http://docs.python.org/library/warnings.html


Copyright
=========

This document has been placed in the public domain.



..
   Local Variables:
   mode: indented-text
   indent-tabs-mode: nil
   sentence-end-double-space: t
   fill-column: 70
   coding: utf-8
   End:

From solipsis at pitrou.net  Fri Jun 19 10:23:01 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Fri, 19 Jun 2009 08:23:01 +0000 (UTC)
Subject: [Python-Dev] draft pep: backwards compatibility
References: <1afaf6160906181917m670663f5ucb21b8bc00c07735@mail.gmail.com>
Message-ID: <loom.20090619T081720-465@post.gmane.org>

Benjamin Peterson <benjamin <at> python.org> writes:
> 
> This policy applys to all public APIs.

applies?

> * The behavior of an API *must* not change between any two consecutive
> releases.
> 
> * A feature cannot be removed without notice between any two consecutive
>   releases.

By induction, this would mean no API could change and no feature could be
removed without notice between any N consecutive releases. Do you really mean
it?

> * Addition of a feature which breaks 3rd party libraries or applications
> should
>   have a large benefit to breakage ratio, and/or the incompatibility should
> be
>   trival to fix in broken code.

There is always the possibility that a new feature breaks existing code, for
example because it relies on a similarly named attribute, or on some obscure
internal condition. I think this should be qualified so that it only applies
when e.g. a fair number of third-party apps or libraries are broken.



From p.f.moore at gmail.com  Fri Jun 19 10:29:23 2009
From: p.f.moore at gmail.com (Paul Moore)
Date: Fri, 19 Jun 2009 09:29:23 +0100
Subject: [Python-Dev] draft pep: backwards compatibility
In-Reply-To: <1afaf6160906181917m670663f5ucb21b8bc00c07735@mail.gmail.com>
References: <1afaf6160906181917m670663f5ucb21b8bc00c07735@mail.gmail.com>
Message-ID: <79990c6b0906190129p5c930fe3l1d93162219e0bbbf@mail.gmail.com>

2009/6/19 Benjamin Peterson <benjamin at python.org>:
> Backwards compatibility seems to be an issue that arises a lot here. I
> think we all have an idea of it is, but we need some hard place to
> point to. So here's my attempt:

Nice :-)

A general point - it's probably worth clarifying that you're referring
to major releases (2.6 -> 2.7 etc.) here. Minor releases have a strict
bugfixes-only policy.

> applications and libraries. ?This is fantastic; it is probably one of a language
> designer's most wishful dreams.

That's a slightly odd wording. I'm not sure I can think of a better
one, though...


> This policy applys to all public APIs. ?These include the C-API, the standard

... applies ...

> * The behavior of an API *must* not change between any two consecutive releases.

With the exception of compatibility warnings as described below.

In practice, I think APIs *do* change, so presumably there must have
been a pair of releases between which the change happened. I see what
you're trying to say, but I think you overstated things a little.

Paul.

From glyph at divmod.com  Fri Jun 19 11:31:30 2009
From: glyph at divmod.com (glyph at divmod.com)
Date: Fri, 19 Jun 2009 09:31:30 -0000
Subject: [Python-Dev] draft pep: backwards compatibility
In-Reply-To: <1afaf6160906181917m670663f5ucb21b8bc00c07735@mail.gmail.com>
References: <1afaf6160906181917m670663f5ucb21b8bc00c07735@mail.gmail.com>
Message-ID: <20090619093130.12555.864027812.divmod.xquotient.12879@weber.divmod.com>


On 02:17 am, benjamin at python.org wrote:
>Backwards compatibility seems to be an issue that arises a lot here. I
>think we all have an idea of it is, but we need some hard place to
>point to. So here's my attempt:

>PEP: 387
>Title: Backwards Compatibility Policy

Thanks for getting started on this.  This is indeed a bikeshed that 
there would be less discussion around if there were a clearer PEP to 
point to.  However, this draft PEP points to the problem rather than the 
solution.  In order to really be useful this needs to answer a whole 
slew of very very specific questions, because the real problem is that 
everybody thinks they know what they mean but in fact we all have 
slightly different definitions of these words.

I've taken a stab at this myself in the past while defining a policy for 
Twisted, which is here: 
<http://twistedmatrix.com/trac/wiki/CompatibilityPolicy>.  I think we 
might be a bit stricter than Python, but I believe our attempt to 
outline these terms specifically is worth taking a look at.

The questions that follow might seem satirical or parodic but I am 
completely serious and each of these terms really does have a variable 
definition.
>* The behavior of an API *must* not change between any two consecutive 
>releases.

What is "behavior"?  Software is composed of behavior.  If no behavior 
changes, then nothing can ever happen.

What is an "API"?  Are you talking about a public name in a Python 
module?  A name covered in documentation?  A name covered by a specific 
piece of documentation?  Is "subclassing" considered an API, or just 
invoking?  What about isinstance()?  What about repr()?  Are string 
formats allowed to change?  For example, certain warnings have caused 
the Twisted test suite to fail because with the introduction of warnings 
at the C level it becomes progressively harder for a Python process to 
control its own stderr stream.  I can't remember the specifics right 
now, but on more than one occasion a python upgrade caused our test 
suite to fail because the expectation was that a process would be able 
to successfully terminate without putting anything on any FD but stdout.

In order for any changes to be possible, there needs to be a clearly 
labeled mine-field: don't touch or depend on these things in your Python 
application or it *will* break every time Python is released.

What are "consecutive" releases?  If we have releases 1, 2, 3, 4, and 
behavior can't change 1->2, 2->3, or 3->4, then no change may ever be 
made.

What does "must not" mean here?  My preference would be "once there is 
agreement that an incompatible change has occurred, there should be an 
immediate revert with no discussion".  Unless of course the change has 
already been released.
>* A feature cannot be removed without notice between any two 
>consecutive
>  releases.

Presumably removal of a feature is a change in behavior, so isn't this 
redundant with the previous point?
>* Addition of a feature which breaks 3rd party libraries or 
>applications should
>  have a large benefit to breakage ratio, and/or the incompatibility 
>should be
>  trival to fix in broken code.

How do you propose to measure the benefit to breakage ratio?  How do you 
propose to define "trivial" to fix?  Most projects, including Python, 
Twisted, Django, and Zope, have an ever-increasing bug count, which 
means that trivial issues fall off the radar pretty easily.

One of the big problems here in detecting and fixing "trivial" changes 
is that they can occur in test code as well.  So if the answer is "run 
your tests", that's not much help if you can't call the actual APIs 
whose behavior has changed.  The standard library would need to start 
providing a lot more verified test stubs for things like I/O in order 
for this to be feasible.
>Making Incompatible Changes
>===========================
>
>It's a fact: design mistakes happen.  Thus it is important to be able 
>to change
>APIs or remove misguided features.  This is accomplished through a 
>gradual
>process over several releases:
>
>1. Discuss the change.  Depending on the size of the incompatibility, 
>this could
>   be on the bug tracker, python-dev, python-list, or the appropriate 
>SIG.  A
>   PEP or similar document may be written.  Hopefully users of the 
>affected API
>   will pipe up to comment.

There are a very large number of users of Python, the large percentage 
of whom do not read python-dev.  A posting on python-list is likely to 
provoke an unproductive pile-on.  I suggest a dedicated list, which 
should ideally be low traffic, "python-incompatible-announce", or 
something like that, so that *everyone* who maintains Python software 
can subscribe to it, even if they're not otherwise interested in Python 
development.
>2. Add a warning [#warnings_]_.  If behavior is changing, a the API may 
>gain a
>   new function or method to perform the new behavior; old usage should 
>raise
>   the warning.  If an API is being removed, simply warn whenever it is 
>entered.
>   DeprecationWarning is the usual warning category to use, but
>   PendingDeprecationWarning may be used in special cases were the old 
>and new
>   versions of the API will coexist for many releases.

Why is PendingDeprecationWarning the special case?

As outlined in the Twisted compatibility proposal, I'd prefer it if 
every warning went through multiple phases: pending, deprecated, gone: 
so that careful developers could release new versions of their software 
to quash noisy warnings *before* the new version of Python that actually 
made them user-visible was released.
>3. Wait for a release.

How does this affect the parallel 2.x/3.x release cycle?
>4. See if there's any feedback.  Users not involved in the original 
>discussions
>   may comment now after seeing the warning.  Perhaps reconsider.

At this point, many developers may already have been porting over to the 
new, non-deprecated API.  I realize that there may be no way around 
that, but it's worth considering.

Thanks for tackling this thorny problem.  Good luck!

From solipsis at pitrou.net  Fri Jun 19 11:21:49 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Fri, 19 Jun 2009 09:21:49 +0000 (UTC)
Subject: [Python-Dev] draft pep: backwards compatibility
References: <1afaf6160906181917m670663f5ucb21b8bc00c07735@mail.gmail.com>
	<20090619093130.12555.864027812.divmod.xquotient.12879@weber.divmod.com>
Message-ID: <loom.20090619T091500-999@post.gmane.org>

<glyph <at> divmod.com> writes:
> 
> In order for any changes to be possible, there needs to be a clearly 
> labeled mine-field: don't touch or depend on these things in your Python 
> application or it *will* break every time Python is released.

I think the "frozen area" should be defined positively (explicit public APIs and
explicitly guaranteed behaviour) rather than negatively (an explicit "mine
field"). Otherwise, we will forget to put some important things in the minefield
and get bitten later when we need to change those things in a
backwards-incompatible way.

For example, I think it was wrong to change the name of a test-skipping unittest
method just so that it wouldn't clash with a method created by Twisted
subclassing unittest (besides, self.skip() was much nicer than self.skipTest()
;-)). Just because some class is public shouldn't prevent us to add new public
methods or attributes to it.

Regards

Antoine.



From glyph at divmod.com  Fri Jun 19 13:26:34 2009
From: glyph at divmod.com (glyph at divmod.com)
Date: Fri, 19 Jun 2009 11:26:34 -0000
Subject: [Python-Dev] draft pep: backwards compatibility
In-Reply-To: <loom.20090619T091500-999@post.gmane.org>
References: <1afaf6160906181917m670663f5ucb21b8bc00c07735@mail.gmail.com>
	<20090619093130.12555.864027812.divmod.xquotient.12879@weber.divmod.com>
	<loom.20090619T091500-999@post.gmane.org>
Message-ID: <20090619112634.12555.1573592942.divmod.xquotient.12922@weber.divmod.com>


On 09:21 am, solipsis at pitrou.net wrote:
><glyph <at> divmod.com> writes:
>>
>>In order for any changes to be possible, there needs to be a clearly
>>labeled mine-field: don't touch or depend on these things in your 
>>Python
>>application or it *will* break every time Python is released.
>
>I think the "frozen area" should be defined positively (explicit public 
>APIs and
>explicitly guaranteed behaviour) rather than negatively (an explicit 
>"mine
>field"). Otherwise, we will forget to put some important things in the 
>minefield
>and get bitten later when we need to change those things in a
>backwards-incompatible way.

This is a false dichotomy; for core developers, the list needs to be 
exhaustive.  Everything that can change needs to be described as either 
compatible or incompatible.

However, the reason I say that the list needs to be phrased as a 
whitelist is that we have to assume *all* people writing Python code, 
who ever want to be able to upgrade Python, need to *completely* 
understand the list of things which they should not depend on.  Every 
piece of documentation they read and every API they find, the assumption 
is that it's going to be compatible and it's not something they need to 
worry about.

It's important to remember that this PEP has a "public" (python 
application programmers) and "private" (python core developer) 
interfaces, too ;-).
>For example, I think it was wrong to change the name of a test-skipping 
>unittest
>method just so that it wouldn't clash with a method created by Twisted
>subclassing unittest (besides, self.skip() was much nicer than 
>self.skipTest()
>;-)). Just because some class is public shouldn't prevent us to add new 
>public
>methods or attributes to it.

I think it would be wrong to have a blanket prohibition on such changes, 
by which I mean additions of names to public namespaces.  Twisted's own 
compatibility possibility would not forbid a similar change.  But in 
that specific case I think it was the right thing to do.  Like it or 
not, a lot of people use Twisted, a lot of people run tests with Trial, 
and we beat stdlib unittest to the punch on the 'skip' testing feature 
by a good 5 years.  We caught the change well before the release, we 
reported it and discussed it.

IMHO this is the best way to deal with incompatible changes, especially 
in the case of superclasses, given Python's subtle and complex 
inheritance semantics.  It's not feasible to provide a policy that 
somehow prohibits subclasses from adding names which may eventually be 
used on a superclass.

Projects which notice test failures with new versions of Python should 
report them so that the features can be adjusted to be compatible, 
assuming the project in question hasn't done anything in egregious 
violation of the compatibility policy (like invoking a private method). 
This lets users, system administrators, and application authors upgrade 
components individually, without worrying about the components further 
down the stack flaking out on them.  It also provides a feedback loop 
for the compatibility policy: if there are things that initially seem 
reasonable, but projects report compatibility issues when they are 
changed, they might need to be added later.

From solipsis at pitrou.net  Fri Jun 19 13:11:11 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Fri, 19 Jun 2009 11:11:11 +0000 (UTC)
Subject: [Python-Dev] draft pep: backwards compatibility
References: <1afaf6160906181917m670663f5ucb21b8bc00c07735@mail.gmail.com>
	<20090619093130.12555.864027812.divmod.xquotient.12879@weber.divmod.com>
	<loom.20090619T091500-999@post.gmane.org>
	<20090619112634.12555.1573592942.divmod.xquotient.12922@weber.divmod.com>
Message-ID: <loom.20090619T110757-94@post.gmane.org>

<glyph <at> divmod.com> writes:
> 
> This is a false dichotomy; for core developers, the list needs to be 
> exhaustive.  Everything that can change needs to be described as either 
> compatible or incompatible.

How do you enumerate "everything that can change"? It does not look like a
finite set to me (but perhaps I'm wrong); and certainly not like a set of a size
reasonable enough to be enumerated in a human-readable way :)




From fuzzyman at voidspace.org.uk  Fri Jun 19 13:13:39 2009
From: fuzzyman at voidspace.org.uk (Michael Foord)
Date: Fri, 19 Jun 2009 12:13:39 +0100
Subject: [Python-Dev] draft pep: backwards compatibility
In-Reply-To: <20090619112634.12555.1573592942.divmod.xquotient.12922@weber.divmod.com>
References: <1afaf6160906181917m670663f5ucb21b8bc00c07735@mail.gmail.com>	<20090619093130.12555.864027812.divmod.xquotient.12879@weber.divmod.com>	<loom.20090619T091500-999@post.gmane.org>
	<20090619112634.12555.1573592942.divmod.xquotient.12922@weber.divmod.com>
Message-ID: <4A3B72E3.3070106@voidspace.org.uk>

glyph at divmod.com wrote:
> [snip...]
>> For example, I think it was wrong to change the name of a 
>> test-skipping unittest
>> method just so that it wouldn't clash with a method created by Twisted
>> subclassing unittest (besides, self.skip() was much nicer than 
>> self.skipTest()
>> ;-)). Just because some class is public shouldn't prevent us to add 
>> new public
>> methods or attributes to it.
>
> I think it would be wrong to have a blanket prohibition on such 
> changes, by which I mean additions of names to public namespaces.  
> Twisted's own compatibility possibility would not forbid a similar 
> change.  But in that specific case I think it was the right thing to 
> do.  Like it or not, a lot of people use Twisted, a lot of people run 
> tests with Trial, and we beat stdlib unittest to the punch on the 
> 'skip' testing feature by a good 5 years.  We caught the change well 
> before the release, we reported it and discussed it.

Just to note that Twisted (along with Bazaar) are one of the few 'good 
citizens' in the Python community running their tests on Python trunk. 
Both projects have caught incompatibilities *before* release of new 
versions which is greatly preferable to discovering them after a 
release. Thanks for this.

Michael Foordt

>
> IMHO this is the best way to deal with incompatible changes, 
> especially in the case of superclasses, given Python's subtle and 
> complex inheritance semantics.  It's not feasible to provide a policy 
> that somehow prohibits subclasses from adding names which may 
> eventually be used on a superclass.
>
> Projects which notice test failures with new versions of Python should 
> report them so that the features can be adjusted to be compatible, 
> assuming the project in question hasn't done anything in egregious 
> violation of the compatibility policy (like invoking a private 
> method). This lets users, system administrators, and application 
> authors upgrade components individually, without worrying about the 
> components further down the stack flaking out on them.  It also 
> provides a feedback loop for the compatibility policy: if there are 
> things that initially seem reasonable, but projects report 
> compatibility issues when they are changed, they might need to be 
> added later.
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: 
> http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk 
>


-- 
http://www.ironpythoninaction.com/
http://www.voidspace.org.uk/blog



From fuzzyman at voidspace.org.uk  Fri Jun 19 13:14:55 2009
From: fuzzyman at voidspace.org.uk (Michael Foord)
Date: Fri, 19 Jun 2009 12:14:55 +0100
Subject: [Python-Dev] draft pep: backwards compatibility
In-Reply-To: <loom.20090619T110757-94@post.gmane.org>
References: <1afaf6160906181917m670663f5ucb21b8bc00c07735@mail.gmail.com>	<20090619093130.12555.864027812.divmod.xquotient.12879@weber.divmod.com>	<loom.20090619T091500-999@post.gmane.org>	<20090619112634.12555.1573592942.divmod.xquotient.12922@weber.divmod.com>
	<loom.20090619T110757-94@post.gmane.org>
Message-ID: <4A3B732F.5080808@voidspace.org.uk>

Antoine Pitrou wrote:
> <glyph <at> divmod.com> writes:
>   
>> This is a false dichotomy; for core developers, the list needs to be 
>> exhaustive.  Everything that can change needs to be described as either 
>> compatible or incompatible.
>>     
>
> How do you enumerate "everything that can change"? It does not look like a
> finite set to me (but perhaps I'm wrong); and certainly not like a set of a size
> reasonable enough to be enumerated in a human-readable way :)
>
>   

And this is why expressing a finite list of things we guarantee won't 
change is a virtually impossible task - unless one of you is 
volunteering to write an official spec for Python and its libraries... 
:-) (Something that would not be bad IMO - just a long and difficult 
task, *especially* if you include the library along with language 
semantics and APIs).

Michael

>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk
>   


-- 
http://www.ironpythoninaction.com/
http://www.voidspace.org.uk/blog



From fuzzyman at voidspace.org.uk  Fri Jun 19 13:16:30 2009
From: fuzzyman at voidspace.org.uk (Michael Foord)
Date: Fri, 19 Jun 2009 12:16:30 +0100
Subject: [Python-Dev] draft pep: backwards compatibility
In-Reply-To: <1afaf6160906181917m670663f5ucb21b8bc00c07735@mail.gmail.com>
References: <1afaf6160906181917m670663f5ucb21b8bc00c07735@mail.gmail.com>
Message-ID: <4A3B738E.3040309@voidspace.org.uk>

Benjamin Peterson wrote:
> Backwards compatibility seems to be an issue that arises a lot here. I
> think we all have an idea of it is, but we need some hard place to
> point to. So here's my attempt:
>   

Should this PEP include a note about features which require new syntax, 
particularly  new keywords?

The policy (AFAICT) is that if new keywords are created they are enabled 
with a future import (with a warning?) in the version they are 
introduced and then enabled by default in the next version.

All the best,

Michael Foord

>
> PEP: 387
> Title: Backwards Compatibility Policy
> Version: $Revision$
> Last-Modified: $Date$
> Author: Benjamin Peterson <benjamin at python.org>
> Status: Draft
> Type: Process
> Content-Type: text/x-rst
> Created: 18-Jun-2009
>
>
> Abstract
> ========
>
> This PEP outlines Python's backwards compatibility policy.
>
>
> Rationale
> =========
>
> As one of the most used programming languages today [#tiobe]_, the Python core
> language and its standard library play a critcal role in thousands of
> applications and libraries.  This is fantastic; it is probably one of a language
> designer's most wishful dreams.  However, it means the development team must be
> very careful not to break this existing 3rd party code with new releases.
>
>
> Backwards Compatibility Rules
> =============================
>
> This policy applys to all public APIs.  These include the C-API, the standard
> library, and the core language including syntax and operation as defined by the
> reference manual.
>
> This is the basic policy for backwards compatibility:
>
> * The behavior of an API *must* not change between any two consecutive releases.
>
> * A feature cannot be removed without notice between any two consecutive
>   releases.
>
> * Addition of a feature which breaks 3rd party libraries or applications should
>   have a large benefit to breakage ratio, and/or the incompatibility should be
>   trival to fix in broken code.
>
>
> Making Incompatible Changes
> ===========================
>
> It's a fact: design mistakes happen.  Thus it is important to be able to change
> APIs or remove misguided features.  This is accomplished through a gradual
> process over several releases:
>
> 1. Discuss the change.  Depending on the size of the incompatibility, this could
>    be on the bug tracker, python-dev, python-list, or the appropriate SIG.  A
>    PEP or similar document may be written.  Hopefully users of the affected API
>    will pipe up to comment.
>
> 2. Add a warning [#warnings_]_.  If behavior is changing, a the API may gain a
>    new function or method to perform the new behavior; old usage should raise
>    the warning.  If an API is being removed, simply warn whenever it is entered.
>    DeprecationWarning is the usual warning category to use, but
>    PendingDeprecationWarning may be used in special cases were the old and new
>    versions of the API will coexist for many releases.
>
> 3. Wait for a release.
>
> 4. See if there's any feedback.  Users not involved in the original discussions
>    may comment now after seeing the warning.  Perhaps reconsider.
>
> 5. The behavior change or feature removal may now be made default or permanent
>    in the next release.  Remove the old version and warning.
>
>
> References
> ==========
>
> .. [#tiobe] TIOBE Programming Community Index
>
>    http://www.tiobe.com/index.php/content/paperinfo/tpci/index.html
>
> .. [#warnings] The warnings module
>
>    http://docs.python.org/library/warnings.html
>
>
> Copyright
> =========
>
> This document has been placed in the public domain.
>
>
> 
> ..
>    Local Variables:
>    mode: indented-text
>    indent-tabs-mode: nil
>    sentence-end-double-space: t
>    fill-column: 70
>    coding: utf-8
>    End:
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk
>   


-- 
http://www.ironpythoninaction.com/
http://www.voidspace.org.uk/blog



From solipsis at pitrou.net  Fri Jun 19 13:22:10 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Fri, 19 Jun 2009 11:22:10 +0000 (UTC)
Subject: [Python-Dev] draft pep: backwards compatibility
References: <1afaf6160906181917m670663f5ucb21b8bc00c07735@mail.gmail.com>	<20090619093130.12555.864027812.divmod.xquotient.12879@weber.divmod.com>	<loom.20090619T091500-999@post.gmane.org>	<20090619112634.12555.1573592942.divmod.xquotient.12922@weber.divmod.com>
	<loom.20090619T110757-94@post.gmane.org>
	<4A3B732F.5080808@voidspace.org.uk>
Message-ID: <loom.20090619T111712-669@post.gmane.org>

Michael Foord <fuzzyman <at> voidspace.org.uk> writes:
> 
> And this is why expressing a finite list of things we guarantee won't 
> change is a virtually impossible task - unless one of you is 
> volunteering to write an official spec for Python and its libraries... 

Well, we *can* enumerate a list of things that are guaranteed not to change -
it's just that the list will be "completely incomplete", and the rest will rely
on application of common sense rather than formal guarantees.

And it actually more or less exists: it's Python's official documentation,
combined with a bit of common sense when interpreting its written text.

Regards

Antoine.



From lukepadawan at gmail.com  Fri Jun 19 13:29:43 2009
From: lukepadawan at gmail.com (Lucas P Melo)
Date: Fri, 19 Jun 2009 08:29:43 -0300
Subject: [Python-Dev] io.BufferedReader.peek() Behaviour in python3.1
In-Reply-To: <4A3AE0D3.2090708@canterbury.ac.nz>
References: <20090617012719.GA19974@cskk.homeip.net>	<4A399CEF.2070202@canterbury.ac.nz>
	<4A3A551A.6060304@gmail.com> <4A3AE0D3.2090708@canterbury.ac.nz>
Message-ID: <4A3B76A7.7060701@gmail.com>

Greg Ewing wrote:
> That's exactly why I think the blocking version should
> keep reading until the requested number of bytes is
> available (or the buffer is full or EOF occurs).
Do you mean that the blocking version should keep waiting for new bytes 
until they show up?
This would not be acceptable, since the program would hang forever most 
of the time (no changes to the buffer would ever occur in this situation 
when there's only the main thread running).

Am I understanding this correctly:
* The blocking version would not do any raw reads.
* The non-blocking version would do.

From glyph at divmod.com  Fri Jun 19 14:20:49 2009
From: glyph at divmod.com (glyph at divmod.com)
Date: Fri, 19 Jun 2009 12:20:49 -0000
Subject: [Python-Dev] draft pep: backwards compatibility
In-Reply-To: <loom.20090619T110757-94@post.gmane.org>
References: <1afaf6160906181917m670663f5ucb21b8bc00c07735@mail.gmail.com>
	<20090619093130.12555.864027812.divmod.xquotient.12879@weber.divmod.com>
	<loom.20090619T091500-999@post.gmane.org>
	<20090619112634.12555.1573592942.divmod.xquotient.12922@weber.divmod.com>
	<loom.20090619T110757-94@post.gmane.org>
Message-ID: <20090619122049.12555.1502244749.divmod.xquotient.12951@weber.divmod.com>


On 11:11 am, solipsis at pitrou.net wrote:
><glyph <at> divmod.com> writes:
>>
>>This is a false dichotomy; for core developers, the list needs to be
>>exhaustive.  Everything that can change needs to be described as 
>>either
>>compatible or incompatible.
>
>How do you enumerate "everything that can change"? It does not look 
>like a
>finite set to me (but perhaps I'm wrong); and certainly not like a set 
>of a size
>reasonable enough to be enumerated in a human-readable way :)

Sorry.  You're right.  To be clear, I mean, "everything that can change" 
as classified by some arbitrary classification system yet to be defined 
:-).

For example, the number of different programmatic entities in Python is 
pretty small: you've got classes, methods, modules, and constants.  The 
ways you can change them is also not too huge: you can add to them, 
remove them, or rename them.  I realize that given Python's flexibility 
when loading code, any system of such classification is going to have 
edge cases, but I doubt those are going to matter that much.

From glyph at divmod.com  Fri Jun 19 14:25:34 2009
From: glyph at divmod.com (glyph at divmod.com)
Date: Fri, 19 Jun 2009 12:25:34 -0000
Subject: [Python-Dev] draft pep: backwards compatibility
In-Reply-To: <4A3B72E3.3070106@voidspace.org.uk>
References: <1afaf6160906181917m670663f5ucb21b8bc00c07735@mail.gmail.com>
	<20090619093130.12555.864027812.divmod.xquotient.12879@weber.divmod.com>
	<loom.20090619T091500-999@post.gmane.org>
	<20090619112634.12555.1573592942.divmod.xquotient.12922@weber.divmod.com>
	<4A3B72E3.3070106@voidspace.org.uk>
Message-ID: <20090619122534.12555.2014907760.divmod.xquotient.12955@weber.divmod.com>

On 11:13 am, fuzzyman at voidspace.org.uk wrote:
>Just to note that Twisted (along with Bazaar) are one of the few 'good 
>citizens' in the Python community running their tests on Python trunk. 
>Both projects have caught incompatibilities *before* release of new 
>versions which is greatly preferable to discovering them after a 
>release. Thanks for this.

And thank *you* for being so responsive to the issues we've reported as 
a result of that.  It's definitely an incentive to remain engaged in 
core development :).

From benjamin at python.org  Fri Jun 19 15:56:12 2009
From: benjamin at python.org (Benjamin Peterson)
Date: Fri, 19 Jun 2009 08:56:12 -0500
Subject: [Python-Dev] draft pep: backwards compatibility
In-Reply-To: <loom.20090619T081720-465@post.gmane.org>
References: <1afaf6160906181917m670663f5ucb21b8bc00c07735@mail.gmail.com>
	<loom.20090619T081720-465@post.gmane.org>
Message-ID: <1afaf6160906190656x2b2243cerbe6e85d75973b209@mail.gmail.com>

2009/6/19 Antoine Pitrou <solipsis at pitrou.net>:
> Benjamin Peterson <benjamin <at> python.org> writes:
>>
>> This policy applys to all public APIs.
>
> applies?

Yes, thanks.
>
>> * The behavior of an API *must* not change between any two consecutive
>> releases.
>>
>> * A feature cannot be removed without notice between any two consecutive
>> ? releases.
>
> By induction, this would mean no API could change and no feature could be
> removed without notice between any N consecutive releases. Do you really mean
> it?

No, I'll reword.

>
>> * Addition of a feature which breaks 3rd party libraries or applications
>> should
>> ? have a large benefit to breakage ratio, and/or the incompatibility should
>> be
>> ? trival to fix in broken code.
>
> There is always the possibility that a new feature breaks existing code, for
> example because it relies on a similarly named attribute, or on some obscure
> internal condition. I think this should be qualified so that it only applies
> when e.g. a fair number of third-party apps or libraries are broken.

I add a few examples.



-- 
Regards,
Benjamin

From benjamin at python.org  Fri Jun 19 16:09:44 2009
From: benjamin at python.org (Benjamin Peterson)
Date: Fri, 19 Jun 2009 09:09:44 -0500
Subject: [Python-Dev] draft pep: backwards compatibility
In-Reply-To: <20090619093130.12555.864027812.divmod.xquotient.12879@weber.divmod.com>
References: <1afaf6160906181917m670663f5ucb21b8bc00c07735@mail.gmail.com>
	<20090619093130.12555.864027812.divmod.xquotient.12879@weber.divmod.com>
Message-ID: <1afaf6160906190709j260c1cc5l4b3e5cce4212742a@mail.gmail.com>

2009/6/19  <glyph at divmod.com>:
>
> On 02:17 am, benjamin at python.org wrote:
>>
>> Backwards compatibility seems to be an issue that arises a lot here. I
>> think we all have an idea of it is, but we need some hard place to
>> point to. So here's my attempt:
>
>> PEP: 387
>> Title: Backwards Compatibility Policy
>
> Thanks for getting started on this. ?This is indeed a bikeshed that there
> would be less discussion around if there were a clearer PEP to point to.
> ?However, this draft PEP points to the problem rather than the solution. ?In
> order to really be useful this needs to answer a whole slew of very very
> specific questions, because the real problem is that everybody thinks they
> know what they mean but in fact we all have slightly different definitions
> of these words.

I don't expect this document at all to be the definitive policy all at
once. I expect it will evolve and be filled in as it is applied in
real life.

>
> I've taken a stab at this myself in the past while defining a policy for
> Twisted, which is here:
> <http://twistedmatrix.com/trac/wiki/CompatibilityPolicy>. ?I think we might
> be a bit stricter than Python, but I believe our attempt to outline these
> terms specifically is worth taking a look at.

Yes, that's helpful. Thanks.

>
> The questions that follow might seem satirical or parodic but I am
> completely serious and each of these terms really does have a variable
> definition.

And will always no matter what we do. It's how natural language works.

>>
>> * The behavior of an API *must* not change between any two consecutive
>> releases.
>
> What is "behavior"? ?Software is composed of behavior. ?If no behavior
> changes, then nothing can ever happen.

I mean that if you pass X and Y into a function and get Z in 2.6, then
you should be able to get Z from passing X and Y in 2.7 even if
there's a new argument that returns Z' if you pass True to it.
(Obviously, this is somewhat simplified, but I hope it helps.)

>>
>> * A feature cannot be removed without notice between any two consecutive
>> ?releases.
>
> Presumably removal of a feature is a change in behavior, so isn't this
> redundant with the previous point?

I wanted to qualify differently from explicit behavior change as I
defined above in this message.

>>
>> * Addition of a feature which breaks 3rd party libraries or applications
>> should
>> ?have a large benefit to breakage ratio, and/or the incompatibility should
>> be
>> ?trival to fix in broken code.
>
> How do you propose to measure the benefit to breakage ratio? ?How do you
> propose to define "trivial" to fix? ?Most projects, including Python,
> Twisted, Django, and Zope, have an ever-increasing bug count, which means
> that trivial issues fall off the radar pretty easily.

Well, if you're tests aren't failing from it, is it an incompatibility?

>
> One of the big problems here in detecting and fixing "trivial" changes is
> that they can occur in test code as well. ?So if the answer is "run your
> tests", that's not much help if you can't call the actual APIs whose
> behavior has changed. ?The standard library would need to start providing a
> lot more verified test stubs for things like I/O in order for this to be
> feasible.
>>
>> Making Incompatible Changes
>> ===========================
>>
>> It's a fact: design mistakes happen. ?Thus it is important to be able to
>> change
>> APIs or remove misguided features. ?This is accomplished through a gradual
>> process over several releases:
>>
>> 1. Discuss the change. ?Depending on the size of the incompatibility, this
>> could
>> ?be on the bug tracker, python-dev, python-list, or the appropriate SIG.
>> ?A
>> ?PEP or similar document may be written. ?Hopefully users of the affected
>> API
>> ?will pipe up to comment.
>
> There are a very large number of users of Python, the large percentage of
> whom do not read python-dev. ?A posting on python-list is likely to provoke
> an unproductive pile-on. ?I suggest a dedicated list, which should ideally
> be low traffic, "python-incompatible-announce", or something like that, so
> that *everyone* who maintains Python software can subscribe to it, even if
> they're not otherwise interested in Python development.

And that won't generate a pile-on?

>> ?DeprecationWarning is the usual warning category to use, but
>> ?PendingDeprecationWarning may be used in special cases were the old and
>> new
>> ?versions of the API will coexist for many releases.
>
> Why is PendingDeprecationWarning the special case?

It should be used for things like string exceptions which are
deprecated but are not removed for a while.

>
> As outlined in the Twisted compatibility proposal, I'd prefer it if every
> warning went through multiple phases: pending, deprecated, gone: so that
> careful developers could release new versions of their software to quash
> noisy warnings *before* the new version of Python that actually made them
> user-visible was released.
>>
>> 3. Wait for a release.
>
> How does this affect the parallel 2.x/3.x release cycle?

I just added something about this.

>>
>> 4. See if there's any feedback. ?Users not involved in the original
>> discussions
>> ?may comment now after seeing the warning. ?Perhaps reconsider.
>
> At this point, many developers may already have been porting over to the
> new, non-deprecated API. ?I realize that there may be no way around that,
> but it's worth considering.
>
> Thanks for tackling this thorny problem. ?Good luck!

I think I'll need it. Thanks. :)



-- 
Regards,
Benjamin

From solipsis at pitrou.net  Fri Jun 19 16:15:23 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Fri, 19 Jun 2009 14:15:23 +0000 (UTC)
Subject: [Python-Dev] draft pep: backwards compatibility
References: <1afaf6160906181917m670663f5ucb21b8bc00c07735@mail.gmail.com>
	<20090619093130.12555.864027812.divmod.xquotient.12879@weber.divmod.com>
	<1afaf6160906190709j260c1cc5l4b3e5cce4212742a@mail.gmail.com>
Message-ID: <loom.20090619T141411-742@post.gmane.org>

Benjamin Peterson <benjamin <at> python.org> writes:
> 
> I mean that if you pass X and Y into a function and get Z in 2.6, then
> you should be able to get Z from passing X and Y in 2.7 even if
> there's a new argument that returns Z' if you pass True to it.

Well, except if returning Z rather than Z' was a bug.

Regards

Antoine.



From rdmurray at bitdance.com  Fri Jun 19 16:30:20 2009
From: rdmurray at bitdance.com (R. David Murray)
Date: Fri, 19 Jun 2009 10:30:20 -0400 (EDT)
Subject: [Python-Dev] draft pep: backwards compatibility
In-Reply-To: <loom.20090619T141411-742@post.gmane.org>
References: <1afaf6160906181917m670663f5ucb21b8bc00c07735@mail.gmail.com>
	<20090619093130.12555.864027812.divmod.xquotient.12879@weber.divmod.com>
	<1afaf6160906190709j260c1cc5l4b3e5cce4212742a@mail.gmail.com>
	<loom.20090619T141411-742@post.gmane.org>
Message-ID: <Pine.LNX.4.64.0906191022300.32513@kimball.webabinitio.net>

On Fri, 19 Jun 2009 at 14:15, Antoine Pitrou wrote:

> Benjamin Peterson <benjamin <at> python.org> writes:
>>
>> I mean that if you pass X and Y into a function and get Z in 2.6, then
>> you should be able to get Z from passing X and Y in 2.7 even if
>> there's a new argument that returns Z' if you pass True to it.
>
> Well, except if returning Z rather than Z' was a bug.

I'm pretty sure there have been cases of keeping buggy behavior in point
releases for backward compatibility reasons.  I think the decision has
depended on the nature, severity, and age of the bug, and the estimated
likelihood that code in the wild would break if the bug were fixed.

--David

From status at bugs.python.org  Fri Jun 19 18:07:03 2009
From: status at bugs.python.org (Python tracker)
Date: Fri, 19 Jun 2009 18:07:03 +0200 (CEST)
Subject: [Python-Dev] Summary of Python tracker Issues
Message-ID: <20090619160703.F35CD782C1@psf.upfronthosting.co.za>


ACTIVITY SUMMARY (06/12/09 - 06/19/09)
Python tracker at http://bugs.python.org/

To view or respond to any of the issues listed below, click on the issue 
number.  Do NOT respond to this message.


 2239 open (+26) / 15895 closed (+14) / 18134 total (+40)

Open issues with patches:   886

Average duration of open issues: 653 days.
Median duration of open issues: 404 days.

Open Issues Breakdown
   open  2210 (+25)
pending    29 ( +1)

Issues Created Or Reopened (40)
_______________________________

Add client side certificate support to httplib                   06/12/09
CLOSED http://bugs.python.org/issue6273    created  jnoller                       
       patch                                                                   

subprocess.Popen() may leak file descriptors                     06/12/09
       http://bugs.python.org/issue6274    created  facundobatista                
       patch                                                                   

let unittest.assertRaises() return the exception object caught   06/14/09
       http://bugs.python.org/issue6275    reopened krisvale                      
       patch, patch, easy, needs review                                        

Fix contextlib.nested DeprecationWarning for test_signal.        06/13/09
CLOSED http://bugs.python.org/issue6276    created  vshenoy                       
       patch                                                                   

Add description of new syntax of with to 2.7 whatsnew document.  06/13/09
CLOSED http://bugs.python.org/issue6277    created  vshenoy                       
       patch                                                                   

http.server, BaseHTTPRequestHandler write string error           06/13/09
CLOSED http://bugs.python.org/issue6278    created  System32                      
                                                                               

datamodel documentation confuses staticmethod with classmethod   06/13/09
CLOSED http://bugs.python.org/issue6279    created  segfaulthunter                
       patch                                                                   

calendar.timegm() belongs in time module, next to time.gmtime()  06/13/09
       http://bugs.python.org/issue6280    created  zooko                         
                                                                               

Bug in hashlib                                                   06/13/09
       http://bugs.python.org/issue6281    created  Eloff                         
                                                                               

In tarfile, compression level cannot be specified                06/14/09
CLOSED http://bugs.python.org/issue6282    created  Tzigi                         
                                                                               

Cannot build extension in amd64 using msvc9compiler              06/14/09
       http://bugs.python.org/issue6283    created  alexsh                        
                                                                               

C/API PyErr_AsUnicode()                                          06/14/09
       http://bugs.python.org/issue6284    created  ideasman42                    
       patch                                                                   

Silent abort on XP help document display                         06/14/09
       http://bugs.python.org/issue6285    created  scott_daniels                 
                                                                               

distutils upload command doesn't work with http proxy            06/15/09
       http://bugs.python.org/issue6286    created  tarek                         
                                                                               

distutils.core.setup does not document the 'license' meta-data   06/15/09
CLOSED http://bugs.python.org/issue6287    created  mattsmart                     
                                                                               

Update contextlib.nested docs in light of deprecation            06/17/09
       http://bugs.python.org/issue6288    reopened ncoghlan                      
                                                                               

compile() raises SyntaxError with confusing message for some dec 06/15/09
CLOSED http://bugs.python.org/issue6289    created  exarkun                       
                                                                               

cPickle can misread data type                                    06/15/09
       http://bugs.python.org/issue6290    created  ac.james                      
                                                                               

TypeError: b2a_base64() argument 1 must be bytes or buffer, not  06/16/09
CLOSED http://bugs.python.org/issue6291    created  milivojm                      
                                                                               

Fix tests to work with -OO                                       06/16/09
       http://bugs.python.org/issue6292    created  collinwinter                  
       patch, easy                                                             

Have regrtest.py echo back sys.flags                             06/16/09
       http://bugs.python.org/issue6293    created  collinwinter                  
       patch, easy                                                             

Improve shutdown exception ignored message                       06/17/09
       http://bugs.python.org/issue6294    created  tjreedy                       
       easy                                                                    

curses doc : getch is blocking by default                        06/17/09
CLOSED http://bugs.python.org/issue6295    created  nojhan                        
                                                                               

Native (and default) tarfile support for setup.py sdist	in distu 06/17/09
       http://bugs.python.org/issue6296    created  michael.foord                 
                                                                               

Added Misc/python.pc to 'distclean' Rule                         06/17/09
       http://bugs.python.org/issue6297    created  jcsalterego                   
       patch                                                                   

http://python.org/download says Python 2.4.5, but I think it mea 06/17/09
CLOSED http://bugs.python.org/issue6298    created  zooko                         
                                                                               

pyexpat build failure on Solaris 10 for 2.6.1/2.6.2              06/17/09
       http://bugs.python.org/issue6299    created  enchanter                     
                                                                               

encode and decode should accept 'errors' as a keyword argument   06/18/09
       http://bugs.python.org/issue6300    created  r.david.murray                
       easy                                                                    

Error in tutorial section 7.2                                    06/18/09
CLOSED http://bugs.python.org/issue6301    created  melbourne                     
                                                                               

email.header.decode_header data types are inconsistent and incor 06/18/09
       http://bugs.python.org/issue6302    created  r.david.murray                
                                                                               

print does not appear in the 3.x documentation index             06/18/09
CLOSED http://bugs.python.org/issue6303    created  r.david.murray                
                                                                               

Confusing error message when passing bytes to print with file po 06/18/09
       http://bugs.python.org/issue6304    created  r.david.murray                
                                                                               

islice doesn't accept large stop values                          06/18/09
       http://bugs.python.org/issue6305    created  thomasguest                   
                                                                               

filecmp.cmp can not compare two files from different OS with the 06/18/09
       http://bugs.python.org/issue6306    created  higer                         
                                                                               

AttributeError in traceback.print_last()                         06/18/09
CLOSED http://bugs.python.org/issue6307    created  complex                       
                                                                               

termios fix for QNX breaks HP-UX                                 06/18/09
       http://bugs.python.org/issue6308    created  haubi                         
                                                                               

Tix needs TCL package require statement                          06/19/09
       http://bugs.python.org/issue6309    created  eantelman                     
                                                                               

Windows "App Paths" key is not checked when installed for curren 06/19/09
       http://bugs.python.org/issue6310    created  techtonik                     
                                                                               

virtual memory error with archivemail                            06/19/09
       http://bugs.python.org/issue6311    created  helgekraak                    
                                                                               

httplib fails with HEAD requests to pages with "transfer-encodin 06/19/09
       http://bugs.python.org/issue6312    created  ezio.melotti                  
                                                                               



Issues Now Closed (29)
______________________

linecache .updatecache fails on utf8 encoded files                544 days
       http://bugs.python.org/issue1685    amaury.forgeotdarc            
       patch, easy                                                             

test_urllib2_localnet fails on MacOS X 10.4.11 (Intel)            332 days
       http://bugs.python.org/issue3407    orsenthil                     
                                                                               

Ill-formed surrogates not treated as errors during encoding/deco  295 days
       http://bugs.python.org/issue3672    hippietrail                   
       patch                                                                   

Local variables not freed when Exception raises in function call   76 days
       http://bugs.python.org/issue5641    georg.brandl                  
                                                                               

Unexpected universal newline behavior (newline duplication) in W   16 days
       http://bugs.python.org/issue6127    alexandre.vassalotti          
       patch                                                                   

subprocess module not compatible with python 2.5                   15 days
       http://bugs.python.org/issue6189    amaury.forgeotdarc            
       patch                                                                   

sdist doesn't include data_files                                   10 days
       http://bugs.python.org/issue6205    purpleidea                    
                                                                               

doctest_aliases doesn't test duplicate removal                      8 days
       http://bugs.python.org/issue6227    amaury.forgeotdarc            
       patch                                                                   

Python compiles dead code                                           6 days
       http://bugs.python.org/issue6250    loewis                        
       patch                                                                   

PyInt_FromSize_t is undocumented.                                   7 days
       http://bugs.python.org/issue6255    georg.brandl                  
                                                                               

distributions built with bdist_msi on 64-bit Windows fail	to ins    3 days
       http://bugs.python.org/issue6258    loewis                        
                                                                               

mmap: don't close file description if fd=-1                         2 days
       http://bugs.python.org/issue6271    ocean-city                    
       patch                                                                   

Add client side certificate support to httplib                      0 days
       http://bugs.python.org/issue6273    jnoller                       
       patch                                                                   

Fix contextlib.nested DeprecationWarning for test_signal.           6 days
       http://bugs.python.org/issue6276    georg.brandl                  
       patch                                                                   

Add description of new syntax of with to 2.7 whatsnew document.     3 days
       http://bugs.python.org/issue6277    rhettinger                    
       patch                                                                   

http.server, BaseHTTPRequestHandler write string error              2 days
       http://bugs.python.org/issue6278    amaury.forgeotdarc            
                                                                               

datamodel documentation confuses staticmethod with classmethod      1 days
       http://bugs.python.org/issue6279    segfaulthunter                
       patch                                                                   

In tarfile, compression level cannot be specified                   1 days
       http://bugs.python.org/issue6282    Tzigi                         
                                                                               

distutils.core.setup does not document the 'license' meta-data      0 days
       http://bugs.python.org/issue6287    tarek                         
                                                                               

compile() raises SyntaxError with confusing message for some dec    0 days
       http://bugs.python.org/issue6289    benjamin.peterson             
                                                                               

TypeError: b2a_base64() argument 1 must be bytes or buffer, not     0 days
       http://bugs.python.org/issue6291    r.david.murray                
                                                                               

curses doc : getch is blocking by default                           0 days
       http://bugs.python.org/issue6295    georg.brandl                  
                                                                               

http://python.org/download says Python 2.4.5, but I think it mea    0 days
       http://bugs.python.org/issue6298    georg.brandl                  
                                                                               

Error in tutorial section 7.2                                       1 days
       http://bugs.python.org/issue6301    pjenvey                       
                                                                               

print does not appear in the 3.x documentation index                0 days
       http://bugs.python.org/issue6303    r.david.murray                
                                                                               

AttributeError in traceback.print_last()                            0 days
       http://bugs.python.org/issue6307    r.david.murray                
                                                                               

current directory in sys.path handles symlinks badly             1665 days
       http://bugs.python.org/issue1074015 marko                         
                                                                               

enhance unittest to define tests that *should* fail              1253 days
       http://bugs.python.org/issue1399935 benjamin.peterson             
       patch                                                                   

keyword and topic help broken in Pythonwin IDE                   1126 days
       http://bugs.python.org/issue1489051 georg.brandl                  
                                                                               



Top Issues Most Discussed (10)
______________________________

 18 io.BufferedReader.peek(): Documentation differs from Implementa   58 days
open    http://bugs.python.org/issue5811   

 13 Backport the IO lib to trunk                                      14 days
open    http://bugs.python.org/issue6215   

  9 Add "daemon" argument to threading.Thread constructor             31 days
open    http://bugs.python.org/issue6064   

  8 Error in tutorial section 7.2                                      1 days
closed  http://bugs.python.org/issue6301   

  8 Update contextlib.nested docs in light of deprecation              2 days
open    http://bugs.python.org/issue6288   

  8 Python compiles dead code                                          6 days
closed  http://bugs.python.org/issue6250   

  7 Native (and default) tarfile support for setup.py sdist	in dist    2 days
open    http://bugs.python.org/issue6296   

  6 Add description of new syntax of with to 2.7 whatsnew document.    3 days
closed  http://bugs.python.org/issue6277   

  6 let unittest.assertRaises() return the exception object caught     5 days
open    http://bugs.python.org/issue6275   

  6 path separator output ignores shell's path separator: /	instead   14 days
open    http://bugs.python.org/issue6208   




From tseaver at palladion.com  Fri Jun 19 19:47:22 2009
From: tseaver at palladion.com (Tres Seaver)
Date: Fri, 19 Jun 2009 13:47:22 -0400
Subject: [Python-Dev] draft pep: backwards compatibility
In-Reply-To: <loom.20090619T081720-465@post.gmane.org>
References: <1afaf6160906181917m670663f5ucb21b8bc00c07735@mail.gmail.com>
	<loom.20090619T081720-465@post.gmane.org>
Message-ID: <h1givm$na5$1@ger.gmane.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Antoine Pitrou wrote:

> There is always the possibility that a new feature breaks existing code, for
> example because it relies on a similarly named attribute, or on some obscure
> internal condition. I think this should be qualified so that it only applies
> when e.g. a fair number of third-party apps or libraries are broken.

I think the recent asyncore changes are a good test case here:  they
broke major consumers of the package (Zope, supervisord), but it wasn't
obvious that the breakage would occur, because the API of the package
wasn't clear:  the apps broken by the change were forced to rely on
stuff that the subsequent maintainer considered "implementation details."


Tres.
- --
===================================================================
Tres Seaver          +1 540-429-0999          tseaver at palladion.com
Palladion Software   "Excellence by Design"    http://palladion.com
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFKO88q+gerLs4ltQ4RAmmkAJ4ghgszBWGruCqONtouAXp82G+blgCgySsL
4ywgpU5137D5isiJ8+d6KtM=
=OksB
-----END PGP SIGNATURE-----


From glyph at divmod.com  Fri Jun 19 20:44:28 2009
From: glyph at divmod.com (glyph at divmod.com)
Date: Fri, 19 Jun 2009 18:44:28 -0000
Subject: [Python-Dev] draft pep: backwards compatibility
In-Reply-To: <1afaf6160906190709j260c1cc5l4b3e5cce4212742a@mail.gmail.com>
References: <1afaf6160906181917m670663f5ucb21b8bc00c07735@mail.gmail.com>
	<20090619093130.12555.864027812.divmod.xquotient.12879@weber.divmod.com>
	<1afaf6160906190709j260c1cc5l4b3e5cce4212742a@mail.gmail.com>
Message-ID: <20090619184428.12555.1883394051.divmod.xquotient.12994@weber.divmod.com>

On 02:09 pm, benjamin at python.org wrote:
>2009/6/19  <glyph at divmod.com>:
>>
>>On 02:17 am, benjamin at python.org wrote:

>>I've taken a stab at this myself in the past while defining a policy 
>>for
>>Twisted

>Yes, that's helpful. Thanks.

Glad you found it useful.
>>The questions that follow might seem satirical or parodic but I am
>>completely serious and each of these terms really does have a variable
>>definition.
>
>And will always no matter what we do. It's how natural language works.

Well, one cannot proceed from the informal to the formal by formal 
means.  I'm pretty sure we can nail down the definitions of these terms 
for the scope of Python core development.
>>What is "behavior"? ?Software is composed of behavior. ?If no behavior
>>changes, then nothing can ever happen.

>I mean that if you pass X and Y into a function and get Z in 2.6, then
>you should be able to get Z from passing X and Y in 2.7 even if
>there's a new argument that returns Z' if you pass True to it.
>(Obviously, this is somewhat simplified, but I hope it helps.)

This definition only makes sense if the interesting thing about a 
function is its return value, and if the only sort of thing you have are 
functions.  What about side-effects, or exceptional conditions?  What 
about interactions with subclasses?  (Changing a class in a library from 
old-style to new-style has serious repercussions, as MRO conflicts can 
result in applications that subclass it.)
>>How do you propose to measure the benefit to breakage ratio? ?How do 
>>you
>>propose to define "trivial" to fix? ?Most projects, including Python,
>>Twisted, Django, and Zope, have an ever-increasing bug count, which 
>>means
>>that trivial issues fall off the radar pretty easily.
>
>Well, if you're tests aren't failing from it, is it an incompatibility?

Well, let's say the tests do fail from it.  Right now, even Twisted 
trunk still doesn't *quite* support Python 2.6, but only on Windows, due 
to stricter checking of the 'mode' argument for files.  But this failing 
test is just not that high priority, so our recommendation is "don't use 
python 2.6 yet on Windows, 2.5 works fine".  My point is that triviality 
is a matter of perspective :).  Eventually somebody will get around to 
it, but 2.6 has been out for a while now.
>>There are a very large number of users of Python, the large percentage 
>>of
>>whom do not read python-dev. ?A posting on python-list is likely to 
>>provoke
>>an unproductive pile-on. ?I suggest a dedicated list, which should 
>>ideally
>>be low traffic, "python-incompatible-announce", or something like 
>>that, so
>>that *everyone* who maintains Python software can subscribe to it, 
>>even if
>>they're not otherwise interested in Python development.
>
>And that won't generate a pile-on?

Well, the etiquette for that specific list could be "keep this low- 
traffic unless you have a serious problem with this change".  Or, better 
yet, make it an announce-only list: the pile-on can still happen on 
python-list, but only the results of the discussion will be announced on 
the incompatible-announce list.

The point is, the notifications about compatibility are really 
important, and sometimes people need to offer feedback about them, but 
not everyone who needs to know about the changes needs to hear about the 
feedback.

From python at rcn.com  Fri Jun 19 21:06:27 2009
From: python at rcn.com (Raymond Hettinger)
Date: Fri, 19 Jun 2009 12:06:27 -0700
Subject: [Python-Dev] draft pep: backwards compatibility
References: <1afaf6160906181917m670663f5ucb21b8bc00c07735@mail.gmail.com>
Message-ID: <8004914DB3B9496BB3DC1A1BFAF4D572@RaymondLaptop1>

Not sure why we need yet another pep on the subject.  Just update PEP 5 if needed.

Also, I think there is a certain amount of wishful thinking here.  Ideally, we could approve a tiny PEP with just a few bullet 
points and it would eliminate the need for all of the complicated decision making that we usually go through.  Ideally, we could 
make all decisions in advance of seeing the facts for a given situation.  ISTM, this pep is wishing away the actual complexity of 
making software design decisions.

The policy for 2.x should probably be different than for 3.x.  ISTM that 3.x has not been fully shaken out and that possibly many 
things will need to change as users start to report problems.  The text vs bytes issue is lurking throughout the release.  The JSON 
module in particular was affected by a half thought out port to 3.0.  And yesterday on #python IRC, one developer reported the email 
package in 3.1 to be unusable and one of its maintainers characterized it as being in need of a serious overhaul (meaning major API 
changes).  In the end, there is going to have to be some thoughtful balancing between making the needed changes and not hurting the 
existing users.  I don't think a small, general purpose PEP like this one can wish that away.

Another sticking point about preserving features across releases arises if the feature itself is hazardous in some way (like have a 
security hole or some such).  The contextlib.nested() function was an example.  It didn't ever really work as advertised for its 
intended purpose (it wasn't truly equivalent to two nested with-statements) and it presented users with the possibility of 
hard-to-spot bugs.  The bugfix for it was to replace it with new syntax.  Unfortunately, the new syntax didn't provide all of the 
functionality of the original.  So, the question arises about whether this particular mine should be left on the battlefield.  We 
resolved the question after a long and thoughtful discussion; I don't think that decision making process could have been solved in 
advance by a bullet-point in a general purpose process PEP.


Raymond


From benjamin at python.org  Fri Jun 19 21:37:45 2009
From: benjamin at python.org (Benjamin Peterson)
Date: Fri, 19 Jun 2009 14:37:45 -0500
Subject: [Python-Dev] draft pep: backwards compatibility
In-Reply-To: <8004914DB3B9496BB3DC1A1BFAF4D572@RaymondLaptop1>
References: <1afaf6160906181917m670663f5ucb21b8bc00c07735@mail.gmail.com>
	<8004914DB3B9496BB3DC1A1BFAF4D572@RaymondLaptop1>
Message-ID: <1afaf6160906191237q67860926m8c1d3a8fa441d8e2@mail.gmail.com>

2009/6/19 Raymond Hettinger <python at rcn.com>:
> Not sure why we need yet another pep on the subject. ?Just update PEP 5 if
> needed.

Hmm. I didn't know about that one.

>
> Also, I think there is a certain amount of wishful thinking here. ?Ideally,
> we could approve a tiny PEP with just a few bullet points and it would
> eliminate the need for all of the complicated decision making that we
> usually go through. ?Ideally, we could make all decisions in advance of
> seeing the facts for a given situation. ?ISTM, this pep is wishing away the
> actual complexity of making software design decisions.

I think it should just serve as a backbone and basis for decisions. It
will certainly not eliminate any complex thinking, but hopefully it
will

More importantly, it shows to the community what our policies are and
what they should expect. Exceptions should be exceptional not the
norm.

>
> The policy for 2.x should probably be different than for 3.x. ?ISTM that 3.x
> has not been fully shaken out and that possibly many things will need to
> change as users start to report problems. ?The text vs bytes issue is
> lurking throughout the release. ?The JSON module in particular was affected
> by a half thought out port to 3.0. ?And yesterday on #python IRC, one
> developer reported the email package in 3.1 to be unusable and one of its
> maintainers characterized it as being in need of a serious overhaul (meaning
> major API changes). ?In the end, there is going to have to be some
> thoughtful balancing between making the needed changes and not hurting the
> existing users. ?I don't think a small, general purpose PEP like this one
> can wish that away.

That is very unfortunate. 2.7 may be the last 2.x release, so this PEP
primarily applies to 3.x anyway. We could add a provision for
experimental or unstable modules, but I don't think those should be
released in the first place.

>
> Another sticking point about preserving features across releases arises if
> the feature itself is hazardous in some way (like have a security hole or
> some such). ?The contextlib.nested() function was an example. ?It didn't
> ever really work as advertised for its intended purpose (it wasn't truly
> equivalent to two nested with-statements) and it presented users with the
> possibility of hard-to-spot bugs. ?The bugfix for it was to replace it with
> new syntax. ?Unfortunately, the new syntax didn't provide all of the
> functionality of the original. ?So, the question arises about whether this
> particular mine should be left on the battlefield. ?We resolved the question
> after a long and thoughtful discussion; I don't think that decision making
> process could have been solved in advance by a bullet-point in a general
> purpose process PEP.

I would say that's pretty close to the procedure in the PEP actually:
"Discuss the change. This may be on python-dev, the tracker...."


-- 
Regards,
Benjamin

From g.brandl at gmx.net  Fri Jun 19 22:13:12 2009
From: g.brandl at gmx.net (Georg Brandl)
Date: Fri, 19 Jun 2009 20:13:12 +0000
Subject: [Python-Dev] draft pep: backwards compatibility
In-Reply-To: <1afaf6160906181917m670663f5ucb21b8bc00c07735@mail.gmail.com>
References: <1afaf6160906181917m670663f5ucb21b8bc00c07735@mail.gmail.com>
Message-ID: <h1grgu$ni7$1@ger.gmane.org>

Benjamin Peterson schrieb:
> Backwards compatibility seems to be an issue that arises a lot here. I
> think we all have an idea of it is, but we need some hard place to
> point to. So here's my attempt:

Yet another rather pointless bikeshed: if this becomes policy, maybe it
should get a PEP number < 100, like PEP 5 or 6.

Georg

-- 
Thus spake the Lord: Thou shalt indent with four spaces. No more, no less.
Four shall be the number of spaces thou shalt indent, and the number of thy
indenting shall be four. Eight shalt thou not indent, nor either indent thou
two, excepting that thou then proceed to four. Tabs are right out.


From benjamin at python.org  Fri Jun 19 22:40:54 2009
From: benjamin at python.org (Benjamin Peterson)
Date: Fri, 19 Jun 2009 15:40:54 -0500
Subject: [Python-Dev] draft pep: backwards compatibility
In-Reply-To: <h1grgu$ni7$1@ger.gmane.org>
References: <1afaf6160906181917m670663f5ucb21b8bc00c07735@mail.gmail.com>
	<h1grgu$ni7$1@ger.gmane.org>
Message-ID: <1afaf6160906191340pbedf3abn2bccb185bdddf15b@mail.gmail.com>

2009/6/19 Georg Brandl <g.brandl at gmx.net>:
> Benjamin Peterson schrieb:
>> Backwards compatibility seems to be an issue that arises a lot here. I
>> think we all have an idea of it is, but we need some hard place to
>> point to. So here's my attempt:
>
> Yet another rather pointless bikeshed: if this becomes policy, maybe it
> should get a PEP number < 100, like PEP 5 or 6.

The point is to avoid a little bike shedding later.



-- 
Regards,
Benjamin

From g.brandl at gmx.net  Fri Jun 19 22:58:04 2009
From: g.brandl at gmx.net (Georg Brandl)
Date: Fri, 19 Jun 2009 22:58:04 +0200
Subject: [Python-Dev] draft pep: backwards compatibility
In-Reply-To: <Pine.LNX.4.64.0906191022300.32513@kimball.webabinitio.net>
References: <1afaf6160906181917m670663f5ucb21b8bc00c07735@mail.gmail.com>	<20090619093130.12555.864027812.divmod.xquotient.12879@weber.divmod.com>	<1afaf6160906190709j260c1cc5l4b3e5cce4212742a@mail.gmail.com>	<loom.20090619T141411-742@post.gmane.org>
	<Pine.LNX.4.64.0906191022300.32513@kimball.webabinitio.net>
Message-ID: <h1gu4s$v07$1@ger.gmane.org>

R. David Murray schrieb:
> On Fri, 19 Jun 2009 at 14:15, Antoine Pitrou wrote:
> 
>> Benjamin Peterson <benjamin <at> python.org> writes:
>>>
>>> I mean that if you pass X and Y into a function and get Z in 2.6, then
>>> you should be able to get Z from passing X and Y in 2.7 even if
>>> there's a new argument that returns Z' if you pass True to it.
>>
>> Well, except if returning Z rather than Z' was a bug.
> 
> I'm pretty sure there have been cases of keeping buggy behavior in point
> releases for backward compatibility reasons.  I think the decision has
> depended on the nature, severity, and age of the bug, and the estimated
> likelihood that code in the wild would break if the bug were fixed.

That is always a difficult issue. There are tons of issues in the tracker
that would be quite easy to fix, but are not touched because nobody wants
to take the blame if it is considered "not buggy enough" for an incompatible
change. But they won't be closed either, because the current behavior
clearly is wrong.

Georg

-- 
Thus spake the Lord: Thou shalt indent with four spaces. No more, no less.
Four shall be the number of spaces thou shalt indent, and the number of thy
indenting shall be four. Eight shalt thou not indent, nor either indent thou
two, excepting that thou then proceed to four. Tabs are right out.


From benjamin at python.org  Fri Jun 19 23:08:24 2009
From: benjamin at python.org (Benjamin Peterson)
Date: Fri, 19 Jun 2009 16:08:24 -0500
Subject: [Python-Dev] draft pep: backwards compatibility
In-Reply-To: <20090619184428.12555.1883394051.divmod.xquotient.12994@weber.divmod.com>
References: <1afaf6160906181917m670663f5ucb21b8bc00c07735@mail.gmail.com>
	<20090619093130.12555.864027812.divmod.xquotient.12879@weber.divmod.com>
	<1afaf6160906190709j260c1cc5l4b3e5cce4212742a@mail.gmail.com>
	<20090619184428.12555.1883394051.divmod.xquotient.12994@weber.divmod.com>
Message-ID: <1afaf6160906191408i49b346eegc9b40fae0cf69576@mail.gmail.com>

2009/6/19  <glyph at divmod.com>:
> On 02:09 pm, benjamin at python.org wrote:
>> 2009/6/19 ?<glyph at divmod.com>:
>>> What is "behavior"? ?Software is composed of behavior. ?If no behavior
>>> changes, then nothing can ever happen.
>
>> I mean that if you pass X and Y into a function and get Z in 2.6, then
>> you should be able to get Z from passing X and Y in 2.7 even if
>> there's a new argument that returns Z' if you pass True to it.
>> (Obviously, this is somewhat simplified, but I hope it helps.)
>
> This definition only makes sense if the interesting thing about a function
> is its return value, and if the only sort of thing you have are functions.
> ?What about side-effects, or exceptional conditions? ?What about
> interactions with subclasses? ?(Changing a class in a library from old-style
> to new-style has serious repercussions, as MRO conflicts can result in
> applications that subclass it.)

I've added a little more about this to the PEP. See what you think.

>>>
>>> How do you propose to measure the benefit to breakage ratio? ?How do you
>>> propose to define "trivial" to fix? ?Most projects, including Python,
>>> Twisted, Django, and Zope, have an ever-increasing bug count, which means
>>> that trivial issues fall off the radar pretty easily.
>>
>> Well, if you're tests aren't failing from it, is it an incompatibility?
>
> Well, let's say the tests do fail from it. ?Right now, even Twisted trunk
> still doesn't *quite* support Python 2.6, but only on Windows, due to
> stricter checking of the 'mode' argument for files. ?But this failing test
> is just not that high priority, so our recommendation is "don't use python
> 2.6 yet on Windows, 2.5 works fine". ?My point is that triviality is a
> matter of perspective :). ?Eventually somebody will get around to it, but
> 2.6 has been out for a while now.

I understand. I think we will just have to provide guidelines for
triviality and decide on a case by case basis.


>>> There are a very large number of users of Python, the large percentage of
>>> whom do not read python-dev. ?A posting on python-list is likely to
>>> provoke
>>> an unproductive pile-on. ?I suggest a dedicated list, which should
>>> ideally
>>> be low traffic, "python-incompatible-announce", or something like that,
>>> so
>>> that *everyone* who maintains Python software can subscribe to it, even
>>> if
>>> they're not otherwise interested in Python development.
>>
>> And that won't generate a pile-on?
>
> Well, the etiquette for that specific list could be "keep this low- traffic
> unless you have a serious problem with this change". ?Or, better yet, make
> it an announce-only list: the pile-on can still happen on python-list, but
> only the results of the discussion will be announced on the
> incompatible-announce list.

I think that's a fine idea, but the PEP is dealing with policy as our
mailing list infrastructure is now.


-- 
Regards,
Benjamin

From aahz at pythoncraft.com  Sat Jun 20 00:07:07 2009
From: aahz at pythoncraft.com (Aahz)
Date: Fri, 19 Jun 2009 15:07:07 -0700
Subject: [Python-Dev] draft pep: backwards compatibility
In-Reply-To: <h1grgu$ni7$1@ger.gmane.org>
References: <1afaf6160906181917m670663f5ucb21b8bc00c07735@mail.gmail.com>
	<h1grgu$ni7$1@ger.gmane.org>
Message-ID: <20090619220707.GA4300@panix.com>

On Fri, Jun 19, 2009, Georg Brandl wrote:
> 
> Yet another rather pointless bikeshed: if this becomes policy, maybe it
> should get a PEP number < 100, like PEP 5 or 6.

+1 -- I was debating whether to say something.
-- 
Aahz (aahz at pythoncraft.com)           <*>         http://www.pythoncraft.com/

"as long as we like the same operating system, things are cool." --piranha

From benjamin at python.org  Sat Jun 20 00:15:31 2009
From: benjamin at python.org (Benjamin Peterson)
Date: Fri, 19 Jun 2009 17:15:31 -0500
Subject: [Python-Dev] draft pep: backwards compatibility
In-Reply-To: <20090619220707.GA4300@panix.com>
References: <1afaf6160906181917m670663f5ucb21b8bc00c07735@mail.gmail.com>
	<h1grgu$ni7$1@ger.gmane.org> <20090619220707.GA4300@panix.com>
Message-ID: <1afaf6160906191515x6b144085n64422aad18eade84@mail.gmail.com>

2009/6/19 Aahz <aahz at pythoncraft.com>:
> On Fri, Jun 19, 2009, Georg Brandl wrote:
>>
>> Yet another rather pointless bikeshed: if this becomes policy, maybe it
>> should get a PEP number < 100, like PEP 5 or 6.
>
> +1 -- I was debating whether to say something.

Is that a +1 to numbering or bike shedding?


-- 
Regards,
Benjamin

From glyph at divmod.com  Sat Jun 20 00:40:36 2009
From: glyph at divmod.com (glyph at divmod.com)
Date: Fri, 19 Jun 2009 22:40:36 -0000
Subject: [Python-Dev] draft pep: backwards compatibility
In-Reply-To: <8004914DB3B9496BB3DC1A1BFAF4D572@RaymondLaptop1>
References: <1afaf6160906181917m670663f5ucb21b8bc00c07735@mail.gmail.com>
	<8004914DB3B9496BB3DC1A1BFAF4D572@RaymondLaptop1>
Message-ID: <20090619224036.12555.889474637.divmod.xquotient.13046@weber.divmod.com>

On 07:06 pm, python at rcn.com wrote:
>Not sure why we need yet another pep on the subject.  Just update PEP 5 
>if needed.

Hmm.  This is a good point; it might make sense to have a more specific 
PEP for library compatibility as opposed to language compatibility 
though, since language compatibility is necessarily a vaguer concept.
>Also, I think there is a certain amount of wishful thinking here. 
>Ideally, we could approve a tiny PEP with just a few bullet points and 
>it would eliminate the need for all of the complicated decision making 
>that we usually go through.  Ideally, we could make all decisions in 
>advance of seeing the facts for a given situation.  ISTM, this pep is 
>wishing away the actual complexity of making software design decisions.

This is not about making design decisions.  This is about how to treat 
the *output* of design decisions.

A really important part of this PEP, as I alluded to previously, is the 
part that tells developers what *they* should be doing if they want 
their python code to function on the next release of the interpreter.

Right now, the rule to write software that will not break with the next 
Python release is "read the minds of the python core committers and hope 
that you do not do anything with the stdlib that they don't like". 
Along with this there are several rules you can infer that are probably 
true most of the time: don't call stuff starting with "_", don't monkey- 
patch anything, don't use dynamic class replacement on objects from 
classes other than your own, don't depend on the depth of inheritance 
hierarchies (for example, no ".__bases__[0].__bases__[0]"), make sure 
your tests run without producing any DeprecationWarnings, be mindful of 
potential namespace conflicts when adding attributes to classes that 
inherit from other libraries.  I don't think all these things are 
written down in one place though.
>Another sticking point about preserving features across releases arises 
>if the feature itself is hazardous in some way (like have a security 
>hole or some such).  The contextlib.nested() function was an example. 
>(...)
>We resolved the question after a long and thoughtful discussion; I 
>don't think that decision making process could have been solved in 
>advance by a bullet-point in a general purpose process PEP.

You are correct that nothing in Python's policy could have dictated how 
this problem could be resolved.  However, the policy can most definitely 
state how to deal with code using contextlib.nested in the interim 
before it has been changed to use the new syntax: to wit, that 
contextlib.nested has to stick around, continue to do the (broken) thing 
that it did before, and emit a DeprecationWarning indicating the new 
syntax.  The existing policy in PEP 5 already does this, but doesn't 
offer guidelines on *how* to do this for lots of different types of 
changes.  For example, how do you issue a deprecation warning for a new 
parameter you want to require application code to accept?  How do you 
change the name of a parameter, to wit, do you need to expect that all 
arguments can validly be used as keyword arguments?  How do you 
determine an appropriate stack-depth, etc?

From aahz at pythoncraft.com  Sat Jun 20 00:26:30 2009
From: aahz at pythoncraft.com (Aahz)
Date: Fri, 19 Jun 2009 15:26:30 -0700
Subject: [Python-Dev] draft pep: backwards compatibility
In-Reply-To: <1afaf6160906191515x6b144085n64422aad18eade84@mail.gmail.com>
References: <1afaf6160906181917m670663f5ucb21b8bc00c07735@mail.gmail.com>
	<h1grgu$ni7$1@ger.gmane.org> <20090619220707.GA4300@panix.com>
	<1afaf6160906191515x6b144085n64422aad18eade84@mail.gmail.com>
Message-ID: <20090619222630.GA14341@panix.com>

On Fri, Jun 19, 2009, Benjamin Peterson wrote:
> 2009/6/19 Aahz <aahz at pythoncraft.com>:
>> On Fri, Jun 19, 2009, Georg Brandl wrote:
>>>
>>> Yet another rather pointless bikeshed: if this becomes policy, maybe it
>>> should get a PEP number < 100, like PEP 5 or 6.
>>
>> +1 -- I was debating whether to say something.
> 
> Is that a +1 to numbering or bike shedding?

+1 to changing the PEP number
-- 
Aahz (aahz at pythoncraft.com)           <*>         http://www.pythoncraft.com/

"as long as we like the same operating system, things are cool." --piranha

From martin at v.loewis.de  Sat Jun 20 18:27:22 2009
From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=)
Date: Sat, 20 Jun 2009 18:27:22 +0200
Subject: [Python-Dev] xmlrpc improvements
In-Reply-To: <930F189C8A437347B80DF2C156F7EC7F057F9DDAF3@exchis.ccp.ad.local>
References: <930F189C8A437347B80DF2C156F7EC7F057F9DDAF3@exchis.ccp.ad.local>
Message-ID: <4A3D0DEA.50002@v.loewis.de>

> I?d really  like to get this stuff in.  The performance gains allowing
> http1.1 and gzip for xmlrpc are quite significant.

I think you really need to get Fredrik Lundh to comment on it. If he
can't predict when he'll be able to review the changes, maybe he can
accept releasing control of xmlrpclib.

Regards,
Martin

From fredrik at pythonware.com  Sat Jun 20 18:34:31 2009
From: fredrik at pythonware.com (Fredrik Lundh)
Date: Sat, 20 Jun 2009 18:34:31 +0200
Subject: [Python-Dev] xmlrpc improvements
In-Reply-To: <4A3D0DEA.50002@v.loewis.de>
References: <930F189C8A437347B80DF2C156F7EC7F057F9DDAF3@exchis.ccp.ad.local> 
	<4A3D0DEA.50002@v.loewis.de>
Message-ID: <368a5cd50906200934n5b15a1aboac83ad1d32f5c3bb@mail.gmail.com>

2009/6/20 "Martin v. L?wis" <martin at v.loewis.de>:
>> I?d really ?like to get this stuff in. ?The performance gains allowing
>> http1.1 and gzip for xmlrpc are quite significant.
>
> I think you really need to get Fredrik Lundh to comment on it. If he
> can't predict when he'll be able to review the changes, maybe he can
> accept releasing control of xmlrpclib.

Pointer to the patch?

</F>

From martin at v.loewis.de  Sat Jun 20 18:57:20 2009
From: martin at v.loewis.de (=?windows-1252?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sat, 20 Jun 2009 18:57:20 +0200
Subject: [Python-Dev] xmlrpc improvements
In-Reply-To: <368a5cd50906200934n5b15a1aboac83ad1d32f5c3bb@mail.gmail.com>
References: <930F189C8A437347B80DF2C156F7EC7F057F9DDAF3@exchis.ccp.ad.local>
	<4A3D0DEA.50002@v.loewis.de>
	<368a5cd50906200934n5b15a1aboac83ad1d32f5c3bb@mail.gmail.com>
Message-ID: <4A3D14F0.7080100@v.loewis.de>

Fredrik Lundh wrote:
>> I think you really need to get Fredrik Lundh to comment on it. If he
>> can't predict when he'll be able to review the changes, maybe he can
>> accept releasing control of xmlrpclib.
> 
> Pointer to the patch?

http://bugs.python.org/issue6267

While I have your attention, please also review

http://bugs.python.org/issue6233

Regards,
Martin

From glyph at divmod.com  Sat Jun 20 19:55:28 2009
From: glyph at divmod.com (glyph at divmod.com)
Date: Sat, 20 Jun 2009 17:55:28 -0000
Subject: [Python-Dev] draft pep: backwards compatibility
In-Reply-To: <1afaf6160906191408i49b346eegc9b40fae0cf69576@mail.gmail.com>
References: <1afaf6160906181917m670663f5ucb21b8bc00c07735@mail.gmail.com>
	<20090619093130.12555.864027812.divmod.xquotient.12879@weber.divmod.com>
	<1afaf6160906190709j260c1cc5l4b3e5cce4212742a@mail.gmail.com>
	<20090619184428.12555.1883394051.divmod.xquotient.12994@weber.divmod.com>
	<1afaf6160906191408i49b346eegc9b40fae0cf69576@mail.gmail.com>
Message-ID: <20090620175528.12555.830517954.divmod.xquotient.13310@weber.divmod.com>


On 19 Jun, 09:08 pm, benjamin at python.org wrote:
>2009/6/19  <glyph at divmod.com>:
>>On 02:09 pm, benjamin at python.org wrote:
>>>2009/6/19 ?<glyph at divmod.com>:

>>?What about side-effects, or exceptional conditions? ?What about
>>interactions with subclasses? ?(Changing a class in a library from 
>>old-style
>>to new-style has serious repercussions, as MRO conflicts can result in
>>applications that subclass it.)

>I've added a little more about this to the PEP. See what you think.

Finally had a chance to take a look.  It's a big improvement, certainly: 
much more specific.  Although I think I have a few quibbles with the 
wording.

For one thing, I don't like the use of the word "reasonable".  Everybody 
thinks *their* exception to the rules is reasonable, but nobody else's 
is.  Besides, if the BDFL thinks a change is really reasonable, do you 
think a PEP is going to stop him? :)

For another, I think it's actually a bit too strict, as worded.  The 
side-effects of a function includes the warnings that it emits; any 
method-call can have side-effects, so you can't change an algorithm *at 
all*.  The only side-effects that I think are important are the ones 
that get invoked on objects that an application gets to inject 
somewhere, or inspect later via a public API.

The word "releases" also needs to be qualified.  Most importantly, 
minor-version and micro-version releases need to be described 
distinctly.

Finally, the PEP should mention its participation in the universe of 
compatibility process PEPs.  It should describe its relationship to at 
least some of PEP 3002, 291, 5, 6, 2, 3001, and 384.
>>My point is that triviality is a matter of perspective :).

>I understand. I think we will just have to provide guidelines for
>triviality and decide on a case by case basis.

A simple litmus test, or some examples, of triviality would be helpful.
>>the pile-on can still happen on python-list, but
>>only the results of the discussion will be announced on the
>>incompatible-announce list.
>
>I think that's a fine idea, but the PEP is dealing with policy as our
>mailing list infrastructure is now.

Hmm... well, maybe everybody should just run their tests against Python 
trunk.  The commits list is a reliable notification mechanism for 
potentially incompatible changes ;-).  Perhaps it would be good to 
mention this specifically in the PEP?  For example, "third party 
projects may request that an incompatible change be reverted, by 
providing evidence of tests failing on x.y+1 that passed on x.y, where 
the code in question does not rely on any details not specified as 
'public' in the section above"?

If more projects did this when there *was* a problem, then it would 
actually be a lot easier to break the policy with confidence. With an 
incompatible change, you could know, "we checked it in, and nobody 
complained".  Whereas right now is nobody complains it's more likely 
that nobody is paying attention.

From collinw at gmail.com  Sat Jun 20 19:49:59 2009
From: collinw at gmail.com (Collin Winter)
Date: Sat, 20 Jun 2009 10:49:59 -0700
Subject: [Python-Dev] draft pep: backwards compatibility
In-Reply-To: <1afaf6160906181917m670663f5ucb21b8bc00c07735@mail.gmail.com>
References: <1afaf6160906181917m670663f5ucb21b8bc00c07735@mail.gmail.com>
Message-ID: <43aa6ff70906201049q26add809sa5893c470cc520d6@mail.gmail.com>

On Thu, Jun 18, 2009 at 7:17 PM, Benjamin Peterson<benjamin at python.org> wrote:
[snip]
> Backwards Compatibility Rules
> =============================
>
> This policy applys to all public APIs. ?These include the C-API, the standard
> library, and the core language including syntax and operation as defined by the
> reference manual.
>
> This is the basic policy for backwards compatibility:
>
> * The behavior of an API *must* not change between any two consecutive releases.

Is this intended to include performance changes? Clearly no-one will
complain if things simply get faster, but I'm thinking about cases
where, say, a function runs in half the time but uses double the
memory (or vice versa).

Collin

From tjreedy at udel.edu  Sat Jun 20 21:02:38 2009
From: tjreedy at udel.edu (Terry Reedy)
Date: Sat, 20 Jun 2009 15:02:38 -0400
Subject: [Python-Dev] draft pep: backwards compatibility
In-Reply-To: <20090619224036.12555.889474637.divmod.xquotient.13046@weber.divmod.com>
References: <1afaf6160906181917m670663f5ucb21b8bc00c07735@mail.gmail.com>	<8004914DB3B9496BB3DC1A1BFAF4D572@RaymondLaptop1>
	<20090619224036.12555.889474637.divmod.xquotient.13046@weber.divmod.com>
Message-ID: <h1jbof$lh1$1@ger.gmane.org>

glyph at divmod.com wrote:

> On 07:06 pm, python at rcn.com wrote:
>> Not sure why we need yet another pep on the subject.  Just update PEP 
>> 5 if needed.

I agree. The draft covers the same ground. Two PEPs on the same subject 
would be redundant where they agree but would  create confusion where 
they do not.

To the extent that the new PEP intends to change existing policy by 
severely curtailing language change, as it appears to, then that *idea* 
should be directly presented and discussed, perhaps on python-list, 
before worrying about wording (bikeshed) details.  In other words, I 
think the discussion should have start out "Here is existing policy (PEP 
5).  I propose to change it like so..." or possibly "Here is existing 
policy in PEP 5. I believe it has defacto changed as evidenced by ... "

In other words, discuss and decide whether the bikeshed should be 
re-painted before worrying about the exact shade.

> Right now, the rule to write software that will not break with the next 
> Python release is "read the minds of the python core committers and hope 
> that you do not do anything with the stdlib that they don't like".

A bit harsh ;-)

> Along 
> with this there are several rules you can infer that are probably true 
> most of the time: don't call stuff starting with "_", don't monkey- 
> patch anything, don't use dynamic class replacement on objects from 
> classes other than your own, don't depend on the depth of inheritance 
> hierarchies (for example, no ".__bases__[0].__bases__[0]"), make sure 
> your tests run without producing any DeprecationWarnings, be mindful of 
> potential namespace conflicts when adding attributes to classes that 
> inherit from other libraries.  I don't think all these things are 
> written down in one place though.

This could be the core of a new PEP Keeeping up with Language Changes.
I think that would be a good thing.

Terry Jan Reedy


From greg.ewing at canterbury.ac.nz  Sun Jun 21 03:26:36 2009
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Sun, 21 Jun 2009 13:26:36 +1200
Subject: [Python-Dev] io.BufferedReader.peek() Behaviour in python3.1
In-Reply-To: <4A3B76A7.7060701@gmail.com>
References: <20090617012719.GA19974@cskk.homeip.net>
	<4A399CEF.2070202@canterbury.ac.nz> <4A3A551A.6060304@gmail.com>
	<4A3AE0D3.2090708@canterbury.ac.nz> <4A3B76A7.7060701@gmail.com>
Message-ID: <4A3D8C4C.1030602@canterbury.ac.nz>

Lucas P Melo wrote:

> Am I understanding this correctly:
> * The blocking version would not do any raw reads.

No, the blocking version would keep doing raw reads
until the buffer contains enough bytes.

-- 
Greg

From j at 3rdengine.com  Sun Jun 21 19:36:40 2009
From: j at 3rdengine.com (Jerry Chen)
Date: Sun, 21 Jun 2009 12:36:40 -0500
Subject: [Python-Dev] Binary Operator for New-Style String Formatting
Message-ID: <1c9b9de40906211036v2eb2c86ajbb14fa646d2d9014@mail.gmail.com>

Hello all,

For better or for worse, I have created a patch against the py3k trunk
which introduces a binary operator '@' as an alternative syntax for
the new string formatting system introduced by PEP 3101 ("Advanced
String Formatting"). [1]

For common cases, this syntax should be as simple and as elegant as
its deprecated [2] predecessor ('%'), while also ensuring that more
complex use cases do not suffer needlessly.

I would just like to know whether this idea will float before
submitting the patch on Roundup and going through the formal PEP
process.  This is my first foray into the internals of the Python
core, and with any luck, I did not overlook any BDFL proclamations
banning all new binary operators for string formatting. :-)

QUICK EXAMPLES

    >>> "{} {} {}" @ (1, 2, 3)
    '1 2 3'

    >>> "foo {qux} baz" @ {"qux": "bar"}
    'foo bar baz'

One of the main complaints of a binary operator in PEP 3101 was the
inability to mix named and unnamed arguments:

    The current practice is to use either a dictionary or a tuple as
    the second argument, but as many people have commented ... this
    lacks flexibility.

To address this, a convention of having the last element of a tuple
as the named arguments dictionary is introduced.

    >>> "{} {qux} {}" @ (1, 3, {"qux": "bar"})
    '1 bar 3'

Lastly, to print the repr() of a dictionary as an unnamed argument,
one would have to append an additional dictionary so there is no
ambiguity:

    >>> "{}" @ {"foo": "bar"}
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
    IndexError: tuple index out of range

    >>> "{}" @ ({"foo": "bar"}, {})
    "{'foo': 'bar'}"

Admittedly, these workarounds are less than clean, but the
understanding is the '@' syntax would indeed be an alternative, so one
could easily fall back to the str.format() method or the format()
function.

IMPLEMENTATION

Code-wise, the grammar was edited per PEP 306 [3], and a
function was introduced in unicodeobject.c as PyUnicode_FormatPrime
(in the mathematical sense of A and A' -- I didn't fully understand or
want to intrude upon the *_FormatAdvanced namespace).

The PyUnicode_FormatPrime function transforms the incoming arguments,
i.e. the operands of the binary '@', and makes the appropriate
do_string_format() call.  Thus, I have reused as much code as
possible.

I have done my development with git by using two branches: 'master'
and 'subversion', the latter of which can be used to run 'svn update'
and merge back into master.  This way my code changes and the official
ones going into the Subversion repository can stay separate, meanwhile
allowing 'svn diff' to produce an accurate patch at any given time.

The code is available at:

    http://github.com/jcsalterego/py3k-atsign/

The SVN patch [4] or related commit [5] are good starting points.

References:

[1] http://www.python.org/dev/peps/pep-3101
[2] http://docs.python.org/3.0/whatsnew/3.0.html
[3] http://www.python.org/dev/peps/pep-0306/
[4] http://github.com/jcsalterego/py3k-atsign/blob/master/py3k-atsign.diff
[5] http://github.com/jcsalterego/py3k-atsign/commit/5c8bdf72d9252cea78af2b7809613f6530e25db4

Thanks,
-- 
Jerry Chen

From steven.bethard at gmail.com  Sun Jun 21 20:08:42 2009
From: steven.bethard at gmail.com (Steven Bethard)
Date: Sun, 21 Jun 2009 14:08:42 -0400
Subject: [Python-Dev] Binary Operator for New-Style String Formatting
In-Reply-To: <1c9b9de40906211036v2eb2c86ajbb14fa646d2d9014@mail.gmail.com>
References: <1c9b9de40906211036v2eb2c86ajbb14fa646d2d9014@mail.gmail.com>
Message-ID: <d11dcfba0906211108l7645c899w3e804408cfcf74b7@mail.gmail.com>

On Sun, Jun 21, 2009 at 1:36 PM, Jerry Chen<j at 3rdengine.com> wrote:
> QUICK EXAMPLES
>
> ? ?>>> "{} {} {}" @ (1, 2, 3)
> ? ?'1 2 3'
>
> ? ?>>> "foo {qux} baz" @ {"qux": "bar"}
> ? ?'foo bar baz'
>
> One of the main complaints of a binary operator in PEP 3101 was the
> inability to mix named and unnamed arguments:
>
> ? ?The current practice is to use either a dictionary or a tuple as
> ? ?the second argument, but as many people have commented ... this
> ? ?lacks flexibility.

The other reason an operator was a pain is the order of operations:

>>> '{0}'.format(1 + 2)
'3'
>>> '%s' % 1 + 2
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: cannot concatenate 'str' and 'int' objects

In general, I don't see any gain in introducing an operator for string
formatting. What's the point? Maybe you save a few characters of
typing, but it sure is easier to Google for "Python string format"
than for "Python @".

A big -1 from me.

Steve
-- 
Where did you get the preposterous hypothesis?
Did Steve tell you that?
        --- The Hiphopopotamus

From solipsis at pitrou.net  Sun Jun 21 20:45:20 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Sun, 21 Jun 2009 18:45:20 +0000 (UTC)
Subject: [Python-Dev] Binary Operator for New-Style String Formatting
References: <1c9b9de40906211036v2eb2c86ajbb14fa646d2d9014@mail.gmail.com>
Message-ID: <loom.20090621T183953-658@post.gmane.org>


Hello,

> For better or for worse, I have created a patch against the py3k trunk
> which introduces a binary operator '@' as an alternative syntax for
> the new string formatting system introduced by PEP 3101 ("Advanced
> String Formatting"). [1]

While many people find the new format() tedious to adapt to, I don't think
adding a third formatting syntax will help us.

Especially given this annoyance:

> Lastly, to print the repr() of a dictionary as an unnamed argument,
> one would have to append an additional dictionary so there is no
> ambiguity:
> 
>     >>> "{}" @ {"foo": "bar"}
>     Traceback (most recent call last):
>       File "<stdin>", line 1, in <module>
>     IndexError: tuple index out of range
> 
>     >>> "{}" @ ({"foo": "bar"}, {})
>     "{'foo': 'bar'}"

Regards

Antoine.



From rdmurray at bitdance.com  Sun Jun 21 20:50:40 2009
From: rdmurray at bitdance.com (R. David Murray)
Date: Sun, 21 Jun 2009 14:50:40 -0400 (EDT)
Subject: [Python-Dev] Binary Operator for New-Style String Formatting
In-Reply-To: <1c9b9de40906211036v2eb2c86ajbb14fa646d2d9014@mail.gmail.com>
References: <1c9b9de40906211036v2eb2c86ajbb14fa646d2d9014@mail.gmail.com>
Message-ID: <Pine.LNX.4.64.0906211440100.32513@kimball.webabinitio.net>

On Sun, 21 Jun 2009 at 12:36, Jerry Chen wrote:
> For better or for worse, I have created a patch against the py3k trunk
> which introduces a binary operator '@' as an alternative syntax for
> the new string formatting system introduced by PEP 3101 ("Advanced
> String Formatting"). [1]

It seems to me that this topic is more appropriate for python-ideas.

That said, I'm -1 on it.  The 'keywords as last item of tuple' reeks
of code-smell to my nose, and I don't think you've addressed all of
the reasons for why a method was chosen over an operator.  Python has a
tradition of having "one obvious way" to do something, so introducing an
"alternative" syntax that you admit is sub-optimal does not seem to me
to have enough benefit to justify breaking that design guideline.

Congratulations on your first foray into the core, though :)

--David

From tjreedy at udel.edu  Sun Jun 21 21:12:16 2009
From: tjreedy at udel.edu (Terry Reedy)
Date: Sun, 21 Jun 2009 15:12:16 -0400
Subject: [Python-Dev] Binary Operator for New-Style String Formatting
In-Reply-To: <1c9b9de40906211036v2eb2c86ajbb14fa646d2d9014@mail.gmail.com>
References: <1c9b9de40906211036v2eb2c86ajbb14fa646d2d9014@mail.gmail.com>
Message-ID: <h1m0mf$gah$1@ger.gmane.org>

Jerry Chen wrote:
> Hello all,
> 
> For better or for worse, I have created a patch against the py3k trunk
> which introduces a binary operator '@' as an alternative syntax for
> the new string formatting system introduced by PEP 3101 ("Advanced
> String Formatting"). [1]
> 
> For common cases, this syntax should be as simple and as elegant as
> its deprecated [2] predecessor ('%'), while also ensuring that more
> complex use cases do not suffer needlessly.
> 
> I would just like to know whether this idea will float before

The place to float trial balloons is the python-ideas list.

> submitting the patch on Roundup and going through the formal PEP
> process.  This is my first foray into the internals of the Python

Even if this particular idea in not accepted, I hope you learned from 
and enjoyed the exercise and will try other forays.

> core, and with any luck, I did not overlook any BDFL proclamations
> banning all new binary operators for string formatting. :-)
> 
> QUICK EXAMPLES
> 
>     >>> "{} {} {}" @ (1, 2, 3)

The only advantage '@' over '.format' is fewer characters.
I think it would be more useful to agitate to give 'format' a one char 
synonym such as 'f'.

One disadvantage of using an actual tuple rather than an arg quasi-tuple 
is that people would have to remember the trailing comma when printing 
one thing. '{}' @ (1,) rather than '{}' @ (a) == '{}' @ a. [If you say, 
'Oh, then accept the latter', then there is a problem when a is a tuple!]

>     '1 2 3'
> 
>     >>> "foo {qux} baz" @ {"qux": "bar"}
>     'foo bar baz'
> 
> One of the main complaints of a binary operator in PEP 3101 was the
> inability to mix named and unnamed arguments:
> 
>     The current practice is to use either a dictionary or a tuple as
>     the second argument, but as many people have commented ... this
>     lacks flexibility.
> 
> To address this, a convention of having the last element of a tuple
> as the named arguments dictionary is introduced.
> 
>     >>> "{} {qux} {}" @ (1, 3, {"qux": "bar"})
>     '1 bar 3'
> 
> Lastly, to print the repr() of a dictionary as an unnamed argument,
> one would have to append an additional dictionary so there is no
> ambiguity:
> 
>     >>> "{}" @ {"foo": "bar"}
>     Traceback (most recent call last):
>       File "<stdin>", line 1, in <module>
>     IndexError: tuple index out of range
> 
>     >>> "{}" @ ({"foo": "bar"}, {})
>     "{'foo': 'bar'}"

This is another disadvantage -- to me a big one.

Formatting is inherently an n-ary function who args are one format and 
an indefinite number of objects to plug in. Packaging the remaining args 
into an object to convert the function to binary is problematical, 
especially in Python with its mix of positional and named args. Even 
without that, there is possible confusion between a package as an arg in 
itself and a package as a container of multiple args. The % formatting 
problem with tuple puns was one of the reasons to seek a replacement.

Terry Jan Reedy

> 
> Admittedly, these workarounds are less than clean, but the
> understanding is the '@' syntax would indeed be an alternative, so one
> could easily fall back to the str.format() method or the format()
> function.
> 
> IMPLEMENTATION
> 
> Code-wise, the grammar was edited per PEP 306 [3], and a
> function was introduced in unicodeobject.c as PyUnicode_FormatPrime
> (in the mathematical sense of A and A' -- I didn't fully understand or
> want to intrude upon the *_FormatAdvanced namespace).
> 
> The PyUnicode_FormatPrime function transforms the incoming arguments,
> i.e. the operands of the binary '@', and makes the appropriate
> do_string_format() call.  Thus, I have reused as much code as
> possible.
> 
> I have done my development with git by using two branches: 'master'
> and 'subversion', the latter of which can be used to run 'svn update'
> and merge back into master.  This way my code changes and the official
> ones going into the Subversion repository can stay separate, meanwhile
> allowing 'svn diff' to produce an accurate patch at any given time.
> 
> The code is available at:
> 
>     http://github.com/jcsalterego/py3k-atsign/
> 
> The SVN patch [4] or related commit [5] are good starting points.
> 
> References:
> 
> [1] http://www.python.org/dev/peps/pep-3101
> [2] http://docs.python.org/3.0/whatsnew/3.0.html
> [3] http://www.python.org/dev/peps/pep-0306/
> [4] http://github.com/jcsalterego/py3k-atsign/blob/master/py3k-atsign.diff
> [5] http://github.com/jcsalterego/py3k-atsign/commit/5c8bdf72d9252cea78af2b7809613f6530e25db4
> 
> Thanks,


From omega_force2003 at yahoo.com  Sun Jun 21 21:22:30 2009
From: omega_force2003 at yahoo.com (omega_force2003 at yahoo.com)
Date: Sun, 21 Jun 2009 12:22:30 -0700 (PDT)
Subject: [Python-Dev] run time error anlysis of python source code
Message-ID: <120485.30890.qm@web34403.mail.mud.yahoo.com>


To All, 
?
?
It appears that one possibility of investigation into the development of a safety-critical variant of the python language would be to conduct run time error analysis of the source code that is responsible for producing the python language.? Therefore, I will now conduct these run time error analysis of the python source code as if the python environment itself was to be utilized as a FADEC controller?within an aircraft engine.? I have?already conducted some analysis already and it appears to?be some concern with memory management within Python.? I will redouble my efforts and determine if I am correct and as a promise.? I will give my findings to everyone to enjoy if they so want it.? I will also give the correct source if anyone would want it for their own purposes.? The source code that I will be evaluating is the one responsible for the newer variants of Python 3.0.
?
I believe that I will name this new variant of the python language as 
?
Apocalypse Python !!
?
I will also develop a web-page for the development and evolution of Apocalypse Python.
If anyone has any questions, please let me know !!!? 
?
Thank You,
?
David Blubaugh
?
?
? 


      
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090621/b0573333/attachment.htm>

From j at 3rdengine.com  Sun Jun 21 21:44:09 2009
From: j at 3rdengine.com (Jerry Chen)
Date: Sun, 21 Jun 2009 14:44:09 -0500
Subject: [Python-Dev] Binary Operator for New-Style String Formatting
In-Reply-To: <h1m0mf$gah$1@ger.gmane.org>
References: <1c9b9de40906211036v2eb2c86ajbb14fa646d2d9014@mail.gmail.com>
	<h1m0mf$gah$1@ger.gmane.org>
Message-ID: <1c9b9de40906211244v2561a758r12751695c8adedc1@mail.gmail.com>

Ah, the people have spoken!

On Sun, Jun 21, 2009 at 2:12 PM, Terry Reedy<tjreedy at udel.edu> wrote:
> The place to float trial balloons is the python-ideas list.

I'll put this one to rest, and as mentioned, will direct any future
suggestions to python-ideas instead of here.

Most of the arguments against my proposal state there is little gain
and much to lose (in terms of clarity or an "obvious way" to go about
string formatting) -- and, I agree.

> The only advantage '@' over '.format' is fewer characters.
> I think it would be more useful to agitate to give 'format' a one char
> synonym such as 'f'.

str.f() would be a great idea.

> One disadvantage of using an actual tuple rather than an arg quasi-tuple is
> that people would have to remember the trailing comma when printing one
> thing. '{}' @ (1,) rather than '{}' @ (a) == '{}' @ a. [If you say, 'Oh,
> then accept the latter', then there is a problem when a is a tuple!]

My code transforms both '{}' @ (a) and '{}' @ a to '{}'.format(a), but
the problem you speak of is probably an edge case I haven't quite
wrapped my head around.

For what it's worth, I spent a bit of time trying to work out the
syntactical quirks, including adapting the format tests in
Lib/test/test_unicode.py to this syntax and ensuring all the tests
passed.  In the end though, it seems to be an issue of usability and
clarity.

> Formatting is inherently an n-ary function who args are one format and an
> indefinite number of objects to plug in. Packaging the remaining args into
> an object to convert the function to binary is problematical, especially in
> Python with its mix of positional and named args. Even without that, there
> is possible confusion between a package as an arg in itself and a package as
> a container of multiple args. The % formatting problem with tuple puns was
> one of the reasons to seek a replacement.

Also (from R. David Murray):

> That said, I'm -1 on it.  The 'keywords as last item of tuple' reeks
> of code-smell to my nose, and I don't think you've addressed all of
> the reasons for why a method was chosen over an operator.  Python has a
> tradition of having "one obvious way" to do something, so introducing an
> "alternative" syntax that you admit is sub-optimal does not seem to me
> to have enough benefit to justify breaking that design guideline.

Well stated (and everyone else).

Just one last note: I think my end goal here was to preserve the
visual clarity and separation between format string and format
parameters, as I much prefer:

"%s %s %s" % (1, 2, 3)

over

"{0} {1} {2}".format(1, 2, 3)

The former is a style I've grown accustomed to, and if % is indeed
being slated for removal in Python 3.2, then I will miss it sorely
(or... just get over it).

Thanks to everyone who has provided constructive criticism and great
arguments.

Cheers,
-- 
Jerry Chen

From martin at v.loewis.de  Sun Jun 21 23:02:12 2009
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sun, 21 Jun 2009 23:02:12 +0200
Subject: [Python-Dev] Binary Operator for New-Style String Formatting
In-Reply-To: <1c9b9de40906211036v2eb2c86ajbb14fa646d2d9014@mail.gmail.com>
References: <1c9b9de40906211036v2eb2c86ajbb14fa646d2d9014@mail.gmail.com>
Message-ID: <4A3E9FD4.1000706@v.loewis.de>

> For better or for worse, I have created a patch against the py3k trunk
> which introduces a binary operator '@' as an alternative syntax for
> the new string formatting system introduced by PEP 3101 ("Advanced
> String Formatting"). [1]

I'd like to join everybody else who said that this would be a change
for the worse. This kind of syntax is one of the most prominent features
of Perl.

$@~ly/your/s!,
Martin

From fredrik at pythonware.com  Sun Jun 21 23:28:57 2009
From: fredrik at pythonware.com (Fredrik Lundh)
Date: Sun, 21 Jun 2009 23:28:57 +0200
Subject: [Python-Dev] xmlrpc improvements
In-Reply-To: <4A3D14F0.7080100@v.loewis.de>
References: <930F189C8A437347B80DF2C156F7EC7F057F9DDAF3@exchis.ccp.ad.local> 
	<4A3D0DEA.50002@v.loewis.de>
	<368a5cd50906200934n5b15a1aboac83ad1d32f5c3bb@mail.gmail.com> 
	<4A3D14F0.7080100@v.loewis.de>
Message-ID: <368a5cd50906211428t301f5599m460e8c2ec2c519a3@mail.gmail.com>

On Sat, Jun 20, 2009 at 6:57 PM, "Martin v. L?wis"<martin at v.loewis.de> wrote:
> Fredrik Lundh wrote:
>>> I think you really need to get Fredrik Lundh to comment on it. If he
>>> can't predict when he'll be able to review the changes, maybe he can
>>> accept releasing control of xmlrpclib.
>>
>> Pointer to the patch?
>
> http://bugs.python.org/issue6267

The xmlrpclib.py changes looks ok.  I'll leave it to other reviewers
to check the rest.

</F>

From eric at trueblade.com  Sun Jun 21 23:40:09 2009
From: eric at trueblade.com (Eric Smith)
Date: Sun, 21 Jun 2009 17:40:09 -0400
Subject: [Python-Dev] Binary Operator for New-Style String Formatting
In-Reply-To: <1c9b9de40906211244v2561a758r12751695c8adedc1@mail.gmail.com>
References: <1c9b9de40906211036v2eb2c86ajbb14fa646d2d9014@mail.gmail.com>	<h1m0mf$gah$1@ger.gmane.org>
	<1c9b9de40906211244v2561a758r12751695c8adedc1@mail.gmail.com>
Message-ID: <4A3EA8B9.3080108@trueblade.com>

I'm against syntax for this, for all the reasons stated by others.

Jerry Chen wrote:
> Just one last note: I think my end goal here was to preserve the
> visual clarity and separation between format string and format
> parameters, as I much prefer:
> 
> "%s %s %s" % (1, 2, 3)
> 
> over
> 
> "{0} {1} {2}".format(1, 2, 3)

If it helps, in 3.1 and 2.7 this can be written as
"{} {} {}".format(1, 2, 3)
I'm not sure it helps for "visual clarity", but it definitely makes the 
typing easier for simple uses.

> The former is a style I've grown accustomed to, and if % is indeed
> being slated for removal in Python 3.2, then I will miss it sorely
> (or... just get over it).

I've basically come to accept that %-formatting can never go away, 
unfortunately. There are too many places where %-formatting is used, for 
example in logging Formatters. %-formatting either has to exist or it 
has to be emulated.

Although if anyone has any suggestions for migrating uses like that, I'm 
interested.

Eric.


From fredrik at pythonware.com  Sun Jun 21 23:39:26 2009
From: fredrik at pythonware.com (Fredrik Lundh)
Date: Sun, 21 Jun 2009 23:39:26 +0200
Subject: [Python-Dev] xmlrpc improvements
In-Reply-To: <4A3D14F0.7080100@v.loewis.de>
References: <930F189C8A437347B80DF2C156F7EC7F057F9DDAF3@exchis.ccp.ad.local> 
	<4A3D0DEA.50002@v.loewis.de>
	<368a5cd50906200934n5b15a1aboac83ad1d32f5c3bb@mail.gmail.com> 
	<4A3D14F0.7080100@v.loewis.de>
Message-ID: <368a5cd50906211439i4f9a22b7mb062df8a8a86c8d7@mail.gmail.com>

On Sat, Jun 20, 2009 at 6:57 PM, "Martin v. L?wis"<martin at v.loewis.de> wrote:

> While I have your attention, please also review
>
> http://bugs.python.org/issue6233

I'm pretty sure that fix is the wrong fix - afaik, _encode is used to
encode tag/attribute names, and charrefs don't work in that context.

</F>

From tjreedy at udel.edu  Sun Jun 21 23:50:37 2009
From: tjreedy at udel.edu (Terry Reedy)
Date: Sun, 21 Jun 2009 17:50:37 -0400
Subject: [Python-Dev] run time error anlysis of python source code
In-Reply-To: <120485.30890.qm@web34403.mail.mud.yahoo.com>
References: <120485.30890.qm@web34403.mail.mud.yahoo.com>
Message-ID: <h1m9vc$7i2$1@ger.gmane.org>

omega_force2003 at yahoo.com wrote:

> It appears that one possibility of investigation into the development of 
> a safety-critical variant of the python language would be to conduct run 
> time error analysis of the source code that is responsible for producing 
> the python language.  Therefore, I will now conduct these run time error 
> analysis of the python source code as if the python environment itself 
> was to be utilized as a FADEC controller within an aircraft engine.  I 
> have already conducted some analysis already and it appears to be some 
> concern with memory management within Python.  I will redouble my 
> efforts and determine if I am correct and as a promise.  I will give my 
> findings to everyone to enjoy if they so want it.  I will also give the 
> correct source if anyone would want it for their own purposes.  The 
> source code that I will be evaluating is the one responsible for the 
> newer variants of Python 3.0.

I hope you mean 3.1 ;-)
3.0 was basically a trial version of Py3.


From tjreedy at udel.edu  Sun Jun 21 23:54:40 2009
From: tjreedy at udel.edu (Terry Reedy)
Date: Sun, 21 Jun 2009 17:54:40 -0400
Subject: [Python-Dev] xmlrpc improvements
In-Reply-To: <368a5cd50906211428t301f5599m460e8c2ec2c519a3@mail.gmail.com>
References: <930F189C8A437347B80DF2C156F7EC7F057F9DDAF3@exchis.ccp.ad.local>
	<4A3D0DEA.50002@v.loewis.de>	<368a5cd50906200934n5b15a1aboac83ad1d32f5c3bb@mail.gmail.com>
	<4A3D14F0.7080100@v.loewis.de>
	<368a5cd50906211428t301f5599m460e8c2ec2c519a3@mail.gmail.com>
Message-ID: <h1ma6v$8on$1@ger.gmane.org>

Fredrik Lundh wrote:
> On Sat, Jun 20, 2009 at 6:57 PM, "Martin v. L?wis"<martin at v.loewis.de> wrote:
>> Fredrik Lundh wrote:
>>>> I think you really need to get Fredrik Lundh to comment on it. If he
>>>> can't predict when he'll be able to review the changes, maybe he can
>>>> accept releasing control of xmlrpclib.
>>> Pointer to the patch?
>> http://bugs.python.org/issue6267
> 
> The xmlrpclib.py changes looks ok.  I'll leave it to other reviewers
> to check the rest.

Comment cc'ed to tracker


From ajaksu at gmail.com  Mon Jun 22 01:05:30 2009
From: ajaksu at gmail.com (Daniel Diniz)
Date: Sun, 21 Jun 2009 20:05:30 -0300
Subject: [Python-Dev] run time error anlysis of python source code
In-Reply-To: <120485.30890.qm@web34403.mail.mud.yahoo.com>
References: <120485.30890.qm@web34403.mail.mud.yahoo.com>
Message-ID: <2d75d7660906211605u4d9cd1c9i6641de5cd2e77d79@mail.gmail.com>

Hi David,

<omega_force2003 at yahoo.com> wrote:
>
> It appears that one possibility of investigation into the development of a
> safety-critical variant of the python language

There is some interesting work related to a safety-critical variant of Python.

> would be to conduct run time
> error analysis of the source code that is responsible for producing the
> python language.

There's been some effort into this too, and the Coverity and Klocwork
based fixes could also be of interest to you.

> Therefore, I will now conduct these run time error
> analysis of the python source code as if the python environment itself was
> to be utilized as a FADEC controller?within an aircraft engine.

Nice, what tools do you have available for this? Any papers that would
be a good start on the topic?

> I
> have?already conducted some analysis already and it appears to?be some
> concern with memory management within Python.? I will redouble my efforts
> and determine if I am correct and as a promise.? I will give my findings to
> everyone to enjoy if they so want it.? I will also give the correct source
> if anyone would want it for their own purposes.? The source code that I will
> be evaluating is the one responsible for the newer variants of Python 3.0.

You want the py3k branch. BTW, take a look at Brett Cannon's work on
Python security, as well as tav's.

> I believe that I will name this new variant of the python language as
>
> Apocalypse Python !!

Oh.

Apocalypse Python !!, you say? Maybe something that conveys a security
message or anything that doesn't relate to the end of the world could
work better.

> I will also develop a web-page for the development and evolution of
> Apocalypse Python.

Ah.

Hm.

David? Don't. I mean, read the mailing lists, take a look at open bug
reports, read the community blogs. You'll get to know how things flow,
you'll figure a nice way to relate your ideas to what people are
discussing and past experiences. Make small contributions that bring
us closer to... Apocalypse Python !!, then you'll have an easier time
to push the idea of going the whole way towards it.

Or just do it, dunno, if it makes you feel better go for it, we all
have our own issues. That's why we have an issue tracker, it's soooo
nice, wanna see it???

> If anyone has any questions, please let me know !!!

OK, I promise.

Daniel

From fijall at gmail.com  Mon Jun 22 01:09:06 2009
From: fijall at gmail.com (Maciej Fijalkowski)
Date: Sun, 21 Jun 2009 17:09:06 -0600
Subject: [Python-Dev] run time error anlysis of python source code
In-Reply-To: <2d75d7660906211605u4d9cd1c9i6641de5cd2e77d79@mail.gmail.com>
References: <120485.30890.qm@web34403.mail.mud.yahoo.com>
	<2d75d7660906211605u4d9cd1c9i6641de5cd2e77d79@mail.gmail.com>
Message-ID: <693bc9ab0906211609w78c831f6g516a3f749ab847b1@mail.gmail.com>

On Sun, Jun 21, 2009 at 5:05 PM, Daniel Diniz<ajaksu at gmail.com> wrote:
> Hi David,
>
> <omega_force2003 at yahoo.com> wrote:
>>
>> It appears that one possibility of investigation into the development of a
>> safety-critical variant of the python language
>
> There is some interesting work related to a safety-critical variant of Python.
>
>> would be to conduct run time
>> error analysis of the source code that is responsible for producing the
>> python language.
>
> There's been some effort into this too, and the Coverity and Klocwork
> based fixes could also be of interest to you.
>
>> Therefore, I will now conduct these run time error
>> analysis of the python source code as if the python environment itself was
>> to be utilized as a FADEC controller?within an aircraft engine.
>
> Nice, what tools do you have available for this? Any papers that would
> be a good start on the topic?
>
>> I
>> have?already conducted some analysis already and it appears to?be some
>> concern with memory management within Python.? I will redouble my efforts
>> and determine if I am correct and as a promise.? I will give my findings to
>> everyone to enjoy if they so want it.? I will also give the correct source
>> if anyone would want it for their own purposes.? The source code that I will
>> be evaluating is the one responsible for the newer variants of Python 3.0.
>
> You want the py3k branch. BTW, take a look at Brett Cannon's work on
> Python security, as well as tav's.

Is py3k branch even passing all tests on all buildbots all the time? I
don't think
svn head is the right thing to look at. Also, it's worth noting that
most big libraries
are 2.x compatible only.

From solipsis at pitrou.net  Mon Jun 22 01:28:06 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Sun, 21 Jun 2009 23:28:06 +0000 (UTC)
Subject: [Python-Dev] run time error anlysis of python source code
References: <120485.30890.qm@web34403.mail.mud.yahoo.com>
	<2d75d7660906211605u4d9cd1c9i6641de5cd2e77d79@mail.gmail.com>
	<693bc9ab0906211609w78c831f6g516a3f749ab847b1@mail.gmail.com>
Message-ID: <loom.20090621T231630-275@post.gmane.org>

Maciej Fijalkowski <fijall <at> gmail.com> writes:
> 
> Is py3k branch even passing all tests on all buildbots all the time?

As much as other branches do (that is, not much, due to the flakiness of some of
the tests and the lack of buildbot maintenance).

> I don't think
> svn head is the right thing to look at. Also, it's worth noting that
> most big libraries are 2.x compatible only.

For projects difficult enough that they won't be finished before a couple of
years, I think it makes sense to target 3.x. Big libraries will hopefully
migrate gradually. SQLAlchemy recently announced that their current development
version is 100% py3k-compatible (http://www.sqlalchemy.org/news.html#item_1).



From greg.ewing at canterbury.ac.nz  Mon Jun 22 02:22:30 2009
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Mon, 22 Jun 2009 12:22:30 +1200
Subject: [Python-Dev] run time error anlysis of python source code
In-Reply-To: <2d75d7660906211605u4d9cd1c9i6641de5cd2e77d79@mail.gmail.com>
References: <120485.30890.qm@web34403.mail.mud.yahoo.com>
	<2d75d7660906211605u4d9cd1c9i6641de5cd2e77d79@mail.gmail.com>
Message-ID: <4A3ECEC6.3020402@canterbury.ac.nz>

Daniel Diniz wrote:

> Apocalypse Python !!, you say? Maybe something that conveys a security
> message or anything that doesn't relate to the end of the world could
> work better.

I guess the idea is meant to be that it's safe enough to
use for something that would result in the end of the
world if it failed !!

Although personally, if something might cause the end
of the world if it failed, I'd prefer not to attempt it
in the first place !!

-- 
Apocalypse Python !! - If you ever need to use it, you're
in deep trouble... !!

Greg !!

From foom at fuhm.net  Mon Jun 22 14:30:49 2009
From: foom at fuhm.net (James Y Knight)
Date: Mon, 22 Jun 2009 08:30:49 -0400
Subject: [Python-Dev] Migration strategy for new-style string formatting
	[Was: Binary Operator for New-Style String Formatting]
In-Reply-To: <4A3EA8B9.3080108@trueblade.com>
References: <1c9b9de40906211036v2eb2c86ajbb14fa646d2d9014@mail.gmail.com>	<h1m0mf$gah$1@ger.gmane.org>
	<1c9b9de40906211244v2561a758r12751695c8adedc1@mail.gmail.com>
	<4A3EA8B9.3080108@trueblade.com>
Message-ID: <631E4C91-41C7-4196-854A-A7462841B319@fuhm.net>

On Jun 21, 2009, at 5:40 PM, Eric Smith wrote:
> I've basically come to accept that %-formatting can never go away,  
> unfortunately. There are too many places where %-formatting is used,  
> for example in logging Formatters. %-formatting either has to exist  
> or it has to be emulated.

It'd possibly be helpful if there were builtin objects which forced  
the format style to be either newstyle or oldstyle, independent of  
whether % or format was called on it.

E.g.
x = newstyle_formatstr("{} {} {}")
x % (1,2,3) == x.format(1,2,3) == "1 2 3"

and perhaps, for symmetry:
y = oldstyle_formatstr("%s %s %s")
y.format(1,2,3) == x % (1,2,3) == "1 2 3"

This allows the format string "style" decision is to be made external  
to the API actually calling the formatting function. Thus, it need not  
matter as much whether the logging API uses % or .format() internally  
-- that only affects the *default* behavior when a bare string is  
passed in.

This could allow for a controlled staged towards the new format string  
format, with a long deprecation period for users to migrate:

1) introduce the above feature, and recommend in docs that people only  
ever use new-style format strings, wrapping the string in  
newstyle_formatstr() when necessary for passing to an API which uses %  
internally.
2) A long time later...deprecate str.__mod__; don't deprecate  
newstyle_formatstr.__mod__.
3) A while after that (maybe), remove str.__mod__ and replace all  
calls in Python to % (used as a formatting operator) with .format() so  
that the default is to use newstyle format strings for all APIs from  
then on.

James

From ziade.tarek at gmail.com  Mon Jun 22 15:23:47 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Mon, 22 Jun 2009 15:23:47 +0200
Subject: [Python-Dev] PEP 376
Message-ID: <94bdd2610906220623k301e7924i5abc0a97d54741ba@mail.gmail.com>

Hello,

We have polished out PEP 376 and its code prototype at Distutils-SIG.
It seems to fullfill now all the requirements,
so I am mailing it here again, for a new round of feedback, if needed.

- the pep : http://svn.python.org/projects/peps/trunk/pep-0376.txt

- the code prototype  : http://bitbucket.org/tarek/pep376/src/tip/pkgutil.py

Notice that if the PEP is accepted at this point, I will :

- focus on making the code work as fast as possible, for directories browsing

- work on the backport and the required patches for setuptools and pip
at the same time, and
  see if I can get some beta-testers that are willing to switch to
this new version to test it extensively before
  2.7/3.2 are out.

Regards
Tarek

-- 
Tarek Ziad? | http://ziade.org

From solipsis at pitrou.net  Mon Jun 22 16:59:24 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Mon, 22 Jun 2009 14:59:24 +0000 (UTC)
Subject: [Python-Dev] PEP 376
References: <94bdd2610906220623k301e7924i5abc0a97d54741ba@mail.gmail.com>
Message-ID: <loom.20090622T142204-980@post.gmane.org>


Hello,

Tarek Ziad? <ziade.tarek <at> gmail.com> writes:
> 
> so I am mailing it here again, for a new round of feedback, if needed.
> 
> - the pep : http://svn.python.org/projects/peps/trunk/pep-0376.txt

Some comments:

  - the **MD5** hash of the file, encoded in hex. Notice that `pyc` and `pyo`
    generated files will not have a hash.

Why the exception for pyc and pyo files?

  - `zlib` and `zlib-2.5.2.egg-info` are located in `site-packages` so the file
    paths are relative to it.

Is it a general rule? That is, the paths in RECORD are always relative to its
grandparent directory?

  The RECORD format
  -----------------

  The `RECORD` file is a CSV file, composed of records, one line per
  installed file. The ``csv`` module is used to read the file, with
  the `excel` dialect, which uses these options to read the file:

  - field delimiter : `,`
  - quoting char :  `"`.
  - line terminator : `\r\n`

Wouldn't it be better to use the native line terminator on the current
platform? (someone might want to edit or at least view the file)

What is the character encoding for non-ASCII filenames? UTF-8?

Are the RECORD file's contents somehow part of the DistributionMetadata?

  - ``DistributionDirectories``: manages ``EggInfoDirectory`` instances.

What is an EggInfoDirectory ?

A plural class name looks strange (I think it's the first time I see one in
the CPython codebase). How about another name? (DistributionPool,
DistributionMap, WorkingSet etc.).

  - ``get_egginfo_file(path, binary=False)`` -> file object

     Returns a file located under the `.egg-info` directory.

     Returns a ``file`` instance for the file pointed by ``path``.

Is it always a file instance or just a file-like object? (zipped distributions
come to mind). Is it opened read-only?

  - ``owner(path)`` -> ``Distribution`` instance or None

    If ``path`` is used by only one ``Distribution`` instance, returns it.
    Otherwise returns None.

This is a bit confusing. If it returns None, it doesn't distinguish between the
case where several Distributions refer to the path, and the case where no
distributions refer to it, does it?

Is there any reason to have this method while file_users(path) already exists?

  A new class called ``DistributionDirectories`` is created. It's a collection of
  ``DistributionDirectory`` and ``ZippedDistributionDirectory`` instances.
  The constructor takes one optional argument ``use_cache`` set to ``True`` by
  default.

You forgot to describe the constructor's signature and what it does exactly.

  ``EggInfoDirectories`` also provides the following methods besides the ones
  from ``dict``::

What is EggInfoDirectories?

  - ``append(path)``

    Creates an ``DistributionDirectory`` (or ``ZippedDistributionDirectory``)
    instance for ``path`` and adds it in the mapping.

  - ``load(paths)``

    Creates and adds ``DistributionDirectory`` (or
    ``ZippedDistributionDirectory``) instances corresponding to ``paths``.

Why are these methods named completely differently although they do almost the
same thing? Besides, append() makes it look like ordering matters. Does it?
(for a naming suggestion, e.g. load(path) and load_all(paths). Or,
even simpler, load(*paths) or load(paths))

  - ``get_file_users(path)`` -> Iterator of ``Distribution`` (or
    ``ZippedDistribution``) instances.

This method is named file_users in another class. Perhaps the naming should be
consistent?

  All these functions use the same global instance of ``DistributionDirectories`` 
  to use the cache.

Is the global instance available to users?

      >>> for path, hash, size in dist.get_installed_files()::
      ...     print '%s %s %d %s' % (path, hash, size)

There's one too many "%s" here.


Thanks for your work!

Antoine.



From ziade.tarek at gmail.com  Mon Jun 22 17:42:24 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Mon, 22 Jun 2009 17:42:24 +0200
Subject: [Python-Dev] PEP 376
In-Reply-To: <loom.20090622T142204-980@post.gmane.org>
References: <94bdd2610906220623k301e7924i5abc0a97d54741ba@mail.gmail.com>
	<loom.20090622T142204-980@post.gmane.org>
Message-ID: <94bdd2610906220842y475aad09l27f5be4b185718a4@mail.gmail.com>

On Mon, Jun 22, 2009 at 4:59 PM, Antoine Pitrou <solipsis at pitrou.net> wrote:

>   - the **MD5** hash of the file, encoded in hex. Notice that `pyc` and
> `pyo`
>    generated files will not have a hash.
>
> Why the exception for pyc and pyo files?


As in PEP 262, since they are produced automatically from a py file,
checking the py file is enough
to decide if the file has changed.


>
>  - `zlib` and `zlib-2.5.2.egg-info` are located in `site-packages` so the
> file
>    paths are relative to it.
>
> Is it a general rule? That is, the paths in RECORD are always relative to
> its
> grandparent directory?



no, they can be located anywhere on the system. But when the paths are
located in the
same directory where the .egg-info directory is located, a relative path is
used. (see the section before this example)

I'll add an example that contains files located elswhere in the system (like
a script and a data file)



>
>
>  The RECORD format
>  -----------------
>
>  The `RECORD` file is a CSV file, composed of records, one line per
>  installed file. The ``csv`` module is used to read the file, with
>  the `excel` dialect, which uses these options to read the file:
>
>  - field delimiter : `,`
>  - quoting char :  `"`.
>  - line terminator : `\r\n`
>
> Wouldn't it be better to use the native line terminator on the current
> platform? (someone might want to edit or at least view the file)


Good idea, I'll change that,


>
>
> What is the character encoding for non-ASCII filenames? UTF-8?
>

Yes, there's a constant in Distutils, called PKG_INFO_ENCODING that will be
used for the file generation.



>
> Are the RECORD file's contents somehow part of the DistributionMetadata?



The DistributionMetadata correspond to the fields defined in PEP 345, e.g.
written in the PKG-INFO file,
which is mentioned in the RECORD file.

We are reworking PEP 345 as well, to add some fields. What did you have in
mind ?


>
>
>  - ``DistributionDirectories``: manages ``EggInfoDirectory`` instances.
>
> What is an EggInfoDirectory ?


Typo (old name), fixing this..


>
>
> A plural class name looks strange (I think it's the first time I see one in
> the CPython codebase). How about another name? (DistributionPool,
> DistributionMap, WorkingSet etc.).


Sure, WorkingSet is nice, it's the name used in setuptools,


>
>
>  - ``get_egginfo_file(path, binary=False)`` -> file object
>
>     Returns a file located under the `.egg-info` directory.
>
>     Returns a ``file`` instance for the file pointed by ``path``.
>
> Is it always a file instance or just a file-like object? (zipped
> distributions
> come to mind). Is it opened read-only?



It's in read-only mode, either "r" either "rb" and in case of a zip file,
it returns a file-like object using zipfile.ZipFile.open.


>
>  - ``owner(path)`` -> ``Distribution`` instance or None
>
>    If ``path`` is used by only one ``Distribution`` instance, returns it.
>    Otherwise returns None.
>
> This is a bit confusing. If it returns None, it doesn't distinguish between
> the
> case where several Distributions refer to the path, and the case where no
> distributions refer to it, does it?
>

The idea of this API is to find out of a distribution "owns" a file, e.g. is
the only distribution
that uses it, so it can be safely removed.


> Is there any reason to have this method while file_users(path) already
> exists?
>

Its just a helper for uninstallers, but file_users() could probably be
enough,
I can remove "owns" if people find it confusing,



>  A new class called ``DistributionDirectories`` is created. It's a
> collection of
>  ``DistributionDirectory`` and ``ZippedDistributionDirectory`` instances.
>  The constructor takes one optional argument ``use_cache`` set to ``True``
> by
>  default.
>
> You forgot to describe the constructor's signature and what it does
> exactly.
>

I'll add that,


>
>  ``EggInfoDirectories`` also provides the following methods besides the
> ones
>  from ``dict``::
>
> What is EggInfoDirectories?


This is a DistributionDirectories, one more instance I forgot to rename,
I'll fix that


>
>
>  - ``append(path)``
>
>    Creates an ``DistributionDirectory`` (or
> ``ZippedDistributionDirectory``)
>    instance for ``path`` and adds it in the mapping.
>
>  - ``load(paths)``
>
>    Creates and adds ``DistributionDirectory`` (or
>    ``ZippedDistributionDirectory``) instances corresponding to ``paths``.
>
> Why are these methods named completely differently although they do almost
> the
> same thing? Besides, append() makes it look like ordering matters. Does it?
> (for a naming suggestion, e.g. load(path) and load_all(paths). Or,
> even simpler, load(*paths) or load(paths))
>

Right, I'll fix that,



>
>  - ``get_file_users(path)`` -> Iterator of ``Distribution`` (or
>    ``ZippedDistribution``) instances.
>
> This method is named file_users in another class. Perhaps the naming should
> be
> consistent?


Right, it used to be get_*, that's a typo. I'll fix it,


>
>
>  All these functions use the same global instance of
> ``DistributionDirectories``
>  to use the cache.
>
> Is the global instance available to users?


No I didn't made it available to avoid concurrency problems,

>
>
>      >>> for path, hash, size in dist.get_installed_files()::
>      ...     print '%s %s %d %s' % (path, hash, size)
>
> There's one too many "%s" here.
>

Fixing it too,



>
>
> Thanks for your work!
>


Thanks for the feedback, I'll commit the fixes asap.

Tarek
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090622/b2094aa6/attachment-0001.htm>

From pje at telecommunity.com  Mon Jun 22 20:48:08 2009
From: pje at telecommunity.com (P.J. Eby)
Date: Mon, 22 Jun 2009 14:48:08 -0400
Subject: [Python-Dev] PEP 376
In-Reply-To: <94bdd2610906220842y475aad09l27f5be4b185718a4@mail.gmail.co
 m>
References: <94bdd2610906220623k301e7924i5abc0a97d54741ba@mail.gmail.com>
	<loom.20090622T142204-980@post.gmane.org>
	<94bdd2610906220842y475aad09l27f5be4b185718a4@mail.gmail.com>
Message-ID: <20090622184512.62DD43A4099@sparrow.telecommunity.com>

At 05:42 PM 6/22/2009 +0200, Tarek Ziad? wrote:
>Wouldn't it be better to use the native line terminator on the current
>platform? (someone might want to edit or at least view the file)
>
>
>Good idea, I'll change that,
>

As long as the file is always *read* with "U" mode, so that you can't 
mess it up, especially if the install is to a directory shared 
between platforms.


>The idea of this API is to find out of a distribution "owns" a file, 
>e.g. is the only distribution
>that uses it, so it can be safely removed.

This could equally well be done by ``owners(path)``, returning a 
sequence of zero or more items.  Any length <> 1 means the file can't 
be safely removed.  Meanwhile, having the data about all the owners 
of a file would also be useful for tools that just want to inspect a 
directory's contents, for example, or to detect conflicts and overwrites.


From benjamin at python.org  Mon Jun 22 21:44:44 2009
From: benjamin at python.org (Benjamin Peterson)
Date: Mon, 22 Jun 2009 14:44:44 -0500
Subject: [Python-Dev] draft pep: backwards compatibility
In-Reply-To: <43aa6ff70906201049q26add809sa5893c470cc520d6@mail.gmail.com>
References: <1afaf6160906181917m670663f5ucb21b8bc00c07735@mail.gmail.com>
	<43aa6ff70906201049q26add809sa5893c470cc520d6@mail.gmail.com>
Message-ID: <1afaf6160906221244x208b1aa3v811483ab6955e038@mail.gmail.com>

2009/6/20 Collin Winter <collinw at gmail.com>:
> Is this intended to include performance changes? Clearly no-one will
> complain if things simply get faster, but I'm thinking about cases
> where, say, a function runs in half the time but uses double the
> memory (or vice versa).

I don't think we can say anything about those cases before hand. We'll
have to decide on a case by case basis.



-- 
Regards,
Benjamin

From benjamin at python.org  Tue Jun 23 00:50:49 2009
From: benjamin at python.org (Benjamin Peterson)
Date: Mon, 22 Jun 2009 17:50:49 -0500
Subject: [Python-Dev] final release procedure
Message-ID: <1afaf6160906221550s5d860086p56da686c61ff2b8e@mail.gmail.com>

Hi everyone,
We're almost to the finish line on 3.1! Assuming that the last release
blocker [1] gets ironed out and no more rear their ugly heads, I will
go ahead. I'm going to freeze and tag the tree on Friday at about
15:00 UTC in order for the binaries to be built and uploaded. The
release announcement should be the next day.

[1] http://bugs.python.org/issue6233

-- 
Regards,
Benjamin

From kevin at bud.ca  Tue Jun 23 03:41:21 2009
From: kevin at bud.ca (Kevin Teague)
Date: Mon, 22 Jun 2009 18:41:21 -0700
Subject: [Python-Dev] PEP 376
In-Reply-To: <94bdd2610906220842y475aad09l27f5be4b185718a4@mail.gmail.com>
References: <94bdd2610906220623k301e7924i5abc0a97d54741ba@mail.gmail.com>
	<loom.20090622T142204-980@post.gmane.org>
	<94bdd2610906220842y475aad09l27f5be4b185718a4@mail.gmail.com>
Message-ID: <A315E325-654E-4827-978A-5B417185F220@bud.ca>

>
>
> A plural class name looks strange (I think it's the first time I see  
> one in
> the CPython codebase). How about another name? (DistributionPool,
> DistributionMap, WorkingSet etc.).
>
> Sure, WorkingSet is nice, it's the name used in setuptools,
>

A WorkingSet and a DistributionDirectories (or whatever it gets named  
to) are different things though, no?

A WorkingSet is "a collection of active distributions", where each  
distribution might come from different distribution directories:

http://peak.telecommunity.com/DevCenter/PkgResources#workingset-objects

Where as DistributionDirectories is a dictionary of locations where  
distributions are installed. The WorkingSet may be comprised of  
distributions from several different locations, and each location may  
contain the same or different versions of the same distribution.

(as far as I understand things ...)

I can't really think of a better name for a dict of distribution  
locations ... but then I'm not averse to a pluralized class name.

Overall though, I think PEP 376 is starting to look very good!


From tcr at freebits.de  Tue Jun 23 08:47:36 2009
From: tcr at freebits.de (Tobias C. Rittweiler)
Date: Tue, 23 Jun 2009 08:47:36 +0200
Subject: [Python-Dev] Adding syntax for units of measure
Message-ID: <873a9rv43b.fsf@freebits.de>


Hi!

Has anyone added special syntax to allow writing numeric literals with
physical units? So you can write 12m + 34cm, and would get 12.34m.

My question is how would you modify the BNF the most sensible way to
allow for this? The above example is simple, but think of 42 km/h.

(For my purposes, modifying the BNF is perfectly reasonable, but if you
can depict a good, and convenient!, way that would not result in
modifying it, I'd like to hear it, too.)

Thanks in advance for your input,

  -T.


From tjreedy at udel.edu  Tue Jun 23 09:00:26 2009
From: tjreedy at udel.edu (Terry Reedy)
Date: Tue, 23 Jun 2009 03:00:26 -0400
Subject: [Python-Dev] Adding syntax for units of measure
In-Reply-To: <873a9rv43b.fsf@freebits.de>
References: <873a9rv43b.fsf@freebits.de>
Message-ID: <h1puic$jji$1@ger.gmane.org>

Tobias C. Rittweiler wrote:

> Has anyone added special syntax to allow writing numeric literals with
> physical units? So you can write 12m + 34cm, and would get 12.34m.

Python-dev is for concrete discussion of development of the next 
versions. Questions and speculative discussion should generally be 
directed to python-list and concrete proposals to python-ideas.

There have been posts on this subject on python-list and you can search 
the archive. That list is mirrored as newsgroup 
gmane.comp.python.general at news.gmane.org, which I believe has its own 
searchable archive.

tjr


From ziade.tarek at gmail.com  Tue Jun 23 10:29:38 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Tue, 23 Jun 2009 10:29:38 +0200
Subject: [Python-Dev] PEP 376
In-Reply-To: <20090622184512.62DD43A4099@sparrow.telecommunity.com>
References: <94bdd2610906220623k301e7924i5abc0a97d54741ba@mail.gmail.com>
	<loom.20090622T142204-980@post.gmane.org>
	<94bdd2610906220842y475aad09l27f5be4b185718a4@mail.gmail.com>
	<20090622184512.62DD43A4099@sparrow.telecommunity.com>
Message-ID: <94bdd2610906230129kac2fcf9rd3ab107499b70821@mail.gmail.com>

2009/6/22 P.J. Eby <pje at telecommunity.com>:
> At 05:42 PM 6/22/2009 +0200, Tarek Ziad? wrote:
>>
>> Wouldn't it be better to use the native line terminator on the current
>> platform? (someone might want to edit or at least view the file)
>>
>>
>> Good idea, I'll change that,
>>
>
> As long as the file is always *read* with "U" mode, so that you can't mess
> it up, especially if the install is to a directory shared between platforms.

Adding that too, thanks;

>
>
>> The idea of this API is to find out of a distribution "owns" a file, e.g.
>> is the only distribution
>> that uses it, so it can be safely removed.
>
> This could equally well be done by ``owners(path)``, returning a sequence of
> zero or more items. ?Any length <> 1 means the file can't be safely removed.
> ?Meanwhile, having the data about all the owners of a file would also be
> useful for tools that just want to inspect a directory's contents, for
> example, or to detect conflicts and overwrites.

that's basically what "get_file_users" does except it's an iterator,

roughly that would be :

def get_file_owners(path):
    return list(get_file_users(paths))

So I am wondering if it worth having it....



-- 
Tarek Ziad? | http://ziade.org

From ziade.tarek at gmail.com  Tue Jun 23 10:38:38 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Tue, 23 Jun 2009 10:38:38 +0200
Subject: [Python-Dev] PEP 376
In-Reply-To: <A315E325-654E-4827-978A-5B417185F220@bud.ca>
References: <94bdd2610906220623k301e7924i5abc0a97d54741ba@mail.gmail.com>
	<loom.20090622T142204-980@post.gmane.org>
	<94bdd2610906220842y475aad09l27f5be4b185718a4@mail.gmail.com>
	<A315E325-654E-4827-978A-5B417185F220@bud.ca>
Message-ID: <94bdd2610906230138o186c5fa5h27ed7d857cd9284b@mail.gmail.com>

On Tue, Jun 23, 2009 at 3:41 AM, Kevin Teague<kevin at bud.ca> wrote:
>>
>>
>> A plural class name looks strange (I think it's the first time I see one
>> in
>> the CPython codebase). How about another name? (DistributionPool,
>> DistributionMap, WorkingSet etc.).
>>
>> Sure, WorkingSet is nice, it's the name used in setuptools,
>>
>
> A WorkingSet and a DistributionDirectories (or whatever it gets named to)
> are different things though, no?
>
> A WorkingSet is "a collection of active distributions", where each
> distribution might come from different distribution directories:
>
> http://peak.telecommunity.com/DevCenter/PkgResources#workingset-objects
>
> Where as DistributionDirectories is a dictionary of locations where
> distributions are installed. The WorkingSet may be comprised of
> distributions from several different locations, and each location may
> contain the same or different versions of the same distribution.

DistributionDirectories can contain directories that are not located
in the same parent
directory, so I find it rather similar besides the "active" feature in
Python doesn't exist (yet)

In any case, maybe picking up a name that is not from setuptools will
be less confusing
for people that uses WorkingSet classes nowadays.

What about using the same names used in Python's site module:
"sitedir" is the name used for
a directory we named DistributionDirectory.

So what about :

DistributionDirectory -> SiteDir
DistributionDirectories ->  SiteDirMap


++
Tarek

From ncoghlan at gmail.com  Tue Jun 23 11:57:53 2009
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Tue, 23 Jun 2009 19:57:53 +1000
Subject: [Python-Dev] PEP 376
In-Reply-To: <94bdd2610906230138o186c5fa5h27ed7d857cd9284b@mail.gmail.com>
References: <94bdd2610906220623k301e7924i5abc0a97d54741ba@mail.gmail.com>	<loom.20090622T142204-980@post.gmane.org>	<94bdd2610906220842y475aad09l27f5be4b185718a4@mail.gmail.com>	<A315E325-654E-4827-978A-5B417185F220@bud.ca>
	<94bdd2610906230138o186c5fa5h27ed7d857cd9284b@mail.gmail.com>
Message-ID: <4A40A721.6090204@gmail.com>

Tarek Ziad? wrote:
> So what about :
> 
> DistributionDirectory -> SiteDir
> DistributionDirectories ->  SiteDirMap

'site' has too many connections to existing concepts for my liking
(site.py, sitesetup.py, site-packages).

Something like DistributionDirectoryMap should cover it.

You could probably get away with shortening "Directory" to "Dir" in the
class names though:

 - Distribution
 - ZippedDistribution
 - DistributionDir
 - ZippedDistributionDir
 - DistributionDirMap

(Shortening Distribution to Dist might also be a possibility, but I
don't think that works well for the two basic classes, and if those use
the long form then shortening it for the *Dir and *DirMap classes would
just look odd)

Cheers,
Nick.


-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------

From lists at cheimes.de  Tue Jun 23 12:51:55 2009
From: lists at cheimes.de (Christian Heimes)
Date: Tue, 23 Jun 2009 12:51:55 +0200
Subject: [Python-Dev] Adding syntax for units of measure
In-Reply-To: <873a9rv43b.fsf@freebits.de>
References: <873a9rv43b.fsf@freebits.de>
Message-ID: <h1qc4b$r6e$1@ger.gmane.org>

Tobias C. Rittweiler schrieb:
> Hi!
> 
> Has anyone added special syntax to allow writing numeric literals with
> physical units? So you can write 12m + 34cm, and would get 12.34m.
> 
> My question is how would you modify the BNF the most sensible way to
> allow for this? The above example is simple, but think of 42 km/h.

You don't need special syntax in order to work with units. You just need
to normalize all input to SI units like meter, second and meter per second:

km=1000.
m=1.
dm=0.1
cm=0.01
mm=0.001

s= 1.
h = 3600.

m_s=m/s
km_h=km/h

length_in_mm = (12*m + 34*cm)/mm
speed = 5*km_h


For what purpose do you want to have physical units in Python syntax? Do
you need to verify your formulas?

Christian


From g.brandl at gmx.net  Tue Jun 23 19:40:28 2009
From: g.brandl at gmx.net (Georg Brandl)
Date: Tue, 23 Jun 2009 17:40:28 +0000
Subject: [Python-Dev] Adding syntax for units of measure
In-Reply-To: <873a9rv43b.fsf@freebits.de>
References: <873a9rv43b.fsf@freebits.de>
Message-ID: <h1r425$d9s$1@ger.gmane.org>

Tobias C. Rittweiler schrieb:
> Hi!
> 
> Has anyone added special syntax to allow writing numeric literals with
> physical units? So you can write 12m + 34cm, and would get 12.34m.
> 
> My question is how would you modify the BNF the most sensible way to
> allow for this? The above example is simple, but think of 42 km/h.
> 
> (For my purposes, modifying the BNF is perfectly reasonable, but if you
> can depict a good, and convenient!, way that would not result in
> modifying it, I'd like to hear it, too.)

Hi,

normally you wouldn't add units to the language itself.  When using them
programmatically, it should be no effort to use a class that represents a
quantity with unit.  This can be made as easy as making "m" an object of
that type, so that you only need to type "2*m" to get two meters.

For the interactive shell, using a wrapper that allows simplified input is
also a possibility, like IPython's "-profile physics" mode, or something like
http://bitbucket.org/birkenfeld/phsh/ which allows you to write

>>> `1 m` + `12 cm`
1.12 m

cheers,
Georg


From forumer at smartmobili.com  Tue Jun 23 20:37:34 2009
From: forumer at smartmobili.com (Vincent R.)
Date: Tue, 23 Jun 2009 20:37:34 +0200
Subject: [Python-Dev] Python 3.0.1 and mingw
Message-ID: <17a4b6870bdf20f0d2b794eb55ee8ba7@mail.smartmobili.com>

Hi,

I wanted to know if you have some patch to compile python 3.x on mingw
platform because I found some
but doesn't work very well :

> make
gcc   -o python.exe \
Modules/python.o \
libpython3.0.a    -lm
Could not find platform independent libraries <prefix>
Could not find platform dependent libraries <exec_prefix>
Consider setting $PYTHONHOME to <prefix>[:<exec_prefix>]
Fatal Python error: Py_Initialize: can't initialize sys standard streams
ImportError: No module named encodings.utf_8


I have some questions about posixmodule.c, config.c and makesetup, 
I can see in posixmodule that you define a INITFUNC like this :

#if (defined(_MSC_VER) || defined(__WATCOMC__) || defined(__BORLANDC__)) &&
!defined(__QNX__)
#define INITFUNC PyInit_nt
#define MODNAME "nt"

#elif defined(PYOS_OS2)
#define INITFUNC PyInit_os2
#define MODNAME "os2"

#else
#define INITFUNC PyInit_posix
#define MODNAME "posix"
#endif

So first I tried to add || defined(____MINGW32____) to declare a PyInit_nt
but after config.c
was still using PyInit_posix. How does makesetup choose to include one
function or another ?
So finally python is using PyInit_posix...


Any idea why I got this compilation error ?


And after 




From aahz at pythoncraft.com  Tue Jun 23 21:39:27 2009
From: aahz at pythoncraft.com (Aahz)
Date: Tue, 23 Jun 2009 12:39:27 -0700
Subject: [Python-Dev] Python 3.0.1 and mingw
In-Reply-To: <17a4b6870bdf20f0d2b794eb55ee8ba7@mail.smartmobili.com>
References: <17a4b6870bdf20f0d2b794eb55ee8ba7@mail.smartmobili.com>
Message-ID: <20090623193927.GA25928@panix.com>

On Tue, Jun 23, 2009, Vincent R. wrote:
>
> I wanted to know if you have some patch to compile python 3.x on mingw
> platform because I found some but doesn't work very well :

This question belongs on comp.lang.python, please re-post there.
-- 
Aahz (aahz at pythoncraft.com)           <*>         http://www.pythoncraft.com/

"as long as we like the same operating system, things are cool." --piranha

From sridharr at activestate.com  Tue Jun 23 21:45:29 2009
From: sridharr at activestate.com (Sridhar Ratnakumar)
Date: Tue, 23 Jun 2009 12:45:29 -0700
Subject: [Python-Dev] PEP 376
In-Reply-To: <4A40A721.6090204@gmail.com>
References: <94bdd2610906220623k301e7924i5abc0a97d54741ba@mail.gmail.com>	<loom.20090622T142204-980@post.gmane.org>	<94bdd2610906220842y475aad09l27f5be4b185718a4@mail.gmail.com>	<A315E325-654E-4827-978A-5B417185F220@bud.ca>	<94bdd2610906230138o186c5fa5h27ed7d857cd9284b@mail.gmail.com>
	<4A40A721.6090204@gmail.com>
Message-ID: <4A4130D9.5040108@activestate.com>

On 09-06-23 02:57 AM, Nick Coghlan wrote:
> Something like DistributionDirectoryMap should cover it.
>
> You could probably get away with shortening "Directory" to "Dir" in the
> class names though:
>
>   - Distribution
>   - ZippedDistribution
>   - DistributionDir
>   - ZippedDistributionDir
>   - DistributionDirMap

'Map' reminds me of an associated hash (or is it the actual map?).

Hence I suggest: DistributionSet

-srid

From skip at pobox.com  Wed Jun 24 06:07:02 2009
From: skip at pobox.com (skip at pobox.com)
Date: Tue, 23 Jun 2009 23:07:02 -0500
Subject: [Python-Dev] Adding syntax for units of measure
In-Reply-To: <h1r425$d9s$1@ger.gmane.org>
References: <873a9rv43b.fsf@freebits.de> <h1r425$d9s$1@ger.gmane.org>
Message-ID: <19009.42598.902021.199401@montanaro.dyndns.org>

    >> Has anyone added special syntax to allow writing numeric literals with
    >> physical units? So you can write 12m + 34cm, and would get 12.34m.
    ...
    Georg> normally you wouldn't add units to the language itself.
    ...
    Georg> For the interactive shell, using a wrapper that allows simplified
    Georg> input is also a possibility, like IPython's "-profile physics"
    Georg> mode, or something like http://bitbucket.org/birkenfeld/phsh/
    Georg> which allows you to write

    >>>> `1 m` + `12 cm`
    Georg> 1.12 m

Also, check out the magnitude module (in PyPI).  I use it to specify the
units of the computation but allow users to input values using units which
are meaningful to them.  So, for example, if a value has units of time they
could enter 1m or 60s and get the same internal value.

Skip

From pje at telecommunity.com  Wed Jun 24 07:10:02 2009
From: pje at telecommunity.com (P.J. Eby)
Date: Wed, 24 Jun 2009 01:10:02 -0400
Subject: [Python-Dev] PEP 376
In-Reply-To: <94bdd2610906230138o186c5fa5h27ed7d857cd9284b@mail.gmail.co
 m>
References: <94bdd2610906220623k301e7924i5abc0a97d54741ba@mail.gmail.com>
	<loom.20090622T142204-980@post.gmane.org>
	<94bdd2610906220842y475aad09l27f5be4b185718a4@mail.gmail.com>
	<A315E325-654E-4827-978A-5B417185F220@bud.ca>
	<94bdd2610906230138o186c5fa5h27ed7d857cd9284b@mail.gmail.com>
Message-ID: <20090624050718.C0DDD3A4099@sparrow.telecommunity.com>

At 10:38 AM 6/23/2009 +0200, Tarek Ziad? wrote:
>What about using the same names used in Python's site module:
>"sitedir" is the name used for
>a directory we named DistributionDirectory.

No, a site dir is a Python-defined directory for site-installed 
packages, and/or a directory where .pth files are processed.  Wrong 
connotations entirely, since packages may be installed to other 
directories, and typically are in e.g. shared hosting environments.

DistributionDirectory is fine by me.  DistributionDirectories sounds 
like what setuptools calls an Environment.


From solipsis at pitrou.net  Wed Jun 24 11:18:35 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Wed, 24 Jun 2009 09:18:35 +0000 (UTC)
Subject: [Python-Dev] PEP 376
References: <94bdd2610906220623k301e7924i5abc0a97d54741ba@mail.gmail.com>	<loom.20090622T142204-980@post.gmane.org>	<94bdd2610906220842y475aad09l27f5be4b185718a4@mail.gmail.com>	<A315E325-654E-4827-978A-5B417185F220@bud.ca>	<94bdd2610906230138o186c5fa5h27ed7d857cd9284b@mail.gmail.com>
	<4A40A721.6090204@gmail.com> <4A4130D9.5040108@activestate.com>
Message-ID: <loom.20090624T091739-378@post.gmane.org>

Sridhar Ratnakumar <sridharr <at> activestate.com> writes:
> 
> On 09-06-23 02:57 AM, Nick Coghlan wrote:
> > Something like DistributionDirectoryMap should cover it.
> >
> > You could probably get away with shortening "Directory" to "Dir" in the
> > class names though:
> >
> >   - Distribution
> >   - ZippedDistribution
> >   - DistributionDir
> >   - ZippedDistributionDir
> >   - DistributionDirMap
> 
> 'Map' reminds me of an associated hash (or is it the actual map?).

Good thing, because it is a dict subclass if you read the PEP :)

+1 with Nick's proposal.

(we are good at finding bikesheds everywhere aren't we?)



From pegasus2000 at email.it  Wed Jun 24 17:09:21 2009
From: pegasus2000 at email.it (Filippo Battaglia)
Date: Wed, 24 Jun 2009 17:09:21 +0200
Subject: [Python-Dev] ndPython: I NEED TO TALK WITH ONE OF THE PYTHON CORE
	DEVELOPERS
Message-ID: <006801c9f4dd$c18294c0$011ea8c0@pegasus>

I need help with an implementation of your
interpreter under Nanodesktop PSPE/PSP.

I'm working to a version of Stackless Python
interpreter (2.5.2) called ndPython. It is
able to work under PSPE and PSP and
it shall be the most powerful interpreter
ever realized before on these platforms.

I need to know something about the C
functions that are recalled by the interpreter
when it executes a .pyc file.

Our version of ndPython is very slow in
execution respect to Carlos's StackLess
Python (another product for PSP).

We believe that the trouble is in any routine
of our Nanodesktop libc that could be
a bottleneck. But we don't know
which can be the interested routine
(string ? memory allocation ?)

Can you tell me which modules provide to
decode and execute a .pyc file ?

The number of modules that are statically
linked in the interpreter can affect its 
performance ? (for example, for a larger
number of strcmp between the keywords). 

The execution of a .pyc file is a memory
allocation intensive task ? I've already
enable the PYMEM option and I've
see that the speed is improved (but
Carlos's StackLess Python remains
faster). 

We've done these tests:

http://www.psp-ita.com/forum/viewtopic.php?t=28323&start=20


from ndpsp import * 
x = time.clock ()
for i in range(1000): 
    pass 
print >> clock, "1.", time.clock()-t 


We've the results:

stackless: 0.0037279 
ndPython: 0.1015625


We don't understand why ndPython is
so slower. 

Please, help us. The product is complete,
but we cannot release cause this problem.

Thank you in advance.
Filippo Battaglia


 
 
 --
 Caselle da 1GB, trasmetti allegati fino a 3GB e in piu' IMAP, POP3 e SMTP autenticato? GRATIS solo con Email.it http://www.email.it/f
 
 Sponsor:
 Vieni in vacanza nei Riccione Family Hotels! Ti aspettano servizi per bambini, mini club e tanto divertimento! Prenota on-line la tua vacanza!
 Clicca qui: http://adv.email.it/cgi-bin/foclick.cgi?mid=9206&d=24-6

From tjreedy at udel.edu  Wed Jun 24 18:10:15 2009
From: tjreedy at udel.edu (Terry Reedy)
Date: Wed, 24 Jun 2009 12:10:15 -0400
Subject: [Python-Dev] ndPython: I NEED TO TALK WITH ONE OF THE PYTHON
	CORE DEVELOPERS
In-Reply-To: <006801c9f4dd$c18294c0$011ea8c0@pegasus>
References: <006801c9f4dd$c18294c0$011ea8c0@pegasus>
Message-ID: <h1tj56$723$1@ger.gmane.org>

SUBJECT LINES IN ALL CAPS CREATE NEGATIVE IMPRESSION

Filippo Battaglia wrote:

[snip questions I cannot answer]

Question about existing, older Python versions should be directed to 
python-list, at least for a first try. Stackless Python is not produced 
by PSF core developers. It has third-party alterations to the core that 
might affect the answers to your question.

> http://www.psp-ita.com/forum/viewtopic.php?t=28323&start=20

You should mention that the above is Italian, not English.
There once was, and maybe still is, a python discussion list in Italian.

> from ndpsp import * x = time.clock ()
> for i in range(1000):    pass print >> clock, "1.", time.clock()-t

When you post code, cut and paste the actual code that you ran. The 
above code will not run. Line-endings are missing. t is not defined. 
Perhaps you meant 't = time.clock()'.

Good luck with your project.

Terry Jan Reedy


From kristjan at ccpgames.com  Wed Jun 24 18:23:57 2009
From: kristjan at ccpgames.com (=?iso-8859-1?Q?Kristj=E1n_Valur_J=F3nsson?=)
Date: Wed, 24 Jun 2009 16:23:57 +0000
Subject: [Python-Dev] ndPython: I NEED TO TALK WITH ONE OF THE
	PYTHON	CORE DEVELOPERS
In-Reply-To: <h1tj56$723$1@ger.gmane.org>
References: <006801c9f4dd$c18294c0$011ea8c0@pegasus>
	<h1tj56$723$1@ger.gmane.org>
Message-ID: <930F189C8A437347B80DF2C156F7EC7F057FB5FB0E@exchis.ccp.ad.local>

The OP was redirected here from the stackless list since his questions were not stackless specific.
Stackless python is in sync with the latest 2.x, 3.x branches as well as the trunk so performance problems he may with performance in general are perhaps best resolved by this lot.

Kristj?n

> -----Original Message-----
> From: python-dev-bounces+kristjan=ccpgames.com at python.org
> [mailto:python-dev-bounces+kristjan=ccpgames.com at python.org] On Behalf
> Of Terry Reedy
> Sent: 24. j?n? 2009 16:10
> To: python-dev at python.org
> Subject: Re: [Python-Dev] ndPython: I NEED TO TALK WITH ONE OF THE
> PYTHON CORE DEVELOPERS
> Question about existing, older Python versions should be directed to
> python-list, at least for a first try. Stackless Python is not produced
> by PSF core developers. It has third-party alterations to the core that
> might affect the answers to your question.
> 


From dhburns at cherokeetel.net  Wed Jun 24 17:59:17 2009
From: dhburns at cherokeetel.net (David H. Burns)
Date: Wed, 24 Jun 2009 10:59:17 -0500
Subject: [Python-Dev] Python for Windows??
Message-ID: <4A424D55.6010404@cherokeetel.net>

I downloaded what claims to be Python for Windows (3.01). The tutorial 
brags a lot about how easy it is to learn, but the tutorials and 
instruction seem to be for a Linux or Unix version. There are three 
executable programs in the Python directory and no indication which 
should be used to start Python. One opens a Dos-like window presumably 
for  "command-line" entry. I can't make anything of it.

I did a straight install on a XP system.

Any help would be appreciated.

Have a good day,

David H. Burns


From fuzzyman at voidspace.org.uk  Wed Jun 24 19:13:32 2009
From: fuzzyman at voidspace.org.uk (Michael Foord)
Date: Wed, 24 Jun 2009 18:13:32 +0100
Subject: [Python-Dev] Python for Windows??
In-Reply-To: <4A424D55.6010404@cherokeetel.net>
References: <4A424D55.6010404@cherokeetel.net>
Message-ID: <4A425EBC.20007@voidspace.org.uk>

David H. Burns wrote:
> I downloaded what claims to be Python for Windows (3.01). The tutorial 
> brags a lot about how easy it is to learn, but the tutorials and 
> instruction seem to be for a Linux or Unix version. There are three 
> executable programs in the Python directory and no indication which 
> should be used to start Python. One opens a Dos-like window presumably 
> for  "command-line" entry. I can't make anything of it.

I'm afraid the mailing list you posted to is for the development *of* 
Python and not the use of Python.

On the Python-tutor or Python-list mailing lists (or their google groups 
/ gmane gateways if you prefer a web interface) you will find people 
willing and able to answer your questions.

All the best,

Michael Foord

>
> I did a straight install on a XP system.
>
> Any help would be appreciated.
>
> Have a good day,
>
> David H. Burns
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: 
> http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk 
>


-- 
http://www.ironpythoninaction.com/
http://www.voidspace.org.uk/blog



From solipsis at pitrou.net  Wed Jun 24 19:41:39 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Wed, 24 Jun 2009 17:41:39 +0000 (UTC)
Subject: [Python-Dev] ndPython
References: <006801c9f4dd$c18294c0$011ea8c0@pegasus>
Message-ID: <loom.20090624T173018-457@post.gmane.org>

Filippo Battaglia <pegasus2000 <at> email.it> writes:
> 
> http://www.psp-ita.com/forum/viewtopic.php?t=28323&start=20
> 
> from ndpsp import * 
> x = time.clock ()
> for i in range(1000): 
>     pass 
> print >> clock, "1.", time.clock()-t 

First, it has been noted that your message would better be sent to python-list.

You must provide a working code example that doesn't need any "ndpsp" import to
work.
Also, since it is about porting Python to a particular platform, please
enlighten us as to what type of CPU it uses, what compiler, what optimization
options were used etc.

As a sidenote, a common way of timing small snippets of Python code is by using
the "timeit" module. From the command line, you could write:

$ python -m timeit "for i in range(1000): pass"
10000 loops, best of 3: 87.2 usec per loop

If you don't spend at least a bit of time writing your message clearly and
giving enough details, nobody will spend their unpaid voluntary time trying to
understand what happens.

Regards

Antoine.



From tjreedy at udel.edu  Wed Jun 24 20:57:17 2009
From: tjreedy at udel.edu (Terry Reedy)
Date: Wed, 24 Jun 2009 14:57:17 -0400
Subject: [Python-Dev] ndPython: I NEED TO TALK WITH ONE OF THE PYTHON
	CORE DEVELOPERS
In-Reply-To: <930F189C8A437347B80DF2C156F7EC7F057FB5FB0E@exchis.ccp.ad.local>
References: <006801c9f4dd$c18294c0$011ea8c0@pegasus>	<h1tj56$723$1@ger.gmane.org>
	<930F189C8A437347B80DF2C156F7EC7F057FB5FB0E@exchis.ccp.ad.local>
Message-ID: <h1tsuc$7rv$1@ger.gmane.org>

Kristj?n Valur J?nsson wrote:
> The OP was redirected here from the stackless list since his
> questions were not stackless specific.

Ok. He should have said so. Third party directions are not always right.

As it turns out, he did post to python-list but ignored the rather 
strong hint about not shouting in the subject line.

> Stackless python is in sync with the latest 2.x, 3.x branches
> as well as the trunk so performance
> problems he may with performance in general are perhaps best resolved
> by this lot.

He said he is using Stackless with the obsolete 2.5.2 release, not even 
the final and best 2.5.4 release with whatever bug fixes and speedups it 
has. Switching to the latter, at least, would be my first suggestion.

Terry


From ziade.tarek at gmail.com  Wed Jun 24 21:54:16 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Wed, 24 Jun 2009 21:54:16 +0200
Subject: [Python-Dev] PEP 376
In-Reply-To: <loom.20090624T091739-378@post.gmane.org>
References: <94bdd2610906220623k301e7924i5abc0a97d54741ba@mail.gmail.com>
	<loom.20090622T142204-980@post.gmane.org>
	<94bdd2610906220842y475aad09l27f5be4b185718a4@mail.gmail.com>
	<A315E325-654E-4827-978A-5B417185F220@bud.ca>
	<94bdd2610906230138o186c5fa5h27ed7d857cd9284b@mail.gmail.com>
	<4A40A721.6090204@gmail.com> <4A4130D9.5040108@activestate.com>
	<loom.20090624T091739-378@post.gmane.org>
Message-ID: <94bdd2610906241254m60d39c2cs99d255dc5223af91@mail.gmail.com>

On Wed, Jun 24, 2009 at 11:18 AM, Antoine Pitrou<solipsis at pitrou.net> wrote:
> Sridhar Ratnakumar <sridharr <at> activestate.com> writes:
>>
>> On 09-06-23 02:57 AM, Nick Coghlan wrote:
>> > Something like DistributionDirectoryMap should cover it.
>> >
>> > You could probably get away with shortening "Directory" to "Dir" in the
>> > class names though:
>> >
>> > ? - Distribution
>> > ? - ZippedDistribution
>> > ? - DistributionDir
>> > ? - ZippedDistributionDir
>> > ? - DistributionDirMap
>>
>> 'Map' reminds me of an associated hash (or is it the actual map?).
>
> Good thing, because it is a dict subclass if you read the PEP :)
>
> +1 with Nick's proposal.
>

I've changed using these names.  Any other feedback with this PEP or
we're good to go ? :)

Regards
Tarek

From mail at timgolden.me.uk  Wed Jun 24 22:05:10 2009
From: mail at timgolden.me.uk (Tim Golden)
Date: Wed, 24 Jun 2009 21:05:10 +0100
Subject: [Python-Dev] Python for Windows??
In-Reply-To: <4A424D55.6010404@cherokeetel.net>
References: <4A424D55.6010404@cherokeetel.net>
Message-ID: <4A4286F6.5090102@timgolden.me.uk>

David H. Burns wrote:
> I downloaded what claims to be Python for Windows (3.01). The tutorial 
> brags a lot about how easy it is to learn, but the tutorials and 
> instruction seem to be for a Linux or Unix version. There are three 
> executable programs in the Python directory and no indication which 
> should be used to start Python. One opens a Dos-like window presumably 
> for  "command-line" entry. I can't make anything of it.

[as pointed out elsewhere, this is the wrong list for
this question, but just to get you going...]

The Python docs have a section on running Python
on Windows. This is the online version:

   http://docs.python.org/using/windows.html

The .chm version should be in c:\python31\doc\python31.chm

TJG


From horos11 at gmail.com  Thu Jun 25 02:12:45 2009
From: horos11 at gmail.com (Edward Peschko)
Date: Wed, 24 Jun 2009 17:12:45 -0700
Subject: [Python-Dev] trace module options.
Message-ID: <5cfa99000906241712n6bfbb167n8b8652522a9be7f0@mail.gmail.com>

All,

I've been looking at the trace module, and although it looks useful, I'm
surprised that there aren't a couple of features that I would have thought
would be fairly basic. So, does trace support (for the --trace option):

    - indentation tracking stacklevel (where each function is prefixed
      by tabs equal to the number of stacklevels deep in the program)

    - output to something other than sys.stdout (eg. output to a file
      specified either by environmental variable or by parameter).

    - mult-threaded programs going to multiple output handles,
      especially in light of the above

    - fully qualified python modules in path: (eg:

          /path/to/module/my_module.py(1): print "HERE"

      instead of

        my_module.py(1): print "HERE".

Ultimately, I'd like to be able to look at two runs of a program
and be able to pinpoint the very first difference
between thembased on the output of their trace runs. As it
stands, I really can't do this.

Of course I could program the above, but I was
hoping to avoid duplicated effort if someone has
already programmed options like this..

Ed

From martin at v.loewis.de  Thu Jun 25 09:30:54 2009
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Thu, 25 Jun 2009 09:30:54 +0200
Subject: [Python-Dev] trace module options.
In-Reply-To: <5cfa99000906241712n6bfbb167n8b8652522a9be7f0@mail.gmail.com>
References: <5cfa99000906241712n6bfbb167n8b8652522a9be7f0@mail.gmail.com>
Message-ID: <4A4327AE.7000907@v.loewis.de>

> Of course I could program the above, but I was
> hoping to avoid duplicated effort if someone has
> already programmed options like this..

In this specific form, the question is off-topic for python-dev,
and better asked on python-list (in particular since it has a wider
audience).

On-topic would have been a disussion of which of these features would
be useful for the trace module to be built-in, how to best provide them,
along with an offer to produce a patch.

Regards,
Martin

From ncoghlan at gmail.com  Thu Jun 25 12:11:10 2009
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Thu, 25 Jun 2009 20:11:10 +1000
Subject: [Python-Dev] Python for Windows??
In-Reply-To: <4A4286F6.5090102@timgolden.me.uk>
References: <4A424D55.6010404@cherokeetel.net>
	<4A4286F6.5090102@timgolden.me.uk>
Message-ID: <4A434D3E.4090802@gmail.com>

Tim Golden wrote:
> The Python docs have a section on running Python
> on Windows. This is the online version:
> 
>   http://docs.python.org/using/windows.html
> 
> The .chm version should be in c:\python31\doc\python31.chm

Which is more easily accessed via the start menu entry Python creates
for you... (as is the interpreter itself for that matter)

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------

From horos11 at gmail.com  Thu Jun 25 14:54:21 2009
From: horos11 at gmail.com (Edward Peschko)
Date: Thu, 25 Jun 2009 05:54:21 -0700
Subject: [Python-Dev] trace module options.
In-Reply-To: <4A4327AE.7000907@v.loewis.de>
References: <5cfa99000906241712n6bfbb167n8b8652522a9be7f0@mail.gmail.com> 
	<4A4327AE.7000907@v.loewis.de>
Message-ID: <5cfa99000906250554p67ab3038y5a3e6be3f9c6a48c@mail.gmail.com>

> On-topic would have been a disussion of which of these features would
> be useful for the trace module to be built-in, how to best provide them,
> along with an offer to produce a patch.

Ok, fair enough - which of the features listed (indentation matching
stacklevel, output to a file (or files wrt threading, fully qualified
output) should be builtin, which should be options, which should be
left out, and based on the above, would a patch like this be accepted?

Ed

(ps - I'll also take it to python-list)

From lanyjie at yahoo.com  Thu Jun 25 17:40:58 2009
From: lanyjie at yahoo.com (Yingjie Lan)
Date: Thu, 25 Jun 2009 08:40:58 -0700 (PDT)
Subject: [Python-Dev] Syntax for parsing tuples/lists into C arrays
Message-ID: <487033.84090.qm@web54208.mail.re2.yahoo.com>


The current C API does not support directly parsing into C arrays. Parsing tuples into C arrays can be greatly facilitated with the introduction of a natural extension of the current parsing syntax. I'd like to propose a syntax extension to facilitate that kind of tasks, and would appreciate your feedbacks. The extension comes from the way strings are parsed in current C API: 

{
const char * s; int i;
PyArg_ParseTyple(args, "s#", &s, &i);
}

Consider for a moment the string as a sequence of char's, which by the example above is parsed into an array of char's. Note that in python, a tuple of some type (e.g. int's), is also a sequence of that type, This perspective provides a natural way to extend the the use of "#" symbol -- what is missing is how to signal a tuple as a sequence, without interfering with current fixed-length tuple parsing syntax. To signal a tuple as a sequence of a given type of objects, "(type:)" can be used. For example, "(i:)" signals a tuple of integers as array of integers. The ":" is chosen to denote possible repetition of the whole pattern inside the parentheses. 

Now consider 3 different scenarios of parsing tuples into arrays.

Scenario (1) Unkown size. Here are some examples:

{
//parsing a tuple and its size
int i, *t;
PyArg_ParseTuple(args, "(i:)#", &t, &i);
}

{
//parsing a matrix of i rows and j columns:
int i, j, **t;
PyArg_ParseTuple(args, "((i:):)##", &t, &i, &j);
}

Here is an informal explanation of "((i:):)##": the '#' can be seen as a peeling operator, which peels away the outer most "(...:)" that denote a "free sequence" (i.e. sequence of unknown size) with some pattern "...".

For types like 's' or its similar types, pretend as if there is an invisible pair of parentheses around it. For example, "(s:)##" matches a tuple of strings of some fixed length. Here the first '#' peels away the parentheses, leaving only "s#", which is equivalent to "(*char*:)#" in concept, where "*char*" denote the character type (not exist in Python). As the second '#' is outside the tuple, it is shared by all strings inside, so they _must_ have the same length.

If there is nothing to peel for a '#', it is considered a dangling '#', an error. Now consider some more interesting examples:

{
int i, *t, *v;
PyArg_ParseTuple(args, "(ii:)#", &t, &v, &i);
//args=(1,2,3,4,5,6)
//t={1,3,5}
//v={2,4,6}
//i=3
}

In that example, a pattern is provided inside the sequence "(ii:)", which can repeat itself multiple times. The repeated times is extracted by the '#' symbol. Below is another example:

{
int i,j, *t, **v;
PyArg_ParseTuple(args, "(i(i:):)##", &t, &v, &i, &j);
//args=(1,(3,),5,(7,))
//t={1,5}
//v={{3},{7}}
//i=2, j=1
}

And another one:

{
int i,j, **t, **v;
PyArg_ParseTuple(args, "((i:)(i:):)##", &t, &v, &i, &j);
//args=((1,),(3,),(5,),(7,))
//t={{1},{5}}
//v={{3},{7}}
//i=2, j=1
}

Note the second '#' is applied to both the free sequences inside the outer tuple. This indicates that all the inside tuples must be of the same length. What if they have their own lengths? We provide a separate matching '#' for each inside tuple:

{
int i,j,k, **t, **v;
PyArg_ParseTuple(args, "((i:)(i:):)#(##)", &t, &v, &i, &j,&k);
//args=((1,),(3,4),(5,),(7,8))
//t={{1},{5}}
//v={{3,4},{7,8}}
//i=2, j=1, k=2
}

The syntax can also deal with more complicated situations, such as an irregular 2-dimensional array, where each row has a different row length:

{
int i, *j, **t;
PyArg_ParseTuple(args, "((i:)#:)#", &t, &j, &i);
for(a=0; a)
//t={{{1,2},{1,2}},{{1,2}}}
//i=2, k=2
//j={2,1}
}

This syntax can be used for parsing a tuple of strings of different lengths:

{
int i, *j;
const char ** s;
PyArg_ParseTuple(args, "(s#:)#", &s, &j, &i);
//args=("ab", "cde")
//s={"ab", "cde"}
//j={2, 3}
//i=2
}


Scenario (2) If you know the size of a tuple before hand, use this syntax: "(*type*integer)", meaning there are exactly 'integer' number of *type* objects in the tuple.

{
int *t;
//
PyArg_ParseTuple(args, "(i5)", &t);
//args = (1,2,3,4,5)
//t = {1,2,3,4,5}
}

A complicated case:
{
int **t;
PyArg_ParseTuple(args, "((i5)2)", &t);
//args=((1,2,3,4,5),(5,6,7,8,9))
//t={{1,2,3,4,5},{5,6,7,8,9}}
}

Another complicated case:

{
int **t, **v, i;
PyArg_ParseTuple(args, "(i3i2:)#", &t,&v,&i);
//args=(1,2,3,4,5,6,7,8,9,10)
//t={{1,2,3},{6,7,8}}
//v={{4,5},{9,10}}
//i=2
}

One more thing: the difference between "(ii)" and "(i2)" is that the former yields two single integers, while the latter gives an array of length two.

Scenario (3) Concatenating arrays while parsing. Sometimes you would like to have a matrix parsed into a single array. We can use a "(+" to signal that the arrays parsed from the elements in that tuple should be concatenated into a single array.

{
int i,j, *t;
PyArg_ParseTuple(args, "(+(i))##", &t, &i, &j);
//args=((1,2,3),(4,5,6))
//t={1,2,3,4,5,6}
//i=2, j=3
}

Another example:

{
int i, *t;
PyArg_ParseTuple(args, "(+(i2))#", &t, &i);
}

Still another example, with strings:
{
int i, *j;
const char * t; //string
PyArg_ParseTuple(args, "(+(+s#2))#", &t, &j, &i);
//args=(("ab","c"),("d","efg"))
//t="abcdefg"
//i=2
////notice the size arrays are concatenated similarly
//j={2,1,1,3} //without '+': j={{2,1},{1,3}}

const char ** sa; 
//The '*' in place of '#' to discard the size info
PyArg_ParseTuple(args, "(+(s*2))", &sa);
//args=(("ab","c"),("d","efg"))
//sa = {"ab","c","d","efg"}

PyArg_ParseTuple(args, "((+s*2))", &sa);
//args=(("ab","c"),("d","efg"))
//sa = { "abc", "defg" }
}

And finally, an example with multi-item pattern:

{
int *t, *v, i;
PyArg_ParseTuple(args, "(+i1i2:)#", &t, &v, &i);
//args=(1,2,3,4,5,6)
//t={1,4}
//v={2,3,5,6}
//i=2
}

Epilogue:

This syntax can be applied to lists: just use "[...:]" instead of "(...:)". It can also be used for building tuples or lists out of arrays.

That's it. Comments?

Yingjie


      

From aahz at pythoncraft.com  Thu Jun 25 19:49:22 2009
From: aahz at pythoncraft.com (Aahz)
Date: Thu, 25 Jun 2009 10:49:22 -0700
Subject: [Python-Dev] Syntax for parsing tuples/lists into C arrays
In-Reply-To: <487033.84090.qm@web54208.mail.re2.yahoo.com>
References: <487033.84090.qm@web54208.mail.re2.yahoo.com>
Message-ID: <20090625174922.GB2323@panix.com>

On Thu, Jun 25, 2009, Yingjie Lan wrote:
>
> The current C API does not support directly parsing into C
> arrays. Parsing tuples into C arrays can be greatly facilitated
> with the introduction of a natural extension of the current parsing
> syntax. I'd like to propose a syntax extension to facilitate that kind
> of tasks, and would appreciate your feedbacks. 

This should probably get reposted to python-ideas.
-- 
Aahz (aahz at pythoncraft.com)           <*>         http://www.pythoncraft.com/

"as long as we like the same operating system, things are cool." --piranha

From martin at v.loewis.de  Thu Jun 25 20:40:47 2009
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Thu, 25 Jun 2009 20:40:47 +0200
Subject: [Python-Dev] Syntax for parsing tuples/lists into C arrays
In-Reply-To: <487033.84090.qm@web54208.mail.re2.yahoo.com>
References: <487033.84090.qm@web54208.mail.re2.yahoo.com>
Message-ID: <4A43C4AF.5000907@v.loewis.de>

> Comments?

I don't think there is any need to support that in the Python core.

Instead, if you find such a functionality useful, feel free to implement
it yourself, and share it with others.

Regards,
Martin

From martin at v.loewis.de  Thu Jun 25 21:30:11 2009
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Thu, 25 Jun 2009 21:30:11 +0200
Subject: [Python-Dev] ndPython: I NEED TO TALK WITH ONE OF THE PYTHON
 CORE	DEVELOPERS
In-Reply-To: <006801c9f4dd$c18294c0$011ea8c0@pegasus>
References: <006801c9f4dd$c18294c0$011ea8c0@pegasus>
Message-ID: <4A43D043.5080406@v.loewis.de>

> Can you tell me which modules provide to
> decode and execute a .pyc file ?

The bytecode loader in in marshal.c; the byte code interpreter is in
ceval.c. Also consider frameobject.c and codeobject.c.

HTH,
Martin

From barry at python.org  Fri Jun 26 15:30:24 2009
From: barry at python.org (Barry Warsaw)
Date: Fri, 26 Jun 2009 09:30:24 -0400
Subject: [Python-Dev] [Python-checkins] r73569 - peps/trunk/pep-0101.txt
In-Reply-To: <20090626124903.C330BE3B84@mail.wooz.org>
References: <20090626124903.C330BE3B84@mail.wooz.org>
Message-ID: <221034BF-8E59-4557-AEED-87A64E85A143@python.org>

On Jun 26, 2009, at 8:49 AM, benjamin.peterson wrote:

> Author: benjamin.peterson
> Date: Fri Jun 26 14:48:55 2009
> New Revision: 73569
>
> Log:
> update release candidate shorthand
>
> Modified:
>   peps/trunk/pep-0101.txt
>
> Modified: peps/trunk/pep-0101.txt
> =
> =
> =
> =
> =
> =
> =
> =
> ======================================================================
> --- peps/trunk/pep-0101.txt	(original)
> +++ peps/trunk/pep-0101.txt	Fri Jun 26 14:48:55 2009
> @@ -66,7 +66,7 @@
>
>     We use the following conventions in the examples below.  Where a  
> release
>     number is given, it is of the form X.YaZ, e.g. 2.6a3 for Python  
> 2.6 alpha
> -    3, where "a" == alpha, "b" == beta, "c" == release candidate.
> +    3, where "a" == alpha, "b" == beta, "rc" == release candidate.
>
>     Final releases are named "releaseXY".  The branch tag is  
> "releaseXY-maint"
>     because this will point to the long lived maintenance branch.   
> The fork

I'm sure this has been discussed but I missed it.  Why was this change  
made?  If nothing else, it breaks many years of tradition.

-Barry

-------------- next part --------------
A non-text attachment was scrubbed...
Name: PGP.sig
Type: application/pgp-signature
Size: 832 bytes
Desc: This is a digitally signed message part
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090626/65737574/attachment.pgp>

From benjamin at python.org  Fri Jun 26 15:35:00 2009
From: benjamin at python.org (Benjamin Peterson)
Date: Fri, 26 Jun 2009 08:35:00 -0500
Subject: [Python-Dev] [Python-checkins] r73569 - peps/trunk/pep-0101.txt
In-Reply-To: <221034BF-8E59-4557-AEED-87A64E85A143@python.org>
References: <20090626124903.C330BE3B84@mail.wooz.org>
	<221034BF-8E59-4557-AEED-87A64E85A143@python.org>
Message-ID: <1afaf6160906260635m3ca2bbbara439991adf70e0f0@mail.gmail.com>

2009/6/26 Barry Warsaw <barry at python.org>:
> I'm sure this has been discussed but I missed it. ?Why was this change made?
> ?If nothing else, it breaks many years of tradition.

I assumed it was the tradition because all the 3.0 and 2.6 candidates had "rc".



-- 
Regards,
Benjamin

From Scott.Daniels at Acm.Org  Fri Jun 26 17:29:22 2009
From: Scott.Daniels at Acm.Org (Scott David Daniels)
Date: Fri, 26 Jun 2009 08:29:22 -0700
Subject: [Python-Dev] [Python-checkins] r73569 - peps/trunk/pep-0101.txt
In-Reply-To: <221034BF-8E59-4557-AEED-87A64E85A143@python.org>
References: <20090626124903.C330BE3B84@mail.wooz.org>
	<221034BF-8E59-4557-AEED-87A64E85A143@python.org>
Message-ID: <h22p3n$ffu$1@ger.gmane.org>

While we are on that, I just noticed:
     http://www.python.org/download/releases/3.1/
Has downloads for 3.1rc2, but show checksums as if for 3.1rc1
The size and checksum is correct for python-3.1rc2.msi,
distinct from that for python-3.1rc1.msi, but are labeled as rc1.
The 32-bit .msi is the only one of the four I checked; I suspect
the other three are similarly mislabeled.

--Scott David Daniels
Scott.Daniels at Acm.Org




From status at bugs.python.org  Fri Jun 26 18:07:17 2009
From: status at bugs.python.org (Python tracker)
Date: Fri, 26 Jun 2009 18:07:17 +0200 (CEST)
Subject: [Python-Dev] Summary of Python tracker Issues
Message-ID: <20090626160717.9D56E78617@psf.upfronthosting.co.za>


ACTIVITY SUMMARY (06/19/09 - 06/26/09)
Python tracker at http://bugs.python.org/

To view or respond to any of the issues listed below, click on the issue 
number.  Do NOT respond to this message.


 2242 open (+17) / 15925 closed (+16) / 18167 total (+33)

Open issues with patches:   889

Average duration of open issues: 657 days.
Median duration of open issues: 408 days.

Open Issues Breakdown
   open  2212 (+17)
pending    30 ( +0)

Issues Created Or Reopened (33)
_______________________________

test_with.py has a couple minor mistakes                         06/19/09
CLOSED http://bugs.python.org/issue6313    created  exarkun                       
       patch                                                                   

logging.basicConfig(level='DEBUG', ...                           06/20/09
CLOSED http://bugs.python.org/issue6314    created  alexl                         
                                                                               

locale._build_localename(locale.getdefaultlocale()) returns 'C.m 06/20/09
CLOSED http://bugs.python.org/issue6315    created  sjt                           
                                                                               

format, str.format don't work well with datetime, date object    06/20/09
CLOSED http://bugs.python.org/issue6316    created  falsetru                      
                                                                               

winsound.PlaySound doesn't accept non-unicode string             06/21/09
       http://bugs.python.org/issue6317    created  ocean-city                    
       patch                                                                   

HTMLParser Attributes Containing Escaped Quotes                  06/21/09
CLOSED http://bugs.python.org/issue6318    created  ericryk                       
                                                                               

bdist_msi runs out of memory for large packages                  06/21/09
CLOSED http://bugs.python.org/issue6319    created  bethard                       
       patch                                                                   

Standard string encodings should include GSM0.38                 06/21/09
       http://bugs.python.org/issue6320    created  jwishnie                      
                                                                               

Reload Python modules when running programs                      06/22/09
       http://bugs.python.org/issue6321    created  samwyse                       
                                                                               

Pdb breakpoints don't work on lines without bytecode             06/22/09
       http://bugs.python.org/issue6322    created  inducer                       
                                                                               

Py3.1 pdb doesn't deal well with syntax errors                   06/22/09
       http://bugs.python.org/issue6323    created  inducer                       
       patch, needs review                                                     

"in" expression falls back to __iter__ before __getitem__        06/22/09
       http://bugs.python.org/issue6324    created  afoglia                       
                                                                               

robotparser doesn't handle URL's with query strings              06/23/09
       http://bugs.python.org/issue6325    created  skybrian                      
                                                                               

Add a "swap" method to list                                      06/23/09
       http://bugs.python.org/issue6326    created  krisvale                      
       patch, patch                                                            

[mimetext] long lines get cut with exclamation mark and newline  06/23/09
       http://bugs.python.org/issue6327    created  martijntje                    
                                                                               

login() function failed in smtplib with message "argument 1 must 06/23/09
CLOSED http://bugs.python.org/issue6328    created  hdvision                      
                                                                               

Fix iteration for memoryviews                                    06/23/09
CLOSED http://bugs.python.org/issue6329    created  rhettinger                    
       patch                                                                   

trunk does not build with --enable-unicode=ucs4                  06/23/09
       http://bugs.python.org/issue6330    created  eric.smith                    
       easy                                                                    

Add unicode script info to the unicode database                  06/23/09
       http://bugs.python.org/issue6331    created  doerwalter                    
       patch                                                                   

typo on man page warning control                                 06/23/09
CLOSED http://bugs.python.org/issue6332    created  lieryan                       
       patch                                                                   

logging: ValueError: I/O operation on closed file                06/24/09
       http://bugs.python.org/issue6333    created  srid                          
                                                                               

3.0/3.1: Bad bug in range() computation (or possible Integer pro 06/24/09
CLOSED http://bugs.python.org/issue6334    created  mfxmfx                        
       patch                                                                   

Add support for mingw                                            06/24/09
       http://bugs.python.org/issue6335    created  smartmobili                   
       patch                                                                   

nb_divide missing in docs                                        06/24/09
       http://bugs.python.org/issue6336    created  flub                          
                                                                               

multiprocessing module: Double close of sys.stdin - ID: 2811568  06/24/09
CLOSED http://bugs.python.org/issue6337    created  subdir                        
       patch                                                                   

Error message displayed on stderr when no C compiler is present  06/24/09
       http://bugs.python.org/issue6338    created  ehuss                         
       patch                                                                   

Some functional errors in turtle.py documentation (missing links 06/24/09
CLOSED http://bugs.python.org/issue6339    created  gregorlingl                   
                                                                               

replace tdemo_chaos.py                                           06/24/09
CLOSED http://bugs.python.org/issue6340    created  gregorlingl                   
                                                                               

io.path.ismount  gives "local variable 'p' referenced before ass 06/25/09
CLOSED http://bugs.python.org/issue6341    created  g.moralis                     
                                                                               

io.path.ismount  gives "local variable 'p' referenced before ass 06/25/09
CLOSED http://bugs.python.org/issue6342    created  g.moralis                     
                                                                               

TimedRotatingFileHandler permission denied rename failure on Win 06/25/09
       http://bugs.python.org/issue6343    created  dnagata                       
                                                                               

mmap.read() crashes when passed a negative argument              06/25/09
       http://bugs.python.org/issue6344    created  amaury.forgeotdarc            
                                                                               

extra characters displayed when writing to sys.stdout.buffer     06/26/09
CLOSED http://bugs.python.org/issue6345    created  metolone                      
                                                                               



Issues Now Closed (31)
______________________

zlib.crc32() and adler32() return value                           263 days
       http://bugs.python.org/issue1202    gregory.p.smith               
       64bit                                                                   

Tix HList class missing method implementation for info_bbox       627 days
       http://bugs.python.org/issue1230    gpolo                         
                                                                               

xml/sax/expatreader.py raises AttributeError when run             202 days
       http://bugs.python.org/issue4490    amaury.forgeotdarc            
       patch                                                                   

test_tcl testLoadTk fails if DISPLAY defined but connect	fails,   105 days
       http://bugs.python.org/issue5450    gpolo                         
       patch                                                                   

pyexpat defines global symbol template_string                      86 days
       http://bugs.python.org/issue5590    doko                          
                                                                               

SimpleXMLRPCServer not suitable for HTTP/1.1 keep-alive            27 days
       http://bugs.python.org/issue6096    loewis                        
       patch, patch, easy, needs review                                        

HTTP/1.1 with keep-alive support for xmlrpclib.ServerProxy         27 days
       http://bugs.python.org/issue6099    loewis                        
       patch, patch, needs review                                              

Backport the IO lib to trunk                                       15 days
       http://bugs.python.org/issue6215    pitrou                        
       patch                                                                   

subprocess.Popen() may leak file descriptors                        7 days
       http://bugs.python.org/issue6274    facundobatista                
       patch                                                                   

Update contextlib.nested docs in light of deprecation               6 days
       http://bugs.python.org/issue6288    ncoghlan                      
                                                                               

virtual memory error with archivemail                               0 days
       http://bugs.python.org/issue6311    loewis                        
                                                                               

test_with.py has a couple minor mistakes                            0 days
       http://bugs.python.org/issue6313    benjamin.peterson             
       patch                                                                   

logging.basicConfig(level='DEBUG', ...                              2 days
       http://bugs.python.org/issue6314    vsajip                        
                                                                               

locale._build_localename(locale.getdefaultlocale()) returns 'C.m    5 days
       http://bugs.python.org/issue6315    r.david.murray                
                                                                               

format, str.format don't work well with datetime, date object       0 days
       http://bugs.python.org/issue6316    falsetru                      
                                                                               

HTMLParser Attributes Containing Escaped Quotes                     0 days
       http://bugs.python.org/issue6318    georg.brandl                  
                                                                               

bdist_msi runs out of memory for large packages                     0 days
       http://bugs.python.org/issue6319    bethard                       
       patch                                                                   

login() function failed in smtplib with message "argument 1 must    0 days
       http://bugs.python.org/issue6328    r.david.murray                
                                                                               

Fix iteration for memoryviews                                       0 days
       http://bugs.python.org/issue6329    rhettinger                    
       patch                                                                   

typo on man page warning control                                    0 days
       http://bugs.python.org/issue6332    georg.brandl                  
       patch                                                                   

3.0/3.1: Bad bug in range() computation (or possible Integer pro    1 days
       http://bugs.python.org/issue6334    mfxmfx                        
       patch                                                                   

multiprocessing module: Double close of sys.stdin - ID: 2811568     0 days
       http://bugs.python.org/issue6337    jnoller                       
       patch                                                                   

Some functional errors in turtle.py documentation (missing links    1 days
       http://bugs.python.org/issue6339    r.david.murray                
                                                                               

replace tdemo_chaos.py                                              1 days
       http://bugs.python.org/issue6340    r.david.murray                
                                                                               

io.path.ismount  gives "local variable 'p' referenced before ass    0 days
       http://bugs.python.org/issue6341    r.david.murray                
                                                                               

io.path.ismount  gives "local variable 'p' referenced before ass    0 days
       http://bugs.python.org/issue6342    amaury.forgeotdarc            
                                                                               

extra characters displayed when writing to sys.stdout.buffer        0 days
       http://bugs.python.org/issue6345    benjamin.peterson             
                                                                               

curselection() in Tkinter.py should return ints                  1997 days
       http://bugs.python.org/issue869780  gpolo                         
       patch                                                                   

Add XML-RPC Fault Interoperability to XMLRPC libraries           1310 days
       http://bugs.python.org/issue1360243 loewis                        
       patch, easy                                                             

Tix ComboBox entry is blank when not editable                     944 days
       http://bugs.python.org/issue1600182 gpolo                         
                                                                               

syslog syscall support for SysLogLogger                           782 days
       http://bugs.python.org/issue1711603 LwarX                         
       patch                                                                   



Top Issues Most Discussed (10)
______________________________

  9 Native (and default) tarfile support for setup.py sdist	in dist    9 days
open    http://bugs.python.org/issue6296   

  9 ElementTree (py3k) doesn't properly encode characters that can'   19 days
open    http://bugs.python.org/issue6233   

  9 Python 2.6 makes .pyc/.pyo bytecode files executable              37 days
open    http://bugs.python.org/issue6070   

  7 3.0/3.1: Bad bug in range() computation (or possible Integer pr    1 days
closed  http://bugs.python.org/issue6334   

  6 cElementTree.iterparse & ElementTree.iterparse return different   15 days
open    http://bugs.python.org/issue6266   

  5 Add a "swap" method to list                                        3 days
open    http://bugs.python.org/issue6326   

  5 Py3.1 pdb doesn't deal well with syntax errors                     4 days
open    http://bugs.python.org/issue6323   

  5 os.popen causes illegal seek on AIX in Python 3.1rc               19 days
open    http://bugs.python.org/issue6236   

  5 test_tcl testLoadTk fails if DISPLAY defined but connect	fails,  105 days
closed  http://bugs.python.org/issue5450   

  5 Document auto __ne__ generation; provide a use case for non-tri  215 days
open    http://bugs.python.org/issue4395   




From fbattaglia at alice.it  Fri Jun 26 17:45:15 2009
From: fbattaglia at alice.it (Filippo Battaglia)
Date: Fri, 26 Jun 2009 17:45:15 +0200
Subject: [Python-Dev] ndPython: I NEED TO TALK WITH ONE OF THE PYTHON
	CORE
Message-ID: <000c01c9f675$19917760$011ea8c0@pegasus>

Thanks for your answers. 
Sorry for the title in upper case. I didn't
want to create troubles. 

:)

I've an important question for you: is it
possible that a large python module,
created using SWIG and with a hundred
of routines, makes slower the execution
(i.e. the job of ceval.c) of the Python
interpreter ?

We've observed that, if we don't import
ndpsp.pyc at startup, the time of execution
of a loop containing the pass instruction
becomes near normal. 

How Python recalls the C functions in
a C wrapper ?

Thanks for your very important help.
Filippo Battaglia




From tjreedy at udel.edu  Fri Jun 26 22:25:02 2009
From: tjreedy at udel.edu (Terry Reedy)
Date: Fri, 26 Jun 2009 16:25:02 -0400
Subject: [Python-Dev] ndPython: I NEED TO TALK WITH ONE OF THE PYTHON
	CORE
In-Reply-To: <000c01c9f675$19917760$011ea8c0@pegasus>
References: <000c01c9f675$19917760$011ea8c0@pegasus>
Message-ID: <h23ar0$vsp$1@ger.gmane.org>

Filippo Battaglia wrote:
> Thanks for your answers. Sorry for the title in upper case. I didn't
> want to create troubles.
> :)
> 
> I've an important question for you: is it
> possible that a large python module,
> created using SWIG and with a hundred
> of routines, makes slower the execution
> (i.e. the job of ceval.c) of the Python
> interpreter ?

If you were running on a PC with what is now considered to be very small 
memory, I would hypothesize that you had filled memory so that the 
interpreter or parts thereof were being swapped in and out of memory 
from and to disk. Is any thing like that possible with the PSP?

Next, I would wonder whether any module, as part of its initialization, 
was doing anything 'unusual' with respect to its interaction with the 
interpreter.
> 
> We've observed that, if we don't import
> ndpsp.pyc at startup, the time of execution
> of a loop containing the pass instruction
> becomes near normal.

What happens if you divide the imported stuff in half?
Do both halves slow it down? Neither? Just one?

The answer to that would be a start to answering whether the specific 
problem is quantitative or qualitative.

Terry Jan Reedy


From flormayer at aim.com  Sat Jun 27 20:43:18 2009
From: flormayer at aim.com (Florian Mayer)
Date: Sat, 27 Jun 2009 18:43:18 +0000
Subject: [Python-Dev] pthread sem PyThread_acquire_lock
Message-ID: <4A466846.1050904@aim.com>

Hello.
As this is my first post I will try to introduce myself as requested in 
the welcome email. If you aren't interested in my person, just continue 
reading at the next paragraph. I'm a student from Vienna/Austria. I 
attend what would match high school in the United States. I have been 
writing Python for about 3 years now and have just began to dig into the 
implementation of CPython.

Now to my real question. I have noticed that PyThread_acquire_lock 
swallows all signals with pthread using sems. Looking at the source code 
it appears that this is intended, but I cannot see the reason for that. 
It seems the pthread sem implementation is the only one doing so. Can 
any of you tell me the reason why it swallows signals?

I have already prepared a patch that introduces a new _noswallow 
function if sem are used and uses this in threadmodule.c. But if there 
isn't any real reason for it to swallow exceptions, I think it would be 
wisest to change that.

Best regards,
Florian Mayer

From martin at v.loewis.de  Sat Jun 27 22:45:59 2009
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sat, 27 Jun 2009 22:45:59 +0200
Subject: [Python-Dev] pthread sem PyThread_acquire_lock
In-Reply-To: <4A466846.1050904@aim.com>
References: <4A466846.1050904@aim.com>
Message-ID: <4A468507.1060503@v.loewis.de>

> Now to my real question. I have noticed that PyThread_acquire_lock
> swallows all signals with pthread using sems. Looking at the source code
> it appears that this is intended, but I cannot see the reason for that.
> It seems the pthread sem implementation is the only one doing so. Can
> any of you tell me the reason why it swallows signals?

Can you please explain what you mean by "swallowing"?
What is the sequence of actions triggering the case you are talking
about, what happens, and what do you expect to happen instead?
Also, how would you fix this (in principle, not what the specific
patch would look like)?

Regards,
Martin

From flormayer at aim.com  Sun Jun 28 00:50:18 2009
From: flormayer at aim.com (Florian Mayer)
Date: Sat, 27 Jun 2009 22:50:18 +0000
Subject: [Python-Dev] pthread sem PyThread_acquire_lock
In-Reply-To: <4A468507.1060503@v.loewis.de>
References: <4A466846.1050904@aim.com> <4A468507.1060503@v.loewis.de>
Message-ID: <4A46A22A.40400@aim.com>

Martin v. L?wis wrote:
 > Can you please explain what you mean by "swallowing"?
The signals don't get through when acquiring the lock.
 > What is the sequence of actions triggering the case you are talking
 > about, what happens, and what do you expect to happen instead?
Create a Lock object, call .acquire and try to press Ctrl-C, the 
keyboardinterrupt is swallowed
 > Also, how would you fix this (in principle, not what the specific
 > patch would look like)?
Remove the loop that explicitly does this in a new function and use that 
one in threadmodule.c for lock.acquire.
 >
 > Regards,
 > Martin
Best regards,
Florian

PS: Sorry for sending this message twice.

From flormayer at aim.com  Sun Jun 28 00:58:19 2009
From: flormayer at aim.com (Florian Mayer)
Date: Sat, 27 Jun 2009 22:58:19 +0000
Subject: [Python-Dev] pthread sem PyThread_acquire_lock
In-Reply-To: <4A468507.1060503@v.loewis.de>
References: <4A466846.1050904@aim.com> <4A468507.1060503@v.loewis.de>
Message-ID: <4A46A40B.5080202@aim.com>

Martin v. L?wis wrote:
> Can you please explain what you mean by "swallowing"?
The signals don't get through when acquiring the lock.
> What is the sequence of actions triggering the case you are talking
> about, what happens, and what do you expect to happen instead?
Easiest way is to create a Queue.Queue and call .get() on it and try to 
Ctrl-C it.
> Also, how would you fix this (in principle, not what the specific
> patch would look like)?
Remove the loop that explicitly does this in a new function and use that
one in threadmodule.c for lock.acquire.
> 
> Regards,
> Martin
Best regards,
Florian

PS: Excuse me, I messed up things once again.


From benjamin at python.org  Sat Jun 27 23:12:10 2009
From: benjamin at python.org (Benjamin Peterson)
Date: Sat, 27 Jun 2009 16:12:10 -0500
Subject: [Python-Dev] [RELEASED] Python 3.1 final
Message-ID: <1afaf6160906271412v3ca3ef9bo4efa4523db3a3685@mail.gmail.com>

On behalf of the Python development team, I'm thrilled to announce the first
production release of Python 3.1.

Python 3.1 focuses on the stabilization and optimization of the features and
changes that Python 3.0 introduced.  For example, the new I/O system has been
rewritten in C for speed.  File system APIs that use unicode strings now handle
paths with undecodable bytes in them. Other features include an ordered
dictionary implementation, a condensed syntax for nested with statements, and
support for ttk Tile in Tkinter.  For a more extensive list of changes in 3.1,
see http://doc.python.org/3.1/whatsnew/3.1.html or Misc/NEWS in the Python
distribution.

To download Python 3.1 visit:

     http://www.python.org/download/releases/3.1/

The 3.1 documentation can be found at:

     http://docs.python.org/3.1

Bugs can always be reported to:

     http://bugs.python.org


Enjoy!

--
Benjamin Peterson
Release Manager
benjamin at python.org
(on behalf of the entire python-dev team and 3.1's contributors)

From python at rcn.com  Sun Jun 28 00:47:56 2009
From: python at rcn.com (Raymond Hettinger)
Date: Sat, 27 Jun 2009 15:47:56 -0700
Subject: [Python-Dev] [RELEASED] Python 3.1 final
References: <1afaf6160906271412v3ca3ef9bo4efa4523db3a3685@mail.gmail.com>
Message-ID: <0035D0ABC9D34D12B5D695005B39D9AC@RaymondLaptop1>



> On behalf of the Python development team, I'm thrilled to announce the first
> production release of Python 3.1.

Sweet!


Raymond

From lists at cheimes.de  Sun Jun 28 00:59:48 2009
From: lists at cheimes.de (Christian Heimes)
Date: Sun, 28 Jun 2009 00:59:48 +0200
Subject: [Python-Dev] [RELEASED] Python 3.1 final
In-Reply-To: <1afaf6160906271412v3ca3ef9bo4efa4523db3a3685@mail.gmail.com>
References: <1afaf6160906271412v3ca3ef9bo4efa4523db3a3685@mail.gmail.com>
Message-ID: <4A46A464.2090704@cheimes.de>

Benjamin Peterson schrieb:
> On behalf of the Python development team, I'm thrilled to announce the first
> production release of Python 3.1.

Gratulations! You did a fantastic job! :)

Christian

PS: I hope that I'm able to spare more of my work time on the
development of Python 3.2 ...

From arcriley at gmail.com  Sun Jun 28 01:12:16 2009
From: arcriley at gmail.com (Arc Riley)
Date: Sat, 27 Jun 2009 19:12:16 -0400
Subject: [Python-Dev] [RELEASED] Python 3.1 final
In-Reply-To: <4A46A464.2090704@cheimes.de>
References: <1afaf6160906271412v3ca3ef9bo4efa4523db3a3685@mail.gmail.com>
	<4A46A464.2090704@cheimes.de>
Message-ID: <bad82a80906271612m5eeda743y1a4d5ed033ba1448@mail.gmail.com>

On Sat, Jun 27, 2009 at 6:59 PM, Christian Heimes <lists at cheimes.de> wrote:

>
> Gratulations! You did a fantastic job! :)


+1 !
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090627/b9807e14/attachment-0001.htm>

From ncoghlan at gmail.com  Sun Jun 28 02:53:49 2009
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sun, 28 Jun 2009 10:53:49 +1000
Subject: [Python-Dev] [RELEASED] Python 3.1 final
In-Reply-To: <1afaf6160906271412v3ca3ef9bo4efa4523db3a3685@mail.gmail.com>
References: <1afaf6160906271412v3ca3ef9bo4efa4523db3a3685@mail.gmail.com>
Message-ID: <4A46BF1D.2090002@gmail.com>

Benjamin Peterson wrote:
> On behalf of the Python development team, I'm thrilled to announce the first
> production release of Python 3.1.

Excellent news!

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------

From g.brandl at gmx.net  Sun Jun 28 12:18:54 2009
From: g.brandl at gmx.net (Georg Brandl)
Date: Sun, 28 Jun 2009 10:18:54 +0000
Subject: [Python-Dev] [RELEASED] Python 3.1 final
In-Reply-To: <1afaf6160906271412v3ca3ef9bo4efa4523db3a3685@mail.gmail.com>
References: <1afaf6160906271412v3ca3ef9bo4efa4523db3a3685@mail.gmail.com>
Message-ID: <h27g2q$hqm$1@ger.gmane.org>

Benjamin Peterson schrieb:
> On behalf of the Python development team, I'm thrilled to announce the first
> production release of Python 3.1.

Congrats!

Georg

-- 
Thus spake the Lord: Thou shalt indent with four spaces. No more, no less.
Four shall be the number of spaces thou shalt indent, and the number of thy
indenting shall be four. Eight shalt thou not indent, nor either indent thou
two, excepting that thou then proceed to four. Tabs are right out.


From martin at v.loewis.de  Mon Jun 29 09:39:32 2009
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Mon, 29 Jun 2009 09:39:32 +0200
Subject: [Python-Dev] 2.4.7: Final 2.4 release
Message-ID: <4A486FB4.9070008@v.loewis.de>

Python 2.4 will become 5 years old this November. I plan to make the
final security release this month or next month. If you want to see
additional patches in Python 2.4, please let us now, or commit them
yourself if you can.

Remember that only security fixes can be considered for inclusion,
and preferably only fixes that have already been applied to later
releases of Python.

Regards,
Martin

From asmodai at in-nomine.org  Mon Jun 29 11:54:36 2009
From: asmodai at in-nomine.org (Jeroen Ruigrok van der Werven)
Date: Mon, 29 Jun 2009 11:54:36 +0200
Subject: [Python-Dev] ndPython: I NEED TO TALK WITH ONE OF THE PYTHON
 CORE
In-Reply-To: <h23ar0$vsp$1@ger.gmane.org>
References: <000c01c9f675$19917760$011ea8c0@pegasus>
	<h23ar0$vsp$1@ger.gmane.org>
Message-ID: <20090629095436.GH65921@nexus.in-nomine.org>

-On [20090626 22:29], Terry Reedy (tjreedy at udel.edu) wrote:
>If you were running on a PC with what is now considered to be very small 
>memory, I would hypothesize that you had filled memory so that the 
>interpreter or parts thereof were being swapped in and out of memory 
>from and to disk. Is any thing like that possible with the PSP?

>From what I know it has either 32 or 64 MB of RAM, depending on the model.
The storage comes in the form of a Memory Stick PRO Duo, but I doubt it is
used in any form as a paging or swap file.
At least playing on my own it only accesses it when I want to save game
data.

-- 
Jeroen Ruigrok van der Werven <asmodai(-at-)in-nomine.org> / asmodai
????? ?????? ??? ?? ??????
http://www.in-nomine.org/ | http://www.rangaku.org/ | GPG: 2EAC625B
Friendship is love without wings...

From ziade.tarek at gmail.com  Mon Jun 29 19:57:56 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Mon, 29 Jun 2009 19:57:56 +0200
Subject: [Python-Dev] PEP 376
Message-ID: <94bdd2610906291057i3d2f533dqcfb3c986a5946835@mail.gmail.com>

Hello,

If no one objects, I'd like to push PEP 376 in the "accepted" status
and go ahead with its implementation,
with continuous feedback at Distutils-SIG as we did to build it.

The next PEPs that are being discussed at Distutils-SIG are :

- the new version of PEP 345 for the inclusion of fields like
"installed_requires" (dependencies)
- PEP 386, that talks about distribution version numbers (because it's
needed to describe dependencies)

While they are still under heavy discussion, I have good hope that
they will both make it for 2.7 and 3.2.

Regards
Tarek

-- 
Tarek Ziad? | http://ziade.org

From phillip.sitbon+python-dev at gmail.com  Mon Jun 29 21:33:00 2009
From: phillip.sitbon+python-dev at gmail.com (Phillip Sitbon)
Date: Mon, 29 Jun 2009 12:33:00 -0700
Subject: [Python-Dev] pthread sem PyThread_acquire_lock
In-Reply-To: <4A46A40B.5080202@aim.com>
References: <4A466846.1050904@aim.com> <4A468507.1060503@v.loewis.de>
	<4A46A40B.5080202@aim.com>
Message-ID: <536685ea0906291233r1b3e8dc4ndf25e7f404a87e22@mail.gmail.com>

I'll do my best to try and explain/contribute, but please feel free to
correct anything I get wrong.

I believe the "swallowing" he's referring to is the ignoring of errno
EINTR. I don't think that's the correct place to handle signals to
begin with- why not just use the signal module to deal with such a
scenario?

http://docs.python.org/dev/library/signal.html#module-signal

AFAIK, ignoring EINTR doesn't preclude the calling of signal handlers.
There are many functions that don't return this value anymore, making
them reentrant. I remember a number of years ago when it wasn't part
of any standard to return EINTR or not, and so the only way to provide
consistent behavior was to ignore it and loop. I'm not sure if that is
still the case.

A great example is reading from a socket. Whether or not it can be
interrupted depends on the platform, so catching Ctrl+C often requires
a timeout loop.

Also, remember that signals are asynchronous in the sense that they
are handled outside the normal execution flow of a program. Checking
for EINTR probably isn't the best way to determine if a signal has
been sent to the program.

Cheers,

Phillip


On Sat, Jun 27, 2009 at 3:58 PM, Florian Mayer <flormayer at aim.com> wrote:
>
> Martin v. L?wis wrote:
>>
>> Can you please explain what you mean by "swallowing"?
>
> The signals don't get through when acquiring the lock.
>>
>> What is the sequence of actions triggering the case you are talking
>> about, what happens, and what do you expect to happen instead?
>
> Easiest way is to create a Queue.Queue and call .get() on it and try to Ctrl-C it.
>>
>> Also, how would you fix this (in principle, not what the specific
>> patch would look like)?
>
> Remove the loop that explicitly does this in a new function and use that
> one in threadmodule.c for lock.acquire.
>>
>> Regards,
>> Martin
>
> Best regards,
> Florian
>
> PS: Excuse me, I messed up things once again.
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/phillip.sitbon%2Bpython-dev%40gmail.com

From martin at v.loewis.de  Mon Jun 29 22:45:09 2009
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Mon, 29 Jun 2009 22:45:09 +0200
Subject: [Python-Dev] PEP 376
In-Reply-To: <94bdd2610906291057i3d2f533dqcfb3c986a5946835@mail.gmail.com>
References: <94bdd2610906291057i3d2f533dqcfb3c986a5946835@mail.gmail.com>
Message-ID: <4A4927D5.4040001@v.loewis.de>

> If no one objects, I'd like to push PEP 376 in the "accepted" status
> and go ahead with its implementation,
> with continuous feedback at Distutils-SIG as we did to build it.

I think this isn't quite the process. In the past, every PEP required
BDFL pronouncement, which you should now seek.

Regards,
Martin

From martin at v.loewis.de  Mon Jun 29 23:28:07 2009
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Mon, 29 Jun 2009 23:28:07 +0200
Subject: [Python-Dev] pthread sem PyThread_acquire_lock
In-Reply-To: <536685ea0906291233r1b3e8dc4ndf25e7f404a87e22@mail.gmail.com>
References: <4A466846.1050904@aim.com> <4A468507.1060503@v.loewis.de>	
	<4A46A40B.5080202@aim.com>
	<536685ea0906291233r1b3e8dc4ndf25e7f404a87e22@mail.gmail.com>
Message-ID: <4A4931E7.1020108@v.loewis.de>

> AFAIK, ignoring EINTR doesn't preclude the calling of signal handlers.

This is my understanding as well - so I don't think Python actually
"swallows" the signal.

> A great example is reading from a socket. Whether or not it can be
> interrupted depends on the platform, so catching Ctrl+C often requires
> a timeout loop.
> 
> Also, remember that signals are asynchronous in the sense that they
> are handled outside the normal execution flow of a program. Checking
> for EINTR probably isn't the best way to determine if a signal has
> been sent to the program.

I think it would be reasonable to support "asynchronous" exceptions,
and Python supports SIGINT fairly well most of the time.

It might be possible to support keyboard interrupts throughout
the system, but changing Python to do so could also cause
incompatibilities. So any change must be done with greatest care,
but simultaneously, should also try to arrange to cover all cases.

Regards,
Martin

From jnoller at gmail.com  Tue Jun 30 03:59:01 2009
From: jnoller at gmail.com (Jesse Noller)
Date: Mon, 29 Jun 2009 21:59:01 -0400
Subject: [Python-Dev] Anyone against having a loop option for regrtest?
Message-ID: <4222a8490906291859o682a4aayb10cf8a7c798be06@mail.gmail.com>

Something that's been helping me squirrel out "wacky" and "fun" bugs
in multiprocessing is running the tests in a loop - sometimes hundreds
of times. Right now, I hack this up with a bash script, but I'm
sitting here wondering if adding a "loop for x iterations" option to
regrtest.py would be useful to others as well.

Any thoughts? Does anyone hate this idea with the power of a thousand suns?

-jesse

From stephen at xemacs.org  Tue Jun 30 04:23:52 2009
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Tue, 30 Jun 2009 11:23:52 +0900
Subject: [Python-Dev]  Anyone against having a loop option for regrtest?
In-Reply-To: <4222a8490906291859o682a4aayb10cf8a7c798be06@mail.gmail.com>
References: <4222a8490906291859o682a4aayb10cf8a7c798be06@mail.gmail.com>
Message-ID: <87tz1ywjbb.fsf@uwakimon.sk.tsukuba.ac.jp>

Jesse Noller writes:

 > Any thoughts? Does anyone hate this idea with the power of a thousand suns?

If somebody has the power of 1000 Suns at their disposal, maybe they
can contribute a few buildbots?

Wishful-thinking-is-the-order-of-the-day-ly y'rs,


From collinw at gmail.com  Tue Jun 30 04:22:08 2009
From: collinw at gmail.com (Collin Winter)
Date: Mon, 29 Jun 2009 19:22:08 -0700
Subject: [Python-Dev] Anyone against having a loop option for regrtest?
In-Reply-To: <4222a8490906291859o682a4aayb10cf8a7c798be06@mail.gmail.com>
References: <4222a8490906291859o682a4aayb10cf8a7c798be06@mail.gmail.com>
Message-ID: <43aa6ff70906291922u4dc687bcm74bcbe38ba2d9a8f@mail.gmail.com>

On Mon, Jun 29, 2009 at 6:59 PM, Jesse Noller<jnoller at gmail.com> wrote:
> Something that's been helping me squirrel out "wacky" and "fun" bugs
> in multiprocessing is running the tests in a loop - sometimes hundreds
> of times. Right now, I hack this up with a bash script, but I'm
> sitting here wondering if adding a "loop for x iterations" option to
> regrtest.py would be useful to others as well.
>
> Any thoughts? Does anyone hate this idea with the power of a thousand suns?

+1 for having this in regrtest. I've wished for this in the past, and
ended up going the bash route, same as you.

Collin

From jnoller at gmail.com  Tue Jun 30 04:41:49 2009
From: jnoller at gmail.com (Jesse Noller)
Date: Mon, 29 Jun 2009 22:41:49 -0400
Subject: [Python-Dev] Anyone against having a loop option for regrtest?
In-Reply-To: <87tz1ywjbb.fsf@uwakimon.sk.tsukuba.ac.jp>
References: <4222a8490906291859o682a4aayb10cf8a7c798be06@mail.gmail.com>
	<87tz1ywjbb.fsf@uwakimon.sk.tsukuba.ac.jp>
Message-ID: <4222a8490906291941x593dd439u269d9791af17183a@mail.gmail.com>

On Mon, Jun 29, 2009 at 10:23 PM, Stephen J. Turnbull<stephen at xemacs.org> wrote:
> Jesse Noller writes:
>
> ?> Any thoughts? Does anyone hate this idea with the power of a thousand suns?
>
> If somebody has the power of 1000 Suns at their disposal, maybe they
> can contribute a few buildbots?
>
> Wishful-thinking-is-the-order-of-the-day-ly y'rs,
>
>

Sorry I had to take mine down. That being said, I need to rummage up a
mac mini with a lot of ram and possibly fire up VMWare
fusion/virtualbox on it to have an os/x/linux/windows buildbot mini
farm.

From 4vinoth at gmail.com  Tue Jun 30 08:07:21 2009
From: 4vinoth at gmail.com (vin vin)
Date: Tue, 30 Jun 2009 11:37:21 +0530
Subject: [Python-Dev] Flow of control - a new way - Idea
In-Reply-To: <6176a14d0906292300r437dee80g9a414ed434da9de5@mail.gmail.com>
References: <6176a14d0906292300r437dee80g9a414ed434da9de5@mail.gmail.com>
Message-ID: <6176a14d0906292307l70db1315vb6cf8c171b38a6eb@mail.gmail.com>

HI all

I am not the too technical guy, but thinking about the new way of
controlling the flow instead of throwing an error.

as of now if we need to break a control and go back, exceptions helps,
but it is not a actual way.

it would be great if we have a control over the frames execution, I mean

A  calls B which calls C which calls D

at that point if we think to move directly to B (what error handler do
if that B has the handler defined of the error), changing the frames
instruction pointer to back to the B's position (in python code without
raising an exception) is a really useful for this web applications.

-----------------------------
How we can achieve this? a simple way?

Actually python interpreter currently doing this.
How?

method *A*

   method *B* <== has exception handler try: except:

      method *C*

         method *D*  <========= got exception here..

now exception handler checks that which caller methods has the exception
handler provided for this exception. in our case method B has this. so, the
instruction pointer has moved back to that method B with the exception.
-------------------

We can have this same functionality* BUT WITHOUT THE USE OF EXCEPTION*.

where it can help?

so many places, & it gives the full power of execution control to the user.

A simple usage of this is :

Currently most of the WSGI frameworks breaking the flow if needed to go on
another route, it simple raising the exception, But if any of the called
modules handled those exceptions it fails the flow. and we can't guide the
extension modules to don't use full try except as it is the pythons power.

What you say?

Please excuse me if we have this control already, (can u explain?)

I hope, currently no languages gives such full control over coders.

-- 
Thanks
Vinoth
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090630/be424683/attachment.htm>

From ncoghlan at gmail.com  Tue Jun 30 14:32:30 2009
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Tue, 30 Jun 2009 22:32:30 +1000
Subject: [Python-Dev] PEP 376
In-Reply-To: <4A4927D5.4040001@v.loewis.de>
References: <94bdd2610906291057i3d2f533dqcfb3c986a5946835@mail.gmail.com>
	<4A4927D5.4040001@v.loewis.de>
Message-ID: <4A4A05DE.2040804@gmail.com>

Martin v. L?wis wrote:
>> If no one objects, I'd like to push PEP 376 in the "accepted" status
>> and go ahead with its implementation,
>> with continuous feedback at Distutils-SIG as we did to build it.
> 
> I think this isn't quite the process. In the past, every PEP required
> BDFL pronouncement, which you should now seek.

Agreed. While Guido is highly likely to just accept the distutils-sig
consensus on something like this, that doesn't eliminate the need for
him to actually *say* that he is approving the PEP on that basis.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------

From ncoghlan at gmail.com  Tue Jun 30 14:36:20 2009
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Tue, 30 Jun 2009 22:36:20 +1000
Subject: [Python-Dev] Flow of control - a new way - Idea
In-Reply-To: <6176a14d0906292307l70db1315vb6cf8c171b38a6eb@mail.gmail.com>
References: <6176a14d0906292300r437dee80g9a414ed434da9de5@mail.gmail.com>
	<6176a14d0906292307l70db1315vb6cf8c171b38a6eb@mail.gmail.com>
Message-ID: <4A4A06C4.1020205@gmail.com>

vin vin wrote:
> 
> HI all
> 
> I am not the too technical guy, but thinking about the new way of
> controlling the flow instead of throwing an error.

This message is too speculative/tentative for python-dev (which is about
concrete development of the next version of Python) or even python-ideas
(which is the list for speculative proposals that may possibly be
considered some day).

I suggest taking this question to the general python list
(python-list at python.org, also available as the newsgroup comp.lang.python).

Regards,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------

From ziade.tarek at gmail.com  Tue Jun 30 14:38:51 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Tue, 30 Jun 2009 14:38:51 +0200
Subject: [Python-Dev] PEP 376
In-Reply-To: <4A4A05DE.2040804@gmail.com>
References: <94bdd2610906291057i3d2f533dqcfb3c986a5946835@mail.gmail.com>
	<4A4927D5.4040001@v.loewis.de> <4A4A05DE.2040804@gmail.com>
Message-ID: <94bdd2610906300538r4c06b142k6c0b5c109e210958@mail.gmail.com>

On Tue, Jun 30, 2009 at 2:32 PM, Nick Coghlan<ncoghlan at gmail.com> wrote:
> Martin v. L?wis wrote:
>>> If no one objects, I'd like to push PEP 376 in the "accepted" status
>>> and go ahead with its implementation,
>>> with continuous feedback at Distutils-SIG as we did to build it.
>>
>> I think this isn't quite the process. In the past, every PEP required
>> BDFL pronouncement, which you should now seek.
>
> Agreed. While Guido is highly likely to just accept the distutils-sig
> consensus on something like this, that doesn't eliminate the need for
> him to actually *say* that he is approving the PEP on that basis.

Sure, I didn't want to bypass the process, I was not really sure about
the right move on this PEP
since it was based on the summit decisions,

I'll wait for Guido decision, thanks.

Cheers
Tarek

-- 
Tarek Ziad? | http://ziade.org

From daniel at stutzbachenterprises.com  Tue Jun 30 15:44:09 2009
From: daniel at stutzbachenterprises.com (Daniel Stutzbach)
Date: Tue, 30 Jun 2009 08:44:09 -0500
Subject: [Python-Dev] Flow of control - a new way - Idea
In-Reply-To: <6176a14d0906292307l70db1315vb6cf8c171b38a6eb@mail.gmail.com>
References: <6176a14d0906292300r437dee80g9a414ed434da9de5@mail.gmail.com>
	<6176a14d0906292307l70db1315vb6cf8c171b38a6eb@mail.gmail.com>
Message-ID: <eae285400906300644j26190421lf4bd4d0a7ba97d5a@mail.gmail.com>

On Tue, Jun 30, 2009 at 1:07 AM, vin vin<4vinoth at gmail.com> wrote:
> at that point if we think to move directly to B (what error handler do
> if that B has the handler defined of the error), changing the frames
> instruction pointer to back to the B's position (in python code without
> raising an exception) is a really useful for this web applications.

If I understand you correctly, the language feature you are looking for is
called a "continuation", which was originally developed in the LISP/Scheme
world (as "call/cc").  I'm not familiar with the details, but I believe you
can get them in Stackless Python.

Hope that helps,

--
Daniel Stutzbach, Ph.D.
President, Stutzbach Enterprises, LLC <http://stutzbachenterprises.com/>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090630/ce29e9b9/attachment.htm>

From guido at python.org  Tue Jun 30 19:21:02 2009
From: guido at python.org (Guido van Rossum)
Date: Tue, 30 Jun 2009 10:21:02 -0700
Subject: [Python-Dev] PEP 376
In-Reply-To: <4A4A05DE.2040804@gmail.com>
References: <94bdd2610906291057i3d2f533dqcfb3c986a5946835@mail.gmail.com> 
	<4A4927D5.4040001@v.loewis.de> <4A4A05DE.2040804@gmail.com>
Message-ID: <ca471dc20906301021m2f5386f1r1e144ba9f1970d60@mail.gmail.com>

On Tue, Jun 30, 2009 at 5:32 AM, Nick Coghlan<ncoghlan at gmail.com> wrote:
> Martin v. L?wis wrote:
>>> If no one objects, I'd like to push PEP 376 in the "accepted" status
>>> and go ahead with its implementation,
>>> with continuous feedback at Distutils-SIG as we did to build it.
>>
>> I think this isn't quite the process. In the past, every PEP required
>> BDFL pronouncement, which you should now seek.
>
> Agreed. While Guido is highly likely to just accept the distutils-sig
> consensus on something like this, that doesn't eliminate the need for
> him to actually *say* that he is approving the PEP on that basis.

Sure. :-)

So what *is* the distutils-sig consensus?

And is there consensus outside of it? (Remember the ipaddr debacle.
It's easy for people to miss an important PEP.)

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From pje at telecommunity.com  Tue Jun 30 19:44:58 2009
From: pje at telecommunity.com (P.J. Eby)
Date: Tue, 30 Jun 2009 13:44:58 -0400
Subject: [Python-Dev] PEP 376
In-Reply-To: <94bdd2610906291057i3d2f533dqcfb3c986a5946835@mail.gmail.co
 m>
References: <94bdd2610906291057i3d2f533dqcfb3c986a5946835@mail.gmail.com>
Message-ID: <20090630174159.731CE3A4099@sparrow.telecommunity.com>

At 07:57 PM 6/29/2009 +0200, Tarek Ziad? wrote:
>Hello,
>
>If no one objects, I'd like to push PEP 376 in the "accepted" status
>and go ahead with its implementation,
>with continuous feedback at Distutils-SIG as we did to build it.

I do have a question about the current draft...  Do zipped 
distributions use EGG-INFO or a project-version.egg-info?  This isn't 
spelled out in the PEP, although I get the general idea that the 
EGG-INFO format isn't supported, and thus the PEP and API do not 
support existing .egg files.  This should probably be made clear, if 
that's the intention.


From p.f.moore at gmail.com  Tue Jun 30 20:37:11 2009
From: p.f.moore at gmail.com (Paul Moore)
Date: Tue, 30 Jun 2009 19:37:11 +0100
Subject: [Python-Dev] PEP 376
In-Reply-To: <ca471dc20906301021m2f5386f1r1e144ba9f1970d60@mail.gmail.com>
References: <94bdd2610906291057i3d2f533dqcfb3c986a5946835@mail.gmail.com>
	<4A4927D5.4040001@v.loewis.de> <4A4A05DE.2040804@gmail.com>
	<ca471dc20906301021m2f5386f1r1e144ba9f1970d60@mail.gmail.com>
Message-ID: <79990c6b0906301137r3728c86atcca4cf2da61cdf7b@mail.gmail.com>

2009/6/30 Guido van Rossum <guido at python.org>:
> And is there consensus outside of it? (Remember the ipaddr debacle.
> It's easy for people to miss an important PEP.)

My impression (as someone who does read the distutils SIG, but in all
honesty has very little practical interest in distutils internals):

- It's very focused on distutils internals. If it has an impact on end
users (as opposed to packagers), it's very hard to discern. The only
hint of such a thing is the mention of an uninstall function. But it's
only an API. So still no end user impact [1] :-(
- The terminology and focus feels setuptools-inspired (my apologies if
that's not the case). Expect pushback from setuptools haters...
- It's quite dense for the casual reader not familiar with the
terminology. I've never managed to read the whole thing through,
personally.

I'd suggest two things:
- Add a section to the PEP describing the purely end user impact of the changes
- Post that to python-list, with a note pointing to the PEP for people
who care about distutils details

If that gets no feedback, you've done as much as you can.

Paul.

[1] I'd actually like it if the PEP defined an uninstall command -
something like "python -m distutils.uninstall packagename". It can be
as minimalist as you like, but I'd like to see it present.

From martin at v.loewis.de  Tue Jun 30 21:18:55 2009
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Tue, 30 Jun 2009 21:18:55 +0200
Subject: [Python-Dev] PEP 376
In-Reply-To: <79990c6b0906301137r3728c86atcca4cf2da61cdf7b@mail.gmail.com>
References: <94bdd2610906291057i3d2f533dqcfb3c986a5946835@mail.gmail.com>	
	<4A4927D5.4040001@v.loewis.de> <4A4A05DE.2040804@gmail.com>	
	<ca471dc20906301021m2f5386f1r1e144ba9f1970d60@mail.gmail.com>
	<79990c6b0906301137r3728c86atcca4cf2da61cdf7b@mail.gmail.com>
Message-ID: <4A4A651F.9000805@v.loewis.de>

> - Post that to python-list, with a note pointing to the PEP for people
> who care about distutils details

If that hasn't been done before (I have not followed), it should be done
right away. PEP 1 makes it a mandatory requirement for acceptance.

Regards,
Martin

From ziade.tarek at gmail.com  Tue Jun 30 21:19:07 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Tue, 30 Jun 2009 21:19:07 +0200
Subject: [Python-Dev] PEP 376
In-Reply-To: <ca471dc20906301021m2f5386f1r1e144ba9f1970d60@mail.gmail.com>
References: <94bdd2610906291057i3d2f533dqcfb3c986a5946835@mail.gmail.com>
	<4A4927D5.4040001@v.loewis.de> <4A4A05DE.2040804@gmail.com>
	<ca471dc20906301021m2f5386f1r1e144ba9f1970d60@mail.gmail.com>
Message-ID: <94bdd2610906301219h3300d610jaf8e376b14de3d42@mail.gmail.com>

2009/6/30 Guido van Rossum <guido at python.org>:
> On Tue, Jun 30, 2009 at 5:32 AM, Nick Coghlan<ncoghlan at gmail.com> wrote:
>> Martin v. L?wis wrote:
>>>> If no one objects, I'd like to push PEP 376 in the "accepted" status
>>>> and go ahead with its implementation,
>>>> with continuous feedback at Distutils-SIG as we did to build it.
>>>
>>> I think this isn't quite the process. In the past, every PEP required
>>> BDFL pronouncement, which you should now seek.
>>
>> Agreed. While Guido is highly likely to just accept the distutils-sig
>> consensus on something like this, that doesn't eliminate the need for
>> him to actually *say* that he is approving the PEP on that basis.
>
> Sure. :-)
>
> So what *is* the distutils-sig consensus?

The consensus is to have one and only one way to install distributions
in Python,
and a way to retrieve information on installed distributions, and their files.


>
> And is there consensus outside of it? (Remember the ipaddr debacle.
> It's easy for people to miss an important PEP.)

I am not sure who you are thinking about,

I am blogging a lot on the topic and I am trying to get key players
involved in this, like os packagers etc.

We've built this PEP in respect with existing tools like setuptools,
etc, and I am sending mails at python-dev
to make sure evereyone involved in python development has a chance to
provide some feedback,


>
> --
> --Guido van Rossum (home page: http://www.python.org/~guido/)
>



-- 
Tarek Ziad? | http://ziade.org

From tjreedy at udel.edu  Tue Jun 30 21:24:34 2009
From: tjreedy at udel.edu (Terry Reedy)
Date: Tue, 30 Jun 2009 15:24:34 -0400
Subject: [Python-Dev] Flow of control - a new way - Idea
In-Reply-To: <6176a14d0906292307l70db1315vb6cf8c171b38a6eb@mail.gmail.com>
References: <6176a14d0906292300r437dee80g9a414ed434da9de5@mail.gmail.com>
	<6176a14d0906292307l70db1315vb6cf8c171b38a6eb@mail.gmail.com>
Message-ID: <h2dopi$93g$1@ger.gmane.org>

vin vin wrote:

Look up 'trampoline', but ask any further questions on python-list.


From ziade.tarek at gmail.com  Tue Jun 30 21:28:43 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Tue, 30 Jun 2009 21:28:43 +0200
Subject: [Python-Dev] PEP 376
In-Reply-To: <79990c6b0906301137r3728c86atcca4cf2da61cdf7b@mail.gmail.com>
References: <94bdd2610906291057i3d2f533dqcfb3c986a5946835@mail.gmail.com>
	<4A4927D5.4040001@v.loewis.de> <4A4A05DE.2040804@gmail.com>
	<ca471dc20906301021m2f5386f1r1e144ba9f1970d60@mail.gmail.com>
	<79990c6b0906301137r3728c86atcca4cf2da61cdf7b@mail.gmail.com>
Message-ID: <94bdd2610906301228s4ac0fb20id44bd144b82af600@mail.gmail.com>

On Tue, Jun 30, 2009 at 8:37 PM, Paul Moore<p.f.moore at gmail.com> wrote:
> - The terminology and focus feels setuptools-inspired (my apologies if
> that's not the case). Expect pushback from setuptools haters...

setuptools implemented *needed* features, like a way for developers to browse
installed packages.

We said during the summit at Pycon that we wanted this feature in
Distutils, (Guido said so)

So I worked in PEP 376 to introduce them.

Now if the fact that we want to introduce the good ideas of setuptools
into distutils,
(problems Phillip resolved) will make people push it back *even* they
are good idead, needed features,
 is something we need to fight against.

> - It's quite dense for the casual reader not familiar with the
> terminology. I've never managed to read the whole thing through,
> personally.

I'll try to add a definitions section,

>
> I'd suggest two things:
> - Add a section to the PEP describing the purely end user impact of the changes
> - Post that to python-list, with a note pointing to the PEP for people
> who care about distutils details
>

Ok.

> If that gets no feedback, you've done as much as you can.

I hope it'll make it one day. If not, I don't understand the goal of
the "Python Language Summit'

>
> Paul.
>
> [1] I'd actually like it if the PEP defined an uninstall command -
> something like "python -m distutils.uninstall packagename". It can be
> as minimalist as you like, but I'd like to see it present.

it's already there:

http://www.python.org/dev/peps/pep-0376/#adding-an-uninstall-function


Regards
Tarek

-- 
Tarek Ziad? | http://ziade.org

From ziade.tarek at gmail.com  Tue Jun 30 21:35:50 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Tue, 30 Jun 2009 21:35:50 +0200
Subject: [Python-Dev] PEP 376
In-Reply-To: <20090630174159.731CE3A4099@sparrow.telecommunity.com>
References: <94bdd2610906291057i3d2f533dqcfb3c986a5946835@mail.gmail.com>
	<20090630174159.731CE3A4099@sparrow.telecommunity.com>
Message-ID: <94bdd2610906301235g60bfea7dsb67a72a04213d8d0@mail.gmail.com>

2009/6/30 P.J. Eby <pje at telecommunity.com>:
> At 07:57 PM 6/29/2009 +0200, Tarek Ziad? wrote:
>>
>> Hello,
>>
>> If no one objects, I'd like to push PEP 376 in the "accepted" status
>> and go ahead with its implementation,
>> with continuous feedback at Distutils-SIG as we did to build it.
>
> I do have a question about the current draft... ?Do zipped distributions use
> EGG-INFO or a project-version.egg-info? ?This isn't spelled out in the PEP,
> although I get the general idea that the EGG-INFO format isn't supported,
> and thus the PEP and API do not support existing .egg files. ?This should
> probably be made clear, if that's the intention.

The intention is to have only one standard for the egg-info, I have
explained it in this section:

http://www.python.org/dev/peps/pep-0376/#egg-info-becomes-a-directory

previous formats will not be supported but that won't break anything of course
since the new APIs will work only over the distribution installed with
the new version
of distutils.



>
>



-- 
Tarek Ziad? | http://ziade.org

From martin at v.loewis.de  Tue Jun 30 21:49:18 2009
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Tue, 30 Jun 2009 21:49:18 +0200
Subject: [Python-Dev] PEP 376
In-Reply-To: <94bdd2610906301228s4ac0fb20id44bd144b82af600@mail.gmail.com>
References: <94bdd2610906291057i3d2f533dqcfb3c986a5946835@mail.gmail.com>	
	<4A4927D5.4040001@v.loewis.de> <4A4A05DE.2040804@gmail.com>	
	<ca471dc20906301021m2f5386f1r1e144ba9f1970d60@mail.gmail.com>	
	<79990c6b0906301137r3728c86atcca4cf2da61cdf7b@mail.gmail.com>
	<94bdd2610906301228s4ac0fb20id44bd144b82af600@mail.gmail.com>
Message-ID: <4A4A6C3E.2090005@v.loewis.de>

Tarek Ziad? wrote:
> On Tue, Jun 30, 2009 at 8:37 PM, Paul Moore<p.f.moore at gmail.com> wrote:
>> - The terminology and focus feels setuptools-inspired (my apologies if
>> that's not the case). Expect pushback from setuptools haters...
> 
> setuptools implemented *needed* features, like a way for developers to browse
> installed packages.

That doesn't imply that setuptools itself is needed. I can browse
installed packages with dpkg -l, or "add-and-remove programs".

> Now if the fact that we want to introduce the good ideas of setuptools
> into distutils,
> (problems Phillip resolved) will make people push it back *even* they
> are good idead, needed features,
>  is something we need to fight against.

Assuming "we" have consensus before we start fighting against others.

FWIW, I abstain from commenting on PEP 376. I don't need it, but it
doesn't seem to hurt having it, especially since "egg-info" already
managed to sneak in.

> I hope it'll make it one day. If not, I don't understand the goal of
> the "Python Language Summit'

Just be patient. Both Python 2.7 and 3.2 are still many months ahead,
so there is no urgency (AFAICT).

Regards,
Martin

From Scott.Daniels at Acm.Org  Tue Jun 30 22:06:07 2009
From: Scott.Daniels at Acm.Org (Scott David Daniels)
Date: Tue, 30 Jun 2009 13:06:07 -0700
Subject: [Python-Dev] PEP 376
In-Reply-To: <94bdd2610906301228s4ac0fb20id44bd144b82af600@mail.gmail.com>
References: <94bdd2610906291057i3d2f533dqcfb3c986a5946835@mail.gmail.com>	<4A4927D5.4040001@v.loewis.de>
	<4A4A05DE.2040804@gmail.com>	<ca471dc20906301021m2f5386f1r1e144ba9f1970d60@mail.gmail.com>	<79990c6b0906301137r3728c86atcca4cf2da61cdf7b@mail.gmail.com>
	<94bdd2610906301228s4ac0fb20id44bd144b82af600@mail.gmail.com>
Message-ID: <h2dqqe$fp1$1@ger.gmane.org>

Tarek Ziad? wrote:
> On Tue, Jun 30, 2009 at 8:37 PM, Paul Moore<p.f.moore at gmail.com> wrote:
>> [1] I'd actually like it if the PEP defined an uninstall command -
>> something like "python -m distutils.uninstall packagename". It can be
>> as minimalist as you like, but I'd like to see it present.
> 
> it's already there:
> 
> http://www.python.org/dev/peps/pep-0376/#adding-an-uninstall-function

That (at least as I read it) is a function, not a command.
If it is a command, give an example of its use from the command line
for us poor "don't want to research" people.  If the following works:

     $ python setup.py uninstall some_package

Then explicitly say so for us poor schlubs.

--Scott David Daniels
Scott.Daniels at Acm.Org


From p.f.moore at gmail.com  Tue Jun 30 22:11:44 2009
From: p.f.moore at gmail.com (Paul Moore)
Date: Tue, 30 Jun 2009 21:11:44 +0100
Subject: [Python-Dev] PEP 376
In-Reply-To: <94bdd2610906301228s4ac0fb20id44bd144b82af600@mail.gmail.com>
References: <94bdd2610906291057i3d2f533dqcfb3c986a5946835@mail.gmail.com>
	<4A4927D5.4040001@v.loewis.de> <4A4A05DE.2040804@gmail.com>
	<ca471dc20906301021m2f5386f1r1e144ba9f1970d60@mail.gmail.com>
	<79990c6b0906301137r3728c86atcca4cf2da61cdf7b@mail.gmail.com>
	<94bdd2610906301228s4ac0fb20id44bd144b82af600@mail.gmail.com>
Message-ID: <79990c6b0906301311l4fd029dcq7ec2b4f427eb6bfb@mail.gmail.com>

2009/6/30 Tarek Ziad? <ziade.tarek at gmail.com>:
> On Tue, Jun 30, 2009 at 8:37 PM, Paul Moore<p.f.moore at gmail.com> wrote:
>> - The terminology and focus feels setuptools-inspired (my apologies if
>> that's not the case). Expect pushback from setuptools haters...
>
> setuptools implemented *needed* features, like a way for developers to browse
> installed packages.

No problem with any of that. Please understand that on the whole, I'm
in favour of the setuptools features. (While I dislike intensely some
of the setuptools "ecosystem", such as easy_install. In fact, one of
my goals in wanting this PEP accepted is to ensure that I can get the
"good bits" of setuptools, without having to buy into the
easy_install, dependency management, automated download infrastructure
that currently goes with it).

> We said during the summit at Pycon that we wanted this feature in
> Distutils, (Guido said so)

"We" in this context denotes the people at the summit. Please remember
that people who weren't there still have an opinion - and it may well
differ. I'm not saying that it *does* differ, just that the view of
the summit should not be viewed as conclusive (unless the way Python
is developed is very different from how I understand it).

> So I worked in PEP 376 to introduce them.

And as with any PEP, that's a difficult and thankless task (I got a
taste myself with PEP 302, and that was far easier), so just for the
record, you have my thanks for doing this.

> Now if the fact that we want to introduce the good ideas of setuptools
> into distutils,
> (problems Phillip resolved) will make people push it back *even* they
> are good idead, needed features,
> ?is something we need to fight against.

But there *are* issues with setuptools. Some users have mentioned them
on a number of occasions. I've raised a few myself. By all means
introduce the good ideas into the core - but make sure people agree
that the ideas *are* good.

And remember - all I said was that people may react against the
setuptools-style terminology. Not that they dislike the *idea*, just
that they might dislike the way it's presented. After all, one common
complaint with setuptools (and one that I agree with) is that its
documentation is obscure to the point of being hostile.

>> - It's quite dense for the casual reader not familiar with the
>> terminology. I've never managed to read the whole thing through,
>> personally.
>
> I'll try to add a definitions section,

Thank you. I'll try to make the time to go through the PEP and comment
more fully.

>> [1] I'd actually like it if the PEP defined an uninstall command -
>> something like "python -m distutils.uninstall packagename". It can be
>> as minimalist as you like, but I'd like to see it present.
>
> it's already there:
>
> http://www.python.org/dev/peps/pep-0376/#adding-an-uninstall-function

No, that defines an API, which as stated in the PEP, "allows a
third-party application to use the uninstall function and make sure
it's the only program that can remove a distribution it has previously
installed". It does NOT define a standard uninstall command, to
complement the standard install command.

Paul.

From ncoghlan at gmail.com  Tue Jun 30 23:11:36 2009
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Wed, 01 Jul 2009 07:11:36 +1000
Subject: [Python-Dev] PEP 376
In-Reply-To: <94bdd2610906301235g60bfea7dsb67a72a04213d8d0@mail.gmail.com>
References: <94bdd2610906291057i3d2f533dqcfb3c986a5946835@mail.gmail.com>	<20090630174159.731CE3A4099@sparrow.telecommunity.com>
	<94bdd2610906301235g60bfea7dsb67a72a04213d8d0@mail.gmail.com>
Message-ID: <4A4A7F88.7080609@gmail.com>

Tarek Ziad? wrote:
> 2009/6/30 P.J. Eby <pje at telecommunity.com>:
>> At 07:57 PM 6/29/2009 +0200, Tarek Ziad? wrote:
>>> Hello,
>>> 
>>> If no one objects, I'd like to push PEP 376 in the "accepted"
>>> status and go ahead with its implementation, with continuous
>>> feedback at Distutils-SIG as we did to build it.
>> I do have a question about the current draft...  Do zipped
>> distributions use EGG-INFO or a project-version.egg-info?  This
>> isn't spelled out in the PEP, although I get the general idea that
>> the EGG-INFO format isn't supported, and thus the PEP and API do
>> not support existing .egg files.  This should probably be made
>> clear, if that's the intention.
> 
> The intention is to have only one standard for the egg-info, I have 
> explained it in this section:
> 
> http://www.python.org/dev/peps/pep-0376/#egg-info-becomes-a-directory
> 
> 
> previous formats will not be supported but that won't break anything
> of course since the new APIs will work only over the distribution
> installed with the new version of distutils.

To address PJE's question in the PEP, it may be worth expanding on this
in the backwards compatibility section explaining how the new distutils
metadata system avoids getting confused by the old pre-standardisation
installation formats (e.g. it may be that the directory names and/or
filenames all deliberately differ from current approaches precisely so
they can coexist without interfering with each other)

Also, I find the following paragraph (near the start of the section
linked above) confusing:

[PEP 376]
> Notice that this change is based on the standard proposed by
> EggFormats, although this standard proposes two ways to install
> files:
> 
> * A self-contained directory that can be zipped or left unzipped and
> contains the distribution files and the .egg-info directory.
> * A distinct .egg-info directory located in the site-packages directory.

This is unclear as to what "this standard" is referring to (since the
PEP itself is proposing a standard, but the sentence is also referring
to the existing EggFormats de facto standard). If the PEP only supports
the latter of the two options (which appears to be the case) it should
say so explicitly.

Another minor nit from the same section:

[PEP 376]
> Any '-' characters are currently replaced with '_'.

This should say something like "Any '-' characters other than the one in
'egg-info' and the one separating the name from the version number are
included in the replacement of non-alphanumeric characters with '_'"

Other questions/comments:

- What is a "local absolute path"? Absolute path I understand, relative
path I understand, but "local absolute" is a novel term to me.

- In DistributionDirMap constructor description: "is not not" -> "is not
None"

 - as Paul Moore suggested, exposing a distutils based uninstall command
at the command line may make sense (and if it doesn't, the PEP should
say why not)

Otherwise, from a non-expert point of view I actually find the PEP
pretty readable and clear, and it all looks like a pretty sensible
design to me.

Regards,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------